The Sound and The Fury for me. If anybody has ever read this you'll understand; the random scene shifts and non sequturs killed me and I could hardly even finish the Bejy section much less understand it.
If you haven't read it, the narrator has down syndrome, no sense of time so different sites and smells set off random scene shifts (there are over a hundred in the first chapter). There are also a couple characters with the same name and Benjy doesn't really distinguish which he is talking about