Yes! I felt a bit the same way when I was reading Connie Willis, too (not coincidentally, these books were all recommended to me by the same person): the book's telling you that you're having a grand time, and I always felt "But am I, though?"

Can we blame Williamson for embellishing That Time I Played the Flute for Seals so he got to pet the seal at the end? Oh, probably we can, but at the same time if I played a flute for a seal I too would feel tempted to end the story "And then I got to pet the seal!" rather than "and then the seal chased me out of the cave!"