This article from Scientific American - Now Hear This! Most People Stink at Listening by Bob Sullivan and Hugh Thompson (an excerpt from their book: The Plateau Effect) - provides a great explanation of why it is so easy to get distracted during a lecture: your brain is ready to process about 400 words per minute or so (like when you are reading), but even someone who speaks rather quickly is only speaking about 125 words per minute. Your brain craves MORE than what the speaker can give you, so you are likely to start thinking about other things. As the authors say:
Here’s the problem: The human brain has the capacity to digest as much as 400 words per minute of information. But even a speaker from New York City talks at around 125 words per minute. That means three-quarters of your brain could very well be doing something else while someone is speaking to you.Read the article to learn just how little people remember of what they listen to, along with some suggestions about what you can try to do to improve your powers of focus and recall.