We have soooo much to remember these days, and what we do have is being constantly written over with new data, that unless it's memorable, we forget, no, we don't really forget, it's just buried deeper.
My guess is that you relive your travels regularly through stories, or photographs. Lunch? Unless it's at a lovely restaurant, I'll struggle to remember.