Hello The Fedora,
If I recall correctly Indy does remember most of the adventure, just not how it ended and how he ended up back on the surface.
It is a rather convenient way out of Indy saying something like "This reminds me of another adventure that took me to an inner world" excuse.
But...