Science

I happen to be extremely nearsighted, and nobody figured that out until I was 8 years old, so that visual memory test thing they did wouldn't have worked well for me.

With that said, my very first memory is actually a smell, from about age 2.

What was that smell? Marijuana, no joke.

Im guessing indexing. You don't know enough of the context of the memories to create a good association and a complete reconstruction of the memory. So there is some very simple recall for hyper detailed things like the structure on my parents first couch. But nothing more complete.

My guess is our brains are hyper plastic at that age - new connections keep forming - and total number of neurons does not really increase. So either the the number of connections per neuron increases (this can only happen to certain extent, due to surface area and nearest number constraints - basically you can not arrange a lot of things close enough to have a stonger bond) or you can have connections rewired (break somewhere, and form somewhere else). And my guess is that latter is prefered beyond a certain point - most of activities of babies is sleeping or basic interactions with outside world, and maybe our brain prefers to keep the basic understanding of the world in memory (maybe because that would be revisited the most).