It released too late and was way too expensive.

I say this as someone who grew up in that time period and has fond nostalgia: it has one of the worst libraries of any console. Depending on how you count (the different regions, the 64DD, what counts as a "game", etc) there were 200-300 N64 games. That may seem like a pretty big difference between 200 and 300, but in comparison the PS1 had, on a conservative count, 4,100 games. If you want to say only 10% of PS1 games we're good that's still more good games than the N64 had games.

There are a handful of titles that will be remembered as some of the greatest games of all time. The two Zelda games, Super Smash Bros, Mario Party, Mario Kart, Paper Mario. Personally I like the Pokemon games too. But the list falls off pretty hard after that.

I love 3D platformers and collect-a-thons, but I could never get into Mario 64, Banjo Kazooie, or Donkey Kong 64. They all feel rudimentary to me, similar to Jumping Flash on the PS1. Maybe it's because the N64's joystick was so uncomfortable and loose. Crash Bandicoot 1 came out in the US before Mario 64 did, and in my opinion it was more fun, looks better, sounds better, and holds up better today. And then there were two more Crash games, plus the Spyro trilogy which I consider even better.

There are "cult classics" for the N64 that I think are only remembered like that because of the lack of other options. Blast Corps for example is a unique and creative little game. It's fun to play for a bit, but was that experience really worth the price of a whole game? It almost feels like it could have been a side mode in something like Twisted Metal.

There's so many games it didn't have. Metal Gear Solid, Castlevania, and Final Fantasy are perhaps the most famous. Even a lot of games it did have were much worse- Resident Evil 2 and the Tony Hawk series are big examples where the cheap storage of the PS1 was clearly better. I remember I had a mediocre PS1 game called Battletanx that was pretty fun. Later on in high school my friend had a modded Xbox that emulated N64 games and I recognized that title, so we played through the co-op. It was still fun, but the textures were mostly replaced with flat colors and it was hard to see what was going on. I thought there may have been an issue with the emulation, or maybe the ROM was for some beta build or a hacked version, but... No, that's just how it looked on the N64.

I didn't mind the 3-prong controller. Honestly just having handles was already an upgrade over the SNES and Genesis. But the controller itself feels so cheap. The buttons all rattle around loosely and feel mushy and unsatisfying to press. The joystick is hard plastic, too tall, and flaccid. The plastic itself is a downgrade compared to its predecessors and to the Dualshock and even Saturn controller.

I still have my N64 and the handful of games I got for it. It had some of the highest highs of any console, but little else.