Super interesting that you enjoy fiction so much. What I struggle with most is that visual language is often very dense in information, but I can't do a lot with it. Imagine something like this:

"Light spilled in through the high windows, tinting the hallway into beautiful autumn colors. It looked as if the sunlight was dancing, but of course nothing moved except the dust suspended in the air."

I would read this and think: cool, I bet this would look amazing if I could see it, but all the information I can actually use from these sentences is "A hallway has high windows, it's maybe morning or evening". Everything else is either visual or obvious to me. So fiction books are more exhausting, because I constantly filter out things that I can't really use. It's like I'm reading a text where a person constantly rambles and can't get to the damn point. I'm really curious how or why this is different for you? Also, I do think fiction books and non-fiction history books are very different. Simply because an author can build a world, story and characters to convey some deeper meaning or overarching theme, or use strong imagery or metaphores. All of that is more uncommon for historic books from my experience. The above example in a history book would probably look something more like "Orange light entered the hallway through the high windows".

Good to know.

Gosh, have to say there are more aphantasics around than I would have guessed.