Computers handle data and files are simply data stored on the physical medium.
While I get your point, I don't think that they are a good example in the least. If we had only one file type that was always structured the same, maybe, but that's not the case. Files are essentially atomic.
Memory is still structured like a file and referenced over addresses, we just call it something else.