I once at a huge slice of boston creme pie from a local deli, I got nearly to the end before I flipped it over and saw the bottom crust was entirely black mold.

I had one of the worst stomach aches of my life for about 6 hours that night and enough gas to power a small town, but no lasting harm. I was also a teenager so likely my digestive system was nigh invulnerable.

It should be understood that most fungus won't survive in your body, but it does produce toxins. Those toxins don't have a hard line between harmless and toxic, so the small amounts of spores you ingest every day do nothing, but eating a whole moldy bread slice could make you pretty sick.