Banana is somehow radioactive too you know. Or granit. Anciant Greece monument often are above the radiation levels allowed in a nuclear power plant.

Also, while you talk about it, lead is far far more toxic than depleted uranium. Many metals are toxic actually, that's why your government monitor water meant for consumption.