My phone doesn't have a 3.5mm headphone jack.

While I use my phone for music I can certainly see some advantages:
physical buttons
smaller and lighter
less distraction than a phone
cheaper to replace if stolen or broken

I have a shuffle just for running.

You can replace your phone with dedicated electronics, which has some advantages, such as better battery life and generally better performance. Like I bet this thing has a better amplifier than a phone has.

And phones have their downsides. They don't last as long and are expensive. Privacy issues. And can be too stimulating and intrusive. For example sometimes you just need to know the time and before you know it you're emailing someone.

When I go hiking I can just as well take a dumb phone, a GPS and an mp3 player with me. Maybe a camera too.

Having something that doesn't connect to the internet makes for a better device in the long run. Plus, as others have mentioned, less distractions.