If you want to mess with different lenses, then yes. a DLSR or mirrorless camera will be the way to go. If you stick with it you will ultimately wind up spending more on lenses than the camera body itself. Don't sweat this too much; it's inevitable, and if you're careful you can keep your lens collection when you move to your next camera. An interchangeable lens camera of some sort will also allow you to engage in that most quintessential of hipster activities, fucking around with pinholes and tilt-shift bendy jigs and mount adapters and vintage glass, and all the other things that are emphatically not current model OEM lenses explicitly designed to work with your camera.
The only other option is to go with an point-and-shoot camera with an integrated lens. Since last time I looked (my last point-and-shoot was a Canon SX130 IS which I still have, and is so old it runs off of two AA cells), these have mutated significantly now that people's smartphone cameras have unequivocally dominated the market niches that used to be occupied by entry level happy snap cams, wind-up disposables, and so forth. Basically all of the point-and-shoots left standing are either novelty instant photo printer cameras or prosumer models offering significant zoom ranges if any of that is your jam, both of these being things that cell phone cameras still can't do.
But this also means they're not exactly cheap.
If you're dedicated to humping a big camera around anyway, a smallish mirrorless body is probably the way to go. Don't feel bad about whipping out your phone camera anyway, for certain subjects. I still do it all the time, even when I have my big camera dangling around my neck — usually because I have some manner of goddamned photohowitzer attached to it and I need to take a picture of something that's not half a mile away or I need a wide angle field of view for something. As Call of Duty taught us all, switching to your sidearm is faster than ~~reloading.~~ Er, swapping lenses.