From your description, I think a hardtail mountain bike would be your best option.
“Gravel bike” covers a range of styles from racing to bikepacking, but in general it’s a drop-bar bike with relaxed geometry that can fit tires at least 40 mm wide. I have one. It’s great for rural New England dirt road and mixed-surface riding. For logging roads, jeep trails, double track, etc. I think a flat-bar bike with even bigger tires works better.
A short-travel full-suspension mountain bike would also work well, but that’s more money and weight.
Tires can make a huge difference. If mud isn’t common where you ride, a set of fast-rolling hard-surface tires can be a great upgrade. I’ve got Continental RaceKings on my mountain bike and have been impressed with them.