OP, since you used to bodybuild, you know that all the things you are getting frustrated with are things you don't need to do...unless you have plans to get back on stage. You don't need boot camp style training, measure every calorie, seem like a second job, etc...
My wife and I are both into bodybuilding/fitness/nutrition. She's hardcore and is toying with the idea of getting on stage. I work very hard, but have no interest in competing. I'm about the same age as you, maybe a little older. I go to the gym 5 days a week regularly....only times I miss are if I'm travelling or on vacation (even then I try to get some work in).
I'm not a big guy, and that's OK. I'm 5'9" and weight hovers around 175-180lb depending on the season. My body fat is in the low-mid teens, again depending on season. I'm pretty muscular for my size. I'm quite regimented in my nutrition. I allow myself one cheat meal per week, and one "snack" per week...my other 25+ meals per week are clean. As you know, if you want the true 6 pack, you need to have 100% of your meals clean, but I don't care enough to do that. I'm very fit, and happy with that.
I've got my share of physical challenges too. Crohns' disease, had surgery on both knees, surgery on one hip (the other is an eventuality), L4/5 and L5/S1 herniations. Every day, something hurts. Not going to be melodramatic and say that without working out I'd be wheelchair bound, because that's not true at all. But I have no doubt that it keeps my issues from being a lot worse.
No boot camp workout crap for me. I'm at the gym at 4:45am every day. I lift hard for about 45-50 minutes, taking proper rest between sets. Have a typical 5 day/week split. Finish off my workout with 10-15 minutes of moderately intense cardio. Done. No special fad workouts. My wife does the same, just does a bit more than me, and she's amazingly ripped. I also don't track every calorie. I used to, many years ago, but I know enough now to have a really good idea of what I'm putting into my body and can hit my splits just fine. I do roughly 275-300g protein/50-75g fat/125-150g carbs. If I wanted to hit the stage, then I'd need to track everything. But my splits are good enough to get me where I want, no feeling like it's a second job.
Best advice I have is to just scale it back. If you're just looking to get back into good shape, you can do it without all the hard core stuff. Like I said, I'm not stage ready, but in all modesty, I look pretty darn good for a guy in his mid-40s. Plenty of muscle, no pot belly, and it's not killing me to do it.