I’m a breakfast lover, so I’ve only been to Boma at breakfast, and it is now a must-do for me!
BUT, to the point of your particular question and situation: Yes, you may be better served spending your dollars at a fancier sit-down restaurant. It sounds as if you may not be a big eater if you don’t typically order appetizers or desserts. So, maybe a buffet isn’t a good fit for you in general?
However, I agree with previous posters in that AKL is beautiful, serene, and completely worth the time. And, like others, I would recommend going to Jiko if you would like to try flavors from Africa. DH and I had a wonderful date-night meal there several years ago. We both generally love to try new things, so that added a special element to it. Whatever you decide, I hope you have an amazing experience!