CX-5 Battery and lights dimming/flickering - need advice

My 2014 CX-5 with 41K on it I am noticing the battery going. My father inlaw looked at it a couple months ago and said the charged seemed a bit low. the dealer looked at it about a month ago and said the battery and alternator are fine, but its possible a cell in the battery is going. I have been noticing lately, it is working a bit harder to start, and the interior lights flicker a bit. Notice it when I get into the car when its dark. The lights above the rear view mirror that come on when the door is open. I also notice when I am putting the windows down and up the head lights dim a bit. So the question is this all battery related? Also the dealer wants $165 for the battery and install. I was going to go to the parts store like Advanced or something get a decent battery and have them install it for free.

Could there be other issues? I am not noticing other lights flickering, etc.

Go to Autozone or other parts store and have them do a load test on the battery. That will tell you for sure if that's it. They will install the battery for free and it will cost less than the dealer too.

Thank you. Doing that now. Is it possible for the alternator to be going at only 41k

Got a new battery, everything checks out now including the alternator. Need till wait till dark regarding the light's