If you are asking the question, then I think you already know how you feel.
Mind, I'm not sure why someone who has never been at Christmas feels really qualified to comment on what it is like.
We have visited over all the major holidays quite a few times. IMO, it isn't the crowds, it is the high costs that you need to understand. Christmas is the MOST EXPENSIVE time of year to visit WDW. If we go, we usually do split stays, and keep our on-site WDW hotel stays short. Universal isn't quite as expensive. Off-property hotel prices are much more reasonable, especially the further you get from Orlando. Tampa to the west, maybe Daytona/St. Augustine to the east. If you are hoping for warm weather, go south towards Ft.Lauderdale/Miami/Naples.
More or less, if you want a relaxing experience, spend time away from the major theme parks. Personally, I feel silly paying $900/night to be at WL if I'm doing a non-park day. I'd rather pay $175 for a super nice place on the beach in Tampa. I find I'm more relaxed when I'm spending less money, but maybe that's just me.
Christmas also should = adjusted expectations. From 10am-9pm the parks will be very busy. We do our 3 FP rides, then spend much of the rest of the day doing quieter attractions. If you only go for headliners at WDW, you'll have to wait in lines. Another option is to go to Universal. Universal isn't quite as crazy as WDW, and they have some fun NYE options.
With the right attitude, Christmas at WDW can be fun, it just isn't the same as going other times of year. A unique part of visiting during Christmas is all the holiday decorations.
The other key thing to know- don't do dumb stuff. Don't go to MK on Christmas Day, because that's the worst of the worst. Think outside the box as much as possible. When does EVERYONE try to eat lunch? Avoid that time...and so forth.
If everyone is game for that kind of flexibility, you can all have fun. If you'll be dragging his around all week against his will....then it is a lot of money to throw away. If you think he won't be flexible, then I highly suggest limiting your WDW time to just one or two days. The rest of the week- take him to place that appeals to HIM. If he likes history, St. Augustine is amazing. If he likes the beach, FL has 1,000. Maybe he'd like NASA? Harry Potter? Great food? FL has a bountiful range of museums, aquariums, and natural areas. The Everglades are amazing to see. Would he enjoy seeing alligators? the Keys? What about an airboat ride? A cruise? The key to fun is finding what you'll both enjoy together.