Purely as devil's advocate I would say the following. First, my MIL is infamous on her own side of the family for laying the blame for any infraction, real or simply perceived on her part, on the spouse with no blood ties. There is currently quite the dustup she created out of whole cloth and blames on her 80-something year old SIL because MIL is positive her 90-year old brother is only agreeing with SIL on something because he has to, not because the feelings he expresses are his own.
Second, it may be that your brother no longer shares the affinity for the volume of family occasions that you, your sister and parents seem to. It sounds like he's the parent of two young children, something that is taxing even for parents who do love their children. Those circumstances also apply to his wife. Maybe in the thick of parenting two young children they're feeling overwhelmed and in need of more "me" time. (I get that you have a newborn and four other children, but I beg of you not to even allow thoughts of you having several children and still making time for extended family. Not all people find the grind of parenthood so easy to roll with. I can guarantee you that I wouldn't have time on a discussion board when either of mine were newborns. That's not trying to take a dig at you, but to point out some people have different needs or even struggles that they may not care to share.)
Third, your last sentence might merit some thought on your part. Are you being a bit irrationally angry over a minor issue? Maybe consider whether there's any room to believe your brother might think it's easier to make a white lie than have to answer a laundry list of rebuttals from you about why they really should come to the party, how much they'll be missed, how much it would mean to your daughter, how much fun the kids would have playing together, etc? Yes, I know you say hey, we can't make it is enough, but the fact you've come up with a counterpoint to so many of the points raised in this discussion make me question your sincerity. That doesn't mean you're a bad person or a selfish person either. It's totally legit that you would simply miss them being there and sincerely wish your nephew could be there to play with his cousins and be disappointed it's not happening. Of course we want to see the people we love as often as we can.