Exactly---not only must everyone get time away, but it all must be at the same time.
We may be a little unusual, because our winter holidays are all quite scripted: Christmas eve open house here, Christmas dinner there, New Years Day open house at the other, and "Outlaw Christmas" somewhere between them. So, the one time everyone is more or less expected to be off of work, school, etc. is the family third rail---it is the one time each year that the extended families on both sides get together, and taking our brood out would be awkward at best.
Once my daughter is out, I'm not anticipating we will get things to line up the rest of the year between various colleges, jobs, internships, etc. I know I certainly didn't when I started college, and it only got worse as the other two started school and we all began moving through our careers. I could get lucky, but my experience is that it is very difficult.
I suppose we could work something out for a long weekend around Thanksgiving.