Every family is different.
A few years ago I went with a large group which included a dear friend of mine and her two small children, her brother, his dear friend and their small children, plus a few assorted others. 'Nanny' was invited along, primarily to look after one family's children, but also assisted with the other (total 1 baby and 2 small children)
There were times on the trip when Nanny came along; other times she didn't. We had dinner in smaller groups, as well as the entire group.
Every family is different, as I stated above. Often cultural aspects come into play as well. We had 3 different nationalities in the group, not including Nanny. One was a family from Asia, another an expat family living in Asia. Plus a German... very mixed group.
Having spent many years as a nanny myself, I considered it a mark of respect from the family when they invited me along to intimate gatherings or vacations. I learned a lot about the Jewish tradition from one family, who included me in their holiday celebrations.
Every family is different.... wait, did I already say that?!