Being from NJ, most of our trips have been during NJ week. The crowds have gotten heavier over the years, but I've found them to be manageable, especially if you stick to the advice of the experts and get to the parks early. Keeping in mind that most NJ schools only have off on Thursday and Friday of that week, the crowds are lighter earlier in the week. Weather-wise, I've found this time of November to be ideal. With the exception of one trip when we hit a bit of a cold spell for part of our trip (but not all of the week-long visit), it's pleasantly warm during the day and cooler in the evenings. Also, Food & Wine is still going on during this time. A definite "pro". I would say that the one "con" is that you must be prompt about reservations, etc., in November. I'm not sure why, but I've had a much harder time getting the dinner reservations and fast-passes I wanted Jersey Week.
Last year, because of a conflict with my daughter's schedule, August was the only time we could go. The temperatures in August definitely required adjustments in our touring. We built in many breaks, ate indoors to cool off whenever possible, and carried water bottles everywhere we went. We also carried cooling towels. Although the comfort-level was not the same in August as in November, we still really enjoyed the trip and actually liked the extra pool time that we planned - we generally didn't spend as much time at the pool in November. The August trip simply required some extra preparation. The rain wasn't really an issue. It went through quickly and we had our ponchos ready to go. If you go in August, I found a few YouTube videos on how to handle the heat while touring and they helped us prepare.