Here is a video I made when we were there in August of last year.... It's my family's favorite place to stay (mine is the contemporary or BLT!)
We have similar questions when we are planning to go, our compromise is to do a split stay. We stay at a hotel that is closer to a particular park that will allow us to focus on that area for a few days and then move over to where we really want to be. While some people think moving is a hassle, we don't mind it as the trade off is that we get the best locations for what it is we are focusing on.
The question of 'worth' is always personal. Is any hotel worth $400+ a night? lol... But I will say this, although I prefer the location of the contemporary and BLT, the Poly is the only hotel that truly feels like 'home' to my family. So 'worth' will all depend on your outlook. As above I have used TA's to get good rates and this last time in July we got the best rates I have ever ever had and we are FL Residents with AP's! So it's worth shopping around.
The benefits of the Poly for my family (yours may differ)
1. Location location location! - Proximity to MK, being on the monorail and close to TTC.
2. Main Loby - very welcoming and a cool place to hang out
3. Amenities - Great pools and food access and lots of options for shops in the main building
4. Stroller access - if you have young kids this is a great location for people with strollers as you can simply get up to the monorail and go straight to MK, or walk to the TTC without ever having to fold it up.
5. Being able to watch and listen to the Happily Ever After fireworks show from the beach (you can see a little of this at the end of the video)
If it were me and it was my money I would not cut the vacation short AND spend $1000 more, if it were a similar price but perhaps $500 extra with the same amount of days then it would be a no brainer. As I said earlier, I would look to see if you can do a split stay for the same amount of time and keep the prices relatively similar. It's definitely possible as its something we do a couple of times a year!