Is it just me or is MDE more broken than usual lately?

We were just in the World last week, and all week we had constant problems with FastPasses not showing up, errors when trying to book new ones, park maps not loading, wait times not loading and most annoyingly, our PhotoPass photos take days to show up. In fact, a set of really important to us photos is completely missing. It's been very frustrating!

Matt
Same issues this month. Some photos showed up immediately and others took days. And I'm not talking just the magic shots that always take a bit longer. Just general pics. FP availability there, gone, there, gone all in the space of a minute. Way worse than usual.
 
Great investigative work @fractalpotato...very interesting info. It’s not just MDE that’s had problems, DCL’s system has been wonky, too. Logged on at midnight Sunday to do on-line check-in and try to secure Palo ressies. Without going into details, it wouldn’t work. Some people had no problems, some people like me couldn’t even check in; we kept getting “service is temporarily down.” It is a COMPLETE mess; it is an embarrassment; and, it needs to be addressed.
 
Great investigative work @fractalpotato...very interesting info. It’s not just MDE that’s had problems, DCL’s system has been wonky, too. Logged on at midnight Sunday to do on-line check-in and try to secure Palo ressies. Without going into details, it wouldn’t work. Some people had no problems, some people like me couldn’t even check in; we kept getting “service is temporarily down.” It is a COMPLETE mess; it is an embarrassment; and, it needs to be addressed.

Thanks! My guess is that the systems were built by the same teams, and probably contracted out. There's likely no way to really address it without a ground up rebuild of the system, which will cost Disney money, especially if it's an outside firm. Until it gets bad enough that it costs them more money, or it gets old enough to get replaced by the next generation, we're stuck with it. Given the rising attendance, prices and paid extras, it is probably plenty cost effective for Disney to leave it broken for the forseeable future :(. Yet another unfortunate result of Igers Disney, willing to liquidate all the goodwill and reputation for profit.

Matt
 
@fractalpotato - Such a shame :sad2: I can’t imagine it can go on much longer (though I’m sure it will). It has to be costing them $$ in IT resources when FPs “disappear” prior to a trip, CM resources for Guest Services plus the possible losses associated with a Guest standing in line waiting for GS, Guest recovery, and then the ill-will and distrust of the system it creates. They are likely going to have an influx of new park visitors once SW:GE opens...they need to get their stuff together.
 
It seems everything IT related. We checked in lat Friday night and wanted to check our MBs were activated but the system at the Swan hotel was down, I think it was Monday you couldn't charge to your MB in EPCOT (which didn't apply to us), we were at AK on Sunday and the photos still aren't showing up, sometimes I couldn't check my FPs or dining etc, I had one FP 2x at EPCOT for a total of 4 Fps showing up which worked in our favor because we used the earlier time. and just other little glitches. We got home last night and I am already thinking about next year. Tried to log onto the Disney site to price out packages and couldn't log on. I agree that it is an embarrassment but don't see it changing.
 
It seems everything IT related.
First, I guess I should point out that Disney has an amazingly complex IT environment. When we stop and think about what they can do for a moment, you have to admit, it is pretty amazing. Scheduling times for attractions, users updating those multiple times throughout the day, and then automatically replacing your fast pass with another one when a ride is down. Integrating hotel and dining reservations, pictures, and those ride times all into a single app, yeah, I guess I have to give Disney some credit there.

Lighting up basically their entire property for Wifi is also no easy task.

On the flip side, there has been a lot of press about what Disney has done with their IT in the past few years. I wonder if that has anything to do with some of the bad coding practices that Matt mentioned above. I too am in IT, and while I am more on the network end, I have had to explain to many developers issues they will encounter when attempting to send a lot of data over a long fat pipe (which is essentially what the internet is). As Matt mentioned, caching is the way to go, and hopefully as they continue to revise their app, they will figure that out. The less data they need to send back and forth, the faster the app will appear to perform. Also the less network utilization they will use, the more other traffic they can handle on their Aruba Wifi.
 
First, I guess I should point out that Disney has an amazingly complex IT environment. When we stop and think about what they can do for a moment, you have to admit, it is pretty amazing. Scheduling times for attractions, users updating those multiple times throughout the day, and then automatically replacing your fast pass with another one when a ride is down. Integrating hotel and dining reservations, pictures, and those ride times all into a single app, yeah, I guess I have to give Disney some credit there.

Lighting up basically their entire property for Wifi is also no easy task.

On the flip side, there has been a lot of press about what Disney has done with their IT in the past few years. I wonder if that has anything to do with some of the bad coding practices that Matt mentioned above. I too am in IT, and while I am more on the network end, I have had to explain to many developers issues they will encounter when attempting to send a lot of data over a long fat pipe (which is essentially what the internet is). As Matt mentioned, caching is the way to go, and hopefully as they continue to revise their app, they will figure that out. The less data they need to send back and forth, the faster the app will appear to perform. Also the less network utilization they will use, the more other traffic they can handle on their Aruba Wifi.
It's been 5 years. They should have it figured out by now.
 
They just don't have the $$ to outlay for better IT.

:ssst:

They have it, they won't spend it because they don't need to. Frankly, from a checkbook perspective it will probably cost them more to fix it than leave it broken. Most users will just stop at the first "all fastpasses are distributed" message. They don't call, they don't complain (enough), they don't know any better. They're the people that pay the most because they book rack rate packages and just don't know any better. As long as they outnumber the fans like us at the rates they do and keep paying, it doesn't matter at all.

As far as recognizing the complexity, I can't go in to the details but I work on systems that are substantially more complex, with much, much tighter requirements. The kind of outages Disney regularly has would literally end our company. They've no excuse, what they're doing is not that complicated and they have the resources. What they're doing is old hat by internet device standards frankly, I give them very little credit.

Matt
 
Wow, I didn't expect such a response to my rant post!

So, I've done a lot of research on the problems, mostly by doing something called a 'man-in-the-middle' attack. Attack is a bit of a misnomer, it's called such because it's often used for nefarious purposes, but really all it means is that I set up my wifi to run through a special piece of software on my laptop. I then configured my phone (and laptop web browser) to trust that software. This allows me to see the otherwise encrypted traffic being sent between the MDE app/web browser and Disney's server.

From there the reverse engineering began, a lot of stuff has really obvious naming schemes, so you can infer what's going on by reading the commands. In addition, I did a lot of things like search for a dining reservation or FastPass, then took notes on what commands were exchanged. This allowed me to start writing my own commands and testing the response. I learned a few things, primarily that their web service (the software that fields requests to MDE) appears to be awful. Every single time the app asks for something simple or benign, like "What FastPasses do I have?" the web service responds with a ton of excess information. Not only does it send you the list of passes, it sends you the list of rides, the list of the rides operating hours, a lot of underlying internal identifiers for the rides, the GPS coordinates of the rides, encoded versions of the little pictures the app shows, all kinds of things. This, at a glance doesn't seem so bad, after all these are all things the app could need to show you. The problem is, it sends it all every time. Which is why it's so slow, and also why it tends to be heavy on data and battery usage! A properly designed application would send them once, if the requesting device indicated it had need for them. In an ideal case, your phone would know which rides it has data for and which it needs data for and could ask only for information that it needs. It can even say "I have information on Small World as of last week, is it up to date?" and only get data if it's stale. This is called caching and is the workhorse of the modern web. MDE seems to be designed to not take advantage of caching at all.

So that explains the slowness and heaviness of it all, but not why it doesn't tend to work well. For that, I also noticed many operations were done as a series of commands, however these commands were done without any indication of what step of the process they were on. I also noticed if you jumbled the order, it not only didn't work, but it tended to do wacky things, or just nothing, with no error message. The web service appears to rely on assumptions on the state of the users device and if something happens to make it go out of sync, it all falls to pieces. Another symptom of this is the way it will sporadically make you log in again, that's the server giving up and trying to get you to start over, but it tries to mask it by storing a cookie of what you were trying to do so it doesn't seem broken, but that often causes it to preserve the out of sync state that broke it in the first place. This explains a lot of the weirdness and why things like Incognito windows solve the problem. Private/Incognito windows automatically force your browser to start from square one and force the process to a known state. In addition, they don't preserve state so when you close and reopen them you start all over again. This mitigates the symptoms without solving the problem.

As far as the remaining problems, my theory for those is that the code is just poorly written, there is a lot of evidence of poor technique and practice in what I've looked at so far. In some cases, when I sent some requests in the wrong order by accident, very low level error messages that implied private information about the underlying software came spewing out. This is frankly terrifying, and if I did such a thing at my job I would likely be fired. All in all, the MDE software displays a startling lack of polish and technique and I'm very disappointed and surprised that Disney would put their name on something so poorly made. I would be absolutely embarrassed and appalled to call this my work and wouldn't even list it on a resume.

Matt

I just want to say Thank You for posting this! Not only is it great fo but you worded it in a way that it was completely understandable to us non-IT folks! It's discouraging to know it's not likely just a bug, but a systemic issue that likely won't have any resolution anytime soon. But at least we know now. Thank you!
 
I just want to say Thank You for posting this! Not only is it great fo but you worded it in a way that it was completely understandable to us non-IT folks! It's discouraging to know it's not likely just a bug, but a systemic issue that likely won't have any resolution anytime soon. But at least we know now. Thank you!
Thanks for the kind words. I'd like to clarify that the things I described aren't directly the cause of the more pervasive problems, they're just indicators of the overall poor quality of the system and it's general slowness. The data that disappears or doesn't sync properly isn't explained directly by my findings, but it stands to reason if the front facing things are written that poorly, the backing software is equally bad.

Matt
 
If I’m understanding fractalpotato correctly, I don’t think this will make a difference, but I just want to be sure... will web browser make any difference, specifically if I’m using Safari/Mac? I know some websites just don’t like macs and I need to go the Chrome route but I haven’t installed that yet on my new computer. Is Chrome the way to go to make ADRs or will I be faced with the same probability of issues with Safari vs. Chrome? Has anyone actually found better luck using iPads over desktops or are we feeling desktops are the more secure way to go here? I mean, dining with princesses for 4 year olds is putting A LOT at stake.....
 
If I’m understanding fractalpotato correctly, I don’t think this will make a difference, but I just want to be sure... will web browser make any difference, specifically if I’m using Safari/Mac? I know some websites just don’t like macs and I need to go the Chrome route but I haven’t installed that yet on my new computer. Is Chrome the way to go to make ADRs or will I be faced with the same probability of issues with Safari vs. Chrome? Has anyone actually found better luck using iPads over desktops or are we feeling desktops are the more secure way to go here? I mean, dining with princesses for 4 year olds is putting A LOT at stake.....

Ironically, in this case, Chrome is actually a bit worse. Due to the fact that the web site depends on the users browsers to maintain the state entirely, either intentionally or by broken design, it's susceptible to slight differences in how those browsers do it. I would guess they've done minimal if any testing with Chrome as it isn't a standard OS browser for Mac/Windows (ChromeOS is rarely officially supported), and as a result, it occasionally fails due to weird quirks. These can be solved using incognito mode (gives you a clean slate) but Safari or Firefox probably works better out of the box.

Matt
 
Totally agree with you there. Our last day in the parks the app decided that DH had no FP for the day, only I did and of course the FPs we had were for rides that had zero fps left.

Thankfully Concierge was able to contact IT for me to get it figured out but that was very stressful. There's enough going on in a Disney day that you don't need to worry about FPs disappearing!
 
You're not alone.... I'm 24 days out and hadn't received any shipment notification for my magic bands. They weren't showing up on MDE (since late January - even before my 180) and I'd called IT several times. They assured me the customization was OK and that they'd ship out. Well, I called back on Monday and they hadn't shipped and weren't even slated to ship out. The IT person had to manually override the system to get them out. I do finally have a tracking number and they should arrive by Saturday, but it was a PITA to deal with.

i have 2 rooms and while one set of bands shipped it took 2 weeks for 2nd set for other room to ship. i also called twice and they just kept saying it was all set and should ship soon. after my 2nd call the next day it had a tracking number
 
Thanks! My guess is that the systems were built by the same teams, and probably contracted out. There's likely no way to really address it without a ground up rebuild of the system, which will cost Disney money, especially if it's an outside firm. Until it gets bad enough that it costs them more money, or it gets old enough to get replaced by the next generation, we're stuck with it. Given the rising attendance, prices and paid extras, it is probably plenty cost effective for Disney to leave it broken for the forseeable future :(. Yet another unfortunate result of Igers Disney, willing to liquidate all the goodwill and reputation for profit.

Matt

Assuming Disney is like almost all other large corporations, they are so entangled in legacy systems that it's near impossible to truly "fix" things, unless as you stated they build from the ground up and that will likely never happen. I've actually wondered if they aren't implemented some infrastructure changes over the last several weeks which are contributing to the higher than normal performance issues people are noting. Every band aid solution they put in place will cause a hiccup in their production.
 
It's been on and off the past few weeks I've noticed, but it seems to be all of Disney's sites when it happens.
 

GET A DISNEY VACATION QUOTE

Dreams Unlimited Travel is committed to providing you with the very best vacation planning experience possible. Our Vacation Planners are experts and will share their honest advice to help you have a magical vacation.

Let us help you with your next Disney Vacation!











facebook twitter
Top