On 95% of the users I will promise it's just hit or miss if the site is slow for you. If you have people that are pulling a lot of threads with a lot of pictures all at once, if you just flat out have lots of users clicking stuff at the same time, you WILL get slow response times. To those that say 'you never have speed problems' you are either VERY lucky or check at lower traffic times of days. I can have days where it is slow all day, I just give up. Could be a combo of traffic between me and the DIS, the DIS itself etc etc. This will be at different locations, computers from old to new, OS's from xp, to vista to win7, IE, chrome, firefox etc etc. It will usually clear up in a few minutes, sometimes it takes longer, sometimes you get a 'done blown up' error (different then just a browser error).
I've checked out some sites that the web masters here go to dedicated to getting BIG vbulletin sites to run as quickly as possible. Knowing the number of views they get, plus some of the businesses they also run, seeing some of the hardware they say they are running on, they do sink some cash into this site, but you still cant be bullet proof.
While cleaning you cache and cookies and stuff sometimes isnt a bad idea, it's not normally the underlying cause of the site running slow.