We're from the Vancouver area, and have made many trips to Vancouver Island. Victoria is indeed a lovely city, and if you have a car, you could also explore Parksville (2 hours drive), as well as Long Beach (4 hours drive). If, however, time is limited, you can definitely spend all your time enjoying Victoria itself!
The "Grand Californian" or "Grand Floridian" of Victoria is The Fairmont Empress hotel. It is
*the* iconic Victoria hotel, dating back to the early 1900's. It's the hotel you see in all those glossy brochures about Victoria, and even has its own
Wikipedia entry! Expensive, but right in the middle of the Inner Harbour area, walkable to many sights.
Five minutes walk from The Empress is The Parkside Hotel and Spa, where we stayed earlier this year. Much more affordable, full kitchenettes (useful for leftovers!), and still within walking distance of the Inner Harbour. No air conditioning, but we had a nice balcony, a nice view, and good air circulation!
There's also a Marriott between the Empress and the Parkside, where we've also stayed -- a reliable choice. I'll attach a map so you see how close everything is: the ferry from Port Angeles if you choose to come from Washington state, the three hotels, the BC Museum, the Legislature... One of the main walkable tourist areas is on Government St. (the north/south street on the waterfront) just a block north of the top of this image.
Air conditioning could be an issue depending on when you come exactly, as well as the construction of the hotel you decide to stay in. The Empress and Marriott will be fine; I have had friends stay at the Parkside during the summer, and I haven't heard any complaints. Some smaller hotels might have problems if you arrive during an especially warm spell. June can get warm, but it's later in the summer when temperatures start to rise more.
Let me know if you have more questions -- I really enjoy sharing the things we've learned from our vacation experiences!