We've done both. I think they are totally different. IMHO, HHI is not a great beach destination, but it is a great family-friendly resort. There are plentiful activities, tons of wildlife, bike rentals and bike paths, and lots of shopping/dining/movies/etc. in the surrounding area and the beach is gentle enough for small children. There are lots of recreational activities. There are two pools.
VB is very much a beach resort. The ocean is visible from the balcony and the waves get wild enough to surf. There is an almost immediate, sharp drop-off, heavy shell deposits and very strong waves - which will likely make the water too much for smallish children, but heaven for older kids who can body surf/skim board/boogie board and adults. There is also a pool. There's not much to do in the surrounding area.
I prefer VB because I like my beaches wild and natural and I just want to spend all my time walking or sitting by the ocean, but HHI may be better for families with small kids/golf/shopping.