The zoo is very spread out and a lot of walking, and to see many of the shows at Sea World you will need to spend the good part of a day there.
If you didn't have young children and didn't mind doing much of Sea World late at night in the summer, you could probably do it, but otherwise I think you would be shortchanging yourselves on both.