Sea Port Village
The Gas Light District
La Jolla Cove
Wild Animal Park
The USS Midway Museum (my former residence)
Torrey Pines Glider Port
45 miles east on I-8 is Sunrise Highway going up to Mt. Laguna. It is has impressive scenic stops to look down at the desert floor.
Also, about an 1.5 hours north of SD is the happiest place on earth (Disneyland).
I definitely recommend the Wild Animal Park, especially if your kids are into that stuff. You get a chance to be up close and personal with the animals.
I think you still might get some whale watching this time of year. The California Grey Whales should be wrapping up the spring migration back up north. Hopefully see some calfs. That is usually better up around Dana Point (about an hour north of SD).
We did the trolley bus tour last year. Its a cool way of seein the city. Going over the bridge was pretty scary but the girls loved it (Mum's the wimp in the family). It's one you can get off and onto the next one.