Not the CN Tower. Gastown is great.  Queen Street District is neat too. Poutine.  Thats all i got.

Check out the Sharp center for Design @ OCAD University

Toronto:

I say YES to the CN Tower - what a view!

City Hall

Chinatown - food

Little Italy on St. Clair West - food (College St. has become Little Lisbon)

Little Lisbon on College St. - food

Greektown on Danforth Ave. - food

Toronto Islands and the ferry boat ride

Casa Loma

"The Beaches"

Univ. of Toronto campus

Riding the TTC subway system

High Park

the zoo

That's all I can think of.  I find Montreal to have more, but Toronto is cool.

^ That all looks like a lot of walking.

I'm also open to suggestions for specific bars and/or restaurants that might be worth checking out, especially places with a good selection of craft beer and/or whisky, and with atmospheres conducive to conversation (i.e., no loud sports bars or tourist traps).

Thanks...

Get a TTC pass for several days.  Many listed items are on subway exits, or close to them.  If not, TO has streetcars which look like the older Muni cars in San Francisco.  As for the bars and restaurants, I can't remember their names, though excellent Canadian and foreign dishes of all kinds are available in TO.  Get the AAA/CAA guide, if you belong, and find the restaurants that way.  Also, I recommend Lonely Planet and Frommer's more budget oriented series.  Fodor's is for high rollers and their idea of 1 or 2 stars is actually higher than that for the general population.  If you walk out of the subway onto Danforth in Greektown, you will be hit in the face with numerous good Greek restaurants.

The only one which I think is a little further east is "The Beaches" but, again, a comprehensive TTC pass should get you there.  Yes, Toronto does require a bit of walking, as many exhibits are set back on expansive grounds from where you exit a subway or streetcar.

I visited Toronto many times(gf lived there) but that was a while ago... but I would love to see Gardiner Museum by KPMB (off Bloor street near Yorkville). If you are into hockey, then you can go to hockey hall of fame and see Calatrava's BCE place. Oh, St. Lawrence market is a cool place to visit, if you guys are foodies. It's very safe city to just walk around at night (or it WAS). 

Montreal > Toronto

Just walk around and get lost.

Check out the ROM 

CN Tower was cool but expensive. 

Little Italy was cool. 

Check out the Crayon Box at UT... I was walking around in the building for fun.

Also try a Poutine... especially if you are not a Canadian. 

Montreal isn't really an option on this trip, since the person I'm visiting lives in Toronto. I'll make it there someday, though.

Normally I'm all about walking... I must have walked well over 100 miles when I spent two weeks in London in 2011, and at least 5 miles in Paris in one afternoon during that same trip. Unfortunately this time, though, our walking distances will be limited because my host has a prosthetic leg that gets very uncomfortable in the summer, and I have an injured hip that gets very uncomfortable whenever I'm awake. It will be a contest to see which of us has the worst limp.

hope you had a good trip- if you didn't get a chance to leave the central downtown, for your next trip here are other suggestions: 

Museum: Art Gallery of Ontario (AGO) the Frank Gehry designed Stair and Gallery are beautiful 

Distillery District- El Catrin has amazing Mexican food

Queens West neighborhood- The Drake (spoken word night) and the Gladstone Hotels

Restaurant: Bar Isabel, one of Grant Van Gameren's restaurants. Mind blowing food

The Royal Ontario Museum 

Hockey Hall of Fame

Casa Loma

and for fun Canada's Wonderland (need a full day)