I would just wander - bring your sketch pad - ditch the camera/phone - the best way to discover this city is to follow your intuition - just stay away from pier 39(strip mall by the bay)

@lietdownl some suggestions have been listed here: http://archinect.com/forum/thread/32514347/architecture-in-san-francisco

also check this for restuarants/bars: http://tiplist.com/united-states/san-francisco-21591/

One more thing - watch out for cyclists - they run red lights and they will hit you - I almost got dropped from one of my PAs who was in a hurry to get to his office

House of Shields. 

that herzog-n-demueron copper thingy (and don'tcha dare miss the turrell skyspace on the west side sculpture garden area please don't miss it!)

while you're in the park check out the science center bumpy roof opposite the copper thingy.  the japanese garden is sort of okay but maybe not worth the price of admission.

and the park is next door to h8 ash-burry and you're also close to the golden bridge so you can hit that up ('tis always cold walkin' out there ..)

another day you can hit up the museums close to AT&T park and walk on that crazy bridge in the rossi museum.  over there you're close to the financial district and transamerpyramidal building and then go the chinatown and finish at PIER 39 you gotta go there!  ghiradelli square and fisherman's wharf too omg!

or just walk around aimlessly and you might find some good stuff.

San Francisco City Hall, by Arthur Brown Jr.

Palace of Fine Arts, Bernard Maybeck

Just spend some time walking some of the lovely neighborhoods there.

I would say the tourist-trap Pier 39 is still a very interesting area, even if for pure kitsch value.

Academy of Sciences - Renzo Piano

De Young Museum - Herzog & de Meuron (both the Piano and H&dM buildings are in Golden Gate Park, a great area!)

Federal Building - Thom Mayne

Saint Mary's Cathedral - Pier Luigi Nervi and Belluschi

There is also a small FLW museum building that is near Union Square.

And if you want to get out of the city for a little bit, there is the Marin Civic Center by FLW just north of the Golden Gate Bridge (the one that inspired the design of Naboo in Star Wars Phantom Menace ;)

Academy of Sciences + De Young is highly advised.

The earthquake show in the planetarium of Academy of Sciences is pretty cool.

william stout bookstore - just leave your cc in the car. just do. 

it's hard not to recommend the gg park, just to see the piano and hdm projects. 

i'd skip the morphosis building - it's not exactly tourist friendly, so you'd mostly get to see the street view. 

anything by maybeck's a plus. there's more of his stuff out near berkeley (maybe too long of a drive/train for just a couple days). 

finally, i've got a ballpark thing, so... just checked. they're in town through sunday; out of town all next week. might not win on that one...

Shalimar Restaurant- On Jones south of Geary, bad neighborhood, and it doesn't exactly look super hygienic, but the BEST PAKISTANI FOOD EVER, or at least pretty damn close.

awesome, thanks everyone and keep it coming...  

greg... im actually visiting my brother who who lives in oakland... so we will be doing the ballpark thing... only, the ballpark will the Colosseum... 

"i'd skip the morphosis building - it's not exactly tourist friendly, so you'd mostly get to see the street view. "

I made the mistake of getting too close and the  Federal protection Service goon squad threw me onto the sidewalk  and took my camera.

The large numbers of homeless begging for money, then insulting you when you ignore them, and the never ending stench of urine is also something you will get to experience in SF.

http://www.stoutbooks.com/cgi-bin/stoutbooks.cgi/index.html

I met Steven Holl here when he dropped by last year supporting his Vanke Center book

Often undermined building, don't miss Yerba Buena Center for the Arts by Fumihiko Maki across from modern art / GAP museum and note the use of super clear glass by Maki. 

must admit even though kubo and FRaC mention some of same buildings prefer the later to the former.

also the william stout bookstore is a must i agree... and I particularly enjoyed a trip around Mare Island and Skaggs Island part of San Pablo Bay National Wildlife Refuge, site of an old navy intelligence post when i last visited.