While I don't do it 100% of the time, I rarely remove the card from the camera. Plug USB cable into camera and computer, turn on camera and a new hard drive shows up. Move (not copy) photos to computer, turn camera off, unplug. Card ready for another round.
Saves wear and tear on the removable disk, eliminates any chance of bending pins on the card socket, or doing something the manual says you shouldn't do.
Unless you're on some long trip where you need to swap disks, it's pretty hard to fill up a 4gb card over a weekend and I shoot RAW + lightest JPG compression so I run about 25 mb aggregate per image. But, even that contingency is covered by dumping to a laptop or external pocket hard drive.
Good advice on leaving the card in the camera as much as possible, using the mini-USB cable to connect to the PC.
SD cards do seem quite fragile, especially the ones that snap in half to make a USB connector.
Keep in mind that the SDHC cards, which look and feel exactly like SD cards, require a special USB card reader. Fortunately, one came with one of the cards I bought, otherwise I'd have to use the camera exclusively to load photos to the 'puter. I swap cards fairly often because of the volume of photos I take (I took 250 of my dog playing with another dog one evening), especially when I'm on a walkabout (1,500 in Yosemite and the Bay Area earlier this month). I keep them in a
LowePro hard case to store them when I'm on the trail. It holds up to 8 cards.