eMusic ($)Emusic has transformed quite a bit over the years. For a while it was about .25 per track, which was an extremely economical way to scoop up some good music on "independent" labels. Now it's about .50 per track or $6 per album, and the selection is extremely broad -- thus, it's still a decent alternative to iTunes if you buy a lot of music each month. I'd still recommend it to the typical music student.
ArchivMusic ($$)Archivmusic is primarily a mail-order CD and DVD company that specializes in Classical music. Why would you use these guys over Amazon, you might ask? Well, they've got the best-organized catalog on the web. This is the first place to go if you want to find out if a certain piece is available on CD. AND, they offer a lot of out-of-print recordings as part of their "ArchivCD" series.
Ariama ($$)I am currently pretty excited about this all-classical downloads service, because it offers recordings in the FLAC format (which is comparable in quality to CDs.)