Prices of components vary enormously, so please treat the following lists as a guide and shop around.
I know some people who buy almost everything on eBay. But beware. Though things are often very cheap there, occasionally they are much more expensive than at other suppliers.
I have listed part codes for the tools, modules, and so on from SparkFun and Adafruit, as these suppliers are very accessible to hobbyists and also provide good accompanying documentation. They also have distributors throughout the world, so you do not have to buy direct from either company if you live outside the U.S.
For other components, I have tried to list product codes for Mouser and DigiKey since these predominate as suppliers to hobbyists in the U.S., ...