Wholesale Supplies Plus is good. They're going to have an inventory reduction sale at the end of this month. I really like them because they have a HUGE variety and good prices. The downside is the variety is SO huge (on things like fragrance oils) that it's hard to narrow it down! They have molds, FOs, EOs, lotion supplies, etc etc.
If you look on the Fragrance subforum, there is a sticky with a link to the Fragrance Review Chart. Members of SMF post their reviews of fragrance oils. It's organized by supplier.
I also really like Brambleberry. They do a lot of testing on their FOs. they post pics of CP (cold process) soaps made with their FOs and report honestly on acceleration, discoloration, etc. They don't have quite the variety of WSP (and also not free shipping), but I really like them.
For natural supplies (EOs, herbs, floral wax) I like Camden Grey. They are in Florida so their shipping to the Southeastern states is pretty fast and cheap. They are my "middle" source for fixed oils (sweet almond, etc). Better than BB and WSP, but not as good as Columbus Foods.
Columbus Foods - aka Soaper's Choice. Fixed oils. Coconut, lard, tallow, avocado, sweet almond. The smallest quantity is a gallon for most oils.
TKB Trading. My favorite source for colors. Most suppliers also sell colors, but TKB has some of the best prices.
Mad Oils - fragrances and colors. I've only used 1 FO I ordered from them so far, but I really like it. I have heard good things about their colors too.
Nurture Soap Supplies - molds, colors and fragrances. I like their colors. I have a few of their FOs but they haven't really blown me away. Their molds have gotten good reviews here.
American Soap Supplies. Home of the original Tall Skinny mold. I love my Tall Skinny molds. Got one FO from them that I'm not thrilled with.
Nature's Garden Soap and Candle. Love their FOs, there colors are pretty good too. They also have wicks, some containers, etc.
SKS Bottle. This is where I get my lipbalm tubes. Best price I've found, if you order 1500. Which is a ludicrous amount, but I give away several hundred tubes of lip balm over the Christmas season as presents. (PS, if you decide you want to make lip balm, get a lip balm filling tray. CHANGED MY LIFE.)
Lye - I googled chemical supply company and bought locally. But you may find lye in your hardware store. It will be with the drain cleaner. Read the ingredients and make sure it is LYE and not something else. The Lye Guy gets good reviews here. Essential Depot gets very mixed reviews here. You can also get lye from Brambleberry and WSP. They sell soap making kits which include everything you need if you just want to try it without investing a lot of stuff.
Equipment - Food scale. (not a postal scale). I've gotten mine for a good price at Bargain Hunt. Silicone spatulas - Big Lots. Bowls and pitchers - look for ones with a 5 or a 2 on the bottom. Soap making pot - stainless steel, take a magnet to check. I've seen them at Fred's Discount Pharmacy. You don't need a good pot - the ones I use have a bottom so thin that anything else I cook in them burns. They are only good for boiling water for pasta or something, or making soap.