I buy Refined Olive Oil (not EVOO) at Sam's Club for $10.88 for 3 liters, which is the cheapest I've found without buying it by the drum and soaps just fine. The only drawback is if you don't have a Sam's Club membership.
You can get LouAna brand Coconut Oil at Walmart for $5.84 for 32 oz. About double what I pay in bulk per pound, but it gets the job done in a pinch. I actually just picked some up tonight because I'm jonesing and not ready to place my big order yet this month
You can get Spectrum brand Organic Sustainable Palm Oil (sold as Vegetable Shortening) in most health food stores. It's almost $7 for 24 oz here in Tampa. It soaps great, even if it is pricey. You could try a local Middle Eastern/Asian market for Vegetable Ghee made from Palm Oil, which would be the same thing. Some of them carry it in bulk. Just make sure it says Vegetable, otherwise it's made from milk (like butter).
A 6 oz bottle of Castor oil can be found in the pharmacy for around $3.
Lard and Tallow are available at just about any grocery store for varying prices.
You can get lye (make sure it says 100% Sodium Hydroxide!) from Ace Hardware (the only place in FL that carries small quantities of it behind the counter), but it's $7.99 for 16 oz. I usually buy my lye from Essential Depot...32lbs for $33 plus shipping = $52.
If you aren't making large quantities of soap and are just making it for yourself, then picking up supplies locally is just fine. If you decide to go large scale, bulk is the best option, and if you don't live locally to a supplier, that means incurring shipping costs, of course, but it still usually works out MUCH cheaper.