If you can drop the "cleansing value" of the soap, this effectively means you are reducing the amount of oils removed from your skin ... squeaky clean skin usually means your skin has been over-cleansed.
Young soap is also a lot more drying than well-aged soap; the soap needs to cure for a minimum of 4 weeks (irrespective of whether it is CP or HP), and sometimes longer (depending on the oils and additives used - soaps high in olive oil and salt bars are examples of two soaps that need to age for many months, not weeks)
For an ordinary soap, reducing the oils that contain the shorter fatty acids (like lauric acid, which is found in coconut, babassu and palm kernal oils) and increasing the superfat slightly can ease that drying effect. Decreasing the amount of soap that you use can also help. All of these suggestions are to reduce the amount of your natural oils that you are removing in one wash - you can still get clean without stripping your skin
.
Sometimes it also takes a few batches to work out what soaping oils suit your skin best.
I personally add (unsaponifiable) ingredients that have a moisturizing effect when the soap is used, but they take some getting used to (they feel different to soaps that just clean the dirt and oils from the skin) - I think DeeAnna mentions lanolin in the thread artemis linked to above - this would be an example of an ingredient that has this effect.
Winter/cold weather dryness can be another issue - some people also formulate different recipes for winter and summer soaps (I don't do this myself, so hopefully some of the people who do might chime in to help you).
Good luck!