I'm confused -- Is this a question about melt and pour soap? Or about a batch of liquid soap you made with KOH?
edit --
Okay, I figured out what you were talking about. Give some thought next time about posting a liquid soap question in the liquid soap forum. I appreciate this is a beginner question, but the vast majority of beginners are making NaOH soap. KOH soap questions in the Beginner forum don't always get answered -- you're more likely to get answers if you post LS questions in the LS forum.
It's not necessarily oil. Glycerin and liquid soap can feel greasy too.
Rub a thin layer of the liquid onto your fingertips and then rinse your fingertips in cold water without rubbing the skin. If the film rinses off fairly clean, it's probably soap. If it doesn't rinse off and leaves a fatty film on the skin, you're right that it's fat.
In any case, my advice is the same -- Stir the liquidy part into the rest of the paste. This happens to me from time to time as well. As long as your soap passes a zap test, this kind of separation is not a big deal.