Welcome, Pat! The best way to help us help you is to post your entire recipe, including all additives, FOs, EOs, and lye solution percentage, as well as your process (CP, HP, etc.). A picture would really help, too!
I think I have experienced this, but a photo of what you are seeing would be helpful. I think mine might be because the fats in the milk were saponified by the lye and then turned brown because the lye solution heat up to the point that the milk discolored. I have no idea if I'm making sense.