On your next batch you could try to outsmart the brown, like I did. I divided up my soap, by pouring half the batter, unscented, into the mold.
The other portion I added oatmeal, milk and honey fragrance, blended gently (did this quickly so the batter wouldn't become too thick ) and then pured that into the mold. I then, mixed ligthly in the mold and smoothed the top. The soap gelled and became very dark looking, but once complete, the soap was a very light, light tan, compared to the dark, dark brown sugar color of the batch I fully scented the day before.
Right now, the much lighter batch smells very pungent.