Yep, those soaps shouldn't be brown unless your fragrance is the culprit. But, oh, gosh, don't throw the soaps away!!! There's probably nothing wrong with them that an "attitude adjustment" won't cure. I mean this as a gentle tease, but there's truth in it -- sometimes we get locked into one way of viewing a situation and later find out the situation is not at all the way we first perceive it to be.
edit: Okay, just read your last post and I'm confused -- you explained the soap was brown before adding the fragrance. Can you explain your method of making your soap, please?