Let some other people smell it. I am one of those who often detects a "play-dough" scent under vanilla scents, but a lot of people don't. I find it particularly strong in BB's OMH Cybilla, for example, but lots of other people don't smell it and love the scent.
Another example, Energy (also by BB) smells disgusting to me and is also widely beloved. OTOH - I like the scent of Ylang Ylang and many don't. You will probably find somebody who will like your yucky soap.
Good idea. I made a batch of spiced mahogany, with fresh coffee grounds, colored with coco powder. I love the FO OOB, but during soaping, mixed with the coffee and cocoa scents, I thought it smelled like a dead animal rotting in a moldy basement. However, most people who have sniffed through my "give away" basket, pick that one above all others. One person tells me every day how good her shower now smells.