I do indeed have a very sensitive smeller for some things (I can smell the alarm pheromone in my bee hives VERY strongly, for instance), and prefer a level of scent in soap I suspect most people wouldn't smell at all. That said, if you cannot smell it, you didn't add enough for you.
If you don't have a small scale for EOs, get one, they are inexpensive. I wouldn't dismiss FO's out of hand. Essential oils are distilled from plant material, and can contain quite a few irritants that synthetic fragrances will not. They also smell different, of course, but tend to be expensive to very expensive in comparison to FOs, and since I assume you don't intend anyone to ingest your soap, the synthetic fragrances will not enter your body at more than tiny trace amounts.
And finally, some EOs just won't stay, they either get destroyed by the lye during saponification or evaporate -- citrus scents are well known for this behavior.