The low percentages of FOs and EOs used in soap, and the fact that they aren't on the skin for very long, make it hard to assess whether they will help with this.
My recommendation would be to make a more cleansing soap recipe, one that is higher in lauric and myristic acids. Those are found mostly in coconut oil, palm kernel oil (not palm oil), and babassu oil, and to a lesser extent, in tallow.
Another recommendation by which some folks swear is to add activated charcoal to your soap. This is known to have a "drawing" effect and has many anecdotal reports of helping with odor, acne, etc.