In my experience, a castile bar (100% olive) will have an olive oil scent. It's not bad (to my nose, anyway), and it does fade some, but it is there for a long time. Neem does not fade.
I am clearly the only one that doesn't mind the smell of making beer soap, I see. I never used sal butter but shea butter does fade. Babassu doesn't have as much of a scent, which is good. I really did think you were talking about a fragrance blend though and was wondering what you blended together.