I do use olive oil, but like to vary. My skin doesn't really like super high OO in soaps (it becomes dry when I use them).
RBO soap makes my skin feel very soft and moisturized, although RBO has accelerated on me a few times (this might be dependent on the brand/batch, not everyone has this experience).
I personally like HO sunflower oil, it's the cheapest oil available to me, traces very slow and doesn't dry my hands (but it doesn't have the amazing skin feel of RBO either)
A lot of this is personal preference, so try different oils and see what you like.