I have no problems with combining my soap making dishes with my food dishes. They are always scraped/wiped out so as not to add to any oily gunk in the dishwasher. As far as I am concerned, oils are not poisonous, and neither is lye ( it's used to make bagels and pretzels).
The fragrance sometimes does linger, but it doesn't contaminate glasses, cutlery or plates. I keep separate silicone spatulas for my soap making because I used to use the household ones, and they do absorb the FOs so end up being a bit too perfume-y for food preparation.