In liquid soap, anything that is a powder is very likely to settle out. If the powder is fine enough, it might be a long time before it settles, but I'd say it's still more likely to settle out than stay permanently suspended.
Another problem with adding starch to liquid soap is you're adding a source of food for microbes to the soap. Anything that adds carbohydrates or proteins -- aloe juice, milk, and your starch -- could cause trouble with bacterial or fungal growth. I don't recommend this.
I agree with Shari (shunt) that most fragrance in soap doesn't last. For one thing, it's not as concentrated as the fragrance in cologne. For another, a lot of the fragrance in soap washes off because of the nature of soap. IMO, if you want the fragrance to stay on the skin longer, make a cologne or lotion or other product that stays on the skin. Fragrances that last a little longer on the skin tend to be ones from woods and spices. Floral fragrances ... not so much.
Also, if you think about it -- a "fixative" such as a clay or starch is supposed to absorb the fragrance into its structure and release it more slowly into the open air. The idea of using a fixative to prolong the scent in soap makes theoretical sense, but, like Shari, I'm skeptical about this.
But assuming this idea actually does work, I'm not sure a fixative would help to transfer that fragrance to the skin. You need the fixative to stick to the skin, right? Or you need the fixative to instantly release the fragrance once the product is used for bathing. Is there any reason to think starch or clay actually does that?
IMO, the use of a fixative might work against your desire for more fragrance on the skin after washing. All those particles with all that fragrance are much more likely to rinse off the skin.