If your bar has any amount of water in it and you aren't finding any solid sugar crystals in it, then you haven't yet added enough sugar. You have to find the amount of sugar that will form a saturated solution in the liquid portion of your bar. As you add more sugar past that point, then some crystals won't dissolve. Until you get to that point, sugar simply won't stay in a solid form.
At room temperature, 179 grams of sugar can be dissolved in 100 mL (100 grams) of water. That's a ~lot~ of sugar, and may make your bar soft and weepy. A typical soap recipe contains about 20% water, give or take. For a batch weighing 1000 g of fat+lye+water, I guess-timate you'd need to add over 360 grams of sugar. The lye dissolved in the water might change the solubility, but this is a reasonable guess to give you an idea of what you're up against.
This solubility issue is one of the reasons why people make salt bars rather than sugar bars. The solubility of salt (NaCl) at room temp is much less -- about 36 grams of salt in 100 mL (100 grams) of water and somewhat less after the lye has been added to the water. If people do want to use sugar, they put it in an anhydrous (no water) product like a scrub or a product with a low amount of water.
Also, if a sugar bar behaves similarly to a salt bar, it won't be very exfoliating even if sugar crystals are present.
Edit: Since you posted in the beginners soap making forum, I assumed you are talking about adding sugar when making a regular lye-based soap, not about adding sugar to a melt-and-pour base or to a non-soap product. If you mean you're adding sugar to a M&P base, then it's still true that you have to add enough sugar to get past the solubility issue.