I've never done it, but I think you'd have a mess on your shower floor as the soap dissolves away from the flakes. When it goes down the drain, it can get lodged in other gunk and partially block your drain. I stay away from a lot of solid additives for this reason.
One option to minimize this might be to very finally chop it with a food processor before adding just a small amount to your batter (this would mostly be for label appeal). Another is to confine your use of the flakes to a small part of the top of the soap (such as an edge or the ridgeline), to get a decorative effect without too mess.