If you can break the layers apart easily, you can re-melt the colors separately. If you tried to swirl each layer, this will be really difficult, and you likely won't get the design you originally wanted in your head. But, if you can separate the colors, pour a thin layer on the bottom, wait for it to harden, then spray it, lightly score it with a skewer or toothpick. Then spray again and pour the next layer. When it's tough enough to take a scoring, score lightly again after spraying (Spray, Score, Spray) and then pour the next layer.
I tried to swirl layers and ended up having to try to break apart the white from the red Moroccan clay I was trying to use, and ended up having to color the white part with alkanet root infusion. But it DID melt, and I DID get a neat look. It wasn't the one I was going for, but it was still pretty.
EDIT: Lightly. I went a little nuts with mine, and through the bottom of the layers as they get used up, you can kind of see the hatch-marks I made in the soap to make it adhere.