Similar to what Meep said, I would freeze your mold prior to adding your soap to it.
What I suggest is making two different 2 lb silicone molds for your one batch, if you possibly can. If you can make enough room in your freezer for both of them, do that, and then, pour your batches, keep them in there for at least 24 hours. I'm impatient, and will unmold early, but I don't suggest that, from experience. I suggest then transferring to the frig, if you can then make room. I don't have issues here, if I can cut soon enough, but you can have worse problems cutting if your soap is too frozen. Just recently, I cut a soap that was way too soft, just out of the freezer. My issue was that the soap really wasn't ready, and while it cut, I got crumbliness from the frozen insides of the soap that wasn't thawed enough. Although, the thawed parts were equally not ready yet. So definitely keep it cold and wait for readiness. This was a goat's milk (among many other things) soap I'm talking about.