Soaps that you put in the fridge/freezer to keep from gelling will take longer to harden since you've slowed down saponification.
Also, some fo's do overheat, which would heat up the soap even if you refrigerate it, add that to the fact that you used a wooden mold (which holds heat well) & that...