I would do some cinnamon and clove, both for the color and the scent. You could also try using coffee or cocoa in small amounts. I like to use instant coffee granuals. I dissolve them in a little hot water and add a bit at a time to the soap batter. I tend to think of gingerbread as being a bit more of a reddish brown than a flat brown, so maybe a little rose clay.