"Brass is a metal alloy that is always made with a combination of copper and zinc. By varying the amount of copper and zinc, brass can be made harder or softer. Other metalsāsuch as aluminum, lead, and arsenicāmay be used as alloying agents to improve machinability and corrosion resistance. "
Aluminum and Sodium Hydroxide are enemies. Even though 99.999999999999% of the lye is used up in HP/CP soap, the microscopic amounts left are the reason why folks are told to NOT use metal racks while their soap cures. And since soap is typically planed, beveled and stamped within the first week if not a few days after it has been cut, using anything but Rubber, Acrylic, or Resin is not a good idea.
Because you no doubt really like your stamp, one thing you can do is place a piece of plastic wrap over the top of your bar and then use your stamp. I would recommend using a good quality brand of plastic wrap as opposed to a cheap one and only stamp once or twice as the stamp will eventually wear through the plastic. If that doesn't work well, you will want to have an acrylic stamp made.