My current iteration of this deodorant is:
49% Babassu oil
33% Magnesium Hydroxide
8.5% E-Wax (Mountain Rose Herbs)
9% Cetyl Alcohol
0.5% Fragrance
As you can see, there is no BTMS-50 in the recipe (ewax, instead) and I do not get any of the pilling you mention - do you think it could be the BTMS-50? Of course, I'm also using a lower percentage of "hardeners," so maybe that could play a part, and I would think your idea of testing out a lower percentage of the BTMS-50/Cetyl is on the right track.
I used e wax and didn't like it. I hope you like it and update us about it.