Absolutely - I was just about to post this very idea. I make a lovers' bath tea, and I do exactly what you said above. I boil water, throw the bag in, remove from heat, and steep about 10 minutes while I fill up the tub. Then pour the "tea," bag and all, into the tub. It's WAY more fragrant...