You first have to decide what kind of scrub you want, because that will determine what type of recipe you need.
1. Do you want it to foam and cleanse? You will need a foamy, cleansing ingredient like soap or syndets. If you don't need foam or cleansing, skip to the next questions.
2. Do you care whether it separates when left to sit? To prevent separation, you will need an emulsifier like e-wax (EDIT: M&P soap base also works, per @lsg’s recipe below). If you don't care about separation, you can simply mix your oils/fats and your scrubby ingredient, and plan to stir to remix it each time before using.
3. Will it be used on rough areas like feet and elbows, or more delicate facial skin? The answer to this question will determine what and how much scrubby ingredient to use, and how finely it must be ground. Facial scrubs are usually scented more lightly than foot scrubs, and will use less comedogenic oils or butters than what might be used on arms, legs, and feet.
My personal favorite scrub recipe is the foaming emulsified sugar scrub recipe that comes with the purchase of Foaming Bath Butter recipe from the DIY Bath & Body shop on Etsy. But there are lots of free recipes out there, too. The Humblebee & Me website has quite a few nice ones, although IIRC, none of them are emulsified as she finds those drying to her skin.
Good luck, and let us know what you end up making, and how you like it!