My dad shared some great wisdom with me when I was restoring my first car, a 1969 Volkswagen Beetle.
“You don’t need to change the oil in your own car, but you need to know how to change the oil in your car.”
He then walked me through my first and last DIY oil change. I learned proper terminology, process and the tools needed. Working on cars with him during my teenage years (my typical role being the official flashlight holder) helped educate me to speak knowledgably to mechanics, which still comes in handy today.
That same background knowledge around predictive modeling is equally helpful. You don’t need to build your own predictive models—that’s a job for the statisticians—but you should know how they are being built.
Most fundraising programs now use predictive modeling to make smarter decisions about who to reach out to. Modeling has been shown to boost the performance of traditional RFM methods (recency, frequency, monetary) that select donors only on giving behaviors. Modeling can identify donors likely to give again, flag those who may be drifting away and pinpoint strong candidates for recurring, mid-level, major, planned or donor-advised fund (DAF) gifts.
But here’s the reality: Many organizations are using models that are black boxes, or opaque systems.
And that matters because the way a model is built, trained and applied has a direct impact on your results.
Not all models are created equal.
If you’re working with an agency or partner on modeling, you should feel confident asking how it works, what goes into it and whether it’s custom to your cause.
Here are six questions to help you do exactly that—and what to listen for in the answers.
Most models incorporate past giving behavior, and for current donors, it is the most predictive. Demographics can add an incremental boost and should go beyond the basic inclusion of age. Ask if the following demographics are evaluated in your models:
These demos typically “pop” as predictive in different model applications. For example, in planned giving applications, knowing whether someone has children is key in identifying a promising lead for a legacy gift.
Are they referring to machine-learning algorithms? If so, which types? Some agencies use “AI” to describe the most basic approaches.
For example, logistic regression is a type of machine-learning algorithm, but its use is outdated in the age of more modern approaches—like Random Forest, XGBoost and Neural Networks—which have been proven to raise money more effectively.
Most models are designed to optimize a single outcome, so tradeoffs are common.
Predicting response rate alone will result in lower average gift size and long-term value, while optimizing average gift can be at the expense of fewer donors.
An ensemble approach combines multiple models to balance desired outcomes and allows you to turn the dial toward the goal of the program or campaign.
A model shouldn’t dictate your strategy. Rather, your strategy shapes how the model is used—across multiple channels, not just direct mail.
Not all donor behaviors are the same.
Food banks donors are very different from other human services donors, who are different from animal welfare donors.
Using one algorithm that was developed on many types of nonprofits will result in less precision.
Cause-specific models are trained to recognize the nuances of your audience, which leads to more relevant targeting and stronger performance over time.
Does your agency partner mail or target using your model score as an easy cutoff, or do they take into account the artful application of a model to your organization’s unique needs?
Easy-to-implement cutoffs oversimplify decision-making and limit your performance long term. Predictive modeling came of age in commercial catalog marketing to maximize short-term efficiency by identifying the customers likely to buy right now. Fundraising is different. Nonprofit organizations must balance current ROI with longer-term donor development and file health.
If we optimize too aggressively, we risk optimizing ourselves out of a program by shrinking the pipeline of emerging supporters who may become tomorrow’s loyal and transformational donors.
More advanced enablement looks at how modeling fits into your broader, multi-channel communication strategy. If we mail fewer donors for a particular campaign, how can we repurpose those dollars in a more impactful channel?
If you can’t see what’s influencing your model, it becomes difficult to trust the output or internally explain it.
To access full transparency beyond the mysterious black box, ask to see the Variable Importance. This output shows which inputs are having the greatest impact on the model’s predictions.
This can also help with the inevitable questions of, “Why did the model select donor X?” Knowing what the model was prioritizing can help demystify the selection process.
Stronger modeling builds a system that helps you make better decisions with limited resources.
That includes:
When these elements come together and consider your campaign goals, segments and channels, you’ll be ensuring your program is efficient but not leaving opportunity behind.
If your current agency can’t clearly answer these questions, it’s worth taking a closer look at your strategy.
If you’re ready to make decisions with confidence, reach out to RKD to discuss what your modeling could be doing differently.