Scoring is the process of applying an algorithmic model built from historical data to a new set of data to uncover practical insights or solve a business problem.
Model development is generally a two-stage process. The first stage is training and validation, during which you apply algorithms to data for which you know the outcomes to uncover patterns between its features and the target variable. The second stage is scoring, in which you apply the trained model to a new dataset. Then, the model returns outcomes in the form of probability scores for classification problems and estimated averages for regression problems. Finally, you deploy the trained model into a production application or use the insights it uncovers to improve business processes.
For example, to score a model meant to predict the likelihood of customer churn:
- Build a churn behavior model using a historical dataset that contains information on which customers churned and other information that you believe contributed to that outcome.
- Apply the model to existing customer data to produce a value, or “score,” that estimates their likelihood to churn.
Different ways to score models include:
- Batch scoring. Useful for when the model’s decisions don’t have to be implemented immediately. For example, a marketer may batch score a model on a list of purchased leads to determine which are most likely to buy their product.
- Real-time scoring. Useful when time is of the essence in realizing value from the model. For example, a bank needs a fraud model to score credit card transactions within milliseconds to quickly deny likely fraudulent transactions.
Scoring is also used to evaluate existing models. By training the model on historical data, using it to score other historical data for which you know the outcome, and comparing the scores to the known values, you determine how well the model performs.