December 9, 2015 at 9:58 AM
Introducing scoring into your business, means the adoption of a reliable and feature-rich scoring and segmentation engine that can calculate predictions or make recommendations that will minimise risk and maximise return on investment. This guide will identify critical features and nice-to-haves when assessing a scoring engine.
For more information about scoring and some of the jargon see what is a scorecard?
This guide is particularly relevant if:
- You are introducing scoring and segmentation into your business,
- You are wondering whether your application processor’s scoring engine or your collection management system’s scoring engine is fit for purpose, or
- You are looking at increasing the decisioning complexity of your decisioning.
What’s essential in a Scorecard?
This posting provides a long list of features that are worthwhile considering when you are looking for a scoring engine for your business. Below we go through what is considered to be Basic, Intermediate and Advanced functionality for scorecard engines.
All engines should allow the basic functionality, but many of the intermediate functionality should also be looked at. For those with more advanced models and strategies, the advanced functions should also be considered.
The following functionality should be parameterised, i.e. not hard-coded or written in code – therefore configurable by a risk manager or business analyst.
Basic Scoring Functionality
- The ability to class into attribute groups. For example taking age and creating groups “18-25”, “26-45”, “46+”.
- Some engines require classing to occur outside of the engine
- The ability to assign numeric weights to each attribute group.
- The ability to sum weightings to achieve a total score.
- The ability to set and apply a scorecard cut-off(s) – essentially determining which accounts should get approved or declined based on score (a referral band is also sometimes required for borderline scores).
- The ability to store the data used for scoring.
This is critical for future development and monitoring.
- The ability to report on key front-end scoring metrics, namely:
- Characteristic stability: is the population you are wanting to apply the model for the same as the population you originally developed the model for
- Population stability: has your scorecard degraded over time due to a change in economic conditions or new origination strategies?
- Final decision analysis
- The ability for the scoring engine to be run in different areas of the business. Whereas scoring may initially be applied for new applications, scoring is typically used later on in other areas (listed below). Does your company have a scoring engine that allows for scalability?
Areas where scoring may need to be applied:
- Application scoring: determining the level of risk for a loan application based on an applicant’s demographic, financial and other data.
- Behavioural scoring: determining the probability of a certain type of behaviour for an applicant or customer, e.g. likelihood to purchase a product, repay a loan, respond to a campaign
- Collections scoring: determining the probability of recovery of a debt from a debtor. Used to prioritise which debtors to contact first based on their probability to repay their debt
- Marketing list risk/response scoring
- Regulatory capital calculations: to determine how much capital is required to be kept in reserve to protect a financial institution against unexpected losses.
- Customer value clustering
- Optimisation models
Intermediate Scoring Functionality
- The ability to define data
- Many basic engines can only accept numeric fields (e.g. marital status = “M” would not be allowed, instead this would need to be converted in the host system to a numeric value).
- More advanced systems will allow a variety of fields including lists. Therefore, for characteristic values you would be able to define expected values, e.g. marital status references the list “M, S, D, W, P” as acceptable values when marital status is passed to the system
- Some advanced systems would allow you to define expected numerical ranges. For example, income should be a positive value; age should be a number between 18-100. This assists in picking up erroneous values that may have been captured and dealing with them appropriately.
- The ability to assign different scorecards based on certain criteria,
- The ability to apply a linear alignment to the scorecard
- The ability to apply a factor (m) and a constant (c) to the scorecard (y=mx+c) to align a scorecard back to original score-to-odds relationship
- The ability to set risk-grades
- Risk grades allow a risk manager to set up a variety of score ranges on which different actions could be applied, e.g. risk-based-pricing: the top score-range could qualify for the best interest rate, the middle band – the standard interest rate, and the lowest band – a premium interest rate.
- The ability to report on key back-end scoring metrics namely (performance data required):
- Characteristic strength analysis, and
- Scorecard strength analysis (Gini, KS, IV, etc.)
- The ability to run segmentation on the population
- The ability to apply multiple scorecards to an account or applicant
- The ability to build and apply policy rules either separately or within decision trees
- The ability to run account based or batch Account based scoring is typically used for application scoring; for behavioural scoring batch scoring is necessary.
- The ability to run “on site” or as a hosted solution
Advanced Scoring Functionality
- The ability to test new strategies (“What if” analysis allows the user to set different strategies (incorporating different old/new strategies utilising scorecards, decision trees, cut-offs and then send through a population to determine the portion of accounts being approved/declined, for example)
- The ability to run regulatory capital (Basel Accord) calculations or provision amounts through scores and trees.
- Ability to set terms-of-business (based on the risk-grade an applicant falls into, terms of business would need to be set which may determine the following: the approve/decline recommendation, interest rate qualified for, maximum term of loan, maximum size of loan, minimum affordability buffer etc.)
- The ability to facilitate machine learning.
- Advance scoring techniques such as machine learning allow the models to change dynamically as results are available. Facilitated machine learning allows for an algorithm that automatically “updates” the models. Having a scorecard that can be updated from the back-end (as well as through the user interface as discussed previously).
- Mathematical optimisation ready: The ability to deploy and apply a host of component models within a mathematical optimisation solution.
- Essentially the requirement would be to apply a sequence of component models (typically regression models) on the entire population and then incorporate the models together to determine a limit/loan-amount that needs to be offered to the customer
- The ability to apply non-linear alignments to the scorecard
- Typically a linear alignment is used to align scorecards. For some mis-aligned scorecard, multiple linear equations or a polynomial may need to be applied.
- The ability to deploy two scorecards into a matrix