The Data Analytics Blog

Our news and views relating to Data Analytics, Big Data, Machine Learning, and the world of Credit.

All Posts

How To Choose A Scoring Engine

December 9, 2015 at 9:58 AM

Introducing scoring into your business, means the adoption of a reliable and feature-rich scoring and segmentation engine that can calculate predictions or make recommendations that will minimise risk and maximise return on investment. This guide will identify critical features and nice-to-haves when assessing a scoring engine.

For more information about scoring and some of the jargon see what is a scorecard. 

This guide is particularly relevant if:

  1. You are introducing scoring and segmentation into your business,
  2. You are wondering whether your application processor’s scoring engine or your collection management system’s scoring engine is fit for purpose, or
  3. You are looking at increasing the decisioning complexity of your decisioning.

What’s essential in a Scorecard?

This posting provides a long list of features that are worthwhile considering when you are looking for a scoring engine for your business. Below we go through what is considered to be Basic, Intermediate and Advanced functionality for scorecard engines.

All engines should allow the basic functionality, but many of the intermediate functionality should also be looked at. For those with more advanced models and strategies, the advanced functions should also be considered.

The following functionality should be parameterised, i.e. not hard-coded or written in code – therefore configurable by a risk manager or business analyst.

Basic Scoring Functionality

  1. The ability to class into attribute groups. For example taking age and creating groups “18-25”, “26-45”, “46+”.
      Some engines require classing to occur outside of the engine
  2. The ability to assign numeric weights to each attribute group.
  3. The ability to sum weightings to achieve a total score.
  4. The ability to set and apply a scorecard cut-off(s) – essentially determining which accounts should get approved or declined based on score (a referral band is also sometimes required for borderline scores).
  5. The ability to store the data used for scoring.
    This is critical for future development and monitoring.
  6. The ability to report on key front-end scoring metrics, namely:
      Characteristic stability: is the population you are wanting to apply the model for the same as the population you originally developed the model for
      Population stability: has your scorecard degraded over time due to a change in economic conditions or new origination strategies?
      Final decision analysis
  7. The ability for the scoring engine to be run in different areas of the business. Whereas scoring may initially be applied for new applications, scoring is typically used later on in other areas (listed below). Does your company have a scoring engine that allows for scalability?

Areas where scoring may need to be applied:

  • Application scoring: determining the level of risk for a loan application based on an applicant’s demographic, financial and other data.
  • Behavioural scoring: determining the probability of a certain type of behaviour for an applicant or customer, e.g. likelihood to purchase a product, repay a loan, respond to a campaign
  • Collections scoring: determining the probability of recovery of a debt from a debtor. Used to prioritise which debtors to contact first based on their probability to repay their debt
  • Marketing list risk/response scoring
  • Regulatory capital calculations: to determine how much capital is required to be kept in reserve to protect a financial institution against unexpected losses.
  • Customer value clustering
  • Optimisation models

Intermediate Scoring Functionality

  1. The ability to define data
      Many basic engines can only accept numeric fields (e.g. marital status = “M” would not be allowed, instead this would need to be converted in the host system to a numeric value).
      More advanced systems will allow a variety of fields including lists. Therefore, for characteristic values you would be able to define expected values, e.g. marital status references the list “M, S, D, W, P” as acceptable values when marital status is passed to the system
      Some advanced systems would allow you to define expected numerical ranges. For example, income should be a positive value; age should be a number between 18-100. This assists in picking up erroneous values that may have been captured and dealing with them appropriately.
  2. The ability to assign different scorecards based on certain criteria,
  3. The ability to apply a linear alignment to the scorecard
      The ability to apply a factor (m) and a constant (c) to the scorecard (y=mx+c) to align a scorecard back to original score-to-odds relationship
  4. The ability to set risk-grades
      Risk grades allow a risk manager to set up a variety of score ranges on which different actions could be applied, e.g. risk-based-pricing: the top score-range could qualify for the best interest rate, the middle band – the standard interest rate, and the lowest band – a premium interest rate.
  5. The ability to report on key back-end scoring metrics namely (performance data required):
      Characteristic strength analysis, and
      Scorecard strength analysis (Gini, KS, IV, etc.)
  6. The ability to run segmentation on the population
  7. The ability to apply multiple scorecards to an account or applicant
  8. The ability to build and apply policy rules either separately or within decision trees
  9. The ability to run account based or batch Account based scoring is typically used for application scoring; for behavioural scoring batch scoring is necessary.
  10. The ability to run “on site” or as a hosted solution

Advanced Scoring Functionality

  1. The ability to test new strategies (“What if” analysis allows the user to set different strategies (incorporating different old/new strategies utilising scorecards, decision trees, cut-offs and then send through a population to determine the portion of accounts being approved/declined, for example)
  2. The ability to run regulatory capital (Basel Accord) calculations or provision amounts through scores and trees.
  3. Ability to set terms-of-business (based on the risk-grade an applicant falls into, terms of business would need to be set which may determine the following: the approve/decline recommendation, interest rate qualified for, maximum term of loan, maximum size of loan, minimum affordability buffer etc.)
  4. The ability to facilitate machine learning.
      Advance scoring techniques such as machine learning allow the models to change dynamically as results are available. Facilitated machine learning allows for an algorithm that automatically “updates” the models. Having a scorecard that can be updated from the back-end (as well as through the user interface as discussed previously).
  5. Mathematical optimisation ready: The ability to deploy and apply a host of component models within a mathematical optimisation solution.
      Essentially the requirement would be to apply a sequence of component models (typically regression models) on the entire population and then incorporate the models together to determine a limit/loan-amount that needs to be offered to the customer
  6. The ability to apply non-linear alignments to the scorecard
      Typically a linear alignment is used to align scorecards. For some mis-aligned scorecard, multiple linear equations or a polynomial may need to be applied.
  7. The ability to deploy two scorecards into a matrix 

predictive analytics guide

Thomas Maydon
Thomas Maydon
Thomas Maydon is the Head of Credit Solutions at Principa. With over 13 years of experience in the Southern African, West African and Middle Eastern retail credit markets, Tom has primarily been involved in consulting, analytics, credit bureau and predictive modelling services. He has experience in all aspects of the credit life cycle (in multiple industries) including intelligent prospecting, originations, strategy simulation, affordability analysis, behavioural modelling, pricing analysis, collections processes, and provisions (including Basel II) and profitability calculations.

Latest Posts

[Slideshare] How To Make Your Business Data Work For You

Common barriers to success: Skills shortage: data scientists are in high demand and in low supply. Companies lack the skills to develop advanced data analytics or machine learning applications. Cost: recruiting and building up or training a team, as well as infrastructure costs are immense. Inefficiency and low ROI on: acquisition campaigns; re-activation and retention campaigns; outbound sales calls and debt collection. Resulting in: No or ineffective use of data. High cost to get insights from data. Low returns from campaigns. What’s the alternative? Machine Learning as a Service (MLaaS): removes infrastructure skills and requirements for machine learning, allowing you to begin benefiting from machine learning quickly with little investment. Subscription based pricing, allowing you to benefit using machine learning while minimising your set-up costs and seeing returns sooner. Answers as a Service: Use historic data and machine learning to allow answers to increase in accuracy with time. MLaaS with predictive models pre-developed to answers specific questions: Genius Call Connect: What is the best time and number to call customers? Genius Customer Growth: Which customers are most likely to respond to cross-sell? Genius Re-activation: Which dormant customers are worth re-activating? Genius Customer Retention: Which customers are most likely to churn? Genius Leads: Which contacts are likely to respond to my campaign? Genius Risk Classifier: Which debtors are most likely to pay or roll? Benefits of Genius: Quick and cost-effective ability to leverage machine learning: Minimal set-up time Minimal involvement from IT Subscription based service Looking to make your data work for your business? Read more on Genius to see how it can help your business succeed. 

5 Must-Join Facebook Pages For Data Science, Machine Learning And Artificial Intelligence In 2019

While LinkedIn has traditionally been thought of as the business or work focussed social platform, Facebook has been making headway into gaining market share in the space as well. With company pages and groups, Facebook is catering to every interest and aspiration that people might have – and combining that with their social interactions and news sources. Facebook aims to give users a one-stop-shop experience, and it’s very good at doing it.

Our 2018 Customer Acquisition And Engagement Blog Roundup

Our final roundup this year covers two of our main topics: customer acquisition and customer engagement. We’ve not covered these topics in depth this year, and so decided to combine these two to provide a roundup of the best of both.