The Data Analytics Blog

Our news and views relating to Data Analytics, Big Data, Machine Learning, and the world of Credit.

All Posts

How To Choose A Scoring Engine

December 9, 2015 at 9:58 AM

Introducing scoring into your business, means the adoption of a reliable and feature-rich scoring and segmentation engine that can calculate predictions or make recommendations that will minimise risk and maximise return on investment. This guide will identify critical features and nice-to-haves when assessing a scoring engine.

For more information about scoring and some of the jargon see what is a scorecard?  

This guide is particularly relevant if:

  1. You are introducing scoring and segmentation into your business,
  2. You are wondering whether your application processor’s scoring engine or your collection management system’s scoring engine is fit for purpose, or
  3. You are looking at increasing the decisioning complexity of your decisioning.

What’s essential in a Scorecard?

This posting provides a long list of features that are worthwhile considering when you are looking for a scoring engine for your business. Below we go through what is considered to be Basic, Intermediate and Advanced functionality for scorecard engines.

All engines should allow the basic functionality, but many of the intermediate functionality should also be looked at. For those with more advanced models and strategies, the advanced functions should also be considered.

The following functionality should be parameterised, i.e. not hard-coded or written in code – therefore configurable by a risk manager or business analyst.

Basic Scoring Functionality

  1. The ability to class into attribute groups. For example taking age and creating groups “18-25”, “26-45”, “46+”.
      Some engines require classing to occur outside of the engine
  2. The ability to assign numeric weights to each attribute group.
  3. The ability to sum weightings to achieve a total score.
  4. The ability to set and apply a scorecard cut-off(s) – essentially determining which accounts should get approved or declined based on score (a referral band is also sometimes required for borderline scores).
  5. The ability to store the data used for scoring.
    This is critical for future development and monitoring.
  6. The ability to report on key front-end scoring metrics, namely:
      Characteristic stability: is the population you are wanting to apply the model for the same as the population you originally developed the model for
      Population stability: has your scorecard degraded over time due to a change in economic conditions or new origination strategies?
      Final decision analysis
  7. The ability for the scoring engine to be run in different areas of the business. Whereas scoring may initially be applied for new applications, scoring is typically used later on in other areas (listed below). Does your company have a scoring engine that allows for scalability?

Areas where scoring may need to be applied:

  • Application scoring: determining the level of risk for a loan application based on an applicant’s demographic, financial and other data.
  • Behavioural scoring: determining the probability of a certain type of behaviour for an applicant or customer, e.g. likelihood to purchase a product, repay a loan, respond to a campaign
  • Collections scoring: determining the probability of recovery of a debt from a debtor. Used to prioritise which debtors to contact first based on their probability to repay their debt
  • Marketing list risk/response scoring
  • Regulatory capital calculations: to determine how much capital is required to be kept in reserve to protect a financial institution against unexpected losses.
  • Customer value clustering
  • Optimisation models

Intermediate Scoring Functionality

  1. The ability to define data
      Many basic engines can only accept numeric fields (e.g. marital status = “M” would not be allowed, instead this would need to be converted in the host system to a numeric value).
      More advanced systems will allow a variety of fields including lists. Therefore, for characteristic values you would be able to define expected values, e.g. marital status references the list “M, S, D, W, P” as acceptable values when marital status is passed to the system
      Some advanced systems would allow you to define expected numerical ranges. For example, income should be a positive value; age should be a number between 18-100. This assists in picking up erroneous values that may have been captured and dealing with them appropriately.
  2. The ability to assign different scorecards based on certain criteria,
  3. The ability to apply a linear alignment to the scorecard
      The ability to apply a factor (m) and a constant (c) to the scorecard (y=mx+c) to align a scorecard back to original score-to-odds relationship
  4. The ability to set risk-grades
      Risk grades allow a risk manager to set up a variety of score ranges on which different actions could be applied, e.g. risk-based-pricing: the top score-range could qualify for the best interest rate, the middle band – the standard interest rate, and the lowest band – a premium interest rate.
  5. The ability to report on key back-end scoring metrics namely (performance data required):
      Characteristic strength analysis, and
      Scorecard strength analysis (Gini, KS, IV, etc.)
  6. The ability to run segmentation on the population
  7. The ability to apply multiple scorecards to an account or applicant
  8. The ability to build and apply policy rules either separately or within decision trees
  9. The ability to run account based or batch Account based scoring is typically used for application scoring; for behavioural scoring batch scoring is necessary.
  10. The ability to run “on site” or as a hosted solution

Advanced Scoring Functionality

  1. The ability to test new strategies (“What if” analysis allows the user to set different strategies (incorporating different old/new strategies utilising scorecards, decision trees, cut-offs and then send through a population to determine the portion of accounts being approved/declined, for example)
  2. The ability to run regulatory capital (Basel Accord) calculations or provision amounts through scores and trees.
  3. Ability to set terms-of-business (based on the risk-grade an applicant falls into, terms of business would need to be set which may determine the following: the approve/decline recommendation, interest rate qualified for, maximum term of loan, maximum size of loan, minimum affordability buffer etc.)
  4. The ability to facilitate machine learning.
      Advance scoring techniques such as machine learning allow the models to change dynamically as results are available. Facilitated machine learning allows for an algorithm that automatically “updates” the models. Having a scorecard that can be updated from the back-end (as well as through the user interface as discussed previously).
  5. Mathematical optimisation ready: The ability to deploy and apply a host of component models within a mathematical optimisation solution.
      Essentially the requirement would be to apply a sequence of component models (typically regression models) on the entire population and then incorporate the models together to determine a limit/loan-amount that needs to be offered to the customer
  6. The ability to apply non-linear alignments to the scorecard
      Typically a linear alignment is used to align scorecards. For some mis-aligned scorecard, multiple linear equations or a polynomial may need to be applied.
  7. The ability to deploy two scorecards into a matrix 

Using machine learning in business - download guide

Thomas Maydon
Thomas Maydon
Thomas Maydon is the Head of Credit Solutions at Principa. With over 17 years of experience in the Southern African, West African and Middle Eastern retail credit markets, Tom has primarily been involved in consulting, analytics, credit bureau and predictive modelling services. He has experience in all aspects of the credit life cycle (in multiple industries) including intelligent prospecting, originations, strategy simulation, affordability analysis, behavioural modelling, pricing analysis, collections processes, and provisions (including Basel II) and profitability calculations.

Latest Posts

The 7 types of credit risk in SME lending

  It is common knowledge in the industry that the credit risk assessment of a consumer applying for credit is far less complex than that of a business that is applying for credit. Why is this the case? Simply put, consumers are usually very similar in their requirements and risks (homogenous) whilst businesses have far more varying risk elements (heterogenous). In this blog we will look at all the different risk elements within a business (here SME) credit application. These are: Risk of proprietors Risk of business Reason for loan Financial ratios Size of loan Risk industry Risk of region Before we delve into this list, it is worth noting that all of these factors need to be deployable as assessment tools within your originations system so it is key that you ensure your system can manage them. If you are on the look out for a loans origination system, then look no further than Principa’s AppSmart. If you are looking for a decision engine to manage your scorecards, policy rules and terms of business then take a look at our DecisionSmart business rules engine. AppSmart and DecisionSmart are part of Principa’s FinSmart Universe allowing for effective credit management across the customer life-cycle.   The different risk elements within a business credit application 1) Risk of proprietors For smaller organisations the risk of the business is inextricably linked to the financial well-being of the proprietors. How small is small? The rule of thumb is companies with up to two to three proprietors should have their proprietors assessed for risk too. This fits in with the SME segment. What data should be looked at? Generally in countries with mature credit bureaux, credit data is looked at including the score (there is normally a score cut-off) and then negative information such as the existence of judgements or defaults; these are typically used within policy rules. Those businesses with proprietors with excessive numbers of “negatives” may be disqualified from the loan application. Some credit bureaux offer a score of an individual based on the performance of all the businesses with which they are associated. This can also be useful in the credit risk assessment process. Another innovation being adopted internationally is the use of psychometrics in credit evaluation of the proprietors. To find out more about adopting credit scoring, read our blog on how to adopt credit scoring.   2) Risk of business The risk of the business should be managed through both scores and policy rules. Lenders will look at information such as the age of company, the experience of directors and the size of company etc. within a score. Alternatively, many lenders utilise the business score offered by credit bureaux. These scores are typically not as strong as consumer scores as the underlying data is limited and sometimes problematic. For example, large successful organisations may have judgements registered against their name which, unlike for consumers, is not necessarily a direct indication of the inability to service debt.   3) Reason for loan The reason for a loan is used more widely in business lending as opposed to unsecured consumer lending. Venture capital, working capital, invoice discounting and bridging finance are just some of many types of loan/facilities available and lenders need to equip themselves with the ability to manage each of these customer types whether it is within originations or collections. Prudent lenders venturing into the SME space for the first time often focus on one or two of these loan types and then expand later – as the operational implication for each type of loan is complex.   4) Financial ratios Financial ratios are core to commercial credit risk assessment. The main challenge here is to ensure that reliable financials are available from the customer. Small businesses may not be audited and thus the financials may be less trustworthy. Financial ratios can be divided into four categories: Profitability Leverage Coverage Liquidity Profitability can be further divided into margin ratios and return ratios. Lenders are frequently interested in gross profit margins; this is normally explicit on the income statement. The EBIDTA margin and operating profit margins are also used as well as return ratios such as return on assets, return on equity and risk-adjusted-returns. Leverage ratios are useful to lenders as they reflect the portion of the business that is financed by debt. Lower leverage ratios indicate stability. Leverage ratios assessed often incorporate debt-to-asset, debt-to-equity and asset-to-equity. Coverage ratios indicate the coverage that income or assets provide for the servicing of debt or interest expenses. The higher the coverage ratio the better it is for the lender. Coverage ratios are worked out considering the loan/facility that is being applied for. Finally, liquidity ratios indicate the ability for a company to convert its assets into cash. There are a variety of ratios used here. The current ratio is simply the ratio of assets to liabilities. The quick ratio is the ability for the business to pay its current debts off with readily available assets. The higher the liquidity ratios the better. Ratios are used both within credit scorecards as well as within policy rules. You can read more about these ratios here.   5) Size of loan When assessing credit risk for a consumer, the risk of the consumer does not normally change with the change of loan amount or facility (subject to the consumer passing affordability criteria). With business loans, loan amounts can range quite dramatically, and the risk of the applicant is normally tied to the loan amount requested. The loan/facility amount will of course change the ratios (mentioned in the last section) which could affect a positive/negative outcome. The outcome of the loan application is usually directly linked to a loan amount and any marked change to this loan amount would change the risk profile of the application.   6) Risk of industry The risk of an industry in which the SME operates can have a strong deterministic relationship with the entity being able to service the debt. Some lenders use this and those who do not normally identify this as a missing element in their risk assessment process. The identification of industry is always important. If you are in manufacturing, but your clients are the mines, then you are perhaps better identified as operating in mining as opposed to manufacturing. Most lenders who assess industry, will periodically rule out certain industries and perhaps also incorporate industry within their scorecard. Others take a more scientific approach. In the graph below the performance of an industry is tracked for two years and then projected over the next 6 months; this is then compared to the country’s GDP. As the industry appears to track above the projected GDP, a positive outlook is given to this applicant and this may affect them favourably in the credit application.                   7) Risk of Region   The last area of assessment is risk of region. Of the seven, this one is used the least. Here businesses,  either on book or on the bureau, are assessed against their geo-code. Each geo-code is clustered, and the projected outlook is given as positive, static or negative. As with industry this can be used within the assessment process as a policy rule or within a scorecard.   Bringing the seven risk categories together in a risk assessment These seven risk assessment categories are all important in the risk assessment process. How you bring it all together is critical. If you would like to discuss your SME evaluation challenges or find out more about what we offer in credit management software (like AppSmart and DecisionSmart), get in touch with us here.

Collections Resilience post COVID-19 - part 2

Principa Decisions (Pty) L

Collections Resilience post COVID-19

Principa Decisions (Pty) L