The Data Analytics Blog

Our news and views relating to Data Analytics, Big Data, Machine Learning, and the world of Credit.

All Posts

Shift happens: Top tips on Scorecard re-alignments

August 5, 2020 at 11:35 AM

Principa employs a variety of best-practice credit scorecard building techniques including mathematical programming, regression modelling, optimal segmentation-seek genetic algorithms and reject inference parceling, amongst others. Through our credit risk scorecards businesses can look to improving their credit risk decisioning by 5-30%.

[FIND OUT MORE] 

What is a credit risk scorecard?

A credit risk scorecard is a mathematical model that predicts the likelihood of a customer defaulting on a loan or facility. Scorecards can use a variety of data from demographic information, credit bureau data, payment behaviour, psychometrics and cellular behaviour.

Principa has extensive experience in monitoring and validating a variety of different scorecards, giving the credit granter comfort in understanding the optimal time to re-align, fine-tune or redevelop a scorecard.

The importance of monitoring reports

Once a scorecard is deployed a regular monitoring regime should be set up. The basic premise of why we monitor is that population and performance shifts occur, and the past is not exactly like the future. One of the outcomes of the scorecard monitoring report might be a scorecard re-alignment.

“The basic premise of why we monitor is that population and performance shifts occur, and the past is not exactly like the future.”

Realignment - negative shift

In this blog we explore the ins and outs of scorecard re-alignment and we give you some of our top tips when it comes to a re-alignment. We address:

  • What is a scorecard re-alignment?
  • Align the scorecard or change the cut-offs?
  • What is a linear re-alignment?
  • What is a non-linear re-alignment?
  • What is marginal bad rate and how is this different to bad rate
  • Use of reject and not-taken-up inference?
  • Decision engine capabilities
  • 2nd re-alignments

What is a scorecard re-alignment?

Scorecards can be built utilising a variety of methods. The final scorecard gives the customer/applicant a score that relates to their expected performance. This is known as the score-odds relationship. To give a popular example, it may be at a score of 660 we expected odds of 15:1 - this means that for every 16 customers scoring 660 points on the scorecard, 1 will end up being a poor payer and 15 will be good payers.

“The final scorecard gives the customer/credit applicant a score that relates to their expected performance. This is known as the score-odds relationship.”

When we set a scorecard strategy, we will set it based on risk-appetite. For originations this might be that we will only accept odds of 15:1 or better, otherwise the business we write would be unprofitable. That means our cut-off should be at 660.

Now two years on we find that customers that scored 660 had odds of 13:1 (i.e. 13 good payers to one bad - customers were actually higher risk than expected) – in fact we may find this across the entire scorecard. That means our strategy of accepting those scoring 660 might be unprofitable. The best-case solution is to re-align the scorecards which really means applying an equation to the score to bring the score to odds back to 660 -> 15:1.

Align the scorecard or change the cut-off

In the example above 15:1 good/bad odds at 660 became 13:1 which meant we were writing unprofitable business at 660. To counteract the shift, we have 2 options:

  1. Change the cut-off(s)
  2. Re-align the scorecard

Changing the cut-off can be a temporary solution. For example, you may have accepted everyone scoring above 660, but now you change the cut-off to 665. But what if you have multiple cut-offs for different products and/or are using risk-based-pricing. In this case an alignment is more suitable.

What is a linear re-alignment?

Linear re-alignments are the most common alignments used in scoring.

New score = (factor) * (old score) + constant

This equation transforms the score to represent the correct score-to-odds relationship. It is not the only re-alignment approach.

What is a non-linear re-alignment?

A linear alignment normally gives an approximate correction for the misaligned scorecard. Often one finds that certain score ranges are flat (i.e. there is no discrimination between good and bad) while other ranges rank order well. Or that some ranges even reverse.

If the ranges are significantly of line, a non-linear alignment may be in order. These alignments would normally utilise either an equation using the exponential function or a polynomial, alternatively it may have different equation for certain ranges – for example two different linear alignments applied to different ranges.

An important consideration before progressing with a non-linear alignment would be to ensure that your scoring engine can accommodate the different alignment functions that you are considering.

“An important consideration before progressing with a non-linear alignment would be to ensure that your scoring engine can accommodate the different alignment functions that you are considering.”

score flatteningMarginal bad rate

Before conducting an alignment, it is important to understand the difference between bad rate and marginal bad rate.

  • Bad rate is the percentage of accounts that perform in an unsatisfactory manner. So, when asked what the bad rate is above 660, this would mean what is the bad rate for the population scoring above 660.
  • Marginal bad rate = is the bad rate at a particular score. So, what is the bad rate of the customers scoring exactly 660. Note this would be higher than the bad rate as stated above.

When conducting a scorecard alignment, always align to the marginal bad rate. This is normally smoothed as there may not be a statistically significant population scoring at a certain score.

Use of reject and not-taken-up inference

This relates to origination scorecards. When conducting an alignment, always remember that your original scorecard was built with reject and possibly not-taken-up inference. This means that your observed bad rate may be lower than your actual bad rate, particularly near the cut-off where a lot of applicants were declined due to policy rules and not the scorecard. Ideally you need to perform an inference exercise to ensure that your alignment to original good-bad-odds is a like-versus-like exercise.

What is commonly experienced is the following:

  • Below cut-off: accepted customers perform better than expected due to positive cherry-picking
  • At cut-off: customers perform better than expected due to policy rules weeding out additional bad customers.
  • In higher score ranges: customers perform worse than expected due to good cohorts of customers not taking up the loan.

For behavioural scores, one has the challenge of action-effect influencing the performance of the scores. For example, you may introduce an aggressive strategy to recover debt from delinquent customers scoring low. This may improve the performance of the lower scores. Similarly, for credit cards, you may offer high limit increases and high authorisation limits to high scoring customers – this may push some of them over the edge resulting in them performing worse than expected. It is worth understanding these effects when you are conducting re-alignment. In extreme cases we have seen credit analytics teams omit segments of their book from the re-alignment exercise.

Decision engine capabilities

DecisionSmart screenshot (realignment blog)

Before conducting a scorecard re-alignment, it is critical to ensure that your scoring engine/decision engine/business rules management system can facilitate alignments. If not, you may need to look for a work-around like amending the score cut-offs.

If the engine can facilitate an alignment, it is also important to know what sort of alignment can be supported. Below is a screenshot from Principa’s DecisionSmart which supports both linear and non-linear alignment. To understand what to look for in a decision engine have a look at our blog on the topic.

Second re-alignments

Another question we have been asked a few times, is: when doing a second re-alignment – do you align against the original score or the last aligned score. Mathematically it does not really matter, but from a deployment perspective it does. Always run the re-alignment against the original scorecard otherwise it will be difficult to deploy.

Re-alignment is a critical part of scorecard maintenance

Re-alignment is a critical part of scorecard maintenance. It is not a straight fo

Analytics ICU

rward exercise as there are quite a few considerations as outlined here. If you would like to know more or would like to find out how Principa can help your organisation with effective scorecard management, consider utilising our Analytics ICU programme. Get in touch with us oninfo@principa.co.za.

 

Thomas Maydon
Thomas Maydon
Thomas Maydon is the Head of Credit Solutions at Principa. With over 17 years of experience in the Southern African, West African and Middle Eastern retail credit markets, Tom has primarily been involved in consulting, analytics, credit bureau and predictive modelling services. He has experience in all aspects of the credit life cycle (in multiple industries) including intelligent prospecting, originations, strategy simulation, affordability analysis, behavioural modelling, pricing analysis, collections processes, and provisions (including Basel II) and profitability calculations.

Latest Posts

How to choose the correct collections chatbot

Principa has a wealth of experience in building and deploying chatbots for the financial services industry. Our custom-built solution is flexible and fully customisable which allows your bot to assume your brand’s persona. We can also seamlessly integrate with existing systems. Click here to find out more. 

Model validation and adjustment

The time is NOW for model validation and adjustment. One of the major premises used in credit scoring is that “the future is like the past”. It’s usually a rational assumption and gives us a reasonable platform on which to build scorecards whether they be application scorecards, behavioural scores, collection scores or financial models. That is reasonable until something unprecedented comes along. You can read about this black swan event in our previous two blogs here and here.

Payment holidays – what did everyone do?

Payment holidays have been used throughout South Africa and around the world to help alleviate the economic stress during the COVID-19 lockdown. In this blog we look at some of the steps taken internationally and by some of South Africa’s major lenders (specifically in the consumer space).