The Data Analytics Blog

Our news and views relating to Data Analytics, Big Data, Machine Learning, and the world of Credit.

All Posts

How to COVID-proof your scorecards with short-outcome machine learning models

July 21, 2020 at 12:20 PM

“Unprecedented” is a term with which we’ve all become quite familiar over the last few months. COVID-19 has changed our society and our economy quite drastically. In predictive analytics “unprecedented” has far reaching implications – simply put it’s difficult to build models when we do not have data that reflects similar trends to what we will expect moving forward.

So, it’s very unlikely that the models that you have deployed – whether it be in originations, account management, IFRS9 or collections are working as expected. In 2 previous blogs here and here we covered how data is changing during the COVID-19 period.

What to do?

The first thing to do would be to conduct a model HealthCheck. Principa’s Analytical ICU will assess the health of your models and triage them into four categories. At the very least you will need to realign your models, but the likelihood is that you may need to fine-tune or rebuild your models completely.Rebuilding your models

If you decide you need to rebuild your models, then you have a challenge at hand.  The diagram below represents a scorecard build time-line. The observation and outcome period are the periods from which we would extract data for the scorecard build. What is evident is that the observation period and outcome period do not coincide with the COVID-19 crisis. This means that the scorecards that you would rebuild may not be suitable for the current economic climate and you are back at square one.There is a solution, though, and that is to adopt machine learning models and to approach the scorecard build with a “short-outcome/strict-performance” methodology.

Short-outcome/ strict-performance

A short-outcome/ strict-performance approach will involve sampling the data from a shorter period of time and use a very strict performance definition (e.g. Any missed payment = ”Bad”).  This approach will allow you to sample from a period that is more representative of the current environment. The time lines below illustrate that a scorecard built in August/September could utilise April/May observations (i.e. the beginning of the COVID-19 period).One of the reasons one uses a long observation period is to take in the full annual cycle. As we are looking at catering for the COVID-19 “cycle” the shorter term is more appropriate. The common good/bad definition (for example “ever 3+” = “Bad”) is better, as it allows for the separation of truly good payers from truly bad payers. The stricter definition means that you’ll pick up technical arrears and a few lazy payers in your bad definition. This may weaken the models slightly, but the gains from being able to model for the COVID-19 period should outweigh the losses from the lesser performance definition. Another challenge with the short-outcome models will be the population size. The modelling approach will be different with the smaller population and we may use coarser classing.

Model longevity and machine learning

Once you deploy these “COVID-19” models, the models should be monitored. Redevelopment will likely need to happen sooner than traditional models. It is therefore suggested that Principa’s Quick-Step machine learning models would be most appropriate here allowing you to leverage off new COVID-19 data every quarter and reducing the cost of a sorecard-build.In our next blog we will be covering Principa’s Quick-Step Machine Learning and why this is a great solution to switch models in-and-out during the COVID-19 recovery period.

To find out how Principa can help you with an Analytics ICU or the building of short-outcome/ strict-performance credit models, contact us on

Contact Us to Discuss Your data analytics Business Requirements

Thomas Maydon
Thomas Maydon
Thomas Maydon is the Head of Credit Solutions at Principa. With over 17 years of experience in the Southern African, West African and Middle Eastern retail credit markets, Tom has primarily been involved in consulting, analytics, credit bureau and predictive modelling services. He has experience in all aspects of the credit life cycle (in multiple industries) including intelligent prospecting, originations, strategy simulation, affordability analysis, behavioural modelling, pricing analysis, collections processes, and provisions (including Basel II) and profitability calculations.

Latest Posts

Solving the Credit Unaware Challenge with Psychometrics

At Principa, we engage with clients and organisations across the entire credit lifecycle and track the focus of the South African credit industry. For nearly ten years the focus has consistently been in the collection space, but recently (since early 2021) this has changed and a large number of our clients are focused on acquisitions and originations.

Predicting Customer Behaviour (PART 2)

In Part One of this two-part blog, we started providing a short overview of just some of the propensity models that Principa has developed. In this Part Two, we continue to look at different types of propensity models available across the customer engagement lifecycle that are used to predict behaviour and solve business problems. 

PART 2: How to Cure the Post Pandemic “Collections” Symptoms

In PART 1 of this two-part series, we explored how the current socio-economic climate resulting from the lingering financial hangover caused by the pandemic is negatively impacting the consumer's ability to settle a debt.