Credit Risk Scoring - Project Details

Model comparison, feature ranking, and scorecard results from the trained pipeline

Interactive dashboard

Open dashboard in full screen

Project information

Predicting and explaining credit risk

This project combines a classification model with survival analysis to answer two questions a retail bank actually needs answered: which customers are likely to default, and when in their lifecycle is that risk highest. Three models (Logistic Regression, Random Forest, XGBoost) were compared, then explained with SHAP, and paired with a Cox Proportional Hazards model to translate risk into interpretable hazard ratios — the kind of dual approach an actuarial or risk team would expect.

Project deep dive

Problem statement

Build an end-to-end probability-of-default (PD) pipeline from raw, multi-table lending data: engineer features across seven linked tables, rank them with Weight-of-Evidence / Information Value, train a staircase of models from an interpretable scorecard up to gradient boosting, and translate the output into a points-based credit scorecard a risk team can actually use — plus a deployed app for live scoring.

Dataset

Home Credit Default Risk (Kaggle) — ~300,000 customers across linked tables:

  • application_train/test, bureau, bureau_balance
  • previous_application, POS_CASH_balance, credit_card_balance
  • installments_payments — ~723 MB, loaded in chunks rather than a single pd.read_csv() call

Methodology

  1. Data prep — loading, profiling, and anomaly cleaning across all seven raw tables; the large installments table is read in chunks with per-chunk dtype downcasting.
  2. Feature engineering — multi-table aggregation and ratio features (e.g. credit-to-income, annuity-to-credit) rolled up from bureau and previous-application history to the applicant level.
  3. WOE / IV ranking — Weight-of-Evidence binning and Information Value scoring to rank predictive strength and prepare features for a linear scorecard.
  4. Model comparison — Logistic Regression (scorecard baseline), Random Forest, XGBoost, and LightGBM trained and tuned on the engineered features.
  5. Scorecard construction — logistic coefficients rescaled into a points-based scorecard, with Population Stability Index (PSI) tracked for drift.
  6. Explainability — SHAP global and local explanations layered on top of the winning model.

What the pipeline measures

MetricWhy it's tracked
AUC-ROCOverall ranking ability
KS statisticMax separation good/bad — standard retail credit metric
Gini2×AUC−1 — industry-standard alternative to AUC
PR-AUCMore informative than ROC-AUC under class imbalance
CalibrationThe PD feeds an Expected Loss calculation, so calibration matters as much as ranking
PSIPopulation/model stability check over time

Tech stack

Python LightGBM XGBoost SHAP optbinning joblib lifelines Plotly Streamlit

The repo is organized as a runnable main_pipeline.py with distinct phases (features → WOE → train → scorecard) — see the GitHub repo → for phase-by-phase run instructions and the 723 MB installments-file handling.