Function validation

In the dcr module, we have included the validation() function, which provides a collection of validation techniques that we will apply throughout this book. The function provides an output of four panels:

  • Upper left panel: summary table of validation metrics;
  • Upper right panel: real-fit diagram by time, i.e., a comparison of the average observed and fitted outcome variable over time;
  • Lower left panel: histogram of the fitted values;
  • Lower right panel: real fit diagram by deciles of the fitted values.

This summary includes the visuals and metrics that we find most useful in validating models for default, payoff, loss rates given default and exposures.

The function validation() has three input arguments:

  • Outcome variable (may be binary or metric);
  • Fitted variable (may be binary or metric);
  • Time variable.

All variables may be numpy arrays or pandas data frames. In a perfect PD model the fitted values are equal to the observed outcomes and the validation function provides the following output:

default_time = data.default_time.values
time = data.time.values

validation(default_time, default_time, time)