Back testing

A margin is a bet on how bad tomorrow could get. The only way to know whether that bet is sized right is to keep checking it against what actually happened. That check is back testing.

The workbook's own words: "CC shall daily conduct back testing of the margins collected vis-a-vis the actual price changes for the contracts being cleared and settled in every segment to assess appropriateness of its margining models."

Back testing sits alongside two related but distinct checks the clearing corporation (CC) must run:

• Stress testing — daily, using at least the standardised methodology for each segment, plus the CC's own scenarios of "extreme but plausible" market conditions, including multiple defaults and forward-looking stress scenarios.
• Reverse stress testing — periodic, working backwards from failure to find what conditions would exhaust the CC's resources.
• Back testing — also daily, but looking backwards: comparing the margin actually collected on a position against the actual price change that followed, to see whether the margin would have covered the loss.

The workbook groups all of this under a wider obligation: the CC must continuously monitor the adequacy of its financial resources (its default waterfall) against the losses its stress tests estimate, and must perform a comprehensive review of its stress testing scenarios, models and assumptions at least monthly — more frequently if the products or markets it clears become more volatile, less liquid, or see a significant rise in position concentration.

A clearing corporation collects initial margin on a cash-settled 91-Day T-Bill futures position sized to cover a 99% one-day VaR. Back testing, run daily, compares that collected margin against what the position actually lost the following day, for every contract in the segment.

Over a month, suppose the actual daily loss exceeded the margin collected on more days than the 99% confidence level would predict — say on 5 out of 22 trading days, well above the roughly 1-in-100 days the 99% VaR model implies. That pattern of back-testing failures is exactly what would prompt the CC to revisit its margining model's assumptions, potentially widening the price scan range used in SPAN margining for that product, rather than waiting for the scheduled monthly review to catch it.

Chapter 7 (Clearing, Settlement and Risk Management of IRD), section 7.12.4 (Stress testing and back testing), sets out back testing, stress testing and reverse stress testing as three distinct obligations of the clearing corporation, followed by the monitoring-of-financial-resources requirement. Expect a question distinguishing back testing (checking margins against what actually happened) from stress testing (simulating what could happen) and reverse stress testing (working backwards from failure).

• Back testing looks backwards at margins already collected; stress testing looks forward at hypothetical scenarios. They are named in the same breath but test opposite directions.
• Back testing is run daily, the same frequency as stress testing — do not assume it is only periodic.
• The comprehensive review of stress-testing scenarios and models is at least monthly, and must be run more often in volatile, illiquid, or concentrated markets.
• The purpose stated is explicitly to assess the appropriateness of the CC's margining models — not to catch individual member defaults.