How the Historical Lead-Relationship Pipeline Works

Abstract

The American Distress Index (ADI) tracks household financial distress through a panel of economic indicators organized into five domains. This paper documents a systematic scanner that tests pairwise combinations of these indicators for lagged statistical relationships. A surviving pair records a historical time offset and association under the five filters below; it does not establish a causal effect or forecast.

The pipeline applies five sequential filters to eliminate spurious correlations:

1. Cross-correlation on raw levels (FDR-corrected at α = 0.05)
2. Cross-correlation on first-differenced series (eliminates trend-driven false positives)
3. Multi-crisis validation across GFC (2007–2009), COVID (2020), and the 2001 recession
4. Granger causality testing (p < 0.05)
5. Out-of-sample replication (calibrate before Q1 2013, validate from Q1 2013 onward, minimum r > 0.3)

The scanner ran 87,281 lagged cross-correlation tests. Those tests produced 5,228 unique relationships that survived the dual correlation filter. 1,031 validated during at least one crisis. The top 30 were tested for Granger causality; 12 passed. Of those, 9 replicated out of sample. The fully validated set is 0.17% of the dual-filter relationships.

Data

The scanner draws from the indicator dataset stored in data/indicators/. Each indicator has quarterly or monthly observations resampled to a common quarterly frequency using last-observation-carried-forward. The available history begins in 2000 for the longest-running inputs, while other series start later and the validation window advances with each research refresh.

Sources include the Federal Reserve Bank of St. Louis Federal Reserve Economic Data (FRED) system, the U.S. Bureau of Labor Statistics (BLS), the National Bureau of Economic Research, the Federal Reserve Bank of New York, and administrative data from mortgage servicer reports. All source data is publicly available and attributable.

Key preprocessing steps:

• Quarterly resampling: Monthly series are resampled to quarterly using end-of-quarter values
• Minimum overlap: Pairs require at least 12 quarters of overlapping observations
• Winsorization: Values are capped at the 95th percentile to prevent COVID-era outliers from dominating the correlation structure
• Maximum lag: Cross-correlations are computed at lags 0 through 16 quarters (4 years)

Methods

Filter 1: Raw Cross-Correlation with FDR Correction

For each pair of indicators (A, B), we compute the Pearson cross-correlation at every lag from 0 to 16 quarters. The null hypothesis is that A at time t and B at time t + lag are uncorrelated.

Testing every candidate pair generates a massive multiple comparisons problem. Raw p-values would produce hundreds of false positives. We apply the Benjamini-Hochberg procedure for False Discovery Rate (FDR) control at α = 0.05, which bounds the expected proportion of false discoveries rather than the probability of any single false discovery. This is more appropriate than Bonferroni correction for exploratory analysis, where we expect a nonzero proportion of true relationships.

The significant candidates advance to the first-difference test rather than publishing from raw-level correlation alone.

Filter 2: First-Differenced Cross-Correlation

Two economic time series that both trend upward over a decade will show spurious correlation even if they’re mechanistically unrelated. This is the “nonsense correlation” problem documented by Granger and Newbold (1974).

We recompute cross-correlations on first-differenced series, the change from one quarter to the next, which strips out shared trends and tests whether changes in A are associated with later changes in B. Only pairs that are significant in both raw levels and first differences survive.

The screening manifest records 5,228 relationships that survive both the raw-level and first-difference filters.

Filter 3: Multi-Crisis Validation

A leading relationship that holds during tranquil periods but breaks during crises has limited practical value. Crises are precisely when leading indicators matter most.

We define three crisis windows:

• Great Financial Crisis (GFC): 2007-Q3 through 2009-Q2
• COVID recession: 2020-Q1 through 2020-Q3
• 2001 recession: 2001-Q1 through 2001-Q4

For each pair, we check whether the leading relationship (correct sign, plausible lag) holds during each crisis window. Pairs must validate during at least one crisis to pass. Of 5,228 dual-filter survivors, 1,031 validated during one or more crises.

The strongest relationships validate across multiple crises. Initial Claims → Unemployment Rate validated during the GFC and COVID. That pair remains inside the current fully validated set, rather than relying on a relationship whose follower has no defined distress pole.

Filter 4: Granger Testing

Cross-correlation establishes co-movement with a time offset. Granger causality goes further: it tests whether lagged values of A contain information about B that is not already contained in B’s own lagged values. In other words, does knowing A’s past improve our forecast of B beyond what B’s own history provides?

We apply grangercausalitytests from the statsmodels library at the optimal lag identified in Filter 1. The null hypothesis is that A does not Granger-cause B. We reject at p < 0.05.

The screening manifest records 30 relationships tested for Granger causality and 12 passes. A pair that fails here may have genuine cross-correlation while the leader’s lagged information adds nothing beyond the follower’s own autoregressive structure under this test.

Filter 5: Out-of-Sample Replication

The four preceding filters use the full sample period. This creates a risk of overfitting: the pipeline might identify relationships that fit historical data perfectly but fail on new data.

We split the sample at Q1 2013:

• Calibration window: observations before Q1 2013 (includes the GFC and 2001 recession)
• Validation window: observations from Q1 2013 onward

We re-estimate the cross-correlation using only the calibration window, then compute the correlation at the same lag on the validation window. Pairs must achieve an out-of-sample correlation |r| > 0.3 to pass.

Of 12 Granger-validated pairs tested out of sample, 9 replicated.

Results

Validated Leading Relationships

Leader	Follower	Lag	r	Crises	Granger F	OOS r
Wage Growth vs CPI Spread	The K-Shape	2q	0.90	2 (GFC, COVID)	3.6	0.93
Initial Unemployment Claims (SA)	Unemployment Rate	1q	0.85	2 (GFC, COVID)	62.9	0.80
Charge-Off Rate on Single-Family Residential Mortgages	Charge-Off Rate on All Loans	0q	0.82	2 (GFC, 2001)	4.0	0.46
Energy CPI	Wage-Inflation Gap	1q	-0.74	2 (GFC, COVID)	6.1	-0.82
The Late Fee	Charge-Off Rate on All Loans	3q	0.66	2 (GFC, 2001)	6.1	0.83
The Wedge	The Pump Tax	0q	0.85	2 (GFC, COVID)	5.0	0.62
The Floor	Motor Vehicle Insurance CPI	3q	-0.79	2 (GFC, COVID)	3.4	-0.87
Initial Unemployment Claims (SA)	The Short Shift	1q	0.70	2 (GFC, COVID)	19.2	0.58
Delinquency Rate on Consumer Loans	The Other Banks	13q	-0.59	2 (GFC, 2001)	2.0	-0.74

Two validated pairs share the same leader: Initial Claims. Initial filings are a real-time proxy for labor-market deterioration. They lead both the unemployment rate and involuntary part-time work by 1 quarter in the current scanner output.

The Wage Growth vs CPI Spread → Wage-Inflation Gap pair records a 2 quarters historical offset. Its out-of-sample r of 0.93 remains close to the full-sample r of 0.90.

In the raw-level historical test, the Credit Card Delinquency → All-Loan Charge-Off pair records the credit-card delinquency level ahead of the aggregate all-loan charge-off level by 3 quarters. That raw-level offset is not the pair’s first-difference lag, and the aggregate series do not show that the same households or loan types moved from one state to another.

The Energy CPI → Wage-Inflation Gap relationship is negative in the scanner (r = -0.74) at the recorded lag. That association does not by itself identify a wage-adjustment mechanism or a causal effect of energy prices.

The Pipeline Funnel

Stage	Count	Survival Rate
Raw lagged cross-correlation tests	87,281	—
Unique relationships (dual filter)	5,228	—
Crisis-validated (1+ crisis)	1,031	19.7% of dual
Granger causality tested	30	Top candidates
Granger passed	12	40.0% of tested
Out-of-sample validated	9	75.0% of Granger
Fully validated	9	0.17% of dual-filter relationships

Cascade Sequences

Beyond pairwise relationships, we inventory transitive chains where the artifact contains an A→B edge and a B→C edge. The cumulative lag is the sum of those recorded pair lags; the chain is not a separately validated A→C causal or predictive effect. The cascade detector identified 23 chains after deduplication and publishes the 23 that span multiple ADI domains.

Top cascade sequences:

Chain	Cumulative Lag	Score
Household Debt Service Ratio → The Other Banks → Real Federal Minimum Wage → Motor Vehicle Insurance CPI	33 quarters	82.3
Consumer Loan Delinquency → The Other Banks → Real Federal Minimum Wage → Motor Vehicle Insurance CPI	30 quarters	80.6
Credit Card Delinquency → The Other Banks → Real Federal Minimum Wage → Motor Vehicle Insurance CPI	33 quarters	80.1

The top recorded chains link debt-service or delinquency measures to The Other Banks, then to real minimum wages and motor-vehicle insurance costs. These are sequences of pairwise statistical edges, not evidence that one series transmits effects through the chain. The committed family-v1 artifact contains indicator-level results only; it does not establish a domain-to-domain lead from Safety Net & Buffer to Delinquency.

Discussion

What This Analysis Establishes

The scanner identifies historical lagged associations among specific indicator pairs. Several surviving pairs include initial unemployment claims, wage-price measures, and credit-card or consumer-loan delinquency as the earlier series.

This is not forecasting. The results document historical lag structures that passed the recorded validation stages. A currently elevated leader can be compared with that history, but the artifact does not produce a future follower value, horizon-specific probability, or guaranteed timing.

Limitations

Correlation is not causation. Initial claims lead the unemployment rate by 1 quarter in the current output, but that lag does not by itself establish a direct causal effect. The relationships may reflect shared upstream causes or institutional mechanisms rather than direct transmission.

Regime dependence. The relationships validated during the GFC and COVID recessions may not hold during a fundamentally different type of crisis, such as a currency crisis, trade war, or technology-driven displacement. The 2001 window provides another historical test, but the sample of recessions is inherently small.

Small crisis sample. Three crises is the maximum available for the post-2000 data period. Statistical power for crisis-specific validation is limited. A relationship that fails in one crisis may still be structurally valid.

COVID distortion. The COVID recession produced unprecedented speed and magnitude in many indicators (initial claims reached 6.1 million in a single week). While winsorization at the 95th percentile mitigates this, some validated relationships may be influenced by the sheer scale of COVID-era movements.

No component-level inference. The scanner tests indicator pairs. It does not aggregate those results into a domain-to-domain lag, so a component-level lead requires a separate committed model artifact before publication.

Relationship to the ADI

The family-v1 ADI uses five equal-weighted domains. Safety Net & Buffer’s 20.0% weight keeps buffer data visible alongside delinquency, default/legal, debt-burden, and labor domains. Scanner results remain tied to the specific indicator pairs recorded in the research artifact and do not transfer automatically to their parent domains.

The 9 validated pairs also inform the frontpage scoring algorithm: indicators that are part of a validated leading relationship receive additional weight in the newsworthiness score when their current values are elevated.

Appendix: Data Access

The full scanner output, including all 5,228 dual-filtered pair records, filter-stage results, and the 9 fully validated relationships, is available via the API:

• JSON: /api/research/leading-indicators.json
• Cascade sequences: Available in data/research/cascade_sequences.json
• Research charts: Static PNGs at /research/heatmap.png, /research/ccf_plots.png, /research/cascade_diagram.png, /research/funnel.png

To reproduce the analysis:

PYTHONPATH=. python3 scripts/research/leading_indicator_scanner.py
PYTHONPATH=. python3 scripts/research/detect_cascades.py
PYTHONPATH=. python3 scripts/research/generate_research_charts.py

All source code is available in the project repository.