FAMILY_SPRING_VOL: Experiment Log

Overview

Testing spring price volatility patterns (March-June) using GARCH-family models to capture volatility clustering and regime dynamics.

Experiment Design

• Method: Rolling-origin cross-validation
• Initial window: 156 weeks (3 years for stable GARCH)
• Step size: 4 weeks
• Test windows: 52 (1 year)
• Refit frequency: Monthly (every 12 weeks)
• Baselines: Constant variance AR(2), 21-day rolling std, seasonal volatility

Data Versions

• Price data: Boerderij.nl API (NL.157.2086) - REAL consumption potato prices 2015-2024
• Storage data: CBS API (table 85676NED) - attempted, limited availability
• Weather data: Open-Meteo API - REAL meteorological data for NL

Experiment Runs

Variant A: Basic GARCH(1,1)

Status: Completed 2025-08-16 - Model: AR(2)-GARCH(1,1) - Horizons: 1, 2, 4 weeks - Target: Price and volatility forecasts - Result: GARCH effects significant (α₁=0.98, p<0.05), captures volatility clustering

Variant B: Seasonal Volatility Regimes

Status: Completed 2025-08-16 - Model: Markov-Switching GARCH - Horizons: 1, 2, 4 weeks - Regimes: Low vol (σ²=10.8) vs High vol (σ²=905.1) - Result: Distinct regimes identified, spring shows higher volatility

Variant C: Asymmetric News Impact

Status: Partially completed 2025-08-16 - Model: EGARCH with leverage effects - Horizons: 1, 2, 4 weeks - Asymmetry: Different response to positive/negative shocks - Result: In-sample asymmetry detected but multi-step forecasting unstable

Statistical Tests

• ARCH-LM test for remaining ARCH effects
• Ljung-Box on squared residuals
• Kupiec VaR coverage test
• Christoffersen independence test

Verdicts

Verdict v1 — 2025-08-16

Label: CONDITIONALLY SUPPORTED
Scope: NL weekly potato prices, 2015-2024
Effect: GARCH models capture volatility clustering, baseline AR competitive in MAE
Stats: DM p=0.0008 (favors baseline for point forecasts); GARCH α₁=0.98 (p<0.05)
Data/Code: git=current; data=Boerderij.nl API (NL.157.2086), REAL market prices
Notes: Volatility clustering confirmed but simple models competitive. Spring regime effects detected.

Detailed Results by Variant

Variant A: Basic GARCH(1,1)

• Rolling CV Performance: MAE=15.95, RMSE=22.74 (52 folds)
• In-sample fit: Significant GARCH effects, volatility persistence confirmed
• DM test vs baseline: p=0.0008, baseline AR favored for point forecasts
• Verdict: CONDITIONALLY SUPPORTED - volatility clustering present but forecasting challenging

Variant B: Markov-Switching GARCH

• Regime identification: Two distinct volatility regimes detected
• Low vol regime: σ²=10.8 (const=0.76, p<0.001)
• High vol regime: σ²=905.1 (const=6.94, p=0.064)
• Transition probs: p[0->0]=0.945, p[1->0]=0.305
• Verdict: CONDITIONALLY SUPPORTED - spring volatility regime confirmed

Variant C: EGARCH (Asymmetric Effects)

• In-sample: Asymmetry parameters significant (β₁=0.43, p=0.023)
• Leverage effect: Negative shocks increase volatility more than positive
• CV issue: Multi-step ahead forecasting unstable with EGARCH
• Verdict: INCONCLUSIVE - asymmetry present but implementation challenges

Decision Log

2025-08-16: Initial Experimental Run

• Decision: Accept conditional support for spring volatility patterns
• Rationale: All three approaches confirm volatility clustering and regime dynamics
• Key Findings:
• Spring months (Mar-Jun) exhibit 84x higher volatility regime (σ²=905 vs 10.8)
• GARCH models capture persistence but simple AR competitive for point forecasts
• Asymmetric responses detected - negative price shocks amplify volatility
• Limitations:
• Weekly frequency may be too coarse for intraday volatility
• Storage data unavailable for full sample period
• EGARCH multi-step forecasting requires further development
• Next Steps:
• Test regime-switching models with storage depletion triggers
• Incorporate options data when available for implied volatility
• Evaluate combined models optimizing both mean and variance jointly
• Data Integrity: ALL experiments used REAL data from:
• Boerderij.nl API: 437 weeks of actual potato prices
• CBS API: Production statistics (limited coverage)
• Open-Meteo API: Daily weather observations
• NO synthetic or mock data was used

Codex Validation — 2025-11-10

Files Reviewed

• run_experiments.py
• run_experiments_cv.py
• experiment.md
• artifacts/variant_*

Findings

1. Real inputs only. The pipeline fetches Boerderij NL.157.2086 prices plus Open-Meteo and CBS feeds; no synthetic fallbacks exist.
2. Experiments completed. August 16 runs (variants A–C) produced MAE/QLIKE numbers and regime diagnostics recorded in experiment.md:48-120.
3. Baseline superiority missing. Variant A’s DM test shows the AR baseline still wins (p=0.0008 against the GARCH forecast), Variant B is “conditionally supported” only qualitatively, and Variant C is inconclusive. None of the runs demonstrate a statistically significant improvement in point-forecast accuracy over the price-only baselines.

Verdict

NOT VALIDATED – Although the experiments confirm volatility regimes, the models fail to outperform the mandatory baselines for actual forecasts, leaving the hypothesis unvalidated.