FAMILY_PARCEL_CONSOLIDATION_MOMENTUM: Experiment Log

Overview

Testing how parcel consolidation momentum creates predictable Dutch potato price movements through efficiency acceleration, market power concentration, and technology adoption amplification using REAL DATA from validated BRP API methodology.

Hypothesis Origins

• Prior experiments:
• FAMILY_PARCEL_DYNAMICS (INCONCLUSIVE): Successfully extracted 34,301 real parcels across 88,879 hectares but focused on static metrics missing consolidation MOMENTUM
• FAMILY_SEASONAL_PLANTING (INCONCLUSIVE): 53% improvement at 30-day showed operational importance but missed consolidation efficiency effects

• FAMILY_PLANTING_INTENSITY_SIGNALS (REFUTED): Validated BRP methodology for large-scale analysis but wrong focus on clustering vs consolidation momentum

• Industry catalyst:

• 2024 Flevoland consolidation: 6,000+ small parcels to ~4,500 larger operational units
• Small farmer exit acceleration creating operational mergers
• Technology adoption thresholds requiring scale (precision agriculture >50ha)

• Supply chain efficiency gains through consolidation documented

• Academic basis:

• Krugman (1991) spatial economics and consolidation effects
• Key & Roberts (2009) farm size efficiency relationships (12-18% cost advantages)

• Brorsen & Anderson (2012) agricultural market momentum transmission

• Data opportunity: BRP API provides temporal consolidation tracking unavailable elsewhere

Revolutionary Innovation: Consolidation Momentum vs Static Analysis

Key Breakthrough: Focus on temporal acceleration of consolidation rather than static consolidation levels:

1. Consolidation Velocity: Rate of merger acceleration (d²/dt²) signals efficiency momentum
2. Market Power Momentum: HHI acceleration indicates concentration effects
3. Technology Adoption Momentum: Large parcel formation velocity enables precision agriculture

Expected Impact: 10-18% improvement vs strongest baseline through forward-looking consolidation signals

Experiment Design

• Method: Rolling-origin cross-validation with mandatory standard baselines
• Initial window: 156 weeks (3 years)
• Step size: 4 weeks
• Test windows: 52 weeks (1 year)
• Refit frequency: Every 12 weeks (quarterly)
• Baselines: MANDATORY - persistent, seasonal_naive, ar2, historical_mean via get_standard_baselines()
• REAL DATA ONLY: BRP parcel boundaries, Boerderij.nl prices, CBS validation, Open-Meteo weather

Data Sources (REAL DATA ONLY)

• BRP API: BRPApi().get_consumption_potato_mask() for consolidation analysis - git:current
• Boerderij.nl API: Product NL.157.2086 (consumption potatoes) - git:current
• CBS API: Table 85676NED for validation context - version 2024-Q4
• Open-Meteo API: Weather controls at 52.55°N, 5.55°E - git:current
• Validated foundation: 34,301 parcels extracted in FAMILY_PARCEL_DYNAMICS
• NO synthetic, mock, or dummy data permitted

Experiment Runs

Variant A: Consolidation Velocity Effects

Status: Ready for EX implementation - Model: Ridge regression with consolidation momentum features - Features: merger_rate_acceleration, efficiency_momentum_index, operational_scaling_signals, price_lags, seasonal/weather controls - Horizons: 30-day, 60-day ahead - Target: Test if 5%+ merger acceleration leads to 8-15% efficiency-driven price improvement - Expected: 10-18% MASE improvement over strongest baseline - Mechanism: Consolidation velocity signals supply chain optimization momentum

Variant B: Market Power Concentration

Status: Ready for EX implementation - Model: Random forest with market concentration features - Features: hhi_momentum, concentration_acceleration, dominance_signals, coordination_potential_index, competitive_pressure_decline, price_lags - Horizons: 30-day, 60-day ahead - Target: Test if 10%+ HHI momentum creates 12-20% price influence via coordination - Expected: 12-20% MASE improvement over strongest baseline - Mechanism: Market power concentration enables strategic price coordination

Variant C: Technology Adoption Momentum

Status: Ready for EX implementation - Model: Gradient boosting with technology adoption features - Features: large_parcel_momentum, scale_threshold_crossings, technology_adoption_acceleration, investment_momentum_signals, price_lags - Horizons: 30-day, 60-day ahead - Target: Test if 15%+ large parcel velocity drives 10-18% yield optimization effects - Expected: 10-18% MASE improvement over strongest baseline - Mechanism: Technology adoption requires scale thresholds - momentum signals crossing points

Implementation Notes

Consolidation Momentum Calculations

# Example: Calculate merger rate acceleration from REAL BRP data
from src.sources.brp.brp_api.brp import BRPApi
import numpy as np

# Validated methodology from FAMILY_PARCEL_DYNAMICS
brp = BRPApi()

# Get annual consolidation metrics
for year in range(2015, 2025):
    parcels_curr = brp.get_consumption_potato_mask(bbox, year_range, grid_shape)
    parcels_prev = brp.get_consumption_potato_mask(bbox, prev_year_range, grid_shape)

    # Calculate momentum metrics
    merger_rate = (parcels_prev_count - parcels_curr_count) / parcels_prev_count
    efficiency_momentum = (avg_size_curr - avg_size_prev) / avg_size_prev

    # Acceleration (second derivative)
    merger_acceleration = merger_rate_curr - merger_rate_prev

Key Innovations and Risks

Revolutionary Innovations

1. Temporal Momentum Focus: First analysis of consolidation VELOCITY vs levels
2. Efficiency Acceleration Signals: Forward-looking consolidation momentum
3. Market Power Dynamics: HHI momentum rather than static concentration
4. Technology Threshold Crossing: Scale-enabled adoption momentum detection
5. Validated Data Foundation: Building on 34,301 real parcel extraction success

Risk Mitigation

1. Annual Data Resolution: Use multiple years and temporal derivatives for robust momentum calculation
2. Consolidation Proxy Validation: Cross-reference with CBS production statistics
3. Technology Adoption Inference: Validate scale thresholds with industry benchmarks
4. Conservative SESOI: 5% threshold despite higher expected improvements

Success Criteria

Statistical Requirements (MANDATORY)

• Standard Baselines: ALL 4 required (persistent, seasonal_naive, ar2, historical_mean) via get_standard_baselines()
• Comparison Strategy: Compare against STRONGEST baseline performer
• Statistical Significance: p < 0.05 via Diebold-Mariano with HLN correction
• Practical Significance: >5% MASE improvement (SESOI)
• Consistency: >70% of CV folds must show improvement

Business Impact

• Enable anticipation of supply chain efficiency improvements
• Predict market structure changes affecting price dynamics
• Guide investment timing for technology adoption
• Improve forecasting through consolidation momentum intelligence

Next Steps for EX Implementation

1. Implement full consolidation momentum calculation pipeline
2. Create rolling CV framework with momentum feature engineering
3. Run MANDATORY standard baselines for all variants
4. Execute experiments with proper statistical testing
5. Compare against strongest baseline performer
6. Record results with complete methodology documentation

Decision Log

• 2025-08-19: Hypothesis family created building on FAMILY_PARCEL_DYNAMICS validated foundation
• Revolutionary focus: Consolidation MOMENTUM vs static metrics
• Expected impact: 10-18% improvement through temporal acceleration signals
• Ready for EX agent implementation with REAL DATA framework established