Skip to content

AN-010: Imhof full pipeline benchmark

Intuition (plain-language)

The literature's workhorse screen (Imhof–Wallimann) reads seven moments off the full bid distribution — powerful, but it needs expensive bid microdata. The cheap award-layer FL screen uses only who participated and lost. Each alone lands near AUC 0.89–0.90; stacked, they reach 0.955. The economically important word is complementarity, not dominance: the two layers observe collusion at different evidentiary stages, so combining them adds genuine information rather than counting the same signal twice.

Question

How does the seven-feature Imhof–Wallimann bid-distribution pipeline perform on the cobidder target, and what is the increment from adding the award-layer score? The joint score is the full-observability upper bound.

Design

  • Sample: pool of 16,779 firms with both award and bid features available in BEC 2009–2019.
  • Specifications:
  • Imhof CV-only: within-tender bid coefficient of variation.
  • Imhof full pipeline: seven-feature bid-distribution set.
  • FL14 alone: binary award-layer indicator.
  • log_tc alone: continuous award-layer score.
  • Joint (Imhof + FL14): stacked features.
  • Joint (Imhof + log_tc): stacked with continuous.
  • Outcome: AUC against the cobidder target; DeLong test for pairwise AUC differences \citep{imhof2018screening,imhof2019detecting,wallimann2023machine}.

Results

Specification AUC 95% CI
Imhof CV-only 0.585 [0.553, 0.616]
Imhof full pipeline 0.888 [0.865, 0.911]
FL14 alone (binary) 0.921 [0.914, 0.928]
log_tc alone (continuous) 0.884 [0.860, 0.908]
Joint Imhof + FL14 0.955 [0.943, 0.967]
Joint Imhof + log_tc 0.962 [0.954, 0.969]

Macros: \valAUCImhofCV, \valAUCImhofFull, \valImhofFLBin, \valImhofFLcont, \valImhofComboBin, \valImhofComboCont, \valImhofPoolN.

Increment magnitudes (AN-015, [AN-049 ref] script 49): \valAUCFLvsImhofDelta = 0.035 (FL14 vs Imhof full, p = 0.014); \valAUCImhofPlusFLDelta = 0.096 (Imhof + FL14 vs Imhof full).

AN-010 Imhof full pipeline + complementarity

Figure: AUC across the six specifications on the five-fold CV pool (N = 16,779; 193 cobidders). Imhof CV-only is chance-level (0.585); Imhof full pipeline 0.888; FL14 alone 0.921; joint (Imhof + FL14) 0.955; joint (Imhof + log_tc) 0.962. The award-layer + bid-layer complementarity adds ~0.07 over either alone.

Interpretation

The Imhof full pipeline is comparable to FL14 alone (0.888 vs 0.921), not dominated by it. The headline is complementarity: the joint score adds roughly 0.05–0.07 AUC over either layer individually. The two information layers operate at different evidentiary stages — award-layer triages, bid-layer evaluates — and their union is the full-observability upper bound.

The pure bid-only CV reading (0.585) is chance-level alone; what makes the Imhof pipeline informative is the inclusion of participation features. The award-layer signal is therefore not redundant; it is necessary for the bid-distribution pipeline to reach AUC 0.888 in the first place.

This is the cost-of-evidence framing of §6: AN-012 reports the operational gatekeeping numbers; the joint score is the counterfactual that gatekeeping approximates at lower cost.

Follow-ups

  • Decomposition of the 0.05–0.07 increment by Imhof feature.
  • Modal-by-modal increment (AN-016).
  • Robustness to alternative bid-distribution feature sets.