Research

Theoretical foundations for causal evaluation and structural alignment

Reading order: Start with the CJE paper and the validated work below. Everything else is grouped by status: work in progress, then essays and positions.

Preprint • December 2025

Causal Judge Evaluation: Calibrated Surrogate Metrics for LLM Systems

Reward calibration, transport audits, and calibration-aware inference. On 4,961 Arena prompts, CJE makes Direct + two-stage calibration the default and reduces oracle labeling costs by 14× (arXiv v3).

Landesberg, E.•

Validated

Empirical results traceable to reproducible artifacts.

Reference Implementation

CJE Technical Report

Complete technical documentation of the CJE method. Covers the Direct Method, reward calibration, calibration-aware inference, diagnostic systems — and the benchmark evidence for why the off-policy estimators (IPS, DR) were removed from the library.

60 min•Technical

Drift Correction

Post-Audit Drift Correction: Offset vs EIF Correction vs Refit

Empirical and theoretical treatment of what to do after a failed transport audit: global offset baseline, policy-specific first-moment correction, and refit escalation under structural drift.

12 min•Technical

Judge Calibration Audit

HealthBench Rubric Judgments Depend on Which Judge You Use

Two custom judges on the HealthBench Consensus meta-evaluation data differ by up to 73.2 percentage points on specific criteria. A one-seed 5% masked-label replay nearly recovers the declared aggregate endpoint, while criterion-level conclusions remain judge-sensitive.

20 min•Technical

In Progress

Real questions with incomplete or withdrawn formalizations.

Diagnostics

Coverage-Limited Efficiency

Why high ESS isn't enough: a coverage diagnostic for off-policy standard errors. The proposed formal lower bound is withdrawn; the coverage diagnosis stands.

10 min•Technical

Closed-Loop Optimization

CLOVER: Closed-Loop Optimization of Valid Evaluators

Formal framework for governed, selective-inference-valid closed-loop optimization of programmable LLM judges. Nested holdouts, calibration uncertainty variance, and active adversarial validation.

50 min•Technical

Assumption Testing

Bridge Validation Protocol (BVP)

Testing the Bridge Assumption (A0): that operational welfare Y aligns with idealized target Y*. Introduces PTE-based validation and SDP-Gov governance framework.

35 min•Technical

Stress Testing

Robustness Under Pressure

Measuring robustness under optimization pressure. Formalizes the Goodhart Point, Optimization Gap, and CLOVER-A stress testing methodology.

25 min•Technical

Value Assessment

Net IDO Impact

Measuring foundation model value as net advantage over realistic alternatives. Competition-creation decomposition: does your model win because it's good, or because alternatives got worse?

30 min•Technical

Essays & Positions

Lenses and positions, not validated results.

Framework

Structural Alignment Theory

A proposed mathematical framework connecting surrogate metrics to true objectives through causal structure. Introduces the Surrogate-Decision Problem, structural assumptions (S1-S3, L1-L2), and bounds on when proxies work.

45 min•Technical

Economic Framework

Rights, Causation, Friction (RCF)

The economic mechanics of AI accountability. Argues that when Fabrication Cost exceeds Verification Cost (F > V), alignment is stable. Introduces the Standard Deliberation Protocol as mechanism design for shifting liability.

20 min•Technical

Protocol Design

Standard Deliberation Protocols

The economics of structured deliberation. SDP raises Fabrication Cost via checkable commitments, lowers Verification Cost via legibility, and increases Detection Probability via the Beckerian multiplier.

12 min•Technical

System Design

Y*-Aligned Systems: Technical Appendix

Formalizes Y*-aligned prompting and judging: anchoring generation and evaluation to the same IDO welfare scale. Argues for optimization compatibility, judge-pool invariance, and OPE transportability.

25 min•Technical

Unified Framework

The Welfare Compiler

Pretraining, RLHF, and evaluation as three regimes of the same welfare estimation problem. Error decomposition into spec error, cross-stage mismatch, and estimation error.

32 min•Technical

State-Space Methods

Continuous Causal Calibration

A state-space framework for fusing biased surrogates with sparse experiments when the world won't stop changing. Introduces the Causal Nyquist Rate and dynamic constraint-set projection.

40 min•Technical

Formal Framework

AI Quality as Surrogacy for Idealized Deliberation

Formal framework with precise definitions, identification results, influence functions, and asymptotic theory for treating AI quality measurement as a surrogate endpoint problem.

25 min•Technical

Related Resources

Blog

Conceptual introductions

CJE Documentation

Implement these methods

Assumptions

When CJE works