Research

Theoretical foundations for causal evaluation and structural alignment

Reading order: Start with the CJE paper for empirical results, then Structural Alignment Theory for the core framework.

Preprint • December 2025

Causal Judge Evaluation: Calibrated Surrogate Metrics for LLM Systems

AutoCal-R, SIMCal-W, and Oracle-Uncertainty-Aware inference. On 4,961 Arena prompts, CJE achieves 99% pairwise ranking accuracy while reducing oracle labeling costs by 14×.

Landesberg, E.•

Core Theory

Start Here

Structural Alignment Theory

The mathematical framework connecting surrogate metrics to true objectives through causal structure. Introduces the Surrogate-Decision Problem, structural assumptions (S1-S3, L1-L2), and bounds on when proxies work.

45 min•Technical

Economic Framework

Rights, Causation, Friction (RCF)

The economic mechanics of AI accountability. When Fabrication Cost exceeds Verification Cost (F > V), alignment is stable. Introduces the Standard Deliberation Protocol as mechanism design for shifting liability.

20 min•Technical

Protocol Design

Standard Deliberation Protocols

The economics of structured deliberation. SDP raises Fabrication Cost via checkable commitments, lowers Verification Cost via legibility, and increases Detection Probability via the Beckerian multiplier.

12 min•Technical

Design Principle

Design-by-Projection

A general mathematical principle for structure-aware estimation. When you know structural constraints but not the functional form, project onto the constraint set. Unifies AutoCal-R and SIMCal-W.

12 min•Technical

Fundamental Bounds

Coverage-Limited Efficiency

Sharp lower bounds on off-policy standard errors. Why high ESS isn't enough. Logger coverage in target-typical regions and shape mismatch both impose efficiency limits.

10 min•Technical

Unified Framework

The Welfare Compiler

Pretraining, RLHF, and evaluation as three regimes of the same welfare estimation problem. Error decomposition into spec error, cross-stage mismatch, and estimation error.

32 min•Technical

Surrogate Methods

Formal Framework

AI Quality as Surrogacy for Idealized Deliberation

Formal framework with precise definitions, identification results, influence functions, and asymptotic theory for treating AI quality measurement as a surrogate endpoint problem.

25 min•Technical

Judge Design

Programmable Proxies: Technical Appendix

Formal framework for designing, calibrating, and auditing LLM judges as programmable surrogate outcomes. Identification results, closed-loop algorithms, and oracle-uncertainty-aware inference.

45 min•Technical

System Design

Y*-Aligned Systems: Technical Appendix

Formalizes Y*-aligned prompting and judging: anchoring generation and evaluation to the same IDO welfare scale. Proves optimization compatibility, judge-pool invariance, and OPE transportability.

25 min•Technical

Dynamic Methods

State-Space Methods

Continuous Causal Calibration

A state-space framework for fusing biased surrogates with sparse experiments when the world won't stop changing. Introduces the Causal Nyquist Rate and dynamic design-by-projection.

40 min•Technical

Governance & Impact

Closed-Loop Optimization

CLOVER: Closed-Loop Optimization of Valid Evaluators

Formal framework for governed, selective-inference-valid closed-loop optimization of programmable LLM judges. Nested holdouts, OUA variance, and active adversarial validation.

50 min•Technical

Value Assessment

Net IDO Impact

Measuring foundation model value as net advantage over realistic alternatives. Competition-creation decomposition: does your model win because it's good, or because alternatives got worse?

30 min•Technical

Implementation

Reference Implementation

CJE Technical Report

Complete technical documentation of the CJE library. Covers Direct Method, IPS, Doubly Robust estimators, AutoCal-R calibration, SIMCal-W weight stabilization, and diagnostic systems.

60 min•Technical

Empirical Validation

Flagship Benchmark

Arena Experiment: 14 Estimators on 5k Evaluations

Comprehensive evaluation of Direct Model, IPS, and Doubly Robust estimators on real Arena data. Direct Model with covariates achieves 99% pairwise ranking accuracy while SNIPS fails completely.

45 min•Technical

Assumption Testing

Bridge Validation Protocol (BVP)

Testing the Bridge Assumption (A0): that operational welfare Y aligns with idealized target Y*. Introduces PTE-based validation and SDP-Gov governance framework.

35 min•Technical

Stress Testing

Robustness Under Pressure

Measuring robustness under optimization pressure. Formalizes the Goodhart Point, Optimization Gap, and CLOVER-A stress testing methodology.

25 min•Technical

Related Resources

Blog

Conceptual introductions

CJE Documentation

Implement these methods

Assumptions

When CJE works