Research

Theoretical foundations for causal evaluation and structural alignment

Reading order: Start with Structural Alignment Theory for the core framework, then explore the Manifesto (why this matters) or dive into specific methods below.

Core Theory

Start Here

Structural Alignment Theory

The mathematical framework connecting surrogate metrics to true objectives through causal structure. Introduces the Surrogate-Decision Problem, structural assumptions (S1-S3, L1-L2), and bounds on when proxies work.

45 min•Technical

Position Paper

Alignment Manifesto

Why alignment is fundamentally an information geometry problem. The case for treating proxy-target relationships as measurable causal structures rather than philosophical puzzles.

20 min•Conceptual

Geometric Structure

Alignment Topology

The geometric structure of the proxy-target mapping space. How calibration creates continuous paths between surrogate metrics and true objectives, and when those paths break.

25 min•Technical

Design Principle

Design-by-Projection

A general mathematical principle for structure-aware estimation. When you know structural constraints but not the functional form, project onto the constraint set. Unifies AutoCal-R and SIMCal-W.

12 min•Technical

Fundamental Bounds

Coverage-Limited Efficiency

Sharp lower bounds on off-policy standard errors. Why high ESS isn't enough—logger coverage in target-typical regions and shape mismatch both impose efficiency limits.

10 min•Technical

Unified Framework

The Welfare Compiler

Pretraining, RLHF, and evaluation as three regimes of the same welfare estimation problem. Error decomposition into spec error, cross-stage mismatch, and estimation error.

32 min•Technical

Geometric Structure

The Geometry of Goodhart's Law

Reward hacking isn't noise—it's a vector direction. Defines the Causal Information Manifold as the geometric subspace where surrogate scores remain predictive of welfare, and why optimization must follow geodesics.

15 min•Technical

Surrogate Methods

Formal Framework

AI Quality as Surrogacy for Idealized Deliberation

Formal framework with precise definitions, identification results, influence functions, and asymptotic theory for treating AI quality measurement as a surrogate endpoint problem.

25 min•Technical

Judge Design

Programmable Proxies: Technical Appendix

Formal framework for designing, calibrating, and auditing LLM judges as programmable surrogate outcomes. Identification results, closed-loop algorithms, and oracle-uncertainty-aware inference.

45 min•Technical

Key Insight

The Surrogate Paradox

Why high-correlation judges fail under optimization. Reward hacking isn't a data quality problem—it's a causal topology problem. Optimization requires causal mediation, not just correlation.

18 min•Conceptual

System Design

Y*-Aligned Systems: Technical Appendix

Formalizes Y*-aligned prompting and judging—anchoring generation and evaluation to the same IDO welfare scale. Proves optimization compatibility, judge-pool invariance, and OPE transportability.

25 min•Technical

Dynamic Methods

State-Space Methods

Continuous Causal Calibration

A state-space framework for fusing biased surrogates with sparse experiments when the world won't stop changing. Introduces the Causal Nyquist Rate and dynamic design-by-projection.

40 min•Technical

Governance & Impact

Closed-Loop Optimization

CLOVER: Closed-Loop Optimization of Valid Evaluators

Formal framework for governed, selective-inference-valid closed-loop optimization of programmable LLM judges. Nested holdouts, OUA variance, and active adversarial validation.

50 min•Technical

Value Assessment

Net IDO Impact

Measuring foundation model value as net advantage over realistic alternatives. Competition-creation decomposition: does your model win because it's good, or because alternatives got worse?

30 min•Technical

Implementation

Reference Implementation

CJE Technical Report

Complete technical documentation of the CJE library. Covers Direct Method, IPS, Doubly Robust estimators, AutoCal-R calibration, SIMCal-W weight stabilization, and diagnostic systems.

60 min•Technical

Empirical Validation

Flagship Benchmark

Arena Experiment: 14 Estimators on 5k Evaluations

Comprehensive evaluation of Direct Model, IPS, and Doubly Robust estimators on real Arena data. Direct Model with covariates achieves 94% pairwise ranking accuracy while SNIPS fails completely.

45 min•Technical

Assumption Testing

Bridge Validation Protocol (BVP)

Testing the Bridge Assumption (A0): that operational welfare Y aligns with idealized target Y*. Introduces PTE-based validation and SDP-Gov governance framework.

35 min•Technical

Stress Testing

Robustness Under Pressure

Measuring robustness under optimization pressure. Formalizes the Goodhart Point, Optimization Gap, and CLOVER-A stress testing methodology.

25 min•Technical

Related Resources

Blog

Conceptual introductions

CJE Documentation

Implement these methods

Glossary

Key terms and assumptions