§01 Setup
Observing is an economic decision
Safe reinforcement learning standardised the cost of violating a constraint as a first-class channel of the environment API, so that whatever the task, a harness can always ask how much harm a policy caused. We have found no established environment API that does the same for the cost of information, although in real engineering the act of observing is itself an economic decision, since an inspection immobilises the asset, a high-fidelity simulation burns days of compute, and a destructive test consumes the very part it was meant to qualify.
COIN (Cost-Of-INformation environments) makes that decision explicit. Every action an agent can take declares whether it changes the system or changes what is known about the system, and carries a price either way; every step reports where the money went, physical, informational or failure; and nothing about the true state is observable except through the channels the environment has declared. The contract is deliberately small, because it was extracted from two working environments rather than designed in advance, and anything those two environments did not force was left out.
The repository is public under Apache-2.0 and labelled what it is: a research preview, v0.4, experimental, published with its defects declared. This note states what the two environments show, what is deliberately not claimed, and where the contract will be decided.
- Project
- coin-envs · v0.4 · Apache-2.0
- Status
- Research preview · experimental · active
- Contract
- 3 rules · typed, priced actions
- Environments
- CrackEnv · TurbofanEnv
- Tuning
- seeds 1000–1299 · every candidate archived
- Final evaluation
- seeds 3000–3499 · run once · 500 episodes
- Next gate
- a third, structurally different environment
- Claims register
- CLAIMS.md · kill conditions
§02 The contract
Three rules, extracted rather than decreed
The contract was written after the two environments below had already run the same reference-policy protocol under it unchanged, and it holds exactly three rules, on the stated ground that two environments cannot justify more. Compliance is not a matter of intent: a generic test file runs the same assertions over every registered environment, so an environment is COIN-compliant precisely when tests/test_contract.py passes for it.
Every action declares its nature and its price. An action is PHYSICAL (it changes the system: continue, derate, repair, replace) or EPISTEMIC (it changes what is known about the system: inspect, test), and carries a non-negative price paid when executed. The kind states the action's primary nature and decides which cost channel its price flows into; side effects in the other direction are permitted rather than forbidden, since a repair that reveals something is realistic.
The information cost is a first-class channel. Every step returns a three-way breakdown, physical, information and failure, with the reward defined as their negated sum, so that an agent or a harness can always answer the question the contract exists for: how much of this policy's cost went into reducing ignorance? The rule is the deliberate analogue, for the price of knowing, of the cost channel that safe reinforcement learning standardised for constraint violation.
No undeclared free observations. Whatever arrives without payment sits under a declared free-observation field, which may be empty and may be rich, the two reference environments spanning that spectrum on purpose; everything else must be bought through a priced epistemic action. Ground truth is never part of the step return: since v0.4, policies receive an AgentView proxy that forwards the declared model and raises on everything else, privileged view, hidden state and random streams included, so that reaching around the boundary is a visible, testable interface breach rather than a quiet possibility.
ActionType(name="ut_inspection", kind=EPISTEMIC, price=3.0) # declared and priced
obs, reward, done, info = env.step(action)
info["costs"] # {"physical": .., "information": .., "failure": ..}
reward == -(physical + information + failure)
policy.act(AgentView(env)) # declared model only
view.privileged_view() # raises: harness-only channelIn-process Python cannot make cheating strictly impossible; the enforceable boundary is that a breach is detectable, and dedicated causal tests cover each mechanism (tests/test_causal.py, 40 tests, no skips). What contract v0.1 refuses to fix is listed with the same care: a parameterised intervention grammar, a registry of information sources with declared validity domains, and safety limits as contract objects all wait on a third environment forcing them.
§03 Environments & protocol
Two environments, one frozen protocol
Every policy hyperparameter, the inspection calendars, the condition-based-maintenance thresholds, the value-of-information windows and the oracle thresholds alike, was tuned on seeds 1000–1299 with every candidate score archived; the code was then frozen and evaluated once, on seeds 3000–3499, which no development run had touched. Failure counts are exact, and every comparison is paired by seed with a 95% confidence interval.
Seeds 0–499 were burned during development; 2000–2499 carried the published v0.3 numbers and are not reused. The receding decision window of the value-of-information planner is itself load-bearing: the archived grid shows the full-horizon ablation at 156.7 per episode on CrackEnv against 37.0 for the selected window, and 182.1 against 58.9 on TurbofanEnv (results/tuning_log.json).
CrackEnv · when nothing useful is free
A hidden fatigue crack grows under cyclic loading, following Paris-Erdogan kinetics with Virkler-style scatter, and nothing useful about it is observable for free: the agent pays for inspections (visual, penetrant or ultrasonic, each with its own probability-of-detection curve, sizing noise, false calls and price) and chooses between continuing, derating, repairing and replacing. Where information must be bought, buying it well is worth a great deal, since the greedy value-of-information policy ends about 44% cheaper than the best tuned calendar, at failure counts too close to separate.
Exactly: 34.25 ± 3.28 against 61.00 ± 2.94 in total cost, a paired difference of +26.75 [+19.06, +34.45]; failures 9 against 7 out of 500, so no safety claim is made on this environment. The cost advantage holds in all nine cells of a failure-cost × inspection-price grid, +20.0 to +59.1 with every interval excluding zero (results/sensitivity.md).
| Policy | Total cost | Failures | Inspections / ep | Δ vs VoI · 95% CI |
|---|---|---|---|---|
| run to failure | 332.64 ± 12.24 | 308/500 | 0 | +298.39 [+275.63, +321.15] |
| periodic replace (k=20) | 92.96 ± 3.70 | 12/500 | 0 | +58.71 [+49.54, +67.89] |
| calendar ultrasonic (k=12, 4 mm) | 61.00 ± 2.94 | 7/500 | 5.0 | +26.75 [+19.06, +34.45] |
| greedy VoI (w=2) | 34.25 ± 3.28 | 9/500 | 6.5 | · |
| oracle threshold (19 mm) | 26.64 ± 0.92 | 0/500 | 0 | −7.61 [−13.67, −1.54] |
| The oracle reads the true state through the harness's privileged channel, explicitly: a tuned reference point, never an optimality bound. | ||||
TurbofanEnv · when free data is rich but biased
The second environment inverts the premise, in the spirit of C-MAPSS, the NASA turbofan-degradation simulation family (sources declared in the repository's third-party notices): onboard sensors report every step for free, noisily, and share an unknown per-episode installation bias, the classic trap of prognostics-and-health-management practice, while paying restores truth through a ground performance test or a borescope inspection. The same contract fits with zero changes to the core, and the result changes shape rather than direction: at nominal sensor quality there is no cost winner, and what the paid tests buy is safety.
Exactly: cost difference CBM − VoI of −7.96 [−17.04, +1.11], the interval containing zero and the free-data policy slightly cheaper on average; failures 2 against 9 out of 500 in favour of the value-of-information policy, at 2.6 paid tests per episode. Paying on a fixed schedule is clearly worse than either, +22.34 [+12.50, +32.18].
| Policy | Total cost | Failures | Paid tests / ep | Δ vs VoI · 95% CI |
|---|---|---|---|---|
| run to failure | 582.12 ± 14.95 | 441/500 | 0 | +518.42 [+491.78, +545.06] |
| periodic overhaul (k=20) | 131.28 ± 9.90 | 54/500 | 0 | +67.58 [+48.40, +86.76] |
| free-sensor CBM (th=0.7, α=0.5) | 55.74 ± 4.39 | 9/500 | 0 | −7.96 [−17.04, +1.11] |
| paid tests on schedule (k=5) | 86.04 ± 5.26 | 15/500 | 12.0 | +22.34 [+12.50, +32.18] |
| greedy VoI (w=4) | 63.70 ± 2.51 | 2/500 | 2.6 | · |
| oracle threshold (0.8) | 40.08 ± 1.21 | 0/500 | 0 | −23.62 [−27.91, −19.33] |
| CBM: condition-based maintenance, a smoothed threshold on the free sensors. VoI: greedy value-of-information planning over a receding window. | ||||
§04 Crossover
The result that needed both environments
Put the two environments side by side and the question sharpens: when is information worth paying for at all? The sensitivity sweep answers with a crossover, interval-backed at every point. While the free sensors are nearly honest, the policy that trusts them wins outright; at the nominal bias, every interval contains zero; and once the bias grows large, the free-data policy collapses while the paying policy barely moves, because its paid measurements do not share the sensors' bias.
At bias σ=0.05 the advantage of free data reaches −14.9 [−23.0, −6.8] and −12.6 [−22.7, −2.6] depending on noise; at σ=0.10 every interval contains zero; at σ=0.20 the free-data policy costs 109 to 116 per episode with 29 to 31 failures per 300, against 61 to 67 and 1 to 4 for the value-of-information policy. Both policies keep their nominal-tuned configurations across the grid, and the free-data family tested is a simple smoothed-threshold rule; whether a smarter estimator with a known-dynamics prior could infer the bias from trajectory shape is recorded in the claims register as an open challenge, not a refuted possibility.
§05 Scope
What is not claimed, and what is open
The claims register states every claim with the test that would kill it, and this page adds none of its own. The register's negative space is as deliberate as its claims.
- No novelty of the ingredients. Planning that mixes physical and epistemic actions is the home ground of the POMDP-maintenance school; value of information goes back to Raiffa; costly observations are active sensing; priced simulator queries are multi-fidelity Bayesian optimisation. The claim is the cross-domain interface and its instrumentation, and the contract itself cites the two closest single-domain precedents: IMP-MARL for structural fatigue and ChemGymRL for chemistry.
- No optimality, no safety superiority, no Gymnasium. The greedy value-of-information policy is a reference whose distance to the oracle is quoted, no safety claim is made on CrackEnv, and the contract explicitly disclaims Gymnasium compatibility, an adapter being future work rather than a promise.
- No realism of absolute costs. Prices follow the normalised-ratio practice of the inspection-planning literature; the ratios are defended, the absolute figures are not the claim.
- Parameter sourcing is partial (C6). The crack parameters sit within published ranges for 2024-T3 aluminium with their deviations declared, and the false-call rates are unverified. On the turbofan side, a calibration study against real NASA FD001 trajectories found real degradation flat for most of life and then sharply accelerating, a fitted acceleration around 36 against the declared 2.0; the declared abstraction was deliberately kept, since integrating the fit would invalidate every published turbofan number, and a calibrated variant with its own tuning is named future work.
- Two verification debts are open (C9, C10). Common random numbers are partial, split streams only; and the 400-particle filter and 24-outcome pre-posterior carry no convergence check yet, with one declared approximation remaining in the failure branch.
The contract's own future is stated as a gate rather than a promise: a third environment, structurally different, the declared candidates being avionics no-fault-found diagnosis with combinatorial fault ambiguity and a design-iteration loop with a multi-fidelity oracle registry, either fits under the three rules or forces the contract to change, and the cross-domain claim stays narrow until that test runs. Learned policies are deliberately absent meanwhile; the oracle gaps, −7.61 [−13.67, −1.54] on the crack environment and −23.62 [−27.91, −19.33] on the turbofan, are the distance a learned method would have to close.
§06 Conclusion
The contract is the contribution
Two environments from different corners of engineering run under the same three rules with no change to the core, and the benchmark they carry cut in both directions, certifying the value-of-information policy where information is scarce while refusing it a cost win where free data is rich. What COIN proposes is smaller and more durable than either result: that the price of knowing deserves the first-class treatment safe reinforcement learning gave the price of harm, because an environment that prices its observations can ask what flat benchmarks cannot, namely how well a policy spends its ignorance-reduction budget.
The preview is published as it stands: claims with kill conditions, seeds burned and declared, defects open where they are open. The third environment will decide whether the contract generalises.
◆ Provenance · what each claim rests on
Figures are quoted as they stand in the repository at v0.4, whose final evaluation ran once on seeds 3000–3499, rather than re-derived for this page; the repository is public and its README lists the commands that regenerate every number above. This is an active research preview: the claims register in the repository, not this page, is the authoritative statement of scope.