Composition-root guard for replica-unsafe state (#271).
Three states, because "is this deployment scaled?" has three honest answers
and collapsing them to two makes one of them useless:
deployment.mode = "multi" — the operator has said there is more
than one replica, so any in-memory shared state is a defect and boot
fails naming every offender.
deployment.mode = "single" — the operator has said there is one.
In-memory state is correct and this says nothing. Warning anyway would
fire on every local run and train people to ignore the warning that
matters.
unset — nothing has been said. This is where the 3am scenario starts,
so it is the state that has to be loud: one consolidated warning naming
what is in memory and what each one costs when scaled.
Which is why deployment.mode has no literal default in HOCON. A baked-in
"single" would make the unset state unreachable and the warning dead code.
This cannot catch the operator who scales without ever setting
deployment.mode — the case the issue describes. A process whose state is
entirely in its own memory has no shared medium through which to observe
peers, so the condition is undetectable from inside precisely when it is
true. The warning and the documentation are what address that; the failure
mode is for operators who have declared their shape.
Composition-root guard for replica-unsafe state (#271).
Three states, because "is this deployment scaled?" has three honest answers and collapsing them to two makes one of them useless:
deployment.mode = "multi"— the operator has said there is more than one replica, so any in-memory shared state is a defect and boot fails naming every offender.deployment.mode = "single"— the operator has said there is one. In-memory state is correct and this says nothing. Warning anyway would fire on every local run and train people to ignore the warning that matters.Which is why
deployment.modehas no literal default in HOCON. A baked-in"single"would make the unset state unreachable and the warning dead code.This cannot catch the operator who scales without ever setting
deployment.mode— the case the issue describes. A process whose state is entirely in its own memory has no shared medium through which to observe peers, so the condition is undetectable from inside precisely when it is true. The warning and the documentation are what address that; the failure mode is for operators who have declared their shape.