A Dry Run Should Prove More Than “Nothing Happened”

Thesis: A useful dry run resolves targets, validates permissions and constraints, produces the exact planned operations, and explains what would block execution.

Dry runs must resolve targets, not guess

A dry run that returns “no changes” is useless if it never looked at the real targets. The first responsibility of a dry run is resolution: expand selectors, follow aliases, and map logical intent to concrete resources. That means turning “region=eu, app=api” into the IPs, instance IDs, database names, or file paths your orchestration will touch.

Failure modes: partial resolution (some IDs missing), stale inventory (cache vs authoritative source), and ambiguous selectors. If resolution fails, the dry run must report which namespace or adapter it tried, the query it used, and whether the data came from cache or authoritative state.

Tradeoff: aggressive live discovery increases fidelity but raises blast radius and latency. Prefer a two-stage approach: a quick authoritative lookup (read-only, scoped least-privilege) followed by optional deeper probes when the operator requests them.

Validate permissions and constraints early

A dry run should tell you whether the actor could actually perform the operations. Permission checks are part of the plan, not a separate runtime surprise.

Concretely, run the same authorization checks the executor will use: service account scopes, IAM policies, reachable SSH keys, API token scopes, and platform rate limits. Where the executor uses delegated credentials, validate the delegation path and token expiry.

Expose constraints too: quota limits, region restrictions, node affinity, and immutable fields. If a change would violate a constraint, the dry run should annotate the planned operation with the violated constraint and the responsible policy (so reviewers can decide to change intent or adjust constraints).

Cost: full permission simulation requires reproducing policy evaluation. If too expensive, at least surface the minimal set of permissions required for each planned operation.

Produce the exact planned operations (and the order)

A dry run should output the precise operations that will run, in the exact order, against the resolved targets. That means commands, API calls with parameters, SQL statements, file-system edits, or orchestration graph nodes with dependencies.

Include: the adapter name, the operation verb, the target identifier, the payload, expected preconditions, and the expected postcondition. For mutable sequences, make explicit whether steps are idempotent and include rollback hints.

Why order matters: many failures are sequence-dependent (migrations, leader elections, draining). Show parallelism and critical path so reviewers can reason about race and blast radius.

Failure mode: a plan that omits side effects (e.g., a job that also updates metadata) creates blind spots. Always call out planned side effects—even ones the operator didn’t intend—so reviewers can assess risk.

Explain what would block execution

A useful dry run is a diagnostic that surfaces blockers, not a wish-list of changes. For each planned operation, include a short, actionable reason if it would be blocked: missing permission, failing precondition, conflicting lock, or an out-of-date dependency.

Blocker examples to expose:

• Permission checks that fail and the exact policy stanza that denied the action.
• Quota or quota-reservation conflicts and which account holds the reservation.
• Concurrent locks or in-flight changes that create a conflict.
• Validation errors from the target (schema mismatch, immutable field change).

Provide remediation steps and a blast-radius estimate for each blocker. If a blocker is an operational decision (e.g., “cluster must be drained”), annotate who or what role is responsible to clear it.

Detect and report drift between plan and execution

Dry runs are most valuable when they reduce the surprise between plan and execution. That requires modeling the execution environment and surfacing expected drift.

Types of drift to report:

• State drift: resource already differs from desired input (what the change would do vs what exists now).
• Temporal drift: checks like token expiry or leader election windows that will change between dry run and execution.
• Environmental drift: configuration or platform-level changes (new admission controllers, changed defaults) that alter behavior.

Include a small “drift risk” score per plan with explicit factors: time-to-execute, volatility of targets, and external dependency stability. Don’t pretend a score is perfect—explain what it assumes and where it lies.

When the drift is large, require an operator confirmation or a staged execution (dry run -> canary -> full rollout).

Dry Run Acceptance Checklist (reusable)

Use this checklist as an acceptance gate before converting a dry run into a live run:

• Resolution and Validation
  • All selectors expanded to concrete targets with authoritative source recorded
  • No ambiguous or missing targets
• Permission Checks
  • Required principals and scopes listed per operation
  • Any failing permission shows denying policy and remediation
• Planned Side Effects
  • Each operation lists intended side effects and idempotency
  • External effects (billing, network changes) marked and acknowledged
• Blocks and Constraints
  • All blocking conditions explained with ownership and remediation
  • Quota, locks, and immutable-field conflicts surfaced
• Drift Assessment
  • Drift risk estimated and assumptions documented
  • If high drift, a staged execution path is defined
• Execution Safety
  • Rollback or compensating action described for each critical step
  • Dry run output cryptographically or procedurally tied to execution request (prevents TOCTOU misuse)

Decision test: If every checked item is true and no blocker is unresolved, the plan is usable for automated execution. Otherwise require manual approval.

Where this recommendation is wrong

If your system operates with completely static resources and a single trusted operator, the overhead of deep dry-run resolution may be unnecessary. Similarly, some real-time systems cannot afford the time to resolve every dependency; there the dry run should focus on permission checks and high-risk preconditions only.

However, in multi-actor, multi-tenant, or highly regulated environments, a weak dry run is worse than none: it gives a false sense of safety while hiding failure modes.

Takeaway

A dry run should be a contract between intent and execution: it resolves targets, validates permissions and constraints, enumerates planned side effects, explains blockers, and measures drift. Treat dry-run output as the primary artifact reviewers use to decide whether to execute. If your dry run can’t answer “what will run, where, and why it might be blocked,” upgrade it before trusting automation to do the work. /contact