Guides

Monitor and observe

Wire OpenTelemetry, inspect audit/operational/debug trails, follow deployments, explain decisions, and alert on invariant alarms.

1. Configure OTel

observability: {
  otel: {
    endpoint: process.env.OTEL_ENDPOINT,
    headers: { "x-honeycomb-team": secret.ref("HONEYCOMB_KEY") }
  }
}

2. Tail operational state

vadyl operational tail --component runtime-fabric
vadyl audit tail --entity Order --since 1h
vadyl debug tail --handler orders.charge --run run_123

Runtime Fabric emits scale-target state, autoscale decisions, sampled measure values, cooldown and hysteresis gates, stale-sample denials, rollout or drain suspension, and substrate capability failures through the same operational and audit trails.

3. Explain decisions

vadyl explain access --entity Order --as user:usr_123
vadyl explain read-plan --entity Order --include customer
vadyl runtime-fabric autoscale explain --project billing --env production --surface EventConsumer

{
  "decision": "Allow",
  "reasonCode": "Access.AllowByRoleAndRowPredicate",
  "publicationVersion": 412
}

4. Alert on invariant alarms

AlarmMeaning
EntityReadCacheDecryptionSkipsProtected cache payload could not be decrypted.
PlatformEventIntentNonRecoverableFailureEvent outbox intent failed beyond recovery policy.
ReservedSubscriptionRejectionsSubscription filter or reserved channel rejected unexpectedly.
BridgeCapabilityDeniedAuthored runtime attempted a capability outside its grant.