0040. Workspace API keys for integrators and outbound webhooks

Status: proposed
Date: 2026-10-30, amended 2026-10-31 (integrations completed) and 2026-11-01 (event versions, fair delivery, the organization’s share; see the amendments below)
Deciders: maintainer (owner review pending)
Extends: ADR 0016, ADR 0017, ADR 0019, ADR 0036, ADR 0041

Observable acceptance criteria

API keys:

An administrator (any member holding api_keys.manage, unrestricted) opens Settings → API keys, creates a key with a name, a set of permissions, all or some properties and an optional expiry, and sees its secret once: gc_live_… in a production workspace, gc_test_… in a sandbox workspace. A reload lists the key with its prefix and last four characters, permissions, properties, creator, creation, expiry and last use, never the secret.
A key can hold only permissions its creator holds in that workspace when it is created, and only those an integration can use (team, audit, keys, webhooks and Channex administration are browser-only and never grantable). A larger request answers 403 auth.forbidden and creates nothing.
Authorization: Bearer gc_live_… authenticates every operation that lists the apiKey scheme, in the key’s workspace only, with the key’s permissions intersected with its creator’s live permissions and property scope, re-read on every request: a creator who loses a permission takes it from the key at once; a creator who leaves the workspace leaves the key answering 401. Another workspace (GuestCraft-Workspace-Id) answers 403; a property outside the key’s properties is refused like for a member limited to selected properties.
A key never reaches the browser-only operations (team, sign-in codes for devices, Channex administration, API keys, webhook endpoints): they answer 401, even next to a valid cookie.
Revocation takes effect on the next request (401); an expired key answers 401. Revoked keys stay listed as revoked.
Creation and revocation are audited (api_key.created, api_key.revoked); a refused use of a known key (revoked, expired, creator without access, another workspace, an operation it may not call) is audited as api_key.denied with the reason, at most once per key and minute per instance. No entry contains the secret or the key’s name.
More than 600 requests in a minute with one key answer 429 auth.rate_limited with Retry-After, on every operation the key reaches, GET /v1/workspaces included. An instance counts a key’s requests while it remembers the key, in one of 10,000 places, at most 100 of one workspace; a key that authenticates while no place can be freed, every place of its workspace or of the instance being held by a key used within the last minute, answers the same 429 until one can. A key that the instance does not remember as authenticated is a guess: before its lookup it takes a permit of its client address, a permit of the instance and one of its address’s 30 failed requests a minute. It holds the permits until its lookup ends, and keeps the failed request spent unless it authenticates and finds a place (so also when it is unknown, revoked or expired, or its creator has no access). An address has one guess looked up at a time and the instance half the connections of its auth pool, whatever the budgets refill meanwhile; any other guess gets the same 429 at once, before any lookup, however many arrive at once, and is not queued. A request of a key whose guess is being looked up, from any address, is no guess: it takes no permit and no failed request, waits for that lookup (at most 100 waiting requests per lookup, besides the one being looked up, beyond which the 429 answers) and gets its outcome, so the parallel first requests of one key on an instance all get the same answer, and only different keys meet the one-guess-per-address permit. A remembered key is never refused because others failed or are being looked up (a shared egress, a NAT, one IPv6 /64), and is still looked up on every request; a key not remembered yet is refused while its address’s failures are spent or the permits it needs are taken, never for failures of other addresses as such, but while a flood of guesses keeps the permits busy it gets in only when a retry finds one free. A request without a known client address takes an instance permit only. All of it lives in memory, per instance.

Outbound webhooks:

An administrator (webhooks.manage, unrestricted) opens Settings → Webhooks and registers an HTTPS endpoint with its event types; the signing secret (whsec_…) is shown once. They edit the URL, event types and description, disable and enable it, rotate the secret (shown once; the previous one keeps signing for 24 hours), delete it, send a test event and see the 50 most recent deliveries with their event, status, attempts, last response status and next attempt. A reload shows the same state. A change refused because the endpoint changed meanwhile (412) reloads it and says so; the next action carries the new version. A refused edit is replaced by the newer version’s address, description and events, which the operator changes again and saves under its ETag: an edit never overwrites a change it has not shown.
An endpoint has at most one test event waiting: another test makes that one due now instead of queuing a second. A disabled endpoint refuses a test with 409 webhook_endpoint.disabled.
Each committed reservation change of the supported types writes a webhook outbox row in the same transaction as the change: a rolled-back change leaves no event and no delivery.
The jobs worker delivers each event to every enabled endpoint subscribed to its type, with the Standard Webhooks headers webhook-id (the event id, stable across retries), webhook-timestamp and webhook-signature (v1,<base64 HMAC-SHA256>), a 10-second timeout, and retries after 1 minute, 5 minutes, 30 minutes, 2, 6, 12, 24 and 24 hours (nine attempts, about 69 hours). An endpoint that has failed without a success for as long as that whole schedule is disabled at that failure (one event that fails every attempt disables it), audited as webhook_endpoint.disabled by the system. A preview deployment, or a worker over a database restored for another instance, sends nothing, even for a delivery job queued before.
An attempt ends after 10 seconds whatever the receiver does (an answer it never finishes, informational answers, a protocol switch); a status outside 100–599 is a failed answer. One endpoint’s outcome never undoes another’s, and a delivery is never attempted more than nine times. Deliveries are deleted 30 days after their creation, also in workspaces with nothing to send.
Destinations that resolve to a private, loopback, link-local, unique-local, multicast, reserved or metadata address are refused at registration when the URL names one, and at connect time after DNS resolution, so DNS rebinding cannot reach them. For IPv6 only the global unicast space (2000::/3, without its special-purpose ranges) counts as public, and an embedded IPv4 address (IPv4-mapped or compatible, NAT64 64:ff9b::/96, 6to4) is checked as itself; the local-use NAT64 prefix, IPv4-translated addresses, SRv6 SIDs and the dummy prefix are refused. A proxy configured in the environment (NODE_USE_ENV_PROXY, HTTPS_PROXY) is never used. Redirects are not followed. Only GUESTCRAFT_WEBHOOKS_ALLOW_PRIVATE=true with GUESTCRAFT_ENV=development allows them, and http: URLs.
Payloads carry identifiers only: no guest name, contact or document.

SDK:

createGuestCraft({ baseUrl, apiKey }) sends the key as a bearer credential, never a cookie.
webhooks.verify(rawBody, headers, secret) in @guestcraft/sdk (MIT, Web Crypto, no AGPL import) returns the parsed event or throws WebhookVerificationError: unknown headers, a timestamp outside five minutes, or no matching signature (compared in constant time). It passes the Standard Webhooks specification’s test vectors.

Context

Chapter 8 of the analysis gives integrations workspace keys with selectable permissions (gc_live_/gc_test_); chapter 9 defines thin, versioned events delivered to webhook_endpoints with Standard Webhooks signatures, exponential retries for up to 72 hours and automatic disabling; chapter 10 adds apiKey to the SDK and webhooks.verify; chapter 18 asks for hashed keys with last use, expiry and immediate revocation, and one outbound HTTP client that blocks private addresses after DNS resolution. ADR 0017 fixed the key design (hashes, visible prefix, GuestCraft-owned scopes, Better Auth’s key routes blocked) and deferred it. ADR 0041 added the second bearer credential, device tokens, behind one early branch of principalOf, and made team, sign-in codes and Channex administration browser-only.

Decision

Permissions. Two catalog codes, workspace-scoped and denied to members limited to selected properties: api_keys.manage and webhooks.manage, both built-in for admin only (migration 0027 adds them to the admin template). Grantable to keys: every catalog code except team.manage, audit.view, api_keys.manage, webhooks.manage, channex.configure, channex.manage_account_mode and messages.configure, whose operations are all browser-only.

API keys (core/integrations, guestcraft.api_keys, migration 0027). A key belongs to one workspace; its row holds the SHA-256 hex digest of the secret, its prefix and last four characters, name, permission codes, optional property list, creator, creation, expiry, last use and revocation. A key is 32 random bytes of the injected random source in base64url behind the prefix chosen by the workspace’s environment. The secret is returned once by the creation; nothing can read it again. Revocation sets revoked_at and keeps the row, so a later use is recognized and audited. The item’s ETag is its version.

Creation checks the requested permissions against the creator’s effective grant in that workspace (ADR 0036): a code the creator does not hold is auth.forbidden; only unrestricted members manage keys, so any property of the workspace may be chosen. The key list returns the permissions the caller may grant (grantable_permissions, { code, group } entries of schema GrantablePermission), which the form offers. Keys are therefore also limited, on every request, to what their creator still holds: the principal of a key is its creator’s live workspace access, narrowed to the key’s permissions and properties. This departs from ADR 0017’s “a person leaving breaks nothing”: the owner asked that a key never exceed its creator’s live permissions, so a creator who leaves disables their keys. An administrator creates a replacement first. Actions taken with a key are audited with the api_key actor, naming the key and its creator, on whose delegation it acts (see the amendment), and the access log line carries the key’s id.

Credential resolution. auth.principalOf gains a second bearer branch beside device tokens: Authorization: Bearer gc_live_…|gc_test_… is resolved by that key alone, through guestcraft.api_key_credential(digest), a narrow security definer function executable by the app role that returns the key’s routing data; the unit of work’s tenant context cannot be set before the workspace is known. A request carrying either bearer credential has no browser session, so it never reaches a browser-only operation, and the cookie origin policy exempts it like a device request. Last use is written at most every five minutes. The resolver answers:

Situation	Answer	Audit
unknown secret	401 `auth.unauthenticated`, a failed request of the client address	none (nothing identifies a workspace)
revoked, expired, or creator without access to the workspace	401, a failed request of the client address	`api_key.denied` (`revoked`, `expired`, `creator_access`)
`GuestCraft-Workspace-Id` names another workspace	403 `auth.forbidden`	`api_key.denied` (`workspace`)
an operation answers 403 to the key	403	`api_key.denied` (`forbidden`, method and route)
the key over 600 requests in a minute; or a key not remembered on this instance while its client address has 30 failed key requests in the last minute or a guess of another key being looked up, its IPv6 /48 has 300, or the instance as many guesses being looked up as half its auth pool’s connections; or a request of a key whose guess 100 other requests already wait for	429 `auth.rate_limited`, `Retry-After: 1`	none
a request of a key whose guess is being looked up	the answer of that lookup: 200 within the key’s own budget once it authenticated, otherwise the same 401, 429 or 503; its 401 spends no failed request of its address	as for that lookup
a key that authenticates while every place of its workspace (100) or of the instance (10,000) holds a key used within the last minute	429 `auth.rate_limited`, `Retry-After: 1`, a failed request of the client address	none
the credential lookup does not finish: a lock on `api_keys` or `workspaces` not granted within 2 seconds, no auth-pool connection within 10 seconds, a connection lost or given up	503 `server.unavailable`, `Retry-After: 2`	none

Denied entries are written by the system actor, at most once per key and minute per instance, so a revoked key in a loop cannot flood the audit log. One function of auth, authenticate, resolves every credential and admits each key request through one budget module (auth/key-budgets.ts) before the digest lookup, so no operation resolves a key without spending a budget. A key that the instance remembers as authenticated spends its own budget, and is forgotten by the first refusal. Any other key is a guess: synchronously, before the lookup’s first await, it takes a permit of its client address, a permit of the instance and a failed request of its address and, for IPv6, of its /48, so concurrent guesses never share them. Each outcome settles them once: both permits come back when the lookup ends, whatever it found or if it failed; the failed requests are given back when the key authenticates and finds a place to be remembered, or when the lookup fails, and stay spent otherwise. Requests of one key at once share its guess’s lookup: keyBudgets(…).lookups<T>() gives each kind of credential (API keys find a principal, device tokens a device session) its map of the guesses being looked up by digest. A request whose digest is in it takes no permit and no failed request: it waits for that lookup, which settles the guess’s admission before any waiting request goes on, and gets its outcome. Once the key authenticated it is remembered, and each waiting request spends its own budget like any request of a remembered key; without a place, each gets the same 429; an unknown or refused key gives each the 401, and a lookup that fails gives each the same failure, 503 with Retry-After: 2 for a lookup that did not finish. Only the lookup holds the permits, and its settlement gives them back once. A waiting request’s 401 spends no failed request of its address: the failure budgets bound the lookups an address causes and the secrets it tests, and that request caused no lookup and presented the secret that the lookup’s address already paid for, so counting it would only make one client with a stale token spend its budget once per parallel request. A request waits as long as the lookup it joined, which the lookup’s own bounds limit (below), with no timer of its own; at most 100 wait for one lookup, and the next is refused at once. A client sends a few requests at once (the mobile app’s Today two, a browser at most six per host over HTTP/1.1), and 100 is a sixth of what the key’s budget then admits in a minute; since one lookup per key holds a permit, at most 100 waiting requests per permit on an instance (200 with the default pool size), holding memory and no connection. A remembered key is not shared: it is looked up for every request, so its revocation answers 401 from the next request. A waiting request gets the outcome of a lookup that started before it arrived, so a revocation committed while that one lookup runs applies from the next request, as it does for a request whose own lookup read the key just before. Because permits are held while their lookups run, no refill admits more: an address has at most one guess looked up and the instance at most half the connections of the pool the lookups use, at least one (the auth pool, half of GUESTCRAFT_DATABASE_POOL_SIZE: two guesses with the default size of ten). Every step of a lookup runs on that pool, one query at a time, so guesses never hold more than half its connections (with a pool of one connection, the guess being looked up holds it). The rest is free of guesses, not reserved: remembered keys and device tokens, sessions and sign-ins share it. The credential function bounds each lock it waits for to 2 seconds (its own lock_timeout, migration 0028, the bound of a unit of work run with boundedLockWait): a call that waits for two locks can take about twice that, and the bound does not limit its CPU or I/O time, for which it has no server-side statement deadline. The other steps of a lookup are transactions with a 10-second statement timeout. The auth pool gives up any query after 12 seconds on the client (query_timeout, with TCP keepalive at the system’s timing), whether the connection is half-open or the server is still working, and waiting for a pool connection can add 10 seconds. Giving up does not cancel a query on the server, so before the function’s bound a lock held for minutes (VACUUM FULL, a non-concurrent REINDEX, a manual ALTER) left one more server connection waiting every 12 seconds per auth connection. Device-token and sign-in-limit statements now run in explicit transactions with local lock_timeout = 2s, statement_timeout = 10s and an idle-in-transaction bound of 30s. Better Auth’s Kysely driver applies those same local bounds to each standalone statement and at the start of each explicit identity transaction. A server-answered statement failure inside an identity transaction rolls back before the connection is released; a checked-out Better Auth connection that reports a transport failure is marked broken, skips rollback, and is destroyed after at most one 12-second client timeout. Failures while the bounded BEGIN or COMMIT gets no server answer likewise destroy the client instead of returning it to the pool. Device-token and sign-in-limit transactions still send rollback after a lost answer, so a half-open connection can cost them up to 24 seconds before destruction. No session/startup settings or app-role defaults are changed, so the bounds work through transaction-mode PgBouncer without affecting other app/worker operations. This needs no migration, inventory/grant change, revision or expand-linter extension. These are per-lock/per-statement bounds, not a total HTTP request deadline; pooler/acquisition queues and a half-open connection still add time. Better Auth sometimes catches adapter failures; a request-local failure scope preserves the classified database failure across those catches, for its handler and cookie-session reads. Better Auth’s logger remains enabled; the driver reduces database failures to sanitized error objects before they reach it. A credential query that does not finish (a lock not granted within its 2 seconds, no pool connection within 10, a connection lost or given up) is a retryable failure of the credential store, which the gateway passes on after the permits come back: 503 server.unavailable with Retry-After: 2. A later step fails as the unit of work classifies it (ADR 0008): its statement timeout answers 500, as in every unit of work; a half-open connection given up after 12 seconds (up to 24 inside a transaction while rollback waits behind the abandoned query) also answers 500 and its checked-out client is destroyed. The client fallback does not cancel work that the server has already received. An IPv6 client’s failures count against its /64, the address Better Auth gives, and against its /48 too, at 300 a minute: ten addresses’ worth for an allocation that holds 65,536 /64s. Keeping the permits busy takes about two lookups per lookup time a second (40,000 a minute at 3 ms, 1,200 at 100 ms), so one /48 holds a few percent of them, a quarter on a database that slow, where before it held them all. Neighbours within one /48, such as an ISP’s customers, the networks of one site or a mobile carrier’s devices, share it as the clients behind one /64 share theirs: their unknown keys can make a neighbour’s key that is not remembered wait for the budget to refill, five lookups a second, while remembered keys are never refused for them.

The device tokens of the mobile app (ADR 0041) are admitted by the same budgets, in one admission the instance shares with API keys (keyBudgets, created once by createAuth): a token remembered as authenticated spends its own 600 requests a minute, at most 100 of one person’s tokens are remembered, and any other token is a guess that takes the same permits and failed requests as a key’s. Both lookups of a token go through it, the principal resolver’s and the device session operations’ (GET and DELETE /device_sessions, which can answer 429 since API revision 3.9.1). So a flood of unknown device tokens holds at most the guesses’ half of the auth pool, as unknown keys do, and a token that the instance does not remember yet gets in like a key: the app’s parallel first requests on an instance, such as the two reads of its Today screen after a restart, share one lookup and all answer. Since API revision 4.6.0 the exchanges of the app’s one-time codes are guesses in the same budgets too (ADR 0041, QR pairing amendment), never remembered. The instance has no per-minute failure budget: its permits bound the database work of guesses. A refused guess is answered 429 at once; it is never queued. While a flood of guesses from many addresses keeps the permits busy, a key that is not remembered gets in only when one of its retries finds a permit free, which is rare under a large flood: a security review measured about 5,000 guesses a second from 4,000 IPv6 /64s refusing a never-used key 15 times out of 15, while remembered keys and sessions answered every time. After a restart that applies to every key until it has authenticated once, and an integration platform that sends K customers’ keys from one egress address warms them up one per round, since its address has one guess looked up at a time (the requests of one key share its lookup; different keys do not): 40 keys take about 40 rounds, 40 seconds with Retry-After: 1. A request without a known client address (no trusted proxy header) takes an instance permit only.

The instance remembers at most 10,000 keys, and at most 100 of one workspace, in order of last use; a key unused for a day is forgotten, and a refused one at its first refusal. A new key of a workspace that holds its 100 takes the place of that workspace’s own least recently used key, never another workspace’s; otherwise, when all 10,000 places are taken, the place of the instance’s least recently used key. Either gives its place only once it has been unused for a minute, when its 600-a-minute budget is full again, so forgetting it never resets a budget. Without such a place the key is refused like a spent one (429, Retry-After: 1), since nothing could count its requests, and its lookup keeps its address’s failed request spent, so that the lookups of a key that keeps finding no place stay within its address’s 30 a minute; it is a guess again on its next request. An integration uses about one key per connected system, and a workspace has at most 20 webhook endpoints: 100 keys used within one minute is far more than a workspace needs, and 100 such workspaces still fit. Before the share, the administrator of one workspace, who may create keys without limit, could hold every place with 10,050 keys each used once a minute, about 200 requests a second, so that the new or idle keys of every other workspace stayed guesses on that instance. The share bounds what one workspace holds; a cap on keys per workspace would add nothing to it, since a cap alone only multiplies the workspaces needed. What remains is the number of workspaces: 100 full workspaces take every place. Only organization administrators create workspaces (auth/workspaces.ts); the amendment of 1 November 2026 adds a share per organization. The failures of at most 10,000 addresses are counted; the least recently used one makes room for a new one unless it has a guess being looked up, in which case the new address goes untracked, bounded by the permits alone. The budgets and the throttle live in memory, per instance, and restart empty: a key’s first request on an instance, and its first after a day unused or after its place went to another key, is a guess, and the key’s other requests that arrive while it is looked up wait for its outcome.

Outbound webhooks (core/integrations, migration 0027). webhook_endpoints holds the URL, an optional description, the subscribed event types, the enabled flag and the reason it was disabled (operator or failing), the start of the current failure streak, a version (the ETag), and the signing secret sealed with the instance’s ENCRYPTION_KEYS (AES-256-GCM, bound to the organization, workspace and endpoint, under a derivation label of its own in server/http/integration-secrets.ts), with the previous secret for 24 hours after a rotation, which the next delivery run after those 24 hours forgets. At most 20 endpoints per workspace. The secret is whsec_ and 32 random bytes in base64, as Standard Webhooks defines it.

Version 1 delivers these types, from the domain events the reservation use cases already write: reservation.created@1, reservation.updated@1, reservation.cancelled@1, reservation.rooms_cancelled@1, reservation.checked_in@1 and reservation.checked_out@1; the test event is webhook.test@1. The event log adapter of db inserts a webhook_outbox row for each of these events in the same statement as the event, so in its transaction and without another round trip. The amendment of 2026-10-31 extends the catalog to Channex imports, folios, rates and room housekeeping status.

The payload is the thin event of chapter 9, with the property so that a receiver can reread the reservation, whose routes are nested under its property:

{
  "id": "evt_01k8…",
  "type": "reservation.checked_in@1",
  "created_at": "2026-10-30T09:12:00.000Z",
  "workspace_id": "ws_01k8…",
  "data": { "object": "reservation", "id": "rsv_01k8…", "property_id": "prop_01k8…", "room_id": "rrm_01k8…" }
}

room_id appears when the change names one room. No guest data, amounts, notes or reasons are sent: receivers reread the reservation with their own key, as chapter 9 decides. The serialized body is stored with the delivery, so every retry sends the same bytes.

Delivery (jobs, server/worker). A system job, webhooks.dispatch, runs every minute: a narrow worker-only function, guestcraft.webhook_workspaces(at, purge_before), names up to 1,000 workspaces that have outbox rows, due deliveries to an enabled endpoint, deliveries older than 30 days, or a previous signing secret whose 24 hours ended, the work that has waited longest first (upkeep waits behind every due send), and the job enqueues webhooks.deliver for each. That job, in the workspace’s unit of work, takes up to 200 outbox rows by deleting them (a concurrent job waits, then finds them gone; a rollback keeps them) and adds one delivery per subscribed enabled endpoint, forgets the previous secrets whose window ended, deletes up to 1,000 expired deliveries, each step in a transaction of its own, then claims up to ten due deliveries at a time with a 60-second lease (for update skip locked), sends them together outside any transaction, and records the outcomes. It takes no new batch after a minute, or once its job’s abort signal fires, so a run ends within about 70 seconds; the next dispatch continues. Delivery is at least once: a crash after a send repeats it with the same webhook-id. A claim skips deliveries that had nine attempts, and fails as timeout one whose ninth lease ended without an outcome.

Delivery is owned by one instance: neither the dispatcher nor the delivery job (asked before its first write and again before each batch, since a queued job outlives the dispatch that made it) calls a receiver, moves, records or deletes anything from a preview deployment (outboundWrites) or over a database whose instance record is another instance’s. Unlike Channex writes, delivery deliberately does not also require a production environment: development instances deliver to test receivers. A copy of a production database used for development must therefore run under its own GUESTCRAFT_INSTANCE_ID; with the production instance’s identifier and ENCRYPTION_KEYS, its worker would sign and deliver the copied events like production.

The API role may change only a delivery’s next_attempt_at (a test made due again: a column grant); attempts, leases, states and outcomes are written by the worker role alone.

Outcomes are recorded per endpoint, endpoints in identifier order, each in a transaction of its own: the endpoint’s row is locked first, then its deliveries in identifier order, each fenced by its attempt number, then its failure streak is updated once. That is the lock order of the whole module, the one administration takes when it locks an endpoint and deletes it with its deliveries; recording deliveries first deadlocked against a concurrent deletion. One endpoint’s failed record never undoes another endpoint’s; the job fails after recording the others. Registration counts the workspace’s endpoints under a transaction lock per workspace, so concurrent registrations cannot pass the limit of 20.

Fairness is partial: each worker process runs at most four delivery jobs at once for every workspace of the instance, a job holds its slot for up to about 70 seconds against slow receivers, and the dispatcher enqueues one job per named workspace each minute even while an earlier one waits, so workspaces with many failing endpoints delay the others’ deliveries by minutes; the amendment of 1 November 2026 adds a per-workspace singleton job and per-endpoint backoff.

webhook-timestamp is the attempt’s time; the signature covers {webhook-id}.{webhook-timestamp}.{body}. Signing (Node’s HMAC) lives in server; verification (Web Crypto) lives in the SDK; both are tested against the specification’s vectors, never against each other only.

SSRF. The only outbound webhook client is server’s guarded transport: node:https through an agent of its own, never the global agent that NODE_USE_ENV_PROXY would point at a proxy, with a lookup that resolves every address and refuses the request if any is not public, checked when the socket connects; literal IP hosts are checked before. Public means: for IPv4, outside Node’s BlockList of IANA’s special-purpose ranges; for IPv6, inside 2000::/3 and outside its special-purpose ranges, or an embedded IPv4 address (IPv4-mapped or compatible, NAT64 64:ff9b::/96, 6to4) that is public itself. No redirect is followed, the response body is read up to 64 KiB and discarded, and a deadline destroys the request after 10 seconds whatever arrived; a protocol switch (101) ends the attempt at once. No static rule can recognise a deployment’s own translation: a NAT64 network-specific prefix (RFC 6052, inside 2000::/3) or ISATAP interface identifiers (…:5efe:a.b.c.d) look public. Recognising them is the operator’s job: a host behind such a translator or tunnel filters private IPv4 destinations there, or at its egress firewall. Registration refuses URLs with credentials, fragments, non-HTTPS schemes and literal or localhost private hosts, with a field-linked webhook_endpoint.invalid.

HTTP contract (API revision 3.9.0 as decided; 3.9.1 later lets GET and DELETE /device_sessions answer 429). Every operation of the principal resolver lists the apiKey bearer scheme next to deviceSession. The new operations are browser-only:

Operation	Permission	Result
`GET /v1/api_keys` (`apiKeys.list`)	`api_keys.manage`	keys of the workspace, 100 per page, `next_cursor`, `grantable_permissions` (each a `GrantablePermission`: code and catalog group)
`POST /v1/api_keys` (`apiKeys.create`)	`api_keys.manage`	201 `{ token, api_key }`, `Location`, `ETag`
`GET /v1/api_keys/{id}` (`apiKeys.get`)	`api_keys.manage`	the key, `ETag`
`POST /v1/api_keys/{id}/revoke` (`apiKeys.revoke`)	`api_keys.manage`, `If-Match`	the revoked key
`GET /v1/webhook_endpoints` (`webhookEndpoints.list`)	`webhooks.manage`	at most 20 endpoints
`POST /v1/webhook_endpoints` (`webhookEndpoints.create`)	`webhooks.manage`	201 `{ secret, webhook_endpoint }`
`GET /v1/webhook_endpoints/{id}` (`webhookEndpoints.get`)	`webhooks.manage`	the endpoint, `ETag`
`PATCH /v1/webhook_endpoints/{id}` (`webhookEndpoints.update`)	`webhooks.manage`, `If-Match`	the endpoint
`DELETE /v1/webhook_endpoints/{id}` (`webhookEndpoints.delete`)	`webhooks.manage`, `If-Match`	204
`POST /v1/webhook_endpoints/{id}/rotate_secret` (`webhookEndpoints.rotateSecret`)	`webhooks.manage`, `If-Match`	`{ secret, webhook_endpoint }`
`POST /v1/webhook_endpoints/{id}/test` (`webhookEndpoints.test`)	`webhooks.manage`	202, the queued delivery, or the waiting test made due now; 409 `webhook_endpoint.disabled`
`GET /v1/webhook_endpoints/{id}/deliveries` (`webhookDeliveries.list`)	`webhooks.manage`	the 50 most recent deliveries

Revocation is an action, not DELETE: the key stays listed. A person’s own device sessions keep the session and device schemes only (PERSON_SECURITY): an API key is not a person’s session. New problems: api_key.not_found (404), api_key.invalid (422, with field), api_key.version_mismatch (412), webhook_endpoint.not_found (404), webhook_endpoint.invalid (422, with field and reason), webhook_endpoint.version_mismatch (412), webhook_endpoint.limit_reached (409), webhook_endpoint.disabled (409) and auth.rate_limited (429), which GET /v1/workspaces answers too.

SDK and screens. createGuestCraft({ baseUrl, apiKey }) sends the key and no cookie; webhooks.verify is independent of the server’s signer. Settings → API keys and Settings → Webhooks use the existing components and the integrations catalog namespace, and appear in the navigation for members holding the permission. The key screen reads only the keys and their grantable permissions; it reads the properties only for a key limited to some of them, so a role with api_keys.manage but without properties.read or team.manage still manages keys (for every property). The role editor and the key form share one permission chooser.

Consequences

An integration acts with a revocable, narrow credential and learns about reservation changes without polling; a person’s departure ends their keys, which the Settings list cannot show yet (it shows the creator).
ENCRYPTION_KEYS now also seals webhook secrets. The maintenance re-encryption (credentials:reencrypt) rewrites the current and previous secret of every endpoint and verifies them with the target key alone, like the Channex values; it reports them as endpointSecrets.
The in-memory budgets are per instance: with several instances (Vercel), a key gets up to 600 requests per minute from each, a client address 30 failed key requests per minute on each, each instance has its own guess permits, and each remembers keys on its own, 100 per workspace.
One extra row per supported reservation change, removed by the next delivery job, even in a workspace without endpoints (the writer may be a member limited to properties, who cannot read endpoints).
Most workspace pages grew by 643 bytes of gzip (490 of Brotli) with this decision, measured on paired production builds against main; Channels, Reports and Roles by 562, 635 and 749 bytes of gzip (Roles also ships the shared permission chooser). About 300 bytes of that are the sidebar’s two new entries with their icons. Since the amendment of ADR 0022 of 2026-10-30, the generated SDK has one module per OpenAPI tag: the four API key operations, the eight webhook operations and their validators ship only with the Settings → API keys and Settings → Webhooks routes. pnpm web:budgets fails when another route’s initial JavaScript defines an operation of either tag, or when those routes define none (tooling/quality/web-route-operations.json); it reads each tag’s operations from openapi/v1.json, so a new operation of the tag needs no new rule. The web app sends operations from their tag modules (@guestcraft/sdk/operations/api-keys, …) through createGuestCraftTransport; a lint rule holds the imports.

Limits and planned work

Organization tokens (gc_org_), client session tokens, the MCP server and OAuth applications arrived with ADR 0047, which also replaces the principal’s apiKeyId with the credential of any of them. Endpoint-chosen event versions, a per-workspace singleton delivery job with per-endpoint backoff and a share of the remembered keys per organization arrived with the amendment of 1 November 2026 below. Budgets shared by every instance are planned; events older than 30 days are not kept. A disabled endpoint keeps its pending deliveries; enabling it again resumes them. A second rotation of a signing secret within 24 hours replaces the previous secret at once: only one previous secret is kept.

Evidence

On 30 October 2026: core unit tests of grants, ceilings, keys’ access, endpoints, the event body, the retry schedule, the time budget of a run and automatic disabling with an in-memory port and a manual clock (packages/core/src/integrations); PostgreSQL through PgBouncer for digests, isolation between workspaces and property contexts, the credential function’s role, last use, the outbox committed or rolled back with its event, and the delivery use case, leases and fences on the real repositories (packages/db/test/integration/integrations.integration.test.ts, with the RLS matrix and catalog covering the four tables); the credential in auth (api-keys.integration.test.ts: never a browser session, creator access re-read, refusals audited); the composed runtime (packages/server/test/integration/integrations.integration.test.ts: shown once, digest only, sandbox gc_test_ keys, another workspace refused and audited, property scope, the creator’s ceiling and live narrowing, browser-only operations refused next to the creator’s cookie, revocation and expiry, a reservation’s outbox row and its signed delivery to a local receiver verified as the specification defines, a test event signed with both secrets after a rotation, a private address refused in production); mobile-sessions.integration.test.ts refuses a device token on the twelve new operations; the signer and the SDK verifier each match the Standard Webhooks test vector; destinations are refused for every IPv4 and IPv6 form and at connect time with a resolver that answers a private address. The Playwright scenario integrations.spec.ts runs the whole workflow on the production build with axe checks and the keyboard.

After the security and code-quality reviews of PR #169, each fix has a regression that failed before it: a delivery job queued before and drained by a preview worker, or by one over another instance’s database, sends nothing and leaves the delivery’s attempts at zero; recording an outcome while an administrator deletes its endpoint no longer deadlocks (40P01 before); an outcome the database refuses for one endpoint keeps another endpoint’s success; a delivery out of attempts is never sent again; one pending test per endpoint (also a unique partial index); retention reaches a workspace with nothing to send; the longest-waiting workspace comes first; four concurrent registrations at 19 endpoints admit one; a receiver answering 101 or a status of 999 or 099 ends its attempt on time and is recorded; a proxy from the environment cannot resolve the host; the local-use NAT64, IPv4-translated, SRv6 and dummy IPv6 ranges are refused; maintenance re-encrypts endpoint secrets; a spent key is refused on GET /v1/workspaces, and 31 unknown keys from one client are refused before any lookup; the key screen works for a role without properties.read or team.manage; a 412 reloads the endpoint and the next change carries its new ETag; endpoints can be edited.

After the re-check reviews, again each with a regression that failed before: 80 concurrent requests with an unknown, revoked, expired or creator-refused key from one address were looked up 30 times, then once after a two-second refill (80 times before, in the unit and on PostgreSQL through PgBouncer); a key that authenticated is answered from an address whose failures are spent (refused before); requests without a known address no longer share one budget; after a 412 the editor shows the newer version’s address, description and events and saves changes to them (the refused draft was sent under the new ETag before); the API role cannot change a delivery’s attempts, state, body or lease (it could before); a previous secret is forgotten once its window ended (kept until the next rotation before).

After the third re-checks, each again with a regression that failed before: while lookups were held, refills admitted more guesses (61 lookups for one address over a minute; now one at a time per address, and three for a pool of six connections, until one ends); a key never used on the instance was refused once twenty addresses had spent 30 failures each, in the unit and on PostgreSQL through PgBouncer (now answered; the per-minute instance budget is gone); a guess’s permits come back exactly once whatever its outcome; a revoked key unused for a day was still remembered and looked up (now a guess, refused before its lookup from a spent address); 10,001 keys were all remembered (now 10,000, and a key used in the last minute keeps its place and its spent budget); an address with a guess being looked up lost its spent budget to 10,000 others; forgetting previous secrets deadlocked (40P01) with a fan-out holding its endpoints in another order (now FOR NO KEY UPDATE, while a rotation committed meanwhile keeps its secret); and every workspace page carried the API key and webhook operations, which put the desk routes over their JS budget.

After the fourth re-checks, again each with a regression that failed before: while every place held a key used within the minute, a new key that authenticated succeeded without any budget (2,000 of 2,000 requests at frozen time, also through the gateway; now 429 until a place frees, then 600 a minute); one workspace with 10,050 keys used within the minute kept another workspace’s new key a guess, whose two requests at once answered 200 and 429 (now it is remembered, and the workspace keeps 100 places); a lookup waiting behind a lock on api_keys held its permits for as long as the lock, on PostgreSQL through PgBouncer (now it ends after 12 seconds and the next guess is looked up); and the bundle text above claimed that the settings operations shipped only with the settings pages.

After the fifth re-check, as follow-ups, each with a regression that failed before on PostgreSQL through PgBouncer: a credential lookup abandoned by the client’s 12-second bound kept waiting behind the lock on the server (16 seconds into a lock, 3 server connections of the app role behind a pooler that lends it one; now none waits more than 2 seconds and the pooler keeps its one); and such a lookup answered 500 server.internal after 12 seconds (now 503 server.unavailable with Retry-After: 2, after 2). Then, again each with a regression that failed before: ten /64s of one IPv6 /48 spent 300 failed requests and an eleventh /64 of it was still looked up (now 429 before its lookup, in the unit, the gateway and through the HTTP runtime); unknown device tokens were looked up without any budget, so the 31st of one address was still looked up, and eight of them waiting for a lock took the whole auth pool, so that a browser session got no answer within 8 seconds (now two wait, six answer 429 at once, and the browser answers 200). Then, again each with a regression that failed before: the parallel first requests of one key or device token on an instance answered one 200 and 429 for the rest (40 through the gateway, five device-token requests through createAuth on PostgreSQL, six through the HTTP runtime, and the two reads of the mobile app’s Today after a restart); now they share one lookup and all answer 200, an unknown key’s all answer 401 and spend one failed request, and behind a lock on api_keys all answer 503 with Retry-After: 2 while the permits come back once (three guesses then: two looked up, one 429).

Amendment of 31 October 2026: integrations completed

Observable acceptance criteria (module wave 1, track INT):

Event catalog. Webhooks deliver, besides the reservation changes, the same-day undo of a check-in or a check-out (reservation.check_in_undone@1, reservation.check_out_undone@1, which reopens a closed folio), reservation.overbooked@1, Channex imports (channex.revision_applied@1, channex.revision_quarantined@1), folios (folio.extra_posted@1, folio.payment_posted@1, folio.refund_posted@1, folio.reversal_posted@1, folio.room_posted@1, folio.policy_posted@1, folio.rerated@1, folio.closed@1, folio.reopened@1), rates (rate_plan.created@1, rate_plan.updated@1, rates.changed@1, rates.restored@1), room housekeeping status (unit.housekeeping_changed@1) and, since ADR 0046 was amended, housekeeping tasks (housekeeping_task.created@1, .assigned@1, .status_changed@1). Bodies keep identifiers only; data.object names the kind of data.id, with property_id and, for folio events and applied revisions, reservation_id.
Events API. GET /v1/events (100 per page, newest first, type, from and until filters, an opaque cursor bound to them) and GET /v1/events/{id} answer the catalog’s events of the last 30 days (analysis chapter 9) to members and API keys holding events.read; Settings → Events shows them with a detail panel.
Replay and redelivery. An administrator redelivers one delivery, or replays the endpoint’s events (all its types or some) of a range within the 30 days, at most 1,000 events per replay; both are audited, refused for a disabled endpoint, and never queue an event the endpoint has pending.
API keys. An action taken with a key is audited as the key acting for its creator. A key is rotated in one operation: the replacement, with the same grant, allow-list and expiry, is shown once and the old key works for a grace period of 0–168 hours (24 by default). A key may be limited to up to 20 client address ranges; a request from elsewhere answers 401 and is audited. The key list says whether each key’s creator still holds its whole grant, part of it, or nothing.
Ownership. An organization owner transfers the ownership to another organization administrator from Settings → Users, confirmed by typing the organization’s name, audited as organization.ownership_transferred (ADR 0030, amended).

One catalog, one place. EVENT_CATALOG in core/integrations/event-catalog.ts lists each public type with the object it names; a public type is the domain event’s own versioned type, so delivering another existing domain event is one entry there (other tracks add none: the catalog belongs to this module). The event log keeps writing a webhook_outbox row for each catalog type in the event’s own statement.

Events are immutable. The same statement writes domain_events.public_subject, the snapshot of the property and reservation the subject names at that moment, found by the locator of its kind of object (repositories/public-events.ts; a new kind of object needs one locator there). The events API, the fan-out and a replay all serialize that snapshot, so one event always has one body: a later change, such as a quarantined Channex revision applied afterwards, never alters an event already written, and the event’s identifier is a true strong validator. The modules that write events are unchanged. Events recorded before migration 0034 have no snapshot: their identifiers are read when they are shown, at most for the 30 days they stay readable. An event whose subject is not of its type’s kind (migration 0009 recorded folio.room_posted about movements) is not public.

Folio closed. Split folios (ADR 0029) give every folio a closed state. folio.closed is the domain event of each folio’s closing transition, with that folio as subject, in the transaction of the change: a member’s close, or the last room’s check-out (settled or with an approved open balance), which closes every folio still open, one event each. The same-day undo of that check-out (reservation.check_out_undone@1) reopens the folios it closed, and a member may reopen a closed folio; each reopening publishes one folio.reopened@1 with the folio as subject. The next final check-out closes them again. (Train wave 1b replaced the earlier check-out heuristic, which assumed one folio per reservation, with these transitions.)

Events API decisions. Events span properties, so events.read is a workspace permission denied to members and keys limited to some properties (ADR 0036); it is built-in for administrators and grantable to keys. domain_events keeps its rows; the API reads only the last 30 days, ordered by event identifier (creation order), using the existing (workspace_id, occurred_at) index for the range. Events of one instant have no order among them. A page’s events are filtered by subject kind in SQL, before its limit, so that a page is full whenever older events remain.

Replays are new deliveries. A redelivery or replay inserts a delivery with origin replay or redelivery and the event’s identifier (the webhook-id), so the attempt fences of existing deliveries never change. Uniqueness moves from a constraint to two partial unique indexes: one fan-out delivery per endpoint and event (origin = 'event'), and at most one pending delivery per endpoint and event of any origin, which also makes concurrent replays queue an event once. A redelivery copies the stored body; a replay serializes the event’s snapshot again, the same body. Replayed deliveries are claimed in no particular order.

API key actor. audit_log.actor_type gains api_key: actor_id stays the creator (filters by person keep finding their delegated actions) and the new api_key_id names the key, with a check that ties them. Core’s Principal carries apiKeyId, and actorOf(principal) builds the actor of every audited write of a request, Messages’ and Channex connections’ included. Every persisted write an API key can make is audited: the one write members leave unaudited, a Messages thread’s read marker (the team’s working state), is audited when a key moves it, as channex.message_thread_read with the thread’s identifier only. Members’ and devices’ read markers stay unaudited.

Rotation. POST /v1/api_keys/{id}/rotate (If-Match) inserts the replacement and sets the old key’s replaced_by and expires_at in one transaction; without grace period the old key expires at once, which the expiry check of migration 0034 allows for a replaced key even at the instant it was created, while a new key must still expire in the future. The rotating member becomes the creator, so the replacement may hold only permissions they hold (the delegation ceiling of ADR 0036); a revoked, expired or already replaced key answers 409 api_key.not_rotatable.

IP allow-lists and admission. api_keys.allowed_ips cidr[] (1 to 20 ranges, canonical CIDR, no host bits), returned by the credential function, which migration 0034 creates again with its lock bound. A key’s lookup, which the requests arriving with it share, finds the key’s facts: what it authenticates, its owner, its ranges and its expiry. Each request is then admitted on its own: its own client address against the ranges, the instant it arrived against the expiry, and its own charge of the key’s budget. A refused request never changes what the key’s other requests get: it answers 401, is audited (ip_address or expired), records no last use, and is charged one failure of its own address only. The instance remembers a key’s facts with the key, so a remembered key refuses an outside address before any lookup and before its budget is spent; once that address’s failures are spent, it gets 429. A lookup started by an outside address still remembers the key for the allowed requests waiting for it. Authorization compares all 128 bits of an IPv6 client address: Better Auth reduces an address to its /64 for its own rate limits, so the gateway reads the whole address, and only the budgets group IPv6 addresses by /64 (and /48).

Creator access. The key list and reads compute creator_access from the creators’ live grants, read through auth after the unit of work ends (never while it holds a connection), with the same narrowing the gateway applies.

Ownership is the organization’s. Workspaces have no owner: they belong to their organization (ADR 0017). Transferring “the workspace or organization ownership” is therefore the transfer of the organization’s owner role; a separate workspace owner would be a new concept needing the owner’s decision.

Contract (API revision 5.3.0): apiKeys.rotate, events.list, events.get, webhookEndpoints.replay, webhookDeliveries.redeliver, team.transferOwnership; new fields allowed_ips, replaced_by, creator_access of ApiKey, origin of WebhookDelivery, actor_type and api_key_id of audit entries; problems api_key.not_rotatable (409), webhook_replay.invalid (422, field, reason), webhook_delivery.not_found (404), webhook_delivery.pending (409), event.invalid (422, field), event.not_found (404). Migration 0034 (contract: it replaces constraints and the credential function).

Evidence (31 October 2026): core unit tests of the catalog, the IP range parser, rotation, allow-lists, creator access, replay, redelivery, the events use case, the api_key actor and folio.closed; the gateway’s allow-list per request, also for requests sharing a lookup; route tests of every new operation; PostgreSQL through PgBouncer for subject resolution of every kind of object, retention, cursor and isolation of events, concurrent replays, redelivery and the pending index, the audit actor’s check, allow-lists through the credential function and rotation (packages/db/test/integration/integrations-events.integration.test.ts); the composed runtime for events of reservations, folio payments and housekeeping listed to a key and delivered, replayed and redelivered, the key’s writes audited as the key, rotation with its grace period, allow-lists and a creator without access (packages/server/test/integration/integrations-catalog.integration.test.ts); device tokens refused on every new browser-only operation; the browser scenario integrations.spec.ts.

After the independent review of PR #285, each fix has a regression that failed before it: an outside address holding a key’s secret no longer leaves the allowed requests sharing its lookup answering 429 (20 of 20 refused before), and is refused before any lookup once the key is remembered; a request that arrives at a key’s expiry while an earlier lookup of the key runs is refused (authenticated before); a /128 allow-list no longer admits another address of its /64, and a /128 or /65 range admits its own address, through the HTTP runtime (all three wrong before, as Better Auth gave the gateway a /64); an event read before and after its quarantined revision is applied, and its replayed body, keep one representation (changed before); a rotation without grace period at the key’s creation instant answers 201 (500 before); a read marker moved by an API key is audited with the key and its creator (unaudited before); Events no longer appends a page of the previous filter that arrives after a new filter’s first page; and the pending Replay button’s announcement is text, so axe passes at once after the replay dialog closes (a dangling aria-labelledby before).

Alternatives considered

A dispatcher reading domain_events by cursor. Rejected: identifiers and recorded_at do not follow commit order, so a cursor skips a transaction that commits late; outbox rows exist only once committed and are deleted when moved.
A trigger that writes deliveries. Rejected: it runs with the writer’s row-level security, which hides endpoints from members limited to properties, and it hides the catalog in SQL.
Keys in guestcraft_auth like device sessions. Rejected: a key is a workspace resource managed by workspace administrators; row-level security keeps one workspace’s keys from another’s.
Better Auth’s API key plugin. Rejected in ADR 0017 for the same reasons.
Keys that survive their creator. The owner’s rule of 2026-10-30 is that a key never exceeds its creator’s live permissions.
fetch with a pinned dispatcher. Node’s built-in fetch cannot take a connect-time lookup without the undici package; node:https can, with no dependency.

Observable acceptance criteria:

Event versions. GET /v1/event_types lists every version of every event (family@version) with the object it names and its state: current, or deprecated with the date until which it stays supported and its successor. An endpoint’s event_types keeps its earlier shape and meaning: the families it receives, each named by its first version (today every type is a first version). It receives each family in one version, pinned when it subscribed to the family (the family’s latest version then) and kept through edits that keep the family; the new event_versions member reports it per family and, on registration or a change, chooses another (Settings → Webhooks offers the choice only when a family has several versions). A later version named in event_types, a version for a family the endpoint does not receive, a version the catalog lacks, or a new subscription to a deprecated version are refused with a field problem, while an endpoint that already receives a deprecated version keeps it and is shown its date and successor. One committed event goes to each endpoint in the version it chose, with the same webhook-id; a delivery keeps the version and body it was queued with, through retries, redeliveries and later edits of the endpoint; a replay sends the endpoint’s chosen version of each event. The events API keeps returning each event as it was recorded.
Retirement. Nothing is retired automatically, no saved choice is silently upgraded or dropped, and no queued delivery is dropped. A deprecated version stays supported for at least six months after its successor ships; retiring it afterwards is a separate, announced, manual decision.
Fair delivery. A workspace has at most one delivery job waiting and one running. An endpoint whose attempts in a batch all failed backs off, on every worker, for as long as it has been failing (30 seconds to an hour), until a success, a test or re-enabling; each batch of ten takes every due endpoint’s oldest delivery before any endpoint’s second, and at most five of one endpoint. A healthy endpoint’s event is delivered in the next run beside an endpoint with a large failing backlog.
The organization’s share. An organization holds at most 1,000 of an instance’s 10,000 remembered keys, across its workspaces’ API keys and client sessions, its tokens and its OAuth applications.

One catalog of versions. EVENT_CATALOG keeps today’s entries, { type, object }: a domain event’s own type, delivered as its thin event, as before; the catalog stays the integrations module’s, and other modules add entries in that shape. In a family of several versions each entry is { type, object, renders, deprecated? }: it renders events stored as its own type thin, and renders says, for every other version of the family (family@n), how it renders the events stored as that one: thin, or with a data mapped from the event’s recorded subject (identifiers only), so a version can say nothing the event did not record. Every version thus receives the family’s events whichever version the producer writes: when a producer moves from @1 to @2, the @1 events still in the outbox or replayed within 30 days reach @2 endpoints, and @1 endpoints receive the events written as @2, with no catalog change at the switch. The catalog is checked when it loads (syntax, uniqueness, each version renders exactly every other version of its family, with the same object, a deprecation needs a date and a current successor), and a test pins every version ever delivered, so none is removed by mistake. Since renderings stay in their family and an endpoint holds one version per family, one event still gives one delivery per endpoint, and the fan-out and pending unique fences of migration 0034 stay valid. Fan-out and replay use one function to pick the endpoint’s version; a replay reads the endpoint’s choice once, under its lock. Endpoint reads keep a saved version even if the catalog lacked it.

An additive contract. /v1 only gains members (ADR 0016), so the version choice is a member of its own: event_types of endpoints and replays stays the closed WebhookEventType enum, with the same values and meaning as before, and names families; event_versions ({"reservation.created": 1}) is new, always present on an endpoint with one entry per family, and optional on registration and changes, where it applies after event_types. A client that sends and reads only event_types sees the same bodies, plus members it ignores, and the same deliveries, since every family has one version. The endpoint still stores one family@version per family (no migration); the core resolves families to versions inside the endpoint’s locked transaction, and a replay names families and sends each in the endpoint’s version. A family is named by its first version so that its name never changes when a later version ships; the latest version is the default for a new subscription because a deprecated version always has a later current one.

Delivery. webhooks.deliver is a pg-boss stately queue keyed by workspace (migration 0067, scheduler_queue_policy_webhook_delivery, the queue-policy form of ADR 0019): one created and one active job per workspace, so the minute’s dispatcher no longer stacks jobs behind a busy workspace, and a job enqueued during a run runs after it, so no wake-up is lost. enqueue returns nothing for a workspace that already has one waiting, which is expected, not an error. The endpoint’s backoff is a column (webhook_endpoints.backoff_until, migration 0066) that the claim and webhook_workspaces honour; it is separate from each delivery’s retry schedule and from the nine-attempt fence, which are unchanged. The claim takes, per batch, at most five deliveries of one endpoint, in turns, and re-states its conditions in the locking select so that a row another claim took meanwhile is checked again. A run fans out up to five passes of 200 outbox rows, so a new event does not wait a whole run behind an outbox backlog.

The organization’s share. A remembered key takes a place in three shares: its owner’s (100: a workspace’s API keys, its client sessions, an organization’s tokens, a person’s device tokens), its organization’s (1,000) and the instance’s (10,000); device tokens have no organization. A new key takes the place of the least recently used key of the narrowest full share, if that key has been unused for a minute; otherwise it is not remembered, and nothing is forgotten or refilled. The administrators of an organization create workspaces at will, so the workspace share alone let one organization multiply its places; a thousand is ten workspaces’ full shares in use within a minute, more than a hotel group’s integrations need, and ten such organizations still fit an instance.

Evidence (1 November 2026): core unit tests of the catalog (versions of one event, events stored as either version reaching endpoints of both, eleven invalid catalogs, families named by their first version), of subscriptions (pinned at subscription and kept after a later version ships, event_versions on registration and changes, each refusal, replays in the pinned versions), fan-out per version, frozen versions and redelivery; API tests that 6.6.0 request bodies reach the use case unchanged and that endpoint answers are 6.6.0’s, byte for byte, plus the new members, and that event_types is the WebhookEventType enum in the document; the server test of the same bodies over PostgreSQL; the PostgreSQL test webhook-versions.integration.test.ts (mixed versions, versions pinned on registration and kept through edits, persisted backoff across two dispatchers, and a 300-delivery failing endpoint beside a healthy one: the healthy event is sent once in the next run, the failing endpoint at most ten times, no delivery twice; removing the backoff condition fails two of its tests); the jobs test singleton.integration.test.ts (eight enqueues from two dispatchers make two jobs, two workers never run one workspace twice, a job enqueued during a run runs after it); key-budget unit tests (15 workspaces of one organization fill exactly 1,000 places while another organization’s key and a device token are still remembered; a refused key forgets no hot key and refills no spent budget).