Running it¶

One process serves everything. Two small workers keep the disk and the database from growing without bound.

Deployment¶

The service is one Hop3 app, described by hop3.toml: a web process, a PostgreSQL addon, a persistent cache directory, and the workers.

cdn migrate                    # before the new code serves traffic
cdn sweep  --interval 300      # evict from the cache past its high-water mark
cdn rollup --interval 3600     # fold statistics, prune the audit log

The web process must bind 0.0.0.0:$PORT, and an ASGI app needs an explicit start command — uvicorn abilian_cdn.app:serve --factory --host 0.0.0.0 --port $PORT. serve rather than create_app because it wraps the application in the host router that CDN_S3_HOST needs.

The proxy in front¶

Two things have to be set on whatever terminates HTTP, and Hop3 has no key for either, so they are set on the server by hand:

• client_max_body_size at least as large as CDN_MAX_UPLOAD_BYTES, or the proxy refuses a large upload before this service can say anything useful about it.
• proxy_request_buffering off, or the proxy writes the whole body to its own disk before the app sees a byte — a 2 GB upload then needs 2 GB of proxy scratch space.

An earlier draft of the deployment notes declared a [proxy] section in hop3.toml. Hop3's schema has no such section and rejects unknown ones, so it failed the deploy rather than being ignored.

The cache volume¶

CDN_CACHE_DIR holds the local copies and their metadata. It is persistent so a restart does not re-fetch everything, and disposable: losing it costs latency, never data.

The sweeper evicts least-recently-read files once the directory passes CDN_CACHE_HIGH_WATER and stops at CDN_CACHE_LOW_WATER. Two marks rather than one, so a full cache does not evict on every single write.

Exclude it from backups.

What to back up¶

PostgreSQL holds the index, the accounts, the tokens, the statistics and the audit log. S3 holds the bytes of every storage zone. Back up the first; the second is already durable.

You also need CDN_SECRET_KEY. Losing it signs everyone out and invalidates every derived S3 credential; the objects are unaffected.

Checking the two halves agree¶

An upload writes the bytes before the index row, and a delete removes them after it, so a crash in between can leave a blob nothing points at or a row pointing at nothing. Neither is visible in normal use: the first quietly costs storage, the second answers 500 to the one person who asks for that file.

cdn reconcile                  # report
cdn reconcile assets --repair  # and fix

It walks the index and the bucket as two sorted streams, so a zone of any size is compared in constant memory. It reports by default and exits non-zero when something disagrees, which makes it a reasonable nightly cron; --repair deletes orphaned blobs, drops dangling rows and corrects sizes from the store.

Health checks¶

The shallow check queries the database on every call, because a check that only proved the process was running would report a service answering every request with a 500 as healthy. The deep one is deliberately not for the load balancer: taking the service out of rotation because a third party is slow turns their bad minute into your outage, while cached objects would have kept serving.

A local object store¶

For development, any S3 works. The test suite and the examples here use Garage:

scripts/garage-dev.sh start

MinIO, Scaleway, OVH and AWS all work the same way — the service holds no assumption beyond S3 itself.