Deploy & run

indice runs in one of two shapes, and the only question that separates them is who may use the management commands.

• Workstation. indice serve on loopback. Whoever reaches the port is the operator and can do everything, with nothing to configure and no login. That is the whole story for local use; see Install and Try it in a minute.
• Server. Behind a TLS-terminating proxy and an identity provider, on two hostnames: one for the site and one for replay, which runs archived pages’ JavaScript where it cannot act as your readers (why). Anyone may read the archive; the people listed in users.yaml may change it. This page is about that shape.

No read-only server shape exists. A server you cannot write to sends you back to the command line the first time you want to fix a finding aid, and a server with an empty users.yaml already reads as read-only to every visitor.

Read this before a public instance

indice is alpha, and what it does and does not defend is written down rather than implied: Securing a server is the operator’s version, Threat Model in the design notes is the full argument, and SECURITY.md lists what is already filed. Several of the open items matter more when your curators are strangers than when they are colleagues, so it is worth ten minutes before you put one on the open web.

Container image

A multi-arch image (linux/amd64 + linux/arm64) is published to the GitHub Container Registry on every release. It carries no default arguments, because serving on a public interface needs an authenticating proxy and indice refuses to start without one, so there is no sensible command to guess. docker run with no arguments prints the help.

The stack below supplies the real command. To run the container yourself, behind a proxy you already have:

docker run -p 127.0.0.1:8080:8080 -p 127.0.0.1:8081:8081 -v indice-data:/data \
  -e INDICE_AUTH_PROXY_SECRET=a-long-random-string \
  -e INDICE_SITE_URL=https://archive.example.org \
  -e INDICE_REPLAY_URL=https://replay.archive.example.org \
  ghcr.io/edsu/indice:latest \
  serve --bind 0.0.0.0:8080 --home /data --auth-proxy-header X-Forwarded-Email

Port 8081 is the replay listener, which your proxy serves on the replay hostname; see below.

/data is indice’s home, holding archive/, collections/, and index/. Mount a volume so it survives restarts.

The stack

compose.yaml runs three containers: Caddy terminates TLS, oauth2-proxy brokers the login against your identity provider, and indice serves the archive.

Three containers for one application deserves a word. indice authenticates nobody: it reads an identity from a request header and checks it against users.yaml. That is what lets one mechanism cover SAML, OIDC and anything else your institution already runs, and it is why the Shibboleth section below needs no oauth2-proxy at all. The price is that something has to turn a login into that header, and for an OIDC issuer oauth2-proxy is the smallest well-tested thing that will. It proxies nothing here, despite the name; Caddy does that, while oauth2-proxy answers one question per request and handles the sign-in and sign-out routes.

Swap it for whatever suits you. The two containers that matter are indice and something terminating TLS.

SITE_ADDRESS=archive.example.org REPLAY_ADDRESS=replay.archive.example.org docker compose up -d

Caddy passes byte-range requests through untouched, so ReplayWeb.page’s ranged reads of large WACZs replay through the proxy. Named volumes persist indice’s /data and Caddy’s certificates. (compose.yaml builds the image from the repo by default; to pull the published image instead, follow the comment in the file.)

Load archives by indexing into the running container:

docker compose cp your.wacz indice:/data/your.wacz
docker compose exec indice indice index --collection "Your Collection" /data/your.wacz

Replay gets a hostname of its own

A server needs two hostnames: one for the site, and one for replay. indice will not start without the second.

Archived pages run their own JavaScript. That is how a replayed page still works, and it means that a page someone got crawled in good faith runs its author’s script in your readers’ browsers. If replay ran on the site’s own origin, that script would be on the same origin as the write API, and could act as whoever views it: a curator replaying a hostile capture could have a collection deleted under their name. So indice serves replay from a second listener, and the browser keeps pages on the replay hostname away from the site. A capture’s script there can reach neither indice’s API as the reader nor the page around it.

A subdomain is the simple choice, say replay.archive.example.org beside archive.example.org. Point both names at the machine; the shipped Caddyfile gets a certificate for each and sends the replay name to indice’s replay listener, with no login and no identity headers.

A separate domain is stronger still, say example-replay.org. A subdomain shares its parent domain with the site, so script on it can set cookies on that parent. indice’s own cookie cannot be planted that way and authorizes nothing anyway, but other services on your domain may rely on cookies of their own. If your institution runs many services under one domain, give replay a domain of its own.

Either way it is configuration, not code: REPLAY_ADDRESS names the hostname to Caddy, and indice frames replay from https:// that hostname (REPLAY_URL overrides it).

A workstation needs none of this. indice serve on loopback runs replay on the next port, 127.0.0.1:8081 beside 127.0.0.1:8080, which is already another origin.

Setting it up

indice speaks to no identity provider and stores no passwords. oauth2-proxy brokers the login against yours over plain OIDC, so Google, Microsoft Entra, Okta, Auth0, Keycloak, Authentik, Dex, GitLab or any issuer with a discovery document will do.

1. Point two DNS names at the machine: the site’s, and replay’s (see above).
2. Register a confidential client with your issuer, with redirect URI https://your-domain/oauth2/callback.
3. Put the details in a .env file next to compose.yaml:

   OIDC_ISSUER_URL=https://accounts.google.com
   OIDC_CLIENT_ID=...
   OIDC_CLIENT_SECRET=...
   OIDC_REDIRECT_URL=https://your-domain/oauth2/callback
   OAUTH2_PROXY_COOKIE_SECRET=      # a 32-char secret from: openssl rand -hex 16
   INDICE_AUTH_PROXY_SECRET=a-long-random-string
   REPLAY_ADDRESS=replay.your-domain
   # SITE_URL=https://your-domain   # defaults to https://$SITE_ADDRESS
   # OIDC_EMAIL_DOMAINS=example.edu    # who may sign in at all; defaults to anyone
   # OAUTH2_PROXY_COOKIE_SECURE=false  # only to run the stack on plain-HTTP localhost
   # BROWSERTRIX_USER=... BROWSERTRIX_PASSWORD=...  # optional: the accession desk's imports
   # ARCHIVEIT_USER=...   ARCHIVEIT_PASSWORD=...

   The import credentials are optional. With them, the accession desk offers a Browsertrix or Archive-It import, and indice import works in the container; see Importing from Browsertrix and Importing from Archive-It.
4. Write a users.yaml saying who may do what. A server will not start without one, for the reason in the next section. It lives at /data/users.yaml inside the container, which is the named volume, so put it there before the first start:

   cat > users.yaml <<'YAML'
   users:
     - id: you@example.org
       role: admin
   YAML
   docker compose create indice                       # make the container without running it
   docker compose cp users.yaml indice:/data/users.yaml

   Editing it later needs no restart: indice notices the file has changed and re-reads it on the next request, so approving a colleague is docker compose cp again.
5. SITE_ADDRESS=your-domain docker compose up -d.

Clicking Log in sends you to your issuer. After you authenticate you come back signed in, with the workroom chrome if users.yaml lists you as a curator or an admin. You may land on the homepage rather than the page you left, because the final hop carries the issuer’s address as its referrer and indice will not redirect to an off-site one.

What it asks your issuer for

openid email, and nothing else. indice identifies people by email address, which is what users.yaml lists and what it canonicalises internally, so it has no use for a profile, a photo or a group list.

The stack uses plain OIDC rather than oauth2-proxy’s GitHub integration, which asks for read:org whether or not you restrict anyone by organization: it calls GitHub’s /user/orgs on every login to populate groups for apps that authorize on them. indice authorizes on users.yaml, so that would be asking visitors for something it never reads. Plain OIDC makes no calls beyond the issuer’s own endpoints.

GitHub publishes no OIDC discovery document for user login, so signing in with a GitHub account now means putting a broker in front, such as Dex with its GitHub connector, which then presents OIDC to oauth2-proxy. That costs a fourth container and buys back a minimal consent screen.

Trying the stack on your machine

You can run the whole stack on your own machine before you have a domain. You still need an OIDC client: with Google, make an OAuth client ID of type “Web application” with http://localhost/oauth2/callback as its redirect URI. Then set these in .env, alongside the client’s details and the two secrets:

OIDC_REDIRECT_URL=http://localhost/oauth2/callback
OAUTH2_PROXY_COOKIE_SECURE=false
REPLAY_ADDRESS=http://replay.localhost
REPLAY_URL=http://replay.localhost
SITE_URL=http://localhost

• Browsers send a Secure cookie only over HTTPS, so on plain HTTP the login loops unless OAUTH2_PROXY_COOKIE_SECURE is false.
• The http:// on REPLAY_ADDRESS tells Caddy to serve that name over plain HTTP instead of getting a certificate for it. REPLAY_URL has to say the same, because its default puts https:// in front of REPLAY_ADDRESS.
• Chrome and Firefox send every *.localhost name to your own machine, so replay.localhost needs no DNS entry.
• SITE_URL is the address your browser uses for the site. indice accepts a write only from pages there, and its default, https:// in front of SITE_ADDRESS, would be wrong on plain HTTP.

Leave SITE_ADDRESS unset. Caddy then serves the site over plain HTTP on port 80, which has to be free.

To keep indice’s home in a directory you can see, rather than in a named volume, add a compose.override.yaml beside compose.yaml:

services:
  indice:
    volumes:
      - ./indice-data:/data

Compose reads compose.override.yaml automatically and merges it over compose.yaml. It matches volumes by their path in the container, so this one replaces the named volume at /data. Put users.yaml in indice-data/ and run docker compose up -d. The image runs as an unprivileged user, UID 65532. Docker Desktop lets it write to a bind mount; on Linux, chown the directory to that UID first.

An override file is the way to keep any local change out of the shipped file, on a server too: a bind mount, other ports, a pinned image tag. git pull then never conflicts with your edits. Compose’s documentation covers how the files merge.

Who gets in, and who can write

Two questions, and two places to answer them.

Your identity provider decides who may sign in. Anyone it authenticates may, and signing in on its own grants nothing beyond public read. To narrow it, set OIDC_EMAIL_DOMAINS to your domain, or restrict who may use the client at the issuer. users.yaml decides who may write. List someone as a curator and they can accession and describe; list them as an admin and they can also deaccession. Everyone else, signed in or not, is a reader. See Who can do what.

We split them so you approve a colleague by editing one line in a file you can commit and diff, instead of reconfiguring your identity provider.

A server refuses to start without a roster

With no users.yaml at all, indice falls back to treating every authenticated user as an admin. That is the right default on a workstation, where there is one of you. On a server it hands the archive, and the delete button, to the first stranger your identity provider admits, so indice stops instead and tells you what to write.

If you want a public read-only archive, that is a roster with nobody in it:

users: []

Anyone may read, nobody may write. An empty roster is an answer; a missing one is a question.

There is no flag to skip this. “Everyone my provider admits is an admin” is not a setting worth keeping within reach on a networked instance, and an opt-out would mostly get used by whoever wanted the error to go away.

Using a different identity provider

The shipped stack is one worked example. indice’s side of it is two request headers, so anything that can authenticate a request and set them will work:

• the authenticated identity, in a header you name with --auth-proxy-header. The shipped stack uses X-Forwarded-Email; Authelia calls it Remote-Email.
• a shared secret in X-Indice-Auth-Secret, matching INDICE_AUTH_PROXY_SECRET. indice trusts the identity header only on requests carrying this secret, so it refuses anything that forged the identity or skipped your proxy.

Besides oauth2-proxy, that covers Authelia, Authentik, Pomerium, and an AWS or GCP load balancer doing OIDC.

Shibboleth

Shibboleth is the common case on campus, and it speaks SAML, so the OIDC stack above is the wrong shape for it. Use the header contract directly instead, and drop oauth2-proxy from the picture.

Run a Shibboleth SP in front of indice, in the usual Apache mod_shib or nginx arrangement. The SP handles SAML with your institution’s IdP and then exposes the released attributes to the backend. Point --auth-proxy-header at whichever one it sets, and add the shared secret in the web server config:

Use a lazy session, so the SP runs on every request without forcing anyone to log in:

<Location />
    AuthType shibboleth
    ShibRequestSetting requireSession 0
    Require shibboleth

    # Never trust a copy the client sent.
    RequestHeader unset X-Forwarded-Email
    RequestHeader unset X-Indice-Auth-Secret
    # Set them only when the SP has a session, so an anonymous reader stays anonymous.
    RequestHeader set X-Forwarded-Email %{mail}e env=mail
    RequestHeader set X-Indice-Auth-Secret "your-shared-secret" env=mail

    ProxyPass http://127.0.0.1:8080/
</Location>

# The one path that forces a login. indice's Log in link points here.
<Location /manage/login>
    ShibRequestSetting requireSession 1
    Require valid-user
</Location>

Replay gets a virtual host of its own, with no SP at all, since replay is the same for everyone and indice’s replay listener reads no identity. Start indice with --site-url https://archive.example.org --replay-url https://replay.archive.example.org:

<VirtualHost *:443>
    ServerName replay.archive.example.org
    # No Shibboleth here, and no identity headers either.
    RequestHeader unset X-Forwarded-Email
    RequestHeader unset X-Indice-Auth-Secret
    ProxyPass / http://127.0.0.1:8081/
</VirtualHost>

requireSession 0 plus Require shibboleth is the SP’s lazy-session mode: it does not send anyone to the IdP, and it fills in attributes whenever a session already exists. That gives indice what it wants, an identity on every request from someone signed in and nothing from someone who is not, so the workroom chrome appears on ordinary pages rather than only under /manage. Protecting the whole site with requireSession 1 instead would force a campus login on every anonymous visitor and make the archive private, which is the subject of the last section rather than this one.

Three things to get right:

Clear the headers before setting them. With a lazy session there is no attribute to read when nobody is signed in, and depending on the Apache version RequestHeader set ... %{mail}e either skips the directive, leaving a client-supplied value in place, or writes the literal (null). The unset lines above close that, and env=mail sets both headers only when the attribute exists. Adjust mail to whichever env var your attribute-map.xml produces.

Choose the attribute with care. A typical attribute release includes eduPersonPrincipalName (alice@example.edu) and mail. Both look like addresses, so indice canonicalises either one, but they are not the same thing: mail is a mailbox and changes when someone changes their address, while eppn is the campus identifier. eppn is the steadier choice for a roster. Some institutions reassign eppn after a person leaves, which is what eduPersonTargetedID and pairwise-id exist to avoid, though those are opaque strings nobody wants to type into users.yaml by hand. Whichever you choose, aliases covers the day an identity changes.

Set the secret in the server config, not from an attribute. It is a static value proving the request came through your SP, so it must not be anything the browser can influence.

If your IdP runs the OIDC OP plugin, officially supported since IdP 4.1 and still maintained, you can skip all of this and point the stack above at it like any other issuer. A SAML-to-OIDC broker such as SATOSA or Keycloak is the third option, and worth it only if you want OIDC for other reasons too.

Two requirements of whatever you put in front:

• Forward the identity on every request, including the ordinary pages. indice draws the workroom chrome on the homepage and the collection pages for a signed-in curator, so it needs to know who you are there too. The shipped Caddyfile asks oauth2-proxy on each request and forwards the answer when there is one.
• Strip any client-supplied copy of both headers, and set the secret only on requests your proxy authenticated.

Using a different web server

Caddy is doing six separable jobs here, and none of them is Caddy-specific. Anything that can do all six works:

1. Route /oauth2/* to your login service.
2. Ask who the caller is on every request, and continue anonymously when the answer is nobody. This is the one people get wrong. Gating only /manage and the write APIs looks right and costs you the workroom chrome on ordinary pages, because indice decides whether to draw the edit controls from the forwarded identity.
3. Inject the identity and the shared secret together, only on a request your proxy authenticated, replacing any copy the client sent.
4. Leave byte ranges alone. No gzip on a ranged response, no buffering of large bodies, or WACZ replay breaks.
5. Send the replay hostname to the replay listener, port 8081 by default (--replay-bind), with no login and with both identity headers stripped, and pass that hostname to indice as --replay-url.
6. Set Strict-Transport-Security on both hostnames, since the proxy is what terminates TLS. indice sets its other headers itself (a Content-Security-Policy per origin, X-Content-Type-Options, Referrer-Policy and X-Robots-Tag), so pass them through rather than replacing them. Send /robots.txt to indice on both hostnames too: each has its own.

On indice’s side, pass the site’s public URL as --site-url. A server will not start without it, and it means indice never relies on the proxy’s Host or X-Forwarded-Host, which nginx and Apache rewrite by default. See What --site-url does.

nginx does this with auth_request, which is the same subrequest mechanism, plus error_page 401 = @anonymous; pointing at a named location that proxies to indice with no identity headers. That named location is the equivalent of the shipped Caddyfile’s handle_errors block. Check proxy_buffering against a large ranged read while you are there.

Apache with mod_auth_openidc speaks OIDC itself, so it replaces Caddy and oauth2-proxy, leaving two containers. It is the same arrangement as the Shibboleth section above: the web server authenticates and sets headers, indice reads them.

Traefik has a ForwardAuth middleware that fits job 2.

One arrangement to avoid: running oauth2-proxy as the reverse proxy itself, with OAUTH2_PROXY_UPSTREAMS pointed at indice and no Caddy. It is tempting because it drops a container, but in that mode it gates everything by default, so you have to list the public routes in --skip-auth-regex. That is a second copy of indice’s route policy living somewhere no test can see it, which is the exact arrangement that broke the public annotations API before.

These are sketches, not tested configs

The shipped Caddyfile is exercised by scripts/smoke-deploy.sh. The alternatives above are not. Proxy configuration fails quietly: Caddy’s documented handle_errors 502 503 504 form, for one, parses and validates and then never matches, so the fallback it describes silently never runs. Test your own against the same checks that script makes.

Keeping an instance private

The server shape serves reads to anyone. If your archive holds in-copyright or embargoed material that should stay off the open web, you can gate reads at the proxy as well. indice does not mind: the identity header then arrives on read requests too, and the roster still decides who may write. Nothing changes inside indice.

The shipped Caddyfile needs three changes first, and all of them are easy to miss because the site looks private once the homepage asks for a login.

Remove the @bytes bypass. That block sends /files/*, /thumb/*, /collection-thumb/*, /replay/* and /assets/* straight to indice with no auth subrequest, because identity cannot change those responses and replay issues a lot of them. On a public archive that is a performance win. On a private one it is the hole: /files/{id} returns the whole WACZ and /thumb/{id} returns page images, so the material you are protecting is served to anyone who has an id, while the pages that merely describe it are behind a login. Delete the block and let those routes go through the catch-all.

Gate the replay host too. It serves /files/{id} and the collection manifests, which are the bytes you are protecting, and the shipped replay block asks nobody who is calling. Give it the same auth subrequest as the site, refusing anyone without a session, and scope oauth2-proxy’s cookie to the parent domain (OAUTH2_PROXY_COOKIE_DOMAINS=.archive.example.org, with OAUTH2_PROXY_WHITELIST_DOMAINS set to match) so that signing in on the site covers both names. Readers sign in on the site; the replay frame then loads with the cookie they already have. This needs replay on a subdomain of the site rather than a separate domain. The replay listener still reads no identity; the proxy alone decides who reaches it. The smoke test does not cover this arrangement, so test it with a WACZ id in hand: /files/{id} on the replay host must refuse a browser that is not signed in.

Replace the handle_errors fallback with a failure. It exists so a dead login service does not take a public archive offline, which means it serves every request anonymously. On a private archive that turns an oauth2-proxy outage into an open door. Swap the body for respond "authentication unavailable" 503.

Once reads are gated, OIDC_EMAIL_DOMAINS becomes load-bearing rather than cosmetic: it is what stops anyone with an account at your issuer from reading the archive.