XOpat — OpenSeadragon-based histology data visualizer

xOpat is a JavaScript application. Two reference backends ship in the repo and either may serve it:

• server/node/ — canonical Node.js backend (see server/node/README.md). Started with npm run s-node (production) or npm run dev (server/utils/node/dev-mode.js).
• server/php/ — legacy PHP backend (entrypoint server/php/index.php → server/php/init.php).

Both backends inject the same runtime configuration into the browser and provide the proxy/auth/storage endpoints the client expects. A high-level integration story lives in ../INTEGRATION.md; operational/deployment docs at https://xopat.org.

Configuration

The viewer always boots from a single object (XOpatRuntimeConfig, see src/types/app.d.ts) carrying params, data, background, visualizations, plugins. The client resolves where that object comes from in this order — first hit wins (src/parse-input.js, xOpatParseConfiguration):

1. POST body, field visualization (legacy alias visualisation also accepted). Canonical delivery for non-trivial sessions; the field carries either a JSON object or a JSON-encoded string. The server advertises POST support via XOpatServerConfig.supportsPost (src/types/config.d.ts).

2. URL hash #<urlencoded-json> — parsed locally. If supportsPost is true the viewer transparently rewrites the navigation into a self-POST (hidden form in parse-input.js) so refreshes/shares stay POST-backed and the address bar is clean.

3. ?visualization=<urlencoded-json> query parameter — same parser as the hash path.

4. ?slides=id1,id2&masks=m1,m2 shorthand — synthesizes one background per slide plus a heatmap-shader visualization per mask (parse-input.js). Convenient for quick links and CI tests.

5. Storage fallback — localStorage["xoSessionCache"] (or sessionStorage["xoSessionCache"]) restores the last successful session if it is < 30 minutes old. The restored config is marked __fromLocalStorage: true so plugins can detect it. Every successful boot writes the current session back to both storages, so an auth-redirect round-trip never loses state.

   Cache scoping. The entry carries __envKey, the deployment cache key (src/classes/app/deployment-key.ts) — a fingerprint of the configuration that decides whether a cached session's data references can still resolve: domain/path/name/version, active_client, slide_protocols, the default background/visualization protocols, the legacy image_group_*/data_group_* fields, and the ids of the plugin and module registries the server actually shipped (a factory protocol such as dicom is registered by a plugin, so a session referencing it is invalid where that plugin is absent). Cosmetic config — themes, UI flags, viewport defaults — is deliberately excluded, or every unrelated tweak would discard the user's session. Without this, two env files served from the same localhost replay each other's sessions and shaders fail with "no protocol resolvable for role visualization".

   The same key stamps __xopat_session__ (the one-shot navigation payload, application-lifecycle-controller.ts) and names the plugin-autoload cookie (_plugins.<key>), so switching XOPAT_ENV no longer resurrects the previous deployment's plugins. It is computed once in initXOpat from the served ENV plus the plugin/module registries and published as window.XOPAT_DEPLOYMENT_KEY. It deliberately does not scope kv:* storage (AppCache, AppCookies, plugin caches), which stays keyed by <ownerUid>::<key>.

   Eviction, not just rejection. An entry that fails any check — key mismatch (including a missing key, which is evidence of nothing), unparseable JSON, expired, or a configuration that no longer parses — is removed from the store it came from, so it stops costing a read on every future boot. The two stores are judged independently and one never evicts the other: localStorage is shared across tabs, so another deployment can overwrite it while this tab's sessionStorage still holds a valid session. localStorage is tried first and capped at 30 minutes; sessionStorage is the fallback and is not aged out, because it dies with its tab and exists to survive auth redirects. A failed restore leaves postData untouched.

   The session carries the key too. UTILITIES.serializeAppConfig stamps __envKey on everything the viewer serializes — which covers transports 1-3 above, because they outlive an ENV swap in a way storage does not: syncSessionToUrl writes the whole session into the address bar hash on every shader edit, and the self-POST rewrite parks the body in the history entry, where a reload re-submits it. Both are read before the boot cache. A session whose stamp does not match this deployment is still opened — deployments that differ only cosmetically fingerprint alike, so a genuinely shared link keeps working — but it warns (messages.sessionOtherDeployment) and is never written into xoSessionCache. Without that last part the boot cache launders it: one stale hash loads, gets saved under the new deployment's key, and is restored legitimately on every later boot. A session with no stamp (an embedding app, a demo link, a hand-written config) is accepted unchanged.

   Operators pin the key with core.client.<active>.cacheKey (legacy aliases: client.sessionCacheKey, setup.sessionCacheKey) — a production keeps one key so nothing is ever invalidated, a development deployment gives each env file its own key, or simply relies on the fingerprint. setup.bypassCache: true disables the restore path entirely. See src/classes/app/deployment-key.ts and the cache-restore block in src/parse-input.js.

A simple form that just POSTs a session JSON into the visualization field is available at /dev_setup on both backends (server/node/index.js — the /dev_setup route, server/php/dev_setup.php, template server/templates/dev-setup.html). Use it during development; in production the embedding application supplies POST data directly.

Plugins may layer additional opening behavior on top of this pipeline — check the relevant plugin README.

Example session

{
  "params": {
    "sessionName": "Demo case 0042",
    "locale": "en"
  },
  "data": [
    {
      "dataID": "path/to/tissue/scan.tif",
      "microns": 0.001,
      "protocol": "dzi",
      "options": { "format": "jpeg" }
    },
    "path/to/annotation.tif",
    "path/to/probability.tif"
  ],
  "background": [
    { "dataReference": 0 }
  ],
  "visualizations": [
    {
      "name": "A visualization setup 1",
      "shaders": {
        "shader_id_1": {
          "name": "Advanced visualization layer",
          "type": "edge",
          "fixed": false,
          "visible": 1,
          "dataReferences": [2, 0],
          "params": {}
        },
        "another_shader_id": {
          "name": "Probability layer",
          "type": "edge",
          "visible": 1,
          "dataReferences": [1],
          "params": { "color": "#fa0058", "use_gamma": 1.0 }
        }
      }
    }
  ],
  "plugins": {
    "recorder": {}
  }
}

data — DataSpecification[] (required)

Each entry is either a bare DataID (string/object the image server understands — most often a UUID4 or file path; objects are used by sources like DICOM) or a DataOverride (DataOverride in src/types/app.d.ts):

• dataID (required) — the underlying DataID.
• options — generic map forwarded to the TileSource (SlideSourceOptions in src/types/app.d.ts). Standard keys: format.
• microns / micronsX / micronsY — pixel size in micrometers.
• magnification — the image's native optical magnification (e.g. 40 for a 40x objective). Omit it (undefined) when unknown: the core then guesses one from the pixel size against a whole-slide optics table, and warns that the image looks like a macro image when it cannot. Set it to null when magnification does not apply to the modality at all — a CT/MR/PT has no objective, and without the explicit null every such image opens with a spurious "this is a macro image" warning and a meaningless magnification ladder. A tile source may declare the same field from getMetadata(); the data specification wins when both are present.
• protocol — name of a registered slide protocol (see Slide protocols below). In non-secure mode a backtick-template string is accepted for back-compat, but is rejected with a warning in secure mode. This is also how a session mixes upstreams with different credentials: the protocol entry owns the HttpClient, hence the auth context, so per-item auth = per-item protocol. A session never names an auth context directly (§7 of AGENTS.md) — see Slide protocols below and AUTH.md.
• imageSmoothingEnabled — when false, tiles for this data source are sampled with gl.NEAREST (blocky pixels at high zoom — useful for label maps or integer-coded segmentation layers). When true or unset (default), tiles use gl.LINEAR. Honored by drawers that implement setTiledImageSmoothingEnabled (currently FlexDrawer); silently ignored otherwise.
• pixelScale — how many pixels of the stack's reference image (its background) one pixel of this image covers. 2 = half-resolution; 512 = each pixel is one 512-px prediction square. OpenSeadragon normalizes every image in the world to viewport width 1, so an overlay lands on its background only when their aspect ratios match — and an overlay covering a whole number of blocks of a slide whose width is not a whole number of blocks never matches: its edge block hangs past the slide, OSD squeezes it back to fit, and every cell ends up slightly small with the error accumulating across the image. Declaring the scale is what lets the overlay overhang instead. Scalar or {x, y} (only x is used — OSD derives height from the image's own aspect). Meaningless on a background, which is the reference. Composes with a virtual-region crop rather than replacing it: the widths multiply. Absent, zero, negative or non-finite means "no opinion" and places the image exactly as before the field existed; session-supplied, so it is range-checked (src/classes/app/overlay-pixel-scale.ts).
• croppingContext — present only on a virtual (cropped) source resolved through the virtual-region protocol; carries the crop rectangle + alignment. Authored by the virtual-viewport machinery, not by hand — see VIRTUAL_VIEWPORTS_SPLIT.md.
• tileSource — deprecated escape hatch for code-only consumers; not serializable.

params — viewer setup (optional)

Aligned with XOpatSetup in src/types/config.d.ts. initXOpat silently drops unknown keys with a console warning (sanitizeAgainst in src/app.ts), so typos vanish quietly — verify names against the type.

Every key below is also a deployment default: core.setup.<key> in env/env.json is deep-merged over src/config.json and becomes APPLICATION_CONTEXT.config.defaultParams, which getOption consults when the session did not set the key and the user has no cached preference. Precedence is params (session/URL payload) → AppCache (user's Settings toggle) → core.setup → the caller's fallback. Because a caller fallback ranks below core.setup, never pass one that repeats the config.json value; declare the default in config.json and call getOption("key").

Key	Type	Default	Notes
sessionName	string	—	Unique session id; overridable by background[i].sessionName.
locale	string	"en"	i18next locale.
theme	"auto" | "light" | "dark"	"auto"	DaisyUI data-theme; "auto" follows the OS preference. ("dark_dimmed" / "dimmed" were never wired up in the v3 UI.)
customBlending	bool	false	Allow user-programmed blending.
debugMode	bool	false	Verbose runtime instrumentation.
webglDebugMode	bool	false	Debug post-processing.
webGlPreferredVersion	string	—	Select WebGL backend version.
webGlPrecision	string	"unorm8"	First-pass render target precision: "unorm8" | "auto" | "float16". "auto" lets float data (16-bit/floating-point TIFF, DICOM parametric maps) render unquantized; the offscreen colour array doubles, per renderer. This is the option the renderer's console notice about precision: 'auto' refers to. Required by plugins/dicom parametric maps and by low-range TIFF windowing.
valueInspectorEnabled	bool	false	Hover value inspector.
visualizationInspectorEnabled	bool	false	Pixel/lens inspector overlay.
visualizationInspectorMode	string	—	Inspector mode (paired with UTILITIES.setVisualizationInspectorMode).
visualizationInspectorRadiusPx	number	—	Inspector radius.
flexInteractionForwarding	string	"auto"	FlexDrawer pointer forwarding, required by shaders that read fr_interaction_* state (e.g. fisheye-lens). "auto" enables it per viewer only while such a visible layer exists; "always" | "never" pin it. Forwarding forces a viewer redraw on every changed pointer move.
visualizationInspectorLensZoom	number	—	Lens zoom factor.
activeBackgroundIndex	number | number[]	0	Initial bg index; array for multi-view. Runtime canonical shape is an array; an explicit [] means "nothing open" (distinct from an absent value, which falls back to this default).
viewport	ViewportSetup | ViewportSetup[]	—	{ point, zoomLevel, rotation? }; single value applies to all viewers or one per viewer in multi-view.
preventNavigationShortcuts	bool	false	Disable xOpat navigation bindings (OSD defaults still apply).
scrollRequiresCtrl	bool	false	Require Ctrl/Cmd + wheel to zoom; plain wheel scrolls the host page. Use for notebook / scrollable-host embeddings. A throttled toast nudges first-time users toward the modifier.
reverseScroll	bool	false	Invert the scroll-to-zoom direction — scroll down to zoom in, scroll up to zoom out. Composes with scrollRequiresCtrl.
snapZoomToMagnification	bool	true	Snap scroll-to-zoom between standard magnification stops (5x/10x/20x/40x…) instead of scaling continuously — only when the slide has a resolved native magnification (calibrated MPP); uncalibrated slides keep continuous zoom. Composes with reverseScroll.
scrollSpeed	number	1	Multiplier on the normalized wheel delta. Above 1 covers more zoom range per wheel turn, below 1 tames an over-sensitive device.
scrollPixelsPerNotch	number	120	Wheel pixels counted as one full zoom step (the conventional mouse notch). Trackpad deltas are smaller and apply fractionally, which keeps them smooth without throttling.
kineticPan	bool	true	Releasing a fast drag lets the slide coast to a stop. The drag itself stays 1:1 with the cursor.
kineticPanFriction	number	0.92	Velocity retained per 1/60 s of coasting; lower stops sooner.
kineticPanMinSpeed	number	300	Drag-release speed (px/s) below which no coast starts.
scaleBar	bool	true	Deprecated, use ui.scaleBar. Requires microns to render.
toolBar	bool	—	Deprecated, use ui.toolBar.
statusBar	bool	—	Deprecated, use ui.statusBar.
ui	XOpatUiSetup	—	Initial visibility of UI components — see table below.
disablePluginsUi	bool	false	Hide the plugin catalogue (browsing and loading new plugins). Loaded plugins keep their menus, settings tabs and view panels.
disablePluginsAutoload	bool	false	Skip the _plugins cookie restore for this session. permaLoad plugins and plugins listed in config.plugins still load. Use to ignore the user's prior manual picks while respecting deployment defaults and session-declared plugins.
grayscale	bool	false	Force grayscale transfer.
tileCache	bool	true	Enable tile caching.
maxImageCacheCount	number	1200	Tile cache size.
backgroundColor	string	—	Hex #RGB/#RGBA canvas clear color (e.g. fluorescence). Transparent if unset. Session-wide; a single slide overrides it with background[i].fill.
permaLoadPlugins	bool	true	Remember loaded plugins across sessions.
bypassCookies	bool	false	Skip cookie-backed user state.
bypassCloseConfirmation	bool	false	Avoid browser close confirmation popup in dirty (data modified) state.
bypassCache	bool	false	Never reuse cached values.
bypassCacheLoadTime	bool	false	Skip the cold-load session restore: a boot with no session of its own does not adopt xoSessionCache. Eviction and saving still run, so a boot that arrives with a session (auth-redirect return, POST, hash) is unaffected. Deployment-side (ENV.setup) — a cold load carries no params by definition.
historySize	number	50	Cap on the history stack (src/classes/history.ts).
isStaticPreview	bool	false	Disable interactive controls for thumbnail/preview embeds.
maxMobileWidthPx	number	—	Responsive breakpoint.
quickActions	Array<string | {id, icon?, label?}>	[]	Actions pinned as icon-only buttons in the top app bar, by AppBar.Actions catalogue key ("tools:core.sync.auto", "view:…", "shortcut:…", "custom:…"). Object entries (icon/label overrides) are honored from ENV core.setup only; a session/user list is reduced to plain ids.
quickActionsMaxVisible	number	5	Pins rendered as buttons before the rest spills into the overflow menu.
quickActionsUserEditable	bool	true	ENV-only lock. false freezes the pin list and hides the Settings → Quick actions card.

params.ui — UI initial visibility

Each flag is the initial visible state at boot. false boots the component collapsed, but the user can still bring it back via the settings menu, the hide-UI button, or the relevant opener. Defaults to true for every key. Reads go through APPLICATION_CONTEXT.getUiOption(key) which also honors the legacy flat aliases (scaleBar / toolBar / statusBar) and the AppCache of user-toggled settings — see XOpatUiSetup in src/types/config.d.ts.

Shorthand: set params.ui: false (or setup.ui: false for a deployment-wide default) to hide every global UI component in one shot — handy for notebook embeddings. params.ui: true is equivalent to leaving the field unset.

Key	Affects
scaleBar	Per-viewer OSD scalebar overlay. Replaces legacy flat scaleBar.
toolBar	Top viewer toolbar. Replaces legacy flat toolBar.
statusBar	Bottom status bar (#viewer-status-bar). Replaces legacy flat statusBar.
mainMenu	Global menu (FullscreenMenus). false boots collapsed; menu-open buttons still work.
navigator	Per-viewer OSD navigator panel.
appBar	Top AppBar chrome — false is equivalent to the hide-UI button being pre-toggled.
globalMenu	Global right-side dock (window.LAYOUT) that hosts plugin tabs (chats, slide-switcher, questionnaire, …). false boots the dock closed; user opens/plugins focus still work.

params.ui.globalMenuMode ("overlay" | "docked", string — not a boolean flag, so it is read directly by MainLayout, not via getUiOption) picks how that dock behaves: "overlay" (default) hides it to a thin edge rail that floats over the viewer on hover/focus; "docked" keeps it as a flex sibling that pushes the viewer. The user's runtime pin toggle (persisted in AppCache) overrides this default.

params.ui.sideMenuTabs (bool | Record<string, boolean>) picks which panels of the per-viewer right-side menu boot open. A boolean applies to every tab; a map is keyed by tab id with "*" as the fallback for tabs it does not name — e.g. {"*": false, "navigator": true} boots a clean viewer with only the navigator open. Also honored by panels plugins append later. Unset means every panel opens. Like sideMenuCompact it is not a boolean flag read via getUiOption; the user's cached per-panel open/closed toggle overrides this deployment default, so put it in the session params.ui when the boot state must be deterministic for returning users. ui.navigator: false still wins for the navigator tab (it hides the OSD navigator element, not just a panel).

background — BackgroundItem[]

Each item is an image group rendered as one OSD layer (BackgroundItem in src/types/app.d.ts):

• dataReference (required) — index into data, or an inline DataID / DataOverride. One reference per background entry.
• shaders (optional) — shader configuration array, same shape as visualization shaders (dataReferences becomes optional). When unset, the renderer synthesizes an implicit identity shader keyed under the background's id. As soon as any entry is set, the implicit identity is replaced. canonical-scene.ts materializes the implicit entry as [{ type: "identity", … }] when a tool edits it, so the change persists across reopens.
• id — unique id; derived from the data path if unset.
• name — tissue name shown in the UI.
• sessionName — overrides params.sessionName for this background.
• visualizationIndex — index into visualizations selected when this background is mounted. Authoritative per-viewer viz binding — the slot's viz follows the bg entry through slot reordering / insertion / deletion. Pass null for "no overlay". Legacy goalIndex is still accepted on read (folded with a one-time warning).
• options — forwarded to the TileSource.
• fill — canvas clear color while this background is open, hex #RGB / #RGBA / #RRGGBB / #RRGGBBAA. Per-background override of params.backgroundColor; unset falls back to it (transparent by default). Use it when one slide in a session needs its own backdrop — a fluorescence slide on black next to a brightfield slide on white. Resolved via BackgroundConfig.resolveFillColor; a malformed value is ignored with a console warning. Applied per viewer before the shader (re)build, so it also reaches the navigator, the offscreen/standalone drawers (thumbnails, region exports, vision inference images) and the Visualization Playground's sandboxed viewer.

Legacy fields protocol, microns, micronsX, micronsY are still accepted at the background level for back-compat, but new code should put them on the DataOverride instead.

visualizations — VisualizationItem[]

WebGL composition goals over the data group (VisualizationItem in src/types/app.d.ts):

• shaders (required) — map of shader id → layer spec:
  • type (required) — one of the renderer's registered shader types (identity, colormap, heatmap, bipolar-heatmap, gridheatmap, edge, threshold, stain-separation, single_channel, group, … ), or any custom-registered shader. Do not hardcode this list: the authoritative, always-current index is visualization.getSchema()["x-shaderCatalog"], which carries each type's name, intent and what it expects from the data.
  • dataReferences (required) — index array into data.
  • visible — 1/0 or boolean.
  • name — UI label.
  • fixed — if false, user can change the shader type; default true.
  • params — shader-specific defaults; invalid entries fall back silently.
• name — goal label.

Which visualization a viewer slot renders is not stored here — it is the per-background visualizationIndex binding above.

Legacy protocol accepted at the visualization level for back-compat — prefer DataOverride.

plugins

Plugin-id → plugin-config map; consult each plugin's README.

Advanced features

Internal parameters. The runtime augments visualization items at runtime with fields that show up in serialized sessions:

• order — shader-id array on a visualization goal; sets render order. All referenced data with visible=1 must be present and valid.
• cache — per-shader, shader-type-dependent value bag (equivalent to default-value overrides). Type-sensitive: writing a wrong-type value will break rendering.

Slide protocols. A protocol is a named entry in ENV.client.slide_protocols (see XOpatClientConfig.slide_protocols in src/types/config.d.ts and SlideProtocolEnvEntry in src/types/slide-protocols.d.ts; registry implementation at src/classes/slide-protocols.ts). Each entry is either:

• a URL template string with data in scope (non-secure mode only — rejected in secure mode), or

• an object { url, tileSourceClass?, tileSourceOptions?, proxy?, baseURL?, auth?, … }. tileSourceClass / tileSourceOptions are described below; every other field is forwarded verbatim to new HttpClient(...), so every metadata + tile request the resulting TileSource issues inherits proxy routing, CSRF tokens, and JWT/auth headers uniformly.

  An auth block needs a transport to bind to: with neither proxy nor baseURL no client is built and the TileSource falls back to an unauthenticated bare fetch — the registry warns once per entry when that happens. With auth.required, the entry also declares the context requirement (so an unclaimed context is reported — core declares every such entry at before-app-init, not lazily on first slide) and its requests wait for that context to finish authenticating instead of racing the login — see AUTH.md. Omit auth.types: they are resolved per request from the auth module owning the context.

  The entry is the unit of credential. One entry → one HttpClient → one auth context, so streaming from two upstreams that need different logins means two entries, and each data item selects its own with protocol:

  // env.client.slide_protocols
  "hosp_a": { "url": "`/slides/${data}`", "proxy": "img", "auth": { "contextId": "hospital-a", "required": true } },
  "hosp_b": { "url": "`/slides/${data}`", "proxy": "img", "auth": { "contextId": "hospital-b", "required": true } }
  // session data: [{ "dataID": "s1", "protocol": "hosp_a" }, { "dataID": "s2", "protocol": "hosp_b" }]

  Resolution carries that client on its result (ResolvedSlideProtocol.client) and the open pipeline threads it to the source it opens, so the two slides above stay on their own credentials even though they share a proxy alias — and every tile keeps using it (tileSource.__xopatHttpClient, which is also what tells a 401 handler which context died). Recovering a client from a URL cannot do that: two entries on one upstream render indistinguishable URLs, so getActiveClientForUrl returns undefined (warning once) for a base URL claimed by two contexts rather than guessing. Ask by id instead — SLIDE_PROTOCOLS.getClientForProtocol("hosp_b").

Explicit tile-source selection (tileSourceClass). By default OpenSeadragon fetches the slide metadata with a generic TileSource and only then picks a class, by scanning its namespace for the first *TileSource whose supports(data, url) matches. Two consequences: the winner depends on script load order when several classes match the same URLs, and the class cannot influence (or even see) its own metadata request — so per-slide options can only be applied afterwards, via setSourceOptions.

Naming a class on the protocol entry makes the registry construct it directly from the rendered URL, skipping autodetection and applying options synchronously before the metadata request fires:

"slide_protocols": {
  "rationai_wsi": {
    "url": "`http://localhost:8080/v3/slides/info?slide_id=${data}`",
    "tileSourceClass": "RationaiStandaloneV3TileSource"
  }
}

The name is resolved by own-property lookup on the global OpenSeadragon (never eval); the class must subclass OpenSeadragon.TileSource and declare static xopatSelfConfiguring = true — the contract documented in src/tile-source.ts (configure this in place from getImageInfo, set ready before raising it, tolerate setSourceOptions being called twice). Anything else logs one warning and degrades to the normal autodetect path. tileSourceOptions adds literal constructor fields.

This is operator-only, by design: it lives on the ENV protocol entry, never on a DataOverride / session bundle, so a third-party or imported session cannot choose which code runs (§ security). Sessions select behaviour the audited way — by naming a registered protocol id.

Option timing. DataOverride.options merged under the background/visualization entry options (the entry wins) reaches the source before its metadata request for tileSourceClass and factory protocols, and again after the item opens; for plain autodetected URLs, only the latter.

DataOverride.protocol (and legacy BackgroundItem.protocol / VisualizationItem.protocol) reference an entry by name ("dzi", "dicomweb", …). Defaults come from default_background_protocol / default_visualization_protocol; the legacy image_group_* / data_group_* env keys are auto-migrated into synthesized __legacy_bg / __legacy_viz entries. Plugins and modules register protocols at runtime via window.SLIDE_PROTOCOLS.register({ id, createTileSource }) — which is why a configured default is validated on use, not when env is ingested: env is read before those scripts run, so a default may legitimately name an entry that does not exist yet.

Use this registry instead of hand-rolling URLs in background.protocol/visualizations.protocol — those evaluations are blocked in secure mode and lose proxy/auth integration.

Asking who owns a slide. To find out which protocol serves a background without building anything, call SLIDE_PROTOCOLS.protocolIdFor({ spec, bgEntry, role: "background" }). Never use resolve(...) for that: it calls createTileSource for factory entries, so an ownership probe would construct a foreign protocol's tile source — and issue its requests — only to discard it. protocolIdFor runs the selection half only, stays silent (a probe repeats for every background), returns "__inline_tile_source" for the deprecated DataOverride.tileSource bypass, and returns undefined where resolve would throw. A module that auto-configures slides must gate on it, so a data id that merely looks like its format (a .tif served over DICOM) is never claimed.

Structure

Each folder ships a README with more detail. The most up-to-date ones are this file, ../plugins/README.md, and ../modules/README.md.

../ (repo root)

• index.html and the server/ tree (Node + PHP entrypoints).
• package.json — s-node, s-node-test, dev, docker-node, docker-php.

./ (src/)

• app.ts — initXOpat(...) entrypoint; builds APPLICATION_CONTEXT, VIEWER_MANAGER, IO_PIPELINE, SLIDE_PROTOCOLS (and SESSION, currently wired off).
• loader.ts — module/plugin loader and the global helpers plugin(id), singletonModule(id), viewerSingletonModule(className, viewerLike).
• parse-input.js — the precedence chain described in Configuration above.
• store.ts — pluggable storage middleware (KV drivers used by the IO pipeline).
• tile-source.ts — common TileSource scaffolding + extension contracts (metadata, thumbnails, HttpClient tile routing, z-stack opt-in — see ZSTACK.md).
• classes/
  • app/ — viewer-open pipeline and canonical-scene round-trip (viewer-open-pipeline.ts, canonical-scene.ts, application-lifecycle-controller.ts, viewer-inspector-controller.ts); focal-plane navigation (viewer-depth-controller.ts, z-plane-prefetcher.ts, see ZSTACK.md); canvas input controllers (viewer-scroll-zoom-controller.ts — wheel normalization and scroll policy, viewer-kinetic-pan-controller.ts — drag momentum, viewer-rotation-controller.ts, viewer-joystick-controller.ts).
  • io/ — IO pipeline implementation (see IO_PIPELINE.md).
  • auth/ — core auth broker XOpatAuth = APPLICATION_CONTEXT.auth (see AUTH.md).
  • session/ — live-collaboration controller (see SESSION.md; currently not instantiated).
  • visualization/ — visualization runtime/registry helpers behind the shader menu and renderer binding.
  • scripting/ + scripting-manager.ts — sandboxed scripting API.
  • tile-sources/ — built-in OpenSeadragon.TileSource implementations registered on the OSD namespace: extended-dzi-tile-source.ts (RationAI DeepZoom ImageArray, auto-detected), empty-tile-source.ts (faulty/empty layer placeholder), preview-slide-source.ts (single decoded image as a one-tile pyramid). Loaded as plain core scripts (config.json js.src.app) after the OSD library and before dist/app.js.
  • slide-protocols.ts — SLIDE_PROTOCOLS registry (per-entry HttpClient, hence per-entry auth context).
  • virtual-region-protocol.ts — the built-in virtual-region factory protocol + CroppedTileSource (see VIRTUAL_VIEWPORTS_SPLIT.md).
  • background-config.ts — BackgroundConfig, the normalized view over a background[i] entry.
  • http-client.ts + remote-endpoint.ts — HttpClient and its transport-agnostic proxy/auth base (see HTTP_CLIENT.md).
  • osd/ — first-party extensions installed onto the OpenSeadragon namespace: tools.ts (viewer.tools), viewport-registration.ts (automatic multi-viewer alignment), scalebar/ (the scalebar, its magnification chrome and ViewportSyncAPI). Side-effect-imported from app.ts.
  • app/tutorial/ — the interactive tutorial overlay behind USER_INTERFACE.Tutorials = APPLICATION_CONTEXT.tutorials (see TUTORIALS.md).
  • network-status.ts — APPLICATION_CONTEXT.networkStatus, the online/offline source of truth.
  • user-roles-core.ts — roles & capability gating (see USER_ROLES.md).
  • history.ts, user.ts.
• external/ — gone. Vendored third-party assets live in libs/, first-party OSD extensions in classes/osd/ (scalebar, tools.ts, viewport-registration.ts), tile sources in classes/tile-sources/. There is no external group in config.json and no requireExternal() on the server.
• workers/ — standalone web-worker entry points fetched by URL, never bundled (registration-worker.js, used by classes/osd/viewport-registration.ts).
• libs/ — vendored libraries: i18next, OpenSeadragon (openseadragon.js), Tailwind CSS, Monaco, Phosphor Icons (phoshor-icons/), plus flex-renderer/ (WebGL renderer). Do not edit libs/ — upstream-only.
• assets/ — style.css, icons, and other static assets.
• types/ — ambient TypeScript declarations (app.d.ts, config.d.ts, globals.d.ts, slide-protocols.d.ts, io.d.ts).

OpenSeadragon (v6+) is bundled under src/libs/openseadragon.js and configured via openSeadragonPrefix / openSeadragon in src/config.json. To run a debug build, point those values at an unminified copy.

../plugins/, ../modules/

User-facing features and shared libraries respectively; both are dynamically loadable via the loader. See their READMEs.

Available API

Make sure you've read ../INTEGRATION.md first.

Globals

Established by src/app.ts and src/loader.ts. These are the supported, ambiently-typed entrypoints:

Global	Where it's set	Purpose
window.APPLICATION_CONTEXT	src/app.ts (createApplicationContext)	Session, config accessors, open pipeline.
window.VIEWER_MANAGER	src/app.ts (new ViewerManager)	Manager for all OSD viewer instances (single- and multi-view).
window.USER_INTERFACE	UI layer	Core generic UI operations (notifications, menus).
window.UTILITIES	UI / inspector controllers	System utilities (inspector toggles, serializers).
window.HttpClient	src/classes/http-client.ts	Auth-aware HTTP client (proxy, JWT, CSRF).
window.SESSION	src/app.ts — currently not instantiated	Live-collaboration SessionSyncController. The feature is parked (see below); the global stays undefined, so always call it as window.SESSION?.….
window.IO_PIPELINE	bootstrapIOPipeline() in src/app.ts	Save/load pipeline; also reachable as APPLICATION_CONTEXT.io.
window.SLIDE_PROTOCOLS	bootstrapSlideProtocols() in src/app.ts	Slide-protocol registry (src/classes/slide-protocols.ts).
window.xmodules	src/loader.ts	Object store of module exports. Use the helpers below — don't reach in directly.
plugin(id)	src/loader.ts	Returns the plugin instance.
singletonModule(id)	src/loader.ts	Returns (and lazily instantiates) the module singleton.
viewerSingletonModule(className, viewerLike)	src/loader.ts	Returns a per-viewer XOpatViewerSingleton.

window.VIEWER is not a stable handle — it tracks whichever viewer most recently took focus, which is the wrong instance whenever multi-view is active. Resolve the right viewer with VIEWER_MANAGER.get(...), with viewerSingletonModule(...), or from e.eventSource on broadcast events. Likewise, do not store long-lived TiledImage references unless you own them, and prefer VIEWER_MANAGER events over reaching for the focused viewer. When you only need to retarget one viewer, use updateViewerSelection(...) instead of rebuilding the whole session. See MULTI_VIEWPORTS.md.

Viewer Open API

xOpat treats viewer opening as an explicit transaction rather than a loose mix of config mutation and OpenSeadragon world edits. The runtime opening pipeline is class-based and lives under src/classes/app/ — viewer rebinding, visualization runtime checks, synthetic-open handling, inspector integration, and session lifecycle all stay there, and src/app.ts is intentionally reduced to bootstrap/composition. The public entrypoints exposed to plugins/modules remain global through window.APPLICATION_CONTEXT.

• APPLICATION_CONTEXT.openViewerWith(data?, background?, visualizations?, bgSpec?, vizSpec?, opts?)
  • Main transaction entrypoint.
  • Can replace or merge session data / background.
  • Can create additional viewers when multiple backgrounds are targeted.
  • vizSpec arrays may contain explicit undefined entries to mean "no visualization for this viewer"; omitted vizSpec still means "keep the current selection".
  • Rebinds navigator title, scalebar reference, measurements, visualization menu, history, and synthetic open events.
• APPLICATION_CONTEXT.updateViewerSelection(viewerIndex, { backgroundIndex?, visualizationIndex? }, opts?)
  • Use when one existing viewer should switch background and/or visualization without rebuilding unrelated viewers.
  • Passing visualizationIndex: null clears the active visualization for that viewer.
  • Delegates to the same open pipeline, keeping history/session synchronization consistent.
• APPLICATION_CONTEXT.replaceVisualizations(visualizations, newData?, activeVizIndex?)
  • Replaces the session visualization list while preserving the rest of the session.
  • Preferred over the older updateVisualization(...) name.

Options are ambiently typed as ViewerOpenOptions and per-viewer patches as ViewerSelectionPatch, so plugins/modules use them without importing from core.

Canonical Scene

src/classes/app/canonical-scene.ts is the single round-trip pair for full session state, exposed publicly as APPLICATION_CONTEXT.scene (typed XOpatSceneApi, so plugins/modules use it without importing from core). Use it whenever you need to capture what is currently rendered and replay it later — playground Apply, session sync's heavy-apply path, scripting export/import, questionnaire page scenes, and draft persistence all go through it. Full-state snapshot/restore must go through this API — never hand-roll config clones. openViewerWith stays the apply primitive for targeted switches (e.g. slide-switcher changing one background).

• scene.serialize(opts?) — captures cfg (data, background, visualizations, active indices) and merges per-shader runtime cache/state from every viewer's renderer back into the structural shader entries. { includeViewport: true } additionally records per-viewer viewers[] overlays ({ uniqueId, viewport }). Returns a CanonicalScene JSON object.
• scene.serializeFromViewer(viewer, init, live?) — single-viewer slice, used by the playground page (passes its namespace-stripped live so renderer ids match the structural ids).
• scene.deserialize(scene, opts) — calls APPLICATION_CONTEXT.openViewerWith(...) with the canonical cfg shape and forwards historyMode / historyLabel. The pipeline rebuilds renderers from the inlined cache — no second per-layer apply pass needed. When the scene carries viewers[] overlays, per-viewer viewports are restored after the open (matched by uniqueId, slot order as fallback).
• scene.snapshotViewport(viewer) / scene.applyViewport(viewer, viewport, animate?) — the blessed per-viewer viewport get/set in the ViewportSetup shape ({ zoomLevel, point, rotation }, same as params.viewport). Consumers with their own wire formats (session sync, recorder) adapt from these instead of reading OSD directly.
• backgroundShaderRendererIds(bg) / visualizationShaderRendererIds(viz) — single source of truth for renderer-id derivation. Bg shader ids follow bgRef.id for index 0 and ${bgRef.id}-N for subsequent entries (mirrors assemble-render-output.ts); viz shader ids are the structural map keys.

Devtools handle: window.__SCENE mirrors APPLICATION_CONTEXT.scene (plus the renderer-id helpers). Inspect the round-trip from the console — e.g. await __SCENE.deserialize(__SCENE.serialize(), { historyMode: "skip" }) should be a visual no-op.

Implicit identity rule. When cfg.background[i].shaders is unset, the renderer synthesizes an identity shader at bg.id. If a tool edits that implicit shader, the canonical-scene serializer materializes it as [{ type: "identity", cache: {…} }] so the change persists across reopens.

Session Restore and Lifecycle

Session bootstrap and restore live in ApplicationLifecycleController.

• Startup restores the last successful session from browser storage when no explicit POST/hash/query session is provided (see Configuration above).
• beginApplicationLifecycle(...) loads required plugins, initializes layers, raises before-app-init, and then opens the requested viewer state.
• Inspector registration is centralized in ViewerInspectorController (no longer mixed into app.ts).

IO Pipeline

window.IO_PIPELINE (also APPLICATION_CONTEXT.io) decouples what modules want to save/load from where it goes. Modules declare capabilities in their include.json (io.capabilities); admin config (ENV.client.io.bindings) binds those to concrete sinks. Plugin authors typically:

• Register bundle-level hooks via this.initIO({ bundleScope, exportBundle, importBundle }).
• Define per-element CRUD resources via this.defineResource({ name, validate, serialize, deserialize }).

The pipeline queues sink dispatch per-resource, supports coalescing, and persists its outbox to IndexedDB. Bundle sinks include file-download, file-upload, post-data, http-rest. See IO_PIPELINE.md for the full design.

Session / Collaboration

Parked. The WebRTC transport is not finished, so src/app.ts leaves window.SESSION uninstantiated (undefined) — the new SessionSyncController() line is commented out, and the companion plugins/session-controls/ UI plugin is not shipped in this tree. Every call site uses window.SESSION?.…, so it is a safe no-op today. The provider contract below is unchanged and the implementation stays under src/classes/session/ — see the TODO at the top of SESSION.md.

window.SESSION is a SessionSyncController singleton enabling real-time peer-to-peer collaboration. Plugins/modules participate by calling window.SESSION?.registerProvider({ id, scope, snapshot, applySnapshot, subscribe, applyDelta }). The sessionCompatible flag in include.json declares participation: "provider" = actively syncs, true = safe but non-syncing, false = incompatible (undeclared plugins trigger a warnings modal). Hosts provision guest URLs via UTILITIES.serializeApp(...) so guests load the host's exact plugin set. Read meta.role in post-event handlers to avoid duplicate side effects on guests. See SESSION.md.

HttpClient

Never use native fetch or XMLHttpRequest for upstream calls — HttpClient (src/classes/http-client.ts) integrates with xOpatUser and injects JWT, CSRF, and proxy paths automatically. See HTTP_CLIENT.md.

const client = new HttpClient({
  proxy: "cerit",           // alias defined in server proxies
  baseURL: "/api/v1",
  // Omit `types` — resolved from the context. `required: true` also makes the
  // client wait for that context to finish authenticating before a request it
  // has no credential for (see AUTH.md "Waiting for a context to settle").
  auth: { contextId: "core", required: true },
  timeoutMs: 30000,         // optional, default 30s
  maxRetries: 3,            // optional, default 3
});

const response = await client.request("data", {
  method: "POST",
  body: { object: "goes here" },
  expect: "json",           // "json" | "text" | "auto"
  // query: { foo: "bar" },
});

Inspector Utilities

Ambiently typed, part of the supported runtime surface:

• UTILITIES.toggleVisualizationInspector(enabled?)
• UTILITIES.setVisualizationInspectorRadius(radiusPx)
• UTILITIES.adjustVisualizationInspectorRadius(deltaPx)
• UTILITIES.setVisualizationInspectorMode(mode)
• UTILITIES.toggleValueInspector(enabled?)

The user-facing controls are registered by ViewerInspectorController into the app-bar Tools category (USER_INTERFACE.AppBar.Tools), not the Edit menu.

Interaction State (shaders that read the pointer)

Shader layers such as fisheye-lens sample screen-space pointer state through the fr_interaction_* GLSL helpers, fed from FlexRenderer.setInteractionState(...).

Observing the pointer is FlexDrawer's own job: it binds its listeners to the viewer container, an ancestor of the Fabric annotation overlay (canvas.upper-canvas), so events arrive by bubbling regardless of what is stacked on top, and it maintains the whole state — position in framebuffer pixels, buttons, drag/click serials. ViewerInteractionController (classes/app/viewer-interaction-controller.ts) only decides when that forwarding is worth paying for, through drawer.setInteractionOptions({enabled, viewerInputCaptureMode}).

The decision is per viewer, re-evaluated after each program build and on every visualization-change: forwarding goes on when a visible layer's class declares static requiresInteraction() === true. The read is tolerant, so a shader registered by a plugin needs no registration here, and a class from an older library build reads as "no". Each changed pointer position costs one forceRedraw; nothing polls.

A layer may still gate on a held mouse button (fisheye-lens defaults to the secondary button; buttonMask: -1, "None (hover)", needs none), and a button held on the canvas is what OpenSeadragon turns into a pan. Hence the core.view.interactionLens hold shortcut (default L): while held, the drawer switches to viewerInputCaptureMode: "drag", which suspends drag/click/flick gestures — wheel zoom keeps working — and restores them on release. Outside the hold, OSD input is untouched.

Policy lives in setup.flexInteractionForwarding ("auto" default | "always" | "never"), with UTILITIES.setInteractionForwarding(mode) as the runtime setter. Isolated/playground viewers (classes/app/setup-isolated-viewer.ts) are outside VIEWER_MANAGER and get no forwarding unless the controller's attach(viewer) is called for them explicitly.

Recovering a visualization whose program fails to link

A GLSL-regenerating action (layer visibility, type/blend change, reorder, cache clear) can produce a fragment program that exceeds the device's uniform budget. The renderer keeps the last program that linked and raises shader-program-failed {key, error, source, shaderIds, snapshot}. classes/app/live-config-sync.ts listens: it caches renderer.getVisualizationSnapshot() on every successful second-pass build, and on failure re-applies that snapshot through drawer.overrideConfigureAll(shaders, order), logs to the app.visualization channel, and shows error.shaderProgramFailedRestored. The debounced config write-back is suppressed while recovering, so a configuration that cannot link never reaches APPLICATION_CONTEXT.config or a session export.

Render Debug (dev only)

APPLICATION_CONTEXT.renderDebug (classes/app/render-debug-controller.ts) records what the renderer was asked to draw and what each pass produced — for the on-screen viewport and every off-screen render (region renders, navigator thumbnails, magic wand, raster sampler). With debugMode on, open it from Tools → Diagnostics → Render debug window.

It is inert until that window is open: hooks are instance-wraps installed on activate() and fully restored on deactivate(), so a normal session pays nothing. Captured descriptors carry counts, view geometry, shader stack and tileSourceIds — never tile pixels. Result imagery is opt-in: a thumbnail per captured frame (checkbox), and a per-frame first-pass layer grid (button) that replaces the flex-renderer's own window.open debug popup.

Off-screen drawers are invisible to the panel unless they announce themselves. A new standalone drawer should add one line at its creation site:

APPLICATION_CONTEXT.renderDebug?.registerDrawer?.(drawer, { label: "my-feature", viewer, kind: "offscreen" });

webglDebugMode is unrelated and unchanged — it still enables the raw library diagnostics, including that popup.

Scripting

src/classes/scripting-manager.ts + src/classes/scripting/ is a Worker-based sandbox exposing scripting namespaces (XOpatApplicationScriptApi, XOpatViewerScriptApi, XOpatVisualizationScriptApi) to user/plugin scripts. Use it for advanced automation and LLM integration; not required for typical plugin development.

UI

Use the new UI components — see ../ui/README.md and ../ui/classes/README.md. Extend BaseComponent and rely on Van.js reactivity instead of manual DOM work. The CORE UI singletons (AppBar, FloatingManager, FullscreenMenus, GlobalTooltip, …) are listed in ../ui/services/README.md.

Reuse the existing components before pulling new dependencies. If you need a DaisyUI element that isn't already wrapped, add it under ui/classes/elements so other plugins can reuse it.

Localization

Driven by i18next. Use $.t('translation_key') at runtime; $.i18n holds the instance. Note that $ is xOpat's i18n namespace, not jQuery — it is a plain object and is not callable (src/classes/app/i18n-dom.ts). HTML can carry data-i18n="key" / data-i18n="[title]key", applied by localizeDom() once i18next initialises. Server-side i18n is available with limited capabilities. Spawned child windows inherit the opener's $.

For plugin localization specifics, see the plugins README.

Embedding the viewer in a custom server

The two reference backends are the documentation:

• PHP — server/php/init.php shows the canonical wiring. The helpers in server/php/inc/core.php (require_libs, require_openseadragon, require_core) and server/php/inc/plugins.php (require_modules, require_plugins) are still the building blocks for embedding xOpat into a PHP host. The browser-side entry is initXOpat(PLUGINS, MODULES, ENV, POST_DATA, PLUGINS_FOLDER, MODULES_FOLDER, VERSION, I18NCONFIG?) (src/app.ts).
• Node — server/node/index.js and server/node/README.md cover the modern integration story: session-cookie CSRF, RPC for plugins/modules, dev-mode hot reload via server/utils/node/dev-mode.js.

Further reading

• Lifecycle events: EVENTS.md
• Auth contexts / brokers / boot barrier: AUTH.md
• HTTP / proxies / token verifiers: HTTP_CLIENT.md
• IO pipeline (save/load): IO_PIPELINE.md, viewer-side storage overview: STORAGE.md
• Roles & capabilities: USER_ROLES.md
• Keyboard shortcuts / keymap: SHORTCUTS.md
• Scripting sandbox: SCRIPTING.md
• Live collaboration (parked): SESSION.md
• Multi-viewport rules: MULTI_VIEWPORTS.md
• Virtual viewport splits: VIRTUAL_VIEWPORTS_SPLIT.md
• Focal-plane z-stacks: ZSTACK.md
• Backends & deployment: ../server/README.md
• NPM-packaged modules/plugins: NPM_MODULES_PLUGINS.md
• Plugin development: ../plugins/README.md
• Module development: ../modules/README.md
• UI design system: ../ui/README.md, ../ui/classes/README.md, ../ui/services/README.md