Bring-Your-Own-Key AI Chat

xOpat ships an AI assistant that can see the current slide and drive the viewer — describe tissue, jump to a region, list annotations — through the safe scripting API. It runs in one of two key modes:

Managed — the deployment holds the provider API key server-side; users just chat.
BYOK (bring your own key) — the deployment registers the provider with no key, and each user supplies their own from the settings panel. The key is stored locally in the browser and sent only to the model provider through the viewer's proxy — never baked into the session or seen by other users.

The demo below runs in BYOK mode: it opens a slide, enables the chat plugin, and launches a short guided tour showing where to paste your key.

Requires a BYOK provider on the demo server

The session enables the chat plugin, but the provider list comes from the server. This demo assumes the deployment registers a chat provider with an empty apiKey (and requiresLogin: false). If the remote viewer has not been updated with such a provider yet, the chat tab still opens but shows no model to pick.

Loading demo…

Open the demo in a new tab ↗

View session configuration

xOpat session config
{
  "params": {
    "bypassCache": true,
    "notificationsPosition": "top"
  },
  "data": [
    "Projects/demo/summer-school-coolab/slide.tiff"
  ],
  "background": [
    {
      "dataReference": 0,
      "visualizationIndex": 0
    }
  ],
  "plugins": {
    "chat-openai-compatible": {},
    "extra-tutorials": {
      "data": [
        {
          "title": "BYOK AI chat: add your key and ask",
          "attach": true,
          "runDelay": 700,
          "confirm": {
            "title": "Bring your own key",
            "message": "This demo runs the AI assistant in <b>BYOK</b> mode — no key ships with it. In under a minute we will show you where to paste your own provider key (it stays in <b>your browser</b>) and how to ask the assistant about the slide.",
            "acceptLabel": "Show me",
            "declineLabel": "Skip",
            "illustrationIcon": "ph-key"
          },
          "content": [
            {
              "next #osd-0": "This is the slide the assistant can inspect — an H&amp;E whole-slide image. Pan with drag, zoom with the wheel."
            },
            {
              "click #viewer-container-menu-b-chat": "Open the <b>AI chat</b> tab in the right-side dock."
            },
            {
              "next #viewer-container-menu-c-chat": "The assistant panel. Pick a <b>provider</b> and <b>model</b> at the top. Because this is BYOK, a provider that needs a key shows a <b>“key required”</b> notice with an <b>Add key</b> link — click it, or use the Plugins menu in the next step."
            },
            {
              "click #visual-menu-b-plugins": "You can also reach it here: open <b>Plugins</b> and choose <b>Chat provider keys</b>. Paste your key and save — it is stored locally in this browser and sent only to the model provider through the viewer proxy, never saved into the session."
            },
            {
              "next #viewer-container-menu-c-chat": "Once a key is stored the input unlocks. Try <i>“Describe what you see on this slide”</i> or ask it to zoom to a region — the assistant can drive the viewer through the scripting API."
            }
          ]
        }
      ]
    }
  }
}

Add your key — step by step

Open the chat. Click the AI chat tab in the right-side dock.
Pick a provider and model at the top of the panel. A BYOK provider shows a “key required” notice with an Add key link.
Paste your key. Use that link, or open Plugins ▸ Chat provider keys from the top app bar. Enter your key and save.
Chat. The input unlocks once a key is stored. Ask “Describe what you see on this slide”, or tell it to zoom to a region — it drives the viewer for you.

Your key never leaves the browser except as an upstream call to the model provider through the viewer proxy. Clearing it (same panel) removes it immediately.

Enabling it in your own session

BYOK is a server decision — you register the provider with an empty key — plus a session that turns the plugin on. The session side is just:

{
  "data": ["path/to/your/slide.tiff"],
  "background": [{ "dataReference": 0 }],
  "plugins": {
    "chat-openai-compatible": {}
  }
}

Expand View session configuration under the viewer above for the full demo JSON (it adds the guided tour via the extra-tutorials plugin). Swap chat-openai-compatible for chat-openai or chat-anthropic to match whichever provider your deployment registers. On the server, leave that provider's apiKey empty to make it BYOK — see the chat plugin READMEs for the providerDefaults block.

For authoring sessions interactively, see the Developer Setup Playground; for shader-focused examples, the Summer School Demos.

Add your key — step by step​

Enabling it in your own session​

Add your key — step by step

Enabling it in your own session