Bitfocus Companion sits at the centre of a lot of production rigs. It turns a Stream Deck or another control surface into buttons that drive vision mixers, audio desks, playback and streaming encoders. Because Companion has an HTTP API, it is now possible to connect an AI assistant to it and ask in plain language for the state of a connection, or for a button to be pressed.
That is useful, and it is also a new way for a show to go wrong. This article sets out the controls we think an AI connection to a control surface needs, using the design of our own open-source Companion MCP server as the worked example.
Why a live control surface is different
Most advice about AI tools assumes a mistake can be corrected. A draft can be rewritten and a wrong summary can be checked. A button on a show surface is different in three ways:
- Actions are immediate. Stopping a stream, cutting to the wrong source or restarting a connection happens in front of an audience.
- Many actions are not idempotent. Pressing a toggle twice is not the same as pressing it once. A retry can undo the action it was meant to complete.
- The operator is busy. Nobody on a show call has spare attention to supervise a tool that needs watching.
A connection that is acceptable for drafting emails is not acceptable here. The design has to assume the assistant will sometimes ask for the wrong thing, and make that harmless.
Start read-only
The most valuable uses are often the ones that change nothing. An assistant that can read Companion can answer questions such as which connections are in an error state, or what a custom variable currently holds, without any ability to affect the show.
Companion MCP is built around that default. Its read tools list connections and their status and read custom and module variables. Its write tools do nothing unless an environment setting, COMPANION_ALLOW_WRITES, is explicitly set to true. A fresh install cannot press anything.
If your first deployment never leaves read-only mode, you have still gained something, and taken on very little risk.
An allowlist is the boundary
When you do enable writes, the important control is not the instruction you give the model. It is a list, held outside the model, of exactly what may be touched.
In Companion MCP that is an allowlist file of specific buttons, variables and connections. A request for anything not on the list is refused, whatever the assistant asks and however the request is phrased. The installer creates the file empty, so the starting position is that nothing can be pressed.
This matters because prompts are not a security boundary. A model can misread a request, or be steered by text it encounters. A list the model cannot edit holds regardless.
Dry runs and confirmation
Two further gates sit in front of a real press:
- Dry run by default. Every write reports what it would send and sends nothing, unless the request explicitly passes
dry_run: false. The assistant can show you its intended action before anything happens. - Confirmation for high-risk actions. Buttons you mark as high risk, and every connection or configuration change, also need
confirm: true. A stream stop should never be one ambiguous sentence away.
Each gate refuses independently and says why. The refusal comes back as a plain reason rather than an error trace, so the assistant can tell the operator which setting stopped it.
Never fire twice
Networks fail at awkward moments. If a request to press a button times out, the honest answer is that nobody knows whether it was pressed.
The tempting behaviour, for software and for AI agents alike, is to try again. On a control surface that is the wrong instinct. Companion MCP makes one attempt and, on a timeout, a reset or a server error, reports the outcome as unknown instead of retrying. A person can then look at the surface and decide, which is the correct place for that decision.
Log before acting
An audit log is only useful if it cannot be skipped. Companion MCP writes an attempt line before every real write, and refuses to write at all if the log cannot be written. Each tool call becomes one JSON line recording whether it succeeded, was refused or failed, with variable values redacted.
After a show, that file answers the questions that matter: what the assistant was asked to do, what it actually did, and what it was stopped from doing.
Keep it on the private network
A control API should not be reachable from the internet. The server refuses a public address unless a separate setting allows it, and it opens no network ports of its own, because the AI client launches it locally and talks to it directly.
A sensible way to introduce it
- Install it against a test instance of Companion, not the show machine.
- Run read-only for a rehearsal. Ask it about connection status and variables and see whether the answers are useful.
- Allowlist a few low-consequence buttons, such as a lower-third or a cue variable. Leave dry runs on and read what it proposes.
- Mark anything that ends or interrupts the show as high risk, or leave it off the list altogether.
- Read the audit log after each session, including the refusals. They show what the assistant tried to do.
- Widen the list one item at a time, and only for actions an operator has asked for more than once.
What to keep in human hands
Some actions gain nothing from delegation. Starting and stopping the stream or the recording, anything touching the main programme output, and any change to connection configuration during a show are quicker and safer as a deliberate press by the operator. The assistant is most useful beside the operator, reading state and handling low-risk routine, not in place of them.
The same pattern applies well beyond Companion. If you are connecting an AI assistant to any system that acts in the real world, look for the same properties: read-only by default, an allowlist the model cannot change, a preview before action, confirmation for anything serious, no automatic retries, and a log you can read afterwards.