When we sat down to design the system behind our platform, we didn’t start from a blank page. We started from something engineering teams already trust and understand: the CI/CD pipeline. It’s a pattern that’s been battle-tested for over a decade, and it turns out the shape of “distribute work reliably across many machines and report back” fits mobile testing almost perfectly — with one extra wrinkle CI/CD never had to deal with: a physical or virtual phone at the end of the chain.
But there’s another way to picture it that maps even more naturally to what’s actually happening: a human QA team. Once you see it that way, every piece of the architecture has an obvious job.
The orchestrator — the QA lead
At the center sits the orchestrator. This is the backend server that owns the big picture: what needs to be tested, in what order, against which devices, and with what AI models involved in the loop. It doesn’t execute anything itself — it decides, assigns, and tracks.
This is exactly what a QA lead does. They don’t personally install every app on every device and tap through every flow. They decide what needs testing, hand it off to their team, and keep track of what came back.
Runners — the testers on the team
Work assigned by the orchestrator gets picked up by runners. There can be as many of these as you need, running wherever makes sense — on your infrastructure, in the cloud, near your device lab. Each runner’s job is simple in concept: take the task the lead assigned, get it executed, report the result.
This is the individual QA tester — the one who actually installs the build, runs through the assigned test, and reports back what happened. Add more runners, and you’ve added more testers to the team. That’s the whole idea.
Drivers — the hands and eyes
But a tester doesn’t test a phone by staring at it and thinking hard. They pick it up. They tap, swipe, type, and look at what’s on screen. That physical layer — the part that actually touches the device — is the driver: a small, platform-specific piece of software that runs directly on the device itself, one implementation for Android, one for iOS.
The driver is the hands and eyes. A human tester doesn’t care whether they’re holding an iPhone or an Android device — a swipe is a swipe, a tap is a tap, they know how to operate the phone in front of them regardless of who made it. The driver gives our runners that same platform-blind fluency: it exposes one unified interface, and every platform-specific detail — the actual mechanics of how you tap or swipe on that particular OS — is absorbed underneath it and never leaks upward. The runner just says “tap here,” the same way whether “here” is on iOS or Android.
Why runners call home, not the other way around
One deliberate design decision: runners initiate the connection to the orchestrator, not the reverse. This mirrors how modern CI/CD runners work, and it buys three things at once:
- Effortless scaling. Spinning up a new runner is just pointing it at the orchestrator — no inbound network configuration required on our end. Onboarding a new tester, not rewiring the office.
- Lower ongoing maintenance. There’s no fragile inbound routing to keep alive as infrastructure changes over time.
- It plays nicely with firewalls and security policies. Most corporate and cloud environments are locked down against inbound connections by default, but outbound-initiated connections are a well-understood, generally accepted pattern. This lets the platform slot into existing infrastructure without asking security teams to open anything up.
The orchestrator’s other job: talking to the outside world
Beyond coordinating runners, the orchestrator is also the integration point for everything outside the core pipeline. That includes AI providers — whether that’s a hosted model from a provider like Anthropic or OpenAI, or a self-hosted model run internally — as well as any operational or feature integrations the platform needs to support day to day. Keeping this at the orchestrator level means the rest of the system — runners, drivers — stays simple and doesn’t need to know anything about which AI provider or integrations are in play. Back in the words of our metaphor: the lead deals with management and tooling, so the testers can just test.
The dashboard — and the agent sitting next to it
Finally, all of this needs to be visible to a human. That’s the dashboard: a front end that connects to the orchestrator and presents everything happening across runners and devices in a clear, usable way — status, and results, in one place. The dashboard doesn’t talk to runners or devices directly — it goes through the orchestrator, which stays the single source of truth for the whole system.
But not everyone wants to look at a screen to get an answer. Alongside the dashboard, the orchestrator also exposes an MCP interface — so instead of opening a browser tab, you can point an AI agent directly at your testing data. Ask it what failed overnight, ask it to summarize this week’s flaky tests, or wire it into your own tooling. You can even point a coding agent at it directly — have it read the failures and go fix the underlying issue in your codebase, closing the loop from “test failed” to “bug fixed” without a human in between. Same underlying data as the dashboard, just a different way in — one built for humans, one built for agents.
Why this shape, and not something custom
None of these pieces individually are exotic. Orchestrator/runner patterns are well understood, and mobile device automation frameworks like Maestro have already proven that a driver-based approach to device control works well in practice. What we’ve done is combine those two proven shapes into one system built specifically for AI-driven test execution at scale — rather than inventing a new architecture from scratch for problems that already have good answers.
Next up, we’ll get into the actual technology choices behind each of these pieces — and why we picked what we picked.
If you’re wondering why we’re building any of this in the first place, our last post covers the bottleneck we’re actually trying to solve.