Introduction

Welcome to The Pilatus Book!

Pilatus is a modular, extendable application framework designed for industrial and computer vision projects. While the core framework is not limited to engineering applications, it is particularly well-suited for projects with autonomous configurable subsystems that interact with one another.

What is Pilatus?

Pilatus provides a robust foundation for building complex, asynchronous applications with the following key features:

Asynchronous Devices: Run asynchronous actors that communicate via message passing
Recipe Management: Switch between different sets of device configurations without restarting the application
Extensibility: Add new extensions without requiring changes to the core framework
API Integration: Easily expose core functionality through various APIs (HTTP, WebSockets, MQTT etc.)
Cross-Platform: Support for multiple platforms (Linux, macOS, Windows) and architectures (x64, ARM)

Who is this book for?

This book is intended for developers who want to:

Build industrial automation or computer vision applications
Avoid wasting time to write boilderplate code
Use existing code for common tasks and easily create reusable internal extensions
Build upon existing code for common tasks and easily package them as reusable internal extensions

Many developers find that proprietary frameworks from individual companies fail to meet their specific requirements and result in vendor lock-in. Pilatus addresses these concerns by being open source under the MPL2 license, which permits proprietary usage with your custom proprietary extensions. The framework provides a growing number of flexible tools that may align with your needs. When they do not, only a small set of flexible core concepts are required by default. Official extensions, such as the Axum webserver, are built on the same APIs that are available to your own extensions.

How to use this book

This book assumes you have a basic understanding of:

Rust programming language
Asynchronous programming concepts

If you're new to Rust, we recommend working through The Rust Programming Language Book first.

Let's get started!

Getting Started

In this chapter, we are developing a pilatus application with a custom image capturing device.

Installation

Prerequisites

Pilatus has almost no system dependencies on purpose, so it stays lightweight and easy to use. You only need:

Rust toolchain (stable version recommended)
Cargo (comes with Rust)
Openssl installation

Extensions with system dependencies for e.g. Opencv or aravis camera are kept in a separate repository.

Your First Application

Let's create a simple Pilatus application to get familiar with the framework.

Creating a New Project

First, create a new Rust project:

cargo new my-pilatus-app
cd my-pilatus-app
cargo add pilatus-rt --git https://github.com/mineichen/pilatus.git
cargo add pilatus-axum-rt --git https://github.com/mineichen/pilatus.git

A Simple Example

Here's a minimal example to get you started, which includes the axum webserver:

fn main() {
    pilatus_rt::Runtime::default()
        .register(pilatus_axum_rt::register)
        .run();
}

Run the app

cargo run

Troubleshooting: Port 80 requires elevated privileges

ERROR: Hosted service 'Main Webserver' failed: Cannot open TCP-Connection for webserver. Is pilatus running already?

By default, Pilatus uses port 80 for the web interface. On some systems, this requires running with elevated privileges (sudo/root). To use a different port, create a data/config.json with the following content
{
  "web": {
    "socket": "0.0.0.0:8080"
  }
}

Test the app

curl http://localhost:80/api/time

This should show you the time on the system where the server is running. Congratulation, you just run your first pilatus app.

Next Steps

Now that you have a basic setup, we can add your first extension

Your First Extension

Create a new project

Pilatus encourages you, to split your code into multiple separate creates. So we are gonna do that here:

cargo init --lib my-pilatus-extension-rt
cargo add pilatus-axum --git https://github.com/mineichen/pilatus.git
cargo add minfac

Create a extension

Pilatus heavily relies on the minfac inversion of control library. It allows making concepts to other code available. We are gonna add a new http route to the my-pilatus-app. To do so, we change the lib.rs to:

#![allow(unused)]
fn main() {
use pilatus_axum::{IntoResponse, ServiceCollectionExtensions};

pub extern "C" fn register(collection: &mut minfac::ServiceCollection) {
    collection.register_web("my-first-extension", |r| {
        r.http("/greet", |m| m.get(get_response))
    });
}

async fn get_response() -> impl IntoResponse {
    "Hello, World!"
}
}

Register the extension in our app

Go to my-pilatus-app project and add this library to the dependencies

cd ../my-pilatus-app
cargo add my-pilatus-extension-rt --path ../my-pilatus-extension-rt

To use your new extension, you just add a my_pilatus_extension_rt to your main.rs. It should now look something like this:

fn main() {
    pilatus_rt::Runtime::default()
        .register(pilatus_axum_rt::register)
        .register(my_pilatus_extension_rt::register)
        .run();
}

The ordering of register-calls is not relevant here. Ordering is only relevant, if your extension would overwrite core services, which is a very advanced topic.

Test your app

curl http://localhost:80/api/my-first-extension/greet

This should output hello world in your terminal.

How this works

Pilatus extensions just register data into the minfac::ServiceCollection (e.g. a struct, TraitObject or even a primitive). In this caseregister_web builds a specific struture with the routing information and registers it to minfac. When the webserver is booting up, it gathers all registered instances of this type and provides them with a axum webserver.

Next Steps

Congratullations, you just wrote your first pilatus extension. If you want a deeper understanding on how the extension system works, take a look at minfac. If you are comfortable to accept, that this feels a little magic right now, I encourage you to just continue. In the next chapter, where we build our first device, you should feel much more comfortable.

Your First Device

Add new dependencies to your library project

cargo add pilatus --git https://github.com/mineichen/pilatus.git --features tokio
cargo add serde --features derive
cargo add anyhow
cargo add tracing

Create camera submodule

We are now finally getting to implement the custom camera. To do so, we'll add a new module camera.rs in our extension library.

use minfac::ServiceCollection;

pub(super) fn register_services(c: &mut ServiceCollection) {
    // TODO: Implement the camera service
}

Add the mod to your lib.rs and call the regsiter_services from within the register function:

pub extern "C" fn register(collection: &mut minfac::ServiceCollection) {
    // ... Existing code
    camera::register_services(collection);
}

Now run cargo run in your app to make sure, everything still works fine.

Register the device

It is now time to create and register a device.

#![allow(unused)]
fn main() {
use minfac::{Registered, ServiceCollection};
use pilatus::{
    UpdateParamsMessageError,
    device::{ActorSystem, DeviceContext, DeviceValidationContext, ServiceBuilderExtensions},
};
use serde::{Deserialize, Serialize};

pub(super) fn register_services(c: &mut ServiceCollection) {
    c.with::<Registered<ActorSystem>>()
        .register_device("my_camera", validator, device);
}

#[derive(Debug, Deserialize, Serialize)]
#[serde(deny_unknown_fields)]
struct Params {
    url: String,
}

async fn validator(ctx: DeviceValidationContext<'_>) -> Result<Params, UpdateParamsMessageError> {
    ctx.params_as::<Params>()
}

async fn device(
    ctx: DeviceContext,
    params: Params,
    actor_system: ActorSystem,
) -> anyhow::Result<()> {
    //actor_system.register(ctx.id).execute(params).await;
    tracing::info!("Start Camera device with params: {:?}", params);
    actor_system.register(ctx.id).execute(params).await;
    tracing::info!("Camera is shutting down");
    Ok(())
}

}

While this compiles already, it's never doing anything yet. Devices are only run, if they are part of a so called recipe. By default, there is only one recipe without any devices at all. In practice, you can add devices to existing recipes via api calls or you can overwrite the default recipe, so it contains the devices your specific app needs. A Recipe can also contain multiple Instances of the same Device (e.g. 2 Cameras). To dig deeper, have a look at Recipes.

Run the device

By default, pilatus writes all of it's files into the data directory. If this folder doesn't exist, it will be created when starting the executable for the first time. During development, its very convenient to have it in the app folder, but in practice, the app is often installed at a different location, where the process itself doesn't have write access (e.g. Programs ond windows). The location might even be installation specific (User/System installation). The data folder should usually not be checked into git and therefore be part of your .gitignore file.

To add a device, we are going to edit the my-pilatus-app/data/recipes/recipes.json file. Your file should look roughly the same, but devices was still empty.

{
  "active_id": "default",
  "active_backup": {
    "created": "2025-11-05T12:27:15.645499788Z",
    "tags": [],
    "devices": {}
  },
  "all": {
    "default": {
      "created": "2025-11-05T12:27:15.645499788Z",
      "tags": [],
      "devices": {
        "11111111-1111-1111-1111-111111111111": {
          "device_type": "my_camera",
          "device_name": "BirdCamera",
          "params": {
            "url": "https://media.istockphoto.com/id/539648544/photo/eastern-bluebird-sialia-sialis-male-bird-in-flight.webp?b=1&s=612x612&w=0&k=20&c=BcPh4xbjrDVTyiErKB8RZFQ3quuME-4vDSnZRu09xCQ="
          }
        },
        "22222222-2222-2222-2222-222222222222": {
          "device_type": "my_camera",
          "device_name": "FoxCamera",
          "params": {
            "url": "https://images.unsplash.com/photo-1474511320723-9a56873867b5?ixlib=rb-4.1.0&ixid=M3wxMjA3fDB8MHxwaG90by1wYWdlfHx8fGVufDB8fHx8fA%3D%3D&auto=format&fit=crop&q=80&w=1172"
          }
        }
      }
    }
  },
  "variables": {}
}

If you run the app again, you should see two tracing infos in the logs indicating, that both are running. Nice, so we are getting close.

Handling messages

The device currently only waits on execute to shut itself down. So we cannot interact with the device yet.

To do so, we are going to create a ActorMessage. ActorMessages are the protocol, which allows interfaces (e.g. Web-route) or other Actors to communicate with one other. As a simple example, we're going to add a GetImageUri message, which only makes sense for cameras, which load images from e.g. file or http.

#![allow(unused)]
fn main() {
struct GetImageUrlMessage;
impl ActorMessage for GetImageUrlMessage {
    type Output = pilatus_axum::http::Uri;
    type Error = anyhow::Error;
}

impl Params {
    async fn handle_image_url(
        &mut self,
        _msg: GetImageUrlMessage,
    ) -> ActorResult<GetImageUrlMessage> {
        Uri::from_str(&self.url).map_err(ActorError::custom)
    }
}

}

Next, we need to tell our device to handle such messages, if someone asks them. This is just one more registration inside the device function. Just replace

#![allow(unused)]
fn main() {
actor_system.register(ctx.id).execute(params).await;

}

with

#![allow(unused)]
fn main() {
 actor_system
    .register(ctx.id)
    .add_handler(Params::handle_image_url)
    .execute(params)
    .await;

}

The actor system makes sure, only one message handler runs at a time and grants mutable access to the device state while the message is beign processed. Notice, that you don't have to deal with mutexes, as this is handled in the ActorSystem.

At this point, we could call this device from another device, but we don't yet have another device. So we are creating a http route to do so.

Add a Http-Route

We are going to expose another http endpoint, which is calling the ActorSystem with the GetImageUrl Message and returns that response via HTTP.

#![allow(unused)]

fn main() {
pub(super) fn register_services(c: &mut ServiceCollection) {
    // ... Device registration
    c.register_web("my_camera", |r| r.http("/image_url", |m| m.get(get_image_url_web)));
}

async fn get_image_url_web(
    InjectRegistered(actor_system): InjectRegistered<ActorSystem>,
    Query(id): Query<DynamicIdentifier>,
) -> impl IntoResponse {
    DeviceResponse::from(
        actor_system
            .ask(id, GetImageUrlMessage)
            .await
            .map(|url| url.to_string()),
    )
}

}

Test the message handler via HTTP

curl http://localhost:8080/api/my_camera/image_url?device_id=11111111-1111-1111-1111-111111111111
curl http://localhost:8080/api/my_camera/image_url?device_id=22222222-2222-2222-2222-222222222222

You shoud see the image urls as output.

Summary

Here is the full code of the camera example

#![allow(unused)]
fn main() {
use std::str::FromStr;

use minfac::{Registered, ServiceCollection};
use pilatus::{
    UpdateParamsMessageError,
    device::{
        ActorError, ActorMessage, ActorResult, ActorSystem, DeviceContext, DeviceValidationContext,
        DynamicIdentifier, ServiceBuilderExtensions,
    },
};
use pilatus_axum::{
    DeviceResponse, IntoResponse, ServiceCollectionExtensions,
    extract::{InjectRegistered, Query},
    http::Uri,
};
use serde::{Deserialize, Serialize};

pub(super) fn register_services(c: &mut ServiceCollection) {
    c.with::<Registered<ActorSystem>>()
        .register_device("my_camera", validator, device);
    c.register_web("my_camera", |r| {
        r.http("/image_url", |m| m.get(get_image_url_web))
    });
}

#[derive(Debug, Deserialize, Serialize)]
#[serde(deny_unknown_fields)]
struct Params {
    url: String,
}

async fn validator(ctx: DeviceValidationContext<'_>) -> Result<Params, UpdateParamsMessageError> {
    ctx.params_as::<Params>()
}

async fn device(
    ctx: DeviceContext,
    params: Params,
    actor_system: ActorSystem,
) -> anyhow::Result<()> {
    tracing::info!("Start Camera device with params: {:?}", params);
    actor_system
        .register(ctx.id)
        .add_handler(Params::handle_image_url)
        .execute(params)
        .await;
    tracing::info!("Camera is shutting down");
    Ok(())
}

struct GetImageUrlMessage;
impl ActorMessage for GetImageUrlMessage {
    type Output = pilatus_axum::http::Uri;
    type Error = anyhow::Error;
}

impl Params {
    async fn handle_image_url(
        &mut self,
        _msg: GetImageUrlMessage,
    ) -> ActorResult<GetImageUrlMessage> {
        Uri::from_str(&self.url).map_err(ActorError::custom)
    }
}

async fn get_image_url_web(
    InjectRegistered(actor_system): InjectRegistered<ActorSystem>,
    Query(id): Query<DynamicIdentifier>,
) -> impl IntoResponse {
    DeviceResponse::from(
        actor_system
            .ask(id, GetImageUrlMessage)
            .await
            .map(|url| url.to_string()),
    )
}
}

Next steps

Congratullations! You implemented your first device and added a external interface to it. I'd recommend you to go to the Core concepts section to find out more about devices and recipes.

Structure

Core projects are devided into two cargo projects. The {project-name} and {project-name}-rt.

rt-projects

Projects with the form {name}-rt mustn't be used as dependencies, except for executables and tests. All code which is required for interfacing lives in the {name} project. The projects without suffix should only have minimal dependencies, so it can be used in other projects as e.g. pilatus-leptos.

If no unstable feature is set on the *-rt crate, it usually only provides a register method, which registers all of its dependencies to minfac

Unstable features

Many pilatus projects have a unstable feature. Code which is available behind these feature flags is not guaranteed to stay stable. They are used to:

Make code available for testing.
- It's ok if tests of your crate break due to breaking changes in e.g. pilatus-rt/unstable. If unstable is only in [dev-dependencies] and not [dependencies], your crate remains stable for others to be used.
Allow for tightly bound crates.
- The GUI will be tightly coupled to a specific backend version internally. But their interface to dependents is just a stable leptos-component.
- {name}-rt projects may depend on their {name} counterpart with active unstable feature. Unstable can e.g. make {name}::DeviceParams available, which is a shining example of a unstable, ever evolving structures you shouldn't depend on in external code. But the UI and the device implementation in your {name}-rt project are allowed to rely on it.

Why do we even have `-rt` projects?

The biggest reason for having *-rt projects is compilation time. RT projects often have significant dependencies. pilatus-axum-rt contains heavy dependencies like tokio, async-zip, image, tower etc. If all this code lived in pilatus-axum, depending crates could only start compiling, when pilatus-axum-rt and all of its dependencies are ready, leading to long dependency chains. For crates on which noone depends on, you could also add a rt feature flag to your {name} project instead. This allows:

fine grained visibility (e.g. Params-fields are only visible in a submodule)
avoiding Orphan-Rule problems

In the open source pilatus project, we don't do that, because the UI is always created in a different repo, so it doesn't make sense. The UI is external, so it can easily be published as MIT without any confusions and because we really want to be able to write a new UI with the same conveniences like the official one used. Furthermore, experiments showed, that most code suddenly was behind a cfg(feature = "rt") flag, which seemed off.

Ignore executables

In projects it often makes sense, not to add your executable to your workspace default-members. If you do, each change of your code triggers a rebuild of the executable, which is very slow to link on e.g. Windows.

Maintain two Compilation modes

This all leads to pilatus having two dependency modes: default and integration

	Default	Integration
Usage	Building, Unittesting	Integration tests and executables
Compilation speed	Faster - excludes heavy runtime dependencies	Slower - includes all runtime dependencies
Invocations	`cargo build`, `cargo test`	`cargo test --test integration`, `cargo run --bin executable`

To maintain these characteristics, follow these guidelines

Don't add rt-deps to your [dev-dependencies], as optional dev-deps are not suported by cargo. Add them optional to [features] and enbable it with the integration feature
RT-Crates are allowed to add unstable feature to their non-rt counterpart
Add rules for your editor, which enables your "integration" feature for rust-analyzer (see this project for zed and vscode)
Check, that unstable is not used in your project, by cargo check ing each project separately. If you run it on the workspace, tests or the executable might enable unstable features

Leptos UI

The frontend code is deliberately developed under the MIT License in a separate repository. You are free to use it as a template for your own frontend and modify it without the need to publish your changes.

The backend is licensed under the MPL-2.0. You may modify it for your own use, but any changes to the backend itself must be shared under the same license, promoting open collaboration and shared progress. The MPL-2.0 is a file-level copyleft license, which means you can combine the backend with proprietary code or extensions without affecting the license of your own proprietary components.

Core Concepts

Recipes

Recipes in Pilatus allow you to define and switch between different configurations of devices without restarting the application.

What are Recipes?

Recipes are complete configurations that define:

Which actors should be active
Metadata like DeviceName, CreationDate, Backup of the active recipe
Device configurations
System parameters

Recipe Management

The recipe system enables:

Hot-swapping configurations
Managing multiple recipe sets
Importing and exporting recipes
Validating configurations

Device System

The device system in Pilatus provides a structured way to manage and interact with hardware and software components.

What are Devices?

Devices represent configurable entities in your system that can:

Send and receive messages
Maintain state
Be started and stopped dynamically

todo!()

Keyboard shortcuts

The Pilatus Book