Introduction
nano-ros is a lightweight ROS 2 client library for embedded real-time systems.
It runs on bare-metal microcontrollers, FreeRTOS, NuttX, ThreadX, and Zephyr,
as well as Linux and *BSD. The entire core stack is no_std compatible.
flowchart TB
App["<b>Application</b><br/>Rust / C / C++ node code"]
Core["<b>nano-ros core</b><br/>Executor · Node · pub/sub · services · actions · CDR"]
RMW["<b>RMW backend</b><br/>Zenoh · XRCE-DDS · Cyclone DDS · custom"]
Plat["<b>Platform</b><br/>clock · heap · threads · sleep · sockets · libc"]
Wire(["ROS 2 / DDS / Zenoh wire"])
App --> Core --> RMW --> Plat
RMW -. wire-compatible .-> Wire
Four layers, swappable independently: the same node code runs over any RMW
backend on any platform. Here is a complete Linux publisher — register a
backend, open the executor, publish std_msgs/String (Hello World: N)
on /chatter once a second:
use core::fmt::Write as _;
use nros::prelude::*;
use std_msgs::msg::String as StringMsg;
fn main() {
// Pick the backend at compile time; this one line registers it.
nros_rmw_zenoh::register().unwrap();
let config = ExecutorConfig::new("tcp/127.0.0.1:7447").node_name("talker");
let mut executor: Executor = Executor::open(&config).unwrap();
let mut node = executor.create_node("talker").unwrap();
let publisher = node.create_publisher::<StringMsg>("/chatter").unwrap();
let mut count = 0i32;
executor
.register_timer(nros::TimerDuration::from_millis(1000), move || {
count += 1;
let mut msg = StringMsg::default();
let _ = write!(msg.data, "Hello World: {count}");
publisher.publish(&msg).unwrap();
})
.unwrap();
executor.spin_blocking(SpinOptions::default()).unwrap();
// This is the imperative Layer-2 API; the starters use the
// declarative `nros::node!` / `nros::main!` form — both are
// first-class (see Concepts → Two-Layer API).
}
The same program in C and C++ is in the First Node guides: Rust · C · C++. When a project grows beyond one node, continue with Multi-Node Project Layout.
Key Features
- Minimal stack — three software layers (application, nano-ros, transport). Lean dependency tree, fast compile times.
- Pluggable middleware — choose Zenoh (agent-less, direct peer communication), XRCE-DDS (agent-based), or Cyclone DDS (RTPS wire-compatible with stock ROS 2) at compile time. Same application code regardless of backend.
- The same source runs on every supported target — a node body written for
one platform compiles unchanged on the others; what differs is build
configuration (
Cargo.toml,.cargo/config.toml,CMakeLists.txt), not the code you write. This is asserted, not asserted-about: theexample_portabilitytest normalizes every platform’s copy of each example and fails if any two differ, so the claim is checked on every run rather than maintained by hope. Two execution models are declared exceptions with written reasons — bare-metal deferred dispatch, and Zephyr’s component shape — and the test names them explicitly rather than hiding them. - Rust-first with C API — the core is written in Rust for memory safety and ergonomics, with a thin C FFI (Foreign Function Interface) layer following rclc conventions.
- True
no_std— runs on bare-metal Cortex-M3; theallocandstdfeatures are opt-in. Whether the IMAGE needs an allocator is the backend’s call, not the core’s: XRCE allocates only at entity setup, zenoh-pico wants an allocator (a bump allocator suffices on bare-metal), and Cyclone DDS allocates per message.scripts/rmw-alloc-sites.pyreports every site. - Standalone tooling —
nros generate-rustproduces message bindings without a ROS 2 installation (bundled interface definitions). - Formally verified — 148 Kani bounded model checking harnesses and 83 Verus deductive proofs cover CDR serialization, scheduling, and protocol correctness.
- ROS 2 compatible — interoperates with standard ROS 2 nodes via
rmw_zenoh_cpp, or directly over RTPS withrmw_cyclonedds_cpp(same wire protocol, no key rewriting). Topics, services, and actions work across the boundary.
Quick board check — does it work on the board I have today?
| Vendor / form factor | Chip | RTOS / no-RTOS | Languages | Example in repo | ROS 2 interop |
|---|---|---|---|---|---|
| ARM MPS2-AN385 (QEMU) | Cortex-M3 | FreeRTOS / bare | Rust C C++ ¹ | examples/qemu-arm-{freertos,baremetal}/ | Verified |
| ST STM32F4-Discovery | Cortex-M4F | bare | Rust ² | out-of-tree — worked example | Untested ⁴ |
| Espressif ESP32-C3 | RISC-V (RV32) | ESP-IDF | Rust C C++ | integrations/nano-ros/ — nightly lane, build-only | Ready ⁵ |
| Espressif ESP32-C3 (QEMU) | RISC-V | bare | Rust | examples/qemu-esp32-baremetal/ | Verified |
QEMU virt RISC-V64 | RV64GC | ThreadX | Rust C C++ | examples/qemu-riscv64-threadx/ | Verified |
| Linux host | x86-64 / aarch64 | ThreadX sim | Rust C C++ | examples/threadx-linux/ | Verified |
| QEMU virt / Cortex-A9 | Cortex-A7 / A9 | NuttX / Zephyr | Rust C C++ | examples/qemu-arm-nuttx/, examples/zephyr/ | Verified |
| Pixhawk 4 / 6X | STM32F7 / H7 | NuttX (PX4) | C++ | integrations/px4/module-template/ | Ready ³ |
| Generic Cortex-M0+/M4/M7 | ≥ 64 KB SRAM | RTOS of choice | Rust C C++ | Use your board’s vendor BSP + integrations shells | Pattern shown |
Legend: Verified = booted + tested in CI. Ready = builds but no
in-CI gate yet — where the row names an examples/<plat>/ directory there
is an app to compile and try; where it names an integrations/ shell there
is build glue but no worked reference app yet.
Footnotes — ¹ MPS2-AN385 bare-metal is Rust-only (nros-c / nros-cpp
need an RTOS for libc / heap). ² STM32F4 is reached through the
customization ladder, not an in-tree crate; a FreeRTOS variant sits on the
shared nros-board-freertos glue. ³ PX4 path is via the
external-module template in integrations/px4/ — C++ only because
PX4’s uORB binding is C++-only. ⁴ Untested = the code path exists and the
port is documented, but no lane in this repo boots it — the hardware is not in
the test rack and QEMU models no STM32 MAC. phase-337 W7.a moved the two
STM32F4 board crates out of the tree for exactly that reason; keeping a row
that says “Verified” for a board nothing verifies is the failure mode the tier
registry exists to prevent. ⁵ The ESP-IDF component path is nightly-support
only: built by the nightly full-matrix lane, never by the per-change or
per-platform ones, and its coverage stops at the image building. The esp-hal row above it is the ESP32
path with a booting CI lane.
Supported platforms (by RTOS)
| Platform | RTOS | Network Stack | Targets |
|---|---|---|---|
| POSIX | Linux / *BSD | OS sockets | x86-64, aarch64 |
| Bare-metal | None | smoltcp | Cortex-M3, ESP32-C3 |
| FreeRTOS | FreeRTOS | lwIP | Cortex-M3 (QEMU) |
| NuttX | NuttX | BSD sockets | Cortex-A7 (QEMU), RISC-V32 (QEMU rv-virt) |
| ThreadX | ThreadX | NetX Duo | RISC-V 64 (QEMU), Linux sim |
| Zephyr | Zephyr | Zephyr sockets | Various boards |
posix there is the platform layer, which is genuinely POSIX-clean. The
host board is named linux, and that is the narrower claim: only Linux is
tested, and nros-board-linux::apply_tier_affinity calls sched_setaffinity
with cpu_set_t, which libc does not define for macOS. macOS is not
supported (AGENTS.md, phase-260).
RMW Backends
nano-ros supports several middleware backends, selected at compile time by adding the backend crate as a dependency:
- Zenoh (
nros-rmw-zenoh) — peer-to-peer via zenoh-pico. No agent process. Compatible with ROS 2rmw_zenoh_cpp. - XRCE-DDS (
nros-rmw-xrce-cffi) — agent-based via Micro-XRCE-DDS. Compatible with micro-ROS agent. - Cyclone DDS (
nros-rmw-cyclonedds) — C++ shim; full RTPS wire-compat with stockrmw_cyclonedds_cpp.
Application code is identical regardless of backend — switch with a single Cargo feature flag or Zephyr Kconfig option.
Project Status
nano-ros is under active development. Core capabilities are functional and exercised in CI; see the platform chapters for per-target detail.
| Capability | Status |
|---|---|
| Pub/Sub | Complete |
| Services | Complete |
| Actions | Complete |
| Parameters | Complete |
| ROS 2 interop | Complete |
| Zenoh backend | Complete |
| XRCE-DDS backend | Complete |
| Cyclone DDS backend | Complete (native + embedded; some embedded action paths in progress) |
| Zephyr support | Complete |
| QEMU bare-metal | Complete |
| C API | Complete |
| C++ API | Complete |
| Message codegen | Complete |
How This Book Is Organized
- Getting Started — install toolchains, build your first app, connect to ROS 2.
- Concepts — understand the architecture, feature system, and backend model.
- Guides — step-by-step walkthroughs for message generation, QEMU testing, and ESP32 development.
- Platforms — per-RTOS setup and configuration.
- Reference — API details, environment variables, build commands, and wire protocol.
- Advanced — formal verification, real-time analysis, safety features, and contributing.
Choose Your Entry
Different readers want different paths through this book. Pick the shoe that fits and jump straight to the right page.
🧪 I’m taking a glance
You heard about nano-ros and want to know in 5 minutes whether it’s worth playing with.
- Start at the “Can I use nano-ros right now?” matrix in the intro. One row per dev board you might have on your desk.
- Then the Project Status paragraph for the maturity signal.
- If you stay interested, jump to one of the starters below.
🌱 I’ve never used ROS
New to ROS entirely? Two ideas carry the whole book: a node is a process-like unit that computes; nodes talk through named topics (publish/subscribe), plus request/reply services. ROS 2 is the ecosystem standardizing those; an RMW (“ROS MiddleWare”) backend is the transport that moves the bytes. nano-ros is a small implementation of that model for microcontrollers. That’s enough to start:
- Install — host toolchain, no ROS 2 required.
- First Project — scaffold a two-node workspace and watch one node hear the other.
- Anatomy explains what you built, in nano-ros’s own terms.
🔧 I want it running on the board in my hand tonight
Honest expectations first: the fastest real win is on your Linux
host (~10 minutes, no ROS 2, no daemon — --rmw cyclonedds), and
every embedded chapter has a QEMU flow that works without hardware.
Real-hardware flashing is currently documented only for out-of-tree
boards (STM32F4 worked example).
The zenoh embedded flows need a ROS 2 install on the host for the
router.
- Check your board’s row in Supported Boards.
- Take the host win: First Project.
- Then your platform’s chapter under Bring Your Own RTOS (FreeRTOS / Zephyr / NuttX / ThreadX / ESP32 / bare-metal).
🤖 I have rclcpp/rclpy nodes and a colcon workspace
Experienced ROS 2 developer porting existing nodes to an MCU:
- Setup Compared to Standard ROS 2
— what stays familiar (package.xml, launch XML,
find_package) and what changes (no install prefix, compile-time RMW). - Porting a ROS 2 C++ node — the rclcpp-compat shim and its limits.
- C / C++ multi-node workspaces — the colcon-shaped workspace flow.
- ROS 2 Interoperability —
wire your MCU node into your existing graph. Zenoh needs
ros-jazzy-rmw-zenoh-cpp(Humble ships no apt package); Cyclone DDS interops withrmw_cyclonedds_cppdirectly, no router. - Migration Guide — concept-to-concept map.
🔌 I have a board (or a vendored SDK tree) on my desk
Already have hardware — an ESP32, an STM32Cube or MCUXpresso project, a Zephyr or NuttX workspace? Two-step path:
- Linux first — First Project
on your host verifies the whole stack in ~10 minutes, with no daemon
and no ROS 2 install (
nros setup native --rmw cyclonedds, one scaffold command,cmake). - Then your target — start at How Integration Works: your RTOS keeps its own build tool (west, make, idf.py, your IDE) and nano-ros plugs into it. One chapter per host build system follows it.
- Company tree, own BSP, forked RTOS? Start at
Integrating into a Vendored Tree
(pinning, air-gapped CI, the patch set you carry), then the
Porting Guide: Build as a CMake
subdirectory,
Custom Board Package,
Vendor Overlay, and the
STM32F4 out-of-tree worked example.
Know up front: adding a platform (new RTOS) or an in-catalog
board still means carrying a small patch set inside your vendored
nano-ros checkout — plan for a fork-with-rebase workflow against
the
nros-v<X.Y.Z>tags.
🚀 I want to get started shipping something
You’ve decided to use nano-ros and want a working system on Linux first, then move it to your target.
- Install — three commands:
./scripts/bootstrap.sh,source ./activate.sh(every new shell), thennros setup native --rmw cyclonedds. - First Project — one scaffolded workspace, C++ and CMake, publishing with nothing else running. Rust variant on the same page.
- Anatomy of What You Just Built — the three package roles and the one configuration file; every later addition is another instance of these.
- Growing: nodes, parameters, more deploy targets — the Multi-Node Projects group; other languages — Rust, C, and Mixed.
- Troubleshooting — First 10 Minutes if anything goes sideways.
- Talking to a ROS 2 system — zenoh (router-based) or Cyclone DDS (direct, routerless) both interop: Choosing an RMW.
🔬 I’m evaluating capabilities
You’re a senior engineer or tech lead assessing nano-ros for adoption. You want to see scope of coverage, performance bounds, verification status, and trade-offs before committing.
- Architecture Overview — the three-layer model.
- Execution Model and Two-Layer API — poll vs callback discipline.
- Per-RMW Feature Matrix — generated from the backend sources: services, events, QoS per backend. Backend Reference for architecture and footprint; Support Status for versions, pins, and CI tiers.
- Scheduling Wiring Matrix — generated per-platform truth: which classes and kernel capabilities are wired where. Scheduling Models is the narrative behind it (Real-Time Analysis the lint/tooling catalogue).
- Static Pool Inventory + Opaque Storage Sizing — memory footprint knobs and their single source of truth.
no_std,alloc, andstd+ Dispatch Strategy — the execution/allocation constraints.- Formal Verification — Kani
- Verus harness coverage.
- Safety Protocol — E2E CRC, EN 50159 mapping.
- Production Readiness Checklist — concrete adoption gates.
- nano-ros vs micro-ROS — head-to-head with the closest peer project.
💼 I’m scoping nano-ros for a fleet / product line
You’re a PM, CTO, or technical buyer. You want license terms, supplier reach, deployment patterns, and risk signals before you write the memo.
- Setup Compared to Standard ROS 2 — the elevator pitch + what stays familiar vs what changes.
- Differences from Standard ROS 2 — feature deltas in plain prose.
- Supported Boards — the procurement matrix (vendor × board × MCU × RTOS × status).
- Choosing an RMW Backend — decision tree.
- Cross-backend Bridges — multi-RMW fleets.
- Safety Protocol — E2E CRC framework + standards mapping.
- Production Readiness Checklist — what you’d ask your pilot team to validate.
- nano-ros vs micro-ROS — license / governance / commercial support comparison.
Still not sure?
Read the Introduction for the one-page overview. Every section above branches from there.
Installation
nano-ros is distributed as source, vendored into the consumer’s
project tree (git submodule, west manifest, ESP-IDF component, etc.)
and built in-tree via add_subdirectory(nano-ros). There is no binary
tarball, no system-wide install step, no find_package(NanoRos), and
no prebuilt nros CLI.
Host prerequisites
Everything toolchain-shaped — rustup, cross-compilers, emulators, the
zenoh router — is provisioned by the steps below. The host itself only
needs the basics: git, curl, a C toolchain, pkg-config, python3.
Debian / Ubuntu:
sudo apt-get update
sudo apt-get install -y git curl ca-certificates build-essential \
cmake pkg-config python3 python3-dev zstd \
python3-catkin-pkg python3-empy python3-lark python3-yaml
Fedora / RHEL:
sudo dnf install -y git curl ca-certificates gcc gcc-c++ make \
cmake pkgconf-pkg-config python3 python3-devel python3-pip zstd
# Fedora ships no rosidl packages — install the codegen Python deps via pip:
pip install --user catkin_pkg 'empy==3.3.4' lark PyYAML
Arch:
sudo pacman -S --needed git curl base-devel cmake python python-pip zstd
# Arch ships no rosidl packages — install the codegen Python deps
# (use a venv or pipx if your Python is externally managed):
pip install --user catkin_pkg 'empy==3.3.4' lark PyYAML
(base-devel is a package group, and it covers pkg-config;
ca-certificates ships with the base system.)
macOS: xcode-select --install, plus brew install cmake pkg-config zstd,
then pip install --user catkin_pkg 'empy==3.3.4' lark PyYAML.
(cmake builds the C/C++ quick start and every workspace root; zstd
unpacks the prebuilt SDK assets nros setup fetches; python3-dev is
needed once, while bootstrap.sh builds the launch resolver — it embeds
CPython to parse ROS launch files. Each tool’s absence produces an error
naming it. The Python packages feed rosidl_adapter, the ROS
message-to-IDL step the CycloneDDS backend uses — on Fedora/Arch/macOS
install catkin_pkg, empy==3.3.4, lark and PyYAML via pip or your
package manager; empy must stay 3.x.)
These few packages are the only ones you install by hand. Per-board OS dependencies later on are declared in the index and printed for your package manager — apt, dnf, pacman or brew — by:
nros setup --system # what is missing + the exact install command
That command needs the nros CLI, which is why the basics above come
first: they are what it takes to clone the repo and build the CLI.
The whole flow, end to end
Four steps take a bare machine to a building project; each is detailed in a section below.
Why step 2 builds the CLI instead of downloading it. nano-ros is a source distribution today: there is no prebuilt
nrosto fetch, so obtaining the tool requires the repository and its toolchain. That is backwards —nrosprovisions dependencies and builds workspaces, so it is meant to be the thing you get first, and the thing that makes cloning this repository unnecessary.Prebuilt binaries were shipped early and withdrawn, because a released binary emitted code that had drifted from the runtime — and drifted generated code compiles, so the failure was silent. Restoring the download is blocked on one question having a checkable answer: can this binary’s output work with this runtime? RFC-0090 answers it with a codegen version that every generated artifact carries and every runtime asserts. Until that ships in a release, step 2 stays.
# 1. Pull the source at a pinned version (or `main` for latest):
git clone --branch nros-v0.5.0 https://github.com/NEWSLabNTU/nano-ros.git
cd nano-ros
# 2. Build the `nros` CLI from source (installs rustup if needed):
./scripts/bootstrap.sh
# 3. Wire the workspace env into this (and every future) shell:
source ./activate.sh # or: direnv allow / source ./activate.fish
# 4. Provision the toolchains + RMW daemon for your target:
nros setup <board> --rmw <zenoh|xrce|cyclonedds> # e.g. `native`
Then point your project’s manifest at the checkout — Pattern A/B
below for CMake (the add_subdirectory / -DNANO_ROS_ROOT guard), or
the Rust-only consumers path dependency. See
Pinning a version for how the tag relates to
nros version.
activate.shis the after-install step, NOT a prereq. Sourcingactivate.sh(oractivate.fish, ordirenv allow) only wires the workspace env into a new shell — it presumes thenrosbinary already exists atpackages/cli/target/release/nros. The bootstrap front door below is what produces that binary; the activate file is what makes it findable in every subsequent shell.
See build-as-subdirectory.md for the canonical user incantation (4-line
CMakeLists.txt). This page walks the surrounding workspace + per-target setup choices.
Do I need ROS 2 installed?
Building, no. Running two zenoh nodes, yes — or use Cyclone. nros sync generates the message bindings from the interface sources vendored
in packages/cli/interfaces/, so nothing about setup, codegen or
compilation needs ROS 2.
Running is where it splits, and the reason is the router. zenoh-pico
connects in client mode, so any two-process example needs a zenoh router
— and since RFC-0075 that router is
rmw_zenoh_cpp/rmw_zenohd, which ships with ROS 2. nano-ros no longer
vendors one. So on a host with no ROS 2:
--rmw cycloneddsworks standalone — Cyclone is in-process, there is no daemon to start. This is the ROS-less path.--rmw zenoh(the default) has no router. Either installros-<distro>-rmw-zenoh-cpp, or pointNROS_RMW_ZENOHDat armw_zenohdyou obtained another way.
If you do have ROS 2, source its setup first — that is what makes the router findable:
source /opt/ros/<distro>/setup.bash
nano-ros reads AMENT_PREFIX_PATH from that, so a ROS installed anywhere works,
including one you built from source or a colcon overlay — it is not assumed to be
under /opt/ros.
Do not expect command -v rmw_zenohd to print anything: rmw_zenohd installs
into <prefix>/lib/rmw_zenoh_cpp/, and setup.bash puts only bin/ on PATH.
ROS’s own way to reach it is ros2 run rmw_zenoh_cpp rmw_zenohd. So a silent
command -v is normal and not a broken install — the
ros2 run rmw_zenoh_cpp rmw_zenohd spelling finds it regardless.
Only what you name is used. nano-ros needs the router that ships with your
rmw_zenoh_cpp: the two share a zenoh build, and a mismatch shows up as
“nothing was delivered” rather than as an error. So the search is
NROS_RMW_ZENOHD, then the prefixes you sourced, then $ROS_DISTRO — and
nothing else. In particular:
PATHis not searched. Azenohdthere — installed from zenoh’s own instructions, or left behind by an older nano-ros — is usually a different and older zenoh. The machine this was written on had two, at v1.4.0 and 1.7.2, against ROS’s 1.8.0./opt/ros/*is not globbed. If you have humble and jazzy installed, guessing between them is not better than saying nothing. Source the one you want, or setROS_DISTRO.
If you do want a specific binary, name it with NROS_RMW_ZENOHD; it is used as
given, with a warning if it is not an rmw_zenoh_cpp router.
If you once used an older nano-ros, run nros doctor: it reports a retired SDK
store entry that is still installed, zenohd included, with the command to
remove it.
| Task | Needs a ROS 2 install? |
|---|---|
nros setup, nros sync, message codegen | No — interface sources are vendored |
| Building any node (Rust / C / C++), any RMW | No |
| Embedded targets (Zephyr, FreeRTOS, NuttX, ThreadX) | No |
| Cyclone DDS backend, build and run | No — idlc comes from the index dist, and there is no daemon |
| Running a multi-process zenoh example | Yes — the router is ROS’s rmw_zenohd (or set NROS_RMW_ZENOHD) |
Verifying with ros2 topic echo / ros2 node list | Yes |
Interop tests against rmw_zenoh_cpp / a real ROS 2 graph | Yes |
Contributor lanes that bridge to ROS 2 (just test-all interop cells) | Yes |
When you source activate.sh without ROS 2 present it says so:
activate.sh: /opt/ros/humble/setup.bash not found — ROS-dependent recipes
will fail (interop tests, `ros2` CLI verification). The setup, codegen
and first-node flows do not need it.
That line is informational on a getting-started host. If you do want the ROS 2 side and your distro has no Humble packages (Arch, for example), a container is the usual answer — nano-ros itself stays on the host.
Pattern A: nano-ros lives inside your ROS 2 workspace
The recommended layout is to clone nano-ros into your workspace’s
src/ directory alongside your packages:
~/ros2_ws/
├── src/
│ ├── nano-ros/ # <-- this repo
│ ├── pkg_a/ # your package(s)
│ ├── pkg_b/
│ └── …
└── (build/, install/, log/ — generated by colcon)
colcon build discovers nano-ros + each user package via package.xml,
builds them in dependency order, and shares one nano-ros build across
every consuming package. Users never run cmake manually — the per-user
package’s CMakeLists.txt does add_subdirectory(../nano-ros nano_ros)
under the hood.
A complete working example lives at
examples/templates/multi-package-workspace/.
Pattern B: nano-ros as a third-party subdirectory
For C/C++ projects that don’t use colcon at all:
~/my_project/
├── CMakeLists.txt
├── src/main.c
└── third_party/
└── nano-ros/ # git submodule
The top-level CMakeLists.txt:
cmake_minimum_required(VERSION 3.22)
project(my_app LANGUAGES C)
# `NROS_RMW` is the user-facing cache var (overridable via
# `-DNROS_RMW=<rmw>`); forward it to `NANO_ROS_RMW`, the var the
# nano-ros add_subdirectory reads. Matches the canonical example
# shape in examples/native/c/talker/CMakeLists.txt.
set(NANO_ROS_PLATFORM posix)
set(NROS_RMW "zenoh" CACHE STRING
"Active RMW (zenoh|xrce|cyclonedds) — selects the backend linked into my_app.")
set(NANO_ROS_RMW "${NROS_RMW}")
add_subdirectory(third_party/nano-ros nano_ros)
add_executable(my_app src/main.c)
target_link_libraries(my_app PRIVATE NanoRos::NanoRos)
nros_platform_link_app(my_app)
nros_platform_link_app transitively wires the selected RMW backend
on POSIX — no explicit nano_ros_link_rmw() call is needed. That is
the entire consumption shape. See
build-as-subdirectory.md for the full
walkthrough.
Provision your toolchain with nros setup
nros setup is the single command that prepares a machine to build
nano-ros for a given board. The cross-compiler, emulator, RMW host
daemon, and any SDK sources a board needs are resolved from a pinned
index and placed in a shared store (~/.nros/sdk). You do not
install cross-toolchains by hand, and you do not need ROS 2 on the
machine.
Packages are prebuilt where the index has a binary for your host, and
built from source otherwise. Source builds are normal, not an error —
but they cost real time and disk, so nros setup announces them before
it starts:
nros setup: 1 package(s) have no prebuilt for linux-x86_64 — BUILDING FROM SOURCE: <name>
Expect minutes (tens of minutes for a large recipe) and hundreds of MB under …
Source recipes build with the workspace’s pinned Rust channel, so a
recipe carrying its own pin does not make rustup fetch a second
toolchain behind your back. nros setup <board> --dry-run prints the
whole plan, prebuilt vs source, without fetching anything — on the
native board it is two source packages (zenoh-pico and mbedtls,
both vendored submodules) and no daemon at all.
1. Get the nros CLI onto PATH
nano-ros is a source distribution — there is no prebuilt nros
download. One front door from a fresh checkout (just is NOT a
prereq; rustup is installed on demand):
Sourcing
activate.shon a host withoutjustprints one line about RTOS SDK path defaults (FREERTOS_DIR,NUTTX_DIR,IDF_PATH, …) not being loaded. Harmless — nothing in the user flows reads them, and embedded builds work with plaincargo/cmake.justis a contributor tool: only the in-treejustrecipes (test lanes, fixture builds) need it (cargo install just).
git clone https://github.com/NEWSLabNTU/nano-ros.git
cd nano-ros
./scripts/bootstrap.sh
Builds the in-tree nros CLI from source at
packages/cli/target/release/nros and leaves it on PATH for this
shell.
Already have cargo? The equivalent by hand (same build, same binary):
git submodule update --init packages/cli/third-party/play_launch
cargo build --release --manifest-path packages/cli/Cargo.toml --bin nros
export PATH="$PWD/packages/cli/target/release:$PATH"
Both produce the per-checkout binary at
packages/cli/target/release/nros. One checkout = one CLI version =
one runtime ABI — no global install, no ~/.nros/bin PATH skew
across worktrees.
2. Activate the workspace (every subsequent shell)
The bootstrap path you ran in §1 left nros on PATH for that shell
only. Every new shell needs the workspace env wired in. Pick whichever
fits your shell — all three wire the same env exports + PATH entries
from the Phase 218.C SSoT activate.sh:
direnv allow # auto-activates on `cd nano-ros` (recommended)
source ./activate.sh # bash / zsh, one-shot per shell
source ./activate.fish # fish, one-shot per shell
The activate file also sources /opt/ros/humble/setup.bash if
present (used by the rmw_zenoh interop tests, and it lets codegen
resolve message packages from the ament index; building with any RMW —
Cyclone included — does NOT require ROS 2, only the pip-installable
rosidl_adapter Python packages from the prerequisites) and exports
NROS_REPO_DIR.
3. Provision a board (+ RMW)
nros setup <board> --rmw <zenoh|xrce|cyclonedds>
nros setup resolves the board’s package set union the RMW’s host
packages and fetches them all — prebuilt cross-toolchain + emulator +
RMW daemon + board SDK sources. --rmw defaults to zenoh.
| Command | Provisions |
|---|---|
nros setup native | host build; zenoh-pico + mbedtls sources (no router — see above) |
nros setup native --rmw xrce | host build; the Micro-XRCE-DDS agent |
nros setup native --rmw cyclonedds | host build; Cyclone DDS runtime + idlc |
nros setup qemu-arm-freertos | arm-none-eabi-gcc, patched qemu-system-arm, FreeRTOS + lwIP sources |
nros setup qemu-arm-nuttx | arm-none-eabi-gcc, qemu, NuttX sources |
nros setup qemu-riscv64-threadx | riscv64-*-gcc, qemu, ThreadX/NetX sources |
nros setup threadx-linux | ThreadX POSIX-sim sources |
nros setup qemu-esp32-baremetal | the ESP32-C3 QEMU + bare-metal (esp-hal) toolchain bits |
nros setup zephyr | the Zephyr west workspace + Zephyr SDK bits |
nros setup mps2-an385 / stm32f4 / qemu-arm-baremetal | bare-metal arm-none-eabi-gcc + qemu |
Useful flags:
nros setup --list # every package in the index + its version
nros setup <board> --dry-run # resolve + print the plan, fetch nothing
nros setup --licenses # license-gated packages + how to install them
nros setup --tool qemu # one tool by name
nros setup --source freertos-kernel # one source package by name
Each board’s exact package set lives in the index; run
nros setup <board> --dry-run to see precisely what a board pulls.
See Supported Boards for the full
board list and nros CLI for every subcommand.
Heads-up before your first example. Every nano-ros example (Linux talker, FreeRTOS talker, …) connects to its RMW host daemon at startup — a zenoh router for zenoh, the Micro-XRCE-DDS agent for xrce. Cyclone DDS is in-process — no separate daemon — so none of this applies if you ran
nros setup … --rmw cyclonedds.For xrce,
nros setup … --rmw xrceinstalls the agent into the nros store (~/.nros/sdk/<tool>/<version>/bin/), and you run it in a dedicated terminal before launching any example.For zenoh, the router is not provisioned by
nros setup: it is ROS 2’srmw_zenoh_cpp/rmw_zenohd, so that nano-ros is tested against the router a ROS 2 deployment actually runs (RFC-0075). Start it with your ROS environment sourced (leave it running for the whole session):ZENOH_CONFIG_OVERRIDE='listen/endpoints=["tcp/127.0.0.1:7447"];scouting/multicast/enabled=false' \ ros2 run rmw_zenoh_cpp rmw_zenohdIt is not configured on the command line —
rmw_zenohdignores argv and readsZENOH_CONFIG_OVERRIDE, as above. To pick a different endpoint, change the port in thelisten/endpointsentry.Without a router the talker blocks forever on
Executor::openwith no output. Default ports:tcp/127.0.0.1:7447on POSIX,tcp/10.0.2.2:7447on QEMU FreeRTOS (Slirp forwards to host). Mismatch = silent hang; see Troubleshooting — First 10 Minutes.
After provisioning, follow the per-platform starter page (FreeRTOS, Zephyr, NuttX, …) for the board’s build + run steps.
Rust-only consumers
nano-ros is source-only — nothing is published to crates.io
(decision 2026-05-14). The full nros crate can’t be published
because it depends transitively on C/C++ submodules (zenoh-pico,
mbedtls); nros-core isn’t carved out for crates.io either, to
avoid a hybrid distribution model with version drift between the
crates.io snapshot and in-repo HEAD.
Rust packages consume nano-ros one of two ways:
- Registry-style names + patch redirection (what every shipped
example uses): declare
nros = { version = "*" }(+ msg crates) and letnros syncwrite the# nros-managed[patch.crates-io]block into.cargo/config.toml, resolving the names into the checkout (NROS_REPO_DIRnames it). Copy the project anywhere; re-runnros syncto re-point. - Explicit path dependency on the in-workspace checkout:
[dependencies]
nros = { path = "src/nano-ros/packages/api/nros",
default-features = false,
features = ["std", "rmw-cffi", "platform-posix", "ros-humble"] }
Each Rust package carries its own .cargo/config.toml patch entries
when needed — see
examples/templates/multi-package-workspace/src/pkg_rust_publisher/
for the pattern.
Pinning a version
Consumers pin the source tree, not a package: clone (or submodule)
at a nros-v<X.Y.Z> git tag —
git clone --branch nros-v0.5.0 https://github.com/NEWSLabNTU/nano-ros.git
# or, as a submodule in your project:
git submodule add -b main https://github.com/NEWSLabNTU/nano-ros third_party/nano-ros
git -C third_party/nano-ros checkout nros-v0.5.0
The tag names the bundle version: the nros CLI built from that
tree prints the same number (nros version → nros 0.5.0; the
scripts/check-version-lockstep.sh gate + bootstrap.sh shell-doctor
enforce binary↔tree agreement), and the tree’s committed
nros-sdk-index.toml pins the exact toolchain/SDK versions
nros setup will provision for it. One tag therefore fixes the whole
surface: runtime source, CLI behavior, and provisioned toolchains.
Upgrading = checking out a newer tag and re-running
./scripts/bootstrap.sh (rebuilds the CLI to match) + nros setup
(reconciles the SDK store against the new index; already-current
packages are no-ops). See
versioning
for how the bundle number is chosen.
Contributor setup (working on nano-ros itself)
Contributors clone the repo and drive everything through just. The
just setup recipes are thin wrappers over the same nros setup index —
just setup <module> calls nros setup <board> under the hood, so the
toolchains a contributor gets are identical to a user’s.
git clone https://github.com/NEWSLabNTU/nano-ros.git
cd nano-ros
./scripts/bootstrap.sh base # path A from §1 above; gets rustup + just + nros
source ./activate.sh # OR: direnv allow / source ./activate.fish
just setup all # provision every supported board's SDK/toolchain
Diagnose missing tools (read-only):
just doctor tier=all
Provision one module:
just setup freertos # → nros setup qemu-arm-freertos
just setup nuttx # → nros setup qemu-arm-nuttx
just setup threadx_linux # → nros setup threadx-linux
Docker environment (contributors)
For a containerized environment with QEMU 7.2+ (in-tree checkout):
just docker build
just docker shell # Interactive shell with all tools
just docker test-qemu # Run QEMU tests in container
Migrating from a pre-140 checkout
If you were on a nano-ros version that still had the (contributor)
just install-local recipe,
see
migration-install-local-removal.md
for the one-page rewrite.
Next Steps
- First Project — the Quick Start continues here: scaffold a two-node workspace and run it (no ROS 2 needed)
- First Node — Rust — build + run a Rust talker on Linux
- First Node — C — build + run a C talker on Linux
- First Node — C++ — build + run a C++ talker on Linux
- Build as Subdirectory — CMake consumption walkthrough
- C API — API entry points and CMake integration
examples/templates/multi-package-workspace/— full mixed C / C++ / Rust example
First Project
One command scaffolds a working multi-node workspace; two more build and run it. C++ and CMake, publishing on your machine, with nothing else running — no router, no daemon, no ROS 2 installation.
This assumes the two-step install is done (nros on
your PATH, nros setup native --rmw cyclonedds run once).
Scaffold
nros new my_robot --workspace
cd my_robot
That wrote 21 files — a complete workspace, not a hello-world stub:
my_robot/
├── CMakeLists.txt # workspace root — names the packages, picks the RMW
└── src/
├── talker_pkg/ # a C++ node: publishes std_msgs/Int32 on /chatter
├── listener_pkg/ # a C++ node: subscribes to /chatter
├── demo_bringup/ # launch file + system.toml — no code
└── robot_entry/ # the binary: boots the launch topology
What each directory is for is the next chapter, Anatomy of What You Just Built. You don’t need it to run.
Build
cmake -S . -B build -DNANO_ROS_ROOT=<path-to-your-nano-ros-checkout>
cmake --build build
The first configure compiles nano-ros’s runtime into the build tree
(~3 minutes); rebuilds are seconds. If you use direnv/activate.sh
from the nano-ros checkout, -DNANO_ROS_ROOT can be omitted — the
NROS_REPO_DIR env it exports is picked up automatically.
Run
./build/src/robot_entry/robot_entry
Published: 0
Received: 0
Published: 1
Received: 1
Published: 2
Received: 2
One process, two nodes: the talker publishes std_msgs/Int32 on
/chatter every 500 ms and the listener prints each one it receives —
the interleaved Received: lines are your proof of delivery — exactly as a ROS 2
composition container would run two composable nodes. Ctrl-C stops it.
Why nothing else needed to be running: the default RMW is CycloneDDS, which discovers peers directly — there is no router or daemon in the picture. The zenoh backend (for talking to a ROS 2 system) and XRCE (for the smallest targets) are one page away: Choosing an RMW, and switching is a one-word edit — Switching RMW in Config.
Same thing in Rust
nros new my_rust_robot --workspace --lang rust
cd my_rust_robot
NROS_REPO_DIR=<path-to-nano-ros> nros sync # Rust-side codegen; once per checkout location
RUST_LOG=info cargo run -p robot_entry
[INFO talker_pkg] Publishing: 0
[INFO listener_pkg] I heard: 0
[INFO talker_pkg] Publishing: 1
(The workspace root is a virtual manifest — cargo run alone has no
default binary, so name the entry package with -p robot_entry. The
Rust talker ticks at 1 s.)
Same workspace shape, same system.toml, same launch file — only the
node implementations and the build tool differ. The extra nros sync
step is the Rust-side message codegen; C++ does the equivalent inside
CMake, which is why the C++ path doesn’t have it.
If it didn’t work
nros: RMW session open failedon a busy machine usually means the DDS port range on yourROS_DOMAIN_IDis contended — pick another:ROS_DOMAIN_ID=57 ./build/src/robot_entry/robot_entry.- Everything else: Troubleshooting — First 10 Minutes.
Where to go next
- Understand the four directories — Anatomy.
- Talk to a real ROS 2 system — ROS 2 Interoperability.
- Put it on a board — How Integration Works.
- Add nodes, parameters, more targets — Project layout and the Multi-Node Projects section.
Anatomy of What You Just Built
Four directories, three roles, one configuration file. Everything you will ever add to this project is another instance of one of these three roles — the shape never changes, it only gains packages.
If you come from ROS 2, each role is something you already know:
| Your workspace | Role | In rclcpp terms |
|---|---|---|
src/talker_pkg/, src/listener_pkg/ | Node package | a composable node (rclcpp_components) |
src/demo_bringup/ | Bringup package | <pkg>_bringup — launch files + config, no code |
src/robot_entry/ | Entry package | the composition container / main() |
Node packages — the code
A node package is one node as a reusable library. The C++ shape is a
class with a configure(nros::Node&) method that binds real member
callbacks:
src/talker_pkg/
├── CMakeLists.txt # nano_ros component library
├── package.xml # ROS package manifest — deps live here
├── include/talker_pkg/Talker.hpp
└── src/Talker.cpp # bind_timer → publish on /chatter
package.xml is standard ROS: message dependencies (std_msgs) are
declared there and code-generated during the build. You never hand-write
message types.
The bringup package — the one configuration file
src/demo_bringup/ holds no code: a launch file and system.toml.
This is where every configuration axis lives:
[system]
name = "demo"
rmw = "cyclonedds" # the RMW axis
domain_id = 0
default_launch = "system.launch.xml"
[[component]] # the topology: which nodes exist
pkg = "talker_pkg"
class = "talker_pkg::Talker"
name = "talker"
[image.native] # one block per BUILDABLE image
board = "native"
[host.native] # WHERE the nodes run; no `nodes` = all
The launch file is the ROS 2 launch XML schema, verbatim —
<node>, <param>, <remap>, <group>, <include> with $(find) /
$(var) substitutions. A launch file from nav2 or Autoware pastes in
unchanged (Python .launch.py is not supported). The scaffold’s launch
even carries a per-topic QoS override to show the plumbing:
<node pkg="talker_pkg" exec="talker" name="talker">
<param name="qos_overrides./chatter.publisher.reliability" value="best_effort"/>
</node>
When you later target hardware, you add an [image.<id>] block per
board — the code and topology stay put. That is the growth rule:
configuration is a new block, never a restructure.
The entry package — the deliverable
src/robot_entry/ is the binary. Its CMakeLists calls
nano_ros_entry(... BRINGUP ... LAUNCH default), which at configure
time resolves the bringup’s launch file and generates a main() that
constructs each declared component, applies launch parameters and QoS
overrides, and runs them on one executor — the launch product is the
binary. There is no separate ros2 launch step; running the entry is
launching the system.
One entry per deployable image. A robot with a perception image and a control image is one workspace, two entry packages, one bringup.
What was generated vs. what is yours
Everything under src/ is yours to edit. The build tree (build/)
holds everything derived: generated message code, the resolved system
model, the entry’s generated main(). Nothing derived is ever
committed — if you can regenerate it, it does not belong in git.
The same anatomy in Rust
nros new --workspace --lang rust produces the identical shape: node
packages are library crates with nros::node!(Talker), the entry is a
one-line nros::main!(launch = "demo_bringup", spin = "forever"), and
the bringup directory is byte-for-byte the same idea. One difference:
Rust message codegen runs as a pre-build step (nros sync) instead of
inside CMake.
Next
- The Quick Start continues: Your own message package
- Add a second talker or your own node: Node packages
- Point it at real hardware: How Integration Works
- The full role reference: Bringup packages · Entry packages
Your own message package
You write a ROS 2 msg package once. The same src/my_msgs/ directory
builds under both:
- a nano-ros build —
rosidl_generate_interfaces(...)is intercepted by nano-ros’s wrapper and routed through nano-ros codegen. No ROS 2 install needed — even therclcpp-shaped consumer below compiles against nano-ros’s source-compat layer. colcon build— upstreamrosidl_default_generatorsproduces the upstream-ROS bindings (this half, and only this half, needs a ROS 2 install; it is the optional cross-build proof at the end).
Different build systems, identical source tree. No nano-ros-specific files in the msg package.
Where this fits. This chapter uses ROS 2 vocabulary (
ament_cmake,rosidl,rclcpp) because its point is that stock ROS shapes work unchanged. If you came straight from First Project and none of that means anything yet, you can skim to “The Rust side” below or skip ahead to First Node — Rust and return later. The standalone tree here is for clarity — in your scaffoldedmy_robotworkspace the same drop-in applies: putsrc/my_msgs/next to your node packages.
The msg package — stock ROS shape
Drop a verbatim ROS msg pkg under src/ of your workspace:
src/my_msgs/
├── package.xml
├── CMakeLists.txt
└── msg/
└── MyMsg.msg
src/my_msgs/package.xml:
<?xml version="1.0"?>
<package format="3">
<name>my_msgs</name>
<version>0.1.0</version>
<description>My ROS 2 msg package</description>
<maintainer email="you@example.org">you</maintainer>
<license>Apache-2.0</license>
<buildtool_depend>ament_cmake</buildtool_depend>
<depend>std_msgs</depend>
<build_depend>rosidl_default_generators</build_depend>
<exec_depend>rosidl_default_runtime</exec_depend>
<member_of_group>rosidl_interface_packages</member_of_group>
<export>
<build_type>ament_cmake</build_type>
</export>
</package>
src/my_msgs/CMakeLists.txt:
cmake_minimum_required(VERSION 3.20)
project(my_msgs)
find_package(ament_cmake REQUIRED)
find_package(rosidl_default_generators REQUIRED)
find_package(std_msgs REQUIRED)
rosidl_generate_interfaces(${PROJECT_NAME}
msg/MyMsg.msg
DEPENDENCIES std_msgs
)
ament_export_dependencies(rosidl_default_runtime)
ament_package()
src/my_msgs/msg/MyMsg.msg:
std_msgs/Header header
string payload
int32 sequence
Zero nano-ros-specific lines. Run colcon build from this dir — upstream
ROS produces a working msg package. Drop the same dir into a nano-ros
build (below) — nano-ros codegen produces the equivalent bindings.
The consumer — stock ROS shape
// src/my_app/src/my_app.cpp
#include <chrono>
#include <memory>
#include <rclcpp/rclcpp.hpp>
#include "my_msgs/msg/my_msg.hpp"
using namespace std::chrono_literals;
class MyNode : public rclcpp::Node {
public:
MyNode() : rclcpp::Node("my_node") {
publisher_ = this->create_publisher<my_msgs::msg::MyMsg>("topic", 10);
timer_ = this->create_wall_timer(500ms, [this]() {
my_msgs::msg::MyMsg m;
m.payload = "hello";
publisher_->publish(m);
});
}
private:
std::shared_ptr<rclcpp::TimerBase> timer_;
std::shared_ptr<rclcpp::Publisher<my_msgs::msg::MyMsg>> publisher_;
};
int main(int argc, char* argv[]) {
rclcpp::init(argc, argv);
rclcpp::spin(std::make_shared<MyNode>());
rclcpp::shutdown();
return 0;
}
src/my_app/CMakeLists.txt:
cmake_minimum_required(VERSION 3.20)
project(my_app LANGUAGES CXX)
find_package(ament_cmake REQUIRED)
find_package(rclcpp REQUIRED)
find_package(my_msgs REQUIRED)
find_package(std_msgs REQUIRED)
add_executable(my_app src/my_app.cpp)
target_link_libraries(my_app
rclcpp::rclcpp
my_msgs::my_msgs
std_msgs::std_msgs
)
ament_package()
The nano-ros umbrella — the only nano-ros-specific file
The two src/ packages are stock ROS. One umbrella CMakeLists.txt at the
workspace root pulls nano-ros in:
cmake_minimum_required(VERSION 3.22)
project(my_workspace LANGUAGES CXX)
set(CMAKE_CXX_STANDARD 14)
# Pull nano-ros.
set(NANO_ROS_PLATFORM posix)
set(NROS_RMW "zenoh" CACHE STRING "Active RMW.")
set(NANO_ROS_RMW "${NROS_RMW}")
add_subdirectory(/path/to/nano-ros nano_ros)
# Point the smart Find-stub at this workspace (must precede the compat
# include so the workspace Find<pkg>.cmake auto-emit picks it up).
set(NROS_INTERFACE_SEARCH_PATH "${CMAKE_SOURCE_DIR}/src")
# Pull the rclcpp source-compat layer (find_package(rclcpp) etc.).
include(/path/to/nano-ros/cmake/compat/NrosRclcppCompat.cmake)
# Bulk-build every workspace msg pkg in topo order. One line, no
# add_subdirectory(src/<pkg>) per pkg.
nros_workspace_interfaces()
# Build the consumer app.
add_subdirectory(src/my_app)
Build:
cmake -B build -S .
cmake --build build -j
./build/src/my_app/my_app
The Rust side
The same msg package feeds a Rust node with no CMake involved: declare
it in your node package’s package.xml (<depend>my_msgs</depend>)
and manifest (my_msgs = { path = "../../generated/my_msgs", version = "0.0.0", default-features = false } — workspace-level generated/,
constant 0.0.0 version), then run nros sync from the workspace
root — the package index finds src/my_msgs/ by its package.xml,
generates the generated/my_msgs crate, and wires the cargo patch
table. The shipped precedent is
examples/workspaces/features/src/custom_msgs/ with its
rust_reading_listener_pkg consumer. Full
detail: Message Generation.
use my_msgs::msg::MyMsg; // generated crate, same field names
Cross-build proof — same source under colcon (optional; needs ROS 2)
cd src && colcon build
src/my_msgs/ produces the upstream my_msgs bindings; src/my_app/
links against the upstream rclcpp + my_msgs::my_msgs. The
nano-ros build above produces the same source linked against
NanoRos::NanoRosCpp through the nano-ros codegen, with the smart
Find-stub forwarding find_package(my_msgs) → my_msgs::my_msgs.
The interface-package search path
find_package(<pkg>) walks three layers, highest priority first:
| Layer | Source | Notes |
|---|---|---|
| 1 | NROS_INTERFACE_SEARCH_PATH | Colon/semicolon-separated colcon-src/-style roots; immediate subdirs with package.xml are candidates. |
| 2 | AMENT_PREFIX_PATH | The standard ROS install-prefix layout (<prefix>/share/<pkg>/{msg,srv,action}/). |
| 3 | Bundled | <nano-ros>/packages/cli/interfaces/<pkg>/ (in-tree bundled definitions). |
Shadowing — a workspace my_msgs and an AMENT my_msgs resolve to the
workspace one, with a message(STATUS ...) line noting the shadow.
Shadowing contract
When two layers carry the same package name (e.g. a workspace
std_msgs and /opt/ros/<distro>/share/std_msgs/), the higher
layer wins — silently and deterministically. Concretely:
-
NROS_INTERFACE_SEARCH_PATH>AMENT_PREFIX_PATH> bundled. The smart Find-stub (cmake/compat/stubs/_NrosFindRosMsgPackage.cmake) walks the three layers in order; the first hit wins. Lower layers are skipped entirely. -
The configure pass emits a
message(STATUS ...)line noting the resolved path, e.g.-- nros: find_package(std_msgs) -> /path/to/workspace/src/std_msgsGrep your configure log for
nros: find_package(<pkg>)to confirm which layer supplied each pkg. -
Intra-workspace shadowing — when two roots under
NROS_INTERFACE_SEARCH_PATHboth ship the same pkg, thenros_workspace_interfaces()bulk orchestrator keeps the earlier-listed one and warns about the shadowed copy. -
No partial overrides. Shadowing is whole-package: a workspace pkg replaces ALL of an AMENT pkg’s interfaces, not a subset. If you only want to add
MyExtraMsg.msgtostd_msgs, ship a separate pkg (e.g.my_std_extra_msgs) — don’t shadow.
Compile-time fail-safe
Shadowing is observed at compile time: if your workspace std_msgs
declares a Marker.msg and your consumer includes
"std_msgs/msg/marker.hpp", the build only succeeds when the
workspace copy is the one linked. AMENT’s std_msgs ships no
Marker.msg, so a fall-through resolves to a missing header. The
build outcome is the strongest evidence — nm on the linked binary
is the symbol-level corroborator.
Reference fixture for shadowing
The canonical smoke proof for the Layer 1 > Layer 2 case ships at
examples/templates/workspace-shadowing/ —
a workspace std_msgs carrying a Marker.msg shadows the
AMENT-installed std_msgs. The fixture’s README.md walks through
the cmake + nm verification. A regression test
(packages/testing/nros-tests/tests/workspace_shadowing.rs)
re-runs the same proof in CI when AMENT is sourced.
Reference fixture
examples/templates/local-msg-package/
ships the exact pattern above end-to-end (two workspace msg pkgs with a
dep between them, plus a consumer node). Use it as a copy-out template.
Next
- Something broke on the way here? Troubleshooting — First 10 Minutes.
- Single-node deep dive: First Node — Rust.
- Grow the workspace: Node packages.
Troubleshooting — First 10 Minutes
The Quick Start runs on --rmw cyclonedds — no router daemon exists
on that path, so “no output” there is never a router problem; skip
section B’s router branches. The zenoh walkthroughs (First Node pages)
additionally assume nros setup native --rmw zenoh has run and the
ROS 2 zenoh router (ros2 run rmw_zenoh_cpp rmw_zenohd) is reachable. When something goes wrong in the first ten minutes, the
error you see usually points at one of the predictable misses below.
Each branch quotes the real stderr you can grep against — not a paraphrase. If your error text matches, the fix on the right is the one to try.
A. Build failures (cargo / cmake)
A1. nros crates don’t resolve (missing / stale patch block)
error[E0432]: unresolved import `nros`
error: failed to load source for dependency `nros`
error: no matching package named `nros` found
The example’s Cargo.toml declares nano-ros crates registry-style
(nros = { version = "*" } — they are not published to crates.io);
the example’s .cargo/config.toml carries the # nros-managed
[patch.crates-io] block that resolves them into a nano-ros
checkout. This error means the patch block is missing or its
relative paths no longer reach a checkout (typical right after
copying the example somewhere else). Fix:
cd <the example dir>
NROS_REPO_DIR=/path/to/nano-ros nros sync
which regenerates the message crates and rewrites the patch block for the example’s current location.
This is not an nros setup issue — nros setup only fetches
the SDK / source-package payload (zenoh-pico, mbedtls, cyclonedds,
…); it does not synthesise missing Cargo dependencies.
A2. nros codegen tool not found
nros (codegen tool) not found on PATH or in packages/cli/target/release/
or ${NROS_HOME:-~/.nros}/bin. nano-ros assumes `nros` is provided
(the CLI lives in-tree at packages/cli/). Build it with:
just setup-cli # or: just setup
(The message names the contributor recipe; ./scripts/bootstrap.sh
runs the same build without just.)
Missing the nros binary on PATH and in the per-checkout location.
It builds from the in-tree sub-workspace; build it, then
activate the workspace:
./scripts/bootstrap.sh # builds packages/cli/target/release/nros
source ./activate.sh # OR: direnv allow / source ./activate.fish
If packages/cli/target/release/nros exists but PATH doesn’t see it —
that’s the [PATH] doctor status (D2 below), not this branch (the
activate file is what puts it on PATH).
The nros binary ships the codegen — there is no separate
nros-codegen build step. CMake examples auto-resolve nros from
PATH / packages/cli/target/release/ / ${NROS_HOME:-~/.nros}/bin/;
-D_NANO_ROS_CODEGEN_TOOL=<path> is an override, not a requirement.
A3. Rust target not installed
error: could not compile … due to previous error
the target `thumbv7m-none-eabi` is not installed
Add it:
rustup target add thumbv7m-none-eabi
# or whichever target the example's `.cargo/config.toml` names
A4. Cross linker not found
error: linker `arm-none-eabi-gcc` not found
The cross toolchain wasn’t provisioned. Run nros setup for your board (it ships a prebuilt arm-none-eabi-gcc):
nros setup qemu-arm-freertos # or qemu-arm-nuttx / mps2-an385 / …
A5. Cyclone DDS runtime missing
ld: cannot find -lddsc
ld: cannot find -lcyclonedds-ddsc
The Cyclone DDS runtime wasn’t provisioned:
nros setup native --rmw cyclonedds
A6. cargo build --features rmw-cyclonedds can’t link
undefined reference to `nros_rmw_cyclonedds_register`
undefined reference to `dds_create_participant`
rmw-cyclonedds cannot link from cargo alone — the Cyclone backend
is C++ + CMake, registered via nros_rmw_cffi_register from a
CMake-built target — wired through CMakeLists.txt +
Corrosion. Use the cmake build path instead:
cd examples/native/c/talker # (or cpp / rust)
cmake -B build-cyclone -DNROS_RMW=cyclonedds
cmake --build build-cyclone
The pure cargo build --features rmw-cyclonedds only succeeds for
the zenoh-pico + xrce backends today.
A7. “current package believes it’s in a workspace”
error: current package believes it's in a workspace when it's not:
current: …/Cargo.toml
workspace: /…/nano-ros/Cargo.toml
cargo walks up the directory tree looking for a workspace root and
adopts the example into the outer nano-ros workspace. Per-example
Cargo.tomls don’t ship an empty [workspace] table yet (tracked
as a known follow-up).
Hits on:
- nested clones / worktrees of nano-ros that share an ancestor path
with the outer
nano-ros/Cargo.toml; - a user vendoring an example into their own workspace.
Workaround on a regular clone: build from the nano-ros root, e.g.
cargo build -p qemu-bsp-talker, instead of cd’ing into the
example dir.
A8. direnv allow reminder
NROS_PLATFORM_CFFI_INCLUDE not set (direnv allow, or build via just)
FREERTOS_PORT not set
(The quoted message is emitted by the build script; “build via just”
in it refers to the contributor recipes.) The
common build sites autoresolve these from
the in-tree checkout, so a fresh cargo build no longer panics on
them in canonical examples. If your custom build site still does,
run direnv allow once after clone, or set the env explicitly
(contributors can build via the in-tree just <plat> recipe).
B. Binary runs but no output
B1. Rust: RMW session open failed — Transport(ConnectionFailed)
[ERROR nros] RMW session open failed — Transport(ConnectionFailed)
The zenoh router isn’t running, or isn’t reachable on the locator the talker is pointed at. The router comes from ROS 2 (nano-ros no longer ships one) — start it in another terminal:
ZENOH_CONFIG_OVERRIDE='listen/endpoints=["tcp/127.0.0.1:7447"];scouting/multicast/enabled=false' ros2 run rmw_zenoh_cpp rmw_zenohd
Default ports: tcp/127.0.0.1:7447 on POSIX,
tcp/10.0.2.2:7451 on QEMU FreeRTOS (Slirp forwards to host),
7452 NuttX, 7453 ThreadX-RV, 7454 ESP32, 7455
ThreadX-Linux, 7447 Zephyr.
B2. C: NROS_CHECK failed: nros_support_init(...) -> -4
NROS_CHECK failed at src/main.c:152: nros_support_init(&app.support, locator, domain_id) -> -4
Process exits 1 (the retval passed to NROS_CHECK_RET).
-4 = NROS_RET_NOT_FOUND — the locator was unreachable (zenohd
not running, or wrong port). Same fix as B1 above.
The C API entry point is nros_support_init, not nros_init
or nros::init — those don’t exist in the C API.
B3. C++: process exits 156 after a nros::init failure
nros::init returned NROS_CPP_RET_TRANSPORT_ERROR (-100)
Same root cause as B1/B2 — zenohd not reachable.
NROS_CPP_RET_TRANSPORT_ERROR = -100 is the C++ result code that
NROS_TRY_RET propagates from main(); on POSIX this becomes
(unsigned char)-100 = 156 as the process exit code. Treat
“exited 156 after starting” as the C++ equivalent of B1.
B4. Override the locator at runtime
When a native talker can’t reach the daemon, override the locator with the canonical env var:
NROS_LOCATOR=tcp/192.168.1.50:7447 ./build/c_talker
# Legacy alias (still accepted): ZENOH_LOCATOR=… ./build/c_talker
ROS_DOMAIN_ID=7 ./build/c_talker # also overridable
The native Rust / C / C++ talkers all read NROS_LOCATOR first, fall
back to ZENOH_LOCATOR, then to the build-time default. Embedded
targets have no runtime env — their locator is compile-baked from the
deploy config ([package.metadata.nros.deploy.<t>] /
the package.xml tuple), so a rebuild is the override.
B5. Binary exits immediately, no error printed
Buffering: setvbuf(stdout, NULL, _IOLBF, 0) if you piped the run.
POSIX terminals flush on newline; piped stdout full-buffers and
may eat short outputs. Add a RUST_LOG=info (Rust) or unbuffer
the C / C++ output (stdbuf -oL).
C. ROS 2 side sees nothing
C1. RMW mismatch
# On the ROS 2 side, default rmw_fastrtps_cpp will NOT see nano-ros:
export RMW_IMPLEMENTATION=rmw_zenoh_cpp # for Zenoh
export RMW_IMPLEMENTATION=rmw_cyclonedds_cpp # for Cyclone
C2. QoS mismatch — echo silent, list sees the topic
ros2 topic list # /chatter shown
ros2 topic echo /chatter # … nothing
nano-ros publishers default to BEST_EFFORT; stock
ros2 topic echo defaults to RELIABLE. The QoS-mismatched
subscriber is created but receives no data. Force best-effort on
the echo:
ros2 topic echo /chatter std_msgs/msg/String --qos-reliability best_effort
D. Doctor + last-resort
D1. Run the doctor first
nros doctor
It prints fixit hints for the toolchain you actually need. (A just-free platform-scoped doctor spelling does not exist today; contributors with an in-tree checkout can scope via the platform recipes.)
D2. [PATH] nros built but not on PATH
The doctor now reports this distinct from [MISSING] when the
binary is built at packages/cli/target/release/nros (or the
transitional ${NROS_HOME:-~/.nros}/bin/nros) but PATH doesn’t see
it. Activate the workspace — it wires PATH:
source ./activate.sh # bash / zsh
# OR
source ./activate.fish # fish
# OR
direnv allow # auto-activates on `cd nano-ros`
Don’t loop on the contributor recipe just workspace cargo-tools — that re-runs the
build which short-circuits on the same PATH miss.
D3. Full sweep (slow)
nros doctor
nros setup --check
Only run this when you’re standing up every supported platform in one go. It walks the provisioning checks and can take a few minutes.
D4. File an issue
When all else fails, include:
- the exact command you ran,
- the full stderr,
rustc --version,cmake --version,qemu-system-arm --version,nros --version.
What success looks like
A correctly-running Rust Linux talker
(examples/native/rust/talker) prints something like this on
stderr (with RUST_LOG=info):
[INFO talker] Publishing: 'Hello World: 1'
[INFO talker] Publishing: 'Hello World: 2'
[INFO talker] Publishing: 'Hello World: 3'
(That is the full expected output — the talker logs one line per publish, nothing else.)
A correctly-running C talker (examples/native/c/talker) prints
on stdout:
Publishing: 'Hello World: 1'
Publishing: 'Hello World: 2'
Publishing: 'Hello World: 3'
A correctly-running C++ talker prints the same
Publishing: 'Hello World: N' line once per second.
The ROS 2 side (ros2 topic echo /chatter std_msgs/msg/String --qos-reliability best_effort with
RMW_IMPLEMENTATION=rmw_zenoh_cpp) should see:
data: 'Hello World: 1'
---
data: 'Hello World: 2'
---
data: 'Hello World: 3'
---
If you see all three of these — talker logging, ROS 2 echo output, and matching counter values — interop is verified end-to-end.
See also
- Install + first build — full setup walkthrough
- First Node — Rust — the canonical Rust starter
- Troubleshooting — broader issue-by-issue reference for post-first-build problems
First Node — Rust (Linux)
Build, run, and verify a single nano-ros publisher node on Linux in
about ten minutes. Uses the canonical Zenoh backend. The zenoh router comes from ROS 2
(ros2 run rmw_zenoh_cpp rmw_zenohd) — nano-ros no longer ships one.
No ROS 2 on this machine? Use --rmw cyclonedds instead: it needs no
router at all (see Choosing an RMW).
Stuck? See Troubleshooting — First 10 Minutes for the common first-build errors.
Prereqs
Pick one path from a fresh checkout — just is NOT a prereq.
A. Front door (bare machine OK — no Rust, no just):
./scripts/bootstrap.sh
Installs rustup if needed and builds the in-tree nros CLI from
source at packages/cli/target/release/nros, leaving it on PATH for
this shell (nano-ros is a source distribution — no prebuilt nros).
B. Already have cargo (equivalent — same build, same binary):
git submodule update --init packages/cli/third-party/play_launch
cargo build --release --manifest-path packages/cli/Cargo.toml --bin nros
export PATH="$PWD/packages/cli/target/release:$PATH"
Every subsequent shell sources the workspace env via one of:
direnv allow # if you use direnv
source ./activate.sh # bash / zsh
source ./activate.fish # fish
Then provision the native host (installs the zenoh client stack into a shared store; the router itself comes from your ROS 2 install):
nros setup native --rmw zenoh
See Install + first build (Linux) for more.
Project layout
The talker is a standalone Cargo package that pulls nano-ros in via a path dependency. Three files matter:
examples/native/rust/talker/
├── Cargo.toml # registry-style deps + `nros sync` patch table
├── package.xml # ROS-style manifest (drives codegen tooling)
├── generated/ # auto-generated message bindings (gitignored)
└── src/
├── main.rs # one line: nros::main!(spin = "forever")
└── lib.rs # the ~65-line talker node body
Native (host) talkers read the locator / domain from environment variables
(NROS_LOCATOR — legacy alias ZENOH_LOCATOR — and ROS_DOMAIN_ID)
— no config file is needed. Embedded targets bake their config from
[package.metadata.nros.deploy.<target>] instead — the shape shows up
under the Embedded Starters section.
The Cargo.toml is the contract that wires nano-ros into your
package. Every example is its own standalone Cargo root (an empty
[workspace] table stops cargo walking up the filesystem), and
nano-ros crates are declared registry-style (version = "*");
the example’s tracked .cargo/config.toml carries an auto-managed
[patch.crates-io] block (written by nros sync) that resolves them
into a nano-ros checkout. Verbatim from
examples/native/rust/talker/Cargo.toml
(trimmed to the docs-relevant fields — the in-tree file also exposes
rmw-cyclonedds / rmw-xrce features for the multi-RMW build path):
[package]
name = "native-rs-talker"
version = "0.1.0"
edition = "2024"
license = "MIT OR Apache-2.0"
publish = false
[[bin]]
name = "talker"
path = "src/main.rs"
[features]
default = ["rmw-zenoh"]
rmw-zenoh = ["dep:nros-rmw-zenoh", "nros-board-linux/rmw-zenoh"]
[dependencies]
nros = { version = "*", default-features = false,
features = ["std", "alloc", "env", "rmw-cffi", "macros"] }
nros-platform-cffi = { version = "*", features = ["posix-c-port"] }
nros-board-linux = { version = "*", default-features = false }
nros-rmw-zenoh = { version = "*", default-features = false,
features = ["std", "platform-posix", "ros-humble"],
optional = true }
# The ONE non-registry dep: your own generated message crate, by path.
std_msgs = { path = "generated/std_msgs", default-features = false }
log = "0.4"
env_logger = "0.11"
[workspace]
Copying this out of the workspace? That is the intended workflow —
nano-ros crates are not published to crates.io, so the version = "*"
deps resolve through the [patch.crates-io] block in
.cargo/config.toml, and nros sync rewrites that block (plus the
generated message crates) for wherever the directory lives now:
cp -r examples/native/rust/talker ~/my-talker && cd ~/my-talker
NROS_REPO_DIR=/path/to/nano-ros nros sync
cargo build && RUST_LOG=info cargo run
Prefer vendoring instead? examples/templates/multi-package-workspace/
documents the path-dep workspace layout.
Configure
Three runtime knobs, each an env override on a built-in default:
| Knob | Default | Env override |
|---|---|---|
| Zenoh locator | tcp/127.0.0.1:7447 | NROS_LOCATOR (legacy alias: ZENOH_LOCATOR) |
| ROS domain ID | 0 | ROS_DOMAIN_ID |
| Zenoh mode | client | NROS_SESSION_MODE (legacy alias: ZENOH_MODE) |
No config file on native. Embedded targets bake these from
[package.metadata.nros.deploy.<target>] instead — see the
Configuration Guide.
Build
cd examples/native/rust/talker
nros sync # once per checkout location: writes the generated/
# message bindings + the [patch.crates-io] table
cargo build # or: cargo build --release
First build pulls dependencies (~3 minutes). Re-builds finish in seconds.
Run
Three terminals (each command below blocks; keep them open):
# Terminal 1 — zenoh router (ROS 2's own). Blocks the shell until Ctrl-C.
source /opt/ros/humble/setup.bash
ros2 run rmw_zenoh_cpp rmw_zenohd # or: just zenohd
# Terminal 2 — the talker. The talker logs via `log::info!`, so set
# RUST_LOG=info — without it `env_logger` only shows errors and the
# `Publishing:` lines stay hidden.
cd examples/native/rust/talker
RUST_LOG=info cargo run
# Expected output (on stderr):
# [INFO talker] Publishing: 'Hello World: 1'
# [INFO talker] Publishing: 'Hello World: 2'
# …
That’s the nano-ros side fully working. Optional step: verify interop with stock ROS 2.
# Terminal 3 — stock ROS 2 with rmw_zenoh_cpp. It talks to the SAME
# router already running in terminal 1 (default tcp/127.0.0.1:7447).
source /opt/ros/humble/setup.bash
export RMW_IMPLEMENTATION=rmw_zenoh_cpp
# Talker publishes best-effort; stock `ros2 topic echo` defaults to
# RELIABLE, so the QoS-mismatched echo silently delivers nothing.
# Force best-effort to receive:
ros2 topic echo /chatter std_msgs/msg/String --qos-reliability best_effort
If ros2 topic echo shows no output despite the talker printing
Publishing:, the routers aren’t peering — confirm both processes
point at the same port (default tcp/127.0.0.1:7447).
Readiness signal. Within ~6 seconds of RUST_LOG=info cargo run
(session open + the first 1 s timer tick), the talker should print
Publishing: 'Hello World: 1' — the count starts at 1, matching the
official ROS 2 demo_nodes_cpp talker. If no Publishing: line in 30
seconds:
- Confirm
RUST_LOGis set. WithoutRUST_LOG=info(ordebug),env_loggerfilters out thePublishing:lines and the run looks silent even when it’s working. - Confirm the router is running (terminal 1). Without it, the talker
blocks on
Executor::openindefinitely. - Re-run with
RUST_LOG=debug cargo runand look for “RMW session open failed” — usually a wrong locator or wrong port. - See Troubleshooting — First 10 Minutes.
GitHub source
Canonical, copy-out:
examples/native/rust/talker/
Copy the directory, run NROS_REPO_DIR=<nano-ros checkout> nros sync
inside it, rename the package, and your starter is ready to modify.
Next
- Add a subscription:
examples/native/rust/listener/ - Generate bindings for custom
.msg/.srv/.actionfiles: Message Generation - Cross-compile for an RTOS: pick the right Embedded Starter from the next section (FreeRTOS / Zephyr / NuttX / ThreadX / ESP32 / Bare-metal Cortex-M3).
First Node — C (Linux)
Build, run, and verify a single nano-ros publisher node on Linux from
C. Uses CMake, the Zenoh backend, and add_subdirectory consumption.
Stuck? See Troubleshooting — First 10 Minutes for the common first-build errors.
Prereqs
Pick one path from a fresh checkout — just is NOT a prereq.
A. Front door (bare machine OK — no Rust, no just):
./scripts/bootstrap.sh
Installs rustup if needed and builds the in-tree nros CLI from
source at packages/cli/target/release/nros, leaving it on PATH for
this shell (nano-ros is a source distribution — no prebuilt nros).
B. Already have cargo (equivalent — same build, same binary):
git submodule update --init packages/cli/third-party/play_launch
cargo build --release --manifest-path packages/cli/Cargo.toml --bin nros
export PATH="$PWD/packages/cli/target/release:$PATH"
Every subsequent shell sources the workspace env via one of:
direnv allow # if you use direnv
source ./activate.sh # bash / zsh
source ./activate.fish # fish
Then provision the native host (installs the zenoh client stack into a
shared store; the router itself comes from your ROS 2 install —
ros2 run rmw_zenoh_cpp rmw_zenohd):
nros setup native --rmw zenoh
See Install + first build (Linux) for more.
Project layout
The talker is a standalone CMake project that pulls nano-ros via
add_subdirectory(<repo-root>). Four files matter:
examples/native/c/talker/
├── CMakeLists.txt # add_subdirectory + targets
├── package.xml # ROS-style manifest (drives codegen tooling)
└── src/
└── main.c # ~100-line talker
On native, locator + domain come from env vars (NROS_LOCATOR,
ROS_DOMAIN_ID) with built-in defaults; embedded targets bake them at
build time via the package.xml <nano_ros deploy=… rmw=…/> tuple (see
Configuration).
The CMake preamble matches the canonical example shape in
examples/native/c/talker/CMakeLists.txt —
five set(...) lines plus the per-target link:
cmake_minimum_required(VERSION 3.22)
project(my_talker LANGUAGES C)
set(CMAKE_C_STANDARD 11)
set(CMAKE_C_STANDARD_REQUIRED ON)
# Pull nano-ros in. Adjust the relative path to your repo root.
# `NROS_RMW` is the user-facing cache var (overridable via
# `-DNROS_RMW=<rmw>`); the example forwards it to `NANO_ROS_RMW`,
# the var the nano-ros add_subdirectory reads.
set(NANO_ROS_PLATFORM posix)
set(NROS_RMW "zenoh" CACHE STRING
"Active RMW (zenoh|xrce|cyclonedds) — selects the backend linked into my_talker.")
set(NANO_ROS_RMW "${NROS_RMW}")
add_subdirectory(<rel-path-to-nano-ros> nano_ros)
# Generate C bindings for std_msgs (transitively depends on
# builtin_interfaces; the dependency is declared explicitly).
nros_generate_interfaces(builtin_interfaces SKIP_INSTALL)
nros_generate_interfaces(std_msgs DEPENDENCIES builtin_interfaces
SKIP_INSTALL)
add_executable(my_talker src/main.c)
target_link_libraries(my_talker PRIVATE
std_msgs__nano_ros_c
NanoRos::NanoRos)
nros_platform_link_app(my_talker)
nros_platform_link_app(my_talker) transitively wires the active
RMW’s strong nros_app_register_backends() stub (calling
nros_rmw_zenoh_register() for the zenoh build above). On POSIX you
do not call nano_ros_link_rmw() explicitly — the platform
module handles it.
The C entry point is int nros_app_main(int argc, char **argv)
(not main); <nros/app_main.h> provides the OS-side main
stub that wires signal handling and forwards to your function.
#include <nros/app_main.h>
#include <nros/init.h>
#include <nros/executor.h>
#include <nros/node.h>
#include <nros/publisher.h>
#include <nros/timer.h>
#include "std_msgs.h"
int nros_app_main(int argc, char** argv) {
// 1. nros_support_init() — opens the zenoh session
// 2. nros_executor_init() / nros_node_init()
// 3. std_msgs_msg_int32_publisher_init() — typed publisher
// 4. nros_timer_init() with a 1 Hz period + publish callback
// 5. nros_executor_spin() until SIGINT
}
Configure
Three runtime knobs:
| Knob | Default | Env override |
|---|---|---|
| Zenoh locator | tcp/127.0.0.1:7447 | NROS_LOCATOR |
| ROS domain ID | 0 | ROS_DOMAIN_ID |
| Node name | talker | hard-coded in source |
On an embedded target the same knobs are compile-baked from the
the package.xml <export><nano_ros deploy="<t>" rmw="<rmw>"/> tuple
call in CMakeLists.txt (see
Configuration).
Build
cd examples/native/c/talker
cmake -B build
cmake --build build
No nros sync here — a C leaf’s message bindings are generated inside
CMake, and unlike the Rust leaves it carries no .cargo/config.toml to
resolve. See
Workflow by Platform and Language.
The first configure pulls and builds nano-ros’s Rust staticlibs (~3 minutes). Re-builds finish in seconds.
Run
Three terminals.
# 1. Start the zenoh router (ROS 2's own):
source /opt/ros/humble/setup.bash
ros2 run rmw_zenoh_cpp rmw_zenohd # or: just zenohd
# 2. Run the talker:
cd examples/native/c/talker
./build/c_talker
# Expected output:
# Publishing: 'Hello World: 1'
# Publishing: 'Hello World: 2'
# Publishing: 'Hello World: 3'
# …
# 3. Verify from stock ROS 2:
source /opt/ros/humble/setup.bash
export RMW_IMPLEMENTATION=rmw_zenoh_cpp
# Talker publishes best-effort; stock `ros2 topic echo` defaults to
# RELIABLE, so the QoS-mismatched echo silently delivers nothing.
# Force best-effort to receive:
ros2 topic echo /chatter std_msgs/msg/String --qos-reliability best_effort
Readiness signal. Within ~6 seconds of ./build/c_talker (session
open + the first 1 s timer tick), the binary should print
Publishing: 'Hello World: 1' on stdout — Rust + C + C++ all start
the count at 1, matching the official ROS 2 demo talker. If no
Publishing: line in 30 seconds:
- Confirm the router is running (terminal 1). Without it,
nros_support_initreturns immediately with-4(NROS_RET_NOT_FOUND— connection refused). - Wrong locator / unreachable host → same
-4signature in stderr. Reachable host but mismatched port → talker hangs on session-open handshake rather than returning a code. - See Troubleshooting — First 10 Minutes.
GitHub source
Canonical, copy-out:
examples/native/c/talker/
Next
- Add a subscription:
examples/native/c/listener/ - Service / action shapes:
service-client/,action-client/ - Custom
.msg/.srv/.action: Message Generation - Cross-compile for an RTOS: pick the right Embedded Starter from the next section.
First Node — C++ (Linux)
Build, run, and verify a single nano-ros publisher node on Linux from
C++14. Uses CMake, the Zenoh backend, and add_subdirectory
consumption.
Stuck? See Troubleshooting — First 10 Minutes for the common first-build errors.
Prereqs
Pick one path from a fresh checkout — just is NOT a prereq.
A. Front door (bare machine OK — no Rust, no just):
./scripts/bootstrap.sh
Installs rustup if needed and builds the in-tree nros CLI from
source at packages/cli/target/release/nros, leaving it on PATH for
this shell (nano-ros is a source distribution — no prebuilt nros).
B. Already have cargo (equivalent — same build, same binary):
git submodule update --init packages/cli/third-party/play_launch
cargo build --release --manifest-path packages/cli/Cargo.toml --bin nros
export PATH="$PWD/packages/cli/target/release:$PATH"
Every subsequent shell sources the workspace env via one of:
direnv allow # if you use direnv
source ./activate.sh # bash / zsh
source ./activate.fish # fish
Then provision the native host (installs the zenoh client stack into a
shared store; the router itself comes from your ROS 2 install —
ros2 run rmw_zenoh_cpp rmw_zenohd):
nros setup native --rmw zenoh
See Install + first build (Linux) for more.
Project layout
The talker is a standalone CMake project that pulls nano-ros via
add_subdirectory(<repo-root>). Two files matter:
examples/native/cpp/talker/
├── CMakeLists.txt # add_subdirectory + targets
└── src/
└── main.cpp # ~70-line talker
Native (host) talkers read the locator + domain from arguments passed to
nros::init(...); no config file is needed. Embedded variants under
examples/<plat>/cpp/talker/ bake theirs from the
package.xml <nano_ros deploy=…/> tuple.
The CMakeLists is the ament shape you already know — this is the
complete canonical file at
examples/native/cpp/talker/CMakeLists.txt,
verbatim (RFC-0048: nano_rosConfig.cmake at the checkout root makes
find_package(nano_ros) resolve source-side, and the message-package
Find-stubs make find_package(std_msgs) generate bindings on the fly):
cmake_minimum_required(VERSION 3.22)
project(cpp_talker LANGUAGES C CXX)
set(CMAKE_CXX_STANDARD 14)
set(CMAKE_CXX_STANDARD_REQUIRED ON)
find_package(nano_ros REQUIRED)
find_package(std_msgs REQUIRED)
nano_ros_add_executable(cpp_talker src/main.cpp)
ament_target_dependencies(cpp_talker std_msgs)
install(TARGETS cpp_talker DESTINATION lib/${PROJECT_NAME})
ament_package()
LANGUAGES C CXX (not CXX alone): the per-target register stub
is a C translation unit, so C must be enabled in this directory scope
or the link fails. Backend selection is a configure-line cache var
(-DNROS_RMW=zenoh|xrce|cyclonedds), not CMake code.
The C++ entry point is int nros_app_main(int argc, char** argv)
(same as C); <nros/app_main.h> provides the OS-side main stub.
The body uses typed nros::Publisher<M> / nros::Subscription<M>
wrappers over the C ABI:
#include <nros/app_main.h>
#include <nros/nros.hpp>
#include "std_msgs.hpp"
#define NROS_TRY_LOG(file, line, expr, ret) \
std::fprintf(stderr, "[nros] %s:%d %s -> %d\n", file, line, expr, (int)ret)
int nros_app_main(int argc, char** argv) {
NROS_TRY_RET(nros::init("tcp/127.0.0.1:7447", 0), 1);
nros::Node node;
NROS_TRY_RET(nros::create_node(node, "talker"), 1);
nros::Publisher<std_msgs::msg::String> pub;
NROS_TRY_RET(node.create_publisher(pub, "/chatter"), 1);
// ... register a timer + spin
}
NROS_TRY_RET short-circuits on any non-OK return code and logs the
expression that failed. Define NROS_TRY_LOG once (any sink — here
std::fprintf) and reuse it across every call site.
Configure
Three runtime knobs:
| Knob | Default | Override |
|---|---|---|
| Zenoh locator | tcp/127.0.0.1:7447 | First arg to nros::init |
| ROS domain ID | 0 | Second arg to nros::init |
| Node name | talker | First arg to nros::create_node |
Reading from env in C++ is std::getenv("NROS_LOCATOR") plus the
same nros::init call — see the GitHub source for the full pattern.
Build
cd examples/native/cpp/talker
cmake -B build # with ./activate.sh sourced, nano_ros_ROOT is
# exported; a copy-out passes
# -Dnano_ros_ROOT=<path-to-nano-ros> explicitly
cmake --build build
First configure builds nano-ros’s Rust staticlibs (~3 minutes). Re-builds finish in seconds.
Run
Three terminals.
# 1. zenoh router (ROS 2's own):
source /opt/ros/humble/setup.bash
ros2 run rmw_zenoh_cpp rmw_zenohd # or: just zenohd
# 2. Run the talker:
cd examples/native/cpp/talker
./build/cpp_talker
# Expected:
# nros C++ Talker
# ===================
# Node created: talker
#
# Publishing messages (Ctrl+C to exit)...
#
# Publishing: 'Hello World: 1'
# Publishing: 'Hello World: 2'
# …
# 3. Verify from stock ROS 2:
source /opt/ros/humble/setup.bash
export RMW_IMPLEMENTATION=rmw_zenoh_cpp
# Talker publishes best-effort; stock `ros2 topic echo` defaults to
# RELIABLE, so the QoS-mismatched echo silently delivers nothing.
# Force best-effort to receive:
ros2 topic echo /chatter std_msgs/msg/String --qos-reliability best_effort
Readiness signal. Within ~6 seconds of ./build/cpp_talker
(session open + the first 1 s timer tick), the binary should print
Publishing: 'Hello World: 1' — Rust + C + C++ all start the count
at 1, matching the official ROS 2 demo talker. If no Publishing:
line in 30 seconds:
- Confirm the router is running (terminal 1). Without it,
nros::initreturns-100(TransportError) — theNROS_TRY_RETmacro logs the failed call to stderr. - Check stderr for
[nros] …/main.cpp:LINE nros::init(...) -> -Ndiagnostics.-3/-100both indicate transport open failed. - See Troubleshooting — First 10 Minutes.
GitHub source
Canonical, copy-out:
examples/native/cpp/talker/
Next
- Add a subscription:
examples/native/cpp/listener/ - Services + actions:
service-client/,action-client/ - Parameters:
parameters/ - Custom
.msg/.srv/.action: Message Generation - Cross-compile for an RTOS: pick the right Embedded Starter from the next section.
Choosing an RMW
Every nano-ros binary picks its RMW (ROS Middleware) backend at build time. There are three: Cyclone DDS, Zenoh (zenoh-pico), and XRCE-DDS. This page helps you pick one; the mechanics of changing your pick live in Switching RMW Backends, and the per-backend deep dive (wire behavior, per-platform network configuration) is in RMW Backends.
Start with Cyclone DDS
For getting started, use Cyclone DDS. It is the one backend that
needs no daemon, router, or agent: a nano-ros node publishes with
nothing else running, and a stock ROS 2 node using rmw_cyclonedds_cpp
sees it directly — same RTPS wire protocol, same discovery, no
translation layer in between.
The reason the other two are not the starting point: nano-ros ships no
bridge process of its own. The zenoh router is ROS 2’s own
rmw_zenohd (see
RFC-0075),
so the zenoh backend requires a ROS 2 installation just to start the
router; the XRCE backend requires a running Micro-XRCE-DDS Agent.
Cyclone DDS requires neither.
Current maturity: pub/sub and services interoperate with stock
rmw_cyclonedds_cpp; status events (liveliness, deadline-miss) are not
wired to Cyclone listeners yet. The full list is in
docs/reference/cyclonedds-known-limitations.md.
Zenoh — ROS 2 interop and small embedded targets
Choose the Zenoh backend (built on zenoh-pico) when:
- You are integrating with a ROS 2 fleet running
rmw_zenoh_cpp. - Your target is a small MCU: ~16 KB of heap and ~100 KB of flash are enough, versus ~32 KB+ RAM for Cyclone DDS.
- Your network cannot do UDP multicast (RTPS discovery needs IGMP; zenoh-pico in client mode runs over plain TCP).
The zenoh backend needs a router, and the router comes from your ROS 2
installation — it is the same rmw_zenohd that ROS 2 zenoh nodes use:
ros2 run rmw_zenoh_cpp rmw_zenohd
That invocation works on any host with rmw_zenoh_cpp installed,
regardless of where ROS 2 lives — never hard-code an install path to
the router binary. See ROS 2 Interop
for the full three-terminal walkthrough.
XRCE-DDS — smallest footprint, serial transport
Choose the XRCE-DDS backend when:
- RAM is the constraint: ~3 KB of client RAM, fully static allocation, no heap required on the MCU.
- You need a serial (UART) transport — the only backend that works on an MCU with no networking hardware. See Serial Transport.
- You are joining an existing micro-ROS / DDS deployment: the agent bridges to any DDS-based RMW.
The MCU cannot participate without the Micro-XRCE-DDS Agent
(MicroXRCEAgent) running on a host; if the agent stops, the device
loses all connectivity.
Capability matrix
| Aspect | Cyclone DDS (cyclonedds) | Zenoh (zenoh) | XRCE-DDS (xrce) |
|---|---|---|---|
| Client RAM | ~32 KB+ (heap required) | ~16 KB+ (heap required) | ~3 KB (fully static) |
| Client Flash | ~150 KB+ | ~100 KB+ | ~75 KB |
| Bridge process | None — RTPS multicast directly | rmw_zenohd router (from your ROS 2 install) | Agent (protocol translator, mandatory) |
| Peer-to-peer | Yes (RTPS native) | Not as shipped — zenoh-pico’s multicast/scouting is compiled out | No (agent always required) |
| Discovery | SPDP / SEDP on UDP multicast or static peer list | Client participates (liveliness tokens) | Agent handles on behalf |
| Entity creation | Client creates directly | Client creates directly | Client requests, agent creates |
| Transports | UDP unicast + multicast (RTPS) | TCP, UDP, TLS | UDP, serial (HDLC framing) |
| Heap allocation | Required (Cyclone uses malloc) | Required (C-level) | None |
| Implementation | C++ wrapper over upstream Cyclone DDS C | Rust + zenoh-pico C | Rust + Micro-XRCE-DDS-Client C |
| ROS 2 interop | Direct against rmw_cyclonedds_cpp (same upstream version) | Via rmw_zenoh_cpp + its router | Via agent + any DDS RMW |
| Failure mode | Peer goes offline = its samples stop arriving | Router crash = lose routing | Agent crash = lose connectivity |
Notes on two rows: zenoh’s router-free peer mode exists upstream but nano-ros compiles zenoh-pico without the multicast transport and scouting, so every deployment reaches its peers through the router; and the upstream Micro-XRCE-DDS client defines a CAN FD transport, but nano-ros builds do not enable it.
Decision summary
- Default / getting started / DDS fleet → Cyclone DDS. Nothing to
run beside your node; direct
rmw_cyclonedds_cppinterop. Needs ~32 KB RAM and a network stack with IGMP. rmw_zenoh_cppfleet, small MCU, or TCP-only network → Zenoh. Needs the router from your ROS 2 install (ros2 run rmw_zenoh_cpp rmw_zenohd).- Tiny MCU, serial link, or micro-ROS deployment → XRCE-DDS. Smallest footprint, zero heap, but the agent is mandatory.
You do not have to pick exactly one per system: a single binary can link multiple backends and bridge between them — see Cross-backend Bridges.
Once you have picked, Switching RMW Backends shows the exact edits per build system.
Switching RMW Backends
nano-ros selects its RMW backend at build time — there is no
RMW_IMPLEMENTATION environment variable to flip at runtime the way
desktop ROS 2 does. Which edit you make depends on how your project is
built. This page lists the exact change per builder; for help picking a
backend in the first place, see Choosing an RMW.
Backend names are the same everywhere: zenoh, xrce, cyclonedds.
When nothing selects a backend, the default is zenoh.
C/C++ workspace (CMake)
One edit. The root CMakeLists.txt of a nano-ros workspace calls
nano_ros_workspace(...); change its BACKEND argument:
nano_ros_workspace(
ORDER_FROM_DEPENDS
BACKEND cyclonedds # zenoh | xrce | cyclonedds
PLATFORM "${NANO_ROS_PLATFORM}"
SUBDIRS ${_ws_subdirs})
Then reconfigure and rebuild — preferably into a fresh build directory, so no object files from the previous backend linger.
Note: when the root hard-codes BACKEND, passing -DNROS_RMW=... on
the configure line is silently overridden — nano_ros_workspace()
stamps the workspace-wide RMW from its BACKEND argument, and the
cache variable loses without a warning. Edit the BACKEND argument;
do not fight it from the command line.
Single C/C++ package (no workspace function)
A standalone package that consumes nano-ros in source form — either
find_package(nano_ros REQUIRED) (the ament-shape entry point,
nano_rosConfig.cmake at the checkout root, located via
nano_ros_ROOT; RFC-0048) or a raw add_subdirectory — for
example a copied-out leaf like examples/native/c/talker/ — picks its
backend on the configure line:
cmake -S . -B build -DNROS_RMW=cyclonedds # zenoh | xrce | cyclonedds
cmake --build build
Use one build directory per backend (build-zenoh/, build-cyclonedds/,
…) if you switch back and forth.
Rust workspace
Today this is three edits that must agree. (A scaffold flag that
bakes all three from one value is planned; until then, treat this as a
checklist.) The example below switches to cyclonedds:
| # | File | Edit |
|---|---|---|
| 1 | src/<bringup_pkg>/system.toml | [system] table: rmw = "cyclonedds" |
| 2 | Entry package Cargo.toml — the board dependency | nros-board-linux = { path = "...", default-features = false, features = ["rmw-cyclonedds"] } |
| 3 | Entry package Cargo.toml — the nros dependency | add "rmw-cyclonedds" to its features list |
Why three places:
system.tomlis the declared, language-agnostic selection — it is what the toolchain and deploy tooling read. See Configuration.- The board crate is what actually links the backend. On the host
board,
nros-board-linux’s features arermw-zenoh(the default),rmw-xrce, andrmw-cyclonedds; each pulls in the corresponding backend crate and registers it during boot. Switching away from zenoh therefore needsdefault-features = falseplus the feature you want. - The
nrosfacade featurermw-cycloneddsis the type-descriptor gate: Cyclone DDS resolves topic types through a runtime descriptor registry, and this feature turns on the hook that registers a message’s descriptor before the topic is created. Without it,create_publisherfails at runtime because the descriptor was never registered. (Zenoh and XRCE do not need a facade feature — steps 1 and 2 are enough for them.)
After editing, re-run nros sync in the workspace and rebuild.
The full-system declaration model — how the one value in system.toml
is lowered to cargo features and CMake cache variables — is specified in
RFC-0031.
Embedded targets
On embedded platforms the backend is fixed at compile time like
everything else — there is nothing to switch on the device. Each
platform surfaces the same choice through its own configuration
mechanism; on Zephyr, for instance, it is a Kconfig choice
(CONFIG_NROS_RMW_ZENOH / CONFIG_NROS_RMW_XRCE /
CONFIG_NROS_RMW_CYCLONEDDS, exactly one selected).
See the platform guides for the concrete per-platform workflow:
and Workflow by Platform and Language for the overview of which builder applies where.
RMW Backends: Zenoh, XRCE-DDS, Cyclone DDS
Capability truth (services / events / QoS per backend) is the generated Per-RMW Feature Matrix; version pins live on Support Status. This page is the architectural narrative.
nano-ros supports three RMW (ROS Middleware) backends for connecting embedded devices to a ROS 2 network. Each backend targets different deployment scenarios and resource constraints. Each Node picks its backend at build time; one binary can link multiple backends and bridge between them — see Cross-backend Bridges for the multi-RMW pattern.
Zenoh (rmw-zenoh)
The Zenoh backend uses zenoh-pico, a lightweight C client for the Zenoh protocol. The MCU participates directly in the Zenoh network – there is no protocol translation layer.
How it works:
- The MCU runs zenoh-pico in client mode, connecting to a zenoh router (
rmw_zenohd, ROS 2’s own) over TCP, UDP, or TLS. - zenoh-pico creates publishers and subscribers directly on the Zenoh network.
- ROS 2 nodes running
rmw_zenoh_cppconnect to the same router, enabling transparent interop. - zenoh itself defines a peer mode in which two zenoh-pico devices talk directly with no
router. nano-ros does not ship it. The shim is compiled without zenoh-pico’s multicast
transport and scouting (
nros_zpico_build::MULTICAST_TRANSPORT = false) because those are three more code paths in a library whose job here is fitting on an MCU, and every nano-ros deployment reaches its peers through a router or an agent.NROS_SESSION_MODE=peeris therefore refused at session open with a message saying so — see issue 0682. Enabling it means flipping that constant, rebuilding the shim, and accepting the footprint.
Key characteristics:
- Client/router topology; zenoh’s router-free peer mode is compiled out (see above)
- The router forwards messages without interpreting them – it is not a protocol translator
- If the router crashes, peers in client mode lose routing but the MCU continues running
- Full ROS 2 graph discovery via liveliness tokens
- Transport options: TCP, UDP, TLS (via
zpico-smoltcpor platform sockets)
XRCE-DDS (rmw-xrce)
The XRCE-DDS backend uses Micro-XRCE-DDS-Client, the same client library used by micro-ROS. It follows an agent-based model where a lightweight client on the MCU delegates entity creation to an agent process.
How it works:
- The MCU runs the XRCE-DDS client, connecting to an Agent process over UDP or serial (UART).
- The client sends requests to the Agent: “create a publisher on topic X with type Y.”
- The Agent creates full DDS entities on behalf of the client and bridges data between the XRCE protocol and the DDS data space.
- ROS 2 nodes using any DDS-based RMW (FastDDS, Cyclone DDS) communicate through the Agent.
Key characteristics:
- Agent is mandatory – the MCU cannot participate in the network without it
- If the Agent crashes, the MCU loses all connectivity
- Fully static memory allocation on the MCU (no heap required)
- Client-side discovery is not supported; the Agent handles it
- Transport options: UDP, serial (HDLC framing)
Cyclone DDS (rmw-cyclonedds)
Maturity status. Cyclone DDS supports pub/sub + service round-trips on the native (host) build (full service implementation in
src/service.cpp); status events (liveliness, deadline-miss, etc.) are NOT wired to Cyclone listeners yet, and the service-availability probe is deferred (service_server_availablereturnsNROS_RMW_RET_UNSUPPORTED— poll-with-retry instead of waiting on it). Wire-level interop with stockrmw_cyclonedds_cpp(Humble) works for topics and services. Full known-limitations list:docs/reference/cyclonedds-known-limitations.md.
The Cyclone DDS backend uses Eclipse Cyclone DDS, the same DDS implementation that ROS 2 ships with via rmw_cyclonedds_cpp. Built as a standalone C++ library at packages/rmw/cyclonedds/nros-rmw-cyclonedds/ that registers itself with the runtime through the C ABI vtable in nros-rmw-cffi.
How it works:
- The application (or platform) calls
nros_rmw_cyclonedds_register()once beforenros::init()— this is automatic when the consumer’s CMake build sets-DNANO_ROS_RMW=cyclonedds. - The runtime stores the vtable pointer; subsequent
nros::init()calls dispatch through it for session, publisher, subscriber, and service operations. dds_create_domain/dds_create_participant/dds_create_topic/dds_create_writer/dds_create_readerare invoked under the hood; Cyclone owns its own RX threads.- ROS 2 nodes using stock
rmw_cyclonedds_cppinteroperate directly — same wire protocol, same discovery, no key rewriting (unlikermw_zenoh’s<domain>/<topic>/<type>/...scheme).
Key characteristics:
- Pure-C++ backend (not a Cargo crate) — Autoware contributors can read and extend the wrapper using the same patterns as
actuation_module/include/common/dds/. - ROS 2 Tier-1 wire compat — pinned to Cyclone DDS tag
0.10.5to matchros-humble-cyclonedds0.10.5 +ros-humble-rmw-cyclonedds-cpp1.3.4. - Static
ddsi_configviadds_create_domain_with_rawconfigskips the XML parser; embedded-friendly. - Discovery via SPDP multicast or unicast peer list (mirrors Cyclone’s standard config knobs).
- Heap required (Cyclone uses
malloc);BUILD_SHARED_LIBS=ONproduceslibddsc.sofor POSIX, static link for embedded. - Services wired (create/recv/reply —
service.cpp); the availability probe (service_server_available) and actions are still unsupported. - No status events yet —
register_subscription_event/register_publisher_event/assert_publisher_livelinessslots are not wired to Cyclone listeners yet.
Build:
Contributors: the in-tree Cyclone DDS build/test lanes are in Per-Platform Contributor Lanes.
Each example picks its RMW via -DNANO_ROS_RMW=cyclonedds at
configure time; the root CMakeLists.txt add_subdirectory’s
packages/rmw/cyclonedds/nros-rmw-cyclonedds/ and links the resulting target
into NanoRos::NanoRos. No build/install/ prefix, no
find_package(NrosRmwCyclonedds) deleted both.
Availability follows from config — the build provisions Cyclone.
Selecting cyclonedds (-DNANO_ROS_RMW=cyclonedds / --features rmw-cyclonedds)
is your whole responsibility; nros_provide_cyclonedds() resolves the library:
a prebuilt install you point at (-DCMAKE_PREFIX_PATH=<install> /
-DCycloneDDS_DIR=), else self-provision from source — by default the pinned
third-party/dds/cyclonedds submodule, or your own checkout via
-DCYCLONEDDS_SOURCE_DIR=<path>. So a bare cmake/cargo build needs no
just cyclonedds pre-step (that is a contributor-only recipe);
freertos / threadx-rv64 / native examples build
Cyclone on demand (sccache-accelerated), gated on the relevant cross toolchain.
On a tier without the toolchain, the embedded-Cyclone tests are filtered out of
test-all (skipped, not failed). idlc is a host tool, found on PATH
(e.g. a ROS 2 install) or via -DIDLC_EXECUTABLE=. See
docs/development/sdk-tiers.md § “CycloneDDS — self-provisioned in CMake”.
Known limitations. Cyclone DDS currently has a 2× CDR roundtrip per message, deferred status-event wiring, pending service request-id correlation, and incomplete Cortex-A/R Zephyr board support. See docs/reference/cyclonedds-known-limitations.md for the full list. ARMv8-R toolchain prep (Cortex-A 64-bit FVP, Cortex-R52 hardware) is in docs/reference/zephyr-armv8r-setup.md.
Comparison
| Aspect | Zenoh (rmw-zenoh) | XRCE-DDS (rmw-xrce) | Cyclone DDS (rmw-cyclonedds) |
|---|---|---|---|
| Client RAM ¹ | ~16 KB+ (heap required) | ~3 KB (fully static) | ~32 KB+ (heap required) |
| Client Flash ¹ | ~100 KB+ | ~75 KB | ~150 KB+ (libddsc.so ~1.4 MB on POSIX, sized down on embedded link) |
| Bridge process | rmw_zenohd (ROS 2’s zenoh router) | Agent (protocol translator) | None — RTPS multicast directly |
| Peer-to-peer | Not as shipped — multicast/scouting compiled out (issue 0682) | No (agent always required) | Yes (RTPS native) |
| Discovery | Client participates | Agent handles on behalf | SPDP / SEDP on UDP multicast or static peer list |
| Entity creation | Client creates directly | Client requests, agent creates | Client creates directly |
| Transport options | TCP, UDP, TLS | UDP, serial (HDLC) | UDP unicast + multicast (RTPS) |
| Heap allocation | Required (C-level) | None | Required (Cyclone uses malloc) |
| Implementation | Rust + zenoh-pico C | Rust + Micro-XRCE-DDS-Client C | C++ wrapper over upstream Cyclone DDS C |
| ROS 2 interop | Via rmw_zenoh_cpp + router | Via Agent + any DDS RMW | Direct against rmw_cyclonedds_cpp (same upstream version) |
| Failure mode | Router crash = lose routing | Agent crash = lose connectivity | Peer goes offline = its samples stop arriving |
| C source files | ~100+ | 28 | Upstream Cyclone (~600+ files, vendored unchanged via submodule) |
¹ Footprint figures are order-of-magnitude planning numbers, not
measurements — they vary with features, message set, and opt level.
Size your own image with arm-none-eabi-size on your build; the RAM
knobs are enumerated in the
Static Pool Inventory.
Multi-backend binaries (bridges)
A single nano-ros binary can link more than one RMW backend and forward traffic between them. The bridge pattern is useful for:
- Translating between protocols — a gateway node running zenoh ingress + DDS egress lets MCU fleets on zenoh-pico talk to an Autoware stack on Cyclone DDS without a separate translator.
- Hard-real-time + best-effort split — a high-priority Node on Cyclone DDS for control loops, a low-priority Node on Zenoh for telemetry, both in one process.
- Bringing up an XRCE Agent-free fleet — bridge XRCE devices to a Zenoh network so they look like first-class participants to stock ROS 2.
The pattern uses Executor::open_with_rmw("<name>", ...) to pin the
primary session and node_builder("name").rmw("<other>").build() to
open additional sessions on other backends. Both backends must be
in the binary’s link line (Cargo manifest deps + register() call
each).
A worked example lives at
examples/bridges/native-rust-zenoh-to-dds/;
the time-triggered variant under
examples/bridges/tt-zenoh-to-xrce/
shows the same pattern under an ARINC-653-style cyclic schedule.
Full walkthrough: Cross-backend Bridges.
RMW Selection
RMW selection is a declared, language-agnostic, per-deploy value —
you set it once in system.toml ([system].rmw, optionally overridden
per image by [image.<id>].rmw) or via a CLI/build flag, and the
toolchain lowers that declaration to each language’s native build
mechanism: a Rust cargo rmw-<x> feature, or a CMake -DNANO_ROS_RMW
cache var. The cargo feature and the CMake var below are the lowering
targets the build uses — not the user-facing knob. A binary links the
common rmw-cffi runtime plus exactly one backend; the linked backend
self-registers through the nros_rmw_vtable_t C ABI, and the registry
walker resolves it at Executor::open. See
RFC-0031
for the full selection-and-lowering model and precedence rules.
Cargo.toml (Rust)
Lowered form of the declared RMW. The selection point is the board crate
(RFC-0031’s phase-248 C5b amendment): an app declares one rmw-<x> feature
that activates the backend dep and forwards to the board crate’s matching
feature. Codegen no longer emits nros/rmw-<x> or nros/platform-<y> — the
board brings both.
[features]
default = ["rmw-zenoh"]
# Each `rmw-*` activates the app-owned optional backend dep and forwards to
# the board crate, which is what actually selects the backend.
rmw-zenoh = ["dep:nros-rmw-zenoh", "nros-board-linux/rmw-zenoh"]
rmw-cyclonedds = [
"nros/rmw-cyclonedds", # the marker feature; see below
"dep:nros-rmw-cyclonedds-sys",
"nros-board-linux/rmw-cyclonedds",
]
In a WORKSPACE you write none of this: [image.<id>].rmw in the bringup’s
system.toml is the declaration, and the Entry pkg’s generated manifest carries
the lowered form. The block above is the standalone copy-out shape, where there
is no bringup to declare it in.
Cyclone DDS backend — Rust runtime sees the generic rmw-cffi C-ABI
vtable; actual Cyclone wiring lives C++-side under
packages/rmw/cyclonedds/nros-rmw-cyclonedds/ and is selected at CMake
configure time via -DNANO_ROS_RMW=cyclonedds. The Rust
manifest only carries the rmw-cffi feature; no Rust shim dep.
[dependencies] nros = { path = “<…>/packages/api/nros”, default-features = false, features = [“std”, “rmw-cffi”, “platform-posix”] }
Each example also calls `<backend>::register()` from `main()` before
`Executor::open` — this drags the rlib's CGU into the binary so the
linkme distributed-slice walker finds the backend. C/C++ builds rely
on the CMake-emitted strong stub from `nano_ros_link_rmw(... RMW <x>)`
instead.
For C++ consumers, the declared RMW lowers to the CMake cache var:
```bash
cmake -S . -B build -DNANO_ROS_RMW=cyclonedds # zenoh / xrce / cyclonedds
Kconfig (Zephyr)
# Zenoh backend
CONFIG_NROS_RMW_ZENOH=y
# XRCE-DDS backend
CONFIG_NROS_RMW_XRCE=y
# Cyclone DDS backend (Cortex-A/R Zephyr targets)
CONFIG_NROS_RMW_CYCLONEDDS=y
Enabling more than one simultaneously produces a compile_error!().
Cyclone DDS — per-platform configuration profile
The Cyclone DDS backend speaks raw RTPS over UDP multicast and unicast. The host networking stack on every RTOS needs IGMP enabled, adequate net- buffer pool sizing for the SPDP / SEDP discovery burst, and a way to join a multicast group. Concrete deltas per platform:
Zephyr (platform-zephyr)
# IGMP for SPDP multicast discovery (239.255.0.1:7400+).
CONFIG_NET_IPV4_IGMP=y
# Multicast group slots — RTPS uses up to 4 builtin groups per
# participant (SPDP + SEDP pubs / subs / topics).
CONFIG_NET_IF_MCAST_IPV4_ADDR_COUNT=4
# Net-buffer pools — defaults of 14 / 36 are too small for the SEDP
# burst when more than two service / action entities exist on each
# participant. Symptom of undersizing: runtime starvation as the
# discovery exchange backs up. Cortex_a9 has 512 MB SRAM; the cost
# is trivial.
CONFIG_NET_PKT_RX_COUNT=256
CONFIG_NET_PKT_TX_COUNT=128
CONFIG_NET_BUF_RX_COUNT=512
CONFIG_NET_BUF_TX_COUNT=256
CONFIG_NET_BUF_DATA_SIZE=512
# Heap — Cyclone DDS (malloc) + heapless mailboxes.
CONFIG_COMMON_LIBC_MALLOC_ARENA_SIZE=4194304
CONFIG_HEAP_MEM_POOL_SIZE=524288
On qemu_cortex_a9, additionally use a board overlay to bump the
GEM driver’s RX / TX descriptor ring (default 32) so a brief drainer
pause doesn’t immediately spill:
&gem0 {
promiscuous-mode;
rx-buffer-descriptors = <128>;
tx-buffer-descriptors = <64>;
};
native_sim is not yet supported — Zephyr’s NSOS driver doesn’t
forward IP_ADD_MEMBERSHIP to the host kernel
(no IPPROTO_IP case in nsos_adapt_setsockopt). Use
qemu_cortex_a9 for DDS-on-Zephyr testing instead.
FreeRTOS + lwIP (platform-freertos)
// FreeRTOSConfig.h / lwipopts.h
#define LWIP_IGMP 1 // SPDP multicast
#define LWIP_SO_RCVTIMEO 1 // Cyclone DDS recv timeouts
#define LWIP_BROADCAST 1
#define IP_REASSEMBLY 1 // RTPS DATA_FRAG fragments
#define MEMP_NUM_NETBUF 32 // discovery burst headroom
NuttX (platform-nuttx)
CONFIG_NET_IGMP=y
CONFIG_NET_BROADCAST=y
CONFIG_NET_UDP_NRECVS=4
CONFIG_NET_RECV_TIMEO=y
ThreadX + NetX Duo (platform-threadx)
tx_user.h / nx_user.h:
#define NX_ENABLE_IGMPV2 // IGMP v2 for RTPS multicast
// NetX BSD layer init must call `bsd_initialize` early — required
// for `setsockopt(SO_RCVTIMEO)` support.
Bare-metal smoltcp (platform-mps2-an385, platform-stm32f4,
platform-esp32-qemu)
#![allow(unused)]
fn main() {
// Bridge config in the board crate.
let mut config = smoltcp::iface::Config::new(...);
config.multicast_groups = vec![Ipv4Address::new(239, 255, 0, 1)];
// MulticastConfig::Strict in smoltcp 0.x; the bridge must expose
// a `join_multicast_group` API.
}
POSIX (platform-posix)
No configuration needed — the kernel does IGMP and setsockopt
support natively. Just ensure the loopback interface is up
(default).
When to Use Which
Choose Zenoh when:
- You need ROS 2 interop with
rmw_zenoh_cpp(the recommended ROS 2 middleware for Jazzy+) - You want peer-to-peer communication without any bridge process
- Your MCU has at least ~16 KB of heap and ~100 KB of flash
- You are using TCP or UDP networking
- You want simpler deployment (the router is a single process started with
ros2 run rmw_zenoh_cpp rmw_zenohd)
Choose XRCE-DDS when:
- Your MCU has very limited RAM (under 8 KB available for middleware)
- You need serial (UART) transport – useful for MCUs without networking hardware
- You are integrating with an existing micro-ROS or DDS deployment
- You want zero heap allocation on the MCU side
Choose Cyclone DDS when:
- You want direct, brokerless ROS 2 interop with no router and no Agent
- Your MCU has at least ~32 KB RAM (heap for Cyclone + RTPS state) and a network stack with IGMP
- You’re integrating with a DDS-based ROS 2 deployment running stock
rmw_cyclonedds_cppand want full RTPS wire-compat without protocol translation - Your tolerance for failure is “samples from peer X stop arriving” rather than “the whole router goes down”
- Note: Cyclone DDS is pub/sub-only today (no services / actions yet)
Any of the three works well for:
- Standard pub/sub and service patterns
- Integration with ROS 2 desktop nodes
- QEMU-based development and testing
Multi-Node Project Layout
You built the single-file talker in
examples/native/rust/talker/:
one main.rs, one Cargo.toml, one package — open a terminal, run
cargo run, done. That shape covers a lot of ground: demos, driver
tests, single-purpose microcontroller apps.
At some point it stops being enough.
When you outgrow one app
Three common triggers:
-
Two or more nodes. You want a talker and a listener, or a sensor driver alongside a control loop. Splitting them into one entry binary per node file works until you need to tune their wiring or deploy them to different boards.
-
A shared launch topology. You want to describe once how nodes are named, remapped, and parameterised — and reuse that description across a dev laptop, a hardware bring-up board, and a sim target.
-
Multiple deploy targets. The same talker logic goes on a native Linux host for integration testing and on an STM32F4 for production. The node logic is identical; only the boot and board differ.
-
Mixed implementation languages. You want to keep a C driver or legacy C node, but host the composed system in a C++ or Rust Entry pkg. The Node-pkg register ABI is language-neutral, so a C Node pkg can link into the same Entry binary as C++ or Rust Node pkgs.
That’s when you split your project into a multi-node workspace.
Canonical layout
Start with the whole project before diving into the parts:
my_robot_ws/
├── Cargo.toml # Rust workspace root, or CMakeLists.txt for C/C++
└── src/
├── talker_pkg/ # Node pkg: reusable node logic, no main()
├── listener_pkg/ # Node pkg: another reusable node
├── robot_bringup/ # Bringup pkg: launch XML + system.toml
└── native_entry/ # Entry pkg: one runnable binary for one board
The roles are deliberately separated:
| Role | Owns | Does not own |
|---|---|---|
| Node pkg | Publishers, subscriptions, timers, services, actions, callback bodies | Board choice, launch topology, main() |
| Bringup pkg | Which nodes run, names, remaps, parameters, per-target topology | Compiled code |
| Entry pkg | Board/runtime selection and the runnable binary | Node behavior |
A typical product has many Node pkgs, one Bringup pkg per logical system,
and one Entry pkg per board or deploy target. The same talker_pkg and
listener_pkg can be linked into a native host Entry pkg for integration
testing and a Cortex-M Entry pkg for hardware.
Reading order
This group starts broad and then drills into each part:
- Project layout — this page: when to split and how the roles fit.
- Node packages — reusable node libraries with
nros::node!. - Bringup packages — launch XML,
system.toml, remaps, parameters. - Entry packages — the board-specific binary that boots the topology.
- C / C++ multi-node workspaces — the same structure through CMake.
- Mixed-language workspaces — C Node pkgs hosted by C/C++ Entry pkgs.
- Role reference — metadata fields and macro forms in reference style.
Prereqs
Pick one path from a fresh checkout — just is NOT a prereq.
A. Front door (bare machine OK — no Rust, no just):
./scripts/bootstrap.sh
Installs rustup if needed and builds the in-tree nros CLI from
source at packages/cli/target/release/nros, leaving it on PATH for
this shell (nano-ros is a source distribution — no prebuilt nros).
B. Already have cargo (equivalent — same build, same binary):
git submodule update --init packages/cli/third-party/play_launch
cargo build --release --manifest-path packages/cli/Cargo.toml --bin nros
export PATH="$PWD/packages/cli/target/release:$PATH"
Every subsequent shell sources the workspace env via one of:
direnv allow # if you use direnv
source ./activate.sh # bash / zsh
source ./activate.fish # fish
Then provision the native host:
nros setup native --rmw zenoh
The three roles in practice
Node pkg — a lib crate that contains one node’s logic. It
declares nros::node!(T) and carries
[package.metadata.nros.node] in its Cargo.toml. It has no
fn main() — that lives in the Entry pkg. One Node pkg per node.
Think of it as a composable building block: the same talker_pkg lib
can be assembled into a native binary and an embedded binary without
any source change.
Bringup pkg — a purely declarative directory that owns the launch
topology. It contains a package.xml, a system.toml (listing which
nodes run, how they’re wired, per-target deploy config), and a
launch/ directory with ROS 2 launch XML. No Cargo.toml, no
compiled code. Naming convention <system>_bringup, matching nav2 /
Autoware / turtlebot3. This role is optional — a workspace with a
single deploy target can fold the launch files and system.toml
directly into the Entry pkg.
Entry pkg — a bin crate that boots a topology on one specific
board. It carries [package.metadata.nros.entry] with deploy = "<board>" and a src/main.rs that is just nros::main!(...). One
Entry pkg per deploy target. The Entry pkg links in all Node pkg libs,
links in the board crate, and hands control to the nano-ros runtime.
The app-node shape you already know (examples/native/rust/talker/) is
effectively a fused Entry + Node pkg: a single package that is both
the logic and the boot point. That fusion is fine — and encouraged —
for single-node work. Only split when you actually need the
flexibility.
ROS 2 ↔ nano-ros command map
If you’re coming from ROS 2, here’s the mapping of the commands you already know:
| ROS 2 | nano-ros | Notes |
|---|---|---|
ros2 pkg create | nros new <name> --platform <plat> [--lang <lang>] | scaffolds a Node pkg |
colcon build | cargo build (Rust) / cmake --build build (C++) | use the underlying tool directly |
ros2 launch <pkg> <file> | cargo run -p <entry_pkg> | composed Entry pkg IS the launch product (Phase 212.N + 222.D); the old launch wrapper was removed in nros 0.5.0 |
| (plan/validate) | nros plan → nros check | resolve + statically check the topology |
ros2 run <pkg> <exe> | run the Entry pkg binary (cargo run) | one Entry pkg per board |
Build verbs (cargo for Rust, cmake for C/C++, west for Zephyr,
idf.py for ESP-IDF) are used directly — there is no CLI build
indirection. The composed Entry pkg binary IS the launch product:
one Entry pkg = one binary = one process. Multi-process orchestration
(equivalent to multiple ros2 launch nodes in separate processes) is
a separate Entry pkg per deploy + a shell script / tmux session, not
a CLI verb.
The app-node shape stays valid
There is no obligation to restructure. If your project is one node on
one board, the app-node shape (src/main.rs + one package = both
logic and boot) is perfectly idiomatic and has no runtime penalty. The
three-role split is a tool for when you need it, not a gatekeeping
requirement.
Where to go next
Walk through the multi-node project model step by step:
- Node packages — scaffold and
implement Node pkgs with
nros::node!. - Bringup packages — declare your topology in a Bringup pkg.
- Entry packages — write the Entry pkg that boots everything together.
- C / C++ multi-node workspaces — use the same project shape with CMake.
For C Node pkgs hosted by a C++ Entry pkg, see Mixed-language workspace.
For the full API reference covering all three roles, see Role reference.
Node packages
This page is part of the Multi-Node Projects group. Previous: Project layout — Next: Bringup packages
A Node pkg is the unit of reusable behaviour in a multi-node workspace.
It is a Rust library — a [lib] crate — that implements one node and
registers it with nros::node!(T).
The Entry pkg is what boots the binary; the Node pkg is what runs inside it.
Prereqs
Pick one path from a fresh checkout — just is NOT a prereq.
A. Front door (bare machine OK — no Rust, no just):
./scripts/bootstrap.sh
Installs rustup if needed and builds the in-tree nros CLI from
source at packages/cli/target/release/nros, leaving it on PATH for
this shell (nano-ros is a source distribution — no prebuilt nros).
B. Already have cargo (equivalent — same build, same binary):
git submodule update --init packages/cli/third-party/play_launch
cargo build --release --manifest-path packages/cli/Cargo.toml --bin nros
export PATH="$PWD/packages/cli/target/release:$PATH"
Every subsequent shell sources the workspace env via one of:
direnv allow # if you use direnv
source ./activate.sh # bash / zsh
source ./activate.fish # fish
Then provision the native host:
nros setup native --rmw zenoh
Scaffolding a Node pkg
Use nros new to create a skeleton:
nros new talker --platform native --lang rust
nros new creates a project skeleton.
For a workspace, move (or create) the result under src/talker_pkg/ so the
workspace root’s Cargo.toml can include it as a member:
# workspace Cargo.toml
[workspace]
resolver = "2"
members = ["src/talker_pkg", "src/listener_pkg", "src/native_entry"]
Node pkg anatomy
A Node pkg has three files:
src/talker_pkg/
├── package.xml # ROS 2 manifest — <exec_depend> per message package
├── Cargo.toml # [lib] + [package.metadata.nros.node] metadata
└── src/lib.rs # impl Node + ExecutableNode; ends with nros::node!(Talker);
No fn main() here — a Node pkg is a library linked into an Entry pkg.
The Entry pkg’s macro-generated runtime owns nros::init, executor open,
RMW registration, and the spin/yield loop.
Cargo.toml — the [package.metadata.nros.node] block
The metadata block is what the nros CLI reads to discover, name, and
wire this node into a topology.
From examples/workspaces/rust/src/talker_pkg/Cargo.toml:
[lib]
path = "src/lib.rs"
[package.metadata.nros.node]
class = "talker_pkg::Talker"
name = "talker"
default_namespace = "/"
[dependencies]
nros = { path = "../../../../../packages/api/nros", default-features = false,
features = ["alloc", "rmw-cffi"] }
The three fields in [package.metadata.nros.node]:
| Field | Purpose |
|---|---|
class | Fully-qualified Rust path to the type that impls Node + ExecutableNode |
name | Default ROS 2 node name (remappable at launch) |
default_namespace | Default namespace (remappable at launch) |
A Node pkg names no platform and no RMW. alloc is the universal
baseline and rmw-cffi is the vtable seam; the concrete backend and the
platform both flow in from the board crate that the sibling Entry pkg depends
on, via cargo feature unification. That is what lets one talker_pkg deploy to
a Linux host and a Cortex-M board without a fork.
src/lib.rs — the node implementation
A Node pkg implements two traits: Node (declarative registration) and
ExecutableNode (per-callback body), then calls nros::node! to export the
trampolines the Entry macro expects.
Here is the essential shape, drawn from
examples/workspaces/rust/src/talker_pkg/src/lib.rs
(see that file for the full worked version):
#![allow(unused)]
fn main() {
use nros::{
CallbackCtx, ExecutableNode, Node, NodeContext, NodeOptions, NodeResult,
TimerDuration,
};
pub struct Talker;
impl Node for Talker {
const NAME: &'static str = "talker";
fn register(ctx: &mut NodeContext<'_>) -> NodeResult<()> {
let mut node = ctx.create_node(NodeOptions::new("talker"))?;
let chatter = node.create_publisher_for_topic::<MyMsg>("/chatter")?;
let _timer =
node.create_timer_for_callback_name("on_tick", TimerDuration::from_millis(1000))?;
node.callback_for_name("on_tick")
.publishes_entity(&chatter)?;
Ok(())
}
}
impl ExecutableNode for Talker {
type State = i32;
fn init() -> Self::State { 0 }
// See the full example for the callback body.
}
nros::node!(Talker); // <-- exports the trampolines; this is the last line
}
Key points:
Node::registeris declarative — it runs once at startup to declare publishers, subscriptions, timers, and callback edges. No message bytes flow here.ExecutableNode::on_callbackis the body — called by the Entry pkg’s executor each time a callback fires.stateis your per-node mutable storage.nros::node!(Talker)must be the last public API call in the file. It generates theextern "C"trampolines the Entry macro imports.- There is no
fn main()in a Node pkg.
package.xml — the ROS 2 manifest
A Node pkg that uses generated message types lists them as <exec_depend>
entries. Minimal example:
<?xml version="1.0"?>
<package format="3">
<name>talker_pkg</name>
<version>0.1.0</version>
<description>Talker node</description>
<maintainer email="dev@example.com">Developer</maintainer>
<license>MIT OR Apache-2.0</license>
<depend>std_msgs</depend>
<export>
<build_type>ament_cargo</build_type>
</export>
</package>
If your node uses no external message packages, the <depend> line can be
omitted.
Services & clients
Server (responds to requests)
Declare the service in Node::register and handle requests in the node body.
-
Rust — declare a service edge in
register; requests dispatch intoExecutableNode::on_callbacklike any other callback (read the request and write the reply through theCallbackCtx). -
C++/C (the
configure(Node&)component shape) — bind a typed member withnros::bind_service:// Response on_request(const Request&) — no hand-rolled CDR. AddTwoInts::Response on_add(const AddTwoInts::Request& req) { AddTwoInts::Response r; r.sum = req.a + req.b; return r; } ::nros::Result configure(::nros::Node& node) { return ::nros::bind_service<AddTwoInts, MyServer, &MyServer::on_add>( node, "/add_two_ints", this); // service-type name from AddTwoInts::TYPE_NAME }bind_servicedeserializes the request into the generatedSvc::Request, calls your typed member, and serializes the returnedSvc::Response— no manual byte offsets. (bind_service_raw<&method>is still available for the rare hand-rolled case.)
Client (issues requests) — call from tick, not on_callback
A node cannot make a blocking service/action client call from
on_callback: that runs while the executor is mid-dispatch, so a blocking call
would deadlock the spin. Client calls go on the separate per-spin hook
ExecutableNode::tick(&mut TickCtx), which the executor runs once per
spin_once between callback dispatch:
#![allow(unused)]
fn main() {
fn on_callback(state: &mut State, cb: Callback<'_>, ctx: &mut CallbackCtx<'_>) {
// ...decide a request is needed; just arm a flag / stash args:
state.want_call = true;
}
fn tick(state: &mut State, ctx: &mut TickCtx<'_>) {
if state.want_call {
state.want_call = false;
let reply = ctx.call_for_name("add_two_ints", &request); // blocking, safe here
// ...use reply
}
}
}
The two-surface pattern (arm in on_callback, call in tick) is the supported
way to drive a client from a declarative node. See
examples/workspaces/rust/.../service_client_pkg for the reference.
Building
From the workspace root, sync generated interfaces first, then let Cargo compile the Node pkgs and Entry pkg:
# From examples/workspaces/rust/ (or your workspace root):
nros sync
nros codegen-system --bringup demo_bringup
cargo build -p native_entry
No per-Node-pkg invocation is needed — the workspace resolver handles dependency ordering.
To cross-compile for an embedded target, pass --target and ensure
.cargo/config.toml in the workspace root sets the right linker and target:
cargo build --target thumbv7em-none-eabihf
Next steps
- Bringup packages — wire the Node pkgs together into a topology.
- Entry packages — build the binary that boots the topology on real hardware or a host process.
- Role reference —
the full reference for all three roles, metadata fields, and the
nros::main!()four forms.
Bringup packages
A Bringup pkg is the declarative glue that ties your Node packages together into a runnable topology. It owns the launch file, the wiring between nodes, and the per-target deploy config — all without any compiled code of its own.
Pre-requisite: You’ve scaffolded your Node packages following the Node packages guide. This page adds the
demo_bringuplayer between them and the Entry package that boots everything.
Prereqs
Pick one path from a fresh checkout — just is NOT a prereq.
A. Front door (bare machine OK — no Rust, no just):
./scripts/bootstrap.sh
Installs rustup if needed and builds the in-tree nros CLI from
source at packages/cli/target/release/nros, leaving it on PATH for
this shell (nano-ros is a source distribution — no prebuilt nros).
B. Already have cargo (equivalent — same build, same binary):
git submodule update --init packages/cli/third-party/play_launch
cargo build --release --manifest-path packages/cli/Cargo.toml --bin nros
export PATH="$PWD/packages/cli/target/release:$PATH"
Every subsequent shell sources the workspace env via one of:
direnv allow # if you use direnv
source ./activate.sh # bash / zsh
source ./activate.fish # fish
Then provision the native host:
nros setup native --rmw zenoh
What a Bringup pkg is
A Bringup pkg is pure declarative — no Cargo.toml, no CMakeLists.txt,
no src/. Its job is to describe which nodes run, how they’re wired, and
where they deploy. Naming convention: <system>_bringup (aliased <system>_launch),
matching nav2 / Autoware / turtlebot3.
It is optional: required only when two or more Entry packages share one
topology. A single-Entry workspace can fold launch/ + system.toml directly
into the Entry pkg.
Anatomy
src/demo_bringup/
├── package.xml # ROS 2 manifest; <exec_depend> per node pkg
├── system.toml # [system] + [[component]] + [image.<id>] + [host.<name>]
├── launch/
│ └── system.launch.xml # ROS 2 launch schema, verbatim
└── config/ # optional — params.yaml, per-target overrides
system.toml — node wiring + deploy targets
The system.toml is the machine-readable topology. It lists every node the
system runs, its class path, and one or more deploy targets (which board, which
RMW, which domain).
Below is a minimal two-node example adapted from the real fixture at
packages/testing/nros-tests/fixtures/orchestration_e2e/demo_pkg_bringup/system.toml:
[system]
name = "demo"
rmw = "zenoh"
domain_id = 0
[[component]]
pkg = "talker_pkg"
class = "talker_pkg::Talker"
name = "talker"
[[component]]
pkg = "listener_pkg"
class = "listener_pkg::Listener"
name = "listener"
[image.native]
board = "native"
[host.native]
Key fields:
| Field | Meaning |
|---|---|
[system] name | Logical system name; used by nros plan/check |
[system] rmw | Default RMW for all components (zenoh, xrce, cyclonedds) |
[system] domain_id | ROS 2 domain (compile-time on embedded, runtime env on host) |
[[component]] pkg | The ROS package name (matches <name> in package.xml) |
[[component]] class | Fully-qualified Rust type (crate::TypeName) |
[[component]] name | Node name at runtime |
[image.<id>] | A buildable image; read by nros build, nros check and Entry codegen |
[image.<id>] board | The board this image is built for; its descriptor supplies the rustc triple |
[host.<name>] | A machine nodes run on. No nodes means every node |
[board_config.<board>] | Site facts for that board (SDK roots, netstack) |
For multi-domain setups or cross-domain bridges add [[domain]] and
[[bridge]] sections — see docs/design/0024-multi-node-workspace-layout.md §11
for the full schema.
launch/system.launch.xml — ROS 2 launch schema
The launch file uses the ROS 2 launch XML schema verbatim — nano-ros reads it with the same parser so existing nav2/Autoware/turtlebot3 XML pastes in and Just Works.
<launch>
<node pkg="talker_pkg" exec="talker" name="talker"/>
<node pkg="listener_pkg" exec="listener" name="listener"/>
</launch>
v1 tag set
| Tag | Purpose |
|---|---|
<launch> | Root element |
<arg name="…" default="…"/> | Declare a launch argument |
<node pkg="…" exec="…" name="…"/> | Instantiate a node |
<param name="…" value="…"/> | Set one parameter inline (nested inside <node>) |
<param from="…param.yaml"/> | Load a ROS parameter FILE (nested inside <node>) |
<remap from="…" to="…"/> | Topic/service remapping (nested inside <node>) |
<group ns="…"> | Namespace a group of nodes |
<include file="…"/> | Nest another launch file |
Substitutions
$(find <pkg>)— resolves to the package’s install/source path$(var <arg>)— expands a launch argument$(env <name>)— reads an environment variable
A richer example using args and remapping (taken from the real fixture):
<launch>
<arg name="talker_name" default="talker" />
<node pkg="talker_pkg" exec="talker" name="$(var talker_name)" output="screen">
<param name="rate_hz" value="25" />
<remap from="chatter" to="/chatter" />
</node>
<node pkg="listener_pkg" exec="listener" name="listener"/>
</launch>
Parameter files
<param from="…"/> takes a standard ROS parameter file. Relative paths resolve
against the launch file’s own directory.
# config/talker.param.yaml
/**: # applies to every node
ros__parameters:
use_sim_time: false
talker: # applies to the node named `talker` only
ros__parameters:
rate_hz: 25
limits:
max_accel: 1.5 # reaches the node as `limits.max_accel`
<node pkg="talker_pkg" exec="talker" name="talker">
<param from="config/talker.param.yaml"/>
<param name="rate_hz" value="50"/> <!-- wins over the file -->
</node>
Precedence matches ROS: the /** wildcard block first, then the
node-specific block, then inline <param name= value=> last. A node block
may be keyed by fully-qualified name (/ns/talker), bare name (talker),
or /**/talker.
These values are resolved and baked into the generated entry at build
time — embedded targets do no runtime file loading, the same approach the
domain ID uses. Editing the YAML re-runs codegen. A from= naming a file
that doesn’t exist is a build error, not a silent skip.
Note: the baked values are seeded through the parameter services, so they reach nodes only when
param_servicesis enabled insystem.toml.
Note: Python
.launch.pyfiles ARE resolved bynros sync(the RFC-0060 resolver executes them with its bundled CPython;$(find-pkg-share)against ROS packages needs a sourcedAMENT_PREFIX_PATH). The XML schema above is the native authoring format.
package.xml
A standard ROS 2 manifest. List each Node package as an <exec_depend>:
<?xml version="1.0"?>
<package format="3">
<name>demo_bringup</name>
<version>0.1.0</version>
<description>Bringup package for the demo system</description>
<maintainer email="you@example.com">Your Name</maintainer>
<license>Apache-2.0</license>
<exec_depend>talker_pkg</exec_depend>
<exec_depend>listener_pkg</exec_depend>
<export>
<build_type>ament_cmake</build_type>
</export>
</package>
No <build_depend> entries — there is nothing to compile.
Workflow: check → run
Once your Bringup pkg is written, use nros check to validate and
cargo run to execute the topology:
# 1. Lint the bringup pkg (pure-declarative check — no Cargo.toml, stray files, etc.)
nros check --bringup src/demo_bringup
# 2. Lint the whole workspace (pkg/class rows, duplicate system.toml, etc.)
nros check --workspace .
# 3. Run the composed Entry binary (boots all nodes in a single process)
ZENOH_CONFIG_OVERRIDE='listen/endpoints=["tcp/127.0.0.1:7447"];scouting/multicast/enabled=false' ros2 run rmw_zenoh_cpp rmw_zenohd & # router — in another shell
cargo run -p native_entry
Both nros check forms pass for the canonical template at
examples/workspaces/rust/.
Caveat —
nros planwith this template
nros plan demo_bringupresolves a topology intoplan.jsonfor static type/QoS checks, but it currently requires pre-collected source-metadata sidecars (record.json+ per-pkg_metadata/*.json). The automatic metadata-build path (nros metadata --build) is not yet wired for lib-only Node pkgs, sonros plandoes not produce a plan straight from this template. Seepackages/testing/nros-tests/fixtures/orchestration_e2e/for the pre-collected-sidecar pipeline.The canonical template README at
examples/workspaces/rust/README.mdis the source of truth for the current CLI state.
Runnable copy-out
examples/workspaces/rust/ is the canonical Rust 3-role workspace that pairs
with this guide. Copy the whole directory out and rename the packages.
nros sync materializes generated message crates, nros codegen-system
bakes the Bringup package, and cargo build -p native_entry builds the Entry
pkg.
The workspace README at examples/workspaces/rust/README.md
documents the exact CLI commands that are verified green today.
When you don’t need a Bringup pkg
If you have a single Entry pkg and don’t plan to share the topology across
multiple boards, fold launch/ + config/ directly into the Entry pkg.
The nros::main! macro’s launch = argument names the bringup package and,
optionally, a launch file within it — the SAME inputs you author. The build
resolves them to a SystemModel under <workspace>/build/nros/models/ (run
nros sync, or let the build system drive it); you never reference the
model file, though you can inspect it there:
#![allow(unused)]
fn main() {
// Multi-node: the bringup's default launch (system.launch.xml)
nros::main!(launch = "demo_bringup");
// A named launch file in the bringup pkg
nros::main!(launch = "demo_bringup:sim.launch.xml");
// A launch-argument binding, declared as [[model]] in system.toml
nros::main!(launch = "demo_bringup:multihost.launch.xml", args = [("host", "robot1")]);
}
If the launch files live inside the Entry pkg itself, point at it by name. The Entry package page covers this in full.
Where to go next
- Entry packages — the
nros::main!macro andnative_entry - Role reference — full reference for all three roles
- Project layout — start here if you haven’t read it yet
Entry packages
An Entry pkg is the binary that boots a topology on a specific board.
Where a Node pkg is a library (no fn main) and a Bringup pkg is purely
declarative, the Entry pkg is the one thing that actually runs: it names a
board, wires the runtime, and — for multi-node setups — points at a Bringup
pkg that describes which nodes should be launched.
You have one Entry pkg per deploy target. A workspace targeting both a native workstation and a bare-metal Cortex-M board has two Entry pkgs that reference the same Node pkgs; only the board and (optionally) the launch target differ.
Prereqs
Pick one path from a fresh checkout — just is NOT a prereq.
A. Front door (bare machine OK — no Rust, no just):
./scripts/bootstrap.sh
Installs rustup if needed and builds the in-tree nros CLI from
source at packages/cli/target/release/nros, leaving it on PATH for
this shell (nano-ros is a source distribution — no prebuilt nros).
B. Already have cargo (equivalent — same build, same binary):
git submodule update --init packages/cli/third-party/play_launch
cargo build --release --manifest-path packages/cli/Cargo.toml --bin nros
export PATH="$PWD/packages/cli/target/release:$PATH"
Every subsequent shell sources the workspace env via one of:
direnv allow # if you use direnv
source ./activate.sh # bash / zsh
source ./activate.fish # fish
Then provision the native host (the canonical first Entry pkg target;
for Zephyr / FreeRTOS / ESP32 swap in the matching nros setup board):
nros setup native --rmw zenoh
Package layout
src/native_entry/
├── package.xml
├── Cargo.toml # [[bin]] + deps on node pkgs + board crate
│ # + [package.metadata.nros.entry]
└── src/main.rs # nros::main!(launch = "demo_bringup");
No library code lives here. The Entry pkg links the Node pkg rlibs and hands
them to the runtime that nros::main!() generates.
Cargo.toml metadata
The [package.metadata.nros.entry] table tells the CLI which deploy target
this binary is built for. The embedded example
examples/qemu-arm-baremetal/rust/talker-rtic/ uses:
[package.metadata.nros.entry]
deploy = "rtic-mps2-an385"
A native Entry pkg that references a Bringup pkg looks like:
[package.metadata.nros.entry]
deploy = "native"
[package.metadata.nros.deploy.native]
board = "posix"
rmw = "zenoh"
domain_id = 0
deploy is the key that nros check and the Entry macro use to
find the board crate and verify the topology. Keep it short and descriptive —
it becomes the identifier in nros plan output and in system.toml’s
[image.<id>] table when you later add a Bringup pkg.
nros::main!() — the forms
#![allow(unused)]
fn main() {
// 1. Single-node self-bringup: reads [package.metadata.nros.entry] deploy
// from Cargo.toml and boots the Node pkg that is the only member of
// this workspace (or the one marked default).
nros::main!();
// 2. Single-node, explicit board type.
nros::main!(board = LinuxBoard);
// 3. Multi-node (CANONICAL): name your INPUT — the Bringup pkg (and
// optionally a launch file inside it; default is its default launch).
// `nros sync` resolves it into a SystemModel build artifact that the
// macro bakes from.
nros::main!(launch = "demo_bringup");
nros::main!(launch = "demo_bringup:variant_b.launch.xml");
// 4. Multi-host slice: point at a PER-HOST model. The launch file gates
// nodes on a `host` launch argument (`if=` conditions), and the model
// is resolved with `host:=robot1`, so it already contains only this
// host's nodes — no extra macro key needed.
nros::main!(launch = "demo_bringup:multihost.launch.xml", args = [("host", "robot1")]);
// 5. DEPRECATED (expert override): name the resolved model ARTIFACT
// directly instead of the input. Skips the sync-freshness contract;
// warns at bake time (phase-330 W7).
nros::main!(model = "demo_bringup");
}
launch and model are mutually exclusive. launch is the canonical
spelling (phase-330 W7): it names your input — the bringup package and
launch file — and the entry consumes the SystemModel that nros sync
resolves from it. The same resolved artifact drives the Linux runtime
(play_launch) and every embedded image, so contract budgets, tiers, and
QoS never drift between runtimes. model names the resolved artifact
directly — a deprecated expert override. Produce/refresh the model with:
$ nros sync # resolves models into <ws>/build/nros/models/<bringup>/
nros sync (the pre-build step every workflow already runs) re-resolves
the model whenever a bringup’s launch XML or system.toml is newer than
it. SystemModels are build artifacts — never commit one (the
check-no-tracked-models gate enforces this).
nros sync is the only verb you need. Under the hood it runs the pinned
nros-launch-resolve helper (RFC-0060 layer 2; contributors build it
in-tree via just setup-launch-resolve) and invokes it by absolute path — never by a
bare name on $PATH, because an unrelated ROS 2 play_launch on PATH used to
win that race and break every platform’s fixture build (issue 0285). The macro reads [package.metadata.nros.entry] at compile
time to select the right board and executor backend; the same model resolves
its tier table for whichever RTOS that board targets. On Embassy / RTIC targets it emits the
framework-specific #[embassy_executor::main] or #[rtic::app] body so your
src/main.rs stays a single line.
The real examples/qemu-arm-baremetal/rust/talker-rtic/src/main.rs collapses
to exactly this:
#![allow(unused)]
#![no_std]
#![no_main]
fn main() {
use panic_semihosting as _;
nros::main!();
}
Escape hatch
If you need more control than the macro provides — custom startup ordering,
hardware init before the runtime, or a fully manual executor loop — you can
bypass nros::main!():
#![allow(unused)]
fn main() {
// Option A: delegate init to the board crate, supply your own closure.
<LinuxBoard as BoardEntry>::run(|runtime| {
let node = runtime.create_node("talker", "/", &Default::default())?;
// ...
Ok(())
});
// Option B: fully manual — no board crate.
let executor = nros::Executor::open(&ExecutorConfig::default())?;
// wire nodes, spin manually ...
}
Option A is the right choice when you need to run something before the first spin (e.g. DMA setup, flash unlock). Option B is there for board-bringup authors adding a new platform.
Running a native Entry pkg
The verified path for the canonical Rust workspace is cargo run -p native_entry.
Start a Zenoh router first, then boot the Entry binary from the workspace root:
# in another shell:
ZENOH_CONFIG_OVERRIDE='listen/endpoints=["tcp/127.0.0.1:7447"];scouting/multicast/enabled=false' ros2 run rmw_zenoh_cpp rmw_zenohd &
cargo run -p native_entry
native_entry opens the executor against the router, registers talker +
listener (composed into a single process), and runs the topology.
The canonical Rust workspace is at examples/workspaces/rust/.
For Zephyr, QEMU, ESP-IDF, and other non-native targets, use the platform’s
native build/run tool.
Running on Zephyr
On Zephyr the RTOS framework is the workflow: west build is the build
verb, Kconfig selects the RMW, and the Entry is an ordinary Zephyr
application. There is no nros build / nros launch build path here, and you
do not type the RMW as a Cargo --features flag or bake the board into the
package.
source ./activate.sh
# Provision message bindings once. This is platform-agnostic workspace
# provisioning (sibling to `west update` / `rosdep`), NOT a compile step —
# the same `nros sync` output feeds every board and every RMW.
nros sync
# west is the build verb. Choose the board with `-b`, and select the RMW with
# the matching Kconfig overlay via -DCONF_FILE. The Entry source never changes.
west build -b native_sim/native/64 src/zephyr_entry \
-- -DCONF_FILE="prj.conf;prj-zenoh.conf"
west build -t run # native_sim; `west flash` for hardware
Swap -b native_sim/native/64 for any other Zephyr board (-b nrf52840dk/nrf52840,
-b stm32f4_disco, …) and prj-zenoh.conf for prj-xrce.conf /
prj-cyclonedds.conf to pick a different RMW — nothing else changes. One
Zephyr Entry pkg (src/zephyr_entry/) covers every Zephyr board: unlike the
board-specific FreeRTOS / ThreadX Entries, Zephyr owns its board abstraction,
so the board is chosen at west build -b time rather than baked into the
package (see One Entry pkg per board).
The Entry source is identical to the native, FreeRTOS, and ThreadX Entries — the same one-line macro, with the committed SystemModel as the single source of truth for the node set:
#![allow(unused)]
fn main() {
// examples/workspaces/rust/src/zephyr_entry/src/lib.rs
nros::main!(launch = "demo_bringup");
}
One Entry pkg per board
Each deploy target gets its own Entry pkg. A workspace that runs on both
native and rtic-mps2-an385 would have two Entry pkgs that share the same
Node pkg library:
| Entry pkg | deploy key | Board crate |
|---|---|---|
native_entry | "native" | nros-board-linux |
mps2_entry | "rtic-mps2-an385" | nros-board-mps2-an385 (rtic feature) |
Both reference the same talker_pkg and listener_pkg Node pkg rlibs. The
board crate provides the BoardEntry impl and any hardware-specific
initialisation; the Node pkgs are board-agnostic.
Zephyr is the exception — one Entry per RTOS, not per board. Zephyr
already owns its board abstraction, so a single zephyr_entry covers
native_sim, nrf52, stm32, aemv8r, … with the board chosen at
west build -b <board> time. Contrast FreeRTOS / ThreadX, whose board crates
are board-specific, so each of those Entries bakes one board. See
Running on Zephyr.
The examples/qemu-arm-baremetal/rust/talker-rtic/ example demonstrates the
embedded shape: deploy = "rtic-mps2-an385" + nros::main!(); on a
no_std / no_main binary that delegates everything to the RticMps2An385
board crate.
C / C++ Entry packages
C and C++ Entry packages use the same role split through CMake. See C / C++ multi-node workspaces.
Where to go next
- Role reference — full reference for all three roles.
- Bringup packages — the
system.toml+ launch XML that an Entry pkg points at. - Node packages — the Node pkgs your Entry pkg links.
- Project layout — the full 3-role picture and when to use it.
C / C++ multi-node workspaces
The four previous chapters
(project layout,
node packages, bringup packages,
entry packages) describe the canonical three-role
node + bringup + entry shape against the Rust path
(nros::node!(…) + nros::main!(launch = …)). This chapter shows the
C and C++ path through the same shape, role-for-role.
The parity gap is closed. Same launch.xml, same package.xml,
same system.toml, same workspace pkg-index — the only thing that
changes language-side is the cmake-fn / macro surface.
TL;DR — side-by-side
| Role | Rust | C / C++ |
|---|---|---|
| Node pkg | lib.rs with nros::node!(MyNode) + [package.metadata.nros.node] in Cargo.toml | Talker.{hpp,cpp} with a configure(::nros::Node&) component method (C++) / NROS_C_COMPONENT (C); CMakeLists.txt calling nano_ros_auto_add_library + nros_components_register_node (RFC-0057) |
| Bringup pkg | package.xml + system.toml + launch/*.launch.xml (no Cargo.toml) | identical (language-agnostic) |
| Entry pkg | src/main.rs with nros::main!(launch = "demo_bringup") | src/main.cpp with NROS_MAIN(nros::board::LinuxBoard, "demo_bringup:system.launch.xml"); CMakeLists.txt calling nano_ros_entry(NAME … BRINGUP "…/demo_bringup" LAUNCH default DEPLOY native) |
| Workspace root | Cargo.toml [workspace] members = […] | CMakeLists.txt calling nano_ros_workspace(BACKEND zenoh PLATFORM posix SUBDIRS src/talker_pkg src/listener_pkg src/native_entry) |
| Build | nros sync + cargo build -p native_entry | nros sync + cmake -S . -B build + cmake --build build |
| Boot | cargo run -p native_entry | ./build/.../native_entry |
The reference C++ workspace ships in-tree at
examples/workspaces/cpp/.
Copy the whole directory, rename the packages.
Workspace layout
Identical structure to the Rust template, swapping Cargo.toml →
CMakeLists.txt:
my_ws/
├── CMakeLists.txt # nano_ros_workspace(SUBDIRS …)
└── src/
├── talker_pkg/ # Node pkg (C++)
│ ├── package.xml
│ ├── CMakeLists.txt # nano_ros_auto_add_library + nros_components_register_node
│ └── src/{Talker.hpp,Talker.cpp}
├── listener_pkg/ # Node pkg (C++)
│ ├── package.xml
│ ├── CMakeLists.txt
│ └── src/{Listener.hpp,Listener.cpp}
├── demo_bringup/ # Bringup pkg (language-agnostic — copy/paste
│ ├── package.xml # works between Rust and C++ workspaces)
│ ├── system.toml
│ └── launch/system.launch.xml
└── native_entry/ # Entry pkg (C++)
├── package.xml
├── CMakeLists.txt # nano_ros_entry(BRINGUP … LAUNCH …)
└── src/main.cpp # NROS_MAIN(…) one-liner
Workspace root
Four declarations:
cmake_minimum_required(VERSION 3.22)
project(my_ws LANGUAGES C CXX)
include(<nano-ros>/cmake/NanoRosWorkspace.cmake)
nano_ros_workspace(
BACKEND zenoh # zenoh | xrce | cyclonedds
PLATFORM posix # posix | … (default posix)
NANO_ROS_ROOT <path-to-nano-ros> # also: -D cache var, $NANO_ROS_ROOT,
# or auto-walk for nros-sdk-index.toml
SUBDIRS src/talker_pkg
src/listener_pkg
src/native_entry
)
nano_ros_workspace() does the heavy lifting in one call:
- Sets
NANO_ROS_PLATFORM=posix+NANO_ROS_RMW=zenoh. add_subdirectory(<nano-ros>)once at root scope (so per-pkg subdirs don’t collide on re-include).include(NanoRosNodeRegister.cmake)+include(NanoRosEntry.cmake)once.add_subdirectory(<each member>)for eachSUBDIRSentry.
Subdir CMakeLists begin with the dual call:
nano_ros_workspace_pkg_guard() # no-op inside a workspace; bootstraps standalone solo
— the cmake equivalent of cargo [workspace] discipline. Every member
compiles standalone (with -DNANO_ROS_ROOT=<path>) or as part of the
workspace; the per-pkg CMakeLists doesn’t change between modes.
Node pkg
A typed component (RFC-0043) — no main(). The pkg ships a class with a
configure(::nros::Node&) method that creates real entities (a Publisher, a
Timer) and binds member callbacks by identity (member-fn-pointer template
param, no string callback name, no interpreter). The Entry pkg constructs the
object and calls configure(node); the executor dispatches the callbacks.
# src/talker_pkg/CMakeLists.txt
cmake_minimum_required(VERSION 3.22)
project(talker_pkg LANGUAGES C CXX)
nano_ros_workspace_pkg_guard()
nros_find_interfaces(LANGUAGE CPP SKIP_INSTALL)
# RFC-0057: the ament shape — the library owns the sources, the register
# owns the identity (keyword parity with rclcpp_components_register_node).
# Interface deps (std_msgs__nano_ros_cpp) are wired automatically.
nano_ros_auto_add_library(talker_lib STATIC src/Talker.cpp)
nros_components_register_node(talker_lib
PLUGIN talker_pkg::Talker # any qualified name — upstream namespaces port verbatim
EXECUTABLE talker
SHAPE configure # this walkthrough uses the configure(Node&) shape
ENTITIES pub:std_msgs/msg/Int32:/chatter timer)
ENTITIES (optional) states what this component’s constructor creates, so the
build can size the executor instead of a human counting call sites. Each entry
is <kind>[:<type>[:<topic>]], with an optional *N repeat on the kind
(pub*5); the kinds are publisher, subscription, timer,
service_server, service_client, action_server, action_client and
guard_condition. Write ENTITIES NONE for a component that creates none.
It is worth stating even though it is optional: NROS_EXECUTOR_MAX_CBS is
derived from it, and the derivation is all-or-nothing. If any component in an
image omits ENTITIES, nothing is derived for that image — a count taken over
only the components that answered is smaller than the image needs, and a short
MAX_CBS fails entity creation at boot. Inspect the composed answer with
nros ws entity-inventory --metadata <build>/nros-metadata.json.
Note that a publisher is inventoried but claims no callback slot, so the
declared entity count and the derived MAX_CBS are legitimately different
numbers.
// src/talker_pkg/include/talker_pkg/Talker.hpp
#pragma once
#include <nros/component.hpp>
#include <nros/nros.hpp>
#include "std_msgs.hpp"
namespace talker_pkg {
class Talker {
::nros::Publisher<std_msgs::msg::Int32> pub_;
::nros::Timer timer_;
int count_ = 0;
void on_tick(); // real body; bound via &Talker::on_tick (no name)
public:
::nros::Result configure(::nros::Node& node);
};
} // namespace talker_pkg
// src/talker_pkg/src/Talker.cpp
#include "talker_pkg/Talker.hpp"
namespace talker_pkg {
void Talker::on_tick() {
std_msgs::msg::Int32 m;
m.data = count_++;
(void)pub_.publish(m);
}
::nros::Result Talker::configure(::nros::Node& node) {
::nros::Result r = node.create_publisher(pub_, "/chatter");
if (!r.ok()) return r;
// Member-fn-pointer-as-template-param → no-alloc trampoline; `this` is ctx.
return ::nros::bind_timer<Talker, &Talker::on_tick>(node, timer_, 1000, this);
}
} // namespace talker_pkg
The Entry pkg constructs Talker in static storage and calls configure(node)
on the real executor — the same component model the Rust nros::node!(Talker) +
the C NROS_C_COMPONENT paths use, so C++, C, and Rust Node pkgs interoperate in
one launch graph.
Scaffold a C++ Node pkg with:
$ nros new --component my-talker --lang cpp --use-case talker
✓ Created nano-ros C++ Node pkg 'my-talker'
Class : my_talker::Talker
Node : talker
Kind : typed component (RFC-0043)
Bringup pkg
Language-agnostic — copy verbatim from the
bringup chapter. package.xml +
system.toml + launch/system.launch.xml. No Cargo.toml, no
CMakeLists.txt. Stock ROS 2 launch.xml from nav2 / Autoware /
turtlebot3 pastes in modulo unsupported tags.
<!-- src/demo_bringup/launch/system.launch.xml -->
<launch>
<node pkg="talker_pkg" exec="talker" name="talker"/>
<node pkg="listener_pkg" exec="listener" name="listener"/>
</launch>
Entry pkg
The C++ Entry pkg’s CMakeLists.txt calls
nano_ros_entry(BRINGUP … LAUNCH …) — you name the input, and the entry
bakes from the SystemModel that nros sync resolved from it (MODEL is
the deprecated artifact-naming override):
# src/native_entry/CMakeLists.txt
cmake_minimum_required(VERSION 3.22)
project(native_entry LANGUAGES C CXX)
nano_ros_workspace_pkg_guard()
nano_ros_entry(
NAME native_entry
SOURCES src/main.cpp # user-authored
BOARD native
BRINGUP "${CMAKE_CURRENT_SOURCE_DIR}/../demo_bringup"
LAUNCH default
DEPLOY native)
At configure time the cmake fn shells nros codegen entry --lang cpp --typed,
emits ${CMAKE_BINARY_DIR}/native_entry_nros_main_generated.cpp (the canonical
int main() body that constructs each launch node’s component + calls
configure(node) on the real executor via LinuxBoard::run_components),
appends it to the target’s sources, and auto-links every
<pkg>_<exec>_component static lib the launch XML named.
BRINGUP + LAUNCH — name your input (canonical)
The canonical spelling names your input: the Bringup package and a
launch selector. nros sync resolves that into a SystemModel build
artifact (under <ws>/build/nros/models/<bringup>/ — never committed;
gate check-no-tracked-models), and the entry bakes from it, so the
same artifact drives the Linux runtime and every embedded image
(contracts, tiers, QoS never drift). MODEL — naming the resolved
artifact file directly — is a deprecated expert override; the two are
mutually exclusive. This is the shape the shipped workspace uses:
nano_ros_add_executable(native_entry
SOURCES src/main.cpp
BOARD native
BRINGUP "${CMAKE_CURRENT_SOURCE_DIR}/../demo_bringup"
LAUNCH default
TYPED
DEPLOY native)
nros sync resolves/refreshes the model whenever the bringup’s launch
XML or system.toml is newer than it. The user’s main.cpp is a single
declarative line:
// src/native_entry/src/main.cpp
#include <nros/main.hpp>
NROS_MAIN(nros::board::LinuxBoard, "demo_bringup:system.launch.xml")
NROS_MAIN(...) is a sentinel macro — the cmake fn owns the generated
body, the user’s TU is documentation + IDE hint.
nros new scaffolds an Entry pkg:
$ nros new my-entry --lang cpp --platform native
Build + boot
# `nros sync` resolves the SystemModel (via the pinned nros-launch-resolve
# helper) whenever the launch XML or system.toml is newer than the model.
nros sync
cmake -S . -B build -DNANO_ROS_ROOT=<path-to-nano-ros>
cmake --build build
./build/src/native_entry/native_entry
The build produces:
src/talker_pkg/libtalker_pkg_talker_component.a— the Node pkg static lib (_componentis the compatibility target suffix).src/listener_pkg/liblistener_pkg_listener_component.a— ditto.src/native_entry/native_entry— the Entry exe, with the generatedint main()+ register-call sequence + Board boot stub linked in.
cmake configure is incremental — pinned CMAKE_CONFIGURE_DEPENDS on
every file nros codegen entry reads (depfile from the CLI), so any
launch.xml / package.xml / Node pkg edit re-runs codegen.
What runs where
| Concern | Lives in |
|---|---|
| Node entities + real callbacks | Node pkg configure(::nros::Node&) |
| Topology + launch args + per-target deploy | Bringup pkg system.toml + launch/*.launch.xml |
int main() + executor init + spin | Generated TU emitted by the Entry pkg’s cmake fn |
| Board + RMW selection | Entry pkg’s nano_ros_entry(BOARD …) arg |
Same partition as the Rust track — the only thing that changes is the syntax the user types into the three pkgs.
C / C++ scaffolding via nros new
| Command | Output |
|---|---|
nros new <name> --lang cpp --platform native | C++ Entry pkg (single-Node self-bringup; swap in a multi-Node LAUNCH arg post-219.D) |
nros new <name> --lang c --platform native | C Entry pkg (same shape) |
nros new --component <name> --lang cpp --use-case talker | C++ Node pkg; --component is the compatibility scaffold flag |
nros new system <name>_bringup --components a,b | Bringup pkg (language-agnostic — works for both Rust and C++ workspaces) |
The C-side compatibility scaffold (nros new --component … --lang c) is
available for pure-C Node pkgs. Pure-C and mixed C/C++ workspace examples
live under examples/workspaces/.
See also
examples/workspaces/cpp/— the canonical reference workspace (talker + listener Node pkgs + Bringup pkg + Entry pkg, all C++).- C++ Entry-pkg roadmap doc — full landing order + acceptance bar.
- Multi-node workspace layout design
§11 — LOCKED canonical shape (
Bringup + Node + Entry).
Mixed-language workspace
nano-ros Node pkgs are linked through a C ABI register trampoline, so one Entry pkg can host Node pkgs written in different languages. The native reference shape is:
- C Node pkg for C code you want to keep in C.
- C++ Entry pkg for the boot harness and generated launch wiring.
- Optional C++ Node pkgs in the same workspace.
- One Bringup pkg with the normal ROS 2 launch XML.
The mixed workspace is in
examples/workspaces/mixed/.
For a pure-C Entry host, use
examples/workspaces/c/.
Layout
my_ws/
├── CMakeLists.txt
└── src/
├── c_talker_pkg/ # C Node pkg
├── cpp_listener_pkg/ # C++ Node pkg
├── demo_bringup/ # launch XML + system.toml
└── native_entry/ # C++ Entry pkg
The root uses the same CMake workspace helper as the C++ track:
include(<nano-ros>/cmake/NanoRosWorkspace.cmake)
nano_ros_workspace(
BACKEND zenoh
PLATFORM posix
SUBDIRS src/c_talker_pkg
src/cpp_listener_pkg
src/native_entry)
C Node pkg
A C Node pkg is a typed component (RFC-0043): no main(). It defines a
state struct + a configure(node, executor, self) function and exports the
C-ABI factory/configure seam via NROS_C_COMPONENT(StateT, configure_fn) — the
typed Entry creates the node and runs configure on the real executor.
# Raw `/chatter` publisher carries the type name as a string → no generated C
# bindings needed (so no nros_find_interfaces for this pkg).
nano_ros_auto_add_library(talker_lib STATIC src/Talker.c)
nros_components_register_node(talker_lib
PLUGIN c_talker_pkg::Talker
EXECUTABLE talker
SHAPE configure
TYPED
DEPLOY native)
#include <stdint.h>
#include <nros/component.h>
typedef struct {
_Alignas(8) uint8_t pub[NROS_C_PUBLISHER_STORAGE_SIZE];
int32_t count;
} c_talker_pkg_t;
static void on_tick(void* ctx) {
c_talker_pkg_t* self = (c_talker_pkg_t*)ctx;
uint8_t buf[8] = {0x00, 0x01, 0x00, 0x00}; /* CDR_LE header + LE int32 */
buf[4] = (uint8_t)self->count;
(void)nros_cpp_publish_raw(self->pub, buf, sizeof(buf));
self->count++;
}
static nros_ret_t talker_configure(const nros_cpp_node_t* node, void* executor,
c_talker_pkg_t* self) {
self->count = 0;
int32_t rc = nros_cpp_publisher_create(node, "/chatter",
"std_msgs::msg::dds_::Int32_", "", nros_c_qos_default(), self->pub);
if (rc != 0) return rc;
size_t timer;
return nros_cpp_timer_create(executor, /*period_ms=*/1000, on_tick, self, &timer);
}
NROS_C_COMPONENT(c_talker_pkg_t, talker_configure)
nano_ros_auto_add_library injects NROS_PKG_NAME (language inferred from the sources), so
NROS_C_COMPONENT exports the __nros_c_component_<pkg>_{create,configure}
seam the typed Entry calls — interoperable with C++ configure(Node&) and Rust
nros::node! components in one launch graph.
Entry pkg
Use a C++ or Rust Entry pkg as the usual host for migration work. In the C++ template:
nano_ros_entry(
NAME native_entry
SOURCES src/main.cpp
BOARD native
MODEL "${CMAKE_CURRENT_SOURCE_DIR}/../demo_bringup/config/system_model.yaml"
DEPLOY native)
The launch file names both Node pkgs:
<launch>
<node pkg="c_talker_pkg" exec="talker" name="talker"/>
<node pkg="cpp_listener_pkg" exec="listener" name="listener"/>
</launch>
The generated Entry translation unit calls each package’s
__nros_component_<pkg>_register symbol and the CMake sidecar links
the matching static libraries. The symbol keeps the legacy
component spelling for ABI compatibility; the user-facing package
role is still Node pkg.
For a pure-C workspace, the Entry pkg uses the same launch-driven shape
with LANG c:
nano_ros_entry(
NAME native_entry
SOURCES src/main.c
BOARD native
MODEL "${CMAKE_CURRENT_SOURCE_DIR}/../demo_bringup/config/system_model.yaml"
LANG c
DEPLOY native)
Scaffolding
# Current compatibility scaffold for Node pkgs:
nros new --component c_talker_pkg --lang c --use-case talker
nros new --component cpp_listener_pkg --lang cpp --use-case listener
nros new system demo_bringup --components c_talker_pkg,cpp_listener_pkg
For a complete working tree, copy the template instead of creating each package separately.
Rust, C, and Mixed-Language Workspaces
The anatomy does not change with the language: node packages hold code, the bringup holds configuration, an entry per image. This page is what changes — per language — inside that fixed shape.
Rust
Scaffold the whole workspace in Rust, or add Rust node packages to an existing one:
nros new my_robot --workspace --lang rust # whole workspace
nros new my_node --component --lang rust # one node pkg, added under src/
A Rust node package is a library crate carrying nros::node!(YourNode);
the entry collapses to one line:
#![allow(unused)]
fn main() {
nros::main!(launch = "demo_bringup", spin = "forever");
}
Two Rust-specific facts:
nros syncbefore the first build, and again after editing.msgfiles or moving the checkout. It generates the message crates and the[patch.crates-io]table that mapsnrosdependencies to your checkout — cargo cannot resolve the workspace without it. C and C++ never need this; their codegen runs inside CMake.- The RMW is spelled in three places (the bringup’s
system.toml, the entry’s board-crate feature, thenrosfacade feature). The scaffold bakes all three from--rmw; changing it later is the checklist in Switching RMW in Config.
C
C node packages join a workspace the same way:
nros new my_c_node --component --lang c
A C node package is the declarative component shape — a configure
function binding callbacks through nros/nros.h — built by the same
per-package CMake as C++. There is no C workspace scaffold; start from
the C++ workspace (nros new --workspace) and add C packages, which is
also what the in-tree reference does.
Mixed — and why it just works
Languages mix per package, not per file, and the bringup does not care:
a [[component]] row names a package and a class/entry point, whatever
implements it. The in-tree reference workspace
(examples/workspaces/mixed/) runs a C talker, a C++ listener, and a
Rust heartbeat under one C++ entry:
mixed/src/
├── c_talker_pkg/ # C — publishes std_msgs/Int32 on /chatter
├── cpp_listener_pkg/ # C++ — subscribes
├── rust_heartbeat_pkg/ # Rust — a timer callback
├── demo_bringup/ # one system.toml for all three
└── native_entry/ # one C++ entry boots them together
Copy it out as a starting point if your project is mixed from day one.
Next
Role Reference: Node, Bringup, and Entry Packages
nano-ros multi-node workspaces split into three kinds of package:
- Node pkg — a reusable, board-agnostic node library. Defines what a node does (publishers, subscribers, timers, services, actions) and registers itself with the
nros::node!(T)macro. Nomain(), no board pick, no deploy config. (Previously called a component package; renamed to Node pkg to match ROS 2 composable-node naming.) - Bringup pkg — pure declarative: owns the launch topology and per-target deploy config. Contains a
package.xml,system.toml,launch/*.launch.xml, and optionalconfig/. NoCargo.toml, no compiled code. Named<system>_bringup. Optional — only required when ≥2 Entry pkgs share one topology; a single-Entry workspace foldslaunch/+system.tomlinto the Entry pkg directly. - Entry pkg — a per-board binary that composes one or more Node pkgs into a runnable system. Owns the board choice (via the
Boardtrait family), the launch file reference, and the deploy/domain/bridge config. Typically ~30 LoC ofmain.rs.
The split exists because a node’s logic is portable across boards, but boot + transport + deploy config is not. One Node pkg can be reused across native POSIX, FreeRTOS, and Zephyr targets by writing one Entry pkg per target.
single-Node convenience: for a single-Node workspace on native hosts, a Node pkg can declare
[package.metadata.nros.entry] deploy = "<board>"in its ownCargo.tomland skip the Entry pkg directory entirely — see Single-Node-pkg convenience below. Embedded boards still need a hand-written Entry pkg.
Node pkg
A Node pkg is a normal Rust library (or C++ static library) with a few nano-ros-specific knobs:
src/talker_pkg/
├── Cargo.toml # [lib] crate-type = ["rlib", "staticlib"]
│ # [package.metadata.nros.node]
│ # class = "talker_pkg::Talker"
├── package.xml # ROS 2 package manifest (<exec_depend> etc.)
├── src/
│ └── lib.rs # impl Node for Talker { … }
│ # nros::node!(Talker);
└── launch/ # OPTIONAL — per-node launch fragment
└── talker.launch.xml
src/lib.rs declares the user class, implements Node +
ExecutableNode (init / on_callback / optional tick), and
ends with nros::node!(Talker); to emit the register trampoline.
Codegen owns the spin loop — your code only describes what the node
has and what its callbacks do.
Key rules:
- No
fn main(). A Node pkg builds asrlib + staticliband is linked into an Entry pkg’s binary. Codegen synthesises the spin driver; you never hand-write one. classmust be a namespace-qualified C++ name. The old 212.L.4 prefix rule (classstarting with the pkg dir name) is retired by RFC-0057 —pkgon the[[component]]row is the identity authority, so upstream namespaces (autoware::x::Node) port verbatim.nros checkstill rejects an unqualifiedclass.- C++ / C analogue:
nano_ros_auto_add_library(<lib> STATIC <srcs>)+nros_components_register_node(<lib> PLUGIN … EXECUTABLE …)(RFC-0057, rclcpp_components keyword parity) + a typed component in the source — C++ aconfigure(::nros::Node&)method, C aNROS_C_COMPONENT(StateT, configure_fn)seam (RFC-0043). Same conceptual shape, no Cargo.toml. package.xmlis mandatory. Even pure-Rust Node pkgs ship one —<exec_depend>lines drive ROS 2 launch discovery when the system runs throughros2 launchoutside the nano-ros toolchain.
Bringup pkg (optional)
A Bringup pkg is pure declarative — it owns the launch topology and per-target deploy config, and contains no compiled code:
src/demo_bringup/
├── package.xml # <name>demo_bringup</name>, <exec_depend> per node
├── system.toml # [system] + [[component]] + [image.<id>] + [host.<name>] (+ [[domain]]/[[bridge]])
├── launch/
│ └── system.launch.xml # ROS 2 launch schema, verbatim
└── config/ # optional — params.yaml, etc.
No Cargo.toml, no CMakeLists.txt, no src/. Naming convention
<system>_bringup (alias <system>_launch), matching nav2 / Autoware /
turtlebot3. It is optional: required only when two or more Entry pkgs
share one topology. A single-Entry workspace folds launch/ + system.toml
into the Entry pkg directly.
launch/*.launch.xml is the ROS 2 launch schema verbatim — <launch>,
<arg>, <node>, <param>, <remap>, <group>, <include>, with
$(find <pkg>) / $(var) / $(env) substitutions. Stock nav2/Autoware
XML pastes in and Just Works; Python .launch.py files are resolved by
nros sync too (the RFC-0060 resolver executes them — see the bringup
tutorial).
See the workspace bringup tutorial.
Entry pkg
An Entry pkg is a binary crate that combines one or more Node pkgs with a board choice, a launch file, and per-board deploy config:
src/robot_entry/
├── Cargo.toml # [[bin]] name = "robot_entry"
│ # [dependencies]
│ # talker_pkg = { path = "../talker_pkg" }
│ # listener_pkg = { path = "../listener_pkg" }
│ # nros-board-linux = { … } # or another family
│ # [package.metadata.nros.entry]
│ # deploy = "native"
│ # [package.metadata.nros.deploy.native]
│ # board = "posix"
│ # rmw = "zenoh"
│ # domain_id = 0
│ # locator = "tcp/127.0.0.1:7447"
├── package.xml # <exec_depend>talker_pkg</exec_depend>, listener_pkg, …
└── src/
└── main.rs # one line: `nros::main!(launch = "demo_bringup");`
The nros::main!() proc-macro reads the launch file
at compile time, walks the workspace pkg-index for each <node pkg=…>
entry, and expands to a fn main() that delegates to
<Board as BoardEntry>::run(...), dispatching one
<pkg>::register(runtime)? call per launch row. The macro has four
forms; pick whichever matches your composition shape:
nros::main!(); // single-node self-bringup (reads [..nros.entry] deploy)
nros::main!(board = LinuxBoard); // single-node, explicit board
nros::main!(launch = "demo_bringup"); // multi-node (CANONICAL): the bringup's default launch
nros::main!(launch = "demo_bringup:sim.launch.xml"); // multi-node, a named launch file
nros::main!( // multi-host: per-host slice via launch args
launch = "demo_bringup:multihost.launch.xml",
args = [("host", "robot1")],
);
// DEPRECATED (expert override): name the resolved model ARTIFACT directly
// instead of the input — skips the sync-freshness contract; bake-time warning.
nros::main!(model = "demo_bringup");
Form-1 (no args) reads
[package.metadata.nros.entry] deploy = "<board>" from this Entry
pkg’s own Cargo.toml and maps the board key
("native"/"freertos"/"zephyr"/…) to the right board crate
via a small lookup table. The launch forms take <bringup>[:<file>]
against launch/<file>.launch.xml + system.toml [system] default_launch; nros sync resolves that input into a SystemModel
under <ws>/build/nros/models/<bringup>/ (a build artifact — never
committed; gate check-no-tracked-models), and the macro bakes from it.
The nros::main!() expansion replaces the older
build.rs + include!(env!("OUT_DIR")/run_plan.rs) shape end-to-end;
new Entry pkgs no longer need a build.rs or a nros-build
build-dep — just nros + the target board crate.
Escape hatch: skip the macro entirely and call
<LinuxBoard as BoardEntry>::run(|runtime| { ... }), or go fully manual
with nros::Executor::open(&ExecutorConfig::default()).
Key rules:
- One Entry pkg per board target. Want to run the same nodes on native POSIX, on a QEMU-MPS2-AN385 FreeRTOS target, and on a real ThreadX board? Three Entry pkgs (
robot_entry_native,robot_entry_qemu_freertos,robot_entry_acme_threadx) sharing the same Node pkgs and (usually) the samelaunch/system.launch.xmlvia symlink or<include>. launch/system.launch.xmlis the canonical name.nros planresolution order:--file <path>→<dir>/launch/<pkg>.launch.xml→<dir>/launch/system.launch.xml→ the single<dir>/launch/*.launch.xml→ synth (only for non-Entry, single-Node pkgs).- Deploy config lives in
Cargo.toml.[package.metadata.nros.deploy.<target>]holds board / RMW / domain / locator per target;[[package.metadata.nros.domain]]and[[package.metadata.nros.bridge]]carry multi-domain topology. - C++ analogue: cmake fn
nano_ros_entry(NAME <name> LANG CXX BRINGUP … LAUNCH …)plusNROS_MAIN(...)(MODELis the deprecated artifact-naming override, mirroring the macro). Metadata flows through${BUILD}/nros-metadata.jsonrather than a sidecar TOML.
Workspace shape
A typical multi-Node workspace, with one Entry pkg per supported board:
my_ws/
├── Cargo.toml # [workspace] members = ["src/talker_pkg", "src/listener_pkg", "src/robot_entry"]
│ # [workspace.metadata.nros] default_system = "demo_bringup"
└── src/
├── talker_pkg/ # Node pkg (lib, nros::node!)
├── listener_pkg/ # Node pkg
├── demo_bringup/ # Bringup pkg (declarative; no Cargo.toml)
└── robot_entry/ # Entry pkg (bin, nros::main!(launch = "demo_bringup"))
cargo build at the workspace root builds everything via cargo’s native scheduler. nros plan reads [workspace.metadata.nros] default_system to pick the Entry pkg (or you pass nros plan robot_entry explicitly).
For C++-majority or mixed workspaces, CMake is the top-level driver instead — see the multi-node workspace design doc.
Single-Node-pkg convenience (cargo run Just Works)
For tiny fixtures and host-side dev loops, a Node pkg can declare itself as its own Entry pkg by adding [package.metadata.nros.entry] deploy = "<board>" to its Cargo.toml, alongside the usual [package.metadata.nros.node] and [package.metadata.nros.deploy.<target>] tables. src/main.rs collapses to one line:
// src/main.rs
nros::main!();
The macro reads deploy = "<board>" from this pkg’s own Cargo.toml,
maps it to the right board crate, and emits
fn main() + <this_pkg>::register(runtime)?; — the latter resolves
through the companion src/lib.rs cargo auto-wires alongside the
binary target. No build.rs, no launch file (one is synthesised
in-memory), no hand-written boot glue. This is the L.7 self-entry
planner path.
Limits of the convenience:
- Native only. Embedded boards (FreeRTOS, ThreadX, Zephyr, bare-metal) still require a hand-written Entry pkg — board init is non-trivial enough that hiding it behind a one-liner does more harm than good.
- One Node. Two or more Node pkgs in the same workspace = author an Entry pkg. The point of the convenience is to skip ceremony for tiny single-node fixtures, not to grow into a multi-node composition root.
Quick reference
| You want… | Use |
|---|---|
| Reusable node logic, board-independent | Node pkg (nros::node!()) |
| Per-board binary that runs N nodes | Entry pkg (main.rs calls BoardEntry::run) |
cargo run on host for a single-node fixture | Node pkg with [package.metadata.nros.entry] deploy = "native" |
| Same nodes on multiple boards | One Node pkg set + one Entry pkg per board |
| Launch topology + per-target deploy config | Bringup pkg (declarative; optional, folds into Entry pkg when single-target) |
| Board hardware bringup | Board trait family (see porting chapter) |
How integration works
You already have an RTOS project that builds and runs — an STM32Cube or NXP MCUXpresso tree, a Pico SDK app, an ESP-IDF project, a vendored Zephyr or NuttX workspace. This section is about adding nano-ros to that project, without changing who is in charge.
The principle
nano-ros is a library your project imports. Your RTOS keeps its own
build tool. You keep building with west, make, idf.py, cmake,
or your IDE, exactly as before. nano-ros never compiles your kernel,
never owns your linker script, and never replaces your build system.
What nano-ros contributes to your build is small and fixed:
- a Rust static library (the client runtime, behind a stable C ABI),
- generated code for your messages and system configuration,
- a thin C shim that your build compiles, so it sees your config
headers (
FreeRTOSConfig.h,lwipopts.h,sdkconfig, …).
The last point is why the shim is source, not a prebuilt: config-header mismatches between your kernel and a prebuilt binary are silent ABI breaks. Your compiler, your headers, your flags.
One shape per host build
Each supported host build system gets a small “shell” — build glue plus the host’s own package manifest — that makes nano-ros look native to it. Pick your row:
| Your build | How nano-ros plugs in | Shell in the repo | Start here |
|---|---|---|---|
Zephyr (west) | west module | zephyr/ (module.yml, Kconfig, CMakeLists.txt) | Zephyr (west module) |
NuttX (make + Kconfig) | apps/external/ app | integrations/nuttx/ (Make.defs, Kconfig, Makefile) | NuttX (apps/external) |
ESP-IDF (idf.py) | IDF component | integrations/nano-ros/ (idf_component.yml, CMakeLists.txt, Kconfig.projbuild) | ESP32 (ESP-IDF component) |
| FreeRTOS, CMake project | add_subdirectory(nano-ros) | cmake/platform/nano-ros-freertos.cmake | Build as a CMake subdirectory, FreeRTOS (QEMU) |
| ThreadX, CMake project | add_subdirectory(nano-ros) | cmake/platform/nano-ros-threadx.cmake | Build as a CMake subdirectory, ThreadX |
PX4 (make px4_…) | EXTERNAL_MODULES_LOCATION copy-out | integrations/px4/module-template/ | PX4 (integration shell) |
PlatformIO (pio run) | library + pre-build codegen script | library.json (repo root) + integrations/platformio/ | integrations/platformio/README.md in the repo |
| IDE project (no CMake) | emitted drop-in sources | planned — no shell ships today | see note below |
Notes on the rows that need them:
- Vendored FreeRTOS — STM32Cube, MCUXpresso, Pico SDK, and upstream
FreeRTOS-Kernelall count. Your SDK builds the kernel and network stack; your CMake project doesadd_subdirectory(nano-ros)and linksNanoRos::NanoRos. See the honest caveat in FreeRTOS: who builds the kernel? - PX4 is a hookless host: its configure step cannot run nano-ros codegen from inside, so codegen runs ahead of the PX4 build, emitting module directories PX4 then builds natively. PlatformIO works the same way, via a pre-build script.
- IDE hosts (STM32CubeIDE, MCUXpresso IDE, IAR, Keil) have no build hook nano-ros can join. A drop-in emitter (prebuilt static library + shim sources + an integration README) is designed but not shipped yet. Today, the practical route is your IDE’s external-CMake or makefile support driving the CMake subdirectory path.
In every row the division of labor is the same: your build owns
compile and link; nano-ros owns the adapter shell and the host-time
code generation. Generated configuration is baked to C at build time
on your host — system.toml and launch files never reach the device.
Two homes for configuration
When you integrate against your own board, you author facts in exactly two places. The split is by what kind of fact it is:
1. A board package — facts intrinsic to the board. A directory in
your workspace with a package.xml and an nros-board.toml: the
board’s name, RTOS platform, target triple, architecture, entry shape.
These are true of every copy of that board on every desk, so they are
reusable and shareable. Discovery: the CLI’s board catalog scans
<nano-ros>/packages/boards/ plus every root named by
NROS_EXTRA_BOARD_PATH (PATH-style list of directories shaped like
packages/boards/), so an out-of-tree board dir is seen without
copying it into the checkout.
2. An [image.<id>] block, and a [board_config.<board>] beside it — facts about this checkout, this
machine, this application. It lives in your bringup package’s
system.toml, next to your nodes and launch files:
[image.my-board]
board = "my-board" # joins to the board package by name
# the rustc triple comes from that board's descriptor
[board_config."my-board"] # site facts: where YOUR SDK lives, which config
sdk.vendor = "{env:VENDOR_SDK_DIR}" # headers are yours — keyed by BOARD
SDK paths, config-header locations, serial ports: anything another developer’s checkout would spell differently belongs here, keyed per deploy target, and never in the board package.
If a fact would survive being published with the board, it is a board fact. If it names a path on your disk, it is a deploy fact.
FreeRTOS: who builds the kernel?
One honest exception to the principle. On the QEMU reference board,
nano-ros can build the FreeRTOS kernel and lwIP for you — point
FREERTOS_DIR at a kernel checkout and
cmake/platform/nano-ros-freertos.cmake compiles it into the image.
That is a convenience for the out-of-the-box demo, and it is the one
place nano-ros compiles an RTOS.
With a vendored kernel — STM32Cube’s Middlewares/Third_Party/ FreeRTOS, MCUXpresso’s or the Pico SDK’s bundled kernel — do not use
that path. Your SDK’s build owns the kernel, its port selection, and
FreeRTOSConfig.h; nano-ros compiles against your include directories
and links in as a library, the same as every other row in the table.
Where to go next
- Zephyr workspace: Zephyr (west module)
- NuttX board: NuttX (apps/external)
- ESP-IDF app: ESP32 (ESP-IDF component)
- PX4 firmware: PX4 (integration shell)
- Plain CMake host (FreeRTOS, ThreadX, POSIX): Build as a CMake subdirectory
- Deploy targets and
system.toml: Deployment Workflow
FreeRTOS (QEMU MPS2-AN385)
Single-node starter on FreeRTOS + lwIP, cross-compiled for Cortex-M3 and booted in QEMU MPS2-AN385. Slirp networking; no host TAP / bridge / sudo. Rust, C, and C++ talkers all live in-tree.
Prereqs.
nros setup qemu-arm-freertosis the single command that prepares your machine for this board. It fetches a prebuilt toolchain set into the shared store at~/.nros/sdk— thearm-none-eabi-gcccross-compiler, the patchedqemu-system-armemulator, and the FreeRTOS kernel + lwIP sources. You do not hand-install a cross-toolchain. The zenoh path DOES need a ROS 2 install for the router (ros2 run rmw_zenoh_cpp rmw_zenohd); the xrce path needs only the Micro-XRCE-DDS agent, whichnros setupinstalls.
Setup
Build the in-tree nros CLI (Phase 218):
./scripts/bootstrap.sh # builds packages/cli/target/release/nros
source ./activate.sh # OR: direnv allow / source ./activate.fish
Then provision the board (--rmw defaults to zenoh; pick xrce
or cyclonedds to match the example you intend to run):
nros setup qemu-arm-freertos --rmw zenoh
This fetches the cross-compiler, the patched qemu-system-arm, and the
FreeRTOS + lwIP sources into ${NROS_HOME:-~/.nros}/sdk (plus the
Micro-XRCE-DDS agent when you pick --rmw xrce). The zenoh router is
NOT installed — it comes from your ROS 2 install (RFC-0075).
Project layout
Each language uses the standard nano-ros canonical example shape —
standalone Cargo (Rust) or CMake (C / C++) project under
examples/qemu-arm-freertos/<lang>/<example>/.
examples/qemu-arm-freertos/
├── rust/talker/ # Cargo package, cross-compile target = thumbv7m-none-eabi
│ ├── Cargo.toml # deps + [package.metadata.nros.deploy.freertos]
│ ├── .cargo/config.toml # target + QEMU runner
│ ├── package.xml
│ ├── generated/ # codegen output — build.rs runs
│ │ # `nros generate-rust` on first
│ │ # `cargo build`; gitignored.
│ └── src/lib.rs # the component class; nros::main! generates the entry
├── c/talker/ # CMake project, add_subdirectory consumption
│ ├── CMakeLists.txt # targets (deploy tuple in package.xml)
│ ├── package.xml
│ └── src/Talker.c
└── cpp/talker/ # CMake C++14 project
├── CMakeLists.txt
├── package.xml
└── src/Talker.cpp
The Rust Cargo.toml pulls the FreeRTOS board crate
(nros-board-mps2-an385-freertos) which wraps the kernel + lwIP +
LAN9118 driver build. The C / C++ CMakeLists.txt follows the
canonical add_subdirectory(<repo-root>) + nano_ros_link_rmw(<target> RMW zenoh) pattern with
NANO_ROS_BOARD = mps2-an385-freertos.
Configure
Deploy config (router locator, domain, RMW) is declared in the build
manifest and baked at compile time — there is no config file on the
device. Verbatim from the in-tree
examples/qemu-arm-freertos/rust/talker/Cargo.toml:
[package.metadata.nros.deploy.freertos]
board = "qemu-mps2-an385"
rmw = "zenoh"
domain_id = 0
locator = "tcp/10.0.2.2:7447"
The C / C++ trees declare the same in their package.xml <export> tuple
(RFC-0048 §4); the connect locator rides the build config
(-DNROS_ENTRY_LOCATOR / the fixture row), not the tuple:
<export>
<build_type>ament_cmake</build_type>
<nano_ros deploy="freertos" board="mps2-an385-freertos" rmw="zenoh"/>
</export>
Task stacks / priorities come from the board crate’s defaults (Cargo features), not a config file — see the Configuration Guide.
The 10.0.2.0/24 subnet is QEMU Slirp’s default; 10.0.2.2 is the
Slirp gateway that forwards to host loopback. No TAP, no sudo.
Ports: the shipped examples dial host port 7447.
Contributors: the prebuilt test fixtures bake different ports — see Per-Platform Contributor Lanes.
Build
# Rust — `nros sync` first, once per checkout location. It writes the
# generated message bindings and the [patch.crates-io] table the leaf's
# .cargo/config.toml includes; without it cargo fails while PARSING the
# manifest, with an error that never names sync (see below).
cd examples/qemu-arm-freertos/rust/talker
nros sync
cargo build --release
# C / C++ — use the cross-toolchain CMake invocation (the `nros` CLI
# on PATH auto-resolves the codegen tool — no `-D_NANO_ROS_CODEGEN_TOOL=`
# needed):
toolchain="$(pwd)/cmake/toolchain/arm-freertos-armcm3.cmake"
cd examples/qemu-arm-freertos/c/talker
cmake -B build -DCMAKE_TOOLCHAIN_FILE="$toolchain" \
-DCMAKE_BUILD_TYPE=Release
cmake --build build --parallel
Contributors: the in-tree fixture build lanes for this platform are in Per-Platform Contributor Lanes.
First Rust build pulls + cross-compiles deps (~5 min). C / C++ build also compiles FreeRTOS kernel + lwIP — first run ~3 min.
If you skipped nros sync, the Rust build stops before it starts, with
failed to load config include '../../../../../nros-patch.toml' and
No such file or directory. That file is generated, not committed —
run nros sync in the leaf and build again. The C / C++ builds above do
not need it; see
Workflow by Platform and Language
for why the requirement is per language rather than per platform.
Run
# 1. Start the router (ROS's `rmw_zenohd`) on the host, on port 7447 —
# the deploy locator the example bakes in its Cargo.toml
# ([package.metadata.nros.deploy.freertos] locator, shown above).
# Slirp forwards guest 10.0.2.2:<p> → host:<p>.
ZENOH_CONFIG_OVERRIDE='listen/endpoints=["tcp/127.0.0.1:7447"];scouting/multicast/enabled=false' \
ros2 run rmw_zenoh_cpp rmw_zenohd &
# 2. Boot the talker in QEMU. The leaf's .cargo/config.toml runner
# wraps qemu-system-arm with the LAN9118 + Slirp wiring:
cd examples/qemu-arm-freertos/rust/talker
cargo run --release
# 3. Verify from stock ROS 2:
source /opt/ros/humble/setup.bash
export RMW_IMPLEMENTATION=rmw_zenoh_cpp
# Talker publishes best-effort; stock `ros2 topic echo` defaults to
# RELIABLE, so the QoS-mismatched echo silently delivers nothing.
# Force best-effort to receive:
ros2 topic echo /chatter std_msgs/msg/String --qos-reliability best_effort
QEMU exits via Ctrl-A x.
Contributors: the in-tree fixture run/test lanes for this platform are in Per-Platform Contributor Lanes.
Readiness signal. Within ~20 seconds of QEMU boot, the talker
should print Publishing: 'Hello World: 1' on its semihosting
stdout — the count starts at 1, matching the official ROS 2 demo
talker. QEMU cold-boot through FreeRTOS init + lwIP DHCP + zenoh
session open typically takes 10–15 s. If no Publishing: line in
30 seconds:
- Confirm the router is running on the host on the port the image
dials (7447 for the copy-out example — Slirp forwards
10.0.2.2:7447→ host:7447). Without it the talker retries the zenoh handshake until QEMU is killed. - Check the talker’s early log for
lwIP DHCP timeoutorFailed to open session. - Bridge tip:
ros2 topic echo /chatterfrom a stock ROS 2 install (withRMW_IMPLEMENTATION=rmw_zenoh_cpp) confirms end-to-end interop. - See Troubleshooting — First 10 Minutes.
GitHub source
Canonical, copy-out:
- Rust:
examples/qemu-arm-freertos/rust/talker/ - C:
examples/qemu-arm-freertos/c/talker/ - C++:
examples/qemu-arm-freertos/cpp/talker/
Next
- Subscriber: peer
listener/directory next to each talker. - Services + actions: peer
service-*/andaction-*/directories. - Real hardware: same code runs on STM32F4-Discovery / NXP-LPC55S69 / TI-MSP432 with a different board crate + linker script; see the Bare-metal Cortex-M3 page for the no-RTOS variant.
- Your own board / RTOS: the Board Integration matrix maps each user profile (Cargo-first, vendor-IDE, Zephyr, ESP-IDF, NuttX, niche fork) to the shortest bring-up path.
- RTOS-specific debugging: FreeRTOS LAN9118 Debugging.
Zephyr (west module)
Single-node starter on Zephyr via the in-tree zephyr/
west module. nano-ros ships as a Zephyr module — west discovers it
from your workspace’s west.yml, drops in a prj.conf Kconfig
surface, and the standard west build / west flash flow takes
care of the rest.
Contributor path? Building nano-ros’s own Zephyr examples straight from this repository (no west-managed workspace) is covered at Zephyr (contributor). The page below is the canonical user entry.
Just want a working starter? Clone the
nano-ros-zephyr-examplerepo (west init -m …) — a manifest + zenoh talker app pinned to a tested Zephyr, with the same quickstart as below baked in. The steps here explain what it does so you can adapt it to your own workspace.
Prereqs. Run
nros setup zephyr --rmw <rmw>once (see Prerequisites below) — it provisions the Zephyr west workspace + Zephyr SDK bits, the emulator, and your RMW’s host daemon into the shared store. No hand-installed Zephyr SDK,west, or cross-toolchain, and no ROS 2 install required. Python: 3.10+ on Zephyr 3.7 LTS, but ≥ 3.12 on Zephyr 4.x (4.x’sfind_package(Python3)requires 3.12 — see the version matrix below). nano-ros’s imported west fragmentzephyr/west.ymlis a manifest-only file — it does NOT pull Zephyr itself; that has to be in your parent manifest (zephyrproject-rtos/zephyr).
Which Zephyr? — 3.7 LTS and 4.x both supported
nano-ros consumes as a module on both the 3.7 LTS line (supported to Jan 2027; the safety-island default) and the current 4.x rolling line. You build against whatever Zephyr your workspace already pins — nano-ros adapts. The two lines differ only in how you select an RMW and apply nano-ros’s Zephyr patches:
| Capability | Zephyr 3.7 LTS | Zephyr 4.x |
|---|---|---|
| Min Python | 3.10 | 3.12 (find_package(Python3)) |
| RMW selection | prj-<rmw>.conf overlay (-DCONF_FILE=...) | -S nros-<rmw> snippet (or the overlay) |
| nano-ros patches | applied during nros setup zephyr provisioning | applied during nros setup zephyr provisioning |
| Examples as samples / Twister | — | samples: + Twister (sample.nano-ros.*) |
| zenoh (native_sim) | ✅ build + e2e | ✅ build + e2e |
| cyclonedds (native_sim) | ✅ build + e2e | ✅ build · publish · receive · multicast-join (stable 2-node run pending a tracked k_mutex fix) |
| xrce | ✅ | build path WIP |
native_sim networking uses NSOS (host loopback) on both lines — no
TAP/bridge/root. The copy-out, snippet, patch-apply, and dual-line build
flows are exercised in CI (contributor lanes just zephyr ci-both,
just zephyr check-copy-out).
Project layout
A Zephyr workspace using nano-ros looks like any other Zephyr project — the nano-ros module sits beside Zephyr, your application sits beside both:
my_zephyr_ws/
├── .west/
├── zephyr/ # cloned by `west init`
├── modules/
│ └── nano-ros/ # imported via west.yml
└── apps/
└── my_app/ # your application
├── CMakeLists.txt
├── prj.conf # Kconfig — selects nros + RMW
├── west.yml # (optional) per-app manifest
└── src/
└── main.c # nros user code
The application CMakeLists.txt is a stock Zephyr app — find_package(Zephyr)
target_sources. Noadd_subdirectory(<nano-ros>)is needed; the module shell handles it onceCONFIG_NROS=yflips on.
A minimal apps/my_app/ looks like this — the shipped
examples/zephyr/c/talker/
quoted directly (only the project/class names are trimmed to my_app), so
this page can’t drift from the real API surface. It is a stateful C
component (a struct + a configure function, RFC-0043 / phase-244.C2) —
not a hand-written main() — because the Zephyr typed carrier generates
the entry point and calls into the component by identity:
# apps/my_app/CMakeLists.txt
cmake_minimum_required(VERSION 3.20.0)
find_package(Zephyr REQUIRED HINTS $ENV{ZEPHYR_BASE})
project(my_app)
set(NANO_ROS_PLATFORM zephyr)
include("${NROS_REPO_DIR}/cmake/NanoRosNodeRegister.cmake")
include("${NROS_REPO_DIR}/cmake/NanoRosVerbs.cmake")
nano_ros_auto_add_library(talker_lib STATIC src/Talker.c)
nros_components_register_node(talker_lib
PLUGIN my_app::Talker
EXECUTABLE talker
SHAPE configure
TYPED
DEPLOY zephyr)
// apps/my_app/src/Talker.c
// `talker_configure` creates a raw publisher on `/chatter` + a timer that
// publishes a CDR-encoded Int32 counter each tick. `NROS_C_COMPONENT` emits
// the C-ABI factory/configure the Zephyr typed Entry carrier calls.
#include <stddef.h>
#include <stdint.h>
#include <stdio.h>
#include <nros/component.h>
typedef struct {
_Alignas(8) uint8_t pub[NROS_C_PUBLISHER_STORAGE_SIZE];
int32_t count;
} talker_t;
static void write_u32_le(uint8_t* p, uint32_t v) {
p[0] = (uint8_t)v;
p[1] = (uint8_t)(v >> 8);
p[2] = (uint8_t)(v >> 16);
p[3] = (uint8_t)(v >> 24);
}
static void on_tick(void* ctx) {
talker_t* self = (talker_t*)ctx;
/* std_msgs/Int32 CDR: 4-byte encapsulation header (CDR_LE) + int32 data. */
uint8_t buf[8];
buf[0] = 0x00;
buf[1] = 0x01;
buf[2] = 0x00;
buf[3] = 0x00;
write_u32_le(buf + 4, (uint32_t)self->count);
if (nros_cpp_publish_raw(self->pub, buf, sizeof(buf)) == 0) {
printf("Published: %d\n", (int)self->count);
}
self->count++;
}
static nros_ret_t talker_configure(const nros_cpp_node_t* node, void* executor, talker_t* self) {
self->count = 0;
int32_t rc = nros_cpp_publisher_create(node, "/chatter", "std_msgs::msg::dds_::Int32_", "",
nros_c_qos_default(), self->pub);
if (rc != 0) {
return rc;
}
size_t timer_handle;
return nros_cpp_timer_create(executor, /*period_ms=*/500, on_tick, self, &timer_handle);
}
NROS_C_COMPONENT(talker_t, talker_configure)
See examples/templates/zephyr-byo/app/src/Talker.c
for the up-to-date source (this is that BYO starter’s talker, only stripped
of its file header comment; the canonical in-tree
examples/zephyr/c/talker/
uses the generated typed std_msgs/String bindings instead of raw CDR).
Note the nros_cpp_* symbol prefix: those are C-ABI
functions from nros-cpp (the cpp is a namespace prefix, not C++
linkage) — a C component links against them directly, and shares the same
executor + node as a C++ component would.
prj.conf is the one shown in Configure below — note it needs
both CONFIG_NROS_C_API=y and CONFIG_NROS_CPP_API=y: the typed
Zephyr carrier that drives this component’s configure is C++, so the
nros-cpp header surface must be on the app include path even though
Talker.c itself is C.
Prerequisites
nros setup provisions the parts nano-ros owns — the RMW host daemon
(zenohd / Micro-XRCE-DDS agent) and the RMW transport submodules
(zenoh-pico + mbedtls for zenoh, the cyclonedds fork) — from a pinned index into
${NROS_HOME:-~/.nros}/sdk / the nano-ros checkout. It does not replace Zephyr’s own SDK,
and interface codegen still needs the ROS message definitions.
- Build the in-tree
nrosCLI (Phase 218, from the nano-ros checkout):./scripts/bootstrap.sh # builds packages/cli/target/release/nros source ./activate.sh # OR: direnv allow / source ./activate.fish - Provision the RMW (daemon + transports) from the nano-ros checkout:
( cd modules/nano-ros && nros setup zephyr --rmw zenoh ) # zenohd + zenoh-pico + mbedtls ( cd modules/nano-ros && nros setup --source px4-rs ) # workspace cargo-load dep - Install the Zephyr SDK the standard Zephyr way (
nros setupdoes not provide it) and expose it —export ZEPHYR_SDK_INSTALL_DIR=/path/to/zephyr-sdk-<ver>(or register it via the SDK’ssetup.sh -c). - Message definitions for codegen. The interface codegen resolves a
package’s
msg/*.msgfromNROS_<PKG>_DIR(e.g.export NROS_STD_MSGS_DIR=/opt/ros/humble/share/std_msgs) — point it at a ROS install or any dir holding the.msgfiles.
The RMW host daemon must be running before an example connects
(for zenoh: ZENOH_CONFIG_OVERRIDE='listen/endpoints=["tcp/127.0.0.1:7447"];scouting/multicast/enabled=false' ros2 run rmw_zenoh_cpp rmw_zenohd; the Micro-XRCE-DDS agent for xrce).
Configure
Add nano-ros (and a Zephyr pin — zephyr/west.yml is manifest-only;
it does not pull Zephyr itself) to your workspace west.yml:
manifest:
remotes:
- name: nano-ros
url-base: https://github.com/NEWSLabNTU
- name: zephyr
url-base: https://github.com/zephyrproject-rtos
projects:
- name: zephyr
remote: zephyr
revision: v3.7.0 # or your chosen 3.7 LTS / 4.x SHA
path: zephyr
import: true # pulls Zephyr's own modules
- name: nano-ros
remote: nano-ros
revision: nros-v0.5.0 # pin a release tag (west needs SOME
# revision — its default `master` does
# not exist here). `main` works for
# tracking tip, but a tag keeps your
# build reproducible.
path: modules/nano-ros
import:
file: zephyr/west.yml # pulls nano-ros's transport deps
Then per-application prj.conf:
CONFIG_NROS=y
CONFIG_NROS_C_API=y
CONFIG_NROS_RMW_ZENOH=y # bool per RMW: NROS_RMW_{ZENOH,XRCE,CYCLONEDDS}
# (ROS edition is a build-time Cargo feature, NOT a Kconfig symbol — do not set
# CONFIG_NROS_ROS_EDITION; Zephyr aborts on the undefined symbol.)
# Required for any networked RMW on QEMU / native_sim:
CONFIG_NETWORKING=y
CONFIG_NET_IPV4=y
CONFIG_NET_TCP=y
CONFIG_NROS=y activates the shell, which maps Kconfig values to
NANO_ROS_* CMake cache vars and add_subdirectory()s the root
nano-ros CMake. NanoRos::NanoRos is linked into your app
library transparently.
Build
If your workspace is a fresh manifest-only dir (no .west/), initialise
it first so west knows which west.yml is the manifest:
cd my_zephyr_ws
west init -l . # one-time; points west at the
# local west.yml in cwd
west update # clones nano-ros + Zephyr into the workspace
(If you started from west init -m <remote>, both calls above are
already done — go straight to west build below.)
nros build on Zephyr: it hands off, it does not take over
The other platforms let nros build own the build: it generates a cargo or
CMake root from your [image.*] declarations and runs the tool. Zephyr is not
like that, on purpose.
You own the west workspace and the application. west init / west update
populate the workspace, your app carries its own CMakeLists.txt and
prj.conf, and west only runs from inside that workspace. None of that is
ours to generate — an app’s Kconfig is authored, not derived (RFC-0065 D5).
So for a Zephyr image nros build:
- resolves the application — the entry package for that image’s board, not the bringup;
- resolves your Zephyr —
$ZEPHYR_BASE, else$NROS_ZEPHYR_WORKSPACE, else an in-treezephyr-workspace/, else a sibling../nano-ros-workspace. Note it is a Zephyr that is needed, not a.west/directory: withZEPHYR_BASEset,west buildruns from anywhere, which is what makes a freestanding application — one outside the west workspace — work; - runs
west buildfrom there with the image’s overlays (EXTRA_CONF_FILE,APPLICATION_CONFIG_DIR) already applied.
If it cannot find a workspace it does not guess. It prints the command and where to run it:
$ nros build demo_bringup:zephyr
Error: no west workspace found, so `west build` cannot be run for `zephyr`.
Zephyr differs from the other drivers: YOU own the west workspace and the
application, and `west` only runs inside a workspace …
Run this from your west workspace:
west build -b native_sim/native/64 …/src/zephyr_entry
Point nros at your workspace:
NROS_ZEPHYR_WORKSPACE=<dir> # the dir containing zephyr/
That is deliberate and it is the same boundary nros setup --system draws when
it composes an install command and prints it rather than running sudo for you:
where the tool is yours, nano-ros hands you the command instead of assuming it
may act.
The plain west build below keeps working and is still the primary flow.
nros build is the convenience that applies an image’s overlays for you; it
never becomes a required layer between you and west.
Starting from nothing: nros new node and nros new entry
A node package is board- and RMW-agnostic — it names no platform, which is what lets the same package be deployed to Linux and to a Cortex-M without an edit:
nros new node talker_pkg
It writes the package and adds the [[component]] row to your bringup. (Use
nros new <name> --platform native only for a standalone runnable project —
it pins a board crate, so an entry on another platform cannot link it.)
The entry: nros new entry
You do not have to write the entry package by hand:
nros new entry zephyr_entry --platform zephyr
nros new entry: scaffolded ./src/zephyr_entry (7 file(s))
declared [image.zephyr_entry] in ./src/demo_bringup/system.toml
Next:
nros sync
nros build demo_bringup:zephyr_entry --zephyr-workspace <dir>
It writes the package — Cargo.toml, CMakeLists.txt, build.rs,
src/lib.rs, prj.conf, prj-zenoh.conf, boards/<board>.conf — and three
things outside it that are easy to miss and invisible until they fail:
- the
[image.*]block, withboard,entryand theconfoverlay the entry’s ownCMakeLists.txtrequires; - an
excludeentry in every enclosing cargo workspace, because a west-builtstaticlibmust not be an ordinary cargo member; - a path dependency per node package the bringup declares, since the entry links them.
--board, --rmw and --bringup override the defaults
(native_sim/native/64, zenoh, and the workspace’s single bringup).
The whole flow, end to end
Five commands, run from your workspace. This is the examples/workspaces/rust
transcript, verbatim:
# 0. tell nros where your west workspace is — the directory containing
# `zephyr/`. On the command line:
#
# nros build demo_bringup:zephyr --zephyr-workspace /path/to/zephyr-workspace
#
# or once for the shell, if you would rather not repeat it:
export NROS_ZEPHYR_WORKSPACE=/path/to/zephyr-workspace
# 1. generate the message bindings this workspace's packages depend on.
# `nros build` refuses with "this workspace has never been synced" until
# you do — it will not silently build against absent bindings.
nros sync
# 2. build. This resolves the application, applies the image's overlays and
# runs `west build` for you.
nros build demo_bringup:zephyr
# 3. start the router ROS itself uses — the one `rmw_zenoh_cpp` ships, so it
# cannot drift from the RMW your ROS 2 nodes use. Separate terminal; it
# keeps running. Listens on tcp/127.0.0.1:7447.
ros2 run rmw_zenoh_cpp rmw_zenohd
# 4. run the image.
./build/zephyr/zephyr.exe
*** Booting Zephyr OS build v3.7.0 ***
<inf> nros_net_wait: Network ready (NSOS — host kernel sockets)
<inf> rust: rustapp: nros: zephyr workspace entry up (2 nodes)
<inf> rust: talker_pkg: talker publishing chatter seq=0
<inf> rust: talker_pkg: talker publishing chatter seq=1
and from a ROS 2 shell on the same router, the nodes are ordinary ROS nodes:
$ export RMW_IMPLEMENTATION=rmw_zenoh_cpp
$ ros2 topic echo /chatter --once
data: 6
---
Step 3 is not optional and its ORDER does not matter, but the locator does: zenoh-pico is a client, so with no router the image gets
<err> rust: nros: zephyr entry — executor open failed: Transport(ConnectionFailed)
which names no port. Both sides default to tcp/127.0.0.1:7447 — the port
rmw_zenohd listens on and a rmw_zenoh_cpp node connects to. Override with
CONFIG_NROS_ZENOH_LOCATOR on the image side (via a conf fragment) and
ZENOH_CONFIG_OVERRIDE='listen/endpoints=["tcp/127.0.0.1:<port>"]' on the
router, and change both.
What an image declares
The Zephyr images in examples/workspaces/* are the worked examples:
[image.zephyr]
board = "native_sim/native/64"
entry = "zephyr_entry" # only when several entries match, see below
conf = ["prj-zenoh.conf"] # the RMW overlay this app requires
confnames the app’s own Kconfig fragments, in order. They are looked for beside the board config dir, then beside the application, then beside the bringup — the app rung matters because a Zephyr app keeps itsprj-<rmw>.confnext to theCMakeLists.txtwest builds. These entriesFATAL_ERRORwithout an RMW overlay, so the image has to say which one.boardis the Zephyr board target — and also a name nano-ros must know. See below.entrynames the application package. Normally leave it out: an entry declares the deploy target it serves —[package.metadata.nros.entry] deployinCargo.toml,nano_ros_add_executable(... DEPLOY zephyr)inCMakeLists.txt— and one package usually claims a given board. Set it when several do (realtime-cpphaszephyr_entryandfvp_entry, bothDEPLOY zephyr, on the same board, for two images that differ in payload). Deriving there is a coin flip, sonros buildrefuses and lists the candidates rather than picking one.
If you live in west: west nros
Everything above assumes you stand in your nano-ros workspace and name the
Zephyr one. If you work the other way round — inside the west workspace — west
already knows where Zephyr is, so it can tell nros for you:
cd ~/zephyr-workspace
west nros build --workspace ~/my_robot demo_bringup:zephyr
No ZEPHYR_BASE, no NROS_ZEPHYR_WORKSPACE, no --zephyr-workspace. Every
argument after nros is passed straight through, so anything nros accepts
works here unchanged.
It picks up automatically in any workspace whose manifest lists nano-ros — the
same route west fvp takes. Two things worth knowing:
- It only exists inside a west workspace. West loads extension commands
from the manifest, so from an unrelated directory you get
west: unknown command "nros". That is why this is an addition rather than a replacement: it swaps which of the two trees you have to name, trading the Zephyr path (which differs per machine) for your own workspace path (which you chose). - It never overwrites a
ZEPHYR_BASEyou set yourself — an exported one is a deliberate choice, and this only fills in a missing answer.
Which boards you can name
board does two jobs from one string: it is passed to west build -b
verbatim, and it is looked up in nano-ros’s board catalog, where a descriptor
carries a set of names:
names = ["zephyr", "native_sim/native/64"]
So it is not free-form. A board the catalog does not know is refused with the
list of ones it does, because the descriptor supplies more than the -b string
— the platform, the toolchain, the entry kind, the declared capabilities. There
is no safe default for those.
A board is a package — put it in your workspace. Drop an nros-board.toml
beside a package’s package.xml and it joins the catalog like any other
package in src/:
my_robot/src/
├── talker_pkg/
└── my_board/
├── package.xml
└── nros-board.toml
# src/my_board/nros-board.toml
[[board]]
names = ["my-board"] # what your images say
west_board = "qemu_cortex_m3" # what west is given
platform = "zephyr"
toolchain = "stable"
platform_feature = "platform-zephyr"
link_kind = "none"
entry_kind = "zephyr-staticlib"
# src/demo_bringup/system.toml
[image.zephyr]
board = "my-board"
No environment variable, nothing outside the tree, nothing copied into the
nano-ros checkout. west_board is what lets the friendly name be the one your
workspace uses — leave it out when your board name already is the west board
target, which is what the shipped descriptors do.
For a board shared by several workspaces there is no single workspace to
put it in, and $NROS_EXTRA_BOARD_PATH still covers that: PATH-style, naming
directories shaped like packages/boards/.
nros new board <name> --for-platform zephyr scaffolds the descriptor. Note
this is the nano-ros descriptor; an out-of-tree Zephyr board definition
(its devicetree and board.yml) is contributed the Zephyr way, through your
module’s board_root.
Adding your own drivers, modules and boards
Everything Zephyr lets you extend, you still extend the Zephyr way. The
application CMakeLists.txt is yours (RFC-0065 D5) and nros build runs
west build against it, so the extension points are untouched:
| you want | you do | nros build involvement |
|---|---|---|
| an out-of-tree module | list(APPEND ZEPHYR_EXTRA_MODULES <dir>) before find_package(Zephyr) | none |
| a driver in that module | zephyr_library() + zephyr_library_sources(), gated on your own Kconfig | none |
| its devicetree binding | dts/bindings/ + settings: {dts_root: .} in the module’s zephyr/module.yml | none |
| an out-of-tree board / SoC / arch / snippet | the matching *_root in the same module.yml (board_root, soc_root, arch_root, snippet_root, module_ext_root) | none |
| turning your Kconfig on for one image | a conf fragment on [image.*] | passed as EXTRA_CONF_FILE |
| a devicetree overlay for one image | an .overlay in the same conf list | passed as EXTRA_DTC_OVERLAY_FILE |
Only the last two rows are ours, and they are how an IMAGE differs from its
siblings — the module itself is a property of the application, not of one
image, so it belongs in the app’s CMakeLists.txt where you would have put it
anyway.
nros build passes exactly four things to west of its own —
APPLICATION_CONFIG_DIR, EXTRA_CONF_FILE, EXTRA_DTC_OVERLAY_FILE,
FILE_SUFFIX, all Zephyr’s own variables — and there is deliberately no
image key for arbitrary -D defines. Something your application always needs
belongs in its CMakeLists.txt, which keeps one place to look.
Customizing the west command itself
Anything after -- goes to west. west build has two argument zones, and
nros build routes each token to the right one using west’s own flag list:
nros build demo_bringup:zephyr -- --pristine # a west option
nros build demo_bringup:zephyr -- -t run # build, then run it
nros build demo_bringup:zephyr -- -DMY_OPT=1 # a cmake option
nros build demo_bringup:zephyr -- -p always -DX=1 # both, in one go
which becomes, respectively, the west option zone and the cmake zone:
west build -b <board> <app> -p always -- -DEXTRA_CONF_FILE=… -DX=1
nros build --dry-run prints that line without running it, so you can always
see exactly what west is being asked to do — and copy it if you would rather
drive west yourself.
Two flags are refused rather than routed, because the image already decides them and a build must not be able to disagree with its own declaration:
$ nros build demo_bringup:zephyr -- --sysbuild
Error: `--sysbuild` is decided by the image, not by the command line.
It comes from the presence of a `sysbuild.conf` beside the application.
Change it there, so one build cannot disagree with the declaration it was
resolved from.
The other is -b/--board, which comes from the image’s board.
Verified by building one. An out-of-tree module with its own Kconfig, a driver
source gated on it, a devicetree overlay declaring a node and a binding shipped
from the module’s dts_root, added to examples/workspaces/rust’s entry and
enabled through the image:
[image.zephyr]
conf = ["prj-zenoh.conf", "prj-user-extra.conf", "user-widget.overlay"]
*** Booting Zephyr OS build v3.7.0 ***
user_extra: out-of-tree driver init, widget-id=42
<inf> rust: rustapp: nros: zephyr workspace entry up (2 nodes)
<inf> rust: talker_pkg: talker publishing chatter seq=0
The driver’s BUILD_ASSERT(DT_NODE_EXISTS(DT_NODELABEL(user_widget))) is what
makes that a test rather than a screenshot: without the overlay reaching the
build, it fails to compile instead of quietly doing nothing.
A mistake in your module surfaces as Zephyr’s own error, unmediated — a wrong
kconfig: path in module.yml reports
ERROR: "kconfig" key in …/extra_module/zephyr/module.yml has value
"../Kconfig" which does not point to a valid Kconfig file.
which is the message you would get from a plain west build, because it is a
plain west build.
The transports + px4-rs come from the prerequisites step
(west update clones nano-ros but not its submodules). With the Zephyr SDK +
NROS_STD_MSGS_DIR exported (also prerequisites), build your app — nros on
PATH is auto-resolved as the codegen tool:
# native_sim (POSIX, no QEMU). The 3.7 line needs the NSOS line overlay; apply
# the NSOS patches first (see "apply nano-ros's patches" below).
overlay="$PWD/modules/nano-ros/cmake/zephyr/native-sim-line-3.7.conf"
west build -b native_sim/native/64 apps/my_app -- -DCONF_FILE="prj.conf;$overlay"
# A real board, e.g. Cortex-A9 (no native_sim overlay):
west build -b qemu_cortex_a9 apps/my_app
(Verified end-to-end on a fresh BYO west workspace: this builds to
zephyr.exe and runs to Published: 0 against a router started with
ZENOH_CONFIG_OVERRIDE='listen/endpoints=["tcp/127.0.0.1:7447"];scouting/multicast/enabled=false' ros2 run rmw_zenoh_cpp rmw_zenohd.
On the Zephyr 4.4 line, find_package(Python3) requires ≥ 3.12 and you
select the RMW with -S nros-zenoh instead of the overlay.)
For a quick sanity check that the module is wired correctly:
west build -t menuconfig # confirm CONFIG_NROS=y is visible
Rust applications
Two things differ for a Rust app (C/C++ apps skip this section):
-
The Rust crate’s
[lib]must be namedrustapp(crate-type = ["staticlib"]) — azephyr-lang-rustcontract: itsrust_cargo_application()linkslibrustapp.a. The Cargo package name is free. -
Generate the interface crates + the
[patch.crates-io]wiring for YOUR layout — do not copy an in-repo example’s.cargo/config.toml: its../../../../packages/core/...paths are repo-relative and break in a copied-out app. From your app dir, run (after the Prerequisitesnros setup, which provides the codegen toolchain + message sources):nros generate-rust --generate-config \ --nano-ros-path "$PWD/../../modules/nano-ros/packages/core"This writes
generated/<pkg>/(the message crates) and a.cargo/config.tomlwhose[patch.crates-io]points thenros-*crates at yourmodules/nano-ros/packages/core/*and the generated interfaces atgenerated/*. Adjust--nano-ros-pathto your workspace’smodules/nano-ros/packages/core(the dir holdingnros-core,nros-node, …). The example apps’ committed.cargo/config.tomlis for the in-tree build only.
Run
# 1. Start the router (ROS's `rmw_zenohd` — nano-ros ships no router
# of its own) on port 7447, the port your app's Kconfig default
# picks up (CONFIG_NROS_ZENOH_LOCATOR defaults to
# "tcp/127.0.0.1:7447"; change either side to match):
ZENOH_CONFIG_OVERRIDE='listen/endpoints=["tcp/0.0.0.0:7447"];scouting/multicast/enabled=false' \
ros2 run rmw_zenoh_cpp rmw_zenohd &
# 2. Boot the app from your BYO west workspace:
# QEMU Cortex-A9:
west build -t run
# native_sim:
./build/zephyr/zephyr.exe
# 3. Verify from stock ROS 2 in another terminal. Match the message
# type to the app you booted: the canonical in-tree talker publishes
# std_msgs/String; the raw BYO Talker.c shown above publishes
# std_msgs/Int32.
source /opt/ros/humble/setup.bash
export RMW_IMPLEMENTATION=rmw_zenoh_cpp
# Talker publishes best-effort; stock `ros2 topic echo` defaults to
# RELIABLE, so the QoS-mismatched echo silently delivers nothing.
# Force best-effort to receive:
ros2 topic echo /chatter std_msgs/msg/String --qos-reliability best_effort # in-tree talker
# ros2 topic echo /chatter std_msgs/msg/Int32 --qos-reliability best_effort # BYO my_app
Contributors: nano-ros’s own zephyr talker has a matching in-tree recipe for the canonical
native_simbuild path — see Per-Platform Contributor Lanes.
The Zephyr boot banner runs first, then the talker fires: the
canonical in-tree talker prints Publishing: 'Hello World: 1',
Publishing: 'Hello World: 2', … (Rust + C + C++ all count from
1, matching the official ROS 2 demo talker); the raw BYO Talker.c
above prints Published: 0, Published: 1, …
Readiness signal. On native_sim, expect the first publish line
(Publishing: 'Hello World: 1' for the in-tree talker;
Published: 0 for the BYO app) within 5 seconds of boot (or
./build/zephyr/zephyr.exe); on qemu_cortex_a9 expect it within
~15 seconds (QEMU cold boot + Zephyr init). If no publish line
in 30 seconds:
- Confirm
CONFIG_NROS=ylit up viawest build -t menuconfig; without it the module shell neveradd_subdirectory’s nano-ros. - Check
CONFIG_NETWORKING=y,CONFIG_NET_IPV4=y,CONFIG_NET_TCP=yinprj.conf— Zephyr networking is opt-in. - Confirm
zenohdreachable from the simulated network (Slirp needs10.0.2.2:7447on QEMU; native_sim uses host loopback). - See Troubleshooting — First 10 Minutes.
Zephyr 4.x build gotchas.
Could NOT find Python3 ... required is at least "3.12"— 4.x needs Python ≥ 3.12. Provision one without sudo (e.g.uv venv --python 3.12 .venv312 && uv pip install west -r zephyr/scripts/requirements.txt) and run west through it (.venv312/bin/python -m west build ...), so the ROS descriptor-codegen subprocess still uses the system ROS Python.attempt to assign the value ... to the undefined symbol ETH_NATIVE_POSIX— that symbol was renamedETH_NATIVE_TAPin 4.x; the version-aware NSOS overlay handles it (contributors: the in-treejust zephyr build-onerecipe applies it automatically).
Zephyr 4.x: select the RMW with a snippet
On 4.x, nano-ros ships west snippets so you pick the RMW on the
build line instead of hand-writing the overlay:
west build -b native_sim/native/64 -S nros-cyclonedds apps/my_app
# ^^^^^^^^^^^^^^^^^^^ nros-zenoh | nros-cyclonedds | nros-xrce
The snippet (shipped via the module’s snippet_root) carries the
RMW-common Kconfig — equivalent to merging prj-<rmw>.conf. The
prj-<rmw>.conf / -DCONF_FILE path still works (and is the only option
on 3.7).
nano-ros patches into your workspace
nano-ros needs a few patches to Zephyr’s native_sim NSOS driver
(recvmsg, IPv4-multicast) so the host-loopback-based examples work.
Both supported lines (3.7 LTS and 4.x) take the same path —
nros setup zephyr reads zephyr/patches.yml and applies each patch
against the workspace’s Zephyr tree, sha256-checked. No extra step
on your side beyond the provisioning command (already run during
Prerequisites).
To re-apply (e.g. after a west update reset the tree):
nros setup zephyr --rmw zenoh
(Earlier nano-ros revisions documented a west patch apply flow on
4.x — that required a workspace-side west extension that doesn’t ship
with stock Zephyr. Phase 208.D.7 / E.9 unified both lines on the
provisioner. Cyclone-DDS-on-Zephyr patches stay baked into the pinned
cyclonedds submodule, not delivered through patches.yml.)
Copy out an example as your starting point
The examples/zephyr/<lang>/<role>/ dirs are copy-out clean — copy one
into your own app tree and it builds against the nano-ros module with no
reference back into the nano-ros repo:
cp -r modules/nano-ros/examples/zephyr/c/talker apps/my_app
# cyclonedds examples need the host idlc + the ROS message dirs:
export NROS_STD_MSGS_DIR=/opt/ros/humble/share/std_msgs # PKG_DIR contract
west build -b native_sim/native/64 -S nros-zenoh apps/my_app
Cyclone idlc and the descriptor-gen scripts are located via the module’s
exported cache vars (NROS_CYCLONE_IDLC, NROS_CYCLONE_SCRIPTS_DIR,
NROS_CYCLONE_CMAKE_DIR); message-package dirs come from NROS_<PKG>_DIR
env (defaulting to /opt/ros/humble/share/<pkg>). No /opt/ros or
repo-relative paths are baked into the example.
See the zephyr/
module dir + its Kconfig
for the canonical in-repo surface.
GitHub source
- Zephyr module shell:
zephyr/ - Worked examples:
examples/zephyr/rust/,examples/zephyr/c/,examples/zephyr/cpp/ - Module manifest:
zephyr/module.yml - Kconfig surface (canonical post-Phase-208.D.7 fold — every
CONFIG_NROS*symbol the doc cites lives here):zephyr/Kconfig - Patches applied by
nros setup zephyr(thewest patchflow on this page was retired in Phase 208.E.9):zephyr/patches.yml
Next
- Pick a real board (Nordic, NXP, STM32, …): swap
-b <board>and add a board-specific overlay to yourprj.conf. - Cyclone DDS on Cortex-A/R: see the DDS section of Choosing an RMW Backend for the required Kconfig deltas.
- Build nano-ros’s own Zephyr examples without west: Zephyr (contributor).
Zephyr (contributor / in-tree workflow)
Looking for the user-facing path? This page covers building nano-ros’s own Zephyr examples from this repository. If you’re consuming nano-ros as a Zephyr module in YOUR Zephyr workspace, see Integration: Zephyr (
westmodule) instead.
Complete setup procedure for Zephyr native_sim testing. Networking uses
NSOS (Native Sim Offloaded Sockets) — each socket call is forwarded to
the host kernel, so tests run on 127.0.0.1 without TAP devices, bridges,
or sudo.
Overview
nros uses an in-tree Zephyr workspace at zephyr-workspace/ (gitignored).
Set $NROS_ZEPHYR_WORKSPACE to install elsewhere.
nros/
├── scripts/zephyr/
│ ├── setup.sh # Initialize workspace
│ ├── migrate-workspace.sh # Move legacy sibling install in-tree
│ ├── downloads/ # SDK tarball cache (gitignored)
│ └── sdk/ # Installed Zephyr SDK (gitignored)
├── zephyr/ # Zephyr module definition
│ ├── Kconfig # RMW backend, API selection, tuning
│ ├── CMakeLists.txt # Transport C sources + nros-c build
│ └── cmake/ # nros_cargo_build(), nros_generate_interfaces()
├── examples/zephyr/
│ ├── rust/ # Rust + zenoh (talker, listener, ...)
│ ├── rust/xrce/ # Rust + XRCE-DDS (talker, listener)
│ ├── c/ # C + zenoh (talker, listener)
│ └── c/xrce/ # C + XRCE-DDS (talker, listener)
├── west.yml # West manifest
└── zephyr-workspace/ # Created by setup.sh (gitignored)
├── nros -> ../ # Symlink back to repo root
├── zephyr/ # Zephyr RTOS v3.7.0
└── modules/ # HALs, zephyr-lang-rust
Migrating from the legacy sibling layout
Legacy setups put the workspace at ../nano-ros-workspace/ with an
in-tree symlink. Both layouts work (contributors: the in-tree
just zephyr recipes auto-detect either), but to consolidate run:
./scripts/zephyr/migrate-workspace.sh --dry-run # preview
./scripts/zephyr/migrate-workspace.sh # execute
Prerequisites
Build the in-tree nros CLI (Phase 218), then let it provision the toolchain:
./scripts/bootstrap.sh # builds packages/cli/target/release/nros
source ./activate.sh # OR: direnv allow / source ./activate.fish
nros setup provides most components prebuilt per platform per RMW — the
cross-compiler, emulator, RMW host daemon, and SDK sources (including the Zephyr
west workspace + Zephyr SDK bits) are fetched from a pinned index into a shared
store at ${NROS_HOME:-~/.nros}/sdk. You do not hand-install a cross-toolchain, and you do not
need ROS 2 installed.
Packages are prebuilt where the index has a binary for your host and built from
source otherwise — zenohd is the notable source build, and zenoh is the default
--rmw. --dry-run prints the plan for your host. See
Installation for the full explanation.
Step 1: Initialize Workspace (One-Time)
nros setup zephyr --rmw zenoh # --rmw defaults to zenoh; xrce | cyclonedds also valid
source ./activate.sh
This provisions:
- The Zephyr west workspace + Zephyr SDK bits
- The emulator
- The RMW host daemon (
zenohdfor zenoh, the Micro-XRCE-DDS agent for xrce) - The in-tree workspace at
zephyr-workspace/(gitignored; auto-detects legacy../nano-ros-workspace/), with nros symlinked in - Rust embedded targets
Contributors: the in-tree
just setup zephyrrecipe still works and now delegates tonros setup zephyrunder the hood.
The RMW host daemon must be running before any example: zenohd for zenoh,
the Micro-XRCE-DDS agent for xrce. nros setup zephyr --rmw <rmw> installs it.
Step 2: Networking
No network setup is required. native_sim uses the NSOS offloaded-sockets
driver, enabled by boards/native_sim_native_64.conf in each example:
CONFIG_ETH_NATIVE_POSIX=n
CONFIG_NET_SOCKETS_OFFLOAD=y
CONFIG_NET_NATIVE_OFFLOADED_SOCKETS=y
With NSOS, Zephyr’s socket API goes straight to host syscalls. Bind to
127.0.0.1 and reach zenohd / the XRCE Agent on the host loopback just
like any other native test. Multiple native_sim processes can coexist
without bridge configuration.
Step 3: Build and Run Zephyr Examples
# Source environment
source zephyr-workspace/env.sh
cd zephyr-workspace
# Rust — `nros sync` first, once per checkout location. It writes the
# generated message bindings and the [patch.crates-io] table the leaf's
# .cargo/config.toml includes. This applies to cargo under WEST too: west
# drives the same leaf, so without sync the build fails while PARSING the
# manifest, with an error that never names sync (issue 0694).
#
# `--no-metadata` because the source-metadata probe builds a host binary that
# path-deps this leaf, dragging in the `zephyr` crate — whose build.rs needs a
# DOTCONFIG from a Zephyr cmake configure that has not run yet, and cannot,
# since this sync is its prerequisite (issue 0318). Without the flag the sync
# still succeeds but prints a "no producer" error the reader cannot act on.
# Codegen and the patch table — the reason to run it — are unaffected.
nros sync --no-metadata ../examples/zephyr/rust/talker
# Build Zephyr talker (Rust + zenoh, default backend).
#
# `../examples/...` rather than `<name>/examples/...`: the workspace's link
# back to the checkout is named after YOUR checkout directory
# (`scripts/zephyr/setup.sh` uses `basename "$NANO_ROS_ROOT"`), so it reads
# `nano-ros/` for a plain `git clone` and `nros/` only if you renamed the
# directory. For the in-tree workspace this page sets up, `..` IS the
# checkout and needs no name at all.
west build -b native_sim/native/64 ../examples/zephyr/rust/talker
# Run (no sudo needed)
./build/zephyr/zephyr.exe
RMW Backend Selection
nros supports three RMW backends on Zephyr, selected via prj.conf:
Zenoh (default)
Connects to a zenoh router. Requires POSIX API for zenoh-pico threads.
CONFIG_NROS=y
# CONFIG_NROS_RMW_ZENOH=y # default, can be omitted
CONFIG_NROS_ZENOH_LOCATOR="tcp/127.0.0.1:7447"
CONFIG_POSIX_API=y
CONFIG_MAX_PTHREAD_MUTEX_COUNT=32
CONFIG_MAX_PTHREAD_COND_COUNT=16
XRCE-DDS
Connects to a Micro-XRCE-DDS Agent over UDP. Requires BSD sockets.
CONFIG_NROS=y
CONFIG_NROS_RMW_XRCE=y
CONFIG_NROS_XRCE_AGENT_ADDR="127.0.0.1"
CONFIG_NROS_XRCE_AGENT_PORT=2018
CONFIG_NET_SOCKETS=y
Cyclone DDS
Brokerless RTPS, wire-compatible with stock ROS 2 (rmw_cyclonedds_cpp).
Cyclone’s source is C++, so CONFIG_CPP=y is required even for Rust
callers. Cyclone is resource-heavy — it needs a large heap, libc malloc
arena, and pthread pools. The bool prerequisites (thread-local storage,
dynamic threads, NET_TCP, …) are selected automatically by
CONFIG_NROS_RMW_CYCLONEDDS in zephyr/Kconfig; the size knobs stay in
prj.conf:
CONFIG_NROS=y
CONFIG_NROS_RMW_CYCLONEDDS=y
CONFIG_CPP=y
CONFIG_POSIX_API=y
CONFIG_NET_IPV4_IGMP=y # RTPS SPDP uses UDP multicast
CONFIG_MAIN_STACK_SIZE=524288
CONFIG_HEAP_MEM_POOL_SIZE=4194304
CONFIG_COMMON_LIBC_MALLOC_ARENA_SIZE=16777216
CONFIG_DYNAMIC_THREAD_STACK_SIZE=32768
On native_sim, add the NSOS host-socket offload
(CONFIG_NET_SOCKETS_OFFLOAD=y + CONFIG_NET_NATIVE_OFFLOADED_SOCKETS=y)
so discovery uses host BSD sockets instead of zeth/TAP. See
examples/zephyr/rust/talker/prj-cyclonedds.conf for the full overlay.
API Selection
Choose between Rust and C APIs via prj.conf:
Rust API (default)
CONFIG_NROS_RUST_API=y
CONFIG_RUST=y
CONFIG_RUST_ALLOC=y
CMakeLists.txt uses rust_cargo_application():
cmake_minimum_required(VERSION 3.20.0)
find_package(Zephyr REQUIRED HINTS $ENV{ZEPHYR_BASE})
project(my_example)
rust_cargo_application()
C API
CONFIG_NROS_C_API=y
CMakeLists.txt uses nros_generate_interfaces():
cmake_minimum_required(VERSION 3.20.0)
find_package(Zephyr REQUIRED HINTS $ENV{ZEPHYR_BASE})
project(my_example)
nros_generate_interfaces(std_msgs "msg/Int32.msg")
target_sources(app PRIVATE src/main.c)
Kconfig Reference
All options are under menuconfig NROS in zephyr/Kconfig.
Common Options
| Option | Type | Default | Description |
|---|---|---|---|
CONFIG_NROS | bool | n | Enable nros module |
CONFIG_NROS_RUST_API | bool | y | Use Rust API |
CONFIG_NROS_C_API | bool | n | Use C API |
CONFIG_NROS_DOMAIN_ID | int | 0 | ROS 2 domain ID |
CONFIG_NROS_INIT_DELAY_MS | int | 2000 | Network init wait (ms) |
Zenoh Options (visible when CONFIG_NROS_RMW_ZENOH=y)
| Option | Type | Default | Description |
|---|---|---|---|
CONFIG_NROS_ZENOH_LOCATOR | string | "tcp/127.0.0.1:7447" | Router address |
CONFIG_NROS_ZENOH_MULTI_THREAD | bool | y | Zenoh-pico multithreading |
CONFIG_NROS_ZENOH_PUBLICATION | bool | y | Publication support |
CONFIG_NROS_ZENOH_SUBSCRIPTION | bool | y | Subscription support |
CONFIG_NROS_ZENOH_QUERY | bool | y | Service client support |
CONFIG_NROS_ZENOH_QUERYABLE | bool | y | Service server support |
CONFIG_NROS_ZENOH_LINK_TCP | bool | y | TCP transport link |
CONFIG_NROS_MAX_PUBLISHERS | int | 8 | Max concurrent publishers |
CONFIG_NROS_MAX_SUBSCRIBERS | int | 8 | Max concurrent subscribers |
CONFIG_NROS_MAX_QUERYABLES | int | 8 | Max concurrent queryables |
CONFIG_NROS_FRAG_MAX_SIZE | int | 2048 | Max reassembled message size |
CONFIG_NROS_BATCH_UNICAST_SIZE | int | 1024 | Max unicast batch size |
CONFIG_NROS_SUBSCRIBER_BUFFER_SIZE | int | -1 (derive) | Per-subscriber buffer, small payload class |
CONFIG_NROS_MAX_LARGE_SUBSCRIBERS | int | -1 (derive) | Blocks in the large payload class |
CONFIG_NROS_SUBSCRIBER_LARGE_SIZE | int | -1 (derive) | Per-sample capacity of the large class |
CONFIG_NROS_SERVICE_BUFFER_SIZE | int | 1024 | Per-service buffer |
-1 means “derive it from the message bounds”
Three of the rows above, plus CONFIG_NROS_SUBSCRIPTION_BUFFER_SIZE, default to
-1. That is not a size: it is how an image says nothing here chose a number,
work it out (phase-403 W8, issue 0940).
Codegen derives an exact serialized-size bound for every message type it
generates and writes it beside the code as nros_message_bounds.cmake. The
build composes those, and the configure prints what it concluded:
-- nros: message-bound sizing DERIVED from 84 types in 11 interface packages (all bounded)
-- nros: largest type std_msgs/msg/Float64MultiArray at 1496 B -> NROS_SUBSCRIPTION_BUFFER_SIZE
-- nros: 0 types over the 2048 B ceiling -> NROS_MAX_LARGE_SUBSCRIBERS
Three things to know before you rely on it:
- Anything you state wins. A value in your
prj.conf, a board.conf, or an environment override outranks the derivation, silently and in both directions. The derived number is a default, never an override. - It is an UPPER BOUND. The inventory covers every type in the interface
closure you LINK, not the ones you subscribe to, so the number is the largest
message the image could receive. That is the safe direction, and it can
still be far above what you need: the line above names
std_msgs/Float64MultiArray, which that image never receives. When the type it names is not one of yours, either state your own value or stop linking the package. - One unbounded type refuses the lot. If any type in the closure has no
derived bound, nothing is derived, every knob keeps its configured value, and
the configure names the types and the members that cost them their bound.
Deriving over the rest would publish a maximum a real sample can exceed, and
that drop is silent on the C++ arena path. The remedy is a bound in the
.msgor aninlinecap innros-codegen.toml; one cap on a declaring type ("std_msgs/Header.frame_id") reaches every message that nests it.
The answer, and why it came out that way, is written to
<build>/nros/message_bound_knobs.cmake.
XRCE Options (visible when CONFIG_NROS_RMW_XRCE=y)
| Option | Type | Default | Description |
|---|---|---|---|
CONFIG_NROS_XRCE_AGENT_ADDR | string | "127.0.0.1" | Agent IP address |
CONFIG_NROS_XRCE_AGENT_PORT | int | 2018 | Agent UDP port |
CONFIG_NROS_XRCE_TRANSPORT_MTU | int | 512 | Transport MTU |
CONFIG_NROS_XRCE_MAX_SUBSCRIBERS | int | 8 | Max concurrent subscribers |
CONFIG_NROS_XRCE_MAX_SERVICE_SERVERS | int | 4 | Max service servers |
CONFIG_NROS_XRCE_MAX_SERVICE_CLIENTS | int | 4 | Max service clients |
CONFIG_NROS_XRCE_BUFFER_SIZE | int | 1024 | Per-slot buffer size |
CONFIG_NROS_XRCE_STREAM_HISTORY | int | 4 | Reliable stream depth (2-16) |
C API Options (visible when CONFIG_NROS_C_API=y)
| Option | Type | Default | Description |
|---|---|---|---|
CONFIG_NROS_C_MAX_HANDLES | int | 16 | Max executor handles |
CONFIG_NROS_C_MAX_SUBSCRIPTIONS | int | 8 | Max subscriptions |
CONFIG_NROS_C_MAX_TIMERS | int | 8 | Max timers |
CONFIG_NROS_C_MAX_SERVICES | int | 4 | Max services |
E2E Testing
Contributors: the in-tree fixture/test lanes for this platform are in Per-Platform Contributor Lanes.
Troubleshooting
| Issue | Solution |
|---|---|
west: command not found | Run pip3 install --user west and add ~/.local/bin to PATH |
Connection refused | Start zenohd / MicroXRCEAgent on the host loopback (e.g. tcp/127.0.0.1:7447) |
Build fails | Source environment: source zephyr-workspace/env.sh |
XRCE Agent not found | Provision the xrce daemon: nros setup zephyr --rmw xrce |
| Zenoh mutex exhaustion | Increase CONFIG_MAX_PTHREAD_MUTEX_COUNT (default 5 is too low) |
Network Architecture
With NSOS, Zephyr sockets are forwarded to host syscalls — there is no emulated L2/L3 stack to configure, no static IP, and no bridge.
┌─────────────────────────────────────────────────────────────┐
│ Host (Linux) │
│ │
│ ┌────────────────────┐ ┌────────────────────────┐ │
│ │ zephyr.exe talker │ │ zephyr.exe listener │ │
│ │ (native_sim+NSOS) │ │ (native_sim+NSOS) │ │
│ └─────────┬──────────┘ └──────────┬─────────────┘ │
│ │ host socket() via NSOS │ │
│ ▼ ▼ │
│ 127.0.0.1 (loopback) │
│ │ │ │
│ ▼ ▼ │
│ ┌────────────────────┐ ┌────────────────────────┐ │
│ │ zenohd │ │ MicroXRCEAgent │ │
│ │ tcp/127.0.0.1:7447 │ │ udp/127.0.0.1:2018 │ │
│ └────────────────────┘ └────────────────────────┘ │
└─────────────────────────────────────────────────────────────┘
Updating the Workspace
To update Zephyr and modules to latest versions specified in west.yml:
cd zephyr-workspace
west update
To completely recreate the workspace:
Contributors: the in-tree workspace-recreate lane is in Per-Platform Contributor Lanes.
NuttX (apps/external)
Single-node starter on NuttX. nano-ros plugs into the standard NuttX
app discovery as an external app under apps/external/nano-ros/,
exposing Kconfig knobs under Application Configuration → External Modules → nano-ros. Use this entry when your NuttX board ships its
own kernel build and you want to add ROS 2 communication.
Contributor path? Building nano-ros’s own NuttX QEMU examples straight from this repository (no NuttX-managed workspace) is covered at NuttX (contributor). The page below is the canonical user entry.
Prereqs. Install the
nrosCLI once per machine, then runnros setup qemu-arm-nuttx --rmw <zenoh|xrce|cyclonedds>(--rmwdefaults tozenoh). This fetches a mostly-prebuilt toolchain set into${NROS_HOME:-~/.nros}/sdk— the NuttX cross-compiler, the emulator, the NuttX sources, and the RMW host daemon — so you do not hand-install a cross-toolchain and do not need a ROS 2 install:./scripts/bootstrap.sh # builds packages/cli/target/release/nros (Phase 218) source ./activate.sh # OR: direnv allow / source ./activate.fish nros setup qemu-arm-nuttx --rmw zenohYou still need a NuttX ≥ nuttx-12 checkout with an
apps/sibling and Python 3.10+ for the NuttX configure scripts.
Project layout
NuttX’s “external apps” pattern places the app shim under
$NUTTX_APPS_DIR/external/<name>/:
$NUTTX_DIR/ # NuttX kernel checkout
$NUTTX_APPS_DIR/ # sibling: apps tree
└── external/
└── nano-ros/ # symlink or submodule of
├── Make.defs # integrations/nuttx/
├── Makefile
├── CMakeLists.txt # (cmake-driven NuttX builds)
└── Kconfig
my_app/ # your application
├── package.xml
├── Cargo.toml | CMakeLists.txt
├── generated/ # Rust codegen — build.rs runs
│ # `nros generate-rust` on first
│ # `cargo build`; gitignored.
└── src/main.{rs,c,cpp}
Wire the shell into your NuttX apps tree. Easiest path:
just setup nuttx # contributor helper: stages the shell +
# example apps into $NUTTX_APPS_DIR/external/
# (delegates to `nros setup qemu-arm-nuttx`
# for the toolchain/SDK provisioning)
This runs scripts/nuttx/stage-external-apps.sh, which writes
$NUTTX_APPS_DIR/external/Make.defs + Kconfig and symlinks the
integration shell in as external/nano-ros. Menuconfig surfaces it
under Application Configuration → External Modules.
Passing --bringup <dir> additionally stages a per-bringup app
(external/<bringup>/, with the bringup tree symlinked alongside as
external/<bringup>-source) — the Phase 212 multi-package workspace
shape. There are no per-EXAMPLE symlinks: that staging loop was
retired in phase-212.M-F.12 when the bringup path superseded it.
If you’d rather wire it yourself (e.g. into a vendored apps tree):
ln -s /path/to/nano-ros/integrations/nuttx \
$NUTTX_APPS_DIR/external/nano-ros
# then copy integrations/nuttx/external-Make.defs.in →
# $NUTTX_APPS_DIR/external/Make.defs and add a matching
# $NUTTX_APPS_DIR/external/Kconfig that `source`s the shell.
Configure
NuttX uses Kconfig as its single source of truth. After the symlink above:
cd $NUTTX_DIR
make menuconfig
# Navigate to:
# Application Configuration → External Modules → nano-ros
# [*] nano-ros ROS 2 client
# RMW backend → zenoh # zenoh | xrce | cyclonedds
# ROS 2 edition → humble
Networking Kconfig requirements live under
Networking Support — enable CONFIG_NET, CONFIG_NET_TCP,
CONFIG_NET_IPv4. For QEMU nsh_smp configurations the defaults
already include these.
Deploy config (RMW / domain id, plus an optional locator override) is
declared in the build manifest and baked at compile time. Verbatim from
the in-tree
examples/qemu-arm-nuttx/rust/talker/Cargo.toml:
[package.metadata.nros.deploy.nuttx]
board = "qemu-armv7a-nsh"
target = "armv7a-nuttx-eabihf"
rmw = "zenoh"
domain_id = 0
The C / C++ variants declare the same in their package.xml <export>
tuple (RFC-0048 §4):
<export>
<build_type>ament_cmake</build_type>
<nano_ros deploy="nuttx" board="nuttx-qemu-arm" rmw="zenoh"/>
</export>
The guest network shape (eth0 10.0.2.30, Slirp gateway 10.0.2.2)
comes from the board crate; add a locator = "tcp/10.0.2.2:<port>"
field to dial a non-default router port. The prebuilt test fixtures
bake distinct per-language allocator ports so parallel suites don’t
collide on one router. Start the router (ROS’s rmw_zenohd) on the
port your app dials — it must listen on 0.0.0.0, not loopback,
because the guest reaches it through QEMU’s Slirp gateway:
ZENOH_CONFIG_OVERRIDE='listen/endpoints=["tcp/0.0.0.0:8200"];scouting/multicast/enabled=false' \
ros2 run rmw_zenoh_cpp rmw_zenohd
Build
cd $NUTTX_DIR
make # full kernel + apps build
The Cargo build of nano-ros’s Rust staticlibs runs as a sub-step of
the NuttX app build; libnros_c.a is linked at the final app
link stage.
For CMake-driven NuttX builds:
cmake -B build -DBOARD=qemu-armv7a \
-DCONFIG=nsh_smp
cmake --build build
Run
# 1. Start the router on the host (Slirp forwards 10.0.2.2:<port> →
# host). It must listen on 0.0.0.0 for the Slirp gateway; 8200 is
# the port the in-tree lanes use — match whatever `locator` your
# app bakes:
ZENOH_CONFIG_OVERRIDE='listen/endpoints=["tcp/0.0.0.0:8200"];scouting/multicast/enabled=false' \
ros2 run rmw_zenoh_cpp rmw_zenohd &
# 2. QEMU NuttX (ARM). For a NuttX-managed workspace where you've
# staged the integration shell + your own app, boot QEMU directly:
qemu-system-arm -M virt -cpu cortex-a7 -nographic \
-icount shift=auto \
-kernel $NUTTX_DIR/nuttx \
-netdev user,id=net0 \
-device virtio-net-device,netdev=net0
# `$NUTTX_DIR/nuttx` is the linked NuttX ELF produced by `make`
# at the NuttX source root — adjust if your workspace puts it
# elsewhere (e.g. an out-of-tree build dir).
# At the NSH prompt, run the example's PROGNAME — the
# `make`-driven build registers every nano-ros example as a
# built-in command via Application.mk's `-Dmain=<PROGNAME>_main`
# rename. Real PROGNAMEs (from
# `packages/testing/nros-tests/tests/nuttx_make_e2e.rs::EXPECTED_PROGNAMES`):
nsh> nuttx_c_talker # C talker
# nsh> nuttx_cpp_talker # C++ talker
# nsh> nuttx_c_listener # ...and listener / service / action variants
# Real hardware: standard NuttX flash flow (openocd / J-Link / etc.)
# 3. Verify from stock ROS 2 in another terminal:
source /opt/ros/humble/setup.bash
export RMW_IMPLEMENTATION=rmw_zenoh_cpp
# Talker publishes best-effort; stock `ros2 topic echo` defaults to
# RELIABLE, so the QoS-mismatched echo silently delivers nothing.
# Force best-effort to receive:
ros2 topic echo /chatter std_msgs/msg/String --qos-reliability best_effort
Contributors: the in-tree fixture/test lanes for nano-ros’s own QEMU examples are in Per-Platform Contributor Lanes.
Readiness signal. After typing the app’s NSH command (e.g.
nuttx_c_talker), expect Publishing: 'Hello World: 1' on the NSH
console within ~6 seconds — Rust + C + C++ talkers all start the
count at 1, matching the official ROS 2 demo talker. If no
Publishing: line:
- Confirm the app actually ran —
psshould show your task. - Confirm networking —
ifconfigshows a configured interface. With the virtio-net + Slirp wiring above,eth0comes up at10.0.2.30(the board crate’s default for the qemu-arm-nuttx examples). - Confirm
zenohdreachable; the deploy locator (or thenros_initarguments) must match the router’s listen port. - See Troubleshooting — First 10 Minutes.
Auto-configure glue (NSH built-in registration)
The make-driven build above relies on a host-side glue layer. In-tree
that is scripts/nuttx/stage-external-apps.sh --bringup <dir>, the only
supported NuttX make path since phase-212 M-F.16 retired the per-example
build-fixtures-make recipe. If you wire a NuttX workspace by hand,
reproduce the same three steps:
- Bring the required symbols into your config. Stock NuttX
qemu-armv7a/nshships withoutCONFIG_NET=y, virtio-net, orTLS_NELEM. If you have no config of your own, copy the board defconfigpackages/boards/nros-board-nuttx-qemu/nuttx-config/arm/defconfigto$NUTTX_DIR/.configand runmake olddefconfig. If you already maintain a vendor defconfig, do NOT clobber it — merge the nano-ros requirements into it instead (CONFIG_NET=y, the virtio-net / netdev options,CONFIG_TLS_NELEM=8,CONFIG_LIBCXXTOOLCHAIN=y,CONFIG_ALLSYMS=n; diff the board defconfig againstnshfor the authoritative list) and re-runmake olddefconfig. - Stage the integration shell. Run
scripts/nuttx/stage-external-apps.sh "$NUTTX_APPS_DIR"to symlinkintegrations/nuttx/into$NUTTX_APPS_DIR/external/(add--bringup <dir>for a workspace app). Remove$NUTTX_APPS_DIR/Kconfigso NuttX’smkkconfig.shrediscovers the newapps/external/Kconfig. - Flip the nano-ros Kconfig knobs via
kconfig-tweak. The recipe enablesNROS,NROS_C_API,NROS_CPP_API, everyNROS_EXAMPLE_<EX>_<LANG>, setsTLS_NELEM=8, disablesLIBCXXNONE+ enablesLIBCXXTOOLCHAIN, and disablesALLSYMSfor the bootstrap link. Re-runmake olddefconfigso the newly-visible dependencies settle, thenmake.
kconfig-tweak ships in the kconfig-frontends package on most
distros. Without it the recipe skips ("NuttX skip: kconfig-tweak not on PATH"); install it before retrying.
GitHub source
- NuttX integration shell:
integrations/nuttx/ - Worked NuttX QEMU examples:
examples/qemu-arm-nuttx/rust/,examples/qemu-arm-nuttx/c/,examples/qemu-arm-nuttx/cpp/ - Kconfig schema:
integrations/nuttx/Kconfig
Next
- Multiple apps: each app declares its own
prognameinApplication Configuration → External Modules; they share the onelibnros_c.abuild via the external-app shell. - DDS on NuttX: bump the netbuffer Kconfig knobs (similar to the Zephyr DDS profile under Choosing an RMW Backend).
- Build nano-ros’s own NuttX QEMU tests without a NuttX-managed workspace: NuttX (contributor).
NuttX (contributor / in-tree workflow)
Looking for the user-facing path? This page covers building nano-ros’s own NuttX examples on QEMU from this repository. If you’re integrating nano-ros into YOUR NuttX app at
apps/external/nano-ros/, see Integration: NuttX external app instead.
nano-ros runs on NuttX, targeting QEMU ARM virt (Cortex-A7 + virtio-net). NuttX provides POSIX-compatible BSD sockets, which makes it the most straightforward RTOS port – zenoh-pico uses the same socket API as on Linux.
Overview
The NuttX platform uses:
- NuttX RTOS – POSIX-compliant real-time OS with BSD socket support
- BSD sockets – standard socket API provided by NuttX’s network stack
- zenoh-pico – Zenoh transport over NuttX sockets (same code path as POSIX)
- virtio-net – NuttX built-in Ethernet driver (no custom driver needed)
Board crate: nros-board-nuttx-qemu (in packages/boards/) — ONE crate serves
both QEMU NuttX boards, arm virt and rv-virt. Everything that differs between
them is data (a defconfig, a toolchain file, a target triple); nothing in the
Rust or C sources is arch-conditional.
Why NuttX Is Simpler Than FreeRTOS
NuttX is the simplest RTOS platform to port because of its strong POSIX
compliance (POSIX.1-2008: pthreads, BSD sockets, select(),
clock_gettime()):
| Aspect | NuttX | FreeRTOS |
|---|---|---|
| zenoh-pico layer | Reuses unix/ platform | Dedicated freertos/ platform |
| Networking | Built-in BSD sockets | External lwIP |
| Ethernet driver | NuttX virtio-net (built-in) | Custom LAN9118 lwIP netif |
| Rust target | armv7a-nuttx-eabi with std | thumbv7m-none-eabi (no_std) |
| Build integration | NuttX build system + cargo | cc crate compiles FreeRTOS + lwIP |
| QEMU machine | virt (Cortex-A7) | mps2-an385 (Cortex-M3) |
Because NuttX supports Rust std, examples use standard fn main() entry
points and println! – no semihosting or custom panic handlers needed.
Setup
nros setup qemu-arm-nuttx provisions everything this board needs —
the NuttX cross-compiler, qemu-system-arm, the NuttX source tree,
and the RMW host daemon — into the shared store at ~/.nros/sdk. No
hand-installed cross-toolchain and no ROS 2 install required.
Build the in-tree nros CLI (Phase 218), then provision the board
(--rmw defaults to zenoh):
./scripts/bootstrap.sh # builds packages/cli/target/release/nros
source ./activate.sh # OR: direnv allow / source ./activate.fish
nros setup qemu-arm-nuttx --rmw zenoh
As a contributor, just setup nuttx remains available and now
delegates to nros setup qemu-arm-nuttx for the toolchain/SDK
provisioning while also staging the external apps.
The NuttX sources land in third-party/nuttx/nuttx/ and
third-party/nuttx/nuttx-apps/. Override the paths with environment
variables if your sources are elsewhere:
| Variable | Default | Description |
|---|---|---|
NUTTX_DIR | third-party/nuttx/nuttx | NuttX RTOS source |
NUTTX_APPS_DIR | third-party/nuttx/nuttx-apps | NuttX apps source |
Prerequisites
The nros setup qemu-arm-nuttx step above provisions the
qemu-system-arm emulator and the arm-none-eabi-gcc cross-compiler
used for NuttX kernel compilation. The one host-side tool you still
supply yourself:
- Rust nightly toolchain (NuttX targets are Tier 3, require
-Z build-std)
Building
Contributors: the in-tree build lanes for this platform are in Per-Platform Contributor Lanes.
The examples link against NuttX’s POSIX layer, which provides sockets, pthreads, and standard I/O.
Available Examples
All examples are in examples/qemu-arm-nuttx/rust/:
| Example | Description |
|---|---|
talker | Publishes std_msgs/String on /chatter |
listener | Subscribes to std_msgs/String on /chatter |
service-server | Serves AddTwoInts on /add_two_ints |
service-client | Calls AddTwoInts on /add_two_ints |
action-server | Serves Fibonacci action on /fibonacci |
action-client | Sends Fibonacci goal on /fibonacci |
RISC-V (rv-virt, riscv32)
The same NuttX path also runs on QEMU’s RISC-V rv-virt board
(rv32imac, riscv32imac-unknown-nuttx-elf), proving the provisioning
path is arch-agnostic. Provision and build with:
nros setup qemu-riscv-nuttx --rmw zenoh
Contributors: the in-tree RISC-V fixture build lanes are in Per-Platform Contributor Lanes.
Board crate: nros-board-nuttx-qemu (in packages/boards/) — the same crate as
the arm board above, selected by deploy = "nuttx-riscv", which picks the
rv-virt [[board]] entry of its nros-board.toml (riscv toolchain, riscv
defconfig, riscv32imac-unknown-nuttx-elf).
The C pub/sub pair is exercised end-to-end under
qemu-system-riscv32, and Rust / C / C++ realtime-tiers workspace
runtime lanes cover the multi-tier scheduling path on this board.
Testing
Contributors: the in-tree test lanes for this platform are in Per-Platform Contributor Lanes.
Tests run under qemu-system-arm -M virt with TAP networking. Each QEMU
instance connects to the host bridge (br-qemu) via TAP devices for
zenohd communication. The test infrastructure builds a NuttX kernel image
with the example app embedded, boots it in QEMU, and verifies message
exchange.
Network Configuration
NuttX QEMU instances use the same IP scheme as other QEMU board crates:
| Role | IP Address | TAP Device |
|---|---|---|
| Talker/Publisher | 192.0.3.10 | tap-qemu0 |
| Listener/Sub | 192.0.3.11 | tap-qemu1 |
| Service Server | 192.0.3.12 | tap-qemu0 |
| Service Client | 192.0.3.13 | tap-qemu1 |
| zenohd (host) | 192.0.3.1 | br-qemu |
Architecture
Board Crate
The nros-board-nuttx-qemu board crate follows the standard Config / run() pattern documented in the Custom Board Package guide. It provides network and node configuration presets (talker(), listener(), server(), client()).
Unlike bare-metal and FreeRTOS board crates, there is no custom hardware
initialization, no network stack setup, and no task creation. NuttX’s kernel
boots the hardware, initializes virtio-net, and starts the application
before main() runs. Because NuttX supports Rust std, examples use standard
fn main() entry points.
NuttX Defconfig
The QEMU board configuration lives in
packages/boards/nros-board-nuttx-qemu/nuttx-config/arm/defconfig
(the rv-virt one sits beside it under riscv/) and enables:
CONFIG_NET– networking subsystemCONFIG_NET_TCP/CONFIG_NET_UDP– TCP/UDP protocolsCONFIG_DRIVERS_VIRTIO+CONFIG_DRIVERS_VIRTIO_NET– virtio EthernetCONFIG_PTHREAD_MUTEX_TYPES– POSIX mutex types (for zenoh-pico)CONFIG_DEV_URANDOM–/dev/urandomfor session ID generationCONFIG_DEFAULT_TASK_STACKSIZE=8192– adequate stack for nros appsCONFIG_BUILD_FLAT– flat memory model (no MMU protection)
Status
NuttX platform support is complete: feature flags, build integration, six Rust examples, E2E network tests, and documentation.
ThreadX (Linux sim / RISC-V64 QEMU)
Single-node starter on Microsoft Azure RTOS ThreadX + NetX Duo (BSD socket layer). Two flavours ship in-tree:
- threadx-linux — ThreadX user-space simulator on Linux. Fast build, host network stack, ideal for development.
- threadx-riscv64 — QEMU
virtmachine with the RISC-V64 GCC toolchain. Full kernel + NetX Duo TCP/IP stack.
Rust, C, and C++ are supported on both flavours.
Contributors: the in-tree fixture/test lanes for this platform are in Per-Platform Contributor Lanes.
See the coverage matrix for the per-RMW cell status.
Prereqs. Install the
nrosCLI once, then runnros setup <board> --rmw <rmw>for the flavour you need (see Setup). It provisions the cross-compiler, emulator, and ThreadX/NetX sources — no hand-installedriscv64cross toolchain orqemu-system-riscv64. The zenoh path additionally needs a ROS 2 install for the router (ros2 run rmw_zenoh_cpp rmw_zenohd); xrce needs only the Micro-XRCE-DDS agent, whichnros setupinstalls.
Setup
nros setup is the single canonical command to prepare a machine to build
nano-ros for a board. Most components are prebuilt per platform per RMW — the
cross-compiler, emulator, and SDK sources (the ThreadX/NetX
sources, and for threadx-linux the POSIX-sim sources) are fetched from a pinned
index into a shared store at ${NROS_HOME:-~/.nros}/sdk. The zenoh router is
NOT among them — it comes from a ROS 2 install (RFC-0075); only the xrce
agent is provisioned by nros setup.
Packages are prebuilt where the index has a binary for your host and built from
source otherwise. --dry-run prints the plan for your host. See
Installation for the full explanation.
Build the in-tree nros CLI (Phase 218):
./scripts/bootstrap.sh # builds packages/cli/target/release/nros
source ./activate.sh # OR: direnv allow / source ./activate.fish
Provision the ThreadX flavour you need (+ the RMW):
nros setup threadx-linux --rmw zenoh # POSIX-sim flavour; --rmw defaults to zenoh
nros setup qemu-riscv64-threadx --rmw zenoh # only if you need the RISC-V64 QEMU flow
source ./activate.sh
The RMW host daemon must be running before any example: for zenoh the
ROS 2 router (ros2 run rmw_zenoh_cpp rmw_zenohd), for xrce the
Micro-XRCE-DDS agent (installed by nros setup … --rmw xrce).
Project layout
Each example is a standalone Cargo or CMake project under
examples/threadx-linux/ and examples/qemu-riscv64-threadx/
(<lang>/<example>/ under each):
examples/threadx-linux/
├── rust/talker/ # Cargo, target = x86_64-unknown-linux-gnu
│ ├── Cargo.toml # deps + [package.metadata.nros.deploy.threadx-linux]
│ ├── package.xml
│ ├── generated/ # codegen output — build.rs runs
│ │ # `nros generate-rust` on first
│ │ # `cargo build`; gitignored.
│ └── src/lib.rs # the component class; nros::main! generates the entry
└── c/talker/ # CMake, add_subdirectory
├── CMakeLists.txt # targets (deploy tuple in package.xml)
├── package.xml
└── src/Talker.c
examples/qemu-riscv64-threadx/
├── rust/talker/ # Cargo, target = riscv64gc-unknown-linux-gnu
│ └── ...
└── c/talker/
└── ...
ThreadX-linux runs as a regular host process — no QEMU. NetX Duo
uses the nx_bsd_* BSD socket shim layered on the host TCP stack
(threadx-linux variant) or on its own NetX Duo TCP/IP stack
(riscv64 variant).
Why there are two board crates and one arch port
The two flavours look like one board with two targets, and they are not. nano-ros ships three ThreadX packages, in the three layers RFC-0064 describes:
| Package | Layer | What it owns |
|---|---|---|
nros-board-threadx | family driver | the kernel/NetX build and the generic run_entry boot path both boards call |
nros-board-threadx-port-riscv64 | arch port | a fork of upstream’s ports/risc-v64/gnu: tx_port.h + five context-switch .S files |
nros-board-threadx-{linux,qemu-riscv64} | board overlay | defaults, console/exit/panic, and the network bring-up |
The arch port is the reason the two boards did not merge. Upstream’s
RISC-V64 port types ULONG as unsigned long — 8 bytes on rv64 — but
NetX Duo’s packet code does ULONG * arithmetic assuming 4-byte words,
so nano-ros retypes it to unsigned int, matching upstream’s own Linux
and AArch64 ports. That retype shifts every field offset inside
TX_THREAD, and the port’s context-switch assembly loads those fields at
hard-coded offsets — so the header change forces the assembly to be
forked with it. threadx-linux needs none of this, because upstream’s
Linux port already types ULONG as 4 bytes.
That is architecture code, not board code: a second RISC-V64 ThreadX
board would need every line of it unchanged, and folding it into a crate
that also serves Linux would mean cfg-gating RISC-V assembly there. If
you are porting to another RISC-V64 ThreadX board, depend on the arch
port and write only the overlay — see
Custom boards.
One rule matters if you build ThreadX outside the shipped recipes: the
arch port’s inc/ must be searched before
$THREADX_DIR/ports/risc-v64/gnu/inc. Get that wrong and the build
succeeds against upstream’s 8-byte ULONG, and the symptom is corrupted
packets at runtime rather than a compiler error. A
_Static_assert(sizeof(ULONG) == 4, …) in
packages/boards/nros-board-common/c/threadx_hooks.c turns that mistake
into a build failure with a name on it.
Configure
Deploy config is declared per flavour in the build manifest and baked at compile time. Both shipped shapes, verbatim:
threadx-linux —
examples/threadx-linux/rust/talker/Cargo.toml:
[package.metadata.nros.deploy.threadx-linux]
board = "threadx-linux"
rmw = "zenoh"
domain_id = 0
# locator/ip default to the board's loopback shape (dial 127.0.0.1)
threadx-riscv64 —
examples/qemu-riscv64-threadx/c/talker/CMakeLists.txt:
cmake_minimum_required(VERSION 3.22)
project(c_talker LANGUAGES C CXX)
find_package(nano_ros REQUIRED)
find_package(std_msgs REQUIRED)
nano_ros_add_executable(c_talker src/main.c)
ament_target_dependencies(c_talker std_msgs)
The deploy coordinate lives in package.xml, not CMake:
<nano_ros deploy="threadx" board="riscv64-qemu" rmw="zenoh"/>.
Network shape (guest IP, gateway, router locator) beyond these fields comes from the board crate’s defaults — see the Configuration Guide.
ThreadX-Linux normally uses a veth pair (tap-tx0) for an isolated
host link, but nros setup threadx-linux does not create the
interface — the test fixtures fall back to a loopback path when
tap-tx0 is absent, which is fine for the happy-path tutorial.
Bring up tap-tx0 by hand (ip link add … type veth …) only when
you need real-network bridging. The QEMU-RISC-V64 fixture uses
Slirp’s default 10.0.2.2 gateway just like the FreeRTOS QEMU flow.
Build
# Single example — `nros sync` first (a hand-run cargo build does
# not do it for you):
cd examples/threadx-linux/rust/talker
nros sync
cargo build --release
Contributors: the in-tree fixture build lanes for both flavours are in Per-Platform Contributor Lanes.
First setup builds ThreadX + NetX Duo (~3 min). Subsequent example builds finish in seconds.
Contributors (in-tree checkout): the just … build-fixtures
recipes run nros sync for you. A
hand-run cargo build in a leaf does not — without it cargo fails
while parsing the manifest, with
failed to load config include '…/nros-patch.toml' and no mention of
sync. See
Workflow by Platform and Language.
Run
# threadx-linux (no QEMU). Step 1 brings up the router (ROS's
# `rmw_zenohd`) on port 9000 — the deploy `locator` the talker bakes
# in its Cargo.toml. Step 2 builds + runs the talker:
ZENOH_CONFIG_OVERRIDE='listen/endpoints=["tcp/0.0.0.0:9000"];scouting/multicast/enabled=false' \
ros2 run rmw_zenoh_cpp rmw_zenohd &
cd examples/threadx-linux/rust/talker && nros sync && cargo run --release
# Expected (per src/lib.rs structured logs):
# Publishing: 'Hello World: 1'
# Publishing: 'Hello World: 2'
# ...
# threadx-riscv64 (QEMU virt). Same shape — the router on 9400 first
# (the riscv64 talker dials tcp/10.0.2.2:9400 through QEMU Slirp):
ZENOH_CONFIG_OVERRIDE='listen/endpoints=["tcp/127.0.0.1:9400"];scouting/multicast/enabled=false' \
ros2 run rmw_zenoh_cpp rmw_zenohd &
# Then boot the built image in QEMU (UART on stdio, Slirp networking):
qemu-system-riscv64 -M virt -m 256M -bios none -nographic \
-global virtio-mmio.force-legacy=false \
-kernel <path-to-built-talker-elf> \
-netdev user,id=net0 \
-device virtio-net-device,netdev=net0,bus=virtio-mmio-bus.0
# **Contributors (in-tree checkout):** `just threadx_riscv64 talker`
# builds and boots this exact shape in one step.
# Verify from stock ROS 2:
source /opt/ros/humble/setup.bash
export RMW_IMPLEMENTATION=rmw_zenoh_cpp
# Talker publishes best-effort; stock `ros2 topic echo` defaults to
# RELIABLE, so the QoS-mismatched echo silently delivers nothing.
# Force best-effort to receive:
ros2 topic echo /chatter std_msgs/msg/String --qos-reliability best_effort
Contributors: the in-tree fixture run/test lanes are in Per-Platform Contributor Lanes.
Readiness signal. threadx-linux: Publishing: 'Hello World: 1'
within a few seconds of cargo run --release on a warm
cache; a cold first run rebuilds the Rust example (~80 s on a
fresh checkout) before the first publish lands. threadx-riscv64
(QEMU): within ~15 seconds of QEMU boot. If no Publishing: line:
- Confirm the router is reachable on the deploy locator
(threadx-linux uses
127.0.0.1; riscv64 QEMU uses10.0.2.2). - threadx-linux: if you brought up
tap-tx0by hand, confirm it is up; without it the fixtures use the loopback fallback, which is fine for this tutorial (nros setup threadx-linuxdoes not create the interface). - See Troubleshooting — First 10 Minutes.
Multi-tier scheduling (phase-297)
ThreadX supports the multi-tier workspace scheduling model
(RFC-0053): declare per-tier ThreadX priorities in the workspace
manifest with [tiers.<name>.threadx] priority tables, and
nros::main! routes the app onto run_tiers — one executor per
tier, all sharing one session, with per-tier stacks carved from the
ThreadX byte pool. ThreadX priorities follow the native convention:
lower number = higher priority. Tiers are sorted descending by
raw priority number, so tiers[0] (the numerically-largest, i.e.
lowest-priority tier) boots first and then adopts its declared
priority. See Scheduling Models
for the mechanics.
GitHub source
- ThreadX-Linux Rust:
examples/threadx-linux/rust/talker/ - ThreadX-Linux C:
examples/threadx-linux/c/talker/ - ThreadX-RISC-V64 Rust:
examples/qemu-riscv64-threadx/rust/talker/ - Board crates:
packages/boards/nros-board-threadx-linux/,packages/boards/nros-board-threadx-qemu-riscv64/
Next
- Subscriber + service + action peers in the same example tree.
- DDS on ThreadX: Cyclone DDS is the surviving DDS backend
(
nros-rmw-cyclonedds, selected via-DNANO_ROS_RMW=cyclonedds); see Choosing an RMW Backend. - Real hardware: same code runs against ThreadX vendor BSPs (Renesas Synergy, MIMXRT, etc.); replace the QEMU board crate with a vendor board crate.
ESP32 (esp-hal, bare-metal Rust)
Single-node starter on ESP32-C3 using the bare-metal esp-hal Rust
path — no ESP-IDF — running under the Espressif QEMU fork (OpenETH
ethernet). For the ESP-IDF component path (C / C++ apps), see
ESP32 (ESP-IDF component).
Prereqs.
nros setup qemu-esp32-baremetalprepares the build: the riscv cross-gcc andespflashfrom a pinned index into the shared store at~/.nros/sdk— you do not hand-install cross-compilers. Running under QEMU additionally needs Espressif’s QEMU fork (nros setup --tool esp32-qemu, source-built) and — for zenoh — a ROS 2 install to provide the router.
Time budget. This chapter is the longest embedded setup in the book (~a dozen steps, two source builds, a nightly toolchain): plan an afternoon, not ten minutes. If you have not had a first win yet, take the First Project host flow first — it is the ten-minute one — and come back.
Setup
Build the in-tree nros CLI (Phase 218):
./scripts/bootstrap.sh # builds packages/cli/target/release/nros
source ./activate.sh # OR: direnv allow / source ./activate.fish
Provision the board (and RMW):
nros setup qemu-esp32-baremetal --rmw zenoh # --rmw defaults to zenoh; xrce | cyclonedds also valid
This pulls the SDK sources nano-ros owns (zenoh-pico + mbedtls
submodules for zenoh; analogous for xrce / cyclonedds) plus the riscv
cross-gcc and espflash into ${NROS_HOME:-~/.nros}/sdk. It does NOT
provide a zenoh router — that is ROS 2’s rmw_zenohd, so the zenoh Run
step below needs a ROS install (--rmw cyclonedds needs no daemon at
all). esp-hal itself is a Cargo dependency the
example pulls in at build time, not a separately-installed toolchain;
the only cross-toolchain you may need to add by hand is the rustup
target — once per host:
rustup target add riscv32imc-unknown-none-elf # ESP32-C3
Project layout
Each example is a standalone Cargo package targeting
riscv32imc-unknown-none-elf (ESP32-C3). The board crate
(nros-board-esp32-qemu) wraps the OpenETH / esp-hal init.
ESP32-S3 (Xtensa) is NOT supported today. The tutorial targets the RISC-V ESP32-C3 only. Xtensa targets do not ship via
rustup(they require theespuptoolchain installer) and the in-tree board crate is RISC-V only; this gap is tracked separately.
examples/qemu-esp32-baremetal/rust/talker/
├── Cargo.toml # deps + [package.metadata.nros.deploy.qemu-esp32-baremetal]
├── .cargo/ # config.toml + nros-board.toml
│ # (target = riscv32imc-unknown-none-elf lives in nros-board.toml)
├── package.xml
├── generated/ # codegen output — build.rs runs
│ # `nros generate-rust` on first
│ # `cargo build`; gitignored.
└── src/ # lib.rs component class + main.rs entry
Configure
Deploy config lives in the app’s Cargo.toml (baked at compile time;
the board’s default Config supplies the remaining smoltcp knobs like
the MAC). The QEMU ESP32 board uses OpenETH ethernet via
nros-board-esp32-qemu. Verbatim from
examples/qemu-esp32-baremetal/rust/talker/Cargo.toml:
[package.metadata.nros.deploy.qemu-esp32-baremetal]
rmw = "zenoh"
domain_id = 0
ip = "10.0.2.50"
gateway = "10.0.2.2"
locator = "tcp/10.0.2.2:9800"
Build
Contributors: the in-tree fixture/test lanes for this platform are in Per-Platform Contributor Lanes.
Copy the example out, generate bindings, build with the pinned nightly, and pack the flash image:
# once per checkout location — bindings + the [patch.crates-io] table:
NROS_REPO_DIR=<path-to-nano-ros> nros sync
# nightly because the board config builds core/alloc from source
# ([unstable] build-std in .cargo/nros-board.toml). The pinned channel
# is tools/rust-toolchain.toml's; any recent nightly with the rust-src
# component works:
# rustup toolchain install nightly && rustup component add rust-src --toolchain nightly
# rustup target add riscv32imc-unknown-none-elf --toolchain nightly
cargo +nightly build --release
# pack the ELF into the flash image QEMU boots (espflash comes from
# `nros setup qemu-esp32-baremetal`, on PATH via activate):
espflash save-image --chip esp32c3 --flash-size 4mb --merge \
target/riscv32imc-unknown-none-elf/release/esp32_qemu_talker talker.bin
First build cross-compiles core/alloc + every dep (~5 min); rebuilds are
seconds. If you build a C-flavored variant, the riscv cross-gcc the zpico
shim needs is also provisioned by the same nros setup.
Run
The esp32c3 machine exists only in Espressif’s QEMU fork — stock
qemu-system-riscv32 knows only virt. The fork is source-built by:
nros setup --tool esp32-qemu # clones + builds espressif/qemu (needs
# libglib2-dev libpixman-dev libgcrypt-dev;
# the command names them if absent)
# 1. Bring up the router (ROS's `rmw_zenohd`) on the port the example
# dials (9800 — the deploy `locator` above):
ZENOH_CONFIG_OVERRIDE='listen/endpoints=["tcp/127.0.0.1:9800"];scouting/multicast/enabled=false' \
ros2 run rmw_zenoh_cpp rmw_zenohd &
# 2. Boot the flash image (the file packed in Build above):
qemu-system-riscv32 -M esp32c3 -icount 3 -nographic \
-drive file=talker.bin,if=mtd,format=raw \
-nic user,model=open_eth
# Serial output:
# Publishing: 'Hello World: 1'
# Publishing: 'Hello World: 2'
Contributors: the in-tree run lane that boots the talker in QEMU is in Per-Platform Contributor Lanes.
# Verify from stock ROS 2 on the same network:
source /opt/ros/humble/setup.bash
export RMW_IMPLEMENTATION=rmw_zenoh_cpp
# Talker publishes best-effort; stock `ros2 topic echo` defaults to
# RELIABLE, so the QoS-mismatched echo silently delivers nothing.
# Force best-effort to receive:
ros2 topic echo /chatter std_msgs/msg/String --qos-reliability best_effort
Readiness signal. QEMU ESP32: ~15 seconds after boot.
If no Publishing: line:
- Wrong locator → talker logs
zenoh open failedand retries. Confirm the router is reachable on the host IP (10.0.2.2:9800). - Confirm
.cargo/nros-board.tomlsetstarget = "riscv32imc-unknown-none-elf"(ESP32-C3). The tutorial does not support ESP32-S3 (Xtensa) yet. - See Troubleshooting — First 10 Minutes.
GitHub source
- QEMU ESP32 talker:
examples/qemu-esp32-baremetal/rust/talker/ - Board crate:
packages/boards/nros-board-esp32-qemu/
Next
- Subscriber + service + action peer directories under the same
examples/qemu-esp32-baremetal/rust/. - ESP-IDF component path for C / C++ apps: ESP32 (ESP-IDF component).
- ESP32-S3 (Xtensa) — not supported today. The Xtensa toolchain
does not ship via
rustup(it requiresespup), and there is no in-tree Xtensa board crate. Stick with ESP32-C3 (RISC-V) for now.
ESP32 (ESP-IDF component)
Support level: nightly only. This path is exercised by the nightly full-matrix lane, never by the per-change or per-platform ones, and its coverage stops at “the image builds” — nothing boots an IDF image in CI. Treat it as a working integration shell you can build on, not as a supported starter. If you want a first ESP32 result, take the esp-hal path, which does boot under QEMU in CI.
Contributors: the lane is
just esp_idf ci(doctor + build-c-port).
Single-node starter on ESP32-family chips via the ESP-IDF
component path — Espressif’s native C / C++ build system. For the
bare-metal Rust (esp-hal) path, see ESP32 (esp-hal).
Prereqs. Two independent toolchains.
ESP-IDF itself — ≥ 5.1, installed through Espressif’s own installer so
idf.pyis onPATH(source $IDF_PATH/export.sh).nros setupdoes not replace this; the IDF toolchain comes fromidf.py install/ Espressif’s tooling.The nano-ros side — the RMW host daemon (and any nano-ros host tools you use for testing) come from the
nrosCLI:./scripts/bootstrap.sh # builds packages/cli/target/release/nros (Phase 218) source ./activate.sh # OR: direnv allow / source ./activate.fish nros setup qemu-esp32-baremetal --rmw zenoh # lands the toolchain # (and for xrce the Micro-XRCE-DDS # agent) in ${NROS_HOME:-~/.nros}/sdk, # AND clones the transport submodules. # The zenoh router is NOT installed — # it comes from ROS 2 # (ros2 run rmw_zenoh_cpp rmw_zenohd) # (zenoh-pico + mbedtls for zenoh) # into the nano-ros checkout # so the IDF build can compile # them in-tree.
Project layout
ESP-IDF apps are CMake projects with idf.py as the orchestrator.
nano-ros plugs in as a component pulled by IDF’s component manager
or by a local path during development.
my_idf_app/
├── CMakeLists.txt # top-level: `project(my_app)`
├── sdkconfig # IDF Kconfig (generated)
├── main/
│ ├── CMakeLists.txt # `idf_component_register(REQUIRES nano-ros …)`
│ ├── idf_component.yml # declares nano-ros as a managed dependency
│ └── app_main.c | app_main.cpp
└── components/ # (optional) local components override
The idf_component.yml is the dependency manifest:
dependencies:
nano-ros:
# During development — local path to your nano-ros clone:
path: ../../../nano-ros/integrations/nano-ros
# Once published to the Espressif Component Registry:
# version: "*"
The shell at integrations/nano-ros/ wraps the nano-ros root CMake
into a standard IDF component, mapping IDF Kconfig knobs to
NANO_ROS_* cache vars.
Configure
After idf.py menuconfig:
Component config → nano-ros
Bringup package name (demo_bringup)
ROS domain ID (0)
RMW backend (xrce) xrce | zenoh
ROS 2 edition (humble) humble | iron | jazzy
Four knobs. The default RMW is XRCE (CycloneDDS is intentionally
absent on ESP — the Kconfig says so). NROS_BRINGUP_NAME drives the
configure-time nros codegen-system step, so it must name a bringup
package that exists in your workspace — a first build with the default
demo_bringup fails if you have none.
Wi-Fi credentials + zenoh locator are NOT in this Kconfig —
provide them via your app’s own Kconfig.projbuild (Espressif’s
standard pattern) or via environment variables, then pass them to
nros::init(locator, domain_id) at startup.
Build
Honesty note: a complete reference app (Wi-Fi + locator wiring on top of the component) is not in-tree yet — the component shell builds and boots, but you author the app glue. For a worked end-to-end flow today, start with the QEMU esp-hal chapter and come back here to embed into your IDF project.
cd my_idf_app
idf.py set-target esp32c3 # or esp32s3, esp32, esp32c6
idf.py build
First build cross-compiles nano-ros’s Rust staticlibs + IDF components (~5 min). Re-builds finish in seconds.
Run
# Flash + monitor:
idf.py -p /dev/ttyUSB0 flash monitor
# Expected serial output:
# I (1234) nano-ros: Wi-Fi connected
# I (1456) nano-ros: zenoh session opened
# I (2567) nano-ros: Publishing: 'Hello World: 1'
# Verify from stock ROS 2 on the same network:
source /opt/ros/humble/setup.bash
export RMW_IMPLEMENTATION=rmw_zenoh_cpp
# Talker publishes best-effort; stock `ros2 topic echo` defaults to
# RELIABLE, so the QoS-mismatched echo silently delivers nothing.
# Force best-effort to receive:
ros2 topic echo /chatter std_msgs/msg/String --qos-reliability best_effort
Contributors (in-tree checkout): for the QEMU ESP32 testing path
see just esp_idf test-c-port — it boots the tests/esp-idf-smoke
C-port project on qemu-system-riscv32 (esp32c3 default) via
Espressif’s patched QEMU.
Readiness signal. After idf.py flash monitor, expect
I (XXXX) nano-ros: Wi-Fi connected followed by
I (XXXX) nano-ros: Publishing: 'Hello World: 1' within 10 seconds
— Rust + C + C++ talkers all start the count at 1, matching the
official ROS 2 demo talker. If no Publishing: line:
- Wi-Fi creds — your app’s own Kconfig (nano-ros’s submenu does not carry them; see Configure above).
- Wrong locator — confirm the host running the router/agent is on the same Wi-Fi subnet (or routable to it). NAT will block discovery.
idf.py menuconfigshows theComponent config → nano-rossubmenu (the component is wired) andCONFIG_NROS_RMWis set to a backend name (xrce/zenoh). There is no separateCONFIG_NROS_ENABLEDtoggle on ESP-IDF; the component’s presence inmain/idf_component.ymlis the on-switch.- See Troubleshooting — First 10 Minutes.
GitHub source
- IDF component shell:
integrations/nano-ros/ - Component manifest:
integrations/nano-ros/idf_component.yml - Kconfig surface:
integrations/nano-ros/Kconfig.projbuild
A complete reference app showing Wi-Fi + zenoh wiring on top of the
component is not in-tree yet; the bare-metal
examples/qemu-esp32-baremetal/rust/talker/
is the closest worked example.
Next
- Bare-metal
esp-halRust path: ESP32 (esp-hal). - Multi-component IDF apps: nano-ros sits next to other Espressif components (network, storage, sensors) — IDF’s component manager resolves them all.
Bare-metal Cortex-M3 (QEMU)
Single-node starter on bare-metal Cortex-M3 (QEMU MPS2-AN385) —
no RTOS, no kernel scheduler. Pure cooperative spin via
zpico_spin_once. Rust only. nros-c / nros-cpp are not
supported on bare-metal targets (they assume a hosted RTOS for
startup / heap / libc); see the
examples coverage matrix
for the policy.
When to use this path: ultra-constrained Cortex-M0+ / M3 / M4 targets with no OS scheduler, no
pthread. If you have FreeRTOS or any RTOS, use the FreeRTOS starter instead — it’s more ergonomic and produces smaller code overall.
Prereqs. Install the
nrosCLI once per machine, then provision this board.nros setupfetches a prebuilt bare-metal toolchain (arm-none-eabi-gcc,qemu-system-arm, the zenoh router) plus the Rustthumbv7m-none-eabitarget into a shared store — no manual cross-compiler install, no ROS 2 needed.
# Build the in-tree nros CLI (Phase 218):
./scripts/bootstrap.sh # builds packages/cli/target/release/nros
source ./activate.sh # OR: direnv allow / source ./activate.fish
# Provision the bare-metal Cortex-M3 board (zenoh RMW is the default):
nros setup qemu-arm-baremetal --rmw zenoh
A board-named variant exists too:
nros setup mps2-an385provisions the same QEMU toolchain under a board-named key (nothing extra for physical hardware — no probe/flash tooling). For a real STM32F4 see the out-of-tree worked example.
Project layout
examples/qemu-arm-baremetal/rust/talker/
├── Cargo.toml # deps + [package.metadata.nros.deploy.qemu-mps2-an385]
├── .cargo/ # config.toml + nros-board.toml
│ # (nros-board.toml carries target = thumbv7m-none-eabi
│ # and the qemu-system-arm ... -kernel runner)
├── package.xml
├── generated/ # codegen output — build.rs runs
│ # `nros generate-rust` on first
│ # `cargo build`; gitignored.
└── src/ # lib.rs component class + main.rs entry
The board crate is nros-board-mps2-an385 (note: no -freertos
suffix — this is the bare-metal variant) which provides:
- Cortex-M3 startup + linker script
- LAN9118 driver for smoltcp
BoardIdle::wfi()for cooperative wait
Direct-exec or RTIC — one crate, two entry shapes
The same crate serves both bare-metal entry models; you pick with a Cargo feature, not a different board crate:
| Entry pkg wants | deploy = | board dep |
|---|---|---|
| direct-exec (inline spin loop) | "qemu-mps2-an385" | nros-board-mps2-an385 = { version = "*", features = ["board-entry"] } |
RTIC (framework-owned #[rtic::app], deferred dispatch) | "rtic-mps2-an385" | nros-board-mps2-an385 = { version = "*", features = ["rtic"] } |
The RTIC surface used to be a separate nros-board-rtic-mps2-an385
crate; phase-337 W6.a folded it in, because it depended on this crate
and re-declared its Config — the two copies had drifted to different
default IPs, which nothing catches until a node fails to reach the
router.
Which network the defaults describe is now explicit rather than
implied. Config::default() is the bridge plan (192.0.3.10/24,
gateway 192.0.3.1); Config::qemu_slirp() is QEMU’s user-mode NAT
(10.0.2.10/24, gateway 10.0.2.2) and is what the RTIC entry boots
from. Either way, an Entry pkg that sets ip/gateway/locator in
its [package.metadata.nros.deploy.<board>] block overrides the
default — which every in-tree example does, and which you should too.
Configure
Deploy config lives in the app’s Cargo.toml and is baked at compile
time — nros::main!() folds it into a DeployOverlay the board’s boot
Config applies. Verbatim from the in-tree
examples/qemu-arm-baremetal/rust/talker/Cargo.toml:
[package.metadata.nros.deploy.qemu-mps2-an385]
locator = "tcp/10.0.2.2:10500"
ip = "10.0.2.10"
gateway = "10.0.2.2"
netmask = "255.255.255.0"
QEMU Slirp networking — no host TAP / bridge / sudo. The zenoh
default port is 7447; this example dials 10500 (the locator
above), so start the router (ROS’s rmw_zenohd) on that port:
ZENOH_CONFIG_OVERRIDE='listen/endpoints=["tcp/127.0.0.1:10500"];scouting/multicast/enabled=false' \
ros2 run rmw_zenoh_cpp rmw_zenohd
or edit the locator above to the port you prefer.
Build
cd examples/qemu-arm-baremetal/rust/talker
nros sync # once per checkout location; writes the generated
# bindings + the [patch.crates-io] table the leaf's
# .cargo/config.toml includes
cargo build --release
First build (~5 min) cross-compiles all of nano-ros’s Rust deps for
thumbv7m-none-eabi. Re-builds finish in seconds.
Contributors (in-tree checkout): the just … build-fixtures
recipes run nros sync for you. A
hand-run cargo build in a leaf does not — without it cargo fails
while parsing the manifest, with
failed to load config include '…/nros-patch.toml' and no mention of
sync. See
Workflow by Platform and Language.
Run
# 1. Bring up the router on the host (Slirp forwards 10.0.2.2:10500 →
# host 127.0.0.1:10500). This example dials 10500, NOT zenoh's
# default 7447 — edit the deploy `locator` in Cargo.toml if you
# want 7447:
ZENOH_CONFIG_OVERRIDE='listen/endpoints=["tcp/127.0.0.1:10500"];scouting/multicast/enabled=false' \
ros2 run rmw_zenoh_cpp rmw_zenohd &
# 2. Boot the talker in QEMU. Invoke qemu-system-arm directly with the
# LAN9118 networking wiring the example expects (a plain `cargo run`
# boots QEMU without networking — the example's runner is bare
# `-kernel`). The patched qemu-system-arm is provisioned by
# `nros setup qemu-arm-baremetal` and reaches PATH via activate.sh:
qemu-system-arm -cpu cortex-m3 -machine mps2-an385 -nographic \
-icount shift=auto \
-semihosting-config enable=on,target=native \
-kernel target/thumbv7m-none-eabi/release/qemu-bsp-talker \
-nic user,model=lan9118
# Expected serial-over-semihosting output (per src/lib.rs):
# Publishing: 'Hello World: 1'
# Publishing: 'Hello World: 2'
# ...
# 3. Verify from stock ROS 2:
source /opt/ros/humble/setup.bash
export RMW_IMPLEMENTATION=rmw_zenoh_cpp
# Talker publishes best-effort; stock `ros2 topic echo` defaults to
# RELIABLE, so the QoS-mismatched echo silently delivers nothing.
# Force best-effort to receive:
ros2 topic echo /chatter std_msgs/msg/String --qos-reliability best_effort
QEMU exits via Ctrl-A x.
Readiness signal. Within ~15 seconds of QEMU boot (no RTOS
init delay, but smoltcp + zenoh handshake still takes a few
seconds), expect Publishing: 'Hello World: 1' on semihosting
stdout — the count starts at 1, matching the official ROS 2 demo
talker. If no Publishing: line:
- Router not running — talker spins on smoltcp poll until killed.
- Wrong LAN9118 emulation flag —
qemu-system-armneeds-nic user,model=lan9118(or equivalent). The example’s.cargo/config.tomlrunner is bare-kernel(so a plaincargo runboots QEMU without networking); the directqemu-system-arminvocation shown above carries the LAN9118 wiring and is the working invocation for this tutorial. If you copy the runner out, mirror those flags. - Cooperative spin starvation — if you added a long-running callback, the entire executor stalls; bare-metal has no preemption.
- See Troubleshooting — First 10 Minutes.
GitHub source
- Bare-metal talker:
examples/qemu-arm-baremetal/rust/talker/ - Board crate:
packages/boards/nros-board-mps2-an385/
Constraints to be aware of
- No
allocby default. Pureno_std+heaplessfor bounded collections. If you needalloc, opt in via theallocfeature on your board crate and supply a#[global_allocator]. - No wake primitive. Cooperative single-thread spin only; the
executor’s
nros_platform_wake_*slots returnUnsupported. - No preemption. A long-running user callback blocks every other dispatchable handle until it returns.
nros-c/nros-cppNOT supported. These wrappers assume hosted-RTOS libc + heap. Pure-Rust API only on this target.
For Cortex-M3 with an RTOS, switch to the FreeRTOS starter.
Next
- Subscriber / service / action peers under the same
examples/qemu-arm-baremetal/rust/tree. - Wake-callback opt-in: the
wake-callback(latency-probe) bench underpackages/testing/nros-bench/wake-latency-cortex-m3/shows how to feed a backend’s transport-notify into the cooperative spin loop on bare-metal. - Real hardware: the same code runs on an STM32F4-Discovery with a board crate of your own and a different linker script — see Worked Example — STM32F4 Out of Tree, which takes that board through the customization ladder end to end.
PX4 Autopilot (external module)
Single-node starter on PX4 Autopilot via the external-module
copy-out template. PX4’s EXTERNAL_MODULES_LOCATION pattern lets
downstream firmware drop in nano-ros without forking PX4 itself.
C++ only — PX4’s uORB binding is C++-only (Rust + C not in the
coverage matrix).
Prereqs. A PX4-Autopilot ≥ v1.16 clone, the matching cross toolchain (e.g.
gcc-arm-none-eabifor Pixhawk targets), andpython3with the PX4 development requirements installed (bash ./Tools/setup/ubuntu.shonce).
Project layout
PX4 external modules live outside the PX4 source tree and are
hooked in at configure time. The template at
integrations/px4/module-template/ has the PX4-required
src/modules/<name>/ shape:
my_drone_firmware/
├── PX4-Autopilot/ # PX4 source tree (submodule)
└── px4-modules/ # passed via EXTERNAL_MODULES_LOCATION
└── nano-ros/ # copy-out from integrations/px4/module-template/
└── src/
├── CMakeLists.txt # populates config_module_list_external
└── modules/
└── nano_ros_app/
├── CMakeLists.txt # px4_add_module(... MAIN nano_ros_app)
└── nano_ros_app.cpp # the user-editable app
Prereq. PX4 is a full-tier dependency. Provision the source trees first to populate
third-party/px4/PX4-Autopilotandthird-party/px4/px4-rs, then initialize PX4’s own submodules.
nros setup --source px4-autopilot --source px4-rs
git -C third-party/px4/PX4-Autopilot submodule update --init \
--depth 1 --recommend-shallow --recursive
PX4’s own Python dependencies come from its Tools/setup scripts
(e.g. PX4-Autopilot/Tools/setup/ubuntu.sh) — run those per PX4’s
documentation if the firmware build complains about missing Python
modules.
Contributors (in-tree checkout): just px4 doctor reports the
provisioning gap on a fresh clone.
Vendor the template into your firmware repo, then point PX4 at its
parent directory + tell the template where nano-ros lives via
NANO_ROS_DIR. The template accepts either form — a CMake cache
variable (-DNANO_ROS_DIR=<path>) takes precedence over an
environment variable (NANO_ROS_DIR=…), then the in-tree default:
cmake -B build -S PX4-Autopilot \
-DCONFIG=px4_fmu-v5_default \
-DEXTERNAL_MODULES_LOCATION=$PWD/px4-modules/nano-ros \
-DNANO_ROS_DIR=$PWD/../nano-ros # point at your nano-ros clone
# Or via environment, if you prefer:
NANO_ROS_DIR=$PWD/../nano-ros cmake -B build -S PX4-Autopilot \
-DCONFIG=px4_fmu-v5_default \
-DEXTERNAL_MODULES_LOCATION=$PWD/px4-modules/nano-ros
(EXTERNAL_MODULES_LOCATION must point at the dir containing
the template’s src/ — that’s px4-modules/nano-ros, not its parent.
PX4’s root CMakeLists.txt does add_subdirectory("${EXTERNAL_MODULES_LOCATION}/src" …),
so the path resolves to px4-modules/nano-ros/src per the layout above.)
Inside the module, the canonical pattern bridges uORB → nano-ros.
The module is a PX4Module subclass that runs in its own work
queue, opens an nros executor, and forwards uORB messages onto a
zenoh / DDS topic (or vice versa). Edit nano_ros_app.cpp to add
your topic bindings.
Configure
The template does not ship a Kconfig overlay (no Kconfig.projbuild
files). Module enablement is implicit once EXTERNAL_MODULES_LOCATION
points at the template’s parent. Pass RMW + ROS-edition selection
via CMake cache vars rather than menuconfig:
cmake -B build -S PX4-Autopilot \
-DCONFIG=px4_fmu-v5_default \
-DEXTERNAL_MODULES_LOCATION=$PWD/px4-modules/nano-ros \
-DNANO_ROS_DIR=$PWD/../nano-ros \
-DNANO_ROS_RMW=zenoh
(Adding a Kconfig overlay so the module appears under
menuconfig → External modules is a follow-up task; for now the
template is always enabled.)
Build
cd PX4-Autopilot
make px4_fmu-v5_default
# Or for the SITL simulator (POSIX target — easier to develop against):
make px4_sitl_default gazebo
The first build cross-compiles nano-ros’s Rust staticlibs alongside PX4’s NuttX kernel + apps (~10 min on a fresh checkout).
Run
# SITL: PX4 boots Gazebo + the autopilot binary
cd PX4-Autopilot
make px4_sitl_default gazebo
# In the PX4 console:
pxh> nano_ros_app start
# Real hardware (Pixhawk): flash via QGroundControl or
# `make px4_fmu-v5_default upload` over the bootloader USB
# Verify from stock ROS 2 on the same network (only after you have
# added uORB → nano-ros forwarders to nano_ros_app.cpp — the shipped
# template forwards nothing on its own):
source /opt/ros/humble/setup.bash
export RMW_IMPLEMENTATION=rmw_zenoh_cpp
ros2 topic echo /vehicle_local_position px4_msgs/msg/VehicleLocalPosition
Readiness signal. After nano_ros_app start in the PX4
console, the shipped template logs nano-ros uORB backend registered and returns immediately — it doesn’t start a
publisher loop or print a “bridge started” line on its own.
You’re expected to edit nano_ros_app.cpp to wire your
uORB → nano-ros forwards. If nano_ros_app reports register failed, check:
- Module didn’t register — the template logs
nros_rmw_uorb_register() -> <rc>on startup (search the PX4 boot log fornros_rmw_uorb_register). A non-zero<rc>usually means a NuttX kernel-config / feature-gate mismatch. - Once you’ve added forwarders,
zenohdmust be reachable from the autopilot’s network (Pixhawk: configured via QGroundControl orparam set). - uORB topic not advertised yet — start the upstream PX4 module
that publishes it (
commander startetc.) first. - See Troubleshooting — First 10 Minutes.
GitHub source
- PX4 external-module template:
integrations/px4/module-template/ - Worked PX4 example:
packages/testing/nros-px4-register-check/ - PX4 integration roadmap notes:
integrations/px4/README.md
Constraints to be aware of
- uORB binding is C++-only. The PX4 example collapses uORB registration to a C++ port; Rust / C variants exist for non-PX4 RTOSes but not for PX4.
- PX4’s NuttX kernel. Underneath, PX4 runs NuttX; if you need to debug at the kernel layer, the NuttX starter page applies too.
- uORB throughput vs zenoh hops. uORB is in-process pub/sub at ~µs latency; zenoh adds network-RTT. Plan accordingly when bridging high-rate streams.
Next
- Add your own uORB topics to the bridge: see the
nano_ros_app.cpptemplate’s topic-table section. - Multi-vehicle: PX4-XRCE-Agent → nano-ros XRCE backend gives you the standard PX4-ROS bridge with nano-ros on the autopilot side.
- For pure-NuttX (no PX4) firmware: see the NuttX starter.
PX4 (integration shell)
Contributor docs? Building nano-ros’s PX4 example from this repository is covered at PX4 (contributor).
nano-ros lifts the PX4 EXTERNAL_MODULES_LOCATION pattern into a
generic copy-out template under
integrations/px4/module-template/. Downstream PX4 apps vendor the
template, customise it, and point EXTERNAL_MODULES_LOCATION at
their copy.
Prereqs
- PX4-Autopilot checkout (
git clone https://github.com/PX4/PX4-Autopilot) - The full PX4 SITL build toolchain (
make px4_sitl_defaultworks standalone)
Contributors (in-tree checkout): for nano-ros’s in-repo PX4 checks:
just setup px4 # `_setup-common` + the px4 module recipe
just px4 doctor
One-liner vendor
cp -r /path/to/nano-ros/integrations/px4/module-template \
./my-px4-modules
Layout (PX4 enforces this exactly)
my-px4-modules/
└── src/
├── CMakeLists.txt (lists module names)
└── modules/
└── nano_ros_app/
├── CMakeLists.txt (px4_add_module)
└── nano_ros_app.cpp (module body)
Rename nano_ros_app consistently in all three places when adapting
for your own module.
Build
export EXTERNAL_MODULES_LOCATION=$PWD/my-px4-modules
export NANO_ROS_DIR=/path/to/nano-ros
make -C $PX4_AUTOPILOT_DIR px4_sitl_default
PX4’s build picks up the module via <location>/src/CMakeLists.txt.
The module CMakeLists pulls in nano-ros C++ headers via
$NANO_ROS_DIR.
Reference
The validated end-to-end SITL pattern (with the Rust-side
nros-rmw-cffi_register weak-stub trick) lives at
packages/testing/nros-px4-register-check/. Use that as the source of truth for the
register-check shape; this template is the generic shell for
downstreams writing their own modules.
Registry note
PX4 has no central module registry. Downstreams vendor the template
via git submodule or cp -r. See
Registry Publishing for
the cross-ecosystem comparison.
ARM FVP (FVP_BaseR_AEMv8R)
Run nano-ros on Arm’s Base_RevC AEMv8-R Fast Models — the
Cortex-A SMP profile under Zephyr 3.7 (board id
fvp_baser_aemv8r/fvp_aemv8r_aarch64/smp). The FVP is the
canonical local proxy for the safety-island reference platforms
that follow the same hwv2 Zephyr shape; pair this chapter with
the Zephyr (west module) starter for
the build half.
Status. The run half — invoking
FVP_BaseR_AEMv8Rand piping UART 0–3 to stdout — is landed.The two standalone talker lanes were retired (issue 0537): their runners had been deleted, so they built images nothing booted. What remains is the part with consumers — the board-crate import surface and the workspace Entry. FVP support is intended to grow back from those. See
docs/roadmap/phase-217-arm-fvp-local-runtime.md.
When to use
- You need to exercise a Cortex-A SMP Zephyr image on a developer laptop without dedicated silicon.
- You’re bringing up a new
hwv2safety-island target — the FVP is the closest in-tree reference for board.cmake + SMP boot + Cyclone DDS shape. - You’re validating stock-Cyclone-DDS RMW interop locally before promoting to a hardware bench.
The FVP is not a replacement for QEMU on Cortex-M (mps2-an385,
mps3-an547); those targets are covered by FreeRTOS (QEMU)
and the Zephyr starter. The FVP is also not the same surface
as Corellium’s AVH cloud FVP — AVH packages firmware as .coreimg
and provisions via a remote API; local FVP loads a raw ELF via
-a cluster0.cpu*=<elf>. AVH is out of scope for this chapter.
Prereqs
- A working Zephyr 3.7 workspace — run
nros setup zephyronce if you haven’t (see the Zephyr starter). - The Arm
Base_RevC AEMv8RFast Models binary — license-gated. Download from developer.arm.com — Arm Ecosystem FVPs after accepting the EULA. nano-ros does not download it (the[gated.arm-fvp]row innros-sdk-index.tomldeclares the tool but the fetch is your responsibility — same policy as the NVIDIA Orin SPE FSP).
After installing, export one of the discovery env vars:
# Preferred — Zephyr's canonical name; takes highest priority.
export ARMFVP_BIN_PATH=/opt/Arm/FastModels/Base_RevC_AEMv8R/models/Linux64_GCC-9.3
# Alternative — directory layout from the gated installer; the
# resolver scans `models/Linux64_GCC-*/` underneath it.
export ARM_FVP_DIR=/opt/Arm/FastModels/Base_RevC_AEMv8R
If neither is set, FVP_BaseR_AEMv8R is discovered via PATH as
a last-ditch fallback.
Installer surface
After extracting the Arm FVP tarball, run the discovery script:
ARM_FVP_DIR=/path/to/extracted/fvp \
scripts/installers/arm-fvp-installer.sh
The installer locates FVP_BaseR_AEMv8R under $ARM_FVP_DIR,
symlinks the containing directory to
~/.nros/sdks/arm-fvp/current/ (atomic via ln -sfn), and
prints the export ARMFVP_BIN_PATH=… line for your shell rc.
Run scripts/installers/arm-fvp-installer.sh --print-env later
to re-emit the export. It never downloads anything — gated-tool
policy.
Doctor check
The FVP is a license-gated tool: nano-ros checks that it resolves via
ARMFVP_BIN_PATH, ARM_FVP_DIR, PATH, or the canonical
~/.nros/sdks/arm-fvp/current/FVP_BaseR_AEMv8R landing path
(the [gated.arm-fvp] entry in nros-sdk-index.toml), and warns —
never hard-fails — when it can’t.
Contributors: the in-tree doctor cross-check and FVP run recipes are in Per-Platform Contributor Lanes.
Build
Contributors: the in-tree FVP build lanes (and the
zephyr.elfpaths they produce) are in Per-Platform Contributor Lanes.
Two standalone talker lanes (
build-fvp-aemv8r-cyclonedds{,-rust}, overexamples/zephyr/{cpp,rust}/talker-aemv8r) used to live here. Their tests were deleted and the lanes later retired, so the chapter no longer documents a lane whose output nobody consumes.
Run
Once the build artifacts and ARM_FVP_DIR / ARMFVP_BIN_PATH are in
place:
Contributors: the in-tree FVP run lanes (and what they do under the hood) are in Per-Platform Contributor Lanes.
Expected output
The Zephyr 3.7 boot banner appears on UART0 first, followed by
the talker. The exact line counts depend on CONFIG_BOOT_BANNER
and your locator config, but the markers to look for are:
*** Booting Zephyr OS build v3.7.0 ***
[00:00:00.xxx,000] <inf> nros: session up (domain 0)
Publishing: 'Hello World: 1'
Publishing: 'Hello World: 2'
...
For the cpp/cyclonedds recipe, the same banner is followed by
Cyclone DDS reader-match logs and std_msgs/String publish lines.
Verify ROS 2 interop by running a sibling listener in another
terminal:
# stock ROS 2 — reads the FVP's Cyclone DDS publisher
ros2 topic echo /chatter std_msgs/msg/String
The same std_msgs/String payload (Hello World: N) + byte-equal
CDR framing must appear on both sides — that’s the stock-RMW
interop contract (byte-compatible with rmw_cyclonedds_cpp).
Cross-references
docs/roadmap/phase-217-arm-fvp-local-runtime.md— the roadmap doc for the FVP runtime slice (contributor detail on what has landed vs what is ongoing).docs/roadmap/archived/phase-117-cyclonedds-rmw.md— the Zephyr FVP build smoke + Cyclone DDS port the runtime exercises.- Environment Variables —
ARM_FVP_DIR/ARMFVP_BIN_PATH— the discovery contract. - Zephyr (west module) — the parent Zephyr starter; the FVP is a board-target slice of it.
examples/workspaces/realtime-cpp/src/fvp_entry/— the workspace Entry this chapter builds and boots.
Native (Host) Build
Use the native (host) target for first experiments, CI smoke tests, and
ROS 2 interoperability checks on Linux or *BSD. It uses OS sockets, the
host process environment, and the same source checkout layout as the
embedded targets.
native is a role — the host build rather than a cross build — not a
reach claim. The reach is linux: the platform layer
(nros-platform-posix) is POSIX-clean, but the host board crate’s
apply_tier_affinity calls sched_setaffinity with cpu_set_t, which
libc does not define for macOS, and only Linux is tested in CI. macOS is
not supported (AGENTS.md, phase-260); *BSD is a supported path that no
lane exercises.
When to Use It
- You want the shortest path to a working nano-ros publisher or subscriber.
- You are validating message generation, CMake integration, or ROS 2 interop before moving to an RTOS.
- Your deployment target is Linux, *BSD, or an embedded Linux system.
Setup
Build the in-tree nros CLI (Phase 218), then provision the native
host. For a host build there is no cross-toolchain to fetch — nros setup native installs the client stack (and for xrce the
Micro-XRCE-DDS agent) into a shared store. The zenoh router is NOT
installed: it comes from a ROS 2 install
(ros2 run rmw_zenoh_cpp rmw_zenohd, RFC-0075). No ROS 2 on this
machine? Use --rmw cyclonedds — it needs no daemon.
# Build the in-tree nros CLI:
./scripts/bootstrap.sh # builds packages/cli/target/release/nros
source ./activate.sh # OR: direnv allow / source ./activate.fish
# Provision the native host (zenoh RMW is the default):
nros setup native --rmw zenoh # or: --rmw xrce / --rmw cyclonedds
native and linux are the two names of this board (native the role,
linux the reach); posix names the platform layer underneath it, not the
board.
For a colcon consumer workspace that already has nano-ros under
src/:
colcon build && source install/setup.bash
Package Layout
For a host application package:
my_native_node/
├── package.xml
├── Cargo.toml # Rust path
├── CMakeLists.txt # C / C++ path
└── src/
└── main.rs # or main.c / main.cpp
Examples are copy-out ready: for Rust, cp -r the example, run
NROS_REPO_DIR=<nano-ros checkout> nros sync, and cargo build;
for C/C++, cp -r and configure with
-DNANO_ROS_ROOT=<nano-ros checkout> (or export NROS_REPO_DIR) —
the example CMakeLists.txt resolves the checkout via its
NANO_ROS_ROOT guard.
Code Example
Rust publisher skeleton:
use core::fmt::Write as _;
use nros::prelude::*;
use std_msgs::msg::String as StringMsg;
fn main() -> Result<(), Box<dyn std::error::Error>> {
let config = ExecutorConfig::from_env().node_name("talker");
let mut executor = Executor::open(&config)?;
let mut node = executor.create_node("talker")?;
let publisher = node.create_publisher::<StringMsg>("/chatter")?;
let mut count = 0i32;
loop {
count += 1;
let mut msg = StringMsg::default();
let _ = write!(msg.data, "Hello World: {count}");
publisher.publish(&msg)?;
executor.spin_once(100);
}
}
Use the full First Node — Rust walkthrough for generated messages and runnable commands.
Build and Run
Start the RMW host daemon (installed by nros setup native). For
zenoh:
ZENOH_CONFIG_OVERRIDE='listen/endpoints=["tcp/127.0.0.1:7447"];scouting/multicast/enabled=false' ros2 run rmw_zenoh_cpp rmw_zenohd
Run the node directly via cargo run (Rust) or
cmake --build build && ./build/<binary> (C/C++), or via
colcon build && source install/setup.bash && ros2 run … if the
package lives in a colcon consumer workspace.
Configuration
Native examples can read runtime settings from the shell:
export ROS_DOMAIN_ID=0
export NROS_LOCATOR=tcp/127.0.0.1:7447
The same fields are compiled into embedded targets through their platform-specific configuration files. See Configuration for the full layering.
Deployment
Host deployment is normal process deployment: install the workspace,
source install/setup.bash, set environment, and run the binary. For
ROS 2 interop, run ROS 2 Interoperability
and set the ROS side to rmw_zenoh_cpp.
Workflow by Platform and Language
You have a target and a language. This page is the sequence of commands that pair implies, and the one step whose absence is the most common first failure.
The rest of the book is organized by where you are going — a starter page per platform. This page is organized by what you type, because the commands vary along a different axis than the pages do: the builder follows from your language, and the toolchain follows from your platform. Neither table below is a summary of the other.
The step that is easy to miss
Rust leaves need nros sync before their first build. C and C++
leaves do not.
A Rust leaf resolves its nano-ros dependencies through a
[patch.crates-io] table in its own .cargo/config.toml, and that
table is generated — .gitignore excludes it, so a fresh clone does not
have it. Sixty-seven Rust example leaves have such a file on a synced
checkout; fifty of them are committed shells that include the
generated central table:
include = [ "../../../../../nros-patch.toml", "nros-board.toml"]
Skipping the sync fails in two different ways, and the quieter one is
worse. Where the config is committed, cargo treats the missing
include as a hard error while parsing the manifest — before it
builds anything, and before any message about nano-ros could appear:
error: failed to parse manifest at `<leaf>/Cargo.toml`
Caused by:
could not load Cargo configuration
Caused by:
failed to load config include `../../../../../nros-patch.toml` from `<leaf>/.cargo/config.toml`
Caused by:
failed to read configuration file `<...>/nros-patch.toml`
Caused by:
No such file or directory (os error 2)
Nothing in those five frames says nros sync. If you see it, this is
what it means.
Where the whole config is generated — the native leaves, for instance — there is no file at all in a fresh clone, so there is no error to read. The patch table simply is not there, and the nano-ros crates your leaf names go looking for themselves on crates.io instead of in your checkout. That one does not announce itself.
Contributors (in-tree checkout): the just recipes —
just <module> build-fixtures and friends — run
nros sync for you, so they work from a fresh clone. It is the
hand-run cd <leaf> && cargo build that needs you to run it yourself.
C and C++ leaves have no .cargo/config.toml at all — not committed,
not generated. Their message bindings are produced inside CMake by
nros_find_interfaces(), and the cargo builds CMake drives resolve
against the repo-root config, which carries no include. Running
nros sync for a C/C++ build is harmless but buys nothing.
You need it once per checkout location, not once per build. Re-run
it after editing a .msg, .srv, or .action file, and — because the
central table it writes holds absolute paths — after moving the
checkout, or after one of the patched crates moves within it.
Which builder your cell uses
Each cell is the builder that nano-ros’s own CI uses for that pair, read
from examples/fixtures.toml — the manifest the fixture builds and the
staleness probe both consume. A dash means the pair has no in-tree
coverage today, not that it is forbidden.
The row names are the manifest’s, which are shorter than the ones you
type: freertos here is the platform whose board is
qemu-arm-freertos and whose examples live in
examples/qemu-arm-freertos/. The per-platform table further down maps
all three spellings.
| platform | rust | c | cpp | mixed |
|---|---|---|---|---|
linux | cargo | cmake | cmake | cmake |
freertos | cargo | cmake | cmake | cmake |
nuttx | cargo | cmake | cmake | — |
nuttx-riscv | cargo | cmake | cmake | — |
threadx-linux | cargo | cmake | cmake | cmake |
threadx-riscv64 | cargo, cmake | cmake | cmake | — |
zephyr | cargo, west | cmake, west | cmake, west | cargo |
zephyr-cortex-m | west | west | west | — |
esp32 | cargo | — | — | — |
qemu-arm-baremetal | cargo | — | — | — |
qemu-esp32-baremetal | cargo | — | — | — |
Where a row lists two builders, both are real and they build different
things: on Zephyr, west builds the single-node examples under
examples/zephyr/<lang>/, and cargo/cmake build the workspace
examples under examples/workspaces/.
The three command shapes
Every cell above is one of these three, plus platform-specific flags that live on the platform’s own starter page.
cargo — Rust, every platform
./scripts/bootstrap.sh # builds the in-tree nros CLI
source ./activate.sh # OR: direnv allow / source ./activate.fish
nros setup <board> --rmw zenoh # toolchain + SDK for the target
cd <leaf>
nros sync # ← the step above; once per checkout
cargo build --release
Some leaves also pin their cross target in that same
.cargo/config.toml ([build] target = "thumbv7m-none-eabi" on the
Cortex-M ones), so no --target on the command line. Others get it from
the platform’s recipe instead — contributors: in an in-tree
checkout, just --list <module> shows which
recipe builds what, and using the recipe avoids having to know.
cmake — C and C++
./scripts/bootstrap.sh
source ./activate.sh
nros setup <board> --rmw zenoh
cd <leaf>
cmake -B build -DCMAKE_TOOLCHAIN_FILE=<toolchain> -DCMAKE_BUILD_TYPE=Release
cmake --build build --parallel
No nros sync. The toolchain file is per platform — see the starter
page. -D_NANO_ROS_CODEGEN_TOOL= is not needed when nros is on PATH;
CMake resolves it.
west — Zephyr single-node examples
Zephyr owns the build. See
Zephyr (west module) for the
module wiring; the Rust leaves under examples/zephyr/rust/ still need
nros sync first, because west drives cargo and cargo reads the leaf
config either way.
In the checkout, or copied out?
The commands above are written as cd <leaf> inside the nano-ros
checkout, which is the fastest way to see something run. For anything
beyond that, copy the example directory out — examples are standalone
copy-out projects with no workspace walk-up, so a copied one builds on
its own.
The distinction matters for one reason: a bare cargo build writes
target/ next to the leaf. In your project that is exactly right. In
the nano-ros checkout it is residue the repo’s own gate rejects
(check-example-leaf-target-dirs) — in-tree builds are expected to go
through the contributor recipes (just <module> …), which write into a
shared build directory
instead. One in-repo cargo build --release of a Cortex-M example
leaves 269 MB behind.
So: exploring in the checkout, prefer the recipe. Building your own
thing, copy the example out and cargo build normally.
Per platform
| platform | grid row | nros setup <board> | examples under | recipes (contributors) | starter page |
|---|---|---|---|---|---|
| Linux host | linux | native | examples/native/ | just native … | Native host build |
| FreeRTOS (QEMU MPS2-AN385) | freertos | qemu-arm-freertos | examples/qemu-arm-freertos/ | just freertos … | FreeRTOS |
| NuttX (Arm) | nuttx | qemu-arm-nuttx | examples/qemu-arm-nuttx/ | just nuttx … | NuttX |
| NuttX (RISC-V) | nuttx-riscv | qemu-riscv-nuttx | examples/qemu-riscv-nuttx/ | just nuttx … | NuttX |
| ThreadX (Linux sim) | threadx-linux | threadx-linux | examples/threadx-linux/ | just threadx_linux … | ThreadX |
| ThreadX (QEMU RISC-V 64) | threadx-riscv64 | qemu-riscv64-threadx | examples/qemu-riscv64-threadx/ | just threadx_riscv64 … | ThreadX |
| Zephyr | zephyr, zephyr-cortex-m | zephyr | examples/zephyr/ | just zephyr … | Zephyr |
| ESP32 | qemu-esp32-baremetal (single-node), esp32 (workspace) | qemu-esp32-baremetal | examples/qemu-esp32-baremetal/ | just esp32 … | ESP32 |
| Bare-metal Cortex-M3 | qemu-arm-baremetal | qemu-arm-baremetal | examples/qemu-arm-baremetal/ | just qemu … | Bare-metal |
| Arm FVP (Cortex-A SMP) | — | zephyr + a license-gated FVP binary | — | — | ARM FVP |
Multi-node workspace examples do not follow that directory rule: they
all live under examples/workspaces/, selected by fixture row rather
than by directory. That is why ESP32 has two grid rows —
qemu-esp32-baremetal for its single-node examples and esp32 for its
share of the workspace ones. Zephyr’s two rows split differently:
zephyr and zephyr-cortex-m build the SAME examples for different
boards.
Contributors (in-tree checkout): each module’s recipes are discoverable rather than memorized:
just --list freertos # every FreeRTOS recipe, grouped
just --list zephyr
More than one node
The sequence above builds one leaf. A project with several nodes adds a Bringup package and an Entry package on top of it, and the build is still one of the three shapes above — see Project layout.
If it does not work
Application Workflow
nano-ros users usually want one path: prepare package, write node, build it, then deploy to target. Use Concepts only when workflow raises a technical question.
1. Prepare Workspace and Package
nano-ros is shipped source-only — vendor it next to (or inside) your
workspace, then provision the board’s toolchain with nros setup:
# Build the in-tree nros CLI (Phase 218), then provision your board (+ RMW):
./scripts/bootstrap.sh base
source ./activate.sh # OR: direnv allow / source ./activate.fish
nros setup native --rmw zenoh # or qemu-arm-freertos, zephyr, …
nros setup provides most components prebuilt per platform per RMW; a few
without a seeded asset (the vendored submodules — zenoh-pico, mbedtls,
RTOS sources) are built from source. nros setup <board> --rmw <rmw> --dry-run
shows which is which on your host — see
Installation.
It does not provide a zenoh router: that is ROS’s rmw_zenohd, so a
multi-process zenoh example needs a ROS 2 install (or NROS_RMW_ZENOHD).
--rmw cyclonedds needs no daemon at all.
For multi-package workspaces (Pattern A — recommended for POSIX +
mixed C / C++ / Rust deployments), put nano-ros and your packages
side-by-side under a shared src/:
~/ros2_ws/
├── src/
│ ├── nano-ros/ # this repo
│ └── my_robot_node/
│ ├── package.xml
│ ├── Cargo.toml # Rust — deps + [package.metadata.nros.*]
│ ├── CMakeLists.txt # C/C++ — targets (deploy tuple in package.xml)
│ └── src/
└── build/ install/ log/ # if you use colcon
For third-party C/C++ projects without colcon (Pattern B), pull
nano-ros in as a third_party/nano-ros/ git submodule and consume
it via add_subdirectory(third_party/nano-ros nano_ros).
See Installation for both patterns. The per-language starter pages document the canonical package shape in their “Project layout” sections: Rust, C, C++. For two or more nodes, use the Multi-Node Projects group: start from the full project layout, then drill into Node, Bringup, and Entry packages.
2. Write Node Code
Choose API language first:
- Rust — use
nros, generated message crates, andExecutor. - C — include
nros/nros.hand generate interfaces withnros_find_interfaces(LANGUAGE C)in CMake (one of five codegen entry points — which to use). - C++ — include
nros/nros.hppand use typed wrappers.
Start with one of the Linux starters above, then adapt to your target via the Embedded Starters section.
3. Generate Messages
If you use custom .msg, .srv, or .action files, generate bindings
inside the workspace/build tree. See
Message Binding Generation.
4. Configure Target
Pick a platform and an RMW backend at build time (compile-time
choice — there is no RMW_IMPLEMENTATION runtime switch on embedded
targets):
| CMake side | Cargo side |
|---|---|
set(NANO_ROS_PLATFORM <plat>) | feature platform-<plat> |
set(NANO_ROS_RMW <rmw>) | feature rmw-<rmw>-cffi (transports auto-pull) |
set(NANO_ROS_BOARD <board>) (optional, embedded only) | nros-board-<board> dep |
Supported pairs: posix / freertos / nuttx / threadx / zephyr / esp32 / baremetal × zenoh / xrce / cyclonedds. Not every cell is
implemented — see the Coverage Matrix.
Runtime configuration (ROS_DOMAIN_ID, NROS_LOCATOR, …) works on
POSIX. Embedded targets bake config from
[package.metadata.nros.deploy.<t>] (Rust) / the package.xml
<nano_ros deploy=…/> tuple (C/C++), plus
Kconfig on Zephyr. See Configuration.
5. Build, Test, Deploy
For each canonical entry point:
# Single Rust example — `nros sync` first, once per checkout location:
cd examples/native/rust/talker
nros sync
cargo run
# Single C/C++ example (CMake + add_subdirectory) — no sync needed:
cd examples/qemu-arm-freertos/cpp/talker
cmake -B build -DCMAKE_TOOLCHAIN_FILE=$PWD/../../../../../cmake/toolchain/arm-freertos-armcm3.cmake
cmake --build build
# Contributors: per-platform multi-example fixture build, in-tree
# checkout only (these run `nros sync` for you):
just freertos build-fixtures
just zephyr build-fixtures
just nuttx build-fixtures
# Contributors: discover full-matrix commands for a platform:
just --group full-matrix --list zephyr
# Multi-node system — sync, bake the bringup, build the entry:
nros sync
nros codegen-system --bringup <bringup-pkg> --out <out-dir>
cargo build -p <entry> # or: cmake --build <dir> --target <entry>
# POSIX-only colcon consumer-workspace build:
colcon build && source install/setup.bash
Which of the three build shapes your target uses, and which need the sync step, is Workflow by Platform and Language.
nros codegen-system resolves the bringup’s system.toml (plus its
launch files) into a SystemModel under the workspace build tree.
SystemModels are build artifacts — never committed, and never named
by an entry package; an entry names its input
(nros::main!(launch = "bringup"), nano_ros_entry(BRINGUP … LAUNCH …))
and the build locates the artifact. nros model-path prints where a
resolved one landed.
nros metadata / nros plan / nros check are the inspection path,
not the build path: they produce and validate an nros-plan.json you
can read with nros explain. You do not need them to build.
For target-specific deployment, go to the matching platform guide. Each guide covers toolchain setup, package layout, code example, build command, run/flash command, and deployment notes.
See Deployment Workflow.
Build as a CMake subdirectory
This is one of the two ways to integrate nano-ros into a C or C++
project, and the lower-level one: the repo’s top-level CMakeLists.txt
exposes everything via add_subdirectory(...). The higher-level,
ament-shaped alternative is find_package(nano_ros REQUIRED) —
nano_rosConfig.cmake at the checkout root, located via
nano_ros_ROOT (RFC-0048) — which every in-tree example now uses.
Either way there is NO install step: phase-140 removed the install
prefix and the old find_package(NanoRos) (capital) config pipeline.
Vendoring the repo into a company tree (pinning, offline CI, upgrade
workflow) has its own page:
Integrating into a Vendored Tree.
Layout
my_app/
├── CMakeLists.txt
├── main.c
└── third_party/
└── nano-ros/ # git clone / git submodule of this repo
User project CMakeLists.txt
cmake_minimum_required(VERSION 3.22)
project(my_app C)
# Pick platform + RMW BEFORE add_subdirectory.
set(NANO_ROS_PLATFORM posix) # posix | freertos | nuttx | threadx | zephyr | baremetal
set(NANO_ROS_RMW zenoh) # zenoh | xrce | cyclonedds
add_subdirectory(third_party/nano-ros nano_ros)
add_executable(my_app main.c)
target_link_libraries(my_app PRIVATE NanoRos::NanoRos)
if(COMMAND nros_platform_link_app) # defined by zephyr/threadx platforms only
nros_platform_link_app(my_app)
endif()
# Optional — generate C bindings for ROS 2 .msg / .srv / .action files.
# (LANGUAGE defaults to CPP; full spelling table → user-guide
# "Message Generation".)
nano_ros_generate_interfaces(std_msgs LANGUAGE C
DEPENDENCIES builtin_interfaces)
target_link_libraries(my_app PRIVATE std_msgs__nano_ros_c)
That’s the whole story for the host-POSIX / zenoh case. CMake’s
transitive target propagation pulls in libnros_c.a,
libnros_rmw_zenoh_staticlib.a, the POSIX platform shim, system
libraries (pthread, dl, m), and the per-build
nros_config_generated.h header automatically.
Cache variables
| Variable | Default | Values |
|---|---|---|
NANO_ROS_PLATFORM | posix | posix, freertos, nuttx, threadx, esp_idf (legacy spellings: freertos_armcm3, nuttx_armv7a, threadx_linux, threadx_riscv64). Zephyr and bare-metal are not add_subdirectory platforms — they enter via the Zephyr module / cargo directly |
NANO_ROS_BOARD | (unset) | required for threadx (threadx-linux or riscv64-qemu) |
NANO_ROS_SKIP_BOOTSTRAP | OFF | ON skips the configure-time bootstrap (git submodule updates + a possible network FetchContent of Corrosion). Required for air-gapped/mirrored CI — pre-seed submodules and install Corrosion first (nros setup --tool corrosion) |
NANO_ROS_FEATURES | (empty) | extra cargo features forwarded to the Rust build |
NANO_ROS_RMW | zenoh | zenoh, dds, xrce, cyclonedds |
NANO_ROS_ROS_EDITION | humble | humble, iron, jazzy |
NANO_ROS_BUILD_CODEGEN | ON | ON / OFF |
Variables MUST be set(...) BEFORE add_subdirectory(...) — the
sub-project consumes them at include time.
What about installing?
Phase-140 deleted every nano-ros-side install(...) rule. nano-ros is
consumed in source form — never out of an installed prefix. If you need a
shippable artefact, your user project owns the install layout; ship
your binary, not nano-ros itself.
For RTOS users who want a more idiomatic surface than raw
add_subdirectory, see the integration shells under
integrations/<rtos>/ — they translate west / esp-idf / NuttX / PX4
manifests into the same root CMake. Each shell is a
~20-line wrapper around add_subdirectory(<repo>).
Worked example
The examples/native/c/talker/CMakeLists.txt is the canonical
copy-out template: it resolves the nano-ros checkout root once
(-DNANO_ROS_ROOT cache var → NROS_REPO_DIR env var → in-repo
walk-up), includes the workspace helpers, generates the message
bindings (nros_find_interfaces), and declares the app via
nano_ros_entry(...) — ~55 lines including codegen + per-app fixup.
All in-tree C/C++ examples follow the same shape.
Integrating nano-ros into a Vendored Tree
This page is for the engineer whose company has its own source tree — a BSP, possibly a forked RTOS, an existing build system — and needs nano-ros to live inside it: pinned, reproducible, buildable offline, upgradable on their schedule. It collects the contracts the other chapters mention in passing into one place.
nano-ros is source-only: nothing is on crates.io, there is no install prefix, and no binary distribution. Vendoring the repository is not a workaround — it is the supported consumption model.
Pinning the checkout
Vendor the repo as a git submodule (or subtree) and pin a release tag:
git submodule add https://github.com/NEWSLabNTU/nano-ros.git third_party/nano-ros
git -C third_party/nano-ros checkout nros-v0.5.0
git add third_party/nano-ros
Rules that keep the pin trustworthy:
- Pin tags (
nros-v<X.Y.Z>), notmain. A moving branch unships the pairing you tested. - Move the pin forward only. A rewind silently drops whatever the skipped commits fixed, and two hex strings in a diff cannot be ordered by eye.
- nano-ros’s own submodules are decisions, not lags. The Cyclone DDS fork tracks the Cyclone that ROS ships; zenoh-pico is pinned at a wire-stable version. Never “update to latest” inside the vendored checkout — take what the tag pinned.
Initialize nano-ros’s submodules non-recursively — the launch toolchain’s third-level runtime submodules are never built by nano-ros:
git -C third_party/nano-ros submodule update --init packages/cli/third-party/play_launch
For build-related submodules you normally do nothing: the configure
step populates the ones your (platform, RMW) pair needs (see
Bootstrap below), or you pre-seed them explicitly for offline builds.
What a fresh vendored checkout needs, in order
Four things, and the order matters:
- The
nrosCLI../scripts/bootstrap.shinside the checkout buildspackages/cli/target/release/nros. Nothing else works without it. - An activated environment for interactive use:
source third_party/nano-ros/activate.shwiresnrosand the SDK-store tools ontoPATHand exportsnano_ros_ROOT. CI can skip this and pass paths explicitly (-Dnano_ros_ROOT=…, absolute CLI path). - Provisioned toolchains.
nros setup <board> --rmw <rmw>fetches the cross-compiler, emulator, and SDK sources for exactly that pair into the shared store at${NROS_HOME:-~/.nros}/sdk.--dry-runprints the plan first. (The zenoh router is deliberately not provisioned — it comes from a ROS 2 install,ros2 run rmw_zenoh_cpp rmw_zenohd.) - Message codegen.
NROS_REPO_DIR=<checkout> nros syncfrom your workspace root — see the next section for why this is not optional.
The Rust side: nros sync is mandatory, and your config is inline
Two things in every Rust leaf are gitignored and generated: the
generated/ message crates and the cargo patch machinery that
resolves registry-style names (nros = { version = "*" }) to the
vendored sources. A fresh clone cannot build a Rust package until
nros sync has run — the failure otherwise is cargo dying during
manifest parse with an error that never names sync.
For an out-of-tree consumer (your workspace, outside the nano-ros
checkout), nros sync writes the [patch.crates-io] table inline
into your leaf’s .cargo/config.toml, with absolute paths, and no
include line. This is deliberate:
- The in-repo examples use a relative
include = ["…/nros-patch.toml"]pointing at a central gitignored file. That shape has three fragile preconditions (cargo ≥ 1.93, a correct relative path, the central file existing) and its failure modes are confusing. Outside the checkout you get the whole table inline, so the only failure mode is loud. - Never copy an in-repo example’s
.cargo/config.tomlinto your tree. You would be copying the relativeincludeline, which cannot resolve from your directory. Copy the package and runnros sync— it writes the correct config for that location. - A moved checkout invalidates the absolute paths — re-run
nros syncafter relocating either the workspace or the vendored nano-ros.
Message dependencies stay path deps pinned 0.0.0
(std_msgs = { path = "generated/std_msgs" } after sync); never
registry-name a message crate — a bare std_msgs = "*" can resolve
against the public crates.io.
The CMake side
Two entry points, both source-backed:
find_package(nano_ros REQUIRED)— the ament-shaped entry (nano_rosConfig.cmakeat the checkout root). Located vianano_ros_ROOT, whichactivate.shexports; CI passes-Dnano_ros_ROOT=<checkout>explicitly.add_subdirectory(third_party/nano-ros)— the lower-level shape. See Build as a CMake subdirectory for the cache-variable table.
Either way, at configure time nano-ros’s bootstrap.cmake runs
tools/setup.sh to populate the source submodules your
(NANO_ROS_PLATFORM, NANO_ROS_RMW) pair needs, and may FetchContent
Corrosion from the network if none is installed. That is correct on a
developer machine and wrong in hermetic CI — see the next section.
Air-gapped / mirrored CI
The offline contract, all of it:
-DNANO_ROS_SKIP_BOOTSTRAP=ON. Disables the configure-time submodule fetch and any network FetchContent. You now own pre-seeding.- Pre-seed the submodules your platform/RMW pair needs in your
mirror (e.g.
zpico-sys/zenoh-pico, the FreeRTOS kernel + lwIP, the Cyclone fork —tools/setup.shderives the per-target list fromnros-sdk-index.toml, which is the single manifest of what comes from where). - Pre-install Corrosion:
nros setup --tool corrosionstages it into the SDK store, where the configure finds it without the network. Verify the configure line printsnano-ros: Corrosion <ver> via <origin>— never infer the version from having run an installer; a stale store entry can shadow a fresh one. - Cache the SDK store. Everything
nros setupfetches lands under${NROS_HOME:-~/.nros}/sdk; pointNROS_HOMEat a shared or cached location and the store is reusable across jobs and machines of the same host triple. - Mirror the asset sources.
nros-sdk-index.tomlnames every asset’s upstream ([tool.*],[source.*]sections). For a full mirror, rehost those URLs and patch the index in your vendored checkout — that patch is part of the carried diff below, and the index is versioned with the tree so the patch rebases cleanly.
The patch set you carry
Be clear-eyed: some extensions live inside the vendored checkout. There is no out-of-tree escape hatch for them today, so a vendored tree carries a small local diff and rebases it across tag bumps:
| You add… | Files inside the checkout |
|---|---|
| A platform (your RTOS) | packages/platform/nros-platform/Cargo.toml (feature + optional dep), nros-platform/src/resolve.rs (type alias), cmake/platform/nano-ros-<name>.cmake |
| A board the tooling can see | none needed since NROS_EXTRA_BOARD_PATH: point it (env, or CMake cache var for nano_ros_use_board()) at a directory of board crates in YOUR tree — same layout as packages/boards/. Board keys stay global (a name under two roots errors). Copying into packages/boards/ remains the fallback |
| A mirrored SDK index | nros-sdk-index.toml URL edits |
An application, a message package, a board overlay crate, and a custom transport all live entirely in your tree — no diff.
Workflow for the diff: keep a branch in your nano-ros fork
(company/nros-v0.5.0-patches), and on upgrade rebase it onto the new
tag rather than merging. The registration edits are a few lines each;
conflicts are rare and loud.
Upgrading the pin
git -C third_party/nano-ros fetch origin --tags
git -C third_party/nano-ros rebase --onto nros-v0.6.0 nros-v0.5.0 company/patches
# build + test, then:
git add third_party/nano-ros
After every pin move, in order: rebuild the nros CLI
(./scripts/bootstrap.sh), re-run nros sync in each consuming
workspace (the sync output is keyed to the CLI), then rebuild. Stale
sync output after an upgrade produces build errors that look like
missing packages, not like staleness.
Checklist
- Submodule pinned to an
nros-vtag, forward-only -
play_launchsubmodule initialized non-recursively - CI passes
-Dnano_ros_ROOT=/NROS_REPO_DIR=explicitly (no reliance on an activated shell) -
nros syncruns in CI before any Rust build; no.cargo/config.tomlcopied from in-repo examples - Air-gapped:
NANO_ROS_SKIP_BOOTSTRAP=ON, submodules pre-seeded, Corrosion staged,NROS_HOMEcached - Local patch set (platform/board/index edits) lives on a fork branch, rebased per tag
- Router comes from the ROS 2 install on the host that runs it — nothing in the vendored tree provides one
Related
- Build as a CMake subdirectory — cache variables, link targets
- How Integration Works — per-RTOS build hosting
- Vendor Overlay Board Crate — board glue that lives in your tree
- Worked Example — STM32F4 Out of Tree
- Install — the interactive first-time flow this page assumes you are automating
Message Binding Generation
nros uses generated Rust bindings for ROS 2 message types. The nros generate-rust command generates no_std compatible bindings from package.xml dependencies.
Overview
The binding generator lives in the in-tree CLI sub-workspace at packages/cli/ and provides:
nrosstandalone binary- Pure Rust,
no_stdcompatible output usingheaplesstypes - Automatic dependency resolution via ament index or bundled interfaces
.cargo/config.tomlgeneration for crate patches
Prerequisites
-
package.xml in project root - Declares ROS interface dependencies
<?xml version="1.0"?> <package format="3"> <name>my_package</name> <version>0.1.0</version> <description>My nros package</description> <maintainer email="dev@example.com">Developer</maintainer> <license>Apache-2.0</license> <depend>std_msgs</depend> <depend>geometry_msgs</depend> <export> <build_type>ament_cargo</build_type> </export> </package> -
nros tool built + on PATH
The
nrosCLI is built from the in-tree sub-workspace atpackages/cli/and put on PATH by the activate file:# From the nano-ros repository root ./scripts/bootstrap.sh # builds packages/cli/target/release/nros source ./activate.sh # OR: direnv allow / source ./activate.fishSee Installation for the full walkthrough.
-
ROS 2 environment (optional for standard types)
Standard interfaces (
std_msgs,builtin_interfaces) are bundled with nros and work without ROS 2. For additional packages (e.g.,geometry_msgs,sensor_msgs), source a ROS 2 environment:source /opt/ros/humble/setup.bash
Workflow
Step 1: Create package.xml
Declare your ROS interface dependencies in <depend> tags:
<depend>std_msgs</depend> <!-- For std_msgs::msg::Int32, String, etc. -->
<depend>example_interfaces</depend> <!-- For service types -->
Step 2: Generate bindings
cd my_project
nros sync
This will:
- Parse
package.xmlto find dependencies - Resolve transitive dependencies (ament index + bundled interfaces)
- Filter to interface packages (those with msg/srv/action)
- Generate bindings to
generated/directory - Write the
[patch.crates-io]entries into.cargo/config.toml
(nros generate-rust is the codegen-only primitive — same steps 1–4
with no patch side-effects; add --generate-config to also write the
patches.)
Step 3: Add dependencies to Cargo.toml
Reference the generated crates using crates.io version specifiers:
[dependencies]
std_msgs = { version = "*", default-features = false }
example_interfaces = { version = "*", default-features = false }
The .cargo/config.toml patches redirect these to local paths.
Why msg crates are RMW-agnostic
Generated msg crates carry only the wire-format data type — no backend-specific code, no per-RMW Cargo feature. A user manifest is the plain pair:
[dependencies]
std_msgs = { version = "*", default-features = false }
nros = { version = "*", features = ["rmw-cyclonedds"] } # RMW choice lives here
Transport choice and message schema are orthogonal concerns and the manifest reflects that. This matches upstream rclcpp + rclrs, which both ship msg packages RMW-agnostic and let the RMW pick which descriptor representation it wants at runtime.
For DDS-based backends that need a per-type descriptor on the wire
(Cyclone DDS today), the nros-rmw-cyclonedds shim builds those
descriptors lazily on first pub/sub for a given message type,
walks the static field schema exposed by nros-serdes (the
Message trait with const TYPE_NAME + const FIELDS), and caches
the result in a bounded no_std registry. No per-msg-pkg backend
code is required.
Tracking + sizing knob (NROS_CYCLONEDDS_MAX_TYPES): see
docs/roadmap/archived/phase-212-ux-cargo-native-and-file-consolidation.md
section 212.K.7.
Git Dependency Workflow
For projects that consume nros as a git dependency (not from within the nros repo), use --nano-ros-git instead of --nano-ros-path:
Step 1: Add git dependency to Cargo.toml:
[dependencies]
nros = { git = "https://github.com/NEWSLabNTU/nano-ros", default-features = false, features = ["std"] }
std_msgs = { version = "*", default-features = false }
Step 2: Create package.xml (same as above).
Step 3: Generate bindings with git patches:
source /opt/ros/humble/setup.bash
nros generate-rust --generate-config --nano-ros-git
This generates .cargo/config.toml with git-based patches:
[patch.crates-io]
nros-core = { git = "https://github.com/NEWSLabNTU/nano-ros" }
nros-serdes = { git = "https://github.com/NEWSLabNTU/nano-ros" }
std_msgs = { path = "generated/std_msgs" }
builtin_interfaces = { path = "generated/builtin_interfaces" }
Step 4: Use in code
#![allow(unused)]
fn main() {
use std_msgs::msg::Int32;
use example_interfaces::srv::{AddTwoInts, AddTwoIntsRequest, AddTwoIntsResponse};
let msg = Int32 { data: 42 };
}
Command Options
nros generate-rust [OPTIONS]
Options:
--manifest <PATH> Path to package.xml [default: package.xml]
-o, --output <DIR> Output directory [default: generated]
--generate-config Generate .cargo/config.toml with [patch.crates-io] entries
(alias: --config)
--nano-ros-path <PATH> Path to nros crates (for config patches, local dev)
--nano-ros-git Use nros git repo for config patches (external users)
--force Overwrite existing bindings
-v, --verbose Enable verbose output
Generated Output Structure
my_project/
├── package.xml # Your dependency declarations
├── Cargo.toml # Your package manifest
├── src/
│ └── main.rs # Your code using generated types
├── generated/ # Generated bindings (do not edit)
│ ├── std_msgs/
│ │ ├── Cargo.toml
│ │ └── src/
│ │ ├── lib.rs # #![no_std]
│ │ └── msg/
│ │ ├── mod.rs
│ │ └── int32.rs
│ └── builtin_interfaces/ # Transitive dependency
│ └── ...
└── .cargo/
└── config.toml # [patch.crates-io] entries
Generated Code Features
no_std by default:
#![allow(unused)]
#![no_std]
fn main() {
pub mod msg;
}
std feature for optional std support:
[features]
default = []
std = ["nros-core/std", "nros-serdes/std"]
heapless types for embedded:
#![allow(unused)]
fn main() {
pub struct String {
pub data: heapless::String<256>,
}
pub struct Arrays {
pub data: heapless::Vec<i32, 64>,
}
}
Service types with Request/Response:
#![allow(unused)]
fn main() {
pub struct AddTwoInts;
pub struct AddTwoIntsRequest { pub a: i64, pub b: i64 }
pub struct AddTwoIntsResponse { pub sum: i64 }
impl RosService for AddTwoInts {
type Request = AddTwoIntsRequest;
type Reply = AddTwoIntsResponse;
}
}
Standalone Package Mode
Examples are configured as standalone packages (excluded from workspace) because each has its own .cargo/config.toml patches. Build each example from its own directory:
cd examples/native/rust/talker && cargo build --features zenoh
cd examples/native/rust/service-client && cargo build --features zenoh
Regenerating Bindings
To regenerate after ROS package updates or dependency changes:
nros generate-rust --force
Bundled Interfaces
nros ships standard .msg files for common packages so codegen works without a
ROS 2 environment:
std_msgs(Bool, Int32, String, Header, etc.)builtin_interfaces(Time, Duration)
These are located at packages/cli/interfaces/. When a ROS 2 environment is sourced,
the ament index takes precedence over bundled files.
Troubleshooting
“Package ‘X’ not found in ament index or bundled interfaces”
- For standard types (
std_msgs,builtin_interfaces): should work without ROS 2 - For other packages: source ROS 2 environment:
source /opt/ros/humble/setup.bash - Check package is installed:
ros2 pkg list | grep X - Install if missing:
sudo apt install ros-humble-X
Build errors with generated code
- Regenerate with
--forceflag - Check nros crate compatibility
Which CMake spelling (one table)
Five codegen entry points exist in the CMake surface. They are NOT interchangeable spellings of one thing — each serves one audience, and two are back-compat only. Pick by what you are doing:
| You are… | Use | Notes |
|---|---|---|
| Consuming an existing msg package from an app/node pkg | find_package(<pkg> REQUIRED) | The recommended default. The smart Find-stub resolves the package through the 3-layer search (workspace → ament → bundled) and generates bindings on the fly. Zero codegen calls in your file. |
| Defining msgs in a message package | rosidl_generate_interfaces(${PROJECT_NAME} msg/… DEPENDENCIES …) | Recommended — the stock ROS shape, intercepted by nano-ros. Your msg pkg carries zero nano-ros-specific lines and cross-builds under colcon unchanged. |
| Defining msgs and you want the nano-ros verb explicitly (or C bindings) | nano_ros_generate_interfaces(<name> <files…> [LANGUAGE C|CPP] [DEPENDENCIES …]) | The ament-shape alias (RFC-0048 §5). Defaults to LANGUAGE CPP like rosidl — pass LANGUAGE C for C. |
| Building a workspace root with several msg pkgs | nros_workspace_interfaces() after set(NROS_INTERFACE_SEARCH_PATH …/src) | Bulk-builds every workspace msg pkg in topological order; one line, no per-pkg add_subdirectory. |
Generating this pkg’s package.xml dependencies (no find_package flow) | nros_find_interfaces([LANGUAGE C|CPP] [SKIP_INSTALL]) | Consume-by-manifest: resolves the transitive interface closure of your own package.xml. Used by the workspace node pkgs. Defaults LANGUAGE CPP. |
| — | nros_generate_interfaces(<target> <files…>) | Deprecated for new code (Phase 210.E.4). The low-level generator everything above routes through; direct calls are back-compat only. |
C Code Generation (CMake)
nano_ros_generate_interfaces(… LANGUAGE C) generates C bindings for .msg, .srv,
and .action files (the function defaults to C++ — the LANGUAGE C
argument is what makes it the C generator). It uses a bundled codegen library — no external nros binary needed.
Prerequisites
nano_ros_generate_interfaces() becomes available automatically once
the consumer’s CMakeLists.txt invokes add_subdirectory(nano-ros).
The codegen tool ships inside the in-tree nros CLI binary
(packages/cli/target/release/nros; the old nros-codegen
submodule is retired); cmake auto-resolves it from
PATH / packages/cli/target/release/ / the transitional
${NROS_HOME:-~/.nros}/bin/. No separate build step.
Usage
See examples/native/c/custom-msg/CMakeLists.txt for a complete example.
Configuration Guide
nano-ros config is language-agnostic and has one home per concern — no
setting lives in two places, and nothing is merged across files. The authored
surfaces are Cargo.toml metadata (Rust) or the CMake nano_ros_* functions
(C/C++), a universal system.toml, package.xml, and launch XML — plus
Kconfig for the embedded build (Zephyr). This guide covers what each file
owns, the embedded deploy config both languages bake at build time, and the
standalone config.toml for hand-written no_std apps.
Two files you may find in old material are retired: the
nros.tomlper-package/workspace file (rejected by the CLI; see below) and the oldconfig.toml[network]/[zenoh]/[scheduling]schema. Design of record: RFC-0004.
One home per concern
| File | Owns | Per |
|---|---|---|
Cargo.toml | Rust build: crate, language deps, the RMW feature menu (rmw-zenoh/rmw-cyclonedds/rmw-xrce); node identity via [package.metadata.nros.node]; entry/boot via [package.metadata.nros.entry]; embedded net config via [package.metadata.nros.deploy.<target>]; workspace membership via [workspace.metadata.nros] | Rust project |
CMakeLists.txt | C/C++ build: targets, language deps, the NROS_RMW option; node/entry registration via nano_ros_auto_add_library + nros_components_register_node (RFC-0057; nano_ros_node_register remains as the compat spelling) / nano_ros_entry | C/C++ project |
.cargo/config.toml | [patch.crates-io] dependency injection only (written by nros sync; local crate + generated-msg paths), plus the cargo [build]/[target]/[env] knobs (target triple, runner, rustflags). No nano-ros runtime config. | Rust project |
package.xml | ROS package identity + msg <depend>s (codegen input for nros generate) | all |
system.toml | System topology — components, deploy targets, domain, RMW, capability axes ([safety], [param_services]), tiers. The language-agnostic universal descriptor (same schema for Rust/C/C++). Optional for single-node (the toolchain synthesises an implicit 1-component system when absent). | bringup pkg |
config.toml (standalone) | Hand-written no_std direct-mode apps only — [node] / [[transport]] / [node.rt], compile-baked via Config::from_toml(include_str!(…)). Apps that use nros::main!() / codegen do not have one — they use the deploy metadata above. | embedded single-node app (no codegen) |
Boundary rule. If a knob changes what is compiled/linked, it lives in the build file (
Cargo.tomlfeature /CMakeLists.txtoption). If it changes the system topology (components, deploy, domain, RMW), it lives insystem.toml. If it is the physical link + router address a target boots with, it lives in thedeployconfig ([package.metadata.nros.deploy.<t>]for Rust / the package.xml<export><nano_ros deploy=… rmw=…/>tuple for C/C++).
Config home by language × scale
Mirrors RFC-0004 §3:
| Single-node | Workspace | |
|---|---|---|
| Rust | Cargo.toml [package.metadata.nros.{node,entry,deploy.<t>}] (+ nros::main!); optional system.toml to pin rmw/domain | root [workspace.metadata.nros] + node [package.metadata.nros.node] + entry [package.metadata.nros.entry] + bringup system.toml |
| C / C++ | CMakeLists.txt + package.xml (the <nano_ros deploy=… rmw=…/> tuple); optional system.toml | nros_components_register_node / nano_ros_entry per pkg + same system.toml + package.xml |
Where a concern has both a native-idiom projection and a system.toml, the
resolution is a fixed precedence ladder, not a merge: explicit CLI/build
flag (--rmw / -DNANO_ROS_*) > system.toml ([image.<id>] > [system])
the per-package projection (
[package.metadata.nros.*]/ CMake) > built-in default.nros config showprints the resolved effective config with per-value provenance;nros checkflags values still sourced from legacy files.
Full design rationale: RFC-0004 (configuration & transports);
RMW backend selection & lowering is RFC-0031.
For the multi-RMW runtime topic-forwarding bridge (a separate file/feature), see
nros-bridge.toml.
Embedded deploy config (codegen apps — the common case)
An embedded app built through nros::main!() (Rust) or the CMake entry
codegen (C/C++) declares its network + router config per deploy target in
its build manifest. The toolchain bakes it at compile time — there is no
config file on the device and nothing is parsed at runtime.
Rust — [package.metadata.nros.deploy.<target>] in the app’s
Cargo.toml. nros::main!() bakes the block into a DeployOverlay that
BoardEntry::run_with_deploy applies onto the board’s boot Config:
# Cargo.toml (e.g. examples/qemu-arm-baremetal/rust/talker)
[package.metadata.nros.node]
class = "talker_pkg::Talker"
name = "talker"
[package.metadata.nros.deploy.mps2-an385]
board = "mps2-an385" # board crate (optional where unambiguous)
rmw = "zenoh"
locator = "tcp/192.168.1.1:7447"
ip = "192.168.1.10"
gateway = "192.168.1.1"
netmask = "255.255.255.0"
# domain_id = 0
C/C++ — the <export><nano_ros …/> tuple in the package’s
package.xml (RFC-0048 §4; the retired nano_ros_deploy() cmake call’s
successor). Domain/locator ride the build config (config.toml /
-DNROS_ENTRY_LOCATOR), not the tuple:
<!-- package.xml (e.g. examples/qemu-arm-freertos/c/talker) -->
<export>
<build_type>ament_cmake</build_type>
<nano_ros deploy="freertos" board="mps2-an385-freertos" rmw="zenoh"/>
</export>
RT/stack/priority for a single-node embedded app comes from board-crate
Cargo features + Kconfig (prj*.conf on Zephyr), not a config file.
Multi-node RT is declared in system.toml ([tiers.<name>.<rtos>] +
[[node_overrides]]) — see RFC-0015.
Periodic timers after a stall
If a tier is preempted past several periods of a periodic timer, the backlog is
coalesced, not replayed: the timer fires ONCE and the missed periods are
counted. Phase is preserved, so activations stay on the declared cadence grid
rather than drifting by each stall. This is the default (TimerOverrunPolicy::Skip)
and matches rclcpp and Zephyr’s k_timer.
Replay is available — Executor::set_timer_overrun_policy(.., CatchUp) — and is
the right choice for a counter or accumulator, where every tick carries state
that must not be lost. It is the wrong choice for a control loop: the replayed
activations compute with stale inputs and emit a burst of commands inside a
fraction of a period.
Overruns are counted per timer (Executor::timer_overruns) and reported as the
timer-overrun-runtime violation. Watch that rather than a publish-rate check:
under CatchUp a replayed burst makes the achieved rate look nominal — measured
100.03 Hz on a declared 100 Hz loop while the tier was stalling for up to 611 ms
at a time — so a rate monitor reports health during exactly the fault it exists
to catch. Rationale and the measurements: RFC-0002 § 4.4a.
Timer periods are microseconds throughout, including across the C ABI.
Standalone config.toml — hand-written no_std apps (no codegen)
A hand-written single-node app that writes its own main() and bypasses
nros::main!() keeps its net config in a file, not hardcoded in Rust: a
sibling config.toml carrying [node] / [[transport]] / [node.rt],
compile-baked with include_str! and parsed by the board crate’s
Config::from_toml.
# config.toml
[node]
domain_id = 0 # ROS 2 domain ID (0–232)
# namespace = "/"
# rmw = "zenoh" # ACTIVE backend; must match a LINKED backend
# (the build file picks what is linked)
# One transport per session. A single ethernet/wifi/serial entry is the
# common case. In-process topic forwarding is the separate [[bridge]] path.
[[transport]]
kind = "ethernet" # ethernet | wifi | serial | can
ip = "10.0.2.10/24" # CIDR — the prefix rides the address
mac = "02:00:00:00:00:00"
gateway = "10.0.2.2"
locator = "tcp/10.0.2.2:7447"
[node.rt] # scheduling / real-time (RTOS); omit for defaults
app_priority = 12
app_stack_bytes = 65536
zenoh_read_priority = 16
zenoh_lease_priority = 16
poll_priority = 16
poll_interval_ms = 5
Per-kind transport fields:
| kind | fields |
|---|---|
ethernet | ip (CIDR), mac, gateway, interface |
wifi | ssid, password, optional static ip/gateway |
serial / can | device, baudrate |
| all | id, rmw, locator |
ip is CIDR (10.0.2.10/24) — the board derives the prefix or netmask from it.
A serial locator carrying # (serial/UART_0#baudrate=115200) is fine — quote it.
Consumption (compile-baked; no filesystem on the target):
use nros_board_mps2_an385::{Config, run};
fn main() -> ! {
run(Config::from_toml(include_str!("../config.toml")), |config| {
let exec = ExecutorConfig::new(config.zenoh_locator)
.domain_id(config.domain_id);
// ...
})
}
Pick by whether the app uses codegen: nros::main!() app → deploy metadata
(above); hand-written main() → config.toml. Only the old
config.toml schema ([network]/[zenoh]/[scheduling]) is retired.
Retired files
nros.toml— retired in full. A workspace-rootnros.tomlis rejected by the CLI (NrosTomlNotSupported; migrate with thenros-v0.5.0tag’snros migrate workspace— the one-shot verb is retired on newer trees, #186); the legacy per-package overlay is a deprecated fallback thatnros checkflags; the embedded-runtime role never shipped (no example ever declared one). If a doc tells you to writenros.toml, it predates the migration — the content belongs indeploymetadata orsystem.toml.- Old
config.tomlschema ([network]/[zenoh]/[scheduling]) — retired; superseded by thedeployclass (net) + board features / Kconfig (RT). The direct-modeconfig.tomlabove is the kept, supported shape.
Build-time environment variables
Read during cargo build (by build.rs) or by justfile recipes. Set in .env
or export in your shell.
SDK paths
Auto-resolved by nros setup <board>; override if SDKs live elsewhere.
| Variable | Default | Description |
|---|---|---|
FREERTOS_DIR | third-party/freertos/kernel | FreeRTOS kernel source |
FREERTOS_PORT | GCC/ARM_CM3 | FreeRTOS portable layer |
LWIP_DIR | third-party/freertos/lwip | lwIP source |
FREERTOS_CONFIG_DIR | Board crate’s config/ | FreeRTOSConfig.h |
NUTTX_DIR / NUTTX_APPS_DIR | third-party/nuttx/… | NuttX RTOS + apps |
THREADX_DIR / NETX_DIR | third-party/threadx/… | ThreadX + NetX Duo |
THREADX_CONFIG_DIR / NETX_CONFIG_DIR | Board crate’s config/ | tx_user.h / nx_user.h |
Buffer-tuning vars (ZPICO_*, XRCE_*, NROS_*) are optional — see the
Environment Variables Reference.
Binary-size knobs (embedded)
On a constrained MCU, two build-time env vars (set in the example’s
.cargo/config.toml [env], like the other NROS_* tuning) shed the parts a
brokered client doesn’t need:
| Variable | Default | Effect |
|---|---|---|
NROS_LINK_IP | on | NROS_LINK_IP=0 on a serial-only node drops the IP link layer — zenoh-pico’s TCP/UDP link C (Z_FEATURE_LINK_TCP/UDP=0) and (with --gc-sections) the smoltcp platform impl. Serial link stays. |
NROS_SMOLTCP_MAX_SOCKETS / NROS_SMOLTCP_MAX_UDP_SOCKETS | 4 / 2 | Sized for DDS RTPS (3 UDP/participant). A zenoh/XRCE client multiplexes everything over one session → set both to 1 to drop the spare socket buffers (≈8 KB each). |
NROS_HEAP_SIZE | per-platform-crate (64 KB nros-platform-mps2-an385, 32 KB nros-platform-stm32f4) | Decimal bytes for the bare-metal static heap. The defaults are generous; size to the RMW’s working set (table below). E.g. NROS_HEAP_SIZE = "24576" on a zenoh-pico node cut the mps2-an385 .data 66 → 25 KB (−41 KB). |
Static-heap sizing by backend (bare-metal FreeListHeap, set via
NROS_HEAP_SIZE):
| Backend | Peak working set | Recommended heap | Notes |
|---|---|---|---|
| zenoh-pico (TCP) | ~16 KB | 24–32 KB (≈2× for fragmentation) | peer middleware; alloc-based session/buffers |
| zenoh-pico (serial) | lighter than TCP | 16–24 KB | no TCP link buffers; verified running at 16 KB |
| XRCE (Micro-XRCE-DDS) | ~3 KB (micro-ROS figure) | ~8 KB | static pools, discovery offloaded to the agent — the RAM-minimal backend; a measured bare-metal XRCE figure is pending an example (no bare-metal XRCE example ships yet — XRCE bare-metal needs a custom-transport injection) |
Measured footprint
Honest, reproducible numbers per (platform, transport, backend, profile) —
built with the in-tree examples, the release profile is cargo’s default
(opt-3), the size profile is the scaffolded [profile.size] (opt-s + lto
strip, see the size-tuned release profile notes). RAM =data + bss. All cells are after--gc-sections+ the size knobs above are applied where noted; the serial cell ships with the recipe below.
| platform | transport | backend | profile | text (flash code) | data | bss | RAM total |
|---|---|---|---|---|---|---|---|
| qemu-arm-baremetal (mps2-an385, cortex-m3) | ethernet (smoltcp) | zenoh-pico | release | 177.4 KB | 67.0 KB | 91.7 KB | 158.7 KB |
| qemu-arm-baremetal | ethernet | zenoh-pico | size | 158.3 KB | 67.0 KB | 91.7 KB | 158.7 KB |
| qemu-arm-baremetal | serial (no IP stack) | zenoh-pico | release | 128.6 KB | 25.2 KB | 75.8 KB | 101.0 KB |
| qemu-arm-baremetal | serial | zenoh-pico | size + recipe | 116.1 KB | 25.2 KB | 75.8 KB | 101.0 KB |
| stm32f4 (thumbv7em-eabihf, cortex-m4) ¹ | ethernet | zenoh-pico | release | 186.9 KB | 13.7 KB | 123.0 KB | 136.7 KB |
| stm32f4 ¹ | ethernet | zenoh-pico | size | 138.1 KB | 13.7 KB | 123.0 KB | 136.7 KB |
| qemu-arm-freertos (cortex-m3 + lwIP, RTOS-reused stack) | ethernet (lwIP) | zenoh-pico | release | 240.6 KB | 10.7 KB | 3.3 MB | 3.3 MB |
| qemu-arm-baremetal (serial) | serial (custom XRCE transport) | XRCE | size, heap 24 KB, tight XRCE pools | 60.3 KB | 25.2 KB (heap 24 KB) | 8.8 KB | ~34 KB |
| micro-ROS reference (XRCE) | serial | XRCE-DDS Client | -Os | < 75 KB | — | ~3 KB | ~3 KB peak |
¹ The two stm32f4 rows are historical: the board crates left the tree in
a later cleanup, so nothing in-tree reproduces them today. They stay because a
Cortex-M4F-with-its-own-stack data point is still the closest published figure
for that class, and deleting a measurement is worse than dating it. The chip
crate the numbers were built against (nros-platform-stm32f4) is still here.
The XRCE row uses tight per-session XRCE pools — set in the
example’s .cargo/config.toml [env] and read by nros-rmw-xrce-cffi’s
build.rs: NROS_XRCE_STREAM_HISTORY=4,
NROS_XRCE_CUSTOM_TRANSPORT_MTU=512, NROS_XRCE_MAX_SUBSCRIBERS=1,
NROS_XRCE_MAX_SERVICE_SERVERS=1, NROS_XRCE_MAX_SERVICE_CLIENTS=1,
NROS_XRCE_SUBSCRIBER_RING_DEPTH=1, NROS_XRCE_BUFFER_SIZE=256. Vendor
defaults grow xrce_session_state_t to ~390 KB (which wouldn’t fit a
24 KB heap); these knobs drop it to ~12 KB. Defaults are unchanged for
hosted / non-tight-RAM consumers — the env vars are pure opt-in.
How to read this:
- The size profile (opt-
s) shrinks.textby ~10–26 % with.bss/.dataunchanged (opt-level doesn’t touch static buffers — those are the env knobs above).-Ozis not used — on smoltcp examples it grows.bss+24 KB by defeating opt-3’s per-socket dead-buffer DCE (see the size-tuning notes above). - Switching ethernet → serial sheds ~50 KB text + ~42 KB
.data(no smoltcp stack, no IP link C, tuned heap) — the structural lever. - FreeRTOS + lwIP cells
.bssis dominated by lwIP’s heap + FreeRTOS task stacks (3 MB is the configured headroom, not nano-ros overhead). - The micro-ROS / XRCE row is a reference, not a nano-ros measurement — no bare-metal XRCE example ships yet (needs a custom-transport injection); the path to parity is XRCE + serial + static pools.
Size-minimal recipe
Smallest measured nano-ros configuration today (qemu-arm-baremetal serial talker, 116 KB text / 101 KB RAM):
# Cargo.toml
[profile.size]
inherits = "release"
opt-level = "s"
lto = "fat"
codegen-units = 1
debug = false
strip = true
# .cargo/config.toml — gc + serial knobs
[target.thumbv7m-none-eabi]
rustflags = [
"-C", "link-arg=--gc-sections", # 204.8 — strip unreferenced fns/data
"-C", "link-arg=-Tlink.x",
]
[env]
NROS_LINK_IP = "0" # 204.7 — drop zenoh-pico TCP/UDP link C
ZPICO_NO_SMOLTCP = "1" # skip smoltcp glue on bare-metal
# Heap floor: the per-entry executor backing is a single ~75 KB
# allocation, so a `nros::main!` image needs ≥128 KB (the #176 board
# default) — the pre-271 24 KB "zenoh-pico working set" figure OOMs at
# boot. HEAP is `.bss` (no flash cost); shrink below the default only on
# a non-`nros::main!` direct-mode image with a measured smaller peak.
NROS_HEAP_SIZE = "131072"
NROS_SMOLTCP_MAX_SOCKETS = "1" # 204.2 — brokered client multiplexes
NROS_SMOLTCP_MAX_UDP_SOCKETS = "1"
Build with cargo build --profile size, or fleet-wide via
NROS_CARGO_PROFILE=size just <plat> build. nros new --platform baremetal
already scaffolds the [profile.size] + the .cargo/config.toml shape (Phase
204.7/204.8); uncomment the serial block when you swap to a serial transport.
The deeper RAM win waits on XRCE on bare-metal (the ~3 KB-class client +
static pools, with discovery offloaded to the agent) — tracked separately;
zenoh-pico’s SUBSCRIBER_BUFFERS + alloc-based session are what keep this row’s
.bss ~76 KB.
Cargo features (which RMW/platform is linked)
Features select the linked RMW backend, platform, and ROS edition. The
deploy config’s rmw (or system.toml [system].rmw) picks which linked
backend is active — the two are different layers (link vs run). Matrix +
mutual-exclusion rules: Platform Model.
[dependencies]
nros = { path = "…/nros", default-features = false, features = [
"rmw-cffi", # generic C-vtable runtime registry
"platform-bare-metal", # or platform-{freertos,nuttx,threadx,zephyr,posix}
"ros-humble", # or ros-iron
"std", "alloc", # optional, target-dependent
] }
# Exactly one RMW backend crate; its registration runs before main:
nros-rmw-zenoh = { path = "…/nros-rmw-zenoh", features = ["platform-bare-metal", "link-tcp", "ros-humble"] }
# …or nros-rmw-xrce-cffi / the cyclonedds CMake backend
Runtime environment (native host only)
On the native host build, ExecutorConfig::from_env() reads at process start
(embedded targets bake their config at build time instead):
| Variable | Description | Default |
|---|---|---|
ROS_DOMAIN_ID | ROS 2 domain ID | 0 |
NROS_LOCATOR | Router address (legacy alias ZENOH_LOCATOR) | tcp/127.0.0.1:7447 |
NROS_SESSION_MODE | client / peer (legacy alias ZENOH_MODE) | client |
ZENOH_TLS_ROOT_CA_CERTIFICATE* | TLS CA cert (path / base64) | (none) |
By deployment scenario
| Scenario | Config source | Cargo features | Notes |
|---|---|---|---|
| Desktop (POSIX) | env (ExecutorConfig::from_env()) | rmw-cffi, platform-posix, std + zenoh dep | run the router locally (ros2 run rmw_zenoh_cpp rmw_zenohd, ZENOH_CONFIG_OVERRIDE='listen/endpoints=["tcp/127.0.0.1:7447"];scouting/multicast/enabled=false') |
| QEMU bare-metal | [package.metadata.nros.deploy.<t>] ip/mac/gateway/locator | rmw-cffi, platform-bare-metal, ros-humble + zenoh | TAP/slirp bridge |
| FreeRTOS hardware | deploy metadata / the package.xml tuple + board features (RT) | …, platform-freertos, … | FREERTOS_DIR/LWIP_DIR |
| ESP32 WiFi | deploy metadata (ssid/password via build env) | …, platform-bare-metal, … | SSID/PASSWORD build env |
| Zephyr module | Kconfig overlay (prj-<rmw>.conf) | (Kconfig → features) | |
Hand-written no_std (no codegen) | standalone config.toml → Config::from_toml | per-board | net config in a file, not in code |
| Minimal RAM (XRCE serial) | config.toml [[transport]] kind="serial" or deploy | … + xrce dep | XRCE_* buffer tuning |
.env
cp .env.example .env # uncomment + adjust; gitignored; auto-loaded by just + direnv
Logging
nano-ros ships a ROS 2 style leveled logging facade
(nros-log)
with the same Logger + severity surface as rclcpp::Logger /
rcutils_logging. Records flow through a single per-platform sink
(PlatformSink) that delegates to whichever
nros-platform-<rtos> is linked into the binary.
Severity ladder
REP-2012 style, matching rcutils_log_severity_t:
| Severity | u8 | When to use |
|---|---|---|
Trace | 0 | Per-instruction granularity; off by default. |
Debug | 1 | Diagnostic information useful while developing. |
Info | 2 | Normal operation events worth surfacing once. |
Warn | 3 | Unexpected but recoverable conditions. |
Error | 4 | Errors the caller should surface; system continues. |
Fatal | 5 | Unrecoverable; system is about to abort. |
The numeric representation is stable and used by the
nros_platform_log_write ABI.
Quick start
Rust
use nros_log::{Logger, Severity};
use nros_log::{nros_info, nros_warn};
static LOGGER: Logger = Logger::new("my_node");
fn main() {
nros_log::register_logger(&LOGGER);
nros_log::init(nros_log::sinks::default());
nros_info!(&LOGGER, "started; domain = {}", 42);
nros_warn!(&LOGGER, "queue depth {} exceeds soft limit", 5);
}
Inside a node, prefer the Node::logger() accessor — it resolves
to the same intern’d entry when the name matches:
#![allow(unused)]
fn main() {
let mut node = executor.create_node("my_node")?;
nros_info!(node.logger(), "subscribed to {}", topic);
}
C
#include <nros/init.h>
#include <nros/node.h>
#include <nros/log.h>
int main(void) {
nros_support_t support = nros_support_get_zero_initialized();
nros_support_init(&support, "tcp/127.0.0.1:7447", 0);
nros_node_t node = nros_node_get_zero_initialized();
nros_node_init(&node, &support, "my_node", "/");
nros_logger_t logger = nros_node_get_logger(&node);
NROS_LOG_INFO(logger, "started; domain=%u", 42);
NROS_LOG_WARN(logger, "queue depth %u exceeds soft limit", 5);
nros_node_fini(&node);
nros_support_fini(&support);
return 0;
}
C++
#include <nros/nros.hpp>
#include <nros/log.hpp>
int main() {
nros::init("tcp/127.0.0.1:7447", 0);
nros::Node node;
nros::create_node(node, "my_node");
auto logger = node.get_logger();
NROS_LOG_INFO(logger, "started; domain=%u", 42);
NROS_LOG_WARN(logger, "queue depth %u exceeds soft limit", 5);
nros::shutdown();
return 0;
}
The first NROS_LOG_* emit auto-installs PlatformSink so C/C++
call sites work without an explicit init step.
Per-platform delivery
| Target | Backend |
|---|---|
| POSIX | fprintf(stderr, "[<LEVEL>] <name>: <msg>\n") |
| Zephyr | LOG_INF / LOG_WRN / etc. (or printk if CONFIG_LOG=n); module nros |
| ESP-IDF | esp_log_write with logger-name = ESP TAG |
| NuttX | syslog(priority, "%s", buf) |
| FreeRTOS | board-registered UART writer fn-ptr |
| ThreadX | board-registered UART writer fn-ptr |
| Bare-metal (mps2-an385) | QEMU semihosting via cortex_m_semihosting::hio::hstderr |
| Bare-metal (stm32f4) | defmt::info!("{=str}", msg) |
| Bare-metal (esp32 / esp32-qemu) | board-registered fn-ptr (esp-println / RTT / serial-jtag) |
Boards on platforms with no native logger (FreeRTOS / ThreadX /
bare-metal Rust ESP32) supply their writer fn-ptr at run() time:
#![allow(unused)]
fn main() {
// from nros-board-mps2-an385-freertos/src/lib.rs
fn register_log_writer() {
unsafe extern "C" fn writer(severity: u8, name_ptr: *const u8, name_len: usize,
msg_ptr: *const u8, msg_len: usize) { … }
unsafe { nros_platform_register_log_writer(Some(writer), None); }
}
}
Filtering
Compile-time ceiling
Cargo features on nros-log (pick at most one; default
max-level-trace):
| Feature | Macros above ceiling that emit |
|---|---|
max-level-trace | trace / debug / info / warn / error / fatal |
max-level-debug | debug / info / warn / error / fatal |
max-level-info | info / warn / error / fatal |
max-level-warn | warn / error / fatal |
max-level-error | error / fatal |
max-level-off | (none) |
Below-ceiling macros expand to (); the format call is
dead-code-eliminated.
Runtime per-logger threshold
#![allow(unused)]
fn main() {
let logger = nros_log::get_logger("my_node");
logger.set_level(nros_log::Severity::Warn); // silences Trace/Debug/Info
}
Default = Severity::Info for any logger constructed via
Logger::new(name).
Buffer size
nros-log formats each record into a stack-resident
heapless::String<N>. N is picked at compile time by the
buffer-size-<N> feature family (default 256). Overflow truncates
and appends …; the macro never panics on a long format string.
| Feature | Per-call-site stack frame |
|---|---|
buffer-size-128 | 128 B |
buffer-size-256 | 256 B (default) |
buffer-size-512 | 512 B |
buffer-size-1024 | 1024 B |
Interop with the log crate
Enable nros-log/log-compat:
#![allow(unused)]
fn main() {
nros_log::log_compat::install_log_crate_bridge()
.expect("log crate already initialized");
// Now `log::info!(...)` calls from ecosystem crates flow through
// nros-log's dispatcher.
// Or fan out the other direction — re-emit nros records via `log`:
static SINKS: &[&dyn nros_log::LogSink] = &[
&nros_log::sinks::PlatformSink,
&nros_log::log_compat::LogCrateSink,
];
nros_log::init(SINKS);
}
Severity maps Trace↔Trace, Debug↔Debug, Info↔Info, Warn↔Warn,
Error↔Error round-trip. Fatal folds into Error one-way (log has
no Fatal).
Working examples
- Rust:
examples/native/rust/logging/ - C:
examples/native/c/logging/ - C++:
examples/native/cpp/logging/
Each demonstrates per-severity macros + the runtime threshold filter.
Reference
packages/core/nros-log/— facade crate.packages/platform/nros-platform-cffi/include/nros/platform.h—nros_platform_log_write/nros_platform_log_flush/nros_platform_register_log_writerABI.packages/api/nros-c/include/nros/log.h— C API surface (NROS_LOG_*macros +nros_log_emit_fmt).packages/api/nros-cpp/include/nros/log.hpp— C++ macros (the legacyNROS_INFO/ etc. file:line printf surface stays alongside the newNROS_LOG_*macros).
/rosout publication is explicitly out of scope today; the
dispatcher’s &'static [&dyn LogSink] shape leaves room for an
add-later RosoutSink that consumes records alongside
PlatformSink.
Simulated Time
A node in a simulation or replaying a bag should run on the simulator’s clock,
not the wall. nano-ros implements ROS 2’s answer to that: rosgraph_msgs/msg/Clock
on /clock, the use_sim_time parameter, and timers that can be told which
clock advances them.
The distinction the names carry
rclcpp draws it in the method name, and so do we:
| you write | it advances with | affected by /clock |
|---|---|---|
create_wall_timer(period, cb) | the platform’s monotonic clock | no |
create_timer(clock, period, cb) | whatever clock is | yes, if it is a ROS-time clock |
A wall timer is the right choice for a watchdog, a transport keep-alive, or anything whose job is to notice that real time has passed — a paused simulator must not pause those. Everything that consumes simulated data wants the other one.
If you are porting from rclcpp, the spelling you already write is the spelling
that works. If you are reading older nano-ros code, note that create_timer
used to mean the wall timer here; it does not any more, and there is no alias.
Reading the time
A node’s clock is ROS time, as in rclcpp:
nros::Time now = node.get_clock()->now();
With no /clock source installed, a ROS-time clock reads system time. That is
the same fallback rclcpp::Clock has, and it is what makes a node written for
simulation still run standalone.
Two predicates report which situation you are in:
node.get_clock()->ros_time_is_active(); // is a /clock source driving this?
node.get_clock()->started(); // has a first sample arrived?
Turning it on
With the parameter, as in ROS 2
Declare use_sim_time and the runtime attaches the source itself — from a
launch file, a params YAML, or ros2 param set:
talker:
ros__parameters:
use_sim_time: true
use_sim_time is reserved: nothing reads its value, the runtime acts on it.
Set it false and /clock samples stop being installed.
The Rust crate needs the sim-time feature, because the subscription costs an
entity slot and an RX buffer:
nros-node = { version = "0.5", features = ["sim-time"] }
Explicitly
#![allow(unused)]
fn main() {
node.install_ros_time_source()?; // subscribes /clock
node.install_ros_time_source_on("/sim/clock")?; // a remapped clock
}
QoS is ClockQoS — best effort, keep-last 1, volatile. A late subscriber wants
the next sample, not a replay of the simulation’s history.
Writing a timer that follows it
C++
nros::Timer t;
NROS_TRY(node.create_timer(t, *node.get_clock(), 100, on_tick));
Rust
#![allow(unused)]
fn main() {
executor.register_timer_on_clock(
TimerDuration::from_millis(100),
TimerClockSource::Ros,
|| { /* … */ },
)?;
}
TimerClockSource::Steady is the default everywhere and stays free: it consumes
the spin delta the executor has already measured. Ros and System read their
clock on every poll of the timer, which on a target is a platform call — that is
why the choice is a separate entry point rather than a defaulted argument.
What happens at the edges
Paused. /clock stops advancing, so the delta is zero and a ROS-time timer
does not fire. Wall timers on the same executor keep their cadence.
Replayed at 0.5x or 2x. The timer tracks the rate, because it advances by the clock’s own delta rather than by elapsed real time.
A jump backwards — a bag looping, a simulator reset — restarts the period. The alternative would be a timer that stays silent for the length of the jump.
A jump forwards is not special-cased. It lands as a backlog, and
TimerOverrunPolicy decides: Skip (the default) coalesces it into one
activation and counts the rest, CatchUp replays every missed period.
Unsubscribing does not reset the clock. A node that stops listening keeps
the last simulated time rather than snapping back to the wall, which every
ROS-time timer would otherwise absorb as a jump backwards. Clearing is
deliberate: Clock::clear_ros_time_override().
Limits worth knowing before you rely on this
One simulated clock per image. The override is process-global, which is the
model nros_core::Clock has always documented. Two nodes in one image cannot be
on different simulated times.
No jump callbacks. rclcpp::Clock::create_jump_callback and the
rcl_jump_threshold_t machinery have no counterpart. The behaviour a jump
produces is fixed (above); you cannot subscribe to the event.
No wait_until_started / sleep_until. RFC-0021 forbids a blocking helper
that does not drive the executor. Poll started() from your spin loop.
Timers only. A ROS-time clock changes what now() returns and what a
ROS-time timer does. It does not retime message delivery, QoS deadlines, or
transport lease timers — those remain on the monotonic clock, which is what you
want for a link that has to stay up while the simulation is paused.
The age monitor is on a different clock from your stamps. RFC-0052’s
max-age-runtime rule compares a message’s header.stamp against an EPOCH
clock (µs since the UNIX epoch, installed by the board via
Executor::set_epoch_clock). If you stamp with node.get_clock()->now() while
a simulator is driving ROS time, the two disagree and the reported ages are
meaningless — usually enormous. Either stamp from the epoch clock, or leave the
age rule off for that topic while running under simulation.
Build Profiles
nano-ros builds both C/C++ and Rust code on your behalf. You choose the optimization level once, in your own build system, and nano-ros propagates it to every crate and library it builds for you.
There are two knobs, not one, because the two languages have different option vocabularies and collapsing them would mean inventing a lossy middle language:
| You are building with | You set | nano-ros propagates it to |
|---|---|---|
| CMake | CMAKE_BUILD_TYPE | your C/C++ code, and (mapped) the Rust it builds |
| Cargo only | a cargo profile | every nano-ros crate, as a normal path dependency |
From a CMake build type
Set CMAKE_BUILD_TYPE as you would in any CMake project. nano-ros derives the
cargo profile from it:
CMAKE_BUILD_TYPE | cargo profile | what it means |
|---|---|---|
Debug | dev | debuggable, minimal optimization |
RelWithDebInfo | nros-relwithdebinfo | opt-level 2, debug info, no LTO — the default |
MinSizeRel | nros-minsizerel | opt-level "s", fat LTO — smallest images |
Release | release | opt-level 3, fat LTO — fastest code |
| (unset) | nros-relwithdebinfo | the development default |
cmake -B build -DCMAKE_BUILD_TYPE=MinSizeRel
cmake --build build
Your C/C++ sources get -Os from CMake as usual, and the Rust nano-ros builds
for you gets --profile nros-minsizerel. You do not need to add anything to
any Cargo.toml — nano-ros supplies the definition of its own nros-*
profiles.
An unrecognized build type is an error rather than a guess, so a custom type never silently produces an optimization level you did not ask for.
Choosing the Rust profile separately
NROS_CARGO_PROFILE overrides the mapping when you want the two halves to
differ — for example a small C/C++ image with debuggable Rust:
cmake -B build -DCMAKE_BUILD_TYPE=MinSizeRel -DNROS_CARGO_PROFILE=nros-relwithdebinfo
Using your own profile
Name any profile you like. When the name is not one of nano-ros’s nros-*
profiles, you own the definition — nano-ros passes the name through and
injects nothing, so your settings are authoritative:
# your-workspace/Cargo.toml
[profile.prod]
inherits = "release"
opt-level = 3
lto = "fat"
cmake -B build -DCMAKE_BUILD_TYPE=Release -DNROS_CARGO_PROFILE=prod
If the profile is not defined, cargo reports
error: profile 'prod' is not defined and points at your manifest — which is
the right file to fix.
A pure-Cargo workspace
Nothing to configure. nano-ros crates are ordinary path dependencies, so your workspace-root profile already governs them:
cargo build --profile prod
Which profile is active?
nros profile resolve --build-type MinSizeRel # -> nros-minsizerel
nros profile dir nros-minsizerel # -> the target/ subdirectory
A CMake configure also prints it:
-- nano-ros: cargo profile `nros-minsizerel` (CMAKE_BUILD_TYPE=MinSizeRel) → target/nros-minsizerel
Notes for embedded targets
- Images grow at the default.
nros-relwithdebinfotrades size for build speed. For flashing to a constrained board, buildMinSizeRel. - Two platforms pin their own profile regardless of what you choose, and
will say so: NuttX Rust images (a codegen miscompile at
lto = "off") and FreeRTOS QEMU images (the emulated Cortex-M3 misses zenoh-pico’s handshake window at low optimization).
Profiling Your Build
When a build feels slow, nros-build-profile tells you where the time went —
codegen vs compile vs link vs flash, which unit dominated, and whether a shared
crate was rebuilt redundantly.
It is a passive, read-only tool. Your build runs exactly as today on its native
toolchain (west, cmake, idf.py, cargo); the profiler only reads the timing
artifacts that build already produced. It never compiles or flashes anything, and it
adds no nros build/test command — nros stays setup + codegen.
How it works
Every backend already emits timing data:
| Backend | Artifact | Opt-in |
|---|---|---|
west, cmake, idf.py | build*/.ninja_log | none — ninja always writes it |
cargo (native, esp32, cross) | target*/cargo-timings/ | cargo build --timings |
The profiler discovers and parses these, then prints a stage table (always) plus an optional per-unit drill-down and diagnostic hints.
Flow 1 — Zephyr / cmake / esp32-idf (no opt-in)
$ west build -b qemu_cortex_a53 examples/zephyr/rust/talker # build/.ninja_log written
$ cargo build -p nros-build-profile # once; builds the profiler bin
$ ./target/debug/nros-build-profile examples/zephyr/rust/talker
Backend: ninja (west) Total: 41.2s
Stage Duration %
codegen 1.1s 3%
compile 33.8s 82%
link 6.0s 15%
hints:
- 1 unit = 61% of compile (libzenoh_pico.a, 20.6s)
Flow 2 — cargo (one flag for per-crate detail)
The coarse table works with any build; the per-crate drill-down needs --timings:
$ cd examples/native/rust/talker && cargo build --timings # target/cargo-timings/ written
$ ./target/debug/nros-build-profile examples/native/rust/talker --deep
Backend: cargo Total: 30.3s
Stage Duration %
codegen 1.2s 4%
compile 24.8s 82%
link 3.9s 13%
slowest units:
zenoh-pico-sys 18.1s ########
nros-node 2.4s #
std_msgs 1.1s #
<80 more> 3.2s
hints:
- nros-c compiled 3× — examples use isolated target/; pool target_dir to reuse the build
If you build cargo without --timings, you still get the coarse table plus a note
telling you how to enable the drill-down — never an error.
Flow 3 — machine-readable (CI regression diffing)
$ ./target/debug/nros-build-profile examples/zephyr/rust/talker --json
wrote examples/zephyr/rust/talker/nros-build-profile.json
A CI step can diff nros-build-profile.json across commits to catch build-time
regressions (a stage’s seconds or a unit’s share jumping).
External (copy-out) projects
For copy-out example projects, run the installed bin the same way:
$ nros-build-profile <project-dir> --deep
Diagnostics
On top of the numbers, the profiler flags known slow patterns:
- a large native (C/C++/
-sys) unit with no incremental → suggest a compiler cache; - the same crate compiled more than once → suggest pooling
target_dir; - the configured job count vs available RAM (the fixture-build OOM budget).
Silence them with --no-hints.
Deployment Workflow
Deployment means different things per target, but the order is stable: prepare toolchain, build package, move binary/firmware to target, then verify ROS 2 communication.
Native (host)
Three equivalent entry points; pick by workspace shape:
# Per-example (Pattern B or any single binary):
cd examples/native/rust/talker
cargo run
# Multi-component system orchestration:
nros metadata my_system
nros plan my_system launch/my_system.launch.py
nros check
cargo run -p robot_entry
# Colcon consumer workspace (Pattern A):
colcon build && source install/setup.bash
ros2 run my_pkg my_node
For interop with stock ROS 2 over Zenoh, run the router ROS ships and point ROS 2 at it:
ros2 run rmw_zenoh_cpp rmw_zenohd
export RMW_IMPLEMENTATION=rmw_zenoh_cpp
nano-ros no longer bundles a router (phase-362 / RFC-0075). rmw_zenohd links
the same libzenohc.so that rmw_zenoh_cpp does, so it cannot drift from the
RMW you are talking to — and it is what a ROS 2 deployment actually runs.
It takes no command-line configuration. To move it off the default tcp/[::]:7447:
ZENOH_CONFIG_OVERRIDE='listen/endpoints=["tcp/127.0.0.1:7447"]' \
ros2 run rmw_zenoh_cpp rmw_zenohd
See Native host build.
RTOS and Bare-Metal
RTOS targets usually produce firmware images or simulator binaries, built with the platform’s own tool from the example / package dir:
cargo build --release # Rust leaves (after `nros sync`)
cmake -B build && cmake --build build # C / C++ leaves
Contributors: the in-tree fixture build/test lanes are in Per-Platform Contributor Lanes.
For real hardware, deployment step becomes flash/load/monitor. For QEMU, deployment is launching simulator with correct network setup.
There is no nros deploy / nros build / nros run verb — Phase 222
removed those wrappers. nros is provisioner + codegen + metadata only;
deployment runs on the vendor’s native tools. The embedded deploy
contract is a documented three-step sequence (per
RFC-0003 §4):
- Bake —
nros codegen-system --bringup <pkg>readssystem.toml+[image.<id>]+launch/*.xmland emits the baked tree underbuild/<board>/. - Build — the vendor tool builds it:
cargo build/cmake --build/west build/idf.py build(contributors: the in-treejust <plat> build*recipes wrap these with the right-Dargs derived from[image.<id>]). - Flash + monitor — the vendor tool again:
probe-rs run/west flash/idf.py flash monitor, or the platform’s QEMU runner.
Platform guides should show:
- package layout,
- setup command,
- toolchain requirements,
- build command,
- run/flash command,
- ROS 2 interop or smoke-test command.
Zephyr
Zephyr deployment uses west:
nros setup zephyr --rmw zenoh
# **Contributors (in-tree checkout):** `just setup zephyr` creates
# zephyr-workspace/ (west init + SDK) — then:
source zephyr-workspace/env.sh # in-tree workspace layout
west build -b native_sim/native/64 nros/examples/zephyr/rust/talker
./build/zephyr/zephyr.exe
Bringing your own west workspace instead? Follow
Zephyr Integration — the
zephyr-workspace/env.sh line above is specific to the in-tree
workspace layout.
ESP32
ESP32 deployment uses the Espressif toolchain and flash tool.
Contributors: the in-tree ESP32 build/run lanes are in Per-Platform Contributor Lanes.
For physical boards: espflash flash --monitor <elf> (the QEMU
chapter’s espflash save-image packs an emulator image instead —
real-hardware bring-up is not yet a documented end-to-end flow).
Verify
After deployment, verify from ROS 2 side:
ros2 topic list
ros2 topic echo /chatter std_msgs/msg/String --qos-reliability best_effort
If discovery works but samples do not arrive, check domain ID, router mode, QoS reliability, and platform network setup.
Cross-backend bridges
Most nano-ros applications pick one RMW backend at compile time and never look back. A cross-backend bridge is the one case where a single binary needs two backends in parallel — one node receiving on backend A, another publishing on backend B, the executor draining both each tick.
Typical reasons:
- Drone link. A flight controller speaks XRCE-DDS over a serial radio link; the ground-station LAN runs standard DDS. A bridge forwards telemetry verbatim between the two.
- Field gateway. Sensor pods talk zenoh-pico over an unreliable Wi-Fi mesh; the ROS 2 datacenter side wants Cyclone DDS. A bridge republishes after a session-buffered drop check.
- Safety carve-out. Mission-critical traffic stays on the deterministic backend; introspection / logging republishes to a best-effort backend so dashboards never starve the critical pipeline.
This chapter walks the model, the build knobs, and the three shipped examples.
The mental model — rclcpp::Node, twice
nano-ros mirrors rclcpp:
Executor exec; // owns the primary session
Node ingress = exec.node_builder("ingress").rmw("xrce").build();
Node egress = exec.node_builder("egress").rmw("cyclonedds").build();
auto sub = ingress.create_subscription<Foo>(...);
auto pub = egress.create_publisher<Foo>(...);
The executor opens one primary session at open* time. Each
extra node_builder(...).rmw(name) call:
- Looks up
namein the registry of linked backends. - If
name == primary_rmwand the locator matches → the Node binds to the primary session (slot 0). No second session opened. - Else → opens a fresh session through that backend’s
open_with_rmwand stores it inextra_sessions[N-1]. The new Node’ssession_idx = N.
spin_once() drains every session in turn. Handles created
through a multi-Session Node route through
the node’s resolved session record instead of the legacy support/session
pointer kept for single-backend callers.
NodeRecord.session_idx is the dispatch key. Print it to
verify which session each Node landed on:
#![allow(unused)]
fn main() {
exec.node(node_in).unwrap().session_idx // 0 = primary
exec.node(node_out).unwrap().session_idx // 1 = first extra
}
Build knobs — three audiences, three shapes
Rust binary
Add both backend crates to Cargo.toml and call each
register() early in main:
[dependencies]
nros = { ..., features = ["rmw-cffi"] }
nros-rmw-zenoh = { ... }
nros-rmw-xrce-cffi = { ... }
fn main() {
nros_rmw_zenoh::register().expect("register zenoh");
nros_rmw_xrce_cffi::register().expect("register xrce");
let mut exec = nros::Executor::open_with_rmw("zenoh",
&nros::ExecutorConfig::from_env())?;
let ingress = exec.node_builder("ingress").rmw("zenoh").build()?;
let egress = exec.node_builder("egress").rmw("xrce").build()?;
// ...
}
Why explicit register()? Stable Rust drops un-referenced
rlib CGUs from the final link even when the backend’s
#[used] static RMW_INIT_ENTRIES exists. The register()
call doubles as the symbol reference that drags the rlib in
and the registration trigger. C / C++ builds dodge this
because --whole-archive pulls every section unconditionally.
C binary
NANO_ROS_RMW=none switches off the root CMake’s
RMW auto-pull. Pull each backend’s staticlib through
corrosion_import_crate and wrap with --whole-archive so
the registry walker finds both names:
set(NANO_ROS_RMW none)
add_subdirectory(${nano_ros_root} nano_ros)
corrosion_import_crate(
MANIFEST_PATH ${nano_ros_root}/Cargo.toml
CRATES nros-rmw-xrce-cffi
NO_DEFAULT_FEATURES
FEATURES std)
corrosion_import_crate(
MANIFEST_PATH ${nano_ros_root}/Cargo.toml
CRATES nros-rmw-zenoh-staticlib
NO_DEFAULT_FEATURES
FEATURES "platform-posix;ros-humble")
target_link_libraries(my_bridge PRIVATE
NanoRos::NanoRos
-Wl,--whole-archive
nros_rmw_xrce_cffi-static
nros_rmw_zenoh_staticlib-static
-Wl,--no-whole-archive)
Then declare + call the register functions early in
nros_app_main:
extern int8_t nros_rmw_xrce_register(void);
extern int8_t nros_rmw_zenoh_register(void);
int nros_app_main(int argc, char** argv) {
if (nros_rmw_xrce_register() != 0) return 1;
if (nros_rmw_zenoh_register() != 0) return 1;
// ... nros_support_init / nros_executor_init / nros_executor_node_init
}
nros_executor_node_init honours per-Node rmw_name +
locator via the nros_node_options_t struct; the executor
sets node.executor so subsequent nros_*_init calls
(publisher, subscription, service, action) route to the right
session.
C++ binary
Same CMake shape as C. The C++ surface (nros-cpp) is a
thin header layer; the linker work is identical. Use the
Executor::node_builder(name) chain:
auto exec = nros::Executor::open_with_rmw("zenoh", cfg);
auto ingress = exec.node_builder("ingress").rmw("zenoh").build();
auto egress = exec.node_builder("egress").rmw("xrce").build();
auto pub = egress.create_publisher<std_msgs::String>("/chatter");
exec.register_subscription_on<std_msgs::String>(ingress, "/chatter",
[&pub](const auto& msg) { pub->publish(msg); });
NROS_RMW environment variable
When a binary links multiple backends, set NROS_RMW to
pin the primary before open():
NROS_RMW=zenoh ./my_bridge
The C-side nros_support_init reads it and routes
Executor::open through open_with_rmw(name, ...). Without
this, the linkme walker returns whichever backend’s ctor
fired first, which is non-deterministic across link
orderings. The bridge then opens another session against
the same backend when the per-Node .rmw(name) matches the
unintended primary — and most singleton-state backends
(XRCE-DDS-Client’s uxrSession, zenoh-pico’s global
g_session) refuse a second open.
The Rust path mirrors this: Executor::open consults
$NROS_RMW first; the per-Node resolve_session_slot
detects the primary-name match and returns slot 0 instead
of opening a duplicate.
Memory + WCET budget
Each extra session adds:
- One
ConcreteSession(RMW-specific; see RMW Backends for sizing). - One
register_wake_signal_on_extrawake-callback slot (stdonly; bare-metal targets share the primary wake). - One round-trip through
spin_onceper backend per tick.
The bridge’s per-tick WCET is the sum across linked backends:
bridge_wcet = Σ poll_wcet_i + Σ dispatch_wcet_j
Read the per-backend numbers in the Real-time budget per backend table and add them up — there is no parallelism between backends inside a single executor.
For deadline-critical bridges, partition by SchedContext instead of running everything on the default Fifo slot:
#![allow(unused)]
fn main() {
let ingress = exec.node_builder("ingress")
.rmw("xrce")
.sched(critical_sc) // RT priority
.build()?;
let egress = exec.node_builder("egress")
.rmw("cyclonedds")
.sched(best_effort_sc)
.build()?;
}
The PiCAS-style per-callback OS-priority dispatcher (gated
behind the scheduler-os-priority feature) routes each Node’s
callbacks to its own OS priority slot so the slow backend cannot
block the fast one.
Shipped examples
examples/bridges/tt-zenoh-to-xrce/
Pure-Rust bridge. Zenoh ingress → XRCE-DDS egress under an
ARINC-653-style time-triggered cyclic schedule. Read this
first — it shows the multi-RMW
Executor::open_with_rmw("zenoh", ...) plus
node_builder.rmw("xrce") per-session pin, with raw byte
forwarding and no codegen.
ZENOH_CONFIG_OVERRIDE='listen/endpoints=["tcp/127.0.0.1:7447"];scouting/multicast/enabled=false' ros2 run rmw_zenoh_cpp rmw_zenohd &
build/xrce-agent/MicroXRCEAgent udp4 -p 8888 &
NROS_XRCE_LOCATOR=udp/127.0.0.1:8888 \
cargo run -p native-rs-bridge-tt-zenoh-to-xrce
examples/bridges/tt-zenoh-to-cyclonedds/
The stock-Cyclone-DDS sibling: same TT schedule, but the egress is
.rmw("cyclonedds"), forwarding onto the DDS databus where a stock
rmw_cyclonedds_cpp (e.g. an Autoware listener) or another nano-ros cyclonedds
node receives the samples. One structural difference from the XRCE variant:
Cyclone rejects a raw publisher whose topic type has no registered
dds_topic_descriptor_t, so the egress type’s schema is staged before the
raw publisher is created —
#![allow(unused)]
fn main() {
// The Cyclone backend installs the registrar during its register():
nros_rmw_cyclonedds_sys::register()?;
// Stage std_msgs/msg/String's schema (NUL-terminated key — it is handed
// straight to Cyclone's C descriptor table):
nros_rmw::register_type_descriptor(
"std_msgs/msg/String\0",
&[nros_serdes::schema::Field { name: "data\0", ty: FieldType::String, offset: 0 }],
)?;
let pub_out = node_out.create_publisher_raw("/chatter", "std_msgs/msg/String", hash)?;
}
XRCE registers lazily from name+hash; Cyclone needs the descriptor up front. The
backend links the vendored CycloneDDS (no -DNANO_ROS_RMW needed for the Rust
binary — the nros-rmw-cyclonedds-sys dep is the selection).
ZENOH_CONFIG_OVERRIDE='listen/endpoints=["tcp/127.0.0.1:7447"];scouting/multicast/enabled=false' ros2 run rmw_zenoh_cpp rmw_zenohd &
ROS_DOMAIN_ID=0 cargo run -p native-rs-bridge-tt-zenoh-to-cyclonedds
# subscribe on Cyclone DDS /chatter (stock ROS 2 or a nano-ros cyclone node on
# the same ROS_DOMAIN_ID) and publish on zenoh /chatter to see bridged samples.
Coverage matrix
Bridge examples live under
examples/bridges/<name>/ (cross-platform, transport-
spanning) or under their canonical
examples/<plat>/<lang>/bridge/<name>/ cell when the bridge
is platform-specific. The
examples README coverage
matrix
lists which <plat> × <lang> combinations ship a bridge today
(a bridge spans RMW backends by nature, so RMW is not a directory axis).
Troubleshooting
| Symptom | Likely cause |
|---|---|
Transport(ConnectionFailed) on open_with_rmw("X", ...) | Backend X’s rlib not pulled into the link line. Rust: add a register() call. C / C++: confirm --whole-archive wraps the staticlib. |
Second node’s .rmw("zenoh") returns Transport(...) | Both nodes try to open against the same singleton-state backend. Set NROS_RMW=zenoh so the primary lands on zenoh + the second Node hits the session-cache (slot 0) instead of double-opening. |
nros_publisher_init -> -7 after nros_executor_node_init | Stale build — the C-side multi-Session dispatch predates nros-v0.5.0; rebuild nros-c after updating. |
Bridge spinning marker never reaches piped test harness | Add setvbuf(stdout, NULL, _IOLBF, 0) at the top of nros_app_main. glibc full-buffers piped stdout; line-buffering surfaces readiness markers before the long-lived spin_period loop. |
See also
- Choosing an RMW Backend — single- backend selection criteria.
- RMW Backends — Host-Language Policy — registry + per-backend sizing.
docs/roadmap/archived/phase-104-multi-backend-bridges.md— design rationale + acceptance criteria.docs/roadmap/archived/phase-156-bridge-runtime-blockers.md— the four sub-bugs that gated the D.3 / D.4 E2E tests + how each was resolved.
QoS, Status Events, and Discovery
This page covers runtime event behavior that application authors need after choosing an RMW backend: liveliness, deadline misses, lost messages, and the places where nano-ros deliberately differs from standard ROS 2 waitset-style event handling.
Transport-level status events let an application observe conditions the underlying middleware detects: a remote node going silent, a publisher missing its rate, a subscriber dropping samples. nano-ros exposes a Tier-1 subset useful for embedded RTOS deployments — liveliness changes, deadline misses, and message loss.
This page describes the event surface, the dispatch model, and the patterns that motivate having events at all on RTOS targets.
The Tier-1 events
Five event kinds. Three on subscribers, two on publishers.
Subscriber events
| Kind | Fires when | Use |
|---|---|---|
LivelinessChanged | A tracked publisher’s liveliness state changes (publisher started asserting / stopped) | Safety-island fail-over: trigger MRM when remote control node goes silent |
RequestedDeadlineMissed | An expected sample didn’t arrive within the configured deadline | Periodic-sensor pattern: 100 Hz topic with 15 ms deadline; fire alarm when late |
MessageLost | The backend dropped one or more samples (typically: ring-buffer overflow, slow consumer) | Diagnostic + adaptive: log, drop more aggressively, coalesce |
Publisher events
| Kind | Fires when | Use |
|---|---|---|
LivelinessLost | This publisher missed its own liveliness assertion deadline | Self-monitoring: detect own task starvation |
OfferedDeadlineMissed | This publisher promised X Hz, fell behind | Self-monitoring: detect overrunning compute |
Registering callbacks
Same shape as message callbacks. Register at construction time;
callback fires from inside spin_once when the backend detects the
event.
Rust
#![allow(unused)]
fn main() {
use core::time::Duration;
let mut sub = node.create_subscription::<SensorReading>("/sensor")?;
// Liveliness — fires when the publisher comes / goes
sub.on_liveliness_changed(|status| {
if status.alive_count == 0 {
log::warn!("sensor publisher went silent");
trigger_failover();
}
})?;
// Deadline — fires when no sample within 15 ms
sub.on_requested_deadline_missed(Duration::from_millis(15), |status| {
metric_inc(&LATE_SAMPLE_COUNT, status.total_count_change);
})?;
// Message lost — fires when backend drops a sample
sub.on_message_lost(|status| {
log::warn!("dropped {} samples", status.total_count_change);
})?;
}
Async equivalents (spin_async / Embassy / tokio):
#![allow(unused)]
fn main() {
let status = sub.next_liveliness_change().await?;
let status = sub.next_deadline_miss().await?;
}
C
static void on_liveliness_changed(
nros_subscription_t *sub,
nros_liveliness_changed_status_t status,
void *user_context) {
if (status.alive_count == 0) {
trigger_failover();
}
}
nros_subscription_set_liveliness_changed_callback(
sub, on_liveliness_changed, NULL);
Same shape on the four other event kinds. Each returns
NROS_RMW_RET_UNSUPPORTED if the active backend doesn’t generate
that event.
C++
sub.on_liveliness_changed([&](nros::LivelinessChangedStatus status) {
if (status.alive_count == 0) trigger_failover();
});
sub.on_requested_deadline_missed(
std::chrono::milliseconds(15),
[&](nros::DeadlineMissedStatus status) {
late_count_ += status.total_count_change;
});
std::function overloads available with NROS_CPP_STD; freestanding
mode uses C function pointers + user-context.
How dispatch works
Events fire from inside drive_io on the executor thread, the same
way message callbacks do:
spin_once(timeout)
└─ session.drive_io(...)
├─ backend RX worker detects message → fires message callback
├─ backend RX worker detects liveliness change → fires liveliness callback
├─ backend timer expires (deadline) → fires deadline callback
└─ return
Same execution context, same priority, same constraints as message
callbacks. The max_callbacks_per_spin knob (see
RTOS Cooperation) covers events too — an
event callback counts as one against the cap.
This means events get the same scheduling treatment as messages.
max_callbacks_per_spin = 1 will fire either one message OR one
event per spin_once, whichever the backend has ready first.
Why a callback model, not a waitset
Upstream rmw.h exposes events via a waitset — register an
rmw_event_t handle in a waitset, call rmw_wait, then rmw_take_event
to pull payload. nano-ros doesn’t have a waitset (see
RMW vs upstream Section 4).
Reusing the existing message-callback path avoids introducing the waitset. Trade-off: applications can’t bulk-poll for events. For the Tier-1 set this isn’t load-bearing — events are rare, callbacks are cheap.
Backend support is uneven
Not every backend can generate every event. Apps must handle
Unsupported errors when registering:
| Backend | LivelinessChanged / Lost | DeadlineMissed | MessageLost |
|---|---|---|---|
| Cyclone DDS | ✗ Not wired through the nano-ros event API yet | ✗ Not wired yet | ✗ Not wired yet |
| XRCE-DDS | ✗ Not exposed (xrce-dds-client API limitation) | ✅ Shim-side clock check (sub: has_data / take_serialized; pub: publish_raw) | ✗ Not exposed |
| zenoh-pico | ✅ Poll-based via zenoh tokens (alive_count ∈ {0,1}) | ✅ Shim-side clock check (<P as PlatformClock>::clock_ms) | ✅ Seq-gap detection from RMW attachment |
| uORB | ✗ No wire-level liveliness | ✗ No rate concept | ✅ Native (host mock + real PX4 via RustSubscriptionCallback publish-counter) |
✓ = wired and tested. 🟡 = surface API works (returns Err while pending), wiring planned. ✗ = not feasible at this layer.
No current backend exposes the full Tier-1 event set through the
nano-ros event API. Apps should call Subscriber::supports_event(kind)
first or design for graceful fallback.
The Subscriber::supports_event(kind) query lets applications check
support before registering:
#![allow(unused)]
fn main() {
if sub.supports_event(EventKind::RequestedDeadlineMissed) {
sub.on_requested_deadline_missed(...)?;
} else {
// fallback: app-side timeout monitoring
}
}
Apps that need cross-backend portability code defensively. Apps pinned to one backend can call register-and-unwrap.
What’s deliberately skipped
Three upstream event types are intentionally absent from the API:
*_MATCHED (publication / subscription matched)
Fires when a remote endpoint appears or disappears. Useful for discovery-tracking dashboards and dynamic-topology apps. Static- topology embedded apps don’t benefit; if a use case appears, the event kind is additive.
*_QOS_INCOMPATIBLE
Fires when publisher and subscriber QoS profiles can’t be
reconciled. nano-ros surfaces this synchronously at create_publisher
/ create_subscription time as NROS_RMW_RET_INCOMPATIBLE_QOS. No
runtime event needed; the failure is visible at startup.
*_INCOMPATIBLE_TYPE
Type-hash mismatch. Same: surfaced synchronously as
NROS_RMW_RET_TOPIC_NAME_INVALID (or a future
NROS_RMW_RET_INCOMPATIBLE_TYPE) at creation. No runtime event.
Patterns
Drone-bridge fail-over on liveliness loss
#![allow(unused)]
fn main() {
let mut sub = node.create_subscription::<VehicleAttitude>("/vehicle_attitude")?;
sub.on_liveliness_changed(|status| {
if status.alive_count == 0 {
// PX4 commander.cpp went silent — trigger MRM
request_minimum_risk_manoeuvre();
} else if status.alive_count_change > 0 {
// commander came back online; clear MRM if appropriate
clear_failover();
}
})?;
}
This pairs with the cross-backend bridge example pattern.
100 Hz sensor with deadline alarm
#![allow(unused)]
fn main() {
let mut sub = node.create_subscription::<SensorReading>("/imu")?;
sub.on_requested_deadline_missed(
Duration::from_millis(15), // expected 100 Hz, allow 15 ms deadline
|status| {
if status.total_count_change > 0 {
log::error!("IMU late: {} missed deadlines", status.total_count_change);
// Optional: enter degraded-mode controller
}
},
)?;
}
Slow-consumer logging
#![allow(unused)]
fn main() {
let mut sub = node.create_subscription::<Pointcloud>("/lidar")?;
sub.on_message_lost(|status| {
log::warn!("dropped {} pointcloud frames (total: {})",
status.total_count_change, status.total_count);
// Tighten downstream filter, reduce work, coalesce, etc.
})?;
}
See also
- RMW API: Differences from upstream
rmw.hSection 8 — the design comparison this page expands on. - RTOS Cooperation —
max_callbacks_per_spintreatment of event callbacks. - Differences from ROS 2 — broader surface comparison.
TX Throughput & Latency Tuning
(zenoh-pico RMW — Zephyr, FreeRTOS, NuttX, ThreadX, POSIX; phases 279/282, issue #145)
Every nano-ros language front-end (Rust, C, C++) publishes through the same
shared transport path (the zpico shim → zenoh-pico), so the knobs on this page
apply uniformly to all of them. They matter most on Zephyr, where the
kernel’s socket layer serializes send/recv per file descriptor: the zenoh-pico
read task holds the session socket for a full receive window
(CONFIG_NROS_ZENOH_SOCKET_TIMEOUT_MS, default 100 ms), so without tuning,
total image tx throughput is capped at roughly one socket send per window —
about 9 msg/s at the default, regardless of how many publishers the image
runs or how fast they publish.
Platform defaults (RFC-0049)
Per-platform defaults for these knobs live in each platform
package’s nros-platform.toml ([knobs.zenoh.tx]), resolved through the
fixed ladder builtin < platform < board < env/Kconfig — an explicit
build-time setting (including an explicit 0 / n) always wins.
| platform | batch | split_lock | flush_ms | rationale |
|---|---|---|---|---|
| Zephyr | on | on | 50 | the per-fd tx ceiling above; measured 15–20× streaming. Requires the issue-0213 fork fix (declarations bypass the batch). zephyr/Kconfig’s NROS_ZENOH_TX_* defaults mirror this (drift-tested) — flip them per-app in prj.conf to opt out |
| everything else | off | off | 50 | no per-fd ceiling (POSIX), no flush thread (ThreadX/bare-metal), or simply unmeasured (FreeRTOS/NuttX — flip is one line in their platform toml after a bench run) |
With batching on, non-express topics pay up to flush_ms of publish
latency — declare low-rate latency-critical publishers tx_express
(below) to bypass the batch. Timer-paced low-rate systems that want the
old behavior image-wide: set CONFIG_NROS_ZENOH_TX_BATCH=n (Zephyr) /
ZPICO_TX_BATCH=0 (cargo/cmake lanes).
The decision tree
Is total publish rate comfortably under ~1 msg per socket window?
├── yes → no tuning needed (the defaults are fine)
└── no
├── Lots of small/medium messages, latency budget ≥ one flush period?
│ └── TX batching: CONFIG_NROS_ZENOH_TX_BATCH=y (or ZPICO_TX_BATCH=1)
│ └── still tx-bound under a tight-loop / bursty publisher?
│ └── add CONFIG_NROS_ZENOH_TX_SPLIT_LOCK=y (ZPICO_TX_SPLIT_LOCK=1)
├── A specific topic needs per-sample latency < the flush period?
│ └── declare THAT publisher express (tx_express QoS) — it bypasses
│ the batch; keep the rest batched
└── Everything is latency-critical at high rate?
└── lower CONFIG_NROS_ZENOH_SOCKET_TIMEOUT_MS (e.g. 5 ms) — more
send windows per second at the cost of read-task wakeups; batching
still composes on top
Measured effect (Zephyr native_sim, 64 B messages, 100 ms socket timeout)
Provenance. Single-run measurements on one development host, on
native_sim— a host-scheduled simulator, so the absolute msg/s figures move with host load; the ratios between rows are the durable result. Re-measure on freshly rebuilt fixtures on a quiet machine before citing any absolute number.
Streaming (tight-loop publisher, 5000 messages, deep-ring listener):
| configuration | talker completes? | throughput | vs baseline |
|---|---|---|---|
| defaults | no (~33 s window) | ~9 msg/s | 1× |
TX_BATCH | no | ~136 msg/s | ~15× |
TX_BATCH + TX_SPLIT_LOCK | yes (27.7 s) | ~181 msg/s | ~20× |
Timer-paced tiers (100 Hz control + 10 Hz telemetry, one session, harness): defaults 8.6 msg/s total → batch + flush thread 34.1 → + split lock 43.2, with the 10 Hz tier at its ideal rate. Delivery integrity is validated in every configuration.
Express-vs-batched (same batching image, 200 messages at 5 ms pacing, native): the batched topic arrives in flush-cadence bursts (21 inter-arrival gaps of ~50 ms), the express topic arrives continuously (zero gaps > 25 ms, max gap 5 ms). On Zephyr an express publisher sends each sample immediately and is therefore paced by the socket window instead — use express for low-rate, latency-sensitive topics (control setpoints, e-stop), never for streams.
The knobs
| knob | Zephyr (Kconfig) | cargo lanes (env at build) | C/C++ cmake lanes | default |
|---|---|---|---|---|
| socket recv window | CONFIG_NROS_ZENOH_SOCKET_TIMEOUT_MS | — (posix doesn’t serialize per-fd) | — | 100 ms |
| tx batching | CONFIG_NROS_ZENOH_TX_BATCH=y | ZPICO_TX_BATCH=1 | -DZPICO_TX_BATCH=1 via NROS_CMAKE_EXTRA_DEFS | off |
| flush cadence | CONFIG_NROS_ZENOH_TX_BATCH_FLUSH_MS | ZPICO_TX_BATCH_FLUSH_MS | -DZPICO_TX_BATCH_FLUSH_MS=<ms> | 50 ms |
| split tx locking | CONFIG_NROS_ZENOH_TX_SPLIT_LOCK=y | ZPICO_TX_SPLIT_LOCK=1 | -DZ_FEATURE_TX_SPLIT_LOCK=1 (shared config) | off |
| per-publisher express | tx_express in the QoS profile (all languages, below) | same | same | off |
All of these are compile-time knobs (embedded images have no runtime config), and all default off — an untuned build is byte-identical to pre-279 behavior.
What each one does
TX batching (TX_BATCH): publishes append to the transport write buffer
and ship as one socket send per flush instead of one send per put. Throughput
then scales with messages-per-flush. A dedicated flush thread (spawned
automatically on multi-threaded platforms; ThreadX stays spin-driven because
it runs no background tasks) flushes every TX_BATCH_FLUSH_MS. Batching adds
up to one flush period of publish latency; service requests/replies always
bypass the batch. Do not enable for timer-paced low-rate systems — with
≤1 message per flush period there is nothing to coalesce and the flush
overhead measurably hurts (a measured negative result).
Flush cadence (TX_BATCH_FLUSH_MS): bounds the extra latency batching
adds. Lower = fresher samples, more (smaller) sends. 50 ms is a good default
for telemetry; drop toward 10–20 ms if your latency budget demands it.
Split tx locking (TX_SPLIT_LOCK, requires TX_BATCH): the flush
steals the pending batch (buffer swap) and performs the socket write under a
separate link mutex, so publishers keep appending while the send is in
flight; a batch-overflow parks the full buffer instead of blocking the
publishing thread. Wire order still equals SN order. This is what lets a
tight-loop publisher actually saturate the path (~181 msg/s above). Fork-local
Z_FEATURE_TX_SPLIT_LOCK; it gates transport-struct fields, so it must be set
in the shared generated zenoh config (the build front-ends above do this
for you — never define it for a single translation unit).
Flush-thread attributes: platforms that need explicit task priorities /
stack sizes can call zpico_set_flush_task_config(priority, stack_bytes)
before session open (mirrors zpico_set_task_config for the read/lease
tasks; FreeRTOS applies name/priority/stack, POSIX-like platforms stack size
only).
Per-publisher express (tx_express)
Marks one publisher’s samples “express”: they carry the zenoh EXPRESS wire flag and bypass tx batching — sent immediately even in a batching image. A transport hint, not a DDS policy: no RxO matching, ignored on subscriptions and by non-batching backends.
Rust:
#![allow(unused)]
fn main() {
// builder
let pub_ = node.publisher("/cmd_vel").typed::<Twist>().tx_express(true).build()?;
// or via the QoS profile
let qos = nros::QoSProfile::RELIABLE.tx_express(true);
let pub_ = node.create_publisher_with_qos::<Twist>("/cmd_vel", qos)?;
}
C:
nros_qos_t qos = NROS_QOS_DEFAULT;
qos.tx_express = 1;
nros_publisher_init_with_qos(&pub, &node, "/cmd_vel", ts, &qos);
C++:
auto pub = node.create_publisher<Twist>("/cmd_vel", nros::QoS().tx_express(true));
Pitfalls
- Benchmark listeners need a deep rx ring. A batched publisher delivers a
whole wire batch as one callback burst; the per-subscriber SPSC ring depth
is compile-time (
ZPICO_SUBSCRIBER_RING_DEPTH, default 4, drop-newest) — build measurement sinks with e.g.ZPICO_SUBSCRIBER_RING_DEPTH=1024or the numbers undercount by >10×. - Express ≠ fast on Zephyr. Express bypasses the batch, so each sample pays the socket-window wait itself. Reserve it for topics whose rate is low but whose latency matters.
- Don’t flush from your own timers/executor threads. The dedicated flush thread exists because flushing from tier threads stalls the very timers that generate the puts (measured slower than no batching).
Benchmarks and procedures: packages/testing/nros-bench/stress-zenoh-zephyr/
(streaming) and packages/testing/nros-tests/tests/w1_zephyr_tx_throughput_measure.rs
(timer tiers). Background: docs/roadmap/archived/phase-279-*.md,
docs/roadmap/phase-282-*.md, and
platform implementation notes.
Serial Transport
This guide covers using serial (UART) transport with nros on embedded targets. Serial transport connects an MCU directly to a zenoh router over UART, without needing an IP stack.
Overview
nros supports two transport mechanisms for connecting embedded devices to a zenoh network:
| Transport | Crate | Use Case |
|---|---|---|
| TCP/UDP (Ethernet/WiFi) | zpico-smoltcp | MCUs with Ethernet MAC or WiFi radio |
| Serial (UART) | zpico-serial | MCUs with only UART, or point-to-point links |
Serial transport uses zenoh-pico’s built-in COBS framing protocol over UART. No IP stack is required — the MCU sends and receives zenoh frames directly over a serial link to a host running the zenoh router (rmw_zenohd).
When to Use Serial
- Small MCUs — Cortex-M0/M0+ with no Ethernet MAC and insufficient RAM for smoltcp
- Point-to-point — Direct UART connection to a host, no network infrastructure needed
- Debugging — Serial output is visible in any terminal, easy to inspect
- Mixed topology — Some nodes on Ethernet, others on serial, all bridged through the router
Architecture
┌──────────┐ UART ┌──────────────┐ TCP ┌──────────┐
│ MCU │───────────────│ rmw_zenohd │─────────────│ ROS 2 │
│ (nros) │ COBS frames │ (router) │ zenoh net │ node │
└──────────┘ └──────────────┘ └──────────┘
The MCU connects to the zenoh router using a serial/... locator. The router bridges serial-connected nodes to the rest of the zenoh network (including ROS 2 nodes using rmw_zenoh).
Platform Support
Serial transport support varies by platform:
| Platform | Serial Implementation | Extra Crate Needed |
|---|---|---|
| Bare-metal (MPS2-AN385) | zpico-serial + UART driver | Yes |
| ESP32-QEMU (esp-hal, no IDF) | zpico-serial path, same as bare-metal | Yes |
| Zephyr | zenoh-pico built-in (uart_poll_in/out) | No |
| FreeRTOS / NuttX | zenoh-pico built-in (POSIX /dev/ttyXXX) | No |
| ThreadX | zenoh-pico built-in (HAL DMA) | No |
On non-bare-metal platforms, serial just works by using a serial/... locator — no extra crates needed. zpico-serial only fills the gap for bare-metal targets where custom FFI symbols replace zenoh-pico’s system layer.
Board Crate Feature Selection
Each board crate uses Cargo features to select the transport:
# Board crates are registry-style names resolved by the `nros sync`
# patch table (see examples/README.md), not path deps:
# Use serial transport (disable default ethernet/wifi)
nros-board-mps2-an385 = { version = "*", default-features = false, features = ["serial"] }
# Use ethernet transport (default)
nros-board-mps2-an385 = { version = "*" }
# Both transports (runtime selection via locator string)
nros-board-mps2-an385 = { version = "*", features = ["serial"] }
Available Features by Board
| Board Crate | Default | Alternative | Both |
|---|---|---|---|
nros-board-mps2-an385 | ethernet | serial | ethernet,serial |
nros-board-esp32-qemu | ethernet | serial | ethernet,serial |
When both features are enabled, the transport is selected at runtime by the zenoh locator string in Config:
"tcp/192.0.3.1:7448"→ Ethernet/WiFi"serial/UART_0#baudrate=115200"→ Serial
Quick Start: QEMU Serial Example
This mirrors what the in-tree serial e2e test
(packages/testing/nros-tests/tests/emulator.rs) actually does: a
socat PTY pair links QEMU’s UART to the router, so both ends exist
before either side starts.
1. Build the Serial Talker
nros sync # materialize generated/ + the patch table
cd examples/qemu-arm-baremetal/rust/serial-talker
cargo build --release
2. Create the PTY pair and start the router
# Two linked PTYs; QEMU gets one end, the router the other:
socat -d -d pty,raw,echo=0,link=/tmp/nros-serial-qemu \
pty,raw,echo=0,link=/tmp/nros-serial-router &
# Router LISTENS on its end of the pair:
ZENOH_CONFIG_OVERRIDE='listen/endpoints=["serial//tmp/nros-serial-router#baudrate=115200"]' \
ros2 run rmw_zenoh_cpp rmw_zenohd &
The router is ROS 2’s
rmw_zenohd(phase-362 retired the vendoredzenohd). It takes no command-line configuration — argv is not parsed, so a--connectflag is not rejected, it is simply unread. Configuration travels inZENOH_CONFIG_OVERRIDE, and the router listens on the serial endpoint; the MCU side dials it.
3. Boot QEMU with UART0 on the other end
The leaf’s default cargo run runner uses semihosting only (no serial
device) — boot QEMU explicitly, wiring UART0 to the pair:
qemu-system-arm -cpu cortex-m3 -machine mps2-an385 \
-display none -monitor none \
-icount shift=auto -semihosting-config enable=on,target=native \
-chardev serial,id=ser0,path=/tmp/nros-serial-qemu \
-serial chardev:ser0 \
-kernel target/thumbv7m-none-eabi/release/serial-talker
(-display none -monitor none, not -nographic — -nographic
implies -serial mon:stdio, which hijacks UART0 for the monitor.)
4. Subscribe from Host
# From a ROS 2 node (the talker publishes best-effort; force the QoS
# or a RELIABLE echo silently receives nothing):
ros2 topic echo /chatter std_msgs/msg/String --qos-reliability best_effort
Configuration
Serial Config
Board crates provide a serial_default() constructor:
#![allow(unused)]
fn main() {
use nros_board_mps2_an385::{Config, run};
let config = Config::serial_default();
// Defaults: baudrate=115200, locator="serial/UART_0#baudrate=115200"
run(config, |config| {
let exec_config = ExecutorConfig::new(config.zenoh_locator)
.domain_id(config.domain_id);
let mut executor = Executor::open(&exec_config)?;
// ...
Ok(())
})
}
Custom Baud Rate
#![allow(unused)]
fn main() {
let config = Config::serial_default()
.with_baudrate(921600)
.with_locator("serial/UART_0#baudrate=921600");
}
Locator Format
Serial locators follow zenoh-pico convention:
| Format | Example | Use Case |
|---|---|---|
serial/<dev>#baudrate=<baud> | serial/UART_0#baudrate=115200 | Device name (Zephyr, ESP-IDF, bare-metal) |
serial/<tx>.<rx>#baudrate=<baud> | serial/0.1#baudrate=115200 | Pin numbers (Arduino) |
QEMU PTY Testing
How It Works
QEMU redirects the emulated UART to a host pseudo-terminal. The in-tree flow uses a socat PTY pair (rather than QEMU’s -serial pty) so both endpoints exist before either process starts — the router listens on one end, QEMU’s UART0 is wired to the other, enabling full end-to-end testing without physical hardware.
┌──────────────────────────────────────────────────────────┐
│ Host │
│ ┌─────────┐ ┌───────────┐ ┌────────────────────┐ │
│ │rmw_zenohd◄──►│ socat PTY │◄──►│ QEMU MPS2-AN385 │ │
│ │ (listen)│ │ pair │ │ -serial chardev:… │ │
│ └────┬────┘ └───────────┘ │ UART0 ──► firmware │ │
│ │ └────────────────────┘ │
│ │ zenoh network │
│ ┌────▼────┐ │
│ │ ROS 2 │ │
│ │ echo │ │
│ └─────────┘ │
└──────────────────────────────────────────────────────────┘
QEMU Flags
The serial example’s .cargo/config.toml uses:
[target.thumbv7m-none-eabi]
runner = "qemu-system-arm -cpu cortex-m3 -machine mps2-an385 -nographic -semihosting-config enable=on,target=native -serial pty -kernel"
Key flags:
-serial pty— Expose UART0 as a host PTY-nographic— No display window-semihosting-config enable=on,target=native— Debug output via semihosting (separate from UART)- No
-netdev/-net— Serial transport doesn’t need Ethernet
-icount shift=auto
For reliable serial communication, add -icount shift=auto to synchronize QEMU’s virtual clock with wall-clock time. Without this, QEMU runs the CPU at full speed, which can cause timing-sensitive serial handshakes to fail:
runner = "qemu-system-arm -cpu cortex-m3 -machine mps2-an385 -nographic -semihosting-config enable=on,target=native -icount shift=auto -serial pty -kernel"
Automated Testing
The integration test test_qemu_serial_pubsub_e2e in packages/testing/nros-tests/tests/emulator.rs automates the full workflow:
- Build the serial-talker example (prebuilt fixture)
- Create socat PTY pairs (both ends exist before either process)
- Start the router with
listen/endpoints=["serial/<pty>#baudrate=115200"] - Boot QEMU via
start_mps2_an385_with_serial(UART0 → the pair;-display none -monitor none) - Subscribe and verify message delivery
Contributors: the in-tree lane that runs this test is in Per-Platform Contributor Lanes.
Baud Rate Tuning
Recommended Rates
| Baud Rate | Use Case |
|---|---|
| 115200 | Default, safe for all hardware |
| 460800 | Higher throughput, most USB-serial adapters |
| 921600 | Maximum for many MCU UARTs |
Higher baud rates increase throughput but may cause framing errors on noisy or long cables. QEMU ignores the baud rate (infinite speed), so rate tuning only matters on physical hardware.
Buffer Sizing
zenoh-pico serial uses a 1500-byte MTU with COBS framing. The maximum wire frame is 1516 bytes. zpico-serial uses a 2048-byte RX ring buffer per port, which accommodates one full frame plus overhead.
For high-throughput scenarios, ensure the MCU’s UART FIFO is drained frequently by calling executor.spin_once() in a tight loop.
Troubleshooting
“Session open failed” or Handshake Timeout
The zenoh serial handshake (Init → Ack) must complete within zenoh-pico’s timeout. Common causes:
- Wrong PTY path — Check the router’s listen endpoint names the same PTY link your QEMU
-chardevuses - Baud rate mismatch — MCU and router must use the same baud rate
- QEMU timing — Add
-icount shift=autoto slow down QEMU’s CPU clock
No Messages Received
- Locator mismatch — Ensure the MCU’s
zenoh_locatormatches what zenohd expects - Domain ID — Both sides must use the same ROS 2 domain ID
- zenohd not bridging — Verify zenohd is connected to the serial port and also listening on TCP for subscribers
UART Pin Conflicts
On physical hardware, ensure the UART TX/RX pins aren’t shared with the debug console. Some boards use UART0 for debug output — use a different UART for zenoh transport, or disable debug prints.
ESP32 Serial
ESP32 uses zenoh-pico’s built-in ESP-IDF serial implementation. No zpico-serial dependency is needed. Select serial transport in the board crate:
nros-board-esp32-qemu = { path = "...", default-features = false, features = ["serial"] }
The default locator is serial/UART_0#baudrate=115200. ESP32’s USB-JTAG-Serial peripheral or UART0/UART1 can be used.
RTIC Integration
RTIC (Real-Time Interrupt-driven Concurrency) is a
concurrency framework for ARM Cortex-M that compiles tasks directly to
hardware interrupt handlers — no RTOS kernel, no task control blocks, no
software scheduler. nano-ros runs on top of RTIC by letting the framework
own fn main, the scheduler, and the dispatchers, while nano-ros
contributes one spin task and (optionally) one callback-dispatch task.
This chapter is the user-facing tutorial for that integration. For the
underlying design — why per-Node dispatch strategies, why tags instead of
closures, why no AsyncNode trait yet — see the sibling internals page
Dispatch Strategy.
When to reach for nros::main!() instead of Pattern A
The Pattern A escape hatch is hand-written #[rtic::app], manual
Executor::open inside #[init], manual spin_once(0) polling task. It looks
like this:
#![allow(unused)]
fn main() {
// Pattern A, written by hand
#[rtic::app(device = mps2_an385_pac, dispatchers = [UARTRX0, UARTTX0])]
mod app {
#[init]
fn init(cx: init::Context) -> (Shared, Local) {
let syst = nros_board_mps2_an385::init_hardware(&config, cx.device, cx.core);
nros_rmw_zenoh::register().expect("Failed to register RMW backend");
let mut executor = Executor::open(&exec_config).unwrap();
let mut node = executor.create_node("talker").unwrap();
let publisher = node.create_publisher::<Int32>("/chatter").unwrap();
net_poll::spawn().unwrap();
publish::spawn().unwrap();
(Shared {}, Local { executor, publisher })
}
#[task(local = [executor], priority = 1)]
async fn net_poll(cx: net_poll::Context) {
loop {
cx.local.executor.spin_once(core::time::Duration::from_millis(0));
Mono::delay(10.millis()).await;
}
}
}
}
That pattern works, but it’s ~90 lines of glue per binary, and it puts the burden of getting RTIC + nros + the dispatcher list right on every example author. The macro collapses it to one line:
#![allow(unused)]
fn main() {
// File: examples/qemu-arm-baremetal/rust/talker-rtic/src/main.rs
#![no_std]
#![no_main]
use panic_semihosting as _;
nros::main!();
}
The nros::main!() proc-macro reads [package.metadata.nros.entry] deploy = "rtic-mps2-an385" from the Entry pkg’s Cargo.toml, sees that
the board’s metadata declares framework = "rtic", and emits a full
#[rtic::app] module — including #[init], __nros_spin, and (if any
deployed Node declares DispatchStrategy::Deferred) __nros_dispatch.
Pick nros::main!() whenever:
- You want a one-line
main.rsand don’t need custom RTIC tasks. - Your Node logic is portable across boards (the Node pkg is framework-agnostic; only the Entry pkg picks RTIC).
- You’re happy with the default dispatcher list from the board crate.
Keep Pattern A when:
- You need fine-grained control of dispatcher priorities, monotonic
setup, or hand-tuned
#[shared]state. - You’re shipping a one-off bring-up binary and don’t want the Node-pkg / Entry-pkg split overhead.
Both paths stay supported. The escape hatch is the “I want full
control” path; nros::main!() is the ergonomic path on top.
The three pkg roles
A nano-ros RTIC workspace is three packages (the 3-pkg-role
taxonomy, per
docs/design/0024-multi-node-workspace-layout.md §11):
my_rtic_robot/
├── Cargo.toml # [workspace] members = [...]
└── src/
├── talker_pkg/ # Node pkg — board-agnostic
│ ├── package.xml
│ ├── Cargo.toml
│ └── src/lib.rs # impl Node for Talker + nros::node!(Talker)
└── talker_entry/ # Entry pkg — picks RTIC board
├── package.xml
├── Cargo.toml # [package.metadata.nros.entry] deploy = "rtic-mps2-an385"
└── src/main.rs # nros::main!();
- Node pkg — declares what the node does (publishers,
subscriptions, services, actions). No
main, no#[rtic::app], no board choice. Builds asrlib + staticliband gets linked into one or more Entry pkgs. - Entry pkg — picks the board crate (
nros-board-mps2-an385,rticfeature), pins the deploy target, and runsnros::main!();. - Bringup pkg — optional, only when ≥2 Entry pkgs share the same
launch/*.launch.xmltopology. Skipped here because we have one binary.
The split exists so the same talker_pkg/ can be deployed under RTIC
on bare-metal Cortex-M, under FreeRTOS on QEMU, and under POSIX on a Linux host
without any per-target Node-pkg fork.
A minimal Node pkg — pub-only Talker
A pub-only Node declares DispatchStrategy::Inline (the default) — it
publishes from its own RTIC task and never enters on_callback.
#![allow(unused)]
fn main() {
// File: src/talker_pkg/src/lib.rs
#![no_std]
use core::time::Duration;
use nros::prelude::*;
use std_msgs::msg::Int32;
pub struct Talker;
pub struct TalkerState {
publisher: Publisher<Int32>,
counter: i32,
}
impl Node for Talker {
const NAME: &'static str = "talker";
// Inline is the default; spelled out here for clarity.
const DISPATCH: DispatchStrategy = DispatchStrategy::Inline;
fn register(ctx: &mut NodeContext<'_>) -> NodeResult<()> {
ctx.create_publisher::<Int32>("/chatter")?;
Ok(())
}
}
impl ExecutableNode for Talker {
type State = TalkerState;
fn init() -> Self::State {
TalkerState { publisher: Publisher::placeholder(), counter: 0 }
}
fn tick(state: &mut Self::State, _ctx: &mut TickCtx<'_>) {
state.counter = state.counter.wrapping_add(1);
let _ = state.publisher.publish(&Int32 { data: state.counter });
}
fn tick_period(_state: &Self::State) -> Option<Duration> {
Some(Duration::from_millis(1000))
}
}
nros::node!(Talker);
}
# File: src/talker_pkg/Cargo.toml
[package]
name = "talker_pkg"
version = "0.1.0"
edition = "2024"
[lib]
crate-type = ["rlib", "staticlib"]
[dependencies]
nros = { workspace = true, default-features = false }
std_msgs = { workspace = true }
[package.metadata.nros.node]
class = "talker_pkg::Talker"
The Entry pkg
# File: src/talker_entry/Cargo.toml
[package]
name = "talker_entry"
version = "0.1.0"
edition = "2024"
[[bin]]
name = "talker_entry"
path = "src/main.rs"
[dependencies]
nros = { workspace = true, default-features = false }
nros-board-mps2-an385 = { workspace = true, features = ["rtic"] }
talker_pkg = { path = "../talker_pkg" }
[package.metadata.nros.entry]
deploy = "rtic-mps2-an385"
[package.metadata.nros.deploy.rtic-mps2-an385]
board = "rtic-mps2-an385"
rmw = "zenoh"
domain_id = 0
locator = "tcp/192.168.1.10:7447"
#![allow(unused)]
fn main() {
// File: src/talker_entry/src/main.rs
#![no_std]
#![no_main]
use defmt_rtt as _;
use panic_probe as _;
nros::main!();
}
That’s the whole Entry pkg. The proc-macro reads deploy = "rtic-mps2-an385", looks up the board crate’s framework = "rtic"
metadata, and expands into a #[rtic::app(device = mps2_an385_pac, dispatchers = [UARTRX0, UARTTX0])] module with auto-generated #[init],
__nros_spin, and per-Node state slots.
DispatchStrategy::Inline vs Deferred
A Node pkg declares DispatchStrategy via the Node::DISPATCH const.
Two variants matter today; FromIsr is reserved as a design slot (see
DispatchStrategy::FromIsr).
Inline — pub-only or tick-only Nodes
#![allow(unused)]
fn main() {
impl Node for Talker {
const NAME: &'static str = "talker";
const DISPATCH: DispatchStrategy = DispatchStrategy::Inline;
// ...
}
}
Inline means: callbacks (if any) fire from the executor’s spin loop —
the same RTIC task that polls the network transport. On RTIC that’s
__nros_spin running at priority = 1.
Pick Inline when:
- The Node has no subscriptions, no service handlers, no action
handlers. Its only output is
Publisher::publishcalls fromtickor from a custom RTIC task you spawn yourself. - The Node has subscriptions, but the per-message work is so cheap (microseconds, no locks, no shared-state touches) that running it inline with the spin loop is acceptable.
The Talker above is the canonical Inline shape: it publishes from
tick at 1 Hz, never receives anything, and never blocks the spin
loop.
Deferred — callback-driven Nodes
#![allow(unused)]
fn main() {
impl Node for Listener {
const NAME: &'static str = "listener";
const DISPATCH: DispatchStrategy = DispatchStrategy::Deferred;
// ...
}
}
Deferred means: when a callback arrives, the spin loop enqueues a callback
token plus context into a heapless::spsc::Queue and returns immediately.
A separate __nros_dispatch RTIC task (typically priority = 2, one
above spin) drains the queue and calls ExecutableNode::on_callback from
its own task context.
Pick Deferred when:
- The Node’s
on_callbackhandler must hold an RTIC lock on a#[shared]resource — running it from the spin task would force every other dispatcher at the spin’s priority to wait on the lock. - The handler does non-trivial work (parses, integrates, posts to a hardware peripheral) and you want it scheduled independently from network polling.
- The Node sits at a different priority tier than the spin task.
The tag-based registration API for Deferred Nodes
Deferred Nodes can’t use the closure form ctx.create_subscription("/chatter", |msg| { ... }) —
the closure captures state by value, and a no-alloc framework
runtime has no place to store an unknown closure type. Instead, you
register with _static and dispatch via tag match in on_callback:
#![allow(unused)]
fn main() {
// File: src/listener_pkg/src/lib.rs
#![no_std]
use nros::prelude::*;
use std_msgs::msg::Int32;
pub struct Listener;
pub struct ListenerState {
sub_chatter: SubscriptionTag,
}
impl Node for Listener {
const NAME: &'static str = "listener";
const DISPATCH: DispatchStrategy = DispatchStrategy::Deferred;
fn register(ctx: &mut NodeContext<'_>) -> NodeResult<()> {
let _tag = ctx.create_subscription_static::<Int32>("/chatter")?;
Ok(())
}
}
impl ExecutableNode for Listener {
type State = ListenerState;
fn init() -> Self::State {
// The macro-emitted glue overwrites the placeholder with the
// real tag at register time.
ListenerState { sub_chatter: SubscriptionTag::placeholder() }
}
fn on_callback(
state: &mut Self::State,
cb: Callback<'_>,
ctx: &mut CallbackCtx<'_>,
) {
if state.sub_chatter == cb {
let msg: Int32 = ctx.downcast().unwrap();
defmt::info!("Received: {}", msg.data);
}
}
}
nros::node!(Listener);
}
The same shape applies to services (ServiceTag /
create_service_static) and actions (ActionTag /
create_action_static). The macro lint (216.A.6) rejects mixing
closure registration into a Deferred Node — the error spans the
offending call and suggests switching to the _static form.
The custom-task escape — nros::main!(custom_tasks = ...)
This form is implemented —
nros::main!parsescustom_tasksalongside the RTIC routing branch.
Real RTIC applications don’t only have nano-ros tasks. You want a
dedicated my_adc task to poll an ADC into a #[shared] buffer, or a
my_ui task to drive an OLED. The custom_tasks = [...] form of
nros::main!() folds your extra task fns into the generated
#[rtic::app] module:
#![allow(unused)]
fn main() {
// File: src/talker_entry/src/main.rs (with custom tasks)
#![no_std]
#![no_main]
use defmt_rtt as _;
use panic_probe as _;
nros::main!(custom_tasks = [my_adc, my_ui]);
#[rtic_task(priority = 3, shared = [adc_data])]
async fn my_adc(mut ctx: my_adc::Context) {
loop {
let sample = read_adc();
ctx.shared.adc_data.lock(|d| *d = sample);
Mono::delay(10.millis()).await;
}
}
#[rtic_task(priority = 2)]
async fn my_ui(_ctx: my_ui::Context) {
loop {
draw_screen();
Mono::delay(33.millis()).await;
}
}
}
The proc-macro extracts the user task tokens verbatim, splices them
into the generated mod __nros_app, and adds their dispatchers to the
RTIC dispatchers = [...] list. Signatures + attributes + priorities
are preserved.
Custom tasks can interact with nano-ros via the #[shared] resources
the macro exposes — typically executor (the spin executor) for raw
publisher access, or a Node’s state slot for direct handler calls.
Specifics will be locked when 216.B.4 lands.
DispatchStrategy::FromIsr — not yet
The third variant of DispatchStrategy is reserved for callbacks that
fire directly from an ISR handler (e.g. a timer pulse triggering a
publish without a scheduler hop). This is a design slot only;
the implementation is deferred.
Landing it requires:
- A reentrancy audit of the dispatch path.
- A lock-free SPSC variant tolerant of ISR-priority producers.
- A per-Node
#[isr_safe]proof contract.
Until that work lands, nros check rejects
DispatchStrategy::FromIsr deployments with a clear diagnostic.
See also
- Dispatch Strategy (internals) — the trichotomy and the per-Node-vs-per-callback rationale.
- Embassy Integration — the async sibling of this chapter.
- Role reference — the 3-pkg-role taxonomy in full.
- Scheduling Models — the RTIC scheduling model in real-time-systems terms.
Embassy Integration
Status: NO in-tree Embassy board — read before relying on this chapter.
nros-board-embassy-stm32f4was the only one, and phase-337 W7.a deleted it: it was registry-labelledscaffold(issue 0248 — entry methodstodo!(), a Deferred image signalling into a channel nothing drained), and RFC-0064’s rule is that a board with no Runtime cell is not a support promise this repo can make. What survives is the seam, which is the part an integrator needs: theEmbassyBoardEntrytrait innros-platformand thenros::main!()Embassy emit branch, plusnros ws check’s Deferred-dispatch lint, which routes on the board crate’sframework = "embassy"metadata.Two consequences for this chapter. (1) Every path below names a board YOU write — see Worked Example — STM32F4 Out of Tree for the board-crate shape. (2)
nros::main!()currently picks the framework from a hardcoded deploy-key table, so an out-of-tree Embassy board falls through toOwnedSpinrather than the Embassy emit; issue 0415 tracks teaching that table to read the sameframeworkmetadatanros ws checkalready reads. Until it closes, use Pattern A. (The RTIC twin of this integration is complete and runtime-tested onmps2-an385— see RTIC Integration.)
Embassy is an async/await framework for embedded
Rust, built around a cooperative executor that polls futures from a
single context (per priority tier). nano-ros runs on top of Embassy by
letting the framework own fn main, the spawner, and the async
executor, while nano-ros contributes one __nros_spin_task async fn
and (optionally) one __nros_dispatch_task async fn.
This chapter is the user-facing tutorial for that integration. For the
underlying design — why per-Node dispatch strategies, why tags instead
of closures, why on_callback stays sync even on async runtimes — see
the sibling internals page
Dispatch Strategy.
What Embassy buys you and where nano-ros fits
Embassy is an alternative to RTOS-style preemptive tasks: every
“task” is an async fn driven by embassy_executor::Spawner::spawn.
Tasks yield at .await points; there’s no scheduler tick, no context
switch overhead beyond a future poll. For nano-ros that means:
- A natural place for I/O-driven background work (SPI bus servicing,
GPIO debounce, sensor frame parsing) that yields at every
.awaiton aembassy_time::Timerorembassy_stm32::spi::Spi::transfer. - A natural place to spawn downstream work from inside a nano-ros callback — the “spawn-from-sync escape” below.
- Cooperative scheduling means a Node
on_callbackruns to completion before the executor polls anything else at the same priority. Keep handlers short, hand off long work to a spawned task.
Pattern A is hand-written #[embassy_executor::main], hand-written
zenoh_poll_task, hand-written publisher_task. It works, but it is ~150
lines and each author re-derives the spawn topology. The 216.C.4 board-entry
path collapses it to:
#![allow(unused)]
fn main() {
// File: <your entry pkg>/src/main.rs
#![no_std]
#![no_main]
use defmt_rtt as _;
use panic_halt as _;
nros::main!();
}
The proc-macro reads [package.metadata.nros.entry] deploy = "<your-embassy-board>" from the Entry pkg’s Cargo.toml, sees that the
board’s metadata declares framework = "embassy", and expands into a
full #[embassy_executor::main] async fn main(spawner: Spawner)
including the spin task spawn, the dispatch task spawn, and the
run_plan registration call.
Why sync on_callback even on Embassy
A natural-feeling Embassy API would make ExecutableNode::on_callback
an async fn. We don’t — for two reasons:
- The no-alloc contract. Async fns desugar to anonymous future
types; storing them generically in the runtime requires either
boxing (
Box<dyn Future>→allocdependency) or const-generic GAT plumbing through every trait. Both add cost without buying anythingSpawner::spawndoesn’t already give us. - Framework-task routing. The runtime already dispatches
callbacks from a framework-owned task (
__nros_dispatch_taskon Embassy,__nros_dispatchon RTIC). The Node author can spawn their own async task from inside the syncon_callback; that task runs under the same executor with no extra plumbing.
So on_callback keeps the same callback-token signature as RTIC, POSIX,
and every other backend. The escape for “I need to await something” is the
spawn-from-sync pattern below.
AsyncNode (an async-on-callback trait via RPITIT) is reserved as a
design slot — see When to wait for
AsyncNode at the end of this chapter.
The three pkg roles
The workspace shape is identical to RTIC (the 3-pkg-role
taxonomy, per
docs/design/0024-multi-node-workspace-layout.md §11):
my_embassy_robot/
├── Cargo.toml # [workspace] members = [...]
└── src/
├── listener_pkg/ # Node pkg — board-agnostic
│ ├── package.xml
│ ├── Cargo.toml
│ └── src/lib.rs # impl Node for Listener + nros::node!(Listener)
└── listener_entry/ # Entry pkg — picks Embassy board
├── package.xml
├── Cargo.toml # [package.metadata.nros.entry] deploy = "<your-embassy-board>"
└── src/main.rs # nros::main!();
The Node pkg stays board-agnostic — listener_pkg/ from the RTIC
chapter could be deployed under Embassy by swapping the Entry pkg.
That’s the point of the split.
Entry pkg
# File: src/listener_entry/Cargo.toml
[package]
name = "listener_entry"
version = "0.1.0"
edition = "2024"
[[bin]]
name = "listener_entry"
path = "src/main.rs"
[dependencies]
nros = { workspace = true, default-features = false }
my-embassy-board = { path = "../../boards/my-embassy-board" }
listener_pkg = { path = "../listener_pkg" }
[package.metadata.nros.entry]
deploy = "my-embassy-board"
[package.metadata.nros.deploy.my-embassy-board]
board = "my-embassy-board"
rmw = "zenoh"
domain_id = 0
locator = "tcp/192.168.1.10:7447"
#![allow(unused)]
fn main() {
// File: src/listener_entry/src/main.rs
#![no_std]
#![no_main]
use defmt_rtt as _;
use panic_halt as _;
nros::main!();
}
DispatchStrategy::Deferred is the common case
Every callback-driven Embassy Node should declare
DispatchStrategy::Deferred:
#![allow(unused)]
fn main() {
impl Node for Listener {
const NAME: &'static str = "listener";
const DISPATCH: DispatchStrategy = DispatchStrategy::Deferred;
// ...
}
}
Deferred means: the spin task pushes signaled callbacks into an
embassy_sync::channel::Channel<NoopRawMutex, _, CHANNEL_CAPACITY>
(default capacity 32). A separate __nros_dispatch_task awaits the
channel receive end and routes each callback to the right Node’s
on_callback. Both tasks run cooperatively on the same Embassy
executor; you can spawn your own tasks alongside them.
Inline is allowed but unusual on Embassy:
- The
nros checklint (Phase 216.D.1) emits a warning forframework = "embassy"withDispatchStrategy::Inlineand suggests switching to Deferred — running callbacks inline on Embassy ties them to the spin task’s poll point, which makes the scheduling cost-vs-benefit confusing. - If your Node is genuinely pub-only (no subscriptions, no
services, no actions), Inline is still fine — the Inline path
simply never enters
on_callback. The warning above does not apply for pure-publisher Nodes.
The spawn-from-sync escape
Real Embassy Nodes need to do async work downstream from a callback: write to an SPI bus, send a UART frame, poll a sensor with timeout. The pattern is:
- Hold an
embassy_executor::SpawneronSelf::State. - From inside the sync
on_callback, callstate.spawner.spawn(handle_downstream(msg)).unwrap(). - The downstream
#[embassy_executor::task] async fnhandles the await-heavy work.
Worked example — Listener that writes received data to SPI
#![allow(unused)]
fn main() {
// File: src/listener_pkg/src/lib.rs
#![no_std]
use embassy_executor::Spawner;
use nros::prelude::*;
use std_msgs::msg::Int32;
pub struct Listener;
pub struct ListenerState {
sub_chatter: SubscriptionTag,
spawner: Spawner,
}
impl Node for Listener {
const NAME: &'static str = "listener";
const DISPATCH: DispatchStrategy = DispatchStrategy::Deferred;
fn register(ctx: &mut NodeContext<'_>) -> NodeResult<()> {
let _tag = ctx.create_subscription_static::<Int32>("/chatter")?;
Ok(())
}
}
impl ExecutableNode for Listener {
type State = ListenerState;
fn init() -> Self::State {
ListenerState {
sub_chatter: SubscriptionTag::placeholder(),
// The macro-emitted glue populates the Spawner from the
// EmbassyBoardEntry init hook; until then, hold a sentinel
// that's overwritten before the first dispatch.
spawner: Spawner::for_current_executor(),
}
}
fn on_callback(
state: &mut Self::State,
cb: Callback<'_>,
ctx: &mut CallbackCtx<'_>,
) {
if state.sub_chatter == cb {
let msg: Int32 = ctx.downcast().unwrap();
// Spawn the async work and return immediately. The
// dispatch task stays unblocked.
state.spawner.spawn(handle_msg(msg)).unwrap();
}
}
}
#[embassy_executor::task(pool_size = 4)]
async fn handle_msg(msg: Int32) {
// Pretend we have an SPI handle stashed somewhere accessible.
// The real wiring depends on your peripheral-access pattern;
// the point is `.await` is free here, even though on_callback
// is sync.
defmt::info!("handling /chatter sample: {}", msg.data);
embassy_time::Timer::after_millis(5).await;
// spi.write(&msg.data.to_le_bytes()).await.unwrap();
}
nros::node!(Listener);
}
Two things to note:
- Spawner is
Copy. Holding aSpawneronSelf::Stateadds no runtime cost beyond an integer copy. You can clone it freely across multipleon_callbackcalls. pool_sizematters.#[embassy_executor::task(pool_size = N)]allocatesNstatic slots. If you spawn faster than the spawned tasks complete,spawn(...)returnsErr(SpawnError::Busy)— handle it (drop the message, log a warning, etc.). The default ispool_size = 1.
When the spawn pool fills up
Two patterns for backpressure:
- Drop on full. Treat the spawned task as best-effort. If the
spawnreturns an error, log it and continue. - Channel-based queue. Pre-spawn one long-lived
#[embassy_executor::task] async fnthat holds anembassy_sync::channel::Channelreceive end. Theon_callbackpushes into the channel’s send end (drop on full or block — your choice). Lets you control queue depth independently ofpool_size.
The right pick depends on whether dropped messages are tolerable for your workload; nano-ros doesn’t impose either.
When to wait for AsyncNode
The spawn-from-sync escape covers every case we know of today. But if
you find yourself writing a lot of one-off pool-of-one spawn dances —
each callback spawning a task whose body is a single .await — that’s
a smell, and we’d want to know.
The design slot for direct async callbacks is AsyncNode (Phase
216.E.2):
#![allow(unused)]
fn main() {
// Design sketch — NOT shipping today.
pub trait AsyncNode: 'static {
const NAME: &'static str;
const DISPATCH: DispatchStrategy = DispatchStrategy::Deferred;
fn register(ctx: &mut NodeContext<'_>) -> NodeResult<()>;
async fn on_callback(
&mut self,
callback: AsyncCallbackToken,
ctx: CallbackCtx,
);
}
}
It would compile only on Embassy targets (RTIC has no async runtime
to drive RPITIT futures into; POSIX + RTOS don’t either), and the
nros::node!() macro would emit a separate
__nros_node_<pkg>_on_callback_async ABI symbol the Embassy dispatch
task picks up. 216.E.2 lands only if real usage shows
spawn-from-sync is consistently painful. If your application is
hitting that case, file an issue with the call pattern that’s
prompting it.
Until then: spawn from sync. It’s two lines per callback and stays fully no-alloc.
See also
- Dispatch Strategy (internals) — the trichotomy and the per-Node-vs-per-callback rationale.
- RTIC Integration — the interrupt-driven sibling of this chapter.
- Role reference — the 3-pkg-role taxonomy in full.
- Scheduling Models — the Embassy cooperative model alongside RTIC, RTOS, and POSIX.
Troubleshooting
Common issues and solutions when working with nros.
zenoh-pico Multiple Client Issues
Symptoms
When running multiple zenoh-pico clients (e.g., a talker and listener) simultaneously connecting to the same router, you may see errors like:
<err> nros_bsp: Failed to open zenoh session: -3
<err> rust: rustapp: BSP init failed: InitFailed(-5)
Or publisher/subscriber declarations fail with:
- Error code
-128(_Z_ERR_GENERIC) - Error code
-78(_Z_ERR_SYSTEM_OUT_OF_MEMORY)
The first client typically succeeds while subsequent clients fail during z_declare_publisher() or z_declare_subscriber().
Root Cause
This is caused by the Z_FEATURE_INTEREST feature in zenoh-pico. The interest protocol implements write filtering to optimize network traffic by only sending data to interested subscribers. However, this feature has issues when multiple clients connect to the same router:
- The router tracks “interest” for each client
- When creating a publisher, zenoh-pico creates a write filter context
- The filter creation calls
_z_session_rc_clone_as_weak()which can fail - This manifests as
_Z_ERR_SYSTEM_OUT_OF_MEMORY(-78) even when memory is available
The failure occurs in zenoh-pico/src/net/filtering.c:
ctx->zn = _z_session_rc_clone_as_weak(zn);
if (_Z_RC_IS_NULL(&ctx->zn)) {
_z_write_filter_ctx_clear(ctx);
z_free(ctx);
_Z_ERROR_RETURN(_Z_ERR_SYSTEM_OUT_OF_MEMORY);
}
Solution
nros disables Z_FEATURE_INTEREST by default for all builds:
Native builds (build.rs):
#![allow(unused)]
fn main() {
let dst = cmake::Config::new(&zenoh_pico_build)
// ...
.define("Z_FEATURE_INTEREST", "0")
.build();
}
Zephyr builds: edit
modules/lib/zenoh-pico/include/zenoh-pico/config.h as shown below.
(Contributors working in the nano-ros checkout: just setup zephyr
applies the same patch to the in-tree workspace automatically.)
Manually edit modules/lib/zenoh-pico/include/zenoh-pico/config.h:
// Change from:
#define Z_FEATURE_INTEREST 1
#define Z_FEATURE_MATCHING 1
// To:
#define Z_FEATURE_INTEREST 0
#define Z_FEATURE_MATCHING 0
Then rebuild with --pristine:
west build -b native_sim/native/64 nros/examples/zephyr/rust/talker -d build-talker --pristine
Note: Z_FEATURE_MATCHING depends on Z_FEATURE_INTEREST, so both must be disabled.
References
- rmw_zenoh_pico disables this feature by default
- zenoh-pico filtering code:
src/net/filtering.c - zenoh-pico interest protocol:
src/session/interest.c
zenoh-pico Version Compatibility
Symptoms
Publisher works but z_publisher_put fails immediately:
zenoh_shim: z_publisher_put failed: -100
<err> rust: rustapp: Publish failed: PublishFailed(-1)
zenoh_shim: z_publisher_put failed: -73
Error codes:
-100:_Z_ERR_TRANSPORT_TX_FAILED- Transport transmission failed-73:_Z_ERR_SESSION_CLOSED- Session closed after first failure
Root Cause
Version mismatch between zenoh-pico and zenohd:
- zenoh-pico: 1.5.1 (in west.yml)
- zenohd: 1.7.x (installed via cargo)
The zenoh protocol may have changed, causing transport-level incompatibilities.
Solution
Compare the two zenoh versions across the seam — nano-ros does not pin the router, because the router is the one ROS ships (phase-362 / RFC-0075):
grep 'define ZENOH_C ' /opt/ros/$ROS_DISTRO/opt/zenoh_cpp_vendor/include/zenoh_configure.h
cat packages/rmw/zenoh/zpico-sys/zenoh-pico/version.txt
Read the HEADER, not dpkg -l: the package version (ros-humble-rmw-zenoh-cpp 0.1.9) is a wrapper version and says nothing about the zenoh inside it.
Under zenoh’s 1.x wire guarantee the two need not match, so a mismatch is evidence rather than a diagnosis. The interop tests print both numbers on a delivery failure for exactly this comparison.
Multiple Zephyr Instance Issues
Ephemeral Port Conflict
Symptom: When running multiple Zephyr native_sim instances (e.g., talker and listener), both sessions fail to establish or one gets -100 (_Z_ERR_TRANSPORT_TX_FAILED) errors.
Root Cause: Zephyr native_sim uses a test entropy source that produces identical random number sequences unless seeded differently. This causes both instances to select the same ephemeral TCP port when connecting to the router, causing a port conflict.
Solution: Pass different --seed values to each native_sim instance:
# Start listener with one seed
./build-listener/zephyr/zephyr.exe --seed=12345
# Start talker with a different seed
./build-talker/zephyr/zephyr.exe --seed=67890
The --seed parameter initializes the test entropy source with a different value, producing different random numbers and thus different ephemeral ports.
For automated tests: The test framework (packages/testing/nros-tests) automatically assigns unique seeds to each process.
Cyclone DDS: Two native_sim Nodes Never Discover Each Other
Symptom: Two native_sim nodes with the Cyclone DDS RMW (e.g. talker + listener) each start, the publisher reports Publishing: 'Hello World: N', but the subscriber never prints an I heard: line — no abort, no error.
Root Cause: Same fixed-entropy issue as above. Cyclone derives each participant’s DDSI GUID prefix from the entropy source; with identical entropy, both processes get the same GUID. DDSI requires per-participant unique GUIDs — a node treats SPDP from a peer that shares its own GUID as a self-announcement and drops it, so the two never form a proxy-participant pair and discovery never closes.
Solution: Pass different --seed values per instance (exactly as above):
./build-listener/zephyr/zephyr.exe --seed=11
./build-talker/zephyr/zephyr.exe --seed=22
With distinct seeds the GUID prefixes differ and SPDP discovery completes. (native_sim Cyclone discovery is unicast-only — the embedded config points <Peers> at 127.0.0.1; native_sim’s NSOS multicast RX fd does not survive Cyclone’s select()-based socket waitset. Real hardware has real entropy and full multicast, so neither workaround is needed off native_sim.)
Network Configuration Issues
TAP Interface Setup (QEMU bare-metal only)
Symptom: QEMU cannot connect to the network.
Solution: Run the network setup script:
sudo ./scripts/qemu/setup-network.sh
This creates and configures the tap0 interface at 192.0.3.1. QEMU
bare-metal guests live on 192.0.3.10+.
Zephyr native_sim networking
Zephyr native_sim uses NSOS (Native Sim Offloaded Sockets) — socket
calls are forwarded to host syscalls, so there is no emulated L2 stack and
no TAP bridge. Point each example at a host-loopback zenohd / XRCE Agent:
ZENOH_CONFIG_OVERRIDE='listen/endpoints=["tcp/127.0.0.1:7447"];scouting/multicast/enabled=false' ros2 run rmw_zenoh_cpp rmw_zenohd # or any host-accessible address
Multiple native_sim instances can coexist without bridge configuration.
Message Too Large / Truncated Messages
Symptoms
TransportError::MessageTooLargewhen receiving service requests or subscription data- Messages arrive but with corrupted or incomplete payloads
- Large messages (images, point clouds) silently disappear
Root Cause
nros uses static buffers at multiple layers, each with configurable size limits. A message must fit through every layer in the path to be delivered intact.
Zenoh backend layers:
| Layer | Posix Default | Embedded Default | Env Var |
|---|---|---|---|
Defragmentation (Z_FRAG_MAX_SIZE) | 65536 | 2048 | ZPICO_FRAG_MAX_SIZE |
Batch size (Z_BATCH_UNICAST_SIZE) | 65536 | 1024 | ZPICO_BATCH_UNICAST_SIZE |
| Per-entity shim buffer | 1024 | 1024 | – (named constant in code) |
User receive buffer (RX_BUF) | 1024 | 1024 | – (const generic) |
XRCE-DDS backend layers:
| Layer | Posix Default | Embedded Default | Env Var |
|---|---|---|---|
| Transport MTU | 4096 | 512 | XRCE_TRANSPORT_MTU |
| Per-entity buffer | 1024 | 1024 | – (named constant in code) |
Solutions
Increase transport-level limits (set before cargo build):
# Zenoh: allow 128 KB reassembled messages
ZPICO_FRAG_MAX_SIZE=131072 cargo build
# XRCE: increase MTU to 8 KB
XRCE_TRANSPORT_MTU=8192 cargo build
Increase per-entity buffer sizes (in code):
#![allow(unused)]
fn main() {
// Zenoh subscriber with 4 KB receive buffer
let sub = node.create_subscriber_sized::<MyMsg, 4096>(SubscriberOptions::new("/topic"))?;
// Zenoh publisher with 4 KB transmit buffer
let pub_ = node.create_publisher_sized::<MyMsg, 4096>(PublisherOptions::new("/topic"))?;
}
Clean rebuild after changing env vars (CMake caches old values):
cargo clean -p zpico-sys # or: cargo clean -p xrce-sys
cargo build
Build Issues
zenoh-pico Submodule Not Found
Symptom:
zenoh-pico submodule not found at /path/to/zenoh-pico. Run: git submodule update --init
Solution:
git submodule update --init --recursive
CMake Cache Stale
Symptom: Changes to CMake defines (like Z_FEATURE_INTEREST) don’t take effect.
Solution: Clean the build cache:
# Clean the zenoh-pico sys crate build cache
cargo clean -p zpico-sys
touch packages/rmw/zenoh/zpico-sys/build.rs
cargo build
For Zephyr builds, clean the west build directory:
rm -rf build/
west build -b native_sim/native/64 <app>
Test Failures
Tests Hang Forever
Symptom: Integration tests hang indefinitely.
Possible causes:
- No zenohd router running: Many tests require a zenohd router
- Port already in use: Another process is using port 7447
- Network misconfigured: TAP or bridge interfaces not set up
Solutions:
# Check if zenohd is running
pgrep zenohd
# Check if port 7447 is in use
ss -tlnp | grep 7447
# Verify QEMU bare-metal TAP (only needed for QEMU bare-metal platforms)
ip addr show tap0
Zephyr Tests Skip
Symptom: All Zephyr tests are skipped.
Cause: The test framework couldn’t find the Zephyr workspace or required components.
Solution: Ensure the Zephyr workspace is properly configured:
# Default location: $repo/zephyr-workspace/ (gitignored, in-tree).
# Legacy setups: $repo/../nano-ros-workspace/ (auto-detected).
ls -la zephyr-workspace
nros doctor
# Or set explicitly
export NROS_ZEPHYR_WORKSPACE=/path/to/zephyr-workspace
# Verify west is available
west --version
Rust/C FFI Issues
Subscriber Callback Crashes
Symptom: When a Zephyr listener receives a message, the application crashes with a segfault or produces garbage values when accessing struct fields. Debug output may show valid pointers but invalid field values:
bsp_zephyr: sub->callback=0x60, sub->user_data=0x4 # These should be valid addresses!
Root Cause: Your C glue stores a pointer to the nros_subscription_t struct when you hand it to a registration call. If the Rust code moves the struct after that (e.g., when returning it from a function), the C side’s pointer becomes dangling.
In Rust, values are moved by default:
#![allow(unused)]
fn main() {
// WRONG - struct will move when returned
pub fn create_subscriber(...) -> Result<BspSubscriber<M>, Error> {
let mut sub = NanoRosSubscriber { ... };
nros_bsp_create_subscriber(&mut sub, ...); // C stores pointer to `sub`
Ok(BspSubscriber { sub, ... }) // `sub` MOVES here - old address is invalid!
}
}
Solution: Use static storage or ensure the struct has a stable address before passing it to C:
use core::mem::MaybeUninit;
use core::ptr::addr_of_mut;
// Wrapper to make static storage Sync
struct StaticSubscriber(MaybeUninit<NanoRosSubscriber>);
unsafe impl Sync for StaticSubscriber {}
static mut SUBSCRIBER_STORAGE: StaticSubscriber =
StaticSubscriber(MaybeUninit::uninit());
fn main() {
let sub_ptr = unsafe {
let storage_ptr = addr_of_mut!(SUBSCRIBER_STORAGE);
let sub = (*storage_ptr).0.as_mut_ptr();
// Initialize fields...
sub
};
// Now the pointer is stable
nros_bsp_create_subscriber(sub_ptr, ...);
}
Alternative approaches:
- Use
Box::new()(requiresalloc) for heap allocation with stable address - Use
Pinto prevent moves - Have the C code use indices instead of pointers
Why this happens: Unlike C where variables have fixed addresses, Rust moves values during assignment and return. When the C code stores a pointer during initialization, it doesn’t know the Rust struct will be moved later.
Function Pointer ABI Mismatch
Symptom: Callbacks passed between Rust and C crash or receive garbage arguments.
Solution: Ensure function pointers use extern "C":
#![allow(unused)]
fn main() {
// CORRECT - uses C calling convention
extern "C" fn my_callback(data: *const u8, len: usize, ctx: *mut c_void) {
// ...
}
// WRONG - uses Rust calling convention (incompatible with C)
fn my_callback(data: *const u8, len: usize, ctx: *mut c_void) {
// ...
}
}
zenoh-pico Error Codes
| Code | Name | Description |
|---|---|---|
| -3 | _Z_ERR_TRANSPORT_OPEN_FAILED | Could not connect to router |
| -78 | _Z_ERR_SYSTEM_OUT_OF_MEMORY | Memory allocation failed (or write filter issue) |
| -128 | _Z_ERR_GENERIC | Generic error |
For the complete list, see zenoh-pico/include/zenoh-pico/utils/result.h.
Setup Compared to Standard ROS 2
This page is for ROS 2 users who already know the normal desktop flow:
install a ROS distro, create a workspace, run rosdep, build with
colcon, and select an RMW at runtime with RMW_IMPLEMENTATION.
nano-ros keeps the workspace and package vocabulary, but changes the setup boundary because it targets embedded and RTOS builds.
Standard ROS 2 Flow
A typical ROS 2 application starts from a distro install:
source /opt/ros/humble/setup.bash
mkdir -p ~/ros2_ws/src
cd ~/ros2_ws
rosdep install --from-paths src --ignore-src -y
colcon build
source install/setup.bash
The middleware implementation is selected at process startup:
export RMW_IMPLEMENTATION=rmw_cyclonedds_cpp
ros2 run my_pkg my_node
That model assumes shared libraries, a hosted OS, and runtime plugin loading.
nano-ros Flow
Where standard ROS 2 installs a distro and resolves system packages with
rosdep, nano-ros provisions a per-board toolchain with one command.
nros setup replaces the distro install + rosdep: it ships prebuilt
toolchains per platform per RMW — the cross-compiler, emulator, RMW host
daemon, and SDK sources for a board are fetched from a pinned index into a
shared store (${NROS_HOME:-~/.nros}/sdk). You do not install cross-toolchains by hand,
and you do not need a ROS distro on the machine.
# 1. Build the in-tree nros CLI (analogous to installing a ROS distro, Phase 218):
./scripts/bootstrap.sh # builds packages/cli/target/release/nros
source ./activate.sh # OR: direnv allow / source ./activate.fish
# 2. Provision a board + RMW (analogous to `rosdep install`):
nros setup native --rmw zenoh
# 3. Generate message bindings, then build + run an example
# (generated/ and .cargo/config.toml are gitignored — a fresh
# clone cannot build without this step):
nros sync
cd examples/native/rust/talker
cargo run
For embedded targets, name the board instead of native; nros setup
fetches the matching prebuilt cross-toolchain + emulator + SDK:
nros setup qemu-arm-freertos --rmw zenoh # arm-none-eabi-gcc, qemu, FreeRTOS+lwIP
nros setup zephyr --rmw zenoh # Zephyr west workspace + SDK bits
nros setup qemu-arm-nuttx --rmw zenoh # arm-none-eabi-gcc, qemu, NuttX
Useful flags: nros setup --list (every package + version),
nros setup <board> --dry-run (resolve + print the plan, fetch nothing),
nros setup --licenses (license-gated packages). See
Supported Boards for the board list and
nros CLI for every subcommand.
Contributors working on nano-ros itself drive the same index through
just—just <module> setupcallsnros setup <board>under the hood, so the provisioned toolchains are identical.
Choosing platform + RMW
Unlike standard ROS 2, RMW and platform are compile-time choices —
there is no runtime RMW_IMPLEMENTATION switch on embedded targets
(no dlopen). The pair is selected via CMake cache vars + Cargo
features:
# Each example is a standalone CMake project that pulls nano-ros in
# via add_subdirectory.
set(NANO_ROS_PLATFORM freertos)
set(NANO_ROS_RMW zenoh)
set(NANO_ROS_BOARD mps2-an385-freertos)
add_subdirectory(<repo-root> nano_ros)
target_link_libraries(my_app PRIVATE NanoRos::NanoRos)
if(COMMAND nros_platform_link_app) # defined by zephyr/threadx platforms only
nros_platform_link_app(my_app)
endif()
nano_ros_link_rmw(my_app RMW zenoh)
Multi-RMW bridges (one binary, two or more backends) use
Executor::open_with_rmw("<name>", ...) + node_builder.rmw("<name>")
— see Cross-backend Bridges.
What Stays Familiar
- Workspace layout: one source checkout next to your packages.
- Package metadata: downstream packages still use
package.xml. - ROS vocabulary: nodes, publishers, subscriptions, services, actions, QoS profiles, parameters, and message packages keep ROS-shaped names.
colcon buildstill works as a consumer-side build for POSIX workspaces that already use it; embedded targets usecmake,cargo,west, oridf.pydirectly.- Interop: POSIX nano-ros nodes can communicate with standard ROS 2 nodes through compatible RMW backends (Zenoh, Cyclone DDS, XRCE).
What Changes
- Source-only. No binary SDK tarball, no crates.io umbrella crate,
no Arduino zip / ESP-IDF binary component / GitHub Releases artifact.
The locked policy is
git clone --branch=v<X.Y.Z>+ in-tree build. - Per-board provisioning, no
rosdep.nros setup <board> --rmw <rmw>is the single setup command. It fetches toolchains (cross-gcc, emulator) and SDK sources for exactly that board+RMW into~/.nros/sdk— no system-wide package install. Most are prebuilt; a few without a seeded asset are built from source —--dry-runsays which on your host. The zenoh router is deliberately NOT provisioned: it comes from a ROS 2 install (ros2 run rmw_zenoh_cpp rmw_zenohd, RFC-0075), so the zenoh path needs ROS 2 on the router host; xrce and cyclonedds do not. (Thejust <module> setuprecipes call the same command for contributors.) - Compile-time RMW + platform. Embedded targets can’t
dlopen, so the RMW and platform combination is locked in by CMake cache vars (NANO_ROS_PLATFORM,NANO_ROS_RMW) and Cargo features at build time. - No install prefix. Phase 140 removed
just install-local; there is nocmake --installstep for nano-ros itself — consumers pull it into their build viaadd_subdirectory(<repo-root>)orfind_package(nano_ros)from the checkout. (Example packages still carry their owninstall(TARGETS …)for colcon compatibility.) The integration shells underintegrations/<rtos>/re-export the same root CMake under each RTOS’s native package manager. - Generated bindings in-tree. Message codegen lands under
<your-package>/generated/(orOUT_DIRfor Cargo builds), not in an installed ROS message library. - Configuration is build-time on embedded. Runtime env vars
(
ROS_DOMAIN_ID,NROS_LOCATOR— legacy aliasZENOH_LOCATOR, …) work on POSIX; embedded targets bake config from[package.metadata.nros.deploy.<t>](Rust) / the package.xml<nano_ros deploy=…/>tuple (C/C++) (CMake), plus Kconfig on Zephyr.
Next Step
Continue with Installation, then run the ROS 2 Interoperability example before moving to a platform-specific guide.
Porting a ROS 2 C++ node to nano-ros
Goal: take a normal ROS 2 C++ node (one that compiles + runs under
colcon build against ros-humble-*) and run it under nano-ros — without
rewriting the source. The rclcpp source-compat layer (nros/rclcpp_compat.hpp +
cmake/compat/NrosRclcppCompat.cmake + nros-diagnostic-updater) is built for
this; the only delta is build-script glue.
The canonical proof lives at
examples/templates/cpp-port-minimal-publisher/ —
the ROS 2 tutorial’s minimal_publisher.cpp vendored unmodified, building
against nano-ros via the three glue lines below.
Two layers of glue
Per-package CMakeLists.txt — zero nano-ros lines
The ported pkg’s CMakeLists.txt carries only stock ROS 2 syntax.
Same file builds under both colcon build AND a nano-ros build:
cmake_minimum_required(VERSION 3.20)
project(my_node LANGUAGES CXX)
find_package(ament_cmake REQUIRED)
find_package(rclcpp REQUIRED)
find_package(std_msgs REQUIRED)
# … any other msg packages the source #include's.
add_executable(my_node src/my_node.cpp)
ament_target_dependencies(my_node rclcpp std_msgs)
ament_package()
find_package(rclcpp) resolves through the rclcpp Find-stub (which
auto-applies the rclcpp_compat.hpp force-include); find_package(std_msgs)
resolves through the smart Find-stub (it walks
NROS_INTERFACE_SEARCH_PATH > AMENT_PREFIX_PATH > bundled); the
ament_target_dependencies compat shim wires both link targets.
Workspace umbrella CMakeLists.txt — one nano-ros include
The umbrella CMakeLists.txt (sits at the workspace root, next to
src/) is the only nano-ros-aware file:
cmake_minimum_required(VERSION 3.22)
project(my_workspace LANGUAGES CXX)
set(CMAKE_CXX_STANDARD 14)
# 1) Pull nano-ros in.
set(NANO_ROS_PLATFORM posix)
set(NROS_RMW "zenoh" CACHE STRING "Active RMW.")
set(NANO_ROS_RMW "${NROS_RMW}")
add_subdirectory("/path/to/nano-ros" nano_ros)
# 2) Point the smart Find-stub at this workspace's src/ (must precede the
# NrosRclcppCompat include so workspace-pkg Find<pkg>.cmake auto-emit
# picks it up).
set(NROS_INTERFACE_SEARCH_PATH "${CMAKE_SOURCE_DIR}/src")
# 3) Drop-in source-compat surface.
include("/path/to/nano-ros/cmake/compat/NrosRclcppCompat.cmake")
# 4) Bulk-build every workspace msg pkg in topo order (one line instead of
# N add_subdirectory(src/<pkg>) lines).
nros_workspace_interfaces()
# 5) Build consumer apps.
add_subdirectory(src/my_node)
No nros_generate_interfaces(<pkg>) calls per consumer — the smart
Find-stub does the codegen at find_package(<pkg>) time.
Reference fixture
examples/templates/local-msg-package/
ships the pattern end-to-end: two workspace msg pkgs (local_msgs,
extra_msgs) with intra-workspace dep + a C++ consumer pulling msgs
from BOTH the workspace AND AMENT (std_msgs, geometry_msgs,
sensor_msgs) via one find_package shape. Cross-build proof: the
same src/ builds under colcon build (CI-gated).
Legacy nros_generate_interfaces(<pkg>) shape
(The full current-vs-legacy spelling table lives in Message Generation.)
Per-package nros_generate_interfaces(std_msgs LANGUAGE CPP SKIP_INSTALL)
calls still work (back-compat preserved) but are deprecated for new
code — they bypass the ROS-convention smart Find-stub + workspace
discovery. Existing in-tree examples will migrate as part of Phase
210.E.3.
What “just works” without source edits
The compat surface covers the patterns a typical ROS 2 C++ node uses:
| rclcpp surface | nano-ros mapping | Notes |
|---|---|---|
class MyNode : public rclcpp::Node | rclcpp::Node shim → nros::Executor + nros::Node | Ctor takes (name). |
std::make_shared<MyNode>() | inherits enable_shared_from_this | shared_from_this() works. |
create_publisher<M>(topic, qos) | shared_ptr-returning wrapper | qos can be rclcpp::QoS(10) or an int. |
create_subscription<M>(topic, qos, callback) | polling pump dispatched from spin* | Capturing lambdas + std::function all work. |
create_wall_timer(period, callback) | wall-timer dispatched from spin* | std::chrono::duration arg, capturing-lambda callback. |
rclcpp::init(argc, argv) / shutdown() / ok() / spin(n) / spin_some(n) | wraps nros::init/shutdown/ok/spin_once | argc/argv ignored. |
RCLCPP_INFO / WARN / ERROR / DEBUG / FATAL | dispatched through NROS_* macros | _THROTTLE variants degrade to plain log. |
rclcpp::QoS / KeepLast(n) / SystemDefaultsQoS() | subclass of nros::QoS with the (depth) ctor | Chainable setters inherited. |
diagnostic_updater::Updater + DiagnosticStatusWrapper | nros-diagnostic-updater shim | Publishes /diagnostics. |
rclcpp_action::Server<A> / Client<A> | aliases for nros::ActionServer/Client<A> | The action call shapes (send_goal_async etc.) match. |
RCLCPP_COMPONENTS_REGISTER_NODE(class) | no-op macro + cmake-side rclcpp_components_register_node() emits a thin int main() per registration | Single-binary embedded. |
find_package(ament_cmake_auto / rclcpp / rclcpp_components / diagnostic_updater / std_msgs / …) | Find-stubs at cmake/compat/stubs/ | ~24 of the most-cited ROS 2 packages stubbed; add your own under cmake/compat/stubs/Find<pkg>.cmake for more. |
What’s documented as “needs adapt” (codegen-side, not surface-side)
These are cosmetic codegen differences nano-ros’s per-package codegen
and the upstream rosidl_default_runtime codegen don’t share; both are
tracked as ROS-convention codegen work.
- Message string fields. nano-ros codegen emits
nros::FixedString<N>, upstream emitsstd::string. Assigning astd::stringneeds a one-token adapter:message.data = s.c_str(). The reverse(std::string{}.c_str())is whatRCLCPP_INFOalready takes. - Generated message header path. CLOSED (alias
headers): nano-ros codegen emits BOTH the per-message form
<std_msgs/msg/string.hpp>(upstream-shape) AND the umbrella<std_msgs/std_msgs.hpp>. Use whichever the original source picks.
What’s out of scope (will need code adapt or a follow-up phase)
rclcpp_lifecycle::LifecycleNode— the compat shim does not map it. But nano-ros ships its own REP-2002 lifecycle surface (nros/lifecycle.hin C,lifecycle-servicesfeature, state machine + lifecycle services — see the C API reference andexamples/native/rust/lifecycle-node/), so port to that rather than hand-rolling configure/activate bookkeeping on a plainNode.- Yaml-loaded parameters.
declare_parameter<T>("name", default)reads from a launch yaml in stock ROS 2. nano-ros has no runtime yaml loader — parameter initials are compile-baked from the launch XML’s<param name="…" value="…"/>entries (RFC-0004 §10), then live in a volatile store the standard parameter services can update until the next boot. Move yaml values into the launch file (or expose them as compile-time constants). tf2,image_transport,pluginlib— out of nano-ros scope. Project- specific helpers (autowareuniverse_utils, PX4 uORB shims) are not nano-ros’s to ship; the porting user vendors them or replaces the call sites with rawnros-cppones.
When the port hits a gap
Known open follow-ups: yaml-loaded parameter baking, LifecycleNode
compat, and the in-tree migration of legacy
nros_generate_interfaces(<pkg>) call sites. If your port surfaces a
new gap not covered by the compat header, file an issue — a fix
lands either tree-side (in cmake/compat/ or
packages/api/nros-cpp/) or as a codegen change.
In-tree regression fixtures:
local-msg-package— mixed workspace (workspace + AMENT msg sources) C++ + Rust consumers.cpp-port-minimal-publisher— ROS 2 tutorialminimal_publisher.cppverbatim.rclcpp-compat-smoke— minimal source-compat regression test.topic-state-monitor-port— multi-sub / wall-timer / diagnostic_updater exercise.
Drop your reduced-case node under examples/templates/<your-port>/ and
add it to CI once the gap closes.
ROS 2 Interop
nano-ros communicates with standard ROS 2 nodes using the rmw_zenoh_cpp middleware. Both sides connect to the same zenohd router and exchange CDR-encoded messages with compatible key expressions.
Prerequisites
- ROS 2 installed. For the zenoh path use
ROS 2 Jazzy:
sudo apt install ros-jazzy-rmw-zenoh-cpp. Humble and Iron ship normw_zenoh_cppapt package — on those, either buildrmw_zenohfrom source as a colcon overlay, or use the Cyclone DDS backend instead (directrmw_cyclonedds_cppinterop, no router at all — see Choosing an RMW) - The zenoh router running:
ros2 run rmw_zenoh_cpp rmw_zenohd
Quick Start
Open three terminals:
# Terminal 1: Start the zenoh router
ZENOH_CONFIG_OVERRIDE='listen/endpoints=["tcp/127.0.0.1:7447"];scouting/multicast/enabled=false' ros2 run rmw_zenoh_cpp rmw_zenohd
# Terminal 2: Run the nano-ros talker
nros sync # materialize generated/ bindings (once)
cd examples/native/rust/talker
RUST_LOG=info cargo run
# Terminal 3: Run a ROS 2 listener
source /opt/ros/humble/setup.bash
export RMW_IMPLEMENTATION=rmw_zenoh_cpp
export ZENOH_CONFIG_OVERRIDE='mode="client";connect/endpoints=["tcp/127.0.0.1:7447"]'
ros2 topic echo /chatter std_msgs/msg/String --qos-reliability best_effort
You should see the ROS 2 listener printing messages published by nano-ros:
data: 'Hello World: 1'
---
data: 'Hello World: 2'
---
The Other Direction
ROS 2 publishers also work with nano-ros subscribers:
# Terminal 2: Run the nano-ros listener instead
cd examples/native/rust/listener
RUST_LOG=info cargo run
# Terminal 3: ROS 2 talker
source /opt/ros/humble/setup.bash
export RMW_IMPLEMENTATION=rmw_zenoh_cpp
export ZENOH_CONFIG_OVERRIDE='mode="client";connect/endpoints=["tcp/127.0.0.1:7447"]'
ros2 topic pub /chatter std_msgs/msg/String "{data: 'Hello from ROS 2'}" --qos-reliability best_effort
Discovery
nano-ros publishes ROS 2-compatible liveliness tokens, so standard ROS 2 tools work:
source /opt/ros/humble/setup.bash
export RMW_IMPLEMENTATION=rmw_zenoh_cpp
export ZENOH_CONFIG_OVERRIDE='mode="client";connect/endpoints=["tcp/127.0.0.1:7447"]'
ros2 topic list # Shows /chatter
ros2 topic info /chatter # Shows publisher/subscriber count
ros2 node list # Shows nano-ros nodes
Configuration
Domain ID
Both sides must use the same ROS domain ID. On the native (host) build
nano-ros reads ROS_DOMAIN_ID from the environment (default: 0). On
embedded targets the domain ID is fixed at compile time — per-example
config.toml / package.xml deploy metadata, or CONFIG_NROS_DOMAIN_ID
on Zephyr (see Workflow):
ROS_DOMAIN_ID=42 cargo run # nano-ros side
ROS_DOMAIN_ID=42 ros2 topic echo /chatter ... # ROS 2 side
QoS
nano-ros defaults to BEST_EFFORT reliability and VOLATILE durability. When using ROS 2 subscribers, match the QoS:
ros2 topic echo /chatter std_msgs/msg/String --qos-reliability best_effort
Without --qos-reliability best_effort, ROS 2 defaults to RELIABLE, which
won’t match the nano-ros publisher’s BEST_EFFORT QoS.
rmw_zenoh Client Mode
By default, rmw_zenoh_cpp uses peer mode and won’t connect to a zenohd
router. Force client mode with:
export ZENOH_CONFIG_OVERRIDE='mode="client";connect/endpoints=["tcp/127.0.0.1:7447"]'
Set this before any ros2 command.
Protocol Details
nano-ros uses the same wire format as rmw_zenoh_cpp:
- Data key expression:
<domain_id>/<topic>/<type>/TypeHashNotSupported(Humble). Example:0/chatter/std_msgs::msg::dds_::String_/TypeHashNotSupported - Liveliness tokens:
@ros2_lv/<domain>/<zid>/0/...for discovery - Message encoding: CDR little-endian with 4-byte encapsulation header
[0x00, 0x01, 0x00, 0x00] - RMW attachment: 33-byte metadata (sequence number, timestamp, GID) appended via Zenoh attachment
ROS 2 Editions
Three editions ship as Cargo features / --ros-edition values (RFC-0056):
ros-humble (default; TypeHashNotSupported key expressions, XCDR1),
ros-iron, and ros-jazzy (both RIHS01 type hashes). Pick the edition
matching the ROS 2 install you interop with — a Humble-edition image
against a Jazzy bus (or vice versa) discovers nothing, because the key
expressions differ.
Troubleshooting
Topic not visible in ros2 topic list:
Ensure both nano-ros and ROS 2 use the same domain ID and that
rmw_zenoh_cpp is in client mode pointing to the same router.
Discovery works but no messages received:
Check that the QoS matches. Use --qos-reliability best_effort on the
ROS 2 side.
rmw_zenoh not connecting:
Verify ZENOH_CONFIG_OVERRIDE is set. Without it, rmw_zenoh uses peer mode
and won’t find the router.
Next Steps
- Architecture — understand the layer model
- RMW Zenoh Protocol — full wire format reference
Differences from standard ROS 2
Coming from rclcpp, rclrs, or rclc? This page calls out where
nano-ros looks the same, where it diverges, and the reason behind each
choice. It is an orientation page — the per-language API references
(Rust / C /
C++) cover the surface itself.
The short version: nano-ros keeps ROS 2’s vocabulary (Node, Publisher, Subscription, Service, Client, ActionServer, ActionClient, Timer, Executor, QoS, message codegen) so existing nodes port cleanly. The trade-offs that come from running on a Cortex-M3 with 64 KB of heap shape every place where the surface diverges.
Same vocabulary, same wire format
- ROS 2 entity model unchanged. A node owns publishers, subscriptions, services, clients, action servers, action clients, timers, and parameters.
- Topic / service / action names follow ROS 2 conventions
(
/talker_node/chatter,/add_two_ints,/fibonacci). - Message types stay rosidl-shaped (
std_msgs/msg/Int32,geometry_msgs/msg/Twist, …). CDR encoding on the wire. - Default backend (
nros-rmw-zenoh) is bit-compatible with the upstreamrmw_zenohROS 2 RMW. A nano-ros publisher and anrclcppsubscriber on the same zenohd router exchange messages without a bridge — see ROS 2 Interoperability.
Where it diverges, and why
1. The executor is the entry point
Standard ROS 2 starts with a global init (rclcpp::init /
rclrs::Context::default_from_env / rclc_support_init) followed by
node creation, then optionally an executor.
nano-ros inverts this: an Executor opens the RMW session and owns the
runtime budget. Every node, publisher, subscription, service, client,
action handle, and timer is allocated out of the executor’s arena.
Executor::open(&config) → Node → Publisher / Subscription / Service / Client / Timer
Why. The arena is fixed-size and known at compile time
(NROS_EXECUTOR_ARENA_SIZE / NROS_EXECUTOR_MAX_CBS). On a 64 KB
heap MCU we cannot afford the indirection of a global allocator behind
every create_publisher call. The executor-as-arena pattern moves the
size negotiation up to the application’s startup code, where it
belongs.
2. Both manual-poll and callback paths are first-class
rclcpp is callback-only — every subscription needs a callback, the
executor dispatches them. rclrs <0.7 was manual-poll only. rclc
exposes both but treats manual-poll as the second-class path.
nano-ros treats them as equals. A subscription created via
node.create_subscription(...) exposes take(). A subscription
registered via executor.node_mut(nid).create_subscription::<M, _>("/topic", callback)
runs the callback during spin_once. Pick whichever fits the control loop.
Why. Embedded apps often want the predictability of an explicit poll inside a real-time loop. Callback dispatch is great for event-driven services but adds indirection that real-time engineers have to bound by hand. Offering both means the application picks.
3. Async pub/sub/service/action
rclcpp has no real async/await. rclrs 0.7+ added async but it
sits next to the synchronous executor, not in it. rclc has none.
nano-ros has it as a first-class path: Executor::spin_async() wakes
on RMW I/O, Subscription::recv().await, Client::call().await,
ActionClient::send_goal().await, etc. Runs on tokio (POSIX),
Embassy (FreeRTOS / RTIC), or any external Future-driver.
Why. Robotic control loops are stitched together from many concurrent waits — sensor data, service replies, action feedback, parameter updates. Async lets you write them as one straight-line function instead of hand-rolling state machines on top of a poll-based executor.
4. Heap is optional
ROS 2 RMW implementations all assume std + a heap. Even rclrs’s
no_std story is “soon”.
nano-ros runs in three modes that map onto target capability:
| Mode | Cargo features | What works |
|---|---|---|
std | std (default on POSIX) | Everything. POSIX threading, full async runtime. |
no_std + alloc | alloc + a #[global_allocator] | Everything except features that need std::sync::Mutex. Used by FreeRTOS / NuttX / ThreadX / Zephyr / ESP32. |
no_std cooperative | no std/alloc platform threading (bare-metal board crates, RTIC apps) | Cooperative single-task — no threading at all. Used by bare-metal MPS2-AN385, single-core RTIC. |
Why. Heap presence is not a binary “embedded yes/no” — it is a spectrum. Stm32-class boards have a heap; Cortex-M0+-class might not. The feature axis lets the same application code target both.
5. Backend selection at compile time, not runtime
Standard ROS 2 uses an RMW_IMPLEMENTATION env var read at process
start. The plugin loader pulls a shared library, dispatches calls
through C function pointers.
nano-ros bakes the backend in at compile time. The consuming
Cargo.toml adds the backend crate directly (nros-rmw-zenoh /
nros-rmw-cyclonedds / nros-rmw-xrce-cffi) alongside nros with the
rmw-cffi feature; CMake options (-DNANO_ROS_RMW=zenoh) decide it
for C/C++ builds. The backend’s #[ctor] registers its vtable with
the nros-rmw-cffi runtime registry before main.
Why.
- Dead-code elimination. A 32 KB Flash budget cannot afford to
link every backend’s C client and pick at runtime. Linking only the
selected backend cuts the binary substantially (the three backends
differ by an order of magnitude in linked C footprint — measure your
own image with
size; no benchmarked figure exists). - No plugin loader. Most embedded targets have no
dlopen. The cost of the plugin abstraction is a permanent overhead with no payoff there. - Cross-compile sanity.
RMW_IMPLEMENTATIONbaked into the binary means the build system already knows which backend’s C client to link — no separate “find shared library at runtime” step.
The trade-off is real: changing backends requires a rebuild. This is the right trade-off for the embedded use case; it would be the wrong trade-off for desktop ROS 2.
A binary can still link multiple backends and run them in
parallel — one node on backend A, another on backend B,
the executor draining both each tick. The
Cross-backend Bridges
chapter walks the build knobs (NROS_RMW=... primary
selector, explicit per-backend register(), NANO_ROS_RMW= none cmake escape hatch) and the three shipped examples.
6. Message codegen lands inside your build, not a sibling library
Standard ROS 2 uses ament + rosidl to compile message packages
(std_msgs, geometry_msgs, …) into separate shared libraries that
your application links against.
nano-ros’s nros generate-rust (Rust) and
nano_ros_generate_interfaces() (C / C++ via CMake — one of five
entry points; see the spelling table) write message
type definitions into your build tree. No _msgs library, no ament
overlay, no colcon workspace required.
Why. Embedded cross-builds without a hosted ROS 2 install need to
generate message types from package.xml + .msg files alone. The
codegen tool ships its own bundled rosidl-flavoured .msg set inside
the nros CLI (built in-tree from packages/cli/, Phase 218), so you
don’t even need the upstream message packages on disk.
7. QoS profile is the full DDS field set; backends advertise per-policy support
Standard ROS 2 supports the full DDS QoS profile family
(reliability, durability, history, depth, deadline,
lifespan, liveliness, liveliness_lease_duration,
avoid_ros_namespace_conventions) and performs profile matching
between endpoints.
nano-ros’s rmw_qos_profile_t carries the same field set; standard
profile constants (NROS_RMW_QOS_PROFILE_DEFAULT, _SENSOR_DATA,
_SERVICES_DEFAULT, _PARAMETERS, _SYSTEM_DEFAULT) match
upstream rmw_qos_profile_* field-for-field. ROS 2 apps porting
across pull the equivalent constant unchanged.
Each backend advertises which policies it can enforce via
Session::supported_qos_policies(). The runtime validates the
requested QoS at entity-create time and returns
IncompatibleQos synchronously when the backend can’t honour
a requested policy:
#![allow(unused)]
fn main() {
if session.supported_qos_policies().contains(QoSPolicyMask::DEADLINE) {
// backend honours deadline; safe to set deadline_ms
} else {
// app handles deadline monitoring itself
}
}
No silent downgrade. The runtime never quietly drops a requested policy. Apps either get the QoS they asked for or a hard error.
Why upstream-shaped struct, not a smaller subset. ROS 2 QoS is the established vocabulary; mismatched APIs make porting painful. The field set is small (24 bytes); apps that don’t request a policy leave its field at zero (“off”). Per-backend implementation is a separate question — which policies actually fire — answered by the support mask.
Why synchronous error instead of runtime event. Upstream’s
RMW_EVENT_REQUESTED_INCOMPATIBLE_QOS event surfaces mismatches
at run time. Most QoS mismatches are configuration errors visible
at startup; the runtime path doesn’t need to handle them. The few
that aren’t (cross-process QoS-mismatched discovery) the wire
protocol handles itself — DDS endpoints negotiate via DDS Discovery,
zenoh endpoints communicate intent through the topic-key encoding.
Manual liveliness assertion. Publishers configured with
MANUAL_BY_TOPIC / MANUAL_BY_NODE liveliness call
assert_liveliness() explicitly to refresh the lease. Available on
every language surface (Rust Publisher<M>::assert_liveliness(), C
nros_publisher_assert_liveliness(&pub), C++
pub.assert_liveliness()). The zenoh backend wires it: a shim-side
keepalive lease that assert_liveliness() refreshes, with
LivelinessLost fired on expiry. Backends without manual-assertion
wiring (xrce, cyclonedds) treat the call as a no-op. See Status events for the runtime-event
side of liveliness, deadline, and message-lost.
Per-backend coverage is documented in RMW vs upstream § 7.
8. No runtime backend swap, no runtime introspection
Standard ROS 2 ships ros2 topic list, ros2 node info, dynamic
endpoints discovery, rmw_get_* introspection.
nano-ros has none of that at runtime. The backend is fixed at compile
time, the wire-protocol introspection is whatever the backend natively
exposes (zenoh-pico’s z_query for SPDP, Cyclone DDS’s SPDP/SEDP
discovery, …). Use the host-side ROS 2 tools for introspection and connect via
the rmw_zenoh interop path.
Why. Every byte of “introspect what’s running” is overhead a microcontroller can’t justify when a host-side ROS 2 environment is one router-hop away.
9. Parameters: node-local server, no descriptors, no callbacks (yet)
Standard ROS 2 (rclcpp::Node) ships a rich parameter surface:
declare_parameter<T> with ParameterDescriptor (description, ranges,
read-only, dynamic typing), set_parameter returning a
SetParametersResult, atomic multi-set, three callback hooks
(pre_set / on_set / post_set), parameter overrides from the
launch file or CLI, and a service-backed remote-introspection surface
(/<node>/get_parameters, /<node>/set_parameters, …).
nano-ros’s nros::ParameterServer<Cap> (C++) and the equivalent C
nros_parameter_server_t keep the vocabulary (declare_parameter<T>,
get_parameter<T>, set_parameter<T>, has_parameter) but trim the
surface aggressively for embedded use.
What we keep
- Same five scalar types:
bool,int64_t,double, string, plus thebool/int64_t/double/byte/stringarray variants on the C side (nros_parameter_*_array). - Same lifecycle: declare → get → set, with declare-once-then-typed-get semantics.
- Optional service-backed exposure (
~/get_parameters/~/set_parameters/~/list_parameters/ …) when theparam-servicesfeature is enabled. This pulls in ROS 2 wire compat: declared parameters are visible toros2 param list /<node>andros2 param set.
What we drop, and why
| Upstream feature | nano-ros status | Why dropped |
|---|---|---|
ParameterDescriptor (description, ranges, read-only, dynamic_typing) | not exposed | descriptor metadata is host-side concern; embedded server enforces type at declare-time, range checks belong in set callbacks (deferred — see below) |
add_pre_set_parameters_callback / add_on_set_parameters_callback / add_post_set_parameters_callback | one combined nros_parameter_callback_t (server-wide, fires after set) | three callbacks → three indirection slots × N subscribers; one callback covers the safety-island validation use case (reject if out of range) |
set_parameter returning SetParametersResult (successful: bool, reason: string) | returns nros_ret_t | string reason would force heap or fixed-buffer; ret code captures the binary outcome |
set_parameters_atomically | not exposed | atomic multi-set requires transaction log; not justified by current embedded use |
declare_parameters (multi-declare with namespace) | not exposed | one-by-one declare is fine for compile-time-known parameter sets |
| Parameter overrides from CLI / launch / yaml | not exposed | embedded apps configure via Kconfig / Config struct; runtime overrides come over the wire via ~/set_parameters (when param-services is on) |
| Storage allocation policy | compile-time <Capacity>, inline storage | no heap; capacity sizing belongs in the application’s startup code, same as the executor arena |
Storage shape difference
| rclcpp | nano-ros | |
|---|---|---|
| Container | std::map<string, ParameterValue> (heap) | nros_parameter_t storage[Capacity] (caller-owned, inline) |
| String value | std::string (heap) | fixed 128-byte slot, copy semantics |
| Array params | std::vector<T> (heap) | caller-owned pointer + length (caller keeps storage alive) |
| Total fixed cost | unbounded | Capacity × sizeof(nros_parameter_t) known at compile time |
Class shape difference
rclcpp::Node owns the parameter store. nano-ros splits them:
// rclcpp
auto node = std::make_shared<rclcpp::Node>("ctrl");
node->declare_parameter<double>("ctrl_period", 0.15);
double v = node->get_parameter("ctrl_period").as_double();
// nano-ros
nros::Node node;
nros::ParameterServer<8> params;
NROS_TRY(nros::Node::create(node, "ctrl"));
NROS_TRY(params.declare_parameter<double>("ctrl_period", 0.15));
double v;
NROS_TRY(params.get_parameter<double>("ctrl_period", v));
Why split. Adding a parameter store to Node would require
templating Node on capacity, which propagates through every
create_publisher / create_subscription site. Composing
ParameterServer<N> alongside the node keeps Node non-templated and
matches the rest of the freestanding C++14 surface (callers own
storage). params.raw() exposes the underlying
nros_parameter_server_t* for future ROS 2 service-backed registration.
Why no Box<dyn FnMut> callback yet. The same constraint that
shapes the QoS event callbacks applies here: nano-ros’s
#[no_std] core forbids alloc-style indirection. A future
descriptor-+-validation-callback path will use a function pointer +
void* user_context pair, registered at declare-time. Tracked under
the upstream parity backlog.
Going further. When upstream’s full parameter surface matters —
describe_parameter, ranges, three-stage validation callbacks,
override files — fall back to running a host-side ROS 2 node that
exposes them and uses the embedded node only as the leaf publisher /
subscriber.
What this means in practice
If you are coming from rclcpp:
- Open an
Executor, then create the node from it. - Decide poll vs. callback per subscription, not globally.
- If the platform has
std,nros::init()looks identical; if it is RTOS / bare-metal, plan the executor arena up front. - Pick
nros-rmw-zenohfor ROS 2 interop; everything else is a different trade-off.
If you are coming from rclrs:
- The umbrella crate is
nros, not split intorclrs_*.nros::preludegives you everything. Executor::open(&config)is the equivalent ofContext::default_from_env()+Executor::new(...).- The async surface is on the executor + handles themselves (re-exported at the crate root). Compatible with tokio out of the box.
If you are coming from rclc:
- Same C names where they map (
nros_node_init,nros_publisher_init,nros_subscription_init). Memory ownership rules are the same — the caller owns storage, the API initialises it. - See the C API reference for the full surface.
Going deeper
- API surface, type by type → per-language references at Rust / C / C++.
- Why the executor / RMW / platform layers split this way → Architecture Overview.
- Cooperative
no_stdmodel → no_std Support.
nano-ros vs micro-ROS
The closest peer project to nano-ros is micro-ROS. Both ship a full embedded ROS 2 framework — client library + RMW + tooling targeted at MCU-class hardware. This page is an apples-to-apples comparison.
Why not compare to rmw_zenoh or ros2_rust? Scope mismatch.
rmw_zenohis a single RMW backend; nano-ros is a full client stack.ros2_rust(rclrs) targets hosted Linux only — it can’t run on the bare-metal or RTOS targets nano-ros and micro-ROS both address. Comparing nano-ros to either would be misleading.
Side-by-side
| Axis | nano-ros | micro-ROS |
|---|---|---|
| Project home | NEWSLab NTU | ROS 2 ecosystem (community + Bosch + eProsima) |
| First release | 2024 | 2019 |
| Primary language | Rust (with C + C++ bindings) | C (rclc) |
| User-facing APIs | Rust + C + C++ | C only (rclc); experimental C++ in rclcpp_lite |
| RMW backend choice | Zenoh, XRCE-DDS, Cyclone DDS — pick at compile time | XRCE-DDS only |
| Network model | Peer-to-peer (Zenoh / Cyclone DDS) or agent-based (XRCE) | Agent-based only |
| Bridge process required? | No for Zenoh / Cyclone DDS; yes for XRCE | Yes (Micro-XRCE-DDS Agent) |
| Supported RTOSes | FreeRTOS, NuttX, ThreadX, Zephyr, ESP-IDF, PX4 (NuttX), POSIX, bare-metal | FreeRTOS, NuttX, Zephyr, ESP-IDF, POSIX; PX4 is the canonical deployment |
no_std core | Yes — entire client stack compiles no_std + heapless | No — rclc requires libc + a heap |
| Heap usage | Optional on bare-metal (XRCE backend is fully static); required for Zenoh / Cyclone DDS | Required (malloc-based DDS-XRCE client) |
| RT scheduling story | SchedContext classes (Fifo/Edf/Sporadic live in dispatch; TT deprecated-cooperating) + per-platform kernel capabilities — the generated Scheduling Wiring Matrix is the per-cell truth | rclc executor with priority callbacks; no SchedContext / EDF / TT story |
| Multi-executor preemption | Executor::open_threaded per-RTOS via PlatformScheduler trait | Single executor per process |
| Multi-backend bridge in one binary | Yes — Executor::open_with_rmw + multi-Node | No (single XRCE session per process) |
| Discovery | Zenoh liveliness, RTPS SPDP, XRCE-via-Agent | XRCE-via-Agent |
| QoS support | Backend-dependent — per-policy table in RMW vs upstream §7 | Subset of XRCE QoS |
| Formal verification | 148 Kani harnesses + 83 Verus proofs (CDR, scheduling, RMW glue) | None published |
| E2E safety protocol | CRC-32/ISO-HDLC + sequence tracking, EN 50159-mapped (safety-e2e feature) | None |
| ROS 2 distro coverage | Humble, Iron, Jazzy (ros-humble/ros-iron/ros-jazzy features, RFC-0056) | Humble, Iron, Jazzy |
| Build system | Cargo + CMake + platform tools (west, idf.py, probe-rs) plus just recipes; C/C++ consume via add_subdirectory(<repo>) | colcon + CMake; per-RTOS meta-build (create/configure/build/flash_firmware.sh) |
| Deploy/config model | Entry packages select board/RMW/deploy shape; Bringup packages own launch topology; platform tools build and flash | colcon.meta (hand-tuned static sizing) + configure_firmware.sh -t <transport> flags + hand-coded rclc app |
| Host-side broker | none (Zenoh P2P / Cyclone DDS brokerless); Agent only for XRCE | Micro-XRCE-DDS Agent always required |
| Release model | Source-only (no crates.io, no precompiled binaries) | Source-only + per-distro Debian packages |
| Code-size (Cortex-M XRCE talker) | ~75 KB flash (XRCE), ~100 KB+ (Zenoh) — order-of-magnitude, not benchmarked | ~30–50 KB (XRCE + rclc, as commonly quoted upstream — not measured here) |
| License | MIT OR Apache-2.0 (dual) | Apache-2.0 |
| Governance | Single-academic-lab maintainership today | ROS 2 community + corporate stewards |
| Commercial support | None as of writing | Bosch + eProsima offer services |
Pick nano-ros when…
- You want Rust as a first-class API, not bolted on. Memory safety + ownership semantics extend to your application code.
- You want multi-backend flexibility — same binary can speak Zenoh + DDS, or you want to pick Zenoh’s peer-to-peer model over XRCE’s agent dependency.
- You need scheduling primitives beyond priority callbacks —
EDF / Sporadic / ARINC-653 TT classes, per-callback runtime
accounting, formal
SchedContextAPI. - You’re targeting safety-aware deployments that benefit from the E2E CRC + sequence tracking and the Kani / Verus harness coverage.
- You want a lean source-only consumption model:
git clone+add_subdirectory(<repo>). No crates.io drift, no pre-built per-distro binaries.
Pick micro-ROS when…
- You’re already in the micro-ROS ecosystem (existing rclc code, established Agent deployment, ROS 2 Jazzy / Iron support).
- Your toolchain is C-only and you don’t want to introduce Rust into the build pipeline.
- Your target’s flash budget is tight (~30 KB ceiling) — the rclc + XRCE client is smaller than nano-ros + Zenoh.
- You need commercial support contracts today (Bosch, eProsima, PIWeb).
Migration sketch
If you’re porting from micro-ROS to nano-ros:
rcl_node_init→Executor::create_node(Rust) /nros_executor_node_init(C).rclc_executor_t→nros::Executor.rclc_publisher_init_default→Node::create_publisher::<M>ornros_publisher_init.- micro-ROS’s
rmw_uros_set_custom_transport→ nano-ros’s custom transport pattern vianros_platform_*C ABI (see Custom Transport). - XRCE Agent deployment stays the same — point nano-ros’s XRCE backend at the same Agent.
See also
- Build / config / deploy workflow comparison — historical comparison of the three workflow axes vs micro-ROS, Zenoh-pico, embedded DDS, and Arduino-ROS.
- Choosing an RMW Backend — the backend capability matrix.
- Production Readiness Checklist — pilot-deployment validation.
- Supported Boards — per-board status.
- Migration Guide — for porters coming from rclcpp / rclrs / rclc.
Migration Guide for ROS 2 Users
This page maps standard ROS 2 concepts to their nano-ros equivalents.
It is not an API reference; use it as a checklist when moving an
existing rclcpp, rclc, or rclrs node toward nano-ros.
Concept map
| You have (ROS 2) | You get (nano-ros) | Where |
|---|---|---|
colcon workspace (src/*) | nano-ros workspace: Node pkgs + one declarative Bringup pkg + one Entry pkg per deploy target | Project layout |
package.xml | kept, verbatim — plus a <nano_ros deploy=…/> export for embedded targets | Anatomy |
.launch.xml / .launch.py | kept — nros sync resolves them (RFC-0060) into a SystemModel the Entry binary bakes in | Bringup packages |
ros2 launch pkg file | run the Entry binary (cargo run -p <entry> / ./build/src/<entry>/<entry>) — the launch product is compiled in | Entry packages |
colcon build | cmake / cargo per workspace root; colcon build still works for POSIX C++ workspaces | C/C++ workspaces |
rosdep install | nros setup <board> --rmw <rmw> | Install |
RMW_IMPLEMENTATION=… at runtime | compile-time backend: -DNROS_RMW=… / cargo feature | Switching RMW |
| generated msg packages (ament index) | nros sync writes generated/ crates + the cargo patch table | Message Generation |
find_package(rclcpp) | find_package(nano_ros) — ament-shaped, source-backed | Porting a C++ node |
ament_target_dependencies(t std_msgs) | kept, verbatim (compat shim + Find-stubs) | same page |
Setup
Standard ROS 2 usually starts from a distro install and a runtime RMW choice. nano-ros starts from a source checkout + a compile-time target tuple:
git clone --branch=v<X.Y.Z> https://github.com/NEWSLabNTU/nano-ros.git
cd nano-ros
./scripts/bootstrap.sh # build the nros CLI (no `just` needed)
source ./activate.sh
nros setup native --rmw zenoh # native/ROS/zenoh quick start
Read Setup Compared to Standard ROS 2 before changing package code.
Node Lifecycle
ROS 2 applications typically call rclcpp::init(), create nodes, then
spin. nano-ros opens an executor first; the executor owns the session,
arena, and runtime budget. Nodes and entities are created from it.
See Differences from Standard ROS 2 and Execution Model and Two-Layer API.
Publishers and Subscriptions
Topic names, message names, and CDR wire encoding stay ROS-shaped. The main porting decision is whether to use:
- polling handles from
Node::create_*, or - callback registration through
Executor::register_*.
Use polling for RTIC, Embassy, or tight RT loops. Use callbacks for desktop-style event-driven nodes.
Services and Actions
Service and action names map cleanly, but nano-ros exposes request / reply and goal / feedback / result paths through explicit handles and promises. Manual-poll paths may require explicit result handling in RTOS loops.
Start with the native examples, then check platform-specific examples for FreeRTOS, Zephyr, or bare-metal timing constraints.
QoS and Events
nano-ros keeps DDS-shaped QoS profile fields, but each backend advertises the policies it can enforce. Unsupported QoS is reported at entity creation instead of being silently downgraded.
See QoS, Status Events, and Discovery and Choosing an RMW Backend.
Message Generation
Standard ROS 2 builds generated message libraries as sibling packages. nano-ros generates Rust, C, or C++ bindings into the workspace or build tree and can use a shared generation cache.
See Message Binding Generation.
Backend Selection
Replace runtime RMW_IMPLEMENTATION=... with a compile-time selection:
zenohfor ROS 2 interop throughrmw_zenoh_cpp(default).xrcefor agent-based micro-ROS-style deployments.cycloneddsfor direct DDS/RTPS interop withrmw_cyclonedds_cpp— no router or agent process at all — where the platform supports the required networking and memory model.
Common Porting Traps
- Assuming the backend can change without rebuilding.
- Creating heap-heavy callbacks on
no_stdtargets withoutalloc. - Expecting ROS 2 graph introspection APIs on constrained targets.
- Forgetting to match ROS domain ID, QoS reliability, or zenoh router mode during interop tests.
- Treating a platform guide as optional when cross-compiling for RTOS or bare-metal targets.
Architecture Overview
nano-ros is a ROS 2 client stack for embedded and RTOS targets. The important architectural idea is separation by responsibility: user code talks to a ROS-shaped API, the core runtime owns entities and serialization, the RMW layer moves bytes, and the platform layer supplies OS or hardware primitives.
Layers
Application
Rust / C / C++ node code
Board package (optional)
Hardware init, network drivers, config loading, entry point
Core runtime
Executor, Node, pub/sub/service/action handles, parameters,
message traits, CDR serialization
RMW backend
Zenoh, XRCE-DDS, Cyclone DDS, or a custom backend
Platform
Clock, allocation, threading, sleep, random, sockets, libc,
and bare-metal network polling
POSIX applications usually depend directly on nros. Embedded
applications often depend on a board package that initializes hardware,
networking, and platform glue before running user code.
Core Runtime
The core runtime is middleware-agnostic. nros-node owns the
Executor, node creation, entity handles, timers, and the two API
styles:
Node::create_*returns handles that the caller polls or awaits.Executor::register_*installs callbacks dispatched byspin_once.
Message and service types are ordinary generated Rust, C, or C++ types with CDR serialization. The RMW backend receives serialized bytes; it does not own rosidl typesupport.
For the API split, see Execution Model and Two-Layer API.
RMW Layer
The RMW layer is the transport boundary. It creates sessions, publishers, subscribers, services, clients, and action channels, then moves serialized samples over the selected wire protocol.
Each Node picks its backend at build time. That compile-time selection
replaces standard ROS 2’s runtime RMW_IMPLEMENTATION plugin loader,
which is not available on many embedded targets. A single binary can
link multiple backends and bridge between them — see
Cross-backend Bridges.
For user-facing backend selection, see Choosing an RMW Backend. For design rationale, see RMW API Design.
Platform Layer
The platform layer supplies the primitives that desktop ROS 2 normally
gets from the operating system: monotonic time, memory allocation,
threading, sleep, random IDs, TCP/UDP sockets, multicast, and libc
helpers. Bare-metal ports may also expose a network_poll() hook so the
runtime can advance smoltcp while waiting.
The platform is selected at compile time together with the RMW backend and ROS edition. See Platform Model for the feature axes and Custom Platform for porting.
Board Packages
A board package combines a platform implementation with hardware setup and drivers. It typically provides:
- a
Configtype loaded fromconfig.tomlor target-specific build settings, - network and clock initialization,
- driver setup for Ethernet, UART, WiFi, or simulator I/O,
- a
run()entry point that starts the scheduler or main loop.
Board packages are optional for POSIX but useful for RTOS and bare-metal targets. See Custom Board Package.
Data Flow
Publishing follows this path:
user message -> CDR serializer -> RMW publish bytes -> transport
Receiving reverses it:
transport -> RMW receive bytes -> CDR deserializer -> user handle/callback
This boundary keeps the transport layer small and lets the same message types work across Rust, C, and C++ APIs.
Where to Go Next
- New user: Setup Compared to Standard ROS 2.
- Application author: Configuration.
- Platform porter: Porting Overview.
- Contributor changing internals: Design Overview.
Two-Layer API
nano-ros exposes the same communication primitives — subscriptions, publishers, services, service clients, action servers, action clients, timers — through two layers with disjoint verbs. Picking between them is a per-entity deployment choice. The two layers share the same session and compose freely.
| Layer 1 — caller polls | Layer 2 — executor dispatches | |
|---|---|---|
| Verb | Node::create_* | Executor::register_* |
| Returns | Owned handle | Handle ID + dispatched closure |
| Receive shape | take() / call() → Promise<T> / try_accept_goal(...) | Closure runs on rx / reply / timer fire |
| Scheduler | Caller (RTIC, embassy, app loop) | Executor::spin_once |
| Typical use | RTIC tasks, single-task FreeRTOS, RT-bounded loops | Callback-shaped apps, multi-handler servers |
Rationale
Embedded callers want both. RTIC owns scheduling and refuses to hand its loop to a library; an embassy task wants to .await a Promise; a FreeRTOS task-per-entity port wants tight take polling. None of those fit the L2 callback model.
But a posix app that handles a dozen topics + a couple of services + an action server wants L2 — write one closure per entity and let spin_once() route events. Forcing it through L1 means writing a manual dispatcher.
The two layers are deliberately separate verb sets so the choice is explicit in source:
node.create_subscription::<M>("/topic")— caller polls. No executor magic.executor.register_subscription::<M, _>("/topic", |m| { ... })— executor dispatches.
The C / C++ FFI mirrors this split: nros_subscription_init (L1) vs nros_executor_register_subscription (L2); nros_subscription_init_polling (L1, inline-storage) vs register_subscription_* (L2, executor arena).
What goes where
graph LR
EX["Executor::open(&config)"] --> NODE["executor.create_node(name)"]
subgraph "L2 — register_*"
direction LR
EX --> R_T["register_timer"]
EX --> R_S["register_subscription"]
EX --> R_SVC["register_service"]
EX --> R_AS["register_action_server"]
EX --> R_AC["register_action_client"]
end
subgraph "L1 — create_*"
direction LR
NODE --> C_P["create_publisher"]
NODE --> C_S["create_subscription"]
NODE --> C_SS["create_service"]
NODE --> C_SC["create_client"]
NODE --> C_AS["create_action_server"]
NODE --> C_AC["create_action_client"]
end
Both layers are wired up by the same call to Executor::open — the register_* family attaches to the executor’s arena, create_* borrows the executor’s session through a short-lived Node. The handles returned by create_* outlive the Node (owned, not borrowed), so the canonical pattern is to scope the Node to creation time only:
let mut executor = Executor::open(&config)?;
let publisher = {
let mut node = executor.create_node("talker")?;
node.create_publisher::<Int32>("/chatter")?
}; // node drops here; publisher outlives it
Picking the right layer
Use L1 (Node::create_*) when:
- The caller has its own scheduling primitive (RTIC dispatcher, embassy task, FreeRTOS task per entity).
- The flow is fundamentally request-response — service-client
Promise<Reply>and action-clientsend_goal→get_resultwork this way. - The receive logic isn’t simple enough to express as a one-shot closure — e.g. a Fibonacci server that publishes feedback in a loop after accepting a goal.
- You want zero allocations and no executor arena overhead (the L2 path stores the closure + its captures inline in the arena; the arena byte budget is per-build-tunable but non-zero).
Use L2 (Executor::register_*) when:
- The flow is event-driven with a simple handler — log every received message, reply to a service synchronously, fire a 1 Hz publish.
- You want
spin_onceto dispatch all handlers from one site (less state to thread through main). - The application is callback-shaped (matches
rclcpp/rclpymental models).
Service clients and action clients stay on L1 in the current code — they have no typed register_*_client<T> API (only register_*_raw byte-level). The typed user path is Node::create_client::<S> + client.call(&req) → Promise<Reply>. The request-response + Promise::await model is the L1 shape.
C / C++ mirror
Both layers cross the FFI cleanly:
- C, L1 polling —
nros_subscription_init_pollingwrites aRawSubscriptioninline innros_subscription_t._opaque.nros_subscription_take_serializedreads from the inline buffer. Same shape for service / service-client / action server / action client. - C, L2 callback —
nros_subscription_initrecords the C callback pointer;nros_executor_register_subscriptionallocates the executor-arena entry. - C++ — typed templates wrap each FFI surface:
nros::Subscription<M>+takefor L1,nros::PollingActionServer<A>for the L1 action path (122.3.d.b), the L2 executor-registered callback model via the existingnros::ActionServer<A>API.
For the per-FFI-function spec, see the Doxygen reference. For the example migration tally (32 examples on L2, 16 intentionally L1), see the unify-api-paths roadmap doc.
Event-driven path (122.3.c.6.e)
L1 polling assumes the caller has a spin loop. RTOS / embassy applications want the kernel to wake them when data lands. The Raw*::register_waker(&Waker) Rust API and the matching *_set_wake_callback(state, cb, ctx) C / C++ FFI plug a wake source into the same L1 handle. RMW backends that support waking (currently zenoh-pico for subscribers, service-clients, and service-servers) fire the callback from their rx path; other backends fall back to polling silently.
For the underlying primitive (ServiceTrait::register_waker in nros-rmw), see the trait doc. For the C wake-callback POD shape (nros_wake_state_t { fn_ptr, ctx }), see the Doxygen reference — same canonical-spec rule as the platform API.
Platform Model
nano-ros configures the library for a specific target through
compile-time choices on three axes. nros itself carries one
generic rmw-cffi runtime registry; the consuming Cargo.toml
adds the chosen backend shim crate directly (one per RMW).
The Three Axes
RMW Backend
The RMW backend determines which middleware transport carries
messages. nros exposes a single rmw-cffi feature — the generic
C-vtable runtime registry. The consuming crate adds the concrete
backend crate as a direct dependency; its #[ctor] registers a
vtable with that registry before main.
| Backend crate | Transport | Description |
|---|---|---|
nros-rmw-zenoh | zenoh-pico | Peer-to-peer via Zenoh protocol; default, ROS-2-interop. |
nros-rmw-xrce-cffi | Micro-XRCE-DDS-Client | Agent-based via DDS-XRCE protocol. |
nros-rmw-cyclonedds | Cyclone DDS | C++ shim; standalone CMake project. |
Unlike the platform axis, RMW is not mutually exclusive — Phase 104 lets one binary register multiple backends (bridge nodes). See Choosing an RMW Backend.
Platform (pick one)
The platform feature selects which OS primitives (threading, mutexes, clock, sleep, network) are linked. Each platform crate provides the symbols that the RMW backend’s C library requires at link time.
| Feature | Target | Threading | Network |
|---|---|---|---|
platform-posix | Linux, *BSD | pthreads | BSD sockets |
platform-zephyr | Zephyr RTOS | k_thread_create | Zephyr sockets |
platform-bare-metal | Cortex-M, RISC-V, ESP32 | Single-threaded | smoltcp |
platform-freertos | FreeRTOS | xTaskCreate | lwIP |
platform-nuttx | NuttX RTOS | pthreads | BSD sockets |
platform-threadx | Azure RTOS / ThreadX | tx_thread_create | NetX Duo |
ROS Edition (pick one)
The ROS edition controls wire-format compatibility with specific ROS 2 releases.
| Feature | Description |
|---|---|
ros-humble | Humble Hawksbill wire format (no type hash) |
ros-iron | Iron Irwini wire format (type hash in key expression) |
Cross-Cutting Features
These features are orthogonal to the three axes above and can be combined freely.
| Feature | Description |
|---|---|
std | Enables std-dependent APIs: spin_blocking(), spin_period(), system clock |
alloc | Enables heap-dependent APIs: boxed callbacks, param-services |
safety-e2e | CRC-32 integrity + sequence tracking (AUTOSAR E2E / EN 50159) |
param-services | ROS 2 parameter service handlers (~/get_parameters, etc.). Implies alloc. |
ffi-sync | Wraps transport FFI calls in critical_section::with() for RTOS reentrancy |
sync-spin | Use spin-lock mutex (default) |
sync-critical-section | Use critical-section mutex (for RTIC, Embassy) |
unstable-zenoh-api | Zero-copy receive path (Zenoh backend only) |
Mutual Exclusivity Enforcement
The platform and ROS-edition axes are mutually exclusive — nros
enforces this at compile time with compile_error!():
#[cfg(all(feature = "platform-posix", feature = "platform-zephyr"))]
compile_error!("Platform features are mutually exclusive — select at most one.");
#[cfg(all(feature = "ros-humble", feature = "ros-iron"))]
compile_error!("`ros-humble` and `ros-iron` are mutually exclusive.");
The RMW axis is not enforced this way — it is the consuming crate’s
choice of which nros-rmw-* dependency to add (one, or several for
bridge nodes).
Boards vs Platforms
Above the platform axis, board crates select a concrete chip + RTOS combination on top of one of the platform features. A board owns the linker layout, the vendor HAL wiring, the boot path, and any chip-specific peripherals (Ethernet PHY, IVC mailboxes, RTC). It does not carry its own platform axis — it pulls in one of the rows above.
| Board crate | Underlying platform | CPU | Notes |
|---|---|---|---|
nros-board-mps2-an385 | bare-metal | Cortex-M3 | QEMU MPS2-AN385 + LAN9118 + smoltcp |
nros-board-esp32-qemu | bare-metal | RISC-V (ESP32-C3) | QEMU ESP32 + OpenETH + smoltcp |
nros-board-orin-spe | freertos | Cortex-R5F | NVIDIA Jetson AGX Orin SPE + FreeRTOS-FSP |
nros-board-orin-spe is the canonical example of the board-over-RTOS
pattern: a Cortex-R5F SPE running NVIDIA’s FreeRTOS V10.4.3 FSP, with
critical-section provided by the canonical
nros_platform_critical_section_{acquire,release} C symbols (Cortex-R
CPSR I-bit body inside the port) — not by a per-CPU Rust feature flag.
See docs/roadmap/phase-100-orin-spe-infra.md for the wiring.
Example Configurations
Each config lists nros (with rmw-cffi + one platform-*) plus
the chosen backend crate. POSIX additionally needs
nros-platform-cffi with posix-c-port for a pure-cargo build.
QEMU bare-metal Cortex-M3 with Zenoh:
[dependencies]
nros = { path = "…/nros", default-features = false, features = [
"rmw-cffi", "platform-bare-metal", "ros-humble",
] }
nros-rmw-zenoh = { path = "…/nros-rmw-zenoh", features = ["platform-bare-metal"] }
Zephyr with XRCE-DDS and safety features:
[dependencies]
nros = { path = "…/nros", default-features = false, features = [
"rmw-cffi", "platform-zephyr", "ros-humble", "alloc", "safety-e2e", "ffi-sync",
] }
nros-rmw-xrce-cffi = { path = "…/nros-rmw-xrce-cffi", features = ["platform-zephyr"] }
Desktop Linux for development and testing:
[dependencies]
nros = { path = "…/nros", default-features = false, features = [
"rmw-cffi", "platform-posix", "ros-humble", "std", "param-services",
] }
nros-rmw-zenoh = { path = "…/nros-rmw-zenoh", features = ["platform-posix", "link-tcp", "ros-humble"] }
nros-platform-cffi = { path = "…/nros-platform-cffi", features = ["posix-c-port"] }
FreeRTOS with lwIP networking:
[dependencies]
nros = { path = "…/nros", default-features = false, features = [
"rmw-cffi", "platform-freertos", "ros-humble",
] }
nros-rmw-zenoh = { path = "…/nros-rmw-zenoh", features = ["platform-freertos"] }
NuttX with XRCE-DDS over serial:
[dependencies]
nros = { path = "…/nros", default-features = false, features = [
"rmw-cffi", "platform-nuttx", "ros-humble",
] }
nros-rmw-xrce-cffi = { path = "…/nros-rmw-xrce-cffi", features = ["platform-nuttx", "xrce-serial"] }
How the pieces link
Phase 208 decoupled nros from concrete backends — nros carries
no RMW crate dependency. Instead:
- RMW. The consuming crate’s
nros-rmw-*dep ships a#[ctor]that callsnros_rmw_<name>_register()at lib load (POSIX.init_array) or the caller invokes it frommain(bare-metal). The backend installs anros_rmw_vtable_tintonros-rmw-cffi’s named registry.Executor::openresolves the registered backend; the concrete session type is alwaysCffiSession(vtable-backed). - Platform.
nros’splatform-*feature resolves tonros-platform-cffi. The canonicalnros_platform_*C symbols ship frompackages/platform/nros-platform-<plat>/src/*.c, linked at the consumer’s build site —posix-c-portfor pure-cargo POSIX builds, the standalonelib<…>.afor CMake builds, or the kernel build system for NuttX / Zephyr. - Alias TUs. C translation units forward each transport C library’s
z_*/uxr_*platform calls to the canonicalnros_platform_*symbols:zpico-sys/c/zpico/platform_aliases.c(default-onplatform-aliasesfeature) for zenoh-pico, andnros-rmw-xrce/src/platform_aliases.c(always compiled intonros-rmw-xrce-cffi) for XRCE-DDS. (These replaced the formerzpico-platform-shim/xrce-platform-shimcrates, deleted in Phase 129.)
The default feature set is std only. No RMW backend or platform is
selected by default — the consuming crate chooses explicitly.
For implementation details on how to add a new platform, see the Porting Guide and Platform API Reference.
Board Integration
How you consume nano-ros depends on what your project’s build system looks like, not on what RTOS you’re targeting. This page maps user profile → recommended path. Pick your row, follow the linked guide.
The architecture behind the matrix lives in
docs/design/0012-board-bsp-integration-architecture.md
(layered model, vendor-BSP scaling, plan).
Consumption matrix
| Profile | Workflow | Recommended path |
|---|---|---|
| Cargo-first Rust user (have RTOS sources) | cargo build --target <triple> produces a single ELF. RTOS kernel built in-tree by Cargo. | Generic board crate + env vars pointing at FREERTOS_DIR / THREADX_DIR / etc. |
| Vendor-IDE user (STM32CubeIDE, MCUXpresso IDE, etc.) | Vendor’s existing FreeRTOS / ThreadX project; nano-ros as an add_subdirectory() library. | add_subdirectory(third_party/nano-ros) from the vendor’s project. |
| Zephyr user (any board) | west + DTS overlays. Vendor HALs come as Zephyr modules. | Zephyr integration shell — projects: entry in your west.yml. |
| ESP-IDF user (any ESP32 chip) | idf.py build. | ESP-IDF integration shell — idf.py add-dependency nano-ros. |
| NuttX user (any board) | NuttX apps/external/ + Kconfig. | NuttX integration shell — symlink under apps/external/. |
| PX4 user | PX4 build pipeline. | PX4 integration shell — EXTERNAL_MODULES_LOCATION. |
| Niche RTOS / vendor fork | Stock RTOS kernel + vendor driver SDK. Cargo-driven build. | Generic board crate + vendor overlay crate (~50 LOC). See the Vendor Overlay cookbook. |
Generic board crate
For Cargo-first users targeting one of the four supported kernel families:
| Kernel | Crate | SDK env vars you set |
|---|---|---|
| FreeRTOS + lwIP | nros-board-freertos | FREERTOS_DIR, FREERTOS_PORT, LWIP_DIR, FREERTOS_CONFIG_DIR |
| ThreadX + NetX-Duo | nros-board-threadx | THREADX_DIR, THREADX_CONFIG_DIR, NETX_DIR, NETX_CONFIG_DIR |
| NuttX | nros-board-nuttx | NUTTX_DIR (kernel built by NuttX itself) |
| bare-metal Cortex-M + smoltcp | (no family crate — copy nros-board-mps2-an385) | BOARD_LINKER_SCRIPT_DIR |
# user_app/Cargo.toml — registry-style names resolved by the
# `nros sync`-managed [patch.crates-io] table (nano-ros is source-only;
# nothing is on crates.io):
[dependencies]
nros-board-freertos = { version = "*", default-features = false }
nros = { version = "*", default-features = false, features = ["rmw-cffi", "platform-freertos", "ros-humble"] }
nros-rmw-zenoh = { version = "*", default-features = false, features = ["platform-freertos"] }
std_msgs = { path = "generated/std_msgs", default-features = false }
#![allow(unused)]
fn main() {
// user_app/src/main.rs — the macro form; it expands to
// `nros_board_freertos::run_entry::<Board, _, _>(...)` dispatch.
#![no_std]
#![no_main]
nros::main!(spin = "forever");
// Manual form, for full control:
// nros_board_freertos::run_entry::<Mps2An385, _, _>(
// Config::default(), None, |runtime| { /* register nodes */ Ok(()) })
}
export FREERTOS_DIR=$HOME/sdk/freertos/kernel
export FREERTOS_PORT=GCC/ARM_CM3
export LWIP_DIR=$HOME/sdk/freertos/lwip
cargo build --release --target thumbv7m-none-eabi
The generic crate’s build.rs compiles the kernel + network stack
- nano-ros platform glue into a single ELF. No vendor driver glue — that’s what overlay crates are for.
Vendor overlay crate
When your board needs vendor-specific drivers (STM HAL,
NXP fsl_*, NVIDIA FSP, Renesas Synergy SSP, …) on top of one of
the generic crates, write a small (~50 LOC) overlay crate that:
- Depends on the matching generic board crate.
- Defines a board marker type and implements the
nros_platformboard traits on it (BoardInit::init_hardware()and siblings); re-exportsConfig. - Provides strong definitions of the generic crate’s weak C network
hooks (
nros_board_register_netif,nros_board_poll_netif). - Pulls vendor HAL
.csources via its ownbuild.rscc-rs invocation.
The full cookbook + the working precedent
(nros-board-mps2-an385-freertos) live in
Vendor Overlay Board Crate.
Why so many paths
Per the
architecture doc,
each RTOS already has its own package manager (Zephyr’s
west + DTS, ESP-IDF’s component registry, NuttX’s
apps/external/, PX4’s EXTERNAL_MODULES_LOCATION). nano-ros
rides those rails instead
of trying to re-invent a single “embedded library” mechanism that
fits no vendor ecosystem cleanly.
The Cargo-first profile (with optional vendor overlay crates) covers the gap where a user is NOT inside an existing RTOS-IDE project and wants to drive their build from Cargo end-to-end. That’s the two-and-a-half rows at the top + bottom of the matrix.
Not on the matrix
- No common driver HAL. STM
HAL_*, NXPfsl_*, Espressifesp_*, RenesasR_*all stay vendor-owned. Overlay crates wrap them; nano-ros doesn’t abstract over them. - No DTS-equivalent for non-Zephyr platforms. Zephyr owns its
board contract; everywhere else, board config is whatever your
vendor IDE produces (CubeMX
.ioc, NuttXdefconfig, ESP-IDFsdkconfig). - No board crate per SKU. Generic + overlay covers the long tail. If you have an exotic board with custom HAL, write a ~50 LOC overlay; nano-ros doesn’t catalog every vendor SKU.
Related reading
- Vendor Overlay Board Crate — the overlay cookbook.
- Custom Board Package — older guide covering monolithic board crates (legacy pattern).
add_subdirectory(third_party/nano-ros)— root CMake entry for vendor-IDE consumers.- Platform Model — Boards vs Platforms;
Layer 1’s
<nros/platform_*.h>contract overlays sit on top of. - RTOS Cooperation — what the runtime expects from the OS underneath the platform layer.
std and alloc Requirements
This document maps which parts of the nano-ros API require the std or alloc
Cargo features. All core crates are #![no_std] by default and gate
std/alloc-dependent code behind feature flags.
Feature Hierarchy
std (default)
└─ alloc
└─ (base no_std)
Enabling std automatically enables alloc. Enabling alloc does not enable
std. With --no-default-features, the entire library compiles without
std or alloc.
Summary by Crate
| Crate | no_std base | alloc additions | std additions |
|---|---|---|---|
| nros-serdes | CDR ser/de for primitives, heapless types, &str, [T; N] | String and Vec<T> ser/de | (none) |
| nros-core | Time, Duration, Clock (atomic fallback), lifecycle, logger, error types, action types | (none) | Clock::now() via SystemTime, std::error::Error impls |
| nros-rmw | All traits, QoS, sync primitives, safety/E2E protocol | handle_request_boxed() (Box<Reply>) | (none) |
| nros-params | ParameterServer, ParameterValue, all parameter types (heapless) | (none) | ParameterVariant impls for std::string::String, std::vec::Vec |
| nros-node | Executor::open(), create_node(), spin_once(), spin_async(), Promise, pub/sub/service/action, timers (fn pointer callbacks) | Boxed timer callbacks, handle_request_boxed(), parameter services | spin_blocking(), spin_period(), ExecutorConfig::from_env(), halt flag |
| nros | Re-exports from above | (same as above) | SpinPeriodResult re-export |
RMW Backend Crates
| Crate | no_std base | alloc additions | std additions |
|---|---|---|---|
| nros-rmw-zenoh | Zenoh-pico RMW implementation, all pub/sub/service/action ops | (none) | (none) |
| zpico-sys | FFI bindings to zenoh-pico C library | (none) | (none) |
| nros-rmw-xrce | XRCE-DDS RMW implementation, all pub/sub/service/action ops | (none) | (none) |
| xrce-sys | FFI bindings to Micro-XRCE-DDS-Client C library | (none) | (none) |
All four backend crates are unconditionally #![no_std] and do not use alloc. The std
feature gates only extern crate std (for macro availability in transport modules) and
is propagated through the feature chain but does not add any API surface.
Detailed API Availability
Always Available (no_std, no alloc)
Executor and Node:
Executor::open(&config)– open RMW sessionExecutorConfig::new(locator)– manual configurationexecutor.create_node(name)– create a nodeexecutor.spin_once(timeout_ms)– single spin iterationexecutor.spin_period(period)– periodic spin withoutstd::thread::sleep
Two-layer API. Phase 122 unified the verb discipline:
- Layer 1 (caller polls) –
Node::create_*returns an owned handle. Caller drivestake/call/try_accept_goal/take_request_rawitself. Good for RTIC, Embassy, task-per-entity FreeRTOS. - Layer 2 (executor dispatches) –
Executor::register_*takes a closure;spin_oncefires it on rx / reply / timer. Good for callback-shaped applications.
Both layers share the same session; mix per entity.
Publish/Subscribe:
- L1 —
node.create_publisher::<M>(topic),node.create_subscription::<M>(topic)(poll withtake()) - L2 —
executor.register_timer(period, || publisher.publish(...)),executor.register_subscription::<M, _>(topic, |msg| { ... }) publisher.publish(&msg)/publish_raw(&bytes)— publish messages
Services:
- L1 —
node.create_service::<S>(name)(poll withhandle_request()),node.create_client::<S>(name)+client.call(&request)→Promise<Reply>(poll withpromise.take()or.await). - L2 —
executor.register_service::<S, _>(name, |req| reply). Service clients keep the L1Promiseshape; the typed callback API isn’t surfaced (onlyregister_service_client_rawexists for byte-level use).
Actions:
- L1 —
node.create_action_server::<A>(name)+try_accept_goal/complete_goal, ornode.create_action_client::<A>(name)+send_goal→Promise<GoalId>/get_result→Promise<(GoalStatus, Result)>. - L2 —
executor.register_action_server::<A, _, _>(name, goal_cb, cancel_cb)returns a handle for publishing feedback and completing goals. Action clients keep the L1Promiseshape for the same reason as service clients.
Async:
executor.spin_async()– async spin loop (drives I/O, dispatches callbacks, yields between iterations)Promise<'a, T, Cli>– allocation-free promise, borrows client’s reply slotPromise::take()– non-blocking poll for replyPromise: Future– implementscore::future::Futurefor.await- Uses only
core::futureandcore::task– no external async runtime dependency
Timers:
TimerHandlewith function pointer callbacks (fn())TimerDuration,TimerMode,TimerState
Serialization:
CdrWriter/CdrReader– CDR serialization to/from byte buffersSerialize/Deserializetraits- Implementations for:
bool,u8-u64,i8-i64,f32,f64,char,&str,[T; N],heapless::String<N>,heapless::Vec<T, N>
Time:
Time::new(),Time::from_nanos(),Time::to_nanos()Time::from_secs_f64(),Time::to_secs_f64()Duration::new(),Duration::from_nanos(),Duration::to_nanos()Duration::from_secs_f64(),Duration::to_secs_f64()Clockwith atomic counter fallback (no wall-clock time)
Parameters (local only):
ParameterServer– store and retrieve parametersParameterValueenum with heapless collectionsParameterDescriptor,ParameterType,ParameterParameterBuilderfor declaring parameters with constraints
Other:
LifecycleState,LifecycleTransition,LifecyclePollingNodeLogger(usescore::sync::atomic)GoalId,GoalStatus,GoalResponse,CancelResponse,CancelReturnCodeQoSProfile,TopicInfo,ServiceInfoSafetyValidator,IntegrityStatus(withsafety-e2efeature)- Sync primitives:
spin::Mutexorcritical-section(feature-selected)
Requires alloc
| API | Location | Why |
|---|---|---|
Serialize/Deserialize for String, Vec<T> | nros-serdes | Heap-allocated containers |
TimerCallback (Box<dyn FnMut() + Send>) | nros-node/timer.rs | Boxed closure for timer callbacks |
Timer::new_with_box(), set_callback_box() | nros-node/timer.rs | Construct/update boxed timer callbacks |
EmbeddedServiceServer::handle_request_boxed() | nros-node/handles.rs | Returns Box<Reply> for large response types |
param-services feature (all of it) | nros-node/parameter_services.rs | Service response types (~1MB) require heap allocation |
Parameter services detail: The param-services feature (which implies alloc)
provides ROS 2 parameter service handlers for ~/get_parameters,
~/set_parameters, etc. Response types like GetParametersResponse contain
heapless::Vec<ParameterValue, 64> – each ParameterValue is large, making the
total response ~1MB. Box<Response> is required to avoid stack overflow.
The core ParameterServer API works without alloc; only the ROS 2 service
protocol layer requires it.
Requires std
| API | Location | Why |
|---|---|---|
Clock::now() (system/steady clock) | nros-core/clock.rs | Uses std::time::SystemTime / UNIX_EPOCH |
std::error::Error for NanoRosError, RclReturnCode | nros-core/error.rs | Trait requires std |
ExecutorConfig::from_env() | nros-node/types.rs | Uses std::env::var() + Box::leak() |
Executor::spin_blocking(options) | nros-node/spin.rs | Uses std::thread::sleep(), Arc<AtomicBool> |
Executor::spin_period(duration) | nros-node/spin.rs | Uses std::time::Instant, std::thread::sleep() |
Executor::halt_flag() | nros-node/spin.rs | Returns Arc<AtomicBool> for cross-thread cancellation |
SpinPeriodResult | nros-node/types.rs | Contains std::time::Duration |
ParameterVariant for std::string::String, std::vec::Vec | nros-params/types.rs | Convenience conversions for std types |
Typical Configurations
Phase 208 decoupled the nros umbrella from concrete RMW crates.
A consuming Cargo.toml lists three path deps: nros (with
rmw-cffi + a platform-* feature), the chosen backend crate
(nros-rmw-zenoh / nros-rmw-xrce-cffi), and —
on POSIX — nros-platform-cffi with posix-c-port so the C
nros_platform_* symbols link into a pure-cargo build. The backend
crate’s #[ctor] registers its vtable before main.
Bare-metal / RTOS (no allocator):
nros = { path = "…/nros", default-features = false, features = ["rmw-cffi", "platform-bare-metal"] }
nros-rmw-zenoh = { path = "…/nros-rmw-zenoh", features = ["platform-bare-metal"] }
Full pub/sub, services, actions, timers (fn pointers), parameters (local).
Async: spin_async(), Promise, take(), .await – all available without std or alloc.
Use spin_once() or spin_period() in your main loop, or spin_async() with an async runtime (Embassy, RTIC v2).
Embedded with allocator (e.g., Zephyr with heap):
nros = { path = "…/nros", default-features = false, features = ["alloc", "rmw-cffi", "platform-zephyr"] }
nros-rmw-zenoh = { path = "…/nros-rmw-zenoh", features = ["platform-zephyr"] }
Adds boxed timer callbacks and handle_request_boxed() for large service replies.
Desktop / Linux:
nros = { path = "…/nros", default-features = false, features = ["std", "rmw-cffi", "platform-posix"] }
nros-rmw-zenoh = { path = "…/nros-rmw-zenoh", features = ["platform-posix", "link-tcp", "ros-humble"] }
nros-platform-cffi = { path = "…/nros-platform-cffi", features = ["posix-c-port"] }
Full API including spin_blocking(), spin_period(), from_env(), system clock.
For async, use an external runtime (tokio current_thread + spawn_local for background spin).
Desktop with parameter services: add param-services to the
nros feature list above. Adds ~/get_parameters,
~/set_parameters, etc. for ros2 param CLI interop.
C-Level Allocation
Both RMW backends compile and link C libraries that perform heap allocation
independently of Rust’s alloc feature. Disabling the Rust alloc feature
eliminates Rust-side heap usage (Box, Vec, String) but does not
eliminate allocation in the C transport layer.
| Backend crate | C Library | Allocator by Platform |
|---|---|---|
nros-rmw-zenoh | zenoh-pico 1.7.2 | POSIX: malloc; Zephyr: k_malloc; FreeRTOS: pvPortMalloc; bare-metal: custom bump allocator |
nros-rmw-xrce-cffi | Micro-XRCE-DDS 3.0.1 | POSIX: malloc; Zephyr: k_malloc; FreeRTOS: pvPortMalloc; NuttX: malloc |
This is by design: the C libraries manage their own session state, stream
buffers, and protocol metadata using platform-provided allocators. The Rust
alloc feature controls only the Rust API surface (boxed callbacks, heap
containers, etc.) and is orthogonal to C-level memory management.
Checking your own image
Because of the row above, a feature list is evidence that an image is heap-free and not proof of it: a vendored C dependency or a weak-symbol fallback can link an allocator without any Cargo feature changing. Ask the linker instead:
scripts/check-no-alloc-image.py path/to/your.elf
It reads the image’s symbol table and fails on any allocation symbol — libc’s
malloc family, Rust’s __rust_alloc family, C++ operator new, and the RTOS
heaps (k_malloc, pvPortMalloc, tx_byte_allocate, kmm_malloc,
heap_caps_malloc) — reporting for each whether the image calls it or
contains it. --tier unified takes --objects instead and enforces the
weaker rule that allocation may happen only inside the nros_platform_*
backend (RFC-0034 D6).
None of the examples shipped in this repo are built this way today — they all
enable alloc — so run it against your own image, not against one of ours.
RTOS Cooperation
nano-ros runs on platforms that span pure cooperative bare-metal through multi-task preemptive RTOS to fully async / Future-driven runtimes. The executor’s spin loop has to cooperate with each of these without imposing a single execution model on all of them.
This page maps the common RTOS / runtime execution profiles to the
configuration knobs the executor exposes. New apps pick a profile
that matches their target’s scheduling discipline; the knobs
translate that choice into bounded behaviour from drive_io.
The execution model spectrum
| Model | Description | Example targets |
|---|---|---|
| Cooperative single-task | One task / thread does all ROS work. No preemption from other tasks (there are none, or they’re all lower priority). Yielding happens at task boundaries. | Bare-metal MPS2-AN385, single FreeRTOS task, single Zephyr thread |
| Preemptive priority | ROS task runs at a fixed priority. Higher-priority tasks preempt mid-call by the kernel. ROS-internal entities (subs, timers, services, GCs) all share that priority — they don’t preempt each other. | Typical FreeRTOS / ThreadX / Zephyr deployment with worker tasks |
| WCET-bounded real-time | Each “task” has a provable worst-case execution time. Tasks are dispatched directly from interrupts; no spin loops in the hot path. | RTIC, Embassy, avionics WCET-validated code |
| Time-triggered cyclic | Fixed schedule. Each cycle does a fixed amount of work in a fixed time slot; ROS gets a fraction of the cycle and must yield. | DO-178C / IEC 61508 controller frames |
| Async runtime | Futures registered with wakers; reactor drives. No spin loop visible to user code. | tokio, Embassy futures, custom async runtimes |
How drive_io behaves by default
The executor’s spin_once(timeout_ms) calls
session.drive_io(...) and lets it drain all ready I/O before
returning. After drive_io returns, the executor processes any
expired timers and triggered guard conditions. If 10 messages
arrived during the wait, all 10 callbacks fire in a single
spin_once, then timer / GC dispatch happens once afterwards.
This is the right default for cooperative single-task apps and
for async-runtime apps using spin_async. Both want
throughput; neither benefits from per-callback scheduling
opportunities.
For the other three models, the default has trade-offs the configuration knobs address.
Configuration knobs
The knobs live on SpinOptions (passed to
Executor::spin_blocking(opts)), not on ExecutorConfig — the config
struct carries identity/transport (locator, domain, node name, clock),
while scheduling shape is a property of each spin call:
| Knob | Default | When to change |
|---|---|---|
SpinOptions::max_callbacks(n) | None (drain all ready work) | Set to 1 for upstream-rclcpp-style “one callback per iteration” — gives the executor a chance to re-check timers / GCs / yield between callbacks |
SpinOptions::timeout_ms(ms) | None (spin forever) | Bound one spin call by wall clock — the time-triggered pattern calls spin_blocking once per cycle with the cycle’s ROS slot as the timeout |
SpinOptions::spin_once() / only_next | off | One round of work then return — the cooperative-loop building block |
Backends opt into one additional behaviour automatically:
Session::next_deadline_ms() tells the executor about the backend’s
next internal event (lease keepalive, heartbeat). The executor caps
drive_io’s timeout against it. No app configuration; transparent
optimization.
Per-model recommended configuration
Cooperative single-task
#![allow(unused)]
fn main() {
// One dedicated task; drain everything each round.
executor.spin_blocking(SpinOptions::default())?;
}
Drain everything; one task, no fairness concern.
Preemptive priority RTOS — recommended
#![allow(unused)]
fn main() {
loop {
executor.spin_blocking(SpinOptions::new().max_callbacks(1))?;
}
}
max_callbacks(1) matches upstream’s rclcpp
single-threaded executor pattern. Each spin_once fires one
callback and then re-checks timers + GCs. ROS-internal entities
share the task priority, but the spin-loop iteration is the
scheduling unit; timer expiries are bounded by one callback’s
WCET, not the sum across all ready callbacks.
If max-callback dispatch latency is still too high in this profile (e.g., a single callback is slow), the remaining bound is that one callback’s WCET — split the slow callback, or move it to its own tier/task (see Scheduling Models).
WCET-bounded real-time (RTIC / Embassy)
Don’t use the spin loop. Use the async path — a buffered subscription
handle whose recv() is an async fn with waker integration:
#![allow(unused)]
fn main() {
let mut sub = node.create_subscription::<MyMsg>("/topic")?;
loop {
let msg = sub.recv().await?; // suspends; waker integration
handle(msg);
}
}
The async path doesn’t go through drive_io at all. Subscriptions
register a Waker; the backend’s RX path wakes the waker; the
async runtime schedules the receiving task. Per-task WCET analysis
applies to each recv().await continuation, not to a spin loop.
Time-triggered cyclic
#![allow(unused)]
fn main() {
// Called once per cycle from the cyclic frame:
executor.spin_blocking(SpinOptions::new().timeout_ms(5))?; // 5 ms ROS slot
}
The cycle gives ROS a fixed wall-clock slot; the timeout_ms bound
returns control when the slot expires regardless of pending work.
Pending work resumes next cycle. (There is no finer-grained
per-callback budget knob today — if a single callback can overrun the
slot, that callback’s WCET is your design constraint.)
Async runtime
drive_io not used in the hot path. The executor’s spin_async
drives futures via wakers; drive_io becomes a polling tick
internally with negligible overhead.
#![allow(unused)]
fn main() {
executor.spin_async().await
}
No knobs apply.
Trade-offs at a glance
| Configuration | Throughput | Per-callback latency | Timer-callback fairness | Code-size cost |
|---|---|---|---|---|
| drain-all (default) | High | Bounded by ALL ready callbacks’ total WCET | Poor under load | Smallest |
max_callbacks(1) | Slightly lower (more spin loop iterations) | Bounded by ONE callback’s WCET | Good | Same — the cap is just an integer |
timeout_ms(N) per cycle | Lower | Bounded by N ms wall clock + one callback’s WCET | Good if N tight | One clock read per iteration |
async / spin_async | Per-future | Per-future Future poll | Cooperative — futures yield voluntarily | Async runtime cost |
Backends and their wait primitives
drive_io’s sleep behaviour is backend- and platform-dependent. The
spin loop’s “where does the thread sleep” question maps as:
| Platform | Sleep primitive in drive_io | When CPU is sleeping |
|---|---|---|
| POSIX | select / epoll_wait with deadline | Inside drive_io |
| Zephyr | k_poll / condvar with deadline | Inside drive_io |
| FreeRTOS | xSemaphoreTake(g_spin_sem, ticks) | Inside drive_io |
| NuttX | sem_timedwait with absolute deadline | Inside drive_io |
| ThreadX | tx_event_flags_get(..., TX_OR, ..., ticks) | Inside drive_io |
Bare-metal smoltcp + BoardIdle | smoltcp poll + wfi() between iterations | Outside drive_io (in the spin loop’s idle hook) |
Bare-metal smoltcp without BoardIdle | smoltcp poll, busy loop | Nowhere — CPU spins |
In all cases the user-visible API is Executor::spin_once(timeout);
the platform-correct sleep happens transparently underneath.
See also
- RMW API Design — the architectural reasons the runtime / RMW boundary is shaped the way it is.
- RMW API: Differences from upstream
rmw.hSection 4 — thedrive_iovsrmw_waitcomparison this page expands on. - no_std Support — heap and threading constraints that shape the cooperative model.
Porting Overview
nano-ros is designed for customization at three independent levels: RMW (transport protocol), Platform (OS or RTOS), and Board (hardware). The core crates define stable interfaces through Rust traits; your implementation lives in your own crates. Be aware of the honest boundary: registering a new platform or an in-catalog board still touches a handful of files inside the nano-ros checkout (the registration steps below), so a vendored tree carries that small diff — plan a fork-with-rebase workflow against the nros-v<X.Y.Z> tags.
Which chapter do I need?
| I want to… | Chapter |
|---|---|
| Add a new transport protocol (MQTT, DDS, custom) | Custom RMW Backend |
| Port to a new RTOS or bare-metal target | Custom Platform |
| Bring up nano-ros on a new MCU board | Custom Board Package |
Most porting work falls into the second or third category. Adding a new RMW backend is rare and substantially more involved.
Quickstart — 2 crates + 2 tomls (RFC-0049)
Scaffold the two packages, fill in the TODOs:
nros new platform myrtos # nros-platform-myrtos/ + nros-platform.toml
nros new board myboard --for-platform myrtos # nros-board-myboard/ + nros-board.toml
nros-platform.toml(platform package) — software-stack facts ([capabilities]), knob defaults ([knobs.*], only flip what you measured), and the zenoh-pico system-layer build block ([build.zenoh]). An empty file is valid: built-in defaults always produce a working build.nros-board.toml(board package) — the RFC-0042 hardware descriptor plus per-board[capabilities]/[knobs]deltas.- Values resolve through a fixed ladder —
builtin < platform < board < env/Kconfig/-D— and an explicit build-time setting (including an explicit0/n) always wins. Debug any knob with:
nros config explain --platform myrtos [--board-toml path/to/nros-board.toml]
# knob value set by
# zenoh.tx.batch false builtin
Kconfig appears only where the host framework is Kconfig-native (Zephyr / NuttX / ESP-IDF packagings) — a hand-wired fragment whose defaults mirror the platform toml (drift-tested). A port to a non-Kconfig RTOS never touches it.
What stays untouched
The following core packages define the interfaces you implement. They compile and work without modification for any new target.
| Package | Role |
|---|---|
nros | Facade crate: re-exports and feature-axis enforcement |
nros-core | Message, service, and action type traits |
nros-serdes | CDR serialization |
nros-node | Executor, Node, pub/sub/service/action handles |
nros-rmw | RMW trait definitions (Session, Publisher, Subscriber, etc.) |
nros-platform | Platform trait definitions and ConcretePlatform type alias |
zpico-sys alias TU | Maps zenoh-pico z_* C symbols to nros_platform_* (default-on platform-aliases) |
nros-rmw-xrce alias TU | Maps XRCE-DDS uxr_* C symbols to nros_platform_* |
nros-c, nros-cpp | C and C++ API wrappers |
These define the interfaces. You implement them; you do not modify them.
The customization boundary
Everything in nano-ros sits on one side of a trait boundary defined in nros-platform/src/traits.rs.
Above the boundary (yours to write): board crates, platform crates, peripheral drivers, and application code.
Below the boundary (fixed): RMW backends, shim crates, core library, executor, and serialization.
Your platform crate implements traits as inherent methods on a zero-sized type. The shim crates automatically forward RMW-layer C symbols to your implementation through the ConcretePlatform type alias – no dynamic dispatch, no generics propagation.
Platform trait requirements by RMW backend
Not every trait is required. The set depends on which RMW backend the application uses.
| Trait | zenoh-pico | XRCE-DDS |
|---|---|---|
PlatformClock | Required | Required |
PlatformAlloc | Required (~64 KB heap) | Not needed |
PlatformSleep | Required | Not needed |
PlatformRandom | Required | Not needed |
PlatformTime | Required | Not needed |
PlatformThreading | Required (multi-threaded platforms) | Not needed |
PlatformTcp | Required | Not needed |
PlatformUdp | Required | Not needed |
PlatformSocketHelpers | Required | Not needed |
PlatformNetworkPoll | Bare-metal only | Not needed |
PlatformUdpMulticast | Desktop platforms only | Not needed |
PlatformLibc | Bare-metal only | Not needed |
XRCE-DDS is significantly simpler to port: it is single-threaded, heap-free, and uses user-provided transport callbacks rather than a full socket API. A minimal XRCE-DDS port requires only PlatformClock and four C function pointers (open, close, read, write).
zenoh-pico requires a complete platform implementation but provides richer functionality (peer-to-peer, scouting, zero-copy receive, actions).
Registration
After implementing the required traits, you register your platform with two changes:
- Add a
platform-<name>feature tonros-platform/Cargo.tomlthat pulls in your crate as an optional dependency. - Add a
ConcretePlatformtype alias innros-platform/src/resolve.rsgated by that feature.
The shim crates pick up the new platform automatically. No changes to RMW backends or core crates are needed.
Existing platform implementations
These serve as reference when writing a new port.
| Platform crate | Target | Threading | Networking |
|---|---|---|---|
nros-platform-posix | Linux, *BSD | pthreads | libc BSD sockets |
nros-platform-freertos | FreeRTOS | FreeRTOS tasks | lwIP |
nros-platform-nuttx | NuttX | pthreads | POSIX sockets |
nros-platform-threadx | ThreadX | ThreadX threads | NetX Duo |
nros-platform-zephyr | Zephyr | Zephyr POSIX | Zephyr sockets |
nros-platform-mps2-an385 | Cortex-M3 bare-metal | Single-threaded | smoltcp |
nros-platform-stm32f4 | STM32F4 bare-metal | Single-threaded | smoltcp |
nros-platform-esp32-qemu | ESP32-C3 (QEMU) bare-metal | Single-threaded | smoltcp + OpenETH |
Further reading
- Custom RMW Backend – implementing a new transport protocol
- Custom Platform – porting to a new RTOS or bare-metal target
- Custom Board Package – bringing up a new MCU board
- Platform API Reference – complete trait signatures and method documentation
- RMW API Reference – RMW trait hierarchy and backend details
- Architecture Overview – concise layer map
- Platform Model – conceptual overview of the three feature axes
The Board Trait Family
The Board trait family is the porting surface for a new MCU or host target. It lives in packages/platform/nros-platform/src/board/ and pins the contract every board crate (nros-board-<board> or a user-authored crate in a downstream Entry pkg) implements. Earlier prototypes used nros-board-common::board_init::*; those legacy traits stay as a transition shim while the in-tree boards finish migrating.
New board crates: implement
nros-platform::board, not the legacy shim. The porting surface for new code is theBoardfamily described on this page (Board=BoardInit+BoardPrint+BoardExit, driven byBoardEntry::run). Thenros-board-common::board_inittrait family is transition-legacy — kept only so not-yet-migrated in-tree boards keep building — and must not be implemented in new board crates. Convergence onto the singlenros-platform::boardfamily is tracked in issue 0243, sequenced with the RTIC/Embassy integration work.
A board impl tells nano-ros four things: how to initialize hardware, how to print a line of text, how to terminate, and (optionally) how to bring a transport up and gate on the network. With those four pieces the BoardEntry::run driver owns the boot lifecycle, and a user Entry pkg main.rs is a ~30 LoC shim.
Where the trait family sits
nros-platform::board
│
├── Board: BoardInit + BoardPrint + BoardExit // super-trait (blanket impl)
│
│
└── BoardEntry: Board
fn run<F, E>(setup: F) -> Result<(), E>
where F: FnOnce(&mut RuntimeCtx) -> Result<(), E>;
BoardInit::init_hardware()— clock tree, pin mux, peripheral wakes. Runs once on boot before allocation. Panicking here is the same as panicking fromfn main()— no recovery.BoardPrint::println(args: core::fmt::Arguments<'_>)— emit a line. Boards wrap whatever stdout makes sense:cortex_m_semihosting::hprintln!, a vendor printf bridge, a UART writer,libc::write(STDOUT_FILENO, …), orprintk.BoardExit::{exit_success, exit_failure}() -> !— terminate cleanly (or with failure). QEMU boards callcortex_m_semihosting::debug::exit; real hardware resets or halts; POSIX shellsstd::process::exit.BoardEntry::run(setup)— the boot driver. Implementations live in the family driver crates (nros-board-{posix,freertos,threadx,…}); user Entry pkgmain.rscalls it.
Board is itself a blanket-implemented super-trait: any type that carries BoardInit + BoardPrint + BoardExit automatically satisfies Board. Concrete board crates do not impl Board directly — they impl the three sub-traits (plus whichever mixins they need).
The BoardEntry::run lifecycle
BoardEntry::run owns the full boot → user-closure → exit flow. The exact body lives in the family driver crate (e.g. nros-board-linux, nros-board-freertos); each family folds its RTOS specifics in, but the order is fixed:
BoardInit::init_hardware()— clocks, pinmux, MMIO setup.- Device bring-up — the family crate’s job, inside
run. Bring the link layer to L2 (Ethernet frames flow / WiFi associated / UART open at baud), then gate on carrier / DHCP if the board has an IP stack. There is no mixin trait for this: see the note below. - Open the executor, build a
RuntimeCtxwith overlay knobs from the launch file / CLI, and invokesetup(&mut runtime). The codegen-emittedrun_plan(runtime)body is whatsetupultimately calls. - Spin the executor to completion (or termination signal).
BoardExit::exit_success()onOk,BoardExit::exit_failure()onError any failed init step.
run returns Result<(), E> rather than ! so unit tests can drive it in a hosted process without exit() killing the test harness — but production boards still call exit_* from inside run’s body after spin returns, so in practice the caller’s Ok(()) arm is unreachable on a real target.
The setup callback is the only place user code runs inside run. Everything else is family-crate boilerplate.
Why there is no
TransportBringup/NetworkWaitmixinEarlier revisions of this page documented two mixin traits and a family-crate blanket impl that would call them:
impl<B: Board + TransportBringup + NetworkWait> BoardEntry for B { fn run … }That could not be built, and the traits were removed in phase-206 W4 (issue 1067). Two reasons, both structural:
- The blanket impl overlaps the direct
BoardEntryimpls that twelve board crates already carry. Rust’s coherence rules do not allow both.- “Skipped if the board doesn’t impl the mixin” is not expressible — Rust has no way to call a method only when the type happens to implement a trait. The order was written as if specialization existed.
Measured before removal:
TransportBringuphad zero implementations and zero call sites;NetworkWaithad one implementation and no callers, because the one place that would have called it — thenros::main!Zephyr arm — routed around it deliberately (ZephyrBoard::wait_link_upcallsstatic inlineZephyr headers with no link symbol, so the native_sim link failed).Devices are still brought up: inside
BoardEntry::run, or the family helper it delegates to. That is the contract, and it is the one that runs.
RuntimeCtx
RuntimeCtx<'a> is the per-invocation overlay context the setup callback receives:
#![allow(unused)]
fn main() {
pub struct RuntimeCtx<'a> {
pub params: &'a [(&'a str, &'a str)], // <param name=… value=…/> + -p name:=value
pub remaps: &'a [(&'a str, &'a str)], // topic/service/action renames
pub env: &'a [(&'a str, &'a str)], // env-style key/value (rarely set on embedded)
}
}
Slice-of-tuples, no_std-safe, no allocation. Codegen owns the storage and passes a &mut RuntimeCtx<'_> whose backing slices live in statics — RuntimeCtx::EMPTY is a const placeholder for launch-less single-node examples or unit tests.
Picking your transport mixins
What you implement on the transport axis depends on what link layers your board exposes:
| Board transport class | Implement | Notes |
|---|---|---|
| Ethernet (smoltcp / lwIP / NetX BSD) | driver up, then DHCP/link gate | both, in run |
| WiFi (ESP32) | same shape — association is L2, DHCP is L3 | both, in run |
| Serial UART only | open at baud | no IP, so no link gate |
| CAN / USB CDC / IVC | link layer only | no IP |
| Bridged-net (threadx-linux veth) | host kernel owns IP | probe the bridge in run |
| Native (host) | None | Host OS owns everything; the family crate’s run skips both mixins |
Boards with multiple transports compose via an internal helper (e.g. a MultiTransport newtype) rather than blanket impls — each transport’s bringup is sequential and order-sensitive (init_link before link_up, sockets only after link).
Worked example — porting a new board
Suppose you’re adding nros-board-acme-cortex-m4-eth, a Cortex-M4 with a UART for println and an MII-attached PHY routed through smoltcp. The crate sits at packages/boards/nros-board-acme-cortex-m4-eth/ and depends on nros-platform, the family crate (nros-board-freertos if FreeRTOS is the RTOS), the matching packages/drivers/<phy>-smoltcp MAC driver, and a vendor HAL crate.
// packages/boards/nros-board-acme-cortex-m4-eth/src/lib.rs
#![no_std]
use nros_platform::board::{
BoardEntry, BoardExit, BoardInit, BoardPrint,
NetworkError, TransportError, RuntimeCtx,
};
pub struct AcmeCortexM4Eth;
impl BoardInit for AcmeCortexM4Eth {
fn init_hardware() {
acme_hal::clocks::init_hse_192mhz();
acme_hal::pinmux::route_uart2();
acme_hal::pinmux::route_eth_mii();
acme_hal::eth::release_phy_reset();
}
}
impl BoardPrint for AcmeCortexM4Eth {
fn println(args: core::fmt::Arguments<'_>) {
// 256-byte stack staging buffer is enough for our log lines;
// pick whatever your UART driver wants.
let mut buf = heapless::String::<256>::new();
let _ = core::fmt::write(&mut buf, args);
let _ = buf.push('\n');
acme_hal::uart2::write_bytes(buf.as_bytes());
}
}
impl BoardExit for AcmeCortexM4Eth {
fn exit_success() -> ! { acme_hal::system::reset() }
fn exit_failure() -> ! { acme_hal::system::halt_with_blinkenlight() }
}
// Device bring-up lives in the BoardEntry::run body — usually by delegating
// to the family helper, which is where the ORDER for that RTOS is fixed:
impl BoardEntry for AcmeCortexM4Eth {
fn run<F, E>(setup: F) -> Result<(), E>
where
F: FnOnce(&mut RuntimeCtx<'_>) -> Result<(), E>,
E: core::fmt::Debug,
{
let cfg = Config::default(); // MAC / IP / netmask / gateway
nros_board_freertos::run_entry::<Self, F, E>(cfg, setup)
}
}
That’s the whole board crate. A downstream Entry pkg consumes it as:
// pkgs/robot_acme_entry/src/main.rs
use nros_board_acme_cortex_m4_eth::AcmeCortexM4Eth;
use nros_platform::board::BoardEntry;
include!(concat!(env!("OUT_DIR"), "/run_plan.rs")); // codegen-emitted
fn main() {
let _ = <AcmeCortexM4Eth as BoardEntry>::run(|runtime| {
run_plan(runtime)
});
}
See the Role reference for the Entry-pkg surface.
Family driver crates
The family crate is where the BoardEntry::run body actually lives. The kernel families with a driver crate:
nros-board-linux— native host; the reach islinux, notposix—apply_tier_affinitycallssched_setaffinitywithcpu_set_t, which libc does not define for macOS.init_transport/wait_link_upno-ops.nros-board-freertos— FreeRTOS-Kernel + lwIP;runspawns the executor task, hands DHCP to lwIP’s hook.nros-board-threadx— ThreadX + NetX BSD; same shape over NetX.nros-board-nuttx— NuttX POSIX layer;init_transportshellsifup-style logic.nros-board-zephyr— carve-out: Kconfig + DTS own BSP; the crate exposes an inherentwait_link_upover<zephyr/net/net_if.h>. The Rust staticlib cannot take overmainon Zephyr.nros-board-esp-idf— ESP-IDF component shape; WiFi association lives ininit_transport, IP lease inwait_link_up.- Direct-exec (Cortex-M / RV32, no RTOS) has no family crate: each board
implements
BoardEntry::runitself with a single-threadzp_readloop. Anros-board-bare-metalfamily driver was written for this shape and no board ever opted into it — 135 of its 161 lines were doc comment — so a cleanup deleted it.nros-board-mps2-an385is the worked reference.
Current state: the trait surface lives in
nros-platform; family driver crates and per-board shims are migrating onto it. Some in-tree boards underpackages/boards/nros-board-*still ride the legacynros-board-common::board_init::*traits — same conceptual shape, different module path.
Cross-references
- Workspace shape + how an Entry pkg consumes a board → Role reference.
- Multi-node composition root →
docs/design/0024-multi-node-workspace-layout.md. - Why the C ABI looks the way it does → Canonical Platform C ABI.
- Platform trait set vs Board trait set — these are different traits with different roles.
Platform*(clock / alloc / sockets / threading) sits below the RMW;Board*sits above the platform and owns the boot lifecycle. A bare-metal board crate typically depends on both: anros-platform-*impl for the platform traits and anros-board-*impl for the board traits.
Custom Board Package
A board crate is the top-level, batteries-included package that application code depends on. It provides three things:
Configstruct – network settings (IP, MAC, gateway), transport selection (Ethernet, serial, or WiFi via Cargo features), and a zenoh locator string.run()entry point – initializes hardware, starts the network stack, and calls a user-provided closure with&Config.- Hardware initialization – PHY/MAC driver setup, interrupt configuration, clock init, and force-linking of platform shim crates.
Users never interact with platform crates or driver crates directly. They depend on a single board crate, call run(), and get a working environment for creating an Executor.
use nros_my_board::{Config, run};
use nros::prelude::*;
run(Config::default(), |config| {
let exec_config = ExecutorConfig::new(config.zenoh_locator)
.domain_id(config.domain_id);
let mut executor = Executor::open(&exec_config)?;
let mut node = executor.create_node("my_node")?;
// publishers, subscriptions, services, actions, timers...
Ok(())
})
Board = platform + hardware
A board crate combines two layers:
- Platform crate (
nros-platform-<os>) – generic RTOS or bare-metal primitives: clock, sleep, threading, memory, random number generation. Implements the traits defined innros-platform. See Custom Platform for how to write one. - Hardware drivers – board-specific peripherals: Ethernet controller, WiFi radio, serial UART. Driver crates in
packages/drivers/(e.g.,lan9118-smoltcp,openeth-smoltcp) implement the smoltcpDevicetrait for a specific Ethernet MAC/PHY.
The board crate glues these together: it depends on the platform crate for OS primitives, on driver crates for peripheral access, and on transport crates (nros-smoltcp, zpico-serial) for bridging the network stack to zenoh-pico. The result is a single dependency that gives application code everything it needs.
Application
|
v
Board crate (nros-my-board)
|
+-- Platform crate (nros-platform-<name>)
+-- Driver crate (lan9118-smoltcp)
+-- Transport bridge (nros-smoltcp or zpico-serial)
+-- zpico-sys (its default-on alias TU supplies zenoh-pico z_* symbols)
Crate structure
A minimal board crate looks like this:
nros-my-board/
Cargo.toml
.gitignore # /target/
src/
lib.rs # extern crate force-links, pub use Config, pub fn run()
config.rs # Config struct with network fields
node.rs # Hardware init sequence + run() implementation
Cargo.toml
[package]
name = "nros-my-board"
version = "0.1.0"
edition = "2024"
[lib]
name = "nros_my_board"
[dependencies]
# Platform primitives (pick one)
nros-platform = { path = "../../platform/nros-platform", features = ["platform-freertos"] }
nros-platform-freertos = { path = "../../platform/nros-platform-freertos" }
# zenoh-pico FFI. Its default-on `platform-aliases` C TU emits every
# `z_*` / `_z_*` symbol zenoh-pico needs, forwarding to `nros_platform_*`.
# (Phase 129 retired the separate `zpico-platform-shim` crate.)
zpico-sys = { path = "../../rmw/zenoh/zpico-sys", default-features = false }
# Ethernet transport (optional, gated by feature)
nros-smoltcp = { path = "../../drivers/net/nros-smoltcp", optional = true }
lan9118-smoltcp = { path = "../../drivers/net/lan9118-smoltcp", optional = true }
smoltcp = { version = "0.12", default-features = false, optional = true, features = [
"medium-ethernet", "proto-ipv4", "socket-tcp", "socket-udp",
] }
# Serial transport (optional, gated by feature)
zpico-serial = { path = "../../rmw/zenoh/zpico-serial", optional = true }
my-uart-driver = { path = "../../drivers/my-uart", optional = true }
# Board-specific dependencies
cortex-m = "0.7"
cortex-m-rt = "0.7"
cortex-m-semihosting = "0.5"
panic-semihosting = "0.6"
[features]
default = ["ethernet"]
ethernet = ["dep:nros-smoltcp", "dep:lan9118-smoltcp", "dep:smoltcp"]
serial = ["dep:zpico-serial", "dep:my-uart-driver"]
lib.rs
The lib module force-links shim crates and re-exports the public API.
#![no_std]
// Force-link: ensures `zpico-sys` (and its default-on `platform-aliases`
// C alias TU, which emits zenoh-pico's `z_*` FFI symbols) is pulled into
// the final binary even though Rust code never calls it directly. Without
// this, the linker drops the rlib and zenoh-pico fails to resolve. Phase
// 129 retired the separate `zpico-platform-shim` keep-alive.
extern crate zpico_sys;
mod config;
mod node;
pub use config::Config;
pub use node::{init_hardware, run};
// Re-export entry macro so examples can use #[entry]
pub use cortex_m_rt::entry;
// Convenience println! that routes to semihosting (QEMU) or UART
pub use cortex_m_semihosting;
#[macro_export]
macro_rules! println {
($($arg:tt)*) => { $crate::cortex_m_semihosting::hprintln!($($arg)*) };
}
pub fn exit_success() -> ! {
cortex_m_semihosting::debug::exit(cortex_m_semihosting::debug::EXIT_SUCCESS);
loop {}
}
pub fn exit_failure() -> ! {
cortex_m_semihosting::debug::exit(cortex_m_semihosting::debug::EXIT_FAILURE);
loop {}
}
config.rs
The Config struct holds all network and transport settings. Fields are
gated by Cargo features so only relevant fields exist for a given build.
#[derive(Clone)]
pub struct Config {
#[cfg(feature = "ethernet")]
pub mac: [u8; 6],
#[cfg(feature = "ethernet")]
pub ip: [u8; 4],
#[cfg(feature = "ethernet")]
pub prefix: u8,
#[cfg(feature = "ethernet")]
pub gateway: [u8; 4],
#[cfg(feature = "serial")]
pub uart_base: usize,
#[cfg(feature = "serial")]
pub baudrate: u32,
pub zenoh_locator: &'static str,
pub domain_id: u32,
}
#[cfg(feature = "ethernet")]
impl Default for Config {
fn default() -> Self {
Self {
mac: [0x02, 0x00, 0x00, 0x00, 0x00, 0x00],
ip: [192, 0, 3, 10],
prefix: 24,
gateway: [192, 0, 3, 1],
#[cfg(feature = "serial")]
uart_base: 0x4000_4000,
#[cfg(feature = "serial")]
baudrate: 115200,
zenoh_locator: "tcp/192.0.3.1:7447",
domain_id: 0,
}
}
}
Provide builder methods (with_ip, with_mac, with_gateway, etc.) and factory presets (listener(), talker()) for common test topologies. See nros-board-mps2-an385/src/config.rs for a complete example including from_toml() parsing.
node.rs
The run() function is the entry point. For bare-metal targets it calls the user closure directly. For RTOS targets it creates a task, starts the scheduler, and runs the closure inside that task.
Bare-metal pattern:
pub fn run<F, E: core::fmt::Debug>(config: Config, f: F) -> !
where
F: FnOnce(&Config) -> Result<(), E>,
{
init_hardware(&config);
match f(&config) {
Ok(()) => exit_success(),
Err(e) => {
hprintln!("Error: {:?}", e);
exit_failure()
}
}
}
RTOS pattern (FreeRTOS example):
pub fn run<F, E: core::fmt::Debug>(config: Config, f: F) -> !
where
F: FnOnce(&Config) -> Result<(), E>,
{
init_hardware(&config);
// Create FreeRTOS task with the user closure
create_app_task(f, config);
// Start the scheduler -- never returns
start_scheduler()
}
The init_hardware() function must follow this order:
- Clock – initialize the hardware timer (must happen before any clock reads)
- Cycle counter – enable DWT or equivalent for timing measurements
- RNG seed – seed PRNG with entropy (IP-based hash, semihosting time, etc.)
- Transport – Ethernet driver + smoltcp, or UART + zpico-serial
- Application – call the user closure
Ethernet peripherals and smoltcp state must live in static mut storage (they are referenced by FFI poll callbacks):
static mut ETH_DEVICE: MaybeUninit<Lan9118> = MaybeUninit::uninit();
static mut NET_IFACE: MaybeUninit<Interface> = MaybeUninit::uninit();
static mut NET_SOCKETS: MaybeUninit<SocketSet<'static>> = MaybeUninit::uninit();
Transport features
Board crates use Cargo features to select the communication transport:
ethernet(typically the default) – TCP/UDP via smoltcp or lwIP. Requires an Ethernet driver crate andnros-smoltcp.serial– UART link viazpico-serial. Requires a UART driver crate.wifi– WiFi via the platform’s native stack (ESP32). The zenoh-pico layer uses OS sockets directly.
At least one transport must be enabled. Enforce this at compile time:
#[cfg(not(any(feature = "ethernet", feature = "serial")))]
compile_error!("Enable at least one transport: `ethernet` or `serial`");
Config fields are #[cfg(feature = "...")]-gated per transport, so the struct only contains fields relevant to the enabled transport. Both ethernet and serial can be enabled simultaneously – the zenoh locator string determines which transport is used at runtime (tcp/... for Ethernet, serial/... for UART).
Reference implementations
Start from the crate closest to your target:
| Board crate | Transport | Platform | Location |
|---|---|---|---|
nros-board-mps2-an385 | Ethernet + Serial | Bare-metal | packages/boards/nros-board-mps2-an385/ |
nros-board-mps2-an385-freertos | Ethernet (lwIP) | FreeRTOS | packages/boards/nros-board-mps2-an385-freertos/ |
nros-board-esp32-qemu | Ethernet (OpenETH) | Bare-metal (esp-hal) | packages/boards/nros-board-esp32-qemu/ |
nros-board-nuttx-qemu | BSD sockets | NuttX | packages/boards/nros-board-nuttx-qemu/ (arm virt and rv-virt) |
nros-board-threadx-qemu-riscv64 | NetX Duo | ThreadX | packages/boards/nros-board-threadx-qemu-riscv64/ |
The bare-metal nros-board-mps2-an385 is the simplest starting point. The FreeRTOS variant shows how to add RTOS task creation and lwIP networking.
See also
- Custom Platform – writing the platform crate that sits underneath a board crate
- Porting Overview – the three customization axes (RMW, platform, board)
- Creating Examples – detailed example layout patterns across native, BSP, and Zephyr targets
Adding a FreeRTOS Board
nano-ros ships one FreeRTOS witness — MPS2-AN385 under QEMU
(nros-board-mps2-an385-freertos, tier 1). This page is the answer to the
only question that matters if your hardware is not that board: what do I
actually have to write?
The answer is measured, not estimated. Everything below is the complete file
set for a hypothetical second board (an NXP S32K344, Cortex-M7), and the line
counts are wc -l of exactly these listings.
| File | Lines | What it is |
|---|---|---|
config/lwipopts.h | 1 | #include the shared defaults |
config/arch/cc.h | 1 | #include the shared defaults |
config/FreeRTOSConfig.h | 3 | CPU clock + NVIC priority bits, then the shared file |
config/s32k344.ld | 7 | memory map + _estack, then the shared section layout |
c/board_s32k344.c | 64 | vector table, Reset_Handler, netif registration |
| per-board delta | 76 | |
src/lib.rs | 57 | the board ZST + four trait impls |
build.rs | 45 | driver + board C + NROS_APP_CONFIG + linker scripts |
Cargo.toml | 27 | dependencies + the rmw-zenoh feature |
| total | 205 |
Read that table honestly: the part that is genuinely about your board is 76 lines. The other 129 is Cargo and trait scaffolding that is identical for every Cortex-M FreeRTOS board and is the next thing worth templating — see “What is still boilerplate” at the bottom.
For comparison, nros-board-mps2-an385-freertos before phase-337 W5 was
~1600 lines plus a 727-line startup.c.
What you do NOT write
The FreeRTOS family crate (nros-board-freertos) owns all of this, and your
board gets it by depending on the crate:
- the FreeRTOS kernel, lwIP and
nros-platform-freertoscompile FreeRTOSConfig.h,lwipopts.h,arch/cc.h— shared defaults, inpackages/boards/nros-board-freertos/config/- the Cortex-M section layout,
nros-freertos-cortex-m.ld - the FreeRTOS hooks (assert / idle / malloc-failed / stack-overflow /
SysTick) and the semihosting helpers —
c/freertos_hooks.c - lwIP bring-up, the task wrappers and the FFI surface —
c/network_glue.c - the C/C++ application boot path —
c/freertos_c_entry.c - the multi-tier entry —
c/freertos_run_tiers.c - the entry driver (
run_entry/run_bare/run_tiers_entry) andConfig
configure_cflags resolves your target’s compiler flags from the [arch.*]
profiles in config/freertos-lwip/nros-platform.toml. Cortex-M3 and Cortex-M7
are declared there; a new arch is a profile block, not a build-script branch.
The files
config/lwipopts.h
#include "../../nros-board-freertos/config/lwipopts.h"
config/arch/cc.h
#include "../../../nros-board-freertos/config/arch/cc.h"
The include is relative because FREERTOS_CONFIG_DIR is a single directory,
not a search path. If your board lives outside this repository and cannot
spell that path, take ladder rung 3 (see The customization
ladder) and own the file — that is what
rung 3 is for, and nothing else in nano-ros changes.
Out-of-tree variant. The worked example’s
build.rsreaches sibling crates with../../walks (manifest.parent().join("nros-board-freertos/config"), drivers,nros-cincludes) — those only resolve insidepackages/boards/. An out-of-tree board crate resolves the nano-ros root ONCE from an env var instead (NROS_REPO_DIR, the same variablenros syncuses) and derives every include from it:let root = PathBuf::from(env::var("NROS_REPO_DIR")?);hal.include(root.join("packages/api/nros-c/include"));— same files, absolute anchor.
config/FreeRTOSConfig.h
Two numbers. Everything else in the 111-line shared file is generic.
#define NROS_BOARD_CPU_CLOCK_HZ 160000000 /* S32K344 CORE_CLK */
#define NROS_BOARD_PRIO_BITS 4 /* Cortex-M7 __NVIC_PRIO_BITS */
#include "../../nros-board-freertos/config/FreeRTOSConfig.h"
config/s32k344.ld
Three numbers. INCLUDE resolves against the linker’s -L search path, which
build.rs populates with OUT_DIR.
MEMORY
{
FLASH (rx) : ORIGIN = 0x00400000, LENGTH = 4M
RAM (rwx) : ORIGIN = 0x20400000, LENGTH = 320K
}
_estack = ORIGIN(RAM) + LENGTH(RAM);
INCLUDE nros-freertos-cortex-m.ld
c/board_s32k344.c
The vector table, the reset handler, and the strong overrides for
network_glue.c’s weak nros_board_register_netif / nros_board_poll_netif
hooks. main is the firmware entry — the Rust main on a Rust image, or
freertos_c_entry.c::main on a C/C++ image. One symbol, one board file.
#include <stdint.h>
#include <string.h>
#include "FreeRTOS.h"
#include "lwip/netifapi.h"
#include "gmac_lwip.h" /* your MAC driver's lwIP netif init + poll */
extern uint32_t _etext, _sdata, _edata, _sbss, _ebss, _estack;
extern void xPortPendSVHandler(void);
extern void vPortSVCHandler(void);
void SysTick_Handler(void); /* nros-board-freertos/c/freertos_hooks.c */
extern int main(void); /* Rust entry, or freertos_c_entry.c::main */
void Reset_Handler(void);
void Default_Handler(void);
static struct netif s_netif;
static struct gmac_config s_cfg;
/* Strong overrides for network_glue.c's weak hooks. */
int nros_board_register_netif(const uint8_t mac[6], const uint8_t ip[4],
const uint8_t netmask[4], const uint8_t gw[4]) {
ip4_addr_t a, m, g;
IP4_ADDR(&a, ip[0], ip[1], ip[2], ip[3]);
IP4_ADDR(&m, netmask[0], netmask[1], netmask[2], netmask[3]);
IP4_ADDR(&g, gw[0], gw[1], gw[2], gw[3]);
s_cfg.base_addr = GMAC_BASE_DEFAULT;
memcpy(s_cfg.mac_addr, mac, 6);
if (netifapi_netif_add(&s_netif, &a, &m, &g, &s_cfg, gmac_lwip_init, tcpip_input) != ERR_OK) {
return -1;
}
netifapi_netif_set_default(&s_netif);
netifapi_netif_set_up(&s_netif);
netifapi_netif_set_link_up(&s_netif);
return 0;
}
void nros_board_poll_netif(void) { gmac_lwip_poll(&s_netif); }
typedef void (*vector_fn)(void);
__attribute__((section(".isr_vector"), used))
const vector_fn isr_vector[] = {
(vector_fn)(uintptr_t)&_estack,
Reset_Handler,
Default_Handler, /* NMI */
Default_Handler, /* HardFault */
Default_Handler, /* MemManage */
Default_Handler, /* BusFault */
Default_Handler, /* UsageFault */
0, 0, 0, 0,
vPortSVCHandler,
Default_Handler, /* DebugMon */
0,
xPortPendSVHandler,
SysTick_Handler,
};
void Reset_Handler(void) {
uint32_t *src = &_etext, *dst = &_sdata;
while (dst < &_edata) *dst++ = *src++;
for (dst = &_sbss; dst < &_ebss;) *dst++ = 0;
(void)main();
for (;;) {}
}
void Default_Handler(void) { for (;;) {} }
If the board has no Ethernet, omit both hook overrides: the weak defaults
in network_glue.c return “no netif”, and the board reaches the router over a
serial locator instead.
build.rs
use std::{env, fs, path::PathBuf};
use nros_board_common::freertos_build::{
add_freertos_includes, add_lwip_includes, app_stack_bytes_from_build_env, configure_cflags,
emit_app_config_tu,
};
use nros_board_common::{BaseConfig, FreertosScheduling};
fn main() {
if nros_board_common::host_probe::skip_cross_build("nros-board-s32k344", &["thumb"]) {
return;
}
let out = PathBuf::from(env::var_os("OUT_DIR").unwrap());
let manifest = PathBuf::from(env::var("CARGO_MANIFEST_DIR").unwrap());
let shared = manifest.parent().unwrap().join("nros-board-freertos/config");
fs::copy(manifest.join("config/s32k344.ld"), out.join("s32k344.ld")).unwrap();
fs::copy(
shared.join("nros-freertos-cortex-m.ld"),
out.join("nros-freertos-cortex-m.ld"),
)
.unwrap();
println!("cargo:rustc-link-search={}", out.display());
let freertos = nros_build_paths::freertos_dir();
let port = freertos.join("portable/GCC/ARM_CM7/r0p1");
let lwip = nros_build_paths::lwip_dir();
let cfg = manifest.join("config");
let mut b = cc::Build::new();
configure_cflags(&mut b);
add_freertos_includes(&mut b, &freertos, &port, &cfg);
add_lwip_includes(&mut b, &lwip);
b.include(manifest.join("../../drivers/net/gmac-lwip/include"));
b.include(manifest.join("../../api/nros-c/include"));
b.file(manifest.join("c/board_s32k344.c"));
b.file(manifest.join("../../drivers/net/gmac-lwip/src/gmac_lwip.c"));
let sched = FreertosScheduling {
app_stack_bytes: app_stack_bytes_from_build_env(),
..FreertosScheduling::default()
};
b.file(emit_app_config_tu(&out, &BaseConfig::default(), &sched));
b.compile("startup");
println!("cargo:rustc-link-lib=static=startup");
}
src/lib.rs
#![allow(unused)]
#![no_std]
fn main() {
extern crate nros_platform as _;
#[cfg(target_os = "none")]
use panic_semihosting as _;
#[cfg(feature = "rmw-zenoh")]
extern crate zpico_sys;
pub use nros_board_freertos::{BaseConfig, Config};
/// Per-board marker for trait dispatch into the `nros_board_freertos::run_*`
/// family driver.
pub struct S32k344;
impl nros_platform::BoardInit for S32k344 {
fn init_hardware() {}
}
impl nros_platform::BoardPrint for S32k344 {
fn println(args: core::fmt::Arguments<'_>) {
use core::fmt::Write;
if let Ok(mut out) = cortex_m_semihosting::hio::hstdout() {
let _ = writeln!(out, "{}", args);
}
}
}
impl nros_platform::BoardExit for S32k344 {
fn exit_success() -> ! {
cortex_m_semihosting::debug::exit(cortex_m_semihosting::debug::EXIT_SUCCESS);
loop {}
}
fn exit_failure() -> ! {
cortex_m_semihosting::debug::exit(cortex_m_semihosting::debug::EXIT_FAILURE);
loop {}
}
}
impl nros_platform::BoardEntry for S32k344 {
fn run<F, E>(setup: F) -> Result<(), E>
where
F: FnOnce(&mut nros_platform::RuntimeCtx<'_>) -> Result<(), E>,
E: core::fmt::Debug,
{
nros_board_freertos::run_entry::<S32k344, F, E>(Config::default(), None, setup)
}
fn run_with_deploy<F, E>(deploy: &nros_platform::DeployOverlay, setup: F) -> Result<(), E>
where
F: FnOnce(&mut nros_platform::RuntimeCtx<'_>) -> Result<(), E>,
E: core::fmt::Debug,
{
let mut config = Config::default();
config.base.apply_overlay(deploy);
nros_board_freertos::run_entry::<S32k344, F, E>(config, deploy.boot_config, setup)
}
}
}
Cargo.toml
[package]
name = "nros-board-s32k344"
version = "0.1.0"
edition = "2024"
license = "MIT OR Apache-2.0"
[dependencies]
nros-board-freertos = { path = "../nros-board-freertos" }
nros-platform = { path = "../../platform/nros-platform", default-features = false, features = [
"platform-freertos",
"global-allocator",
] }
zpico-sys = { path = "../../rmw/zenoh/zpico-sys", default-features = false, features = [
"freertos",
"platform-aliases",
], optional = true }
cortex-m-semihosting = "0.5"
panic-semihosting = "0.6"
[build-dependencies]
nros-board-common = { path = "../nros-board-common", features = ["build-helpers"] }
nros-build-paths = { path = "../../tooling/nros-build-paths" }
cc = "1"
[features]
default = ["rmw-zenoh"]
rmw-zenoh = ["dep:zpico-sys", "nros-board-freertos/rmw-zenoh"]
The one thing that is not a line count
The MAC driver. board_s32k344.c above assumes a gmac_lwip.c exists — an
lwIP netif driver for your chip’s Ethernet MAC. nano-ros’s reference is
packages/drivers/net/lan9118-lwip/src/lan9118_lwip.c at ~507 lines. If your
vendor SDK already ships an lwIP netif (most do), you wire it up in the four
lines of nros_board_register_netif and write none of it. If it does not,
that driver is the real cost of the port, and it is a cost nano-ros cannot
remove — it is between your silicon and lwIP.
If your vendor SDK owns the whole build (S32DS, MCUXpresso, ESP-IDF,
PlatformIO), you likely want an integration shell instead of a board
crate: see integrations/s32ds/ and RFC-0064. The shell composes
freertos_platform from artefacts the vendor build already produced and
nano-ros runs board-less — no board crate at all.
What is still boilerplate
Measured above: 129 of the 205 lines are not about the board.
src/lib.rs(57):BoardPrint/BoardExitare pure Cortex-M semihosting andBoardEntryis a two-line delegation to the family driver. Every Cortex-M FreeRTOS board writes the same file with a different type name. A declarative macro innros-board-freertoswould take this to one line.Cargo.toml(27) andbuild.rs(45): the dependency set and the build sequence are fixed; only the driver paths, the FreeRTOS port directory and the linker-script name vary.
Phase-337 W5 templated the C and config half (299 lines of headers and 135 lines of linker script became 12 and 7). The Rust half is the same exercise, not yet done. Until it is, quote 205 lines for a second FreeRTOS board, not 80.
Vendor Overlay Board Crate
An overlay crate is a small Cargo crate that depends
on a generic per-kernel board crate (nros-board-freertos,
nros-board-threadx, nros-board-nuttx — there is no bare-metal
family crate; that one was deliberately deleted)
and patches the deltas a specific vendor board / fork needs:
- Vendor HAL source files (NXP
fsl_*, STMHAL_*, NVIDIA FSP, …). - Per-chip linker script + startup assembly.
- Custom kernel-config header (
FreeRTOSConfig.h,tx_user.h). - Custom network-stack glue (vendor Ethernet driver wired to lwIP / NetX-Duo).
- Custom clock-tree / pin-mux init.
This page documents the contract: what the generic crate exposes, what the overlay overrides, and how to distribute an overlay. (nano-ros is source-only — nothing is published to crates.io — so overlays are consumed as path or git dependencies.)
Why overlays
nano-ros’s generic board crates cover the “stock RTOS source + your own drivers” workflow. Vendor SDKs (NXP MCUXpresso, STM32Cube, Espressif ESP-IDF, Renesas Synergy, NVIDIA FSP) ship forked kernels + custom drivers; bolting those into a generic crate would force a build-script branch per vendor. The overlay pattern keeps the generic crate clean: nano-ros ships the kernel-family scaffolding, vendors / community ship the per-fork glue.
See docs/roadmap/phase-152-board-bsp-abstraction-layer.md for the
phase that landed the architecture.
Contract
A generic per-kernel board crate (taking nros-board-freertos as the
worked case) exposes:
| Item | Type | Purpose |
|---|---|---|
Config / BaseConfig | structs | Network + transport config; overlay can extend. |
run_bare / run_entry / run_tiers_entry | generic functions | Entry points, generic over a board marker type. Initialise kernel + network, then call your closure inside the app task. |
nros_platform::BoardInit | trait | One required method — init_hardware(), called once before the executor opens. Your marker type implements it (plus BoardPrint / BoardExit / BoardEntry from the same family). |
nros_board_register_netif / nros_board_poll_netif | weak C hooks (c/network_glue.c) | The network attach points: the overlay provides strong definitions wiring the vendor Ethernet driver into lwIP. |
The overlay’s build.rs:
- Inherits the generic crate’s
FREERTOS_DIR/THREADX_DIR/ etc. env-var contract (overlay doesn’t override unless needed). - Adds vendor HAL
.csources via its owncc::Build. - Optionally regenerates the linker script (e.g. STM32F4 vs STM32F7 sector layout).
Minimal overlay shape
#![allow(unused)]
fn main() {
// nros-board-stm32f4-freertos/src/lib.rs
#![no_std]
// Re-export the generic config types from the kernel-family crate.
pub use nros_board_freertos::{BaseConfig, Config};
/// Per-board marker for trait dispatch into `nros_board_freertos::run_*`.
pub struct Stm32F4;
impl nros_platform::BoardInit for Stm32F4 {
fn init_hardware() {
// HAL_RCC_OscConfig + HAL_RCC_ClockConfig + pin mux + ...
}
}
// (implement BoardPrint / BoardExit / BoardEntry the same way —
// `packages/boards/nros-board-mps2-an385-freertos/src/lib.rs` is the
// working precedent to copy.)
/// The network attach point is a strong definition of the generic
/// crate's WEAK C hook (c/network_glue.c) — typically provided from a
/// C file your build.rs compiles, wiring the vendor Ethernet driver
/// into lwIP:
///
/// int nros_board_register_netif(const uint8_t *mac, ...);
/// void nros_board_poll_netif(void);
}
// nros-board-stm32f4-freertos/build.rs
use std::{env, path::PathBuf};
fn main() {
let stm_hal_dir = env::var("STM32_HAL_DIR")
.expect("set STM32_HAL_DIR to your STMicroelectronics HAL source dir");
let mut hal = cc::Build::new();
hal.flag("-mcpu=cortex-m4")
.flag("-mthumb")
.flag("-mfpu=fpv4-sp-d16")
.flag("-mfloat-abi=hard")
.include(format!("{stm_hal_dir}/Inc"));
for f in &[
"Src/stm32f4xx_hal_eth.c",
"Src/stm32f4xx_hal_uart.c",
"Src/stm32f4xx_hal_rcc.c",
// ...
] {
hal.file(format!("{stm_hal_dir}/{f}"));
}
hal.compile("stm32f4_hal");
// Board-specific linker script wired via the generic crate's
// BOARD_LINKER_SCRIPT_DIR env var.
println!(
"cargo:rustc-env=BOARD_LINKER_SCRIPT_DIR={}",
PathBuf::from(env::var("CARGO_MANIFEST_DIR").unwrap())
.join("config")
.display()
);
println!("cargo:rerun-if-env-changed=STM32_HAL_DIR");
}
# nros-board-stm32f4-freertos/Cargo.toml
[package]
name = "nros-board-stm32f4-freertos"
version = "0.1.0"
edition = "2024"
license = "MIT OR Apache-2.0"
authors = ["Your Name <you@example.com>"]
description = "STM32F4 + FreeRTOS overlay on nros-board-freertos"
repository = "https://github.com/<you>/nros-board-stm32f4-freertos"
[dependencies]
# Path into your vendored nano-ros checkout (or a git dep on your fork).
# nano-ros is source-only: these names are NOT on crates.io.
nros-board-freertos = { path = "../nano-ros/packages/boards/nros-board-freertos" }
nros-platform = { path = "../nano-ros/packages/platform/nros-platform", default-features = false }
[build-dependencies]
cc = "1.0"
User application code stays identical to the generic-crate case
except for the [dependencies] line:
#![allow(unused)]
fn main() {
use nros_board_stm32f4_freertos::{Config, Stm32F4};
// The `nros::main!(board = Stm32F4)` macro emits this dispatch; the
// manual form is:
nros_board_freertos::run_entry::<Stm32F4, _, _>(Config::default(), None, |runtime| {
// register your nodes on `runtime`
Ok(())
})
}
Canonical in-tree precedent
packages/boards/nros-board-mps2-an385-freertos/ is the canonical
“stock kernel + custom Ethernet driver” overlay, and the ONLY in-tree
one (the former orin-spe overlay was removed):
- Re-exports
BaseConfig+Configfromnros-board-freertos. - Defines the
Mps2An385marker and implements thenros_platformboard traits on it. build.rsadds the LAN9118 driver C sources + per-board linker script.- Provides the strong
nros_board_register_netif/nros_board_poll_netifdefinitions binding LAN9118 into lwIP.
Read it for working code; Adding a FreeRTOS Board walks the same contract step by step.
Naming convention
Name the crate nros-board-<vendor>-<chip-or-board>-<rtos>.
Examples:
nros-board-stm32f4-freertosnros-board-stm32h7-threadxnros-board-nxp-mimxrt1064-freertosnros-board-renesas-synergy-s7g2-threadxnros-board-nordic-nrf5340-zephyr(rare — Zephyr generally owns board contract via DTS; only needed when a non-Zephyr nano-ros consumer wants to target an nRF board outside the Zephyr build)
The nros-board- prefix is the informal namespace; keep it so a
reader can tell a board crate from an app crate at a glance.
What overlays DO
- ✅ Re-export
Config+run(or extendConfigwith vendor- specific fields and re-implementrunif needed). - ✅ Add vendor HAL C sources via
cc::Build. - ✅ Provide strong definitions of the generic crate’s weak C hooks
(
nros_board_register_netif,nros_board_poll_netif). - ✅ Ship board-specific config files (linker script,
FreeRTOSConfig.h,tx_user.h). - ✅ Read vendor-SDK env vars (
STM32_HAL_DIR,NXP_SDK_DIR,NV_SPE_FSP_DIR) and inject paths into cc-rs.
What overlays DON’T
- ❌ Re-implement kernel build glue (that’s the generic crate’s job).
- ❌ Add features that should live in the generic crate (push them upstream instead).
- ❌ Duplicate
nros-platform-<rtos>registration (the generic crate handles it). - ❌ Override
nros-rmw-*selection (user picks RMW via Cargo features onnros, same as any nano-ros consumer). - ❌ Ship a fork of zenoh-pico / Cyclone DDS / mbedTLS (use the upstream’s manifest).
Testing an overlay locally
# 1. Clone or scaffold the overlay crate next to your application.
git clone https://github.com/<you>/nros-board-<your-vendor>-<rtos>
# 2. Point your application's Cargo.toml at it (path dep for dev).
[dependencies]
nros-board-<your-vendor>-<rtos> = { path = "../nros-board-<your-vendor>-<rtos>" }
# 3. Build with the vendor SDK env vars set.
export FREERTOS_DIR=$HOME/sdk/freertos/kernel
export FREERTOS_PORT=GCC/ARM_CM4F
export LWIP_DIR=$HOME/sdk/freertos/lwip
export STM32_HAL_DIR=$HOME/sdk/stm32cube/STM32F4xx_HAL_Driver
cargo build --release --target thumbv7em-none-eabihf
Skeleton template
templates/overlay-board/ ships a minimal skeleton:
Cargo.toml.template— deps on the generic kernel crate.src/lib.rs.template— marker type + trait-impl and hook stubs.build.rs.template— cc-rs HAL-source injection scaffold.README.md— env vars + setup recipe.
Copy the directory, rename the placeholder, and fill in the
vendor-specific bits. See templates/overlay-board/README.md for
the per-file walkthrough.
Distributing an overlay
nano-ros is source-only (decision 2026-05-14): nothing — including
board crates — is on crates.io, so do not cargo publish. Ship the
overlay as a git repository and have consumers take it as a git or
path dependency next to their vendored nano-ros checkout. Tag
releases against the nros-v<X.Y.Z> tag of nano-ros you developed
against, and state that pairing in your README — the board-trait
surface is versioned with the tree, not semver-independent.
Related
docs/design/0012-board-bsp-integration-architecture.md— the layered model + the consumption matrix.docs/roadmap/phase-152-board-bsp-abstraction-layer.md— the phase doc.- Custom Board Package — older guide; covers monolithic board crates before the overlay split.
- Custom Platform —
nros-platform-<rtos>guide (the Layer 1 contract overlays rely on).
Worked Example — STM32F4 Out of Tree
This chapter takes one real board all the way through the customization ladder: an STM32F429/F407 with Ethernet, built and maintained entirely outside the nano-ros repository.
It exists because that is the honest arrangement for this board. nano-ros
carried nros-board-stm32f4 and nros-board-rtic-stm32f4 in-tree for a long
time, and no CI lane ever booted either of them — the hardware is not in the
test rack and QEMU does not model the STM32 MAC. An in-tree crate reads as a
support promise, so the two crates left the supported matrix (RFC-0064,
phase-337 W7.a) and became this page instead. The hardware works exactly as well
as it did; what changed is who verifies it, and that is now you.
If you want the general shape rather than this specific board, read Custom Board Package and Vendor Overlay Board Crate first — this chapter assumes them and only shows the STM32F4-specific decisions.
What this board actually needs
Four facts drive everything below. They came from the retired crate’s own descriptor:
| Fact | Value |
|---|---|
| Target | thumbv7em-none-eabihf |
| Link | -C link-arg=-Tlink.x, plus a memory map for the chip |
| Net stack ownership | nanoros-owned — nano-ros supplies both the MAC driver and the IP stack |
| Transports | Ethernet (STM32 MAC + smoltcp) or serial (USART) |
The third row is the expensive one, and it is why this board costs more than a Zephyr or NuttX port. On an RTOS-owned board the host ecosystem already brings up the NIC and the IP stack, and nano-ros only asks it for a socket. Here, nothing else brings up the NIC — so the driver and the stack are part of your board, not part of the platform. Budget for that before anything else: RFC-0064 measures a MAC driver at roughly 500 lines when one does not already exist.
Which rung you are on
The ladder has three rungs, and this board sits on different ones depending on what you are changing.
| What you want | Rung | What you write |
|---|---|---|
| A different chip in the F4 family (F407 vs F429) | 1 — declare | one chip value + its memory map |
| A different PHY, same MAC | 2 — hook | the board’s Ethernet init function |
| A proprietary link instead of Ethernet | 1 + 2 | link.custom + the transport vtable |
| An IP stack that is not smoltcp | 3 — escape | the platform ABI’s net functions directly |
Rung 3 always exists, and it carries a rule worth stating plainly: if you ship an artifact by hand, the generator does not overwrite it and the drift gate does not flag it. You can adopt generated glue one file at a time and keep the files you have reason to own.
The shape of the crate
Your board crate depends on nano-ros; nano-ros does not depend on it. That is the whole point of taking it out of tree, and it is what lets you pin a nano-ros release and upgrade on your own schedule.
my-stm32f4-board/
Cargo.toml # depends on nros + the bare-metal platform
nros-board.toml # the descriptor the `nros` CLI reads
build.rs # emits the linker script / memory map
memory.x # FLASH + RAM origin and length for your chip
src/lib.rs # the board ZST, its trait impls, and `run()`
Compose your Config on nros_board_common::BaseConfig rather than declaring
the network fields again — it carries {mac, ip, netmask, gateway, zenoh_locator, domain_id} and the DeployOverlay merge, and keeps your board
aligned with every other one when those fields change. Board-specific settings
(a UART base address, a PHY address) stay on your own struct beside it:
#![allow(unused)]
fn main() {
pub struct Config {
pub base: nros_board_common::BaseConfig,
pub phy_addr: u8,
}
}
BaseConfig stores the netmask; prefix() and with_prefix() convert if
you think in CIDR.
The descriptor
nros-board.toml is what makes the nros CLI able to generate an entry for
your board. The retired crate’s descriptor is a working template — the fields
that matter for a bare-metal Cortex-M board with its own stack:
[[board]]
names = ["my-stm32f4"]
platform = "stm32"
toolchain = "stable"
platform_feature = "platform-bare-metal"
link_kind = "none"
entry_kind = "board-run"
supported_netstacks = ["smoltcp"]
chip = "stm32f429"
board_crate = "my-stm32f4-board"
# entry_kind = "board-run" needs the matching [board.entry] block —
# copy the one from packages/boards/nros-board-mps2-an385/nros-board.toml
# and swap the crate name.
[board.entry]
crate_name = "my_stm32f4_board"
signature = "#[my_stm32f4_board::entry]\nfn main() -> !"
cargo_config = '''
[build]
target = "thumbv7em-none-eabihf"
[target.thumbv7em-none-eabihf]
rustflags = ["-C", "link-arg=-Tlink.x"]
'''
supported_netstacks names who brings up the link (here your crate’s
smoltcp glue — nothing outside it). Beware unknown keys: the descriptor
parser ignores what it does not know rather than erroring, so a typo’d
key silently does nothing.
What you take on
Be clear-eyed about the trade, because it is the reason this page exists rather than a crate:
- You own verification. No nano-ros lane boots this board. A nano-ros release being green says nothing about your board; only your hardware does.
- You own the driver. A MAC or PHY change is yours to make and yours to test.
- You gain release control. Your board no longer moves when nano-ros refactors its board tree — you upgrade when you choose, and a breaking change in the seam is visible as a version bump instead of a surprise.
The seam you depend on is deliberately narrow: the platform ABI is 92 C functions, and no rung of the ladder requires nano-ros to know what your stack is. That is what makes an out-of-tree board sustainable rather than a fork.
If you want it back in tree
The bar is a witness: a lane that boots the board and asserts real delivery.
That means hardware in the test rack or a QEMU machine that models the MAC. A
board with no Runtime cell cannot be tier 1 or 2 —
scripts/check-board-tiers.py enforces exactly that, which is what moved these
crates out in the first place.
Importing a Board Crate
This chapter is for consumers of a nano-ros board crate – vendors and downstream applications (the Autoware Safety Island archetype) who want to bring a board into their own Zephyr app with as little glue as possible. If you are writing a new board crate, see The Board Trait Family and Custom Board Package instead.
Goal
Consume a nano-ros board crate from a downstream Zephyr application with a single CMake call. The consumer’s build script SHOULD NOT carry any of:
- a hand-curated
EXTRA_CONF_FILElisting the board crate’sprj.confand per-board Kconfig snippets - a hand-curated
DTC_OVERLAY_FILE - a hardcoded
BOARD=<id>string - a hardcoded
-DNANO_ROS_RMW=<rmw>define - a hand-rolled FVP / qemu launch shell
All of those are owned by the board crate. The consumer’s CMakeLists.txt declares which board it wants; nano-ros layers the rest in.
Prereqs
Before importing a board crate, the consumer needs:
- A Zephyr 3.7+ workspace managed by west. Zephyr 3.7 is the floor (the official in-tree
zephyr-lang-rustmodule did not exist below it, so nothing earlier can link the nano-ros Rust staticlib); newer LTS lines work too. - All gated SDK packages for the target board. Run
nros doctor --board <name>to check; missing items can be installed withnros setup <board>(ornros setup --tool <t>for a single dependency). Seenros setupin the build commands reference. - nano-ros listed as a project in your
west.yml, and nano-ros’s Zephyr module exported onZEPHYR_EXTRA_MODULESso Zephyr picks up its Kconfig + DTS roots.
A minimal west.yml fragment:
manifest:
remotes:
- name: newslab
url-base: https://github.com/NEWSLabNTU
projects:
- name: nano-ros
remote: newslab
revision: nros-v0.5.0 # pin a release tag — a moving branch
# unships your tested pairing (see
# installation.md "Pinning a version")
path: deps/nano-ros
import: false
self:
west-commands: deps/nano-ros/scripts/west-commands.yml
And the matching environment:
export ZEPHYR_EXTRA_MODULES="$PWD/deps/nano-ros/zephyr"
The one-call pattern
A consumer CMakeLists.txt should look like this and nothing more:
cmake_minimum_required(VERSION 3.20)
# Single import call. Layers in BOARD, EXTRA_CONF_FILE, DTC_OVERLAY_FILE,
# NANO_ROS_RMW default, and the runner hint.
nano_ros_use_board(fvp-aemv8r-smp)
find_package(Zephyr REQUIRED HINTS $ENV{ZEPHYR_BASE})
project(my_app LANGUAGES CXX)
target_sources(app PRIVATE src/main.cpp)
Call order matters. nano_ros_use_board() MUST precede find_package(Zephyr ...). Zephyr reads BOARD, EXTRA_CONF_FILE, and DTC_OVERLAY_FILE during its find_package call; setting them after that point has no effect. The pattern above is the canonical shape – copy it verbatim and only fill in the board name.
nano_ros_use_board() is shipped by the nano-ros Zephyr module. It is available the moment ZEPHYR_EXTRA_MODULES includes deps/nano-ros/zephyr; no extra include() is required.
Out-of-tree boards. The lookup searches
<nano-ros>/packages/boards/plus every root named byNROS_EXTRA_BOARD_PATH(environment, or a CMake cache var) — so a board crate in YOUR tree is found without copying it into the checkout. Each root has the same layout aspackages/boards/. Board keys are global: a name resolving under more than one root is a fatal ambiguity, never shadowed by search order.
What the call layers in
| Source | Effect |
|---|---|
BOARD | Set to NROS_BOARD_ZEPHYR_ID (from the board crate’s board.cmake) if the user did not pass -DBOARD=... on the command line. |
EXTRA_CONF_FILE | The board crate’s prj.conf and any per-board Kconfig fragments (e.g. an HWv2 snippet) are appended. Any consumer-supplied EXTRA_CONF_FILE is preserved and layered AFTER the board’s. |
DTC_OVERLAY_FILE | The board crate’s per-board DTS overlay is appended. Consumer overlays are preserved and layered after. |
NANO_ROS_RMW | Defaulted to NROS_BOARD_DEFAULT_RMW (from board.cmake) when the consumer did not pass -DNANO_ROS_RMW=.... |
NROS_BOARD_RUNNER | Cached for west fvp run (or another runner extension command) to pick up the right simulator binary / target-launcher. |
NROS_BOARD_RUST_SUPPORT_MODULE | Appended to ZEPHYR_EXTRA_MODULES when the board carries a Rust support module. (Not yet in the board.cmake ↔ Cargo.toml drift-audit field list — drift here is unaudited.) |
The values themselves come from a single source of truth – the board crate’s board.cmake – which a drift audit (nros board info --check-drift) keeps in sync with the crate’s Cargo.toml metadata.
Per-app overrides
The one-call pattern still gives the consumer escape hatches:
- Pin a different Zephyr board id. Pass
-DBOARD=<other>on the CMake /west buildcommand line;nano_ros_use_board()notices the user value and emits a warning rather than clobbering it. - Pick a different RMW. Pass
-DNANO_ROS_RMW=<rmw>(zenoh,xrce, orcyclonedds– see Backend Reference). The board crate’s default is used only when nothing is set. - Layer extra Kconfig / DTS. Set
EXTRA_CONF_FILEandDTC_OVERLAY_FILEafter the call tonano_ros_use_board(). They will be applied on top of the board crate’s contributions, not in place of them.
nano_ros_use_board(fvp-aemv8r-smp)
# Extra app-specific Kconfig layered on top of the board's defaults.
list(APPEND EXTRA_CONF_FILE "${CMAKE_CURRENT_SOURCE_DIR}/boards/fvp-aemv8r-smp.conf")
find_package(Zephyr REQUIRED HINTS $ENV{ZEPHYR_BASE})
project(my_app LANGUAGES CXX)
Running
For boards whose runner is an FVP (the AEMv8-R archetype):
west build -d build
west fvp run -d build
west fvp run consults NROS_BOARD_RUNNER and the FVP resolver to locate the simulator binary, applies the board’s launch arguments, wires UART to stdout, and exits cleanly on Ctrl-C.
For any other runner the Zephyr-native command works unchanged:
west build -d build -t run
There is no need for a hand-written build.sh or run_fvp.sh shell wrapper.
Inspecting the manifest
To see exactly what nano_ros_use_board() will layer in for a given board:
nros board info fvp-aemv8r-smp
This prints the resolved Zephyr board id, the prj.conf + Kconfig fragment list, the DTS overlay, the default RMW, and the runner hint – everything the call would set.
To audit that the board.cmake mirror matches the canonical Cargo.toml metadata:
nros board info fvp-aemv8r-smp --check-drift
This exits 0 when the two agree and emits a field-by-field diff (with a non-zero exit code) when they don’t. The drift audit is the same one CI runs for every packages/boards/nros-board-* shipping a board.cmake.
Anti-patterns
Things that LOOK reasonable but the one-call pattern obviates:
- Don’t hand-list the board’s
prj.confinEXTRA_CONF_FILE. It is already there; doing it again either duplicates or fights the layering order. - Don’t hardcode
BOARD=<id>in abuild.sh. The call sets it; hardcoding it short-circuits thenros board infoinspection and the drift audit. - Don’t carry your own copy of
boards/<id>.conforboards/<id>.overlaymirroring the board crate’s. Vendor a delta only – the base ships in the crate. - Don’t reimplement the FVP runner as a shell script.
west fvp runcoversFVP_BaseR_AEMv8Rand the other supported simulators with the right CLI flags, the same waywest buildcovers compilation. - Don’t
include()files fromdeps/nano-ros/cmake/directly. The public surface isnano_ros_use_board(); anything else is internal and may move.
Migrating an existing hand-glued consumer
Most downstream consumers (the ASI archetype is the canonical example) carry years of accumulated glue. The migration is mechanical:
- Find the per-board Kconfig and overlay entries in the current
CMakeLists.txt/build.sh. Anything listing the board crate’sprj.conf, board overlay, or HWv2 snippet – delete it. Keep only entries that point at the consumer’s own deltas. - Find any hardcoded
BOARD=<id>string (in CMakeset(BOARD ...), inwest build -b <id>, or inbuild.sh). Delete it.nano_ros_use_board()will set it fromboard.cmake. - Replace
find_package(Zephyr) + ... + manual FVP launchwith the canonical pattern above – onenano_ros_use_board()call, thenfind_package(Zephyr), thenproject(), thentarget_sources(app ...). Replace the FVP shell script call withwest fvp run -d build. - Keep only app-specific deltas in
boards/<id>.conf/boards/<id>.overlay. For the ASI archetype this typically means Autoware-msg sizing (large topic / participant memory) and the application’s own GPIO map – everything generic moves into the board crate.
After migration, the consumer’s CMake should be roughly twenty lines, with nano_ros_use_board() as the only nano-ros-specific call.
Cross-references
- The
BoardTrait Family – for implementers of a new board crate. This chapter is the consumer-side dual. - Custom Board Package – the full board-crate authoring guide.
- Vendor Overlay Board Crate – the lighter “I just want to override one field” path.
- Build commands reference –
nros setup,nros doctor,nros board info. - RMW backends – the menu of values for
-DNANO_ROS_RMW=....
Custom Platform
This guide walks through porting nano-ros to a new RTOS or bare-metal environment. A “platform” provides the OS-level primitives that nano-ros needs at runtime: clock, memory, sleep, threading, and networking. The core library is #![no_std] and makes zero platform calls directly – everything flows through your platform crate.
Canonical interface spec. The function-pointer signatures, parameter docs, ownership rules, blocking allowance, and failure modes for every method live in the platform-cffi Doxygen reference. Read it side-by-side with this guide — the tables below summarise the set of traits you need; the Doxygen documents the contract each call must obey.
Quick differences table. For a per-platform comparison (clock source, allocator, threading, networking, multicast support) across the platforms nano-ros already supports, see Platform Differences.
What you implement
All platform traits are defined in nros-platform/src/traits.rs. Your platform crate implements some or all of them as inherent methods on a zero-sized type (ZST). The set you need depends on your RMW backend.
Required for all backends
| Trait | Methods | Purpose |
|---|---|---|
PlatformClock | clock_ms(), clock_us() | Monotonic time. Must use a hardware timer or OS tick – never a software counter that only advances when polled. |
Required for zenoh-pico (rmw-zenoh)
| Trait | Methods | Purpose |
|---|---|---|
PlatformAlloc | alloc(), realloc(), dealloc() | Heap allocation. zenoh-pico needs ~64 KB. |
PlatformSleep | sleep_us(), sleep_ms(), sleep_s() | Delay. On bare-metal with smoltcp, poll the network during busy-wait. |
PlatformRandom | random_u8() through random_u64(), random_fill() | PRNG for session IDs and protocol nonces. |
PlatformTime | time_now_ms(), time_since_epoch() | Wall-clock time for logging. Return monotonic time if no RTC. |
PlatformThreading | Tasks, mutexes, recursive mutexes, condvars (19 methods) | OS threading primitives. Single-threaded platforms provide no-op stubs. |
Networking
| Trait | Methods | Purpose |
|---|---|---|
PlatformTcp | open(), read(), send(), close(), … | TCP client and server sockets. |
PlatformUdp | open(), read(), send(), close(), … | UDP unicast sockets. |
PlatformSocketHelpers | set_non_blocking(), accept(), close(), wait_event() | Socket utility operations. |
Optional
| Trait | When needed |
|---|---|
PlatformUdpMulticast | Desktop platforms using zenoh scouting. Not needed for embedded client mode. |
PlatformNetworkPoll | Bare-metal platforms using smoltcp. Called during sleep to process packets. |
PlatformLibc | Bare-metal targets without a C runtime. Provides strlen, memcpy, etc. |
For full method signatures, see the Platform API Reference.
Wiring into nros
Five files need changes to register a new platform. This example adds a fictional “MyOS” platform.
1. Create the platform crate
packages/platform/nros-platform-myos/
Cargo.toml
src/
lib.rs
The crate must have zero nros-* dependencies. It may depend on your RTOS bindings, HAL crates, or embedded-alloc.
2. Add the feature to nros-platform
In packages/platform/nros-platform/Cargo.toml:
[features]
platform-myos = ["dep:nros-platform-myos"]
[dependencies]
nros-platform-myos = { version = "0.1.0", path = "../nros-platform-myos", optional = true }
3. Add the ConcretePlatform alias
In packages/platform/nros-platform/src/resolve.rs:
#![allow(unused)]
fn main() {
#[cfg(feature = "platform-myos")]
pub type ConcretePlatform = nros_platform_myos::MyOsPlatform;
}
4. Propagate through the nros facade
In packages/api/nros/Cargo.toml, add platform-myos to the feature list so users can write nros = { features = ["rmw-zenoh", "platform-myos"] }.
5. Register your platform as an ABI marker
The platform shim crates are gone (Phase 129). Each transport C library’s
platform symbols now come from a default-on C alias translation unit that
forwards to the canonical nros_platform_* ABI — zpico-sys’s
platform-aliases feature for zenoh-pico (z_* / _z_*), and
nros-rmw-xrce’s always-compiled src/platform_aliases.c for XRCE-DDS
(uxr_*). You do not activate or configure a shim.
What you add is a pure ABI marker feature so each transport crate’s
build.rs can do per-platform source selection (e.g. strip the vendor
system/<rtos>/system.c, gate the alias TU’s network section to bare-metal):
# packages/rmw/zenoh/zpico-sys/Cargo.toml (rmw-zenoh)
[features]
myos = [] # ABI marker only — no shim dependency
Only zenoh needs one. XRCE-DDS has no per-platform feature to add:
phase-129.C.1 deleted all of nros-rmw-xrce-cffi’s platform-<rtos>
features, and its build selection is driven by target_os instead, over
a platform-blind transport_nros_udp.c. There is also no crate left to
add one to — the xrce-sys crate was deleted in phase-321 W1.d and the
directory is now a submodule host for micro-xrce-dds-client/ and
micro-cdr/, whose README says in as many words: do not re-add a crate
here. If your port targets XRCE only, skip this step entirely.
The alias TU (default-on platform-aliases) covers the full z_* surface —
memory, sleep, random, time, yield, threading, condvar, clock, and network.
zpico-sys’s build.rs defines NROS_PLATFORM_ALIASES and emits the network
wrappers only where the vendor stack doesn’t already provide them (bare-metal).
If your RTOS’s vendor system.c already supplies these symbols natively (as on
Orin SPE’s FSP FreeRTOS), turn platform-aliases off for that platform to
avoid duplicate-symbol link errors.
Rust path
This is the recommended approach. Create a ZST and implement each capability as inherent methods (not trait impls). nros-platform-cffi exposes these methods as the canonical nros_platform_* C symbols, which the alias TUs call.
Skeleton
#![allow(unused)]
fn main() {
// packages/platform/nros-platform-myos/src/lib.rs
#![no_std]
use core::ffi::c_void;
/// Zero-sized type implementing platform methods for MyOS.
pub struct MyOsPlatform;
// -- Clock --
impl MyOsPlatform {
pub fn clock_ms() -> u64 {
// Call your RTOS tick API, e.g.:
// unsafe { myos_get_tick_count() as u64 }
todo!()
}
pub fn clock_us() -> u64 { Self::clock_ms() * 1000 }
}
// -- Alloc --
impl MyOsPlatform {
pub fn alloc(size: usize) -> *mut c_void {
// unsafe { myos_malloc(size) }
todo!()
}
pub fn realloc(ptr: *mut c_void, size: usize) -> *mut c_void {
// If your RTOS lacks realloc: alloc new, copy, free old
todo!()
}
pub fn dealloc(ptr: *mut c_void) {
// unsafe { myos_free(ptr) }
todo!()
}
}
// -- Sleep --
impl MyOsPlatform {
pub fn sleep_us(us: usize) { Self::sleep_ms(us.div_ceil(1000)); }
pub fn sleep_ms(ms: usize) {
// unsafe { myos_thread_sleep(ms as u32) }
todo!()
}
pub fn sleep_s(s: usize) { Self::sleep_ms(s * 1000); }
}
// -- Threading (stubs for single-threaded, real impls for RTOS) --
impl MyOsPlatform {
pub fn mutex_init(m: *mut c_void) -> i8 {
// Create a mutex via your RTOS API. Store the handle in `m`.
// Return 0 on success, -1 on failure.
todo!()
}
pub fn mutex_lock(m: *mut c_void) -> i8 { todo!() }
pub fn mutex_unlock(m: *mut c_void) -> i8 { todo!() }
// ... remaining threading methods (see traits.rs for the full list)
}
}
Key points
- Inherent methods, not trait impls. The shim calls
ConcretePlatform::clock_ms()directly. The traits intraits.rsdocument the contract, but the ZST uses inherentimplblocks. c_voidpointers for handles. Mutex, condvar, and task handles are opaque#[repr(C)]structs sized to hold your RTOS handle. Cast the*mut c_voidto your internal type.- Recursive mutexes are required. zenoh-pico locks the same mutex recursively. On FreeRTOS this maps to
xSemaphoreCreateRecursiveMutex; on pthreads,PTHREAD_MUTEX_RECURSIVE. - Seed the PRNG. A deterministic seed (like FreeRTOS
rand()starting from 1) causes duplicate zenoh session IDs across QEMU instances. Seed from hardware entropy, IP address, or semihosting wall-clock.
Reference implementation
packages/platform/nros-platform-freertos/src/lib.rs is a complete real-world example covering all categories: clock via xTaskGetTickCount, heap via pvPortMalloc/vPortFree, sleep via vTaskDelay, xorshift32 PRNG, and full threading with tasks, recursive mutexes, and condvars built on counting semaphores.
C/C++ path
If your platform is easier to implement in C, use the nros-platform-cffi C ABI. It is the canonical layer between nros and any C-implemented platform port.
1. Implement the platform symbols
The canonical header lives at
packages/platform/nros-platform-cffi/include/nros/platform.h. It declares
roughly 45 free extern "C" functions — one per platform capability.
Your port supplies a definition for each; the linker resolves them
into the nros binary directly. There is no runtime registration call.
Browse the rendered reference at
/api/platform-cffi/ for per-function
return-value, threading, and blocking conventions.
Include the header in your port:
// my_platform.c
#include <nros/platform.h>
Then define each symbol:
uint64_t nros_platform_clock_ms(void) {
return myos_get_ticks(); // your RTOS tick API
}
void *nros_platform_alloc(size_t size) {
return myos_malloc(size);
}
void nros_platform_dealloc(void *ptr) {
myos_free(ptr);
}
void *nros_platform_realloc(void *ptr, size_t size) {
return myos_realloc(ptr, size);
}
int8_t nros_platform_mutex_init(void *m) {
myos_mutex_t *mx = (myos_mutex_t *)m;
*mx = myos_mutex_create();
return (*mx != NULL) ? 0 : -1;
}
/* ... define every other symbol declared in <nros/platform.h> ... */
2. Link
Compile your translation unit into a static or object library and link it into the nros binary alongside the nros static library. No registration step is required at boot:
int main(void) {
myos_init();
/* nros calls nros_platform_* symbols directly */
nros_executor_t exec;
nros_executor_open(&exec, &config);
/* ... */
}
3. Build configuration
Enable the platform-cffi feature instead of a platform-specific feature:
nros = { features = ["rmw-zenoh", "platform-cffi"] }
The
rmw-zenohfeature is the lowering of the declared RMW: the backend is declared once insystem.toml([system].rmw/[image.<id>].rmw) and the toolchain emits the cargo feature. The feature is the build mechanism, not the user-facing knob — see RFC-0031.
All symbols declared in <nros/platform.h> are required. For capabilities your platform does not support (e.g., threading on single-threaded bare-metal), supply stubs that return 0 for mutex/condvar operations and -1 for nros_platform_task_init.
Networking
There are two paths for providing TCP/UDP sockets to zenoh-pico.
Option A: Rust networking (preferred)
Implement PlatformTcp, PlatformUdp, and PlatformSocketHelpers on your ZST. These methods map to your OS socket API (BSD sockets, lwIP, NetX Duo, etc.).
#![allow(unused)]
fn main() {
impl MyOsPlatform {
pub fn tcp_open(sock: *mut c_void, endpoint: *const c_void, timeout_ms: u32) -> i8 {
// Parse endpoint, call connect(), store fd in sock
todo!()
}
pub fn tcp_read(sock: *const c_void, buf: *mut u8, len: usize) -> usize {
// Call recv() on the socket fd
todo!()
}
pub fn tcp_send(sock: *const c_void, buf: *const u8, len: usize) -> usize {
// Call send() on the socket fd
todo!()
}
pub fn tcp_close(sock: *mut c_void) {
// Close the socket fd
}
}
}
Activate the network shim feature in zpico-sys so the shim provides the _z_open_tcp, _z_read_tcp, etc. C symbols by forwarding to your Rust methods.
For bare-metal with smoltcp, use nros-smoltcp (in packages/drivers/) as the networking driver. It provides PlatformTcp and PlatformUdp implementations using smoltcp’s TCP/UDP sockets. The MAC/PHY driver lives in a sibling driver crate (e.g. lan9118-smoltcp, openeth-smoltcp) and implements smoltcp’s Device trait; nros-smoltcp consumes that Device and exposes the transport to zenoh-pico. Your platform crate implements PlatformNetworkPoll so the sleep loop can process packets — the platform crate stays free of smoltcp itself.
Option B: Keep zenoh-pico’s C network.c
If your platform already has a working zenoh-pico network.c (e.g., freertos/lwip/network.c or unix/network.c), you can compile it directly instead of implementing the Rust networking traits.
In this case, do not activate the network shim feature in zpico-sys. Instead, link the appropriate network.c through your build system. The C file provides the _z_open_tcp, _z_read_tcp, etc. symbols directly, bypassing the Rust shim.
This is the approach used by platforms with mature C networking stacks (lwIP on FreeRTOS, BSD sockets on NuttX, NetX Duo on ThreadX).
Common pitfalls
- Poll-driven clocks. If the clock only advances when you call a function, timeouts and keep-alives break silently. Use a free-running hardware timer.
- Stack overflow on RTOS. The
Executorhas an inline arena on the task stack. Use at least 16384 words (64 KB) for the application task on action examples. - Deterministic PRNG seeds. Duplicate zenoh session IDs cause silent connection failures. Seed from a source that varies across instances.
- Missing recursive mutexes. zenoh-pico re-enters the same mutex. Non-recursive mutexes deadlock.
- QEMU clock drift. Use
-icount shift=autofor QEMU targets so the virtual clock tracks wall time during WFI.
Next steps
- Custom Board Package – create a board crate that ties your platform to specific hardware
- Platform API Reference – complete method signatures for all traits
- Platform Porting Pitfalls – QEMU networking and timing issues
Adding a Platform (CMake Layer)
Scope. This page covers the CMake-side glue: how the build system picks the right platform shim, links the right archives, and dispatches per-app fixups (linker scripts, startup files, ISR vectors). For the runtime traits a new platform must implement in Rust / platform-cffi, see Custom Platform.
Since, porting a new platform to nano-ros is one new file:
nano-ros/cmake/platform/nano-ros-<plat>.cmake
Adding a platform no longer touches the root CMakeLists.txt, never
edits per-example trees, and never duplicates platform helpers across
20+ examples.
Module contract (§A)
Every cmake/platform/nano-ros-<plat>.cmake must expose:
| Symbol | Kind | Purpose |
|---|---|---|
NanoRos::Platform | ALIAS library | Aliased to the platform-specific INTERFACE target; linked into NanoRos::NanoRos umbrella by the root CMakeLists. |
nros_platform_<plat>_iface | INTERFACE library | Carries the platform staticlib + host-system libs + transitive deps. |
nros_platform_link_app(target) | function | Per-app fixups: linker script, startup objects, ISR vectors, RTOS-specific final-link tweaks. Empty for POSIX. |
NROS_PLATFORM_LINK_FEATURES | cache variable | Default link-feature set for this platform (e.g. tcp udp_unicast). |
The root CMakeLists.txt dispatches via:
include("${CMAKE_CURRENT_SOURCE_DIR}/cmake/platform/nano-ros-${NANO_ROS_PLATFORM}.cmake")
When the user’s project sets NANO_ROS_PLATFORM=foo before
add_subdirectory(nano-ros), your module at
cmake/platform/nano-ros-foo.cmake runs and supplies the four contract
elements above.
Minimal skeleton
# cmake/platform/nano-ros-foo.cmake
if(DEFINED _NROS_PLATFORM_FOO_INCLUDED)
return()
endif()
set(_NROS_PLATFORM_FOO_INCLUDED TRUE)
set(NROS_PLATFORM_LINK_FEATURES tcp udp_unicast
CACHE STRING "Default link features for the Foo platform")
# Build / pull in the platform staticlib. May be add_subdirectory(...)
# into packages/platform/nros-platform-foo/ for a Cargo + cmake hybrid, or
# may declare an IMPORTED target pointing at a prebuilt RTOS archive.
add_subdirectory(
"${CMAKE_CURRENT_LIST_DIR}/../../packages/platform/nros-platform-foo"
nros_platform_foo_build)
add_library(nros_platform_foo_iface INTERFACE)
if(TARGET nros_platform_foo)
target_link_libraries(nros_platform_foo_iface INTERFACE nros_platform_foo)
endif()
if(NOT TARGET NanoRos::Platform)
add_library(NanoRos::Platform ALIAS nros_platform_foo_iface)
endif()
function(nros_platform_link_app target)
# Per-app fixups go here. Linker scripts, startup objects, ISR
# vectors. Delegate to cmake/board/nano-ros-board-${NANO_ROS_BOARD}.cmake
# when the platform supports multiple boards.
if(DEFINED NANO_ROS_BOARD)
include("${CMAKE_CURRENT_LIST_DIR}/../board/nano-ros-board-${NANO_ROS_BOARD}.cmake")
if(COMMAND nros_board_link_app)
nros_board_link_app(${target})
endif()
endif()
endfunction()
That’s it. The root CMakeLists picks up the new platform automatically
— the validation block resolves cmake/platform/nano-ros-foo.cmake and
fatals out only when no such file exists.
Board overlays
When your platform spans multiple boards (different MCUs, link scripts, peripheral inits), carve a board overlay:
nano-ros/cmake/board/nano-ros-board-<board>.cmake
Board overlays provide nros_board_link_app(target) and run
inside nros_platform_link_app when the user sets NANO_ROS_BOARD.
The board layer owns:
- Linker script selection (
target_link_options(${target} PRIVATE -T<script>)) - Startup object emission (
crt0.o, vector table, etc.) - MCU-specific final-link flags
See cmake/board/nano-ros-board-mps2-an385.cmake for a working example.
Where the existing layer-2 helpers live
For RTOS ports that compose the kernel + netstack + glue inside CMake
(FreeRTOS, ThreadX, NuttX), the per-RTOS helper functions
(nros_freertos_build_kernel, nros_threadx_compose_platform, …)
stay at packages/api/nros-c/cmake/nros-<rtos>.cmake. The
modules include(...) those helpers — the per-platform file is the
dispatch entry, not the implementation.
See also
- Custom Platform — Rust-side trait implementations (clock, alloc, threading, networking)
- Custom Board Package — the Rust board crate that carries the linker script and board-specific drivers
- Build as a CMake subdirectory
— user-facing intro to
add_subdirectory(nano-ros)
Custom Transport
nano-ros lets you plug a custom transport (USB-CDC, BLE GATT, RS-485
with framing, ring-buffer loopback, semihosting bridge, …) at runtime,
without changing the board crate, Cargo features, or rebuilding. This
is the runtime equivalent of micro-ROS’s
rmw_uros_set_custom_transport(framing, params, open, close, write, read).
When to use
- You have a serial-over-USB device that doesn’t fit
serial,ethernet, orwifi. - You’re bridging through a host-side proxy (semihosting, RTT, OpenOCD).
- You’re prototyping a transport on a target where the static path isn’t built yet.
- You want a single firmware image that picks transport at boot from a config block.
If your transport fits one of the prebuilt static variants
(platform-posix, platform-freertos, platform-nuttx,
platform-threadx, platform-zephyr, platform-bare-metal), prefer
that — the runtime hook trades binary-size optimisation for
flexibility.
API layering (L0 / L1 / L2)
The custom transport surface is the project’s first canonical-C-ABI
interface, designed per
docs/design/0006-portable-rmw-platform-interface.md:
| Layer | Owns | Crates / files |
|---|---|---|
| L0 — canonical C ABI | #[repr(C)] struct + abi_version: u32 field + four unsafe extern "C" fn pointers + user_data: *mut c_void | nros-rmw::custom_transport (Rust source); <nros/transport.h> is the cbindgen-emitted C header |
| L1 — language wrappers | mechanical glue, no new design decisions | nros-rmw::set_custom_transport (Rust); nros_set_custom_transport (C); nros::set_custom_transport (C++) |
| L2 — typed app API | n/a — transport is platform-side | — |
All design decisions live at L0. A new feature — say, a flush
callback — lands in the L0 struct first; L1 wrappers follow
mechanically.
The same canonical-C-ABI pattern was generalised to
the full RMW backend surface (nros_rmw_vtable_t). For the
host-language policy that decides whether a given backend lives in
Rust, C, or C++, see RMW Backends — Host-Language Policy.
ABI versioning
The abi_version: u32 field at the head of the struct is mandatory.
Consumers must fill in NROS_TRANSPORT_OPS_ABI_VERSION_V1 (Rust:
nros_rmw::NROS_TRANSPORT_OPS_ABI_VERSION_V1; C:
NROS_TRANSPORT_OPS_ABI_VERSION_V1; C++: filled in automatically by
nros::set_custom_transport). Mismatched versions are rejected at
registration time with NROS_RMW_RET_INCOMPATIBLE_ABI (-14); the
slot stays whatever it was before the bad call.
The version bumps under two rules:
- Major (e.g.
V1→V2): existing fields are removed or reordered. Old consumers fail cleanly via the version check. - Minor (struct gains an appended fn pointer / data field):
version stays the same. New consumers detect the new fn via the
size of the trailing
_reservedregion. Today there’s no such appendage — V1 is the inaugural version.
Implementing in another language
The L0 struct is plain C ABI (#[repr(C)] Rust ↔ struct C).
Any language with C-FFI support can author both sides of the
boundary — Zig, Python (ctypes / cffi), Lua-FFI, Go (cgo),
Swift (@_cdecl), etc.
The reference implementation of “custom transport written in pure C”
lives at
packages/rmw/cffi/tests/c_stubs/c_stub_transport.c.
~80 LOC; no Rust headers / cbindgen output / Rust types involved on
the C side. Use it as a template for ports to other languages.
The corresponding Rust integration test
(tests/c_stub_transport.rs)
exercises the round-trip: register the C-built struct → drive each
fn pointer from Rust → confirm the C-side counters bumped → confirm
abi_version mismatch is rejected. Run via:
cargo test -p nros-rmw-cffi --features c-stub-test --test c_stub_transport
API
Rust
#![allow(unused)]
fn main() {
use core::ffi::c_void;
use nros_rmw::{NrosTransportOps, set_custom_transport};
unsafe extern "C" fn my_open(_ud: *mut c_void, _params: *const c_void) -> i32 { 0 }
unsafe extern "C" fn my_close(_ud: *mut c_void) {}
unsafe extern "C" fn my_write(_ud: *mut c_void, buf: *const u8, len: usize) -> i32 {
// hand `buf[..len]` to the underlying medium, return 0 on success
0
}
unsafe extern "C" fn my_read(_ud: *mut c_void, buf: *mut u8, len: usize, timeout_ms: u32) -> i32 {
// read up to `len` bytes within `timeout_ms`, return non-negative count
0
}
unsafe {
set_custom_transport(Some(NrosTransportOps {
abi_version: nros_rmw::NROS_TRANSPORT_OPS_ABI_VERSION_V1,
_reserved: 0,
user_data: my_uart_handle as *mut c_void,
open: my_open,
close: my_close,
write: my_write,
read: my_read,
})).expect("abi_version must match runtime");
}
}
C
#include <nros/transport.h>
static nros_ret_t my_open(void *ud, const void *params) {
(void)params;
return my_uart_open((my_uart_t *)ud);
}
static void my_close(void *ud) { my_uart_close((my_uart_t *)ud); }
static nros_ret_t my_write(void *ud, const uint8_t *buf, size_t len) {
return my_uart_write((my_uart_t *)ud, buf, len);
}
static int32_t my_read(void *ud, uint8_t *buf, size_t len, uint32_t timeout_ms) {
return my_uart_read((my_uart_t *)ud, buf, len, timeout_ms);
}
int main(void) {
nros_transport_ops_t ops = {
.abi_version = NROS_TRANSPORT_OPS_ABI_VERSION_V1,
._reserved = 0,
.user_data = &g_uart,
.open = my_open,
.close = my_close,
.write = my_write,
.read = my_read,
};
nros_set_custom_transport(&ops);
// ... continue with nros_support_init, nros_node_init, etc.
}
C++
#include <nros/transport.hpp>
nros::TransportOps ops;
ops.user_data = &g_uart;
ops.open = [](void *ud, const void*) -> int { return my_uart_open((MyUart*)ud); };
ops.close = [](void *ud) { my_uart_close((MyUart*)ud); };
ops.write = [](void *ud, const uint8_t *buf, std::size_t len) -> int {
return my_uart_write((MyUart*)ud, buf, len);
};
ops.read = [](void *ud, uint8_t *buf, std::size_t len, std::uint32_t to) -> std::int32_t {
return my_uart_read((MyUart*)ud, buf, len, to);
};
auto r = nros::set_custom_transport(ops);
NROS_TRY(r);
Captureless lambdas only. All four fields are raw C function pointers, not
std::function. Pass per-instance state throughuser_data.
Threading contract
| Constraint | Rationale |
|---|---|
read and write MAY be invoked concurrently from different threads on threaded backends. | The zenoh-pico backend runs a dedicated read-task thread on multi-threaded platforms (POSIX, Zephyr, FreeRTOS, NuttX), so read runs there while the application/tx thread drives write. Single-threaded platforms (bare-metal, ThreadX) instead serialise both through the spin-once / drive_io path. A custom transport must therefore be full-duplex safe: do not serialise read and write behind one lock held across a blocking read, or the read thread will starve the writer and deadlock session declaration. Most media (TCP, full-duplex UART/USB-CDC) already allow concurrent read and write; mirror that. With std::net::TcpStream, store the stream directly and use &TcpStream (which implements Read/Write) from both callbacks rather than a Mutex<TcpStream>. |
| Callbacks must NOT be invoked from interrupt context. | The runtime path may take internal locks; ISR context could deadlock. Wrap ISR-driven hardware in a queue + read poller. |
user_data must outlive the transport’s active period. | The runtime never copies it. Lifetime is from the first callback invocation through close returning. |
set_custom_transport must be called BEFORE nros_support_init. | The active backend reads the slot during Rmw::open. Calling after init is implementation-defined — backends may reject with NROS_RET_ALREADY_INIT. |
Return-code conventions
open/writereturn0(NROS_RMW_RET_OK) on success, a negativenros_ret_t(e.g.NROS_RMW_RET_TIMEOUT,NROS_RMW_RET_ERROR) on failure.readreturns the non-negative byte count on success (may be less thanlen); a negativenros_ret_ton error / timeout.closereturns nothing — failures during teardown are best-effort.
Framing
Some transports need wire-level framing (HDLC for serial, length-prefix for stream sockets). The active backend decides whether framing is applied; the user vtable just sees raw bytes.
- XRCE-DDS: pass
framing=trueto the backend’sinit_transport_from_custom_ops(framing)for byte-stream transports (UART, USB-CDC). Passframing=falsefor packet-oriented transports (UDP, BLE GATT). - zenoh-pico: framing is built into the wire protocol — always
framing=falseregardless of the underlying medium.
Backend coverage
| Backend | Status |
|---|---|
| XRCE-DDS | ✅ Wired. nros_rmw_xrce::init_transport_from_custom_ops(framing) pulls the registered vtable into uxr_set_custom_transport_callbacks via four C trampolines. |
| zenoh-pico | 🟡 Deferred (zenoh). zenoh-pico’s custom-link API needs a per-platform _z_link_t shim; tracked separately. Zenoh users with custom transports today fork a zpico-platform-* crate. |
| Cyclone DDS | ➖ N/A. Cyclone DDS binds the platform’s network stack directly (POSIX / NSOS / lwIP / NetX-Duo BSD sockets) rather than going through the nano-ros custom-transport vtable; “porting” Cyclone to a new link means providing those sockets, not a transport plug-in. |
Loopback test
packages/rmw/xrce/nros-rmw-xrce/tests/custom_transport.rs exercises the
slot lifecycle + the XRCE bridge round-trip with stub callbacks (no
real session). Run via:
cargo test -p nros-rmw-xrce --features platform-posix \
--test custom_transport
See also
<nros/transport.h>— C header<nros/transport.hpp>— C++ headernros_rmw::custom_transport— Rust sourcedocs/roadmap/phase-115-runtime-transport-vtable.md— phase doc- Custom platform — when you need more than just transport
Porting a Custom RMW Backend
nano-ros ships with three RMW backends – zenoh-pico, Micro-XRCE-DDS, and Cyclone DDS. To add your own transport (MQTT, a proprietary bus, etc.), implement a small set of traits or fill in a C function table.
Two paths are available:
- Rust path – implement the
nros-rmwtraits directly. - C/C++ path – fill in
nros_rmw_vtable_tand register it at startup vianros-rmw-cffi.
What you implement
Your backend provides concrete types for six traits:
Rmw -- factory: opens a Session from RmwConfig
Session -- connection lifecycle, creates handles
Publisher -- send CDR-encoded messages
Subscription -- non-blocking receive (poll-based)
ServiceTrait -- receive requests, send replies
ClientTrait -- send requests, poll for replies
(Phase-301 aligned the vocabulary with upstream rmw: Subscriber →
Subscription, ServiceServerTrait → ServiceTrait,
ServiceClientTrait → ClientTrait, and the create methods below
match.)
Most methods have default implementations. The required methods per trait are:
| Trait | Required methods |
|---|---|
Rmw | open() |
Session | create_publisher(), create_subscription(), create_service(), create_client(), close() |
Publisher | publish_raw(), buffer_error(), serialization_error() |
Subscription | take_serialized(), deserialization_error() |
ServiceTrait | take_request(), send_response() |
ClientTrait | send_request_raw(), take_response_raw() |
For full trait signatures and associated types, see RMW API Reference.
Use nros-platform for networking
Call ConcretePlatform::tcp_open() / udp_bind() from nros-platform
rather than OS sockets directly. This makes your backend portable across
every platform (POSIX, Zephyr, FreeRTOS, NuttX, ThreadX, bare-metal). If
your transport library already abstracts networking (like zenoh-pico does),
you can use its own I/O layer instead.
Rust path
1. Create the crate
Create packages/myproto/nros-rmw-myproto/ with nros-rmw and
nros-core as dependencies (both default-features = false for
no_std support). Follow the std/alloc feature forwarding pattern
used by the existing backends.
2. Implement the traits
#![allow(unused)]
#![no_std]
fn main() {
use nros_rmw::*;
#[derive(Default)]
pub struct MyProtoRmw {
// Optional pre-open configuration (agent address, TLS CA, serial
// device handle, …) can live here and move into the Session at
// `open` time. Every backend in-repo implements `Default` so the
// caller can spell the common case as
// `MyProtoRmw::default().open(&config)`.
}
impl Rmw for MyProtoRmw {
type Session = MyProtoSession;
type Error = TransportError;
fn open(self, config: &RmwConfig) -> Result<MyProtoSession, TransportError> {
todo!() // Parse config.locator, connect, map config.domain_id
}
}
pub struct MyProtoSession { /* connection state */ }
impl Session for MyProtoSession {
type Error = TransportError;
type PublisherHandle = MyProtoPub;
type SubscriptionHandle = MyProtoSub;
type ServiceHandle = MyProtoServer;
type ClientHandle = MyProtoClient;
fn create_publisher(&mut self, t: &TopicInfo, q: QoSProfile)
-> Result<MyProtoPub, TransportError> { todo!() }
fn create_subscription(&mut self, t: &TopicInfo, q: QoSProfile)
-> Result<MyProtoSub, TransportError> { todo!() }
fn create_service(&mut self, s: &ServiceInfo, q: QoSProfile)
-> Result<MyProtoServer, TransportError> { todo!() }
fn create_client(&mut self, s: &ServiceInfo, q: QoSProfile)
-> Result<MyProtoClient, TransportError> { todo!() }
fn close(&mut self) -> Result<(), TransportError> { todo!() }
fn drive_io(&mut self, timeout_ms: i32) -> Result<(), TransportError> {
let _ = timeout_ms; Ok(()) // poll network, dispatch to buffers
}
}
pub struct MyProtoPub;
impl Publisher for MyProtoPub {
type Error = TransportError;
fn publish_raw(&self, data: &[u8]) -> Result<(), TransportError> { todo!() }
fn buffer_error(&self) -> TransportError { TransportError::BufferTooSmall }
fn serialization_error(&self) -> TransportError { TransportError::SerializationError }
}
pub struct MyProtoSub;
impl Subscription for MyProtoSub {
type Error = TransportError;
fn take_serialized(&mut self, buf: &mut [u8])
-> Result<Option<usize>, TransportError> { todo!() }
fn deserialization_error(&self) -> TransportError { TransportError::DeserializationError }
}
pub struct MyProtoServer;
impl ServiceTrait for MyProtoServer {
type Error = TransportError;
fn take_request<'a>(&mut self, buf: &'a mut [u8])
-> Result<Option<ServiceRequest<'a>>, TransportError> { todo!() }
fn send_response(&mut self, seq: i64, data: &[u8])
-> Result<(), TransportError> { todo!() }
}
pub struct MyProtoClient;
impl ClientTrait for MyProtoClient {
type Error = TransportError;
fn send_request_raw(&mut self, req: &[u8])
-> Result<(), TransportError> { todo!() }
fn take_response_raw(&mut self, buf: &mut [u8])
-> Result<Option<usize>, TransportError> { todo!() }
}
}
Pick the right TransportError variant
The runtime maps every TransportError variant to a named
rmw_ret_t constant at the C boundary. Picking a specific
variant gives the caller actionable information instead of an opaque
“failed somehow”:
| Variant | When to use |
|---|---|
Timeout | Bounded wait elapsed (drive_io). |
WouldBlock | Resource momentarily unavailable; caller should retry. Distinct from NoData — NoData means the queue is empty, WouldBlock means it’s contended. |
NoData | Non-blocking receive found the queue empty. |
BufferTooSmall | Caller’s &mut [u8] smaller than the next message. |
MessageTooLarge | Incoming message exceeds the backend’s static capacity. |
InvalidArgument | NULL pointer, out-of-range value, missing required config. |
Unsupported | Backend genuinely cannot perform this operation (e.g., uORB has no service surface). |
IncompatibleQos | Endpoints’ QoS profiles differ in a way the backend cannot reconcile. |
TopicNameInvalid | Topic / service name failed validation (empty, too long, illegal characters). |
BadAlloc | Allocation failed on a heap-equipped backend. |
LoanNotSupported | Lending requested on an entity that doesn’t support it. |
ConnectionFailed / Disconnected | Transport-level failures. |
PublisherCreationFailed / SubscriberCreationFailed / ServiceServerCreationFailed / ServiceClientCreationFailed | Catch-all for entity creation when no more specific code applies. |
Backend(&str) / BackendDynamic(String) | Backend-specific diagnostic that doesn’t map to any of the above. |
The Rust trait surface is the source of truth; the C header
<nros/rmw_ret.h> exposes the matching NROS_RMW_RET_* constants
for C porters.
Factory shape
Rmw::open consumes self, not a &self. That shape asks every
backend to treat its factory type as a value that carries any
pre-open configuration (agent address, serial device, TLS CA, …)
and moves that state into the returned Session:
// Default constructor (picks config from `&RmwConfig`):
let session = MyProtoRmw::default().open(&config)?;
// Explicit constructor when the backend has pre-open state
// that isn't in the middleware-agnostic `RmwConfig`:
let session = MyProtoRmw::with_endpoint("10.0.0.1", 7447).open(&config)?;
Conventions:
- Every backend implements
Default. Keeps the common call site short and lets generic code build a factory without knowing the backend type. - Provide
new(...)/with_*(...)helpers for backend-specific pre-open state. Don’t bake it intoRmwConfig— that type is the middleware-agnostic contract. If your backend needs an agent IP, a serial device path, or a certificate slot, take it on the factory constructor. - Read your own environment variables in
<Backend>::from_env()if you want zero-boilerplate POSIX configuration. The shippedExecutorConfig::from_env()only reads the middleware-agnosticNROS_LOCATOR/NROS_SESSION_MODE/ROS_DOMAIN_ID; anything backend-specific (e.g.NROS_XRCE_AGENT) stays on the backend side. - Post-open state lives in
Session, never instatic mut. Theopen(self, …)signature makes it natural to move the configured transport into theSessionreturn value, which then owns the connection for the rest of its lifetime. A backend that still usesstatic mutsession-global state will fail any multi-session test (backend.open(...)twice in one process should succeed).
3. Wire into nros
Three changes are needed to integrate the new backend:
a) In nros/Cargo.toml, add a feature and optional dependency:
rmw-myproto = ["dep:nros-rmw-myproto", "nros-node/rmw-myproto"]
b) In nros-node, add the concrete session type alias:
#![allow(unused)]
fn main() {
#[cfg(feature = "rmw-myproto")]
pub type ConcreteSession = nros_rmw_myproto::MyProtoSession;
}
c) Add compile_error! guards to enforce mutual exclusivity with the
other backends (see existing guards in nros-node/src/session.rs).
Applications then select your backend with
nros = { features = ["rmw-myproto", "platform-posix"] }.
C/C++ path
If your transport library is C or C++, use nros-rmw-cffi — a vtable
of C function pointers that map one-to-one onto the Rust trait methods.
The hand-written header lives at
packages/core/nros-rmw-abi/include/nros/rmw_vtable.h. Browse the
rendered reference at /api/rmw-cffi/ for
per-field return-value, threading, and blocking conventions.
1. Fill in the vtable
#include <nros/rmw_vtable.h>
#include <nros/rmw_ret.h>
#include <nros/rmw_entity.h>
// -- Session lifecycle --
static rmw_ret_t my_create_session(const char *locator, uint8_t mode,
uint32_t domain_id, const char *node_name,
rmw_session_t *out) {
/* Connect. On success: write out->backend_data with your session
* pointer, return NROS_RMW_RET_OK. On failure: return one of the
* named codes (NROS_RMW_RET_INVALID_ARGUMENT, NROS_RMW_RET_TIMEOUT,
* NROS_RMW_RET_BAD_ALLOC, NROS_RMW_RET_ERROR). out->node_name is
* already set by the runtime — read it for diagnostics. */
}
static rmw_ret_t my_destroy_session(rmw_session_t *session) { /* ... */ }
static rmw_ret_t my_drive_io(rmw_session_t *session, int32_t timeout_ms) {
/* Dispatch network I/O for up to timeout_ms; return NROS_RMW_RET_OK
* on success, NROS_RMW_RET_TIMEOUT / NROS_RMW_RET_ERROR otherwise. */
}
// -- Publisher --
static rmw_ret_t my_create_publisher(
rmw_session_t *session,
const char *topic, const char *type_name, const char *type_hash,
uint32_t domain_id, const rmw_qos_profile_t *qos,
const rmw_publisher_options_t *options, /* hints (tx_express); NULL = defaults */
rmw_publisher_t *out) {
/* Runtime has already filled out->topic_name / type_name / qos.
* Backend writes out->backend_data with its publisher handle.
*
* Do NOT write out->can_loan_messages: the runtime DERIVES it from
* whether your vtable fills borrow_loaned_message, and overwrites
* whatever you put there (issue 0814). You advertise the
* loan_publish / commit_publish primitive by IMPLEMENTING it, which
* is the one statement that cannot disagree with itself. To refuse a
* loan for one particular publisher, return RMW_RET_UNSUPPORTED from
* borrow_loaned_message. */
}
static void my_destroy_publisher(rmw_publisher_t *publisher) { /* ... */ }
static rmw_ret_t my_publish_raw(rmw_publisher_t *publisher,
const uint8_t *data, size_t len) { /* ... */ }
// -- Subscription --
static rmw_ret_t my_create_subscription(
rmw_session_t *session,
const char *topic, const char *type_name, const char *type_hash,
uint32_t domain_id, const rmw_qos_profile_t *qos,
const rmw_subscription_options_t *options, /* hints (rx_buffer_hint); NULL = defaults */
rmw_subscription_t *out) { /* ... */ }
static void my_destroy_subscription(rmw_subscription_t *subscription) { /* ... */ }
static int32_t my_take_serialized(rmw_subscription_t *subscription,
uint8_t *buf, size_t buf_len) {
/* >= 0 = bytes received (0 = no data),
* negative rmw_ret_t (e.g. NROS_RMW_RET_NO_DATA, _BUFFER_TOO_SMALL). */
}
static int32_t my_has_data(rmw_subscription_t *subscription) { /* 1 = yes, 0 = no */ }
// -- Service (server side) --
static rmw_ret_t my_create_service(
rmw_session_t *session,
const char *service, const char *type_name, const char *type_hash,
uint32_t domain_id, const rmw_qos_profile_t *qos,
rmw_service_t *out) { /* ... */ }
static void my_destroy_service(rmw_service_t *server) { /* ... */ }
static int32_t my_take_request(rmw_service_t *server,
uint8_t *buf, size_t buf_len, int64_t *seq_out) { /* ... */ }
static int32_t my_has_request(rmw_service_t *server) { /* 1 = yes, 0 = no */ }
static rmw_ret_t my_send_reply(rmw_service_t *server,
int64_t seq, const uint8_t *data, size_t len) { /* ... */ }
// -- Client --
static rmw_ret_t my_create_client(
rmw_session_t *session,
const char *service, const char *type_name, const char *type_hash,
uint32_t domain_id, const rmw_qos_profile_t *qos,
rmw_client_t *out) { /* ... */ }
static void my_destroy_client(rmw_client_t *client) { /* ... */ }
static rmw_ret_t my_send_request_raw(rmw_client_t *client,
const uint8_t *request, size_t req_len) {
/* Send without blocking for the reply; return NROS_RMW_RET_OK. */
}
static int32_t my_take_response_raw(rmw_client_t *client,
uint8_t *reply_buf, size_t reply_buf_len) {
/* >= 0 = reply bytes, NROS_RMW_RET_NO_DATA = no reply yet. */
}
static const nros_rmw_vtable_t MY_RMW = {
.create_session = my_create_session,
.destroy_session = my_destroy_session,
.drive_io = my_drive_io,
.create_publisher = my_create_publisher,
.destroy_publisher = my_destroy_publisher,
.publish = my_publish_raw,
.create_subscription = my_create_subscription,
.destroy_subscription = my_destroy_subscription,
.take = my_take_serialized,
.has_data = my_has_data,
.create_service = my_create_service,
.destroy_service = my_destroy_service,
.take_request = my_take_request,
.has_request = my_has_request,
.send_response = my_send_reply,
.create_client = my_create_client,
.destroy_client = my_destroy_client,
.send_request = my_send_request_raw,
.take_response = my_take_response_raw,
};
2. Register before opening a session
int main(void) {
nros_rmw_cffi_register(&MY_RMW); // before any nros call
/* now use the nano-ros C or C++ API normally */
}
Build the static library with the matching feature combo:
cargo build -p nros-c --features rmw-cffi,platform-posix,ros-humble
3. Lifecycle and threading contract
The Rust traits behind this vtable
(nros_rmw::Session,
Publisher, …) document the
per-method contract: thread safety, buffer ownership, blocking
allowance. The C vtable inherits the same rules:
- The vtable itself is registered once and read concurrently. Function pointers must be safe to invoke from any executor thread.
drive_iomay block up totimeout_ms; it must not hold application-visible locks across the wait.publish,take, andsend_responsemay run concurrently from different executor threads — your backend handles serialisation.take,take_request, andtake_responseare non-blocking: return0/NROS_RMW_RET_NO_DATAif nothing is ready. The executor will retry afterdrive_io.- There is no blocking client slot (phase-301 deleted the deprecated
call_raw):send_request+take_responseis the one request/reply path; the executor drives the wait.
All strings are null-terminated and borrowed (caller owns the
storage). Entities are typed structs with an opaque backend_data
slot — the runtime fills metadata fields (topic_name, qos, …)
before calling create_*; the backend writes backend_data. Return
convention: NROS_RMW_RET_OK = success, negative = named
rmw_ret_t constant, positive = byte count (only on
take_*).
Example: local echo RMW
Loops published messages back to subscribers – no real transport. Only pub/sub shown; service types are no-op stubs.
#![allow(unused)]
fn main() {
static mut ECHO_BUF: [u8; 1024] = [0; 1024];
static mut ECHO_LEN: usize = 0;
pub struct EchoPub;
impl Publisher for EchoPub {
type Error = TransportError;
fn publish_raw(&self, data: &[u8]) -> Result<(), TransportError> {
unsafe {
let len = data.len().min(ECHO_BUF.len());
ECHO_BUF[..len].copy_from_slice(&data[..len]);
ECHO_LEN = len;
}
Ok(())
}
fn buffer_error(&self) -> TransportError { TransportError::BufferTooSmall }
fn serialization_error(&self) -> TransportError { TransportError::SerializationError }
}
pub struct EchoSub;
impl Subscription for EchoSub {
type Error = TransportError;
fn take_serialized(&mut self, buf: &mut [u8]) -> Result<Option<usize>, TransportError> {
unsafe {
if ECHO_LEN == 0 { return Ok(None); }
let len = ECHO_LEN;
buf[..len].copy_from_slice(&ECHO_BUF[..len]);
ECHO_LEN = 0;
Ok(Some(len))
}
}
fn deserialization_error(&self) -> TransportError { TransportError::DeserializationError }
}
}
Wire EchoPub/EchoSub into an EchoSession the same way as the
skeleton above – create_publisher returns Ok(EchoPub), etc. The
Rmw::open() impl just returns Ok(EchoSession) unconditionally.
What the ROS 2 ecosystem expects
Implementing the six traits compiles and runs, but a backend that stops
there will not interoperate cleanly with ros2 CLI, RQt, or
rmw_zenoh_cpp nodes. Real ROS 2 interop requires four extra
invariants the traits do not express:
1. Discovery / liveliness tokens
ros2 node list, ros2 topic list, ros2 service list rely on
discovery traffic. How you emit it depends on the backend protocol:
- Zenoh-flavoured backends: publish a liveliness token per endpoint
under
@ros2_lv/<domain>/<zid>/<entity_kind>/<id>/…. See rmw-zenoh-protocol.md for the exact key grammar. - DDS-flavoured backends: use the SPDP/SEDP discovery traffic that your DDS stack provides, plus the ROS 2–specific USER_DATA payload (node name, namespace, enclave).
If your backend is brand new (not wire-compatible with zenoh or DDS),
you still need some discovery channel for ros2 CLI tools to find
your endpoints. The traits currently do not cover this — discovery
happens inside create_publisher / create_subscription /
create_service / create_client as a side effect.
2. RMW attachments (per-message metadata)
Every published message carries ROS 2 metadata that consumers read
through MessageInfo:
| Field | Size | Meaning |
|---|---|---|
sequence_number | 8 bytes | int64 LE — monotonic per publisher |
timestamp | 8 bytes | int64 LE — source nanoseconds |
gid | 16 bytes | random per publisher, constant over its lifetime |
- Zenoh: the attachment rides alongside the payload as a zenoh
Attachment. Humble uses a simple concatenation; Jazzy onward uses a VLE-encodedgid_lengthprefix. - DDS: sample identity and source timestamp fall out of the DDS sample info — your backend only has to forward them.
If you skip this, subscriptions built with the .message_info() builder
(node.subscription(topic).generic(...).message_info().build(cb), or
.typed::<M>().message_info()) on consumers always report
MessageInfo::default(), and downstream features (safety-e2e checks,
source-timestamp ordering) silently degrade.
3. Actions decompose into five underlying channels
ROS 2 actions are not a transport primitive — they are a pattern built on services and topics. Each action server exposes:
| Sub-entity | Kind | Name suffix |
|---|---|---|
send_goal | Service | _action/send_goal |
cancel_goal | Service | _action/cancel_goal |
get_result | Service | _action/get_result |
feedback | Topic (pub) | _action/feedback |
status | Topic (pub) | _action/status |
A backend that implements the six core traits automatically supports
actions — nros-node composes the five channels itself. The only
backend-specific piece is the key / topic construction, which the
Session::create_service and create_publisher methods already
handle.
4. Type hashes
| ROS 2 distro | Type hash |
|---|---|
| Humble | literal string "TypeHashNotSupported" |
| Iron / Jazzy / Rolling | RIHS01_<sha256_hex> computed from the IDL |
Humble is the DEFAULT, not the ceiling: --ros-edition {humble,iron,jazzy,rolling}
selects the edition, and iron/jazzy emit the real REP-2011 hash — computed
in-tree, not read out of an ament index
(phase-41,
complete). A new backend inherits that: TopicInfo::type_hash arrives already
filled in for the selected edition, so the backend’s job is to put it in the
right place on its own wire, not to compute it. On zenoh that place is the
keyexpr tail; get it wrong and delivery fails silently, because a peer that
disagrees about the hash simply never matches.
5. QoS mapping
ROS 2 QoS (reliability, durability, history, depth) maps differently onto each backend:
- Zenoh has reliable/best-effort and a
KEEP_LAST(N)/KEEP_ALLbuffering policy — direct mapping. - DDS has native QoS — almost 1:1.
- Custom backends must either honour the requested QoS or document
which fields are ignored. A
best_effortpublisher matched with areliablesubscriber is a QoS mismatch in ROS 2 — the transport must refuse the subscription (or flag it at runtime) rather than silently lose messages.
Further reading
- RMW API Reference – full trait signatures, QoS profiles, error types, configuration structs.
- RMW API Design – architectural motivation and comparison with the ROS 2 rmw interface.
- RMW API: Differences from upstream
rmw.h– side-by-side C-API comparison for porters coming from upstream ROS 2 RMW backends; covers the renames, the collapses, and the reason for each. - Zenoh-pico Symbol Reference – FFI symbol mapping for the zenoh-pico backend (useful as a reference for how an existing backend is structured).
Platform Porting Pitfalls
Hard-won lessons from porting nros to bare-metal ARM (QEMU MPS2-AN385), ESP32-C3, STM32F4, FreeRTOS, NuttX, ThreadX, and Zephyr. Read this before adding a new platform or debugging mysterious failures on an existing one.
Memory Layout
DMA buffer placement
On MCUs with multiple SRAM regions, DMA descriptors and buffers must be in DMA-accessible memory. The STM32F4 has 64 KB of CCM RAM that is NOT DMA-accessible — placing Ethernet descriptors there causes silent data corruption with no error interrupt.
Fix: Use linker script sections to force placement:
/* stm32f4.x — place DMA descriptors in SRAM1, not CCM */
.eth_descriptors (NOLOAD) : ALIGN(4) {
*(.eth_descriptors .eth_descriptors.*)
} > RAM
In Rust, annotate the buffer:
#![allow(unused)]
fn main() {
#[link_section = ".eth_descriptors"]
static mut ETH_DMA_DESC: MaybeUninit<[u32; 256]> = MaybeUninit::uninit();
}
Stack sizing
Bare-metal and RTIC applications share a single stack. FreeRTOS/Zephyr give each task its own stack. In both cases, zenoh-pico’s internal processing needs significant stack space:
| Context | Minimum stack |
|---|---|
| Bare-metal / RTIC main stack | 8 KB |
| FreeRTOS zenoh read task | 2 KB |
| FreeRTOS zenoh lease task | 1 KB |
| Zephyr zpico work queue thread | 2 KB |
Stack overflow on embedded targets causes silent corruption — there is no
guard page. FreeRTOS can detect overflow with configCHECK_FOR_STACK_OVERFLOW=2
and the high-water-mark API (uxTaskGetStackHighWaterMark), but only after the
fact.
Heap sizing
zenoh-pico allocates heap memory during session open, publisher/subscriber creation, and message processing. Typical heap consumption:
| Platform | Minimum heap |
|---|---|
| MPS2-AN385 (bare-metal) | 64 KB |
| ESP32-C3 | 64 KB (~16 KB static + dynamic) |
| STM32F4 | 64 KB |
| FreeRTOS (lwIP + zenoh) | 256 KB (configTOTAL_HEAP_SIZE) |
Undersized heaps cause _Z_ERR_SYSTEM_OUT_OF_MEMORY (-78) during
z_open() or entity creation, not during message I/O.
Rust struct move invalidation
When Rust returns a struct from a function, it moves the value to a new
address. If C code stored a pointer to the original location (e.g., during
init()), that pointer is now dangling.
Symptom: Callback crashes or garbage field values after struct is returned from an init function.
Fix: Use static storage, Box::pin(), or index-based references instead of
raw pointers. See Troubleshooting: Subscriber Callback Crashes.
Clock Sources
Use hardware timers, not poll-driven counters
Every platform must provide a monotonic millisecond clock via smoltcp_clock_now_ms() (for
smoltcp timestamping) and z_clock_now() / z_clock_elapsed_*() (for zenoh-pico timeouts).
This clock must be backed by a hardware timer or OS clock — never by a software counter
that only advances when spin() or poll() is called.
A poll-driven clock causes:
- Incorrect timeouts: zenoh-pico uses
z_clock_elapsed_ms()for session keep-alive, query timeout, and transport timeout. If the clock only advances during poll, idle periods (waiting for network I/O, sleeping between spins) are invisible to the timeout logic. - Timeout storms: After a long idle period, the next poll advances the clock by a large jump, causing multiple timeouts to fire simultaneously.
- Broken RTIC integration: RTIC tasks yield with
Mono::delay()between spins. A poll-driven clock doesn’t advance during the delay, making all timeouts effectively infinite.
Platform clock status
| Platform | Clock source | Notes |
|---|---|---|
| MPS2-AN385 (zpico) | CMSDK APB Timer0 | Hardware 25 MHz free-running counter. Wraps at ~171s, extended via WRAP_COUNT. |
| ESP32-C3 | esp_hal::time::Instant | Hardware timer, no manual updates needed. |
| ESP32-C3 (QEMU) | esp_hal::time::Instant | Works with QEMU -icount 3. |
| STM32F4 | ARM DWT cycle counter | Hardware-backed. update_from_dwt() reads elapsed DWT cycles and advances the software counter. Must be called at least once per ~25.6s (at 168 MHz) to avoid missing DWT wraps. |
| FreeRTOS | FreeRTOS kernel tick | OS-managed via xTaskGetTickCount(). |
| NuttX | POSIX clock_gettime() | OS-managed. |
| Zephyr | Zephyr kernel tick | OS-managed via k_uptime_get(). |
| POSIX (native) | std::time::Instant | OS-managed. |
| XRCE MPS2-AN385 | CMSDK APB Timer0 | Same hardware timer as zpico MPS2-AN385. Call init_hardware_timer() before use. |
Porting checklist for new platforms
- Identify a hardware timer — SysTick, a general-purpose timer, or DWT cycle counter. On RTOS platforms, use the OS tick API.
- Export
smoltcp_clock_now_ms() -> u64— called byzpico-smoltcpfor TCP/IP timestamping. - Export zenoh-pico clock symbols —
z_clock_now(),z_clock_elapsed_us/ms/s(),z_clock_advance_us/ms/s(). These read from the same underlying counter. - Never advance the clock inside
smoltcp_network_poll()— the poll callback runs during network I/O and is not a reliable time source. Read the hardware timer directly. - Handle timer wraps — 32-bit timers wrap. Track wrap count in an atomic or use a 64-bit hardware counter if available.
QEMU clock synchronization
QEMU’s virtual clock races ahead of wall-clock time during WFI, causing hardware
timer-backed timeouts to fire before TAP network I/O completes. This is solved
with -icount shift=auto, which makes virtual time advance at wall-clock speed
during CPU sleep states.
See QEMU icount reference for the full explanation, parameter reference, and tradeoffs.
Networking
Ephemeral port conflicts
smoltcp’s ephemeral port allocator starts at port 49152 with a static counter that resets to zero on each boot. When QEMU instances are killed and restarted, the new instance picks the same source port, creating a 4-tuple collision with stale host-side TCP sockets.
Symptom: ConnectionFailed or TransportOpenFailed on the second test run.
ss -tnap shows FIN-WAIT-1 sockets from the previous QEMU instance.
Fix (test infrastructure): Kill stale host TCP sockets between test runs:
#![allow(unused)]
fn main() {
// Clean stale TCP connections to QEMU IPs
for ip in &["192.0.3.10", "192.0.3.11"] {
let _ = Command::new("ss")
.args(["-K", "dst", ip])
.status();
}
}
Fix (Zephyr native_sim): Pass different --seed values to each instance so
the entropy source produces different ephemeral ports:
./build-listener/zephyr/zephyr.exe --seed=12345
./build-talker/zephyr/zephyr.exe --seed=67890
QEMU single-threaded I/O starvation
QEMU emulates the CPU and processes TAP network I/O in one thread. If the guest never executes WFI (Wait For Interrupt), QEMU never yields to its I/O event loop, and the TAP file descriptor is never serviced.
Symptom: First few packets work (buffered), then all networking stops.
Services and actions time out. Pub/sub may appear to work because subscriber
callbacks fire inline during zp_read().
Fix: In RTIC, yield between spin_once() calls:
#![allow(unused)]
fn main() {
// CORRECT — yields to idle task → WFI → QEMU processes TAP
cx.local.executor.spin_once(0);
Mono::delay(10.millis()).await;
// WRONG — busy-loops, starving QEMU I/O
loop {
cx.local.executor.spin_once(0);
}
}
In bare-metal (no RTIC), call cortex_m::asm::wfi() in your idle loop.
TAP device assignment
Each QEMU peer must use a different TAP device. Two QEMU instances on the same TAP cause packet collision and loss.
QEMU talker → tap-qemu0 ─┐
├── qemu-br (192.0.3.1/24)
QEMU listener → tap-qemu1 ─┘
Subscriber startup ordering
Zenoh does not buffer messages for unknown subscribers. If the publisher starts before the subscriber’s declaration propagates through the router, early messages are lost.
Rule: Start subscriber first, wait 5 seconds for stabilization, then start publisher.
TAP queue discipline
The default fq_codel qdisc drops packets when QEMU emulation is slow (CoDel’s
default 5ms target interprets emulation pauses as congestion, and per-flow
scheduling disrupts zenoh-pico’s service reply timing). This causes TCP data
segments to be dropped, breaking zenoh session establishment.
Fix: Use pfifo instead:
sudo tc qdisc replace dev tap-qemu0 root pfifo limit 1000
sudo tc qdisc replace dev tap-qemu1 root pfifo limit 1000
pfifo queues packets without dropping, which QEMU needs to absorb bursts
during emulation pauses. Stale packets from killed QEMU processes accumulate
in the queue but are harmless — the firmware seeds smoltcp’s ephemeral port
from the host’s wall clock via ARM semihosting SYS_TIME, so each QEMU run
uses a different source port and stale packets are silently ignored.
See TAP qdisc analysis for why
fq_codel and noqueue don’t work for QEMU TAP devices.
zenoh-pico
Z_FEATURE_INTEREST causes multi-client failures
When multiple zenoh-pico clients connect to the same router, the interest
protocol’s write filter creation can fail with -78
(_Z_ERR_SYSTEM_OUT_OF_MEMORY) even when memory is available.
Fix: Disable in all builds:
#![allow(unused)]
fn main() {
// build.rs
cmake::Config::new(&zenoh_pico_path)
.define("Z_FEATURE_INTEREST", "0")
.define("Z_FEATURE_MATCHING", "0")
.build();
}
Z_FEATURE_MATCHING depends on Z_FEATURE_INTEREST — both must be disabled.
See Troubleshooting: zenoh-pico Multiple Client Issues.
Blocking TCP reads in cooperative schedulers
_z_read_tcp() can block for up to SOCKET_TIMEOUT_MS (default 10 seconds).
In cooperative schedulers (RTIC, bare-metal main loop), this blocks the entire
system since C FFI cannot yield.
Impact: Pub/sub works (callbacks fire inline during zp_read()), but
services and actions fail because z_get() query replies need multiple
zp_read() cycles, and the 5-second query timeout expires while _z_read_tcp
blocks.
Current status: zpico_spin_once(0) is non-blocking (single_read=true),
but the underlying z_get() timeout is still 5 seconds. RTIC service/action E2E
tests are #[ignore]d pending a fully non-blocking TCP read path.
Version pinning
The zenoh router is ROS 2’s own (ros2 run rmw_zenoh_cpp rmw_zenohd,
RFC-0075) — nano-ros no longer vendors one, so the router version always
matches the rmw_zenoh_cpp your ROS nodes use. zenoh-pico is pinned at
zpico-sys/zenoh-pico/ (1.7.2); its wire protocol is stable across
zenoh 1.x, so it interoperates with a newer ROS-shipped router
(issue 0291). A transport-level failure (-100,
_Z_ERR_TRANSPORT_TX_FAILED) that looks like a network issue can still
be a protocol-major mismatch — check both versions before debugging
the network.
ESP32-C3 (RISC-V)
picolibc errno TLS crash
picolibc’s errno.h uses __thread (thread-local storage), which crashes on
bare-metal RISC-V because TLS is not initialized. The crash is a hard fault
with no useful backtrace.
Fix: Shadow errno.h with a non-TLS version:
// errno_override.h — no __thread, just a global
extern int errno;
#define EPERM 1
#define ENOENT 2
// ... subset of errno codes
Include this with -include errno_override.h in CFLAGS before picolibc headers.
Flash image format
ESP32-C3 QEMU requires a merged flash image, not a raw binary:
espflash save-image --merge --chip esp32c3 target/riscv32imc-unknown-none-elf/release/app app.bin
The --merge flag creates a 4 MB image with bootloader, partition table, and
application combined.
FreeRTOS
Task priority inversion
The LAN9118 RX FIFO poll task and zenoh-pico read task must run at the same priority (typically priority 4). If the read task has higher priority, it monopolizes the CPU, preventing RX FIFO drain. TCP keep-alives are missed, and zenoh sessions expire.
Similarly, lwIP’s tcpip_thread must run at the same priority. Lower priority
stalls packet processing.
Recursive mutexes required
zenoh-pico’s FFI layer requires recursive mutexes. Enable in FreeRTOS config:
#define configUSE_RECURSIVE_MUTEXES 1
Interrupt priority constraints
On Cortex-M3 (3-bit priority, 8 levels): ISRs at priority >= 5
(configLIBRARY_MAX_SYSCALL_INTERRUPT_PRIORITY) cannot call FreeRTOS APIs.
Ethernet IRQ handlers that signal tasks must use a lower (numerically higher)
priority.
Build System
Parallel test build races
When nextest runs test files in parallel and multiple tests build the same example with different features, cargo fingerprinting creates race conditions — one test overwrites the binary another test is about to execute.
Fix: Use --target-dir for each feature variant:
#![allow(unused)]
fn main() {
Command::new("cargo")
.args(["build", "--release", "--features", "safety-e2e"])
.arg("--target-dir").arg("target-safety")
.status()?;
}
Add each target dir to the example’s .gitignore:
/target/
/target-safety/
/target-zero-copy/
CMake cache invalidation
Changes to CMake defines (like Z_FEATURE_INTEREST=0) are cached. Changing the
value without cleaning the build has no effect.
Fix:
# Cargo (native)
cargo clean -p zpico-sys && cargo build
# Zephyr (west)
west build --pristine
Feature flag mutual exclusivity
The three platform axes — RMW backend, platform, ROS edition — are mutually
exclusive within each axis. Enabling two platforms (e.g.,
platform-posix,platform-zephyr) causes compile errors with confusing messages
about duplicate symbol definitions, not a clear “pick one” error.
Test Infrastructure
Orphan process prevention
When nextest is killed (Ctrl-C, OOM, timeout), child processes (zenohd, QEMU) can become orphans that hold ports and TAP devices.
Fix: Use PR_SET_PDEATHSIG(SIGKILL) in the pre-exec hook:
#![allow(unused)]
fn main() {
use std::os::unix::process::CommandExt;
let mut cmd = Command::new("zenohd");
unsafe {
cmd.pre_exec(|| {
// Kill this child when parent dies
libc::prctl(libc::PR_SET_PDEATHSIG, libc::SIGKILL);
// New process group for tree cleanup
libc::setpgid(0, 0);
Ok(())
});
}
}
Important: Do not use pkill to clean up zenohd — other agents or users
may have their own zenohd instances. Use process groups and death signals
instead.
Sequential test execution for shared resources
Tests that use QEMU networking (TAP devices, zenohd on fixed ports) must run sequentially. Use nextest test groups:
# .config/nextest.toml
[test-groups.qemu-network]
max-threads = 1
Stale TCP connection cleanup
Between sequential QEMU tests, host-side TCP sockets may linger in FIN-WAIT-1
(the QEMU peer was killed before completing the TCP close handshake). These
cause 4-tuple collisions when the next QEMU instance reuses the same source port.
Fix: Call ss -K dst <ip> between tests to force-destroy stale sockets:
#![allow(unused)]
fn main() {
pub fn cleanup_tap_network() {
for ip in &["192.0.3.10", "192.0.3.11"] {
let _ = Command::new("ss")
.args(["-K", "dst", ip])
.stdout(Stdio::null())
.stderr(Stdio::null())
.status();
}
std::thread::sleep(Duration::from_secs(1));
}
}
Design Overview
This section explains why nano-ros has the shape it does. It is not a tutorial (see Getting Started) and not an API reference (see the Reference chapters). It is the rationale a contributor needs to evaluate proposed changes, and the context a porter needs to make judgment calls when the Porting guides don’t cover their case.
Three design choices shape almost everything else:
1. The RMW layer was rewritten, not adopted
ROS 2’s rmw.h assumes a libc heap, an OS scheduler with preemptable threads, dynamic loaders, and middleware-owned background dispatch. None of those hold on a Cortex-M3 with 64 KB of RAM. nano-ros defines its own RMW abstraction (nros-rmw) that pushes I/O buffers to the caller, replaces wait sets with explicit drive_io(), and selects the backend at LINK time — reached through a registered C ABI vtable (RFC-0054), so a backend can be written in C or C++ as well as Rust. Graph discovery was dropped originally and is being added back deliberately, as visitor callbacks rather than caller-owned tables (phase-376 W4).
2. Future/Promise unifies blocking and async, with no internal spin
rclcpp owns the spin loop (rclcpp::spin(node) blocks the thread it was called on); rclc has no future/promise type at all. Neither model works on a single-threaded MCU. nano-ros uses a single Promise<T> value that supports three patterns – callback-driven polling, blocking-with-timeout, and .await – and never owns the spin loop. Every blocking helper takes the executor as an argument and drives it internally.
3. Platform traits are independent and contract-first
zenoh-pico needs a heap, threading, networking, and a clock. XRCE-DDS only needs a clock and four C function pointers. Bare-metal has no scheduler. Each capability is a separate trait the platform may stub if unsupported. Every method has an explicit behavior contract: blocking? may-fail? unsupported-fallback?
How to read this section
If you are evaluating a new feature: start with the design page closest to the change (RMW for transport-level changes, Client Library for executor or callback-shape changes, Platform API for OS-primitive changes). Each page lists the constraints the current shape satisfies; a new design must satisfy them or argue for relaxing one.
If you are porting nano-ros to new hardware or a new transport: the Porting chapters tell you what to implement. Read the matching design page to understand why each trait exists, which is what you need when the porting guide leaves a judgment call to you.
Client Library Model: nros vs rclc / rclcpp / rclrs
The RMW layer decouples the transport from the client library. The client library is the user-facing API on top of RMW: nodes, executors, publishers, subscribers, services, clients, callbacks, futures. ROS 2 ships three: rclc (C, MCU-focused), rclcpp (C++, desktop), and rclrs (Rust, experimental). nano-ros adds a fourth – the nros-node crate plus its nros-c / nros-cpp wrappers.
This page explains the shape of nros-node and how it differs from the three official client libraries. For trait signatures see Rust API / C API / C++ API.
Side-by-side
| Concern | rclc | rclcpp | rclrs | nros |
|---|---|---|---|---|
| Target | MCU (Micro-XRCE-DDS) | Desktop / robot | Desktop (alpha) | MCU + RTOS + desktop |
| Language | C99 | C++17 | Rust (std) | Rust (no_std) + C + C++14 |
| Node ownership | App owns node + executor | shared_ptr<Node> + Executor | Arc<Node> + executor | Executor owns the session; node borrows from it |
| Executor model | Single-threaded, polled | Single-/multi-threaded, owned thread | Single-threaded, polled | Single-threaded, polled |
| Spin entry point | rclc_executor_spin_some(&exec, timeout) | rclcpp::spin(node) (blocks, owns thread) | executor.spin() (blocks current thread) | executor.spin_once(timeout_ms) |
| Blocking primitive | None (callback-only) | std::shared_future::wait_for(timeout) | Promise::wait(timeout) (Tokio-style) | Promise::wait(&mut executor, timeout_ms) |
| Async primitive | None | std::shared_future + spin_until_future_complete | impl Future | Promise<T> (impls Future + manual poll + executor-driven blocking wait) |
| Heap requirement | Optional (static allocator) | Required | Required | Optional (caller buffers; only zenoh-pico transport heap) |
| Threading requirement | None | Required (std::thread) | Required (std::thread) | None (single-threaded valid) |
Future / Promise as the unifying primitive
The same Promise<T> value serves three usage patterns. None of the other client libraries achieves this with a single type.
Pattern 1 – callback-driven (no blocking call):
let promise = client.call(&request)?;
// ... continue running other work ...
executor.spin_once(10);
if let Some(reply) = promise.take()? {
handle(reply);
}
Pattern 2 – blocking with timeout:
let mut promise = client.call(&request)?;
let reply = promise.wait(&mut executor, 5000)?; // blocks up to 5s, drives I/O
Pattern 3 – async runtime (Tokio, Embassy, smol):
let reply = client.call(&request)?.await; // Promise: Future<Output = T>
Contrast this with the alternatives:
- rclcpp uses
std::shared_future<Response>, which is bound tostd::threadandstd::condition_variable. There is no way to drive it on a cooperative single-threaded MCU. The blocking helperrclcpp::spin_until_future_complete()requires an owned thread. - rclc has no future or promise concept. The only way to receive a service reply is to register a callback that fires from
rclc_executor_spin_some(). Building request/response chains becomes an explicit state machine. - rclrs has
impl Futurebut no callback-poll path; you must.awaitfrom a Tokio task. No blocking-with-timeout helper that drives I/O internally.
In nano-ros, the same Promise carries a borrow of the standalone service client and a sequence number; take() polls it, wait() polls it in a loop while spinning the executor, and the Future impl integrates with any executor that calls register_waker(&Waker) underneath.
No internal spin
rclcpp::spin(node) blocks the calling thread inside the library and owns the dispatch loop. This works on Linux but cannot work on bare-metal: there is no thread to give up, and the application loop must remain in user code (for power management, interrupt servicing, smoltcp polling, RTIC integration).
nano-ros never owns the spin loop. The user always writes the loop:
loop {
executor.spin_once(10);
// user can also: poll smoltcp, service interrupts, run other tasks, sleep
}
Convenience wrappers (spin(count), spin_blocking(opts), spin_period(duration)) exist for desktop-style use cases, but each is implemented as a spin_once() loop that the user could write by hand. On no_std targets they aren’t available – the user writes the loop.
This is the reason every blocking API in nros takes &mut Executor. Promise::wait, Stream::wait_next, Client::call_blocking, the C++ Future::wait(executor.handle(), ...), and the C nros_client_call(..., timeout_ms) all internally call spin_once() to keep I/O moving while waiting. They cannot rely on a background thread doing it for them, because there is none.
Executor is the session owner, not a singleton
In rclcpp, the Executor is a separate object that you attach nodes to (exec.add_node(node)). The node owns its own RMW context. Multiple executor classes (SingleThreadedExecutor, MultiThreadedExecutor, StaticSingleThreadedExecutor) exist as parallel implementations.
In nano-ros, the Executor is the RMW session owner. Calling Executor::open(&config) opens the transport; nodes are derived from the executor:
let mut executor = Executor::open(&config)?;
let mut node = executor.create_node("my_node")?;
let pub_ = node.create_publisher::<Int32>("/topic")?;
There is exactly one executor type. There is no separate “context” object, no add/remove-node lifecycle, no executor selection at runtime. The single-threaded model is baked in – adding a multi-threaded variant would require redesigning the callback dispatch, which we deliberately avoid to keep the bare-metal path viable.
The trade-off: you cannot share one transport session between two executors. In practice no embedded application wants this, and on desktop you can run multiple processes if you need it.
Explicit spin in blocking ops – why pass the executor?
Every blocking operation takes &mut Executor (Rust) or an executor handle (C/C++). This looks redundant when the executor is also the entity that created the operation. The reason is borrow-checker hygiene plus single-threaded I/O.
The standalone communication handles (StandaloneClient, StandaloneSubscription, the Promise returned from call) borrow from the session. They cannot also borrow the executor at the same time, because the executor owns the session. Passing &mut executor at the wait call – after the promise has been created and the borrow released – is the only way to get a mutable executor reference while a promise is in flight.
The deeper reason: there is no other thread that can drive I/O. If Promise::wait() did not have the executor, it could not call spin_once() – the network would freeze and the wait would always time out. By forcing the executor parameter, the API makes the I/O dependency explicit and impossible to forget.
Language parity
The same Future/Promise + explicit-executor model is preserved in C and C++ wrappers.
Rust uses Promise<'_, T> directly. It implements core::future::Future and exposes take() + wait(&mut executor, ms).
C++ wraps the promise as nros::Future<T>:
auto fut = client.send_request(req);
ResponseType resp;
NROS_TRY(fut.wait(executor.handle(), 5000, resp));
Future::wait() takes void* executor_handle (from executor.handle() or nros::global_handle()) for the same reason the Rust API takes &mut executor. There is no global executor singleton; the handle is explicit.
C uses paired _async and blocking entry points:
// Blocking: drives the executor internally, returns reply or timeout.
nros_call_service(client, &req, sizeof(req), &reply, sizeof(reply), 5000);
// Async: returns immediately, result via callback registered ahead of time.
nros_action_send_goal_async(client, &goal, sizeof(goal));
nros_action_client_set_result_callback(client, on_result);
// User keeps calling nros_spin_once() to drive callbacks.
The C API has no Future/Promise type because C lacks generics, but the pattern is the same: an _async send-without-wait paired with a callback (or polled status), and a blocking call that drives the executor for you.
Summary
The four design choices that shape nano-ros’s client library:
- Single executor type that owns the session – no separate context object, no executor variants.
- No internal spin – the user always owns the spin loop; blocking helpers exist but are wrappers.
- Explicit
&mut Executoron every blocking op – the API enforces the I/O dependency at the call site. - Future/Promise as the unifying primitive – one type for callback-drive, blocking-with-timeout, and
.await.
These choices are what make the same client library viable on a Cortex-M3 and a Linux workstation without a separate “MCU client library” like rclc.
RMW API Design: nros-rmw vs ros2/rmw
nano-ros defines its own RMW (ROS Middleware) abstraction in the nros-rmw crate. While it serves the same purpose as the official ros2/rmw interface – decoupling the client library from the transport backend – it is designed for no_std embedded systems and uses a fundamentally different approach.
This page documents the architectural differences and trade-offs. For trait signatures and the available backends, see RMW API Reference. For implementing a new backend, see Custom RMW Backend.
Why We Revised rmw.h
rmw.h was designed for ROS 2 on Linux: a process with a libc heap, an OS scheduler, dynamic loaders, and middleware-owned background threads. None of those assumptions hold on a Cortex-M3 with 64 KB of RAM. Each constraint below drove a specific change.
Heap availability
rmw.h heap-allocates everywhere – handles, serialized message buffers, wait sets, type support tables. Bare-metal targets often have no allocator; RTOS targets have allocators with hard total budgets (~16-256 KB) that must cover the application as well.
nros-rmw moves all I/O buffers to the caller. publish_raw(&[u8]) and take_serialized(&mut [u8]) operate on slices that the caller stack- or statically-allocates. Type metadata is a string-only TopicInfo struct, not a pointer-laden rosidl_message_type_support_t table. What the abstraction itself allocates is nothing. What the BACKENDS allocate is
enumerated by scripts/rmw-alloc-sites.py, which reports each site with its
file and line and — the distinction that decides whether you care — whether it
sits on the STEADY-STATE path (per message, so it is latency and a heap that
must sustain traffic) or in CREATE/INIT (bounded, so it lands in startup):
| backend | steady-state | create / init |
|---|---|---|
| Cyclone DDS | 6 | 6 |
| XRCE-DDS | 0 | 9 |
| uORB | 0 | 3 |
Run the script rather than trusting this table; it is the re-runnable source and
this is a snapshot of it. Two caveats it states and this page inherits: it
counts nano-ros’s OWN sites, so allocations inside Cyclone below dds_write and
inside zenoh-pico’s z_malloc are real but not listed (they are not ours to
remove), and zenoh-pico’s internal transport buffers (~64 KB) reach the image
through PlatformAlloc, where a bump allocator suffices on bare-metal.
Threading model
rmw.h assumes the middleware owns threads. rmw_wait() blocks the calling thread on a wait set; some implementations also spawn internal dispatch threads that fire callbacks asynchronously. Bare-metal has no scheduler; cooperative RTOS configurations can’t tolerate hidden threads.
nros-rmw replaces rmw_wait with Session::drive_io(timeout_ms) – a single call the executor invokes from its own (and only) thread. There is no wait set object, and no entity is implicitly polled by the middleware. The application drives all I/O explicitly. For async runtimes, subscribers and service clients expose register_waker(&Waker) so the transport’s C receive callback can wake a Rust future without a wait set abstraction.
Single-threaded callback dispatch
rmw.h permits multi-threaded executors and reentrant callbacks. Cooperative single-threaded targets cannot guarantee atomicity around RMW state without locks they don’t have.
nros-rmw assumes a single-threaded executor that owns the session for its lifetime. Callbacks run sequentially on the executor thread; no callback can preempt another. This eliminates the need for internal locking around publisher state, subscriber buffers, or service queues – a measurable code-size and runtime win on MCUs.
No dynamic discovery tables
rmw.h provides rmw_get_topic_names_and_types(), rmw_count_publishers(), rmw_get_node_names(), and similar graph-introspection APIs. These require maintaining a dynamic discovery cache, which costs heap and CPU continuously even when nothing reads it.
nros-rmw carried none of these for most of its life: discovery happened at the transport layer (zenoh liveliness, XRCE-DDS session establishment) and was never surfaced as queryable graph state.
That is changing. Phase-376 W4 added the graph-enumeration slots to the ABI —
get_node_names, get_topic_names_and_types, count_publishers and the
by-node variants — as visitor callbacks rather than allocated
names-and-types arrays, so a backend walks its own discovery data and the
caller never owns a table. The slots are declared; backend wiring is the
in-flight part. The generated
Per-RMW Feature Matrix is the current
per-backend truth, and it is derived from the vtables rather than from this
page.
Compile-time backend selection
rmw.h selects backends at runtime via dlopen() of librmw_*.so. This requires a dynamic loader (no embedded MCU has one) and forces every call through a vtable.
nros-rmw selects the backend at link time via Cargo features, and reaches
it through a C ABI vtable (nros_rmw_vtable_t, RFC-0054) that the backend
hands the runtime once via nros_rmw_cffi_register() before any session is
created. There is no loader, no .so, and no path search: the only backends
reachable are the ones linked into the image.
This page used to claim the opposite — “no vtables, no dynamic dispatch”,
monomorphized through Rust generics. That was true before RFC-0054 made the C
headers the SSoT so a backend could be written in C or C++ (Cyclone DDS is);
nros-rmw’s Rust traits are still the surface a Rust backend implements, and
rust_adapter.rs in nros-rmw-cffi is what turns such an impl into the
vtable the runtime calls. The cost is one indirect call per operation; what it
buys is a backend boundary that is not Rust-only.
Architectural Pattern
| Aspect | ROS 2 rmw | nros-rmw |
|---|---|---|
| Language | C API (rmw/rmw.h) | Rust traits |
| Dispatch | Runtime plugin loading (shared library via rmw_implementation) | Link-time selection; calls cross a C ABI vtable registered at startup (RFC-0054) |
no_std | No (requires libc, heap, POSIX) | Yes — but “no heap” is a property of the BACKEND, not of the abstraction (see below) |
| Error model | rmw_ret_t integer codes | nros_rmw_ret_t at the ABI, using upstream rmw’s VALUES (phase-376 W3.d); TransportError on the Rust side |
ROS 2 selects the RMW backend at runtime by loading a shared library (e.g.,
rmw_fastrtps_cpp.so). This enables switching backends without recompilation
but requires a dynamic loader — which no MCU has.
nros-rmw links exactly the backends the image was built with and dispatches through a registered vtable. What that removes is the loader and the relocation work at startup, not the indirect call.
On heap: the abstraction adds none — I/O buffers are caller-owned, and
entity handles are inline. The BACKENDS are another matter, and this page
overstated it for years (“no heap”, flatly). Issue 0777 established that;
scripts/rmw-alloc-sites.py now answers it precisely and repeatably. Only
Cyclone DDS allocates on the steady-state path in nano-ros’s own code (6 sites);
XRCE and uORB allocate at entity/transport setup only. Plan the heap budget from
your backend’s row in that report — and remember it excludes what the middleware
libraries do underneath, which for Cyclone and zenoh-pico is a general allocator
call per message regardless.
Object Model
ROS 2
ROS 2 rmw has a deep initialization hierarchy:
rmw_init() → rmw_context_t
→ rmw_create_node() → rmw_node_t
→ rmw_create_publisher() → rmw_publisher_t*
→ rmw_create_subscription() → rmw_subscription_t*
→ rmw_create_service() → rmw_service_t*
→ rmw_create_client() → rmw_client_t*
Nodes are first-class RMW objects. Each rmw_node_t carries its own context, name, namespace, and security credentials. The RMW layer is responsible for node lifecycle and graph participation.
nros-rmw
nros-rmw is flatter – there is no node at the RMW level:
Rmw::open(&RmwConfig) → Session
→ session.create_publisher(&TopicInfo, QoSProfile) → Self::PublisherHandle
→ session.create_subscription(&TopicInfo, QoSProfile) → Self::SubscriptionHandle
→ session.create_service(&ServiceInfo, QoSProfile) → Self::ServiceHandle
→ session.create_client(&ServiceInfo, QoSProfile) → Self::ClientHandle
Node lives one layer up in nros-node. It is purely a namespace and liveliness concern – it borrows the session from the executor and creates typed communication handles. The RMW layer only knows about sessions and communication endpoints.
Serialization Boundary
This is the most significant design difference.
ROS 2: The rmw layer operates on pre-serialized data. rcl and rosidl handle CDR serialization before calling rmw_publish() with an rmw_serialized_message_t. The rmw layer never sees typed messages – it only moves byte buffers. Type metadata is passed separately via rosidl_message_type_support_t structs.
nros-rmw: The traits include both raw and typed methods:
pub trait Publisher {
// Raw: caller handles serialization
fn publish_raw(&self, data: &[u8]) -> Result<(), Self::Error>;
// Typed: serialize + publish in one call
fn publish<M: RosMessage>(&self, msg: &M, buf: &mut [u8]) -> Result<(), Self::Error>;
}
pub trait Subscription {
fn take_serialized(&mut self, buf: &mut [u8]) -> Result<Option<usize>, Self::Error>;
fn take<M: RosMessage>(&mut self, buf: &mut [u8]) -> Result<Option<M>, Self::Error>;
}
The typed methods have default implementations that call the raw methods with CDR serialization/deserialization from nros-serdes. This keeps the RMW layer self-contained – no separate serialization layer is needed.
Type metadata uses simple structs (TopicInfo { name, type_name, type_hash }) instead of C type support function tables.
I/O and Readiness Model
ROS 2: Uses rmw_wait() with a wait set (rmw_wait_set_t) containing subscriptions, services, clients, guard conditions, and events. The caller constructs a wait set, adds handles, and blocks until any handle is ready. This is similar to select()/epoll().
nros-rmw: Uses a single drive_io(timeout_ms) method on the Session trait:
pub trait Session {
fn drive_io(&mut self, timeout_ms: i32) -> Result<(), Self::Error> {
let _ = timeout_ms;
Ok(())
}
}
This is a pull-based model: the executor calls drive_io() to poll the network and dispatch incoming data to internal subscriber buffers, then checks each entity with has_data(). There is no wait set – the executor iterates its dispatch table directly.
For async integration, subscribers and service clients expose register_waker(&Waker) instead of guard conditions. The transport backend calls waker.wake() from its C receive callback, bridging to Rust Future waking without the wait set abstraction.
Memory Model
ROS 2: Heap-allocates handles, messages, and serialization buffers. rmw_serialized_message_t wraps a dynamically-sized rcutils_uint8_array_t. Loaned message APIs (rmw_borrow_loaned_message, rmw_take_loaned_message) provide optional zero-copy for transports that support shared memory.
nros-rmw: Uses caller-provided &mut [u8] buffers everywhere. All receive and serialize operations write into stack-allocated or statically-allocated buffers:
// Caller provides the buffer
let mut buf = [0u8; 512];
let msg: Option<MyMsg> = subscriber.take(&mut buf)?;
Zero-copy receive is supported via process_raw_in_place(), which invokes a closure with a reference to the subscriber’s internal receive buffer, avoiding the copy into a caller-provided buffer. This is gated behind the unstable-zenoh-api feature.
QoS Settings
ROS 2 rmw_qos_profile_t includes:
| Field | ROS 2 | nros-rmw |
|---|---|---|
| History (keep last/all) | Yes | Yes |
| Depth | Yes | Yes |
| Reliability (reliable/best-effort) | Yes | Yes |
| Durability (volatile/transient local) | Yes | Yes |
| Deadline | Yes | No |
| Lifespan | Yes | No |
| Liveliness (automatic/manual) | Yes | No |
avoid_ros_namespace_conventions | Yes | No |
nros-rmw provides the four QoS policies that zenoh-pico and XRCE-DDS can actually enforce. The time-based policies (deadline, lifespan, liveliness) are omitted because the supported transports do not implement them.
Standard QoS profiles (QOS_PROFILE_DEFAULT, QOS_PROFILE_SENSOR_DATA, QOS_PROFILE_SERVICES_DEFAULT, etc.) match their ROS 2 equivalents for interoperability.
Service Client Model
ROS 2: Service clients are always asynchronous at the rmw level. rmw_send_request() sends a request and returns a sequence number. The reply is retrieved later via rmw_take_response(), typically driven by rmw_wait().
nros-rmw: The same async split (phase-301 deleted the deprecated blocking call_raw path — like upstream, there is no blocking call at the RMW level):
pub trait ClientTrait {
// Async: send request, poll for reply separately
fn send_request_raw(&mut self, request: &[u8]) -> Result<(), Self::Error>;
fn take_response_raw(&mut self, reply_buf: &mut [u8]) -> Result<Option<usize>, Self::Error>;
}
Blocking waits are composed above the RMW by the executor (which keeps driving drive_io between polls).
APIs Present in ROS 2 rmw but Absent in nros-rmw
Phase-376 is closing much of this table: several rows below have gained ABI slots, and the ones that are DECLINED now carry a written reason rather than silence. Treat the generated Per-RMW Feature Matrix as current and this table as the design rationale behind the original omissions.
| ROS 2 rmw API | Purpose | Why absent |
|---|---|---|
rmw_node_t / rmw_create_node() | Node lifecycle at RMW level | No longer absent (phase-376 W5/B1): entities are created ON a node, upstream’s shape — create_publisher and its siblings take a const rmw_node_t *. Node lifecycle still lives above the RMW in nros-node. |
rmw_wait_set_t / rmw_wait() | Multiplexed readiness waiting | Replaced by drive_io() + per-entity has_data() |
rmw_guard_condition_t | Wake wait set from application code | Replaced by register_waker(&Waker) |
rmw_event_t | QoS event callbacks (deadline missed, etc.) | Partly present: subscription_event_init / publisher_event_init are ABI slots and zenoh wires them; the other backends leave them NULL. |
rmw_get_topic_names_and_types() | Graph introspection | Slot declared (phase-376 W4) as a VISITOR callback — the backend walks its own discovery data, the caller owns no table. Backend wiring in flight. |
rmw_get_node_names() | Node discovery | Same — declared as a visitor. |
rmw_count_publishers() / rmw_count_subscribers() | Graph statistics | Same — declared, wiring in flight. |
rosidl_message_type_support_t | C type support tables for serialization | Replaced by TopicInfo string metadata |
rmw_serialize() / rmw_deserialize() | Standalone serialization | CDR handled by nros-serdes |
rmw_borrow_loaned_message() | Zero-copy shared memory publish | Not supported (smoltcp/zenoh-pico don’t use shared memory) |
| Content-filtered topics | Server-side topic filtering | Not supported |
APIs Present in nros-rmw but Absent in ROS 2 rmw
| nros-rmw API | Purpose |
|---|---|
Publisher::publish<M>(msg, buf) | Typed publish with built-in CDR serialization |
Subscription::take<M>(buf) | Typed receive with built-in CDR deserialization |
Subscription::process_raw_in_place(f) | Zero-copy in-place processing via closure |
Subscription::take_validated() | E2E safety validation (CRC-32 + sequence tracking) |
ServiceTrait::handle_request<S>() | Typed request handling with automatic CDR roundtrip |
Session::drive_io(timeout_ms) | Explicit network polling (ROS 2 rmw relies on middleware threads) |
Summary
The core difference is that ROS 2 rmw is a C plugin interface designed for desktop systems with dynamic linking, heap allocation, and OS threading. nros-rmw is a Rust trait hierarchy designed for MCUs with static dispatch, stack allocation, and cooperative scheduling. The trade-off is flexibility (ROS 2 can swap backends at runtime) vs efficiency (nros eliminates all abstraction overhead at compile time).
Despite these differences, the two are wire-compatible when using the same transport. An nros node using nros-rmw-zenoh communicates with a ROS 2 node using rmw_zenoh_cpp through the same router, with matching QoS profiles and CDR encoding.
The zenoh pairing (phase-362 / RFC-0075)
The router is the one ROS ships — ros2 run rmw_zenoh_cpp rmw_zenohd. It
links the same libzenohc.so that rmw_zenoh_cpp does, so it cannot drift from
the RMW you are actually talking to, and it is what a ROS 2 deployment runs.
nano-ros no longer ships a router of its own.
The table below is data to diff a future failure against, not a constraint we
enforce. rmw_zenoh_cpp lives on your machine, installed by your distro; we
cannot pin it, and the interesting number is not the ROS package version anyway.
| side | component | observed |
|---|---|---|
| host | rmw_zenoh_cpp router + RMW | zenoh-c 1.6.2 (ROS 2 Humble, measured 2026-08-16) |
| firmware | zenoh-pico | 1.7.2 |
Read the zenoh version from the header, never from the package manager:
/opt/ros/<distro>/opt/zenoh_cpp_vendor/include/zenoh_configure.h #define ZENOH_C "…"
packages/rmw/zenoh/zpico-sys/zenoh-pico/version.txt
The ROS package version (ros-humble-rmw-zenoh-cpp 0.1.9) is a wrapper
version and says nothing about the zenoh inside it — issue 0609 measured that
same package moving its vendored zenoh 1.2.0 → 1.8.0 in a patch-level bump, and
reading the package version instead of the header is what produced a wrong
version claim in that issue’s first filing.
The two sides need not match. Under zenoh’s 1.x wire guarantee a firmware pin should move for its own reasons — footprint, features, fixes — rather than to chase a host package.
RMW API: Differences from upstream rmw.h
This page is for developers who already know upstream ROS 2’s
rmw/rmw.h and want to write — or just
understand — a backend for nano-ros. Side-by-side: what rmw.h looks
like, what nros-rmw-cffi looks like, and why nano-ros diverges.
The C signatures shown for nano-ros come from
<nros/rmw_vtable.h>. The Rust trait
counterparts are an alternative entry point for porters who already
work in Rust; this page sticks to the C-vtable surface throughout.
Phase-301 rename. The vtable historically diverged from rmw’s vocabulary (
subscriber,service_server,service_client,open/close). Phase-301 (issues 0240/0241) aligned it: the entity types are nowrmw_subscription_t/rmw_service_t/rmw_client_t, the session slots arecreate_session/destroy_session, and the deprecated blockingcall_rawslot is deleted (send_request+take_responseis the one request/reply path). Transport hints (tx_express,rx_buffer_hint) moved out ofrmw_qos_profile_tinto per-create options structs. This page uses the new names throughout.
Looking for the exhaustive list? This page is the rationale — the divergences worth understanding before writing a backend. The complete table, every upstream symbol with both signatures side by side and the reason for each difference, is RMW API vs upstream — every symbol, and it is GENERATED, so it cannot drift from the headers the way prose can.
TL;DR
| Concern | upstream rmw.h | nros-rmw-cffi |
|---|---|---|
| Plugin loading | dlopen("librmw_*.so") at runtime | Single vtable registered at init |
| Init sequence | rmw_init_options_t → rmw_context_t → entities | One create_session() call, returns the session |
| Entity types | rmw_publisher_t / rmw_subscription_t / rmw_service_t / rmw_client_t | Typed-with-opaque-tail: rmw_publisher_t / _subscription_t / _service_t / _client_t (visible metadata + opaque backend_data) |
| Wait | rmw_wait(waitset, timeout) blocks the caller | drive_io(session, timeout_ms) drives I/O once |
| Serialization | typesupport-driven (rosidl) | Pre-serialized CDR bytes only |
| Graph queries | rmw_get_topic_names_and_types, … | None |
| QoS profiles | Full DDS profile match between endpoints | Same field set; per-backend support advertised; synchronous IncompatibleQos on create instead of runtime mismatch event |
| DDS events | rmw_event_t (rmw_take_event) | None |
| Loaned messages | Optional rmw_borrow_loaned_message | First-class loan_publish / loan_recv |
| Error returns | rmw_ret_t (RMW_RET_OK, …) | rmw_ret_t (NROS_RMW_RET_OK, …) — same named-constant style |
1. Plugin loading vs. compile-time backend
Upstream. rmw_implementation resolves the backend at process
start by opening a shared library:
RCUTILS_DLLIMPORT
const char * rmw_get_implementation_identifier(void);
The runtime calls dlopen against the path encoded by
RMW_IMPLEMENTATION and binds every rmw_* symbol through the loader.
nano-ros. The backend is linked into the binary. A C backend registers its vtable once before any nros call:
#include <nros/rmw_vtable.h>
static const nros_rmw_vtable_t MY_VTABLE = { ... };
int main(void) {
nros_rmw_cffi_register(&MY_VTABLE);
nros_init(...);
}
Why. Most embedded targets have no dynamic loader. Even where
dlopen exists, a 32 KB Flash budget can’t afford to link every
backend’s C client and pick at runtime. Compile-time selection cuts
the binary by 60–80 % and lets the linker drop unused entity paths.
2. Init sequence
Upstream. Three-step init:
rmw_init_options_t options = rmw_get_zero_initialized_init_options();
rmw_init_options_init(&options, allocator);
rmw_context_t context = rmw_get_zero_initialized_context();
rmw_init(&options, &context);
/* ... use context to create rmw_node_t, then rmw_publisher_t, … */
rmw_shutdown(&context);
nano-ros. One step — a session covers what upstream splits across options + context:
rmw_session_t session = {0};
rmw_ret_t ret = vtable->create_session(locator, mode, domain_id, node_name, &session);
/* ... use &session to create_publisher / create_subscription / … */
vtable->destroy_session(&session);
Why. Upstream’s split is useful when an application owns multiple RMW instances with different options. Nano-ros assumes one session per process — one transport, one wire-protocol, one set of QoS defaults — so the options/context separation buys nothing.
3. Entity handles
Upstream. Every entity is a typed C struct with backend-private
state hidden behind void * data:
typedef struct RMW_PUBLIC_TYPE rmw_publisher_t {
const char * implementation_identifier;
void * data;
const char * topic_name;
rmw_publisher_options_t options;
bool can_loan_messages;
} rmw_publisher_t;
nano-ros. Hybrid: typed-with-opaque-tail.
Each entity is a typed C struct exposing the metadata the runtime
actually reads (topic name, type name, QoS, lending capabilities)
inline; backend-private state stays behind an opaque backend_data
pointer.
typedef struct rmw_publisher_t {
const char * topic_name; /* borrowed; outlives the publisher */
const char * type_name; /* borrowed */
rmw_qos_profile_t qos;
bool can_loan_messages; /* matches upstream's field of the same name */
uint8_t _reserved[7]; /* forward-compat; must be zero */
void * backend_data; /* opaque */
} rmw_publisher_t;
rmw_ret_t (*create_publisher)(
rmw_session_t * session,
const char * topic_name, const char * type_name, const char * type_hash,
uint32_t domain_id, const rmw_qos_profile_t * qos,
const rmw_publisher_options_t * options, /* transport hints; NULL = defaults */
rmw_publisher_t * out); /* runtime-allocated; backend fills */
Same shape for rmw_subscription_t (whose
rmw_subscription_options_t carries rx_buffer_hint, as the
publisher’s carries tx_express). Service entities
(rmw_service_t, rmw_client_t) and
rmw_session_t have no qos and no can_loan_messages —
service-level QoS doesn’t generalise across non-DDS backends
(see QoS, Section 7)
and service request/reply uses the byte-buffer API rather than the
loan primitive.
Forward-compat reserved bytes. Each entity carries an explicit
_reserved[N] byte block (sized to fill the natural alignment slot
before backend_data). New fields up to N bytes can be added later
without changing struct size or the offset of any field after
backend_data. Backends and runtime keep these bytes zero.
Storage ownership. The runtime allocates the entity-struct shell;
the backend writes its backend_data into the runtime-supplied
out-parameter at create_* time (can_loan_messages is derived by the
runtime — see below). The backend never mallocs a
struct shell — embedded targets cannot afford a per-entity heap
allocation. destroy_* releases only the backend’s backend_data;
the shell stays valid until the runtime drops its owner.
Differences from upstream’s rmw_publisher_t.
- Borrowed strings, not backend-owned copies. Upstream’s
topic_namepoints to a backend-allocated string copied atcreate_publishertime. Ours points to caller (runtime) storage that outlives the publisher — no allocation per entity. - No
implementation_identifierfield. Backend selection is compile-time (see Section 1); there’s no plugin loader to dispatch through, so no need to identify which backend owns a struct. can_loan_messagesmatches upstream in shape, but the runtime DERIVES it (issue 0814). Same bool, same name, same meaning —trueif the backend exposes the loan primitive (the CDR-byte zero-copy loan path) — except that a backend does not write it. Its value is exactlyvtable->borrow_loaned_message != NULL, because that is the same fact stated twice, and the two spellings had drifted in both directions: nothing outside a test ever wrotetrue(so a zenoh build whose loans worked advertised that it could not loan), while a backend writingtruewith a NULL slot was believed. This paragraph used to claim the runtime “dispatches the publish path with no per-call branch” from the flag; it never did — dispatch reads the vtable slot on every call.depth: uint16_t. Upstream uses 32-bit; embedded queue depths are 1–100, the 16-bit width saves 2 bytes × N entities.- Explicit
_reserved[N]bytes. Upstream uses an embeddedrmw_publisher_options_tstruct as the extension point; we reserve raw bytes inline. Same forward-compat property — new fields up to N bytes don’t break ABI — without the indirection.
Why this shape. Fully-opaque void * (the previous nano-ros
design) forced every introspection through a vtable callback.
Upstream’s “expose every field, backend keeps them in sync” forces
duplicated state. The typed-with-opaque-tail middle ground exposes
exactly the fields the runtime reads — no callback indirection — and
keeps backend implementation state private. The struct layout is
ABI; adding or reordering fields is a major-version bump.
4. drive_io vs. rmw_wait
Upstream. Clients block on a waitset that aggregates entities:
rmw_ret_t rmw_wait(
rmw_subscriptions_t * subscriptions,
rmw_guard_conditions_t * guard_conditions,
rmw_services_t * services,
rmw_clients_t * clients,
rmw_events_t * events,
rmw_wait_set_t * wait_set,
const rmw_time_t * wait_timeout);
The middleware can also spawn its own background threads that fire callbacks asynchronously.
nano-ros. The executor calls a single drive-I/O entry point:
rmw_ret_t (*drive_io)(rmw_session_t * session, int32_t timeout_ms);
The backend dispatches whatever receive / send / wakeup work is
pending and returns within timeout_ms. There is no waitset, no
guard-condition aggregation, no implicit middleware thread.
How the two models differ in practice
The two designs distribute work across different layers:
| Phase | Upstream rmw_wait | nano-ros drive_io |
|---|---|---|
| Build the wait set | Executor rebuilds a waitset every spin_once, adds every entity | Executor registers entities once at construction; no per-spin rebuild |
| Block | rmw_wait blocks the thread on a kernel waitable (DDS WaitSet, condvar, kqueue) until any entity is ready or timeout | drive_io blocks (sleep-model backends) or polls (poll-model backends) for up to timeout_ms |
| Signal readiness | Wait primitive raises per-entity ready flags; backend writes flags into the waitset’s status arrays | Backend’s RX worker pulls bytes; the data is “ready” by virtue of being received |
| Dispatch user callback | Executor’s spin_once picks one ready entity, calls rmw_take, fires its user callback | Backend’s RX worker / drive_io loop fires user callbacks while it has work to do |
Per spin_once | Exactly one user callback runs (single-threaded executor) | All ready callbacks run before drive_io returns |
The upstream model separates wait (kernel-blockable) from dispatch (executor-controlled). nano-ros today fuses them — the backend handles both inside one call.
What this buys, what this costs
Fusing wins on:
- No per-spin waitset rebuild. Upstream’s
add_handles_to_wait_setallocates and walks every entity each iteration. On a 100-entity executor at 1 kHz that’s 100 000 per-second add/remove operations plus heap churn. nano-ros registers entities once; the backend tracks them statically. - No kernel-waitable per entity. Upstream’s per-entity ready flags need a per-entity wakeable resource (DDS Condition, eventfd, pipe). nano-ros’s backend uses one wait primitive per session; per-entity tracking is in user-space backend state.
- Backend RX worker fires callbacks directly. zenoh-pico’s
_z_session_read_taskinvokes user callbacks during its read. nano-ros’sdrive_iodrains it; upstream’srmw_zenohwould still have to round-trip throughrmw_take.
Fusing costs on scheduling control. Upstream’s “one callback per
spin_once” rule gives the executor an opportunity between every
two callbacks to:
- Re-check timer expirations
- Re-check guard conditions
- Yield to higher-priority work (multi-threaded executor)
- Apply per-callback priority ordering
nano-ros’s drive_io runs all ready callbacks back-to-back, then
the spin loop processes timers + GCs between drive_io calls.
For a 100 ms drive_io call that fires 10 sub callbacks in 80 ms,
a timer that should have fired 5 ms in is delayed 75 ms.
Where this fits each RTOS execution model
| Execution model | Fits drive_io today? |
|---|---|
| Cooperative single-task (one task does ROS, no priority competition) | Yes — no other task to preempt; entity scheduling fairness is moot |
| Async / tokio / Embassy (futures, wakers) | Yes — spin_async drives futures; drive_io not used in the hot path |
| Preemptive priority RTOS, ROS at one priority (FreeRTOS / ThreadX / Zephyr typical) | Partial — kernel preemption from higher-priority tasks works; ROS-internal entity scheduling is batch-FIFO, timer expiries can be delayed by long sub callbacks at the same priority |
| WCET-bounded real-time (RTIC, DO-178C) | No — drive_io has unbounded execution time; callers needing per-callback WCET use the async path with explicit Waker integration instead |
| Time-triggered cyclic | No — no way to bound drive_io to a fixed wall-clock budget |
The “Yes” rows are where nano-ros ships today. The “Partial” / “No” rows are addressed by the work below.
How RTOS cooperation will improve
Three forward-looking knobs land incrementally as the
drive_io interface is extended. None breaks the default behaviour;
each is opt-in for apps that need it.
-
Backend-internal-deadline visibility.
Session::next_deadline_ms()tells the executor when the backend’s next internal event (lease keepalive, heartbeat, ACK retransmit) is due. The executor capsdrive_io’s timeout against it so the call doesn’t return sooner than expected on otherwise-quiet links. Saves one round-trip per quiet period. -
Per-call user-callback cap.
drive_ioacceptsmax_callbacks: an upper bound on user callbacks fired per call. Setting it to1reproduces upstream’s “one callback perspin_once” pattern. The runtime spin loop callsdrive_ioagain to drain pending work, with timer / GC checks between iterations. Closes the priority-inversion footgun for preemptive priority RTOS. -
Wall-clock budget per call.
drive_ioacceptstime_budget_ms: a wall-clock cap that bounds total time spent firing callbacks. Time-triggered cyclic apps configure a fixed slot per cycle;drive_ioyields when the slot expires even ifmax_callbacksisn’t reached.
Once the cap (knob 2) ships, an additional refinement moves timer
and guard-condition dispatch into the backend’s drive_io loop so
the cap applies uniformly across all callback sources, not just
backend-RX-driven ones. This unifies the dispatch path and makes
max_callbacks = 1 mean “exactly one callback per spin_once,
regardless of whether it’s a sub, service, timer, or guard
condition.”
For the per-RTOS-model recommendations (which knobs to set, which defaults to use), see the RTOS Cooperation concepts page.
Why drive_io and not rmw_wait. Cooperative single-task runtimes
(bare-metal, single-threaded RTOS) have one execution context. A
waitset abstraction with kernel-blockable per-entity resources
doesn’t fit — there is no kernel to provide them. drive_io makes
the cooperative model explicit and lets backends pick the most
efficient wait primitive available on their target (kernel block on
multi-threaded; cooperative yield + WFI on bare-metal). The
scheduling-control trade-off the upstream waitset gives up is
recovered through the optional knobs above for apps that need it.
5. Serialization: CDR bytes, not typesupport
Upstream. Publish/take APIs accept typed messages plus typesupport pointers, and each backend implements serialization-from-typesupport itself:
rmw_ret_t rmw_publish(
const rmw_publisher_t * publisher,
const void * ros_message,
rmw_publisher_allocation_t * allocation);
The backend dereferences ros_message according to a
rosidl_message_type_support_t table to compute the wire bytes.
nano-ros. The runtime serializes upstream of the RMW. Backends receive already-CDR-encoded bytes:
rmw_ret_t (*publish)(rmw_publisher_t * publisher,
const uint8_t * data, size_t len);
int32_t (*take)(rmw_subscription_t * subscription,
uint8_t * buf, size_t len);
Why. rosidl typesupport is heavy: dynamic dispatch through a function-pointer table per field type, dependence on dynamic allocators for nested sequences, megabytes of generated symbol tables linked even for one message. Splitting the layers means:
- Codegen (
nros_generate_interfaces()from CMake) writes a fixed-shape C struct per message and a single<MsgType>_serialize_cdr(...)function. - The runtime calls it before handing bytes to the RMW.
- The RMW backend stays transport-only — no rosidl, no typesupport, no allocator coupling.
6. No graph cache
Upstream. Graph introspection sits in the RMW:
rmw_ret_t rmw_get_topic_names_and_types(...);
rmw_ret_t rmw_count_publishers(...);
rmw_ret_t rmw_get_node_names(...);
Implementations maintain a discovery cache that costs heap and CPU continuously even when nothing reads it.
nano-ros. None of the above. Backends do whatever discovery their
transport mandates (zenoh liveliness, XRCE-DDS session establishment),
but no get_topic_names / count_publishers / node_names exists in
the vtable.
Why. Graph introspection is fundamentally a host-side debugging
need. An MCU has no terminal, no ros2 topic list. Wire-protocol
interop with rmw_zenoh_cpp means standard ROS 2 tools running on a
laptop can introspect the same domain as the MCU — at zero cost on
the MCU.
7. QoS: full DDS-shaped profile, per-backend support advertised
Upstream. Full DDS QoS profile family with profile matching between endpoints:
typedef struct RMW_PUBLIC_TYPE rmw_qos_profile_t {
rmw_qos_history_policy_t history;
size_t depth;
rmw_qos_reliability_policy_t reliability;
rmw_qos_durability_policy_t durability;
rmw_time_t deadline;
rmw_time_t lifespan;
rmw_qos_liveliness_policy_t liveliness;
rmw_time_t liveliness_lease_duration;
bool avoid_ros_namespace_conventions;
} rmw_qos_profile_t;
The backend negotiates compatibility
(rmw_qos_profile_check_compatible) and surfaces mismatches as
runtime events.
nano-ros. Same field set, packed into 24 bytes:
typedef struct rmw_qos_profile_t {
uint8_t reliability;
uint8_t durability;
uint8_t history;
uint8_t liveliness_kind;
uint16_t depth;
uint16_t _reserved0;
uint32_t deadline_ms; /* 0 = infinite */
uint32_t lifespan_ms; /* 0 = infinite */
uint32_t liveliness_lease_ms; /* 0 = infinite */
uint8_t avoid_ros_namespace_conventions;
uint8_t _reserved1[3];
} rmw_qos_profile_t;
Standard profile constants
(NROS_RMW_QOS_PROFILE_DEFAULT, _SENSOR_DATA,
_SERVICES_DEFAULT, _SYSTEM_DEFAULT, _PARAMETERS) match
upstream rmw_qos_profile_* field-for-field, so applications
porting from rclcpp / rclrs can pull the equivalent profile
constant unchanged.
_SYSTEM_DEFAULT is the one where “field-for-field” is worth
spelling out: like upstream’s, it names no concrete policy at
all. Every field is the sentinel — NROS_RMW_{RELIABILITY, DURABILITY,HISTORY,LIVELINESS}_SYSTEM_DEFAULT are all 0, and
the depth sentinel is 0 too, matching
RMW_QOS_POLICY_DEPTH_SYSTEM_DEFAULT. The backend resolves it at
entity-create time, and the answer differs per backend, which
is the whole point of the name: cyclonedds resolves to
RELIABLE / VOLATILE / KEEP_LAST(1), mirroring what
rmw_cyclonedds_cpp does with the same sentinel; zenoh resolves
the depth to the subscriber ring the shim actually enforces; xrce
resolves the three policies the same way and leaves the depth
absent from the wire for the Agent to fill. Upstream’s own two
reference RMWs disagree here as well — rmw_cyclonedds_cpp picks
depth 1, rmw_zenoh_cpp picks 42 — so a profile constant that
baked a number would be wrong for every backend but one. Ask for
_DEFAULT if you want a stated reliable / volatile /
keep-last(10) profile. (issue 0829)
Per-backend support, no silent downgrade
Each backend advertises which policies it can honour via
Session::supported_qos_policies(), returning a QoSPolicyMask
bitfield. Policies a backend can’t enforce are explicit.
#![allow(unused)]
fn main() {
pub trait Session {
fn supported_qos_policies(&self) -> QoSPolicyMask {
QoSPolicyMask::CORE // reliability + durability VOLATILE + history + depth
}
}
}
The runtime validates the requested QoS against the backend’s mask
at entity-create time. Requesting a policy the backend doesn’t
support returns TransportError::IncompatibleQos
(NROS_RMW_RET_INCOMPATIBLE_QOS at the C boundary) synchronously.
There is no silent degradation — applications either get the
requested QoS or a hard error.
Apps that need cross-backend portability check the mask at startup:
#![allow(unused)]
fn main() {
if session.supported_qos_policies()
.contains(QoSPolicyMask::DEADLINE)
{
pub.create_with_qos(...deadline_ms = 100, ...);
} else {
// app-side fallback: timeout monitoring in user code
}
}
Manual liveliness assertion
For LIVELINESS_MANUAL_BY_TOPIC and LIVELINESS_MANUAL_BY_NODE,
publishers call assert_liveliness() explicitly:
#![allow(unused)]
fn main() {
pub.assert_liveliness()?; // refresh this publisher's lease
}
C side: nros_publisher_assert_liveliness(&pub). C++ side:
pub.assert_liveliness(). No-op for AUTOMATIC and NONE kinds.
Differences from upstream’s matching
Upstream surfaces QoS mismatches via runtime events
(RMW_EVENT_REQUESTED_INCOMPATIBLE_QOS). nano-ros surfaces them
synchronously at create time as IncompatibleQos. The mismatch
is a configuration error visible at startup; the runtime path
doesn’t need to handle it.
Two related choices:
- No profile matching between publisher and subscriber. Each endpoint requests the QoS it wants from its backend; the backend enforces locally. Cross-endpoint compatibility is the wire protocol’s concern — DDS endpoints negotiate via DDS Discovery, zenoh endpoints communicate intent via the topic-key encoding, uORB endpoints share an in-process queue. nano-ros’s executor doesn’t run a profile-matching pass.
- Wire metadata per backend. Lifespan needs per-sample timestamps; liveliness needs a keepalive mechanism. Each backend uses its native attachment / sample-info mechanism (Zenoh attachments, DDS RTPS sample-info, XRCE session pings) — no cross-backend metadata header.
Per-backend QoS coverage
The mask actually advertised by each backend’s
Session::supported_qos_policies():
| Backend | Reliability + Durability + History/Depth | Deadline | Lifespan | Liveliness Automatic | Liveliness Manual | Liveliness Lease | avoid_ros_namespace_conventions |
|---|---|---|---|---|---|---|---|
| Cyclone DDS | ✅ Native | ✅ Native | ✅ Native | ✅ Native | ✅ Native | ✅ Native | ✅ honoured |
| XRCE-DDS | ✅ Native (binary uxrQoS_t) | ✅ Shim-side clock check (sub: RequestedDeadlineMissed; pub: OfferedDeadlineMissed) + agent-side via FastDDS XML profile | ✅ Agent-side via FastDDS XML profile | ✅ Native | ✅ Configured via XML | ✅ Configured via XML | ✅ honoured |
| zenoh-pico | ✅ Shim-emulated | ✅ Clock-based check (sub + pub) | ✅ Subscriber-side filter using attachment timestamp | ✅ Trivial via session keepalive | ✅ Shim-side keepalive timer at pub publish path | ✅ Honoured | n/a (no /rt/ prefix) |
| uORB | ✅ CORE only (intra-process, no wire) | ❌ No rate concept | ❌ No expiry concept | ❌ No wire-level liveliness | ❌ | ❌ | n/a |
Key takeaways:
- The default QoS profile (RELIABLE + VOLATILE + KEEP_LAST(10) + AUTOMATIC) works on every backend.
- Apps that need extended QoS but want to stay backend-portable: check
supported_qos_policies()at startup and degrade gracefully. - For full DDS QoS: Cyclone DDS is the native DDS path; XRCE-DDS auto-routes through FastDDS XML profile when extended policies are set. Zenoh-pico fills the gap with shim-side emulation. uORB is intra-process only.
See Status events for how the deadline / liveliness / message-lost policies translate into runtime events, and User guide → Configuration for code examples.
8. Status events: callback-on-entity, Tier-1 subset
Upstream. Event APIs surface DDS-shaped notifications via a waitset-take pattern:
rmw_ret_t rmw_subscription_event_init(
rmw_event_t * event, const rmw_subscription_t * sub,
rmw_event_type_t type);
/* Add event_handle to a waitset alongside subscriptions. */
rmw_ret_t rmw_wait(...);
/* Poll fired events. */
rmw_ret_t rmw_take_event(
const rmw_event_t * event_handle,
void * event_info, bool * taken);
Eleven event types covering liveliness, deadline, QoS-incompatibility, match, message-lost, type-incompatibility — all dispatched through the waitset.
nano-ros. Callback-on-entity for a Tier-1 subset (liveliness
changes, deadline misses, message lost). Skips the waitset. Skips
Tier-2 (MATCHED) and Tier-3 (QOS_INCOMPATIBLE,
INCOMPATIBLE_TYPE) — see “What’s skipped” below.
#![allow(unused)]
fn main() {
sub.on_liveliness_changed(|status| {
if status.alive_count == 0 { trigger_failover(); }
})?;
sub.on_requested_deadline_missed(
Duration::from_millis(15),
|status| metric_inc(&LATE_SAMPLE_COUNT, status.total_count_change),
)?;
sub.on_message_lost(|status| log::warn!("dropped {}", status.total_count_change))?;
}
C side mirrors with nros_subscription_set_*_callback functions.
What lands
| Event | Producer | Subscriber callback / Publisher callback |
|---|---|---|
| Liveliness changed | sub | on_liveliness_changed(LivelinessChangedStatus) |
| Liveliness lost | pub | on_liveliness_lost(DeadlineMissedStatus) |
| Requested deadline missed | sub | on_requested_deadline_missed(deadline, DeadlineMissedStatus) |
| Offered deadline missed | pub | on_offered_deadline_missed(deadline, DeadlineMissedStatus) |
| Message lost | sub | on_message_lost(MessageLostStatus) |
What’s skipped
MATCHED— embedded apps usually have static topology; rarely load-bearing. Add the kind if a discovery-tracking app shows up; additive.QOS_INCOMPATIBLE/INCOMPATIBLE_TYPE— these surface at create time, not as runtime events. The existingrmw_ret_tcodes (NROS_RMW_RET_INCOMPATIBLE_QOS) carry the diagnostic synchronously fromcreate_publisher/create_subscription. No event needed.
Dispatch — callback-on-entity, not waitset-take
Events fire from inside the existing drive_io callback-dispatch
path. The backend’s RX worker detects an event in the same place it
detects messages; runs the registered callback; loops. No separate
waitset, no per-call take.
This reuses the message-callback dispatch model; events count
against the max_callbacks cap from Section 4 the same way message
callbacks do.
Why callback-on-entity instead of waitset-take. The waitset-take pattern requires a waitset abstraction nano-ros deliberately doesn’t have (see Section 4). Replacing it with per-entity callbacks reuses existing machinery, matches the message-callback ergonomics users already know, and keeps the bounded-storage property — each registered event-callback is a fixed-size struct embedded in the entity, no per-call allocation.
The trade-off: users can’t bulk-poll all events at once. For the Tier-1 events this isn’t load-bearing — events are rare, callbacks are cheap.
Backend coverage
Coverage is uneven and surfaces through Subscription::supports_event
(Rust) / register_*_event returning NROS_RMW_RET_UNSUPPORTED
(C). Apps must handle “not supported” — not every backend will
generate every event:
| Backend | Liveliness | Deadline | Message lost |
|---|---|---|---|
| Cyclone DDS | 🟡 Not wired through nano-ros events yet | 🟡 Not wired yet | 🟡 Not wired yet |
| XRCE-DDS | ❌ XRCE protocol carries no session→client liveliness callback | 🟡 Sub: shim-side clock check on take; pub: shim-side check on publish. LivelinessChanged / LivelinessLost not feasible. | ❌ topic_callback carries no per-sample sequence |
| zenoh-pico | ✅ Sub: per-publisher count via zpico_liveliness_get_count + wildcard keyexpr query; pub: shim-side keepalive timer fires LivelinessLost from publish when MANUAL_BY_* lease expires. | ✅ Clock-based check at sub + pub, rate-limited to ≤ 1 fire per deadline period | ✅ Sequence-gap detection from RMW attachment |
| uORB | ❌ No wire-level liveliness | ❌ No rate concept | ✅ Native: RustSubscriptionCallback publish-counter delta on host mock + real PX4 |
assert_liveliness() (manual): not all backends expose manual
liveliness through nano-ros yet. Unsupported backends’ default is
Ok(()) (no-op) since they don’t honour MANUAL_BY_TOPIC /
MANUAL_BY_NODE liveliness kinds.
9. Loaned messages first-class
Upstream. Optional, often unimplemented:
rmw_ret_t rmw_borrow_loaned_message(
const rmw_publisher_t * publisher,
const rosidl_message_type_support_t * type_support,
void ** ros_message);
rmw_ret_t rmw_return_loaned_message_from_publisher(
const rmw_publisher_t * publisher,
void * loaned_message);
A backend that doesn’t implement these returns
RMW_RET_UNSUPPORTED and the client falls back to a copying path.
nano-ros. Lending is a separate vtable surface. When the backend
supports zero-copy publish, it implements loan_publish_* /
commit_publish_*; when it supports zero-copy receive, it implements
loan_recv_* / release_recv_*. The runtime checks the function
pointers for non-NULL once at session open and takes the lending
path for the lifetime of the session.
Why. Zero-copy isn’t optional on embedded — every avoidable copy is a copy of a CDR-encoded sensor frame from a 64 KB heap. Promoting the lending surface to first-class makes it visible at compile time (missing pointer = compile error against the lending build) and lets the runtime pre-arrange arena slots once instead of probing on every publish.
10. Error returns
Upstream. Single error type for everything:
typedef int32_t rmw_ret_t;
#define RMW_RET_OK 0
#define RMW_RET_ERROR 1
#define RMW_RET_TIMEOUT 2
#define RMW_RET_UNSUPPORTED 3
/* ... */
Pointer-returning calls indicate failure with NULL and set a
thread-local error string via rmw_set_error_string.
nano-ros. Same named-constant style (<nros/rmw_ret.h>),
different sign convention, no thread-local error string:
typedef int32_t rmw_ret_t;
#define NROS_RMW_RET_OK 0
#define NROS_RMW_RET_ERROR -1
#define NROS_RMW_RET_TIMEOUT -2
#define NROS_RMW_RET_BAD_ALLOC -3
#define NROS_RMW_RET_INVALID_ARGUMENT -4
#define NROS_RMW_RET_UNSUPPORTED -5
#define NROS_RMW_RET_INCOMPATIBLE_QOS -6
#define NROS_RMW_RET_TOPIC_NAME_INVALID -7
#define NROS_RMW_RET_NODE_NAME_NON_EXISTENT -8
#define NROS_RMW_RET_LOAN_NOT_SUPPORTED -9
#define NROS_RMW_RET_NO_DATA -10
#define NROS_RMW_RET_WOULD_BLOCK -11
#define NROS_RMW_RET_BUFFER_TOO_SMALL -12
#define NROS_RMW_RET_MESSAGE_TOO_LARGE -13
Two return-shape conventions, picked by call shape:
| Returns | Success | Failure |
|---|---|---|
rmw_ret_t + entity-struct out-param (create_session, create_publisher, create_subscription, …) | NROS_RMW_RET_OK, out->backend_data non-NULL | negative named constant |
rmw_ret_t (destroy_session, drive_io, publish, send_response, …) | NROS_RMW_RET_OK | negative named constant |
int32_t byte count (take, take_request, take_response) | >= 0 (bytes received) | negative rmw_ret_t |
Differences from upstream.
- Negative for error. Upstream uses positive integer codes
(
RMW_RET_ERROR = 1); nano-ros uses negative so the byte-count convention can be unified into the sameint32_treturn. - No thread-local error string. No
rmw_set_error_string, normw_get_error_string. The thread-local heap allocation that pattern needs is unaffordable on embedded targets. Backends log verbose diagnostics at the failure site through the platform’sprintkequivalent — never buffered, never thread-local. - Smaller code-set. 13 codes total (vs upstream’s ~25). Phase
set started from upstream’s and dropped codes that don’t apply
(e.g., DDS event codes,
RMW_RET_NODE_INVALID). Adding a code is a<nros/rmw_ret.h>header change only.
Why same style. Named constants make switch statements
possible at call sites; bare negative ints don’t.
Publisher GID & message-info: functional-parity, different shape (issue 0242)
Two upstream rmw.h capabilities exist in nano-ros in a different
shape — the functionality is present, but the upstream-exact C entry
points are deliberately not mirrored. Recorded here so the divergence
reads as a decision, and updated (2026-07-27) to describe what nano-ros
actually provides today.
Message-info: surfaced via a builder, not a take_with_info slot
Upstream’s rmw_take_with_info fills an rmw_message_info_t (source +
received timestamp, publisher GID, publication + reception sequence
numbers) alongside the payload.
nano-ros provides the same information — MessageInfo carries all five
fields — but through a different seam:
- Application API (present). A subscription opts into message-info
with the
message_info()builder:
C mirrors this with#![allow(unused)] fn main() { node.subscription::<Msg>("/topic") .message_info() .build(|msg, info: Option<&MessageInfo>| { if let Some(i) = info { log::info!("from {:?} at {:?}", i.publisher_gid(), i.source_timestamp()); } })?; }nros_executor_register_subscription_raw_with_info(the callback receives the payload plus the attachment/metadata arguments directly — there is no info struct on the C side). - What’s carved out. The hot
takevtable slot returns bytes ONLY. The metadata rides a separateMessageInfoSlotside-channel that the backend populates from its native attachment / sample-info (Zenoh attachment, DDS sample-info, XRCE topic callback) and the runtime pairs with the payload. Atake_with_info-shaped vtable slot would widen the ABI of the byte-count-return take path (int32_tbytes) for no functional gain — the side-channel already delivers the info to themessage_info()builder.
Publisher identity: per-message GID, no standalone query
Upstream exposes rmw_get_gid_for_publisher(pub, out_gid) and stamps
every sample with the writer’s GID.
nano-ros does carry a publisher GID where the wire supports it: a
zenoh publisher generates a 16-byte GID at create time
(RmwAttachment::generate_gid) and stamps it into the per-sample
attachment; the subscriber extracts it into MessageInfo.publisher_gid,
so per-message publisher attribution is observable via
info.publisher_gid() (populated on the zenoh path; zero-filled on
backends whose wire carries no writer GID).
- What’s carved out. The standalone
rmw_get_gid_for_publisherQUERY — reading a publisher’s own GID back through a C API — is not provided. The publisher holds its GID internally, but no in-tree consumer needs to query it: the one need, bridge loop-suppression / dedup, is served by thebridge_originattachment, and per-message attribution is already covered byMessageInfo.publisher_gid.
If a consumer shows up
Both carved-out shapes — a take_with_info vtable slot and a
get_publisher_gid query slot — can land later as NULL-able optional
vtable slots (the established tail-append extension pattern, RFC-0035)
without an ABI break, the day a concrete consumer needs the upstream-exact
entry point rather than the builder / per-message forms above.
See also
- RMW API Design — deeper architectural rationale (heap, threading, dispatch model) shared across all the points above.
- Custom RMW Backend — step-by-step guide to writing a backend in C against this vtable.
<nros/rmw_vtable.h>Doxygen — full C reference for every function pointer above.packages/rmw/zenoh/nros-rmw-zenoh— canonical reference port. Read the source for a worked example.
Platform API Design
The platform API (nros-platform) sits below the RMW backend. It
exposes the OS or hardware primitives that zenoh-pico and XRCE-DDS
need: a clock, optionally a heap, optionally threading, optionally
networking. This page explains why the trait surface is grouped
the way it is.
Canonical interface spec. The function-pointer signatures, parameter docs, ownership rules, blocking allowance, and failure modes for every platform entry live in the platform-cffi Doxygen reference. This doc is rationale, not spec — when the two disagree, the Doxygen wins. The Rust trait crate
nros-platform-apimirrors the C signatures one-for-one.
For per-platform behaviour comparison, see Platform Differences. For writing a new port, see Custom Platform.
Trait groups and rationale
The traits cluster into seven concern groups. Each group is independent: a platform can provide some and stub others, and the RMW backend declares which it actually needs.
Time – PlatformClock
Every backend needs a monotonic clock. zenoh-pico uses milliseconds for socket timeouts and lease management; it uses microseconds for finer-grained protocol math (KeepAlive intervals, lease expiry calculations). We expose both clock_ms and clock_us rather than a single clock_ns because:
- 32-bit MCUs without a hardware microsecond tick cannot serve
clock_nsaccurately. Synthesizing nanoseconds from a 1 ms tick is a lie that hides clock resolution from the caller. u64nanoseconds wraps after ~584 years, but on a Cortex-M0 every multiply-divide on a 64-bit value costs cycles that the lease task does not have to spare.- The two functions can share a single hardware source:
clock_usreturns the raw counter,clock_msdivides by 1000. Platforms with a slow tick (1 ms) implementclock_usasclock_ms * 1000and accept the resolution loss.
The clock is monotonic and wraparound-free for system lifetime. There is no failure mode – if the platform cannot produce time, it cannot run nano-ros at all.
Memory – PlatformAlloc
Only zenoh-pico needs PlatformAlloc. XRCE-DDS does not allocate. The trait is a thin malloc/realloc/free shim because zenoh-pico’s internal buffer types expect that contract.
Bare-metal platforms back this with a bump allocator (linked-list-allocator or embedded-alloc). RTOS platforms back it with pvPortMalloc (FreeRTOS), tx_byte_allocate (ThreadX), k_malloc (Zephyr), or malloc (NuttX, POSIX). zenoh-pico’s working set is ~64 KB total; the allocator must have at least that much budget, ideally more.
Threading – PlatformThreading
Three sub-areas: tasks (spawn/join/exit), mutexes (regular + recursive), and condition variables. Single-threaded targets (bare-metal) provide stub implementations: task_init returns -1 (so zenoh-pico’s lease task spawn fails gracefully and the application drives lease-keepalive itself), and mutex_lock/condvar_wait are no-ops that always succeed.
The condvar API is the load-bearing one: zenoh-pico’s blocking z_get and the C++ Future::wait both block on a condvar that the receive callback signals. On single-threaded platforms there is no thread to block, so the blocking C++ wait paths are not used (the C++ action client status note covers the migration to non-blocking polling).
Sleep / Random / Wall-time
Three small zenoh-pico-only traits grouped for convenience:
PlatformSleep– delay APIs. On bare-metal with smoltcp, the implementation must callnetwork_poll()while busy-waiting, otherwise packets queue and the lease times out.PlatformRandom– a 32-bit xorshift PRNG seeded with hardware entropy (or a user-supplied seed). Used for session IDs and protocol nonces, not cryptography.PlatformTime– wall-clock time for log timestamps. On bare-metal with no RTC, return monotonic time as a fallback.
Networking – PlatformTcp / PlatformUdp / PlatformSocketHelpers
zenoh-pico’s network layer is split into three traits because the original C interface (unix/network.c) has three concerns:
- TCP and UDP each have their own
open/read/send/closebecause the backend opens different socket types for each. PlatformSocketHelperscarries the cross-cutting operations –set_non_blocking,accept, genericclose, andwait_event– that apply to either socket family.
Sockets and endpoints are opaque *mut c_void pointers; their underlying types vary per platform (POSIX int, lwIP struct netconn*, Zephyr socket descriptor, smoltcp SocketHandle). The shim layer auto-detects the type sizes from C headers at build time so the FFI boundary stays type-erased.
PlatformUdpMulticast is split out as a fourth networking trait because embedded targets that connect to a fixed tcp/host:port locator never multicast and should not pay the code-size cost of multicast plumbing.
NetworkPoll – PlatformNetworkPoll
Bare-metal only. network_poll() advances the smoltcp state machine, processing pending RX/TX. Platforms with kernel-level networking (Linux, lwIP-on-FreeRTOS, NetX-on-ThreadX, Zephyr sockets) drive their own NIC and don’t need this.
PlatformSleep and the wait_event helper both call network_poll() while waiting, so packets keep flowing during otherwise-idle time. Without this hook, smoltcp would only receive when the application explicitly asked for it – a recipe for dropped TCP segments.
Libc – PlatformLibc
zenoh-pico uses strlen, memcpy, errno, etc. directly. Bare-metal targets that link picolibc or newlib-nano get these for free. Targets without a C runtime (some no_std builds) provide the trait, which forwards to Rust implementations of the same functions.
This trait exists because the alternative – patching zenoh-pico to call platform shims for every libc function – would require modifying upstream sources we don’t control.
Why clock_ms and clock_us, not clock_ns
Summarized from above:
| API | Pros | Cons | Verdict |
|---|---|---|---|
clock_ns only | Single function, finest resolution | 64-bit math on every call; lies on 1 ms-tick MCUs | Rejected |
clock_ms only | Cheap, fits zenoh-pico’s lease math | Insufficient resolution for sub-millisecond protocol timing | Insufficient |
Both clock_ms and clock_us | Each call is cheap and honest about its resolution | Two functions to implement | Chosen |
Behavior contracts
Each trait below has a contract table. Columns:
- Method – name (matches the trait definition).
- Blocking? – whether the method may suspend the caller.
- May fail? – whether the method has a meaningful failure mode.
- Unsupported fallback – what to do when the platform cannot provide the capability.
- Notes – extra constraints (monotonicity, reentrancy, side effects).
PlatformClock
| Method | Blocking? | May fail? | Unsupported fallback | Notes |
|---|---|---|---|---|
clock_ms | No | No | Required for any backend | Monotonic; wraparound-free for system lifetime |
clock_us | No | No | Required for any backend | Same monotonic base as clock_ms |
PlatformAlloc
| Method | Blocking? | May fail? | Unsupported fallback | Notes |
|---|---|---|---|---|
alloc | No | Yes (null) | Required for zenoh-pico | Caller checks for null and propagates as RMW error |
realloc | No | Yes (null) | Required for zenoh-pico | Existing block must be preserved on failure |
dealloc | No | No | Required for zenoh-pico | dealloc(null) is a no-op |
PlatformSleep
| Method | Blocking? | May fail? | Unsupported fallback | Notes |
|---|---|---|---|---|
sleep_us | Yes | No | Required for zenoh-pico | Bare-metal must call network_poll() during busy-wait |
sleep_ms | Yes | No | Required for zenoh-pico | Same |
sleep_s | Yes | No | Required for zenoh-pico | Same; typically implemented as sleep_ms(s * 1000) |
PlatformRandom
| Method | Blocking? | May fail? | Unsupported fallback | Notes |
|---|---|---|---|---|
random_u8 / _u16 / _u32 / _u64 | No | No | Required for zenoh-pico | xorshift32 is sufficient; not cryptographic |
random_fill | No | No | Required for zenoh-pico | Fills len bytes; no upper bound check |
PlatformTime (wall-clock)
| Method | Blocking? | May fail? | Unsupported fallback | Notes |
|---|---|---|---|---|
time_now_ms | No | No | Required for zenoh-pico | Return clock_ms if no RTC |
time_since_epoch | No | No | Required for zenoh-pico | Return (monotonic_s, 0) if no RTC |
PlatformThreading – tasks
| Method | Blocking? | May fail? | Unsupported fallback | Notes |
|---|---|---|---|---|
task_init | No | Yes (-1) | Return -1 on single-threaded | zenoh-pico’s lease task must degrade gracefully |
task_join | Yes | Yes | Return 0 (success) | Single-threaded never spawned a task to join |
task_detach | No | Yes | Return 0 | Same |
task_cancel | No | Yes | Return 0 | Same |
task_exit | No | No | No-op | Caller is the only thread |
task_free | No | No | No-op | No allocation to free |
PlatformThreading – mutexes (regular and recursive)
| Method | Blocking? | May fail? | Unsupported fallback | Notes |
|---|---|---|---|---|
mutex_init / mutex_rec_init | No | Yes | Return 0 | Single-threaded has no mutex state |
mutex_drop / mutex_rec_drop | No | Yes | Return 0 | Same |
mutex_lock / mutex_rec_lock | Yes | Yes | Return 0 (success) | Single-threaded: no contention possible |
mutex_try_lock / mutex_rec_try_lock | No | Yes | Return 0 | Always “succeeds” on single-threaded |
mutex_unlock / mutex_rec_unlock | No | Yes | Return 0 | Same |
PlatformThreading – condition variables
| Method | Blocking? | May fail? | Unsupported fallback | Notes |
|---|---|---|---|---|
condvar_init | No | Yes | Return 0 | No state on single-threaded |
condvar_drop | No | Yes | Return 0 | Same |
condvar_signal | No | Yes | Return 0 | No waiter to wake |
condvar_signal_all | No | Yes | Return 0 | Same |
condvar_wait | Yes | Yes | Return 0 | Single-threaded must use polling instead – avoid this path |
condvar_wait_until | Yes | Yes (timeout) | Return 0 immediately | Same; blocking C++ Future::wait deadlocks on single-threaded (use non-blocking polling instead) |
PlatformTcp
| Method | Blocking? | May fail? | Unsupported fallback | Notes |
|---|---|---|---|---|
create_endpoint | Yes (DNS) | Yes | Required for zenoh-pico | Backed by getaddrinfo or platform equivalent |
free_endpoint | No | No | Required | Mirrors freeaddrinfo |
open | Yes | Yes | Required | Connect with timeout in ms |
listen | No | Yes | Optional (server mode) | Bare-metal client typically returns -1 |
close | No | No | Required | Shutdown + close |
read | No (after set_non_blocking) | Yes (usize::MAX) | Required | Returns 0 if no data; must be non-blocking for zenoh-pico’s poll loop |
read_exact | Yes | Yes | Required | Used for length-prefixed framing |
send | Yes | Yes | Required | May block on socket buffer full |
PlatformUdp
| Method | Blocking? | May fail? | Unsupported fallback | Notes |
|---|---|---|---|---|
create_endpoint | Yes (DNS) | Yes | Required for zenoh-pico | Same as TCP but SOCK_DGRAM |
free_endpoint | No | No | Required | |
open | No | Yes | Required | UDP socket open is non-blocking |
close | No | No | Required | |
read | No | Yes | Required | recvfrom; returns 0 if no datagram |
read_exact | Yes | Yes | Required | Rarely used (UDP is message-oriented) |
send | Yes | Yes | Required | sendto |
PlatformSocketHelpers
| Method | Blocking? | May fail? | Unsupported fallback | Notes |
|---|---|---|---|---|
set_non_blocking | No | Yes | Required | Critical: enables non-blocking read |
accept | No (after set_non_blocking) | Yes | Optional | Server-mode only |
close | No | No | Required | Generic socket close |
wait_event | Yes | Yes | Required | Multi-threaded: yields to scheduler. Single-threaded: spins + network_poll |
PlatformUdpMulticast (optional)
| Method | Blocking? | May fail? | Unsupported fallback | Notes |
|---|---|---|---|---|
mcast_open | Yes | Yes | Stub returns -1 | Skipped on embedded targets without scouting |
mcast_listen | Yes | Yes | Stub returns -1 | Same |
mcast_close | No | No | No-op | |
mcast_read / mcast_read_exact | Varies | Yes | Stub returns usize::MAX | |
mcast_send | Yes | Yes | Stub returns usize::MAX |
PlatformNetworkPoll (bare-metal only)
| Method | Blocking? | May fail? | Unsupported fallback | Notes |
|---|---|---|---|---|
network_poll | No | No | OS-driven platforms: no-op | Advances smoltcp once; called by sleep_* and wait_event |
PlatformLibc (bare-metal without libc)
| Method | Blocking? | May fail? | Unsupported fallback | Notes |
|---|---|---|---|---|
strlen, strcmp, strncmp, strchr, strncpy | No | No | Linker provides if libc present | Same semantics as the C standard |
memcpy, memmove, memset, memcmp, memchr | No | No | Same | Same |
strtoul | No | Yes (errno) | Same | Used by zenoh-pico to parse locator strings |
errno_ptr | No | No | Same | Returns pointer to thread-local (or static) errno |
Cross-cutting rules
Two contract rules apply to every trait method:
-
Reentrancy. Methods may be called from any context the executor enters: a publisher callback, a service handler, or directly from user code. Implementations must not assume a particular calling thread or critical-section state. Single-threaded platforms get this for free; RTOS platforms must use reentrant primitives.
-
No panics across the FFI boundary. All trait methods are exposed to C through the shim crates. Panicking through C is undefined behavior. Implementations return error codes (or
usize::MAXfor byte counts) instead of panicking. The exception isPlatformClock– if the clock cannot be read, the system is fundamentally broken and there is nothing useful to return.
Codegen — the pack pipeline
nano-ros generates ROS 2 message/service/action types (and their per-package
scaffolding) from .msg/.srv/.action files. The generator is a four-stage
pipeline (RFC-0068): parse → resolve → lower → render. This page covers the
last stage — render — and how to change or add a target language.
Render = a data pack + a runtime template
Every backend renders through one minijinja environment
(packages/cli/rosidl-codegen/src/render.rs) over data packs under
packages/cli/rosidl-codegen/packs/:
| pack | output |
|---|---|
packs/c/ | C headers + sources |
packs/rmw/ | RRR-compatible Rust message layer |
packs/rust/ | idiomatic (rclrs-style) Rust |
packs/nros/ | embedded (no_std) Rust |
packs/cpp/ | C++ headers + the Rust FFI glue |
packs/scaffold/ | per-package Cargo.toml / lib.rs / build.rs |
A pack is just .jinja templates. The Rust side hands each template a
serde-serialized view struct (the render context) and never spells a type
itself — the type strings are composed in the pack by registered filters:
c_type/c_array_suffix,cpp_type/cpp_array_suffix,cpp_repr_c_type/cpp_view_repr_typerust_type_rmw/rust_type_idiomatic,nros_typesnake_case
The view struct carries only neutral facts (the parsed field_type, resolved
capacity, storage mode, current_package, …); the filter maps those to the
language’s syntax. This is RFC-0068’s “what vs how” seam: what a type is lives in
the IR; how a language spells it lives in the pack + its filter.
Changing a template
Edit the .jinja under packs/<lang>/ and rebuild the CLI (just setup-cli).
The codegen fingerprint (RFC-0061) hashes every bundled pack’s content, so any
template edit marks the affected fixtures stale — no separate bookkeeping.
Overriding a pack at runtime — no rebuild
Point the renderer at an external directory of .jinja files: a file named
<template-name> or <template-name>.jinja there overrides the bundled pack
of that name; anything absent falls back to bundled.
export NROS_TEMPLATE_DIR=/path/to/my/pack
nros generate-rust … # uses the override, no recompile
(Equivalently, rosidl_codegen::render::set_template_dir(dir) from Rust, called
once before the first render.)
The stable template names are the keys of PACKS in render.rs (e.g.
message.h, message_nros.rs, cargo.toml, _field.jinja). tests/ external_pack_smoke.rs proves the override + fallback.
Do not set
NROS_TEMPLATE_DIRduring fixture or CI builds. The fingerprint hashes the bundled packs; an external override would silently produce output that disagrees with the recorded fingerprint.
Adding a language
Adding a target language is a pack + a spelling filter, not a rewrite:
- add its
.jinjatemplates (a newpacks/<lang>/) and its rows inPACKS; - if it needs type spelling the existing filters don’t cover, register one more
filter wrapping a
*_spellingfunction intypes.rs; - add the generator entry that builds the view struct and calls
render::render("<template-name>", &ctx).
No per-language type logic lives in the builders — the packs and their filters own it. Implemented by phase-335 (RFC-0068).
Canonical Platform C ABI
nano-ros separates the platform layer (clock, sleep, allocator,
threading, critical section, network, timer, …) from the rest of the
code via a single canonical C ABI of free extern "C" symbols.
Every supported port — POSIX, FreeRTOS, NuttX, ThreadX, Zephyr,
ESP-IDF, bare-metal Cortex-M — implements the same symbols against
its host kernel. RMW backends, codegen output, and the
nros-node runtime all link against the ABI; nobody links against a
specific port’s Rust crate.
This page is the contract: what the surface looks like, how to add a
port, and why the shape is what it is. It is the implementation
companion to Platform Model (user-
facing axis description) and
docs/design/0006-portable-rmw-platform-interface.md
(L0/L1/L2 design rationale across RMW + platform).
Surface
Three hand-written headers under
packages/platform/nros-platform-cffi/include/nros/:
| Header | Purpose | Symbol count |
|---|---|---|
platform.h | Core kernel surface: clock, sleep, alloc + heap stats, threading, scheduler, time, yield, random, critical section, opaque wake primitive. | 59 |
platform_net.h | Network surface: TCP/UDP/multicast socket helpers, endpoint resolution, IVC. | 29 |
platform_timer.h | Periodic timer surface (nros_platform_timer_*). | 8 |
Every symbol has the prefix nros_platform_. A drift gate
(scripts/check-platform-abi-mirror.sh) walks each header and asserts
that the unsafe extern "C" {} mirror block in
packages/platform/nros-platform-cffi/src/lib.rs declares the same set —
no symbol can land in C without its Rust mirror, no Rust mirror can
declare a symbol without a C header decl. just check runs the gate
on every CI build.
Why free symbols (not a vtable struct)
The RMW layer uses a NrosRmwVtable fn-ptr struct because a single
binary registers multiple backends at runtime (bridge nodes). Platforms
are different: one platform per binary, resolved at link time. A
vtable would add an indirection on every clock read, every mutex lock,
every socket send — overhead with no upside, because there is nobody to
swap. Free symbols let the linker resolve calls direct, let LTO inline
across the boundary, and let static analysis treat them like any other
extern.
The full rationale is in
docs/design/0006-portable-rmw-platform-interface.md
under “Platform ABI: free symbols (no vtable)”.
How a port is built
A port can be written in either of two ways. The choice is per-port and not visible to consumers — both shapes resolve to the same symbols.
Path A — Rust trait + macro export
The Rust crate implements PlatformClock, PlatformAlloc,
PlatformThreading, … on a marker type, then invokes one of the
export macros:
#![allow(unused)]
fn main() {
use nros_platform_api::{
PlatformClock, PlatformSleep, PlatformAlloc, PlatformThreading,
PlatformCriticalSection,
nros_platform_export, nros_platform_export_net, nros_platform_export_timer,
};
pub struct PosixPlatform;
impl PlatformClock for PosixPlatform { /* … */ }
impl PlatformSleep for PosixPlatform { /* … */ }
// … etc.
nros_platform_export!(PosixPlatform);
nros_platform_export_net!(PosixPlatform);
nros_platform_export_timer!(PosixPlatform);
}
The macros expand to the full set of #[unsafe(no_mangle)] pub extern "C" fn nros_platform_* bodies forwarding to the trait calls.
This is what nros-platform-posix does today. It is also what the
bare-metal board crates (nros-platform-mps2-an385,
nros-platform-stm32f4, nros-platform-esp32, nros-platform-esp32-qemu)
use: there is no host kernel to write idiomatic C against, so the
single-task stub impls are written in Rust and exported via the macro.
Path B — pure C port
The kernel-side ports (FreeRTOS, NuttX, ThreadX, Zephyr, ESP-IDF) write
the bodies directly in src/platform.c, src/net.c, src/timer.c
inside each packages/platform/nros-platform-<rtos>/ directory. The Rust
side has no implementation file — the build script (or parent build
system: NuttX make, Zephyr west module, ESP-IDF cmake) compiles the C
sources against the kernel’s headers.
Pure-C bodies are the more natural fit when the kernel already speaks
C and ships C headers (xSemaphoreCreate*, k_sem_*, tx_thread_*):
the impl is one-to-one with the kernel call, no FFI dance through
Rust’s calling convention. The shared crate
nros-platform-critical-section is the canonical example for a
single-symbol shim: the Rust side just calls the externs and registers
the result with critical_section::set_impl!.
Adding a new port — checklist
- Decide A vs B. Greenfield host build with a Rust crate available? Path A. Vendor RTOS with a C SDK and no Rust toolchain on the build machine? Path B.
- Mirror, don’t extend. Implement every symbol in the three
headers. The drift gate fails the build otherwise. If the kernel
genuinely cannot provide a primitive, return the documented
sentinel value (
-1fortask_initon single-task RTOS,0formutex_*on single-core no-preempt hardware) rather than skipping the symbol. - Write the smoke test. Land a
tests/<port>-c-smoke/mini-app that links the new C port against the canonical headers and calls a representative symbol from each capability group. Wire it intojust <port> test-c-port. The smoke tests are the runtime parity layer that the drift gate cannot enforce. - For platforms that emit
critical_section::Impl(i.e. anything that consumes acritical-section-using crate), make sure the binary pullsnros-platform-critical-sectiononce — it does thecritical_section::set_impl!(PlatformCs)registration against the canonical externs. Binaries that don’t need the global registration don’t pay for it. - Update the drift gate’s smoke list (the
HEADERS_REQUIRE_MACROarray inscripts/check-platform-abi-mirror.sh) if you add a new header. Adding a symbol to an existing header needs no script change — the grep is generic.
Capability groups
Within platform.h, the symbols are grouped by trait. Each group is
exported by one Rust trait + one C-port section + one drift-gate
match. Adding a capability means adding a row to every column.
| Capability | Rust trait | C section | Symbols |
|---|---|---|---|
| Clock | PlatformClock | clock_ms | nros_platform_clock_ms |
| Sleep | PlatformSleep | sleep_ms | nros_platform_sleep_ms |
| Alloc | PlatformAlloc | malloc/realloc/free | nros_platform_alloc{,_realloc,_free} |
| Threading | PlatformThreading | mutex/condvar/task | nros_platform_{mutex,condvar,task}_* |
| Critical section | PlatformCriticalSection | per-CPU interrupt mask | nros_platform_critical_section_{acquire,release} |
| Scheduler | PlatformScheduler | task hints | nros_platform_scheduler_* |
| Time | PlatformTime | wall-clock | nros_platform_time_ns |
| Yield | PlatformYield | cooperative yield | nros_platform_yield_now |
| Random | PlatformRandom | best-effort RNG | nros_platform_random_* |
| Wake | PlatformThreading (wake methods) | opaque binary-semaphore | nros_platform_wake_{init,drop,wait_ms,signal,signal_from_isr,storage_size,storage_align} |
platform_net.h covers TCP/UDP/multicast/IVC, and platform_timer.h
covers periodic timers. The shapes follow the same pattern: trait →
macro → header → drift gate.
Status and architecture
The platform tier today is:
- One canonical header per capability family (
platform.h,platform_net.h,platform_timer.h). Every port mirrors them exactly; a drift gate fails the build on divergence. - Pure-C ports under
nros-platform-{posix,freertos,nuttx,threadx,zephyr,esp-idf}— the per-RTOS Rust platform crates were retired in favour of one C body per RTOS. critical_sectionpromoted to a canonical platform capability owned by every port’s C body;nros-platform-critical-sectionis the global-registration shim.- An opaque-storage wake primitive (
nros_platform_wake_*) — a binary semaphore that lets the executor block on RMW activity without burning a thread.
The current canonical surface is 59 + 29 + 8 = 96 symbols across three headers, mirrored exactly by the Rust extern block, exported by six ports plus four bare-metal board crates, and gated by one drift script. Adding a capability touches all four columns in one PR.
The scalar / opaque-struct boundary (RFC-0034 D2)
Not every kernel service unifies the same way. The ABI splits into two classes, and the split is a design boundary, not tech debt:
-
Scalar services — fully unified. Alloc, sleep, clock, time, yield, random. Their ABI is plain scalars (sizes, millisecond counts, byte buffers) with no host-type in the signature, so a single canonical
nros_platform_*symbol works for every RTOS and every consumer (core, RMW, vendored zenoh-pico/XRCE) funnels through it. The D8 gate (check-no-direct-kernel-alloc.sh) enforces this for allocation; the same pattern extends to the other scalars (the vendoredz_*/uxr_*funnel is the CI-relink-gated remainder of phase-230). -
Opaque-struct services — stay per-RTOS-vendored. Threading (
mutex/condvar/task), the wake primitive, and network sockets carry RTOS-defined structs (SemaphoreHandle_t,TX_MUTEX,struct k_mutex, socket/NX_*control blocks) whose layout the vendor owns. A single C ABI cannot name those types portably, so each port vendors its own body and nano-ros exposes only opaque storage (*_storage_size/*_storage_align+ an init/drop pair). This is why the ThreadX board’stx_byte_allocatethread-stack / NetX-pool sites are not D8 violations — they are the task and net opaque services, allowed on a documented symbol-scoped lint allowlist.
Escape hatch (if an opaque service must move later). The way out is
NOT to widen the C ABI with host types; it is to pin a canonical
fixed-layout struct in platform.h, give each port a compile-time
size_probe / _Static_assert that the canonical layout is ≥ the
vendor struct (so opaque storage stays sound), and let the port
memcpy/place the vendor object inside the canonical slot. Net is the
first candidate — platform_net.h already exposes 29 symbols over an
opaque socket handle — but it is deferred (RFC-0034 “out of scope”)
until there is a consumer that needs portable socket structs. Threads
and sync are the least likely to move (deepest struct coupling).
Measuring Static Memory
The Static Pool Inventory tells you
which knobs exist and what they cost at their defaults. It cannot tell you what
your image costs — your knobs differ, your backend differs, and several pools
are sized by a sizeof that no comment can see.
For that, measure the image you built:
just mem-report path/to/your/binary
You get RAM broken down three ways — by symbol, by crate, and by declared pool — plus the section totals, so you can see how much is not attributable to any symbol:
RAM (.bss + .data), by section: 357,154 bytes
RAM attributed to symbols: 342,962 bytes
unattributed (padding, linker reservations, symbol-less data): 14,192 bytes (4.0%)
That last line matters. Alignment padding and linker-script reservations are real RAM that no symbol names, so a budget built only from a list of pools will come up short.
Finding what to cut
The per-symbol list is sorted, so the first few lines are usually the whole story:
## top 5 RAM symbols
144,128 40.4% nros_rmw_zenoh::shim::service::SERVICE_BUFFERS
131,072 36.7% nros_rmw_zenoh::shim::subscriber::LARGE_PAYLOADS
32,768 9.2% nros_rmw_zenoh::shim::subscriber::SMALL_PAYLOADS
Cross-reference each name against the pool inventory to find the knob that moves it. Most of the large ones are pools with a knob you can set at build time.
Be aware that today these pools are sized by which backend you link, not by what your node actually does: a publisher-only node still reserves the service and large-payload pools in full. If your image looks far larger than the entities you created would suggest, that is expected rather than a misconfiguration on your side — see issue 0827. Turning the corresponding knobs down is the current remedy.
Showing a saving
Take a baseline, change something, and compare:
just mem-report my-binary --json > before.json
# ... tune a knob, rebuild ...
just mem-report my-binary --baseline before.json
Symbols that moved are annotated with their delta. This is also how a change to nano-ros itself should report a memory saving: as a measured difference between two named images, not as an estimate.
Cross-compiled images
The tool prefers llvm-nm, which reads ELF files for any target; your host’s
GNU nm is built for one target family and refuses a cross-built image with
“File format not recognized”.
You probably already have llvm-nm without knowing it: rustup ships it as part
of the llvm-tools component, under the toolchain’s own bin directory rather
than on your PATH, and the tool looks there. If it reports that it found no
usable nm, run:
rustup component add llvm-tools
RMW Backends — Host-Language Policy
This page documents which language each RMW backend is implemented in,
and the rule that decides it. The matrix was originally frozen
2026-05-07; the L tier (+ L.8,
landed 2026-05-12) collapsed the public RMW surface so every backend
now reaches the runtime via the same nros_rmw_vtable_t bridge. The
underlying implementations still ship in whichever language their
upstream library prefers, but consumers never see that — they only
see the C vtable.
The rule (post-115.L)
Every backend installs itself via the
nros_rmw_vtable_tbridge. The underlying library’s host language is an implementation detail of the per-backend-cffishim; it does not appear on the consumer surface.
The original rule (a backend’s host language matches its underlying
library’s native language unless overridden) is still how we pick
the inside of each -cffi shim — but the shim itself is uniform:
a small Rust or C++ TU that fills in a vtable and calls
nros_rmw_cffi_register(&vtable) once at startup.
Hierarchy
nros-core (Rust) ──→ Rmw trait (internal; bridged by RustBackendAdapter<R>)
└──→ nros-rmw-cffi (C ABI bridge, registry)
↓ nros_rmw_vtable_t (~17 fn ptrs)
├──→ nros-rmw-zenoh (wraps Rust nros-rmw-zenoh)
├──→ nros-rmw-xrce-cffi (links C nros-rmw-xrce)
├──→ nros-rmw-cyclonedds (C++ direct, no Rust)
└──→ nros-rmw-uorb (C++ direct, no Rust)
The shims are the canonical consumer surface. Public Cargo features
on nros / nros-c / nros-cpp (rmw-{zenoh,xrce}-cffi,
cffi-{zenoh-cffi,xrce-c}) all route through the same
nros_rmw_vtable_t runtime. The pre-L.7 direct-Rust-trait features
(rmw-zenoh, rmw-xrce, rmw-uorb) are gone.
Phase 169 (2026-05-19) — dust-dds retired. The nros-rmw-dds
Rust shim and the dust-dds upstream Rust DDS implementation
have been removed (Phase 169.4). Cyclone DDS is the sole DDS
backend; the nros-rmw-cyclonedds shim registers under its
canonical name "cyclonedds" only. The previous "dds" generic
slot is not aliased — callers always select Cyclone by its
specific name (NROS_RMW=cyclonedds, target_link_libraries(... NanoRos::Rmw::cyclonedds), etc.).
Any language with stable C-ABI interop (C, C++, Zig, Rust,
Go-via-cgo, Python-via-ctypes…) can implement a backend by filling
in the vtable and calling nros_rmw_cffi_register(&vtable) once at
startup.
Decision matrix (post-115.L, updated by Phase 171)
| Backend | Underlying lib | Underlying lang | Shim crate | Verdict |
|---|---|---|---|---|
| Cyclone DDS | Cyclone DDS | C / C++ | nros-rmw-cyclonedds (C++ direct vtable; canonical DDS backend) | keep |
| XRCE | micro-XRCE-DDS-Client | C | nros-rmw-xrce-cffi (Rust shim over the C nros-rmw-xrce static lib; 115.K.2 ported) | keep |
| zenoh-pico | zenoh-pico | C | nros-rmw-zenoh (Rust → vtable via RustBackendAdapter<ZenohRmw>) | keep |
| uORB | PX4 module SDK | C++ | nros-rmw-uorb (C++ direct vtable; 115.K.4 port replaces legacy nros-rmw-uorb Rust crate) | keep |
Dust-DDS was retired in Phase 169 (2026-05-19) after repeated bring-up failures on embedded targets. It is intentionally absent from the active backend table; Cyclone DDS now fills the DDS slot.
Rust-backend cffi shape
For backends whose upstream library is Rust (zenoh-pico) the cffi
shim ships as a tiny crate that calls
RustBackendAdapter::<UnderlyingRmw>::register(). The adapter
monomorphizes a static nros_rmw_vtable_t over the Rust Rmw
trait impl and installs it into the C registry. Consumer code never
sees the trait surface; it only sees the vtable.
The legacy direct-Rust-trait crates (nros-rmw-zenoh,
nros-rmw-xrce) stay in the workspace as internal-only
implementation libs of these shims. They have no public Cargo
feature reaching them after.
C-/C++-backend cffi shape
For backends whose upstream library is C/C++ (Cyclone DDS, uORB,
XRCE) the cffi shim is a standalone CMake project that builds a
static C/C++ library and registers a nros_rmw_vtable_t at startup
via nros_rmw_cffi_register. No RustBackendAdapter is involved.
The Rust runtime sees these via the same registry; the
NANO_ROS_RMW=<name> CMake selector flips a build-time macro that
ensures the register call is wired into nros::init.
Cyclone DDS: runtime type introspection (Phase 212.K.7)
The Cyclone DDS shim does not require per-type backend code on the
generated msg crate. Phase 212.K.7 inverted the original design where
each msg crate carried an optional cyclonedds Cargo feature plus a
Cyclone-specific descriptor sidecar.
In the runtime-introspection design every generated msg crate is
purely the wire-format data type (#[derive]d struct + a tiny impl nros_serdes::Message exposing const TYPE_NAME + const FIELDS).
On the first typed create_publisher<M> / create_subscription<M>
for a given M, the nros-rmw-cyclonedds shim walks the static
field schema, builds a Cyclone ddsi_sertype via Cyclone’s
dynamic-type C API, and caches the pointer in a bounded
heapless::FnvIndexMap<u64, NonNull<ddsi_sertype>, MAX_TYPES>
guarded by a platform-selected mutex. Subsequent uses of the same
M are hash-map hits.
End state: no <msg-pkg>/cyclonedds Cargo feature anywhere, no
per-msg-pkg backend code, no codegen branching on the active RMW.
The shape matches upstream rclcpp’s introspection typesupport + rclrs’s
plain <pkg> = "*" consumer manifest.
Sizing knob: NROS_CYCLONEDDS_MAX_TYPES (default 32), wired through
the existing nros-sizes build probe (same pattern as
EXECUTOR_OPAQUE_U64S). See section 212.K.7 of
docs/roadmap/phase-212-ux-cargo-native-and-file-consolidation.md
for the work-item ledger.
When to revisit
This matrix is a snapshot. Update it when any backend’s situation changes:
- A new backend lands → add a row + shim-crate + verdict.
- An existing backend’s underlying library changes language (e.g. zenoh-pico ships a Rust port upstream) → swap the shim shape (Rust adapter vs. C/C++ direct) but the vtable bridge stays.
The rule stays. Only per-backend verdicts and shim shapes move.
Registry + naming
nros-rmw-cffi holds a fixed-size named registry of backend
vtables. Each backend registers under a canonical name at
process startup:
| Backend | Name | Registered by |
|---|---|---|
| zenoh-pico | "zenoh" | nros_rmw_zenoh_register() (auto-ctor on POSIX) |
| micro-XRCE-DDS-Client | "xrce" | nros_rmw_xrce_register() (C ctor on POSIX) |
| Cyclone DDS | "cyclonedds" | nros_rmw_cyclonedds_register() (Phase 169.5 — canonical DDS backend; no generic "dds" alias) |
| uORB | "uorb" (future) | TBD |
Naming policy
- Lowercase ASCII identifying the protocol / wire format.
Not the transport variant —
"xrce"covers both XRCE-UDP and XRCE-serial; the transport is selected via the locator (udp/...vsserial:/dev/...). - Stable across releases. Renaming a registered name is a breaking change for bridge code that selects backends by string.
- No
"default"for new backends. The string"default"is reserved for the legacy single-argnros_rmw_cffi_registershim — single-backend builds where the backend’s specific name doesn’t matter.
Capacity
Registry size: NROS_RMW_MAX_BACKENDS build-time env var consumed
by nros-rmw-cffi/build.rs. Default 8. Range [1, 64]. Set lower
on Cortex-M0+ (where each slot’s ~40 B costs); set higher for
bridge nodes with 4+ backends. Hitting the cap = subsequent
nros_rmw_cffi_register_named returns NROS_RMW_RET_ERROR.
Default-backend convention
Executor::open and any create_node call without an explicit
.rmw(name) selector use the first-registered backend — the
nano-ros equivalent of ROS 2’s RMW_IMPLEMENTATION. Single-
backend binaries with one auto-registering backend Just Work
without user code mentioning the backend’s name.
The user-facing knob is a declared, language-agnostic, per-deploy
value (system.toml [system].rmw / [image.<id>].rmw, or a
CLI/build flag) that the toolchain lowers to each language’s
native link mechanism. The Cargo feature / shim dep and the CMake
cache var below are those lowering targets — what the build uses,
not how a user picks a backend (see
RFC-0031):
- Cargo (Rust): the declared RMW lowers to the
nrosrmw-<x>feature plus the matchingnros-rmw-<x>shim dep in the consumer’s[dependencies]. Linking the shim crate is what registers the backend. - CMake:
cmake -DNANO_ROS_RMW=zenoh ...(C/C++ users). Thenano_ros_link_rmw(... RMW zenoh)helper auto-generates the per-targetnros_app_register_backends()strong stub.
No runtime env-var override; selection is fixed at link time (RTOS-friendly, matches our static-link world).
Symbol-survival mechanism
Backend register symbols must survive linker dead-strip. Four mechanisms, layered:
linkmedistributed-slice (/ 128.H.2) — each backend contributes anRMW_INIT_ENTRIESentry through thenros_rmw_register_backend!macro.nros_support_init/Executor::openwalks the slice and calls each entry. Canonical on the hosted targets whose linker sectionslinkmerecognises — of which nano-ros builds only thenativehost board. Macro expands to a no-op on RTOS targets wherelinkmecan’t recognise the section (NuttX, Zephyr, ESP-IDF, FreeRTOS bare-metal).- Rust ctor (legacy fallback):
#[unsafe(link_section = ".init_array")] #[used] static AUTO_REGISTER_CTOR.#[used]keeps rustc from dead-stripping. - C ctor (legacy fallback):
__attribute__((constructor)) static void nros_rmw_<name>_register_ctor. Same survival via.init_arraywalk by libc startup. - CMake strong stub (landed): the
nano_ros_link_rmw(<target> RMW <name>)helper atcmake/NanoRosLink.cmake:62-117emits an auto-generated TU per target that defines a strongnros_app_register_backends()calling every linked RMW’snros_rmw_<name>_register(). The weak default inlibnros_c_weak_stubs.ais overridden. This is the canonical path on every RTOS wherelinkmecan’t survive. - Explicit user call (Rust no_std bridges):
nros_rmw_<name>::register()frommain()— drags the rlib’s CGU into the binary so the linkme entry is reachable. Seeexamples/bridges/native-rust-zenoh-to-dds/.
Bare-metal + RTOS targets that don’t run .init_array rely on
(4). Pure-Rust no_std binaries with multiple backends rely on (5).
POSIX builds get (1) + (2) + (3) for free.
Ctor ordering
POSIX .init_array runs ctors in link order, not in any
user-controlled sequence. When multiple backends auto-register
in one binary, the first to fire owns the default slot —
the one selected by Executor::open() / nros::init() with no
.rmw(name) argument. The order is reproducible per link
graph but not portable across linkers (lld vs. mold vs. gold)
or build configs (LTO can reorder via --print-icf-sections
collapse) and must not be relied on for correctness.
Disambiguation is the user’s job in multi-backend binaries. Use the named entry points:
- Rust:
Executor::open_with_rmw("zenoh", &cfg)for the primary session;node_builder("name").rmw("cyclonedds").build()for additional Nodes. - C:
nros_node_init_exwithnros_node_options_t.rmw_nameset. - C++:
nros::Executor::open_with_rmw(...)andnros::NodeBuilder::rmw(...)mirror the Rust API (Phase 104.C.9).
The examples/bridges/native-rust-zenoh-to-cyclonedds/ demo shows
the pattern end-to-end: both Zenoh and Cyclone DDS backend ctors fire
at lib-load (so the registry has both "zenoh" and "cyclonedds"
slots populated), and open_with_rmw("zenoh", ...) plus
node_builder("egress").rmw("cyclonedds") pin each session to its
intended backend without depending on link-order luck.
Single-backend builds keep the legacy ergonomics — only one ctor fires, the default-slot convention picks it up, and no user-visible name is ever required. The cost of naming is paid only when multiple backends coexist.
Real-time budget per backend
The poll loop’s worst-case execution time is dominated by the
backend’s transport drain. Bridge users summing
bridge_wcet = Σ poll_i + Σ dispatch_j need each backend’s
contribution; this table captures the current best-effort
estimates from per-backend microbenchmarks
(packages/testing/nros-bench/wcet-cycles-qemu/,
packages/testing/nros-bench/wake-latency-cortex-m3/) +
heap-usage stats from cargo build --release symbol
dumps.
| Backend | poll_wcet_us | Buffer-pool size | Notes |
|---|---|---|---|
zenoh-pico (nros-rmw-zenoh) | ~50–200 µs nominal on Cortex-M3 (FreeRTOS QEMU — emulated timing, host-load-dependent; no hardware measurement exists) | Z_BATCH_UNICAST_SIZE (default 6500 B/peer) + 4 KB per subscription buffered ring | Wake-cb collapses idle wait to kernel xSemaphore post — sub-poll-period latency when transport notifies. POSIX cv-wait path same shape, ~1 µs notify-to-dispatch. |
XRCE-DDS (nros-rmw-xrce-cffi) | ~100–500 µs per uxr_run_session_time on POSIX; agent-round-trip dominates over local poll. Bare-metal targets pay the same poll cost. | STREAM_HISTORY (4) × UXR_CONFIG_UDP_TRANSPORT_MTU (512 B default) ≈ 2 KB/stream; one input + one output stream per session | Poll-only — set_wake_callback slot is NULL; spin_once cv-wait still wakes on its deadline. Agent does the reliable-retransmit accounting; client adds ~10 µs per stream per tick. |
Cyclone DDS (nros-rmw-cyclonedds) | ~150–600 µs on POSIX; C++ listener callback latency depends on Cyclone’s reader-cache scan. | Cyclone’s RTPS history per the DDS QoS History.depth (default 10) + Cyclone’s own DDSI buffer pool (~32 KB default) | Listener-side set_wake_callback wiring is follow-up — today the C++ vtable sets the slot NULL. Memory footprint dominated by Cyclone itself, not the nano-ros shim. |
Bridge users: sum the poll_wcet_us for every backend the
bridge process opens, then add per-callback dispatch budget
(typically <10 µs for the executor’s arena dispatch + the
user callback’s own work). A bridge_picas_priority regression
test (blocked on the PiCAS dispatcher) will eventually pin a
bar to this table.
Per-backend README.md files live at
packages/{zpico/nros-rmw-zenoh,dds/nros-rmw-cyclonedds,xrce/nros-rmw-xrce-cffi}/README.md
(when present); reach out to the backend’s maintainer for
fresh microbench numbers on a different target class than
the ones above.
See also
- roadmap doc — Appendix D carries LOC sizing, port shapes, and risk notes.
- Custom Transport porting guide — how the transport vtable composes with the RMW vtable.
packages/rmw/cyclonedds/nros-rmw-cyclonedds/— reference layout for the C++ vtable consumer.packages/rmw/uorb/nros-rmw-uorb/— reference layout for the C++ vtable consumer with PX4 SDK integration.
ROS 2 rmw_zenoh Interoperability
This document describes how nros communicates with standard ROS 2 nodes using rmw_zenoh_cpp.
Status: WORKING
nros <-> ROS 2 rmw_zenoh communication is fully operational as of January 2025.
Architecture
┌─────────────────────────┐ ┌─────────────────────────┐
│ ROS 2 Node │ │ nros Node │
│ (rmw_zenoh_cpp) │<-------->│ (zenoh-pico) │
│ │ Zenoh │ │
│ ros2 topic echo │ Router │ native-rs-talker │
│ /chatter │ (zenohd) │ │
└─────────────────────────┘ └─────────────────────────┘
Both nodes connect to the same Zenoh router (zenohd). zenoh’s router-free peer mode is not
compiled into the nano-ros shim — NROS_SESSION_MODE=peer is refused at session open (issue 0682).
Quick Start
# Terminal 1: Start zenoh router
ZENOH_CONFIG_OVERRIDE='listen/endpoints=["tcp/127.0.0.1:7447"];scouting/multicast/enabled=false' ros2 run rmw_zenoh_cpp rmw_zenohd
# Terminal 2: Run nros talker
cargo run -p native-rs-talker --features zenoh -- --tcp 127.0.0.1:7447
# Terminal 3: Run ROS 2 listener
source /opt/ros/humble/setup.bash
export RMW_IMPLEMENTATION=rmw_zenoh_cpp
export ZENOH_CONFIG_OVERRIDE='mode="client";connect/endpoints=["tcp/127.0.0.1:7447"]'
ros2 topic echo /chatter std_msgs/msg/String --qos-reliability best_effort
Protocol Requirements
1. Data Topic Key Expression Format
For ROS 2 Humble, the data key expression format is:
<domain_id>/<topic_name>/<type_name>/<type_hash>
Important: For Humble, use TypeHashNotSupported as the type hash (no RIHS01_ prefix).
Example:
0/chatter/std_msgs::msg::dds_::String_/TypeHashNotSupported
Note: Newer ROS 2 versions (Iron+) use RIHS01_<hash> format with actual type hashes.
Implementation: TopicInfo::to_key() in the transport traits module
2. Liveliness Token Format
rmw_zenoh uses Zenoh liveliness tokens for discovery. Without these, ros2 topic list won’t show nros topics.
Node Liveliness Token:
@ros2_lv/<domain>/<zid>/0/0/NN/%/%/<node_name>
Publisher Liveliness Token:
@ros2_lv/<domain>/<zid>/0/11/MP/%/%/<node_name>/<topic>/<type>/RIHS01_<hash>/<qos>
Subscriber Liveliness Token:
@ros2_lv/<domain>/<zid>/0/11/MS/%/%/<node_name>/<topic>/<type>/RIHS01_<hash>/<qos>
Key differences from data keyexprs:
- Liveliness tokens do use
RIHS01_prefix - Topic names use
%instead of/(e.g.,%chatternot/chatter) - ZenohId must be in LSB-first hex format
Implementation: Ros2Liveliness in the transport/zenoh module
3. QoS String Format
The QoS portion of liveliness tokens uses this format:
reliability:durability:history,depth:deadline_sec,deadline_nsec:lifespan_sec,lifespan_nsec:liveliness,liveliness_sec,liveliness_nsec
Values:
- Reliability: 1=RELIABLE, 2=BEST_EFFORT
- Durability: 1=TRANSIENT_LOCAL, 2=VOLATILE
- History: 1=KEEP_LAST
For BEST_EFFORT/VOLATILE with KEEP_LAST depth 1:
2:2:1,1:,:,:,,
Important: Empty values default to RELIABLE which causes QoS mismatch. Always specify explicit values.
4. CDR Message Format
Messages use CDR (Common Data Representation) little-endian encoding:
[0x00, 0x01, 0x00, 0x00] // CDR encapsulation header (LE)
[... CDR payload ...] // Message data
Implementation: the serdes crate
5. RMW Attachment Format
rmw_zenoh requires metadata attached to each published message. For Humble, the serialization format is:
| Field | Size | Format |
|---|---|---|
| sequence_number | 8 bytes | int64, little-endian |
| timestamp | 8 bytes | int64, little-endian (nanoseconds) |
| gid_length | 1 byte | VLE encoded (value: 16) |
| gid | 16 bytes | Random bytes, constant per publisher |
Total: 33 bytes
The serialization uses Zenoh’s ze_serializer to ensure compatibility with zenoh-cpp’s deserializer.
Implementation: serialize_rmw_attachment() in zenoh-pico/src/serializer.rs
6. ZenohId Format
The ZenohId in liveliness tokens must be formatted in LSB-first (little-endian) hex format.
#![allow(unused)]
fn main() {
// Correct: LSB-first
fn to_hex_string(&self) -> String {
let mut hex = String::new();
for byte in self.bytes.iter() { // LSB first
write!(&mut hex, "{:02x}", byte).ok();
}
hex
}
}
Common Issues and Solutions
Issue: Discovery works but no messages received
Symptom: ros2 topic list shows the topic, ros2 topic info shows correct QoS, but ros2 topic echo receives nothing.
Cause: Data keyexpr format mismatch.
Solution: For Humble, don’t use RIHS01_ prefix in data keyexprs:
- Wrong:
0/chatter/std_msgs::msg::dds_::String_/RIHS01_TypeHashNotSupported - Correct:
0/chatter/std_msgs::msg::dds_::String_/TypeHashNotSupported
Issue: Topic not visible in ros2 topic list
Symptom: nros publishes but ROS 2 doesn’t see the topic.
Cause: Liveliness token format incorrect.
Solutions:
- Check ZenohId is LSB-first hex format
- Ensure topic name uses
%prefix (e.g.,%chatter) - Verify QoS string has explicit values, not defaults
Issue: QoS incompatibility warnings
Symptom: ROS 2 logs QoS incompatibility or subscriber doesn’t receive.
Cause: Publisher and subscriber QoS don’t match.
Solution: Use BEST_EFFORT for both:
ros2 topic echo /chatter std_msgs/msg/String --qos-reliability best_effort
Issue: rmw_zenoh not connecting to router
Symptom: rmw_zenoh uses peer mode by default, not connecting to zenohd.
Solution: Force client mode:
export ZENOH_CONFIG_OVERRIDE='mode="client";connect/endpoints=["tcp/127.0.0.1:7447"]'
Implementation Status
| Component | Status | Location |
|---|---|---|
| Data keyexpr format | Done | TopicInfo::to_key() |
| Liveliness tokens | Done | Ros2Liveliness |
| QoS format | Done | 2:2:1,1:,:,:,, |
| CDR serialization | Done | serdes crate |
| RMW attachment | Done | serialize_rmw_attachment() |
| ZenohId format | Done | ZenohId::to_hex_string() |
| Node integration | Done | ConnectedNode |
Version Compatibility
| ROS 2 Version | Type Hash Format | Status |
|---|---|---|
| Humble | TypeHashNotSupported | Working |
| Iron+ | RIHS01_<sha256> | Needs type hash computation |
References
FreeRTOS LAN9118 Debugging
This page collects the low-level FreeRTOS + lwIP + QEMU LAN9118 notes that are useful when the platform guide is not enough.
Stack
- Board: QEMU MPS2-AN385, Cortex-M3, 25 MHz.
- Ethernet: LAN9118 MMIO at
0x4020_0000, IRQ 13. - Software: FreeRTOS, lwIP threaded mode (
NO_SYS=0), zenoh-pico over BSD sockets.
Default task layout:
| Priority | Task | Role |
|---|---|---|
| 4 | tcpip_thread | lwIP protocol processing |
| 4 | poll task | drains LAN9118 RX FIFO into lwIP |
| 4 | zenoh read / lease | zenoh-pico background I/O |
| 3 | app task | nano-ros executor and user code |
| 0 | idle | must execute WFI for QEMU networking |
The poll task must run at least as high as the zenoh read task; if it cannot drain the RX FIFO, TCP keep-alives are missed and zenoh sessions expire.
LAN9118 Registers
Useful direct registers:
| Offset | Name | Purpose |
|---|---|---|
0x00 | RX_DATA_PORT | RX FIFO data |
0x20 | TX_DATA_PORT | TX FIFO data |
0x40 | RX_STAT_PORT | RX packet status |
0x50 | ID_REV | chip ID / revision |
0x54 | IRQ_CFG | interrupt output configuration |
0x58 | INT_STS | interrupt status |
0x6C | RX_CFG | RX configuration |
0x70 | TX_CFG | TX configuration |
0x7C | RX_FIFO_INF | RX FIFO status / bytes used |
0x80 | TX_FIFO_INF | TX FIFO free space |
0xA4 | MAC_CSR_CMD | indirect MAC CSR command |
0xA8 | MAC_CSR_DATA | indirect MAC CSR data |
Indirect MAC CSR registers include MAC_CR (TXEN/RXEN), ADDRH,
ADDRL, MII_ACC, and MII_DATA. Always wait for the CSR busy bit
to clear before starting another access.
QEMU Delivery Model
QEMU routes LAN9118 traffic through a legacy hub:
guest LAN9118 NIC <-> QEMU hub <-> backend (Slirp / TAP / socket)
When the guest transmits, QEMU delivers the frame synchronously through
the hub. Slirp or TAP replies, such as ARP responses, may arrive in the
RX FIFO before the guest TX register write returns. This is useful for
diagnostics: check RX_FIFO_INF immediately after TX, before yielding
to FreeRTOS.
QEMU drops frames that arrive before MAC_CR_RXEN is enabled, and its
LAN9118 model does not flush queued packets when RXEN changes. Enable
RX before traffic starts.
Common Pitfalls
Diagnostic shows zero RX packets. The poll task may have consumed
the packet before diagnostics read the register. Check RX_FIFO_INF
immediately after TX or temporarily prevent the poll task from running.
Network init hangs. lwIP threaded mode requires the FreeRTOS
scheduler to be running before tcpip_init(). Initialize networking
from a task, not before vTaskStartScheduler().
HardFault on first context switch. The Cortex-M3 FreeRTOS port
expects vPortSVCHandler, xPortPendSVHandler, and
xPortSysTickHandler at the vector table positions. Avoid wrapper
functions except for the guarded SysTick case.
lwIP core-lock assertion. Keep LWIP_TCPIP_CORE_LOCKING=0; the
setup path calls netif functions from the app task after tcpip_init().
Intermittent stalls. Always write IRQ_CFG_DEFAULT (0x22000111)
during LAN9118 init, even when polling.
Corrupted RX frames. RX_STAT_PORT packet length includes the
4-byte FCS. Read all aligned FIFO words, including CRC bytes, but pass
only pkt_len - 4 bytes to lwIP.
Second QEMU node cannot connect. Seed rand() uniquely per node.
zenoh-pico uses z_random_fill() for session IDs; identical QEMU boots
otherwise produce duplicate IDs.
lwIP ASSERT: Invalid mbox. The application stack is too small.
The executor arena lives on the FreeRTOS task stack; use 64 KB for
service and action examples.
Action client times out on result. Manual-polling action servers
created with create_action_server() must call
try_handle_get_result() explicitly after completing a goal.
No inbound frames in QEMU. The idle hook must execute WFI so QEMU’s main event loop can service network file descriptors:
void vApplicationIdleHook(void) {
__asm__ volatile("wfi");
}
Debugging Tips
- Use ARM semihosting for early
printfoutput. - Launch QEMU with
-monitor telnet:127.0.0.1:4444,server,nowaitand runinfo networkorinfo qtree. - Send a diagnostic ARP request and check for immediate RX FIFO data.
- Confirm
ID_REV,MAC_CRTXEN/RXEN,TX_FIFO_INF,RX_FIFO_INF, andINT_STSduring bring-up.
References
- FreeRTOS-Plus-TCP MPS2_AN385 driver.
- Zephyr
drivers/ethernet/eth_smsc911x.c. - ARM CMSIS LAN9220 Ethernet driver templates.
- QEMU
hw/net/lan9118.c. - nano-ros
packages/drivers/net/lan9118-smoltcp/.
Patched qemu-system-arm Binary
nano-ros ships a project-local QEMU build that the test harness uses
in preference to whatever qemu-system-arm is on $PATH. This
chapter explains why it exists, how it gets built, how a test picks
it up, and how to add a new patch.
Why
Two production-blocking issues motivated the patched build:
-
LAN9118 RX FIFO drain bug (mainline QEMU through at least 11.0). Under sustained burst RX, the LAN9118 emulator’s
lan9118_can_receive()can stop returning true even after the guest drains the FIFO, so frames silently disappear. bisected this and ships the fix asthird-party/qemu/patches/0001-hw-net-lan9118-add-can_receive-flush-on-FIFO-drain.patch. Bare-metal MPS2-AN385 RTPS / Zenoh tests on the system QEMU sporadically fail without it. -
-netdev dgram,local.type=unix,…requires QEMU 7.2+. Ubuntu jammy ships QEMU 6.2, and the dgram-tunnel pattern is how NuttX / ThreadX DDS multi-instance tests cross-deliver frames between two QEMU processes after retired the broken-netdev socket,mcast=cross-process path. With a too-old system QEMU, those tests fall back to[SKIPPED].
Both problems disappear once tests use the patched build at
build/qemu/bin/qemu-system-arm.
Layout
third-party/qemu/qemu/ # submodule, pinned to stable-11.0
third-party/qemu/patches/ # patch series, applied on top
0001-hw-net-lan9118-…patch
build/qemu/bin/qemu-system-arm # final installed binary
build/qemu/share/qemu/… # firmware, etc.
The submodule URL and pin live in .gitmodules:
[submodule "third-party/qemu/qemu"]
path = third-party/qemu/qemu
url = https://gitlab.com/qemu-project/qemu.git
branch = stable-11.0
stable-11.0 is QEMU 11.0.x — already past the 7.2 cutoff for
-netdev dgram unix and recent enough that the patch series is
small.
Build
just qemu setup-qemu (pulled in by just setup qemu and
just setup all) does the full
build:
just qemu setup-qemu
The recipe:
- Inits the submodule shallowly if it is not already present.
- Short-circuits when
build/qemu/bin/qemu-system-armis newer than every file underthird-party/qemu/patches/(touch a patch to force a rebuild). - Resets the submodule to its pinned tip, applies every
.patchunderthird-party/qemu/patches/in alphabetical order viagit apply. - Configures with
--target-list=arm-softmmu(no other arches, no docs, no tools) and--enable-slirp. make -j$(nproc) && make installintobuild/qemu/.
End-to-end cost is roughly ten minutes the first time and ~150 MB of disk. Subsequent runs are no-ops.
just qemu doctor reports the build status and clearly distinguishes
the patched build from the system fallback.
How tests pick it up
The single resolver lives in
packages/testing/nros-tests/src/qemu.rs:
#![allow(unused)]
fn main() {
pub fn qemu_system_arm_path() -> std::ffi::OsString { … }
pub fn qemu_system_arm_cmd() -> std::process::Command { … }
}
Selection order:
QEMU_SYSTEM_ARMenv var — developer override / CI pin.- Project-local
<workspace>/build/qemu/bin/qemu-system-armwhen it exists (auto-detected viaCARGO_MANIFEST_DIRwalk to the workspaceCargo.toml). - System
qemu-system-armon$PATH— kept as fallback so a minimal install still produces a clean[SKIPPED]rather than an exec error.
Every Command::new("qemu-system-arm") in the test crates goes
through this helper. New tests must use it.
just/qemu-baremetal.just, just/nuttx.just and just/freertos.just
do the same in shell, gating their qemu-system-arm invocations
through:
QEMU_BIN := absolute_path("build/qemu") / "bin/qemu-system-arm"
# inside a recipe:
{{ if path_exists(QEMU_BIN) == "true" { QEMU_BIN } else { "qemu-system-arm" } }} -M virt …
Smoke test
packages/testing/nros-tests/tests/qemu_patched_binary.rs asserts:
qemu_system_arm_path()resolves to eitherQEMU_SYSTEM_ARMor<workspace>/build/qemu/bin/qemu-system-arm.- The patched binary reports version ≥ 7.2.
-netdev helpadvertisesdgram(the multi-instance backend).
Tests use nros_tests::skip! when the patched build is absent —
a fresh clone without just qemu setup-qemu surfaces a clear
[SKIPPED] with the suggested remedy instead of silently passing.
Adding a new patch
- Land the upstream fix or write a downstream-only patch against the pinned submodule tip.
- Save it as a numbered file under
third-party/qemu/patches/(e.g.0002-hw-net-…patch). Keep one logical change per patch. - Bump any inline comment that names specific patches.
- Touch the patch file (or just commit it) —
just qemu setup-qemudetects the patch is newer than the installed binary and rebuilds. - Bump the relevant CI cache key (see) so other machines also rebuild.
Submodule pin bump
When upstream rolls a new stable branch and an existing patch either lands upstream or no longer applies cleanly:
- Edit
.gitmodulesbranch = stable-…to the new branch name. cd third-party/qemu/qemu && git fetch && git checkout origin/stable-…- Re-run
just qemu setup-qemu. If a patch fails to apply, either drop it (landed upstream) or rebase it onto the new tip. - Commit the submodule SHA bump together with any patch-series updates.
Scope
Only qemu-system-arm is unified. qemu-system-riscv32 is
Espressif’s fork (different upstream, different patches);
qemu-system-riscv64 and qemu-system-aarch64 ship no patches
today. Other arches stay on the system binary until they
accumulate their own patches; the helper pattern is easy to copy
when that happens.
Opaque Storage Sizing
nano-ros exposes a C and C++ API on top of Rust types whose layout is
chosen by the Rust compiler. The C/C++ side has to allocate storage for
those types — by value, on the user’s stack or BSS — without knowing
their exact byte size. The runtime makes Rust the single source of
truth for those sizes: the values flow from core::mem::size_of of
the real Rust types into auto-generated C/C++ headers, with no
hand-tuned constants.
The pattern
Every size that crosses the Rust / C boundary follows the same path:
-
Export from
nros::sizes— thenrosumbrella crate defines apub const FOO_SIZE: usize = core::mem::size_of::<T>();and emits a#[used] static __NROS_SIZE_FOO: [u8; FOO_SIZE] = [0u8; FOO_SIZE]with a#[no_mangle]symbol whose storage size in the rlib isFOO_SIZE. The two artefacts come from a singleexport_size!macro invocation:#![allow(unused)] fn main() { // packages/api/nros/src/sizes.rs export_size!(pub PUBLISHER_SIZE = RmwPublisher); export_size!(pub SUBSCRIBER_SIZE = RmwSubscriber); export_size!(pub EXECUTOR_SIZE = nros_node::Executor); // ...etc } -
Probe from consumer build scripts —
nros-c/build.rsandnros-cpp/build.rsuse the helper cratenros-sizes-buildto find the compilednrosrlib (viacargo metadata+ a glob overtarget/<triple>/<profile>/deps/) and read the__NROS_SIZE_*symbol storage sizes with theobjectcrate. No subprocess, no llvm-nm; pure Rust. -
Emit
#define NROS_FOO_SIZEinto a generated header —nros_config_generated.h(C) andnros_cpp_config_generated.h(C++) carry the probe values.types.hincludes the generated config transitively, so every nros C header seesNROS_*_SIZEautomatically. -
C/C++ structs use the macros —
typedef struct nros_publisher_t { /* ... */ _Alignas(8) uint8_t _opaque[NROS_PUBLISHER_SIZE]; } nros_publisher_t;class Publisher { alignas(8) uint8_t storage_[NROS_PUBLISHER_SIZE]; /* ... */ };The Rust side reads/writes the same bytes via
&mut *(opaque as *mut RmwPublisher). C and Rust agree on the size by construction.
What the SSoT covers today
The nros::sizes module exports:
| Symbol | Type | Used by |
|---|---|---|
SESSION_SIZE | RmwSession | nros_support_t._opaque |
PUBLISHER_SIZE | RmwPublisher | nros_publisher_t._opaque, nros::Publisher<M>::storage_ |
SUBSCRIBER_SIZE | RmwSubscriber | nros::Subscription<M>::storage_ |
SERVICE_CLIENT_SIZE | RmwServiceClient | nros::Client<S>::storage_ |
SERVICE_SERVER_SIZE | RmwServiceServer | nros::Service<S>::storage_ |
EXECUTOR_SIZE | nros_node::Executor | nros_executor_t._opaque, nros::Executor::storage_ |
GUARD_CONDITION_SIZE | nros_node::GuardCondition | nros_guard_condition_t._guard_opaque, nros::GuardCondition::storage_ |
LIFECYCLE_CTX_SIZE | nros_node::lifecycle::LifecyclePollingNodeCtx | nros_lifecycle_state_machine_t._opaque_storage |
ACTION_SERVER_INTERNAL_SIZE | ActionServerInternalLayout | nros_action_server_t._internal |
CPP_ACTION_SERVER_SIZE | CppActionServerLayout | nros::ActionServer<A>::storage_ |
CPP_ACTION_CLIENT_SIZE | CppActionClientLayout | nros::ActionClient<A>::storage_ |
Plus three *Internal C-API shim structs (ServiceServerInternal,
ServiceClientInternal, ActionClientInternal) that cbindgen now
emits directly into nros_generated.h because they’re #[repr(C)] —
the C side just embeds them as typed fields, no opaque storage at all.
Layout-mirror trick
Some downstream types — nros-c::ActionServerInternal,
nros-cpp::CppActionServer, nros-cpp::CppActionClient — embed C-API
pointer types (*mut nros_action_server_t, *const nros_goal_handle_t)
that aren’t visible from the nros umbrella crate. They can’t be
referenced from nros::sizes directly, but their byte size only
depends on the field shape: pointers are pointers, Option<extern "C" fn> collapses to a fn-pointer-sized slot via niche optimization, etc.
The fix is a layout-mirror struct in nros::sizes:
#![allow(unused)]
fn main() {
// packages/api/nros/src/sizes.rs
#[repr(C)]
#[doc(hidden)]
pub struct ActionServerInternalLayout {
pub handle: nros_node::ActionServerRawHandle,
pub executor_ptr: *mut c_void,
pub c_goal_callback: unsafe extern "C" fn(
*mut c_void, *const c_void, *const u8, usize, *mut c_void,
) -> i32,
// ...same field shape as the real `ActionServerInternal`,
// with C-API pointers replaced by `*mut c_void`...
}
export_size!(pub ACTION_SERVER_INTERNAL_SIZE = ActionServerInternalLayout);
}
Downstream then asserts byte-equivalence at compile time:
#![allow(unused)]
fn main() {
// packages/api/nros-c/src/opaque_sizes.rs
const _: () = assert!(
size_of::<crate::action::ActionServerInternal>()
== size_of::<nros::sizes::ActionServerInternalLayout>(),
"ActionServerInternal size diverges from nros::sizes::ActionServerInternalLayout — \
update the layout mirror in `nros/src/sizes.rs`"
);
}
The tripwire: any field-shape change in the real wrapper (adding a field, changing a pointer to a value, etc.) must be paired with an update to the mirror. The build fails immediately if they diverge.
How sizing works today
- All four
*Internalshim types are#[repr(C)]and embedded as typed fields in their outernros_*_tstructs. - All seven C++ wrappers (Publisher, Subscription, Service, Client,
ActionServer, ActionClient, GuardCondition) use the
NROS_*_SIZE/NROS_CPP_*_STORAGE_SIZEprobe macros. types.hships zero*_OPAQUE_U64Smacros; the four consumer module headers route through the probe.- The remaining hand-coded “upper bound” assertions (e.g., the
EXECUTOR_OPAQUE_U64Senvelope check innros-c/build.rs) are defence-in-depth: they fire if a Rust type accidentally exceeds a configured envelope (which would normally also be flagged by the config knobs).
Adding a new size export
When a new Rust handle type needs to cross the FFI boundary:
- Add the
export_size!(pub FOO_SIZE = nros_node::Foo)line tonros/src/sizes.rs. (IfFoolives in a downstream crate thatnroscan’t import, define aFooLayoutmirror struct first and add a byte-equivalence assert in the owning crate.) - Add a
let probe_foo = probed.get("FOO_SIZE").copied().unwrap_or(0)line tonros-c/build.rs(and/ornros-cpp/build.rs), and emit#define NROS_FOO_SIZE {probe_foo}into the generated config header. - Use
NROS_FOO_SIZEin the C/C++ struct that needs the storage.
Cross-target verification: the same probe runs for every target
triple. There’s no per-platform branching — size_of::<T>() resolves
to the right value at the target’s compile time.
See also
packages/tooling/nros-sizes-build/src/lib.rsfor the rlib probe implementation.packages/api/nros/src/sizes.rsfor the canonical exports.
Scheduling Models
This chapter introduces the real-time scheduling models used by the platforms nano-ros supports. Understanding these models helps you make informed decisions about task priorities, deadline guarantees, and platform selection for your application.
Background: What Is Real-Time Scheduling?
A real-time scheduler decides which task (or thread) runs on the CPU at any given moment. The key property is predictability — not raw speed. A system that always finishes a 10 ms task in exactly 10 ms is more “real-time” than one that finishes it in 1 ms but occasionally takes 50 ms.
Two key concepts recur across all models:
- Priority: a numeric value that determines which task runs when multiple tasks are ready. Higher-priority tasks preempt lower-priority ones.
- Preemption: the ability to interrupt a running task to run a higher-priority one. Non-preemptive (cooperative) systems only switch tasks at explicit yield points.
Hard vs. Soft Real-Time
| Type | Guarantee | Consequence of missed deadline |
|---|---|---|
| Hard | Deadline must never be missed | System failure (safety hazard) |
| Soft | Deadline should rarely be missed | Degraded quality (dropped frame, late message) |
nano-ros targets soft real-time by default (bounded message latency, bounded memory). Hard real-time is achievable on RTIC and with careful priority assignment on RTOS platforms.
Scheduling Algorithms
Fixed-Priority Preemptive (FPP)
The most common RTOS scheduling algorithm. Each task has a static priority assigned at creation time. The scheduler always runs the highest-priority ready task. When a higher-priority task becomes ready, it immediately preempts the running task.
Time ──────────────────────────────────────►
┌───────┐ ┌───────┐
High │ Task A│ │ Task A│ (preempts B when ready)
└───┬───┘ └───┬───┘
│ ┌───┐ ┌───┐ │ ┌───┐
Low └►│ B │ │ B │ └►│ B │ (runs when A is blocked)
└───┘ └───┘ └───┘
Schedulability analysis uses Rate-Monotonic Analysis (RMA): assign higher priority to tasks with shorter periods. RMA is optimal for independent periodic tasks — if any fixed-priority assignment can meet all deadlines, RMA can too.
The response time of a task under FPP is:
R_i = C_i + Σ ⌈R_i / T_j⌉ × C_j (for all higher-priority tasks j)
Where C_i is worst-case execution time, T_j is the period of
higher-priority task j. The task meets its deadline if R_i ≤ D_i.
Used by: FreeRTOS, ThreadX, NuttX (SCHED_FIFO), Zephyr (preemptive threads), RTIC.
Round-Robin (Time-Sliced)
Tasks at the same priority level share CPU time in equal time slices (quanta). When a task’s quantum expires, the scheduler switches to the next same-priority task. Tasks at different priority levels still follow FPP rules.
Time ──────────────────────────────────────►
┌──┐┌──┐┌──┐┌──┐┌──┐┌──┐
Pri 3 │A ││B ││A ││B ││A ││B │ (equal time slices)
└──┘└──┘└──┘└──┘└──┘└──┘
Round-robin prevents starvation among equal-priority tasks but adds
scheduling jitter (a task may wait up to (N-1) × quantum before running,
where N is the number of same-priority tasks).
Used by: NuttX (SCHED_RR), FreeRTOS (when configUSE_TIME_SLICING=1),
ThreadX (when time_slice > 0).
Cooperative (Non-Preemptive)
Tasks run until they explicitly yield the CPU. No preemption occurs. This eliminates the need for locks (no race conditions) but requires every task to yield frequently. A single task that runs too long blocks all others.
Time ──────────────────────────────────────►
┌──────────┐ ┌────┐
Task A │ runs │ │ │ (runs until yield())
└────┬─────┘ └──┬─┘
│ ┌────────┐ │ ┌──────┐
Task B └►│ runs │ └►│ runs │ (gets CPU only when A yields)
└────────┘ └──────┘
Used by: Zephyr (cooperative threads via K_PRIO_COOP), bare-metal
super-loops, Embassy async executor.
Interrupt-Driven (Hardware Scheduling)
The CPU’s interrupt controller acts as a hardware scheduler. Each interrupt source has a hardware priority level. When an interrupt fires, the hardware saves context and jumps to the handler — no software scheduler overhead. Nested interrupts provide preemption between priority levels.
On ARM Cortex-M, the Nested Vectored Interrupt Controller (NVIC) provides:
- Up to 256 priority levels (typically 8–16 usable)
- Zero-cycle context switch for tail-chaining interrupts
- Deterministic latency (12 cycles to handler entry on Cortex-M3)
NVIC
IRQ Priority ┌──────────────────┐
0 (highest) │ SysTick │──► timing / OS tick
1 │ UART RX │──► message receive
2 │ Timer │──► periodic publish
3 (lowest) │ PendSV │──► background work
└──────────────────┘
This is the most deterministic scheduling model — no jitter from software task switching, no priority inversion, and worst-case latency is bounded by the longest critical section that disables interrupts.
Used by: RTIC (exclusively), bare-metal interrupt handlers.
Earliest Deadline First (EDF)
A dynamic-priority algorithm: the task with the nearest deadline always runs next. EDF is optimal — it can schedule any task set that is schedulable by any algorithm, up to 100% CPU utilization (vs. ~69% for RMA with harmonic periods).
Time ──────────────────────────────────────►
┌───┐ ┌───┐
Task A │ d=5│ │d=15│ (deadline 5, then 15)
└───┘ └───┘
┌────┐ ┌────┐
Task B │d=10│ │d=20│ (deadline 10, then 20)
└────┘ └────┘
EDF is harder to analyze than FPP (no simple closed-form response time) and harder to implement (priority changes every scheduling decision). Overload behavior is also less predictable — under FPP, low-priority tasks miss deadlines first; under EDF, deadline misses can cascade unpredictably.
Used by: Zephyr (CONFIG_SCHED_DEADLINE + k_thread_deadline_set())
as the kernel-native capability, and the portable executor’s
SchedClass::Edf ready-set inside dispatch. Which platform gets which
is decided by the realizer — see the generated
Scheduling Wiring Matrix.
Platform Scheduling Comparison
RTIC (ARM Cortex-M)
RTIC is not an RTOS — it is a concurrency framework that compiles directly to hardware interrupt handlers. There is no scheduler, no task control blocks, and no context switch overhead.
| Property | Value |
|---|---|
| Model | Interrupt-driven (NVIC hardware scheduling) |
| Preemption | Yes — hardware interrupt nesting |
| Priority levels | 4–16 (depends on Cortex-M variant) |
| Priority direction | Lower number = higher priority (ARM convention) |
| Context switch | 12 cycles (Cortex-M3 tail-chain) |
| Mutual exclusion | Stack Resource Policy (SRP) — compile-time deadlock-free |
| Scheduling analysis | Fully static — all priorities and resources known at compile time |
RTIC’s key innovation is the Stack Resource Policy (SRP): shared resources are protected by ceiling-based priority elevation, not locks. The compiler proves at build time that no deadlock or unbounded priority inversion can occur. This gives interrupt-driven scheduling the safety of a cooperative model.
#![allow(unused)]
fn main() {
#[rtic::app(device = stm32f4xx_hal::pac)]
mod app {
#[task(priority = 2, shared = [sensor_data])]
async fn read_sensor(mut ctx: read_sensor::Context) {
ctx.shared.sensor_data.lock(|data| {
*data = read_adc(); // runs at ceiling priority
});
}
#[task(priority = 1, shared = [sensor_data])]
async fn publish(mut ctx: publish::Context) {
ctx.shared.sensor_data.lock(|data| {
node.publish(*data); // cannot deadlock — SRP guarantee
});
}
}
}
nano-ros integration: RTIC tasks call nano-ros directly — no executor
needed. Each RTIC task can own a publisher, subscription, or service handle.
See examples/qemu-arm-baremetal/rust/talker-rtic/.
FreeRTOS
The most widely deployed RTOS. Uses fixed-priority preemptive scheduling with optional round-robin time-slicing for same-priority tasks.
| Property | Value |
|---|---|
| Model | Fixed-priority preemptive (FPP) |
| Preemption | Yes — configUSE_PREEMPTION=1 (default) |
| Priority levels | configMAX_PRIORITIES (typically 8–32) |
| Priority direction | Higher number = higher priority |
| Context switch | ~80 cycles on Cortex-M (PendSV handler) |
| Mutual exclusion | Mutexes with optional priority inheritance |
| Time slicing | Optional — configUSE_TIME_SLICING |
FreeRTOS tasks are created with xTaskCreate(), specifying a priority and
stack size. The scheduler runs the highest-priority ready task. When
multiple tasks share a priority and time-slicing is enabled, they
round-robin at tick boundaries.
Priority inheritance: FreeRTOS mutexes optionally support priority inheritance to mitigate priority inversion. When a low-priority task holds a mutex needed by a high-priority task, the low-priority task temporarily inherits the high priority until it releases the mutex.
Without inheritance: With inheritance:
H ──blocks──► H ──blocks──►
M ──runs────► L ──promoted──► (runs at H's priority)
L ──holds mutex L ──releases──► H runs immediately
(M preempts L → (no M preemption → bounded inversion)
unbounded inversion)
nano-ros task layout on FreeRTOS:
| Task | Default Priority | Stack | Role |
|---|---|---|---|
| nros_app | 3 (Normal) | 64 KB | Executor, callbacks, spin |
| net_poll | 4 (AboveNormal) | 1 KB | Poll LAN9118 RX FIFO |
| zenoh read | 4 (AboveNormal) | 5 KB | Socket read, message decode |
| zenoh lease | 4 (AboveNormal) | 5 KB | Keep-alive, lease monitor |
| tcpip_thread | 4 (AboveNormal) | 4 KB | lwIP protocol processing |
ThreadX (Azure RTOS)
ThreadX is designed for deeply embedded systems with a unique preemption-threshold feature not found in other RTOSes.
| Property | Value |
|---|---|
| Model | Fixed-priority preemptive (FPP) with preemption-threshold |
| Preemption | Yes — with configurable threshold per thread |
| Priority levels | 32 (0–31) |
| Priority direction | Lower number = higher priority |
| Context switch | ~60 cycles (optimized assembly for each architecture) |
| Mutual exclusion | Mutexes with priority inheritance |
| Time slicing | Per-thread configurable (time_slice parameter) |
The preemption-threshold is ThreadX’s distinguishing feature. Each thread has two priority values:
- Priority: determines scheduling order (which thread runs next)
- Preemption-threshold: the minimum priority that can preempt this thread
Thread A: priority=10, preempt_threshold=5
→ Scheduled based on priority 10
→ Only threads with priority 0–4 can preempt it
→ Threads with priority 5–9 must wait, even though they're higher priority
Thread B: priority=10, preempt_threshold=10 (threshold = priority)
→ Normal behavior — any higher-priority thread can preempt
This effectively creates non-preemptive regions without disabling interrupts. A thread performing a critical sequence of operations can set a low preemption-threshold to prevent most preemption while still allowing the highest-priority threads through.
The academic basis is the dual-priority model: preemption-threshold reduces context switches (and thus stack usage) while preserving schedulability. Research shows it can reduce RAM requirements by 30–50% compared to pure FPP, because fewer threads need independent stacks for preemption frames.
nano-ros on ThreadX: multi-tier run_tiers is real (phase-297,
RFC-0053) — one executor per tier over one shared session, with
per-tier stacks allocated from the ThreadX byte pool via
nros_threadx_create_task. The tier’s preempt_threshold is passed
natively at tx_thread_create (a -1 sentinel means “use the
priority”, i.e. normal preemption). Tiers are resolved descending by
raw priority number, so on ThreadX (lower number = higher priority)
tiers[0] is the lowest-priority tier; the boot tier runs first and
re-prioritizes itself to its declared priority.
NuttX
NuttX is a POSIX-compliant RTOS, meaning it implements the full
pthread and sched APIs from IEEE 1003.1. This makes it the most
portable platform — standard POSIX real-time scheduling is well-understood
and widely taught.
| Property | Value |
|---|---|
| Model | POSIX SCHED_FIFO (FPP), SCHED_RR, or SCHED_SPORADIC |
| Preemption | Yes (FIFO/RR), configurable |
| Priority levels | 1–255 (POSIX sched_param.sched_priority) |
| Priority direction | Higher number = higher priority |
| Context switch | Kernel-managed, architecture-dependent |
| Mutual exclusion | POSIX mutexes with PTHREAD_PRIO_INHERIT or PTHREAD_PRIO_PROTECT |
| Scheduling policy | Per-thread via sched_setscheduler() |
NuttX supports three POSIX scheduling policies:
SCHED_FIFO (First-In-First-Out): Pure fixed-priority preemptive. A task runs until it blocks, yields, or is preempted by a higher-priority task. Tasks at the same priority run in FIFO order — no time-slicing.
SCHED_RR (Round-Robin): Same as SCHED_FIFO but with time-slicing for same-priority tasks. Each task gets a time quantum before the scheduler switches to the next task at that priority.
SCHED_SPORADIC (NuttX extension): Implements the sporadic server algorithm for aperiodic event handling. A task alternates between a high “normal” priority and a low “background” priority based on its execution budget:
Budget = 5ms, Replenish period = 20ms
Time ──────────────────────────────────────────────────►
┌─────┐ ┌─────┐
High │5ms │ │5ms │ (budget)
└──┬──┘ └──┬──┘
│ ┌──────────────┐ │
Low └►│ background │ └►... (budget exhausted)
└──────────────┘
▲ replenish after 20ms
The sporadic server bounds the interference that an aperiodic task imposes on periodic tasks, making it analyzable with standard RMA techniques. This is valuable for event-driven ROS callbacks that fire at irregular intervals.
Priority inversion protocols: NuttX supports both POSIX priority
inheritance (PTHREAD_PRIO_INHERIT) and priority ceiling
(PTHREAD_PRIO_PROTECT):
| Protocol | How it works | Trade-off |
|---|---|---|
| Priority inheritance | Holder inherits waiter’s priority | Dynamic — may chain through multiple locks |
| Priority ceiling | Mutex has fixed ceiling priority; holder runs at ceiling | Static — simpler analysis, avoids chained inversion |
nano-ros on NuttX: Currently runs with kernel defaults (no explicit
scheduling policy). Future work will add SCHED_FIFO and priority
configuration.
Zephyr
Zephyr provides the most flexible scheduling model, supporting cooperative threads, preemptive threads, and EDF in a single system.
| Property | Value |
|---|---|
| Model | Cooperative + preemptive + optional EDF |
| Preemption | Configurable per-thread |
| Priority levels | Cooperative: K_PRIO_COOP(0) to K_PRIO_COOP(N) (negative values); Preemptive: K_PRIO_PREEMPT(0) to K_PRIO_PREEMPT(N) (positive values) |
| Priority direction | Lower number = higher priority |
| Context switch | Architecture-dependent (~100 cycles on Cortex-M) |
| Mutual exclusion | k_mutex with priority inheritance |
| Meta-IRQ | Ultra-high-priority threads that preempt even cooperative threads |
Zephyr’s scheduling model has three tiers:
Priority Number Line:
◄── higher priority lower priority ──►
│ │ │
│ Meta-IRQ │ Cooperative │ Preemptive
│ (negative, │ (negative, │ (0 or positive)
│ special) │ non-preemptible) │
│ │ │
Cooperative threads (K_PRIO_COOP): Cannot be preempted by other
threads (only by interrupts). They run until they explicitly yield or
block. This is useful for critical sections that must not be interrupted
by other threads, while still allowing hardware interrupts.
Preemptive threads (K_PRIO_PREEMPT): Standard FPP behavior. Can
be preempted by any higher-priority thread (cooperative or preemptive).
Meta-IRQ threads (CONFIG_NUM_METAIRQ_PRIORITIES): Special
ultra-high-priority threads that can preempt even cooperative threads.
Used for work that must complete with interrupt-like urgency but needs
thread context (e.g., stack, blocking calls). This fills the gap between
ISR context (limited API) and thread context (preemptible).
Deadline scheduling: Zephyr optionally supports EDF via
CONFIG_SCHED_DEADLINE. Threads call k_thread_deadline_set() to
declare their next deadline. Among threads at the same priority level,
the scheduler picks the one with the earliest deadline. This allows EDF
within a priority band while preserving FPP across bands — a hybrid
approach that combines EDF’s optimality with FPP’s predictable overload
behavior.
nano-ros on Zephyr: Currently uses a single main thread with
default priority. The async service example uses Embassy’s executor
with kernel-backed waking (zephyr::embassy::Executor).
Priority Inversion
Priority inversion is a well-studied problem in real-time systems. It occurs when a high-priority task is indirectly blocked by a low-priority task through a shared resource, while a medium-priority task runs unimpeded.
The classic example (from the Mars Pathfinder incident, 1997):
High-priority task ──► blocks on mutex held by Low
Medium-priority task ──► preempts Low (doesn't need mutex)
Low-priority task ──► holds mutex, can't run (M preempts)
Result: High is blocked by Medium indefinitely
Solutions
| Solution | Approach | Platforms |
|---|---|---|
| Priority inheritance | Mutex holder inherits highest waiter’s priority | FreeRTOS, ThreadX, NuttX, Zephyr |
| Priority ceiling | Mutex has fixed ceiling; holder runs at ceiling | NuttX (PTHREAD_PRIO_PROTECT) |
| SRP (Stack Resource Policy) | Compile-time ceiling, zero runtime overhead | RTIC |
| Preemption-threshold | Limit which tasks can preempt | ThreadX |
| Lock-free design | Avoid shared resources entirely | nano-ros single-slot buffers |
nano-ros mitigates priority inversion architecturally: subscriptions use single-slot buffers with atomic overwrites — no mutex needed between publisher and subscriber tasks. The executor processes callbacks in a single task, eliminating inter-task resource sharing for most use cases.
Choosing a Scheduling Model
| Criterion | RTIC | FreeRTOS | ThreadX | NuttX | Zephyr |
|---|---|---|---|---|---|
| Determinism | Best (hardware) | Good (FPP) | Good (FPP+threshold) | Good (POSIX FPP) | Good (FPP+coop+EDF) |
| Worst-case latency | 12 cycles | ~80 cycles | ~60 cycles | Kernel-dependent | ~100 cycles |
| Priority inversion | Impossible (SRP) | Inheritance | Inheritance + threshold | Inheritance + ceiling | Inheritance |
| Analysis tools | Compile-time proofs | RMA/RTA | RMA/RTA + threshold | POSIX standard RMA | RMA + EDF analysis |
| Flexibility | Low (ARM only) | Medium | Medium-High | High (POSIX) | Highest |
| RAM overhead | Lowest (no TCBs) | Low | Low (threshold reduces stacks) | Medium (kernel) | Medium (kernel) |
Use RTIC when: you need hard real-time on ARM Cortex-M, want compile-time scheduling proofs, and can live without dynamic task creation.
Use FreeRTOS when: you need a widely supported RTOS with a small footprint and a large ecosystem. Good for projects where portability across MCU vendors matters more than advanced scheduling features.
Use ThreadX when: you need deterministic scheduling with reduced RAM (preemption-threshold), or your project targets Azure IoT infrastructure. ThreadX is also safety-certified (IEC 61508 SIL 4, ISO 26262 ASIL D).
Use NuttX when: you want POSIX compatibility (reuse Linux-targeted code on embedded), need SCHED_SPORADIC for aperiodic events, or want PTHREAD_PRIO_PROTECT for static priority ceiling analysis.
Use Zephyr when: you need maximum scheduling flexibility (cooperative + preemptive + EDF in one system), want a Linux Foundation backed project with broad hardware support, or need meta-IRQ for interrupt-like thread priorities.
Further Reading
- Liu, C.L. and Layland, J.W. (1973). “Scheduling Algorithms for Multiprogramming in a Hard-Real-Time Environment.” Journal of the ACM, 20(1), 46–61. — The foundational paper for Rate-Monotonic Analysis.
- Sha, L., Rajkumar, R., and Lehoczky, J.P. (1990). “Priority Inheritance Protocols: An Approach to Real-Time Synchronization.” IEEE Transactions on Computers, 39(9), 1175–1185. — Priority inheritance and priority ceiling protocols.
- Baker, T.P. (1991). “Stack-Based Scheduling of Realtime Processes.” Real-Time Systems, 3(1), 67–99. — The Stack Resource Policy used by RTIC.
- Wang, Y. and Saksena, M. (1999). “Scheduling Fixed-Priority Tasks with Preemption Threshold.” RTCSA. — The theoretical basis for ThreadX’s preemption-threshold.
- Buttazzo, G.C. (2011). Hard Real-Time Computing Systems: Predictable Scheduling Algorithms and Applications. Springer. — Comprehensive textbook covering FPP, EDF, and hybrid approaches.
Dispatch Strategy
Phase 216 design rationale. This chapter explains why
DispatchStrategyis shaped the way it is — the trichotomy, the per-Node granularity, the tag-based callback API, the__nros_node_<pkg>_dispatch_strategy()ABI symbol, and the backward-compat contract. For the user-facing tutorials see RTIC Integration and Embassy Integration.
The Inline / Deferred / FromIsr trichotomy
A nano-ros Node declares Node::DISPATCH: DispatchStrategy to tell the
codegen + lint layers how its callbacks need to be delivered:
#![allow(unused)]
fn main() {
// File: packages/platform/nros-platform/src/board/dispatch.rs
#[repr(u8)]
#[derive(Copy, Clone, Debug, PartialEq, Eq, Hash)]
pub enum DispatchStrategy {
Inline = 0,
Deferred = 1,
FromIsr = 2,
}
}
The variants:
-
Inline— Callbacks fire from the executor’s spin loop, in the same task that drives transport I/O. Default for every Node (preserves every pre-Phase-216 Node pkg unchanged). Served by every runtime: POSIX, FreeRTOS, NuttX, Zephyr, ThreadX, bare-metal, RTIC (proxied via__nros_dispatchtask when needed), Embassy (likewise — though Inline is unusual under cooperative async; see the matrix below). -
Deferred— Callbacks land in a board-side queue (heapless::spsc::Queueon RTIC,embassy_sync::channel::Channelon Embassy). A framework-owned dispatch task drains the queue and drivesExecutableNode::on_callbackfrom its own task context, decoupling callback latency from network polling. Required for callback-driven Nodes on RTIC + Embassy. -
FromIsr— Callbacks fire directly from an ISR handler. Design slot only — implementation deferred to Phase 216.E.1. Reserved so the lint matrix has a stable discriminant to reject against; current builds error out at thenros checklayer.
Framework × Strategy matrix
The matrix nros check (Phase 216.D.1) enforces:
┌──────────────────────────────────────────────────────┐
│ DispatchStrategy │
│ Inline │ Deferred │ FromIsr │
┌─────────────┼──────────────┼─────────────────┼─────────────────────┤
│ posix │ OK │ OK │ ERR: no ISRs │
│ rtos │ OK │ OK │ ERR: no ISRs │
│ rtic │ WARN: pref. │ OK (canonical) │ FUTURE (216.E.1) │
│ │ Deferred │ │ │
│ embassy │ WARN: pref. │ OK (canonical) │ ERR: no ISR exec │
│ │ Deferred │ │ │
└─────────────┴──────────────┴─────────────────┴─────────────────────┘
- POSIX + RTOS: both Inline and Deferred work. Inline is the default because the executor’s spin loop is the natural place to dispatch callbacks on hosted targets.
- RTIC + Embassy: Deferred is canonical. Inline is permitted (the
framework adapter still serves it) but
nros checkwarns: callback-driven Nodes that run inline tie their handler latency to spin task scheduling, which is rarely what you want under hardware-interrupt or async-cooperative scheduling. FromIsron POSIX/RTOS: rejected — there’s no meaningful “ISR context” for nano-ros to dispatch from on a hosted OS.FromIsron RTIC: future — needs the reentrancy audit + SPSC rework + per-Node#[isr_safe]contract called out in Phase 216.E.1.FromIsron Embassy: rejected — Embassy has no concept of “callback fires from an ISR handler”; ISR-driven work hands off to an async task viaembassy_sync::signal::Signalor similar.
Why per-Node, not per-callback
A Node that wants its /chatter subscription to run inline but its
/heartbeat subscription to run deferred is conceptually expressible —
we could put DispatchStrategy on each create_subscription call.
We don’t, for two reasons:
- The lint matrix collapses. Per-callback strategies multiply the
(framework, strategy)matrix by the number of callbacks per Node. The error surface grows quadratically and the messages get harder to phrase. Per-Node keeps each Node either fully Inline or fully Deferred — one strategy to reason about per pkg. - Implementation simplicity. The dispatch task drains a single
queue and routes each entry by
CallbackIdto a single Node’son_callback. Per-callback would mean tagging each entry with its own strategy at enqueue time and forking the dispatch path. Adds weight for a feature no real user has asked for yet.
If a real user demonstrates a Node that genuinely wants mixed strategies — typically because one subscription handler must hold a high-priority lock while another doesn’t — Phase 216.E.3 is the slot to reconsider. Until then: YAGNI.
Why FromIsr is a design slot, not an impl
The FromIsr discriminant exists in the enum today so that:
- The
[repr(u8)]discriminants are stable.Inline = 0,Deferred = 1,FromIsr = 2are wire-frozen — the__nros_node_<pkg>_dispatch_strategy()ABI symbol returns au8thatnros checkreads without linking the Node crate. AddingFromIsrlater (when 216.E.1 lands) would either renumber existing discriminants (breaking already-compiled Node binaries) or require a new ABI symbol. - The lint matrix has somewhere to point. Without the variant in the
enum, the
nros checkmatrix would have to special-case “user wroteFromIsrbut it doesn’t exist yet” via spelling-comparison rather than enum match. Worse error messages, worse evolvability.
The actual implementation needs three pieces that aren’t there yet:
- Reentrancy audit. Every step of the dispatch path —
signal_callback, queue push, RMW raw-CDR buffer ownership — must be re-entrant against ISR-priority producers. Today’s path assumes thread-context callers. - Lock-free SPSC variant.
heapless::spsc::Queueis single- producer / single-consumer at thread priority. ISR-priority producer + thread-priority consumer needs a stronger ordering contract (memory barrier on the ISR-side push at minimum, possibly a different queue type). - Per-Node
#[isr_safe]proof contract.on_callbackmust not call anything that can block, panic, or allocate. Statically proving that for arbitrary user code is a documentation + attribute exercise we haven’t undertaken.
The substrate Phase 214.J built (atomic_waker for cross-task
notification) is the building block for the SPSC variant; landing
214.J was a precondition for being able to even prototype
FromIsr. The full impl is deferred until a real ISR-driven
driver demands it.
Tag-based callback API rationale
The closure-based registration API:
#![allow(unused)]
fn main() {
// Pre-216.A.4 — still valid for Inline Nodes.
ctx.create_subscription("/chatter", |msg: Int32| {
handle(msg);
});
}
works fine when the callback runs synchronously on the spin task: the
closure captures state by value (or by &mut) and gets called inline
during spin_once. The captured environment lives on the spin
task’s stack frame, no heap, no boxing.
Deferred dispatch breaks that assumption. The callback now fires from
a different task than the one that called register, which means
the closure environment must either:
- Outlive both tasks (require
'staticcapture —move ||everywhere, but state still needs somewhere to live), or - Be stored generically in the runtime, which means erasing the
closure type to
Box<dyn FnMut(...)>(alloc-dependent) or toextern "C" fn(no captures at all).
Both options conflict with the no-alloc + framework-task-routed contract. So Phase 216.A.4 introduces tags:
#![allow(unused)]
fn main() {
// File: packages/api/nros/src/dispatch_tag.rs
pub struct SubscriptionTag(&'static str);
pub struct ServiceTag(&'static str);
pub struct ActionTag(&'static str);
impl From<SubscriptionTag> for CallbackId<'static> { /* ... */ }
impl PartialEq<Callback<'_>> for SubscriptionTag { /* ... */ }
}
The tag carries only the &'static str callback identifier — zero
runtime cost, no captures, FFI-safe. State lives on
ExecutableNode::State; the macro-emitted init() body wires the
tag fields by calling the _static registration variants
(create_subscription_static / create_service_static /
create_action_static) and storing the returned tag onto
Self::State.
Dispatch then matches the tag against the Callback<'_> event:
#![allow(unused)]
fn main() {
fn on_callback(state: &mut Self::State, cb: Callback<'_>, ctx: &mut CallbackCtx<'_>) {
if state.sub_chatter == cb {
let msg: Int32 = ctx.downcast().unwrap();
// ...
} else if state.sub_heartbeat == cb {
let beat: Heartbeat = ctx.downcast().unwrap();
// ...
}
}
}
The PartialEq<Callback<'_>> impl on each tag type does the
comparison in &'static str terms — O(ptr_eq) in the common case
when the runtime hands back the same &'static the user registered.
Inline keeps closures. The closure-vs-tag split is enforced by the
macro lint (216.A.6): a Deferred Node using create_subscription (the
closure form) fails to compile with a clear error pointing at the
registration call and suggesting the _static form. An Inline Node
using _static is allowed — but Deferred → closure is rejected.
Zero migration cost for pre-216 Node pkgs (all defaulted to Inline)
and a forced-correct API for the Deferred path.
The __nros_node_<pkg>_dispatch_strategy() ABI symbol
The nros::node!() macro emits, per Node pkg:
#![allow(unused)]
fn main() {
// Emitted by nros::node!(Talker) — Phase 216.A.5.
#[unsafe(no_mangle)]
extern "C" fn __nros_node_talker_pkg_dispatch_strategy() -> u8 {
<Talker as ::nros::Node>::DISPATCH as u8
}
}
Three consumers care about this symbol:
-
nros check(Phase 216.D.1). Statically inspects the Node crate’s.rmetaor links the staticlib + reads the symbol viadlsym/GetProcAddress(host-side check; embedded targets only read it from.rmeta). Compares against the Entry pkg’s boardframeworkmetadata using the matrix above; rejects mismatches atnros checktime, before the user runscargo build. -
The
nros::main!()proc-macro (Phase 216.B.3 / C.3). When expanding the Entry pkg’smain.rsit walks the registered Node list and reads each pkg’s strategy. If any Node is Deferred, the generated#[rtic::app]/#[embassy_executor::main]body includes the__nros_dispatchtask; if all are Inline, the dispatch task is omitted (zero overhead for Inline-only workspaces). -
Future runtime diagnostic tools. A
nros doctorornros topologystyle introspection tool can read the symbols from a linked binary and print the (pkg, strategy) table without having to re-parse the source. Useful for post-mortem on a binary you didn’t build yourself.
The extern "C" + [repr(u8)] ABI is the contract. It must not
break across nano-ros versions — adding a new strategy variant means
adding a new discriminant (FromIsr = 2 was reserved up front
exactly to avoid this), never renumbering an existing one.
The trait surface split (post-214.K.1)
Phase 214.K.1 renamed the board-side dispatch sink from NodeRuntime
to NodeDispatchRuntime (the user-facing sink kept the
NodeRuntime name, which is now in packages/api/nros/src/node.rs).
Phase 216 lands its new methods on NodeDispatchRuntime:
#![allow(unused)]
fn main() {
// File: packages/platform/nros-platform/src/board/runtime.rs
pub trait NodeDispatchRuntime {
// ... existing methods unchanged ...
fn signal_callback(&mut self, _cb_id: CallbackId<'_>, _ctx: &mut CallbackCtx<'_>) {
panic!("signal_callback not implemented for Inline runtime");
}
fn dispatch_strategy(&self) -> DispatchStrategy {
DispatchStrategy::Inline
}
}
}
Both methods are defaulted — zero-touch for the existing Inline
impls (ExecutorNodeRuntime in nros, NullNodeRuntime in
nros-platform). A Deferred runtime overrides both:
#![allow(unused)]
fn main() {
// File: packages/boards/nros-board-mps2-an385/src/rtic.rs (sketch)
impl NodeDispatchRuntime for RticRuntime {
fn dispatch_strategy(&self) -> DispatchStrategy {
DispatchStrategy::Deferred
}
fn signal_callback(&mut self, cb_id: CallbackId<'_>, ctx: &mut CallbackCtx<'_>) {
// SAFETY: SPSC producer is single-threaded by RTIC priority assignment.
self.queue_producer.enqueue((cb_id, ctx.snapshot())).unwrap();
// Wake the dispatch task; RTIC will schedule it at its declared priority.
__nros_dispatch::spawn().ok();
}
}
}
signal_callback’s default panic is the right behavior: the Inline
path never calls it (callbacks flow through the existing inline
trampoline). If a Deferred Node ends up on an Inline runtime — for
example because the user manually picked the wrong board — the panic
is louder than silently dropping the callback. The lint at
Phase 216.D.1 prevents this combination from compiling in the first
place; the panic is a belt-and-braces backstop.
Backward compatibility
Two contracts:
- Defaulted associated const.
Node::DISPATCHisconst DISPATCH: DispatchStrategy = DispatchStrategy::Inline;in the trait definition. Edition 2024 supports defaulted associated consts as stable, so every pre-216impl Node for ...block that doesn’t mentionDISPATCHcontinues to compile and is treated asInline. - Closure API preserved on the Inline path. The Inline runtime
keeps the closure-based registration path
(
create_subscription,create_service,create_action). The macro lint only rejects closure use whenDISPATCH = Deferred. Every Phase 212 Node pkg using closures stays valid without changes.
The migration shape for a Phase 212 Node that wants to move to Deferred is:
#![allow(unused)]
fn main() {
// Before — Phase 212 Inline-by-default.
impl Node for Listener {
const NAME: &'static str = "listener";
fn register(ctx: &mut NodeContext<'_>) -> NodeResult<()> {
ctx.create_subscription::<Int32>("/chatter", |msg| {
defmt::info!("Received: {}", msg.data);
})?;
Ok(())
}
}
// After — Phase 216 Deferred.
impl Node for Listener {
const NAME: &'static str = "listener";
const DISPATCH: DispatchStrategy = DispatchStrategy::Deferred;
fn register(ctx: &mut NodeContext<'_>) -> NodeResult<()> {
let _tag = ctx.create_subscription_static::<Int32>("/chatter")?;
Ok(())
}
}
impl ExecutableNode for Listener {
type State = ListenerState;
fn init() -> Self::State {
ListenerState { sub_chatter: SubscriptionTag::placeholder() }
}
fn on_callback(state: &mut Self::State, cb: Callback<'_>, ctx: &mut CallbackCtx<'_>) {
if state.sub_chatter == cb {
let msg: Int32 = ctx.downcast().unwrap();
defmt::info!("Received: {}", msg.data);
}
}
}
}
The migration adds three things: the DISPATCH const, the tag-typed
State field, and the on_callback body. The closure body in the
“before” version becomes the if state.sub_chatter == cb { ... }
branch in the “after” version — same code, hoisted to a method.
See also
- RTIC Integration — user-facing tutorial for the RTIC side of dispatch.
- Embassy Integration — user-facing tutorial for the Embassy side of dispatch + spawn-from-sync.
- Scheduling Models — the real-time scheduling backdrop against which dispatch strategy choices are made.
docs/roadmap/phase-216-baremetal-framework-integration.md— the locked spec.packages/platform/nros-platform/src/board/dispatch.rs— theDispatchStrategyenum.packages/api/nros/src/dispatch_tag.rs— the tag types.
Real-Time Analysis
This chapter describes static analysis methods and tools for detecting anti-patterns that violate real-time guarantees in Rust embedded applications.
Overview
Real-time systems require deterministic execution times. Common anti-patterns that break real-time guarantees include:
| Anti-Pattern | Problem | Detection Method |
|---|---|---|
| Unbounded loops | Infinite execution time | Clippy + custom lints |
| Recursion | Stack overflow, unbounded depth | cargo-call-stack |
| Heap allocation | Non-deterministic timing, fragmentation | no_std + forbid patterns |
| Blocking I/O | Unbounded wait times | Custom lints |
| Missing timeouts | Operations can hang forever | Custom lints |
| Large stack frames | Stack overflow | cargo-call-stack |
Built-in Clippy Lints
Loop and Iteration Lints
# Enable all loop-related lints
cargo clippy -- \
-D clippy::infinite_iter \
-D clippy::while_immutable_condition \
-D clippy::never_loop \
-D clippy::empty_loop
| Lint | Detects |
|---|---|
infinite_iter | Iterator chains guaranteed to be infinite |
while_immutable_condition | Loop conditions that can never change |
never_loop | Loops that exit on first iteration |
empty_loop | loop { } without body (use loop { hint::spin_loop() }) |
Memory and Performance Lints
cargo clippy -- \
-W clippy::large_stack_arrays \
-W clippy::large_types_passed_by_value \
-W clippy::box_collection \
-W clippy::rc_buffer
Recommended Clippy Configuration
Create clippy.toml in your project root:
# Maximum size for stack-allocated arrays (bytes)
array-size-threshold = 512
# Warn on types larger than this passed by value
trivial-copy-size-limit = 16
# Cognitive complexity threshold
cognitive-complexity-threshold = 15
Running Clippy for Real-Time Code
# Strict mode for real-time code
cargo clippy --all-targets -- \
-D warnings \
-D clippy::all \
-W clippy::pedantic \
-D clippy::infinite_iter \
-D clippy::while_immutable_condition \
-A clippy::module_name_repetitions
Stack Analysis with cargo-call-stack
Installation and Usage
# Install (requires nightly)
cargo +nightly install cargo-call-stack
# Build with stack size info
cd examples/qemu-arm-baremetal/rust/talker-rtic
RUSTFLAGS="-Z emit-stack-sizes" cargo +nightly build --release
# Generate call graph with stack sizes
cargo +nightly call-stack --release > call_graph.dot
# Visualize (requires graphviz)
dot -Tsvg call_graph.dot -o call_graph.svg
Interpreting Results
The output shows:
- Each function’s stack frame size
- Call graph relationships
- Maximum stack depth through any path
- Cycles (recursion) in the call graph
Example output:
digraph {
"main" [label="main\n256 bytes"]
"zenoh_poll" [label="zenoh_poll\n128 bytes"]
"publisher_task" [label="publisher_task\n512 bytes"]
"main" -> "zenoh_poll"
"main" -> "publisher_task"
}
Limitations
- Requires fat LTO (
lto = "fat"in Cargo.toml) - Limited support for programs linking
std - Indirect calls (function pointers, trait objects) may not be analyzed
- Best for embedded
no_stdprograms
Preventing Heap Allocation
Method 1: no_std Without Allocator
The simplest approach – don’t provide a global allocator:
#![allow(unused)]
#![no_std]
#![no_main]
fn main() {
// No #[global_allocator] defined
// Attempting to use Box, Vec, String will fail to compile
}
Method 2: Compile-Time Enforcement
#![allow(unused)]
#![no_std]
#![forbid(unsafe_code)] // Also prevents custom allocators
fn main() {
use heapless::{Vec, String}; // Static-sized alternatives
// This will NOT compile:
// let v = alloc::vec::Vec::new(); // Error: no allocator
// This works:
let v: heapless::Vec<u8, 256> = heapless::Vec::new();
}
Method 3: Custom Lint (Dylint)
For projects that need alloc but want to restrict usage in certain modules:
#![allow(unused)]
fn main() {
// In real-time critical code, add:
#![deny(clippy::disallowed_methods)]
}
With clippy.toml:
disallowed-methods = [
{ path = "alloc::vec::Vec::push", reason = "Use heapless::Vec in RT code" },
{ path = "alloc::boxed::Box::new", reason = "No heap in RT code" },
{ path = "alloc::string::String::new", reason = "Use heapless::String" },
]
Detecting Missing Timeouts
Pattern: I/O Operations Without Timeout
Anti-pattern:
#![allow(unused)]
fn main() {
// BAD: No timeout - can block forever
let data = socket.read(&mut buf)?;
}
Correct pattern:
#![allow(unused)]
fn main() {
// GOOD: Explicit timeout
socket.set_read_timeout(Some(Duration::from_millis(100)))?;
let data = socket.read(&mut buf)?;
}
Manual Audit Checklist
For code review, check that these operations have timeouts:
TcpStream::connect()– useconnect_timeout()socket.read()/socket.write()– set socket timeoutchannel.recv()– userecv_timeout()- zenoh operations – configure session timeout
- Any blocking syscall
Detecting Unbounded Loops
Clippy Detection
cargo clippy -- -D clippy::infinite_iter -D clippy::while_immutable_condition
Manual Patterns to Audit
Potentially unbounded:
#![allow(unused)]
fn main() {
// BAD: No termination guarantee
loop {
if let Some(msg) = queue.pop() {
process(msg);
}
}
// BAD: Condition may never be false
while !flag.load(Ordering::Relaxed) {
do_work();
}
}
Bounded alternatives:
#![allow(unused)]
fn main() {
// GOOD: Bounded iteration count
for _ in 0..MAX_ITERATIONS {
if let Some(msg) = queue.pop() {
process(msg);
} else {
break;
}
}
// GOOD: Timeout-based termination
let deadline = Instant::now() + Duration::from_millis(10);
while Instant::now() < deadline {
if let Some(msg) = queue.pop() {
process(msg);
} else {
break;
}
}
}
Detecting Recursion
cargo-call-stack Cycle Detection
# Will report cycles in call graph
cargo +nightly call-stack --release 2>&1 | grep -i "cycle"
Clippy Recursion Lint
# Warn on unconditional recursion
cargo clippy -- -D clippy::unconditional_recursion
Manual Pattern Detection
#![allow(unused)]
fn main() {
// BAD: Direct recursion
fn process(node: &Node) {
process(&node.child); // May overflow stack
}
// BAD: Mutual recursion
fn a() { b(); }
fn b() { a(); }
// GOOD: Iterative with explicit stack
fn process(root: &Node) {
let mut stack: heapless::Vec<&Node, 32> = heapless::Vec::new();
stack.push(root).ok();
while let Some(node) = stack.pop() {
// Process node
if stack.push(&node.child).is_err() {
// Handle stack full - bounded!
break;
}
}
}
}
Advanced: Custom Dylint Lints
Setting Up Dylint
# Install dylint
cargo install cargo-dylint dylint-link
# Create a new lint library
cargo dylint new realtime_lints
cd realtime_lints
Recommended Custom Lints for Real-Time
| Lint | Purpose |
|---|---|
blocking_in_async | Detect std::thread::sleep, blocking I/O in async |
unbounded_loop_in_task | Flag loops without bounds in RTIC tasks |
heap_in_interrupt | Detect allocation in #[interrupt] handlers |
mutex_in_interrupt | Flag std::sync::Mutex in interrupt context |
float_in_isr | Warn about FP operations in ISRs (soft-float targets) |
missing_timeout | I/O operations without timeout |
Tool Summary
| Tool | Detects | Effort | CI-Ready |
|---|---|---|---|
| Clippy | Infinite iters, recursion, large arrays | Low | Yes |
| cargo-call-stack | Stack usage, recursion cycles | Medium | Yes |
| Miri | Undefined behavior, memory errors | Low | Yes |
| Dylint | Custom patterns (blocking, heap, etc.) | High | Yes |
| MIRAI | Abstract interpretation, all paths | High | Experimental |
| no_std | Prevents std heap allocation | Low | Automatic |
| heapless | Static collections | Low | N/A |
Quick Reference: Lint Flags
# Copy-paste for strict real-time checking
cargo clippy -- \
-D warnings \
-D clippy::all \
-D clippy::infinite_iter \
-D clippy::while_immutable_condition \
-D clippy::never_loop \
-D clippy::empty_loop \
-D clippy::unconditional_recursion \
-W clippy::large_stack_arrays \
-W clippy::large_types_passed_by_value \
-W clippy::cognitive_complexity
WCET Benchmark Harness (no measurements yet)
A Worst-Case Execution Time (WCET) benchmark harness for nros core operations, built around DWT cycle counting on Cortex-M3. It has produced no numbers to date — every in-tree run is under QEMU, where DWT reads zero (see the limitation below), so the tables in this section list the benchmark categories, not results. Producing real baselines needs physical hardware (e.g. STM32F4 @ 168 MHz).
Measurement Platform
| Property | Value |
|---|---|
| Target | ARM Cortex-M3 (thumbv7m-none-eabi) |
| Machine | QEMU lm3s6965evb (for infrastructure validation) |
| Clock | DWT CYCCNT (cycle-accurate on real hardware) |
| Iterations | 100 per benchmark (10 warmup) |
| Optimization | release profile (opt-level = "s") |
DWT limitation on QEMU: QEMU’s Cortex-M3 emulation does not implement the DWT cycle counter – all reads return 0. The benchmark infrastructure is validated by the fact that it compiles, runs, and produces structured output. Actual cycle counts must be collected on real hardware (e.g., STM32F4 at 168 MHz, STM32F7 at 216 MHz).
Benchmark Categories
CDR Serialization:
| Function | Notes |
|---|---|
serialize Int32 | Single i32 field |
deserialize Int32 | Single i32 field |
serialize Time | Two fields (i32 + u32) |
roundtrip Int32 | Serialize + deserialize |
| serialize w/ header | CDR encapsulation header + Int32 |
Node API:
| Function | Notes |
|---|---|
Node::new() | StandaloneNode creation |
create_publisher() | Register Int32 publisher |
serialize_message() | Node-level serialize to buffer |
Safety E2E:
| Function | Notes |
|---|---|
crc32 (64B) | CRC-32/ISO-HDLC, 64-byte payload |
crc32 (256B) | CRC-32/ISO-HDLC, 256-byte payload |
crc32 (1024B) | CRC-32/ISO-HDLC, 1024-byte payload |
validate() | SafetyValidator sequence check |
| full pipeline (128B) | Extract attachment + CRC + validate |
Sanity Bound
The harness asserts a 100,000-cycle ceiling per function for typical payloads — a design assertion baked into the benchmark, not a measured bound (no measured WCET exists yet; see above). Extract the harness’s artifact with just qemu wcet-artifact:
- CRC-32 is O(n) in payload size with a single table lookup per byte
SafetyValidator::validate()is O(1) – a few comparisons and an increment- The full pipeline combines CRC computation with attachment parsing and validation
Running the Benchmark
# Build all QEMU examples (includes rs-wcet-bench)
just qemu build
# Run the WCET benchmark
just qemu test-wcet
Static WCET Analysis
For certification (ISO 26262 ASIL C/D, DO-178C DAL A/B), dynamic measurement is insufficient – static WCET analysis is required. Candidate tools:
| Tool | Type | License | Notes |
|---|---|---|---|
| Platin | Static (IPET) | Open source | Analyzes LLVM IR + machine code |
| aiT | Static (abstract interpretation) | Commercial (AbsInt) | Industry standard for DO-178C / ISO 26262 |
| SWEET | Static (flow analysis) | Academic | Research tool; good for Cortex-M |
| Chronos | Static (IPET) | Open source | Academic; limited embedded target support |
Recommended path: Use Platin for open-source WCET bounding on the LLVM IR produced by rustc. For certification evidence, aiT provides the strongest tool qualification story (pre-qualified per ISO 26262 Part 8).
References
- Clippy Lints Reference
- cargo-call-stack
- Dylint - Custom Lints
- MIRAI Abstract Interpreter
- heapless Crate
- Embedded Rust Book
- RTIC Book
Build System & Caching
Audience. Contributors working on the repo build/test matrix. End users never need this — they consume nano-ros with a single
add_subdirectory(<repo-root>), no install step, nofind_package. This handbook documents the other build world: the in-repojustorchestration that builds + tests every platform × RMW, and the caching that keeps it incremental and correct.Canonical sources this page summarises:
CLAUDE.md(“Build”, “Build tiers”, “Build parallelism”), Phase 176 (unified jobserver), Phase 181 (fixture SSOT
- Ninja), Phase 177.9 (staleness probes).
TL;DR cheat-sheet
just setup # one-time SDK/toolchain install (tiered; see CLAUDE.md)
direnv allow # once after clone — else zpico-sys/build.rs panics
just build # workspace + transports (fast inner loop)
just build-examples # + every example
just build-all # + test fixtures (everything test-all consumes)
just build-test-fixtures # just the fixtures + writes the test-all stamp
just build-test-fixtures lane=native # only what tier 1 (`just ci`) runs
just test-unit # ~5s ⊂
just test-integration # ~30s ⊂
just test # ⊂
just test-all # heavy QEMU/Zephyr/ROS-interop/miri/codegen
just <plat> build|build-all|test|ci # narrow to one platform first
NROS_BUILD_JOBS=8 just build-all # cap parallelism
Rule of thumb: a platform-specific failure → run the narrow
just <plat> build-all (or closest build/build-examples/build-fixtures)
before the root just build-all. Always just ci after a task. Never sudo
— if a step needs it, tell the user.
Generators — Ninja by default
The repo’s CMake builds use Ninja (-G Ninja), falling back to Make only
when ninja is absent. Ninja gives reliable incremental rebuilds and fits the
Phase 176 fifo jobserver; Make was dropped from the staleness path because
make -q mis-reported up-to-dateness.
- Configure-once:
scripts/build/cmake-incremental.sh::nros_cmake_configure_if_neededconfigures a build dir only when itsCMakeCache.txt/ generated build system is missing;cmake --buildthen handles reconfigure onCMakeListschanges. - Generator-mismatch wipe: a dir configured with the other generator is
rm -rf’d and reconfigured (you can’t switch generators in place). - Pinned tools: the unified jobserver needs
make ≥ 4.4+ninja ≥ 1.13(apt’s 4.3/1.10 lack the fifo jobserver). Both are SDK-store tools —nros setup --tool make(source tarball; upstream ships no binary) andnros setup --tool ninja(upstream prebuilt zip) — andscripts/sdk-path-tools.txtputs them onPATH(incl. agmake→ make-4.4 alias). Resolve an exact path withnros sdk-path <tool>.
Per-RMW build dirs = cache isolation
Each example builds into a per-RMW directory with its own cargo target dir:
examples/<plat>/<lang>/<example>/
build-zenoh/ cargo/ … # -DNROS_RMW=zenoh
build-xrce/ cargo/ … # -DNROS_RMW=xrce
build-cyclonedds/ cargo/ … # -DNROS_RMW=cyclonedds
Because each RMW (and its Corrosion cargo target dir) is physically separate,
selecting a different RMW cannot collide with another RMW’s cache — there is
no shared target dir to invalidate. Platform is fixed per build dir
(-DNANO_ROS_PLATFORM=<plat> at configure). This layout is why the old manual
cache-invalidation idea (archived Phase 145) was retired: the directory shape
makes it unnecessary. (Rust-only examples build via plain cargo into
target-<rmw>/; the per-RMW principle is the same.)
Parallelism — NROS_BUILD_JOBS + the unified jobserver
One knob scales every parallel recipe:
NROS_BUILD_JOBS=8 just build-all # default: nproc
Unified jobserver (Phase 176). just build-all auto-routes to
build-all-jobserver when the pinned make 4.4 + ninja 1.13 are present: a
single GNU-make fifo jobserver spans cargo + build-script cc + ninja-via-west
- cmake, allocating tokens dynamically instead of a static per-tool split. Under
the jobserver (
NROS_JOBSERVER=1) recipes drop their explicit-j/--parallel/CMAKE_BUILD_PARALLEL_LEVELso the tools inherit the pool.
- Same artifacts either way; without the pinned tools it falls back to a static split.
NROS_NO_JOBSERVER=1forces the static path.- Never re-introduce a hardcoded
parallel --jobs <n>without threading${NROS_BUILD_JOBS:-N}through.
See docs/roadmap/archived/phase-176-unified-jobserver-build-orchestration.md.
Build & test tiers
Each tier is a strict superset of the previous:
| Build | contains |
|---|---|
build | workspace + transports |
build-examples | ⊃ build + every example |
build-all | ⊃ build-examples + test fixtures |
| Test | wall-clock | contains |
|---|---|---|
test-unit | ~5 s | unit |
test-integration | ~30 s | ⊃ + integration |
test | — | ⊃ |
test-all | minutes | ⊃ + heavy QEMU/Zephyr/ROS-interop + doc + miri + C codegen |
Per-platform: just <plat> build ⊂ build-examples ⊂ build-fixtures ⊂ build-all
and just <plat> test|test-all|ci. <plat> = target families
(qemu, zephyr, board groups). Support services (zenohd, cyclonedds) are
not platform scopes. Orchestration lives in justfile + just/*.just.
Fixture SSOT — examples/fixtures.toml
Per-fixture build options (features, --target-dir, env, per-RMW variants,
cmake -D defs, cross target) live in one manifest,
examples/fixtures.toml,
consumed by both the build recipes and the test-all staleness probe — so the
build and the probe use identical options (no feature-thrash).
- Reader:
scripts/build/fixtures-manifest.py list --platform <p> --lang <l> [--rmw <r>] [--for-probe]. Emits\x1f-separated records (unit-separator, not tab — tab is IFS-whitespace). - Builder:
scripts/build/fixtures-build.sh <plat> [lang] [rmw]— one shared loop; rust cells go throughcargo build, C/C++ cells throughcmake --build(configure-once + per-RMW dir). Platform-wide-Ddefs (toolchain, codegen tool, SDK dirs) are injected by the recipe viaNROS_CMAKE_EXTRA_DEFS. - Cross-toolchain platforms inject
RUSTUP_TOOLCHAIN/ SDK env via the recipe (e.g. ESP32-C3 = riscv32imc on the workspace nightly + build-std).
Staleness discipline
A prebuilt fixture that’s silently stale would let test-all run against the
wrong binary. The discipline:
-
Coverage gate.
build-test-fixturesstampstarget/nextest/.fixtures-builtwith the lane and the fixture coordinates it built, not just a timestamp;test-all’s_require-fixturesfast-fails (~1 s) if the stamp is absent or does not cover the run implied byNROS_FIXTURE_LANE(bypass:NROS_SKIP_FIXTURE_CHECK=1). So alane=nativebuild satisfiesjust ci(tier 1) and is correctly rejected by an unscopedtest-all. A pre-issue-0393 timestamp-only stamp is read aslane=all, which is what it meant.Note “the run”, not “the lane” (issue 0482). A lane’s coordinates are what it keeps fresh; what must exist is decided by which tests the recipe executes. The two are answered separately, and
nros_lane_build_laneis the mapping between them (CiLane::run_scopeis the declaration;lane-coords <lane> --build-laneprints it).Each lane narrows its run in the way its cost actually lives:
lane run narrowed by build lane tier 1 test/binary NAME ( NROS_TEST_SCOPE=native)nativetier 2, nightly fixture COORDINATE ( NROS_TEST_COORDS)itself Tier 2 needs the second kind because it is 1-wise over platform: every platform is in the lane, so a platform-token filter excludes nothing, and the saving is in lang × rmw within a platform, which test names do not encode (issues 0357, 0482). So the narrowing happens in the fixture resolver (phase-340 W3,
nros_tests::fixtures::lane): a resolved artifact is attributed back to itsexamples/fixtures.tomlrow throughrow_artifact_root(), and a row outside the lane reports SKIPPED instead of missing. Build and run then apply the samerow_coord()predicate to the same coordinate file, sojust build-test-fixtures lane=tier2 && just ci-matrixis a coherent pair. Before W3 tier 2 ran the whole suite, and alane=tier2build satisfied the preflight for a run that then failed STALE on 34 unbuilt coordinates.The skip is keyed on the coordinate, never on the artifact being absent: an in-lane fixture that is missing or stale still fails hard, and a path the manifest cannot attribute (Zephyr west leaves, the compile-check lane — both built module-level, so nothing is omitted) is never skipped.
The preflight enforces the pairing rather than trusting it: accepting a build narrower than
allrequiresNROS_TEST_COORDSto be set and to hold exactly that lane’s coordinates, so a hand-runNROS_FIXTURE_LANE=tier2 just test-allis refused instead of reproducing issue 0482. A full build needs no narrowing, soNROS_FIXTURE_LANE=tier2on top oflane=allstill works — scope the freshness gate, keep the run wide. -
Rust cells.
scripts/test/rust-fixture-stale.shreuses cargo’s own fingerprint —cargo build … --message-format=jsonreports"fresh":falsefor a stale unit, so the probe both detects and self-heals. -
C/C++ cells.
scripts/test/cmake-fixture-stale.shruns the incrementalcmake --build(near-no-op when fresh) and flags cells that actually rebuilt. (ninja -nis unusable here — Corrosion’s cargo step is an always-run custom command, so it always reports pending.) -
Probe opt-out. Cells needing a recipe-injected toolchain the probe can’t supply (e.g. the ESP32 nightly + build-std) set
skip_probe = true; the reader’s--for-probeomits them so they don’t toolchain-thrash. -
Source-list / drift gates.
zpico-sys(vendored zenoh-pico, Phase 136.6) andnros-rmw-xrce-cffi(vendored uxr/micro-cdr, Phase 145.4) verify each vendored source root resolves to a real dir with.cfiles and panic with agit submodule update --inithint on drift. cbindgen build scripts (nros-c,nros-cpp,zpico-sys) emitcargo:rerun-if-changed=cbindgen.tomlsrc/.
-
Probe runs in preflight.
_check-fixtures-staleruns both probes (rust + cmake) over the manifest before thetest-allnextest stage; it warns + self-heals rather than hard-failing.
Patched QEMU
QEMU networked tests use a patched qemu-system-arm (icount + MPS2 fixes) built
by just qemu setup-qemu into build/qemu/bin/. The harness picks it up
automatically (nros_tests::qemu::qemu_system_arm_cmd()); the system binary is
the fallback. See Patched qemu-system-arm.
See also
- Build Commands — the user-facing quick reference.
- zpico-sys Build Architecture — the zenoh-pico cross-compile.
CLAUDE.md— the authoritative build/test-tier + parallelism policy.- Phase docs (
docs/roadmap/archived/): 176 (jobserver), 181 (fixture SSOT + Ninja), 177.9 (staleness probes).
CLI lives in the monorepo (Phase 218)
The nros CLI lives in-tree at packages/cli/ as a sub-workspace
— own Cargo.toml, own Cargo.lock, own member crates (nros-cli,
nros-cli-core, nros-build, cargo-nano-ros, rosidl-*,
colcon-cargo-ros2, nros-msg-to-idl). It is built per checkout via
just setup-cli → packages/cli/target/release/nros, then put on
PATH by the activate file. See also:
Why sub-workspace, not root member
The runtime workspace’s no_std feature-unification view (Phase
214.F.3) is the project’s most fragile invariant — a single workspace
member adding a std-activating dep without target-gating turns the
embedded build red on a path the test matrix doesn’t always exercise.
The CLI’s deps (clap, askama, syn, ureq, …) are aggressively
std-only; making them workspace siblings of the runtime crates would
either (a) require target-gating every single one or (b) effectively
disable the F.3 guard.
The sub-workspace shape keeps the two surfaces categorically separate.
The pattern mirrors packages/testing/nros-{tests,bench,smoke}/,
which carved out the same way for the same reason.
Why per-checkout, not ~/.nros/bin
Confirmed during the Phase 218 UX walkthrough: contributors with
multiple nano-ros worktrees (phase branches, ASI integration trees,
downstream forks) need each tree to point at its own CLI. A global
install would silently version-skew across trees the moment the user
cds — and because the CLI’s codegen format is structurally bound to
the nros-core / nros-c / nros-cpp crates in the same checkout, a
global CLI is a known footgun.
The per-checkout shape makes the activate file the single switch — no
which nros surprise across cwd boundaries. One checkout = one CLI
version = one runtime ABI, by construction; no version-pin matrix to
police.
Resolution chain
scripts/build/cargo.sh::nros_cli_bin resolves the active binary:
$NROS_CLI(explicit override)nroson$PATH(activate file puts the repo-local one here)packages/cli/target/release/nros(per-checkout)${NROS_HOME:-~/.nros}/bin/nros(transitional, removed once every active branch lands on 218)- error
The Phase 218.E ABI guard checks the in-tree CLI build matches the
runtime crates’ codegen ABI hash; opt out via NROS_SKIP_VERSION_CHECK=1
when intentionally bisecting a CLI mismatch.
See also
zpico-sys Build Architecture
zpico-sys is nano-ros’s Rust *-sys crate for zenoh-pico. After (2026-05-18) the build path is a single cc-rs invocation
driven entirely by a TOML manifest. Earlier phases shipped a
CMake/cc-rs hybrid with per-RTOS Rust functions; the unified path
collapses that surface to one consumer + one data file.
The manifest
packages/rmw/zenoh/zpico-sys/zenoh_platforms.toml declares every
per-platform datum the build needs. Two top-level table groups:
[platform.<name>]— one block per supported platform (posix,zephyr,freertos-lwip,nuttx,threadx,bare-metal,generic,orin-spe).[arch.<name>]— reusable target-arch compiler-flag profiles shared across platforms that target the same CPU family (cortex-m3,cortex-m4f,cortex-a7,cortex-r5-softfp,riscv32imc,riscv32gc,riscv64gc).
Per-platform fields
| Field | Type | Use |
|---|---|---|
inherits | string | Merge from another [platform.*] block before applying this one. |
defines | list[str] | Unconditional cc::Build::define(name, None). |
defines_kv | map | cc::Build::define(name, Some(value)). |
defines_env | map | Value from env, falls back to default. |
include | list[str] | Glob roots under zenoh-pico/src/. The drift gate validates these exist. |
exclude | list[str] | Drop entries from include matches. |
extra_sources | list[ExtraSource] | Additional .c files outside zenoh-pico/src/. See below. |
required_env | list[RequiredEnv] | SDK paths the build needs. See below. |
include_paths | list[str] | Header search paths; interpolated. |
include_paths_conditional | list[ConditionalPath] | Header paths gated by when. |
arch | string | list[str] | Name(s) of [arch.*] block(s) to apply. Single name (TOML scalar) for single-arch platforms; list (TOML array) for multi-arch platforms like bare-metal (cortex-m3 + riscv32imc). build.rs walks the list in order and applies the first arch whose target_match hits the build target.. |
compile | table | opt_level / warnings / cflags. |
pic | bool | cc::Build::pic override (NuttX flat builds use false). |
link.* | map | Per-link-feature policy. Values: true / false (force on/off) or "feature" (defer to CARGO_FEATURE_LINK_<X>). |
mbedtls | string | pkg-config / vendored / none. |
system_libs | list[str] | cargo:rustc-link-lib= entries. |
rerun_if_env_changed | list[str] | cargo:rerun-if-env-changed=… triggers. |
Per-arch fields
| Field | Type | Use |
|---|---|---|
target_match | string | Substring or <prefix>* glob the target triple must match. |
target_exclude | string | Veto when the target triple contains this substring (e.g. cortex-m3 excludes thumbv7em so Cortex-M4 doesn’t pick it). |
cflags | list[str] | Compiler flags. |
needs_picolibc | bool | Add picolibc sysroot’s include/ to the search path. |
needs_errno_override | bool | Generate + prepend the errno-override shadow header (RISC-V picolibc TLS-errno workaround). |
needs_riscv_compiler | bool | Probe for a RISC-V cross-cc. |
Interpolation tokens
Every path / include_paths / extra_sources.path /
include_paths_conditional.path string is run through a small
interpolator before being passed to cc-rs:
| Token | Value |
|---|---|
{nros} | CARGO_MANIFEST_DIR (zpico-sys/). |
{out} | OUT_DIR. |
{src} | zenoh-pico/src/ under {nros}. |
{env:VAR} | Value of env var VAR; build fails with a clear error if unset. |
when matcher
Conditional includes carry a when table. Each field is optional;
present fields ALL must hold for the matcher to fire.
| Field | Meaning |
|---|---|
target_match | Substring (or <prefix>* glob) that must be in the target triple. |
target_not | Substring that must NOT be in the target triple. Special value "embedded" matches any of the known embedded RTOSes. |
if_env | Env var that must be set (any value). |
ExtraSource shape
extra_sources = [
{ path = "{nros}/c/platform/threadx/task.c" },
{ path = "{nros}/c/platform/threadx/log_uart.c",
if_env = "NROS_ZPICO_LOG_TO_UART",
with_define = ["ZENOH_LOG_PRINT", "zpico_log_print"] },
]
if_env skips the file when the env var is absent. with_define
adds the matching cc::Build::define(name, Some(value)) whenever
the source is included (use name alone for unconditional None).
RequiredEnv shape
required_env = [
{ name = "THREADX_DIR",
help = "ThreadX kernel source. Run just setup threadx_linux or export THREADX_DIR=$PWD/third-party/threadx/kernel",
validate_subdir = "common/inc" },
]
Missing env vars panic at build time with the help string;
present env vars whose value lacks the validate_subdir subdir
also panic with the offending path printed. No silent fallthroughs.
Adding a new platform
Adding a new RTOS is a TOML edit:
- Pick an
[arch.<name>]block for the target CPU. If the family isn’t already there (or compiler flags differ), add an[arch.*]block. Reusable across platforms. - Write the
[platform.<your_rtos>]block withdefines+include+extra_sources+required_env+include_paths. - The Cargo feature for the new platform must be added in
Cargo.tomland wired into the existing match inbuild.rsthat maps Cargo feature → manifest platform name. - Verify with
cargo build -p zpico-sys --features <your_rtos> --target <triple>.
No build.rs edits needed for the data layer.
Adding a new target arch
Adding a new CPU family is one [arch.*] block + any of the
needs_picolibc / needs_errno_override /
needs_riscv_compiler flags it requires. Existing platforms that
need to target it set arch = "<your_arch>" (or extend their
existing arch = [...] list with the new name).
For platforms that span multiple architectures (e.g. bare-metal
covers cortex-m3 for qemu-arm-baremetal / stm32f4 AND
riscv32imc for ESP32-C3), declare every arch the platform
supports and let build.rs’s first-match dispatch pick the right
one per target triple:
[platform.bare-metal]
arch = ["cortex-m3", "riscv32imc"] # first arch_matches wins
This is what makes cargo check work on both
qemu-arm-baremetal/rust/zenoh/talker and
esp32/rust/zenoh/{listener,talker} from the same platform
entry — the picolibc sysroot wired up by
arch.riscv32imc.needs_picolibc = true is added to the cc-rs
-I list only when the build target is riscv32imc-*.
mbedTLS source policy
Manifest’s mbedtls field per-platform:
pkg-config— discover via thepkg-configbuild crate. POSIX hosts only. The build script also synthesizes.pcfiles when the host doesn’t ship them (Ubuntu’slibmbedtls-devis the motivating case).vendored— pull from the in-treembedtls/submodule sources. Bare-metal / RTOS targets without a system mbedTLS.none—link-tlsusers on this platform get a link error; the platform doesn’t support TLS.
The selection only matters when link-tls is on (controlled by
the CARGO_FEATURE_LINK_TLS env var).
Source-drift gate
Every include root in zenoh_platforms.toml must (a) resolve to
a real directory under zenoh-pico/src/, (b) contain ≥1 .c file
or sub-directory. A failed check panics build-time with the
offending key + expected path. owns this; full
set-equality vs. the resolved cc-rs source list lands as a
follow-up.
Manifest-driven consumer
build.rs’s build_zenoh_pico_unified consumes a
ResolvedPlatform (the merged inherits chain). It:
- Validates
required_env+validate_subdir. - Generates the version header in
{out}/zenoh-pico-version/. - Applies the
[arch.*]profile if itstarget_match/target_excludepredicates pass. - Adds core sources (8 protocol subdirs +
system/common) and per-platformextra_sources(withif_env/with_define). - Sets include paths (unconditional + conditional after
matches). - Adds defines (
defines/defines_kv/defines_env). - Handles mbedTLS per the manifest’s
mbedtlsfield. - Applies shim slot counts.
- Applies compile settings +
picoverride. - Compiles to
libzenohpico.a. - Registers
rerun_if_env_changed.
The consumer is the only function that actually invokes cc-rs
for zenoh-pico. Every per-platform delta lives in the TOML.
Related
docs/roadmap/phase-136-zpico-sys-unified-build.md— the phase doc that drove this refactor.book/src/concepts/platform-model.md— Boards vs Platforms; manifest is the platform-side knob.book/src/internals/rmw-backends.md— RMW host-language policy.
Formal Verification
nano-ros uses three complementary verification tools to ensure correctness of its core libraries:
- Kani – bounded model checking with 148 harnesses across 6 crates. Checks memory safety, integer overflow, and panic-freedom within bounded inputs.
- Verus – deductive (unbounded) proofs with 83 verified properties. Proves functional correctness of scheduling, serialization, time arithmetic, and safety protocols.
- Miri – runtime undefined behavior detection for unsafe code.
All three run via just recipes:
just verify # Run both Kani + Verus
just verify-kani # Kani only (~3 min)
just verify-verus # Verus only (~1 sec)
just test-miri # Miri UB detection
Kani
Kani is a bounded model checker that translates Rust code to a verification IR and exhaustively explores all possible executions up to a given bound.
just verify-kani
148 harnesses are spread across 6 crates (counts as of the last audit — regenerate with grep -rc '#[kani::proof' packages/):
| Crate | Focus |
|---|---|
nros-serdes | CDR serialization round-trip, buffer bounds |
nros-core | Time arithmetic, type invariants |
nros-params | Parameter value constraints, range checking |
nros-c | FFI boundary safety |
nros-ghost-types | Buffer state machine transitions (overflow/lock) |
nros-node | Executor scheduling, subscription handling |
Kani checks three properties by default:
- Memory safety – no out-of-bounds access, no use-after-free
- Overflow freedom – no integer overflow on arithmetic operations
- Panic freedom – no reachable
panic!,unwrap(), orassert!failure
A typical run takes approximately 3 minutes on a modern machine.
Miri
Miri is an interpreter for Rust’s Mid-level IR that detects undefined behavior at runtime:
just test-miri
Miri catches issues that neither the compiler nor Kani can detect:
- Invalid pointer dereferences
- Uninitialized memory reads
- Violations of aliasing rules (Stacked Borrows / Tree Borrows)
- Data races in concurrent code
If Miri fails with “contains outdated or invalid JSON”, clean the cache:
rm -rf target/miri
Verus
Verus is a deductive verification tool for Rust. Unlike Kani (which checks bounded executions), Verus proves properties hold for all possible inputs using SMT solving.
Quick Reference
just verification verus # Provision the pinned Verus release into the SDK store (or `just verification setup` for kani + verus)
just verify-verus # Run all Verus proofs
just verify # Run both Kani + Verus
Verification crate: packages/verification/nros-verification/
Type Specifications
Every type used inside verus! { } that is defined outside the macro needs an
external_type_specification. How you write it determines whether Verus treats
the type as transparent or opaque.
Transparent enums (variant matching allowed)
Use external_type_specification without external_body:
#![allow(unused)]
fn main() {
use nros_node::TriggerCondition;
verus! {
#[verifier::external_type_specification]
pub struct ExTriggerCondition(TriggerCondition);
// Now we can match on variants in spec functions:
pub open spec fn trigger_eval_spec(cond: TriggerCondition, ready: Seq<bool>) -> bool {
match cond {
TriggerCondition::Any => ...,
TriggerCondition::All => ...,
TriggerCondition::Always => true,
TriggerCondition::One(index) => ...,
}
}
} // verus!
}
This works for enums whose variants are visible (public). Verus sees the full variant structure and allows pattern matching in specs and proofs.
Opaque types (no internal access)
Use external_type_specification with external_body:
#![allow(unused)]
fn main() {
#[verifier::external_type_specification]
#[verifier::external_body]
pub struct ExSomeOpaqueType(SomeOpaqueType);
}
Use this for types with private fields, complex internals, or when you only need to pass them around without inspecting their structure. You cannot match on variants or access fields.
Structs with public fields
Use external_type_specification without external_body – fields become
accessible in specs:
#![allow(unused)]
fn main() {
#[verifier::external_type_specification]
pub struct ExDuration(nros_core::time::Duration);
// Fields are accessible:
pub assume_specification[ Duration::to_nanos ](self_: &Duration) -> (n: i64)
ensures n == (self_.sec as i64) * 1_000_000_000i64 + (self_.nanosec as i64);
}
Linking Specs to Production Code
assume_specification
Axiomatically declares a contract on a production function. The contract is trusted (not verified by Verus) – a human auditor must confirm the spec matches the implementation.
#![allow(unused)]
fn main() {
pub assume_specification[ TriggerCondition::evaluate ](
self_: &TriggerCondition, // &self becomes a named parameter
ready: &[bool],
) -> (ret: bool)
ensures
ret == trigger_eval_spec(*self_, ready@);
}
Rules:
&selfmust be written asself_: &Type(named parameter, not method syntax)ready@converts&[bool]toSeq<bool>(Verus view conversion)*self_dereferences&TriggerConditiontoTriggerConditionfor spec matching- The return value must be named:
-> (ret: bool)
Ghost models
For types that can’t be imported (private fields, feature-gated behind C FFI, etc.), create a ghost model – a manually maintained mirror:
#![allow(unused)]
fn main() {
/// Ghost representation of TimerState (mirrors nros_node::timer::TimerState).
pub struct TimerGhost {
pub period_ms: u64,
pub elapsed_ms: u64,
pub mode: TimerModeGhost,
pub canceled: bool,
}
}
Ghost models have weaker guarantees than assume_specification. Correctness
relies on manual comparison with the production source code.
Trust Levels
Every proof falls into one of three trust levels:
| Level | Mechanism | What’s trusted | Strength |
|---|---|---|---|
| Formally linked | assume_specification + external_type_specification | The spec matches the impl (human audit of ~4 lines) | Strongest |
| Ghost model | Manual struct/enum mirror | Line-by-line correspondence with production source | Medium |
| Pure math | Arithmetic identities | Only the math itself | Weakest (no code link) |
Document the trust level of each proof in the module-level doc comment.
Pitfalls
verify = true on production crates
Never add [package.metadata.verus] verify = true to a production crate
that contains function pointers, dyn Trait, or closures. Verus will attempt
THIR erasure on all items in the crate and panic at erase.rs:237.
Only the verification crate (nros-verification) should have verify = true.
Production crates are used as regular dependencies – Verus compiles them with
standard rustc without attempting verification.
vstd dependency
vstd is published on crates.io (vstd = "0.0.0-2026-02-08-0120"). Use the
registry version, not a path dependency. The pre-built Verus release does not
include the full source tree needed for path dependencies (missing
dependencies/prettyplease etc.).
Items outside verus! are external
Any type, function, or trait defined outside the verus! { } macro is treated
as external by Verus. To use it in specs, you must register it with
external_type_specification or reference it via assume_specification.
Closures and iterators
Verus cannot verify code containing closures (including .iter().any(|&r| r))
or function pointers. Mark such items with #[verifier::external] if they’re
in a crate being verified, or keep them in production code that Verus never
touches.
Edition 2024
Verus’s bundled rustc (1.93.0) supports edition 2024. The verification crate
uses edition = "2024" to match the nros workspace. No special
configuration is needed.
Adding a New Proof
- Identify the production function and its source location
- Determine the trust level:
- Can you import the type? Use
external_type_specification - Is the type behind a feature gate with C FFI? Use a ghost model
- Is it pure arithmetic? Pure math proof
- Can you import the type? Use
- Write the spec function inside
verus! { } - If formally linking: add
assume_specificationand document which source lines the auditor should compare - Write the proof function with
ensuresclauses - Run
just verify-verusto check - Update the module doc comment with the new proof’s trust level
File Organization
packages/verification/nros-verification/src/
├── lib.rs # Module declarations
├── scheduling.rs # Timer + trigger + executor proofs
├── communication.rs # CDR serialization safety proofs
├── cdr.rs # CDR round-trip integrity proofs
├── time.rs # Duration/Time arithmetic proofs
├── action.rs # GoalStatus state machine proofs
├── params.rs # ParameterValue + range proofs
├── progress.rs # Executor progress / liveness proofs
├── safety.rs # Safety-protocol proofs
└── e2e.rs # End-to-end data path proofs
Proofs are organized by what they guarantee to the application developer, not by source crate. A single proof module may reference types from multiple production crates.
E2E Safety Protocol
The
safety-e2efeature described in this document has been implemented. It is available innros-rmw,nros-node, and the top-levelnroscrate.
Problem Statement
nano-ros treats zenoh as a trusted transport. Messages are serialized (CDR), transmitted over zenoh, and deserialized without any integrity verification. In safety-critical deployments (Autoware safety island), the transport channel must be treated as untrusted – the EN 50159 “black channel” principle.
Without E2E protection:
- No CRC or checksum on message payloads (corruption undetected)
- No subscriber-side sequence tracking (message loss undetected)
- No duplicate detection (message repetition undetected)
- No freshness validation (stale messages accepted silently)
- No source authentication (masquerade undetected)
Existing Infrastructure
The nano-ros transport layer carries metadata that partially addresses these concerns, but historically only on the publisher side.
RMW Attachment (33 bytes)
Every published message includes a zenoh attachment with:
| Offset | Size | Field | Purpose |
|---|---|---|---|
| 0-7 | 8 | sequence_number (i64 LE) | Monotonically increasing per publisher |
| 8-15 | 8 | timestamp (i64 LE, nanos) | Publication time |
| 16 | 1 | VLE length (always 16) | GID size prefix |
| 17-32 | 16 | rmw_gid | Random per-publisher identifier |
This exists for rmw_zenoh_cpp interoperability. Without safety-e2e, the subscriber parses it into MessageInfo but does not validate sequence continuity, freshness, or source identity.
Message Data Flow
Publisher side:
User msg -> CdrWriter::serialize() -> CDR payload (with 4-byte header)
CDR payload -> ShimPublisher::publish_raw()
-> Compute seq++, timestamp
-> Serialize 33-byte RMW attachment
-> zenoh publish(payload, attachment)
Subscriber side:
zenoh callback -> Copy payload + attachment to static SubscriberBuffer
User calls take_serialized() -> Copy from static buffer to user buffer
User buffer -> CdrReader::deserialize() -> User msg
(Attachment parsed into MessageInfo but NOT validated without safety-e2e)
Key Observations
- Sequence numbers exist but are only checked by subscribers when
safety-e2eis enabled. - Timestamps exist but freshness validation requires a clock source (deferred).
- Publisher GID exists but source authentication requires a registration mechanism.
- CRC is added by the
safety-e2efeature. It is computed over the CDR payload and appended to the attachment. - The attachment travels out-of-band from the CDR payload in zenoh. This is beneficial – the CRC in the attachment provides a diverse check (different data paths).
EN 50159 Threat Model
EN 50159 defines 7 threat classes for communication over untrusted channels:
| Threat | EN 50159 Defense | Integration Approach |
|---|---|---|
| Corruption | CRC | CRC-32 covering CDR payload, stored in extended attachment |
| Repetition | Sequence number | Subscriber tracks expected sequence, flags duplicates |
| Deletion | Seq + timeout | Subscriber detects sequence gaps |
| Insertion | Seq + auth | Sequence validation rejects unexpected messages |
| Resequencing | Sequence number | Subscriber validates monotonic sequence |
| Delay | Timestamp/timeout | Subscriber compares message timestamp to current time |
| Masquerade | Authentication | Subscriber validates expected source GID |
5 of 7 defenses only require subscriber-side validation of data that already exists in the attachment. Only CRC requires new publisher-side computation.
CRC Architecture
The safety-e2e feature extends the zenoh attachment from 33 to 37 bytes:
Existing attachment (33 bytes):
[seq:8][timestamp:8][vle:1][gid:16]
Extended attachment (37 bytes):
[seq:8][timestamp:8][vle:1][gid:16][crc32:4]
The CRC covers the CDR payload bytes (not the attachment itself).
Algorithm: CRC-32/ISO-HDLC
Standard Ethernet CRC (polynomial 0xEDB88320 reflected), used by both AUTOSAR E2E Profile 1/2 and EN 50159. The 1KB lookup table is const-generated at compile time. Deterministic execution time for WCET analysis.
Interoperability
- rmw_zenoh_cpp reads exactly 33 bytes using VLE parsing – extra bytes are ignored
- nano-ros without
safety-e2ereads 33 bytes – succeeds normally - nano-ros with
safety-e2ereads 37 bytes – extracts CRC from bytes 33-36
Subscriber-Side Validation
Sequence Tracking
The subscriber maintains an expected_seq counter (initialized to -1). On each received message:
- First message: initializes
expected_seqtomessage_seq + 1 - Contiguous:
message_seq == expected_seq– normal delivery - Duplicate:
message_seq < expected_seq– flagged inIntegrityStatus - Gap:
message_seq > expected_seq– gap count reported inIntegrityStatus
CRC Validation
If the attachment is longer than 33 bytes, the subscriber extracts the CRC-32 from bytes 33-36 and compares it against a locally computed CRC of the received payload. If the attachment is 33 bytes (legacy or ROS 2 publisher), crc_valid is None.
Feature Integration
The safety-e2e feature flag propagates through the crate hierarchy:
nros (top-level) -> nros-node -> nros-rmw
safety-e2e safety-e2e safety-e2e
Changes by layer
nros-rmw (core changes):
safetymodule: CRC-32 function,IntegrityStatustype,SafetyValidatorstate trackerShimPublisher::publish_raw(): computes CRC and extends attachment to 37 bytesSubscriberBuffer: attachment buffer sized to 37 bytesShimSubscriber:SafetyValidatorfield,take_validated()method
nros-node (API surface):
ShimNodeSubscription:take_validated()returning the message +IntegrityStatus
Unchanged:
- CDR serialization (
nros-serdes) – payload format unchanged - Core types (
nros-core) – no new traits needed - Zenoh backend (
nros-rmw-zenoh) – attachment handling supports variable sizes - Existing
take()API – unchanged behavior
Memory Impact
| Component | Without safety-e2e | With safety-e2e |
|---|---|---|
| CRC-32 lookup table | 0 | +1024 bytes (.rodata) |
| Subscriber attachment buffers (8x) | 8 x 33 = 264 bytes | 8 x 37 = 296 bytes (+32) |
| SafetyValidator per subscriber (8x) | 0 | 8 x ~24 = 192 bytes |
| Total | – | +1248 bytes |
ROS 2 Interoperability
| Sender | Receiver | CRC | Sequence | Result |
|---|---|---|---|---|
| nano-ros (safety) | nano-ros (safety) | Validated | Tracked | Full E2E protection |
| nano-ros (safety) | nano-ros (no safety) | Ignored | Not tracked | Works, no protection |
| nano-ros (safety) | ROS 2 (rmw_zenoh) | Ignored | Not tracked | Works, no protection |
| nano-ros (no safety) | nano-ros (safety) | None | Tracked | Partial (seq only) |
| ROS 2 (rmw_zenoh) | nano-ros (safety) | None | Tracked | Partial (seq only) |
No interoperability is broken. Safety degrades gracefully when one side doesn’t support it.
AUTOSAR E2E Comparison
| Feature | AUTOSAR E2E P01 | AUTOSAR E2E P02 | nano-ros E2E |
|---|---|---|---|
| CRC | CRC-8 (8-bit) | CRC-8 (8-bit) | CRC-32 (32-bit, stronger) |
| Counter | 4-bit (0-14) | 4-bit (0-14) | 64-bit (i64, no rollover in practice) |
| Data ID | 16-bit | 16-bit | 128-bit GID (stronger) |
| Alive counter | Yes | Yes | Yes (via sequence) |
| Timeout | Configurable | Configurable | Deferred (needs clock abstraction) |
| State machine | E2E_SM | Same | Simpler (valid/invalid per message) |
nano-ros E2E is stronger than AUTOSAR P01/P02 in CRC and counter width, but currently lacks the state machine and timeout features.
Future Extensions
- Freshness validation: Requires a
MonotonicClocktrait and platform implementations. The existingtimestampfield already carries the publication time. - Source authentication: The
rmw_gid(16-byte random publisher ID) can be used for masquerade detection if the subscriber registers expected sources. - Watchdog supervision: Separate from E2E protocol – monitors alive heartbeats, deadline completion, and logical execution flow.
- Formal verification: CRC correctness, sequence tracking invariants, and
IntegrityStatus::is_valid()are candidates for Verus proofs.
Production Readiness Checklist
A copy-out checklist for teams piloting nano-ros toward production deployment. Each box is a concrete validation step, not a marketing claim. The book documents the framework’s intent + plumbing; this page is what your team needs to confirm on your target before shipping.
Why a separate checklist? nano-ros is production-capable, but some acceptance items are hardware-gated (P99 latency on real Cortex-M3, multicast on real silicon, NuttX SCHED_SPORADIC under kernel config) and can’t be validated in CI. The checklist gives you the steps to close those gaps for your deployment.
1. Real-time metrics (hardware-validated)
The book intentionally quotes almost no latency numbers, because the in-tree harnesses run under QEMU, where DWT cycle counters read zero and wall-clock percentiles track host load. For production claims, measure on your actual silicon.
- End-to-end P99 latency (publisher → executor callback)
on your target MCU at its production clock + load. Target:
≤ design budget. Tooling:
wake-latency-cortex-m3bench atpackages/testing/nros-bench/wake-latency-cortex-m3/. - Worst-case stack depth per task. Tool:
cargo-call-stack,cargo-stack-sizes, or the ARM stack-analyzer for C/C++. - Heap fragmentation pattern over 24 h at nominal load.
Spot-check with
mallinfoor your RTOS’s heap-stats API. - Wake latency (transport-rx interrupt → first user callback dispatched). Required if you use the async / poll- blocked spin path.
- Spin-loop budget overrun rate under sustained pub load.
Executor::spin_once(timeout)returns the overrun count.
2. Platform-specific validation
Per-RTOS gaps that the book documents as “tested in CI” cover reference boards — your actual board / kernel-config combination may differ.
- Multicast / IGMP if using DDS (RTPS). Confirm SPDP discovery actually fires on your RTOS + driver. Untested on FreeRTOS + ThreadX as of writing.
- Clock wraparound + extension correctness on long-running
deployments. The platform’s
nros_platform_time_now_msmust handle u32 wrap (49.7 days) and u64 extend. - Allocator behavior under memory pressure. Boot-time alloc
OK on most RTOSes; mid-run alloc only on
stdPOSIX. Confirm your hot paths don’t allocate. - Network packet loss recovery. Drop 5% of packets in your lab and confirm the talker / listener recovers.
- Critical-section regions are short. The platform’s
nros_platform_critical_section_*ABI is the IRQ-disable surface; long critical sections starve other ISRs.
3. RMW backend certification
- Backend version pinned to a tested tuple. Zenoh-pico
1.7.2 (matches
rmw_zenoh_cpp). Cyclone DDS 0.10.5 (matchesros-humble-cyclonedds). XRCE-DDS Micro-Client at its workspace pin. - All required QoS policies supported by your backend. The
per-policy table is in
RMW API: Differences from upstream rmw.h §7;
the authoritative source is each backend’s
supported_qos_policies(). - Discovery stability over your network topology. Zenoh-pico in client mode needs the router reachable; loss of router = lost routing but local node lives. XRCE needs Agent uptime. Cyclone DDS discovers via multicast SPDP.
- Bridge stability if multi-backend. Confirm no memory bloat over 72 h with two registered RMWs running.
- Cyclone DDS limitations checked. Status events and some stock-ROS interop slices are still in progress; embedded RTOS ports remain gated on a hosted Cyclone runtime.
4. Safety + formal verification
-
just verify-kaniclean against your build. 148 bounded harnesses; non-trivial coverage of CDR + scheduling + RMW glue. -
just verify-verusclean. 83 deductive proofs. - CRC32 attached if using
safety-e2efeature. The 37-byte attachment is transparent to stock ROS 2 (ignored gracefully) and detected by other nano-ros nodes. - Timeout bounds on every blocking call.
spin_oncetimeout,Promise::wait_for(timeout),recv_timeout. NoWAIT_FOREVER. - Parameter store capacity ≥ declared parameter count
([
param-services] feature gate). - Stack overflow detection enabled by your platform
(FreeRTOS
configCHECK_FOR_STACK_OVERFLOW, Zephyr stack sentinels, NuttXCONFIG_DEBUG_STACK).
5. Interop testing
- Publish from nano-ros, subscribe with stock ROS 2.
RMW_IMPLEMENTATION=rmw_zenoh_cppfor Zenoh,rmw_cyclonedds_cppfor Cyclone,rmw_fastrtps_cppfor DDS (interop tier). - Message type compatibility for any custom
.msgyou’ve added. Round-trip a sample message through ROS 2’srosbag2to confirm wire-level parity. - QoS profile matching. Mismatched reliability / durability / history kill discovery silently on DDS / RTPS.
- Lifecycle callbacks fire on node startup / shutdown if
you’ve opted into
lifecycle-services. - Cross-RTOS interop: if your fleet mixes RTOSes (e.g. Zephyr sensor + FreeRTOS actuator + POSIX coordinator), confirm all three sides see each other.
6. Failure recovery
- Agent / router restart: kill
zenohd(Zenoh) orMicroXRCEAgent(XRCE) mid-run. Confirm reconnection. For DDS / Cyclone this is N/A (no central process). - Network partition → reconnection. Block the talker’s
egress with
iptablesfor 30 s, then unblock. Verify the listener resumes within your design SLA. - Heap exhaustion path: graceful degradation OR clean crash + restart? If hosted-RTOS + watchdog, restart is usually correct. If bare-metal, you probably have no restart story — confirm your design assumes this.
- Stack overflow detection triggers a panic / fault rather than silent corruption.
- Power loss mid-write (if persisting state). Not nano-ros’s concern, but mention it in your design review.
7. Operational concerns
- Bootloader + OTA strategy. Out of scope for nano-ros but mandatory for fleet deployments — name it explicitly in your project plan.
- Log / diagnostics exfiltration.
nros-logprovides the logging surface; pick a sink (UART, RTT, semihosting, or ROS 2/rosoutover the wire). - Time synchronization (NTP, PTP, RTC). nano-ros doesn’t ship a time-sync layer; your fleet design must.
- Watchdog coverage: the executor’s
spin_periodreports overruns, but it doesn’t pet a hardware watchdog. Wire one manually.
8. License + governance
- License: MIT OR Apache 2.0 (dual). Both permissive, no GPL copyleft, OK for proprietary firmware. Confirm your legal team is comfortable.
- Third-party dependencies: zenoh-pico (Eclipse), Cyclone
DDS (Eclipse), Micro-XRCE-DDS-Client (Apache-2). Vendored
dependencies carry license files in each
third-party/*/LICENSE. - Patent grant: Apache 2.0 carries an explicit patent grant; MIT does not. Most adopters rely on the Apache half.
- Support model: nano-ros has no commercial support entity as of writing. Plan accordingly — either staff in-house expertise or contract a consultancy.
- Roadmap visibility: track the project’s roadmap directory in the upstream repo. Items are numbered and dated.
9. Maintainability pledge
The single-maintainer signal in §8 is a real adoption risk. Add explicit mitigations to your project plan rather than treating “open-source” as the answer:
- Maintenance horizon: record the date you adopted and
the upstream commit cadence (e.g.
git log --since=… --oneline | wc -lover the last 90 days). Re-check quarterly. - Escalation path: identify the primary maintainer
contact (from
Cargo.tomlauthors+ GitHub commit history). For CVEs, use a public GitHub security advisory. - Governance model: nano-ros is BDFL-style (benevolent-dictator-for-life) with no formal RFC / vote process. Major design decisions are documented in the roadmap directory; align your expectations accordingly.
- Fork mitigation: the dual MIT / Apache-2.0 license lets your team fork and maintain independently if upstream goes dormant. Budget for that possibility, not the assumption that it will happen.
- Distro tracking: nano-ros targets ROS 2 Humble today. Iron / Jazzy support depends on the type-hash work tracked in the upstream roadmap (no public ETA yet). If your product must ship on Iron / Jazzy at launch, plan to contribute the type-hash port or wait for upstream.
Scoring rubric
For each section above, count [x] boxes as your readiness score.
Suggested gates:
Sections carry 4–6 boxes each (45 total):
| Score per section | Status |
|---|---|
| All boxes | Production-ready for that axis |
| Missing one | Pilot deployment OK; close the gap before scale |
| Missing two | Lab / prototype only |
| Missing three+ | Block on these items first |
Sum across all 9 sections (§1–9). Below ~32 / 45 you have foundational work to do; above ~40 / 45 you’re at production quality on every axis where nano-ros can be validated externally.
See also
- Real-Time Analysis — RT scheduling background + response-time formulas.
- Formal Verification — Kani + Verus harnesses.
- Safety Protocol — E2E CRC + sequence tracking.
- Choosing an RMW Backend — backend capability matrix.
- Supported Boards — per-board status + caveats.
Zenoh-pico Symbol Reference
This page documents the ~55 FFI symbols that zenoh-pico requires at link time.
These symbols are provided by zpico-sys’s C alias translation unit
(c/zpico/platform_aliases.c, built by the default-on platform-aliases
feature), which forwards each z_* / _z_* call to the canonical
nros_platform_* ABI. When porting to a new platform, you implement an
nros-platform-<name> crate (see Custom Platform)
that supplies the nros_platform_* symbols — you do not provide the
z_* symbols directly.
Historical note: this mapping used to live in a separate
zpico-platform-shimcrate; Phase 129 deleted it and folded the forwarders into thezpico-sysalias TU.
This page serves as a reference for understanding what the alias TU maps
and what capabilities your nros-platform-<name> crate must provide.
Platform crate structure
packages/platform/nros-platform-<name>/
├── Cargo.toml
└── src/
├── lib.rs
├── clock.rs
├── memory.rs
├── sleep.rs
├── random.rs
├── time.rs
├── threading.rs # no-op stubs or real RTOS impl
├── socket_stubs.rs # if using smoltcp
└── network.rs # if using smoltcp
Cargo.toml must have zero nros-* dependencies. It may depend on:
- Hardware HAL crate (e.g.,
stm32f4xx-hal,esp-hal) zpico-smoltcp(if using smoltcp networking)embedded-alloc(for heap on bare-metal)- RTOS bindings crate
Required FFI symbols
Clock (critical)
The clock is the most important primitive. zenoh-pico uses it for session keep-alive, query timeouts, and transport timeouts. It must be backed by a hardware timer or OS tick — never by a software counter that only advances when polled.
// Monotonic clock — returns an opaque timestamp (lower 32 bits of ms)
usize z_clock_now(void);
// Elapsed time since a previous z_clock_now() value
c_ulong z_clock_elapsed_us(usize *time);
c_ulong z_clock_elapsed_ms(usize *time);
c_ulong z_clock_elapsed_s(usize *time);
// Advance a timestamp value by a duration
void z_clock_advance_us(usize *clock, c_ulong duration);
void z_clock_advance_ms(usize *clock, c_ulong duration);
void z_clock_advance_s(usize *clock, c_ulong duration);
If using smoltcp (bare-metal networking), also provide:
// Called by zpico-smoltcp for TCP/IP timestamping
u64 smoltcp_clock_now_ms(void);
Implementation checklist:
- Identify a hardware timer (SysTick, GPT, DWT) or use the OS tick API
(
xTaskGetTickCount,k_uptime_get,clock_gettime) - Handle 32-bit timer wraps — track a wrap count in an atomic or use a 64-bit counter
- Never advance the clock inside
smoltcp_network_poll()— read the hardware timer directly - Verify with QEMU: use
-icount shift=autoto synchronize virtual time with wall-clock time
Reference implementations:
| Platform | Clock source | File |
|---|---|---|
| MPS2-AN385 | CMSDK APB Timer0 (25 MHz) | nros-platform-mps2-an385/src/clock.rs |
| STM32F4 | ARM DWT cycle counter | nros-platform-stm32f4/src/clock.rs |
| ESP32-C3 (QEMU) | esp_hal::time::Instant | nros-platform-esp32-qemu/src/clock.rs |
| FreeRTOS | xTaskGetTickCount() | Use OS tick directly |
| NuttX | clock_gettime(CLOCK_MONOTONIC) | POSIX API |
Memory
zenoh-pico allocates heap during session open and entity creation. Typical minimum: 64 KB heap.
void *z_malloc(usize size);
void *z_realloc(void *ptr, usize size);
void z_free(void *ptr);
Options:
embedded-allocFreeListHeap(bare-metal)- RTOS heap (
pvPortMalloc/tx_byte_allocate) - System
malloc(POSIX, NuttX)
Sleep
i8 z_sleep_us(usize time);
i8 z_sleep_ms(usize time);
i8 z_sleep_s(usize time);
All return 0 (_Z_RES_OK). On bare-metal, busy-wait using the clock.
On RTOS, delegate to vTaskDelay / tx_thread_sleep / k_sleep.
If using smoltcp, poll the network stack during busy-wait sleep to avoid missing packets.
Random
zenoh-pico needs randomness for session IDs, SN initialization, and scouting nonces.
u8 z_random_u8(void);
u16 z_random_u16(void);
u32 z_random_u32(void);
u64 z_random_u64(void);
void z_random_fill(void *buf, usize len);
A simple xorshift32 PRNG is sufficient. Seed it with hardware entropy (RNG peripheral, ADC noise, semihosting wall-clock time) during init.
Time
System time (wall clock). Used for logging and z_time_now_as_str().
u64 z_time_now(void);
const char *z_time_now_as_str(char *buf, c_ulong buflen);
c_ulong z_time_elapsed_us(u64 *time);
c_ulong z_time_elapsed_ms(u64 *time);
c_ulong z_time_elapsed_s(u64 *time);
i8 _z_get_time_since_epoch(ZTimeSinceEpoch *t);
Where ZTimeSinceEpoch is:
#[repr(C)]
struct ZTimeSinceEpoch {
u32 secs,
u32 nanos,
}
On bare-metal without an RTC, return monotonic time or zeros.
Threading
For single-threaded platforms (bare-metal, RTIC), provide no-op stubs. For RTOS platforms, implement real task/mutex/condvar operations.
Task operations:
i8 _z_task_init(ZTask *task, ZTaskAttr *attr, void*(*fun)(void*), void *arg);
i8 _z_task_join(ZTask *task);
i8 _z_task_detach(ZTask *task);
i8 _z_task_cancel(ZTask *task);
void _z_task_exit(void);
void _z_task_free(ZTask **task);
Mutex operations:
i8 _z_mutex_init(ZMutex *m);
i8 _z_mutex_drop(ZMutex *m);
i8 _z_mutex_lock(ZMutex *m);
i8 _z_mutex_try_lock(ZMutex *m);
i8 _z_mutex_unlock(ZMutex *m);
Recursive mutex operations:
i8 _z_mutex_rec_init(ZMutexRec *m);
i8 _z_mutex_rec_drop(ZMutexRec *m);
i8 _z_mutex_rec_lock(ZMutexRec *m);
i8 _z_mutex_rec_try_lock(ZMutexRec *m);
i8 _z_mutex_rec_unlock(ZMutexRec *m);
Condition variable operations:
i8 _z_condvar_init(ZCondvar *cv);
i8 _z_condvar_drop(ZCondvar *cv);
i8 _z_condvar_signal(ZCondvar *cv);
i8 _z_condvar_signal_all(ZCondvar *cv);
i8 _z_condvar_wait(ZCondvar *cv, ZMutex *m);
i8 _z_condvar_wait_until(ZCondvar *cv, ZMutex *m, u64 *abstime);
Single-threaded stubs: Return 0 for all mutex/condvar operations.
Return -1 for _z_task_init (task creation is not supported).
RTOS implementations: Map to xTaskCreate/xSemaphoreCreateMutex
(FreeRTOS), tx_thread_create/tx_mutex_create (ThreadX), or
pthread_create/pthread_mutex_init (NuttX, POSIX). zenoh-pico requires
recursive mutexes (configUSE_RECURSIVE_MUTEXES=1 on FreeRTOS).
Sockets
If using smoltcp (bare-metal), socket operations are handled by
zpico-smoltcp. Your platform crate provides thin shims:
#[repr(C)]
struct ZSysNetSocket { i8 _handle, bool _connected }
i8 _z_socket_set_non_blocking(const ZSysNetSocket *sock); // Return 0
i8 _z_socket_accept(const ZSysNetSocket *in, ZSysNetSocket *out); // Return -1
void _z_socket_close(ZSysNetSocket *sock);
i8 _z_socket_wait_event(void *peers, ZMutexRecRef *mutex); // Return 0
If using OS sockets (POSIX, NuttX, Zephyr), zenoh-pico’s built-in socket layer handles everything — no socket stubs needed.
libc stubs (bare-metal only)
Bare-metal targets without a C runtime need standard C library functions:
usize strlen(const char *s);
int strcmp(const char *s1, const char *s2);
int strncmp(const char *s1, const char *s2, usize n);
char *strchr(const char *s, int c);
char *strncpy(char *dest, const char *src, usize n);
void *memcpy(void *dest, const void *src, usize n);
void *memmove(void *dest, const void *src, usize n);
void *memset(void *dest, int c, usize n);
int memcmp(const void *s1, const void *s2, usize n);
void *memchr(const void *s, int c, usize n);
c_ulong strtoul(const char *nptr, char **endptr, int base);
int *__errno(void);
RTOS platforms (FreeRTOS, NuttX, ThreadX, Zephyr) ship their own libc — you do not need these stubs.
Network poll callback (smoltcp only)
If using smoltcp for networking, provide a poll callback that
zpico-smoltcp calls to process network events:
void smoltcp_network_poll(void);
And a Rust API for the board crate to register the network state:
#![allow(unused)]
fn main() {
pub unsafe fn set_network_state(
iface: *mut Interface,
sockets: *mut SocketSet<'static>,
device: *mut (),
);
pub unsafe fn clear_network_state();
}
Step-by-step procedure
- Create the platform crate (
nros-platform-<name>) – see Custom Platform for the full guide - Implement and verify the clock — this is the #1 cause of porting
failures. Print
clock_ms()in a loop and verify monotonic advance - Implement remaining primitives — memory, random, sleep, time, threading, sockets. Each module is independent
- Wire into
nros-platform— add a feature andConcretePlatformalias - Create the board crate — see Custom Board Package
- Add the platform feature to
nroswith mutual exclusivity checks - Write an example — see Creating Examples
- Add test infrastructure —
just test-<name>recipe + nextest group
Platform capability summary
| Capability | Bare-metal | RTOS (FreeRTOS/ThreadX) | POSIX-like (NuttX/Zephyr) |
|---|---|---|---|
| Clock | Hardware timer | OS tick API | clock_gettime |
| Memory | embedded-alloc | RTOS heap | System malloc |
| Sleep | Busy-wait + poll | vTaskDelay | nanosleep |
| Threading | No-op stubs | Real tasks + mutexes | pthreads |
| Sockets | smoltcp shims | lwIP or NetX sockets | BSD sockets |
| Random | Seeded xorshift | RTOS RNG or xorshift | /dev/urandom |
| libc | Hand-written stubs | RTOS libc | System libc |
| Network poll | smoltcp_network_poll | Stack-specific poll | Not needed |
Common pitfalls
See Platform Porting Pitfalls for detailed failure modes including poll-driven clocks, DMA buffer placement, QEMU I/O starvation, recursive mutexes, stack sizing, and heap sizing.
XRCE-DDS Symbol Reference
XRCE-DDS has a minimal platform abstraction layer. It is single-threaded,
heap-less, and delegates networking to user-provided transport callbacks.
The 2-3 required FFI symbols (uxr_millis, uxr_nanos) are provided by
nros-rmw-xrce’s C alias translation unit (src/platform_aliases.c,
compiled into nros-rmw-xrce-cffi on every target), which forwards them to
the canonical nros_platform_* ABI. When porting to a new platform, you
implement an nros-platform-<name> crate (see
Custom Platform) that supplies the
nros_platform_* symbols — you do not provide the uxr_* symbols
directly.
Historical note: this mapping used to live in a separate
xrce-platform-shimcrate; Phase 129 deleted it and moved theuxr_*forwarders into thenros-rmw-xrcealias TU.
Platform crate structure
The XRCE-DDS clock symbols are a subset of what the platform crate provides.
Your nros-platform-<name> crate provides the canonical clock primitives
(nros_platform_time_now_ms, nros_platform_clock_us) and the alias TU
maps them to the uxr_* symbols XRCE-DDS expects.
packages/platform/nros-platform-<name>/
├── Cargo.toml
└── src/
└── lib.rs # clock + other primitives
Cargo.toml must have zero nros-* dependencies. It may depend on:
- Hardware HAL crate (e.g.,
stm32f4xx-hal,esp-hal) xrce-smoltcp(if using smoltcp networking)
Required FFI symbols
Clock (2 symbols)
XRCE-DDS uses the clock for session timeouts and time synchronization.
// Monotonic millisecond clock
i64 uxr_millis(void);
// Monotonic nanosecond clock
i64 uxr_nanos(void);
These must be backed by a hardware timer or OS tick — same rules as the zenoh-pico clock.
Reference implementations:
| Platform | Clock source | File |
|---|---|---|
| MPS2-AN385 | CMSDK APB Timer0 (25 MHz) | nros-platform-mps2-an385/src/clock.rs |
| Zephyr | k_uptime_get() | xrce-zephyr/src/xrce_zephyr.c |
| POSIX | clock_gettime(CLOCK_MONOTONIC) | Built-in time.c from XRCE library |
smoltcp clock (1 symbol, if using smoltcp)
If using xrce-smoltcp for bare-metal networking:
u64 smoltcp_clock_now_ms(void);
This is the same symbol used by zpico-smoltcp — shared clock
implementations work for both RMW backends.
Transport callbacks
Instead of requiring socket FFI symbols, XRCE-DDS uses a custom transport
abstraction. You register four callbacks via
uxr_set_custom_transport_callbacks():
bool open(uxrCustomTransport *transport);
bool close(uxrCustomTransport *transport);
size_t write(uxrCustomTransport *transport, const uint8_t *buf, size_t len, uint8_t *errcode);
size_t read(uxrCustomTransport *transport, uint8_t *buf, size_t len, int timeout_ms, uint8_t *errcode);
These are typically provided by a transport crate, not the platform crate:
| Transport | Crate | Description |
|---|---|---|
| smoltcp UDP | xrce-smoltcp | Bare-metal networking via smoltcp |
| Zephyr UDP | xrce-zephyr | Zephyr BSD sockets |
| POSIX UDP | Built-in | Uses XRCE library’s POSIX transport |
The transport crate registers its callbacks during initialization. Your platform crate does not need to implement sockets.
What you do NOT need
Unlike zenoh-pico, XRCE-DDS does not require:
- Memory symbols — XRCE-DDS is heap-less; all buffers are statically sized
- Sleep symbols — polling is driven by
uxr_run_session_time() - Random symbols — no session ID randomization needed
- Threading symbols — single-threaded by design
- libc stubs — Micro-XRCE-DDS has minimal C dependencies
This makes XRCE-DDS the easiest RMW backend to port to new platforms.
Platform-conditional compilation in xrce-sys
The xrce-sys crate uses Cargo features to select platform behavior:
| Feature | Effect |
|---|---|
posix | Compiles built-in time.c (uses clock_gettime, BSD sockets) |
bare-metal | Skips time.c; expects uxr_millis()/uxr_nanos() from platform crate |
zephyr | Compiles xrce_zephyr.c (Zephyr kernel APIs) |
freertos | RTOS-specific; skips time.c |
nuttx | RTOS-specific; skips time.c |
threadx | RTOS-specific; skips time.c |
When adding a new platform, you may need to add a feature to xrce-sys that
skips the default time.c compilation and lets your platform crate provide
the clock symbols.
Step-by-step procedure
- Create the platform crate —
nros-platform-<name>/(see Custom Platform) - Implement the canonical clock primitives —
nros_platform_time_now_ms()/nros_platform_clock_us(); the alias TU (nros-rmw-xrce/src/platform_aliases.c) maps these touxr_millis()/uxr_nanos()for you - Implement
smoltcp_clock_now_ms()if using smoltcp transport - Add a feature to
xrce-sysfor the new platform if needed - Choose or implement a transport crate — reuse
xrce-smoltcpfor bare-metal, or implement a new transport if the platform has its own networking stack - Create the board crate — see Custom Board Package
- Add the platform feature to
nroswith mutual exclusivity checks - Write an example and tests
Example: bare-metal MPS2-AN385
The simplest reference is nros-platform-mps2-an385. It supplies the
canonical nros_platform_* clock symbols; the XRCE alias TU
(nros-rmw-xrce/src/platform_aliases.c) then derives uxr_millis /
uxr_nanos from them — you never hand-write the uxr_* symbols:
/* nros-rmw-xrce/src/platform_aliases.c — provided for you, not the porter */
int64_t uxr_millis(void) {
return (int64_t) nros_platform_time_now_ms();
}
int64_t uxr_nanos(void) {
return (int64_t) nros_platform_clock_us() * 1000;
}
So a porter only implements the canonical primitives in their platform crate
(nros_platform_time_now_ms, nros_platform_clock_us), plus
smoltcp_clock_now_ms() if using the smoltcp transport. The MPS2-AN385
platform reads these from a hardware timer with wrap detection.
Where clock_ms() reads from a hardware timer with wrap detection.
Common pitfalls
- Agent connectivity — XRCE-DDS requires a Micro-XRCE-DDS Agent running on the host. Ensure the agent is reachable from the target before debugging the platform layer.
- Reliable stream history —
STREAM_HISTORYmust be >= 2 (recommend 4). History=1 fails to recycle slots between separateuxr_run_session_until_all_statuscalls. - Flush after
request_data—uxr_buffer_request_datamust be flushed withuxr_run_session_timeimmediately after being called. Unflushed requests in the reliable output stream cause intermittent timeouts.
Creating Examples
This guide describes the canonical example shape adopted (layout) + (consumption). When adding a new example, copy the nearest existing peer; the per-platform working examples are the authoritative templates.
Canonical layout
Every example is a self-contained, copy-out project under one of:
| Path | Used for |
|---|---|
examples/<plat>/<lang>/<example>/ | The standard cell. RMW is selected at build time (Cargo features / -DNANO_ROS_RMW= / Kconfig overlay), not encoded in the path. A single-package “app” example here is the canonical starter shape; the multi-package workspace shape (Node + Bringup + Entry pkgs) kicks in at ≥2 nodes — see Multi-Node Projects. |
examples/bridges/<name>/ | Cross-RMW gateway examples (one binary, multiple backends). |
examples/templates/<name>/ | Multi-platform copy-out recipes (e.g. multi-package-workspace). |
The <plat> × <lang> coverage matrix (RMW chosen at build time) is authoritative in
examples/README.md.
Intentionally empty cells (bare-metal C/C++, PX4 Rust, …) are listed
in the same file; do not fill them without lifting the underlying
constraint.
Variant naming uses suffix form so peers sort together:
talker-rtic, service-client-async, talker-rtic-mixed. Avoid
parallel parent directories like async-*/ or rtic-*/.
Non-example binaries
Tests / benches / smokes are not under examples/. They live
under packages/testing/:
| Use | Location |
|---|---|
| Performance, fairness, stress, large-msg | packages/testing/nros-bench/<name>/ |
| Driver / board bringup smoke (no nros API) | packages/testing/nros-smoke/<name>/ |
| Fixture binaries built by integration tests | packages/testing/nros-tests/bins/<name>/ |
Consumption shape
Rust (native or Cargo cross-target)
Each example is a standalone Cargo package with empty [workspace]
table — it does not participate in any walking-up workspace:
[package]
name = "native-rs-zenoh-talker"
edition = "2024"
publish = false
[[bin]]
name = "talker"
path = "src/main.rs"
[dependencies]
nros = { version = "*", default-features = false,
features = ["std", "rmw-cffi", "ros-humble"] }
nros-rmw-zenoh = { version = "*",
features = ["std", "platform-posix", "ros-humble"] }
[workspace]
nano-ros crates are declared registry-style (phase-277 W6): they are
not on crates.io, so the example’s tracked .cargo/config.toml carries
the # nros-managed [patch.crates-io] block resolving them into the
checkout. After adding/renaming nros deps or msg <depend> rows, re-run
NROS_REPO_DIR=<repo root> nros sync in the example dir and commit the
rewritten .cargo/config.toml. This is what makes the copy-out promise
real — a copied example re-runs nros sync at its new location.
cargo build / cargo run from inside the example directory is the
canonical invocation. There is no workspace-wide cargo build that
picks up examples — they are explicitly out-of-workspace.
C / C++ (CMake)
Each example is a standalone CMake project that pulls nano-ros via
add_subdirectory(<repo-root>). The canonical four-line
preamble:
cmake_minimum_required(VERSION 3.22)
project(my_example LANGUAGES C CXX)
set(NANO_ROS_PLATFORM <plat>)
set(NANO_ROS_RMW <rmw>)
set(NANO_ROS_BOARD <board>) # embedded only
# Phase-277 W6 standard root guard: cache var > NROS_REPO_DIR env > walk-up.
if(NOT DEFINED NANO_ROS_ROOT)
if(DEFINED ENV{NROS_REPO_DIR} AND NOT "$ENV{NROS_REPO_DIR}" STREQUAL "")
set(NANO_ROS_ROOT "$ENV{NROS_REPO_DIR}")
else()
get_filename_component(NANO_ROS_ROOT
"${CMAKE_CURRENT_SOURCE_DIR}/<rel-path-to-repo-root>" ABSOLUTE)
endif()
endif()
add_subdirectory("${NANO_ROS_ROOT}" nano_ros)
add_executable(my_example src/main.c)
target_link_libraries(my_example PRIVATE NanoRos::NanoRos)
nros_platform_link_app(my_example)
nano_ros_link_rmw(my_example RMW <rmw>)
nano_ros_link_rmw emits the strong-stub nros_app_register_backends()
that calls every linked RMW’s nros_rmw_<x>_register() symbol — the
auto-registration path for targets where linkme’s distributed-slice
contribution isn’t picked up by the linker (FreeRTOS, NuttX, Zephyr,
ESP-IDF).
There is no find_package(NanoRos) path — it was deleted along
with just install-local, every install(...) rule, and every
Config.cmake.in template.
Per-example contents
examples/<plat>/<lang>/<example>/
├── package.xml # ROS-style manifest for the example
├── Cargo.toml | CMakeLists.txt # Rust or C/C++ build entry
├── .cargo/config.toml # Rust only — target + .cargo patches
├── src/ # main.rs / main.c / main.cpp
├── generated/ # codegen output for any custom msgs
└── README.md # usage instructions
Each example’s Cargo.toml / CMakeLists.txt builds in isolation —
no workspace reliance, no path heuristics walking up the source tree.
Message generation
Examples with custom .msg, .srv, or .action files generate
bindings in-tree under generated/. The generated/ directory is
gitignored per-example (only packages/interfaces/rcl-interfaces/
generated bindings live in git Arduino bundle exception
aside).
source /opt/ros/humble/setup.sh # for rosidl tooling
nros generate-rust # or generate-c / generate-cpp / generate-all
For BSP / cross-target examples that maintain their own
.cargo/config.toml, pass --config --nano-ros-path <relative>:
nros generate-rust --config --nano-ros-path ../../../packages
The --config flag uses ConfigPatcher to idempotently add
[patch.crates-io] entries while preserving existing [build] /
[target.*] sections.
For CMake consumers:
nros_find_interfaces(LANGUAGE C SKIP_INSTALL)
nano_ros_generate_interfaces(... LANGUAGE C)
See Message Generation for the
full reference + package.xml schema.
Adding a new example — checklist
- Pick the canonical cell. Confirm
<plat>/<lang>/<name>isn’t in the “intentionally empty” list inexamples/README.md. - Copy the nearest peer. Identical-RMW + adjacent-platform is
the lowest-risk template (e.g. copy
examples/qemu-arm-freertos/c/talkerto make a new FreeRTOS C/zenoh example). - Update names +
package.xml. RenameCargo.toml’snameand[[bin]]entries (Rust) orproject(...)andadd_executable(...)targets (CMake). - Regenerate bindings. Run
nros generate-rust-*against the newpackage.xml. Custom messages need their ownpackage.xmlin the consuming example. - Build standalone.
cargo buildorcmake -B build && cmake --build buildfrom the example directory. No walking-up workspace allowed. - Wire the test fixture (optional). If the example needs an E2E
gate, add a builder in
packages/testing/nros-tests/src/fixtures/binaries/<plat>.rsthat runscargo build/cmake --buildand points the test at the resulting binary. - Update
examples/README.mdcoverage matrix if you filled a previously-empty cell.
Per-platform notes
| Platform | Source file shape | Build command | Notes |
|---|---|---|---|
native | src/main.rs, src/main.c, src/main.cpp | cargo run / cmake --build | Full std. Pattern A or B. |
qemu-arm-baremetal | src/main.rs with #[entry] | cargo run (runner = qemu-system-arm …) | No std. Pure Cortex-M3. |
qemu-arm-freertos | src/main.rs / src/main.cpp / src/main.c | cargo run (Rust) or cmake --build (C/C++) | FreeRTOS kernel + lwIP. |
nuttx | src/main.rs / src/main.c | cmake --build (NuttX export tarball) | NuttX kernel. |
threadx-linux / threadx-riscv64 | src/main.rs | cmake --build | ThreadX + NetX Duo. |
esp32 | src/main.rs | cargo run (esp-hal) | bare-metal esp-hal. |
zephyr | src/lib.rs (staticlib) or src/main.cpp | west build | Kconfig + west module. |
Per-platform deep-dives — toolchain setup, Kconfig variables, runner scripts — live in the Platform Guides.
See Also
- Build as a CMake subdirectory
- Message Generation
examples/README.md— coverage matrix + intentionally-empty cellsexamples/templates/multi-package-workspace/— Pattern A copy-out template
Per-Platform Contributor Lanes
These are the in-tree just recipes contributors use to build fixture
sets, boot QEMU lanes, and run per-platform test sweeps from a nano-ros
checkout. Users never need them — the user path (setup, build, run with
the vendor’s own tools) is on each platform’s own page, which links back
here per platform. Every lane assumes source ./activate.sh (or
direnv allow) in the checkout first.
FreeRTOS
Lanes for FreeRTOS (QEMU MPS2-AN385).
The prebuilt test fixtures bake per-language allocator ports instead
of 7447 (the copy-out examples’ port) so suites run in parallel;
just freertos talker boots the fixture, so pair it with
just freertos zenohd (listens on 7800, the Rust pub/sub fixture port).
just freertos build-fixtures builds every in-tree zenoh + DDS example
across Rust / C / C++ in one pass.
just freertos zenohd & starts the router on the fixture port (7800)
and just freertos talker boots the prebuilt talker fixture in QEMU
instead of the copy-out example. For batch testing without manual QEMU
launches, just freertos test runs every E2E (pub/sub, service,
action) against a temporary in-test zenohd.
just freertos build / just freertos test build and exercise the
in-tree fixtures.
ESP32
Lanes for ESP32 (esp-hal, bare-metal Rust).
Build:
# QEMU ESP32 (qemu-system-riscv32). `just esp32 build-qemu` (which
# `just esp32 talker` depends on) builds the QEMU-board variant; the
# example's build.rs invokes `nros generate-rust` automatically, so
# the `generated/` dir populates on first build (gitignored).
just esp32 build-qemu
Run (the router must already be listening on the port the example dials — see the platform page):
# Boot the talker binary in qemu-system-riscv32 (esp32c3):
just esp32 talker
# Expected serial output (per src/lib.rs):
# Publishing: 'Hello World: 1'
# Publishing: 'Hello World: 2'
# ...
The just esp32 talker recipe re-runs build-qemu every invocation,
so a first / cold run adds ~25 s of build time on top of the ~15 s
readiness signal.
The shorter deployment spelling:
just esp32 build
just esp32 talker
NuttX
Lanes for NuttX (QEMU) and Integration: NuttX external app.
Build (arm QEMU):
just nuttx build
This cross-compiles all NuttX examples for armv7a-nuttx-eabi using
cargo +nightly build --release.
Build (RISC-V rv-virt):
just nuttx build-riscv-c # C example fixtures
just nuttx build-riscv-rust # Rust example fixtures
Test:
just nuttx test # arm QEMU integration tests
just nuttx test-all # including the networked E2E lanes
Run: for nano-ros’s own in-tree QEMU examples, just nuttx zenohd &
starts the router on the fixture port (8200) and just nuttx talker
wraps qemu-system-arm with the right wiring. talker there is the
Rust variant; the C / C++ variants boot through the make-driven path
described under “Auto-configure glue” in
Integration: NuttX external app.
Zephyr
Lanes for Zephyr (native_sim) and Zephyr Integration (west module).
E2E testing:
# Zenoh examples
just zephyr build # Build Rust zenoh examples
just zephyr build-c # Build C zenoh examples
just zephyr test # Run zenoh E2E tests
# XRCE examples
just zephyr build-xrce # Build all XRCE examples (Rust + C)
just zephyr test-xrce # Run XRCE E2E tests
# All examples
just zephyr build-all # Build everything
just zephyr ci # Doctor + test (CI shortcut)
nano-ros’s own zephyr talker has a matching recipe for the canonical
native_sim build path: just zephyr talker, paired with
just zephyr zenohd & (listens on the fixture port 7400).
To completely recreate the in-tree Zephyr workspace:
just setup zephyr --force
Arm FVP
Lanes for ARM FVP (FVP_BaseR_AEMv8R).
Doctor: nros doctor --board fvp-aemv8r-smp runs the FVP resolution
cross-check (it delegates part of its report to just doctor, so it
needs just on PATH). The just zephyr run-fvp-ws-entry /
run-fvp-board-import recipes do the equivalent inline via
scripts/zephyr/resolve-fvp-bin.sh and skip with a clear hint when the
binary can’t be found.
Build: just zephyr build-fvp-all runs the FVP build lanes.
# The workspace C++ RT-tiers Entry — the canonical ASI-consumption
# reference (nano_ros_use_board + run_tiers).
just zephyr build-fvp-ws-entry
# The minimal board-crate IMPORT surface: nano_ros_use_board() and a
# trivial printk app, so the import path can be checked on its own.
just zephyr build-fvp-board-import
Each recipe shells west build -b fvp_baser_aemv8r/fvp_aemv8r_aarch64/smp
inside the zephyr-workspace/ directory and produces zephyr.elf at:
zephyr-workspace/build-fvp-ws-entry/zephyr/zephyr.elfzephyr-workspace/build-fvp-board-import/zephyr/zephyr.elf
Run (once the build artifacts and ARM_FVP_DIR / ARMFVP_BIN_PATH are
in place):
# Boot the workspace RT-tiers Entry (prints `[ctrl] tick=` / `[telem] tick=`).
just zephyr run-fvp-ws-entry
# Boot the board-import smoke (prints `nros: smoke ok`).
just zephyr run-fvp-board-import
Under the hood the recipe:
- Verifies
west+ the Zephyr workspace +zephyr.elfexist; skips with a hint otherwise. - Resolves the FVP binary directory via
scripts/zephyr/resolve-fvp-bin.sh(priority order:ARMFVP_BIN_PATH→ARM_FVP_DIR/models/Linux64_GCC-*/→dirname $(command -v FVP_BaseR_AEMv8R)). - Exports
ARMFVP_BIN_PATH=<dir>and shellswest build -d <build-dir> -t run, which drives Zephyr’scmake/emu/armfvp.cmaketarget with the canonicalboards/arm/fvp_baser_aemv8r/board.cmake-Cflags — UART 0–3 piped to stdout, GICv3, cache state, NUM_CORES fromCONFIG_MP_MAX_NUM_CPUS. No flags are duplicated in thejustrecipe.
Exit cleanly with Ctrl-C.
ThreadX
Lanes for ThreadX (Linux sim / RISC-V64 QEMU).
just <flavour> build-fixtures produces threadx_cpp_* and
riscv64_threadx_cpp_* binaries alongside the Rust + C ones:
just threadx_linux build-fixtures # build all rust + c examples
just threadx_riscv64 build-fixtures
just threadx_riscv64 talker boots the prebuilt riscv64 talker
fixture in qemu-system-riscv64 with the virtio-net + Slirp wiring
baked in. For batch testing, just threadx_linux test runs every
pubsub / service / action against an in-test zenohd.
Cyclone DDS
Lanes for the Cyclone DDS backend
(Choosing an RMW Backend). These are
contributor-only recipes — a bare cmake / cargo consumer build
needs no just cyclonedds pre-step; the consumer build self-provisions
Cyclone from source.
just setup cyclonedds # build Cyclone DDS from third-party/dds/cyclonedds (tag 0.10.5)
just cyclonedds build-rmw # build packages/rmw/cyclonedds/nros-rmw-cyclonedds
just cyclonedds test # run the CTest harness
Serial transport (QEMU)
Lane for Serial Transport: the
integration test test_qemu_serial_pubsub_e2e in
packages/testing/nros-tests/tests/emulator.rs automates the full
QEMU serial pub/sub workflow. Run it with:
source ./activate.sh
just qemu build-fixtures # the test consumes a prebuilt
# qemu-arm-baremetal fixture
cargo nextest run -p nros-tests --test emulator test_qemu_serial_pubsub_e2e
(A bare cargo nextest counts a skipped-precondition test as a FAILURE;
the contributor lane just test-all is what rewrites those into skips.
Read the panic text before treating a red here as a regression.)
Contributing
This chapter covers the development workflow for nano-ros.
Development Setup
Choose the setup tier. base is enough for most code review and
native development; all prepares the full test-all matrix:
just setup # print setup choices
just setup base # workspace tools + zenohd
just setup all # all supported SDKs/services
source ./activate.sh
The test and fixture stages need the vendored-source UNION, not one board.
nros setup <board> --rmw <x>provisions one board+rmw’s sources — right for building an app for that board. Butjust testlinks every RMW’s-syscrate andjust build-test-fixturesresolves component graphs that path-dep platform sources (e.g.nuttx/libc) even for a native component, so the contributor build needs the whole set. Both recipes preflight it and, if a source is missing, name the exact remedy —nros setup --source <name>, ornros setup --build-sourcesto provision the union at once (issue 0390).
Build the entire workspace including examples:
just build
Use groups to inspect the command surface:
just --group main --list
just --group full-matrix --list
just --group full-matrix --list zephyr
Quality Checks
Always run just ci after completing a task that needs the full
maintainer gate. This runs the normal checks plus the full test matrix
entry points:
just ci
For a narrower local iteration, run just check and just test.
For platform-specific CI, use just <platform> ci (e.g., just freertos ci).
Testing
nano-ros has several test tiers, each with its own just recipe:
| Recipe | What it tests | External deps |
|---|---|---|
just test-unit | Unit tests (no external deps) | None |
just test-integration | Rust integration tests, no heavy QEMU/Zephyr (builds zenohd automatically) | None |
just test | test-unit + test-integration (default dev tier) | None |
just test-doc | rustdoc doctests for the nros umbrella crate | None |
just test-miri | Undefined behavior detection (standalone) | None |
just qemu test | QEMU bare-metal examples | qemu-system-arm |
just freertos test | FreeRTOS QEMU E2E | qemu-system-arm + arm-none-eabi-gcc |
just nuttx test | NuttX QEMU E2E | nightly + qemu-system-arm |
just threadx_linux test | ThreadX Linux user-space E2E | gcc |
just threadx_riscv64 test | ThreadX RISC-V QEMU E2E | qemu-system-riscv64 |
just zephyr test | Zephyr E2E (native_sim + NSOS) | west |
just esp32 test | ESP32-C3 QEMU E2E | nightly + qemu-system-riscv32 |
just test-all | Everything: test + heavy QEMU/Zephyr/ROS-interop + test-doc + test-miri + C codegen | All of the above |
All test recipes accept a verbose argument for live output.
Test organization
- Reusable Rust tests go in
packages/testing/nros-tests/tests/ - Shell-based test scripts go in
tests/with corresponding justfile entries - Temporary tests can be run directly in the shell, then converted to proper tests once validated
Build isolation
Nextest runs each test file as a separate process in parallel. When multiple tests build the same example with different features, use --target-dir to isolate output directories (e.g., target-safety/, target-zero-copy/). Add new target dirs to the example’s per-directory .gitignore.
Code Style
Rust Edition 2024
nano-ros uses Rust edition 2024, which requires:
unsafe extern "C" { ... }– extern blocks require theunsafekeyword#[unsafe(no_mangle)]–no_manglerequires theunsafeattribute- Unsafe operations inside
unsafe fnneed explicitunsafe { ... }blocks
The nros-c crate keeps #![allow(unsafe_op_in_unsafe_fn)] due to 420+ FFI operations.
Unsafe conventions
- Minimize unsafe usage; prefer safe abstractions
- Document safety invariants with
// SAFETY:comments - Use wrapper types (like
SubscriberBufferRef) to encapsulate unsafe access patterns
no_std patterns
All core crates support #![no_std] with optional std/alloc features:
#[cfg(feature = "alloc")]gatesVec/Boxusage#[cfg(feature = "std")]gates std-only featuresheapless::Vecis used inno_stdcontexts
Unused variables
- Rename to
_namewith a comment explaining why - Use
#[allow(dead_code)]for test struct fields
Message Types
Message types are always generated, never hand-written. Use the codegen tool:
nros generate-rust
Bundled interface definitions live in packages/cli/interfaces/. Example generated/ directories are gitignored and recreated by just generate-bindings. Only the core pre-generated packages under packages/interfaces/* (rcl-interfaces, lifecycle-msgs, diagnostic-msgs) check their generated/ into git.
System Packages
Never install system packages or run sudo directly. If a system dependency is needed, document what the user should install.
Temporary Files
Create temporary files in the project’s tmp/ directory (git-ignored), not in /tmp.
PR Workflow
- Create a branch from
main - Make changes
- Run
just ci– all checks must pass - Submit for review
Verification
For changes to core crates, run the formal verification suite:
just verify # Kani + Verus
just verify-kani # Kani bounded model checking (~3 min)
just verify-verus # Verus deductive proofs (~1 sec)
nros CLI reference
The nros binary is the user entry point for nano-ros: scaffolding,
message codegen, SDK provisioning, topology planning, static checks,
diagnostics, and board introspection.
The old cargo nano-ros … cargo subcommand has been removed. Use nros
directly.
Install
The nros CLI ships from the in-tree sub-workspace at packages/cli/
nano-ros is a source distribution — there is no
prebuilt nros download. One front door — bootstrap
(builds the CLI from source; installs rustup if needed), then activate
the workspace to put it on PATH:
./scripts/bootstrap.sh
source ./activate.sh # OR: direnv allow / source ./activate.fish
Equivalent, if you already have cargo (same build, same binary):
git submodule update --init packages/cli/third-party/play_launch
then cargo build --release --manifest-path packages/cli/Cargo.toml --bin nros
(bootstrap runs both for you; just setup-cli is the internal alias).
The resulting binary lives at packages/cli/target/release/nros. One
checkout = one CLI version = one runtime ABI; contributors with
multiple nano-ros worktrees get per-tree CLIs with no global PATH
skew. Once on PATH, nros --help lists every verb. The transitional
${NROS_HOME:-~/.nros}/bin/nros install location remains supported
for users mid-migration from the pre-218 release-fetch shape.
Verbs
nros setup [<board>] [--rmw <rmw>] [--tool <name>] [--source <name>] [--prefix <dir>] [--list] [--licenses] [--dry-run]
Provision the toolchain/SDK for a board. nros setup is the single
canonical provisioning command. Most components are prebuilt per platform
per RMW (cross-compiler, emulator, RMW host daemon, SDK sources), fetched
from a pinned index into a shared store (${NROS_HOME:-~/.nros}/sdk). No
hand-installed cross-toolchains; no ROS distro required.
Packages are prebuilt where the index has a binary for your host and built
from source otherwise; the vendored submodules (zenoh-pico, mbedtls,
RTOS sources) are the source builds. --dry-run prints the plan without
fetching. Full explanation:
Installation.
Note setup does not provision a zenoh router: since RFC-0075 that is
ROS’s rmw_zenoh_cpp/rmw_zenohd. See
Do I need ROS 2 installed?.
nros setup native --rmw zenoh # host build (router comes from ROS)
nros setup qemu-arm-freertos --rmw xrce # arm-none-eabi-gcc, qemu, FreeRTOS, XRCE agent
nros setup zephyr # Zephyr west workspace + SDK bits
| Flag | Effect |
|---|---|
<board> | resolve + fetch this board’s package set (see Supported Boards) |
--rmw <zenoh|xrce|cyclonedds> | also provision the RMW’s host daemon/tool (default zenoh); resolves board ∪ rmw packages |
--tool <name> | install a single tool by name (e.g. --tool qemu) |
--source <name> | provision a single source package by name (repeatable) |
--prefix <dir> | install a --tool here instead of the shared store |
--list | list every package in the index + its version |
--licenses | show license-gated packages + how to install them |
--dry-run | resolve + print the plan without fetching anything |
Board names: native, posix, qemu-arm-baremetal, mps2-an385,
stm32f4, qemu-arm-freertos, qemu-arm-nuttx, qemu-riscv64-threadx,
threadx-linux, qemu-esp32-baremetal, zephyr, and more —
run nros setup --list or nros setup <board> --dry-run.
nros init [<dir>]
Generate a project CMakePresets.json (into <dir>, default the current
directory) that includes the per-board preset fragments nros setup <board> wrote under ~/.nros/presets/ (RFC-0048 §6). After it,
cmake --preset <board> cross-configures a nano-ros ament package with no
hand-set -DCMAKE_TOOLCHAIN_FILE / -Dnano_ros_ROOT. Idempotent — re-run
after provisioning a new board to pick up its fragment.
nros setup qemu-arm-nuttx
nros init # writes ./CMakePresets.json
cmake --preset qemu-arm-nuttx
nros new <name> --platform <plat> [--rmw <rmw>] [--lang <lang>] [--use-case <case>] [--force]
Scaffold a new nano-ros project. Emits a colcon-compatible
package.xml plus a hello-world Rust / C / C++ skeleton tuned for the
chosen platform.
| Flag | Values | Default |
|---|---|---|
--platform | native, freertos, nuttx, threadx, zephyr, esp32, posix, baremetal | (required) |
--rmw | zenoh, xrce, cyclonedds | zenoh |
--lang | rust, c, cpp | rust |
--use-case | talker, listener, service, action | talker |
--force | overwrite an existing directory | off |
Image / host mode: nros new --image <name> [--board <b>] scaffolds an [image.<name>] — a buildable image (RFC-0065 D6) — into the bringup package’s system.toml; nros new --host <name> scaffolds a [host.<name>], a machine nodes run on. Both accept [--bringup <pkg>] [--from-launch <path>] [--from-profile <name>]. The bringup package is discovered automatically when the workspace exposes exactly one; pass --bringup <pkg> to pick one when there are several. --from-launch also seeds the bringup [system].default_launch; --from-profile forks an existing block of the same table.
An image names a --board and derives its rustc triple from that board’s descriptor, so there is no --target. A host takes no board: a host says WHERE nodes run, and a board is what an image is BUILT for — passing one is refused rather than written.
nros new --deploy <name> is DEPRECATED (issue 0951): [deploy.*] split into those two tables. It still works, dispatched by the --kind it always took — self scaffolds a host, anything else an image — and says which flag to use instead.
nros generate <lang> [--manifest <path>] [--output <dir>] [--ros-edition <edition>] [--force] [--verbose] [--generate-config]
Generate ROS 2 message bindings from a package.xml. Rust users should
prefer the direct nros generate-rust command; C and C++ users normally
use the CMake integration.
| Argument | Values | Default |
|---|---|---|
<lang> | rust, c, cpp, all | (required) |
--manifest | path to package.xml | package.xml |
--output | output directory | generated |
--ros-edition | humble, iron | humble |
--generate-config | emit .cargo/config.toml patches (Rust only) | off |
nros generate-rust [--manifest <path>] [--output <dir>] [--generate-config] [--nano-ros-path <p> | --nano-ros-git] [--force] [--verbose]
The Rust-only codegen primitive. Same binding generation as
nros generate rust, but with no side-effects unless asked:
--generate-config (alias --config) additionally writes the
[patch.crates-io] entries, pointing at a local checkout
(--nano-ros-path) or the GitHub repo (--nano-ros-git). Most
in-tree workflows use nros sync instead, which wraps this plus the
patch write per consumer.
nros generate-px4-msgs
Generate CDR-serializable px4_msgs::msg::* bindings from a PX4
.msg tree — the XRCE/PX4 interop path (see the
PX4 starter).
nros codegen <resolve-deps|cyclonedds-descriptors|entry> …
Internal build-tool interface — invoked by the CMake integration and
build.rs scripts, not meant for direct use. Subcommands resolve
message-package dependency closures, emit CycloneDDS type descriptors,
and generate C/C++ entry glue. (nros codegen-system below is the
user-facing system bake.)
nros metadata <system_pkg> [--workspace <path>] [--out-dir <dir>] [--metadata <existing.json>] [--build [--nano-ros-workspace <path>]] — walk a colcon-style workspace under <workspace>/src/
collecting component source metadata into
build/<system_pkg>/nros/source-metadata.json. The result feeds
nros plan. With --build, any declared component (component_nros.toml)
missing its source-metadata is produced by the metadata-mode build:
nano-ros compiles + runs the component against an in-memory recorder to
emit the JSON. --build needs the nano-ros workspace
(--nano-ros-workspace or NROS_WORKSPACE).
nros plan <system_pkg> <launch_file> [LAUNCH_ARGS...] [options] — resolve a ROS 2 launch file (or a precomputed
play_launch_parser record.json) plus the source metadata into a
typed build/<system_pkg>/nros/nros-plan.json IR. Picks per-component
SchedContext bindings, node graph wiring, parameter remaps, and
generated-package layout.
| Flag | Use |
|---|---|
--record <file> | Skip launch-file parsing; consume an existing record.json |
--workspace <path> | Override the workspace root (default $PWD) |
--out-dir <dir> | Override build/<system_pkg>/nros/ |
--metadata <file> | Reuse a prior source-metadata.json |
--manifest <file> | ROS launch manifest YAML overlay |
--nros-toml <file> | nano-ros deployment overlay (nros.toml) |
nros check [plan]
Validate an nros-plan.json (default build/nros/nros-plan.json):
static checker — catches unconnected required topics, conflicting QoS,
missing parameters, and SchedContext binding errors before the platform
build runs. A .toml argument is instead validated as a root nros.toml
(the workspace deployment SSOT) — [system]/[image.<id>]/[host.<name>] shape,
default-deploy + system references, bridge endpoints, etc.
nros explain [<plan>]
Render a generated nros-plan.json in human-readable form.
nros explain # reads build/nros/nros-plan.json
nros explain path/to/nros-plan.json # explicit path
Run after nros plan to inspect the resolved component graph, topic
wiring, parameter bindings, and SchedContext assignments before
committing to a platform build.
nros config show --system <pkg> [--workspace <dir>]
Prints the resolved effective config for a bringup system (rmw,
domain_id, locator, and the safety / param_services / lifecycle
capability axes) with a per-value provenance column — system.toml [section] vs the built-in default. If a legacy per-package nros.toml
overlay still declares any of those blocks, it is flagged DEPRECATED by
name (RFC-0004 §3.1). Omit the <pkg> value to default to the workspace’s
default_system (or the sole bringup); --workspace defaults to the cwd.
The nros check command surfaces the same overlay warnings when validating
a system.toml.
The legacy
nros config show/check --config <path>reader forconfig.tomlwas removed:config.tomlis retired (RFC-0004 §8) and no example ships one. Embedded runtime config lives in[package.metadata.nros.deploy.<t>].
nros sync
Codegen all *.msg packages + write the [patch.crates-io] config to match
the declared deps — for a standalone package or a colcon-style workspace
(picks single-pkg vs workspace mode by layout). The patch lands in each Rust
consumer’s .cargo/config.toml (never edits Cargo.toml).
Pre-cargo step; run once after editing *.msg files, then cargo build works.
eval "$(nros ws env)" # add src/ to NROS_INTERFACE_SEARCH_PATH
nros sync # codegen msg pkgs + write [patch.crates-io] → .cargo/config.toml
nros generate-rust stays the low-level codegen-only primitive (no patch side
effects).
nros ws <subcommand>
Workspace-level message-package utilities (the env / status / list /
clean / doctor helpers; sync was promoted to top-level nros sync).
eval "$(nros ws env)" # add src/ to NROS_INTERFACE_SEARCH_PATH
nros ws status # freshness check (non-fatal): n up-to-date / n stale / n missing
nros ws list # list discovered msg + Rust-consumer pkgs
nros ws clean # remove generated/ + auto-managed patch entries
nros ws doctor # lint workspace pkgs (package.xml markers, stale patches, …)
| Subcommand | Description |
|---|---|
env | Print shell export adding <dir> (default ./src) to NROS_INTERFACE_SEARCH_PATH |
status | Non-fatal freshness check — sibling of nros sync --check |
list | List discovered msg + Rust-consumer pkgs (kind, name, dir per row) |
clean | Remove generated/ + auto-managed [patch.crates-io] entries from each .cargo/config.toml; leaves user-written keys + sections alone |
doctor | Lint: warn on missing <member_of_group> rosidl_interface_packages</member_of_group>, malformed package.xml, missing nros-managed [patch.crates-io] entries in the authority .cargo/config.toml |
nros codegen-system [--bringup <pkg>] [--target <triple>] [--out <dir>] [--launch <file>] [--ahead-of-vendor <pio|px4>]
Host-time system bake: reads <bringup>/system.toml +
<bringup>/launch/system.launch.xml and emits the baked compile-time C
config + component registration glue consumed by every embedded RTOS
adapter.
nros codegen-system --bringup demo_bringup
nros codegen-system --bringup demo_bringup --target thumbv7em-none-eabihf
nros codegen-system --bringup demo_bringup --ahead-of-vendor pio # + PlatformIO library.json
nros codegen-system --bringup demo_bringup --ahead-of-vendor px4 # + PX4 module dirs
| Flag | Description |
|---|---|
--workspace <path> | Workspace root (default: cwd) |
--bringup <pkg> | Bringup pkg name or path. Defaults to [workspace.metadata.nros].default_system |
--target <triple> | Target triple for cross-compile bake context |
--out <dir> | Output directory; nros-system/ subdir created inside. Default: <workspace>/build/<bringup>/ |
--launch <file> | Multi-launch disambiguation: pick <bringup>/launch/<file> (--file is an alias) |
--exec <exec> | <node exec="…"> override for synthesised launches |
--ahead-of-vendor | pio: emit PlatformIO library.json; px4: emit one PX4-native nros_<component>/ module dir per component |
nros doctor [--platform <name>] [--workspace <path>]
Shells out to just doctor from the auto-detected workspace root.
The justfile orchestrates every per-module doctor recipe (just nuttx doctor, just zephyr doctor, …) and is the source of truth for
“healthy”. --platform <name> scopes the check to a single module.
nros board list [--workspace <path>]
Enumerate every nros-board-* crate under packages/boards/. Output
columns are name | description; structured chip | flash | ram | supported_rmw fields are deferred to a future board-descriptor TOML.
nros version
Print toolchain + library versions.
nros completions <shell>
Emit shell completion scripts to stdout.
| Shell | Install snippet |
|---|---|
bash | nros completions bash > ~/.local/share/bash-completion/completions/nros |
zsh | nros completions zsh > "${fpath[1]}/_nros" |
fish | nros completions fish > ~/.config/fish/completions/nros.fish |
powershell | nros completions powershell > $PROFILE.parent\nros.ps1 |
Comparison: nros vs. just
| Want to … | Use |
|---|---|
| Scaffold a project | nros new … |
Generate Rust bindings (+ .cargo patches) | nros sync (primitive: nros generate-rust) |
| Plan + check a multi-component system from a ROS 2 launch file | nros metadata → nros plan → nros check |
| Build / flash / run | Platform tools: cargo, cmake --build, west, idf.py, probe-rs, or focused just <platform> … recipes |
| Orchestrate the workspace (setup, doctor, CI, multi-platform sweeps) | just … |
nros routes through the nros-cli-core library. just recipes that
wrap user-flow operations call into nros for consistency; internal
recipes (build matrices, CI orchestration) keep their current shape.
Release pipeline status
There is no crates.io / Arduino zip / ESP-IDF binary / GitHub Releases
tarball channel. Per the archive decision (2026-05-19), nano-ros is
consumed by git clone --branch=v<X.Y.Z> plus the in-tree build
recipes documented at
Installation. (A hidden,
maintainer-only nros release verb exists behind the release cargo
feature; the default build does not carry it.)
Downstream RTOS package managers (Zephyr west,
ESP-IDF idf_component_yml, NuttX Kconfig) consume the same source
tree via the integration shells under integrations/<rtos>/.
Rust API
The Rust user-facing surface lives in the nros umbrella crate. Generated by
rustdoc --no-deps; always reflects the current main branch.
API reference
nrosrustdoc — start here.
That single crate re-exports every type a user application needs:
Executor, Node, Publisher<M>, Subscription<M>, Service<S>,
Client<S>, ActionServer<A>, ActionClient<A>, Timer, GuardCondition,
plus the nros::prelude glob-import.
Where to start
- New to nano-ros? →
nros::prelude - Two-layer API (L1 polling vs L2 callback)? → Two-Layer API — concept page with the verb discipline and per-layer use cases.
- Executor-driven app? →
nros::Executor - Async/
spin_async? → the async surface lives onnros::Executorand the entity handles (recv().await,call().await) — see the crate root docs
Generating locally
cargo doc --no-deps -p nros --open
Internals (porting only)
These crates are not part of the user API. They surface only when writing a custom RMW backend or porting to a new platform — see Custom RMW Backend / Custom Platform.
nros_rmw— RMW trait surface (porters)nros_platform_api— platform abstraction traits (porters)nros-rmw-zenoh— zenoh-pico backend internals
C API
The C user-facing surface lives in packages/api/nros-c/include/nros/*.h.
Generated by Doxygen; always reflects the current main branch.
API reference
C API Doxygen — start here.
The umbrella header nros/nros.h pulls in every
public surface a user application needs.
Where to start
nros/nros.h— convenience umbrella includenros/init.h—nros_support_init,nros_support_fininros/node.h—nros_node_init,nros_node_fininros/executor.h— spin loop, executor lifecyclenros/publisher.h/subscription.h— pub/subnros/service.h/client.h— servicesnros/action.h— actionsnros/parameter.h— parameter servicesnros/lifecycle.h— REP-2002 lifecycle states
Two-layer API. Every entity exposes two parallel C entry-point sets:
nros_*_init(Layer 1, caller polls) andnros_executor_register_*(Layer 2, executor callback). Layer 1 grew an_init_polling+take_*_raw/send_*_rawfamily for inline-storage callers (no executor arena). Layer 2 keeps the existingnros_*_init+ executor-register pair. See Two-Layer API for the verb discipline and per-layer trade-offs.
Event-driven wake callbacks. Each L1 polling entity supports
nros_*_set_wake_callback(entity, &state, cb, ctx)for event- driven RTOS / embassy callers. State is a caller-ownednros_wake_state_tPOD; the backend firescb(ctx)on rx / reply / request arrival. Per-channel for actions (set_{goal,cancel,get_result}_wake_callbackon the server,set_{goal_response,cancel_response,result,feedback}_wake_callbackon the client). See the Doxygen for signatures.
Async action client
<nros/action.h> exposes two parallel entry-point families for sending
goals, requesting results, and cancelling:
- Async (non-blocking):
nros_action_send_goal_async(),nros_action_get_result_async(),nros_action_cancel_goal(),nros_action_try_recv_feedback(). Return immediately. Replies are delivered through the callbacks registered withnros_action_client_set_goal_response_callback(),nros_action_client_set_feedback_callback(), andnros_action_client_set_result_callback(). The callbacks fire fromnros_executor_spin_some()— your spin loop is the only place callbacks run. - Blocking convenience:
nros_action_send_goal()andnros_action_get_result(). These call the async variant and then drivenros_executor_spin_some()internally on a wall-clock budget until the reply lands (or 15 s / 30 s timeout). They take thenros_executor_t*explicitly because the action client stores its handle as an opaque slot inside the executor (registered vianros_executor_register_action_client()). The same applies tonros_action_client_wait_for_action_server()andnros_action_client_action_server_is_ready(). Calling any of them from inside a dispatch callback returnsNROS_RET_REENTRANT.
The equivalent service client helper nros_client_call() does not
take an explicit executor — the client stashes the executor pointer
during nros_executor_register_client() and recovers it internally —
but it follows the same async-then-spin contract.
Canonical pattern: declare the executor, register the client, then
either call the blocking helper or drive nros_executor_spin_some()
yourself between async calls until your callback flags the result. See
the example layouts in examples/native/c/action-client/ and
examples/qemu-arm-freertos/c/action-client/.
Phase 122.3 added a separate L1 polling family
(nros_action_client_init_polling + the _raw send/recv siblings)
that stores ActionClientCore inline in the
nros_action_client_t._opaque slot and skips the executor entirely;
see the Doxygen for the full L1 verb list.
CMake integration
Out-of-tree projects link the NanoRos::NanoRos CMake target and
generate C bindings with nano_ros_generate_interfaces(<pkg> LANGUAGE C)
(the function defaults to LANGUAGE CPP; the full which-spelling table
is in Message Generation) —
covered in
docs/reference/c-api-cmake.md.
Generating locally
doxygen packages/api/nros-c/Doxyfile
xdg-open target/doc/c-api/html/index.html
Internals (porting only)
The rmw-cffi and platform-cffi Doxygen sites are not part of
the user API — they document the C vtables for porters writing a custom RMW
backend or platform port. See RMW API and
Platform API.
C++ API
The C++ user-facing surface lives in
packages/api/nros-cpp/include/nros/*.hpp. Generated by Doxygen;
always reflects the current main branch.
API reference
C++ API Doxygen — start here.
The umbrella header nros/nros.hpp pulls in every
public surface a user application needs.
Where to start
nros/nros.hpp— convenience umbrella includenros::init/nros::shutdown— session lifetimenros::Node— node + create_publisher/subscription/service/client/action_*nros::Publisher<M>/Subscription<M>nros::Service<S>/Client<S>nros::ActionServer<A>/ActionClient<A>— L2 callback model (executor-arena registered).nros::PollingActionServer<A>/PollingActionClient<A>— L1 polling model. Caller drivestake_*/accept_goal/complete_goalfrom a spin loop.nros::Future<T>— async result handlenros::Executor,Timer,GuardConditionnros::ParameterServer<Cap>— node-local typed parameter store (bool/int64_t/double/const char*); compose alongside aNode. See Differences from ROS 2 §9 for what is intentionally smaller thanrclcpp.
The library is freestanding C++14 — no STL, no exceptions. Define
NROS_CPP_STD if you want the optional std::string / std::function /
std::chrono overloads.
Two-layer API. The
nros::ActionServer<A>/ActionClient<A>templates are L2 callback handles (set callbacks vianros_cpp_action_server_set_callbacks, dispatched by the executor). The newnros::PollingActionServer<A>/PollingActionClient<A>are L1 polling templates with the same typed serdes glue but caller-driven control flow. Subscriptions / services / service clients are already L1-shaped in nros-cpp. See Two-Layer API for the per-layer rationale.
Event-driven wake callbacks.
PollingActionServer<A>andPollingActionClient<A>expose a nestedWakeStatePOD and typedset_{goal,cancel,get_result,goal_response, cancel_response,result,feedback}_wake_callback(state, cb, ctx)methods. Pair annros_cpp_wake_state_twith each entity / channel to wake on rx instead of polling.
Generating locally
doxygen packages/api/nros-cpp/Doxyfile
xdg-open target/doc/cpp-api/html/index.html
RMW API
The RMW (ROS middleware) vtable is the porting boundary between nano-ros and a concrete pub/sub transport (zenoh-pico, XRCE-DDS, Cyclone DDS, uORB, …). RMW is internal — user applications use the Rust / C / C++ APIs, not the vtable directly.
Canonical reference
The C vtable nros_rmw_vtable_t is the source of truth. Every
function pointer’s brief, parameter docs, ownership rules
(buffer-borrowed vs caller-owned), blocking / non-blocking
classification, return-code conventions, and lifetime contract for
loaned slots live in the Doxygen output.
To regenerate locally:
just doc-rmw-cffi # produces target/doxygen/rmw-cffi/
This page does not duplicate the interface specification — read the Doxygen for that.
Reference implementations
Concrete backends. Each crate’s README.md walks the
implementation; the source is the worked example to copy.
| Backend | Source | Notes |
|---|---|---|
| zenoh-pico | packages/rmw/zenoh/nros-rmw-zenoh | Default. C transport via zenoh-pico. Native zero-copy publish via z_bytes_from_static_buf. |
| micro-XRCE-DDS-Client | packages/rmw/xrce/nros-rmw-xrce | C-only shim; agent-based. |
| Cyclone DDS | packages/rmw/cyclonedds/nros-rmw-cyclonedds | C++ shim; standalone CMake project. |
| PX4 uORB | packages/rmw/uorb/nros-rmw-uorb | Typed-trampoline registry over PX4 uORB. |
The zenoh-pico shim is the canonical reference port.
Writing a custom backend
- Conceptual walkthrough: Custom RMW Backend.
- Coming from upstream
rmw.h? → RMW API: Differences from upstreamrmw.h. - Coverage status vs upstream
rmw.h:docs/research/rmw-c-abi-coverage.md.
Platform API
The platform API is the porting boundary between nano-ros and a concrete OS / RTOS / bare-metal target. Each platform provides a clock, optionally a heap, optionally threading, optionally networking. Platform is internal — user applications use the Rust / C / C++ APIs, not the platform vtable directly.
Canonical reference
The SSoT C headers in
packages/platform/nros-platform-api/include/nros/ (platform.h +
platform_net.h, platform_timer.h, platform_zephyr.h; RFC-0054 —
Rust consumes committed bindgen output from them) are the source of
truth. Every symbol’s brief, parameter docs, ownership rules,
blocking / non-blocking classification, and ISR-safe contract live in
the Doxygen compiled from those headers.
To regenerate locally:
just doc-platform-cffi # produces target/doxygen/platform-cffi/
This page does not duplicate the interface specification — read the Doxygen for that.
Reference implementations
Each row is a complete worked example. The crate’s README.md
walks the implementation; the source is the worked solution to
copy.
| Crate | Target | Source |
|---|---|---|
nros-platform-posix | Linux / *BSD | packages/platform/nros-platform-posix |
nros-platform-nuttx | NuttX RTOS | packages/platform/nros-platform-nuttx |
nros-platform-freertos | FreeRTOS | packages/platform/nros-platform-freertos |
nros-platform-threadx | Azure RTOS / ThreadX | packages/platform/nros-platform-threadx |
nros-platform-zephyr | Zephyr RTOS | packages/platform/nros-platform-zephyr |
nros-platform-mps2-an385 | Cortex-M3 (QEMU) | packages/platform/nros-platform-mps2-an385 |
nros-platform-stm32f4 | STM32F4 | packages/platform/nros-platform-stm32f4 |
nros-platform-esp32-qemu | ESP32-C3 (QEMU) | packages/platform/nros-platform-esp32-qemu |
The POSIX implementation is the canonical reference port.
Writing a custom platform
- Conceptual walkthrough: Custom Platform.
- Per-platform behaviour matrix: Platform Differences.
Platform Differences
This page covers OS-port differences (clock, allocator, threading, sockets). For per-RMW feature coverage — services, events, QoS — see the Per-RMW Feature Matrix.
Per-platform behaviour comparison. The Doxygen reference defines what each vtable entry must do; this page covers how each shipped platform crate fulfils that contract and which optional capabilities each platform exposes.
At a glance
| Platform | Clock source | Allocator | Threading | Networking | UDP multicast | Wall-clock | Notes |
|---|---|---|---|---|---|---|---|
| POSIX | clock_gettime(CLOCK_MONOTONIC) | libc malloc | pthreads | libc BSD sockets (Rust) | Yes | clock_gettime(CLOCK_REALTIME) | Canonical reference port. |
| NuttX | POSIX alias | libc malloc | pthreads | zenoh-pico unix/network.c (C) | Yes | POSIX | Most paths inherit POSIX behaviour. |
| FreeRTOS | xTaskGetTickCount | pvPortMalloc | FreeRTOS tasks | lwIP via freertos-lwip-sys (Rust) | — | Tick-based, no RTC | Multicast gated on lwIP LWIP_IGMP=1 (untested). |
| Zephyr | k_uptime_get | k_malloc | Zephyr POSIX pthreads | Zephyr POSIX sockets (Rust) | Yes (native_sim via NSOS) | clock_gettime(CLOCK_REALTIME) | native_sim multicast routes through the NSOS IPPROTO_IP patch. |
| ThreadX | tx_time_get | tx_byte_allocate | ThreadX threads | NetX Duo BSD network.c (C) | — | tx_time_get fallback | Multicast gated on NetX Duo nx_igmp_* (untested). |
| Bare-metal (MPS2-AN385) | CMSDK Timer0 | bump allocator | single-threaded | nros-smoltcp (Rust) | Yes (smoltcp 0.12 IGMP) | monotonic fallback | Cortex-M3 QEMU. |
| Bare-metal (STM32F4) | DWT cycle counter | bump allocator | single-threaded | nros-smoltcp (Rust) | Yes | monotonic fallback | Cortex-M4F. |
| Bare-metal (ESP32-S3) | esp_timer_get_time | bump allocator | single-threaded | nros-smoltcp (Rust) | Yes | monotonic fallback | Xtensa LX7 (build-only board crate). |
| Bare-metal (ESP32-C3 QEMU) | esp_timer_get_time | bump allocator | single-threaded | nros-smoltcp (Rust) | Yes | monotonic fallback | RISC-V. |
Canonical interface contract. What “Yes” / “—” / specific functions mean — buffer ownership, blocking allowance, valid value ranges — is in the platform-cffi Doxygen reference. This page is a comparison table, not a spec.
Behaviour notes by trait group
Time (PlatformClock)
All ports surface a monotonic clock. Resolution varies:
- POSIX / NuttX: nanosecond-resolution
clock_gettime, exposed at ms + µs. - FreeRTOS:
xTaskGetTickCountticks (typically 1 ms).clock_us=clock_ms × 1000— the resolution lie is documented. - Zephyr:
k_uptime_getms +k_cyc_to_us_floor64(k_cycle_get_64())µs. - ThreadX:
tx_time_getticks (default 100 Hz; bump viaTX_TIMER_TICKS_PER_SECONDfor finer resolution). - Bare-metal (Cortex-M): hardware timer counter (CMSDK Timer0, DWT cycle counter). µs is the native resolution; ms is
µs / 1000. - Bare-metal (ESP32):
esp_timer_get_timereturns µs natively.
The clock is monotonic and wraparound-free for the duration of nros::init → nros::shutdown. Platforms with 32-bit timers run a software extender on overflow.
Memory (PlatformAlloc)
Only zenoh-pico calls PlatformAlloc::{alloc, realloc, free}. XRCE-DDS does not allocate. Recommended heap budget: 64 KB minimum for zenoh-pico’s working set; bare-metal ports typically allocate 128 KB+ via a bump allocator (linked-list-allocator or embedded-alloc).
The RTOS allocators (pvPortMalloc, tx_byte_allocate, k_malloc, libc malloc) honour their respective heap-region configurations. On bare-metal the heap region is defined in the linker script and consumed at startup by the bump allocator.
Threading (PlatformThreading)
Multi-threaded ports (POSIX, NuttX, FreeRTOS, Zephyr, ThreadX) expose real tasks, mutexes, and condition variables. zenoh-pico’s lease task and read task spawn into these. Single-threaded bare-metal ports stub task_init to return -1 so the spawn fails gracefully; the application drives spin_once from a single context.
Recursive mutexes are required by zenoh-pico. FreeRTOS uses xSemaphoreCreateRecursiveMutex; ThreadX uses tx_mutex_create(..., TX_INHERIT, ...); POSIX uses PTHREAD_MUTEX_RECURSIVE.
Sleep / Random / Wall-time (PlatformSleep, PlatformRandom, PlatformTime)
PlatformSleep on RTOS ports uses the kernel’s tickless sleep (vTaskDelay, tx_thread_sleep, k_msleep, libc usleep). On bare-metal with smoltcp, the sleep helper also runs network_poll() in a busy loop — packets must keep flowing during otherwise-idle waits or the zenoh lease times out.
PlatformRandom is a 32-bit xorshift PRNG seeded with hardware entropy (getrandom, RNG peripheral) or a build-time constant on platforms without entropy. Used for session IDs and protocol nonces; not cryptographic.
PlatformTime returns wall-clock for log timestamps. Bare-metal platforms with no RTC fall back to the monotonic clock — log timestamps then count from boot, not Unix epoch.
Networking (PlatformTcp, PlatformUdp, PlatformSocketHelpers, PlatformUdpMulticast, PlatformNetworkPoll)
The split into four traits maps directly onto zenoh-pico’s unix/network.c interface. Each port wires the four traits independently:
- C-backed networking (NuttX, ThreadX): the platform shim forwards into zenoh-pico’s bundled
unix/network.c(NuttX) or a NetX-Duo equivalent (ThreadX) — the C code talks to BSD-shape sockets directly. - Rust-backed networking (POSIX, FreeRTOS, Zephyr, bare-metal): the platform crate implements TCP / UDP in Rust. POSIX wraps libc, FreeRTOS wraps lwIP via
freertos-lwip-sys, Zephyr wraps the POSIX layer, bare-metal goes throughnros-smoltcp.
PlatformUdpMulticast (RTPS SPDP, zenoh scouting) ships fully wired on POSIX, NuttX, Zephyr, and bare-metal (smoltcp 0.12 IGMP group join). FreeRTOS and ThreadX have no multicast yet — gated by lwIP’s IGMP=1 (untested) and NetX Duo’s nx_igmp_* (untested).
PlatformNetworkPoll is bare-metal only. The implementation advances the smoltcp state machine; without it, smoltcp would only receive when the application explicitly asked. PlatformSleep and the wait_event helper both run network_poll() while waiting so packets continue flowing.
Libc (PlatformLibc)
zenoh-pico uses strlen, memcpy, errno, etc. directly. Bare-metal targets that link picolibc or newlib-nano satisfy these for free; the trait exists for no_std targets without a C runtime, which forward to Rust implementations.
Common pitfalls
For platform-specific gotchas (DMA buffer placement, ephemeral port conflicts, picolibc errno TLS, QEMU clock synchronization, Z_FEATURE_INTEREST mutex exhaustion, etc.), see Platform Porting Pitfalls.
For the design rationale (why these trait groups, why both ms and µs, why split PlatformNetworkPoll), see Platform API Design.
For writing a new port, see Custom Platform. For the canonical interface spec, see the Doxygen reference.
Support Status
What you get, at which version, tested how hard — the analogue of
Rust’s platform-support page. Version strings on this page are
parsed from nros-sdk-index.toml and rust-toolchain.toml (the
pinning SSoTs), so they cannot drift from what nros setup
actually installs.
CI tiers — what “supported” means
| Tier | Lane | Runs | Guarantee |
|---|---|---|---|
| 1 | ci (every change) | Host-native builds + native fixtures, all gates | The logic and the seams are sound. Says nothing about embedded targets. |
| 2 | ci-matrix (when core/codegen/cmake move) | 1-wise cover: every platform, language, RMW, and kind appears at least once (~28 % of the coordinate space) | Each axis value builds and runs somewhere — but pairings are NOT covered. |
| 2-nightly | ci-matrix-nightly | Pairwise cover (~70 %) | Platform×language and RMW×language interactions surface here, within a day. |
| 3 | ci-full (pre-release) | The whole matrix | Everything the tree claims, verified together. |
Per-board tier membership is generated on its own page: Board Support Tiers. The hand-maintained procurement view (including boards with no in-tree crate) is Supported Boards.
RTOS / platform source pins
nros setup <board> fetches exactly these (shared store
${NROS_HOME:-~/.nros}/sdk):
| Component | Pinned version | Used by |
|---|---|---|
| FreeRTOS kernel | 10.6.2 | FreeRTOS boards (QEMU MPS2-AN385, overlays) |
| lwIP | 2.2.0 | FreeRTOS networking |
| ThreadX | 6.4.1 | ThreadX (Linux sim + QEMU RISC-V64) |
| NetX Duo | 6.4.x | ThreadX networking |
| NuttX kernel | 12.13.0 | NuttX (QEMU armv7a / rv32) |
| Zephyr SDK | 0.16.8 (+ 1.0.1 for the 4.4 rolling line) | Zephyr module builds |
| mbedTLS | 3.x | zenoh TLS transport |
These pins are decisions, not lags — they move only with a reason, tested together (see Integrating into a Vendored Tree).
RMW backends — versions and interop pairing
| Backend | Client side (pinned) | Host side | Pairing rule |
|---|---|---|---|
| Zenoh | zenoh-pico 1.7.2 (vendored submodule) | Router = the one ROS ships: ros2 run rmw_zenoh_cpp rmw_zenohd — nano-ros deliberately vendors no router (RFC-0075) | zenoh-pico’s wire protocol is stable across zenoh 1.x, so the pinned client interops with a newer ROS-shipped router (issue 0291) |
| XRCE-DDS | Micro-XRCE-DDS client 2.4.3-nros1 + Micro-CDR 2.0.x | Agent 2.4.3-nros1 (installed by nros setup … --rmw xrce) | Client and agent are pinned to the same upstream release |
| Cyclone DDS | fork 0.10.5-nros1 — upstream 0.10.5 + carried patches (fork delta) | Talks RTPS directly to rmw_cyclonedds_cpp | Pinned to the Cyclone ROS ships (ros-humble-cyclonedds 0.10.5) — upstream 11.x is deliberately NOT a rebase target until a distro migration (issue 0507) |
Per-feature coverage across the three backends: Per-RMW Feature Matrix.
ROS 2 editions
| Edition | Feature / flag | Status | Notes |
|---|---|---|---|
| Humble | ros-humble (default) | Supported | TypeHashNotSupported key expressions, XCDR1. No rmw_zenoh_cpp apt package — zenoh interop on Humble needs a source overlay, or use Cyclone DDS. |
| Iron | ros-iron | Supported | RIHS01 type hashes. Same zenoh-apt caveat as Humble. |
| Jazzy | ros-jazzy | Supported | RIHS01; ships ros-jazzy-rmw-zenoh-cpp — the zenoh-interop reference edition. XCDR2/appendable is tracked separately (phase-303). |
Pick the edition of the ROS system you interop with — mismatched editions discover nothing (different key expressions).
Toolchains
| Tool | Pinned version |
|---|---|
| Rust | stable channel (rust-toolchain.toml); pinned cross targets: thumbv7em-none-eabihf, armv7r-none-eabihf, aarch64-unknown-none. rustfmt runs on nightly (nightly-only options). |
Arm GCC (arm-none-eabi) | 13.2-nros1 |
RISC-V GCC (riscv-none-elf) | 14.2-nros1 |
| QEMU (patched, Arm) | 11.0.0-nros2 |
| QEMU (Espressif fork, ESP32-C3) | esp-develop-nros1 |
| espflash | 4.5.0-nros1 |
| Corrosion (CMake↔Cargo) | 0.6.1-nros1 (≥ 0.6.0 required — older versions share one build dir across workspace roots and break the mixed link) |
| OpenOCD | 0.12.0-nros1 |
| sccache | 0.8.2-nros1 |
-nros<N> suffixes mark builds re-packaged (or patched) for the SDK
store; the base version is the upstream release.
Related
- Install — how these get onto
a machine (
nros setup, prebuilt-vs-source rules) - Per-RMW Feature Matrix
- Board Support Tiers (generated) · Supported Boards (procurement view)
- Integrating into a Vendored Tree — pinning and upgrading these from a company tree
RMW API — every upstream symbol, side by side
What ROS 2’s rmw asks an implementation for, what nano-ros provides, and
the reason wherever the two differ. Derived from three sources — the ROS 2
signature extract, the nano-ros ABI headers, and the authored reason map
docs/reference/rmw-api-map.toml that just check rmw-api-parity also reads —
so a slot changing shape moves this page in the same commit or fails the gate.
For the prose rationale behind the big divergences, see
RMW API: Differences from upstream rmw.h.
How to read a row
The signature says which surface it is on. nano-ros answers an upstream symbol in one of two ways, and C syntax already distinguishes them:
rmw_ret_t (*count_publishers)(...) // vtable slot — a function pointer,
// may differ per backend
rmw_ret_t rmw_compare_gids_equal(...) // global — a plain exported function,
// defined once for the image
Slot names drop the rmw_ prefix because that is genuinely the name in
nros_rmw_vtable_t; showing rmw_count_publishers on the right would flatter
the comparison.
The chip says what we DID with the symbol, which is a different question from what the signature shows:
| chip | means |
|---|---|
● same | identical signature and name |
● re-shaped | one slot, one symbol, different signature |
◆ re-mapped | answered, but NOT 1:1 — decomposed, merged, or off this seam. The arrows under it name what provides the capability |
✕ not-supported | a decision, permanent; the reason names the constraint |
○ not-implemented | a gap, with the issue tracking it |
same and re-shaped are derived from the signatures, so the map cannot
assert a match the types deny. The other three are authored, because no
signature can say whether an absent symbol was decomposed or dropped — and
not-implemented must name an issue, so silence cannot turn a gap into a
decision.
Marks show the difference. Red — upstream takes it, we do not. Green — we
take it, upstream does not. Yellow — the name differs from the mechanical one
(upstream minus rmw_). A row with no marks is identical on both sides and
carries no reason, because there is nothing to explain.
Argument names appear on the right only. The ROS 2 side is read from a
signature extract that drops parameter names on purpose — a renamed argument is
not an ABI difference, and reporting one trains people to skim. Ours are kept
because they are the only place a reader learns what an argument means:
size_t twice in a row is not a signature anyone can act on. The comparison
itself runs on types, so a name never colours a row.
An empty right-hand cell reads rejected when the symbol is deliberately absent, or answered elsewhere when the capability ships outside the RMW seam — in the executor, in codegen, or inside a backend. Both carry the reason.
A dimmed right-hand cell is an inert slot: declared in the vtable, written and read by nothing. A reserved shape, not a working capability (issue 0800).
What is being compared
Not rmw.h against rmw_vtable.h — that comparison is wrong twice. Upstream
declares 177 RMW_PUBLIC functions, but most are utilities rmw itself
defines; an implementation links those, so comparing against 177 manufactures
~90 phantom gaps. And our vtable is only the backend seam: plenty of what
upstream calls rmw lives one layer up in the executor, one layer down in a
backend, or in codegen.
So the contract is empirical — the 88 rmw_* symbols that
librmw_fastrtps_cpp.so and librmw_zenoh_cpp.so both define. Two independent
implementations with identical symbol sets is a better definition of “what an
rmw must provide” than any reading of the headers.
| vtable | global | executor | platform | build time | serdes | runtime | none | total | |
|---|---|---|---|---|---|---|---|---|---|
| same | 12 | 1 | 13 | ||||||
| re-shaped | 33 | 1 | 34 | ||||||
| re-mapped | 16 | 4 | 1 | 3 | 3 | 1 | 28 | ||
| not supported — by decision | 13 | 13 | |||||||
| total | 61 | 2 | 4 | 1 | 3 | 3 | 1 | 13 | 88 |
Read a row for what we did, a column for where it lives. Only not implemented should shrink over time; not supported is the one line that is a decision rather than a state, so it is expected to stay.
Every contract symbol
| ROS 2 | nano-ros | reason |
|---|---|---|
rmw_ret_t rmw_borrow_loaned_message( const rmw_publisher_t *, const rosidl_message_type_support_t *, void * * ) |
rmw_ret_t (*borrow_loaned_message)( const rmw_publisher_t *publisher, size_t requested_len, rmw_mut_byte_span_t *out_slot, rmw_loan_token_t **out_token ) |
● re-shaped
types are resolved at BUILD time — Upstream resolves a type through a runtime typesupport pointer. Ours are resolved by codegen before the image exists, so the seam carries bytes and a type hash rather than a pointer to a type description that would have to be walked at runtime.bytes, not an untyped pointer — Upstream's `void *` is the message, interpreted through the typesupport pointer beside it. With types resolved at build time there is nothing to interpret it WITH, so the seam carries an explicit byte range and its length — which is also what makes the buffer's capacity checkable at the call. |
rmw_ret_t rmw_client_request_publisher_get_actual_qos( const rmw_client_t *, rmw_qos_profile_t * ) |
rmw_ret_t (*client_request_publisher_get_actual_qos)( const rmw_client_t *client, rmw_qos_profile_t *qos ) |
● same
|
rmw_ret_t rmw_client_response_subscription_get_actual_qos( const rmw_client_t *, rmw_qos_profile_t * ) |
rmw_ret_t (*client_response_subscription_get_actual_qos)( const rmw_client_t *client, rmw_qos_profile_t *qos ) |
● same
|
rmw_ret_t rmw_client_set_on_new_response_callback( rmw_client_t *, rmw_event_callback_t, const void * ) |
— |
✕ not-supported · by decision
as rmw_service_set_on_new_request_callback (issue 0960). |
rmw_ret_t rmw_compare_gids_equal( const rmw_gid_t *, const rmw_gid_t *, bool * ) |
rmw_ret_t rmw_compare_gids_equal( const rmw_gid_t *gid1, const rmw_gid_t *gid2, bool *result ) |
● same
|
rmw_ret_t rmw_context_fini(rmw_context_t *) |
rmw_ret_t (*destroy_session)(rmw_session_t *session) |
◆ re-mapped · 2 upstream → 1 slot
renamed — the slot is destroy_session.the SESSION is the seam — As `rmw_node_t`: upstream's context is the process-wide init state, and ours is the session the slot is being called on. |
rmw_ret_t rmw_count_publishers( const rmw_node_t *, const char *, size_t * ) |
rmw_ret_t (*count_publishers)( const rmw_session_t *session, const char *topic_name, size_t *count ) |
● re-shaped
the SESSION is the seam — Upstream passes the node into almost every call. Our vtable is scoped to a session handle that already knows its node, so re-passing it would ask the caller to carry an identity the callee holds — the same argument that decided `handle-owns-node` in the C API, one layer down. |
rmw_ret_t rmw_count_subscribers( const rmw_node_t *, const char *, size_t * ) |
rmw_ret_t (*count_subscribers)( const rmw_session_t *session, const char *topic_name, size_t *count ) |
● re-shaped
the SESSION is the seam — Upstream passes the node into almost every call. Our vtable is scoped to a session handle that already knows its node, so re-passing it would ask the caller to carry an identity the callee holds — the same argument that decided `handle-owns-node` in the C API, one layer down. |
rmw_client_t * rmw_create_client( const rmw_node_t *, const rosidl_service_type_support_t *, const char *, const rmw_qos_profile_t * ) |
rmw_ret_t (*create_client)( const rmw_node_t *node, const rmw_service_type_support_t *type_support, const char *service_name, uint32_t domain_id, const rmw_qos_profile_t *qos, rmw_client_t *out ) |
● re-shaped
types are resolved at BUILD time — As the message typesupport. |
rmw_guard_condition_t * rmw_create_guard_condition(rmw_context_t *) |
— |
◆ re-mapped · executor
executor: EntryKind::GuardCondition
|
rmw_node_t * rmw_create_node( rmw_context_t *, const char *, const char * ) |
rmw_ret_t (*create_node)( rmw_session_t *session, const char *name, const char *namespace_, rmw_node_t *out ) |
● re-shaped
the SESSION is the seam — As `rmw_node_t`: upstream's context is the process-wide init state, and ours is the session the slot is being called on. |
rmw_publisher_t * rmw_create_publisher( const rmw_node_t *, const rosidl_message_type_support_t *, const char *, const rmw_qos_profile_t *, const rmw_publisher_options_t * ) |
rmw_ret_t (*create_publisher)( const rmw_node_t *node, const rmw_message_type_support_t *type_support, const char *topic_name, uint32_t domain_id, const rmw_qos_profile_t *qos, const rmw_publisher_options_t *options, rmw_publisher_t *out ) |
● re-shaped
types are resolved at BUILD time — Upstream resolves a type through a runtime typesupport pointer. Ours are resolved by codegen before the image exists, so the seam carries bytes and a type hash rather than a pointer to a type description that would have to be walked at runtime. |
rmw_service_t * rmw_create_service( const rmw_node_t *, const rosidl_service_type_support_t *, const char *, const rmw_qos_profile_t * ) |
rmw_ret_t (*create_service)( const rmw_node_t *node, const rmw_service_type_support_t *type_support, const char *service_name, uint32_t domain_id, const rmw_qos_profile_t *qos, rmw_service_t *out ) |
● re-shaped
types are resolved at BUILD time — As the message typesupport. |
rmw_subscription_t * rmw_create_subscription( const rmw_node_t *, const rosidl_message_type_support_t *, const char *, const rmw_qos_profile_t *, const rmw_subscription_options_t * ) |
rmw_ret_t (*create_subscription)( const rmw_node_t *node, const rmw_message_type_support_t *type_support, const char *topic_name, uint32_t domain_id, const rmw_qos_profile_t *qos, const rmw_subscription_options_t *options, rmw_subscription_t *out ) |
● re-shaped
types are resolved at BUILD time — Upstream resolves a type through a runtime typesupport pointer. Ours are resolved by codegen before the image exists, so the seam carries bytes and a type hash rather than a pointer to a type description that would have to be walked at runtime. |
rmw_wait_set_t * rmw_create_wait_set( rmw_context_t *, size_t ) |
— |
◆ re-mapped · executor
executor: the arena entry table, allocated once
|
rmw_ret_t rmw_deserialize( const rmw_serialized_message_t *, const rosidl_message_type_support_t *, void * ) |
— |
◆ re-mapped · 1 → 2
nros-serdes: Deserialize / DeserializeView
codegen packs
|
rmw_ret_t rmw_destroy_client( rmw_node_t *, rmw_client_t * ) |
rmw_ret_t (*destroy_client)(rmw_client_t *client) |
● re-shaped
the SESSION is the seam — Upstream passes the node into almost every call. Our vtable is scoped to a session handle that already knows its node, so re-passing it would ask the caller to carry an identity the callee holds — the same argument that decided `handle-owns-node` in the C API, one layer down. |
rmw_ret_t rmw_destroy_guard_condition(rmw_guard_condition_t *) |
— |
◆ re-mapped · executor
executor: EntryKind::GuardCondition
|
rmw_ret_t rmw_destroy_node(rmw_node_t *) |
rmw_ret_t (*destroy_node)(rmw_node_t *node) |
● same
|
rmw_ret_t rmw_destroy_publisher( rmw_node_t *, rmw_publisher_t * ) |
rmw_ret_t (*destroy_publisher)(rmw_publisher_t *publisher) |
● re-shaped
the SESSION is the seam — Upstream passes the node into almost every call. Our vtable is scoped to a session handle that already knows its node, so re-passing it would ask the caller to carry an identity the callee holds — the same argument that decided `handle-owns-node` in the C API, one layer down. |
rmw_ret_t rmw_destroy_service( rmw_node_t *, rmw_service_t * ) |
rmw_ret_t (*destroy_service)(rmw_service_t *server) |
● re-shaped
the SESSION is the seam — Upstream passes the node into almost every call. Our vtable is scoped to a session handle that already knows its node, so re-passing it would ask the caller to carry an identity the callee holds — the same argument that decided `handle-owns-node` in the C API, one layer down. |
rmw_ret_t rmw_destroy_subscription( rmw_node_t *, rmw_subscription_t * ) |
rmw_ret_t (*destroy_subscription)(rmw_subscription_t *subscription) |
● re-shaped
the SESSION is the seam — Upstream passes the node into almost every call. Our vtable is scoped to a session handle that already knows its node, so re-passing it would ask the caller to carry an identity the callee holds — the same argument that decided `handle-owns-node` in the C API, one layer down. |
rmw_ret_t rmw_destroy_wait_set(rmw_wait_set_t *) |
— |
◆ re-mapped · executor
executor: the arena entry table, allocated once
|
rmw_ret_t rmw_event_set_callback( rmw_event_t *, rmw_event_callback_t, const void * ) |
rmw_ret_t (*publisher_event_init)( const rmw_publisher_t *publisher, rmw_event_type_t kind, uint32_t deadline_ms, rmw_status_event_callback_t cb, void *user_context ) |
◆ re-mapped · 2 upstream → 1 slot
renamed — the slot is publisher_event_init.events are a slot family, not an event object — Upstream models an event as an object you take from. Ours are per-kind slots on the session (`take_event` and the event mask in `rmw_event.h`), so the object that would carry the kind is the slot itself. one wake callback per session — Upstream installs a callback per entity. Ours is `set_wake_callback` on the session: one executor drives several backends, so the wake has to be attributable to a session rather than to whichever entity fired. bytes, not an untyped pointer — Upstream's `void *` is the message, interpreted through the typesupport pointer beside it. With types resolved at build time there is nothing to interpret it WITH, so the seam carries an explicit byte range and its length — which is also what makes the buffer's capacity checkable at the call. |
bool rmw_feature_supported(rmw_feature_t) |
bool (*feature_supported)(rmw_feature_t feature) |
◆ re-mapped
per-capability slots, answered by nullity or a dedicated probe
|
rmw_ret_t rmw_fini_publisher_allocation(rmw_publisher_allocation_t *) |
— |
✕ not-supported · by decision
as above |
rmw_ret_t rmw_fini_subscription_allocation(rmw_subscription_allocation_t *) |
— |
✕ not-supported · by decision
as above |
rmw_ret_t rmw_get_client_names_and_types_by_node( const rmw_node_t *, rcutils_allocator_t *, const char *, const char *, rmw_names_and_types_t * ) |
rmw_ret_t (*get_client_names_and_types_by_node)( const rmw_session_t *session, const char *node_name, const char *node_namespace, rmw_names_and_types_visitor_t visitor ) |
● re-shaped
the SESSION is the seam — Upstream passes the node into almost every call. Our vtable is scoped to a session handle that already knows its node, so re-passing it would ask the caller to carry an identity the callee holds — the same argument that decided `handle-owns-node` in the C API, one layer down.no allocator at this seam — This ABI does not allocate. Upstream hands an `rcutils_allocator_t` so the implementation can size and own the result; ours writes into a caller-owned byte range whose capacity the caller already knows. An allocator parameter cannot cross a seam that has nothing to give it. visitor, not an owning out-param — Upstream fills a heap-owning out-parameter the caller must `fini`. We pass a VISITOR callback invoked once per entry, with borrowed strings valid for the call. No allocation, bounded work, and nothing for a caller to leak — which is what makes the graph family available on a target with no allocator at all. |
rmw_ret_t rmw_get_gid_for_publisher( const rmw_publisher_t *, rmw_gid_t * ) |
rmw_ret_t (*get_gid_for_publisher)( const rmw_publisher_t *publisher, rmw_gid_t *gid ) |
● same
|
const char * rmw_get_implementation_identifier(void) |
const char * (*get_implementation_identifier)(void) |
◆ re-mapped · runtime
runtime: the registry name
|
rmw_ret_t rmw_get_node_names( const rmw_node_t *, rcutils_string_array_t *, rcutils_string_array_t * ) |
rmw_ret_t (*get_node_names)( const rmw_session_t *session, rmw_node_visitor_t visitor ) |
◆ re-mapped · 2 upstream → 1 slot
the SESSION is the seam — Upstream passes the node into almost every call. Our vtable is scoped to a session handle that already knows its node, so re-passing it would ask the caller to carry an identity the callee holds — the same argument that decided `handle-owns-node` in the C API, one layer down.visitor, not an owning out-param — As `rmw_names_and_types_t` — an owning array upstream, a visitor here. |
rmw_ret_t rmw_get_node_names_with_enclaves( const rmw_node_t *, rcutils_string_array_t *, rcutils_string_array_t *, rcutils_string_array_t * ) |
rmw_ret_t (*get_node_names)( const rmw_session_t *session, rmw_node_visitor_t visitor ) |
◆ re-mapped · 2 upstream → 1 slot
renamed — the slot is get_node_names.the SESSION is the seam — Upstream passes the node into almost every call. Our vtable is scoped to a session handle that already knows its node, so re-passing it would ask the caller to carry an identity the callee holds — the same argument that decided `handle-owns-node` in the C API, one layer down. visitor, not an owning out-param — As `rmw_names_and_types_t` — an owning array upstream, a visitor here. |
rmw_ret_t rmw_get_publisher_names_and_types_by_node( const rmw_node_t *, rcutils_allocator_t *, const char *, const char *, bool, rmw_names_and_types_t * ) |
rmw_ret_t (*get_publisher_names_and_types_by_node)( const rmw_session_t *session, const char *node_name, const char *node_namespace, bool no_demangle, rmw_names_and_types_visitor_t visitor ) |
● re-shaped
the SESSION is the seam — Upstream passes the node into almost every call. Our vtable is scoped to a session handle that already knows its node, so re-passing it would ask the caller to carry an identity the callee holds — the same argument that decided `handle-owns-node` in the C API, one layer down.no allocator at this seam — This ABI does not allocate. Upstream hands an `rcutils_allocator_t` so the implementation can size and own the result; ours writes into a caller-owned byte range whose capacity the caller already knows. An allocator parameter cannot cross a seam that has nothing to give it. visitor, not an owning out-param — Upstream fills a heap-owning out-parameter the caller must `fini`. We pass a VISITOR callback invoked once per entry, with borrowed strings valid for the call. No allocation, bounded work, and nothing for a caller to leak — which is what makes the graph family available on a target with no allocator at all. |
rmw_ret_t rmw_get_publishers_info_by_topic( const rmw_node_t *, rcutils_allocator_t *, const char *, bool, rmw_topic_endpoint_info_array_t * ) |
rmw_ret_t (*get_publishers_info_by_topic)( const rmw_session_t *session, const char *topic_name, bool no_mangle, rmw_topic_endpoint_info_visitor_t visitor ) |
● re-shaped
the SESSION is the seam — Upstream passes the node into almost every call. Our vtable is scoped to a session handle that already knows its node, so re-passing it would ask the caller to carry an identity the callee holds — the same argument that decided `handle-owns-node` in the C API, one layer down.no allocator at this seam — This ABI does not allocate. Upstream hands an `rcutils_allocator_t` so the implementation can size and own the result; ours writes into a caller-owned byte range whose capacity the caller already knows. An allocator parameter cannot cross a seam that has nothing to give it. visitor, not an owning out-param — `rmw_topic_endpoint_info_array_t` and `rmw_network_flow_endpoint_array_t` are heap-owning arrays with their own `fini`. Visited one entry at a time instead. |
const char * rmw_get_serialization_format(void) |
const char * (*get_serialization_format)(void) |
◆ re-mapped
vtable: get_serialization_format, per session
|
rmw_ret_t rmw_get_serialized_message_size( const rosidl_message_type_support_t *, const rosidl_runtime_c__Sequence__bound *, size_t * ) |
— |
◆ re-mapped · serdes
nros-serdes: size.rs (size_bound / max_serialized_size)
|
rmw_ret_t rmw_get_service_names_and_types( const rmw_node_t *, rcutils_allocator_t *, rmw_names_and_types_t * ) |
rmw_ret_t (*get_service_names_and_types)( const rmw_session_t *session, rmw_names_and_types_visitor_t visitor ) |
● re-shaped
the SESSION is the seam — Upstream passes the node into almost every call. Our vtable is scoped to a session handle that already knows its node, so re-passing it would ask the caller to carry an identity the callee holds — the same argument that decided `handle-owns-node` in the C API, one layer down.no allocator at this seam — This ABI does not allocate. Upstream hands an `rcutils_allocator_t` so the implementation can size and own the result; ours writes into a caller-owned byte range whose capacity the caller already knows. An allocator parameter cannot cross a seam that has nothing to give it. visitor, not an owning out-param — Upstream fills a heap-owning out-parameter the caller must `fini`. We pass a VISITOR callback invoked once per entry, with borrowed strings valid for the call. No allocation, bounded work, and nothing for a caller to leak — which is what makes the graph family available on a target with no allocator at all. |
rmw_ret_t rmw_get_service_names_and_types_by_node( const rmw_node_t *, rcutils_allocator_t *, const char *, const char *, rmw_names_and_types_t * ) |
rmw_ret_t (*get_service_names_and_types_by_node)( const rmw_session_t *session, const char *node_name, const char *node_namespace, rmw_names_and_types_visitor_t visitor ) |
● re-shaped
the SESSION is the seam — Upstream passes the node into almost every call. Our vtable is scoped to a session handle that already knows its node, so re-passing it would ask the caller to carry an identity the callee holds — the same argument that decided `handle-owns-node` in the C API, one layer down.no allocator at this seam — This ABI does not allocate. Upstream hands an `rcutils_allocator_t` so the implementation can size and own the result; ours writes into a caller-owned byte range whose capacity the caller already knows. An allocator parameter cannot cross a seam that has nothing to give it. visitor, not an owning out-param — Upstream fills a heap-owning out-parameter the caller must `fini`. We pass a VISITOR callback invoked once per entry, with borrowed strings valid for the call. No allocation, bounded work, and nothing for a caller to leak — which is what makes the graph family available on a target with no allocator at all. |
rmw_ret_t rmw_get_subscriber_names_and_types_by_node( const rmw_node_t *, rcutils_allocator_t *, const char *, const char *, bool, rmw_names_and_types_t * ) |
rmw_ret_t (*get_subscriber_names_and_types_by_node)( const rmw_session_t *session, const char *node_name, const char *node_namespace, bool no_demangle, rmw_names_and_types_visitor_t visitor ) |
● re-shaped
the SESSION is the seam — Upstream passes the node into almost every call. Our vtable is scoped to a session handle that already knows its node, so re-passing it would ask the caller to carry an identity the callee holds — the same argument that decided `handle-owns-node` in the C API, one layer down.no allocator at this seam — This ABI does not allocate. Upstream hands an `rcutils_allocator_t` so the implementation can size and own the result; ours writes into a caller-owned byte range whose capacity the caller already knows. An allocator parameter cannot cross a seam that has nothing to give it. visitor, not an owning out-param — Upstream fills a heap-owning out-parameter the caller must `fini`. We pass a VISITOR callback invoked once per entry, with borrowed strings valid for the call. No allocation, bounded work, and nothing for a caller to leak — which is what makes the graph family available on a target with no allocator at all. |
rmw_ret_t rmw_get_subscriptions_info_by_topic( const rmw_node_t *, rcutils_allocator_t *, const char *, bool, rmw_topic_endpoint_info_array_t * ) |
rmw_ret_t (*get_subscriptions_info_by_topic)( const rmw_session_t *session, const char *topic_name, bool no_mangle, rmw_topic_endpoint_info_visitor_t visitor ) |
● re-shaped
the SESSION is the seam — Upstream passes the node into almost every call. Our vtable is scoped to a session handle that already knows its node, so re-passing it would ask the caller to carry an identity the callee holds — the same argument that decided `handle-owns-node` in the C API, one layer down.no allocator at this seam — This ABI does not allocate. Upstream hands an `rcutils_allocator_t` so the implementation can size and own the result; ours writes into a caller-owned byte range whose capacity the caller already knows. An allocator parameter cannot cross a seam that has nothing to give it. visitor, not an owning out-param — `rmw_topic_endpoint_info_array_t` and `rmw_network_flow_endpoint_array_t` are heap-owning arrays with their own `fini`. Visited one entry at a time instead. |
rmw_ret_t rmw_get_topic_names_and_types( const rmw_node_t *, rcutils_allocator_t *, bool, rmw_names_and_types_t * ) |
rmw_ret_t (*get_topic_names_and_types)( const rmw_session_t *session, bool no_demangle, rmw_names_and_types_visitor_t visitor ) |
● re-shaped
the SESSION is the seam — Upstream passes the node into almost every call. Our vtable is scoped to a session handle that already knows its node, so re-passing it would ask the caller to carry an identity the callee holds — the same argument that decided `handle-owns-node` in the C API, one layer down.no allocator at this seam — This ABI does not allocate. Upstream hands an `rcutils_allocator_t` so the implementation can size and own the result; ours writes into a caller-owned byte range whose capacity the caller already knows. An allocator parameter cannot cross a seam that has nothing to give it. visitor, not an owning out-param — Upstream fills a heap-owning out-parameter the caller must `fini`. We pass a VISITOR callback invoked once per entry, with borrowed strings valid for the call. No allocation, bounded work, and nothing for a caller to leak — which is what makes the graph family available on a target with no allocator at all. |
rmw_ret_t rmw_init( const rmw_init_options_t *, rmw_context_t * ) |
rmw_ret_t (*create_session)( const char *locator, uint8_t mode, uint32_t domain_id, const char *node_name, const rmw_session_options_t *options, rmw_session_t *out ) |
● re-shaped
renamed — the slot is create_session.build-time POD, no init/copy/fini — Upstream's options own heap and carry an allocator, which is why they need the init/copy/fini trio. Ours are a build-time POD passed by value. the SESSION is the seam — As `rmw_node_t`: upstream's context is the process-wide init state, and ours is the session the slot is being called on. |
rmw_ret_t rmw_init_options_copy( const rmw_init_options_t *, rmw_init_options_t * ) |
— |
◆ re-mapped · build time
build time: `copy` is `=`
|
rmw_ret_t rmw_init_options_fini(rmw_init_options_t *) |
— |
◆ re-mapped · build time
build time: `fini` is nothing
|
rmw_ret_t rmw_init_options_init( rmw_init_options_t *, rcutils_allocator_t ) |
— |
◆ re-mapped · build time
build time: rmw_init_options_t is a POD
|
rmw_ret_t rmw_init_publisher_allocation( const rosidl_message_type_support_t *, const rosidl_runtime_c__Sequence__bound *, rmw_publisher_allocation_t * ) |
— |
✕ not-supported · by decision
upstream's first two parameters are a `rosidl_message_type_support_t *` and a `rosidl_runtime_c__Sequence__bound *`. The sequence bound is declined ABI-wide (it is a rosidl runtime type), and the third parameter is an `rmw_publisher_allocation_t`, whose body is `{const char *implementation_identifier; void *data;}` — an opaque handle whose contents only the implementation that made it can read. Nothing crosses this seam usefully. THREE earlier reasons here were wrong, and the corrections are the record worth keeping: 'pools are baked' (false for four backends of five — issue 0777, now RESOLVED with the finding that every deviation reason built on that clause was false); 'upstream pre-sizes an rcutils_allocator_t the caller owns' (false — verified against Humble's rmw/types.h); and, until phase-406, a tail deferring the capability question to 0777 as though it were still open. It is not: 0777 closed, and the pre-sizing CAPABILITY for cyclonedds is a separate question that needs its own issue if anyone wants it, not a citation here. |
rmw_ret_t rmw_init_subscription_allocation( const rosidl_message_type_support_t *, const rosidl_runtime_c__Sequence__bound *, rmw_subscription_allocation_t * ) |
— |
✕ not-supported · by decision
as above |
const rmw_guard_condition_t * rmw_node_get_graph_guard_condition(const rmw_node_t *) |
rmw_ret_t (*node_get_graph_guard_condition)( rmw_session_t *session, rmw_event_callback_t callback, const void *user_data ) |
✕ not-supported · by decision
inert — declared, written and read by nothing.the SESSION is the seam — Upstream passes the node into almost every call. Our vtable is scoped to a session handle that already knows its node, so re-passing it would ask the caller to carry an identity the callee holds — the same argument that decided `handle-owns-node` in the C API, one layer down. |
rmw_ret_t rmw_publish( const rmw_publisher_t *, const void *, rmw_publisher_allocation_t * ) |
rmw_ret_t (*publish)( const rmw_publisher_t *publisher, rmw_byte_span_t payload ) |
◆ re-mapped · 2 upstream → 1 slot
bytes, not an untyped pointer — Upstream's `void *` is the message, interpreted through the typesupport pointer beside it. With types resolved at build time there is nothing to interpret it WITH, so the seam carries an explicit byte range and its length — which is also what makes the buffer's capacity checkable at the call.pre-allocation declined ABI-wide — `rmw_publisher_allocation_t` / `rmw_subscription_allocation_t` are upstream's pre-sizing handles. Declined at this seam because their first parameters are a typesupport pointer and a sequence bound, both declined ABI-wide. The CAPABILITY question — can a backend pre-size — is live for cyclonedds alone, and issue 0777 is CLOSED, so it is no longer a place to route it: 0777 resolved with the finding that every deviation reason built on "pools are baked" was false, which settles the reason and not the capability. Anyone who wants pre-sizing for cyclonedds needs a NEW issue; a citation to a resolved one is not a reason. |
rmw_ret_t rmw_publish_loaned_message( const rmw_publisher_t *, void *, rmw_publisher_allocation_t * ) |
rmw_ret_t (*publish_loaned_message)( const rmw_publisher_t *publisher, rmw_loan_token_t *token, size_t actual_len ) |
● re-shaped
bytes, not an untyped pointer — Upstream's `void *` is the message, interpreted through the typesupport pointer beside it. With types resolved at build time there is nothing to interpret it WITH, so the seam carries an explicit byte range and its length — which is also what makes the buffer's capacity checkable at the call.pre-allocation declined ABI-wide — `rmw_publisher_allocation_t` / `rmw_subscription_allocation_t` are upstream's pre-sizing handles. Declined at this seam because their first parameters are a typesupport pointer and a sequence bound, both declined ABI-wide. The CAPABILITY question — can a backend pre-size — is live for cyclonedds alone, and issue 0777 is CLOSED, so it is no longer a place to route it: 0777 resolved with the finding that every deviation reason built on "pools are baked" was false, which settles the reason and not the capability. Anyone who wants pre-sizing for cyclonedds needs a NEW issue; a citation to a resolved one is not a reason. |
rmw_ret_t rmw_publish_serialized_message( const rmw_publisher_t *, const rmw_serialized_message_t *, rmw_publisher_allocation_t * ) |
rmw_ret_t (*publish)( const rmw_publisher_t *publisher, rmw_byte_span_t payload ) |
◆ re-mapped · 2 upstream → 1 slot
renamed — the slot is publish.no allocator at this seam — `rmw_serialized_message_t` is an `rcutils_uint8_array_t`, which carries an allocator. The byte range and its length cross instead. pre-allocation declined ABI-wide — `rmw_publisher_allocation_t` / `rmw_subscription_allocation_t` are upstream's pre-sizing handles. Declined at this seam because their first parameters are a typesupport pointer and a sequence bound, both declined ABI-wide. The CAPABILITY question — can a backend pre-size — is live for cyclonedds alone, and issue 0777 is CLOSED, so it is no longer a place to route it: 0777 resolved with the finding that every deviation reason built on "pools are baked" was false, which settles the reason and not the capability. Anyone who wants pre-sizing for cyclonedds needs a NEW issue; a citation to a resolved one is not a reason. |
rmw_ret_t rmw_publisher_assert_liveliness(const rmw_publisher_t *) |
rmw_ret_t (*publisher_assert_liveliness)(const rmw_publisher_t *publisher) |
● same
|
rmw_ret_t rmw_publisher_count_matched_subscriptions( const rmw_publisher_t *, size_t * ) |
rmw_ret_t (*publisher_count_matched_subscriptions)( const rmw_publisher_t *publisher, size_t *subscription_count ) |
● same
|
rmw_ret_t rmw_publisher_event_init( rmw_event_t *, const rmw_publisher_t *, rmw_event_type_t ) |
rmw_ret_t (*publisher_event_init)( const rmw_publisher_t *publisher, rmw_event_type_t kind, uint32_t deadline_ms, rmw_status_event_callback_t cb, void *user_context ) |
◆ re-mapped · 2 upstream → 1 slot
events are a slot family, not an event object — Upstream models an event as an object you take from. Ours are per-kind slots on the session (`take_event` and the event mask in `rmw_event.h`), so the object that would carry the kind is the slot itself. |
rmw_ret_t rmw_publisher_get_actual_qos( const rmw_publisher_t *, rmw_qos_profile_t * ) |
rmw_ret_t (*publisher_get_actual_qos)( const rmw_publisher_t *publisher, rmw_qos_profile_t *qos ) |
● same
|
rmw_ret_t rmw_publisher_get_network_flow_endpoints( const rmw_publisher_t *, rcutils_allocator_t *, rmw_network_flow_endpoint_array_t * ) |
— |
✕ not-supported · by decision
DECLINED, phase-407: reporting the transport's 5-tuples is diagnostics, and the value is in a multi-homed deployment with dynamic discovery — the case least like this ABI's target, where one interface and a static config mean the answer is already in the file you wrote. Landed in phase-376 W5 for shape parity, never filled, never read (issue 0956). Cheap to revisit for Cyclone alone if `ros2` tooling ever needs to introspect a nano-ros node: it is read-only and has no wire effect. |
rmw_ret_t rmw_publisher_wait_for_all_acked( const rmw_publisher_t *, rmw_time_t ) |
rmw_ret_t (*publisher_wait_for_all_acked)( const rmw_publisher_t *publisher, uint32_t timeout_ms ) |
✕ not-supported · by decision
inert — declared, written and read by nothing.milliseconds, not a struct — `rmw_time_t` is a `{sec, nsec}` pair. This ABI carries `uint32_t` milliseconds throughout — the resolution every supported platform clock actually offers, and the unit the deadline arithmetic already uses. |
rmw_ret_t rmw_qos_profile_check_compatible( const rmw_qos_profile_t, const rmw_qos_profile_t, rmw_qos_compatibility_type_t *, char *, size_t ) |
rmw_ret_t rmw_qos_profile_check_compatible( rmw_qos_profile_t publisher_profile, rmw_qos_profile_t subscription_profile, rmw_qos_compatibility_type_t *compatibility, char *reason, size_t reason_size ) |
● re-shaped
by pointer, not by value — Upstream passes two `rmw_qos_profile_t` BY VALUE. The struct is ~80 bytes, and a by-value copy of it crosses the seam twice on every compatibility check — on a target where the call may run from a 512-byte task stack. Ours takes `const rmw_qos_profile_t *`, which is the same information without the copies. |
rmw_ret_t rmw_return_loaned_message_from_publisher( const rmw_publisher_t *, void * ) |
rmw_ret_t (*return_loaned_message_from_publisher)( const rmw_publisher_t *publisher, rmw_loan_token_t *token ) |
● re-shaped
bytes, not an untyped pointer — Upstream's `void *` is the message, interpreted through the typesupport pointer beside it. With types resolved at build time there is nothing to interpret it WITH, so the seam carries an explicit byte range and its length — which is also what makes the buffer's capacity checkable at the call. |
rmw_ret_t rmw_return_loaned_message_from_subscription( const rmw_subscription_t *, void * ) |
rmw_ret_t (*return_loaned_message_from_subscription)( const rmw_subscription_t *subscription, rmw_loan_token_t *token ) |
● re-shaped
bytes, not an untyped pointer — Upstream's `void *` is the message, interpreted through the typesupport pointer beside it. With types resolved at build time there is nothing to interpret it WITH, so the seam carries an explicit byte range and its length — which is also what makes the buffer's capacity checkable at the call. |
rmw_ret_t rmw_send_request( const rmw_client_t *, const void *, int64_t * ) |
rmw_ret_t (*send_request)( const rmw_client_t *client, rmw_byte_span_t request, int64_t *sequence_id ) |
● re-shaped
bytes, not an untyped pointer — Upstream's `void *` is the message, interpreted through the typesupport pointer beside it. With types resolved at build time there is nothing to interpret it WITH, so the seam carries an explicit byte range and its length — which is also what makes the buffer's capacity checkable at the call. |
rmw_ret_t rmw_send_response( const rmw_service_t *, rmw_request_id_t *, void * ) |
rmw_ret_t (*send_response)( const rmw_service_t *server, int64_t seq, rmw_byte_span_t response ) |
● re-shaped
visitor, not an owning out-param — As `rmw_service_info_t` — the sequence number and writer GUID travel in the borrowed view rather than a separate out-param.bytes, not an untyped pointer — Upstream's `void *` is the message, interpreted through the typesupport pointer beside it. With types resolved at build time there is nothing to interpret it WITH, so the seam carries an explicit byte range and its length — which is also what makes the buffer's capacity checkable at the call. |
rmw_ret_t rmw_serialize( const void *, const rosidl_message_type_support_t *, rmw_serialized_message_t * ) |
— |
◆ re-mapped · 1 → 2
nros-serdes: Serialize
codegen packs
|
rmw_ret_t rmw_service_request_subscription_get_actual_qos( const rmw_service_t *, rmw_qos_profile_t * ) |
rmw_ret_t (*service_request_subscription_get_actual_qos)( const rmw_service_t *service, rmw_qos_profile_t *qos ) |
● same
|
rmw_ret_t rmw_service_response_publisher_get_actual_qos( const rmw_service_t *, rmw_qos_profile_t * ) |
rmw_ret_t (*service_response_publisher_get_actual_qos)( const rmw_service_t *service, rmw_qos_profile_t *qos ) |
● same
|
rmw_ret_t rmw_service_server_is_available( const rmw_node_t *, const rmw_client_t *, bool * ) |
rmw_ret_t (*service_server_is_available)( const rmw_client_t *client, bool *out_available ) |
● re-shaped
the SESSION is the seam — Upstream passes the node into almost every call. Our vtable is scoped to a session handle that already knows its node, so re-passing it would ask the caller to carry an identity the callee holds — the same argument that decided `handle-owns-node` in the C API, one layer down. |
rmw_ret_t rmw_service_set_on_new_request_callback( rmw_service_t *, rmw_event_callback_t, const void * ) |
— |
✕ not-supported · by decision
DECLINED, phase-407: readiness belongs to the EXECUTOR here, not to the backend. `set_wake_callback` says "something on this session moved" and `spin_once` then walks its entry table calling `has_data` per entity — one vtable call each, on images whose entity count is single digits. Upstream's per-entity callback carries `number_of_events`, so honouring it means every backend tracking per-entity pending DEPTH: new state on a target where static RAM is already the pressure (issue 0827 measures a talker reserving 275 KB), bought for a scan nobody has measured as costly and a consumer that does not exist. If rclcpp `EventsExecutor` compatibility is ever wanted, that is a phase with a stated goal, not three slots kept warm (issue 0960). |
rmw_ret_t rmw_set_log_severity(rmw_log_severity_t) |
rmw_ret_t (*set_log_severity)(rmw_log_severity_t severity) |
● same
|
rmw_ret_t rmw_shutdown(rmw_context_t *) |
rmw_ret_t (*destroy_session)(rmw_session_t *session) |
◆ re-mapped · 2 upstream → 1 slot
renamed — the slot is destroy_session.the SESSION is the seam — As `rmw_node_t`: upstream's context is the process-wide init state, and ours is the session the slot is being called on. |
rmw_ret_t rmw_subscription_count_matched_publishers( const rmw_subscription_t *, size_t * ) |
rmw_ret_t (*subscription_count_matched_publishers)( const rmw_subscription_t *subscription, size_t *publisher_count ) |
● same
|
rmw_ret_t rmw_subscription_event_init( rmw_event_t *, const rmw_subscription_t *, rmw_event_type_t ) |
rmw_ret_t (*subscription_event_init)( const rmw_subscription_t *subscription, rmw_event_type_t kind, uint32_t deadline_ms, rmw_status_event_callback_t cb, void *user_context ) |
● re-shaped
events are a slot family, not an event object — Upstream models an event as an object you take from. Ours are per-kind slots on the session (`take_event` and the event mask in `rmw_event.h`), so the object that would carry the kind is the slot itself. |
rmw_ret_t rmw_subscription_get_actual_qos( const rmw_subscription_t *, rmw_qos_profile_t * ) |
rmw_ret_t (*subscription_get_actual_qos)( const rmw_subscription_t *subscription, rmw_qos_profile_t *qos ) |
● same
|
rmw_ret_t rmw_subscription_get_content_filter( const rmw_subscription_t *, rcutils_allocator_t *, rmw_subscription_content_filter_options_t * ) |
— |
✕ not-supported · by decision
as rmw_subscription_set_content_filter — declined with it, since a getter for a filter nothing can set is not a capability (issue 0956). |
rmw_ret_t rmw_subscription_get_network_flow_endpoints( const rmw_subscription_t *, rcutils_allocator_t *, rmw_network_flow_endpoint_array_t * ) |
— |
✕ not-supported · by decision
as rmw_publisher_get_network_flow_endpoints (issue 0956). |
rmw_ret_t rmw_subscription_set_content_filter( rmw_subscription_t *, const rmw_subscription_content_filter_options_t * ) |
— |
✕ not-supported · by decision
DECLINED, phase-407: network-side content filtering needs a DDS feature our pinned Cyclone does not have. 0.10.5 exposes `dds_set_topic_filter{,_and_arg,_extended}` — a LOCAL sample callback applied after the message arrives — and not `dds_create_topic_filtered`, which is the one that propagates the predicate to the writer. Shipping the local filter under this name would promise saved BANDWIDTH and deliver saved CPU, which is the class of deviation reason W4 spent its time undoing. The slot existed from phase-376 W5 to 2026-08-31 and no backend ever filled it (issue 0956). |
rmw_ret_t rmw_subscription_set_on_new_message_callback( rmw_subscription_t *, rmw_event_callback_t, const void * ) |
— |
✕ not-supported · by decision
as rmw_service_set_on_new_request_callback — and note the header had argued, correctly, that `set_wake_callback` does NOT cover this trio: it is session-scoped and serves subscriptions, services and clients identically. That is why these are DECLINED rather than re-mapped: nothing else answers them, and we have decided not to (issue 0960). |
rmw_ret_t rmw_take( const rmw_subscription_t *, void *, bool *, rmw_subscription_allocation_t * ) |
rmw_ret_t (*take)( const rmw_subscription_t *subscription, rmw_mut_byte_span_t *out, bool *taken ) |
◆ re-mapped · 2 upstream → 1 slot
bytes, not an untyped pointer — Upstream's `void *` is the message, interpreted through the typesupport pointer beside it. With types resolved at build time there is nothing to interpret it WITH, so the seam carries an explicit byte range and its length — which is also what makes the buffer's capacity checkable at the call.pre-allocation declined ABI-wide — `rmw_publisher_allocation_t` / `rmw_subscription_allocation_t` are upstream's pre-sizing handles. Declined at this seam because their first parameters are a typesupport pointer and a sequence bound, both declined ABI-wide. The CAPABILITY question — can a backend pre-size — is live for cyclonedds alone, and issue 0777 is CLOSED, so it is no longer a place to route it: 0777 resolved with the finding that every deviation reason built on "pools are baked" was false, which settles the reason and not the capability. Anyone who wants pre-sizing for cyclonedds needs a NEW issue; a citation to a resolved one is not a reason. |
rmw_ret_t rmw_take_event( const rmw_event_t *, void *, bool * ) |
rmw_ret_t (*subscription_take_event)( const rmw_subscription_t *subscription, rmw_event_type_t kind, rmw_event_payload_t *out, bool *taken ) |
● re-shaped
renamed — the slot is subscription_take_event.events are a slot family, not an event object — Upstream models an event as an object you take from. Ours are per-kind slots on the session (`take_event` and the event mask in `rmw_event.h`), so the object that would carry the kind is the slot itself. bytes, not an untyped pointer — Upstream's `void *` is the message, interpreted through the typesupport pointer beside it. With types resolved at build time there is nothing to interpret it WITH, so the seam carries an explicit byte range and its length — which is also what makes the buffer's capacity checkable at the call. |
rmw_ret_t rmw_take_loaned_message( const rmw_subscription_t *, void * *, bool *, rmw_subscription_allocation_t * ) |
rmw_ret_t (*take_loaned_message)( const rmw_subscription_t *subscription, rmw_byte_span_t *out_view, rmw_loan_token_t **out_token, bool *taken ) |
● re-shaped
bytes, not an untyped pointer — Upstream's `void *` is the message, interpreted through the typesupport pointer beside it. With types resolved at build time there is nothing to interpret it WITH, so the seam carries an explicit byte range and its length — which is also what makes the buffer's capacity checkable at the call.pre-allocation declined ABI-wide — `rmw_publisher_allocation_t` / `rmw_subscription_allocation_t` are upstream's pre-sizing handles. Declined at this seam because their first parameters are a typesupport pointer and a sequence bound, both declined ABI-wide. The CAPABILITY question — can a backend pre-size — is live for cyclonedds alone, and issue 0777 is CLOSED, so it is no longer a place to route it: 0777 resolved with the finding that every deviation reason built on "pools are baked" was false, which settles the reason and not the capability. Anyone who wants pre-sizing for cyclonedds needs a NEW issue; a citation to a resolved one is not a reason. |
rmw_ret_t rmw_take_loaned_message_with_info( const rmw_subscription_t *, void * *, bool *, rmw_message_info_t *, rmw_subscription_allocation_t * ) |
rmw_ret_t (*take_loaned_message_with_info)( const rmw_subscription_t *subscription, rmw_byte_span_t *out_view, rmw_loan_token_t **out_token, bool *taken, rmw_message_info_t *message_info ) |
◆ re-mapped · 1 → 2
take_loaned_message
the attachment on the message already taken
bytes, not an untyped pointer — Upstream's `void *` is the message, interpreted through the typesupport pointer beside it. With types resolved at build time there is nothing to interpret it WITH, so the seam carries an explicit byte range and its length — which is also what makes the buffer's capacity checkable at the call. pre-allocation declined ABI-wide — `rmw_publisher_allocation_t` / `rmw_subscription_allocation_t` are upstream's pre-sizing handles. Declined at this seam because their first parameters are a typesupport pointer and a sequence bound, both declined ABI-wide. The CAPABILITY question — can a backend pre-size — is live for cyclonedds alone, and issue 0777 is CLOSED, so it is no longer a place to route it: 0777 resolved with the finding that every deviation reason built on "pools are baked" was false, which settles the reason and not the capability. Anyone who wants pre-sizing for cyclonedds needs a NEW issue; a citation to a resolved one is not a reason. |
rmw_ret_t rmw_take_request( const rmw_service_t *, rmw_service_info_t *, void *, bool * ) |
rmw_ret_t (*take_request)( const rmw_service_t *server, rmw_mut_byte_span_t *request, int64_t *seq_out, bool *taken ) |
● re-shaped
visitor, not an owning out-param — The request/response metadata upstream writes into a caller-owned struct arrives here through the same borrowed-view discipline as the graph family.bytes, not an untyped pointer — Upstream's `void *` is the message, interpreted through the typesupport pointer beside it. With types resolved at build time there is nothing to interpret it WITH, so the seam carries an explicit byte range and its length — which is also what makes the buffer's capacity checkable at the call. |
rmw_ret_t rmw_take_response( const rmw_client_t *, rmw_service_info_t *, void *, bool * ) |
rmw_ret_t (*take_response)( const rmw_client_t *client, rmw_mut_byte_span_t *reply, int64_t *seq_out, bool *taken ) |
● re-shaped
visitor, not an owning out-param — The request/response metadata upstream writes into a caller-owned struct arrives here through the same borrowed-view discipline as the graph family.bytes, not an untyped pointer — Upstream's `void *` is the message, interpreted through the typesupport pointer beside it. With types resolved at build time there is nothing to interpret it WITH, so the seam carries an explicit byte range and its length — which is also what makes the buffer's capacity checkable at the call. |
rmw_ret_t rmw_take_sequence( const rmw_subscription_t *, size_t, rmw_message_sequence_t *, rmw_message_info_sequence_t *, size_t *, rmw_subscription_allocation_t * ) |
rmw_ret_t (*take_sequence)( const rmw_subscription_t *subscription, uint8_t *buf, size_t per_msg_cap, size_t max_msgs, size_t *out_lens, size_t *taken ) |
● re-shaped
visitor, not an owning out-param — `rmw_take_sequence`'s message and info sequences are caller-owned arrays sized by an allocator. The bounded form is a repeated take, which is what the executor does.pre-allocation declined ABI-wide — `rmw_publisher_allocation_t` / `rmw_subscription_allocation_t` are upstream's pre-sizing handles. Declined at this seam because their first parameters are a typesupport pointer and a sequence bound, both declined ABI-wide. The CAPABILITY question — can a backend pre-size — is live for cyclonedds alone, and issue 0777 is CLOSED, so it is no longer a place to route it: 0777 resolved with the finding that every deviation reason built on "pools are baked" was false, which settles the reason and not the capability. Anyone who wants pre-sizing for cyclonedds needs a NEW issue; a citation to a resolved one is not a reason. |
rmw_ret_t rmw_take_serialized_message( const rmw_subscription_t *, rmw_serialized_message_t *, bool *, rmw_subscription_allocation_t * ) |
rmw_ret_t (*take)( const rmw_subscription_t *subscription, rmw_mut_byte_span_t *out, bool *taken ) |
◆ re-mapped · 2 upstream → 1 slot
renamed — the slot is take.no allocator at this seam — `rmw_serialized_message_t` is an `rcutils_uint8_array_t`, which carries an allocator. The byte range and its length cross instead. pre-allocation declined ABI-wide — `rmw_publisher_allocation_t` / `rmw_subscription_allocation_t` are upstream's pre-sizing handles. Declined at this seam because their first parameters are a typesupport pointer and a sequence bound, both declined ABI-wide. The CAPABILITY question — can a backend pre-size — is live for cyclonedds alone, and issue 0777 is CLOSED, so it is no longer a place to route it: 0777 resolved with the finding that every deviation reason built on "pools are baked" was false, which settles the reason and not the capability. Anyone who wants pre-sizing for cyclonedds needs a NEW issue; a citation to a resolved one is not a reason. |
rmw_ret_t rmw_take_serialized_message_with_info( const rmw_subscription_t *, rmw_serialized_message_t *, bool *, rmw_message_info_t *, rmw_subscription_allocation_t * ) |
rmw_ret_t (*take_with_info)( const rmw_subscription_t *subscription, rmw_mut_byte_span_t *message, bool *taken, rmw_message_info_t *message_info ) |
◆ re-mapped · 2 upstream → 1 slot
take
the attachment on the message already taken
take_with_info.inert — declared, written and read by nothing. no allocator at this seam — `rmw_serialized_message_t` is an `rcutils_uint8_array_t`, which carries an allocator. The byte range and its length cross instead. pre-allocation declined ABI-wide — `rmw_publisher_allocation_t` / `rmw_subscription_allocation_t` are upstream's pre-sizing handles. Declined at this seam because their first parameters are a typesupport pointer and a sequence bound, both declined ABI-wide. The CAPABILITY question — can a backend pre-size — is live for cyclonedds alone, and issue 0777 is CLOSED, so it is no longer a place to route it: 0777 resolved with the finding that every deviation reason built on "pools are baked" was false, which settles the reason and not the capability. Anyone who wants pre-sizing for cyclonedds needs a NEW issue; a citation to a resolved one is not a reason. |
rmw_ret_t rmw_take_with_info( const rmw_subscription_t *, void *, bool *, rmw_message_info_t *, rmw_subscription_allocation_t * ) |
rmw_ret_t (*take_with_info)( const rmw_subscription_t *subscription, rmw_mut_byte_span_t *message, bool *taken, rmw_message_info_t *message_info ) |
◆ re-mapped · 2 upstream → 1 slot
take
the attachment on the message already taken
bytes, not an untyped pointer — Upstream's `void *` is the message, interpreted through the typesupport pointer beside it. With types resolved at build time there is nothing to interpret it WITH, so the seam carries an explicit byte range and its length — which is also what makes the buffer's capacity checkable at the call. pre-allocation declined ABI-wide — `rmw_publisher_allocation_t` / `rmw_subscription_allocation_t` are upstream's pre-sizing handles. Declined at this seam because their first parameters are a typesupport pointer and a sequence bound, both declined ABI-wide. The CAPABILITY question — can a backend pre-size — is live for cyclonedds alone, and issue 0777 is CLOSED, so it is no longer a place to route it: 0777 resolved with the finding that every deviation reason built on "pools are baked" was false, which settles the reason and not the capability. Anyone who wants pre-sizing for cyclonedds needs a NEW issue; a citation to a resolved one is not a reason. |
rmw_ret_t rmw_trigger_guard_condition(const rmw_guard_condition_t *) |
— |
◆ re-mapped · platform
platform: the wake primitive, via GuardCondition::trigger
|
rmw_ret_t rmw_wait( rmw_subscriptions_t *, rmw_guard_conditions_t *, rmw_services_t *, rmw_clients_t *, rmw_events_t *, rmw_wait_set_t *, const rmw_time_t * ) |
— |
◆ re-mapped · 1 → 5
has_data
has_request
drive_io
set_wake_callback
next_deadline_ms
|
Reproduce
| command | asks |
|---|---|
just check rmw-api-parity | is every contract symbol classified |
just check rmw-abi-shape | does the vtable mirror it — name, args, return |
just check rmw-slot-producers | which slots anything actually writes or reads |
python3 scripts/rmw-api-parity.py --contract | re-derive the contract from an installed impl |
Per-RMW Feature Matrix
One table per question the capability pages used to answer in
contradictory prose. Derived from the backend sources — the two
C vtables (vtable.cpp,
vtable.c) and the zenoh shim’s trait
overrides — so a backend gaining or losing a slot moves this page in
the same commit or fails the gate.
wired = the backend implements it. — = not wired: the runtime
surfaces UNSUPPORTED or falls back where a fallback exists (the
vtable comments name which). The rows below the parity marker are
the phase-376 W4 surface — slots declared in the ABI whose backend
wiring is the in-flight campaign; all-dash rows there mean
declared, not yet wired anywhere, and they flip automatically as
backends land implementations.
Session / entity capabilities
| Capability | Zenoh | XRCE-DDS | Cyclone DDS |
|---|---|---|---|
| Publish / subscribe | wired | wired | wired |
| Services (server side) | wired | wired | wired |
| Service clients | wired | wired | wired |
| Server-availability probe | wired | — | — |
| Status events (deadline / liveliness / lost) | wired | — | — |
| Manual liveliness assert | wired | — | — |
Event-driven wake (set_wake_callback) | wired | — | — |
Deadline hint (next_deadline_ms) | wired | — | — |
| Zero-copy loan API | — | — | — |
Batch receive (take_sequence) | wired | — | wired |
| Streamed publish | wired | wired | — |
| Connectivity ping | wired | wired | — |
| Identity / feature probe | — | — | — |
| Publisher GID / matched counts | — | — | — |
| Actual-QoS read-back | — | — | — |
| Wait-for-acked | — | — | — |
| Take-with-info | — | — | — |
| Entity new-data callbacks | — | — | — |
| Graph introspection (names/types/counts) | — | — | — |
Node-layer features
These live in nros-node on top of pub/sub + services — they are
not backend slots, so the rule, not a vtable, decides the row:
| Feature | Zenoh | XRCE-DDS | Cyclone DDS | Rule |
|---|---|---|---|---|
| Actions | yes | yes | untested | Built in nros-node on pub/sub + services. Cyclone has both since service.cpp, but no action example runs on it in CI — see known limitations. |
| Parameters (+ param services) | yes | yes | yes | Node-layer services (RFC-0004); available wherever services are. |
| Lifecycle (REP-2002) | yes | yes | yes | Node-layer state machine + services (lifecycle-services feature). |
QoS policies
A backend advertises the policies it can enforce via
supported_qos_policies(); requesting an unadvertised policy fails
entity creation loudly (INCOMPATIBLE_QOS) — no silent
downgrade. Per-policy semantics: RMW vs upstream §7.
| Policy | Zenoh | XRCE-DDS / Cyclone DDS (via C ABI) ¹ |
|---|---|---|
CORE | ✓ | ✓ |
DURABILITY_TRANSIENT_LOCAL | — | ✓ |
DEADLINE | ✓ | ✓ |
LIFESPAN | ✓ | ✓ |
LIVELINESS_AUTOMATIC | ✓ | ✓ |
LIVELINESS_MANUAL_BY_TOPIC | ✓ | ✓ |
LIVELINESS_MANUAL_BY_NODE | ✓ | ✓ |
LIVELINESS_LEASE | ✓ | ✓ |
AVOID_ROS_NAMESPACE_CONVENTIONS | — | ✓ |
¹ The C-ABI backends’ mask is asserted by the runtime shim
(packages/rmw/cffi/src/lib.rs), not reported by the backend — the
vtable has no supported_qos_policies slot yet (TODO 115.K.2.x).
Treat the column as the runtime’s assumption; the backend’s own
enforcement happens at entity creation.
Related
- Choosing an RMW Backend — the decision tree
- Backend Reference — architecture, footprint, transports per backend
- Support Status — versions, pins, and CI tiers
Scheduling Wiring Matrix
Which scheduling capabilities nano-ros delivers per platform,
and which portable executor classes are live in dispatch —
derived from the two artifacts the runtime executes, not from
prose. The per-platform rows are parsed from sched_caps_for()
(nros-orchestration-ir), the exact function the workload realizer
calls to decide Native-vs-Degrade for a deployment; its rule, quoted
from the source: caps describe what nano-ros DELIVERS, not what
the OS could do (issue 0261).
Kernel capabilities per platform
| Platform | Kernel EDF | Reservation / sporadic server | Preemption threshold | SMP affinity | Priority levels | Low number = high priority |
|---|---|---|---|---|---|---|
| Linux host | — | — | — | ✓ | 99 | — |
| Zephyr | ✓ | — | — | ✓ | 32 | ✓ |
| FreeRTOS | — | — | — | ✓ | 16 | — |
| ThreadX | — | — | ✓ | ✓ | 32 | ✓ |
| NuttX | — | ✓ | — | ✓ | 255 | — |
| Bare-metal (RTIC / NVIC) | — | — | — | — | 8 | — |
Notes carried from the source arms: the Linux row is deliberately
conservative — no SCHED_DEADLINE consumer exists, so edf is
false even though Linux could; its affinity is sched_setaffinity,
which is Linux and not POSIX (absent from libc’s apple module), so the
row’s reach is linux while the platform layer beneath it stays
posix; Zephyr’s EDF is
CONFIG_SCHED_DEADLINE; NuttX’s reservation is POSIX
SCHED_SPORADIC; ThreadX’s preemption threshold is native; the
bare-metal row is NVIC hardware priorities with SRP ceilings (RTIC),
single core. These are PLATFORM DEFAULTS and there is no per-image
override today: [deploy.<board>] edf = <bool> / cores = <n> was
documented here but never authorable — the knob read an extras bag no
table wrote, and upstream’s deny_unknown_fields rejected the keys
outright. Retired with issue 0951; issue 0259 wants the override to
exist and needs an authoring path first.
Portable executor classes (SchedClass)
These run inside the executor on every platform — they need no
kernel support, which is what makes them portable; where the kernel
HAS the native capability (table above), the realizer places the
workload natively instead. “Dispatch sites” counts
SchedClass::<X> references in the executor’s spin path — a class
with zero sites is a type, not a behavior.
| Class | Dispatch sites in spin.rs | Status |
|---|---|---|
Fifo | 2 | Live — the default class; bucketed FIFO ready-set. |
Edf | 1 | Live — earliest-deadline-first ready-set ordering. |
Sporadic | 5 | Live — budget/period enforcement in dispatch. |
TimeTriggered | 1 | Accepted but deprecated in favour of cooperating TT-with-other-classes (see the enum’s deprecation note); dispatches as Fifo. |
Related
- Scheduling Models — the narrative: how tiers, transport priorities, and these classes land on each kernel
- RTOS Cooperation — spin-loop shapes per execution model
- Support Status · Per-RMW Feature Matrix
Supported Boards
Procurement-grade compatibility matrix. Each row lists a real
vendor + board model and reports nano-ros’s status on it. Rows
marked Tested boot in a CI lane — note this maps to the tier
registry’s tier 1 (in just ci) or tier 2 (nightly/matrix lanes
only): MPS2-AN385, the RISC-V64 ThreadX virt machine, and ESP32-C3
QEMU are tier 2, so “Tested” there means the nightly matrix, not every
push. Ready rows compile and run but have no in-CI gate yet;
Build-only rows compile but cannot be booted by any CI lane
(hardware or a license-gated model is required); Untested rows
compile per the architecture support but no one has reported booting
nano-ros on them. Per-crate tier truth is the generated
Board Support Tiers.
For in-tree board packages, see the generated Board Support Tiers — it is checked against the test matrix, the fixture rows and the nightly lanes by
just check board-tiers. This page stays hand-maintained because it also covers parts that have never had a crate in this tree (Nordic, NXP, TI…), which the registry cannot express.It has drifted before — the ARM FVP row claimed Tested while its model is license-walled and runs in no CI lane at all.
| Vendor | Board | MCU / SoC | Arch | Default RTOS | Status | Example / board crate |
|---|---|---|---|---|---|---|
| ARM | MPS2-AN385 (QEMU) | Cortex-M3 | Armv7-M | FreeRTOS / bare | Tested | examples/qemu-arm-freertos/, examples/qemu-arm-baremetal/ |
| STMicro | STM32F4-Discovery | STM32F407 | Cortex-M4F | bare | Out of tree | Worked example — no in-CI lane boots it |
| STMicro | STM32H7-Nucleo | STM32H743 | Cortex-M7F | FreeRTOS / Zephyr | Ready | Use FreeRTOS / Zephyr starter with nros-board-freertos overlay |
| STMicro | Pixhawk 4 (FMUv5) | STM32F765 | Cortex-M7F | NuttX (PX4) | Ready | integrations/px4/module-template/ |
| STMicro | Pixhawk 6X / 6C | STM32H753 | Cortex-M7F | NuttX (PX4) | Ready | integrations/px4/module-template/ |
| Nordic | nRF52840-DK | Cortex-M4F | Armv7E-M | Zephyr | Untested | Zephyr starter — supply -b nrf52840dk_nrf52840 |
| Nordic | nRF5340-DK | Cortex-M33 (dual) | Armv8-M | Zephyr | Untested | Zephyr starter — supply -b nrf5340dk_nrf5340_cpuapp |
| Espressif | ESP32-C3 (QEMU) | RISC-V (RV32IMC) | RISC-V | bare (esp-hal) | Tested | examples/qemu-esp32-baremetal/rust/ |
| Espressif | ESP32-C3-DevKit | RISC-V (RV32IMC) | RISC-V | ESP-IDF | Ready (build-only, nightly lane) | integrations/nano-ros/ — built by full-matrix only; nothing boots an IDF image in CI |
| Espressif | ESP32-S3-DevKit | Xtensa LX7 | Xtensa | ESP-IDF | Not supported (no board crate) | Xtensa is not a supported target today — see ESP32 chapter |
| Espressif | ESP32-C6-DevKit | RISC-V | RISC-V | ESP-IDF | Untested | Same ESP-IDF path as C3 |
| NXP | LPC55S69-EVK | Cortex-M33 | Armv8-M | Zephyr | Untested | Zephyr -b lpcxpresso55s69_cpu0 |
| NXP | MIMXRT1170-EVK | Cortex-M7 + M4 | Armv7-M | FreeRTOS / Zephyr | Untested | FreeRTOS starter + vendor BSP |
| TI | LP-CC1352P7 | Cortex-M4F | Armv7E-M | FreeRTOS / TI-RTOS | Untested | FreeRTOS starter + TI driver overlay |
| RP2040 | Raspberry Pi Pico | Cortex-M0+ | Armv6-M | bare / FreeRTOS | Untested | Bare-metal Cortex-M3 path — Cortex-M0+ has only 4 NVIC priority levels (per-callback OS-priority dispatch is disqualified — pub/sub still works fine) |
| QEMU | virt RISC-V64 | rv64gc | RISC-V | ThreadX | Tested | examples/qemu-riscv64-threadx/ |
| QEMU | Cortex-A9 (Versatile) | Cortex-A9 | Armv7-A | Zephyr | Tested | Zephyr -b qemu_cortex_a9 |
| QEMU | arm virt | Cortex-A7 | Armv7-A | NuttX | Tested | examples/qemu-arm-nuttx/ |
| QEMU | rv-virt RISC-V32 | rv32imac | RISC-V | NuttX | Tested | just nuttx build-riscv-c / build-riscv-rust (nros setup qemu-riscv-nuttx) |
| NVIDIA | Jetson Orin SPE | Cortex-R5 | Armv7-R | FreeRTOS | Not supported (no board crate) | Board crate removed (phase-337); FreeRTOS starter + vendor BSP is the path back |
| Arm FVP | Base_RevC AEMv8R (SMP) | AEMv8-R, AArch64 profile | Armv8-R | Zephyr 3.7 | Build-only | See ARM FVP getting-started chapter; just zephyr build-fvp-ws-entry / build-fvp-board-import + their run- siblings. Runtime is maintainer-run (just zephyr verify-fvp-runtime) — the model is license-gated, so no CI lane can boot it |
| Linux host | (sim) | x86-64 / aarch64 | x86 / Arm | ThreadX sim | Tested | examples/threadx-linux/ |
| Linux host | (native) | x86-64 / aarch64 | x86 / Arm | POSIX | Tested | examples/native/ |
How to add a new board
- Pick the matching RTOS path. Cortex-M3 / M4 / M7 + RTOS → use FreeRTOS or Zephyr starter. Cortex-M0+ → bare-metal starter (limited; no NVIC priority headroom). Cortex-A / RISC-V64 → NuttX or Zephyr. Xtensa / RISC-V32 + Wi-Fi → ESP-IDF or esp-hal.
- Find or write a board crate. Existing crates under
packages/boards/nros-board-*/cover most QEMU + reference dev kits. Real-hardware boards need a thin board crate that supplies startup, linker script, andBoardIdle::wfi()(bare-metal) or wraps the RTOS’s BSP (FreeRTOS / Zephyr). - Run the existing example tree. Each row above points at the
canonical example dir. Cross-compile the talker / listener and
verify against stock ROS 2 +
RMW_IMPLEMENTATION=rmw_zenoh_cpp. - Report back. Open an issue with the working build + flash + run commands so the row moves from Untested / Ready to Tested.
Caveats by chip family
- Cortex-M0+ (RP2040, STM32F0, nRF51): only 4 NVIC priority levels. Per-callback OS-priority dispatch (a research scheduler shape originally proposed by Choi et al. as PiCAS, RTAS ’21) is disqualified on this class; nano-ros’s user-space EDF / FIFO scheduler is the only option. Pub/sub works fine.
- Xtensa ESP32 / ESP32-S2 / ESP32-S3: needs the
esp-rsfork of rustc (rustup target adddoes not cover Xtensa; install via https://github.com/esp-rs/rust-build). - Cortex-A9 / A53: hosted-RTOS only (NuttX, Zephyr). Heap + libc required; bare-metal Cortex-A is not in the coverage matrix.
- PX4 boards: NuttX is the underlying kernel; the PX4 module template is the canonical entry path — see PX4 Autopilot.
See also
- Quick board check (intro) — hobbyist-grade one-liner per chip.
- Embedded Starters — per-RTOS walkthrough.
- Platform Differences — per-platform capability deltas.
Board Support Tiers
What nano-ros promises for each in-tree board package, and why.
Every tier here is checked against evidence by just check board-tiers: a board cannot claim tier 1 or 2 without runtime cells in the test matrix, cannot claim tier 3 while holding runtime cells, and cannot be omitted from this table at all. That gate exists because the previous hand-written table drifted into claiming a license-gated target was CI-tested.
For procurement guidance on parts that have no crate in this tree (Nordic, NXP, TI and friends), see Supported Boards. That table is hand-maintained and covers hardware this one cannot describe; this table is generated and covers what is actually in the repo.
Tier 1 — supported
just ci exercises it, it has asserted runtime coverage, and a nightly lane builds it. A regression fails before merge.
| Board package | Platform | Maintainers | Notes |
|---|---|---|---|
nros-board-linux | Linux | unassigned | The reference platform: the densest Runtime coverage of any target, all three RMWs. phase-337 W8.a merged the native shim and the posix family driver into this one crate — the registry carried TWO rows for one implementation because board_path_for mapped both keys to the same ZST. |
nros-board-threadx-linux | ThreadxLinux | unassigned | |
nros-board-zephyr | ZephyrNativeSim | unassigned | CAVEAT: only ever built for native_sim/native/64. No real Zephyr hardware board is built by anything. |
Tier 2 — verified
Asserted runtime coverage and a nightly lane, but not in just ci. Breakage is found within a day.
| Board package | Platform | Maintainers | Notes |
|---|---|---|---|
nros-board-esp32-qemu | Esp32Qemu | unassigned | Real two-way QEMU e2e (esp32_emulator.rs). |
nros-board-freertos-posix | FreertosPosix | unassigned | |
nros-board-mps2-an385 | QemuBaremetal | unassigned | phase-337 W6.a folded nros-board-rtic-mps2-an385 in as the rtic feature — one crate, two entry shapes (direct-exec + RTIC), one witness row. |
nros-board-mps2-an385-freertos | FreertosMps2 | unassigned | |
nros-board-nuttx-qemu | NuttxArm | unassigned | |
nros-board-nuttx-qemu | NuttxRiscv | unassigned | HONEST LABEL: C runtime-proven; Rust and C++ Pubsub are explicit CarveOuts and every EntryPubsub row is BuildOnly. |
nros-board-threadx-qemu-riscv64 | ThreadxRiscv64 | unassigned | All three Action rows are BuildOnly by a deliberate wall-clock choice (182.5), not by breakage. |
nros-board-zephyr | ZephyrQemuCortexM | unassigned | C and C++ pubsub are Runtime; service/action are BuildOnly pending the runner’s peer half. NO RUST ROW: zephyr-lang-rust cannot compile for any board whose devicetree has gpio nodes (issue 0432), so a rust cell would not build and BuildOnly would be a lie. Tier 2 rather than 1 because just ci is host-only and this needs QEMU. |
Tier 3 — build-only
Compiles. No CI lane can boot it — real hardware or a license-gated model is required. Nobody has booted it recently unless a maintainer did so by hand.
| Board package | Platform | Maintainers | Notes |
|---|---|---|---|
nros-board-s32z270-freertos | — | unassigned | S32Z270 RTU Cortex-R52 (ASI phase-4 W5.b consumer). No matrix_platform: no witness exists for this board, and borrowing FreertosMps2’s would claim its Runtime cells. Link-completeness is proven by the workspace cmake lane with the CRx_No_GIC default port. |
nros-board-zephyr | Fvp | unassigned | Gated SDK: arm-fvp. License-gated model; runtime is maintainer-run via just zephyr verify-fvp-runtime. Kept in-tree: it is the ASI reference consumer’s target (phase-292), a real downstream user the CI evidence cannot see. phase-337 W9.a folded the nros-board-fvp-aemv8r-smp CRATE into nros-board-zephyr/boards/fvp-aemv8r-smp/ — the crate’s Rust half (a Config and a run loop) had zero consumers, so what it actually shipped was a prj.conf, a DTS overlay and a board.cmake: a config bundle wearing a Cargo.toml. Third row for this crate, which is exactly what W1.c’s (crate, matrix_platform) keying is for. |
Scaffold — not supported
Structurally incomplete. Distinct from tier 3: tier 3 is finished-but-unverified, scaffold is unfinished. Do not build on these.
| Board package | Platform | Maintainers | Notes |
|---|---|---|---|
nros-board-mps3-an536-freertos | — | unassigned |
Infrastructure — not boards
Shared traits, family drivers, ABI mirrors, PACs, and descriptor directories. No platform, no tier promise.
| Board package | Platform | Maintainers | Notes |
|---|---|---|---|
linux | — | unassigned | nros-board.toml descriptor directory, not a crate (renamed from posix by phase-337 W8.d). phase-321 W2 moves these out of packages/boards/. |
mps2-an385-pac | — | unassigned | IRQ enum only. Kept because RTIC’s #[rtic::app(device = ...)] needs a nameable path. |
nros-board-cffi | — | unassigned | Header ABI mirror with ZERO in-tree C consumers; kept alive by its own drift gate. Spec artifact, not a library. |
nros-board-common | — | unassigned | Build-helper library plus one trait; 2180 of 2252 code lines are behind build-helpers. |
nros-board-freertos | — | unassigned | Real family driver: the MPS2 overlay is pub use nros_board_freertos::Config and carries zero config code. The pattern the forked boards should copy. |
nros-board-nuttx | — | unassigned | NuttX FAMILY DRIVER, not a façade: owns run_entry / run_tiers / the tier spin loops, which the one board overlay (nros-board-nuttx-qemu) delegates to. phase-337 W3 merged the two board crates below it; this layer stays. |
nros-board-threadx | — | unassigned | Real family driver: a 1120-line generic entry.py both ThreadX boards call into. |
nros-board-threadx-port-riscv64 | — | unassigned | NOT a board: RFC-0064 layer 2, the ThreadX RISC-V64/GNU ARCH PORT. Upstream types ULONG as 8 bytes there; NetX Duo’s packet code does ULONG * arithmetic assuming 4, and retyping it shifts every TX_THREAD offset the context-switch assembly loads — so the fork is a header plus five .S files, and it is arch, not board. Extracted from nros-board-threadx-qemu-riscv64 by phase-337 W4.a; a second riscv64 ThreadX board would need every line of it unchanged. threadx-linux has none because upstream’s Linux port already types ULONG as 4 bytes — which is why the two ThreadX boards did NOT merge. |
zephyr | — | unassigned | nros-board.toml descriptor directory, not a crate. |
Maintainers
A tier is a promise, and just check board-tiers requires someone to have made it: tier 1 needs 3 named maintainers, tier 2 needs 2, tier 3 needs 1 (phase-375 W1). Rust’s target-tier policy is the model — a named person per target, and demotion when the maintainer becomes unreachable; that mechanism, not the directory layout, is what actually retires abandoned targets.
Rows that predate the rule are grandfathered by scripts/board-maintainer-baseline.json, a list that only shrinks. So the rule binds a NEW board immediately and an existing one as its owner is found — a gate that failed all 22 rows on the day it landed would have been bypassed instead. Recording an owner is the point; inventing one would be worse than leaving the row at the tier it can carry.
Environment Variables Reference
Configuration File
All environment variables can be set in a .env file at the project root:
cp .env.example .env
# Edit .env — uncomment and adjust values as needed
- justfile —
.envis auto-loaded. Missing file silently ignored. - direnv —
.envrcsources.envif present. - Manual —
set -a; source .env; set +abeforecargo build.
Variables in .env take precedence over justfile defaults but are overridden by explicit shell exports.
Runtime Configuration
Examples use ExecutorConfig::from_env() for configuration:
| Variable | Description | Default |
|---|---|---|
ROS_DOMAIN_ID | ROS 2 domain ID | 0 |
NROS_LOCATOR | RMW locator (tcp/…, udp/…, serial/…, or tls/…) | tcp/127.0.0.1:7447 |
NROS_SESSION_MODE | Session mode: client or peer | client |
ZENOH_TLS_ROOT_CA_CERTIFICATE | Path to CA certificate (PEM) for TLS | (none) |
ZENOH_TLS_ROOT_CA_CERTIFICATE_BASE64 | Base64-encoded CA certificate for TLS | (none) |
ZENOH_TLS_VERIFY_NAME_ON_CONNECT | Verify server hostname in TLS (true/false) | (none) |
Deprecated legacy names:
ZENOH_LOCATORandZENOH_MODEare still accepted (they fall back toNROS_LOCATOR/NROS_SESSION_MODE) but will print a one-line deprecation warning to stderr. Migrate to theNROS_*names.ZENOH_TLS_*names are kept because TLS is currently zenoh-specific.
TLS Notes
- POSIX: requires
libmbedtls-dev(just setup basechecks it). File-path and base64 cert loading are both supported. - Bare-metal: only
ZENOH_TLS_ROOT_CA_CERTIFICATE_BASE64is supported (no filesystem). The certificate is embedded at build time. - The
link-tlsCargo feature must be enabled on both the example and thenroscrate.
Build-Time Configuration
| Variable | Description | Required |
|---|---|---|
ZENOH_PICO_DIR | CMake install prefix for pre-built zenoh-pico (use with system-zenohpico feature on zpico-sys) | Only with system-zenohpico |
SSID | WiFi network name for ESP32 examples | Required for build-examples-esp32 |
PASSWORD | WiFi password for ESP32 examples | Required for build-examples-esp32 |
NROS_EXTRA_BOARD_PATH | Extra board-search roots (PATH-style : separated). Each entry is a directory shaped like packages/boards/ — subdirs carrying nros-board.toml crates or */boards/<name>/board.cmake bundles. Read by the nros CLI’s board catalog AND nano_ros_use_board() (also settable as a CMake cache var there). Board keys stay global: a name found under two roots is an error, never shadowed. | (unset) |
ARM FVP (FVP_BaseR_AEMv8R)
License-gated — nano-ros does not download the binary. Set one of the discovery vars after accepting the Arm EULA and installing locally. See the ARM FVP getting-started chapter for the end-to-end build+run walk-through.
| Variable | Description | Default |
|---|---|---|
ARMFVP_BIN_PATH | Directory containing FVP_BaseR_AEMv8R (Zephyr-canonical, highest priority). | (unset) |
ARM_FVP_DIR | Install root; resolver scans models/Linux64_GCC-*/ underneath. Matches sdk-index. | (unset) |
If neither is set, scripts/zephyr/resolve-fvp-bin.sh falls back
to dirname $(command -v FVP_BaseR_AEMv8R). Phase 217.A —
just zephyr run-fvp-ws-entry / run-fvp-board-import skip gracefully when
the binary can’t be resolved.
After extracting the Arm-provided tarball, run
scripts/installers/arm-fvp-installer.sh with ARM_FVP_DIR set
to the extraction root — it locates FVP_BaseR_AEMv8R, symlinks
the directory to ~/.nros/sdks/arm-fvp/current/, and prints the
export ARMFVP_BIN_PATH=… line for your shell rc. Verify with
nros doctor --board fvp-aemv8r-smp, which cross-checks the
[gated.arm-fvp] entry in nros-sdk-index.toml and warns (never
hard-fails — license-gated) when the binary is missing.
FreeRTOS / NuttX / ThreadX SDK Paths
These are auto-resolved by justfile recipes (defaulting to external/ paths from just setup freertos / just setup nuttx / just setup threadx_linux). Override via env vars if sources are elsewhere.
| Variable | Default | Description |
|---|---|---|
FREERTOS_DIR | third-party/freertos/kernel | FreeRTOS kernel source |
FREERTOS_PORT | GCC/ARM_CM3 | FreeRTOS portable layer |
LWIP_DIR | third-party/freertos/lwip | lwIP source |
FREERTOS_CONFIG_DIR | Board crate’s config/ | FreeRTOSConfig.h + lwipopts.h |
NUTTX_DIR | third-party/nuttx/nuttx | NuttX RTOS source |
NUTTX_APPS_DIR | third-party/nuttx/nuttx-apps | NuttX apps source |
THREADX_DIR | third-party/threadx/kernel | ThreadX kernel source |
THREADX_CONFIG_DIR | Board crate’s config/ | ThreadX config (tx_user.h) |
NETX_DIR | third-party/threadx/netxduo | NetX Duo source |
NETX_CONFIG_DIR | Board crate’s config/ | NetX Duo config (nx_user.h) |
Buffer Tuning
All optional – platform-appropriate defaults apply if unset. See Configuration for deployment-scenario guidance and platform guides for target-specific sizing.
Zenoh-pico (ZPICO_*)
| Variable | Description | Default | Crate |
|---|---|---|---|
ZPICO_FRAG_MAX_SIZE | Max reassembled message size after defragmentation | 65536 / 2048 | zpico-sys |
ZPICO_BATCH_UNICAST_SIZE | Max unicast batch size before fragmentation | 65536 / 1024 | zpico-sys |
ZPICO_BATCH_MULTICAST_SIZE | Max multicast batch size | 8192 / 1024 | zpico-sys |
ZPICO_MAX_PUBLISHERS | Max concurrent publishers in zenoh shim | 8 | zpico-sys |
ZPICO_MAX_SUBSCRIBERS | Max concurrent subscribers in zenoh shim | 8 | zpico-sys |
ZPICO_MAX_QUERYABLES | Max concurrent queryables in zenoh shim | 8 | zpico-sys |
ZPICO_MAX_LIVELINESS | Max concurrent liveliness tokens in zenoh shim | 16 | zpico-sys |
ZPICO_MAX_PENDING_GETS | Max concurrent in-flight service calls | 4 | zpico-sys |
ZPICO_SUBSCRIBER_BUFFER_SIZE | Per-subscriber static buffer in zenoh shim | 1024 | nros-rmw-zenoh |
ZPICO_SUBSCRIBER_LARGE_SIZE | large size-class slot, for subscriptions whose type does not fit the small block | 16384 | nros-rmw-zenoh |
ZPICO_MAX_LARGE_SUBSCRIBERS | How many large-class blocks this image reserves. 0 is legal (phase-403 W4): an image whose subscribed types all fit ZPICO_SUBSCRIBER_BUFFER_SIZE declares 0 and stops paying RING_DEPTH x LARGE_SIZE — 65,536 B at the defaults — for a class it never routes into. A hint no class can hold then fails create_subscription rather than dropping every sample. | 2 | nros-rmw-zenoh |
ZPICO_SERVICE_BUFFER_SIZE | Per-service-server static buffer in zenoh shim | 1024 | nros-rmw-zenoh |
ZPICO_GET_REPLY_BUF_SIZE | Stack buffer for service client replies | 4096 | zpico-sys |
ZPICO_GET_POLL_INTERVAL_MS | Single-threaded polling interval in zenoh_shim_get() | 10 | zpico-sys |
NROS_SMOLTCP_MAX_SOCKETS | Max concurrent TCP sockets (smoltcp); brokered default since Phase 204.2. Legacy alias: ZPICO_SMOLTCP_MAX_SOCKETS. | 1 (brokered) | nros-smoltcp |
NROS_SMOLTCP_MAX_UDP_SOCKETS | Max concurrent UDP sockets (smoltcp); 1 by default, 4 with the nros-smoltcp/rtps feature (Phase 204.2). Legacy alias: ZPICO_SMOLTCP_MAX_UDP_SOCKETS. | 1 (brokered) | nros-smoltcp |
NROS_SMOLTCP_BUFFER_SIZE | Per-socket staging buffer (smoltcp). Legacy alias: ZPICO_SMOLTCP_BUFFER_SIZE. | 2048 | nros-smoltcp |
NROS_SMOLTCP_CONNECT_TIMEOUT_MS | TCP connection timeout (smoltcp). Legacy alias: ZPICO_SMOLTCP_CONNECT_TIMEOUT_MS. | 30000 | nros-smoltcp |
NROS_SMOLTCP_SOCKET_TIMEOUT_MS | TCP read/write timeout (smoltcp). Legacy alias: ZPICO_SMOLTCP_SOCKET_TIMEOUT_MS. | 10000 | nros-smoltcp |
XRCE-DDS (NROS_XRCE_*)
These are read by nros-rmw-xrce-cffi’s build script, either from the
environment or — on Zephyr — from CONFIG_<name> in .config. They set C
#defines of the same name without the NROS_ prefix; the define is not
itself an environment variable, so exporting XRCE_BUFFER_SIZE has no effect
and reports no error.
| Variable | Description | Default | Min | Crate |
|---|---|---|---|---|
NROS_XRCE_BUFFER_SIZE | Per-slot receive buffer. See the note below — this is the receive ceiling. | 1024 | 64 | nros-rmw-xrce |
NROS_XRCE_SUBSCRIBER_RING_DEPTH | Queued samples per subscriber | 32 | 1 | nros-rmw-xrce |
NROS_XRCE_MAX_SUBSCRIBERS | Max concurrent subscribers | 8 | 1 | nros-rmw-xrce |
NROS_XRCE_MAX_SERVICE_SERVERS | Max concurrent service servers | 4 | 1 | nros-rmw-xrce |
NROS_XRCE_MAX_SERVICE_CLIENTS | Max concurrent service clients | 4 | 1 | nros-rmw-xrce |
NROS_XRCE_STREAM_HISTORY | Reliable stream history depth; sizes the per-session output buffer | 16 | 4 | nros-rmw-xrce |
NROS_XRCE_CUSTOM_TRANSPORT_MTU | Custom transport MTU; stream buffers are MTU x STREAM_HISTORY | 4096 | 128 | nros-rmw-xrce |
Dropping MAX_SUBSCRIBERS, MAX_SERVICE_* and SUBSCRIBER_RING_DEPTH to 1,
BUFFER_SIZE to 256, STREAM_HISTORY to 4 and CUSTOM_TRANSPORT_MTU to 512
takes the session struct from ~390 KB to ~10-20 KB.
NROS_XRCE_BUFFER_SIZE is the XRCE receive ceiling
A subscriber’s receive ring entry is a fixed uint8_t data[XRCE_BUFFER_SIZE],
so a subscription’s own buffer size cannot raise it: asking for
create_subscription_sized::<M, 16384> still stops at 1024, because the 16384
buffer is the destination of a copy that never happens. A sample that does not
fit is refused on take with MessageTooLarge.
The static cost is XRCE_SUBSCRIBER_RING_DEPTH x XRCE_BUFFER_SIZE per
subscriber, which is why the default is small — raise it deliberately, and
consider lowering the ring depth at the same time.
Raising it stops helping at the transport MTU: at 4096 samples arrive corrupted rather than refused, with no error and no counter. That is issue 0819, open at time of writing. Keep payloads comfortably below the MTU.
Compile-time only (not environment variables)
These are #defines in packages/rmw/xrce/nros-rmw-xrce/src/internal.h with no
env or Kconfig knob. Changing them means editing that header.
| Define | Description | Default |
|---|---|---|
XRCE_ENTITY_CREATION_TIMEOUT_MS | Timeout for entity creation | 1000 |
XRCE_SERVICE_REPLY_TIMEOUT_MS | Per-attempt service reply timeout | 50 |
XRCE_SERVICE_REPLY_TOTAL_MS | Total service reply budget | 5000 |
XRCE_SESSION_FLUSH_TIMEOUT_MS | Session flush timeout | 100 |
XRCE_SESSION_CREATION_RETRIES | Session creation retries | 3 |
XRCE_MAX_PENDING_REPLIES | In-flight service replies | 4 |
XRCE_SERVICE_REQUEST_RING_DEPTH | Queued service requests per server | 4 |
XRCE_DEFAULT_AGENT_PORT | Default agent port | 2018 |
Core (NROS_*)
| Variable | Description | Default | Crate |
|---|---|---|---|
NROS_EXECUTOR_MAX_CBS | Max executor callback slots (compile-time fixed array size) | 4 | nros-node |
NROS_EXECUTOR_ARENA_SIZE | Executor arena size in bytes (compile-time fixed array size) | 4096 | nros-node |
NROS_SUBSCRIPTION_BUFFER_SIZE | Default subscription/service buffer size (bytes) | 1024 | nros-node |
NROS_EXECUTOR_MAX_HANDLES | Max handles in a C API executor | 16 | nros-c |
NROS_MAX_SUBSCRIPTIONS | Max subscriptions in a C API executor | 8 | nros-c |
NROS_MAX_TIMERS | Max timers in a C API executor | 8 | nros-c |
NROS_MAX_SERVICES | Max services in a C API executor | 4 | nros-c |
NROS_LET_BUFFER_SIZE | Buffer size for LET semantics per handle | 512 | nros-c |
NROS_MESSAGE_BUFFER_SIZE | Max buffer size for subscription/service data | 4096 | nros-c |
NROS_MAX_CONCURRENT_GOALS | Max concurrent goals per action server (compile-time constant, not env-var configurable) | 4 | nros-c |
NROS_MAX_PARAMETERS | Max parameters in parameter server | 32 | nros-params |
NROS_MAX_PARAM_NAME_LEN | Max parameter name length | 64 | nros-params |
NROS_MAX_STRING_VALUE_LEN | Max string parameter value length | 256 | nros-params |
NROS_MAX_ARRAY_LEN | Max parameter array length | 32 | nros-params |
NROS_MAX_BYTE_ARRAY_LEN | Max byte array parameter length | 256 | nros-params |
Static pool inventory
Every build-time sizing knob nano-ros reads, with the default it uses when you do not set it. Set them as environment variables at BUILD time.
This page exists because knowing a knob exists is the hard part. Issue 0271 audited a 256 KB image that already tuned nine of these and still carried ~145 KB of defaults it did not know to change — every knob it needed was already there. Four separate features had each added a static pool with a knob, silently.
Pools with a computed size
Bytes are at the DEFAULTS below, computed from the pool’s own declared
arithmetic (// nros-pool: in the source). Change a knob and the figure
moves with it.
| pool | bytes at default | formula | declared in |
|---|---|---|---|
LARGE_PAYLOADS | — (knob ZPICO_SUBSCRIBER_RING_DEPTH has a computed default) | ZPICO_MAX_LARGE_SUBSCRIBERS * ZPICO_SUBSCRIBER_RING_DEPTH * ZPICO_SUBSCRIBER_LARGE_SIZE | packages/rmw/zenoh/nros-rmw-zenoh/src/shim/subscriber.rs:200 |
SLOTS | 8,192 | NROS_RMW_SUBSCRIBER_SLOTS * 1024 | packages/rmw/cffi/src/rust_adapter.rs:111 |
SMALL_PAYLOADS | — (knob ZPICO_SUBSCRIBER_RING_DEPTH has a computed default) | ZPICO_MAX_SUBSCRIBERS * ZPICO_SUBSCRIBER_RING_DEPTH * NROS_SUBSCRIBER_BUFFER_SIZE | packages/rmw/zenoh/nros-rmw-zenoh/src/shim/subscriber.rs:199 |
Every sizing knob
A knob with no pool row above is still tunable; it simply has not declared its byte cost yet. Absence of a figure is not a claim that it is free.
| knob | default | read by |
|---|---|---|
NROS_EXECUTOR_ACTION_CLIENTS | computed — see packages/core/nros-node/build.rs:114 | packages/core/nros-node |
NROS_EXECUTOR_ARENA_SIZE | computed — see packages/core/nros-node/build.rs:251 | packages/core/nros-node |
NROS_EXECUTOR_MAX_CBS | 4 | packages/core/nros-node |
NROS_EXECUTOR_MAX_NODES | 4 | packages/core/nros-node |
NROS_EXECUTOR_MAX_SC | 8 | packages/core/nros-node |
NROS_EXECUTOR_MAX_SHUTDOWN_CBS | 2 | packages/core/nros-node |
NROS_FREERTOS_APP_STACK_KB | computed — see packages/tooling/nros-platform-config/src/platform_config.rs:893 | packages/tooling/nros-platform-config |
NROS_FREERTOS_HEAP_KB | computed — see packages/tooling/nros-platform-config/src/platform_config.rs:892 | packages/tooling/nros-platform-config |
NROS_KEYEXPR_STRING_SIZE | computed — see packages/tooling/nros-platform-config/src/platform_config.rs:1061 | packages/tooling/nros-platform-config |
NROS_LET_BUFFER_SIZE | computed — see packages/tooling/nros-build-helpers/src/c.rs:119 | packages/tooling/nros-build-helpers |
NROS_MAX_ARRAY_LEN | 32 | packages/core/nros-params |
NROS_MAX_BYTE_ARRAY_LEN | 256 | packages/core/nros-params |
NROS_MAX_PARAMETERS | 32 | packages/core/nros-params |
NROS_MAX_PARAM_NAME_LEN | 64 | packages/core/nros-params |
NROS_MAX_STRING_VALUE_LEN | 256 | packages/core/nros-params |
NROS_PARAM_SERVICE_BUFFER_SIZE | 4096 | packages/core/nros-node |
NROS_RMW_MAX_BACKENDS | 8 | packages/rmw/cffi |
NROS_RMW_MAX_NODES | 4 | packages/rmw/cffi |
NROS_RMW_MESSAGE_INFO_SLOTS | 64 | packages/rmw/cffi |
NROS_RMW_SUBSCRIBER_SLOTS | 8 | packages/rmw/cffi |
NROS_RUNTIME_COMPONENT_SLOT_BYTES | computed — see packages/api/nros/build.rs:47 | packages/api/nros |
NROS_RUNTIME_MAX_CELL_ENTITIES | computed — see packages/api/nros/build.rs:70 | packages/api/nros |
NROS_RUNTIME_MAX_CLASS_INSTANCES | computed — see packages/api/nros/build.rs:58 | packages/api/nros |
NROS_RUNTIME_MAX_COMPONENTS | computed — see packages/api/nros/build.rs:41 | packages/api/nros |
NROS_SERVICE_TIMEOUT_MS | 30000 | packages/rmw/zenoh/nros-rmw-zenoh |
NROS_SMOLTCP_BUFFER_SIZE | computed — see packages/drivers/net/nros-smoltcp/build.rs:57 | packages/drivers/net/nros-smoltcp |
NROS_SMOLTCP_CONNECT_TIMEOUT_MS | computed — see packages/drivers/net/nros-smoltcp/build.rs:62 | packages/drivers/net/nros-smoltcp |
NROS_SMOLTCP_MAX_SOCKETS | computed — see packages/drivers/net/nros-smoltcp/build.rs:44 | packages/drivers/net/nros-smoltcp |
NROS_SMOLTCP_MAX_UDP_SOCKETS | computed — see packages/drivers/net/nros-smoltcp/build.rs:52 | packages/drivers/net/nros-smoltcp |
NROS_SMOLTCP_SOCKET_TIMEOUT_MS | computed — see packages/drivers/net/nros-smoltcp/build.rs:67 | packages/drivers/net/nros-smoltcp |
NROS_SUBSCRIBER_BUFFER_SIZE | computed — see packages/core/nros-node/build.rs:180 | packages/core/nros-node (conflicting defaults — see below) |
NROS_SUBSCRIPTION_BUFFER_SIZE | 1024 | packages/core/nros-node |
NROS_XRCE_CUSTOM_TRANSPORT_MTU | computed — see packages/rmw/xrce/nros-rmw-xrce-cffi/build.rs:429 | packages/rmw/xrce/nros-rmw-xrce-cffi |
NROS_XRCE_STREAM_HISTORY | computed — see packages/rmw/xrce/nros-rmw-xrce-cffi/build.rs:240 | packages/rmw/xrce/nros-rmw-xrce-cffi |
NROS_ZEPHYR_HEAP_SIZE | computed — see packages/tooling/nros-platform-config/src/platform_config.rs:891 | packages/tooling/nros-platform-config |
ZPICO_BATCH_MULTICAST_SIZE | computed — see packages/tooling/nros-platform-config/src/platform_config.rs:1081 | packages/tooling/nros-platform-config |
ZPICO_BATCH_UNICAST_SIZE | computed — see packages/tooling/nros-platform-config/src/platform_config.rs:1080 | packages/tooling/nros-platform-config |
ZPICO_FRAG_MAX_SIZE | computed — see packages/tooling/nros-platform-config/src/platform_config.rs:1082 | packages/tooling/nros-platform-config |
ZPICO_GET_POLL_INTERVAL_MS | computed — see packages/tooling/nros-platform-config/src/platform_config.rs:1084 | packages/tooling/nros-platform-config |
ZPICO_GET_REPLY_BUF_SIZE | computed — see packages/tooling/nros-platform-config/src/platform_config.rs:1083 | packages/tooling/nros-platform-config |
ZPICO_LEASE_TASK_PRIORITY | 16 | packages/rmw/zenoh/nros-zpico-build |
ZPICO_MAX_LARGE_SUBSCRIBERS | 2 | packages/rmw/zenoh/nros-rmw-zenoh |
ZPICO_MAX_LIVELINESS | 16 | packages/rmw/zenoh/nros-zpico-build |
ZPICO_MAX_PENDING_GETS | 4 | packages/rmw/zenoh/nros-zpico-build |
ZPICO_MAX_PUBLISHERS | 8 | packages/rmw/zenoh/nros-zpico-build |
ZPICO_MAX_QUERYABLES | computed — see packages/rmw/zenoh/nros-zpico-build/src/runner.rs:497 | packages/rmw/zenoh/nros-zpico-build |
ZPICO_MAX_SESSIONS | 1 | packages/rmw/zenoh/nros-zpico-build |
ZPICO_MAX_SUBSCRIBERS | 8 | packages/rmw/zenoh/nros-zpico-build |
ZPICO_PUBLISHER_TX_BUFFER_SIZE | 1024 | packages/rmw/zenoh/nros-rmw-zenoh |
ZPICO_READ_TASK_PRIORITY | 16 | packages/rmw/zenoh/nros-zpico-build |
ZPICO_SERVICE_BUFFER_SIZE | 1024 | packages/rmw/zenoh/nros-rmw-zenoh |
ZPICO_SUBSCRIBER_LARGE_SIZE | 16384 | packages/rmw/zenoh/nros-rmw-zenoh |
ZPICO_SUBSCRIBER_RING_DEPTH | computed — see packages/rmw/zenoh/nros-rmw-zenoh/build.rs:50 | packages/rmw/zenoh/nros-rmw-zenoh |
ZPICO_SUBSCRIBER_SIZE_THRESHOLD | 2048 | packages/rmw/zenoh/nros-rmw-zenoh |
Conflicting defaults
These knobs are read in more than one place with DIFFERENT defaults, so setting one moves only part of the tree — the issue-0135 split-brain shape one layer up.
NROS_SUBSCRIBER_BUFFER_SIZE
Configuration surface
One fact can be spelled several ways. Only one of them is a place to make a decision; the rest carry that decision across a layer boundary. This page separates the two, because a list that does not is a list of nine ways to set a locator (issue 0934).
Every symbol on this page is one a user may set. Kconfig says so
itself: each carries a PROMPT, which is what a prompt means. An earlier
draft split these into public and carrier by asking who read them, and got
the layer wrong — cmake reading ${CONFIG_X} to emit ZPICO_X=<value> is
cmake CARRYING the choice, not owning it.
The real carriers are one layer down and are NOT Kconfig symbols: the
ZPICO_* / NROS_* environment variables, the -D cache variables and
the compile definitions this page’s symbols feed. Setting one of those by
hand bypasses the question Kconfig asked.
The evidence column is where each value is consumed, so a symbol with no
reader stands out: a prompt asking a question whose answer is discarded is
worse than no prompt.
80 Kconfig symbols — 79 settable, 1 provided for application code, 0 derived, 0 dead.
Facts with more than one spelling
Where a decision has an authority OUTSIDE Kconfig, that authority is the place to set it and these symbols carry it.
RMW backend
Set it here: [system].rmw in the bringup’s system.toml
| symbol | default | class | evidence |
|---|---|---|---|
CONFIG_NROS_RMW_ZENOH | — | public | cmake, rust-source, kconfig select/depends |
CONFIG_NROS_RMW_XRCE | — | public | c-source, cmake, rust-source |
CONFIG_NROS_RMW_CYCLONEDDS | — | public | cmake, rust-source, kconfig select/depends |
nano_ros_workspace(BACKEND …) and -DNROS_RMW are the cmake carriers. cmake already warns that BACKEND silently beats the cache var; nothing yet compares either against [system].rmw.
ROS domain id
Set it here: [system].domain_id in system.toml
| symbol | default | class | evidence |
|---|---|---|---|
CONFIG_NROS_DOMAIN_ID | 0 | public | c-source, cmake, rust-source, kconfig select/depends |
CONFIG_NROS_CYCLONE_DOMAIN_ID | NROS_DOMAIN_ID | public | c-source, cmake, rust-source |
CONFIG_NROS_CYCLONE_DOMAIN_ID defaults FROM NROS_DOMAIN_ID — the one derivation edge in this cluster. Pinning it to a literal is the phase-180 split-brain (CLAUDE.md).
Router / agent endpoint
Set it here: [system].locator in system.toml
| symbol | default | class | evidence |
|---|---|---|---|
CONFIG_NROS_ZENOH_LOCATOR | tcp/127.0.0.1:7447 | public | c-source, cmake, rust-source |
CONFIG_NROS_XRCE_AGENT_ADDR | 127.0.0.1 | public | c-source, rust-source |
CONFIG_NROS_XRCE_AGENT_PORT | 2018 | public | c-source, rust-source |
Zephyr is exempt from the cmake precedence chain by design (see the exemption comments in NanoRosEntry.cmake — no line ref, they move), so the Kconfig spelling is the carrier there rather than an override.
Executor sizing
Set it here: environment knobs at build time (see the pool inventory)
| symbol | default | class | evidence |
|---|---|---|---|
CONFIG_NROS_EXECUTOR_MAX_CBS | -1 | public | forwarded-to-cargo, cmake, rust-source |
CONFIG_NROS_EXECUTOR_MAX_SC | 8 | public | forwarded-to-cargo, cmake |
CONFIG_NROS_EXECUTOR_MAX_NODES | -1 | public | forwarded-to-cargo, cmake |
CONFIG_NROS_EXECUTOR_ARENA_SIZE | 0 | public | forwarded-to-cargo, cmake, rust-source |
CONFIG_NROS_EXECUTOR_MAX_SHUTDOWN_CBS | 2 | public | forwarded-to-cargo, cmake, rust-source |
CONFIG_NROS_EXECUTOR_ACTION_CLIENTS | -1 | public | forwarded-to-cargo, cmake |
CONFIG_NROS_SUBSCRIPTION_BUFFER_SIZE | -1 | public | forwarded-to-cargo, cmake |
CONFIG_NROS_PARAM_SERVICE_BUFFER_SIZE | 4096 | public | forwarded-to-cargo, cmake |
CONFIG_NROS_MAX_PARAMETERS | 32 | public | forwarded-to-cargo, cmake, rust-source |
On Zephyr the Kconfig spelling IS how a user sets these — it is a carrier only in the sense that the env var is what the build script reads. Issue 0460 is what happens when the bridge breaks.
Public — set these
Read by compiled code or forwarded to a build script. On Zephyr the
CONFIG_ spelling IS the way to state these; the env var is the carrier.
| symbol | default | evidence |
|---|---|---|
CONFIG_NROS_BATCH_UNICAST_SIZE | 1024 | forwarded-to-cargo, cmake, rust-source |
CONFIG_NROS_BOOT_REPORT | n | cmake, rust-source |
CONFIG_NROS_CODEGEN_TOOL | — | cmake |
CONFIG_NROS_CPP_API | — | cmake, kconfig select/depends |
CONFIG_NROS_CYCLONE_CONFIG_XML | — | c-source |
CONFIG_NROS_C_API | — | cmake |
CONFIG_NROS_FAULT_LOG | n | cmake, kconfig select/depends |
CONFIG_NROS_FAULT_LOG_FLASH | n | c-source |
CONFIG_NROS_FRAG_MAX_SIZE | 2048 | forwarded-to-cargo, cmake, rust-source |
CONFIG_NROS_GET_POLL_INTERVAL_MS | 10 | forwarded-to-cargo, cmake, rust-source |
CONFIG_NROS_GET_REPLY_BUF_SIZE | 4096 | forwarded-to-cargo, cmake, rust-source |
CONFIG_NROS_GRAPH_CACHE_SIZE | 65536 | cmake |
CONFIG_NROS_MAX_LARGE_SUBSCRIBERS | -1 | forwarded-to-cargo, cmake |
CONFIG_NROS_MAX_LIVELINESS | 16 | forwarded-to-cargo, cmake, rust-source |
CONFIG_NROS_MAX_PENDING_GETS | 4 | forwarded-to-cargo, cmake, rust-source |
CONFIG_NROS_MAX_PUBLISHERS | -1 | forwarded-to-cargo, cmake, rust-source |
CONFIG_NROS_MAX_QUERYABLES | -1 | forwarded-to-cargo, cmake, rust-source |
CONFIG_NROS_MAX_SUBSCRIBERS | -1 | forwarded-to-cargo, cmake, rust-source |
CONFIG_NROS_RMW_SUBSCRIBER_SLOTS | -1 | forwarded-to-cargo, cmake |
CONFIG_NROS_RUST_API | NROS_RMW_ZENOH | cmake, kconfig select/depends |
CONFIG_NROS_SERVICE_BUFFER_SIZE | 1024 | forwarded-to-cargo, cmake, rust-source |
CONFIG_NROS_SNTP_EPOCH | — | c-source, rust-source, kconfig select/depends |
CONFIG_NROS_SNTP_SERVER | 10.0.2.2 | c-source |
CONFIG_NROS_SNTP_TIMEOUT_MS | 3000 | c-source |
CONFIG_NROS_SUBSCRIBER_BUFFER_SIZE | -1 | forwarded-to-cargo, cmake, rust-source |
CONFIG_NROS_SUBSCRIBER_LARGE_SIZE | -1 | forwarded-to-cargo, cmake |
CONFIG_NROS_SUBSCRIBER_RING_DEPTH | 4 | forwarded-to-cargo, cmake |
CONFIG_NROS_TRACE_CALLBACKS | — | cmake |
CONFIG_NROS_TRANSPORT_SERIAL | — | kconfig select/depends |
CONFIG_NROS_XRCE_BUFFER_SIZE | 1024 | forwarded-to-cargo, cmake |
CONFIG_NROS_XRCE_MAX_SERVICE_CLIENTS | 4 | forwarded-to-cargo, cmake |
CONFIG_NROS_XRCE_MAX_SERVICE_SERVERS | -1 | forwarded-to-cargo, cmake |
CONFIG_NROS_XRCE_MAX_SUBSCRIBERS | -1 | forwarded-to-cargo, cmake |
CONFIG_NROS_XRCE_STREAM_HISTORY | 16 | forwarded-to-cargo, cmake |
CONFIG_NROS_XRCE_TRANSPORT_MTU | 512 | forwarded-to-cargo, cmake, rust-source |
CONFIG_NROS_ZENOH_AUTO_RECONNECT | y | cmake |
CONFIG_NROS_ZENOH_LEASE_FACTOR | 3 | c-define, cmake |
CONFIG_NROS_ZENOH_LEASE_MS | 10000 | c-define, cmake |
CONFIG_NROS_ZENOH_LEASE_PRIORITY | 200 | c-define, cmake, rust-source |
CONFIG_NROS_ZENOH_LINK_CAN | n | cmake |
CONFIG_NROS_ZENOH_LINK_ISOTP | n | cmake, kconfig select/depends |
CONFIG_NROS_ZENOH_LINK_SERIAL | n | cmake, kconfig select/depends |
CONFIG_NROS_ZENOH_LINK_TCP | y | cmake |
CONFIG_NROS_ZENOH_LINK_UDP_MULTICAST | n | cmake |
CONFIG_NROS_ZENOH_LINK_UDP_UNICAST | n | cmake |
CONFIG_NROS_ZENOH_LINK_WS | n | cmake |
CONFIG_NROS_ZENOH_MULTI_THREAD | y | cmake, kconfig select/depends |
CONFIG_NROS_ZENOH_PUBLICATION | y | cmake |
CONFIG_NROS_ZENOH_QUERY | y | cmake |
CONFIG_NROS_ZENOH_QUERYABLE | y | cmake |
CONFIG_NROS_ZENOH_RAWETH_TRANSPORT | n | cmake |
CONFIG_NROS_ZENOH_READ_PRIORITY | 200 | c-define, cmake, rust-source |
CONFIG_NROS_ZENOH_SCOUTING | n | cmake |
CONFIG_NROS_ZENOH_SOCKET_TIMEOUT_MS | 100 | c-define, cmake |
CONFIG_NROS_ZENOH_SUBSCRIPTION | y | cmake |
CONFIG_NROS_ZENOH_TX_BATCH | y | cmake, rust-source, kconfig select/depends |
CONFIG_NROS_ZENOH_TX_BATCH_FLUSH_MS | 50 | forwarded-to-cargo, cmake, rust-source |
CONFIG_NROS_ZENOH_TX_SPLIT_LOCK | y | cmake, rust-source |
CONFIG_NROS_ZEPHYR_HEAP_SIZE | 65536 | forwarded-to-cargo, c-source, cmake, rust-source |
CONFIG_NROS_ZEPHYR_STACKS_IN_DTCM | n | cmake |
CONFIG_NROS_ZEPHYR_TASK_SLOTS | 4 | c-define, c-source, cmake |
CONFIG_NROS_ZEPHYR_TASK_STACK_SIZE | 0 | c-define, cmake |
Provided for application code
No reader in THIS tree, and not dead: the guides teach an application
to use these in its own main(). Kconfig generating the macro is the
service. Each names where that contract is written down.
| symbol | default | evidence |
|---|---|---|
CONFIG_NROS_INIT_DELAY_MS | 2000 | docs/guides/cpp-api.md:451 shows an application calling zpico_zephyr_wait_network(CONFIG_NROS_INIT_DELAY_MS) from its own main(); docs/guides/zephyr-setup.md:220 documents it as a knob. |
Quick Reference
Finding Commands
Root commands are grouped by audience:
just --list # grouped root command overview
just --groups # group names
just --group main --list # normal development workflows
just --group full-matrix --list # build-all / fixtures / test-all / ci
just --group setup --list # setup and doctor entry points
just --group maintenance --list # clean and generated-binding commands
just --group docs --list # book and API docs
Platform and backend commands stay namespaced:
just --group full-matrix --list zephyr
just zephyr build-fixtures
just zephyr build-all
just qemu setup-network # QEMU TAP networking only
just zephyr help # Zephyr-specific help
Install the local book tooling before previewing docs:
just setup-docs # mdbook + mdbook-mermaid
just book-serve # serve book/src with live reload
just book # full deployed preview with API docs
Test profiling & slow-test reporting
just test and just test-all always print the slowest tests after the
run summary — the top 20 by duration (binary, test name, time, status),
parsed from the active profile’s target/nextest/<profile>/junit.xml. No
flag needed; it is part of the normal output.
Deeper profiling is opt-in and adds nextest’s experimental event/output recording plus artifact export. It preserves the normal nextest execution model (same filters, cargo profile, parallelism) — it only records and exports, so leave it off for routine runs:
NROS_NEXTEST_RECORD=1 just test
NROS_NEXTEST_RECORD=1 just test-all
Each profiled run writes a timestamped directory under tmp/ with a
stable -latest symlink:
tmp/nextest-profile-test-YYYYMMDD-HHMMSS/ tmp/nextest-profile-test-latest -> …
tmp/nextest-profile-test-all-YYYYMMDD-HHMMSS/ tmp/nextest-profile-test-all-latest -> …
Artifacts in that directory:
nextest-run.zip— portable recording archive; replay with full captured output (incl. successful tests) viacargo nextest replay.nextest-trace.json— Chrome/Perfetto timeline (slot/group occupancy, idle slots, retries, long-pole tests). Canonical concurrency artifact.junit.xml— copy of the run’s JUnit report.env.txt,command.txt— the knobs and exact command used.
Knobs:
| Variable | Effect |
|---|---|
NROS_NEXTEST_RECORD=1 | Enable recording + artifact export. |
NROS_NEXTEST_RECORD_DIR=<path> | Override the output dir (and its -latest link). |
NROS_NEXTEST_TRACE_GROUP_BY=slot|binary | Perfetto grouping; default slot (wall-clock/concurrency view). |
NROS_NEXTEST_REPLAY_LOG=1 | Also write nextest-replay.log (full captured stdout/stderr — can be large on chatty tests). Off by default; rely on the portable archive otherwise. |
NROS_NEXTEST_RECORD_KEEP_STATE=1 | Keep the temp NEXTEST_STATE_DIR (<dir>/state); removed after export otherwise. |
NROS_NEXTEST_RUN_PROFILE=fail-fast | Stop at the first failure instead of the default --no-fail-fast full report (uses target/nextest/fail-fast/junit.xml). |
Overhead and retention: recording adds event/output-store writes during
the run, and nextest-run.zip (plus an enabled nextest-replay.log) can
get sizable on output-heavy suites — keep it opt-in for local runs and
prune old tmp/nextest-profile-* directories. Recording uses a
profile-local NEXTEST_STATE_DIR, so it never pollutes the user’s global
nextest record store. Do not reach for --no-capture to inspect
output: it serializes execution and skews every timing. Use the replay
archive instead.
Manual Testing
# Base setup builds/checks the in-tree zenoh router.
just setup base
source ./activate.sh
# Terminal 1: Router
ZENOH_CONFIG_OVERRIDE='listen/endpoints=["tcp/127.0.0.1:7447"];scouting/multicast/enabled=false' ros2 run rmw_zenoh_cpp rmw_zenohd
# Terminal 2: Talker
cd examples/native/rust/talker && RUST_LOG=info cargo run --features zenoh
# Terminal 3: Listener
cd examples/native/rust/listener && RUST_LOG=info cargo run --features zenoh
UDP Transport
On the native (host) build, zenoh-pico has built-in UDP support via OS sockets:
# Use UDP instead of TCP for the zenoh locator
NROS_LOCATOR=udp/127.0.0.1:7447 cargo run --features zenoh
On bare-metal, enable the link-udp-unicast feature to use UDP over smoltcp:
nros = { features = ["rmw-zenoh", "platform-bare-metal", "link-tcp", "link-udp-unicast"] }
The
rmw-zenohfeature here is the lowering of the declared RMW — you declare the backend once insystem.toml([system].rmw/[image.<id>].rmw) and the toolchain sets the cargo feature; the feature is what the build uses, not the user-facing selector. See RFC-0031.
TLS Transport
TLS layers on top of TCP using mbedTLS. Requires a self-signed certificate (or real CA cert) and the link-tls Cargo feature.
Generate a test certificate:
openssl req -x509 -newkey ec -pkeyopt ec_paramgen_curve:prime256v1 \
-keyout key.pem -out cert.pem -days 365 -nodes -subj "/CN=localhost"
Native (host) (requires libmbedtls-dev – installed by just setup base):
# Terminal 1: Router with TLS
ZENOH_CONFIG_OVERRIDE='listen/endpoints=["tls/localhost:7447"];scouting/multicast/enabled=false' ros2 run rmw_zenoh_cpp rmw_zenohd \
--cfg 'transport/link/tls/listen_certificate:"cert.pem"' \
--cfg 'transport/link/tls/listen_private_key:"key.pem"'
# Terminal 2: Talker with TLS
NROS_LOCATOR=tls/localhost:7447 \
ZENOH_TLS_ROOT_CA_CERTIFICATE=cert.pem \
cargo run -p native-rs-talker --features link-tls
Bare-metal (QEMU ARM):
On bare-metal, only base64-encoded certificates are supported (no filesystem).
Build with --features link-tls:
# Build TLS-enabled examples
cd examples/qemu-arm-baremetal/rust/talker
cargo build --release --features link-tls
The CA certificate must be passed via ZENOH_TLS_ROOT_CA_CERTIFICATE_BASE64 at runtime
(through the zenoh config), or embedded in the binary at build time.
ROS 2 Interop
# Terminal 1: Router
ZENOH_CONFIG_OVERRIDE='listen/endpoints=["tcp/127.0.0.1:7447"];scouting/multicast/enabled=false' ros2 run rmw_zenoh_cpp rmw_zenohd
# Terminal 2: nros talker
cd examples/native/rust/talker && RUST_LOG=info cargo run --features zenoh
# Terminal 3: ROS 2 listener
source /opt/ros/humble/setup.bash
export RMW_IMPLEMENTATION=rmw_zenoh_cpp
ros2 topic echo /chatter std_msgs/msg/String --qos-reliability best_effort
Actions
# Terminal 1: Router
ZENOH_CONFIG_OVERRIDE='listen/endpoints=["tcp/127.0.0.1:7447"];scouting/multicast/enabled=false' ros2 run rmw_zenoh_cpp rmw_zenohd
# Terminal 2: Action server (Fibonacci example)
cd examples/native/rust/action-server && cargo run
# Terminal 3: Action client
cd examples/native/rust/action-client && cargo run
Zephyr action tests:
just zephyr build-all # Build all Zephyr examples (Rust + C + C++ + XRCE)
just zephyr test # Run zenoh E2E tests (covers actions)
Docker Development Environment
Docker provides QEMU 7.2 (from Debian bookworm) which fixes TAP networking issues present in Ubuntu 22.04’s QEMU 6.2.
# One-time setup: add yourself to docker group
sudo usermod -aG docker $USER
# Log out and back in, or run: newgrp docker
# Build and use Docker environment
just docker build # Build nano-ros-qemu image
just docker shell # Interactive shell
just docker test-qemu # Run QEMU tests in container
just docker help # Show all Docker commands
QEMU Bare-Metal Testing
Run bare-metal Cortex-M3 examples on QEMU (MPS2-AN385 machine with LAN9118 Ethernet).
# Build prerequisites
just qemu build-zenoh-pico # Build zenoh-pico for ARM Cortex-M3
just qemu build # Build all QEMU examples
# Non-networked tests (no setup required)
just qemu test # Bare-metal QEMU integration tests
# Networked talker/listener test (Docker Compose - recommended)
just docker test-qemu # Runs zenohd, talker, listener in separate containers
Docker Compose Architecture:
┌─────────────────────────────────────────────────────────────┐
│ Docker Network: 172.20.0.0/24 │
│ │
│ ┌─────────────┐ ┌─────────────┐ ┌─────────────────────┐ │
│ │ zenohd │ │ talker │ │ listener │ │
│ │ 172.20.0.2 │ │ 172.20.0.10 │ │ 172.20.0.11 │ │
│ │ │ │ ┌───────┐ │ │ ┌───────────────┐ │ │
│ │ │ │ │ QEMU │ │ │ │ QEMU │ │ │
│ │ │ │ │ ARM │──┼──┼──│ ARM │ │ │
│ │ │ │ │ TAP │ │ │ │ TAP │ │ │
│ │ │ │ └───────┘ │ │ └───────────────┘ │ │
│ └──────▲──────┘ └──────┼──────┘ └─────────┼───────────┘ │
│ └────────────────┴───────────────────┘ │
│ NAT to zenohd │
└─────────────────────────────────────────────────────────────┘
Manual networked test (3 terminals, requires host TAP setup):
# Terminal 1: Setup network + start router
just qemu setup-network # Requires sudo
ZENOH_CONFIG_OVERRIDE='listen/endpoints=["tcp/0.0.0.0:7447"];scouting/multicast/enabled=false' ros2 run rmw_zenoh_cpp rmw_zenohd
# Terminal 2: Talker (192.0.2.10)
bin=build/cargo-fixtures/qemu-arm-baremetal/thumbv7m-none-eabi/nros-relwithdebinfo
./scripts/qemu/launch-mps2-an385.sh --tap tap-qemu0 --binary "$bin/qemu-bsp-talker"
# Terminal 3: Listener (192.0.2.11)
./scripts/qemu/launch-mps2-an385.sh --tap tap-qemu1 --binary "$bin/qemu-bsp-listener"
Build those first with just qemu build-fixtures (or just build-test-fixtures lane=native).
The artifact directory is computed, not fixed. Since phase-340 P2 a
platform’s cargo fixtures build into a shared group directory under
build/cargo-fixtures/<group>/ rather than each leaf’s own target/,
and the profile is nros-relwithdebinfo, not release. The group name
can carry a hash suffix when a row builds with distinct env, so treat
the path above as what it is on a default build, not as a guarantee —
scripts/build/fixtures-target-dir.sh (nros_fixture_row_artifact_dir)
is the single source of truth, and it is what the test side uses to
find the same file. That helper takes the row’s cargo args and env as
arguments, so a script that did not itself run the build should ask for
the row by id — nros_fixture_row_artifact_dir_by_id <row-id> reads all
three key fields from the manifest, and cannot disagree with the builder
about which group a row landed in (issue 1025).
Run just qemu help for more options.
Zephyr Setup
just setup zephyr # Initialize workspace at $repo/zephyr-workspace/
just zephyr test # Run zenoh tests (native_sim uses NSOS on host loopback)
just zephyr test-xrce # Run XRCE tests
The workspace lives at $repo/zephyr-workspace/ by default (gitignored).
Set $NROS_ZEPHYR_WORKSPACE to install elsewhere. Legacy sibling installs
at ../nano-ros-workspace/ are auto-detected; run
./scripts/zephyr/migrate-workspace.sh to consolidate.
See Zephyr for full details.
nros-bridge.toml — bridge configuration schema
Consumed by nros::run_from_config (Rust) and the future C/C++ mirror
(nros::run_from_config in nros-cpp).
Not the orchestration
nros.toml. This is the bridge runtime config (multi-RMW byte forwarding). It is namednros-bridge.tomlto avoid colliding with the orchestrationnros.toml(component/system build config) — different lifecycle, different schema.
The file lives next to the binary (or anywhere the program can read it) and selects:
- Which RMW backends to open, under what locator / domain.
- How those backends are wired into bridge entries that forward traffic between them.
The binary’s Cargo.toml (or target_link_libraries) still selects
which backends are linked — the TOML file selects which of the
linked backends are used in this run. Backend names in the file
that don’t match a linked backend surface as
ConfigError::OpenSession.
Top-level structure
# nros-bridge.toml — sibling of the binary
[[node]]
name = "field"
rmw = "zenoh"
locator = "tcp/10.0.0.1:7447"
[[node]]
name = "control"
rmw = "cyclonedds"
locator = "domain=0"
[[bridge]]
type = "std_msgs/Int32"
type_hash = "RIHS01_..."
from = { node = "field", topic = "/sensor/raw" }
to = { node = "control", topic = "/sensor/raw" }
Run via:
fn main() -> Result<(), nros_bridge::ConfigError> {
nros_bridge::run_from_config("nros-bridge.toml")
// or, via the umbrella feature: nros::run_from_config("nros-bridge.toml")
}
[[node]]
One block per backend session. The first [[node]] becomes the
primary session (extra_sessions[0] in Executor::open_multi); the
rest open as extras keyed by rmw.
| Key | Type | Default | Notes |
|---|---|---|---|
name | string | required | Logical name. Bridge entries reference this. |
rmw | string | required | Canonical backend name: "zenoh", "xrce", "cyclonedds". Must match a backend the binary linked in. |
locator | string | "" | Backend-specific locator. Zenoh uses tcp/..., udp4/..., serial/...; DDS uses domain=<n>; XRCE uses udp4://...:port. |
domain_id | u32 | 0 | ROS domain id passed to the backend. |
namespace | string | "/" | Default namespace applied to handles created on this node. |
Locator scheme grammar (zenoh)
| Scheme | Example | Meaning |
|---|---|---|
tcp/<host>:<p> | tcp/10.0.0.1:7447 | TCP unicast |
udp/<host>:<p> | udp/10.0.0.1:7447 | UDP unicast |
serial/<dev> | serial//dev/ttyUSB0 | UART (host-side) or board UART (bare-metal) |
tls/<host>:<p> | tls/router.example.org:7447 | TLS over TCP (requires link-tls on backend) |
DDS / XRCE follow their native locator conventions; consult each backend’s docs for the exact form.
[[bridge]]
One block per topic forwarded between two [[node]]s. Each bridge
creates a RawSubscription on the source side and an
EmbeddedRawPublisher on the destination side and pumps bytes once
per executor tick.
| Key | Type | Default | Notes |
|---|---|---|---|
type | string | required | ROS type name ("std_msgs/Int32"). Backends use it for liveliness / discovery. |
type_hash | string | "" | ROS 2 RIHS type hash for the typed binding. May be left empty if both sides ignore it. |
from | endpoint | required | { node = "<name>", topic = "<topic>" }. The source node + topic. |
to | endpoint | required | { node = "<name>", topic = "<topic>" }. The destination node + topic. |
Bidirectional bridges are two [[bridge]] entries — one in each
direction. The runtime stamps a bridge_origin attachment field on
forwarded frames and drops on receive when the origin matches the
local backend, so an echo pair does not loop.
Error semantics
run_from_config returns ConfigError:
| Variant | Cause |
|---|---|
Io | File read failed (missing / unreadable). |
Parse | TOML malformed or a required field missing. |
UnknownNode | A [[bridge]] references a node name no [[node]] declared. |
OpenSession | Executor::open_multi rejected a spec — usually a backend name no RMW_INIT_ENTRIES entry registered under. |
BuildNode | create_node_on failed (registry exhausted, name too long, …). |
BuildEntity | Creating the source subscription or destination publisher failed (typically backend rejection of the topic name / type / QoS). |
Any error short-circuits the runtime; bridges that opened cleanly before the failure are not pumped.
Linked-backend matrix
| Backend name | Cargo dep that contributes the RMW_INIT_ENTRIES entry |
|---|---|
zenoh | nros-rmw-zenoh = { ... } |
xrce | nros-rmw-xrce-cffi = { ... } |
cyclonedds | nros-rmw-cyclonedds static lib (CMake side, --whole-archive) |
Add a backend to your Cargo.toml (or target_link_libraries) to
make its name available in the TOML; remove it to fence off use even
when the TOML file lists it (yields OpenSession at startup).
Migrating off just install-local
TL;DR. deleted just install-local, the build/install/
prefix, every install(...) rule, every Config.cmake.in template,
and the find_package(NanoRos) consumption path. nano-ros is now
consumed exclusively via add_subdirectory(<repo-root>) from the
user’s CMakeLists.txt.
If your project was on a pre-140 checkout, this page is the one-page rewrite that gets you onto the supported shape.
Before (pre-140)
cmake_minimum_required(VERSION 3.16)
project(my_app C)
find_package(NanoRos REQUIRED CONFIG)
nano_ros_generate_interfaces(std_msgs SKIP_INSTALL)
add_executable(my_app src/main.c)
target_link_libraries(my_app PRIVATE
std_msgs__nano_ros_c
NanoRos::NanoRos)
Build:
# In the nano-ros checkout:
just install-local # populates build/install/
# In the user project:
cmake -S . -B build -DCMAKE_PREFIX_PATH=<nano-ros>/build/install
cmake --build build
After (post-140)
cmake_minimum_required(VERSION 3.22)
project(my_app C)
set(NANO_ROS_PLATFORM posix)
set(NANO_ROS_RMW zenoh)
add_subdirectory(<path-to-nano-ros> nano_ros)
nano_ros_generate_interfaces(std_msgs SKIP_INSTALL)
add_executable(my_app src/main.c)
target_link_libraries(my_app PRIVATE
std_msgs__nano_ros_c
NanoRos::NanoRos)
nros_platform_link_app(my_app)
Build:
# No nano-ros-side install step. Just configure your project.
cmake -S . -B build
cmake --build build
What changed
| Pre-140 | Post-140 |
|---|---|
find_package(NanoRos REQUIRED CONFIG) | add_subdirectory(<path-to-nano-ros>) after set(NANO_ROS_PLATFORM ...) + set(NANO_ROS_RMW ...) |
just install-local | (deleted) — nano-ros builds in-tree per-example via Corrosion |
cmake -DCMAKE_PREFIX_PATH=<nano-ros>/build/install | (not needed) |
nano_ros_link_platform(target) / nano_ros_link_rmw(target) | (folded into NanoRos::NanoRos’s INTERFACE link libraries) |
nros_freertos_compose_platform(target) / nros_threadx_compose_platform(target) | nros_platform_link_app(target) — single per-app fixup hook |
find_package(NrosRmwCyclonedds CONFIG REQUIRED) | (auto-wired when NANO_ROS_RMW=cyclonedds) |
cmake_minimum_required(VERSION 3.16) | cmake_minimum_required(VERSION 3.22) (matches root nano-ros) |
RTOS-specific notes
- Zephyr. Consume nano-ros via the
zephyr/module — dropnano-rosinto yourwest.yml, setCONFIG_NROS=y+CONFIG_NROS_RMW="zenoh"inprj.conf. - ESP-IDF.
integrations/nano-ros/is a component manifest — add it to youridf_component.yml. - NuttX.
integrations/nuttx/is aapps/external/shim — symlink (or copy) and enable viamake menuconfig. - PX4.
integrations/px4/module-template/is aEXTERNAL_MODULES_LOCATIONtemplate — copy out, edit the user glue, point PX4 at it.
Each shell internally does
add_subdirectory(<nano-ros>) with the right
NANO_ROS_PLATFORM cache var set; user-side code is the same as
the C/C++ snippet above.
What you lose
- A pre-built
<prefix>/lib/libnros_c.ayou could link from arbitrary external projects. Post-140, each consumer builds the staticlib in its own build tree (Corrosion handles caching via the Cargo target dir). - The
build/install/share/nano-ros/interfaces/bundled-interface drop. Codegen now resolves interface files via the ament index (when colcon-discovered) or directly frompackages/cli/interfaces/inside the nano-ros checkout. - The
cmake --install buildstep. There is nothing nano-ros-side to install. Your own project ships its binary; nano-ros is a static dependency.
Why this change
find_package(NanoRos) was a Debian-style “library installed
once, consumed by many projects” model. The RTOS workflows that
nano-ros actually targets (Zephyr west, ESP-IDF,
NuttX apps/external, PX4 EXTERNAL_MODULES_LOCATION) all
consume dependencies as source trees inside the user’s workspace —
not from an installed prefix. The install path drifted out of sync
with the cargo path more than once (UDP multicast
linker error was the proof) and added ~30 s warm / ~10 min cold to
every test-all run. Removing it collapses two surfaces into one.
See docs/roadmap/phase-140-install-local-rip-off.md for the full
design notes and the audit table that drove the deletion.