Start Here

This is nano-ros Internals, the book for people who change nano-ros itself. If you want to use nano-ros (write nodes, bring up a board, add a board or an RMW backend from outside the tree), read the nano-ros Book instead. Everything there works from an installed release, while everything here assumes a checkout.

The two books and the records

The three numbered series are records. This book is narrative: it explains how the parts work and links to the record that decided each one, without copying it.

The books are split by reader, not by topic (RFC-0123). When a subject has a user side and a contributor side, each book has its own page and the two link to each other. For example, the Book’s ROS 2 interop page says what to do, and RMW Internals → Zenoh protocol says how it works.

First hour

1. Set up a checkout: Contributing.
2. Learn the shape of the system: Architecture, then the repo map.
3. Learn the verbs. just with no arguments lists about eight, grouped by when you need them, and just check <name> runs one gate. Build Commands has the full list.
4. Before you open a pull request, run just check fast.

Where a new fact goes

When you learn something that will still be true later, file it where the next person will look:

• a design decision goes in an RFC;
• planned work goes in a phase doc;
• a bug or limitation goes in an issue;
• how something works, for contributors, goes on a page of this book;
• how to use something goes on a page of the nano-ros Book.

CLAUDE.md and AGENTS.md at the repo root are routers for coding agents. They point here and into docs/, and they deliberately hold no design detail.

Architecture

The whole-system design is written once, in docs/design/ARCHITECTURE.md. That document narrates the system end to end and links into the RFC behind each part. This page is a reading order for it, not a second copy.

Read in this order

1. The layers. Application, nano-ros core, RMW backend, platform. Each layer can be swapped on its own. RFC-0001 has the crate stack and the directory map.
2. The feature axes. RMW × platform × ROS edition (ARCHITECTURE §2, RFC-0005, RFC-0006).
3. How an image is built. There are three roads (cargo, CMake, west), and each carries knobs to the compiler differently: book-internals/src/pipeline/canonical-build-path.md. nros configure is the single resolver every road calls (RFC-0120).
4. How an image runs. A target is a launcher or flasher, a console and a link, executed from one RunPlan (RFC-0118).
5. What a board is. A board is a descriptor plus a registry row. Every support claim is generated from the registry (RFC-0114).
6. The CLI’s shape. A user tier and an nros dev tier (RFC-0117), extended by runner and command packages (RFC-0119).

The user-facing view of the same system, with fewer moving parts, is the Book’s Architecture Overview.

What backs the Book’s claims

The Book’s introduction states its benefits without evidence, for readers. This is the evidence, and the place to update when a claim changes.

• “One node, any board.” The example_portability test normalizes every platform’s copy of each example and fails if any two differ. Two execution models are declared exceptions, with written reasons: bare-metal deferred dispatch, and Zephyr’s component shape. The test names them rather than hiding them.
• “Fits a microcontroller.” The core is no_std; alloc and std are opt-in features. Whether an image needs an allocator is the backend’s call: XRCE allocates only at entity setup, zenoh-pico wants an allocator (a bump allocator suffices on bare-metal), and Cyclone DDS allocates per message. scripts/rmw-alloc-sites.py reports every allocation site.
• “Checked, not only tested.” Kani harnesses and Verus proofs cover CDR serialization, scheduling and protocol correctness. The current counts and how to run them are in Formal Verification.

Repo Map

What each top-level directory holds. The packages/ breakdown is kept in one place, the “Directory map” in RFC-0001, and check-package-directories checks it against the tree, so it is not repeated here.

Contributing

This chapter covers the development workflow for nano-ros.

Development Setup

This is the contributor setup: a checkout of the repository and the CLI built from it. If you only want to use nano-ros, install a release instead — Installation; every user chapter of this book assumes the installed CLI and no checkout.

Inside a nano-ros checkout the tree’s own build is the only correct nros: a released nros run against a checkout is refused outright, because it emits its own generated code and this tree’s runtime is the one that has to compile it (RFC-0090). One checkout = one CLI version = one runtime ABI — no global install, no ~/.nros/bin PATH skew across worktrees.

The host prerequisites are the ones in Installation § Host prerequisites (python3-dev is needed here: bootstrap.sh builds the launch resolver, which embeds CPython to parse ROS launch files).

From a fresh checkout

# 1. Pull the source (`main`, or a release tag `nros-<version>`):
git clone https://github.com/NEWSLabNTU/nano-ros.git
cd nano-ros

# 2. Build the `nros` CLI from source (installs rustup if needed):
./scripts/bootstrap.sh

# 3. Wire the workspace env into this (and every future) shell:
source ./activate.sh          # or: direnv allow / source ./activate.fish

# 4. Provision the toolchains + RMW daemon for your target:
nros setup <board> --rmw <zenoh|xrce|cyclonedds>    # e.g. `linux`

bootstrap.sh builds the in-tree nros CLI from source at packages/cli/target/release/nros and leaves it on PATH for that shell (just is NOT a prerequisite; rustup is installed on demand). Already have cargo? The equivalent by hand:

git submodule update --init packages/cli/third-party/play_launch
cargo build --release --manifest-path packages/cli/Cargo.toml --bin nros
cargo build --release --manifest-path packages/cli/nros-launch-resolve/Cargo.toml
export PATH="$PWD/packages/cli/target/release:$PATH"

Both builds are needed for a workspace flow: nros configure and nros build call nros-launch-resolve to resolve launch files, so a bare cargo build --bin nros alone is not equivalent.

Every subsequent shell

Every new shell needs the workspace env wired in. All three wire the same env exports + PATH entries from activate.sh:

direnv allow                  # auto-activates on `cd nano-ros` (recommended)
source ./activate.sh          # bash / zsh, one-shot per shell
source ./activate.fish        # fish, one-shot per shell

activate.sh is the after-install step, not a prerequisite: it presumes the nros binary already exists at packages/cli/target/release/nros. It also sources /opt/ros/humble/setup.bash if present (used by the rmw_zenoh interop tests, and it lets codegen resolve message packages from the ament index). It exports no checkout root: the in-tree nros knows the checkout it was built in. Without ROS 2 it says so:

activate.sh: /opt/ros/humble/setup.bash not found — ROS-dependent recipes
  will fail (interop tests, `ros2` CLI verification). The setup, codegen
  and first-node flows do not need it.

On a host without just it also prints one line about RTOS SDK path defaults (FREERTOS_DIR, NUTTX_DIR, THREADX_DIR, …) not being loaded — only the in-tree just recipes (test lanes, fixture builds) read them (cargo install just).

Provisioning through just

The just setup recipes are thin wrappers over the same nros setup index — just setup <module> calls nros setup <board> under the hood, so the toolchains a contributor gets are identical to a user’s:

just setup freertos           # → nros setup mps2-an385-freertos
just setup nuttx              # → nros setup qemu-armv7a-nuttx
just setup threadx_linux      # → nros setup threadx-linux
just doctor tier=all          # diagnose missing tools (read-only)

A checkout’s version is its tag: the CLI built from a nros-<version> tree prints the same number (scripts/check-version-lockstep.sh and bootstrap.sh shell-doctor enforce binary↔tree agreement), and the tree’s committed nros-sdk-index.toml pins the toolchains. Upgrading = checking out a newer tag, re-running ./scripts/bootstrap.sh, then nros setup.

Docker environment

For a containerized environment with QEMU 7.2+ (in-tree checkout):

just docker build
just docker shell      # Interactive shell with all tools
just docker test-qemu  # Run QEMU tests in container

Setup tiers

Choose the setup tier. base is enough for most code review and native development; all prepares the full test-all matrix:

just setup               # print setup choices
just setup base          # workspace tools + zenohd
just setup all           # all supported SDKs/services
source ./activate.sh

The test and fixture stages need the vendored-source UNION, not one board. nros setup <board> --rmw <x> provisions one board+rmw’s sources — right for building an app for that board. But just test links every RMW’s -sys crate and just build-test-fixtures resolves component graphs that path-dep platform sources (e.g. nuttx/libc) even for a native component, so the contributor build needs the whole set. Both recipes preflight it and, if a source is missing, name the exact remedy — nros setup --source <name>, or nros setup --build-sources to provision the union at once (issue 0390).

Build the entire workspace including examples:

just build

Use groups to inspect the command surface:

just --group main --list
just --group full-matrix --list
just --group full-matrix --list zephyr

Quality Checks

Always run just ci after completing a task that needs the full maintainer gate. This runs the normal checks plus the full test matrix entry points:

just ci

For a narrower local iteration, run just check and just test.

For platform-specific CI, use just <platform> ci (e.g., just freertos ci).

Testing

nano-ros has several test tiers, each with its own just recipe:

All test recipes accept a verbose argument for live output.

Test organization

• Reusable Rust tests go in packages/testing/nros-tests/tests/
• Shell-based test scripts go in tests/ with corresponding justfile entries
• Temporary tests can be run directly in the shell, then converted to proper tests once validated

Build isolation

Nextest runs each test file as a separate process in parallel. When multiple tests build the same example with different features, use --target-dir to isolate output directories (e.g., target-safety/, target-zero-copy/). Add new target dirs to the example’s per-directory .gitignore.

Code Style

Rust Edition 2024

nano-ros uses Rust edition 2024, which requires:

• unsafe extern "C" { ... } – extern blocks require the unsafe keyword
• #[unsafe(no_mangle)] – no_mangle requires the unsafe attribute
• Unsafe operations inside unsafe fn need explicit unsafe { ... } blocks

The nros-c crate keeps #![allow(unsafe_op_in_unsafe_fn)] due to 420+ FFI operations.

Unsafe conventions

• Minimize unsafe usage; prefer safe abstractions
• Document safety invariants with // SAFETY: comments
• Use wrapper types (like SubscriberBufferRef) to encapsulate unsafe access patterns

no_std patterns

All core crates support #![no_std] with optional std/alloc features:

• #[cfg(feature = "alloc")] gates Vec/Box usage
• #[cfg(feature = "std")] gates std-only features
• heapless::Vec is used in no_std contexts

Unused variables

• Rename to _name with a comment explaining why
• Use #[allow(dead_code)] for test struct fields

Message Types

Message types are always generated, never hand-written. Use the codegen tool:

nros dev codegen rust

Bundled interface definitions live in packages/cli/interfaces/. Example generated/ directories are gitignored and recreated by just generate-bindings. Only the core pre-generated set checks its generated/ into git: ONE driver package at packages/interfaces/, one output tree at packages/interfaces/generated/humble/, six crates, regenerated with just generate-interfaces on a ROS 2 ament host.

System Packages

Never install system packages or run sudo directly. If a system dependency is needed, document what the user should install.

Temporary Files

Create temporary files in the project’s tmp/ directory (git-ignored), not in /tmp.

PR Workflow

1. Create a branch from main
2. Make changes
3. Run just ci – all checks must pass
4. Submit for review

Verification

For changes to core crates, run the formal verification suite:

just verify          # Kani + Verus
just verify-kani     # Kani bounded model checking (~3 min)
just verify-verus    # Verus deductive proofs (~1 sec)

Writing the Book

This page is for anyone who writes or reorganizes a page of the nano-ros Book, the user book under book/src/. It is the charter of RFC-0124 put as a checklist, followed by one template for each kind of page. Read the checklist before you open a template, and check your page against it before you ask for review.

check-book-audience reads the gated rules. It reports them today (phase-537 W0) and refuses them once phase-537 W13 lands:

python3 scripts/check-book-audience.py --charter

The checklist

Who and why

1. Three purposes, stated budgets. Every page serves at least one of education, field solutions and industrial, within its budget: a 15-minute install and labs under two hours; 30 minutes from box to ros2 topic echo; reproducible, auditable, offline builds (RFC-0116 D1).
2. Two books by reader. Application developers and porters read the Book. Contributors read this book, Internals. Contributor material in a user page moves here.

What may be in the Book

3. A board enters the Book only with an emulator twin, or a track that replays its chapter (RFC-0115 D4). QEMU mps2-an385 is the concepts board. Gated: a board page whose registry rows are neither Verified nor an emulator twin of a named board is reported.
4. Claims follow the registry. Board facts and status words are generated, never typed. A board, feature or integration with no registry row gets no user page. Gated (rule E refuses a typed status word).
5. Only the user tier. No nros dev, no checkout steps, no internal ids in prose. RFC numbers appear only in a closing “Design background”. Gated (rules D and check-user-surface-ids refuse).

How it is organized

6. Sections follow what the reader is doing: Start here, Get started on your board, Tutorial, User guide, Background, Reference, Extending, Releases. RFC-0124 Appendix A names every page and its sources. A page not in Appendix A needs an RFC amendment first.
7. One topic, one page per type. Tutorial, how-to, explanation and reference are separate pages that link to each other. A topic has at most one page of each type. Gated: the declared type must fit the section, and two pages may not declare the same type and topic.
8. Starters before depth. Every admitted board has a starter that needs only Install, twin-and-hardware boards first.
9. Reference is generated from its owner: the contract format, the CLI, the APIs, system.toml, boards. A hand-written table that copies a generated one is deleted.

How it is written

10. Procedure first. A task page is steps. Each step is a command or an edit, followed by its expected output. The page ends with a complete working example.

11. Concepts at the point of need. Explain a concept only if it is uncommon outside nano-ros and central inside it: the bringup, the image, the contract, static pools, the execution tiers. Start from the example in front of the reader, not from the model. For what a ROS 2 user knows, link to the ROS 2 documentation.

12. The simplest working path first. Serial before ethernet, with WiFi stated as deferred. The first chapter names no launch file, image or entry. One run verb, nros run, with nros flash --monitor on hardware.

13. Say plainly where nano-ros differs from ROS 2. Never imply a symmetry that is not there.

14. Every runnable step is probed on its row’s evidence. A step that cannot run yet carries an awaits line on its track and fails loudly. Nothing is skipped. Gated: a shell fence with no probe= is reported.

15. Budgets. Gated:

    Type	Words of prose
    how-to	300–1,500
    tutorial chapter	1,500–3,000
    explanation	800–2,500
    reference	no limit, but tables and lookups only

    A page under 300 words is merged into its parent. Rust, C and C++ variants of one step are tabs on one page, not parallel pages.

Voice

Follow the Google developer documentation style guide: second person, present tense, active voice, sentence-case headings, and no “simply”, “just” or “easy”. Name a thing once, then use that name throughout.

Declare the type and level

Every hand-written user page carries one declaration on the line after its title. Generated pages are reference pages and carry none.

# Configure zenoh

<!-- type: how-to · level: 2 -->

Add · topic: <slug> when a topic has pages of several types, so the gate can tell that each type appears once:

<!-- type: tutorial · level: 2 · topic: contract -->

Topic levels

A page may assume only topics at its own level or below. It lists them on a “needs first” line, and the gate reports a link to a higher level.

A reference page carries the level of the most advanced topic it describes. Anyone may look it up.

Templates

Each template gives the headings in order. Replace the comments. Keep the headings a reader scans for, even when a section is one line.

Tutorial chapter

One chapter of the one Tutorial project, run on a Verified row, small enough for a two-hour lab.

# <What the reader does, as a verb phrase>

<!-- type: tutorial · level: <1|2> -->

**Needs first:** [<previous chapter>](<link>)

<!-- One paragraph: what the reader has at the end, stated as output they will see. -->

## <Step 1, a verb phrase>

<!-- The command or edit. Then the output they should see. -->

## <Step n>

## What just happened

<!-- The ONE nano-ros concept this chapter introduces, explained from the
     example the reader just ran. Link to Background for the full model. -->

## Next

<!-- The next chapter, and the User guide pages that go deeper. -->

How-to

One task, for a reader who already knows what they want.

# <Task, as a verb phrase: "Use a serial link">

<!-- type: how-to · level: <2|3|4> -->

**Needs first:** [<page>](<link>)

<!-- One sentence: when you would do this. -->

## Before you start

<!-- What must already be true: board, packages, a working project. -->

## Steps

1. <!-- A command or edit. -->

   <!-- Its expected output. -->

## Example

<!-- One complete, working example the reader can copy whole. -->

## If it doesn't work

<!-- The two or three failures a reader actually meets, each with its fix. -->

## See also

<!-- The reference page for the options; the Background page for the why. -->

Board starter

The Tutorial’s chapter 1 done on one board, needing only Install. About 1,500 words. The status header and the list of starters are generated from the registry; do not type them.

# Get started on <board>

<!-- type: how-to · level: 1 -->

**Needs first:** [Install](<link>)

## What you need

## Provision

## Scaffold the project

## Run it in the <emulator or twin>

## Flash the board

<!-- Only when the row names `hardware`. -->

## See it in ROS 2

## If it doesn't work

## Next

Platform page

What the platform’s starter leaves out: configuration, board notes, deeper tasks. Needs only Tutorial 1–2 and the starter. It sits in the User guide group its row’s generated status word names.

# <Platform>

<!-- type: how-to · level: <2|3> -->

**Needs first:** [Get started on <board>](<link>)

## Configure the platform

## Board notes

## <Deeper task, one heading each>

## Reference

<!-- Links to the generated board pages. -->

Explanation

A Background page: the nano-ros “why” a reader needs before production work.

# <The thing explained, as a noun phrase>

<!-- type: explanation · level: 3 -->

<!-- Open with a concrete example the reader has already run or will run:
     a build, a log line, a measured number. Then explain the model behind
     it. No steps. -->

## <Part of the model>

## Where nano-ros differs from ROS 2

<!-- Stated plainly, when it differs. -->

## Design background

<!-- The only place an RFC number appears. -->

Reference

Lookup only: tables, keys, options. If the page’s content has an owner (the CLI’s help, a schema, a generator), generate the page from it instead.

# <Thing> reference

<!-- type: reference · level: <level of its most advanced topic> -->

<!-- One sentence: what this lists and where it is generated from. -->

| Name | Type | Default | Meaning |
| --- | --- | --- | --- |

Before you ask for review

State in the PR description, for every page you add or rewrite:

• its type, level and section in RFC-0124 Appendix A;
• the principles you checked it against, and any you could not meet yet, with the reason (for example, a step waiting on a CLI change, with its awaits line);
• the output of python3 scripts/check-book-audience.py --charter for that page.

Moves go through book/page-map.toml and scripts/book-move.py, so every moved page keeps a redirect. Land the work in section-sized PRs, rebased onto main rather than merging main in.

Quick Reference

The .env file

All environment variables can be set in a .env file at the checkout root:

cp .env.example .env
# Edit .env — uncomment and adjust values as needed

• justfile — .env is auto-loaded. Missing file silently ignored.
• direnv — .envrc sources .env if present.
• Manual — set -a; source .env; set +a before cargo build.

Variables in .env take precedence over justfile defaults but are overridden by explicit shell exports. The variables themselves are listed in the user book’s Environment Variables.

Finding Commands

Root commands are grouped by audience:

just --list                         # grouped root command overview
just --groups                       # group names
just --group main --list            # normal development workflows
just --group full-matrix --list     # build-all / fixtures / test-all / ci
just --group setup --list           # setup and doctor entry points
just --group maintenance --list     # clean and generated-binding commands
just --group docs --list            # book and API docs

Platform and backend commands stay namespaced:

just --group full-matrix --list zephyr
just zephyr build-fixtures
just zephyr build-all
just qemu setup-network             # QEMU TAP networking only
just zephyr help                    # Zephyr-specific help

Install the local book tooling before previewing docs:

just setup-docs                     # mdbook + mdbook-mermaid
just book-serve                     # serve book/src with live reload
just book                           # full deployed preview with API docs

API documentation

The API reference the user book links (book/src/api/…) is Doxygen and rustdoc output built from the tree:

just doc                 # all of the below
just doc-rust            # rustdoc
just doc-c               # target/doxygen/c/
just doc-cpp             # C++ API
just doc-rmw-cffi        # RMW C ABI
just doc-platform-cffi   # target/doxygen/platform-cffi/

The Doxygen recipes need doxygen and skip with a warning when it is not installed (nros setup --system composes the install command).

Test profiling & slow-test reporting

just test and just test-all always print the slowest tests after the run summary — the top 20 by duration (binary, test name, time, status), parsed from the active profile’s target/nextest/<profile>/junit.xml. No flag needed; it is part of the normal output.

Deeper profiling is opt-in and adds nextest’s experimental event/output recording plus artifact export. It preserves the normal nextest execution model (same filters, cargo profile, parallelism) — it only records and exports, so leave it off for routine runs:

NROS_NEXTEST_RECORD=1 just test
NROS_NEXTEST_RECORD=1 just test-all

Each profiled run writes a timestamped directory under tmp/ with a stable -latest symlink:

tmp/nextest-profile-test-YYYYMMDD-HHMMSS/      tmp/nextest-profile-test-latest -> …
tmp/nextest-profile-test-all-YYYYMMDD-HHMMSS/  tmp/nextest-profile-test-all-latest -> …

Artifacts in that directory:

• nextest-run.zip — portable recording archive; replay with full captured output (incl. successful tests) via cargo nextest replay.
• nextest-trace.json — Chrome/Perfetto timeline (slot/group occupancy, idle slots, retries, long-pole tests). Canonical concurrency artifact.
• junit.xml — copy of the run’s JUnit report.
• env.txt, command.txt — the knobs and exact command used.

Knobs:

Overhead and retention: recording adds event/output-store writes during the run, and nextest-run.zip (plus an enabled nextest-replay.log) can get sizable on output-heavy suites — keep it opt-in for local runs and prune old tmp/nextest-profile-* directories. Recording uses a profile-local NEXTEST_STATE_DIR, so it never pollutes the user’s global nextest record store. Do not reach for --no-capture to inspect output: it serializes execution and skews every timing. Use the replay archive instead.

Manual Testing

# Base setup builds/checks the in-tree zenoh router.
just setup base
source ./activate.sh

# Terminal 1: Router
ZENOH_CONFIG_OVERRIDE='listen/endpoints=["tcp/127.0.0.1:7447"];scouting/multicast/enabled=false' ros2 run rmw_zenoh_cpp rmw_zenohd

# Terminal 2: Talker
cd examples/linux/rust/talker && nros configure && nros build
RUST_LOG=info ./build/linux/target/debug/talker

# Terminal 3: Listener
cd examples/linux/rust/listener && nros configure && nros build
RUST_LOG=info ./build/linux/target/debug/listener

Each leaf’s RMW comes from its system.toml [system] rmw, and its artifact path is build/<image-id>/target/[<triple>/]<profile>/<bin>.

UDP Transport

On the native (host) build, zenoh-pico has built-in UDP support via OS sockets:

# Use UDP instead of TCP for the zenoh locator
NROS_LOCATOR=udp/127.0.0.1:7447 ./build/linux/target/debug/talker

On bare-metal, enable the link-udp-unicast feature to use UDP over smoltcp:

nros = { features = ["rmw-zenoh", "platform-bare-metal", "link-tcp", "link-udp-unicast"] }

The rmw-zenoh feature here is the lowering of the declared RMW — you declare the backend once in system.toml ([system].rmw / [image.<id>].rmw) and the toolchain sets the cargo feature; the feature is what the build uses, not the user-facing selector. See RFC-0031.

TLS Transport

TLS layers on top of TCP using mbedTLS. Requires a self-signed certificate (or real CA cert) and the link-tls Cargo feature.

Generate a test certificate:

openssl req -x509 -newkey ec -pkeyopt ec_paramgen_curve:prime256v1 \
  -keyout key.pem -out cert.pem -days 365 -nodes -subj "/CN=localhost"

Native (host) (requires libmbedtls-dev – installed by just setup base):

# Terminal 1: Router with TLS
ZENOH_CONFIG_OVERRIDE='listen/endpoints=["tls/localhost:7447"];scouting/multicast/enabled=false' ros2 run rmw_zenoh_cpp rmw_zenohd \
  --cfg 'transport/link/tls/listen_certificate:"cert.pem"' \
  --cfg 'transport/link/tls/listen_private_key:"key.pem"'

# Terminal 2: Talker with TLS
cd examples/linux/rust/talker && nros configure
cd .. && cargo build --manifest-path talker/Cargo.toml \
      --config talker/build/linux/nros-cargo.toml --features link-tls
NROS_LOCATOR=tls/localhost:7447 \
  ZENOH_TLS_ROOT_CA_CERTIFICATE=cert.pem \
  ./talker/build/linux/target/debug/talker

link-tls is a transport feature of the leaf’s own crate, so it stays a --features flag; the RMW is not — that is [system] rmw in system.toml.

Bare-metal (QEMU ARM):

On bare-metal, only base64-encoded certificates are supported (no filesystem). Build with --features link-tls:

# Build TLS-enabled examples
cd examples/mps2-an385-baremetal/rust/talker && nros configure
cd .. && cargo build --release --features link-tls \
      --manifest-path talker/Cargo.toml \
      --config talker/build/mps2-an385-baremetal/nros-cargo.toml

The CA certificate must be passed via ZENOH_TLS_ROOT_CA_CERTIFICATE_BASE64 at runtime (through the zenoh config), or embedded in the binary at build time.

ROS 2 Interop

# Terminal 1: Router
ZENOH_CONFIG_OVERRIDE='listen/endpoints=["tcp/127.0.0.1:7447"];scouting/multicast/enabled=false' ros2 run rmw_zenoh_cpp rmw_zenohd

# Terminal 2: nros talker
cd examples/linux/rust/talker && nros configure && nros build
RUST_LOG=info ./build/linux/target/debug/talker

# Terminal 3: ROS 2 listener
source /opt/ros/humble/setup.bash
export RMW_IMPLEMENTATION=rmw_zenoh_cpp
ros2 topic echo /chatter std_msgs/msg/String --qos-reliability best_effort

Actions

# Terminal 1: Router
ZENOH_CONFIG_OVERRIDE='listen/endpoints=["tcp/127.0.0.1:7447"];scouting/multicast/enabled=false' ros2 run rmw_zenoh_cpp rmw_zenohd

# Terminal 2: Action server (Fibonacci example)
cd examples/linux/rust/action-server && nros configure && nros build
./build/linux/target/debug/action-server

# Terminal 3: Action client
cd examples/linux/rust/action-client && nros configure && nros build
./build/linux/target/debug/action-client

Zephyr action tests:

just zephyr build-all          # Build all Zephyr examples (Rust + C + C++ + XRCE)
just zephyr test               # Run zenoh E2E tests (covers actions)

Docker Development Environment

Docker provides QEMU 7.2 (from Debian bookworm) which fixes TAP networking issues present in Ubuntu 22.04’s QEMU 6.2.

# One-time setup: add yourself to docker group
sudo usermod -aG docker $USER
# Log out and back in, or run: newgrp docker

# Build and use Docker environment
just docker build              # Build nano-ros-qemu image
just docker shell              # Interactive shell
just docker test-qemu          # Run QEMU tests in container
just docker help               # Show all Docker commands

QEMU Bare-Metal Testing

Run bare-metal Cortex-M3 examples on QEMU (MPS2-AN385 machine with LAN9118 Ethernet).

# Build prerequisites
just qemu build-zenoh-pico    # Build zenoh-pico for ARM Cortex-M3
just qemu build                # Build all QEMU examples

# Non-networked tests (no setup required)
just qemu test                # Bare-metal QEMU integration tests

# Networked talker/listener test (Docker Compose - recommended)
just docker test-qemu         # Runs zenohd, talker, listener in separate containers

Docker Compose Architecture:

┌─────────────────────────────────────────────────────────────┐
│              Docker Network: 172.20.0.0/24                  │
│                                                             │
│  ┌─────────────┐  ┌─────────────┐  ┌─────────────────────┐  │
│  │   zenohd    │  │   talker    │  │      listener       │  │
│  │ 172.20.0.2  │  │ 172.20.0.10 │  │    172.20.0.11      │  │
│  │             │  │  ┌───────┐  │  │  ┌───────────────┐  │  │
│  │             │  │  │ QEMU  │  │  │  │     QEMU      │  │  │
│  │             │  │  │ ARM   │──┼──┼──│     ARM       │  │  │
│  │             │  │  │ TAP   │  │  │  │     TAP       │  │  │
│  │             │  │  └───────┘  │  │  └───────────────┘  │  │
│  └──────▲──────┘  └──────┼──────┘  └─────────┼───────────┘  │
│         └────────────────┴───────────────────┘              │
│                    NAT to zenohd                            │
└─────────────────────────────────────────────────────────────┘

Manual networked test (3 terminals, requires host TAP setup):

# Terminal 1: Setup network + start router
just qemu setup-network                    # Requires sudo
ZENOH_CONFIG_OVERRIDE='listen/endpoints=["tcp/0.0.0.0:7447"];scouting/multicast/enabled=false' ros2 run rmw_zenoh_cpp rmw_zenohd

# Terminal 2: Talker (192.0.2.10)
bin=build/cargo-fixtures/baremetal/thumbv7m-none-eabi/nros-relwithdebinfo
./scripts/qemu/launch-mps2-an385.sh --tap tap-qemu0 --binary "$bin/qemu-bsp-talker"

# Terminal 3: Listener (192.0.2.11)
./scripts/qemu/launch-mps2-an385.sh --tap tap-qemu1 --binary "$bin/qemu-bsp-listener"

Build those first with just qemu build-fixtures (or just build-test-fixtures lane=linux).

The artifact directory is computed, not fixed. Since phase-340 P2 a platform’s cargo fixtures build into a shared group directory under build/cargo-fixtures/<group>/ rather than each leaf’s own target/, and the profile is nros-relwithdebinfo, not release. The group name can carry a hash suffix when a row builds with distinct env, so treat the path above as what it is on a default build, not as a guarantee — scripts/build/fixtures-target-dir.sh (nros_fixture_row_artifact_dir) is the single source of truth, and it is what the test side uses to find the same file. That helper takes the row’s cargo args and env as arguments, so a script that did not itself run the build should ask for the row by id — nros_fixture_row_artifact_dir_by_id <row-id> reads all three key fields from the manifest, and cannot disagree with the builder about which group a row landed in (issue 1025).

Run just qemu help for more options.

Zephyr Setup

just setup zephyr       # Initialize workspace at $repo/zephyr-workspace/
just zephyr test        # Run zenoh tests (native_sim uses NSOS on host loopback)
just zephyr test-xrce   # Run XRCE tests

The workspace lives at $repo/zephyr-workspace/ by default (gitignored). Set $NROS_ZEPHYR_WORKSPACE to install elsewhere. Legacy sibling installs at ../nano-ros-workspace/ are auto-detected; run ./scripts/zephyr/migrate-workspace.sh to consolidate.

See Zephyr for full details.

CLI lives in the monorepo (Phase 218)

The nros CLI lives in-tree at packages/cli/ as a sub-workspace — own Cargo.toml, own Cargo.lock, own member crates (nros-cli, nros-cli-core, nros-build, cargo-nano-ros, rosidl-*, colcon-cargo-ros2, nros-msg-to-idl). It is built per checkout via just setup-cli → packages/cli/target/release/nros, then put on PATH by the activate file. See also:

• Phase 218 roadmap doc
• Phase 218 design spec

Why sub-workspace, not root member

The runtime workspace’s no_std feature-unification view (Phase 214.F.3) is the project’s most fragile invariant — a single workspace member adding a std-activating dep without target-gating turns the embedded build red on a path the test matrix doesn’t always exercise. The CLI’s deps (clap, askama, syn, ureq, …) are aggressively std-only; making them workspace siblings of the runtime crates would either (a) require target-gating every single one or (b) effectively disable the F.3 guard.

The sub-workspace shape keeps the two surfaces categorically separate. The pattern mirrors packages/testing/nros-{tests,bench,smoke}/, which carved out the same way for the same reason.

Why per-checkout, not ~/.nros/bin

Confirmed during the Phase 218 UX walkthrough: contributors with multiple nano-ros worktrees (phase branches, ASI integration trees, downstream forks) need each tree to point at its own CLI. A global install would silently version-skew across trees the moment the user cds — and because the CLI’s codegen format is structurally bound to the nros-core / nros-c / nros-cpp crates in the same checkout, a global CLI is a known footgun.

The per-checkout shape makes the activate file the single switch — no which nros surprise across cwd boundaries. One checkout = one CLI version = one runtime ABI, by construction; no version-pin matrix to police.

Resolution chain

scripts/build/cargo.sh::nros_cli_bin resolves the active binary:

1. $NROS_CLI (explicit override)
2. nros on $PATH (activate file puts the repo-local one here)
3. packages/cli/target/release/nros (per-checkout)
4. ${NROS_HOME:-~/.nros}/bin/nros (transitional, removed once every active branch lands on 218)
5. error

The Phase 218.E ABI guard checks the in-tree CLI build matches the runtime crates’ codegen ABI hash; opt out via NROS_SKIP_VERSION_CHECK=1 when intentionally bisecting a CLI mismatch.

See also

• Build System & Caching
• SDK Tiers
• Environment Variables

The CLI’s developer tier

The user-facing nros verbs are documented in the user book’s CLI reference. This page holds the rest: building the CLI from a checkout, the hidden nros dev … verbs that cmake, west, build.rs and just call, the contributor-only nros update, and how nros relates to the just recipes. nros help --all and nros dev --help list the dev tier (RFC-0117 D1, D3).

Building the CLI from the checkout

Inside a nano-ros checkout, build the CLI from the checkout. That is not a fallback: in a checkout the tree’s own build is the only binary this tree accepts, because a released nros emits its own generated code and this runtime is what compiles it (RFC-0090, phase-431 W1). just doctor fails when another nros shadows it. The CLI ships from the in-tree sub-workspace at packages/cli/; bootstrap builds it (installing rustup if needed), then activate the workspace to put it on PATH:

./scripts/bootstrap.sh
source ./activate.sh        # OR: direnv allow / source ./activate.fish

Equivalent, if you already have cargo (same build, same binaries): git submodule update --init packages/cli/third-party/play_launch, then cargo build --release --manifest-path packages/cli/Cargo.toml --bin nros and cargo build --release --manifest-path packages/cli/nros-launch-resolve/Cargo.toml — the launch resolver the workspace flow needs (bootstrap runs all three for you; just setup-cli + just setup-launch-resolve are the internal aliases).

The resulting binary lives at packages/cli/target/release/nros. One checkout = one CLI version = one runtime ABI; contributors with multiple nano-ros worktrees get per-tree CLIs with no global PATH skew. Once on PATH, nros --help lists every verb. ${NROS_HOME:-~/.nros}/bin/nros is where install.sh puts a release; inside a checkout just doctor fails when that one shadows the tree’s own build.

nros doctor run inside a checkout also forwards to just doctor (or just <platform> doctor with --platform) after its own native checks; when just is not on PATH it says it skipped those contributor checks and still exits on the native result.

Dev-tier verbs

nros dev codegen rust [--manifest <path>] [--output <dir>] [--ros-edition <e>] [--codegen-config <f>] [--rename <old=new>] [--force] [--verbose]

The Rust-only codegen primitive. Same binding generation as nros interface generate rust, and with no side-effects: it writes generated/ and nothing else. Use it when you want bindings and only bindings; use nros sync for an actual project, because a leaf also needs its generated build settings.

--generate-config (alias --config), --nano-ros-path and --nano-ros-git still parse and are legacy. They wrote a leaf .cargo/config.toml — the file RFC-0098 retired. The patch rows now live in the generated build/<image>/nros-cargo.toml that nros sync writes, alongside the board’s cargo settings and the image’s derived [env], and cargo reads that through --config. (nros sync’s own --nano-ros-path is documented in its help as a no-op kept for back-compat.)

nros dev codegen <resolve-deps|cyclonedds-descriptors|entry> …

Internal build-tool interface — invoked by the CMake integration and build.rs scripts, not meant for direct use. Subcommands resolve message-package dependency closures, emit CycloneDDS type descriptors, and generate C/C++ entry glue. (nros dev codegen system below is the user-facing system bake.)

nros dev metadata <system_pkg> [--workspace <path>] [--out-dir <dir>] [--metadata <existing.json>] [--build [--nano-ros-workspace <path>]] — walk a colcon-style workspace under <workspace>/src/

collecting component source metadata into build/<system_pkg>/nros/source-metadata.json. The result feeds nros plan. With --build, any declared component (component_nros.toml) missing its source-metadata is produced by the metadata-mode build: nano-ros compiles + runs the component against an in-memory recorder to emit the JSON. --build needs the nano-ros workspace (--nano-ros-workspace or NROS_WORKSPACE).

nros ws <subcommand>

Workspace-level message-package utilities (the env / status / list / clean / doctor helpers; sync was promoted to top-level nros sync).

eval "$(nros dev ws env)"   # add src/ to NROS_INTERFACE_SEARCH_PATH
nros ws status          # freshness check (non-fatal): n up-to-date / n stale / n missing
nros list            # list discovered msg + Rust-consumer pkgs
nros clean           # remove generated/ + auto-managed patch entries
nros ws doctor          # lint workspace pkgs (package.xml markers, stale patches, …)
nros dev lock managed-deps    # write the `# nros-managed` message rows from package.xml

nros dev codegen system [--bringup <pkg>] [--target <id> | --for-entry <pkg>] [--out <dir>] [--launch <file>] [--ahead-of-vendor <pio|px4>]

Host-time system bake: reads <bringup>/system.toml + <bringup>/launch/system.launch.xml and emits the baked compile-time C config + component registration glue consumed by every embedded RTOS adapter.

nros dev codegen system --bringup demo_bringup
nros dev codegen system --bringup demo_bringup --target zephyr_native_sim  # an [image.<id>]
nros dev codegen system --bringup demo_bringup --for-entry zephyr_app      # the image claiming it
nros dev codegen system --bringup demo_bringup --ahead-of-vendor pio   # + PlatformIO library.json
nros dev codegen system --bringup demo_bringup --ahead-of-vendor px4   # + PX4 module dirs

nros dev sdk root [--workspace <dir>] [--nano-ros-path <dir>] [--explain]

Print the nano-ros SDK root this nros resolves — the directory holding the runtime sources and nano_rosConfig.cmake that a build compiles against: an explicit --nano-ros-path, else a nano-ros checkout found by walking up from --workspace (default: the current directory), else this binary’s own SDK (a release’s share/nano-ros, or the checkout an in-tree nros was built in). --explain names the rung that answered on stderr; stdout stays the bare path.

nros dev sdk root --explain

Dev-tier verbs the user book no longer names

The user book teaches only the user tier (RFC-0117 D2, RFC-0123 D9). These dev-tier verbs used to appear in user chapters; the user step that replaced each is named beside it.

Old spellings

nros help --aliases prints every older spelling that still parses and the spelling it stands for. The table lives in packages/cli/nros-cli-core/src/cmd/user.rs (ALIASES, user tier) and dev.rs (dev tier). An old user spelling prints one note on stderr naming the new one, only when stderr is a terminal; a dev-tier alias prints nothing.

nros toolchain manifest: with no arguments it reads the manifest of the toolchain the binary belongs to; --write <path> (with --store-version, --index <date>, --nano-ros <commit>) stamps one. The release workflow calls the binary it just built, so a manifest cannot claim a codegen version its own binary does not emit.

Front-door gaps

A user step with no user-tier nros spelling. The user book either works around each (named below) or says no command exists yet. Each wants a front-door verb or an explicit decision that it stays out of the user tier.

• The SDK root — closed (phase-532 W9). nros locate nano-ros is the user spelling: it answers through the nano_ros_root ladder anywhere, and in a pinned project it is the pinned version’s root. The user book uses "$(nros locate nano-ros)" wherever a host tool needs the path. The scaffolds nros new writes and the probe fences still spell nros dev sdk root, which gives the same answer.
• A store tool’s directory. nros locate tool covers qemu-system-* only. The Zephyr SDK (ZEPHYR_SDK_INSTALL_DIR) and the Arm FVP model directory (ARMFVP_BIN_PATH) have no user-tier spelling.
• The boot-report reader (below).
• The mem-report tool (mem-report).
• A QEMU -nic value for a script that starts QEMU itself (nros ws qemu-nic).
• The census, for a hand-run west/cmake build (nros dev image entity-census take); the configure prints the command when it refuses.
• The NuttX apps-tree staging is a script in the SDK root (scripts/nuttx/stage-external-apps.sh), not a verb.

Decoding a boot report

The user book’s Observing a Board With No Console page saves the NROS_BOOT_REPORT record with a debug probe; decoding it needs the repository’s reader:

read ADDR LEN <<< "$(python3 scripts/read-boot-report.py --addr-only build/zephyr/zephyr.elf)"
pyocd commander -t s32k344 -c halt -c "savemem $ADDR $LEN report.bin"
python3 scripts/read-boot-report.py build/zephyr/zephyr.elf report.bin

--addr-only prints the address and length from the ELF’s own symbol table, so a relinked image is never read at a stale address. --heap-headroom <dump> is the heap-headroom gate; --help lists the fields.

Contributor verbs

nros update (--pins [--prune] | --crate <n> [--precise <v>] | --source <n> [--ref <sha>] [--allow-rewind] | --tool <n> [--to-version <v>] | --west <m> [--revision <r>]) [--nano-ros-path <dir>]

Move ONE pin of a nano-ros checkout’s pin set — a registry crate, a vendored source, a host tool or a west project — and leave one diff to review. --pins regenerates the registry pin set from the locked roots. A contributor verb: an installed SDK’s pin set is read-only.

nros and just

nros vs. just

nros routes through the nros-cli-core library. just recipes that wrap user-flow operations call into nros for consistency; internal recipes (build matrices, CI orchestration) keep their current shape.

Release pipeline status

The nros CLI is released as a GitHub Releases asset (nros-<host>.tar.zst, carrying the CLI and its SDK root) and installed with scripts/install.sh — Installation. There is no crates.io, Arduino zip or ESP-IDF binary channel. (A hidden, maintainer-only nros release verb exists behind the release cargo feature; the default build does not carry it.) Downstream RTOS package managers (Zephyr west, ESP-IDF idf_component_yml, NuttX Kconfig) consume the same source tree via the integration shells under integrations/<rtos>/.

SDK Tiers for Setup

Setup is split by customer workflow. scripts/bootstrap.sh is the entrypoint for a fresh checkout with no just installed yet; once just is available, all paths delegate to the same recipes.

scripts/bootstrap.sh                 # install/check just, then show setup choices
scripts/bootstrap.sh base            # first-time quick start
scripts/bootstrap.sh platform zephyr # focused platform developer
scripts/bootstrap.sh all             # contributor / full test-all setup

just setup                           # print setup choices
just setup base                      # base quick-start tier
just setup all                       # full contributor / test-all tier
just <platform> setup                # focused platform setup
just doctor                          # diagnose base tier
just doctor tier=all                 # diagnose full tier

just setup with no argument is a menu and does not fetch/install. NROS_SETUP_TIER still overrides the default no-argument just doctor tier. Valid tiers are base and all. Legacy aliases minimal and default map to base; extended and everything map to all.

Tiers

cli (Phase 218 monorepo merge) is the first step of the base tier — just setup-cli runs cargo build --release --manifest-path packages/cli/Cargo.toml --bin nros, producing the per-checkout packages/cli/target/release/nros. Every downstream just <module> setup recipe assumes this binary is present (it is the canonical provisioner). packages/cli/ is a sub-workspace (own Cargo.toml, own Cargo.lock) so its host-only deps (clap, askama, syn, ureq) stay outside the runtime no_std feature-unification view (Phase 214.F.3). Per-checkout, not ~/.nros/bin — contributors with multiple worktrees get tree-isolated CLIs with no global PATH skew. A source ./activate.sh / direnv allow puts the resulting binary on PATH.

Corrosion (the CMake ↔ Rust/Cargo bridge corrosion_import_crate() uses) is installed by just workspace install-corrosion to ~/.nros/sdk/corrosion/ whenever the workspace module runs — i.e. on every tier (base and all), since workspace is the first module in both branches of justfile::_orchestrate. The pin is in just/workspace.just::CORROSION_VERSION; the SDK-index counterpart [tool.corrosion] in nros-sdk-index.toml pins the same tag so nros setup --tool corrosion --prefix $HOME/.nros/sdk/corrosion lands an equivalent install. Header/config-only at install time, well under the 500 MB / 5 min default tier policy, idempotent (stamp-file gated). Phase 212.D + H.4 mixed Rust+C++ workspace tests (phase212_d_workspace_metadata::cmake_mixed_corrosion_bridge_builds) green-path here once the workspace module runs.

The launch parser is not a provisioned tool. It is linked into nros-launch-resolve, built from the packages/cli/third-party/play_launch submodule by just setup-launch-resolve (and shipped beside nros in a release). The standalone play_launch_parser store tool was retired by issue 1413: no build path ran it, and it carried a second, lagging pin.

all is intentionally explicit because it pulls many submodules and installs large SDKs. A module never moves between tiers without bumping this document and the orchestrator switch in justfile::_orchestrate.

Policy: when does a module join base?

A module joins base when all of:

1. Expected by first-time users. It supports native examples, standard ROS/zenoh interop, or core workspace development.
2. Moderate surprise. It does not fetch large platform SDKs such as PX4-Autopilot, ESP-IDF, Zephyr, or full RTOS trees.
3. Idempotent on re-run. No destructive ops, no sudo, no network-only stages that flap.

A module joins all when it is needed by full-matrix development or just test-all, but is too heavy or platform-specific for the default quick start.

A module stays opt-in entirely (neither tier) when it pulls in private SDKs, requires sudo, or carries license-restricted bits. ARM FVP, NVIDIA SDK Manager, Cadence Tensilica toolchain, proprietary vendor BSPs all fall here. Contributors needing them run just <module> setup explicitly out-of-band.

CycloneDDS — self-provisioned in CMake (Phase 186)

nros-rmw-cyclonedds consumes Cyclone via find_package(CycloneDDS) and never compiled it itself. Phase 186 moved provisioning into the build system: the backend’s nros_provide_cyclonedds() resolves Cyclone in order —

1. an already-defined CycloneDDS::ddsc target,
2. find_package(CycloneDDS CONFIG) — a prebuilt install on CMAKE_PREFIX_PATH / CycloneDDS_DIR (a user install),
3. self-provision from source — add_subdirectory(${CYCLONEDDS_SOURCE_DIR}) (defaults to the pinned third-party/dds/cyclonedds submodule; a user points it at their own checkout), with the per-platform flags staged in cmake/platform/nano-ros-<plat>.cmake and sccache as the compiler launcher.

So a bare cmake/cargo build self-provisions Cyclone with no just cyclonedds pre-step — freertos / threadx-rv64 / native all build it on demand (just <plat> build-fixtures), gated on the relevant cross toolchain. There is no longer a build/cyclonedds-<rtos>-install artifact (the old cross-probe scripts are deleted); the targets link a static ddsc (no runtime libddsc.so, so no rpath and no risk of ld.so substituting a mismatched system /opt/ros Cyclone).

Availability still follows the tier that ships the cross toolchains:

• all tier (ships arm-none-eabi-gcc via freertos, RV64 GCC via threadx_riscv64): the embedded-Cyclone test-all cases build + pass.
• base tier (no cross toolchains): the embedded-Cyclone tests are filtered out of test-all (gated on command -v <cross-cc>, Phase 185.2/ 186.4) so they report skipped, not failed. ThreadX-RV64 Cyclone fixtures stay behind the experimental NROS_THREADX_RV64_CYCLONEDDS_FIXTURES=1.

Host build/install (the cyclonedds setup module) is still built — for the backend standalone CI (just cyclonedds build-rmw/test/ci), the Zephyr-Cyclone host idlc, and the not-yet-migrated threadx-linux host-Cyclone path. Example builds no longer depend on it (Phase 186 “host build.sh” follow-up tracks full removal).

Run test-all in the all tier for full Cyclone coverage.

Adding a new module

When introducing a new RTOS / SDK module:

1. Land just/<module>.just with setup (idempotent) and doctor (read-only) recipes.
2. Add mod <module> 'just/<module>.just' to justfile.
3. Decide whether it belongs in base, all, or platform-only setup.
4. Add run <module> to the matching tier branch of _orchestrate in justfile.
5. Update the table in §Tiers in this doc.
6. Update AGENTS.md’s “Build, Test, and Development Commands” if the change is user-facing.

If the module is opt-in entirely, skip steps 3–5 and add a note to its own README explaining why.

Cold-start verification

For full-matrix cold start, run:

rm -rf build target*
scripts/bootstrap.sh all
source ./setup.bash
just build-test-fixtures
just test-all

For quick-start cold start, use scripts/bootstrap.sh without all.

Relation to Cargo features

Tiers describe what gets installed (host SDKs, toolchains, clones). Cargo features describe what gets compiled inside the nano-ros workspace. The two axes are orthogonal: a contributor may install tier=extended (every SDK) but build with --no-default-features (minimum nros feature set).

Relation to packaged releases (Phase 139.8)

Downstream consumers of a packaged nano-ros release (ESP-IDF component registry, PlatformIO library, Zephyr west.yml, NuttX-apps, PX4 external module) never invoke just setup. They pull a pinned release through their RTOS’s package manager and the host-SDK install is handled by the RTOS toolchain. Tier policy applies to local development of nano-ros itself, not to consumers. See registry-publishing.md.

nano-ros versioning

Model: JetPack-style bundle. One version number covers the whole project — runtime crates, the nros CLI, the C/C++ surfaces, the example trees, the board-glue cmake. A release bundles the artifacts behind that number; consumers either:

• clone the repo at a nros-v<X.Y.Z> tag (runtime + examples + cmake),
• AND/OR fetch the matching CLI binary from the GitHub release page (the Phase 218.G nros-<triple>.tar.gz assets).

The model is named after NVIDIA’s JetPack: a single SDK version (JetPack 5.1.2) covers CUDA + cuDNN + TensorRT + the L4T kernel + the rest — distributed as one bundle, not as N independently-versioned crates on PyPI.

Why not crates.io publish?

The runtime crates carry deep C/C++ deps via *-sys shims — zenoh-pico, Cyclone DDS, micro-XRCE-DDS, mbedtls, FreeRTOS/Zephyr/ NuttX/ThreadX RTOS surfaces. crates.io can’t carry those cleanly: build scripts assume a configured workspace with SDK paths exported (see .envrc / activate.sh), and the SDK source itself is gitignored

• index-driven (nros-sdk-index.toml).

A user who tries cargo add nros-core from a third-party project gets a crate that won’t link in any meaningful environment. Better to ship the whole thing as a bundle, where the SDK provisioning, the codegen toolchain, the runtime, and the examples all move together.

No nano-ros crate is published to crates.io. Every workspace member should carry publish = false (defensive; prevents accidental publish on a stray cargo publish).

Consumers pin nano-ros runtime crates through their project’s [patch.crates-io] table, redirected to a local path (or to a git submodule of the tagged release). The Phase 212.M-F.21 nros sync verb writes those entries automatically.

Single workspace.package.version

Two Cargo.toml files carry a [workspace.package].version field:

These two MUST stay in lockstep. scripts/check-version-lockstep.sh (run as just check version-lockstep, on the fast lane) errors if they diverge. just release-bump <X.Y.Z> updates both atomically.

Inside each workspace, every member crate inherits the version via version.workspace = true. New crates added to either workspace should follow this pattern.

Bumping the bundle

Three rules:

1. Patch bump (0.4.0 → 0.4.1) — any non-ABI-breaking change to ANY subsystem (CLI bug fix, runtime perf fix, board-glue cleanup, docs-only changes that ship with release notes).
2. Minor bump (0.4.x → 0.5.0) — any ABI-affecting change to nros-core, nros-c, nros-cpp, or any of the RMW vtable surfaces. The Phase 218.E ABI guard will reject pairings that straddle a minor boundary, forcing consumers to update.
3. Major bump (0.x.y → 1.0.0) — declared stability commitment; reserved for the post-rclrs-0.7-parity moment.

Patches are cheap (no crates.io republish). Resist the temptation to batch unrelated changes into a single bump just to “save a version number” — the bundle model treats version numbers as low-cost labels.

Bump workflow

just release-bump 0.4.1                # sed both files + lockstep check
git diff Cargo.toml packages/cli/Cargo.toml
git commit -am 'release: nros-v0.4.1'
git tag nros-v0.4.1
git push origin main nros-v0.4.1

The tag publishes no artifacts — nano-ros is a source distribution (phase-288 D1/D2; the pre-288 release.yml prebuilt pipeline is deleted). A tag is the consumer’s pin: it fixes the runtime source, the CLI the tree builds (nros version lockstep), and the toolchain versions in the committed nros-sdk-index.toml. See the book’s Pinning a version section.

ABI guard (Phase 218.E)

nros dev codegen / nros dev codegen rust reads the consumer’s Cargo.lock, finds the resolved nros-core version, and compares it to the CLI binary’s compile-time env!("CARGO_PKG_VERSION"). Strict equality. Mismatch → exit non-zero with both versions printed.

NROS_SKIP_VERSION_CHECK=1 opts out (warned to stderr for CI visibility).

The strict comparison is the bundle model’s enforcement mechanism: if a consumer has nros-core 0.4.0 in their lockfile and runs the nros 0.5.0 binary, the codegen / runtime ABI assumption is broken and the build SHOULD fail loud. The maintainer’s job (via just release-bump) is to keep the CLI and the runtime moving as one unit so the guard stays quiet on the tracked tag.

When the guard fires within the nano-ros tree itself (during local development on a branch where the version hasn’t been bumped yet), the NROS_SKIP_VERSION_CHECK=1 env in your shell session is the documented relief valve.

Baseline

Bundle version 0.4.0 is the Phase 218 monorepo-merge baseline. It was chosen so that:

• It signals a discontinuity from nros-v0.3.7 (the last standalone nros-cli tag — see the archived github.com/NEWSLabNTU/nros-cli history).
• It doesn’t claim post-1.0 stability the project hasn’t yet established.
• 0.x.y SemVer convention (everything pre-1.0 is “API may change”) matches the current development pace.

Related

• Phase 218 doc: docs/roadmap/phase-218-merge-cli-into-monorepo.md
• Phase 218 design spec: docs/superpowers/specs/2026-06-04-cli-monorepo-merge-design.md
• ABI guard impl: packages/cli/nros-cli-core/src/abi_guard.rs
• Lockstep check: scripts/check-version-lockstep.sh
• Bump recipe: just release-bump in the root justfile
• (The pre-288 release.yml prebuilt workflow is deleted — tags publish no artifacts.)

CI conventions for nano-ros

A new GitHub Actions workflow should work on its first push. The Phase 196 bring-up of zephyr-dual-line.yml cost seven stacked CI rounds because each workflow re-discovered the same fresh-runner gaps (no submodules, no ROS, no Python 3.12, wrong runner OS). This page is the checklist + copy-paste snippets that close that gap class. Pair it with the live workflows in .github/workflows/ — they are the worked examples.

Some workflow names on this page are HISTORICAL. The fresh-runner discipline below is current and still load-bearing; the files it was learned from were consolidated by the phase-253 CI reorg and again by phase-395. Every such name is listed in the “Historical workflow names” table at the foot of this page, with what replaced it. A name NOT in that table must name a file that exists — check-ci-doc-workflow-refs enforces exactly that, because a doc citing a workflow nobody can open sends a reader looking for a file rather than for the rule it was illustrating.

The mental model: the runner is a fresh clone, nothing else

A GitHub runner has the repo at the recorded commit and the runner image’s default tools. It has no submodules, no ROS, no SDKs, and an OS-pinned Python. Every assumption your laptop satisfies silently is a CI failure. Walk the job as if you just cloned into an empty container.

Conventions

1. Init only the submodules you need — never recursive-all

actions/checkout@v4 initializes no submodules. Init the specific paths the job touches; never submodules: recursive on checkout (the platform/RTOS submodules are large and fork-pinned — a full recursive init is slow and pulls trees the job will never read).

- uses: actions/checkout@v4
- name: Init needed submodules
  run: git submodule update --init --recursive packages/codegen
  # add others a job actually builds, e.g.:
  #   third-party/dds/cyclonedds packages/rmw/zenoh/zpico-sys/zenoh-pico

The rule — build the in-tree nros, then provision via it; never hand git submodule update. A user builds the nros CLI from source (scripts/bootstrap.sh — nano-ros is a source distribution, phase-288 D1/D2) and runs nros setup to provision sources from nros-sdk-index.toml — they don’t git submodule update their libc / zenoh-pico / cyclonedds fork. CI simulates exactly that. There is no hand-init step:

- name: Build the nros CLI (source, per-checkout)
  run: |
    git submodule update --init packages/cli/third-party/play_launch
    cargo build --release --manifest-path packages/cli/Cargo.toml --bin nros
    echo "$PWD/packages/cli/target/release" >> "$GITHUB_PATH"
# board's whole toolchain + source set:
- run: nros setup <board> --rmw <rmw>
# or a specific source (submodule) by name:
- run: nros setup --source <name> [--source <name>…]

The build’s own interface codegen resolves this same per-checkout nros (via $NROS_CLI / PATH / scripts/build/cargo.sh::nros_cli_bin) as its codegen tool. Caching the built binary is safe (see the source-cache caveat below); caching nros setup --source dirs is NOT (phase-253).

If a build needs a source the index doesn’t provision, that is an index/nros bug to fix (add the [source.*] entry / teach nros), never a git submodule line in the workflow. Examples: dep-chain.yml (per-board), ci.yml (px4-rs via --source), zephyr-dual-line.yml (zenoh-pico + cyclonedds-src + px4-rs via --source).

2. ROS 2 for the interface codegen

nros dev codegen rust / nros dev codegen resolve a message package’s msg/*.msg via AMENT_PREFIX_PATH from a sourced ROS 2. Provide it and source before any build that codegens interfaces:

- name: Install ROS 2 Humble
  uses: ros-tooling/setup-ros@v0.7
  with:
    required-ros-distributions: humble
# ... then in the build step:
- run: source /opt/ros/humble/setup.bash && <build command>

3. Runner OS follows the ROS distro

ros-tooling/setup-ros keys off the runner OS. Humble’s baseline is jammy, so any job that sources ROS 2 Humble must runs-on: ubuntu-22.04 (not the floating ubuntu-latest, which has moved to noble). Pure-lint jobs that need no ROS can stay on ubuntu-latest.

4. Python 3.12 via uv for the Zephyr line

Zephyr 4.4 needs Python ≥3.12; scripts/zephyr/provision-py312-venv.sh requires uv with no fallback. Add it before the Zephyr setup:

- uses: astral-sh/setup-uv@v5

5. Build the in-tree nros CLI — there is no released binary

nano-ros is a source distribution (phase-288 D1/D2): the CLI is built per-checkout from packages/cli/ (one checkout = one CLI = one runtime ABI). Every lane that needs nros runs the same cargo build --release --manifest-path packages/cli/Cargo.toml --bin nros (see §1). The pre-288 prebuilt route (NEWSLabNTU/nros-cli Releases + install.sh, Phase 195) is retired.

6. Canonical codegen invocation: nros dev codegen --args-file

The low-level codegen entrypoint is the codegen subcommand (nros dev codegen --args-file …); the old top-level nros --args-file … was removed in Phase 195. Build glue (*.cmake, *.just, *.sh) that drops the subcommand silently breaks interface generation. The codegen-convention.yml lint (scripts/ci/codegen-invocation-check.sh) enforces this — don’t reintroduce the bare form.

7. Concurrency: cancel-in-progress per ref

Push-storms (a maintainer landing several commits) otherwise queue redundant runs. Cancel the in-flight run for the same ref:

concurrency:
  group: <workflow-name>-${{ github.ref }}
  cancel-in-progress: true

Note the side effect: rapid pushes show cancelled runs for the superseded commits — that is the dedup working, not a failure. Judge a commit by the run on its own SHA.

8. Path-filter triggers to what the workflow proves

Scope on.push/on.pull_request paths: to the inputs the job actually validates, plus the workflow file itself. Keeps the matrix off unrelated commits. Always include workflow_dispatch: so a workflow can be re-run by hand.

Cost discipline

• Validate the dep chain, don’t rebuild the world. Full per-platform builds are expensive; dep-chain.yml proves every (board, rmw) cell resolves (nros setup + codegen + cargo tree, no compile) in one cheap job. Reserve full builds for the sparse lanes (zephyr-dual-line, just build-all).
• Cache the heavy SDKs. A job that re-installs the ~1 GB Zephyr SDK and west-updates from scratch every run should cache scripts/zephyr/sdk + the workspace (Phase 196.3 follow-up).

Preconditions must fail loud

Mirroring the repo-wide test rule (CLAUDE.md): a CI step whose precondition is unmet must exit non-zero, never warn-and-pass. scripts/ci/* check AMENT_PREFIX_PATH, the nros binary, etc. up front and exit 1 with a fix hint. A green check must mean the thing was actually validated.

Split CI into a core lane + per-platform lanes

CI is split so each workflow provisions only what it validates — keeping per-workflow minutes low and failures isolated to one platform:

• Core-libraries lane (ci.yml, job core-libs) — the portable no_std core crates cross-checked on bare embedded targets. No SDKs, no submodules; the only setup is a rustup target add. Split by target (one job per target, parallel), each a single cargo check over the crates compatible with it (e.g. nros-rmw-cffi needs atomic CAS, so it’s checked on thumbv7m but not riscv32imc).
• Cross-platform resolution lane (dep-chain.yml) — proves every (board, rmw) dep chain resolves via one cheap job (nros setup per board pulls only that board’s tools; ROS installed once). No compiles.
• Per-platform build lanes — one workflow per platform, each pulling only that platform’s SDK + submodules (zephyr-dual-line.yml is the template). The heavy lanes; add a new one per platform rather than fattening a shared job.

The worked examples

Historical workflow names

These files no longer exist. They are named above and in the prose because the lessons were learned from them; do not go looking for them.

ROS 2 on a non-Ubuntu host (Arch, Fedora, NixOS …)

nano-ros itself needs no ROS 2 (see installation.md): setup, nros sync, codegen and the first-node flows all work without it. ROS 2 is required only for the interop side — ros2 CLI verification, the rmw_zenoh_cpp / cyclone / fastrtps interop cells, and the ROS-bridging lanes of just test-all.

The problem: ROS 2 ships apt packages for one Ubuntu LTS per edition (humble → 22.04, jazzy → 24.04) and nothing else. On Arch the AUR ros2-humble package source-builds the whole tree against the rolling python/boost/OpenCV and breaks whenever those move — not worth the time. The repo also hardcodes /opt/ros/<distro>/setup.bash in activate.sh and in packages/testing/nros-tests/src/ros2.rs, so a relocated prefix (RoboStack, nix-ros-overlay) does not drop in either.

The route that works: an Ubuntu distrobox sharing your home and network. /opt/ros/humble exists inside it, the checkout keeps ONE absolute path, and every documented lane runs unchanged.

Setup

sudo pacman -S --needed distrobox        # or dnf/apt equivalent

# podman is not required if you already use docker:
DBX_CONTAINER_MANAGER=docker distrobox create -n ros2 -i ubuntu:22.04 \
    --volume /path/to/your/checkout/parent:/path/to/your/checkout/parent -Y

# ROS 2 Humble + the packages the interop lanes use:
distrobox enter ros2 -- bash scripts/dev/ros2-distrobox-setup.sh

Mount the checkout at the same absolute path it has on the host. Different paths on the two sides is the issue-0375 hazard: nros sync writes absolute paths, and every path-keyed cache then splits in two.

ros-humble-desktop is ~2 GB of apt; the script also installs the book’s host prerequisites, rmw_cyclonedds_cpp, rmw_fastrtps_cpp, rmw_zenoh_cpp, domain_bridge, example_interfaces, rosidl_adapter, colcon and rosdep, then verifies each.

rmw_zenoh_cpp is what the zenoh e2e lanes run as their router (phase-362 W1). Without it they report [SKIPPED:capability], which reads as green — three issues were closed “unverifiable” that way on 2026-08-18. just doctor reports which router it can see.

Where the box’s storage lands — check / before you create it

ros-humble-desktop plus the three RMWs is roughly 3–5 GB of image layers and apt content, and BOTH docker and rootless podman keep that under / by default (/var/lib/docker, ~/.local/share/containers). On a workstation whose root filesystem is nearly full, the documented distrobox create above will either fail partway through the apt step or leave the system with no headroom.

Measured on the maintainer host, 2026-08-18: / at 97 % (9.1 GB free) while /mnt/wd — the disk holding the checkout — had 218 GB. Three ways out, in the order worth trying:

1. Put rootless podman’s storage on the big disk. A user-level ~/.config/containers/storage.conf with graphroot pointing at a directory on that disk; no sudo, reversible by deleting the file, and / is untouched. Create the box with the default manager afterwards (drop the DBX_CONTAINER_MANAGER=docker prefix).

2. Use docker exactly as documented and accept the space on /. Simplest and matches the Setup section verbatim — fine when / has ≥ 15 GB free.

3. Install ros-humble-ros-base instead of desktop, keeping rmw-{zenoh,cyclonedds,fastrtps}-cpp. Roughly half the size, and a divergence from scripts/dev/ros2-distrobox-setup.sh — so when a lane later wants a package desktop would have pulled, that divergence is what you will be chasing. Prefer 1 or 2 unless disk is genuinely scarce.

Whichever you pick, the checkout still has to be mounted at the SAME absolute path it has on the host (issue 0375) — see “Two paths to the same checkout”.

What a host without the box gives up

Not a nicety. On a host with no ROS install, just ci on 2026-08-18 reported 176 [SKIPPED:capability] tests — the whole zenoh-router-dependent surface, including every linux_api interop case — and they are counted as skips only because test-all rewrites them. The check-required-features-tests lane (issue 0652) runs a BARE cargo nextest, where the same skips are FAILURES, so tier 1 cannot go green on such a host at all. The box is what turns that surface from “unverified, quietly” into coverage.

Build where you run — clone INSIDE the box (issue 1248)

The box is an ordinary Linux that happens to have ROS. Treat it as one:

distrobox enter ros2
git clone https://github.com/NEWSLabNTU/nano-ros      # NOT --recurse-submodules
cd nano-ros
. scripts/dev/ros2-box-env.sh      # gives this toolchain its own store
just setup-cli && just ci gate

Never --recurse-submodules, here or anywhere (RFC-0097 D10). The submodules are large and most are irrelevant to any one task, and a recursive clone also drags in play_launch’s layer-3 runtime submodules, which nano-ros never builds. scripts/bootstrap.sh initialises the one submodule the CLI build needs (packages/cli/third-party/play_launch), scoped — run it once if you use just setup-cli, which builds but does not init. Everything else comes from nros setup --source <name>, which fetches shallow (--depth 1) rather than deepening to reach a pin.

Never build on the host and run in the box, and never mirror one tree into the other. Host and box differ in compiler and libc, nothing checks that shared artifacts agree, and the failures are loader errors naming a symbol version — which read like code bugs (issues 0400, 0401, 0759).

scripts/dev/ros2-box-sync.sh used to rsync the host tree into a <checkout>-box sibling, and it is retired. Its rules excluded build output by directory NAME, the tree has tracked SOURCE with those names, and the two cannot be told apart by name because provisioned source and build output share directories. That collision ate tracked files five times — packages/cli/ build-support/ (the box could not compile nros at all), Zephyr’s drivers/i2c/target/ (could not configure any Zephyr image), then #1229, #0925 and #1243. Every one of them was a bug in the COPY, not in nano-ros. A git clone has none of them.

The one thing the box still needs from us

A distrobox shares $HOME with the host by design, so ~/.nros and ~/.cargo are the same directory in both — two toolchains, one store. That is not a box concept; it is the general rule a store belongs to one toolchain meeting an environment where a different machine does not imply a different $HOME. ros2-box-env.sh sets NROS_HOME and CARGO_INSTALL_ROOT for the inner toolchain and does nothing else.

The build system asks exactly one question about an environment: are ROS 2 ament packages discoverable here, so message packages can be found? Whether this Linux is bare metal, a container or a VM is the operator’s business and is not encoded anywhere.

Known rough edges

• zstd is not in a stock Ubuntu 22.04, and the prebuilt dists are .tar.zst, so the first prebuilt install fails with tar (child): zstd: Cannot exec. sudo apt-get install zstd inside the box. → issue 0385
• rmw_zenoh_cpp is an apt package on humble+ (ros-<distro>-rmw-zenoh-cpp; this note used to say humble had none). Install it like the others — there is no source overlay to build any more (RFC-0075, amended 2026-08-19).
• If the box is in play, EVERY job runs in the box — against a tree cloned there. Not a style preference: the two sides have different compilers and a different libc, and nothing in the build system checks that the toolchains agree. Do not half-share — a host-side just setup-cli or a host-side fixture build reaches into the same directories. Issue 0759 refused sharing one checkout; issue 1248 retired the rsync mirror that was the other way to get a second tree, because git clone does the same thing without a rule set that cannot distinguish source from build output by name.
• A box created BEFORE that note has no router, and nothing says so loudly. The setup script installs rmw_zenoh_cpp today; a box built earlier does not gain it, and the only symptom is cells reporting [SKIPPED] zenohd failed to start … no rmw_zenoh_cpp/rmw_zenohd found — a skip, so the run stays green (2026-08-22: every router-needing sched cell skipped here, and a measurement script that keys on its cell’s own output line read those skips as FAILURES, which is the opposite misread and the reason it was noticed at all). Check with just doctor, fix with sudo apt-get install ros-<distro>-rmw-zenoh-cpp inside the box.
• The vendored make and nros-launch-resolve are host binaries in a shared tree, exactly like the CLI. The pinned make used to link the build host’s guile and died here as libguile-3.0.so.1: cannot open shared object file (fixed: the index’s source recipe configures --without-guile, so one binary serves both sides — rebuild it once in the box with nros setup --tool make). nros-launch-resolve is per-environment by design (issue 0409): a stale one fails nros sync with a TOML unknown field error naming a field the current schema does have, which reads as a broken system.toml rather than a stale binary. just setup-launch-resolve in the box.
• ROS’s setup.bash dies under set -u (AMENT_TRACE_SETUP_FILES: unbound variable). Any script sourcing it needs set +u around the source.
• The stale-CLI guard fires after any pull that touched CLI sources (in-tree nros CLI is STALE — its sources changed since it was built). Rebuild it in the box and re-run nros_box_publish.
• source in a pipeline runs in a subshell — source env.sh | tail leaves your environment untouched and looks exactly like a broken activation.

Verified on this route

just test-unit in the box: 817 tests, 816 passed, 2 skipped, plus one [SKIPPED] second session refused — shim built with ZPICO_MAX_SESSIONS=1 reported as a failure by the bare-nextest quirk above. No box-specific failures. (Since issue 1704 that test is compiled #[ignore] in a single-session build, so it counts among the skipped rather than the failed.)

Arch host (glibc 2.44) + Ubuntu 22.04.5 box (glibc 2.35), 2026-08-01: nros setup for zenoh and cyclonedds, nros sync, a cyclone-backed examples/linux/rust/talker build, and ros2 topic echo /chatter receiving its messages over rmw_cyclonedds_cpp with ros2 topic list showing /chatter.

Build System & Caching

Audience. Contributors working on the repo build/test matrix. End users never need this — they consume nano-ros with a single add_subdirectory(<repo-root>), no install step, no find_package. This handbook documents the other build world: the in-repo just orchestration that builds + tests every platform × RMW, and the caching that keeps it incremental and correct.

Canonical sources this page summarises: CLAUDE.md (“Build”, “Build tiers”, “Build parallelism”), Phase 176 (unified jobserver), Phase 181 (fixture SSOT

• Ninja), Phase 177.9 (staleness probes).

TL;DR cheat-sheet

just setup                 # one-time SDK/toolchain install (tiered; see CLAUDE.md)
direnv allow               # once after clone — else zpico-sys/build.rs panics

just build                 # workspace + transports (fast inner loop)
just build-examples        # + every example
just build-all             # + test fixtures (everything test-all consumes)
just build-test-fixtures   # just the fixtures + writes the test-all stamp
just build-test-fixtures lane=linux   # only what the tier-1 lane (`just ci`) runs

just test-unit             # ~5s          ⊂
just test-integration      # ~30s         ⊂
just test                  #              ⊂
just test-all              # heavy QEMU/Zephyr/ROS-interop/miri/codegen

just <plat> build|build-all|test|ci   # narrow to one platform first
NROS_BUILD_JOBS=8 just build-all       # cap parallelism

Rule of thumb: a platform-specific failure → run the narrow just <plat> build-all (or closest build/build-examples/build-fixtures) before the root just build-all. Always just ci after a task. Never sudo — if a step needs it, tell the user.

Generators — Ninja by default

The repo’s CMake builds use Ninja (-G Ninja), falling back to Make only when ninja is absent. Ninja gives reliable incremental rebuilds and fits the Phase 176 fifo jobserver; Make was dropped from the staleness path because make -q mis-reported up-to-dateness.

• Configure-once: scripts/build/cmake-incremental.sh::nros_cmake_configure_if_needed configures a build dir only when its CMakeCache.txt / generated build system is missing; cmake --build then handles reconfigure on CMakeLists changes.
• Generator-mismatch wipe: a dir configured with the other generator is rm -rf’d and reconfigured (you can’t switch generators in place).
• Pinned tools: the unified jobserver needs make ≥ 4.4 + ninja ≥ 1.13 (apt’s 4.3/1.10 lack the fifo jobserver). Both are SDK-store tools — nros setup --tool make (source tarball; upstream ships no binary) and nros setup --tool ninja (upstream prebuilt zip) — and scripts/sdk-path-tools.txt puts them on PATH (incl. a gmake → make-4.4 alias). Resolve an exact path with nros dev sdk path <tool>.

Per-RMW build dirs = cache isolation

Each example builds into a per-RMW directory with its own cargo target dir:

examples/<plat>/<lang>/<example>/
  build-zenoh/       cargo/   …   # -DNROS_RMW=zenoh
  build-xrce/        cargo/   …   # -DNROS_RMW=xrce
  build-cyclonedds/  cargo/   …   # -DNROS_RMW=cyclonedds

Because each RMW (and its Corrosion cargo target dir) is physically separate, selecting a different RMW cannot collide with another RMW’s cache — there is no shared target dir to invalidate. Platform is fixed per build dir (-DNROS_PLATFORM=<plat> at configure). This layout is why the old manual cache-invalidation idea (archived Phase 145) was retired: the directory shape makes it unnecessary. (Rust-only examples build via plain cargo into target-<rmw>/; the per-RMW principle is the same.)

Parallelism — NROS_BUILD_JOBS + the unified jobserver

One knob scales every parallel recipe:

NROS_BUILD_JOBS=8 just build-all     # default: nproc

Unified jobserver (Phase 176). just build-all auto-routes to build-all-jobserver when the pinned make 4.4 + ninja 1.13 are present: a single GNU-make fifo jobserver spans cargo + build-script cc + ninja-via-west

• cmake, allocating tokens dynamically instead of a static per-tool split. Under the jobserver (NROS_JOBSERVER=1) recipes drop their explicit -j/--parallel/CMAKE_BUILD_PARALLEL_LEVEL so the tools inherit the pool.

• Same artifacts either way; without the pinned tools it falls back to a static split.
• NROS_NO_JOBSERVER=1 forces the static path.
• Never re-introduce a hardcoded parallel --jobs <n> without threading ${NROS_BUILD_JOBS:-N} through.

Stall watchdog (issue 1403). Every fixture/jobserver make (the fixture make drivers, fixtures-build.sh, workspace-fixtures-build.sh, the jobserver-pool, build-all-jobserver) runs under scripts/build/make-stall-watchdog.py. If make sits with no live child and no CPU for NROS_JOBSERVER_STALL_SECS (default 600; 0 disables; sampled every NROS_JOBSERVER_STALL_POLL_SECS, default 15), the watchdog writes jobserver-stall-<pid>-<time>.txt into that build’s log dir (make’s wchan/syscall/status or stack, its fds, the jobserver FIFO and its FIONREAD token count, every other process holding a jobserver, a gdb backtrace when ptrace allows), kills make and its descendants — by pid, nothing else — and exits 75 with NO VERDICT: jobserver stall — <file>. That is a lane that never answered, not a build failure; just nightly-triage counts it as NO VERDICT. Attach the file to issue 1403. A normal build is untouched: same exit status, no output.

See docs/roadmap/archived/phase-176-unified-jobserver-build-orchestration.md.

Build & test tiers

Each tier is a strict superset of the previous:

Per-platform: just <plat> build ⊂ build-examples ⊂ build-fixtures ⊂ build-all and just <plat> test|test-all|ci. <plat> = target families (qemu, zephyr, board groups). Support services (zenohd, cyclonedds) are not platform scopes. Orchestration lives in justfile + just/*.just.

Fixture SSOT — examples/fixtures.toml

Per-fixture build options (features, --target-dir, env, per-RMW variants, cmake -D defs, cross target) live in one manifest, examples/fixtures.toml, consumed by both the build recipes and the test-all staleness probe — so the build and the probe use identical options (no feature-thrash).

• Reader: scripts/build/fixtures-manifest.py list --platform <p> --lang <l> [--rmw <r>] [--for-probe]. Emits \x1f-separated records (unit-separator, not tab — tab is IFS-whitespace).
• Builder: scripts/build/fixtures-build.sh <plat> [lang] [rmw] — one shared loop; rust cells go through cargo build, C/C++ cells through cmake --build (configure-once + per-RMW dir). Platform-wide -D defs (toolchain, codegen tool, SDK dirs) are injected by the recipe via NROS_CMAKE_EXTRA_DEFS.
• Cross-toolchain platforms inject RUSTUP_TOOLCHAIN / SDK env via the recipe (e.g. ESP32-C3 = riscv32imc on the workspace nightly + build-std).

Staleness discipline

A prebuilt fixture that’s silently stale would let test-all run against the wrong binary. The discipline:

• Coverage gate. build-test-fixtures stamps target/nextest/.fixtures-built with the lane and the fixture coordinates it built, not just a timestamp; test-all’s _require-fixtures fast-fails (~1 s) if the stamp is absent or does not cover the run implied by NROS_FIXTURE_LANE (bypass: NROS_SKIP_FIXTURE_CHECK=1). So a lane=linux build satisfies just ci (the tier-1 lane) and is correctly rejected by an unscoped test-all. A pre-issue-0393 timestamp-only stamp is read as lane=all, which is what it meant.

  Note “the run”, not “the lane” (issue 0482). A lane’s coordinates are what it keeps fresh; what must exist is decided by which tests the recipe executes. The two are answered separately, and nros_lane_build_lane is the mapping between them (CiLane::run_scope is the declaration; lane-coords <lane> --build-lane prints it).

  Every lane narrows its run by fixture COORDINATE (NROS_TEST_COORDS), and each is its own build lane. The tier-1 lane used to narrow by test/binary NAME (NROS_TEST_SCOPE=linux); phase-395 W19 moved it to coordinates, and issue 1900 retired the name filter’s reader, which no runner had set since. Each test asks the lane BEFORE it probes the host, so a test of a platform the lane did not select reports [SKIPPED:lane] on every host; scripts/test/check-out-of-lane-deselected.py proves it on the just ci gate lane.

  A name filter could never have expressed the tier-2 lane anyway: it is 1-wise over platform, every platform is in the lane, so a platform-token filter excludes nothing, and the saving is in lang × rmw within a platform, which test names do not encode (issues 0357, 0482). So the narrowing happens in the fixture resolver (phase-340 W3, nros_tests::fixtures::lane): a resolved artifact is attributed back to its examples/fixtures.toml row through row_artifact_root(), and a row outside the lane reports SKIPPED instead of missing. Build and run then apply the same row_coord() predicate to the same coordinate file, so just build-test-fixtures lane=tier2 && just ci-matrix is a coherent pair. Before W3 the tier-2 lane ran the whole suite, and a lane=tier2 build satisfied the preflight for a run that then failed STALE on 34 unbuilt coordinates.

  The skip is keyed on the coordinate, never on the artifact being absent: an in-lane fixture that is missing or stale still fails hard, and a path the manifest cannot attribute (Zephyr west leaves, the compile-check lane — both built module-level, so nothing is omitted) is never skipped.

  The preflight enforces the pairing rather than trusting it: accepting a build narrower than all requires NROS_TEST_COORDS to be set and to hold exactly that lane’s coordinates, so a hand-run NROS_FIXTURE_LANE=tier2 just test-all is refused instead of reproducing issue 0482. A full build needs no narrowing, so NROS_FIXTURE_LANE=tier2 on top of lane=all still works — scope the freshness gate, keep the run wide.

• Rust cells. scripts/test/rust-fixture-stale.sh reuses cargo’s own fingerprint — cargo build … --message-format=json reports "fresh":false for a stale unit, so the probe both detects and self-heals.

• C/C++ cells. scripts/test/cmake-fixture-stale.sh runs the incremental cmake --build (near-no-op when fresh) and flags cells that actually rebuilt. (ninja -n is unusable here — Corrosion’s cargo step is an always-run custom command, so it always reports pending.)

• Probe opt-out. Cells needing a recipe-injected toolchain the probe can’t supply (e.g. the ESP32 nightly + build-std) set skip_probe = true; the reader’s --for-probe omits them so they don’t toolchain-thrash.

• Source-list / drift gates. zpico-sys (vendored zenoh-pico, Phase 136.6) and nros-rmw-xrce-cffi (vendored uxr/micro-cdr, Phase 145.4) verify each vendored source root resolves to a real dir with .c files and panic with a git submodule update --init hint on drift. cbindgen build scripts (nros-c, nros-cpp, zpico-sys) emit cargo:rerun-if-changed=cbindgen.toml

  • src/.

• Probe runs in preflight. _check-fixtures-stale runs both probes (rust + cmake) over the manifest before the test-all nextest stage; it warns + self-heals rather than hard-failing.

Patched QEMU

QEMU networked tests use a patched qemu-system-arm (icount + MPS2 fixes) built by just qemu setup-qemu into build/qemu/bin/. The harness picks it up automatically (nros_tests::qemu::qemu_system_arm_cmd()); the system binary is the fallback. See Patched qemu-system-arm.

Cyclone DDS in the tree

nros_provide_cyclonedds() self-provisions Cyclone from the pinned third-party/dds/cyclonedds submodule when no prebuilt install is given, so a bare cmake/cargo build needs no pre-step. just cyclonedds is a contributor-only recipe that builds it ahead of time. On a tier without the cross toolchain, the embedded-Cyclone tests are filtered out of test-all (skipped, not failed). Details: book-internals/src/dev-env/sdk-tiers.md § “CycloneDDS — self-provisioned in CMake”.

See also

• Build Commands — the user-facing quick reference.
• zpico-sys Build Architecture — the zenoh-pico cross-compile.
• CLAUDE.md — the authoritative build/test-tier + parallelism policy.
• Phase docs (docs/roadmap/archived/): 176 (jobserver), 181 (fixture SSOT + Ninja), 177.9 (staleness probes).

Codegen — the pack pipeline

nano-ros generates ROS 2 message/service/action types (and their per-package scaffolding) from .msg/.srv/.action files. The generator is a four-stage pipeline (RFC-0068): parse → resolve → lower → render. This page covers the last stage — render — and how to change or add a target language.

Render = a data pack + a runtime template

Every backend renders through one minijinja environment (packages/cli/rosidl-codegen/src/render.rs) over data packs under packages/cli/rosidl-codegen/packs/:

Each pack directory carries a pack.toml that declares it: its language, its registry_order, its templates as { key, file } rows, and — for a pack that names C-family artifacts — the header_extension / guard_suffix / source_extension that generator::naming derives every artifact name from. build.rs reads every manifest and generates the template registry and the Surface enum, so a pack directory is a pack because it exists and describes itself (phase-469 W1). Nothing in render.rs names a template.

A pack is just .jinja templates. The Rust side hands each template a serde-serialized view struct (the render context) and never spells a type itself — the type strings are composed in the pack by registered filters:

• c_type / c_array_suffix, cpp_type / cpp_array_suffix, cpp_repr_c_type / cpp_view_repr_type
• rust_type_rmw / rust_type_idiomatic, rclc_type
• snake_case

The view struct carries only neutral facts (the parsed field_type, resolved capacity, storage mode, current_package, …); the filter maps those to the language’s syntax. This is RFC-0068’s “what vs how” seam: what a type is lives in the IR; how a language spells it lives in the pack + its filter.

Changing a template

Edit the .jinja under packs/<lang>/ and rebuild the CLI (just setup-cli). The codegen fingerprint (RFC-0061) hashes every bundled pack’s content, so any template edit marks the affected fixtures stale — no separate bookkeeping.

Overriding a pack at runtime — no rebuild

Point the renderer at an external directory of .jinja files: a file named <template-name> or <template-name>.jinja there overrides the bundled pack of that name; anything absent falls back to bundled.

export NROS_TEMPLATE_DIR=/path/to/my/pack
nros dev codegen rust …          # uses the override, no recompile

(Equivalently, rosidl_codegen::render::set_template_dir(dir) from Rust, called once before the first render.)

The stable template names are the templates keys in each packs/<dir>/pack.toml (e.g. message.h, message_nros.rs, cargo.toml, _field.jinja); nros’s own copy of the generated registry is at target/**/build/rosidl-codegen-*/out/ pack_registry.rs. tests/external_pack_smoke.rs proves the override + fallback.

Do not set NROS_TEMPLATE_DIR during fixture or CI builds. The fingerprint hashes the bundled packs; an external override would silently produce output that disagrees with the recorded fingerprint.

Adding a language

Step 0 — decide which SURFACES you need

This is the step that sizes the work, and skipping it is why “add a language” sounds bigger than it is. A pack is a (language × surface) pair, not a language: Rust has four message packs, and the cpp pack emits Rust as well as C++.

There are two independent axes. On the message side: idiomatic, embedded-idiomatic (no_std), FFI/ABI (repr(C)), bridge glue, packaging. On the entry side: the entry TU, and the component seam. A language can take an entry surface with no message surface (it consumes another language’s messages) or the reverse.

RFC-0091 §6 has the full table, including what each surface drags in — an FFI surface inherits the cross-language memory-agreement gate, a bridge surface is two spellings of one type that must move together. A language can ship with two surfaces and gain the rest later: a Zig component installed by a C or C++ entry needs the idiomatic and FFI message surfaces and the component seam, and nothing else.

Step 1 — the message pack

1. add its .jinja templates (a new packs/<lang>/) and a pack.toml declaring them. No Rust edit: the registry is discovered from the manifest, and a pack directory with no manifest — or a .jinja no manifest claims — fails the build naming the file. If the pack names C-family artifacts, its three naming fields make it a generator::naming surface and the Surface variant is generated for it;
2. if it needs type spelling the existing filters don’t cover, add a filter set for the language (rosidl_codegen::filters::FILTER_SETS) wrapping a *_spelling function in types.rs. The filter set is all the Rust that type SPELLING takes; the set is keyed by the pack that CALLS the filter, not by the syntax it emits — cpp_repr_c_type returns Rust;
3. add a generator per kind (generator/{msg,srv,action}.rs, or one file like generator/cpp.rs) that builds the context, names the output files and calls render::render("<template-name>", &ctx), and its arm in nros interface generate (cmd/generate.rs).

Step 2 — the entry pack, if the language writes entries

An entry pack is DATA. Since phase-494 there is no per-language entry emitter: one renderer (codegen/entry/emit/mod.rs) lowers the plan once into nros_entry_lower::LoweredEntry and hands THAT to the pack’s template.

1. packs/entry/<surface>/entry.<ext>.jinja over LoweredEntry. Nodes (with their component kind, resolved name and namespace, params, remaps, QoS), board family and boot shape, the C-ABI runner names, the executor layout (one executor, sched contexts, or per-tier setups), tier rows, the boot blob and the monitor tables all arrive computed and RAW — a template decides where a value goes and spells it, never how to quote it;
2. packs/entry/<surface>/pack.toml — the extension, whether the TU is C-family, the entry template, its templates = [{ key, file }] rows, and what the renderer must know: context = "lowered-entry", the filters its templates call, the component kinds it constructs (components), and whether it calls the board’s C-ABI runners (c_abi_runners). CMake reads it through nros dev codegen entry-pack. The build discovers the directory — there is no registry row to add;
3. one variant on Language in nros-lang. Every consumer sees it — one enumeration;
4. only if the language needs a spelling no existing filter provides (the set is c_str, pkg_ident, cpp_board_class): one function and one row in codegen/entry/filters.rs. A spelling is a correctness property, so it is Rust; everything else about the pack is not.

testdata/entry-packs/zig/ is a whole entry pack written this way — a manifest and one template — and a unit test renders every golden C plan through it.

Step 3 — the goldens

Add the coordinate to the entry golden harness (codegen/entry/golden.rs; its Emitter enum names the entry point a row renders through), run NROS_UPDATE_GOLDEN=1 cargo test -p nros-cli-core --lib codegen::entry::golden, and read the diff. The generated source is a file, not a claim.

just check entry-pack-conformance refuses a half-wired pack in EITHER family — entry and message; the name is historical (see the recipe comment). A directory with no manifest, a language pack missing the fields its consumer needs, a registry key or registry_order claimed twice, a template file that does not exist, a .jinja no manifest claims, a Language variant nothing renders, or a language whose bytes are recorded in no golden — and, since phase-494, a new codegen/entry/emit_<x>.rs, because an entry language is a pack. Run it before you believe the pack works.

What this does NOT make cheap

The toolchain story — how CMake compiles the language, how it links libnros, how its components declare themselves. The codegen cost becomes a pack; the build-integration cost does not. Budget for it separately.

No per-language TYPE SPELLING lives in the message builders — the packs and their filters own it, and since phase-469 W1 so do the artifact-naming suffixes (file and guard names). The builders still hold the per-language Rust that is a RULE rather than a parameter — which generator builds which context, and the ROS kind word, which is identical on every surface. An entry language brings no emitter (phase-494): its pack and, at most, a spelling filter. Implemented by phase-335 (RFC-0068), phase-432 (RFC-0091), phase-469 and phase-494.

The canonical build path

What this is. A map of how a nano-ros image actually gets built: the three roads, what carries a configuration knob to the compiler on each, where a build finds a resource it did not produce, and which RFC owns each decision. It exists because the decisions are all made and all scattered — several RFCs own a slice each, and nobody owned the picture. The rule that ties them together is RFC-0103’s: every fact has one authored home, every other copy is generated from it, and every road reads the resolved answer.

What this is not. A new decision. Every rule here is already settled somewhere; this file points at the owner and explains how the pieces meet. Where it disagrees with an RFC, the RFC wins and this file is the bug.

Every number lives in a command, not in this file:

python3 scripts/nros-build-wiring.py          # roads, scripts, knob sources, readers
python3 scripts/nros-build-wiring.py --roads  # just the road table
python3 scripts/nros-build-wiring.py --scripts  # the build.rs census, by ROLE

That is deliberate. A hand-authored census is the failure mode this repository has recorded more than any other — the package-directory list that named four directories which no longer existed while omitting four that did (issue 1211), the RMW parity map that disagreed with the shape tool by 25 symbols, the sizes-header family. A table of counts reads as current and answers about the tree somebody measured once.

The five stages

nros build (RFC-0065 D1) is one pipeline, whatever the road:

1. DISCOVER   walk package.xml → pkg index → topological order
2. RESOLVE    the image: argument > [system] default_images > list and fail
3. PREFLIGHT  toolchains / SDKs / sources present?
4. GENERATE   msg bindings + system model + the ROOT BUILD FILE → build/<coord>/
5. EXEC       cmake --build / cargo build / west build — stderr untouched

The road is chosen at stage 4/5 by the board, never by the language mix (RFC-0065 D3). A workspace that crosses languages goes to cmake, because cargo can be consumed as a cmake target through Corrosion and cmake cannot be consumed as a cargo target (RFC-0024 §6.3). Choosing can also fail: one combination — an esp32 board with a graph that crosses languages — has no road and says so, rather than naming a tool that cannot build it.

The three roads, and the thing that actually differs

The stages are shared. What is not shared is how a resolved knob reaches the compiler — and that is the whole of this document, because each road carries it differently and each carrier has produced its own delivery defect:

The road names below are Driver variants (Cargo, CMake, West in packages/cli/nros-cli-core/src/builder/plan.rs); check-build-wiring-roads holds this table and the enum to each other in both directions.

One road emits no root on purpose: a Zephyr app is already a complete cmake project and its Kconfig overlays are user intent, so stage 4 emits a root only where a root would otherwise be hand-written (RFC-0065 D3).

The road that was deleted, and the test that finds the next one

There was a fourth, idf.py, chosen for platform = "esp32" whenever the graph crossed languages. Its carrier cell was empty — and that is what ended it (RFC-0065 D3 amendment, 2026-09-28). Every other road’s cell names a mechanism that has produced a measured defect; an empty one says the road delivers no resolved knob at all, which is issue 0460’s class applied to a whole road rather than to one lane of one.

So the question this table asks of a new road is not “does the tool work” but what carries a knob on it, and how would I know if nothing did. A road that cannot answer is not a capability with a missing test; it is a claim. nros build now REFUSES that combination, naming issue 1525, because answering cmake instead would be the same claim in a tool that also has no esp32 road.

There WAS a fourth carrier that is not a road. On NuttX, <nros/nros_config_generated.h> resolved to a committed snapshot, not the per-build header, so a template edit that the generated artifacts consume was a two-file change (issue 1115) — forgetting it broke every NuttX C and C++ image from a clean clone for two days — and the snapshot’s sizes fell below the build four times (issue 1568 measured a live overrun). Issue 1569 retired it: the NuttX FFI build reads the per-build headers from the nros-c/nros-cpp links channels (DEP_NROS_{C,CPP}_CONFIG_INCLUDE), which also order it after both writers. The snapshot survives as *_buildless.h for header-only checks that compile with no build, behind NROS_CONFIG_BUILDLESS, which check-config-fallback-macros forbids any build to define.

The ladder: what may state a value

RFC-0049 orders the rungs builtin < platform < board < env. A higher rung that states a value wins; a rung that is silent is not a zero.

One name per knob (RFC-0103 D6, phase-484 W5a). Every knob is one row in the registry packages/tooling/nros-zephyr-build/knobs.toml: id <group>.<name>, env NROS_<GROUP>_<NAME> (a knob of one RMW is NROS_<RMW>_<X>), Kconfig symbol CONFIG_NROS_<ID> unless the row names another. Everything else is generated from that file: the Rust KNOBS table and its derived KCONFIG_PAIRS, cmake’s NanoRosKnobRegistry.cmake, and the knob section of zephyr/Kconfig. An older spelling (ZPICO_MAX_PUBLISHERS) is a RETIRED row (phase-484 W8): every reader on every road — a build script, a configure, the CLI’s [image.<id>] env — refuses one that is set, in the registry’s one refusal sentence, naming the canonical spelling. The row stays one release as that error table; check-retired-knob-spellings keeps every env reader and writer off the old names (the same words stay as the zenoh-pico shim’s C macro names).

The app rung’s two spellings, one set of rows (phase-484 W8, RFC-0103 D6). An image states a registry knob as [image.<id>.knobs.<group>] <name> = <value>, the shape a board’s [knobs.<group>] has (the key is the row’s toml key, else its id split at the first _). cmd::leaf_settings::app_rung validates each key against the registry (knob_by_image_key: unknown key, type, range, names are errors naming it) and renders it to the knob’s env spelling, which is the row an [image.<id>] env entry states. Every carrier below is fed from those rows, so moving a row between the two spellings changes no byte of any carrier (measured on 29 rows: 575 dumped files identical, the west Kconfig.from-system-toml differing only in its provenance comment). An env row naming a registry knob, by its env name or by the Kconfig symbol west renders it to, still works and prints one IMAGE_ENV_DEPRECATED_TEMPLATE warning; stating a knob both ways is an error. The gate no_tracked_system_toml_states_a_registry_knob_as_an_env_row (check-cli-tests) keeps the tree on the knobs spelling.

The app rung’s carrier, per road (issue 1712). Cargo: the generated nros-cargo.toml [env]. CMake: --config <build>/nros/nros-image-env.toml on every cargo command the configure spawns (written by nros dev image leaf-system --image-env-out), the same unforced [env], so an exported variable still wins and a changed row re-runs the build scripts that read it. West (phase-481 W1, RFC-0098 D11, phase-484 W5b): the nano-ros module’s module_ext_root hook writes two files. Knobs go to <build>/nros/Kconfig.from-system-toml, which the module’s prompt-less knob section osources first, so the image’s [image.<id>.knobs] (or env) value is the symbol’s default and reaches $DOTCONFIG. A one-build override is -DNROS_OVERRIDE_<ID>=<v>; a -DCONFIG_<knob> or a knob line in a conf file is refused, naming system.toml. The image’s RMW choice, endpoint and domain stay in the conf fragment <build>/nros/<image>.conf, LAST in EXTRA_CONF_FILE, because a Kconfig choice cannot be a prompt-less default (W5b’s M1). A row with no Kconfig symbol rides the same unforced --config [env] file on the Zephyr cargo commands. A Zephyr leaf’s conf files hold no nano-ros knob at all (gate check-leaf-conf-nros-knobs). A workspace image whose configuration differs gets its own cmake configure (build/<coord>-cfg<hash>/, D12).

Beside the ladder sit two things that are facts, not rungs, and the distinction matters because a fact has no $DOTCONFIG counterpart:

• Derived entity facts — NROS_DECLARED_*, NROS_ENTITY_*, computed by cmake for this image from the resolved SystemModel. Absent means “no answer”, never “zero” (issue 1122).
• The sizing descriptor — <build>/nros/sizing/<entry>.toml, per-FIELD stated-or-refused (RFC-0100 D4). Read it through nros-sizing-descriptor, never by hand, and never re-carry one of its facts through an env knob.

A platform that resolves to no descriptor is an ERROR (phase-468 W1): every board-declared name has a descriptor or an explicit declaration that it has no rungs, and an unanswered name panics rather than warning.

Where a resource comes from: the location ladder

A build also needs things it did not produce: vendored source trees, cross toolchains, the nros CLI, generated artifacts. Before phase-484 a resource’s location was decided by 13 mechanisms, and 13 resources were found two or more ways (issue 1767, archived with a per-item table). RFC-0103 D3 gives them one ladder, nros_build_paths::locate, with these rungs, highest first:

Every road asks the same ladder, never a copy of it:

The image’s resolve records the answers in resolved.toml [locations] (path, rung, and transient for an environment answer). That file is a record, not a carrier: build scripts re-ask the ladder.

Two roots, one name each (phase-484 W1/W2d): the store is $NROS_HOME (NROS_STORE and NROS_SDK_STORE are refused, naming it), and the checkout is found by --nano-ros-path (-DNROS_REPO_DIR in cmake), then a walk up from the workspace, then the nros binary’s own location (a release’s share/nano-ros, or the checkout an in-tree build sits in). No environment variable names it. activate.sh exports no location at all: it puts nros on PATH and nothing else, so a linked worktree cannot inherit another checkout’s trees (issue 1280).

The last three resources found more than one way went through the same fix, each with a gate that refuses a new hand-written path:

• the nros and nros-launch-resolve binaries: nros_cli_bin [--checkout-only], nros locate tool, and in Rust nros_cli_in_checkout / launch_resolver_from (issue 1850, check-host-tool-path-literals);
• QEMU: nros_qemu_bin and the nros-qemu-ladder crate (CLI + tests; issue 1851, check-qemu-one-ladder);
• the generated-header directory names: nros locate artifact and cmake/NanoRosArtifactLayout.cmake, which read nros_build_paths::artifact (issue 1852, check-artifact-path-literals).

Where a selection is stated

The board, RMW, domain, locator, IP address and transport are stated in the image’s system.toml ([image.<id>], [system]). The board’s descriptor supplies the defaults through [board.net]: every QEMU board joins the one qemu network (10.0.2.0/24, gateway 10.0.2.2), and a router default is derived from the gateway as tcp/<gateway>:7447 (phase-484 W4b). The board crate reads generated BOARD_NET_* constants rather than restating them.

On the cargo road nros::main! bakes the image’s selections into a deploy overlay, and every board family merges it over its own defaults with BaseConfig::apply_overlay before opening the session. That merge is the only way the image’s locator reaches a board, because BootConfig::over_board_defaults lets the board’s value win. NuttX once skipped it and every NuttX Rust image dialled an empty locator (issue 1836), so check-image-locator-bake now checks that each family’s crates apply the overlay rather than trusting that they do. ROS_DOMAIN_ID and NROS_LOCATOR remain runtime overrides; NROS_BOARD, ZENOH_LOCATOR and a build-time NROS_DOMAIN_ID are not inputs.

Where a value must NOT be stated

Three of these are gated, because each was written by hand at least once:

• Not in examples/**/.cargo/ — an example’s build configuration is generated from one board choice (RFC-0098 D1). Gate: check-example-cargo-dirs.
• Not in a committed SystemModel — models are build artifacts. Gate: check-no-tracked-models.
• Not in a second spelling of a derivation — one formula, and its INPUTS derived once too (issue 1025). Gate: check-fixture-artifact-dir-inputs.

Reading a knob: the two shapes

A Kconfig knob reaching the Zephyr C lane and not the Rust one is issue 0460, the hazard this whole area exists to contain. Readers come in two shapes and they are not interchangeable:

• DERIVED — the Kconfig name is built from the env name as CONFIG_{env_name}. A knob such a reader names is a knob it resolves, so “does the file mention it” is a sound test.
• TABULATING — an authored table, needed where the two vocabularies differ (NROS_ZENOH_SUBSCRIBER_RING_DEPTH ↔ CONFIG_NROS_SUBSCRIBER_RING_DEPTH). Here a mention proves nothing: three knobs were mentioned with no table row and silently compiled crate defaults (issue 1490).

Held by check-kconfig-knob-forwarding. Both shapes now go through one resolution function, nros_zephyr_build::knob("<ENV>") with the Kconfig and env sources as inputs (phase-468 W4). Since phase-484 W5a the TABULATING table is no longer authored: KCONFIG_PAIRS is derived from the knob registry, so a missing row is a missing registry row, which check-zephyr-kconfig-knobs holds against the rendered Kconfig.

How to prove a knob is delivered

The baseline cannot show a delivery failure. Unset, the Kconfig default and the crate default are usually the same number, so “delivered” and “fell back to the same value” are one observation. Use a non-default probe:

# append to the leaf's prj.conf, rebuild, read the generated const, then REVERT
CONFIG_NROS_SUBSCRIBER_RING_DEPTH=7

That is how issue 1490 was found and how its fix was accepted (4 → 7).

Who reads a carrier: the build scripts

A carrier delivers to a build script, and there are 66 of them. What each is FOR is a ROLE, printed by --scripts; phase-471 replaced a set of capability LETTERS with those roles because a letter was a grep for a TOOL and a tool is not a role — cc::Build put three first-party test C files in the same class as the FreeRTOS kernel, and left cyclonedds-sys (≈200k lines of vendored C through cmake::Config) reading as “re-roots paths and nothing else”.

Two things in that output are load-bearing here and not obvious:

• source_origin — where a C-compiling script’s sources come from. Three answers exist: a path-valued SDK variable, a workspace-relative path, and a links= hand-off (DEP_*) from the crate that already resolved the tree. Only the first is exposed to issue 1280, because only the first can be inherited from another checkout. Prefer the links hand-off where one is available: it also orders the two build scripts, which a shared environment variable cannot do.
• the PATH RESOLUTION section — issue 1280’s BUILD-SCRIPT half, which no gate covers (check-inherited-checkout-paths holds the shell half and names no build.rs). It read 5 before issue 1527 and reads 0 now. A build script that resolves one of these variables goes through nros_build_paths; a private helper that takes the name as an argument is the shape that hid two of the five, because no literal-matching probe can see inside it.

The classes, the outlier verdicts and the migration order are phase-471.

Which RFC owns what

The recurring failure, stated once

Every defect named above is the same shape at a different site: a value that was resolved correctly and delivered to only some of the things that read it. 0460 (Kconfig reaches C, not Rust), 1115 (the header reaches every road but NuttX), 0491 (one directory, three spellings), 1025 (one formula, two derivations of its inputs), 1490 (the table that resolves, missing a row).

So the question to ask of any new wiring is never “is the value right” — it is which roads carry it, and how would I know if one did not. The answer is a non-default probe on each road, not a reading of the code.

Codegen Type Mapping Reference

How nros dev codegen rust maps ROS 2 message fields to Rust, C, and C++ types.

Overview

Every ROS message generates a single owned Rust struct — no lifetime parameters, no dual Msg<'a> / MsgOwned variants. Unbounded fields (string, uint8[], etc.) use fixed-capacity heapless types. All generated types implement Serialize, Deserialize, RosMessage, Default, and Clone.

For zero-copy access to large payloads (images, point clouds), use the raw subscription API and read CDR fields directly with CdrReader instead of fully deserializing into owned structs.

Rust type mapping

Primitive fields

String fields

heapless::String<N> is re-exported from nros_core::heapless for use in generated code and in user code without a direct heapless dependency.

Sequence fields

The default sequence capacity is 64 elements. For large-payload messages where you need more elements at runtime, use the raw subscription API with CdrReader to read fields without materializing the whole vector.

Fixed-size arrays

Arrays with more than 32 elements use a manual Default impl instead of deriving, since Default is only derived for arrays up to [T; 32].

Nested message fields

Nested message types are embedded inline:

Traits

Every generated message type implements:

RosMessage requires Serialize + Deserialize:

#![allow(unused)]
fn main() {
pub trait RosMessage: Sized + nros_serdes::Serialize + nros_serdes::Deserialize {
    const TYPE_NAME: &'static str;
    const TYPE_HASH: &'static str;
}
}

Zero-copy access via raw subscription API

For large messages, deserializing into an owned struct (e.g., Image { data: heapless::Vec<u8, 64> }) truncates at 64 elements. Use the raw API and CdrReader instead:

#![allow(unused)]
fn main() {
use nros_core::{CdrReader, RosMessage};

// Read only needed fields without full deserialization
executor.add_subscription_buffered_raw::<65536>(
    "/camera/image",
    Image::TYPE_NAME,
    Image::TYPE_HASH,
    QoSProfile::default(),
    |cdr: &[u8]| {
        let mut r = CdrReader::new_with_header(cdr).unwrap();
        // Skip header fields by reading and discarding them
        let _stamp_sec: u32 = r.read_u32().unwrap_or(0);
        let _stamp_nsec: u32 = r.read_u32().unwrap_or(0);
        let _frame_id_len: u32 = r.read_u32().unwrap_or(0);
        // ... read only what you need
        let height: u32 = r.read_u32().unwrap_or(0);
        let width: u32 = r.read_u32().unwrap_or(0);
        let pixel_data_len = r.read_u32().unwrap_or(0) as usize;
        let pixels = r.read_slice_u8(pixel_data_len).unwrap_or(&[]);
        process_pixels(width, height, pixels);
    },
)?;
}

The cdr slice borrows from the triple buffer slot and is only valid for the duration of the callback — do not store a reference to it.

C type mapping

The C deserializer sets pointer+length fields to point into the CDR buffer (valid for the callback duration). The C serializer reads from the pointer+length fields.

C++ type mapping

rclcpp::Span<T> and rclcpp::StringView are freestanding C++14 types defined in rclcpp/span.hpp. They provide data(), size(), begin(), end(), and operator[] — same API as std::span / std::string_view.

Service types

Service request and response types are regular message structs that also implement Serialize and Deserialize. Nested message fields in service types are embedded inline (same as standalone messages).

Foo implements RosService:

#![allow(unused)]
fn main() {
impl RosService for Foo {
    type Request = FooRequest;
    type Reply = FooResponse;
    const SERVICE_NAME: &'static str = "...";
    const SERVICE_HASH: &'static str = "...";
}
}

Action types

Action goal, result, and feedback types are regular message structs. The action marker type implements RosAction:

#![allow(unused)]
fn main() {
impl RosAction for MyAction {
    type Goal = MyActionGoal;
    type Result = MyActionResult;
    type Feedback = MyActionFeedback;
    const ACTION_NAME: &'static str = "...";
    const ACTION_HASH: &'static str = "...";
}
}

Goal, Result, and Feedback all implement Serialize + Deserialize (required by the RosMessage bound on RosAction associated types).

Package name remapping

nros dev codegen rust --rename old_pkg=new_crate_name renames:

• Output directory
• [package] name in Cargo.toml
• Dependency names and paths in Cargo.toml
• use old_pkg:: references in Rust source

Used by nano-ros to generate nros-rcl-interfaces from rcl_interfaces.

RMW Backends — Host-Language Policy

This page documents which language each RMW backend is implemented in, and the rule that decides it. The matrix was originally frozen 2026-05-07; the L tier (+ L.8, landed 2026-05-12) collapsed the public RMW surface so every backend now reaches the runtime via the same nros_rmw_vtable_t bridge. The underlying implementations still ship in whichever language their upstream library prefers, but consumers never see that — they only see the C vtable.

The rule (post-115.L)

Every backend installs itself via the nros_rmw_vtable_t bridge. The underlying library’s host language is an implementation detail of the per-backend -cffi shim; it does not appear on the consumer surface.

The original rule (a backend’s host language matches its underlying library’s native language unless overridden) is still how we pick the inside of each -cffi shim — but the shim itself is uniform: a small Rust or C++ TU that fills in a vtable and calls nros_rmw_cffi_register_named(name, &vtable) once at startup.

Hierarchy

nros-core (Rust) ──→ Rmw trait (internal; bridged by RustBackendAdapter<R>)
                        └──→ nros-rmw-cffi   (C ABI bridge, registry)
                                ↓ nros_rmw_vtable_t  (~17 fn ptrs)
                                ├──→ nros-rmw-zenoh    (wraps Rust nros-rmw-zenoh)
                                ├──→ nros-rmw-xrce-cffi     (links C nros-rmw-xrce)
                                ├──→ nros-rmw-cyclonedds    (C++ direct, no Rust)
                                └──→ nros-rmw-uorb      (C++ direct, no Rust)

The shims are the canonical consumer surface. Public Cargo features on nros / nros-c / nros-cpp (rmw-{zenoh,xrce}-cffi, cffi-{zenoh-cffi,xrce-c}) all route through the same nros_rmw_vtable_t runtime. The pre-L.7 direct-Rust-trait features (rmw-zenoh, rmw-xrce, rmw-uorb) are gone.

Phase 169 (2026-05-19) — dust-dds retired. The nros-rmw-dds Rust shim and the dust-dds upstream Rust DDS implementation have been removed (Phase 169.4). Cyclone DDS is the sole DDS backend; the nros-rmw-cyclonedds shim registers under its canonical name "cyclonedds" only. The previous "dds" generic slot is not aliased — callers always select Cyclone by its specific name (NROS_RMW=cyclonedds, target_link_libraries(... NanoRos::Rmw::cyclonedds), etc.).

Any language with stable C-ABI interop (C, C++, Zig, Rust, Go-via-cgo, Python-via-ctypes…) can implement a backend by filling in the vtable and calling nros_rmw_cffi_register_named(name, &vtable) once at startup.

Decision matrix (post-115.L, updated by Phase 171)

Dust-DDS was retired in Phase 169 (2026-05-19) after repeated bring-up failures on embedded targets. It is intentionally absent from the active backend table; Cyclone DDS now fills the DDS slot.

Rust-backend cffi shape

For backends whose upstream library is Rust (zenoh-pico) the cffi shim ships as a tiny crate that calls RustBackendAdapter::<UnderlyingRmw>::register(). The adapter monomorphizes a static nros_rmw_vtable_t over the Rust Rmw trait impl and installs it into the C registry. Consumer code never sees the trait surface; it only sees the vtable.

The legacy direct-Rust-trait crates (nros-rmw-zenoh, nros-rmw-xrce) stay in the workspace as internal-only implementation libs of these shims. They have no public Cargo feature reaching them after.

C-/C++-backend cffi shape

For backends whose upstream library is C/C++ (Cyclone DDS, uORB, XRCE) the cffi shim is a standalone CMake project that builds a static C/C++ library and registers a nros_rmw_vtable_t at startup via nros_rmw_cffi_register_named. No RustBackendAdapter is involved. The Rust runtime sees these via the same registry; the NROS_RMW=<name> CMake selector flips a build-time macro that ensures the register call is wired into nros::init.

Cyclone DDS: runtime type introspection (Phase 212.K.7)

The Cyclone DDS shim does not require per-type backend code on the generated msg crate. Phase 212.K.7 inverted the original design where each msg crate carried an optional cyclonedds Cargo feature plus a Cyclone-specific descriptor sidecar.

In the runtime-introspection design every generated msg crate is purely the wire-format data type (#[derive]d struct + a tiny impl nros_serdes::Message exposing const TYPE_NAME + const FIELDS). On the first typed create_publisher<M> / create_subscription<M> for a given M, the nros-rmw-cyclonedds shim walks the static field schema, builds a Cyclone ddsi_sertype via Cyclone’s dynamic-type C API, and caches the pointer in a bounded heapless::FnvIndexMap<u64, NonNull<ddsi_sertype>, MAX_TYPES> guarded by a platform-selected mutex. Subsequent uses of the same M are hash-map hits.

End state: no <msg-pkg>/cyclonedds Cargo feature anywhere, no per-msg-pkg backend code, no codegen branching on the active RMW. The shape matches upstream rclcpp’s introspection typesupport + rclrs’s plain <pkg> = "*" consumer manifest.

Sizing knob: NROS_CYCLONEDDS_MAX_TYPES (default 32), wired through the existing nros-sizes build probe (same pattern as EXECUTOR_OPAQUE_U64S). See section 212.K.7 of docs/roadmap/phase-212-ux-cargo-native-and-file-consolidation.md for the work-item ledger.

When to revisit

This matrix is a snapshot. Update it when any backend’s situation changes:

• A new backend lands → add a row + shim-crate + verdict.
• An existing backend’s underlying library changes language (e.g. zenoh-pico ships a Rust port upstream) → swap the shim shape (Rust adapter vs. C/C++ direct) but the vtable bridge stays.

The rule stays. Only per-backend verdicts and shim shapes move.

Registry + naming

nros-rmw-cffi holds a fixed-size named registry of backend vtables. Each backend registers under a canonical name at process startup:

Naming policy

• Lowercase ASCII identifying the protocol / wire format. Not the transport variant — "xrce" covers both XRCE-UDP and XRCE-serial; the transport is selected via the locator (udp/... vs serial:/dev/...).
• Stable across releases. Renaming a registered name is a breaking change for bridge code that selects backends by string.
• No "default" for new backends. It was the implicit name of the legacy single-arg nros_rmw_cffi_register_named, which phase-482 W6 deleted; name the protocol instead.

Capacity

Registry size: NROS_RMW_MAX_BACKENDS build-time env var consumed by nros-rmw-cffi/build.rs. Default 8. Range [1, 64]. Set lower on Cortex-M0+ (where each slot’s ~40 B costs); set higher for bridge nodes with 4+ backends. Hitting the cap = subsequent nros_rmw_cffi_register_named returns NROS_RMW_RET_ERROR.

Default-backend convention

Executor::open and any create_node call without an explicit .rmw(name) selector use the first-registered backend — the nano-ros equivalent of ROS 2’s RMW_IMPLEMENTATION. Single- backend binaries with one auto-registering backend Just Work without user code mentioning the backend’s name.

The user-facing knob is a declared, language-agnostic, per-deploy value (system.toml [system].rmw / [image.<id>].rmw, or a CLI/build flag) that the toolchain lowers to each language’s native link mechanism. The Cargo feature / shim dep and the CMake cache var below are those lowering targets — what the build uses, not how a user picks a backend (see RFC-0031):

• Cargo (Rust): the declared RMW lowers to the nros rmw-<x> feature plus the matching nros-rmw-<x> shim dep in the consumer’s [dependencies]. Linking the shim crate is what registers the backend.
• CMake: cmake -DNROS_RMW=zenoh ... (C/C++ users). The nano_ros_link_rmw(... RMW zenoh) helper auto-generates the per-target nros_app_register_backends() strong stub.

No runtime env-var override; selection is fixed at link time (RTOS-friendly, matches our static-link world).

Symbol-survival mechanism

Backend register symbols must survive linker dead-strip. Four mechanisms, layered:

1. linkme distributed-slice (/ 128.H.2) — each backend contributes an RMW_INIT_ENTRIES entry through the nros_rmw_register_backend! macro. rclc_support_init / Executor::open walks the slice and calls each entry. Canonical on the hosted targets whose linker sections linkme recognises — of which nano-ros builds only the linux host board. Macro expands to a no-op on RTOS targets where linkme can’t recognise the section (NuttX, Zephyr, FreeRTOS, bare-metal).
2. Rust ctor (legacy fallback): #[unsafe(link_section = ".init_array")] #[used] static AUTO_REGISTER_CTOR. #[used] keeps rustc from dead-stripping.
3. C ctor (legacy fallback): __attribute__((constructor)) static void nros_rmw_<name>_register_ctor. Same survival via .init_array walk by libc startup.
4. CMake strong stub (landed): the nano_ros_link_rmw(<target> RMW <name>) helper at cmake/NanoRosLink.cmake:62-117 emits an auto-generated TU per target that defines a strong nros_app_register_backends() calling every linked RMW’s nros_rmw_<name>_register(). The weak default in libnros_c_weak_stubs.a is overridden. This is the canonical path on every RTOS where linkme can’t survive.
5. Explicit user call (Rust no_std bridges): nros_rmw_<name>::register() from main() — drags the rlib’s CGU into the binary so the linkme entry is reachable. See the gateway fixture packages/testing/nros-tests/bins/bridge-zenoh-to-xrce-fwd/.

Bare-metal + RTOS targets that don’t run .init_array rely on (4). Pure-Rust no_std binaries with multiple backends rely on (5). POSIX builds get (1) + (2) + (3) for free.

Ctor ordering

POSIX .init_array runs ctors in link order, not in any user-controlled sequence. When multiple backends auto-register in one binary, the first to fire owns the default slot — the one selected by Executor::open() / nros::init() with no .rmw(name) argument. The order is reproducible per link graph but not portable across linkers (lld vs. mold vs. gold) or build configs (LTO can reorder via --print-icf-sections collapse) and must not be relied on for correctness.

Disambiguation is the user’s job in multi-backend binaries. Use the named entry points:

• Rust: Executor::open_with_rmw("zenoh", &cfg) for the primary session; node_builder("name").rmw("cyclonedds").build() for additional Nodes.
• C: rclc_node_init_ex with rcl_node_options_t.rmw_name set.
• C++: rclcpp::Executor::open_with_rmw(...) and rclcpp::NodeBuilder::rmw(...) mirror the Rust API (Phase 104.C.9).

The packages/testing/nros-tests/bins/bridge-zenoh-to-cyclonedds-fwd/ gateway fixture (run by the mixed-RMW bridge e2e) shows the pattern end-to-end: both Zenoh and Cyclone DDS backend ctors fire at lib-load (so the registry has both "zenoh" and "cyclonedds" slots populated), and open_with_rmw("zenoh", ...) plus node_builder("egress").rmw("cyclonedds") pin each session to its intended backend without depending on link-order luck.

Single-backend builds keep the legacy ergonomics — only one ctor fires, the default-slot convention picks it up, and no user-visible name is ever required. The cost of naming is paid only when multiple backends coexist.

Real-time budget per backend

The poll loop’s worst-case execution time is dominated by the backend’s transport drain. Bridge users summing bridge_wcet = Σ poll_i + Σ dispatch_j need each backend’s contribution; this table captures the current best-effort estimates from per-backend microbenchmarks (packages/testing/nros-bench/wcet-cycles-qemu/, packages/testing/nros-bench/wake-latency-cortex-m3/) + heap-usage stats from cargo build --release symbol dumps.

Bridge users: sum the poll_wcet_us for every backend the bridge process opens, then add per-callback dispatch budget (typically <10 µs for the executor’s arena dispatch + the user callback’s own work). A bridge_picas_priority regression test (blocked on the PiCAS dispatcher) will eventually pin a bar to this table.

Per-backend README.md files live at packages/{zpico/nros-rmw-zenoh,dds/nros-rmw-cyclonedds,xrce/nros-rmw-xrce-cffi}/README.md (when present); reach out to the backend’s maintainer for fresh microbench numbers on a different target class than the ones above.

See also

• roadmap doc — Appendix D carries LOC sizing, port shapes, and risk notes.
• Custom Transport porting guide — how the transport vtable composes with the RMW vtable.
• packages/rmw/cyclonedds/nros-rmw-cyclonedds/ — reference layout for the C++ vtable consumer.
• packages/rmw/uorb/nros-rmw-uorb/ — reference layout for the C++ vtable consumer with PX4 SDK integration.

zpico-sys Build Architecture

zpico-sys is nano-ros’s Rust *-sys crate for zenoh-pico. Its build script is one line; the work is in packages/rmw/zenoh/nros-zpico-build (runner::run), a library so its parsing and source selection can be unit-tested outside Cargo’s build-script harness. It compiles the vendored zenoh-pico C with a single cc-rs invocation, driven by data rather than by per-RTOS Rust code.

Zephyr is the exception: its west/cmake lane compiles the same vendored tree through zephyr/cmake/nros_rmw_zenoh.cmake, and the cargo build of zpico-sys for a Zephyr image compiles none of it (see compiled_by below).

Where the data lives

Three files, each owned by the thing it describes. None of them is the zenoh_platforms.toml older docs mention; that central manifest was split up by phase-290 and phase-349 W2b.

A platform names no backend (RFC-0072 §9): what a platform needs from any vendored C build stays in its own file, and what zenoh-pico needs on that platform is a row in zenoh’s descriptor. A build joins the two.

zenoh-sources.txt

The one source list, read by both the cargo lane and the cmake lane. Records are group <name> <condition>, dir <group> <tree> <path> (every .c under <path>, recursively) and src <group> <tree> <path> (exactly one file). The core is nine dir records under zenoh-pico/src/, condition always; the two other groups are Zephyr-only. The file’s header explains the format and why it lists directories rather than files. Gate: just check zenoh-source-manifest — every .c under zenoh-pico/src/ must be covered by a record or sit under a system/<platform>/ tree some port row selects, so a new upstream DIRECTORY fails the gate.

The platform half — nros-platform.toml

Seven files: config/{bare-metal,generic}/ and packages/platform/nros-platform-{freertos,nuttx,posix,threadx,zephyr}/. Most of each file is the RFC-0049 knob ladder ([capabilities], [knobs.*]); the C-build keys are:

Profiles listed today: cortex-m3, cortex-m4f, riscv32imc (bare-metal); cortex-m3, cortex-m7, cortex-r5, cortex-r52 (FreeRTOS); cortex-a7 (NuttX); riscv64gc (ThreadX). posix, zephyr and generic list none. Gate: just check arch-profile-resolution checks every supported triple against these lists.

The search path is $NROS_PLATFORMS_DIR (colon-separated, searched first), then packages/platform/, then config/. An empty tree is fatal, not a fallback to builtins.

[arch.<name>]

The backend half — [port.<platform>]

Rows exist for freertos, nuttx, posix, threadx, zephyr, bare-metal and generic; the key is a platform name or alias. A row is deny_unknown_fields: one that names arch_profiles, compile or required_env is a platform fact in the wrong file and fails to load. The schema is PortRow in packages/tooling/nros-platform-config/src/manifest.rs.

The system_libs, exclude and link keys that older docs list were parsed and read by nothing. They were removed (issue 2031), so a row naming one is refused. pthread on POSIX is emitted by the runner itself.

How a build resolves its platform

1. Pick the name. From zpico-sys’s Cargo feature: threadx, nuttx, freertos, zephyr, bare-metal. With none of those, a hosted target (Linux, the BSDs, Android) is posix; macOS must pass the feature.
2. Join. PlatformsTree::as_platform_manifest builds one PlatformEntry per platform file: its port row (or an empty one) plus the file’s inherits, arch_profiles, compile and required_env. A row that names no platform, or two rows that name the same one, is an error.
3. Merge the inherits chain, parent first (PlatformManifest::for_platform):
   • lists append, parent’s entries first;
   • maps (defines_kv, defines_env) take the child’s value per key;
   • mbedtls, pic, compiled_by and each compile field take the child’s value if set, else the parent’s, so an unset child never downgrades a parent;
   • arch_profiles is replaced, not appended, when the child sets it.
4. Check every platform, not just the selected one: each must resolve, and each port’s include directories must exist. A bad row anywhere fails every zenoh build, which is what keeps the ports nobody is building honest.

Shared shapes

when matcher

Every present field must hold; an empty matcher is always true.

ExtraSource

extra_sources = [
  { path = "{src}/system/freertos/system.c" },
  { path = "{src}/system/freertos/lwip/network.c",
    when = { capability = { ip_stack = true } } },
  { path = "{nros}/c/platform/threadx/log_uart.c",
    if_env = "NROS_ZPICO_LOG_TO_UART",
    with_define = ["ZENOH_LOG_PRINT", "zpico_log_print"] },
]

when and if_env both gate the file. with_define is [name] or [name, value], defined whenever the file is compiled.

RequiredEnv

required_env = [
  { name = "THREADX_CONFIG_DIR",
    help = "Directory containing tx_user.h. Example: packages/boards/nros-board-threadx-linux/config" },
]

A missing variable panics with help. With validate_subdir, a present variable whose value lacks that subdirectory panics naming the path. An entry with a when is required only where the matcher holds, so a FreeRTOS board with no network stack does not need an lwIP SDK.

Interpolation tokens

Paths in extra_sources, include_paths and include_paths_conditional pass through manifest::interpolate:

The compile

build_zenoh_pico_unified in nros-zpico-build/src/runner.rs is the only place that invokes cc-rs for zenoh-pico, and it is skipped when the row says compiled_by = "platform". In order:

1. Validate required_env (honouring when).
2. Generate the version header into {out}/zenoh-pico-version/.
3. Apply the first [arch.*] profile from arch_profiles that admits the target.
4. Add the zenoh-sources.txt groups this lane selects, then the row’s extra_sources (when, if_env, with_define).
5. Add include paths: the generated config, zenoh-pico’s own, the version header and the platform C ABI headers, then the row’s paths (unconditional, then conditional).
6. Add defines: defines, defines_conditional, defines_kv, defines_env.
7. Handle mbedTLS per mbedtls, when link-tls is on.
8. Apply shim slot counts and compile settings, then the pic override.
9. Compile libzenohpico.a and register rerun_if_env_changed.

The zpico shim (c/zpico/zpico.c) is a separate compile in build_c_shim, and shares the generated zenoh config header with the library (issue 0135).

Adding a platform

1. The platform file. Create nros-platform.toml beside the platform crate (or in a directory on $NROS_PLATFORMS_DIR): names, and the C-build keys if the platform cross-compiles. Reuse an existing [arch.*] profile by naming it in arch_profiles, or define one.
2. The port row. Add [port.<name>] to nros-rmw-zenoh/nros-rmw.toml: the ZENOH_* platform define, the system/<x>/ sources as extra_sources, and include paths. If the platform’s own build compiles zenoh-pico, set compiled_by = "platform".
3. The feature. Add the zpico-sys Cargo feature and its arm in runner.rs’s platform_name derivation.
4. Check. cargo build -p zpico-sys --features <name> --target <triple>, then just check zenoh-source-manifest and just check arch-profile-resolution.

A new CPU family is one [arch.*] block in each platform file that admits it, plus its name in their arch_profiles. A multi-arch platform lists every profile and lets the first match pick: bare-metal lists cortex-m3, cortex-m4f and riscv32imc, so one platform serves mps2-an385 and the ESP32-C3.

TX tuning benchmarks

The user page TX Throughput & Latency Tuning states the per-platform tx defaults. The measurements behind them: packages/testing/nros-bench/stress-zenoh-zephyr/ (streaming) and packages/testing/nros-tests/tests/w1_zephyr_tx_throughput_measure.rs (timer tiers). Background: docs/roadmap/archived/phase-279-*.md, docs/roadmap/archived/phase-282-*.md and issue #145. The Zephyr default relies on the zenoh-pico fork fix from issue 0213 (declarations bypass the batch). See also book-internals/src/platform/implementation-notes.md.

Related

• docs/roadmap/archived/phase-136-zpico-sys-unified-build.md — the unified cc-rs build.
• docs/roadmap/archived/phase-349-rtos-integration-shells.md — W2b, the platform/backend split.
• RFC-0049 (docs/design/0049-hierarchical-platform-board-config.md) — the platform and board files and their ladder.
• book/src/concepts/platform-model.md — boards vs platforms.
• book-internals/src/rmw/backend-host-language-policy.md — RMW host-language policy.

ROS 2 rmw_zenoh Interoperability

This page describes how nros’s zenoh backend speaks the rmw_zenoh_cpp wire protocol: key expressions, liveliness tokens, QoS strings, CDR framing and the per-message attachment. For the user steps (starting the router, running a talker against ros2 topic echo, matching domain ID, QoS and edition), see the book’s ROS 2 Interop page; this page does not repeat them.

Status: WORKING

nros <-> ROS 2 rmw_zenoh communication is fully operational as of January 2025.

Architecture

┌─────────────────────────┐          ┌─────────────────────────┐
│   ROS 2 Node            │          │   nros Node             │
│   (rmw_zenoh_cpp)       │<-------->│   (zenoh-pico)          │
│                         │  Zenoh   │                         │
│   ros2 topic echo       │  Router  │   native-rs-talker      │
│   /chatter              │ (zenohd) │                         │
└─────────────────────────┘          └─────────────────────────┘

Both nodes connect to the same Zenoh router (zenohd). zenoh’s router-free peer mode is not compiled into the nano-ros shim — NROS_SESSION_MODE=peer is refused at session open (issue 0682).

Protocol Requirements

1. Data Topic Key Expression Format

For ROS 2 Humble, the data key expression format is:

<domain_id>/<topic_name>/<type_name>/<type_hash>

Important: For Humble, use TypeHashNotSupported as the type hash (no RIHS01_ prefix).

Example:

0/chatter/std_msgs::msg::dds_::String_/TypeHashNotSupported

Note: Newer ROS 2 versions (Iron+) use RIHS01_<hash> format with actual type hashes.

Implementation: TopicInfo::to_key() in the transport traits module

2. Liveliness Token Format

rmw_zenoh uses Zenoh liveliness tokens for discovery. Without these, ros2 topic list won’t show nros topics.

Node Liveliness Token:

@ros2_lv/<domain>/<zid>/0/0/NN/%/%/<node_name>

Publisher Liveliness Token:

@ros2_lv/<domain>/<zid>/0/11/MP/%/%/<node_name>/<topic>/<type>/RIHS01_<hash>/<qos>

Subscriber Liveliness Token:

@ros2_lv/<domain>/<zid>/0/11/MS/%/%/<node_name>/<topic>/<type>/RIHS01_<hash>/<qos>

Key differences from data keyexprs:

• Liveliness tokens do use RIHS01_ prefix
• Topic names use % instead of / (e.g., %chatter not /chatter)
• ZenohId must be in LSB-first hex format

Implementation: Ros2Liveliness in the transport/zenoh module

3. QoS String Format

The QoS portion of liveliness tokens uses this format:

reliability:durability:history,depth:deadline_sec,deadline_nsec:lifespan_sec,lifespan_nsec:liveliness,liveliness_sec,liveliness_nsec

Values:

• Reliability: 1=RELIABLE, 2=BEST_EFFORT
• Durability: 1=TRANSIENT_LOCAL, 2=VOLATILE
• History: 1=KEEP_LAST

For BEST_EFFORT/VOLATILE with KEEP_LAST depth 1:

2:2:1,1:,:,:,,

Important: Empty values default to RELIABLE which causes QoS mismatch. Always specify explicit values.

3b. TRANSIENT_LOCAL — the @adv cache keyexpr

phase-455 W5 / issue 1341. This section is MEASURED, from a RUST_LOG=zenoh=debug router watching a stock rmw_zenoh_cpp 0.1.9 TRANSIENT_LOCAL pair on 2026-09-13, not read out of upstream source.

rmw_zenoh_cpp 0.1.9 builds its endpoints on zenoh’s ze_advanced_publisher / ze_advanced_subscriber (confirmed with nm -D --undefined-only librmw_zenoh_cpp.so), so transient-local durability is served by QUERY-ON-MATCH rather than by anything in the wire format above:

Declare queryable  <domain>/<topic>/<type>/<hash>/@adv/pub/<zid>/<eid>/_
Declare subscriber <domain>/<topic>/<type>/<hash>
Declare subscriber <domain>/<topic>/<type>/<hash>/@adv/pub/**
Route query    for <domain>/<topic>/<type>/<hash>/@adv/**
Route query    for <domain>/<topic>/<type>/<hash>/@adv/pub/<zid>/<eid>/_

• The PUBLISHER’s cache is a queryable at <topic keyexpr>/@adv/pub/<zid>/<eid>/_. @adv is zenoh-ext’s own prefix — the key format ${remaining:**}/@adv/${entity:*}/${zid:*}/${eid:*}/${meta:**} is a literal string in libzenohc.so; pub is the entity and the trailing _ is the empty metadata chunk.
• The SUBSCRIBER issues a GLOBAL history query at <topic keyexpr>/@adv/** when it is created. That intersects any publisher’s cache key, and it is what serves a late joiner.
• The second query is late-joiner DETECTION: the subscriber also declares a liveliness subscriber on <topic keyexpr>/@adv/pub/** and queries each publisher whose @adv liveliness token appears after it.
• The reply carries the TOPIC keyexpr, not the queryable’s, and the same attachment a live publication would have. The advanced subscriber accepts a reply keyexpr differing from the query’s, which is what makes that legal.

nano-ros serves the publisher half of this (shim/publisher.rs::transient_local) and declares no @adv liveliness token, so it answers the global query and not the per-publisher one. The subscriber half — querying a stock transient-local publisher on match — is not implemented, and shim/qos.rs::admit refuses TRANSIENT_LOCAL on a subscription rather than pretending.

4. CDR Message Format

Messages use CDR (Common Data Representation) little-endian encoding:

[0x00, 0x01, 0x00, 0x00]  // CDR encapsulation header (LE)
[... CDR payload ...]     // Message data

Implementation: the serdes crate

5. RMW Attachment Format

rmw_zenoh requires metadata attached to each published message. For Humble, the serialization format is:

Total: 33 bytes

The serialization uses Zenoh’s ze_serializer to ensure compatibility with zenoh-cpp’s deserializer.

Implementation: serialize_rmw_attachment() in zenoh-pico/src/serializer.rs

6. ZenohId Format

The ZenohId in liveliness tokens must be formatted in LSB-first (little-endian) hex format.

#![allow(unused)]
fn main() {
// Correct: LSB-first
fn to_hex_string(&self) -> String {
    let mut hex = String::new();
    for byte in self.bytes.iter() {  // LSB first
        write!(&mut hex, "{:02x}", byte).ok();
    }
    hex
}
}

Common Issues and Solutions

These are failures of the protocol encoding itself, seen while developing the backend.

Issue: Discovery works but no messages received

Symptom: ros2 topic list shows the topic, ros2 topic info shows correct QoS, but ros2 topic echo receives nothing.

Cause: Data keyexpr format mismatch.

Solution: For Humble, don’t use RIHS01_ prefix in data keyexprs:

• Wrong: 0/chatter/std_msgs::msg::dds_::String_/RIHS01_TypeHashNotSupported
• Correct: 0/chatter/std_msgs::msg::dds_::String_/TypeHashNotSupported

Issue: Topic not visible in ros2 topic list

Symptom: nros publishes but ROS 2 doesn’t see the topic.

Cause: Liveliness token format incorrect.

Solutions:

1. Check ZenohId is LSB-first hex format
2. Ensure topic name uses % prefix (e.g., %chatter)
3. Verify QoS string has explicit values, not defaults

User-level failures (QoS mismatch, rmw_zenoh_cpp left in peer mode, domain ID or edition mismatch) are covered in the book’s ROS 2 Interop → Troubleshooting.

Implementation Status

Version Compatibility

The edition is chosen per image (ros_edition in system.toml); a Humble-edition image and a Jazzy bus do not discover each other, because the key expressions differ.

References

• rmw_zenoh design docs
• Zenoh serialization RFC
• zenoh-pico

RMW API — every upstream symbol, side by side

What ROS 2’s rmw asks an implementation for, what nano-ros provides, and the reason wherever the two differ. Derived from three sources — the ROS 2 signature extract, the nano-ros ABI headers, and the authored reason map docs/reference/rmw-api-map.toml that just check rmw-api-parity also reads — so a slot changing shape moves this page in the same commit or fails the gate.

For the prose rationale behind the big divergences, see RMW API: Differences from upstream rmw.h.

How to read a row

The signature says which surface it is on. nano-ros answers an upstream symbol in one of two ways, and C syntax already distinguishes them:

rmw_ret_t (*count_publishers)(...)   // vtable slot — a function pointer,
                                     // may differ per backend
rmw_ret_t rmw_compare_gids_equal(...) // global — a plain exported function,
                                     // defined once for the image

Slot names drop the rmw_ prefix because that is genuinely the name in nros_rmw_vtable_t; showing rmw_count_publishers on the right would flatter the comparison.

Each side is spelled the way its own headers spell it. Upstream writes const rmw_publisher_t * publisher, we write const rmw_publisher_t *publisher, and the two columns keep their own convention rather than being normalised to a third that neither project uses. Parameter NAMES are shown on both sides; the comparison behind the highlighting is on TYPES only, because a renamed argument is not an ABI difference.

The chip says what we DID with the symbol, which is a different question from what the signature shows:

same and re-shaped are derived from the signatures, so the map cannot assert a match the types deny. The other three are authored, because no signature can say whether an absent symbol was decomposed or dropped — and not-implemented must name an issue, so silence cannot turn a gap into a decision.

Marks show the difference. Red — upstream takes it, we do not. Green — we take it, upstream does not. Yellow — the name differs from the mechanical one (upstream minus rmw_). A row with no marks is identical on both sides and carries no reason, because there is nothing to explain.

Argument names appear on the right only. The ROS 2 side is read from a signature extract that drops parameter names on purpose — a renamed argument is not an ABI difference, and reporting one trains people to skim. Ours are kept because they are the only place a reader learns what an argument means: size_t twice in a row is not a signature anyone can act on. The comparison itself runs on types, so a name never colours a row.

An empty right-hand cell reads rejected when the symbol is deliberately absent, or answered elsewhere when the capability ships outside the RMW seam — in the executor, in codegen, or inside a backend. Both carry the reason.

A dimmed right-hand cell is an inert slot: declared in the vtable and read by nothing. A reserved shape, not a working capability (issue 0800) — and some of them ARE filled by a backend, which is an implementation no caller reaches rather than a capability (phase-428 W8).

What is being compared

Not rmw.h against rmw_vtable.h — that comparison is wrong twice. Upstream declares 177 RMW_PUBLIC functions, but most are utilities rmw itself defines; an implementation links those, so comparing against 177 manufactures ~90 phantom gaps. And our vtable is only the backend seam: plenty of what upstream calls rmw lives one layer up in the executor, one layer down in a backend, or in codegen.

So the contract is empirical — the 88 rmw_* symbols that librmw_fastrtps_cpp.so and librmw_zenoh_cpp.so both define. Two independent implementations with identical symbol sets is a better definition of “what an rmw must provide” than any reading of the headers.

Read a row for what we did, a column for where it lives. Only not implemented should shrink over time; not supported is the one line that is a decision rather than a state, so it is expected to stay.

Vocabulary — why we say session where upstream says context

An rmw_context_t is two things in one struct: the options the process was initialised with (rmw_init_options_t — domain id, enclave, security, discovery) and an opaque impl the backend allocates to hold its running state. Upstream can fuse them because both are per-process, heap-allocated at rmw_init and freed at rmw_shutdown.

Ours are not one thing. The options half is a build-time POD — that is the recorded reason rmw_init_options_{init,copy,fini} are declined, since “copy” is = and “fini” is nothing — and it lives on nros::Context. The running half is a Session, one per backend, created and destroyed through vtable slots, and an image may hold several at once: the bridge entries run a zenoh ingress and a Cyclone or XRCE egress in one process. There is no per-process singleton to name.

So the rename is unavailable in both directions. Context is already taken by the half that is not the backend state, and calling the backend state context would promise a one-per-process object that a bridge image visibly does not have. The word session is also the one the transports themselves use (a zenoh session, an XRCE-DDS session), which is what a reader of create_session is actually looking at.

Read rmw_context_t on this page as our Context plus one or more Sessions, and the rmw_init/rmw_shutdown rows follow from that.

Every contract symbol

rmw_ret_t
rmw_borrow_loaned_message(
  const rmw_publisher_t * publisher,
  const rosidl_message_type_support_t * type_support,
  void * * ros_message
)

rmw_ret_t
(*borrow_loaned_message)(
  const rmw_publisher_t *publisher,
  size_t requested_len,
  rmw_mut_byte_span_t *out_slot,
  rmw_loan_token_t **out_token
)

rmw_ret_t
rmw_client_request_publisher_get_actual_qos(
  const rmw_client_t * client,
  rmw_qos_profile_t * qos
)

rmw_ret_t
(*client_request_publisher_get_actual_qos)(
  const rmw_client_t *client,
  rmw_qos_profile_t *qos
)

rmw_ret_t
rmw_client_response_subscription_get_actual_qos(
  const rmw_client_t * client,
  rmw_qos_profile_t * qos
)

rmw_ret_t
(*client_response_subscription_get_actual_qos)(
  const rmw_client_t *client,
  rmw_qos_profile_t *qos
)

rmw_ret_t
rmw_client_set_on_new_response_callback(
  rmw_client_t * client,
  rmw_event_callback_t callback,
  const void * user_data
)

rmw_ret_t
(*client_set_on_new_response_callback)(
  rmw_client_t *client,
  rmw_event_callback_t callback,
  const void *user_data
)

rmw_ret_t
rmw_compare_gids_equal(
  const rmw_gid_t * gid1,
  const rmw_gid_t * gid2,
  bool * result
)

rmw_ret_t
rmw_compare_gids_equal(
  const rmw_gid_t *gid1,
  const rmw_gid_t *gid2,
  bool *result
)

rmw_ret_t
rmw_context_fini(rmw_context_t * context)

rmw_ret_t
(*destroy_session)(rmw_session_t *session)

rmw_ret_t
rmw_count_publishers(
  const rmw_node_t * node,
  const char * topic_name,
  size_t * count
)

rmw_ret_t
(*count_publishers)(
  const rmw_session_t *session,
  const char *topic_name,
  size_t *count
)

rmw_ret_t
rmw_count_subscribers(
  const rmw_node_t * node,
  const char * topic_name,
  size_t * count
)

rmw_ret_t
(*count_subscribers)(
  const rmw_session_t *session,
  const char *topic_name,
  size_t *count
)

rmw_client_t *
rmw_create_client(
  const rmw_node_t * node,
  const rosidl_service_type_support_t * type_support,
  const char * service_name,
  const rmw_qos_profile_t * qos_policies
)

rmw_ret_t
(*create_client)(
  const rmw_node_t *node,
  const rmw_service_type_support_t *type_support,
  const char *service_name,
  uint32_t domain_id,
  const rmw_qos_profile_t *qos,
  rmw_client_t *out
)

rmw_guard_condition_t *
rmw_create_guard_condition(rmw_context_t * context)

rmw_node_t *
rmw_create_node(
  rmw_context_t * context,
  const char * name,
  const char * namespace_
)

rmw_ret_t
(*create_node)(
  rmw_session_t *session,
  const char *name,
  const char *namespace_,
  rmw_node_t *out
)

rmw_publisher_t *
rmw_create_publisher(
  const rmw_node_t * node,
  const rosidl_message_type_support_t * type_support,
  const char * topic_name,
  const rmw_qos_profile_t * qos_profile,
  const rmw_publisher_options_t * publisher_options
)

rmw_ret_t
(*create_publisher)(
  const rmw_node_t *node,
  const rmw_message_type_support_t *type_support,
  const char *topic_name,
  uint32_t domain_id,
  const rmw_qos_profile_t *qos,
  const rmw_publisher_options_t *options,
  rmw_publisher_t *out
)

rmw_service_t *
rmw_create_service(
  const rmw_node_t * node,
  const rosidl_service_type_support_t * type_support,
  const char * service_name,
  const rmw_qos_profile_t * qos_profile
)

rmw_ret_t
(*create_service)(
  const rmw_node_t *node,
  const rmw_service_type_support_t *type_support,
  const char *service_name,
  uint32_t domain_id,
  const rmw_qos_profile_t *qos,
  rmw_service_t *out
)

rmw_subscription_t *
rmw_create_subscription(
  const rmw_node_t * node,
  const rosidl_message_type_support_t * type_support,
  const char * topic_name,
  const rmw_qos_profile_t * qos_policies,
  const rmw_subscription_options_t * subscription_options
)

rmw_ret_t
(*create_subscription)(
  const rmw_node_t *node,
  const rmw_message_type_support_t *type_support,
  const char *topic_name,
  uint32_t domain_id,
  const rmw_qos_profile_t *qos,
  const rmw_subscription_options_t *options,
  rmw_subscription_t *out
)

rmw_wait_set_t *
rmw_create_wait_set(
  rmw_context_t * context,
  size_t max_conditions
)

rmw_ret_t
rmw_deserialize(
  const rmw_serialized_message_t * serialized_message,
  const rosidl_message_type_support_t * type_support,
  void * ros_message
)

rmw_ret_t
rmw_destroy_client(
  rmw_node_t * node,
  rmw_client_t * client
)

rmw_ret_t
(*destroy_client)(rmw_client_t *client)

rmw_ret_t
rmw_destroy_guard_condition(rmw_guard_condition_t * guard_condition)

rmw_ret_t
rmw_destroy_node(rmw_node_t * node)

rmw_ret_t
(*destroy_node)(rmw_node_t *node)

rmw_ret_t
rmw_destroy_publisher(
  rmw_node_t * node,
  rmw_publisher_t * publisher
)

rmw_ret_t
(*destroy_publisher)(rmw_publisher_t *publisher)

rmw_ret_t
rmw_destroy_service(
  rmw_node_t * node,
  rmw_service_t * service
)

rmw_ret_t
(*destroy_service)(rmw_service_t *server)

rmw_ret_t
rmw_destroy_subscription(
  rmw_node_t * node,
  rmw_subscription_t * subscription
)

rmw_ret_t
(*destroy_subscription)(rmw_subscription_t *subscription)

rmw_ret_t
rmw_destroy_wait_set(rmw_wait_set_t * wait_set)

rmw_ret_t
rmw_event_set_callback(
  rmw_event_t * event,
  rmw_event_callback_t callback,
  const void * user_data
)

rmw_ret_t
(*publisher_event_init)(
  const rmw_publisher_t *publisher,
  rmw_event_type_t kind,
  uint32_t deadline_ms,
  rmw_status_event_callback_t cb,
  void *user_context
)

bool
rmw_feature_supported(rmw_feature_t feature)

bool
(*feature_supported)(rmw_feature_t feature)

rmw_ret_t
rmw_fini_publisher_allocation(rmw_publisher_allocation_t * allocation)

rmw_ret_t
rmw_fini_subscription_allocation(rmw_subscription_allocation_t * allocation)

rmw_ret_t
rmw_get_client_names_and_types_by_node(
  const rmw_node_t * node,
  rcutils_allocator_t * allocator,
  const char * node_name,
  const char * node_namespace,
  rmw_names_and_types_t * service_names_and_types
)

rmw_ret_t
(*get_client_names_and_types_by_node)(
  const rmw_session_t *session,
  const char *node_name,
  const char *node_namespace,
  rmw_names_and_types_visitor_t visitor  /* visit(ctx, name, types, types_count) */
)

rmw_ret_t
rmw_get_gid_for_publisher(
  const rmw_publisher_t * publisher,
  rmw_gid_t * gid
)

rmw_ret_t
(*get_gid_for_publisher)(
  const rmw_publisher_t *publisher,
  rmw_gid_t *gid
)

const char *
rmw_get_implementation_identifier(void)

const char *
(*get_implementation_identifier)(void)

rmw_ret_t
rmw_get_node_names(
  const rmw_node_t * node,
  rcutils_string_array_t * node_names,
  rcutils_string_array_t * node_namespaces
)

rmw_ret_t
(*get_node_names)(
  const rmw_session_t *session,
  rmw_node_visitor_t visitor  /* visit(ctx, node_name, node_namespace, enclave) */
)

rmw_ret_t
rmw_get_node_names_with_enclaves(
  const rmw_node_t * node,
  rcutils_string_array_t * node_names,
  rcutils_string_array_t * node_namespaces,
  rcutils_string_array_t * enclaves
)

rmw_ret_t
(*get_node_names)(
  const rmw_session_t *session,
  rmw_node_visitor_t visitor  /* visit(ctx, node_name, node_namespace, enclave) */
)

rmw_ret_t
rmw_get_publisher_names_and_types_by_node(
  const rmw_node_t * node,
  rcutils_allocator_t * allocator,
  const char * node_name,
  const char * node_namespace,
  bool no_demangle,
  rmw_names_and_types_t * topic_names_and_types
)

rmw_ret_t
(*get_publisher_names_and_types_by_node)(
  const rmw_session_t *session,
  const char *node_name,
  const char *node_namespace,
  bool no_demangle,
  rmw_names_and_types_visitor_t visitor  /* visit(ctx, name, types, types_count) */
)

rmw_ret_t
rmw_get_publishers_info_by_topic(
  const rmw_node_t * node,
  rcutils_allocator_t * allocator,
  const char * topic_name,
  bool no_mangle,
  rmw_topic_endpoint_info_array_t * publishers_info
)

rmw_ret_t
(*get_publishers_info_by_topic)(
  const rmw_session_t *session,
  const char *topic_name,
  bool no_mangle,
  rmw_topic_endpoint_info_visitor_t visitor  /* visit(ctx, info) */
)

const char *
rmw_get_serialization_format(void)

const char *
(*get_serialization_format)(void)

rmw_ret_t
rmw_get_serialized_message_size(
  const rosidl_message_type_support_t * type_support,
  const rosidl_runtime_c__Sequence__bound * message_bounds,
  size_t * size
)

rmw_ret_t
rmw_get_service_names_and_types(
  const rmw_node_t * node,
  rcutils_allocator_t * allocator,
  rmw_names_and_types_t * service_names_and_types
)

rmw_ret_t
(*get_service_names_and_types)(
  const rmw_session_t *session,
  rmw_names_and_types_visitor_t visitor  /* visit(ctx, name, types, types_count) */
)

rmw_ret_t
rmw_get_service_names_and_types_by_node(
  const rmw_node_t * node,
  rcutils_allocator_t * allocator,
  const char * node_name,
  const char * node_namespace,
  rmw_names_and_types_t * service_names_and_types
)

rmw_ret_t
(*get_service_names_and_types_by_node)(
  const rmw_session_t *session,
  const char *node_name,
  const char *node_namespace,
  rmw_names_and_types_visitor_t visitor  /* visit(ctx, name, types, types_count) */
)

rmw_ret_t
rmw_get_subscriber_names_and_types_by_node(
  const rmw_node_t * node,
  rcutils_allocator_t * allocator,
  const char * node_name,
  const char * node_namespace,
  bool no_demangle,
  rmw_names_and_types_t * topic_names_and_types
)

rmw_ret_t
(*get_subscriber_names_and_types_by_node)(
  const rmw_session_t *session,
  const char *node_name,
  const char *node_namespace,
  bool no_demangle,
  rmw_names_and_types_visitor_t visitor  /* visit(ctx, name, types, types_count) */
)

rmw_ret_t
rmw_get_subscriptions_info_by_topic(
  const rmw_node_t * node,
  rcutils_allocator_t * allocator,
  const char * topic_name,
  bool no_mangle,
  rmw_topic_endpoint_info_array_t * subscriptions_info
)

rmw_ret_t
(*get_subscriptions_info_by_topic)(
  const rmw_session_t *session,
  const char *topic_name,
  bool no_mangle,
  rmw_topic_endpoint_info_visitor_t visitor  /* visit(ctx, info) */
)

rmw_ret_t
rmw_get_topic_names_and_types(
  const rmw_node_t * node,
  rcutils_allocator_t * allocator,
  bool no_demangle,
  rmw_names_and_types_t * topic_names_and_types
)

rmw_ret_t
(*get_topic_names_and_types)(
  const rmw_session_t *session,
  bool no_demangle,
  rmw_names_and_types_visitor_t visitor  /* visit(ctx, name, types, types_count) */
)

rmw_ret_t
rmw_init(
  const rmw_init_options_t * options,
  rmw_context_t * context
)

rmw_ret_t
(*create_session)(
  const char *locator,
  uint8_t mode,
  uint32_t domain_id,
  const char *node_name,
  const rmw_session_options_t *options,
  rmw_session_t *out
)

rmw_ret_t
rmw_init_options_copy(
  const rmw_init_options_t * src,
  rmw_init_options_t * dst
)

rmw_ret_t
rmw_init_options_fini(rmw_init_options_t * init_options)

rmw_ret_t
rmw_init_options_init(
  rmw_init_options_t * init_options,
  rcutils_allocator_t allocator
)

rmw_ret_t
rmw_init_publisher_allocation(
  const rosidl_message_type_support_t * type_support,
  const rosidl_runtime_c__Sequence__bound * message_bounds,
  rmw_publisher_allocation_t * allocation
)

rmw_ret_t
rmw_init_subscription_allocation(
  const rosidl_message_type_support_t * type_support,
  const rosidl_runtime_c__Sequence__bound * message_bounds,
  rmw_subscription_allocation_t * allocation
)

const rmw_guard_condition_t *
rmw_node_get_graph_guard_condition(const rmw_node_t * node)

rmw_ret_t
(*node_get_graph_guard_condition)(
  rmw_session_t *session,
  rmw_event_callback_t callback,
  const void *user_data
)

rmw_ret_t
rmw_publish(
  const rmw_publisher_t * publisher,
  const void * ros_message,
  rmw_publisher_allocation_t * allocation
)

rmw_ret_t
(*publish)(
  const rmw_publisher_t *publisher,
  rmw_byte_span_t payload
)

rmw_ret_t
rmw_publish_loaned_message(
  const rmw_publisher_t * publisher,
  void * ros_message,
  rmw_publisher_allocation_t * allocation
)

rmw_ret_t
(*publish_loaned_message)(
  const rmw_publisher_t *publisher,
  rmw_loan_token_t *token,
  size_t actual_len
)

rmw_ret_t
rmw_publish_serialized_message(
  const rmw_publisher_t * publisher,
  const rmw_serialized_message_t * serialized_message,
  rmw_publisher_allocation_t * allocation
)

rmw_ret_t
(*publish)(
  const rmw_publisher_t *publisher,
  rmw_byte_span_t payload
)

rmw_ret_t
rmw_publisher_assert_liveliness(const rmw_publisher_t * publisher)

rmw_ret_t
(*publisher_assert_liveliness)(const rmw_publisher_t *publisher)

rmw_ret_t
rmw_publisher_count_matched_subscriptions(
  const rmw_publisher_t * publisher,
  size_t * subscription_count
)

rmw_ret_t
(*publisher_count_matched_subscriptions)(
  const rmw_publisher_t *publisher,
  size_t *subscription_count
)

rmw_ret_t
rmw_publisher_event_init(
  rmw_event_t * rmw_event,
  const rmw_publisher_t * publisher,
  rmw_event_type_t event_type
)

rmw_ret_t
(*publisher_event_init)(
  const rmw_publisher_t *publisher,
  rmw_event_type_t kind,
  uint32_t deadline_ms,
  rmw_status_event_callback_t cb,
  void *user_context
)

rmw_ret_t
rmw_publisher_get_actual_qos(
  const rmw_publisher_t * publisher,
  rmw_qos_profile_t * qos
)

rmw_ret_t
(*publisher_get_actual_qos)(
  const rmw_publisher_t *publisher,
  rmw_qos_profile_t *qos
)

rmw_ret_t
rmw_publisher_get_network_flow_endpoints(
  const rmw_publisher_t * publisher,
  rcutils_allocator_t * allocator,
  rmw_network_flow_endpoint_array_t * network_flow_endpoint_array
)

rmw_ret_t
rmw_publisher_wait_for_all_acked(
  const rmw_publisher_t * publisher,
  rmw_time_t wait_timeout
)

rmw_ret_t
(*publisher_wait_for_all_acked)(
  const rmw_publisher_t *publisher,
  uint32_t timeout_ms
)

rmw_ret_t
rmw_qos_profile_check_compatible(
  const rmw_qos_profile_t publisher_profile,
  const rmw_qos_profile_t subscription_profile,
  rmw_qos_compatibility_type_t * compatibility,
  char * reason,
  size_t reason_size
)

rmw_ret_t
rmw_qos_profile_check_compatible(
  rmw_qos_profile_t publisher_profile,
  rmw_qos_profile_t subscription_profile,
  rmw_qos_compatibility_type_t *compatibility,
  char *reason,
  size_t reason_size
)

rmw_ret_t
rmw_return_loaned_message_from_publisher(
  const rmw_publisher_t * publisher,
  void * loaned_message
)

rmw_ret_t
(*return_loaned_message_from_publisher)(
  const rmw_publisher_t *publisher,
  rmw_loan_token_t *token
)

rmw_ret_t
rmw_return_loaned_message_from_subscription(
  const rmw_subscription_t * subscription,
  void * loaned_message
)

rmw_ret_t
(*return_loaned_message_from_subscription)(
  const rmw_subscription_t *subscription,
  rmw_loan_token_t *token
)

rmw_ret_t
rmw_send_request(
  const rmw_client_t * client,
  const void * ros_request,
  int64_t * sequence_id
)

rmw_ret_t
(*send_request)(
  const rmw_client_t *client,
  rmw_byte_span_t request,
  int64_t *sequence_id
)

rmw_ret_t
rmw_send_response(
  const rmw_service_t * service,
  rmw_request_id_t * request_header,
  void * ros_response
)

rmw_ret_t
(*send_response)(
  const rmw_service_t *server,
  int64_t seq,
  rmw_byte_span_t response
)

rmw_ret_t
rmw_serialize(
  const void * ros_message,
  const rosidl_message_type_support_t * type_support,
  rmw_serialized_message_t * serialized_message
)

rmw_ret_t
rmw_service_request_subscription_get_actual_qos(
  const rmw_service_t * service,
  rmw_qos_profile_t * qos
)

rmw_ret_t
(*service_request_subscription_get_actual_qos)(
  const rmw_service_t *service,
  rmw_qos_profile_t *qos
)

rmw_ret_t
rmw_service_response_publisher_get_actual_qos(
  const rmw_service_t * service,
  rmw_qos_profile_t * qos
)

rmw_ret_t
(*service_response_publisher_get_actual_qos)(
  const rmw_service_t *service,
  rmw_qos_profile_t *qos
)

rmw_ret_t
rmw_service_server_is_available(
  const rmw_node_t * node,
  const rmw_client_t * client,
  bool * is_available
)

rmw_ret_t
(*service_server_is_available)(
  const rmw_client_t *client,
  bool *out_available
)

rmw_ret_t
rmw_service_set_on_new_request_callback(
  rmw_service_t * service,
  rmw_event_callback_t callback,
  const void * user_data
)

rmw_ret_t
(*service_set_on_new_request_callback)(
  rmw_service_t *service,
  rmw_event_callback_t callback,
  const void *user_data
)

rmw_ret_t
rmw_set_log_severity(rmw_log_severity_t severity)

rmw_ret_t
(*set_log_severity)(rmw_log_severity_t severity)

rmw_ret_t
rmw_shutdown(rmw_context_t * context)

rmw_ret_t
(*destroy_session)(rmw_session_t *session)

rmw_ret_t
rmw_subscription_count_matched_publishers(
  const rmw_subscription_t * subscription,
  size_t * publisher_count
)

rmw_ret_t
(*subscription_count_matched_publishers)(
  const rmw_subscription_t *subscription,
  size_t *publisher_count
)

rmw_ret_t
rmw_subscription_event_init(
  rmw_event_t * rmw_event,
  const rmw_subscription_t * subscription,
  rmw_event_type_t event_type
)

rmw_ret_t
(*subscription_event_init)(
  const rmw_subscription_t *subscription,
  rmw_event_type_t kind,
  uint32_t deadline_ms,
  rmw_status_event_callback_t cb,
  void *user_context
)

rmw_ret_t
rmw_subscription_get_actual_qos(
  const rmw_subscription_t * subscription,
  rmw_qos_profile_t * qos
)

rmw_ret_t
(*subscription_get_actual_qos)(
  const rmw_subscription_t *subscription,
  rmw_qos_profile_t *qos
)

rmw_ret_t
rmw_subscription_get_content_filter(
  const rmw_subscription_t * subscription,
  rcutils_allocator_t * allocator,
  rmw_subscription_content_filter_options_t * options
)

rmw_ret_t
rmw_subscription_get_network_flow_endpoints(
  const rmw_subscription_t * subscription,
  rcutils_allocator_t * allocator,
  rmw_network_flow_endpoint_array_t * network_flow_endpoint_array
)

rmw_ret_t
rmw_subscription_set_content_filter(
  rmw_subscription_t * subscription,
  const rmw_subscription_content_filter_options_t * options
)

rmw_ret_t
rmw_subscription_set_on_new_message_callback(
  rmw_subscription_t * subscription,
  rmw_event_callback_t callback,
  const void * user_data
)

rmw_ret_t
(*subscription_set_on_new_message_callback)(
  rmw_subscription_t *subscription,
  rmw_event_callback_t callback,
  const void *user_data
)

rmw_ret_t
rmw_take(
  const rmw_subscription_t * subscription,
  void * ros_message,
  bool * taken,
  rmw_subscription_allocation_t * allocation
)

rmw_ret_t
(*take)(
  const rmw_subscription_t *subscription,
  rmw_mut_byte_span_t *out,
  bool *taken
)

rmw_ret_t
rmw_take_event(
  const rmw_event_t * event_handle,
  void * event_info,
  bool * taken
)

rmw_ret_t
(*subscription_take_event)(
  const rmw_subscription_t *subscription,
  rmw_event_type_t kind,
  rmw_event_payload_t *out,
  bool *taken
)

rmw_ret_t
rmw_take_loaned_message(
  const rmw_subscription_t * subscription,
  void * * loaned_message,
  bool * taken,
  rmw_subscription_allocation_t * allocation
)

rmw_ret_t
(*take_loaned_message)(
  const rmw_subscription_t *subscription,
  rmw_byte_span_t *out_view,
  rmw_loan_token_t **out_token,
  bool *taken
)

rmw_ret_t
rmw_take_loaned_message_with_info(
  const rmw_subscription_t * subscription,
  void * * loaned_message,
  bool * taken,
  rmw_message_info_t * message_info,
  rmw_subscription_allocation_t * allocation
)

rmw_ret_t
(*take_loaned_message_with_info)(
  const rmw_subscription_t *subscription,
  rmw_byte_span_t *out_view,
  rmw_loan_token_t **out_token,
  bool *taken,
  rmw_message_info_t *message_info
)

rmw_ret_t
rmw_take_request(
  const rmw_service_t * service,
  rmw_service_info_t * request_header,
  void * ros_request,
  bool * taken
)

rmw_ret_t
(*take_request)(
  const rmw_service_t *server,
  rmw_mut_byte_span_t *request,
  int64_t *seq_out,
  bool *taken
)

rmw_ret_t
rmw_take_response(
  const rmw_client_t * client,
  rmw_service_info_t * request_header,
  void * ros_response,
  bool * taken
)

rmw_ret_t
(*take_response)(
  const rmw_client_t *client,
  rmw_mut_byte_span_t *reply,
  int64_t *seq_out,
  bool *taken
)

rmw_ret_t
rmw_take_sequence(
  const rmw_subscription_t * subscription,
  size_t count,
  rmw_message_sequence_t * message_sequence,
  rmw_message_info_sequence_t * message_info_sequence,
  size_t * taken,
  rmw_subscription_allocation_t * allocation
)

rmw_ret_t
(*take_sequence)(
  const rmw_subscription_t *subscription,
  uint8_t *buf,
  size_t per_msg_cap,
  size_t max_msgs,
  size_t *out_lens,
  size_t *taken
)

rmw_ret_t
rmw_take_serialized_message(
  const rmw_subscription_t * subscription,
  rmw_serialized_message_t * serialized_message,
  bool * taken,
  rmw_subscription_allocation_t * allocation
)

rmw_ret_t
(*take)(
  const rmw_subscription_t *subscription,
  rmw_mut_byte_span_t *out,
  bool *taken
)

rmw_ret_t
rmw_take_serialized_message_with_info(
  const rmw_subscription_t * subscription,
  rmw_serialized_message_t * serialized_message,
  bool * taken,
  rmw_message_info_t * message_info,
  rmw_subscription_allocation_t * allocation
)

rmw_ret_t
(*take_with_info)(
  const rmw_subscription_t *subscription,
  rmw_mut_byte_span_t *message,
  bool *taken,
  rmw_message_info_t *message_info
)

rmw_ret_t
rmw_take_with_info(
  const rmw_subscription_t * subscription,
  void * ros_message,
  bool * taken,
  rmw_message_info_t * message_info,
  rmw_subscription_allocation_t * allocation
)

rmw_ret_t
(*take_with_info)(
  const rmw_subscription_t *subscription,
  rmw_mut_byte_span_t *message,
  bool *taken,
  rmw_message_info_t *message_info
)

rmw_ret_t
rmw_trigger_guard_condition(const rmw_guard_condition_t * guard_condition)

rmw_ret_t
rmw_wait(
  rmw_subscriptions_t * subscriptions,
  rmw_guard_conditions_t * guard_conditions,
  rmw_services_t * services,
  rmw_clients_t * clients,
  rmw_events_t * events,
  rmw_wait_set_t * wait_set,
  const rmw_time_t * wait_timeout
)

Reproduce

RMW vs upstream: the decision ledger

The user page RMW API: Differences from upstream rmw.h states what is true of the vtable today. This page keeps the trail behind those statements: which phase, issue or RFC made each decision, and what the page used to say before the decision landed. Read it before changing a divergence, so the reason it exists is not lost.

Issue files live under docs/issues/archived/ unless noted; phase docs under docs/roadmap/archived/.

Vocabulary and slot shape

• rmw names. The vtable once used its own vocabulary (subscriber, service_server, service_client, open/close). Phase-301 (phase-301-rmw-shape-alignment.md, issues 0240 and 0241) aligned it with upstream: rmw_subscription_t / rmw_service_t / rmw_client_t, create_session / destroy_session, transport hints moved out of rmw_qos_profile_t into per-create options structs, and the deprecated blocking call_raw slot was deleted. The Rust traits were renamed in the same phase (Subscriber → Subscription, ServiceServerTrait → ServiceTrait, ServiceClientTrait → ClientTrait).
• can_loan_messages is derived (issue 0814). The flag and the borrow_loaned_message slot had drifted in both directions: nothing outside a test ever wrote true (so a zenoh build whose loans worked advertised that it could not loan), while a backend writing true with a NULL slot was believed. The page also once claimed the runtime “dispatches the publish path with no per-call branch” from the flag; it never did.
• Return codes use upstream rmw’s values at the ABI (phase-376 W3.d, phase-376-rmw-api-parity.md).
• Graph slots were added as visitor callbacks in phase-376 W4; node entities (rmw_node_t) arrived in phase-376 W5/B1.
• Heap is a backend property (issue 0777, user page RMW API Design). That page said “no heap” flatly for years; scripts/rmw-alloc-sites.py now counts the allocation sites per backend.
• zenoh version reading (issue 0609): the ROS package version of rmw_zenoh_cpp moved its vendored zenoh 1.2.0 → 1.8.0 in a patch bump, and reading the package version instead of zenoh_configure.h produced a wrong version claim in that issue’s first filing. The router pairing itself is phase-362 / RFC-0075.

QoS

• _SYSTEM_DEFAULT names no concrete policy (issue 0829: the two system-default presets disagreed on depth).
• The QoS mask default is EMPTY (phase-428 W9, phase-428-api-porting-principle-sweep.md). It was QoSPolicyMask::CORE — four policies granted to any implementation that had written no code — until measurement found the zenoh shim honouring none of the four. The check-qos-mask-derivation gate now requires each bit to be backed by a claim sited on the code that reads the profile field.
• The C ABI answers per backend (issue 1329). The cffi route used to answer the UNION of what any supported RMW honours, which over-claimed for each of them.
• Manual liveliness answers UNSUPPORTED where there is no per-topic lease (phase-467 RMW gap-closure study, Row 7, phase-467-rmw-gap-closure-design-study.md). The unsupported default used to be Ok(()) whatever the kind, which reported an assertion that never happened.

Status events

• Matched and incompatible-type events (issue 1972). MATCHED was skipped until issue 1972 ruled it ADOPT; INCOMPATIBLE_TYPE came with it.

XRCE-DDS NULL slots (section 9b of the user page)

just check rmw-slot-producers prints how many slots each backend fills. The rows marked filled on the user page were filled by:

None of the gaps was trivial enough to land with the rulings: the closest, *_get_actual_qos, needed a measurement first, which issue 2107 made (an Agent-side profile cannot override a BIN-created entity’s QoS) and which found that a requester/replier ignores its requested QoS altogether.

Publisher GID, message-info and service info

The section records issue 0242 (parity gaps: publisher GID, message info), updated 2026-07-27.

• C rclc_executor_add_subscription_with_info registrations became removable in issue 1668.
• take_with_info as the C/C++ backend channel, and one GID from one source for get_gid() and MessageInfo::publisher_gid: issue 1495.
• received_timestamp stamping: issue 2135.
• Service info on the server side: issues 2081 (C ABI request info) and 1942 (rclrs ServiceInfo counterpart, zenoh side table).
• Client side: take_response_with_info from issue 2136; sequence-id correlation from issue 0778 (and 2009 for RequestCorrelator); the callback client and C++ PollClient::take_response header from issue 2148.

The cyclonedds fork: what it carries, and why

nano-ros builds Cyclone DDS from a fork (NEWSLabNTU/cyclonedds, branch nano-ros) rather than from the ROS package. This page records exactly what that fork adds, so the delta is a deliberate, reviewable thing rather than an accumulation nobody can enumerate.

The commit list itself, regenerated whenever the pin moves, is forks/cyclonedds.md; this page keeps the reasons.

The delta is carried on purpose. It is not tech debt awaiting an upstream PR — see “Why upstreaming does not retire it” below, which is the part that is easy to get wrong.

The pin tracks ROS, not upstream

The pin matches the cyclonedds that ROS ships, and must. A ROS node on the host links the distro’s Cyclone through rmw_cyclonedds_cpp; an embedded image speaking a different Cyclone is the same drift class as issue 0609, where a vendored zenoh pin fell behind the one rmw_zenoh_cpp actually used. Moving the fork to upstream master would be a wire-and-ABI divergence from every ROS peer we exist to talk to.

So the fork is rebased when ROS’s cyclonedds moves (a distro migration), not when upstream releases.

What the fork adds

27 commits, 56 files, +2623/−142 against the 0.10.x boundary 5041f356. Eight groups.

Re-measured 2026-08-29 and it had drifted BOTH ways: the headline said 15 while the tables below listed 17, and ae14b312 appeared in neither. A count that disagrees with its own table is worse than no count — it reads as the authoritative number. The two verification recipes at the bottom are the check; run them rather than trusting this line.

1. ThreadX / NetX Duo port (6 commits) — a platform upstream does not have

Almost entirely additive — new ddsrt/{sync,threads,sockets,time,heap, ifaddrs,process}/threadx/ trees plus the DDSRT_WITH_THREADX selection arms. Retired only if upstream accepts a ThreadX port, which nobody has offered.

2. Zephyr platform gaps (6 commits)

Upstream does have a Zephyr port (ports/zephyr, WITH_ZEPHYR); these are places where it does not survive contact with Zephyr’s native IP stack / NSOS.

3. Lock-pool scaling (3 commits) — the pair this issue was opened about, plus a 4.x fix to it

404fc5d9 repairs a09babf3 on Zephyr 4.x, where k_condvar_wait takes the mutex back only when woken, so a timed-out wait returned it unlocked and the GC thread’s next unlock aborted (-EINVAL). It takes the mutex back only when the waiter does not already own it, because 3.7 still takes it back itself.

The substantive ones. Zephyr’s POSIX pthread_mutex_t/pthread_cond_t are handles into fixed static pools (CONFIG_MAX_PTHREAD_MUTEX_COUNT and friends). Cyclone puts a mutex in every entity — one per addrset, three per writer — and the count scales with the remote graph (proxy entities, SEDP announcements), not just what this node creates. That makes “how large a ROS graph can this board join” a compile-time RAM constant.

Measured (issue 0371): a 33-node Autoware graph exhausted 16384 slots ~19 s in and died as an anonymous abort(); clearing it needed 131072 slots ≈ 4.1 MiB static. Raising the Kconfig knob is the stock workaround and is a bad trade — the RAM cost is proportional to a worst case you cannot know at build time, and getting it wrong fails deep inside discovery with nothing to point at.

The two changes attack the term from different sides and neither alone was enough: with striping only, 2048 slots still exhausted on the remaining per-proxy-entity locks. The native backend removes ddsrt sync from the pool entirely; striping stops the addrset term scaling at all.

Both carry a behavioural asymmetry worth remembering: k_mutex is recursive where a POSIX NORMAL mutex deadlocks, so a self-relock bug hangs on Linux and passes on Zephyr. See also book-internals/src/platform/implementation-notes.md.

4. Diagnostics for the failure above (3 commits)

Stock Cyclone ignores the return of pthread_mutex_init. That is defensible on a desktop and useless here: it converts pool exhaustion into an anonymous abort() seconds later, which is exactly how 0371 cost as much time as it did.

5. FreeRTOS (3 commits)

With LWIP_NETCONN_SEM_PER_THREAD=1 every thread touching the socket API needs its own netconn semaphore. ddsrt creates its own threads and never called lwip_socket_thread_init(), so the first socket call from one asserted sem != NULL. thread_start_routine is the point they have in common. Found by phase-370 W4, the first work to boot an embedded Cyclone image at all.

8665a51f is the same TLS gap seen from the log path, and it covers ThreadX too. GCC compiles __thread as emulated TLS on a bare-metal newlib toolchain, whose gthreads are single-threaded, so vlog’s scratch buffer is one object shared by every task. vlog still took the sink lock for READING, so two threads tracing at once could advance the shared pos past the buffer and vsnprintf wrote without a bound. It now takes the lock exclusively on those two ports. Found tracing issue 0997, as interleaved records; that issue’s own cause was the FreeRTOS FPU context, not this.