Skip to content

Copper Runtime Overview

Copper is a deterministic, data-oriented runtime for robotics. You describe your system declaratively, Copper compiles the schedule, and every run is bit-for-bit reproducible.

Task Graph

The task graph defines your system topology, inter-task communication, and message types. It is expressed in RON and drives how Copper builds the runtime.

See Copper RON Configuration Reference for the full schema.

Runtime Generator

Copper creates an execution plan from the task graph metadata. At startup it preallocates a pool of CopperLists and initializes every message in place with an empty (None) payload and valid metadata. This applies to synchronous, asynchronous, and parallel execution.

Each execution cycle reuses an initialized slot: the runtime updates the list identifier and lifecycle state, resets per-cycle metadata, and retains payload storage for tasks to reuse. Tasks publish a value with set_payload(...) or clear it with clear_payload(). Pool initialization is paid once at startup.

Two visualizers answer different questions:

  • just graph shows topology: components, ports, connections, and resources.
  • just sched shows the exact generated per-CopperList process order with one scheduler model in both projections. Serial is the main CopperList executor with one worker, including woven anytime refinements; parallel is the staggered generated-stage wavefront. Background nodes appear as poll/dispatch gateways in either projection, with arrows into aligned named-pool worker lanes whose open-ended bars expose work that may persist across later CopperLists. Large in-flight depths show a six-CopperList window (CL n through CL n+5) while reporting the real configured depth. Repeated configured resource targets are highlighted as potential contention, including background jobs and skew outside the nominal wavefront. The columns are ordinal positions, not measured time.

When a config contains missions, just sched stacks every mission by default; pass the mission argument to select one. just sched-log uses the local example log and typed logreader; pass explicit log and logreader arguments to select other inputs. Observed task segments are packed back-to-back while remaining proportional to recorded process duration, with carried-forward stale slots excluded. Observed overlap on the same declared resource is a risk signal, not proof of lock contention; serialization, queue wait, wrapper overhead, and CPU/worker identity cannot be isolated from the current log. Multi-Copper plans remain unsupported.

Zero-Copy Data Logging

Messages between tasks are recorded without copying data. Logs can be replayed exactly, making debugging and certification practical.

Generated applications encode CopperList metadata as a lossless, independently decodable block. Presence planes omit absent TOVs, process times, status strings, origins, and payloads; timestamps use one absolute nanosecond base plus exact deltas; repeated status and origin values are referenced within that CopperList. Payload serialization, including configured CuLogCodec implementations, remains separate and unchanged.

This compressed encoding is the generated default for both std and no_std. Applications that are more CPU-bound than I/O-bound can enable the cu29/flat-copperlist-encoding Cargo feature to restore the previous per-message flat encoding. Writers and typed readers for a log must be built with the same choice, consistent with Copper's existing version-matched log compatibility model.

CopperLists serialize their id and msgs; their lifecycle state remains runtime-only. Binary logs, Serde exports, Python exports, and remote-debug snapshots all omit it. Decoding reconstructs BeingSerialized internally rather than restoring recorded lifecycle history.

Unified-log encapsulation version 2 describes file and section layout, including static metadata and construction context. Decode messages with the producing application's typed logreader or its embedded catalog. Logstream carries no version fields in packets, records, RLC fragments, or session manifests. Compressed metadata keeps the same bytes while reusing cu-bincode 2.1's selective Uleb128<T> wrapper for timestamp deltas and metadata backreferences. Ordinary bincode integers and payload encoding keep their existing representation.

Logstream session manifests carry identity, receiver requirements, application schema and the producing section context using standard bincode encoding. Requirements contain only symbol size, RLC field, window symbols, maximum complete CopperList record bytes, and optional feedback requirements: report interval and destination key. Feedback timeout and adaptation bounds stay in the local sender configuration. The receiver validates those values and enforces its own decoder and buffering limits. RaptorQ geometry comes from packet OTI and uses receiver-local object limits. Destination ID, MTU, pacing, sender memory budget, repair policy, recovery interval, and sender object bounds remain local to the sender. For the ground test profile (1128-byte symbols, GF(256), window 64, 65536-byte records), one-way requirements take 11 bincode bytes versus 40 for the former sender plan: 29 bytes saved per repeated manifest. Savings vary with policy values and destination-name length. Schema checks remain intact; recovery points still bind the exact manifest record by digest. Use matching sender and receiver builds.

Logstream packet headers omit packet sequence counters: FEC symbol identifiers and record identities provide recovery and deduplication. RLC source headers use 32 bytes and repair headers use 36 bytes; RaptorQ headers use 51 bytes. RLC carries object identity, record length, and fragment index inside its protected source fragments, and transmits only the active FEC ID bytes. The protected fragment header uses 20 bytes: magic (4), object ID (8), record length (4), and fragment index (4). Fragment count and payload length are derived from that geometry and the configured symbol capacity; fragment kind is implicitly CopperList. The reassembled record must still match that kind and identity and pass digest verification. This frees 7 bytes per source fragment compared with the previous 27-byte header, allowing more record data in each fixed-size symbol and sometimes fewer fragments. RaptorQ transmits 11 OTI bytes, omitting the reserved zero byte at index 5 of the library's 12-byte representation. The receiver restores that byte before validating geometry and decoding; encoding rejects a nonzero reserved byte. This saves one byte per RaptorQ source or repair packet. Preallocated symbol capacity stays at 1128 bytes. At a 1200-byte MTU, packets are at most 1160 bytes for RLC sources, 1164 for RLC repairs, and 1179 for RaptorQ. Packet payload length is derived from the complete packet extent supplied by the carrier, saving two bytes per packet. Serial and transparent-radio adapters frame byte streams into complete packets before decoding. Semantic records use a 45-byte header: magic (4), kind (1), object ID (8), and BLAKE3 digest (32). Payload length is derived from the complete reassembled record, saving eight bytes per record. The digest still binds kind, object ID, payload length as a big endian u64, and payload, so recovery-point digest references are unchanged. Truncated payloads and appended bytes fail digest verification; receiver allocation bounds still apply to the complete framed record before assembly. Data packets and feedback reports use the same transport integrity contract, saving four bytes per packet in both directions. Neither LogStream format carries an inner CRC. Delivery, ordering, and uniqueness are not guaranteed; feedback controllers reject stale and duplicate reports using their sequence and counters. UDP supplies packet integrity through the network stack. Raw serial supplies neither packet boundaries nor integrity. The framing adapter between LogStream and raw serial appends a four-byte big endian CRC32C over the complete unescaped packet, then escapes packet and checksum between delimiters. The adapter verifies and strips the checksum before delivery, discards damaged frames, and resynchronizes at the next delimiter. This turns corruption into packet loss before FEC. A 514-byte framing buffer permits packets up to 252 bytes under worst-case escaping: (N - 2) / 2 - 4. Account for framing/checksum bytes, escaping, and serial start/stop bits when budgeting link throughput. Both serial peers need the new framing; the older adapter that relied on the common CRC is incompatible. Direct decoder callers must supply complete, carrier-verified packets. Record digests and recovery-point digest references retain their content binding. Sender and receiver must use the matching wire layout.

Generated LogStream twins seal application metadata when opening their receiver archive. With cu29/self-describing-logs, the twin also writes the shared decode catalog from its compiled missions and payload recipes. Build sender and twin from the same application definitions and encoding features to obtain identical catalogs. Received captures can then be decoded by the standalone log extractor. Archive configuration capture is deferred; the effective configuration string is empty.

With LogStream enabled, the shared catalog includes streamed payload recipes even when onboard task logging is disabled.

Set .with_log_rollover(bytes) on the twin builder to retain recent data within a fixed archive capacity. This uses the unified logger's OverwriteOldest policy and retains static metadata and the decode catalog. Capacity must exceed the configured section size, be a multiple of 512 bytes, and fit static metadata plus one archive section. For example, .with_log_rollover(256 * 1024 * 1024) bounds the archive slabs to 256 MiB.

Each sender session belongs to one construction. Its repeated manifest carries the onboard section context: run ID, instance ID and mission index. All receiver archive sections preserve that context across rollover, and late joins recover it from the manifest. Select the twin generated for the sender's active mission; a new construction requires a new session and archive. Lifecycle transitions and failures remain runtime events in the onboard log.

Fast Structured Logging

Structured log strings are interned and indexed at compile time. Runtime logging avoids string construction overhead and keeps hot paths fast.

Remote structured logs

With cu29/logstream and a configured log_streaming destination, generated runtimes forward structured entries alongside CopperLists and keyframes. Each destination gets a bounded copy of the bytes from the existing local serialization pass; framing, FEC, and transmission run on background workers. Local logging continues when streamed entries exceed their buffer limit or are shed under pressure.

Message templates and parameter names travel as interned numeric IDs. Typed parameter values retain their native encoding. The receiver uses the producing application's cu29_log_index to reconstruct readable text and archives the original binary entries in StructuredLogLine sections.

A generated twin exposes take_log_reader() for an independent log view after archival, with a default capacity of 64 entries. Configure it with .with_log_capacity(capacity) before .spawn(). Pausing or dropping the view leaves recording active. In examples/cu_logstream_demo, run just telemetry, then just sender in another terminal: the robot emits simulated encoder temperature and supply-voltage diagnostics with info!(ctx, ...), and the remote log pane displays them. These numeric diagnostics travel only in structured logs; task messages carry joint angles. Use --log-index <path> to select the sender build's index on another machine.

SDK Capabilities

  • Task interface and lifecycle traits for implementing new algorithms, sensors, and actuators.
  • Compile-time runtime generation with deterministic scheduling.
  • Log readers that export Copper logs using Rust Serde formats.
  • Structured log reader for efficient debug and telemetry data.
  • Experimental Python support for offline log analysis and single-task prototyping. See Python Support.
  • A growing set of components and drivers (see the Copper Component Catalog).
  • Deterministic log replay and resimulation.
  • A minimal simulation environment for hardware-free testing.

Received native log archives

The feature-gated logstream receiver discovers a sender's manifest through any CuStreamRx resource. SessionRouter buffers a bounded number of packets before manifest discovery and accepts keyframes and anchors in any arrival order. A restart boundary is usable only after its manifest and keyframe digests and CopperList ids agree. Missing startup history is recorded as a LateJoin gap; outages and expired recovery windows also produce explicit inclusive gaps.

cu29_logstream::NativeArchive<P> derives the expected schema from the generated application dataset type, validates each received CopperList, and appends its canonical bytes to the ordinary native CopperList section. Only verified keyframes enter FrozenTasks. A StreamContinuity section records the manifest, gaps, verified anchors, and receiver finalization. Use a separate archive path for each sender/session. This slice requires the matching full-capture native codec.

Ordinary application logreaders can read the archive. fsck reports source gaps and verified anchors, and cu29_export::stream_continuity_reader exposes those records programmatically. Generated recorded replay and indexed state replay reject missing CopperList history unless a keyframe restores state at the next boundary. A restart does not fill the missing history or prove completeness of an unobserved tail.

Generated exact-output replay aligns the next allocated CopperList id with the recorded recovery boundary and drains pending output between offline iterations. This preserves IDs after a gap and prevents the replay harness from overrunning the bounded output pool. Production real-time execution keeps its nonblocking handoff. Replay harnesses explicitly restore task state from verified keyframes.

Receiver allocation and file I/O stay off the real-time path. The startup packet cache and completed recovery-object cache have explicit receiver-local bounds in addition to RLC and RaptorQ bounds. Run just logstream-receiver-check in the source checkout. The two-process UDP demo compares the received archive with the onboard log and exercises ordinary logreader/replay workflows. Run just in examples/cu_logstream_demo for a clean link, or select loss, outage, late, restart, or idle with just run. Logs stay under the example's own logs/ directory. just logstream-demo-check from the root runs all scenarios, including a real receiver process restart while the sender continues. The display shows archived records, verified anchors, and explicit gaps.

Generated streaming destinations now have one background sender owning the carrier, FEC state, bounded queues, and a shared bitrate/burst budget. Byte-deficit scheduling gives replay and recovery weights 3:1. Ordinary packets expire after the configured latency from encoded-record admission. All work remains after the existing real-time ownership handoff; no runtime object waits for transmission.

The manifest and latest complete keyframe/anchor/CopperList-boundary bundle repeat on a 250 ms local deadline, with overlapping requests coalesced. A transfer finishes before a newer bundle replaces it. just run idle loses the initial transmission, then recovers one captured CopperList with no subsequent task captures.

Pacing uses RobotClock and requires no ground/robot time synchronization. Generated real-link senders use a running clock when application time is mocked. Shutdown stops repetition and drains only to the configured latency deadline. Sender counters and failures are recorded through Copper structured logging. The destination budget covers explicit sender buffers and continuous state; thread stacks, channel/allocator bookkeeping, and object-size-bounded RaptorQ scratch allocations remain additional. Four pending CL boundaries and two pending keyframes bound cross-worker skew; pressure replaces the oldest pending entries.

CuFeedbackTx/CuFeedbackRx are logical packet interfaces: a shared duplex carrier or separate endpoints can supply them. Autonomous operation uses OneWay<T>. Optional validated receiver feedback adjusts repair rate within configured limits; feedback requests share the sender scheduler and retention bounds. Run just logstream-feedback-check for feedback and adaptive FEC checks.

Native ground-station telemetry

The UDP demo now includes an optional Ratatui screen. In examples/cu_logstream_demo, start just telemetry, then just sender in another terminal. Use fresh log paths for repeated runs. The manual sender runs for about a minute. Space pauses the view; q closes the receiver and its archive. The screen shows captured joint angles, the reconstructed arm pose, encoder charts, remote structured logs, packet freshness, archive progress, recovery anchors, and separate source-gap and UI-overwrite counts.

The std-only cu29_logstream::telemetry::telemetry_channel moves successfully archived, already decoded frames into a bounded circular buffer. User code waits on ready().await, wait_timeout, or a registered scheduling waker, then pulls try_read() on its own thread. It receives a stable borrowed typed frame and an exact count of unread overwritten samples, resuming at the oldest retained frame. Status is coalesced separately and remains readable while the view is paused. There are no receiver-thread data callbacks. The user owns chart history, projections, storage, and widgets. The demo uses generated named task accessors.

Storage is preallocated for the ring and one in-flight reader frame; payload-owned allocations are additional. Recording never waits for slow or disconnected readers. Both share a process failure boundary. Only successfully archived captures enter this initial telemetry path; a writer error is reported rather than retried or hidden.

Run just logstream-telemetry-check at the repository root for notification/overrun tests, archive equality with fast/stalled/disconnected readers, terminal layouts, and all six existing UDP recovery/replay scenarios. The UI dependency is behind the example's tui feature; telemetry is std-only and adds no work to the robot's real-time task path.

Live Copper twin

The demo graph is encoders -> kinematics. The ordinary Kinematics Copper task computes the arm pose from joint angles on the robot and in generated ground-side replay. Its payload is never transmitted, including repeated anchor boundaries and FEC repairs. The robot's onboard log contains the full output for comparison. The telemetry screen identifies the reconstructed pose and labels it pose payload not transmitted.

Declare the static contract in the same RON used by the robot and ground build:

(id: "kinematics", type: "cu_logstream_demo::tasks::Kinematics",
 streaming: (replay: reconstruct)),

Reconstruction recomputes outputs from captured inputs, clock, and frozen state. It supports ordinary synchronous logged tasks using the lossless native compressed codec and full handle capture. Every producer feeding a reconstructed task must have logging enabled; invalid connections fail compilation. Use deterministic task code without external side effects when reconstructing on another target.

A ground station declares #[copper_runtime(config = "copperconfig.ron", sim_mode = true)]. The generated application exposes a twin builder:

```rust,ignore let (mut twin, mut frames) = Ground::twin(rx) .with_log_path("logs/received.copper") .spawn()?;

// On the UI or analysis thread: frames.wait_timeout(std::time::Duration::from_millis(50)); while let Some(update) = frames.try_read() { render(&update.frame.copperlist); } let status = twin.stop()?; ```

rx is any CuStreamRx, such as the receive half of a UDP resource. Copper owns session routing, native recording, the bounded replay worker, status publication, and shutdown. The caller owns the frame reader and presentation. Pausing or dropping that reader never blocks recording. Dropping the twin stops and joins its workers; stop() also reports receiver errors and final counters. archive_only() records without running a twin. Each handle accepts one sender session and a fresh log path. Configure the twin by chaining .with_slab_size(bytes), .with_section_size(bytes), .with_receiver_limits(limits), .with_replay_capacity(capacity), .with_frame_capacity(capacity), and .with_log_capacity(capacity) before .spawn(). SessionRouterLimits::default() supplies one sender using the 1200-byte-MTU, 64-symbol FEC profile, 4 KiB records and 64 KiB finite objects. Override record and finite-object bounds for the application's largest serialized CopperList and keyframe. Defaults are 16 MiB slabs, 128 KiB sections, 32 replay events and pending captures per buffer, and 64 unread reconstructed CopperLists and structured log entries per reader buffer.

Sections must fit the larger configured record/object bound plus a 512-byte header and 32 bytes of continuity envelope allowance, and must fit in a slab. Both sizes must be multiples of 512 bytes. Prefer slabs roughly 10–100 times larger than sections so each backing file holds many sections and file turnover stays low. Larger slabs reserve more disk space at a time. The archive grows with additional slabs. Local settings are validated before workers start; sender record requirements are checked when the manifest arrives. Size retention and concurrency counts together with byte bounds for the receiver's memory budget.

Replay buffers absorb temporary reconstruction slowdowns; overflow interrupts replay until a matching recovery point is available while recording continues. A frame is one reconstructed CopperList containing captured inputs and locally computed outputs for one iteration. Frame buffering lets UI or analysis readers pause. Structured log buffering separately holds the robot's logging calls, such as info! and debug!, for twin.take_log_reader(). Both reader buffers replace the oldest unread item when full; the archive keeps the received data. Higher counts retain more message payloads and log entries in memory. The receiver example uses named constants with explanations of the storage and buffer sizing.

Production sends the existing native CopperList format with selected payloads omitted. The native codec already carries original/captured presence. The receiver records received native bytes before replay; synthesized outputs remain in the ground-side frame. Sender and receiver use the same application schema and Copper version. Packet framing, FEC, and verified recovery points preserve stream continuity across gaps.

Copper restores keyframes, injects captured inputs, executes reconstructible tasks and restores sender metadata before downstream tasks run. Existing source gaps and replay queue overflows require a matching recovery anchor. These continuity checks are separate from checking whether deterministic task code produced the right result. The generated ground runtime disables its own logging and transport transmitters; its archive is owned by the twin receiver.

Reconstruction correctness checks are entirely opt-in, even in debug Rust builds. Enable cu29/logstream-verify (the demo calls it verify-reconstruction) on both ends for development. Only this feature compiles in hashing and a fixed 32-byte digest trailer covering the omitted outputs and their payload presence. Hashing runs on the existing sender output worker and the ground replay worker, using borrowed payloads with no intermediate allocation. Production captures have no trailer or digest storage. A normal receiver rejects debug trailers explicitly; a verification receiver also accepts normal captures and labels them Reconstructed, never Verified.

Only debug captures that pass comparison are labeled Verified. A mismatch suppresses reconstructed frames until the next matching anchor; native recording continues. Debug digests are consumed live and do not add archive sections or change offline log readers. Use just telemetry-verify and just sender-verify to run the development checks. just resim reconstructs ordinary capture archives offline; the demo's verification command compares the result against the full onboard log.

Run just logstream-twin-check for allocation/native-format checks, both verification modes, worker lifecycle, reader isolation, and the UDP loss/recovery/replay scenarios.