Keyboard shortcuts

Press ← or → to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

Self-Describing Logs

v1.3 Suppose you receive a robot’s .copper recording and want to plot wheel speed. With an application-specific logreader, you need the Rust payload types used by that application. A self-describing log carries a catalog that tells an offline reader how to decode those payloads, including field names and storage units. You can inspect the recording with Copper’s standalone tools and export the samples without building the robot’s logreader.

Use this feature when you share recordings with another team, keep logs for later analysis, or collect recordings from several applications. This feature is experimental; use the checkout named in the edition notice for the examples below.

1. Try a complete recording

From the root of a copper-rs checkout, enter the wheel-sensor example:

cd examples/cu_self_describing_logs
just

This records ten cycles in logs/wheel.copper, displays the embedded catalog, and checks the recording with fsck --deep. The graph is a wheel source, a filter, and a sink. Each cycle captures the source and filter payloads; the sink contributes its execution metadata.

Look at the catalog’s WheelSample fields. Alongside encoder ticks it describes distance, speed, a timestamp, acceleration, temperatures, and sensor state. The reader obtains those names and types from the recording itself.

Export the captured cycles from the same directory:

just extract --export-format jsonl > samples.jsonl

Each JSONL line is one CopperList, with its id and ordered msgs. A message contains its payload, timestamps, and capture information. Check that samples.jsonl has ten lines, with source and filtered wheel samples in each cycle. Exporting Data continues with catalog dumps, CSV, and payload-size diagnostics using this same recording.

2. Enable it in your application

The project templates expose an application feature. For an existing app, add this forwarding entry to its Cargo.toml:

[features]
self-describing-logs = ["cu29/self-describing-logs"]

Keep the usual build script calling cu29_build::setup() and your ordinary builder:

let app = App::builder()
    .with_log_path("logs/app.copper", Some(32 * 1024 * 1024))?
    .build()?;

Run your application’s recording binary with the feature enabled:

cargo run --features self-describing-logs

Copper builds and saves the catalog during application construction, before initializing resources. It describes the output types declared in your RON graph, including types from dependency crates. One catalog covers every compiled mission and records each mission’s output-slot order.

Payload capture still follows your logging configuration. Keep enable_task_logging: true and enable logging on the outputs you want to inspect. A catalog describes a type; a task with logging: (enabled: false) does not contribute its payload bytes to the recording.

3. Describe your payloads

Ordinary encoding derives supply the descriptions when the feature is enabled. For example, a wheel payload can use Copper’s units directly:

use cu29::bincode::{Decode, Encode};
use cu29::prelude::*;
use cu29::units::si::f32::{Length, Velocity};

#[derive(Clone, Debug, Default, Serialize, Deserialize, Encode, Decode, Reflect)]
#[bincode(crate = "cu29::bincode")]
pub struct WheelSample {
    pub ticks: u32,
    pub distance: Length,
    pub speed: Velocity,
    pub timestamp: CuTime,
}

The catalog follows nested fields and enum variants automatically. A reusable payload crate can add #[bincode(describe)] to provide descriptions even when the recording feature is disabled.

Units tell you how to interpret the stored value. A length created from centimetres is stored in metres, velocity uses m·s⁻¹, and CuTime stores nanoseconds. Use the catalog’s storage unit when plotting or converting exported numbers. The unit used to construct a Rust quantity does not change its storage unit.

If you implement Encode by hand, also implement ValueDecode with a recipe that matches the bytes your encoder writes. For an Orientation wrapper whose encoder writes exactly an array of four f32 values:

impl cu29::bincode::ValueDecode for Orientation {
    const DECODE: &'static cu29::bincode::ValueDecodeSpec =
        <[f32; 4] as cu29::bincode::ValueDecode>::DECODE;
}

Check that the recipe decodes the complete encoded value, including consecutive values. A custom logging codec also needs a recipe for its recorded representation. If startup rejects a task/type/codec combination, check that encoder’s description.

4. Read your own recording

From the copper-rs repository root, pass the recording’s base path to the standalone reader. Replace logs/app.copper below with your recording’s path:

just logextract logs/app.copper catalog
just logextract logs/app.copper extract-copperlists --export-format jsonl > samples.jsonl
just logextract logs/app.copper fsck --deep

just logextract builds the standalone reader with catalog support. Keep the complete slab family with the recording. Structured text reconstruction still uses the producing build’s cu29_log_index.

For recordings containing several runs and for detailed validation reports, continue to Exporting Data. For dictionary-based analysis, see Python Support.

Recording on an embedded target

The producer feature supports no_std. Schema serialization and compression finish at startup using bounded working memory, without allocating a complete catalog buffer. The recording loop keeps its ordinary native encoding pass. Your logger backend still determines its own storage allocation requirements.

The startup traversal supports up to 256 distinct reachable native types, including nested field types. A graph exceeding that limit returns a startup error. Host-side catalog readers enable cu29/decode-catalog and use std to allocate decoded value trees; cu29-export/self-describing-logs selects that reader feature automatically.

Append reuses a matching catalog, and rollover retains it so surviving samples remain decodable. See recording retention for the storage budget and the history that rollover can reclaim.

The wheel example provides the complete application. The format reference describes the encoding for readers that implement their own tooling.