Master Backends =============== .. note:: ``MockMasterEx``, ``RecordingMaster``, ``ReplayMaster``, and the ``BusPolicies`` bundle (barriers/cache/retry) have all shipped and are covered in detail below. ``SimMaster`` is **alpha** — its semantics today match :class:`MockMaster`, with the behavioural-simulator divergence (RDL side-effects honoured natively) landing in a future batch. A handful of SoC-level helpers (``soc.attach_master``, ``soc.barrier``, ``soc.batch``, ``soc.trace``, ``soc.lock``) and the ``async_session`` surface remain part of the design sketch in ``docs/IDEAL_API_SKETCH.md`` §13. The shape and naming are stable; a class that is not yet implemented will not appear in the autogenerated reference at the foot of this page. Overview -------- A master is the bus binding for the generated SoC tree. It is the only component that talks to real hardware (or pretends to, for tests), and it is the piece you swap when you move from a unit test to a JTAG bring-up or from a lab notebook to a CI replay. Three principles drive the design: - **Replaceable.** The same generated SoC tree runs against ``MockMaster``, ``OpenOCDMaster``, ``SSHMaster``, a simulator, or a recorded trace. Switching masters does not require code changes anywhere else. - **Observable.** Every read and every write goes through the master, so it is the right place to attach traces, coverage observers, retry policies, and recording wrappers. - **Explicit.** Bus semantics (barriers, caching, retries, batching) are first-class, named, and discoverable. Nothing important happens by accident. Built-in masters ---------------- Entries marked *(alpha)* have shipped a thin implementation but still point at a richer behavioural target — they appear in examples so the intended shape is concrete. ``MockMaster`` ^^^^^^^^^^^^^^ In-memory master backed by a dict. The default for unit tests, REPL exploration, and notebook examples — every read returns the value last written (or zero), no side effects, no hooks. Reach for ``MockMaster`` when the test only needs a place to land bytes. For tests that need register-level read/write hooks, RDL side-effect semantics (``rclr`` clear-on-read, ``woclr`` write-1-to-clear), or bulk memory preloading, use :class:`MockMasterEx` (described below) instead. ``MockMasterEx`` ^^^^^^^^^^^^^^^^ A subclass of :class:`MockMaster` that adds the surface tests actually need. Lives in :mod:`peakrdl_pybind11.masters.mock_extensions` and is also re-exported at ``peakrdl_pybind11.masters`` for convenience. * ``on_read(reg, fn)`` / ``on_write(reg, fn)`` — install per-register callbacks. ``reg`` may be a register handle (its ``info.address`` is resolved automatically) or a raw integer address. * ``preload(mem, ndarray)`` — bulk-seed a memory region with an iterable or numpy array of word values; the default ``word_size`` of 4 matches a 32-bit memory layout. * ``mark_rclr(reg)`` / ``mark_woclr(reg)`` — explicitly tag an address with read-clear-on-read or write-1-to-clear semantics. When a hook is installed against a register handle whose ``info.on_read`` is ``"rclr"`` (or ``info.on_write`` is ``"woclr"``), the tag is inferred automatically; ``mark_*`` is the raw-address equivalent. Pick :class:`MockMasterEx` whenever a test needs to react to a write or assert on hook-observed values; pick :class:`MockMaster` when in-memory storage is enough. ``OpenOCDMaster`` ^^^^^^^^^^^^^^^^^ Talks to an OpenOCD server over its TCL/RPC port for JTAG- or SWD-attached silicon. Use it for bring-up at a workstation with a debug probe wired to the target. ``SSHMaster`` ^^^^^^^^^^^^^ Tunnels register access through an SSH connection. Useful when the bus is exposed by a Linux helper running on the target SoC, on a remote tester, or on a lab machine you reach through a jump host. ``CallbackMaster`` ^^^^^^^^^^^^^^^^^^ Lightweight master that forwards every transaction to user-supplied callables. Handy for ad-hoc bridging (e.g. piping to a custom UART or PCIe shim) without subclassing the base. ``SimMaster`` *(alpha)* ^^^^^^^^^^^^^^^^^^^^^^^ The intent is a bus-functional model that drives a co-simulated RTL design (cocotb, verilator, etc.) and natively honours RDL side-effects (``rclr``, ``W1C``, ``singlepulse``, ...). Today the source flags it as alpha-quality: behaviour matches :class:`MockMaster` plus a constructor that pre-seeds the in-memory store, so a test written for ``MockMaster`` runs against ``SimMaster`` unchanged. Tests that depend on side-effect simulation should keep using the :class:`MockMasterEx` mock-with-callbacks pattern until the divergence lands in a future batch. ``ReplayMaster`` ^^^^^^^^^^^^^^^^ Replays a recorded session against the same SoC tree. Used for regression tests, post-mortems, and deterministic CI replays of recorded bring-up runs. Strict mode is the default; mismatches raise ``ReplayMismatchError`` (see :ref:`tracing-and-replay`). ``RecordingMaster`` ^^^^^^^^^^^^^^^^^^^ Wraps any other master and records every transaction to a file. The recorded log is consumed by ``ReplayMaster`` and may be either a JSON array (the default for ``.save("trace.json")``) or NDJSON (one event per line, used for crash-safe streaming via ``file=``). Composing masters ----------------- Most chips do not have a single bus. JTAG owns the debug peripherals, a memory-mapped DMA owns SRAM, and a mock fills in for absent silicon. The SoC supports multiple masters, each scoped to a region. .. code-block:: python from peakrdl_pybind11.masters import ( MockMaster, OpenOCDMaster, SSHMaster, SimMaster, ReplayMaster, RecordingMaster, ) # Single master for the whole SoC soc = MySoC.create(master=OpenOCDMaster("localhost:6666")) # Per-region routing soc = MySoC.create() soc.attach_master(jtag, where="peripherals.*") soc.attach_master(mem_master, where="ram") soc.attach_master(MockMaster(), where=lambda node: node.info.is_external) soc.attach_master(other, where=(0x4000_0000, 0x4000_FFFF)) The ``where=`` selector accepts: - a **glob** string against the node path (``"peripherals.*"``), - a **callable** ``(node) -> bool`` for arbitrary predicates, - an **address-range** tuple ``(lo, hi)`` over absolute addresses. Masters must serve **disjoint** regions; the SoC raises ``RoutingError`` on overlap and on any address that no attached master claims. Transactions as objects ----------------------- The master accepts a list of reified transactions, so you can script the bus without going through the typed tree (handy for tools that have an address but not a node, or for re-issuing a recorded sequence). .. code-block:: python from peakrdl_pybind11 import Read, Write, Burst txns = [ Read(0x4000_1000), Write(0x4000_1004, 0x42), Burst(0x4000_2000, count=128, op="read"), ] results = soc.master.execute(txns) For typed-tree users, ``soc.batch()`` coalesces a sequence of operations into one master command when the backend supports queuing: .. code-block:: python with soc.batch() as b: b.uart.control.write(1) b.uart.data.write(0x55) # All transactions sent at exit; one round-trip if the master can queue. .. _bus-policies-bundle: The ``BusPolicies`` bundle -------------------------- Barrier ordering, read caching, and retry/backoff are bundled into a single :class:`peakrdl_pybind11.runtime.bus_policies.BusPolicies` container that wraps a master's ``read`` / ``write`` once and keeps the three policies in a defined order: cache lookup → retry loop → barrier fence → master. The SoC layer attaches a bundle to every master at attach time, but the bundle is also reachable directly when you need to configure a policy from your own code: .. code-block:: python from peakrdl_pybind11.runtime import attach_master_extension from peakrdl_pybind11.masters import MockMaster master = MockMaster() bundle = attach_master_extension("bus_policies", master) # bundle is a BusPolicies instance with three policy objects: bundle.barriers # BarrierPolicy — fences and "auto" raw-fencing bundle.cache # CachePolicy — TTL slots + cached_window blocks bundle.retry # RetryPolicy — backoff, on_disconnect, give-up The bundle can also be imported directly when the registry seam is not needed: .. code-block:: python from peakrdl_pybind11.runtime.bus_policies import BusPolicies bundle = BusPolicies(master=master) Note that :class:`BusPolicies` mutates the master in place — it captures the original ``read`` / ``write`` and replaces them with delegations through the policy chain. Wrapping the same master twice would double-wrap; if you need to re-install, reconstruct the master first. The next three sections walk through what each policy on the bundle does in practice; see :doc:`/concepts/bus_layer` for the conceptual depth on barriers, caching, and retry semantics. Barriers and fences ------------------- Many masters queue or coalesce writes; some buses post writes asynchronously. A barrier forces all in-flight writes on the relevant master(s) to drain before the next read. .. code-block:: python # Per-master scope (default and usually what you want) soc.uart.barrier() # masters serving the uart subtree soc.master.barrier() # the explicit single-master form soc.barrier() # masters used at the current call site # SoC-wide scope (rare; needed when read on master B depends on a # write that went out via master A) soc.barrier(scope="all") soc.global_barrier() # alias of the above; reads better in scripts Auto-barrier policy is set per SoC. The default flushes the relevant master only before a read-after-write on the **same** master, since flushing every master on every barrier is expensive when masters serve disjoint regions. .. code-block:: python soc.set_barrier_policy("auto") # default: same-master raw fences soc.set_barrier_policy("none") # opt out, faster, you fence yourself soc.set_barrier_policy("strict") # barrier before every read AND write soc.set_barrier_policy("auto-global") # auto-fences across all masters (paranoid) Read coalescing and cache policy -------------------------------- Tight polling loops often hammer the same register. The master can cache recent reads when it is safe to do so. .. code-block:: python soc.uart.status.cache_for(50e-3) # within 50 ms, return cached value soc.uart.status.invalidate_cache() with soc.cached(window=10e-3): # any read inside this block may serve from a 10 ms cache window run_polling_loop() Caching is **refused** for side-effecting reads (``info.on_read != NONE``) and for volatile registers (``info.is_volatile``). The master does not attach a cache to those nodes even if asked, and an explicit ``cache_for(...)`` call on a destructive register raises. Bus error recovery ------------------ Real hardware misbehaves. The master applies a configurable retry policy and surfaces a structured error when it gives up. .. code-block:: python soc.master.set_retry_policy( retries=3, backoff=0.05, # exponential backoff base, in seconds on=("timeout", "nack"), # underlying errors that trigger retry on_giveup="raise", # also: "log", "panic" ) # Per-call override soc.uart.control.read(retries=10) # Reconnect hook (e.g. re-attach JTAG, replay last N transactions) soc.master.on_disconnect(lambda m: m.reconnect()) When the policy gives up, the call raises ``BusError`` carrying the failed transaction, the retry count, the underlying exception, and the node path / absolute address. That is enough for a CI run to triage why it died. .. _tracing-and-replay: Tracing and replay ------------------ Every transaction passes through the master, which makes the master the right place to record and replay sessions. .. code-block:: python with soc.trace() as t: soc.uart.control.write(0x42) soc.uart.status.read() print(t) # 2 transactions, 8 bytes # wr @0x40001000 0x00000042 (uart.control) # rd @0x40001004 -> 0x00000001 (uart.status) t.save("session.json") soc2 = MySoC.create(master=ReplayMaster.from_file("session.json")) ``RecordingMaster(inner, file=None)`` ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ For long-running lab sessions, wrap the live master in :class:`RecordingMaster` so the transcript lands on disk as it happens. Two on-disk shapes are supported and chosen by extension: * **JSON array** — a single document, written by ``.save("trace.json")`` (the default for any extension other than ``.ndjson`` / ``.jsonl``). Compact and easy to load with one ``json.loads``; not crash-safe (the file is rewritten in one go). * **NDJSON** — one event per line, used by both the streaming ``file=`` argument and ``.save("trace.ndjson")`` / ``.save("trace.jsonl")``. Each event is flushed per-op so a long-running session that crashes still leaves a usable trace, and concatenating multiple NDJSON traces yields a valid combined log. .. code-block:: python from peakrdl_pybind11.masters import RecordingMaster # Stream NDJSON to disk as the run unfolds. soc.attach_master(RecordingMaster(jtag, file="run.ndjson")) # Or capture in memory, then dump as JSON or NDJSON depending on path. rec = RecordingMaster(jtag) soc.attach_master(rec) ... rec.save("trace.json") # JSON array rec.save("trace.ndjson") # NDJSON, one event per line The recording is a stream of the same transaction objects the master already speaks, so it round-trips exactly under :class:`ReplayMaster`. ``ReplayMaster.from_file(path)`` ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ :meth:`ReplayMaster.from_file` auto-detects the format (NDJSON vs JSON array) and is **strict by default**: every requested transaction must match the next recorded event in order (op kind + address + width; writes also compare value). A mismatch raises ``ReplayMismatchError`` — a distinct exception class from :class:`BusError` so tests can catch "the recording doesn't match what the driver did" without conflating it with "the bus itself errored". Pass ``strict=False`` for the looser mode that serves matching reads from the recording and silently ignores extras. .. code-block:: python from peakrdl_pybind11.masters import ReplayMaster, ReplayMismatchError replay = ReplayMaster.from_file("run.ndjson") # strict mode soc = MySoC.create(master=replay) try: run_driver_sequence(soc) except ReplayMismatchError as exc: # exc.expected is the recorded Event (or None if exhausted); # exc.actual is a {"op", "address", "width", ["value"]} dict. print(f"recording diverged: expected {exc.expected}, got {exc.actual}") # Loose mode for a quick "what would the driver do here?" probe. loose = ReplayMaster.from_file("run.ndjson", strict=False) Mock master with hooks ---------------------- The hook-driven, side-effect-aware mock is :class:`MockMasterEx` — :class:`MockMaster` itself is a plain dict-backed store and does not have ``on_read`` / ``on_write`` / ``preload``. Tests that want to express behaviour rather than bookkeeping should import the extension class: .. code-block:: python import numpy as np from peakrdl_pybind11.masters import MockMasterEx mock = MockMasterEx() mock.on_read(soc.uart.intr_status, lambda addr: 0b101) mock.on_write(soc.uart.data, lambda addr, val: stdout.append(val)) mock.preload(soc.ram, np.arange(1024, dtype=np.uint32)) soc.attach_master(mock) The mock honours the clear-on-read (``rclr``) and write-one-to-clear (``woclr``) semantics declared in RDL. When you pass a register handle whose ``info.on_read`` is ``"rclr"`` (or ``info.on_write`` is ``"woclr"``) to ``on_read`` / ``on_write``, the side-effect tag is inferred automatically — a register declared ``rclr`` clears under the mock just as it would on silicon. When you only have a raw integer address, ``mock.mark_rclr(addr)`` and ``mock.mark_woclr(addr)`` apply the tags explicitly. Concurrency ----------- The master holds a re-entrant lock by default; multi-threaded callers do not tear up shared state. For sequences that must not be interleaved even with other transactions on the same master, take the lock explicitly: .. code-block:: python with soc.lock(): soc.dma.config.write(...) soc.dma.start.pulse() # nothing else hits this master until the block exits The async dual surface exposes ``aread``, ``awrite``, and ``amodify`` on every node. It is opened with a session context so cancellation, back-pressure, and reconnection are well-defined: .. code-block:: python async with soc.async_session(): await soc.uart.control.awrite(0x1) v = await soc.uart.status.aread() Custom masters -------------- To add a new backend, inherit from ``MasterBase`` and implement the read/write surface. The base class handles the routing, retry, observation, and lock plumbing — the subclass only needs to know how to move bytes. .. code-block:: python from peakrdl_pybind11.masters import MasterBase class MyCustomMaster(MasterBase): def read(self, address, width): ... # one bus read; raise BusError on failure return value def write(self, address, width, value): ... # one bus write; raise BusError on failure # Optional: implement burst, peek, barrier, reconnect, on_disconnect # if your transport supports them. Default fallbacks loop / raise # NotSupportedError as appropriate. Reference --------- .. automodule:: peakrdl_pybind11.masters :members: :undoc-members: :show-inheritance: :special-members: __init__