Model & observers

ActionConfig

Speed/steering limits of the action space (src/actions.py::ActionConfig). Defaults mirror Python; the experiment YAMLs set v_fast 0.41, v_slow 0.17.

ActionSpec

One row of the discrete action table: name, commanded v, commanded omega (src/actions.py::ActionSpec).

DuckieAction

Canonical continuous action (v_cmd, ω_cmd) in simulator command magnitudes (m/s, rad/s). Discrete macro-actions project onto it; SAC/TD3 act in the Box [0, -w0] × [v_fast, w0].

MacroAction

The 7 discrete macro-actions for tabular solvers (index order fixed by build_action_table in src/actions.py; exact table values are verified in FJ2).

The mapping is fast_left=(v_fast,+w0), fast_straight=(v_fast,0), fast_right=(v_fast,-w0), slow_left=(v_slow,+w0), slow_straight=(v_slow,0), slow_right=(v_slow,-w0), brake=(0,0).

action_to_wheels(action_id::Integer, cfg=ActionConfig()) -> NTuple{2,Float32}

Look up macro-action action_id ∈ 0:6 and convert it to clipped wheel commands (src/actions.py::action_to_wheels). Out-of-range ids raise ArgumentError (Python: ValueError).

build_action_table(cfg=ActionConfig()) -> NTuple{7,ActionSpec}

A = {fast/slow × left/straight/right, brake} with the Python order (src/actions.py::build_action_table): fast_left, fast_straight, fast_right, slow_left, slow_straight, slow_right, brake. left means +w0 (this is the sign convention the reference policies were trained with).

vw_to_wheels(v, omega, wheel_base) -> NTuple{2,Float32}

Inverse differential-drive kinematics: u_L = v - L·ω/2, u_R = v + L·ω/2, clipped to ±1 (src/actions.py::vw_to_wheels). Arithmetic is Float64, rounded to Float32 before clipping, exactly like np.clip(np.array([l, r], dtype=np.float32), -1, 1).

OBSERVATION_NAMES

The 15 privileged continuous-state features, in order. The 15th feature (stop_hold_progress) was appended later and is the append-only element of the feature list (src/continuous_state.py::OBSERVATION_NAMES).

ContinuousState

15-component privileged state (d, phi, v, kappa, stop_present, d_stop, sigma_stop, duck_present, duck_longitudinal, duck_lateral, duck_v_longitudinal_relative, duck_v_lateral_relative, duck_active, duck_crossing_available, stop_hold_progress) used by SAC/TD3 (src/continuous_state.py::ContinuousState).

stop_hold_progress is a feature of the stop tracker's dwell counter; the canonical dwell memory lives in DuckieWorldState.

ContinuousStateConfig

Normalization and detection-gate parameters for the continuous state (src/continuous_state.py::ContinuousStateConfig). The three duck_detection_* fields default to disabled (older SAC behaviour); the SAC/TD3 experiment YAMLs enable the gate (range 1.20, corridor 0.60, forward_only true).

DuckRelativeState

Geometry of the nearest duckie in the ego lane frame (src/continuous_state.py::DuckRelativeState).

D_BINS, TRACKING_ERROR_BINS, V_BINS

Bin edges of the tabular discretizer (src/discretizer.py), in Python order. np.digitize(x, bins) (right-open bins, bins[i-1] ≤ x < bins[i]) equals searchsortedlast(bins, x), i.e. the count of edges ≤ x.

STATE_SHAPE, Q_SHAPE

Number of bins per dimension of the discretized state (7 dims), and the Q-table shape STATE_SHAPE + (7,) for the macro-action dimension.

digitize(x, bins) -> Int

NumPy np.digitize semantics (default right=false): the number of bin edges ≤ x, i.e. searchsortedlast(bins, x). Valid on monotone bins.

discretize(state::RawState) -> NTuple{7,Int}

Exact 7-D index of the Q-table for a RawState (src/discretizer.py::discretize):

s_bar = (bin(d), bin(phi + d), bin(v), tile, bin(d_stop), sigma_stop, duck)

e = phi + d is the tracking error; binning it (not phi alone) removes the state aliasing between poses with equal heading but different offsets. d_stop classes: none → 0, > 1.0 → 1, ≥ 0.3 → 2, else 3.

The range guard mirrors Python (IndexError(index)); with the four bin edges of D_BINS/TRACKING_ERROR_BINS/V_BINS and valid enums it is unreachable (max digitize = length(bins) < shape), exactly as in the reference.

build_continuous_state(raw, kappa, duck, stop_hold_progress=0.0) -> ContinuousState

Assemble the 15-component continuous state from the tabular RawState projection, the signed look-ahead curvature and the (already gated) duckie relative state (src/continuous_state.py::build_continuous_state).

The two env-dependent inputs — duck (from duck_relative_state) and kappa (from signed_curvature_ahead) — are produced by the FJ3 dynamics; this pure assembly is what FJ2 pins.

continuous_observation_space() -> (low, high)

Normalization bounds of the 15-D observation (src/continuous_state.py): low = [-1,-1,0,-1,0,0,0,0,-1,-1,-1,-1,0,0,0], high = 1 everywhere (Float32). Returned as (low, high) — the Python wrapper is gym.spaces.Box(low, high, dtype=np.float32).

encode_continuous_state(state, cfg) -> Vector{Float32}

Normalize a ContinuousState into the 15-D SAC/TD3 observation vector (src/continuous_state.py::encode_continuous_state). Arithmetic is performed in Float64 and rounded to Float32 exactly as NumPy's np.array(..., dtype=np.float32). Non-finite observations raise ArgumentError (Python: ValueError).

gate_duck_visibility(duck, cfg) -> DuckRelativeState

Apply the optional duckie detection gate (src/continuous_state.py): with the gate disabled the nearest duckie is always reported at any distance; with it enabled, a duckie outside the forward corridor, out of range, or behind the ego is reported as absent, giving information parity with the tabular classify_duck.

_directed_curve(curves, forward) -> Union{Nothing,Matrix{Float64}}

continuous_state.py::_directed_curve: the tile curve whose end-to-end heading best aligns with forward (1e-12 normalization variant); nothing when the tile carries no curves.

classify_duck(world, cfg) -> DuckThreat

state.py::classify_duck: maximum pedestrian threat over the visible duckies inside the ego's forward corridor (ahead >= 0, |lateral| <= corridor, planar distance <= duck_max_distance); crossing vs side by pedestrian_active, near vs far by duck_near_distance.

distance_to_next_stop(world, cfg) -> Union{Nothing,Float64}

state.py::distance_to_next_stop — the distance half of next_stop_candidate.

duck_relative_state(world, ego_speed, controller_cfg) -> DuckRelativeState

continuous_state.py::duck_relative_state: geometry of the nearest visible duckie in the ego's continuous lane frame. ego_speed is raw.v. controller_cfg === nothing reproduces controller=None (crossing_available = true); otherwise availability follows DuckController.crossing_available(i): (limit <= 0 || crossings_started[i] < limit) && crossing_armed[i] with i the duckie's index in world.ducks (Python: identity lookup in controller.ducks).

get_continuous_state(world, raw, state_cfg, continuous_cfg;
    controller_cfg=nothing, stop_hold_progress=0.0) -> ContinuousState

continuous_state.py::build_continuous_state applied to the latent world: gated duckie geometry + signed look-ahead curvature assembled onto the tabular raw projection by the FJ2-pinned pure build_continuous_state.

get_raw_state(world, cfg=StateConfig(); sigma_stop=nothing)
    -> (RawState, lane_fallback)

state.py::get_raw_state: extract the 7-component tabular state from the latent world. Returns the state together with the updated lane-fallback memory (env._mdp_last_lane_position in Python — updated on a successful lane query, left unchanged on NotInLane); the caller threads it back into the next DuckieWorldState. sigma_stop === nothing reads world.stop_memory.sigma_stop (Python env._mdp_sigma_stop).

lane_frame_continuous(world) -> (forward, right)

continuous_state.py::_lane_frame: the SAC/TD3 variant — forward normalized with the 1e-12 threshold (zeroed below it) and right normalized too.

lane_frame_tabular(world) -> (forward, right)

state.py::_lane_frame: forward/right axes of the ego's lane frame used by the tabular extraction. right is NOT normalized (np.cross of a unit forward with ŷ is unit anyway, but the bits are the raw cross product).

next_stop_candidate(world, cfg) -> (distance, index)

state.py::next_stop_candidate: nearest stop sign ahead of and facing the ego, filtered by lateral offset and orientation. Returns (nothing, nothing) when no candidate passes; index is the 0-based position within world.stop_signs (the reference index runs over env.objects, but only its identity across decisions matters — the StopTracker uses it solely for change detection).

signed_curvature_ahead(world, state_cfg, continuous_cfg) -> Float64

continuous_state.py::signed_curvature_ahead: signed curvature of the directed lane at the look-ahead tile, clipped to ±max_abs_curvature; 0.0 when no drivable tile or no curves exist (unlike the tabular tile_ahead, this never raises).

tile_ahead(world, cfg) -> TileType

state.py::tile_ahead: curvature class at the look-ahead probe point (pos + tile_lookahead * forward), falling back to the current tile when the probe is off-map/non-drivable. Raises ArgumentError (Python ValueError) when no tile exists at all — get_raw_state catches it as STRAIGHT.

_normalize_gt0(vector) -> Vector{Float64}

src/state.py::_normalize: divide by the Euclidean norm if it is > 0, else return the vector unchanged (unlike the 1e-12 variant used in continuous_state.py, which zeroes the vector instead).

classify_tile(drivable, kind) -> TileType

Classify a tile by its map kind (src/state.py::classify_tile): any straight/3way*/4way tile is STRAIGHT; curve_left/curve_right map directly; anything else (or a non-drivable tile) raises ArgumentError (Python: ValueError). kind is compared case-insensitively.

ego_relative_curve(curves, forward, threshold) -> TileType

Turn a tile's directed Bezier curves into the kappa_t class correct for the ego's approach (src/state.py::_ego_relative_curve). A curve_left tile can carry two lanes turning in opposite directions; the sign of the y-component of the cross product of the tangent at t = 0.10 and t = 0.90 of the directed curve (the one best aligned with forward) decides left vs right.

terminal_lane_fallback(last_d, last_phi) -> (d, phi)

Fallback lane position once the ego leaves the lane (src/state.py::_terminal_lane_fallback): a 0.25-offset d and π/2 heading error, with signs taken from the last valid lane position stored by get_raw_state (_mdp_last_lane_position, default (1.0, 1.0)); a zero-valued reference falls back to sign +1 via Python's or 1.0.

DuckThreat

Pedestrian threat class reported by classify_duck (Python src/state.py). Values match the Python enum integers.

RawState

7-component lane-relative state (d, phi, v, tile, d_stop, sigma_stop, duck) extracted from the latent simulator state (src/state.py::get_raw_state).

• d: signed lane offset, clipped to ±0.25 m (right of lane negative).
• phi: signed heading error, clipped to ±π/2 (right of tangent negative).
• v: max(0, ego speed) m/s.
• tile: curvature class of the directed look-ahead tile.
• d_stop: distance to the nearest valid stop line (ahead - sign_to_line_offset, clipped at 0), or nothing when no sign passes the orientation/lateral filters.
• sigma_stop: stop-compliance memory bit.
• duck: maximum threat over visible duckies in the forward corridor.

StateConfig

Parameters of the 7-component raw-state extraction. Defaults mirror src/state.py::StateConfig exactly (including duck_max_distance = 2.0 and duck_corridor_width = 0.35, which the four experiment YAMLs override to 1.20/0.60).

TileType

Curvature class of the directed lane ahead of the ego, as computed by _ego_relative_curve in the Python src/state.py. Enum values match the Python integer values used in raw_state_to_dict.