Skip to content

feat(observability): Phase-0 baseline-observability package + real-run validation - #60

Open
yueqin22 wants to merge 3 commits into
makerspet:mainfrom
yueqin22:feat/phase0-baseline-observability
Open

feat(observability): Phase-0 baseline-observability package + real-run validation#60
yueqin22 wants to merge 3 commits into
makerspet:mainfrom
yueqin22:feat/phase0-baseline-observability

Conversation

@yueqin22

Copy link
Copy Markdown

Summary

Adds a Phase-0 ROS 2 (ament_python) observability package for the OOMWOO MVP baseline, so that reproducible-run metadata and the §5.2 KPIs can be captured and reported without hand-rolled scripts.

  • metrics.py — framework-agnostic KPI aggregation (LiDAR rate, TF latency quantiles, odometry drift, cmd_vel staleness, bumper events, map free ratio, recovery latency).
  • metrics_collector.py — real rclpy node subscribing to the baseline topic surface (/scan /odom /tf /cmd_vel /map /bumper_* /oomwoo_status); degrades gracefully when ros_gz_interfaces is not installed (bumper KPIs disabled until ros-humble-ros-gz-interfaces is present).
  • run_metadata.py — captures ROS_DISTRO/RMW/Git SHA/seed/scenario into run_metadata.json.
  • scenario_registry.py + config/scenario_registry.yaml — the §5.1 eight baseline scenarios.
  • launch/baseline_record.launch.py, scripts/baseline_report.py (dict + CLI), scripts/system_sampler.py (Linux /proc), scripts/run_experiment.sh, scripts/demo_publisher.py + scripts/demo_run.sh for verification without a Gazebo robot.
  • test/test_metrics.py (10 logic) + test/test_node_smoke.py (2 node) = 12 tests.

Verification (WSL + ROS 2 Humble)

  • colcon build OK; colcon test 12/12 pass.
  • metrics_collector runs as a real node fed by synthetic /scan /odom /tf /cmd_vel /map @5Hz; produces DEMO_BASELINE_REPORT.md (scan 5.00 Hz meets target, odom drift ≈1.0, TF p95 ≈15 ms).
  • Three node-only bugs caught by the real run and fixed: wrong QoS constant (qos_profile_sensor_data), record_cmd_vel not counting samples, and ros_gz_interfaces import crash → graceful degradation.

Notes / follow-ups

  • bumper KPIs need sudo apt-get install ros-humble-ros-gz-interfaces (auto-enabled once present).
  • Full Gazebo robot run (EKF+SLAM, boustrophedon, recovery) is a later phase; this PR delivers the baseline-observability foundation only.

See jihua20260826.md §7/§9 for the plan and execution log.

veni and others added 3 commits August 27, 2026 13:15
…OMWOO

Add a ROS 2 (ament_python) package that captures reproducible-run metadata
and Phase-0 KPIs (LiDAR rate, TF latency, odometry drift, cmd_vel staleness,
bumper events, map free ratio, recovery latency) from the MVP baseline topic
surface, plus a markdown report generator and a synthetic-topic demo publisher
for local verification without a Gazebo robot.

Verified in WSL + ROS 2 Humble: colcon build OK, colcon test 12/12 pass
(10 logic + 2 node smoke tests), launch file parses, and metrics_collector
runs as a real node fed by synthetic /scan /odom /tf /cmd_vel /map messages.

Includes graceful degradation when ros_gz_interfaces is absent (bumper KPIs
disabled until ros-humble-ros-gz-interfaces is installed). See
scripts/colcon_validate.sh for the one-shot build/test/run validation.

Co-Authored-By: WorkBuddy <workbuddy@tencent.com>
…lling

demo_run.sh previously killed the run_metadata node after a fixed 2s sleep,
which could interrupt rclpy.init() during DDS cold-start and drop the
run-metadata JSON (broken 'cp' at the end). Wait for the one-shot node to
self-exit so the metadata file is always written.
…ive sim

Scaffold for running oomwoo_baseline collector against the official OOMWOO
Gazebo simulation (makerspet/oomwoo:jazzy-dev container + VcXsrv). Builds
the package and launches baseline_record.launch.py with use_sim_time:=true
to record /scan /odom /tf /cmd_vel /bumper_* /map from the running world.
@makers-pet

Copy link
Copy Markdown
Collaborator

Hi @yueqin22 , thank you for the PR!

Would it be possible for you to self-host it - and change the PR to a link to your repo (and a description)?

FYI, my Claude a few things

  • package.xml build-system mismatch (real bug): it declares <build_type>ament_python</build_type> but <buildtool_depend>ament_cmake</buildtool_depend>. An ament_python package shouldn't depend on ament_cmake — that'll confuse the build. (Should be no buildtool_depend, or ament_python.)
  • Placeholder maintainer: oomwoo-baseline oomwoo-contrib@example.com — wants the real contributor identity.
  • rosbag2 undeclared: the description says it "records baseline topics via rosbag2," but there's no rosbag2 exec_depend. If the launch/scripts use it, add the dep.
  • Tested on Humble, declares Jazzy: package.xml says Jazzy (good), but verification was WSL + ROS 2 Humble. Worth a Jazzy build/test, and it assumes a topic surface (/oomwoo/status, /bumper_*) that should be checked against the real robot.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants