Skip to content

python/iot3/mobility: ETSI time fixes, expose more GNSS details, extend support for CAM - #586

Open
ymorin-orange wants to merge 23 commits into
Orange-OpenSource:masterfrom
ymorin-orange:nyma/iot3-gpsd-devices
Open

python/iot3/mobility: ETSI time fixes, expose more GNSS details, extend support for CAM#586
ymorin-orange wants to merge 23 commits into
Orange-OpenSource:masterfrom
ymorin-orange:nyma/iot3-gpsd-devices

Conversation

@ymorin-orange

@ymorin-orange ymorin-orange commented Aug 18, 2026

Copy link
Copy Markdown
Member

Changes

Close #582
Close #583
Close #585
Close #587

Test

How to test

Note: in the following, lines starting with $ are to be executed on your
machine, as a non-root user;
lines starting with (docker)$ are to be executed in the Docker container;
lines starting with (docker)🐍 $ are to be executed in the Docker container,
in the python venv.

  1. Prepare a test environment:
    1. be sure to have unrestricted access to test.mosquitto.org (IPv4 and IPv6)
    2. be sure to have an MQTT broker that listens locally on port 1883,
      with no credentials and no ACL; if not, run your own:
      $ mosquitto
      
    3. in another terminal, prepare a collector implementing
      the OpenTelemetry API, on localhost. If you don't have one,
      you may use an existing one, like:
      $ docker container run \
          --rm \
          -p 16686:16686 \
          -p 4318:4318 \
          jaegertracing/all-in-one:1.58
      
      then open a browser on the Jaegger UI
      (or that of your own collector if you have one):
      http://localhost:16686/
      
    4. in another terminal, start a fake gpsd process
      (no need for an actual GNSS device):
      $ TMPDIR=/tmp gpsfake -q -P 2948 -u -n -c 0.1 tests/data/NMEA.log
      
    5. in another terminal, start a container with Python 3.11
      and the necessary packages:
      $ docker container run \
          --detach \
          --name iot3 \
          --rm \
          -ti \
          --network host \
          -e http_proxy \
          -e https_proxy \
          -e no_proxy \
          --user $(id -u):$(id -u) \
          --mount type=bind,source=$(pwd),destination=$(pwd) \
          --workdir $(pwd) \
          python:3.11.15-slim-trixie \
          /bin/bash -il
      
      $ docker container exec -u 0:0 iot3 apt update
      $ docker container exec -u 0:0 iot3 apt install -y git build-essential curl socat
      $ docker container exec -u 0:0 iot3 install -d -m 0755 /run/zoneinfo
      $ docker container exec -u 0:0 iot3 curl -o /run/zoneinfo/leap-seconds.list 'https://data.iana.org/time-zones/data/leap-seconds.list' 
      $ docker container exec -d iot3 socat UNIX-LISTEN:/tmp/mqtt.socket,fork TCP4:localhost:1883
      
      $ docker container attach iot3
      
      Note: the socat command exposes the MQTT broker listening
      on localhost:1883 (see 1.2., above), inside the
      container listening on the UNIX socket /tmp/mqtt.socket.
    6. prepare a Python 3.11 environment, with tests dependencies
      and packages' dependencies:
      (docker)$ python3.11 -m venv /tmp/venv
      (docker)$ . /tmp/venv/bin/activate
      (docker)🐍 $ pip --disable-pip-version-check --no-cache-dir install \
                      -r python/requirements-tests.txt \
                      $(sed -r -e '/.*"(.+(==|>=|<=).+)".*/!d; s//\1/; s/ //g;' \
                        python/*/pyproject.toml
                       )
      
      Note: we install the packages dependencies manually,
      and do not rely on pip to do so, because our Python packages depend
      one on the others by git hash, as they are not published on PyPi yet,
      so installing one of our packages may overwrite another.
  2. Build the Python packages:
    (docker)🐍 $ for pkg in python/*/; do
                    python -m build -w "${pkg}" || break
                 done
    
  3. Install the Python packages:
    (docker)🐍 $ pip --disable-pip-version-check --no-cache-dir \
                        install --no-deps python/*/dist/*.whl
    
  4. Run the IoT3 Core SDK tests:
    (docker)🐍 ./python/iot3/tests/test-iot3-core
    (docker)🐍 ./python/iot3/tests/test-iot3-core-all
    (docker)🐍 ./python/iot3/tests/test-iot3-core-mqtt
    (docker)🐍 ./python/iot3/tests/test-iot3-core-otel
    (docker)🐍 ./python/iot3/tests/test-iot3-mobility
    (docker)🐍 ./python/iot3/tests/test-iot3-mobility-gnss
    (docker)🐍 ./python/iot3/tests/test-iot3-mobility-message
    
    Note: if test.mosquitto.org is unreachable, skip the test
    ./python/iot3/tests/test-iot3-core-mqtt.
  5. Test CAM emitedby its_vehicle:
    1. start an MQQT client listening to CAM:
      $ mosquitto_sub -t '#' -C 1 -F %J |jq .
      
    2. create a config file:
      (docker)🐍 cat >/tmp/its-vehicle.cfg <<-_EOF_
      [general]
      station-uuid = ora_car_1234
      station-type = passengerCar
      length = 4.335
      width = 2.257
      type = CAM
      depth = 26
      messages = cam cpm denm
      depth-sub-cam = 18
      depth-sub-cpm = 18
      depth-sub-denm = 15
      mirror-self = False
      report-freq = 5.0
      topic-pub-prefix = default/inQueue/v2x/
      topic-sub-prefix = default/outQueue/v2x/
      speed-thresholds = 8.33 19.44 27.78
      
      [broker.main]
      host = localhost
      port = 1883
      tls = false
      
      [gpsd]
      port=2948
      persistence = 1.0
      _EOF_
      
    3. start its-vehicle:
      (docker)🐍 its-vehicle -c /tmp/its-vehicle.cfg
      
    4. when the MQTT client dumps a message, terminate its-vehicle with Ctrl-C

Expected results

  1. The test environment is ready
  2. The packages were all built successfully
  3. The packages were all installed successfully
  4. The tests all succeed (no exception is raised)
  5. The CAM are version 2.4.0; they have dimensions and station type, and the
    station_id is the third part of the source_uuid:
      "tst": "2026-08-20T12:48:14.862572+0200",
      "topic": "default/inQueue/v2x/cam/ora_car_1234/0/3/1/3/1/0/3/1/2/3/2/3/1/3/3/1/3/0/3/3/0/2/0/2/2/2",
      "qos": 0,
      "retain": 0,
      "payloadlen": 853,
      "payload": {
        "message_type": "cam",
        "message_format": "json/raw",
        "source_uuid": "ora_car_1234",
        "timestamp": 1787222894862,
        "version": "2.4.0",
        "message": {
          "protocol_version": 1,
          "station_id": 1234,
          "generation_delta_time": 33942,
          "basic_container": {
            "station_type": 5,
            "reference_position": {
              "latitude": 533613367,
              "longitude": -65056200,
              "altitude": {
                "value": 11690,
                "confidence": 11
              },
              "position_confidence_ellipse": {
                "semi_major": 1957,
                "semi_minor": 1957,
                "semi_major_orientation": 0
              }
            }
          },
          "high_frequency_container": {
            "basic_vehicle_container_high_frequency": {
              "heading": {
                "value": 317,
                "confidence": 127
              },
              "speed": {
                "value": 1,
                "confidence": 127
              },
              "drive_direction": 2,
              "vehicle_length": {
                "value": 43,
                "confidence": 4
              },
              "vehicle_width": 23,
              "longitudinal_acceleration": {
                "value": 161,
                "confidence": 102
              },
              "curvature": {
                "value": 1023,
                "confidence": 7
              },
              "curvature_calculation_mode": 2,
              "yaw_rate": {
                "value": 32767,
                "confidence": 8
              }
            }
          }
        }
      }
    }
    

@ymorin-orange
ymorin-orange requested a review from tigroo August 18, 2026 17:36
@ymorin-orange
ymorin-orange force-pushed the nyma/iot3-gpsd-devices branch 7 times, most recently from ca9761b to 598b9cc Compare August 20, 2026 10:35
When we terminate the GNSS provider, we forcibly close the underlying
filedescriptors, which causes the socket to raise an OSError exception
(rightfully so).

Cat that exception so that we canterminate the loop cleanly.

Signed-off-by: Yann E. MORIN <yann.morin@orange.com>
Every so often, test.mosquitto.org is unreachable, or the broker is not
working, or publishing or subscribing does not work.

For most MQTT tests, we can use the local mosquitto broker (which the
python PR template documents), rather than test.mosquitto.org.

Only for tests that require TLS or websockets, do we need
test.mosquitto.org. If that is not available, document skipping the
test.

Signed-off-by: Yann E. MORIN <yann.morin@orange.com>
Test for unknown altitude already handled in the first CAM test.

Signed-off-by: Yann E. MORIN <yann.morin@orange.com>
When the altitude is unknown (None), it does not make sense to have a
good altitude confidence...

Signed-off-by: Yann E. MORIN <yann.morin@orange.com>
Signed-off-by: Yann E. MORIN <yann.morin@orange.com>
Closes Orange-OpenSource#587

Signed-off-by: Yann E. MORIN <yann.morin@orange.com>
Closes: Orange-OpenSource#585

Signed-off-by: Yann E. MORIN <yann.morin@orange.com>
Signed-off-by: Yann E. MORIN <yann.morin@orange.com>
Signed-off-by: Yann E. MORIN <yann.morin@orange.com>
Closes Orange-OpenSource#582

Signed-off-by: Yann E. MORIN <yann.morin@orange.com>
Closes Orange-OpenSource#583

Signed-off-by: Yann E. MORIN <yann.morin@orange.com>
The StationType name was inherited from the legacy schemas, and ETSI has
renamed it TrafficParticipantType instead.

We already used that in the generic ETSI message, but derived objects
were still using the legacy type.

Switch those over now.

We keep the legacy name for extenal users that may still depend on the
old type, and since they are equivalent, we do not want them to switch
(yet).

Signed-off-by: Yann E. MORIN <yann.morin@orange.com>
Signed-off-by: Yann E. MORIN <yann.morin@orange.com>
It is used as the defualt MQTT client-ID, just before we check it is
set... So better check it right after parsing (and transforming) the
configuration.

Signed-off-by: Yann E. MORIN <yann.morin@orange.com>
No need to iterate on the object, which iterates on all sections, and
exclude the default one; just iterate on sections(), which already
excludes it.

Signed-off-by: Yann E. MORIN <yann.morin@orange.com>
Signed-off-by: Yann E. MORIN <yann.morin@orange.com>
For conciseness and legacy, we call it station-type, which is
what it was named previously, and that the SDK still expects.

Signed-off-by: Yann E. MORIN <yann.morin@orange.com>
Signed-off-by: Yann E. MORIN <yann.morin@orange.com>
Signed-off-by: Yann E. MORIN <yann.morin@orange.com>
Signed-off-by: Yann E. MORIN <yann.morin@orange.com>
Signed-off-by: Yann E. MORIN <yann.morin@orange.com>
Signed-off-by: Yann E. MORIN <yann.morin@orange.com>
Signed-off-by: Yann E. MORIN <yann.morin@orange.com>
@ymorin-orange
ymorin-orange marked this pull request as ready for review August 20, 2026 11:03
@ymorin-orange ymorin-orange self-assigned this Aug 20, 2026
@ymorin-orange ymorin-orange added the Python Python code label Aug 20, 2026
@github-project-automation github-project-automation Bot moved this to Backlog in Kanban Aug 20, 2026
@ymorin-orange

Copy link
Copy Markdown
Member Author

@mathieu1fb Just because you were so curious this morning, here's the PR! ;-)

@ymorin-orange
ymorin-orange force-pushed the nyma/iot3-gpsd-devices branch from 598b9cc to d47dcca Compare August 20, 2026 11:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Python Python code

Projects

Status: Backlog

1 participant