Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
44 changes: 44 additions & 0 deletions quality/coverage/coverage_justifications.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -93,3 +93,47 @@ justifications:
Coverage tool incorrectly marks this line as not covered. Lines immediately before and after are covered.
Covered by the test BidirectionalTransportSocketFixture.DispatchThreadDeliversQueuedMessageToHandler
inside score/mw/com/gateway/transport_layer/sample/bidirectional_transport_test.cpp

- id: ivshmem-bind-shm-to-bar-shm-ctl-special-success
category: defensive_programming
platforms: [qnx]
reason: >
QNX x86_64 only: shm_ctl_special() needs a real ivshmem device and WB-backed shared memory,
so the mock-fd test path always hits EBADF. Fallback to standard shm_ctl() on failure.
locations:
- file: score/mw/com/gateway/transport_layer/qemu/ivshmem/ivshmem_typed_memory_provider.cpp
line_start: 321
line_end: 329

- id: ivshmem-write-directory-entry-idempotent
category: defensive_programming
platforms: [qnx]
reason: >
Idempotent guard in WriteDirectoryEntry() requires multi-VM concurrent write race;
single-process tests cache allocations on first write, preventing re-entry.
locations:
- file: score/mw/com/gateway/transport_layer/qemu/ivshmem/ivshmem_typed_memory_provider.cpp
line_start: 222
line_end: 230

- id: ivshmem-bar-discovery-hardware-path
category: defensive_programming
platforms: [qnx]
reason: >
QNX PCI API calls require real ivshmem device. Tests with no hardware hit pci_device_find→PCI_BDF_NONE
and cannot mock PCI subsystem APIs.
locations:
- file: score/mw/com/gateway/transport_layer/qemu/ivshmem/ivshmem_bar_discovery.cpp
line_start: 45
line_end: 133

- id: qemu-worldwritable-ctor-gcc-artifact
category: defensive_programming
platforms: [qnx]
reason: >
GCC/GCOV x86_64 QNX emits separate counter for WorldWritable{} on continuation lines;
QCC omits sequence-point counter on trivial temporaries.
locations:
- file: score/mw/com/gateway/transport_layer/qemu/qemu_hypervisor_transport.cpp
line_start: 356
line_end: 357
Original file line number Diff line number Diff line change
Expand Up @@ -78,7 +78,7 @@ def __init__(
intervm=None,
vm_index=0,
max_boot_attempts=3,
boot_timeout=100,
boot_timeout=180,
):
super().__init__(path_to_qemu_image, available_ram, available_cores, port_forwarding=port_forwarding)
# Replace the base's default Qemu with our ivshmem-capable subclass.
Expand Down Expand Up @@ -119,6 +119,9 @@ def start(self):
self.stop()
except Exception: # pylint: disable=broad-except
logger.exception("Failed to stop the wedged QEMU before retrying")
if attempt < self._max_boot_attempts:
logger.info("Waiting 5 s before next boot attempt to let resources settle")
time.sleep(5)
raise RuntimeError(
f"VM never booted into a usable state after {self._max_boot_attempts} attempts: {last_error}"
)
Expand Down
5 changes: 4 additions & 1 deletion quality/integration_testing/integration_testing.bzl
Original file line number Diff line number Diff line change
Expand Up @@ -219,8 +219,11 @@ def dual_qemu_integration_test(
# Two VMs require even more resources than a single one.
if "size" not in kwargs:
kwargs["size"] = "enormous"

# Dual-QEMU boots two guests and can fall back to slower TCG in CI, so give
# it more wall-clock budget than a single-VM integration test.
if "timeout" not in kwargs:
kwargs["timeout"] = "moderate"
kwargs["timeout"] = "long"

# Driving two real QNX guests under KVM has rare, environment-induced boot
# nondeterminism (e.g. a guest occasionally wedging during device bring-up).
Expand Down
8 changes: 5 additions & 3 deletions score/mw/com/gateway/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -78,10 +78,12 @@ files in its domain:
2. The transport-layer config file referenced by the `config-path` field of the `transport-layer` section in
`mw_com_gateway_config.json`. Its content is transport-specific and is parsed during
`GatewayApplication::Setup()` by the selected transport (via `TransportFactory::Create()`). For the bundled
`sample_hypervisor` transport this is the hypervisor-socket config (remote IP, local/remote ports, request
timeout) — see its
`sample_hypervisor` and `qemu_hypervisor` transports this is the hypervisor-socket config (remote IP,
local/remote ports, request timeout) — see its
[schema](transport_layer/sample/configuration/mw_com_gateway_sample_transport_config_schema.json) and
[example](transport_layer/sample/configuration/example/mw_com_gateway_sample_transport_config.json).
[example](transport_layer/sample/configuration/example/mw_com_gateway_sample_transport_config.json). For the
QEMU transport, the same file also carries the optional `ivshmem.preferred-bar-num` setting (default BAR2)
used when mapping the shared-memory BAR.
3. `mw_com_config.json` — standard `mw::com` deployment config (provided as for all `mw::com` applications).
Required by the `mw::com` runtime to resolve the `InstanceSpecifier`s referenced in
`mw_com_gateway_config.json` into concrete deployment info (binding, number of slots, ASIL level, allowed
Expand Down
4 changes: 1 addition & 3 deletions score/mw/com/gateway/gateway_application/BUILD
Original file line number Diff line number Diff line change
Expand Up @@ -30,9 +30,7 @@ cc_library(
"@score_baselibs//score/language/futurecpp",
"@score_baselibs//score/mw/log",
],
visibility = [
"//score/mw/com/gateway:__subpackages__",
],
visibility = ["//score/mw/com/gateway:__subpackages__"],
deps = [
":gateway_core",
"//score/mw/com/gateway/gateway_application/configuration:gateway_configuration",
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,7 @@ cc_library(
srcs = ["gateway_configuration.cpp"],
hdrs = ["gateway_configuration.h"],
features = COMPILER_WARNING_FEATURES,
visibility = ["//score/mw/com/gateway:__subpackages__"],
deps = [
"@score_baselibs//score/language/futurecpp",
],
Expand All @@ -50,6 +51,7 @@ cc_library(
implementation_deps = [
"@score_baselibs//score/mw/log",
],
visibility = ["//score/mw/com/gateway:__subpackages__"],
deps = [
":gateway_configuration",
"@score_baselibs//score/json",
Expand Down
5 changes: 5 additions & 0 deletions score/mw/com/gateway/transport_layer/BUILD
Original file line number Diff line number Diff line change
Expand Up @@ -56,7 +56,12 @@ cc_library(
],
deps = [
":transport",
"//score/memory/shared",
"//score/mw/com/gateway/gateway_application:gateway_core",
"//score/mw/com/gateway/transport_layer/qemu:qemu_hypervisor_transport",
"//score/mw/com/gateway/transport_layer/qemu/configuration:qemu_transport_config_parser",
"//score/mw/com/gateway/transport_layer/qemu/ivshmem:ivshmem_bar_discovery",
"//score/mw/com/gateway/transport_layer/qemu/ivshmem:ivshmem_typed_memory_provider",
"//score/mw/com/gateway/transport_layer/sample:bidirectional_transport",
"//score/mw/com/gateway/transport_layer/sample:sample_hypervisor_transport",
"//score/mw/com/gateway/transport_layer/sample/configuration:hypervisor_socket_configuration",
Expand Down
53 changes: 53 additions & 0 deletions score/mw/com/gateway/transport_layer/qemu/BUILD
Original file line number Diff line number Diff line change
@@ -0,0 +1,53 @@
# *******************************************************************************
# Copyright (c) 2026 Contributors to the Eclipse Foundation
#
# See the NOTICE file(s) distributed with this work for additional
# information regarding copyright ownership.
#
# This program and the accompanying materials are made available under the
# terms of the Apache License Version 2.0 which is available at
# https://www.apache.org/licenses/LICENSE-2.0
#
# SPDX-License-Identifier: Apache-2.0
# *******************************************************************************
load("@rules_cc//cc:defs.bzl", "cc_library")
load("@score_baselibs//score/language/safecpp:toolchain_features.bzl", "COMPILER_WARNING_FEATURES")
load("//quality/unit_testing:unit_testing.bzl", "cc_unit_test")

cc_library(
name = "qemu_hypervisor_transport",
srcs = ["qemu_hypervisor_transport.cpp"],
hdrs = ["qemu_hypervisor_transport.h"],
features = COMPILER_WARNING_FEATURES + [
"aborts_upon_exception",
],
visibility = ["//score/mw/com/gateway:__subpackages__"],
deps = [
"//score/memory/shared",
"//score/mw/com/gateway/gateway_application:gateway_core",
"//score/mw/com/gateway/transport_layer:transport",
"//score/mw/com/gateway/transport_layer/qemu/ivshmem:ivshmem_typed_memory_provider",
"//score/mw/com/gateway/transport_layer/sample:i_bidirectional_transport",
"//score/mw/com/gateway/transport_layer/sample:sample_hypervisor_transport",
"//score/mw/com/gateway/transport_layer/sample/messages:gateway_messages",
"@score_baselibs//score/mw/log",
],
)

cc_unit_test(
name = "qemu_hypervisor_transport_test",
srcs = ["qemu_hypervisor_transport_test.cpp"],
features = COMPILER_WARNING_FEATURES,
deps = [
":qemu_hypervisor_transport",
"//score/mw/com/gateway/gateway_application:gateway_core_mock",
"//score/mw/com/gateway/transport_layer:transport_error",
"//score/mw/com/gateway/transport_layer/qemu/ivshmem:ivshmem_typed_memory_provider_mock",
"//score/mw/com/gateway/transport_layer/sample:bidirectional_transport_mock",
"@score_baselibs//score/mw/log",
"@score_baselibs//score/mw/log:recorder_mock",
] + select({
"@platforms//os:qnx": ["@score_baselibs//score/os/mocklib/qnx:mman_mock"],
"//conditions:default": [],
}),
)
57 changes: 57 additions & 0 deletions score/mw/com/gateway/transport_layer/qemu/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,57 @@
<!--
*******************************************************************************
Copyright (c) 2026 Contributors to the Eclipse Foundation

See the NOTICE file(s) distributed with this work for additional
information regarding copyright ownership.

This program and the accompanying materials are made available under the
terms of the Apache License Version 2.0 which is available at
https://www.apache.org/licenses/LICENSE-2.0

SPDX-License-Identifier: Apache-2.0
*******************************************************************************
-->

# QEMU Transport Layer

Inter-VM communication and shared memory exchange for LoLa gateway using **ivshmem** (Inter-VM Shared Memory BAR) for zero-copy data transfer between QEMU virtual machines. Extends the [sample transport layer](../sample/README.md) with ivshmem-based memory sharing.

## What It Provides

- **Message Communication** — Service provisioning and lifecycle events via TCP over QEMU's intervm network (reuses sample transport layer)
- **Shared Memory** — Zero-copy data exchange through ivshmem BAR mapping
- **Cross-VM Coordination** — Shared directory for consistent memory region access

## Main Components

- **QemuHypervisorTransport** — Orchestrates message and memory layers
- **IvshmemTypedMemoryProvider** — Manages named shared memory backed by ivshmem BAR

## Configuration

See [`configuration/example/mw_com_gateway_qemu_transport_config.json`](./configuration/example/mw_com_gateway_qemu_transport_config.json)

The config contains the usual `hypervisor-socket` block plus an optional `ivshmem` block.
`ivshmem.preferred-bar-num` selects which PCI BAR to map for the shared-memory region and
defaults to BAR2 for QEMU `ivshmem-plain`.

## How It Works

**Offering a service (Source VM):**
- Allocate CTRL and DATA shared memory regions via ivshmem BAR ([`IvshmemTypedMemoryProvider::AllocateNamedTypedMemory`](./ivshmem/ivshmem_typed_memory_provider.h))
- Register allocation offsets in the shared directory (visible to both VMs)
- Send service availability notification to remote gateway over TCP
- Remote VMs can now discover and map to the same physical memory regions

**Consuming a service (Destination VM):**
- Receive service notification and look up memory offsets in the shared directory
- Bind local shared memory objects to those discovered offsets in ivshmem BAR ([`IvshmemTypedMemoryProvider::AllocateNamedTypedMemoryAtOffset`](./ivshmem/ivshmem_typed_memory_provider.h))
- Both VMs now access the exact same physical memory

See [`QemuHypervisorTransport`](./qemu_hypervisor_transport.h) for the orchestration logic.

## Supported Platforms

- **QNX 7.1+** — Full ivshmem support
- **Linux** — Message layer only
51 changes: 51 additions & 0 deletions score/mw/com/gateway/transport_layer/qemu/configuration/BUILD
Original file line number Diff line number Diff line change
@@ -0,0 +1,51 @@
# *******************************************************************************
# Copyright (c) 2026 Contributors to the Eclipse Foundation
#
# See the NOTICE file(s) distributed with this work for additional
# information regarding copyright ownership.
#
# This program and the accompanying materials are made available under the
# terms of the Apache License Version 2.0 which is available at
# https://www.apache.org/licenses/LICENSE-2.0
#
# SPDX-License-Identifier: Apache-2.0
# *******************************************************************************

load("@rules_cc//cc:defs.bzl", "cc_library")
load("@score_baselibs//score/language/safecpp:toolchain_features.bzl", "COMPILER_WARNING_FEATURES")
load("//quality/unit_testing:unit_testing.bzl", "cc_unit_test")

package(
default_visibility = ["//score/mw/com/gateway/transport_layer/qemu:__subpackages__"],
)

exports_files(["example/mw_com_gateway_qemu_transport_config.json"])

cc_library(
name = "qemu_transport_config_parser",
srcs = ["qemu_transport_config_parser.cpp"],
hdrs = [
"qemu_transport_config_parser.h",
"qemu_transport_configuration.h",
],
features = COMPILER_WARNING_FEATURES,
implementation_deps = [
"@score_baselibs//score/mw/log",
],
visibility = ["//score/mw/com/gateway/transport_layer:__subpackages__"],
deps = [
"//score/mw/com/gateway/transport_layer/qemu/ivshmem:ivshmem_bar_discovery",
"@score_baselibs//score/json",
"@score_baselibs//score/language/futurecpp",
],
)

cc_unit_test(
name = "qemu_transport_config_parser_test",
srcs = ["qemu_transport_config_parser_test.cpp"],
deps = [
":qemu_transport_config_parser",
"@googletest//:gtest",
"@score_baselibs//score/json",
],
)
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
{
"hypervisor-socket": {
"remote-ip": "10.0.2.2",
"local-port": 45001,
"remote-port": 45002,
"request-timeout-ms": 5000
},
"ivshmem": {
"preferred-bar-num": 2
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,84 @@
/********************************************************************************
* Copyright (c) 2026 Contributors to the Eclipse Foundation
*
* See the NOTICE file(s) distributed with this work for additional
* information regarding copyright ownership.
*
* This program and the accompanying materials are made available under the
* terms of the Apache License Version 2.0 which is available at
* https://www.apache.org/licenses/LICENSE-2.0
*
* SPDX-License-Identifier: Apache-2.0
********************************************************************************/
#include "score/mw/com/gateway/transport_layer/qemu/configuration/qemu_transport_config_parser.h"

#include "score/json/json_parser.h"
#include "score/mw/log/logging.h"

#include <score/assert.hpp>

#include <string_view>

namespace score::mw::com::gateway::qemu
{
namespace
{

using std::string_view_literals::operator""sv;

constexpr auto kIvshmemConfigurationKey = "ivshmem"sv;
constexpr auto kPreferredBarNumKey = "preferred-bar-num"sv;

} // namespace

auto ParseQemuTransportConfig(const std::string_view path) noexcept -> QemuTransportConfiguration
{
const score::json::JsonParser json_parser_obj;
// NOLINTNEXTLINE(score-banned-function): AoU of score::json::JsonParser — caller must guarantee path integrity.
auto json_result = json_parser_obj.FromFile(path);
if (!json_result.has_value())
{
::score::mw::log::LogFatal("lola")
<< "Parsing qemu transport config file" << path << "failed with error:" << json_result.error().Message()
<< ": " << json_result.error().UserMessage() << " . Terminating.";
std::terminate();
}
return ParseQemuTransportConfig(std::move(json_result).value());
}

auto ParseQemuTransportConfig(score::json::Any json) noexcept -> QemuTransportConfiguration
{
auto top_level_object = json.As<score::json::Object>();
if (!top_level_object.has_value())
{
::score::mw::log::LogFatal("lola")
<< "Parsing qemu transport configuration failed: Expected top-level JSON object. Terminating.";
std::terminate();
}

const auto& obj = top_level_object.value().get();

std::uint32_t preferred_bar_num = ivshmem::kDefaultIvshmemBarNum;
const auto ivshmem_entry = obj.find(kIvshmemConfigurationKey);
if (ivshmem_entry != obj.cend())
{
const auto ivshmem_obj_result = ivshmem_entry->second.As<score::json::Object>();
if (ivshmem_obj_result.has_value())
{
const auto& ivshmem_obj = ivshmem_obj_result.value().get();
const auto preferred_bar_num_entry = ivshmem_obj.find(kPreferredBarNumKey);
if (preferred_bar_num_entry != ivshmem_obj.cend())
{
const auto preferred_bar_num_result = preferred_bar_num_entry->second.As<std::uint32_t>();
if (preferred_bar_num_result.has_value())
{
preferred_bar_num = preferred_bar_num_result.value();
}
}
}
}

return QemuTransportConfiguration{preferred_bar_num};
}

} // namespace score::mw::com::gateway::qemu
Loading
Loading