test: formalize classic S7 real PLC acceptance testing

.github/ISSUE_TEMPLATE/plc-test-result.yml

@@ -0,0 +1,71 @@
+name: Real PLC test result
+description: Record one reproducible python-snap7 acceptance run against a real PLC
+title: "[hardware test]: "
+labels: ["hardware-test"]
+body:
+  - type: markdown
+    attributes:
+      value: |
+        Attach the generated JSON and JUnit XML. Review all attachments before uploading. Never include PLC addresses, credentials, private keys, plant names, or unreviewed packet captures.
+  - type: input
+    id: source
+    attributes:
+      label: Source revision
+      description: Immutable 40-character commit SHA and, if applicable, release version.
+      placeholder: abcdef0123456789abcdef0123456789abcdef01
+    validations:
+      required: true
+  - type: dropdown
+    id: result
+    attributes:
+      label: Overall result
+      options: [pass, fail, partial]
+    validations:
+      required: true
+  - type: input
+    id: plc
+    attributes:
+      label: PLC family, model, and order code
+      placeholder: S7-1500, CPU 1511-1 PN, 6ES7 ...
+    validations:
+      required: true
+  - type: input
+    id: firmware
+    attributes:
+      label: Firmware and security mode
+      placeholder: V2.9, PUT/GET enabled, non-optimized fixture DBs
+    validations:
+      required: true
+  - type: input
+    id: host
+    attributes:
+      label: Host environment
+      placeholder: Python 3.13, Linux x86_64
+    validations:
+      required: true
+  - type: input
+    id: started
+    attributes:
+      label: Run time (UTC)
+      placeholder: 2026-09-11T12:34:56Z
+    validations:
+      required: true
+  - type: textarea
+    id: artifacts
+    attributes:
+      label: Sanitized report artifacts
+      description: Drag the generated JSON and JUnit XML here. State any manually reviewed capture separately.
+    validations:
+      required: true
+  - type: textarea
+    id: skips
+    attributes:
+      label: Skips, failures, and notes
+      description: Include machine-readable skip reasons and whether this result supersedes an earlier issue.
+  - type: checkboxes
+    id: safety
+    attributes:
+      label: Safety confirmation
+      options:
+        - label: I reviewed the attachments and they contain no PLC address, credentials, private keys, plant/site identifiers, or proprietary PLC program.
+          required: true

CHANGES.md

@@ -6,6 +6,8 @@ CHANGES
 
 Major release: new `s7commplus` package with S7CommPlus protocol support.
 
+* Add a safe, Gherkin-based real-PLC acceptance program with restored scratch
+  writes and sanitized JSON/JUnit reports (#840).
 * Return S7 item errors when pure-Python server reads target unregistered areas
   or ranges outside registered memory instead of fabricating successful data.
 * Echo calling and called TSAP parameters in pure-Python server connection

README.rst

@@ -50,5 +50,8 @@ Documentation
 Read the full documentation at
 `python-snap7.readthedocs.io <https://python-snap7.readthedocs.io/en/latest/>`_.
 
+The safe volunteer hardware-test procedure is in
+`Real-PLC acceptance testing <doc/real-plc-testing.rst>`_.
+
 For native communication with S7-1200 and S7-1500 controllers without PUT/GET,
 see the standalone `s7commplus <https://github.com/gijzelaerr/s7commplus>`_ package.

doc/index.rst

@@ -34,6 +34,7 @@ Welcome to python-snap7's documentation!
    :caption: Development
 
    development
+   real-plc-testing
 
 .. toctree::
    :maxdepth: 2

doc/real-plc-testing.rst

@@ -0,0 +1,87 @@
+Real-PLC acceptance testing
+===========================
+
+The versioned Gherkin specifications in ``tests/features/real_plc`` exercise the
+same client APIs and canonical 37-byte fixture used by the diagnostic end-to-end
+tests. ``pytest-bdd`` was selected because it keeps pytest as the runner, reuses
+fixtures, maps tags to markers, and supports the existing JUnit workflow.
+
+Safety model
+------------
+
+Use only a dedicated, non-production test PLC. The default runner selects
+``@smoke`` and is read-only. It never enables writes or administrative actions.
+Scratch writes require ``--allow-write``, a separate DB (DB2 by default), and are
+wrapped in a guard that saves, restores, and reads back the original bytes from
+fixture teardown even when a step fails or pytest handles an interruption.
+Hard process termination or power loss cannot run cleanup, so inspect the scratch
+DB before reuse after either event.
+
+Administrative actions (CPU start/stop, clock changes, program transfers, or
+protection changes) require ``--allow-admin`` and must never target an in-service
+PLC. No administrative scenarios are included in the initial suite.
+
+PLC preparation
+---------------
+
+Import ``tests/plc_setup/e2e_test_dbs.scl`` into TIA Portal, or recreate its exact
+layout. It defines read-only DB1 and scratch DB2. On S7-1200/1500, disable
+``Optimized block access`` for both fixture DBs. Enable PUT/GET when the classic
+S7 protocol and the PLC security policy require it. Defaults are
+rack 0, slot 1, and TCP port 102. Apply the least privilege that permits the
+selected scenarios; do not weaken a PLC that is in service.
+
+Running and reporting
+---------------------
+
+Install the test extras, then run the safe profile. Values supplied to reportable
+metadata must not contain addresses, credentials, certificate paths, plant names,
+or other site identifiers::
+
+  uv sync --extra test
+  uv run python tools/run_real_plc_acceptance.py \
+    --plc-ip YOUR_PRIVATE_ADDRESS \
+    --tester @YOUR_HANDLE \
+    --plc-family S7-1500 \
+    --plc-model "CPU 1511-1 PN" \
+    --plc-firmware V2.9 \
+    --plc-security-mode "PUT/GET, non-optimized fixture DBs"
+
+The runner prints a Gherkin-aware terminal summary and writes JUnit XML plus a
+schema-versioned, sanitized JSON report beneath ``real-plc-results/``. The JSON
+records exact source state, host environment, PLC metadata, scenario tags,
+pass/fail/skip results, and bounded diagnostics. It deliberately never reads the
+PLC address into the report. Review both artifacts before publishing them.
+
+Add ``--allow-write`` only after confirming DB2 is disposable scratch space.
+``--allow-admin`` remains separate so write permission never implies permission
+for operational actions.
+
+Result policy
+-------------
+
+File one ``Real PLC test result`` issue per tester, PLC configuration, source
+revision, and run. Attach both generated artifacts and apply exactly one result
+label:
+
+* ``hardware-test``
+* ``test-result: pass``, ``test-result: fail``, or ``test-result: partial``
+
+``pass`` means every selected applicable scenario passed. ``fail`` means at least
+one selected scenario failed. ``partial`` means the run completed with capability
+skips or an incomplete selection; unsupported capabilities are not failures.
+Reruns get a new issue and link the earlier result. Close the earlier issue as
+superseded only after the replacement artifacts exist. A result becomes stale
+when the tested code, feature schema, relevant protocol implementation, PLC
+firmware, or PLC configuration changes—not merely with age.
+
+Useful searches:
+
+* `Open hardware failures <https://github.com/gijzelaerr/python-snap7/issues?q=is%3Aissue+is%3Aopen+label%3Ahardware-test+label%3A%22test-result%3A+fail%22>`_
+* Search a release or commit by adding its version or SHA to ``is:issue label:hardware-test``.
+
+For a release candidate, prioritize representative classic S7 PLC families
+before expanding the host matrix; hosted CI already covers supported Python
+versions and operating systems without hardware. Test evidence stays in issues.
+Volunteer coordination may use Discussions, but is not a substitute for an
+attached structured result.

pyproject.toml

@@ -32,7 +32,7 @@ Homepage = "https://github.com/gijzelaerr/python-snap7"
 Documentation = "https://python-snap7.readthedocs.io/en/latest/"
 
 [project.optional-dependencies]
-test = ["pytest", "pytest-asyncio", "pytest-cov", "pytest-html", "hypothesis", "mypy", "types-setuptools", "ruff", "tox", "tox-uv", "types-click", "uv"]
+test = ["pytest", "pytest-asyncio", "pytest-bdd>=8.1,<9", "pytest-cov", "pytest-html", "hypothesis", "mypy", "types-setuptools", "ruff", "tox", "tox-uv", "types-click", "uv"]
 cli = ["rich", "click" ]
 demo = ["psutil", "rich", "click"]
 doc = ["sphinx", "sphinx_rtd_theme"]
@@ -54,6 +54,12 @@ markers =[
     "client",
     "common",
     "e2e: end-to-end tests requiring a real PLC connection",
+    "real_plc: versioned real-PLC acceptance scenario",
+    "smoke: read-only real-PLC acceptance scenario",
+    "write: real-PLC scenario that changes and restores scratch data",
+    "plc_write: legacy alias for real-PLC tests that modify scratch data",
+    "administrative: disruptive real-PLC scenario requiring separate opt-in",
+    "classic_s7: scenario using the classic S7 protocol path",
     "hypothesis: property-based tests using Hypothesis",
     "logo",
     "mainloop",

tests/conftest.py

@@ -2,9 +2,15 @@
 
 import socket
 import sys
+from pathlib import Path
+from typing import Any
 
 import pytest
 
+from tests.real_plc.reporting import RealPLCReport, report_metadata
+
+_REAL_PLC_REPORT = RealPLCReport()
+
 
 def get_free_tcp_port() -> int:
     """Return a TCP port that is free *right now* on 127.0.0.1.
@@ -76,6 +82,26 @@ def pytest_addoption(parser: pytest.Parser) -> None:
         default=2,
         help="Read-write DB number for e2e tests (default: 2)",
     )
+    parser.addoption(
+        "--allow-plc-write",
+        action="store_true",
+        default=False,
+        help="Allow tests that modify and restore the dedicated scratch DB",
+    )
+    parser.addoption(
+        "--allow-plc-admin",
+        action="store_true",
+        default=False,
+        help="Allow disruptive administrative tests on a dedicated non-production PLC",
+    )
+    parser.addoption("--plc-report-json", default="", help="Write a sanitized real-PLC JSON report")
+    parser.addoption("--tester", default="", help="GitHub handle of the volunteer running the test")
+    parser.addoption("--plc-family", default="", help="Reportable PLC family, such as S7-1500")
+    parser.addoption("--plc-model", default="", help="Reportable PLC model")
+    parser.addoption("--plc-order-code", default="", help="Reportable PLC order code")
+    parser.addoption("--plc-firmware", default="", help="Reportable PLC firmware version")
+    parser.addoption("--plc-security-mode", default="", help="Reportable access/TLS mode; never enter credentials")
+    parser.addoption("--plc-tia-configuration", default="", help="Reportable TIA settings, without site details")
 
 
 def pytest_configure(config: pytest.Config) -> None:
@@ -84,6 +110,9 @@ def pytest_configure(config: pytest.Config) -> None:
         "markers",
         "e2e: mark test as end-to-end test requiring real PLC connection",
     )
+    global _REAL_PLC_REPORT
+    _REAL_PLC_REPORT = RealPLCReport()
+    config._real_plc_report = _REAL_PLC_REPORT
 
 
 def pytest_collection_modifyitems(config: pytest.Config, items: list[pytest.Item]) -> None:
@@ -92,8 +121,6 @@ def pytest_collection_modifyitems(config: pytest.Config, items: list[pytest.Item
     for mod_name in [
         "tests.test_client_e2e",
         "test_client_e2e",
-        "tests.test_s7_e2e",
-        "test_s7_e2e",
     ]:
         e2e = sys.modules.get(mod_name)
         if e2e is not None:
@@ -104,14 +131,26 @@ def pytest_collection_modifyitems(config: pytest.Config, items: list[pytest.Item
             e2e.DB_READ_ONLY = int(config.getoption("--plc-db-read"))
             e2e.DB_READ_WRITE = int(config.getoption("--plc-db-write"))
 
-    # Skip e2e tests if flag not provided
-    if config.getoption("--e2e"):
-        return
-
-    skip_e2e = pytest.mark.skip(reason="Need --e2e option to run end-to-end tests")
     for item in items:
-        if "e2e" in item.keywords:
-            item.add_marker(skip_e2e)
+        if "e2e" in item.keywords and not config.getoption("--e2e"):
+            item.add_marker(pytest.mark.skip(reason="Need --e2e option to run end-to-end tests"))
+        if ("write" in item.keywords or "plc_write" in item.keywords) and not config.getoption("--allow-plc-write"):
+            item.add_marker(pytest.mark.skip(reason="WRITE_OPT_IN_REQUIRED: pass --allow-plc-write"))
+        if "administrative" in item.keywords and not config.getoption("--allow-plc-admin"):
+            item.add_marker(pytest.mark.skip(reason="ADMIN_OPT_IN_REQUIRED: pass --allow-plc-admin"))
+
+
+def pytest_runtest_logreport(report: Any) -> None:
+    """Capture final BDD outcomes without recording connection secrets."""
+    _REAL_PLC_REPORT.record(report)
+
+
+def pytest_sessionfinish(session: pytest.Session, exitstatus: int) -> None:
+    """Write the optional structured report after all scenario outcomes are known."""
+    del exitstatus
+    report_path = session.config.getoption("--plc-report-json")
+    if report_path:
+        session.config._real_plc_report.write(Path(report_path), report_metadata(session.config))
 
 
 @pytest.fixture(scope="session")

tests/features/real_plc/connection.feature

@@ -0,0 +1,17 @@
+@real_plc
+Feature: Connect to and identify a real PLC
+
+  Background:
+    Given I am using a dedicated test PLC
+    And the test configuration contains no secrets in reportable fields
+
+  @smoke
+  @classic_s7
+  Scenario: Establish and close a session
+    Given the client uses the classic S7 protocol
+    When I connect to the configured PLC
+    Then the client reports that it is connected
+    And the negotiated protocol and security mode are recorded
+    And the PLC identity and CPU state are recorded when available
+    When I disconnect
+    Then the client reports that it is disconnected

tests/features/real_plc/lifecycle.feature

@@ -0,0 +1,19 @@
+@real_plc
+Feature: Recover from normal connection lifecycle events
+
+  @smoke
+  @classic_s7
+  Scenario: Reconnect after a clean disconnect
+    Given the client uses the classic S7 protocol
+    And I connected to and disconnected from the PLC
+    When I reconnect with the same configuration
+    Then a known read succeeds
+
+  @smoke
+  @classic_s7
+  Scenario: Repeated operations do not corrupt the session
+    Given the client uses the classic S7 protocol
+    And I am connected to the PLC
+    When I read the canonical fixture repeatedly
+    Then every read succeeds with the expected value
+    And disconnect completes cleanly

tests/features/real_plc/read.feature

@@ -0,0 +1,19 @@
+@real_plc
+Feature: Read a known test data block
+
+  @smoke
+  @classic_s7
+  Scenario: Read the canonical fixture
+    Given the client uses the classic S7 protocol
+    And the read-only test DB has the documented canonical layout
+    When I read the complete fixture DB
+    Then INT, REAL, BYTE, WORD, DWORD, DINT, CHAR and BOOL values match the fixture
+    And individual reads return the same values as the complete-block read
+
+  @smoke
+  @classic_s7
+  Scenario: Read multiple values in one request
+    Given the client uses the classic S7 protocol
+    And the read-only test DB has the documented canonical layout
+    When I read values of different sizes in one multi-variable request
+    Then every value and result code matches the fixture

tests/features/real_plc/report.schema.json

@@ -0,0 +1,32 @@
+{
+  "$schema": "https://json-schema.org/draft/2020-12/schema",
+  "$id": "https://github.com/gijzelaerr/python-snap7/blob/master/tests/features/real_plc/report.schema.json",
+  "title": "python-snap7 real-PLC acceptance result",
+  "type": "object",
+  "additionalProperties": false,
+  "required": ["schema_version", "started_at", "ended_at", "overall_result", "source", "environment", "tester", "plc", "scenarios"],
+  "properties": {
+    "schema_version": {"const": "1.0"},
+    "started_at": {"type": "string", "format": "date-time"},
+    "ended_at": {"type": "string", "format": "date-time"},
+    "overall_result": {"enum": ["pass", "fail", "partial"]},
+    "source": {"type": "object"},
+    "environment": {"type": "object"},
+    "tester": {"type": "string"},
+    "plc": {"type": "object"},
+    "scenarios": {
+      "type": "array",
+      "items": {
+        "type": "object",
+        "required": ["scenario_id", "status", "tags", "duration_seconds", "diagnostic"],
+        "properties": {
+          "scenario_id": {"type": "string"},
+          "status": {"enum": ["passed", "failed", "skipped"]},
+          "tags": {"type": "array", "items": {"type": "string"}},
+          "duration_seconds": {"type": "number", "minimum": 0},
+          "diagnostic": {"type": ["string", "null"]}
+        }
+      }
+    }
+  }
+}