Feature/key simplification by 777arc · Pull Request #150 · sigmf/sigmf-python

777arc · 2026-04-29T22:12:02Z

Going to stick AI on it

- add sigmf/keys.py with all core: field key constants and file extension constants - rename 8 abbreviated keys to match their core: string values (e.g. FLO_KEY -> FREQ_LOWER_EDGE_KEY, HASH_KEY -> SHA512_KEY) - export all key constants at sigmf.* module level via from .keys import * - deprecate old names via metaclass __getattr__ with DeprecationWarning - update all internal modules (sigmffile, siggen, archivereader, validate, converters) to reference keys directly - archive.py now imports extension constants from keys instead of defining them - add TestDeprecatedKeyAliases test class - bump version 1.10.0 -> 1.11.0

Copilot

Pull request overview

This PR refactors SigMF metadata key constants into a dedicated sigmf.keys module (re-exported at sigmf.*), introduces deprecation warnings for class-level key access, and expands archive support to include compressed SigMF archives plus a new sigmf.tofile() convenience writer.

Changes:

Add sigmf.keys as the canonical location for key/extension constants, and deprecate SigMFFile.*_KEY/old alias names with warnings.
Add compressed archive read/write support (.sigmf.gz, .sigmf.xz, .sigmf.zip) and update archive APIs to accept a compression argument.
Add sigmf.tofile() convenience function and update tests/docs to use the new key constants and archive behaviors.

Reviewed changes

Copilot reviewed 25 out of 25 changed files in this pull request and generated 8 comments.

Show a summary per file

File	Description
tests/testdata.py	Update test fixtures to use module-level key constants.
tests/test_validation.py	Update validation tests for renamed key constants.
tests/test_sigmffile.py	Add tests for `sigmf.tofile()` and deprecation behavior; update key usage.
tests/test_siggen.py	Update generator tests to use module-level key constants.
tests/test_ncd.py	Update NCD tests for renamed hash/index keys.
tests/test_hashing.py	Update hashing tests to use `SHA512_KEY`.
tests/test_collection.py	Add `sigmf` import (currently unused).
tests/test_attributes.py	Update attribute-access tests to use module-level key constants.
tests/test_archivereader.py	Update archive reader tests for key constant refactor.
tests/test_archive.py	Add compressed archive roundtrip/behavior tests and memmap assertion.
tests/conftest.py	Add `sigmf` import for test environment consistency.
sigmf/validate.py	Switch validator ordering/extension checks to new key constants.
sigmf/sigmffile.py	Core refactor: deprecating key access, key list centralization, archive/tofile API updates, add `sigmf.tofile()`.
sigmf/siggen.py	Update generator metadata/annotation building to use `keys.*`.
sigmf/keys.py	New module defining key constants, valid key lists, archive extensions, and deprecated alias mapping.
sigmf/hashing.py	Extend hashing API with `offset`/`size` parameters (needs fixes).
sigmf/convert/wav.py	Converter updated to use `keys.*` constants.
sigmf/convert/signalhound.py	Converter updated to use `keys.*` constants.
sigmf/convert/blue.py	Converter updated to use `keys.*` constants.
sigmf/archivereader.py	Add zip + compressed tar support; add memmap path for uncompressed tar.
sigmf/archive.py	Add compressed archive writing + zip support; add extension/compression resolution helpers.
sigmf/init.py	Export keys at module level; bump version; export `tofile`.
docs/source/quickstart.rst	Document `sigmf.tofile()` and update frequency-edge key names.
docs/source/advanced.rst	Update docs for renamed keys and add compressed archive documentation.
README.md	Document reading compressed archives and writing with `sigmf.tofile()`.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2026-04-29T22:17:55Z

+        if data_buffer is None:
+            raise SigMFFileError("No .sigmf-data file found in archive!")
+        return json_contents, data_buffer, data_size_bytes


_read_tar_obj() only errors if the .sigmf-data member is missing. If the .sigmf-meta member is missing, json_contents remains None and later initialization fails with a less clear exception. Raise SigMFFileError when json_contents is still None after scanning the archive.

Copilot · 2026-04-29T22:17:55Z

+                raw = zf.read(member_name)
+                data_size_bytes = len(raw)
+                data_buffer = io.BytesIO(raw)
+


_read_zip_obj() only errors if the .sigmf-data member is missing. If the .sigmf-meta member is missing, json_contents remains None and later initialization fails with a less clear exception. Raise SigMFFileError when json_contents is still None after scanning the archive.

Suggested change

if json_contents is None:

raise SigMFFileError("No .sigmf-meta file found in archive!")

Copilot · 2026-04-29T22:17:55Z

@@ -1,3 +1,4 @@
+import sigmf


Unused import sigmf was added here but this module doesn’t reference sigmf anywhere. Dropping it avoids unnecessary imports and keeps the test file clean.

Suggested change

import sigmf

Copilot · 2026-04-29T22:17:56Z

            self.sigmffile.dump(handle)
        if isinstance(self.sigmffile.data_buffer, io.BytesIO):
-            # write data buffer to archive
            self.sigmffile.data_file = data_path


This mutates the caller’s SigMFFile (self.sigmffile.data_file = data_path) when archiving from an in-memory buffer, but the temp directory is removed at the end of __init__. That leaves the original object pointing at a non-existent data_file. Avoid mutating the passed-in object (use a local variable or restore the original data_file).

Suggested change

self.sigmffile.data_file = data_path

Copilot · 2026-04-29T22:17:56Z

+        SigMFFile.FREQ_LOWER_EDGE_KEY: 914995000.0,
+        SigMFFile.FREQ_UPPER_EDGE_KEY: 915005000.0,
        SigMFFile.COMMENT_KEY: "example annotation",


These docs still use SigMFFile.*_KEY constants in examples. With the new deprecation descriptors, accessing key constants via SigMFFile emits DeprecationWarning; examples should prefer module-level constants (e.g., sigmf.FREQ_LOWER_EDGE_KEY / sigmf.SAMPLE_START_KEY) to avoid teaching the deprecated API.

Copilot · 2026-04-29T22:17:56Z

+        annotation_start_idx = annotation[SigMFFile.SAMPLE_START_KEY]
+        annotation_length = annotation[SigMFFile.SAMPLE_COUNT_KEY]
        annotation_comment = annotation.get(SigMFFile.COMMENT_KEY, "[annotation {}]".format(adx))


The examples use SigMFFile.SAMPLE_START_KEY / SigMFFile.SAMPLE_COUNT_KEY. Since class-level key access now emits DeprecationWarning, the docs should use module-level constants (e.g., sigmf.SAMPLE_START_KEY, sigmf.SAMPLE_COUNT_KEY) so copied code doesn’t start warning immediately.

Copilot · 2026-04-29T22:17:56Z

    if filename is not None:
        fileobj = open(filename, "rb")
-        bytes_to_hash = Path(filename).stat().st_size
+        if size is not None:
+            bytes_to_hash = size
+        else:
+            bytes_to_hash = Path(filename).stat().st_size
+        fileobj.seek(offset)
    elif fileobj is not None:


When offset is non-zero and size is None, bytes_to_hash should be based on the remaining bytes (e.g., file_size - offset), otherwise the loop will read to EOF and then read() returns b'' repeatedly, causing an infinite loop. Adjust bytes_to_hash and/or add an explicit EOF guard in the loop.

Copilot · 2026-04-29T22:17:57Z

+    if toarchive:
+        # create archive only — no loose files
+        meta.archive(str(fns["base_fn"]), compression=compression)
+        data_path.unlink()


In the toarchive path, the function unlinks the just-written *.sigmf-data file but still returns the original SigMFFile instance that was constructed from that path. This can make the returned object unusable (and may fail on Windows if the file is still open/mem-mapped). Consider returning a SigMFFile reopened from the archive, or avoid deleting the file until the returned object no longer depends on it.

Suggested change

data_path.unlink()

data_path.unlink()

return fromarchive(str(fns["archive_fn"]))

Teque5 added 5 commits April 24, 2026 20:51

allow compression extensions

5b1992d

add new method to allow easy sigmf write directly from numpy array

f13847e

increment to v1.10.0

c0147fc

deprecate class-level key access; warn once per session

0d31967

777arc requested a review from Copilot April 29, 2026 22:12

Copilot started reviewing on behalf of 777arc April 29, 2026 22:12 View session

Merge branch 'main' into feature/key-simplification

aae5529

777arc marked this pull request as ready for review April 29, 2026 22:16

777arc marked this pull request as draft April 29, 2026 22:16

Copilot AI reviewed Apr 29, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Feature/key simplification#150

Feature/key simplification#150
777arc wants to merge 6 commits intomainfrom
feature/key-simplification

777arc commented Apr 29, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI Apr 29, 2026

Uh oh!

Copilot AI Apr 29, 2026

Uh oh!

Copilot AI Apr 29, 2026

Uh oh!

Copilot AI Apr 29, 2026

Uh oh!

Copilot AI Apr 29, 2026

Uh oh!

Copilot AI Apr 29, 2026

Uh oh!

Copilot AI Apr 29, 2026

Uh oh!

Copilot AI Apr 29, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants



	if json_contents is None:
	raise SigMFFileError("No .sigmf-meta file found in archive!")

	data_path.unlink()
	data_path.unlink()
	return fromarchive(str(fns["archive_fn"]))

Conversation

777arc commented Apr 29, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Copilot AI Apr 29, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 29, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 29, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 29, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 29, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 29, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 29, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 29, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants