Skip to content

tools/gen_code_table.py uses fragile name-pattern categorization instead of tag data #43

Description

@sumanjeet0012

tools/gen_code_table.py re-derives codec categories (multihash, multiaddr, ipld, etc.) from name patterns instead of using the tag data that is available in the upstream CSV. This duplicates the same fragile logic as _get_tag() and can misclassify codecs.

Problem

In tools/gen_code_table.py (lines 80–165), codecs are categorized using 8 separate pattern lists:

multihash_patterns = ["sha", "keccak", "blake", "md4", "md5", "ripemd", ...]
multiaddr_patterns = ["ip4", "tcp", "udp", "dns", "quic", ...]
ipld_patterns = ["cid", "raw", "dag-", "eth-", "bitcoin", ...]
# ... 5 more pattern lists

Each codec name is tested against these patterns using substring matching:

if any(h in name for h in multihash_patterns):
    categories["multihash"].append(...)
elif any(a in name for a in multiaddr_patterns):
    categories["multiaddr"].append(...)

Problems:

  1. Duplicates _get_tag() logic — same fragile pattern matching in two places
  2. Can misclassify — a codec named "sha2-transport" would match multihash_patterns before transport
  3. Missing categories — codecs in categories like multisig, nonce, vlad, shelter, softhash, multikey all fall into "other" because there are no patterns for them
  4. Maintenance burden — adding a new codec category requires updating pattern lists in TWO files (gen_code_table.py and code.py)

Proposed Solution

  1. Once constants.py includes tag metadata (see related issue), update gen_code_table.py to read the tag directly:

    for name, info in sorted(CODECS.items(), key=lambda x: x[1]["prefix"]):
        tag = info.get("tag", "other")
        categories[tag].append((const_name, info["prefix"], name))
  2. Remove all 8 pattern lists from gen_code_table.py.

  3. Use the tag as the section header in the generated code_table.py:

    # Multihash
    SHA2_256: Code = Code(0x12)  # sha2-256
    ...
    # Multiaddr
    IP4: Code = Code(0x04)  # ip4
    ...

Related

  • Issue: constants.py entries lack tag metadata
  • Issue: Replace hardcoded _get_tag() with data-driven TAG_TABLE
  • Generator: tools/gen_code_table.py

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions