Skip to content

refactor: name the module for the data it fetches, not for the login - #135

Merged
skishchampi merged 2 commits into
masterfrom
refactor/microdata-download-api
Aug 17, 2026
Merged

refactor: name the module for the data it fetches, not for the login#135
skishchampi merged 2 commits into
masterfrom
refactor/microdata-download-api

Conversation

@skishchampi

Copy link
Copy Markdown
Contributor

otp_download_portal named the authentication step. A developer looking for
bulk microdata would never find it. The name also promised a generic OTP driver.

The module is not generic. It hardcodes year ids, report ids, two hosts and one
sentinel. The OTP is one of five traps, not the identity of the module.

Open question for review

This is the third name for this file, and the naming rule is still not settled
here. The rule the repo now follows: the module names the mechanism a developer
must implement; the scheme, the department and the host go in the docstring.

Two readings compete:

  1. Name the artefact and the transportmicrodata_download_api, as in
    this PR.
  2. Split it like aspnet / aspnet_cascade — put the reusable mechanics in
    their own module and keep the portal as data. The reusable half is real: the
    API base is compiled into a JavaScript bundle and must be recovered by
    grepping it, and auth is captcha then OTP then a JWT bearer. Both recur
    across ministry single-page apps. The instance half is the year ids, the
    report ids, the two hosts and the 99 sentinel.

Reviewers: say which, or name a better third.

Verification

1,428 passed, ruff clean. No shim, because the module is unreleased under
either name.

otp_download_portal named the authentication step. A developer looking for bulk
microdata would never find it, and the name promised a generic OTP driver.

The module is not generic. It hardcodes year ids, report ids, two hosts and one
sentinel. The OTP is one of five traps, not the identity of the module.

It is now microdata_download_api. The summary states the artefact and the
transport. The OTP, the captcha and the JavaScript-bundle API base stay in the
docstring, where the context belongs.

No shim. The module is unreleased under either name.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 1226758362

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@@ -1,5 +1,8 @@
# SPDX-License-Identifier: MIT
"""Download bulk microdata from a portal that gates it behind a mobile OTP.
"""Download bulk statistical microdata as per-year CSV archives.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Keep the previous module path importable

Any caller that adopted commoner_probe.otp_download_portal from the preceding feature commit now gets ModuleNotFoundError, because this rename removes that module without the compatibility shim used for every other module rename in this package. Retain an otp_download_portal.py deprecation shim that aliases the new module, and add the mapping to tests/test_module_aliases.py, so consumers do not fail merely by upgrading.

Useful? React with 👍 / 👎.

The rule this repo now follows: the module names the software, and the
docstring names the deployment. geoserver names GeoServer. dspace names DSpace.
nada names the World Bank's National Data Archive.

This portal runs no such product. It is a bespoke build, so the name states the
stack instead: a JWT-authenticated REST API behind an Angular single-page app.

Three names in one day, and none of them shipped. udise named a programme.
otp_download_portal named the login. microdata_download_api named the payload.
Each told a developer nothing about what to implement.

The docstring now leads with the reusable half. The page renders only after
JavaScript runs. The build compiles the API base into one bundle, so a grep for
URL literals finds nothing, and the bundle must be grepped for the variable.
The auth flow is captcha, then OTP, then a bearer token.

The deployment stays below it, as data: the ministry, the two hosts, the year
and report ids, and the all-India sentinel of 99.
@skishchampi

Copy link
Copy Markdown
Contributor Author

On the P1 — the previous path does not need a shim. Checked, not assumed.

The claim is that a caller may have adopted commoner_probe.otp_download_portal.
No caller can have. Evidence, gathered this turn:

  1. The path is in no release. It appears in no tag. git cat-file -e against
    v0.14.9, v0.14.3 and v0.14.2 returns absent for all three. It was introduced
    by feat: extract points from a WMS-only GeoServer, and reach OTP-gated microdata #134, merged today.

  2. Every consumer pins a release, so none of them tracks master:

    repo pin
    theright2read commoner-probe[http,pdf]==0.14.3
    academiaindia commoner-probe[academia,http]==0.14.3 (two files)
    public-finance commoner-probe[budget]==0.14.3
    commoner-analyse commoner-probe==0.14.3
    narcotrek git+...@v0.14.3
    zero-hour git+...@v0.14.9

    Both git pins name a TAG, not a branch.

The name it replaced, udise, is in no release either. A shim for a path that
lived under an hour and shipped in nothing would be permanent clutter, and the
rule for removing one is a major version.

Where the finding does apply, it was already honoured. The 21 modules
renamed in #132 were all released, so every one of them kept a shim, and
tests/test_module_aliases.py asserts the identity of each.

The name changed again in this push, to spa_jwt_api. The module drives a
JWT-authenticated REST API behind an Angular single-page app, and that is what a
developer must implement. The ministry, the hosts, the year ids and the
all-India sentinel stay in the docstring as deployment facts.

@skishchampi
skishchampi merged commit 8a28377 into master Aug 17, 2026
9 checks passed
@skishchampi
skishchampi deleted the refactor/microdata-download-api branch August 17, 2026 02:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant