Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
4 changes: 4 additions & 0 deletions .clusterfuzzlite/build.sh
Original file line number Diff line number Diff line change
@@ -1,5 +1,9 @@
#!/bin/bash -eu

# SPDX-FileCopyrightText: 2026 Xquik contributors
#
# SPDX-License-Identifier: Apache-2.0

export PYTHONPATH="$SRC/x-twitter-scraper-python/src"

compile_python_fuzzer \
Expand Down
4 changes: 4 additions & 0 deletions .clusterfuzzlite/fuzz_querystring.py
Original file line number Diff line number Diff line change
@@ -1,3 +1,7 @@
# SPDX-FileCopyrightText: 2026 Xquik contributors
#
# SPDX-License-Identifier: Apache-2.0

from __future__ import annotations

import sys
Expand Down
53 changes: 40 additions & 13 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -18,15 +18,15 @@ permissions:
contents: read

concurrency:
group: ci-${{ github.workflow }}-${{ github.ref }}
group: ci-${{ github.event.pull_request.number || github.ref }}
cancel-in-progress: true

jobs:
lint:
timeout-minutes: 10
name: lint
timeout-minutes: 15
name: Lint & Licensing
runs-on: ubuntu-24.04
if: (github.event_name == 'push' || github.event.pull_request.head.repo.fork) && (github.event_name != 'push' || github.event.head_commit.message != 'codegen metadata')
if: github.event_name == 'pull_request' || github.event.head_commit.message != 'codegen metadata'
steps:
- name: Check out repository
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
Expand All @@ -45,10 +45,13 @@ jobs:
- name: Run lints
run: ./scripts/lint

- name: Check licensing
uses: fsfe/reuse-action@676e2d560c9a403aa252096d99fcab3e1132b0f5 # v6.0.0

build:
if: (github.event_name == 'push' || github.event.pull_request.head.repo.fork) && (github.event_name != 'push' || github.event.head_commit.message != 'codegen metadata')
if: github.event_name == 'pull_request' || github.event.head_commit.message != 'codegen metadata'
timeout-minutes: 10
name: build
name: Build & Reproducibility
runs-on: ubuntu-24.04
steps:
- name: Check out repository
Expand All @@ -62,17 +65,19 @@ jobs:
enable-cache: false
version: "0.11.6"

- name: Install dependencies
run: uv sync --locked --all-extras
- name: Verify reproducible distributions
run: ./scripts/check-reproducible

- name: Build distributions
run: SOURCE_DATE_EPOCH=946684800 uv build

- name: Run build
run: uv build
- name: Check package metadata
run: uvx --from twine==6.2.0 twine check dist/*

test:
timeout-minutes: 10
name: test
timeout-minutes: 20
name: Tests & Coverage
runs-on: ubuntu-24.04
if: github.event_name == 'push' || github.event.pull_request.head.repo.fork
steps:
- name: Check out repository
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
Expand All @@ -90,3 +95,25 @@ jobs:

- name: Run tests
run: ./scripts/test

- name: Enforce coverage
run: ./scripts/coverage

security:
timeout-minutes: 15
name: Dependency Security
runs-on: ubuntu-24.04
steps:
- name: Check out repository
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
persist-credentials: false

- name: Install uv
uses: astral-sh/setup-uv@11f9893b081a58869d3b5fccaea48c9e9e46f990 # v8.3.2
with:
enable-cache: false
version: "0.11.6"

- name: Audit locked dependencies
run: ./scripts/audit
13 changes: 12 additions & 1 deletion .github/workflows/publish-pypi.yml
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,7 @@ jobs:
- name: Check out repository
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
fetch-depth: 0
persist-credentials: false

- name: Install uv
Expand All @@ -31,8 +32,18 @@ jobs:
enable-cache: false
version: "0.11.6"

- name: Verify reproducible distributions
run: ./scripts/check-reproducible

- name: Verify release tag and ancestry
shell: bash
run: |
package_version="$(python3 -c 'import tomllib; print(tomllib.load(open("pyproject.toml", "rb"))["project"]["version"])')"
test "$GITHUB_REF_NAME" = "v$package_version"
git merge-base --is-ancestor "$GITHUB_SHA" refs/remotes/origin/main

- name: Build distributions
run: uv build
run: SOURCE_DATE_EPOCH=946684800 uv build

- name: Check package metadata
run: uvx --from twine==6.2.0 twine check dist/*
Expand Down
130 changes: 61 additions & 69 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
@@ -1,115 +1,107 @@
## Setting up the environment
# Contributing

### With `uv`
Thank you for improving the Xquik Python SDK.

We use [uv](https://docs.astral.sh/uv/) to manage dependencies because it will automatically provision a Python environment with the expected Python version. To set it up, run:
Read [GOVERNANCE.md](GOVERNANCE.md) before proposing major changes.

```sh
$ ./scripts/bootstrap
```
Follow the shared [Xquik contribution policy][contribution-policy].

## Set Up

Or [install uv manually](https://docs.astral.sh/uv/getting-started/installation/) and run:
Install `uv`, then run:

```sh
$ uv sync --all-extras
./scripts/bootstrap
```

You can then run scripts using `uv run python script.py` or by manually activating the virtual environment:
The bootstrap installs supported Python runtimes and locked dependencies.

```sh
# manually activate - https://docs.python.org/3/library/venv.html#how-venvs-work
$ source .venv/bin/activate
Without `uv`, install `requirements-dev.lock` inside an isolated environment.

# now you can omit the `uv run` prefix
$ python script.py
```
Never commit credentials or runtime environment files.

## Generated Code

### Without `uv`
Most SDK files come from the public OpenAPI contract.

Alternatively if you don't want to install `uv`, you can stick with the standard `pip` setup by ensuring you have the Python version specified in `.python-version`, create a virtual environment however you desire and then install dependencies using this command:
Preserve generated method names and response contracts.

Avoid generated-file changes when a generator fix exists.

Place stable examples outside generated directories.

## Verify Changes

Run focused tests while editing.

Run every gate before requesting review.

```sh
$ pip install -r requirements-dev.lock
./scripts/lint
./scripts/test
./scripts/coverage
./scripts/audit
uvx --from reuse==6.2.0 reuse lint
./scripts/check-reproducible
```

## Modifying/Adding code
Statement coverage must remain at least 90%.

Most of the SDK is generated code. Modifications to code will be persisted between generations, but may
result in merge conflicts between manual patches and changes from the generator. The generator will never
modify the contents of the `src/x_twitter_scraper/lib/` and `examples/` directories.
Branch coverage must remain at least 80%.

## Adding and running examples
Add regression tests for every corrected defect.

All files in the `examples/` directory are not modified by the generator and can be freely edited or added to.
`TEST_API_BASE_URL` accepts literal loopback addresses only.

```py
# add an example to examples/<your-example>.py
This guard prevents tests from mutating remote services.

#!/usr/bin/env -S uv run python
```
## Build From Source

Build both public distributions:

```sh
$ chmod +x examples/<your-example>.py
# run the example against your api
$ ./examples/<your-example>.py
uv build
```

## Using the repository from source
Install the wheel inside a clean environment for integration testing.

If you’d like to use the repository from source, you can either install from git or link to a cloned repository:
## Submit Changes

To install via git:
Keep pull requests focused.

```sh
$ pip install git+ssh://git@github.com/Xquik-dev/x-twitter-scraper-python.git
```
Explain user-visible behavior and public contract effects.

Alternatively, you can build from source and install the wheel file:
Link relevant issues and public API contracts.

Building this package will create two files in the `dist/` directory, a `.tar.gz` containing the source files and a `.whl` that can be used to install the package efficiently.
Use clear Conventional Commit subjects when practical.

To create a distributable version of the library, all you have to do is run this command:
Sign every commit with the Developer Certificate of Origin.

```sh
$ uv build
# or
$ python -m build
git commit --signoff
```

Then to install:
Another human must review maintainer-authored, nontrivial changes.

```sh
$ pip install ./path-to-wheel-file.whl
```
Reviewers follow the shared [review policy][review-policy].

## Running tests
Address every review comment before merging.

```sh
$ ./scripts/test
```
## Report Security Issues

## Linting and formatting
Never disclose suspected vulnerabilities in public issues.

This repository uses [ruff](https://github.com/astral-sh/ruff) and
[black](https://github.com/psf/black) to format the code in the repository.
Follow [SECURITY.md](SECURITY.md) for private reporting.

To lint:
## Releases

```sh
$ ./scripts/lint
```
Publish an immutable `v*` release after its commit reaches `main`.

To format and fix all ruff issues automatically:
The protected `pypi` environment uses PyPI trusted publishing.

```sh
$ ./scripts/format
```
Manual workflow runs must target a version tag.

## Publishing and releases
[contribution-policy]: https://github.com/Xquik-dev/.github/blob/main/CONTRIBUTING.md
[review-policy]: https://github.com/Xquik-dev/.github/blob/main/REVIEWING.md

Publish an immutable `v*` GitHub release after the release commit reaches `main`.
The [Publish PyPI workflow](https://github.com/Xquik-dev/x-twitter-scraper-python/actions/workflows/publish-pypi.yml)
builds the distributions separately, then publishes them through PyPI trusted
publishing from the protected `pypi` environment. Manual runs must target a
version tag.
Xquik is an independent third-party service. Not affiliated with X Corp. "Twitter" and "X" are trademarks of X Corp.
69 changes: 69 additions & 0 deletions GOVERNANCE.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,69 @@
# Governance

## Scope

This repository publishes the Xquik Python SDK.

The public OpenAPI specification defines its external contract.

## Roles

Contributors propose changes, tests, documentation, and reviews.

Maintainers hold repository administration and release permissions.

GitHub organization access remains the authority for maintainer permissions.

## Decisions

Use public issues for non-sensitive design discussions.

Use pull requests for reviewed repository changes.

Preserve generated API contracts unless the public specification changes.

Prefer consensus for user-visible and governance decisions.

Document unresolved tradeoffs in the pull request.

## Reviews

Every nontrivial change requires review before release.

Maintainer-authored changes require another human reviewer.

Security-sensitive changes require an explicit security review.

Reviewers follow the shared [review policy][review-policy].

## Releases

Release automation prepares version and changelog updates.

Maintainers verify tests, security gates, and release metadata.

Release tags must reference the reviewed default-branch commit.

Python wheel and source archives must remain reproducible.

## Maintainer Changes

Sustained contributors may request maintainer consideration.

Existing maintainers evaluate judgment, reliability, and security practices.

Remove inactive access when continuity and security permit.

Document material governance changes through reviewed pull requests.

## Continuity

At least 2 maintainers should hold required release access.

Current public evidence does not yet prove that threshold.

Track this limitation in [OPENSSF.md](OPENSSF.md).

[review-policy]: https://github.com/Xquik-dev/.github/blob/main/REVIEWING.md

Xquik is an independent third-party service. Not affiliated with X Corp. "Twitter" and "X" are trademarks of X Corp.
Loading
Loading