Skip to content

Add repository and activity timestamp indexes for issues and pull requests - #501

Open
EngCaioFonseca wants to merge 1 commit into
chaoss:mainfrom
EngCaioFonseca:fix/issue-pr-activity-indexes
Open

EngCaioFonseca wants to merge 1 commit into
chaoss:mainfrom
EngCaioFonseca:fix/issue-pr-activity-indexes

Conversation

@EngCaioFonseca

@EngCaioFonseca EngCaioFonseca commented Sep 28, 2026 •

Copy link
Copy Markdown
Contributor

Description

  • Please include a summary of the change.

Workers and metrics frequently filter issues and pull requests by repo_id and creation or update timestamps. The existing indexes cover repo_id or (repo_id, data_collection_date), so basically existing indexes can narrow by repository, but cannot efficiently narrow by the activity timestamps used by workers and metrics

This PR fixes #
The solution/fix to this issue was to add four composite B-tree indexes as follows:

data.issues (repo_id, updated_at)
data.issues (repo_id, created_at)
data.pull_requests (repo_id, pr_updated_at)
data.pull_requests (repo_id, pr_created_at)

Update the SQLAlchemy models and introduce Alembic migration 45, with concurrent index creation and removal to allow collection writes during migration.

Notes for Reviewers

Signed commits

  • Yes, I signed my commits.

Generative AI disclosure

Please select one option:

  • This contribution was NOT assisted or created by Generative AI tools.
  • This contribution was assisted or created by Generative AI tools.

If AI tools were used, please provide details below:
- What tools were used? GPT 6 Astra
- How were these tools used? Initial draft.
- Did you review these outputs before submitting this PR? Yes.

Signed-off-by: Caio Fonseca <engcaiofonseca@protonmail.com>
@EngCaioFonseca
EngCaioFonseca force-pushed the fix/issue-pr-activity-indexes branch from bede7f1 to 98b4afa Compare September 28, 2026 22:18
@MoralCode MoralCode added the database Related to the unifed data model/schema label Sep 29, 2026
Comment on lines +18 to +21
with op.get_context().autocommit_block():
op.create_index(
'issues_idx_repo_id_updated_at', 'issues', ['repo_id', 'updated_at'],
schema='data', postgresql_concurrently=True,

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

was this migration generated with an AI tool, or alembic's built-in --autogenerate flag?

@MoralCode

Copy link
Copy Markdown
Contributor

related to #445

@MoralCode MoralCode added this to the v1.2 milestone Sep 29, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

database Related to the unifed data model/schema

Projects

Development

Successfully merging this pull request may close these issues.

2 participants