Skip to content

perf: optimize find_all_tags_for with caching and indexing#127

Merged
magajh merged 4 commits into
masterfrom
maga/optimize-find-all-tags-for
Nov 24, 2025
Merged

perf: optimize find_all_tags_for with caching and indexing#127
magajh merged 4 commits into
masterfrom
maga/optimize-find-all-tags-for

Conversation

@magajh

@magajh magajh commented Jan 7, 2025

Copy link
Copy Markdown
Contributor

Description

This is a backport of the changes made in #126 to solve this issue #119

This PR introduces optimizations to the find_all_tags_for method and related functionality to improve performance when working with large datasets in the Tag model. Specifically:

  1. Caching for ContentType Lookups:

    • Added cache to store ContentType objects for frequently used model names. This reduces redundant database queries for ContentType.objects.get(), especially in high-traffic scenarios or repeated calls.
  2. Database Indexes:

    • Added an index on the target_type and target_object_id fields in the Tag model.
    • This index improves query performance for operations that filter tags by target_type and target_object_id, particularly in the find_all_tags_for method.
  3. Optimized Query Logic:

    • Removed the list() call on target.values_list("id", flat=True) in the find_all_tags_for method.
    • This ensures the query is fully managed by the database, avoiding unnecessary memory overhead and improving scalability for large result sets.

Why These Changes Were Made

  • The find_all_tags_for method was identified as a performance bottleneck, particularly for large datasets in the Tag table (~6 million records).
  • Redundant ContentType lookups were causing unnecessary database queries.
  • Queries filtering by target_type and target_object_id lacked an index, leading to slower performance on large datasets.

@magajh magajh requested a review from a team as a code owner January 7, 2025 13:16
@magajh magajh force-pushed the maga/optimize-find-all-tags-for branch from ff6b935 to 12f0b58 Compare November 17, 2025 12:49
@magajh

magajh commented Nov 17, 2025

Copy link
Copy Markdown
Contributor Author

@eduNEXT/heimdall just so you know these changes were already deployed some time ago to the nimbus saas repository to fix #119, and they are working fine

@DeimerM

DeimerM commented Nov 19, 2025

Copy link
Copy Markdown

Hi @magajh thanks a lot for your help! LGTM!

@magajh magajh merged commit 9d7a329 into master Nov 24, 2025
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants