diff --git a/README.md b/README.md index 545f378e..b73747c3 100644 --- a/README.md +++ b/README.md @@ -24,7 +24,7 @@ -Semble is a code search library built for agents. It returns the exact code snippets they need instantly, using ~99% fewer tokens than grep+read. Indexing and searching a full codebase end-to-end takes under a second for most repos, matching the retrieval quality of a code-specialized transformer while indexing ~380x faster and querying ~17x faster (see [benchmarks](#benchmarks)). Everything runs on CPU with no API keys, GPU, or external services. Use it as an MCP server, a CLI tool via AGENTS.md, or a dedicated sub-agent, and any coding agent (Claude Code, Cursor, Codex, OpenCode, etc.) gets instant access to any repo. +Semble is a code search library built for agents. It returns the exact code snippets they need instantly, using ~99% fewer tokens than grep+read. Indexing and searching a full codebase end-to-end takes under a second for most repos, matching the retrieval quality of a code-specialized transformer while indexing ~480x faster and querying ~23x faster (see [benchmarks](#benchmarks)). Everything runs on CPU with no API keys, GPU, or external services. Use it as an MCP server, a CLI tool via AGENTS.md, or a dedicated sub-agent, and any coding agent (Claude Code, Cursor, Codex, OpenCode, etc.) gets instant access to any repo. @@ -78,7 +78,7 @@ semble install --agent claude --type mcp subagent --yes ## Main Features - **Fast**: indexes an average repo in ~500 ms and answers queries in ~1 ms, all on CPU. -- **Accurate**: NDCG@10 of 0.854 on our [benchmarks](#benchmarks), on par with code-specialized transformer models, at a fraction of the size and cost. +- **Accurate**: NDCG@10 of 0.855 on our [benchmarks](#benchmarks), on par with code-specialized transformer models, at a fraction of the size and cost. - **Token-efficient**: returns only the relevant chunks, using [~99% fewer tokens than grep+read](#benchmarks). - **Zero setup**: runs on CPU with no API keys, GPU, or external services required. - **MCP server**: works with Claude Code, Cursor, Codex, OpenCode, VS Code, and any other MCP-compatible agent. @@ -259,7 +259,7 @@ We benchmark quality and speed across ~1,250 queries over 63 repositories in 19 -The quality benchmark (left) scores retrieval quality (NDCG@10) against total latency; semble matches the quality of the 137M-parameter [CodeRankEmbed](https://huggingface.co/nomic-ai/CodeRankEmbed) while indexing 380x faster. The token efficiency benchmark (right) measures how many tokens each method needs to reach a given recall level; semble uses 99% fewer tokens on average and hits 97% recall at only 2k tokens, while grep+read needs a full 100k context window to reach 85%. See [benchmarks](benchmarks/README.md) for per-language results, ablations, and full methodology. +The quality benchmark (left) scores retrieval quality (NDCG@10) against total latency; semble matches the quality of the 137M-parameter [CodeRankEmbed](https://huggingface.co/nomic-ai/CodeRankEmbed) while indexing 480x faster. The token efficiency benchmark (right) measures how many tokens each method needs to reach a given recall level; semble uses 99% fewer tokens on average and hits 96% recall at only 2k tokens, while grep+read needs a full 100k context window to reach 85%. See [benchmarks](benchmarks/README.md) for per-language results, ablations, and full methodology. ## How it works diff --git a/assets/images/speed_vs_ndcg_cold.png b/assets/images/speed_vs_ndcg_cold.png index ebc55d54..0f9932db 100644 Binary files a/assets/images/speed_vs_ndcg_cold.png and b/assets/images/speed_vs_ndcg_cold.png differ diff --git a/assets/images/speed_vs_ndcg_warm.png b/assets/images/speed_vs_ndcg_warm.png index 768193c2..2cad0701 100644 Binary files a/assets/images/speed_vs_ndcg_warm.png and b/assets/images/speed_vs_ndcg_warm.png differ diff --git a/assets/images/token_efficiency.png b/assets/images/token_efficiency.png index 6d663511..335471ae 100644 Binary files a/assets/images/token_efficiency.png and b/assets/images/token_efficiency.png differ diff --git a/benchmarks/README.md b/benchmarks/README.md index bd45ae21..ea230ffa 100644 --- a/benchmarks/README.md +++ b/benchmarks/README.md @@ -17,7 +17,7 @@ Quality and speed across all methods. | Method | NDCG@10 | Index | Query p50 | | -------------------- | --------: | ---------: | ----------: | -| **semble** | **0.854** | **306 ms** | **0.91 ms** | +| **semble** | **0.855** | **240 ms** | **0.69 ms** | | CodeRankEmbed | 0.839 | 116 s | 16 ms | | ColGREP | 0.693 | 5.4 s | 122 ms | | BM25 | 0.673 | 47 ms | 0.17 ms | @@ -33,7 +33,7 @@ Quality and speed across all methods. | :-----------------------------------------------------------------: | :-----------------------------------------------------------------: | | _Time to first result (index + query) vs NDCG@10_ | _Query latency on a warm index vs NDCG@10_ | -semble matches the NDCG@10 of the 137M-param CodeRankEmbed while winning index time by ~380x and query latency by ~17x. +semble matches the NDCG@10 of the 137M-param CodeRankEmbed while winning index time by ~480x and query latency by ~23x. NDCG@10 is averaged across all queries. Speed numbers use one repo per language, CPU only: cold-start index time and warm query p50 (median across 5 consecutive runs). @@ -50,7 +50,7 @@ For each query: tokens consumed at first relevant hit, or 32k if the method neve | Method | Expected tokens | Savings | | ------------------- | --------------: | ------------: | | ripgrep + read file | 45,587 | baseline | -| **semble** | **348** | **99% fewer** | +| **semble** | **353** | **99% fewer** | ### Recall at fixed token budgets @@ -58,7 +58,7 @@ A relevant file is "covered" once any retrieved unit comes from it. | Method | 500 | 1k | 2k | 4k | 8k | 16k | 32k | | ------------------- | --------: | --------: | --------: | --------: | --------: | --------: | --------: | -| **semble** | **0.842** | **0.923** | **0.967** | **0.988** | **0.995** | **0.995** | **0.995** | +| **semble** | **0.843** | **0.921** | **0.963** | **0.989** | **0.995** | **0.996** | **0.996** | | ripgrep + read file | 0.001 | 0.008 | 0.037 | 0.086 | 0.207 | 0.374 | 0.583 |
@@ -74,26 +74,26 @@ NDCG@10 per language, sorted by semble. CRE = CodeRankEmbed, cbm = codebase-memo | Language | semble | CRE | ColGREP | zvec-grep | ck | cbm | grepai | probe | cs | ripgrep | | ----------- | --------: | --------: | --------: | ---------: | --------: | --------: | --------: | --------: | --------: | --------: | -| javascript | 0.917 | **0.925** | 0.823 | 0.760 | 0.772 | 0.770 | 0.675 | 0.588 | 0.171 | 0.176 | -| cpp | **0.915** | 0.897 | 0.626 | 0.677 | 0.687 | 0.630 | 0.731 | 0.375 | 0.262 | 0.126 | -| zig | **0.913** | 0.911 | 0.474 | 0.664 | 0.511 | 0.766 | 0.755 | 0.369 | 0.121 | 0.000 | -| scala | 0.909 | **0.925** | 0.765 | 0.799 | 0.717 | 0.704 | 0.330 | 0.392 | 0.111 | 0.180 | +| scala | 0.921 | **0.925** | 0.765 | 0.799 | 0.717 | 0.704 | 0.330 | 0.392 | 0.111 | 0.180 | +| javascript | 0.911 | **0.925** | 0.823 | 0.760 | 0.772 | 0.770 | 0.675 | 0.588 | 0.171 | 0.176 | +| go | **0.910** | 0.713 | 0.785 | 0.512 | 0.458 | 0.506 | 0.722 | 0.410 | 0.183 | 0.133 | | ruby | **0.909** | 0.905 | 0.708 | 0.758 | 0.738 | 0.689 | 0.643 | 0.382 | 0.255 | 0.230 | -| go | **0.895** | 0.713 | 0.785 | 0.512 | 0.458 | 0.506 | 0.722 | 0.410 | 0.183 | 0.133 | -| elixir | **0.894** | 0.893 | 0.808 | 0.811 | 0.786 | 0.506 | 0.669 | 0.412 | 0.397 | 0.134 | -| csharp | **0.885** | 0.848 | 0.614 | 0.609 | 0.548 | 0.775 | 0.277 | 0.392 | 0.248 | 0.117 | -| python | 0.867 | **0.878** | 0.777 | 0.704 | 0.721 | 0.643 | 0.634 | 0.488 | 0.305 | 0.202 | -| swift | **0.860** | 0.845 | 0.710 | 0.709 | 0.672 | 0.630 | 0.429 | 0.280 | 0.151 | 0.160 | -| php | **0.858** | 0.847 | 0.663 | 0.583 | 0.615 | 0.608 | 0.402 | 0.340 | 0.180 | 0.123 | -| rust | **0.856** | 0.754 | 0.662 | 0.541 | 0.419 | 0.454 | 0.519 | 0.242 | 0.193 | 0.162 | -| java | **0.849** | 0.790 | 0.641 | 0.685 | 0.606 | 0.554 | 0.386 | 0.536 | 0.136 | 0.198 | -| bash | 0.825 | **0.834** | 0.706 | 0.725 | 0.677 | 0.768 | 0.723 | 0.226 | 0.170 | 0.000 | -| lua | 0.823 | **0.829** | 0.798 | 0.736 | 0.738 | 0.591 | 0.699 | 0.336 | 0.050 | 0.000 | -| kotlin | 0.821 | **0.823** | 0.637 | 0.628 | 0.587 | 0.611 | 0.478 | 0.335 | 0.170 | 0.166 | -| haskell | 0.765 | **0.811** | 0.683 | 0.689 | 0.733 | 0.624 | 0.483 | 0.313 | 0.160 | 0.000 | -| c | 0.741 | **0.771** | 0.676 | 0.598 | 0.606 | 0.655 | 0.555 | 0.384 | 0.175 | 0.000 | -| typescript | **0.706** | 0.671 | 0.430 | 0.476 | 0.456 | 0.455 | 0.394 | 0.354 | 0.145 | 0.128 | -| **overall** | **0.854** | 0.839 | 0.693 | 0.670 | 0.642 | 0.630 | 0.561 | 0.387 | 0.200 | 0.126 | +| elixir | **0.905** | 0.893 | 0.808 | 0.811 | 0.786 | 0.506 | 0.669 | 0.412 | 0.397 | 0.134 | +| zig | 0.899 | **0.911** | 0.474 | 0.664 | 0.511 | 0.766 | 0.755 | 0.369 | 0.121 | 0.000 | +| cpp | 0.881 | **0.897** | 0.626 | 0.677 | 0.687 | 0.630 | 0.731 | 0.375 | 0.262 | 0.126 | +| python | 0.876 | **0.878** | 0.777 | 0.704 | 0.721 | 0.643 | 0.634 | 0.488 | 0.305 | 0.202 | +| csharp | **0.872** | 0.848 | 0.614 | 0.609 | 0.548 | 0.775 | 0.277 | 0.392 | 0.248 | 0.117 | +| php | **0.864** | 0.847 | 0.663 | 0.583 | 0.615 | 0.608 | 0.402 | 0.340 | 0.180 | 0.123 | +| swift | **0.848** | 0.845 | 0.710 | 0.709 | 0.672 | 0.630 | 0.429 | 0.280 | 0.151 | 0.160 | +| bash | **0.840** | 0.834 | 0.706 | 0.725 | 0.677 | 0.768 | 0.723 | 0.226 | 0.170 | 0.000 | +| lua | **0.835** | 0.829 | 0.798 | 0.736 | 0.738 | 0.591 | 0.699 | 0.336 | 0.050 | 0.000 | +| java | **0.828** | 0.790 | 0.641 | 0.685 | 0.606 | 0.554 | 0.386 | 0.536 | 0.136 | 0.198 | +| rust | **0.826** | 0.754 | 0.662 | 0.541 | 0.419 | 0.454 | 0.519 | 0.242 | 0.193 | 0.162 | +| kotlin | 0.803 | **0.823** | 0.637 | 0.628 | 0.587 | 0.611 | 0.478 | 0.335 | 0.170 | 0.166 | +| haskell | 0.775 | **0.811** | 0.683 | 0.689 | 0.733 | 0.624 | 0.483 | 0.313 | 0.160 | 0.000 | +| c | 0.750 | **0.771** | 0.676 | 0.598 | 0.606 | 0.655 | 0.555 | 0.384 | 0.175 | 0.000 | +| typescript | **0.742** | 0.671 | 0.430 | 0.476 | 0.456 | 0.455 | 0.394 | 0.354 | 0.145 | 0.128 | +| **overall** | **0.855** | 0.839 | 0.693 | 0.670 | 0.642 | 0.630 | 0.561 | 0.387 | 0.200 | 0.126 | ## Ablations @@ -101,20 +101,20 @@ NDCG@10 per language, sorted by semble. CRE = CodeRankEmbed, cbm = codebase-memo | Retrieval | Raw | + ranking | | ---------------------- | ----: | --------: | -| BM25 | 0.675 | 0.834 | -| potion-code-16M-v2 | 0.650 | 0.821 | -| BM25 + potion-code-16M-v2 | — | **0.854** | +| BM25 | 0.686 | 0.838 | +| potion-code-16M-v2 | 0.652 | 0.824 | +| BM25 + potion-code-16M-v2 | — | **0.855** |
By query category | Mode | Architecture | Semantic | Symbol | | ---------------------------------- | -----------: | --------: | --------: | -| BM25 raw | 0.628 | 0.676 | 0.719 | -| potion-code-16M raw | 0.626 | 0.666 | 0.629 | -| semble BM25 (+ ranking) | 0.770 | 0.819 | 0.957 | -| semble potion-code-16M (+ ranking) | 0.757 | 0.808 | 0.943 | -| **semble hybrid** | **0.802** | **0.846** | **0.958** | +| BM25 raw | 0.642 | 0.680 | 0.748 | +| potion-code-16M raw | 0.636 | 0.671 | 0.620 | +| semble BM25 (+ ranking) | 0.794 | 0.817 | 0.962 | +| semble potion-code-16M (+ ranking) | 0.768 | 0.819 | 0.917 | +| **semble hybrid** | **0.808** | **0.847** | **0.954** |
diff --git a/benchmarks/plot.py b/benchmarks/plot.py index 4801611c..378f2523 100644 --- a/benchmarks/plot.py +++ b/benchmarks/plot.py @@ -103,9 +103,9 @@ class _Method(TypedDict): }, { "name": "semble", - "ndcg10": 0.8544, - "index_ms": 305.6, - "query_p50_ms": 0.91, + "ndcg10": 0.8546, + "index_ms": 240.3, + "query_p50_ms": 0.69, "color": "#1a5fa8", "params_m": 16, }, diff --git a/benchmarks/results/semble-ablations-5fbcd035d594.json b/benchmarks/results/semble-ablations-5fbcd035d594.json new file mode 100644 index 00000000..fb338f82 --- /dev/null +++ b/benchmarks/results/semble-ablations-5fbcd035d594.json @@ -0,0 +1,9520 @@ +{ + "tool": "semble-ablations", + "model": "minishlab/potion-code-16M-v2", + "by_mode": { + "semble-auto": { + "avg_ndcg10": 0.8546, + "avg_p50_ms": 1.0, + "avg_tokens": 1778.9 + }, + "semble-balanced": { + "avg_ndcg10": 0.8546, + "avg_p50_ms": 1.0, + "avg_tokens": 1778.3 + }, + "semble-bm25": { + "avg_ndcg10": 0.8381, + "avg_p50_ms": 0.7, + "avg_tokens": 1835.4 + }, + "semble-semantic": { + "avg_ndcg10": 0.8237, + "avg_p50_ms": 0.8, + "avg_tokens": 1610.7 + }, + "unranked-auto": { + "avg_ndcg10": 0.7117, + "avg_p50_ms": 0.3, + "avg_tokens": 1776.9 + }, + "unranked-balanced": { + "avg_ndcg10": 0.7098, + "avg_p50_ms": 0.3, + "avg_tokens": 1778.6 + }, + "unranked-bm25": { + "avg_ndcg10": 0.6861, + "avg_p50_ms": 0.3, + "avg_tokens": 1801.8 + }, + "unranked-semantic": { + "avg_ndcg10": 0.6524, + "avg_p50_ms": 0.3, + "avg_tokens": 1664.0 + } + }, + "repos": [ + { + "repo": "abseil-cpp", + "language": "cpp", + "mode": "semble-auto", + "chunks": 16813, + "tokens": 1502, + "ndcg5": 0.8621567083124564, + "ndcg10": 0.8799670676678575, + "p50_ms": 1.3537289923988283, + "p90_ms": 13.019679504213855, + "p95_ms": 13.354321193764918, + "p99_ms": 13.599864243878983, + "index_ms": 4034.962958015967, + "by_category": { + "architecture": 1.0, + "semantic": 0.8732894235571433, + "symbol": 0.8333333333333334 + } + }, + { + "repo": "abseil-cpp", + "language": "cpp", + "mode": "semble-balanced", + "chunks": 16813, + "tokens": 1497, + "ndcg5": 0.8621567083124564, + "ndcg10": 0.8799670676678575, + "p50_ms": 1.2594795261975378, + "p90_ms": 13.101429806556553, + "p95_ms": 13.419537781737745, + "p99_ms": 13.453673985786736, + "index_ms": 4034.962958015967, + "by_category": { + "architecture": 1.0, + "semantic": 0.8732894235571433, + "symbol": 0.8333333333333334 + } + }, + { + "repo": "abseil-cpp", + "language": "cpp", + "mode": "semble-bm25", + "chunks": 16813, + "tokens": 1580, + "ndcg5": 0.859968142854855, + "ndcg10": 0.859968142854855, + "p50_ms": 1.0185620049014688, + "p90_ms": 11.718279123306274, + "p95_ms": 11.94350971491076, + "p99_ms": 12.83223473583348, + "index_ms": 4034.962958015967, + "by_category": { + "architecture": 1.0, + "semantic": 0.8466241904731402, + "symbol": 0.8333333333333334 + } + }, + { + "repo": "abseil-cpp", + "language": "cpp", + "mode": "semble-semantic", + "chunks": 16813, + "tokens": 1380, + "ndcg5": 0.8228302093894448, + "ndcg10": 0.8228302093894448, + "p50_ms": 1.0826459911186248, + "p90_ms": 11.996337492018938, + "p95_ms": 12.495587480952963, + "p99_ms": 12.838917485205457, + "index_ms": 4034.962958015967, + "by_category": { + "architecture": 0.8154648767857288, + "semantic": 0.8217116289478296, + "symbol": 0.8333333333333334 + } + }, + { + "repo": "abseil-cpp", + "language": "cpp", + "mode": "unranked-auto", + "chunks": 16813, + "tokens": 1561, + "ndcg5": 0.7765543652555861, + "ndcg10": 0.8018249849182094, + "p50_ms": 0.7410625112242997, + "p90_ms": 0.8540670096408576, + "p95_ms": 0.8973294985480608, + "p99_ms": 1.0555995092727242, + "index_ms": 4034.962958015967, + "by_category": { + "architecture": 1.0, + "semantic": 0.773721542686053, + "symbol": 0.8102255193577976 + } + }, + { + "repo": "abseil-cpp", + "language": "cpp", + "mode": "unranked-balanced", + "chunks": 16813, + "tokens": 1561, + "ndcg5": 0.7765543652555861, + "ndcg10": 0.8018249849182094, + "p50_ms": 0.6914790137670934, + "p90_ms": 0.7512167096138, + "p95_ms": 0.7568606844870374, + "p99_ms": 0.778804900473915, + "index_ms": 4034.962958015967, + "by_category": { + "architecture": 1.0, + "semantic": 0.773721542686053, + "symbol": 0.8102255193577976 + } + }, + { + "repo": "abseil-cpp", + "language": "cpp", + "mode": "unranked-bm25", + "chunks": 16813, + "tokens": 1631, + "ndcg5": 0.7594417513399824, + "ndcg10": 0.7861353670201771, + "p50_ms": 0.6720000237692147, + "p90_ms": 0.7183377572800964, + "p95_ms": 0.729447856429033, + "p99_ms": 0.7552559807663783, + "index_ms": 4034.962958015967, + "by_category": { + "architecture": 1.0, + "semantic": 0.7850875140030903, + "symbol": 0.6487982101190621 + } + }, + { + "repo": "abseil-cpp", + "language": "cpp", + "mode": "unranked-semantic", + "chunks": 16813, + "tokens": 1486, + "ndcg5": 0.6999554830799519, + "ndcg10": 0.73005848264635, + "p50_ms": 0.6713334878440946, + "p90_ms": 0.7231163792312145, + "p95_ms": 0.7274419913301244, + "p99_ms": 0.7544219755800441, + "index_ms": 4034.962958015967, + "by_category": { + "architecture": 0.7153382790366966, + "semantic": 0.6780328729902405, + "symbol": 1.0 + } + }, + { + "repo": "aeson", + "language": "haskell", + "mode": "semble-auto", + "chunks": 788, + "tokens": 1990, + "ndcg5": 0.7586215664331644, + "ndcg10": 0.768292886614028, + "p50_ms": 2.9114169883541763, + "p90_ms": 5.978920514462522, + "p95_ms": 10.443720570765443, + "p99_ms": 17.694311323575665, + "index_ms": 141.74112502951175, + "by_category": { + "architecture": 0.6297424628754914, + "semantic": 0.81447636119354 + } + }, + { + "repo": "aeson", + "language": "haskell", + "mode": "semble-balanced", + "chunks": 788, + "tokens": 1990, + "ndcg5": 0.7586215664331644, + "ndcg10": 0.768292886614028, + "p50_ms": 2.8962290089111775, + "p90_ms": 5.922687466954819, + "p95_ms": 10.415333329001449, + "p99_ms": 17.665100285084907, + "index_ms": 141.74112502951175, + "by_category": { + "architecture": 0.6297424628754914, + "semantic": 0.81447636119354 + } + }, + { + "repo": "aeson", + "language": "haskell", + "mode": "semble-bm25", + "chunks": 788, + "tokens": 1933, + "ndcg5": 0.7855159313011154, + "ndcg10": 0.7855159313011154, + "p50_ms": 2.090645517455414, + "p90_ms": 4.244424711214387, + "p95_ms": 10.27505832025782, + "p99_ms": 17.21474530408158, + "index_ms": 141.74112502951175, + "by_category": { + "architecture": 0.5896918237758785, + "semantic": 0.8507906338095277 + } + }, + { + "repo": "aeson", + "language": "haskell", + "mode": "semble-semantic", + "chunks": 788, + "tokens": 1749, + "ndcg5": 0.7389349643930563, + "ndcg10": 0.7389349643930563, + "p50_ms": 2.0519585232250392, + "p90_ms": 4.0638458100147625, + "p95_ms": 9.380024633719593, + "p99_ms": 16.574437731760543, + "index_ms": 141.74112502951175, + "by_category": { + "architecture": 0.6297424628754914, + "semantic": 0.7753324648989113 + } + }, + { + "repo": "aeson", + "language": "haskell", + "mode": "unranked-auto", + "chunks": 788, + "tokens": 2090, + "ndcg5": 0.9008891280402999, + "ndcg10": 0.9008891280402999, + "p50_ms": 0.2828124852385372, + "p90_ms": 0.3093670413363725, + "p95_ms": 0.31230236345436424, + "p99_ms": 0.31246051366906613, + "index_ms": 141.74112502951175, + "by_category": { + "architecture": 0.8035565121611998, + "semantic": 0.9333333333333333 + } + }, + { + "repo": "aeson", + "language": "haskell", + "mode": "unranked-balanced", + "chunks": 788, + "tokens": 2090, + "ndcg5": 0.9008891280402999, + "ndcg10": 0.9008891280402999, + "p50_ms": 0.28016645228490233, + "p90_ms": 0.2970749861560762, + "p95_ms": 0.3068750083912164, + "p99_ms": 0.3087750228587538, + "index_ms": 141.74112502951175, + "by_category": { + "architecture": 0.8035565121611998, + "semantic": 0.9333333333333333 + } + }, + { + "repo": "aeson", + "language": "haskell", + "mode": "unranked-bm25", + "chunks": 788, + "tokens": 2109, + "ndcg5": 0.85, + "ndcg10": 0.8657732438392864, + "p50_ms": 0.2771459985524416, + "p90_ms": 0.29155363445170224, + "p95_ms": 0.29416901525110006, + "p99_ms": 0.29800018761307, + "index_ms": 141.74112502951175, + "by_category": { + "architecture": 0.6630929753571457, + "semantic": 0.9333333333333333 + } + }, + { + "repo": "aeson", + "language": "haskell", + "mode": "unranked-semantic", + "chunks": 788, + "tokens": 1777, + "ndcg5": 0.9065464876785729, + "ndcg10": 0.9065464876785729, + "p50_ms": 0.27718752971850336, + "p90_ms": 0.29089130694046617, + "p95_ms": 0.29147856403142214, + "p99_ms": 0.29242932330816984, + "index_ms": 141.74112502951175, + "by_category": { + "architecture": 0.8261859507142916, + "semantic": 0.9333333333333333 + } + }, + { + "repo": "aiohttp", + "language": "python", + "mode": "semble-auto", + "chunks": 1469, + "tokens": 1565, + "ndcg5": 0.8223865779223699, + "ndcg10": 0.8550948458996863, + "p50_ms": 0.9534170385450125, + "p90_ms": 3.079374961089343, + "p95_ms": 3.4031670074909925, + "p99_ms": 6.132733402773741, + "index_ms": 204.30829201359302, + "by_category": { + "architecture": 0.8634033929769788, + "semantic": 0.7681297792282846, + "symbol": 1.0 + } + }, + { + "repo": "aiohttp", + "language": "python", + "mode": "semble-balanced", + "chunks": 1469, + "tokens": 1556, + "ndcg5": 0.8223865779223699, + "ndcg10": 0.8550948458996863, + "p50_ms": 0.9774579666554928, + "p90_ms": 3.339166985824704, + "p95_ms": 3.545417042914778, + "p99_ms": 5.330549797508867, + "index_ms": 204.30829201359302, + "by_category": { + "architecture": 0.8634033929769788, + "semantic": 0.7681297792282846, + "symbol": 1.0 + } + }, + { + "repo": "aiohttp", + "language": "python", + "mode": "semble-bm25", + "chunks": 1469, + "tokens": 1580, + "ndcg5": 0.7814325379643664, + "ndcg10": 0.7911650330876276, + "p50_ms": 0.4864170332439244, + "p90_ms": 2.0660420414060354, + "p95_ms": 2.1751660387963057, + "p99_ms": 4.322065971791746, + "index_ms": 204.30829201359302, + "by_category": { + "architecture": 0.7758109690170575, + "semantic": 0.6870876568578643, + "symbol": 1.0 + } + }, + { + "repo": "aiohttp", + "language": "python", + "mode": "semble-semantic", + "chunks": 1469, + "tokens": 1502, + "ndcg5": 0.7999405515584438, + "ndcg10": 0.8087298704527089, + "p50_ms": 0.526249990798533, + "p90_ms": 2.5481670163571835, + "p95_ms": 2.9238330316729844, + "p99_ms": 6.572866591159258, + "index_ms": 204.30829201359302, + "by_category": { + "architecture": 0.8920109923338452, + "semantic": 0.7488055925744415, + "symbol": 0.8 + } + }, + { + "repo": "aiohttp", + "language": "python", + "mode": "unranked-auto", + "chunks": 1469, + "tokens": 1587, + "ndcg5": 0.5501403369720637, + "ndcg10": 0.596446214359299, + "p50_ms": 0.41691696969792247, + "p90_ms": 0.8813340100459754, + "p95_ms": 0.9282079990953207, + "p99_ms": 0.9755415841937065, + "index_ms": 204.30829201359302, + "by_category": { + "architecture": 0.6334090614525779, + "semantic": 0.5671522367441948, + "symbol": 0.5974273881358959 + } + }, + { + "repo": "aiohttp", + "language": "python", + "mode": "unranked-balanced", + "chunks": 1469, + "tokens": 1594, + "ndcg5": 0.5468392206898443, + "ndcg10": 0.5931450980770796, + "p50_ms": 0.28779200511053205, + "p90_ms": 0.4242919967509806, + "p95_ms": 0.44949998846277595, + "p99_ms": 0.5701335961930455, + "index_ms": 204.30829201359302, + "by_category": { + "architecture": 0.6334090614525779, + "semantic": 0.5671522367441948, + "symbol": 0.5835626997505745 + } + }, + { + "repo": "aiohttp", + "language": "python", + "mode": "unranked-bm25", + "chunks": 1469, + "tokens": 1575, + "ndcg5": 0.5929982984763218, + "ndcg10": 0.6210493997724876, + "p50_ms": 0.33554097171872854, + "p90_ms": 0.5412499886006117, + "p95_ms": 0.6011249497532845, + "p99_ms": 0.7087249774485828, + "index_ms": 204.30829201359302, + "by_category": { + "architecture": 0.5561583570773782, + "semantic": 0.6686665713454596, + "symbol": 0.6261859507142915 + } + }, + { + "repo": "aiohttp", + "language": "python", + "mode": "unranked-semantic", + "chunks": 1469, + "tokens": 1509, + "ndcg5": 0.5700093898052172, + "ndcg10": 0.5891990628336807, + "p50_ms": 0.28491602279245853, + "p90_ms": 0.32416702015325427, + "p95_ms": 0.35266700433567166, + "p99_ms": 0.5177661892957987, + "index_ms": 204.30829201359302, + "by_category": { + "architecture": 0.7273805133632569, + "semantic": 0.5515541511840623, + "symbol": 0.4635058730615869 + } + }, + { + "repo": "alamofire", + "language": "swift", + "mode": "semble-auto", + "chunks": 1302, + "tokens": 1533, + "ndcg5": 0.9775325271359823, + "ndcg10": 0.9775325271359823, + "p50_ms": 0.66329151741229, + "p90_ms": 3.216908336617054, + "p95_ms": 4.184841367532501, + "p99_ms": 5.657468261779283, + "index_ms": 205.70454199332744, + "by_category": { + "architecture": 1.0, + "semantic": 0.9591500493381495, + "symbol": 1.0 + } + }, + { + "repo": "alamofire", + "language": "swift", + "mode": "semble-balanced", + "chunks": 1302, + "tokens": 1523, + "ndcg5": 0.9775325271359823, + "ndcg10": 0.9775325271359823, + "p50_ms": 0.917520490474999, + "p90_ms": 3.363933577202262, + "p95_ms": 4.407016342156568, + "p99_ms": 5.707503252779132, + "index_ms": 205.70454199332744, + "by_category": { + "architecture": 1.0, + "semantic": 0.9591500493381495, + "symbol": 1.0 + } + }, + { + "repo": "alamofire", + "language": "swift", + "mode": "semble-bm25", + "chunks": 1302, + "tokens": 1604, + "ndcg5": 0.969504606050801, + "ndcg10": 0.969504606050801, + "p50_ms": 0.5209585069678724, + "p90_ms": 2.8024661063682297, + "p95_ms": 3.4025649394607185, + "p99_ms": 5.010312988306393, + "index_ms": 205.70454199332744, + "by_category": { + "architecture": 0.9732402630493958, + "semantic": 0.9518519392607121, + "symbol": 1.0 + } + }, + { + "repo": "alamofire", + "language": "swift", + "mode": "semble-semantic", + "chunks": 1302, + "tokens": 1471, + "ndcg5": 0.9815464876785729, + "ndcg10": 0.9815464876785729, + "p50_ms": 0.5615830013994128, + "p90_ms": 3.2491915975697347, + "p95_ms": 3.815926477545874, + "p99_ms": 5.4785852984059575, + "index_ms": 205.70454199332744, + "by_category": { + "architecture": 1.0, + "semantic": 0.966448159415587, + "symbol": 1.0 + } + }, + { + "repo": "alamofire", + "language": "swift", + "mode": "unranked-auto", + "chunks": 1302, + "tokens": 1561, + "ndcg5": 0.7671261445939673, + "ndcg10": 0.7780465164348754, + "p50_ms": 0.2869169693440199, + "p90_ms": 0.31272495980374515, + "p95_ms": 0.330504184239544, + "p99_ms": 0.34766726486850524, + "index_ms": 205.70454199332744, + "by_category": { + "architecture": 0.8211640327667377, + "semantic": 0.7361307482179362, + "symbol": 0.8333333333333334 + } + }, + { + "repo": "alamofire", + "language": "swift", + "mode": "unranked-balanced", + "chunks": 1302, + "tokens": 1559, + "ndcg5": 0.7671261445939673, + "ndcg10": 0.7924997577507699, + "p50_ms": 0.3352915227878839, + "p90_ms": 0.6589496741071345, + "p95_ms": 0.7094166066963227, + "p99_ms": 0.7765497139189391, + "index_ms": 205.70454199332744, + "by_category": { + "architecture": 0.8211640327667377, + "semantic": 0.7361307482179362, + "symbol": 0.8815108043863146 + } + }, + { + "repo": "alamofire", + "language": "swift", + "mode": "unranked-bm25", + "chunks": 1302, + "tokens": 1540, + "ndcg5": 0.8077080121794307, + "ndcg10": 0.8282997042012024, + "p50_ms": 0.35758400917984545, + "p90_ms": 0.5357539048418403, + "p95_ms": 0.5437045008875431, + "p99_ms": 0.6069744960404931, + "index_ms": 205.70454199332744, + "by_category": { + "architecture": 0.8301208360345026, + "semantic": 0.8250574159927763, + "symbol": 0.8333333333333334 + } + }, + { + "repo": "alamofire", + "language": "swift", + "mode": "unranked-semantic", + "chunks": 1302, + "tokens": 1514, + "ndcg5": 0.7875006699908373, + "ndcg10": 0.8086401617111699, + "p50_ms": 0.29860451468266547, + "p90_ms": 0.5145334114786237, + "p95_ms": 0.521611055592075, + "p99_ms": 0.5220222019124776, + "index_ms": 205.70454199332744, + "by_category": { + "architecture": 0.8769765845238192, + "semantic": 0.719176702461862, + "symbol": 0.9384882922619097 + } + }, + { + "repo": "axios", + "language": "javascript", + "mode": "semble-auto", + "chunks": 299, + "tokens": 1395, + "ndcg5": 0.855657359638273, + "ndcg10": 0.866577731479181, + "p50_ms": 0.6078125152271241, + "p90_ms": 2.003629098180682, + "p95_ms": 2.2011098044458777, + "p99_ms": 2.432055567624047, + "index_ms": 46.320166962686926, + "by_category": { + "architecture": 0.6276256451932702, + "semantic": 0.91038071151303, + "symbol": 1.0 + } + }, + { + "repo": "axios", + "language": "javascript", + "mode": "semble-balanced", + "chunks": 299, + "tokens": 1400, + "ndcg5": 0.855657359638273, + "ndcg10": 0.866577731479181, + "p50_ms": 0.6171044951770455, + "p90_ms": 2.0596329821273685, + "p95_ms": 2.153324699611403, + "p99_ms": 2.1854985581012443, + "index_ms": 46.320166962686926, + "by_category": { + "architecture": 0.6276256451932702, + "semantic": 0.91038071151303, + "symbol": 1.0 + } + }, + { + "repo": "axios", + "language": "javascript", + "mode": "semble-bm25", + "chunks": 299, + "tokens": 1392, + "ndcg5": 0.855241398894884, + "ndcg10": 0.855241398894884, + "p50_ms": 0.46874998952262104, + "p90_ms": 1.5414249966852367, + "p95_ms": 1.558945799479261, + "p99_ms": 1.6132219799328595, + "index_ms": 46.320166962686926, + "by_category": { + "architecture": 0.5508861261352197, + "semantic": 0.9278219274690644, + "symbol": 1.0 + } + }, + { + "repo": "axios", + "language": "javascript", + "mode": "semble-semantic", + "chunks": 299, + "tokens": 1405, + "ndcg5": 0.8694059321288214, + "ndcg10": 0.8860725987954883, + "p50_ms": 0.4615210054907948, + "p90_ms": 1.6068294062279167, + "p95_ms": 1.710555562749505, + "p99_ms": 1.777277523651719, + "index_ms": 46.320166962686926, + "by_category": { + "architecture": 0.7141062092205258, + "semantic": 0.9056578810896814, + "symbol": 1.0 + } + }, + { + "repo": "axios", + "language": "javascript", + "mode": "unranked-auto", + "chunks": 299, + "tokens": 1437, + "ndcg5": 0.6537874476332355, + "ndcg10": 0.7073668498595211, + "p50_ms": 0.2414585032965988, + "p90_ms": 0.26166673051193357, + "p95_ms": 0.27327534626238054, + "p99_ms": 0.3072223009075969, + "index_ms": 46.320166962686926, + "by_category": { + "architecture": 0.5948896184542856, + "semantic": 0.691832697173534, + "symbol": 0.8243991050595311 + } + }, + { + "repo": "axios", + "language": "javascript", + "mode": "unranked-balanced", + "chunks": 299, + "tokens": 1444, + "ndcg5": 0.6722409599546626, + "ndcg10": 0.7267137850083284, + "p50_ms": 0.24872898939065635, + "p90_ms": 0.2676792209967971, + "p95_ms": 0.2806024276651442, + "p99_ms": 0.29862049268558616, + "index_ms": 46.320166962686926, + "by_category": { + "architecture": 0.5948896184542856, + "semantic": 0.691832697173534, + "symbol": 0.8888888888888888 + } + }, + { + "repo": "axios", + "language": "javascript", + "mode": "unranked-bm25", + "chunks": 299, + "tokens": 1399, + "ndcg5": 0.6222352819387953, + "ndcg10": 0.6463598434355532, + "p50_ms": 0.2400419907644391, + "p90_ms": 0.3204794018529356, + "p95_ms": 0.36232024431228654, + "p99_ms": 0.5622640531510111, + "index_ms": 46.320166962686926, + "by_category": { + "architecture": 0.484197324898015, + "semantic": 0.6206906293701827, + "symbol": 0.8199990966482242 + } + }, + { + "repo": "axios", + "language": "javascript", + "mode": "unranked-semantic", + "chunks": 299, + "tokens": 1436, + "ndcg5": 0.6998149480791669, + "ndcg10": 0.7833415321317966, + "p50_ms": 0.23508348385803401, + "p90_ms": 0.2639868762344122, + "p95_ms": 0.2677416166989133, + "p99_ms": 0.28091471118386835, + "index_ms": 46.320166962686926, + "by_category": { + "architecture": 0.7991549802711949, + "semantic": 0.8321335171386328, + "symbol": 0.6969756811720434 + } + }, + { + "repo": "axum", + "language": "rust", + "mode": "semble-auto", + "chunks": 1034, + "tokens": 1657, + "ndcg5": 0.7815464876785729, + "ndcg10": 0.8151300908732605, + "p50_ms": 0.8129795314744115, + "p90_ms": 3.3843710785731673, + "p95_ms": 3.471860141144135, + "p99_ms": 4.636972060543483, + "index_ms": 163.39195799082518, + "by_category": { + "architecture": 0.8, + "semantic": 0.8288662191964322, + "symbol": 0.7854023957026741 + } + }, + { + "repo": "axum", + "language": "rust", + "mode": "semble-balanced", + "chunks": 1034, + "tokens": 1654, + "ndcg5": 0.7815464876785729, + "ndcg10": 0.8151300908732605, + "p50_ms": 0.6873959791846573, + "p90_ms": 3.3376250066794455, + "p95_ms": 3.471177141182126, + "p99_ms": 4.902669009752569, + "index_ms": 163.39195799082518, + "by_category": { + "architecture": 0.8, + "semantic": 0.8288662191964322, + "symbol": 0.7854023957026741 + } + }, + { + "repo": "axum", + "language": "rust", + "mode": "semble-bm25", + "chunks": 1034, + "tokens": 1674, + "ndcg5": 0.7685709273335515, + "ndcg10": 0.7685709273335515, + "p50_ms": 0.5297499883454293, + "p90_ms": 2.218325278954581, + "p95_ms": 2.453783302917146, + "p99_ms": 3.377689469489267, + "index_ms": 163.39195799082518, + "by_category": { + "architecture": 0.6496411846762655, + "semantic": 0.8077113387680258, + "symbol": 0.8102255193577976 + } + }, + { + "repo": "axum", + "language": "rust", + "mode": "semble-semantic", + "chunks": 1034, + "tokens": 1565, + "ndcg5": 0.7746141434859123, + "ndcg10": 0.7903873873251988, + "p50_ms": 0.49581247731111944, + "p90_ms": 2.0314164110459387, + "p95_ms": 2.1347794856410482, + "p99_ms": 3.1130894587840867, + "index_ms": 163.39195799082518, + "by_category": { + "architecture": 0.7861353116146785, + "semantic": 0.7968005259704043, + "symbol": 0.7718216255952429 + } + }, + { + "repo": "axum", + "language": "rust", + "mode": "unranked-auto", + "chunks": 1034, + "tokens": 1658, + "ndcg5": 0.7315464876785729, + "ndcg10": 0.749356847033974, + "p50_ms": 0.2861669927369803, + "p90_ms": 0.3412872087210417, + "p95_ms": 0.35066461714450275, + "p99_ms": 0.413966507767327, + "index_ms": 163.39195799082518, + "by_category": { + "architecture": 0.5974273881358959, + "semantic": 0.7916666666666666, + "symbol": 0.8333333333333334 + } + }, + { + "repo": "axum", + "language": "rust", + "mode": "unranked-balanced", + "chunks": 1034, + "tokens": 1664, + "ndcg5": 0.7280803155822426, + "ndcg10": 0.7458906749376437, + "p50_ms": 0.3228335117455572, + "p90_ms": 0.38804608047939837, + "p95_ms": 0.40257260261569183, + "p99_ms": 0.5373145063640548, + "index_ms": 163.39195799082518, + "by_category": { + "architecture": 0.5974273881358959, + "semantic": 0.7916666666666666, + "symbol": 0.8102255193577976 + } + }, + { + "repo": "axum", + "language": "rust", + "mode": "unranked-bm25", + "chunks": 1034, + "tokens": 1672, + "ndcg5": 0.7180676558073393, + "ndcg10": 0.7347343224740059, + "p50_ms": 0.26833300944417715, + "p90_ms": 0.3214121737983078, + "p95_ms": 0.3277624928159639, + "p99_ms": 0.32795249426271766, + "index_ms": 163.39195799082518, + "by_category": { + "architecture": 0.5528019782813453, + "semantic": 0.7858897131727828, + "symbol": 0.8333333333333334 + } + }, + { + "repo": "axum", + "language": "rust", + "mode": "unranked-semantic", + "chunks": 1034, + "tokens": 1637, + "ndcg5": 0.6065464876785729, + "ndcg10": 0.652717895444333, + "p50_ms": 0.26172897196374834, + "p90_ms": 0.279124645749107, + "p95_ms": 0.2814499574014917, + "p99_ms": 0.29399000864941627, + "index_ms": 163.39195799082518, + "by_category": { + "architecture": 0.7244796319302442, + "semantic": 0.5942441461309548, + "symbol": 0.7670099985546605 + } + }, + { + "repo": "bash-it", + "language": "bash", + "mode": "semble-auto", + "chunks": 1319, + "tokens": 1344, + "ndcg5": 0.6056128427182248, + "ndcg10": 0.6525424269728923, + "p50_ms": 1.448958006221801, + "p90_ms": 2.0759753067977726, + "p95_ms": 2.1869812248041867, + "p99_ms": 3.5242962109623455, + "index_ms": 493.6942500062287, + "by_category": { + "architecture": 0.8154648767857288, + "semantic": 0.6044855749416566, + "symbol": 0.7123212623289701 + } + }, + { + "repo": "bash-it", + "language": "bash", + "mode": "semble-balanced", + "chunks": 1319, + "tokens": 1344, + "ndcg5": 0.6056128427182248, + "ndcg10": 0.6525424269728923, + "p50_ms": 1.3088124978821725, + "p90_ms": 1.4810255845077336, + "p95_ms": 1.511504192603752, + "p99_ms": 1.5552680508699268, + "index_ms": 493.6942500062287, + "by_category": { + "architecture": 0.8154648767857288, + "semantic": 0.6044855749416566, + "symbol": 0.7123212623289701 + } + }, + { + "repo": "bash-it", + "language": "bash", + "mode": "semble-bm25", + "chunks": 1319, + "tokens": 1388, + "ndcg5": 0.6531436073493369, + "ndcg10": 0.6794815941968672, + "p50_ms": 0.9484170004725456, + "p90_ms": 1.1577624711208045, + "p95_ms": 1.1971166502917185, + "p99_ms": 1.263489707489498, + "index_ms": 493.6942500062287, + "by_category": { + "architecture": 0.75, + "semantic": 0.6647055973121498, + "symbol": 0.6896918237758785 + } + }, + { + "repo": "bash-it", + "language": "bash", + "mode": "semble-semantic", + "chunks": 1319, + "tokens": 1238, + "ndcg5": 0.5951156862641558, + "ndcg10": 0.6298731021090032, + "p50_ms": 1.04302100953646, + "p90_ms": 1.1553378019016236, + "p95_ms": 1.1577354423934594, + "p99_ms": 1.1708143103169277, + "index_ms": 493.6942500062287, + "by_category": { + "architecture": 0.75, + "semantic": 0.5883848402538977, + "symbol": 0.6896918237758785 + } + }, + { + "repo": "bash-it", + "language": "bash", + "mode": "unranked-auto", + "chunks": 1319, + "tokens": 1418, + "ndcg5": 0.617035370681913, + "ndcg10": 0.654649358203016, + "p50_ms": 0.6683130050078034, + "p90_ms": 0.7827048364561052, + "p95_ms": 0.8281857473775746, + "p99_ms": 0.9215707285329698, + "index_ms": 493.6942500062287, + "by_category": { + "architecture": 1.0, + "semantic": 0.5213478682518915, + "symbol": 0.8630929753571458 + } + }, + { + "repo": "bash-it", + "language": "bash", + "mode": "unranked-balanced", + "chunks": 1319, + "tokens": 1418, + "ndcg5": 0.617035370681913, + "ndcg10": 0.654649358203016, + "p50_ms": 0.6274580082390457, + "p90_ms": 0.7604493759572507, + "p95_ms": 0.7989707926753908, + "p99_ms": 0.8156269520986825, + "index_ms": 493.6942500062287, + "by_category": { + "architecture": 1.0, + "semantic": 0.5213478682518915, + "symbol": 0.8630929753571458 + } + }, + { + "repo": "bash-it", + "language": "bash", + "mode": "unranked-bm25", + "chunks": 1319, + "tokens": 1478, + "ndcg5": 0.5875696397737991, + "ndcg10": 0.6378422881146872, + "p50_ms": 0.5682290066033602, + "p90_ms": 0.7272416609339417, + "p95_ms": 0.7911732507636771, + "p99_ms": 0.8406682516215368, + "index_ms": 493.6942500062287, + "by_category": { + "architecture": 0.6577324383928644, + "semantic": 0.6162080251872786, + "symbol": 0.6861353116146786 + } + }, + { + "repo": "bash-it", + "language": "bash", + "mode": "unranked-semantic", + "chunks": 1319, + "tokens": 1298, + "ndcg5": 0.5761615245008577, + "ndcg10": 0.6157261968610789, + "p50_ms": 0.26324953068979084, + "p90_ms": 0.29936666833236814, + "p95_ms": 0.31517528113909066, + "p99_ms": 0.3172014618758112, + "index_ms": 493.6942500062287, + "by_category": { + "architecture": 1.0, + "semantic": 0.5241941490170444, + "symbol": 0.7 + } + }, + { + "repo": "bats-core", + "language": "bash", + "mode": "semble-auto", + "chunks": 97, + "tokens": 1505, + "ndcg5": 0.8661732909393883, + "ndcg10": 0.8661732909393883, + "p50_ms": 0.5249995156191289, + "p90_ms": 0.5931413965299726, + "p95_ms": 0.6065252848202363, + "p99_ms": 0.6138714618282393, + "index_ms": 17.10820896551013, + "by_category": { + "architecture": 0.7230866159492644, + "semantic": 0.9432199620879166 + } + }, + { + "repo": "bats-core", + "language": "bash", + "mode": "semble-balanced", + "chunks": 97, + "tokens": 1505, + "ndcg5": 0.8661732909393883, + "ndcg10": 0.8661732909393883, + "p50_ms": 0.5183540051802993, + "p90_ms": 0.5909581203013659, + "p95_ms": 0.5944485863437876, + "p99_ms": 0.5961897427914664, + "index_ms": 17.10820896551013, + "by_category": { + "architecture": 0.7230866159492644, + "semantic": 0.9432199620879166 + } + }, + { + "repo": "bats-core", + "language": "bash", + "mode": "semble-bm25", + "chunks": 97, + "tokens": 1471, + "ndcg5": 0.8530803155822426, + "ndcg10": 0.8530803155822426, + "p50_ms": 0.44249999336898327, + "p90_ms": 0.4820707952603698, + "p95_ms": 0.4977163771400228, + "p99_ms": 0.5206432921113446, + "index_ms": 17.10820896551013, + "by_category": { + "architecture": 0.7043823654390561, + "semantic": 0.9331484425824198 + } + }, + { + "repo": "bats-core", + "language": "bash", + "mode": "semble-semantic", + "chunks": 97, + "tokens": 1437, + "ndcg5": 0.8346268032608155, + "ndcg10": 0.8504000471001019, + "p50_ms": 0.46608300181105733, + "p90_ms": 0.5349118902813644, + "p95_ms": 0.5930062761763111, + "p99_ms": 0.6045012531103566, + "index_ms": 17.10820896551013, + "by_category": { + "architecture": 0.706627804336741, + "semantic": 0.927815870126527 + } + }, + { + "repo": "bats-core", + "language": "bash", + "mode": "unranked-auto", + "chunks": 97, + "tokens": 1497, + "ndcg5": 0.9530803155822426, + "ndcg10": 0.9530803155822426, + "p50_ms": 0.24718750501051545, + "p90_ms": 0.2713333058636636, + "p95_ms": 0.2957704768050462, + "p99_ms": 0.3138204815331846, + "index_ms": 17.10820896551013, + "by_category": { + "architecture": 0.9186680797247704, + "semantic": 0.9716099810439582 + } + }, + { + "repo": "bats-core", + "language": "bash", + "mode": "unranked-balanced", + "chunks": 97, + "tokens": 1497, + "ndcg5": 0.9530803155822426, + "ndcg10": 0.9530803155822426, + "p50_ms": 0.25014596758410335, + "p90_ms": 0.272200332256034, + "p95_ms": 0.27637925231829286, + "p99_ms": 0.30114305438473815, + "index_ms": 17.10820896551013, + "by_category": { + "architecture": 0.9186680797247704, + "semantic": 0.9716099810439582 + } + }, + { + "repo": "bats-core", + "language": "bash", + "mode": "unranked-bm25", + "chunks": 97, + "tokens": 1508, + "ndcg5": 0.8880929753571458, + "ndcg10": 0.9047596420238125, + "p50_ms": 0.24054149980656803, + "p90_ms": 0.26567078894004226, + "p95_ms": 0.26723927876446396, + "p99_ms": 0.26771426026243716, + "index_ms": 17.10820896551013, + "by_category": { + "architecture": 0.9472756790816368, + "semantic": 0.8818663913003685 + } + }, + { + "repo": "bats-core", + "language": "bash", + "mode": "unranked-semantic", + "chunks": 97, + "tokens": 1479, + "ndcg5": 0.8530803155822426, + "ndcg10": 0.8530803155822426, + "p50_ms": 0.24568746448494494, + "p90_ms": 0.26195000973530114, + "p95_ms": 0.272035421221517, + "p99_ms": 0.2953734825132414, + "index_ms": 17.10820896551013, + "by_category": { + "architecture": 0.804418536224494, + "semantic": 0.8792828121594918 + } + }, + { + "repo": "cats", + "language": "scala", + "mode": "semble-auto", + "chunks": 2393, + "tokens": 1478, + "ndcg5": 0.9113147192765458, + "ndcg10": 0.9113147192765458, + "p50_ms": 1.5996044967323542, + "p90_ms": 5.902458302443847, + "p95_ms": 6.323799715028146, + "p99_ms": 8.42469357710797, + "index_ms": 615.3576669748873, + "by_category": { + "architecture": 0.8065735963827292, + "semantic": 0.9, + "symbol": 1.0 + } + }, + { + "repo": "cats", + "language": "scala", + "mode": "semble-balanced", + "chunks": 2393, + "tokens": 1480, + "ndcg5": 0.9113147192765458, + "ndcg10": 0.9113147192765458, + "p50_ms": 1.4887709694448858, + "p90_ms": 5.565937515348196, + "p95_ms": 5.766212515300142, + "p99_ms": 7.399642522213978, + "index_ms": 615.3576669748873, + "by_category": { + "architecture": 0.8065735963827292, + "semantic": 0.9, + "symbol": 1.0 + } + }, + { + "repo": "cats", + "language": "scala", + "mode": "semble-bm25", + "chunks": 2393, + "tokens": 1527, + "ndcg5": 0.8928612069551187, + "ndcg10": 0.8928612069551187, + "p50_ms": 1.3085624668747187, + "p90_ms": 4.54218327649869, + "p95_ms": 4.801139267510735, + "p99_ms": 5.941994261811486, + "index_ms": 615.3576669748873, + "by_category": { + "architecture": 0.8065735963827292, + "semantic": 0.8630929753571458, + "symbol": 1.0 + } + }, + { + "repo": "cats", + "language": "scala", + "mode": "semble-semantic", + "chunks": 2393, + "tokens": 1402, + "ndcg5": 0.8928612069551187, + "ndcg10": 0.8928612069551187, + "p50_ms": 1.4404165267478675, + "p90_ms": 4.761274671182037, + "p95_ms": 5.4902020609006295, + "p99_ms": 6.762474044226108, + "index_ms": 615.3576669748873, + "by_category": { + "architecture": 0.7143060347755936, + "semantic": 0.9, + "symbol": 1.0 + } + }, + { + "repo": "cats", + "language": "scala", + "mode": "unranked-auto", + "chunks": 2393, + "tokens": 1545, + "ndcg5": 0.7546735197420095, + "ndcg10": 0.7546735197420095, + "p50_ms": 0.32010453287512064, + "p90_ms": 0.3892162814736367, + "p95_ms": 0.4188973223790527, + "p99_ms": 0.4756130417808889, + "index_ms": 615.3576669748873, + "by_category": { + "architecture": 0.6671758726532829, + "semantic": 0.7732230839010749, + "symbol": 0.782089344202718 + } + }, + { + "repo": "cats", + "language": "scala", + "mode": "unranked-balanced", + "chunks": 2393, + "tokens": 1533, + "ndcg5": 0.7831396918383398, + "ndcg10": 0.7831396918383398, + "p50_ms": 0.32054149778559804, + "p90_ms": 0.3806208376772702, + "p95_ms": 0.3886038553901017, + "p99_ms": 0.4250207846052944, + "index_ms": 615.3576669748873, + "by_category": { + "architecture": 0.6671758726532829, + "semantic": 0.7732230839010749, + "symbol": 0.8769765845238192 + } + }, + { + "repo": "cats", + "language": "scala", + "mode": "unranked-bm25", + "chunks": 2393, + "tokens": 1540, + "ndcg5": 0.593223943480546, + "ndcg10": 0.6191193949861911, + "p50_ms": 0.30264598899520934, + "p90_ms": 0.36857976811006665, + "p95_ms": 0.38647742185276, + "p99_ms": 0.4025955166434869, + "index_ms": 615.3576669748873, + "by_category": { + "architecture": 0.46036960056071, + "semantic": 0.5779049990338064, + "symbol": 0.793643251190486 + } + }, + { + "repo": "cats", + "language": "scala", + "mode": "unranked-semantic", + "chunks": 2393, + "tokens": 1483, + "ndcg5": 0.7345495600271248, + "ndcg10": 0.7599231731839274, + "p50_ms": 0.29708349029533565, + "p90_ms": 0.3624793840572238, + "p95_ms": 0.3738722938578576, + "p99_ms": 0.3846080170478672, + "index_ms": 615.3576669748873, + "by_category": { + "architecture": 0.519528495522843, + "semantic": 0.798941972801572, + "symbol": 0.8551549589285763 + } + }, + { + "repo": "chi", + "language": "go", + "mode": "semble-auto", + "chunks": 508, + "tokens": 1436, + "ndcg5": 0.8243610443449215, + "ndcg10": 0.8699498574344997, + "p50_ms": 0.6240209913812578, + "p90_ms": 2.102387318154797, + "p95_ms": 2.19170362688601, + "p99_ms": 2.2910407464951272, + "index_ms": 99.80416699545458, + "by_category": { + "architecture": 0.7546946133749226, + "semantic": 0.9444444444444444, + "symbol": 0.8769765845238192 + } + }, + { + "repo": "chi", + "language": "go", + "mode": "semble-balanced", + "chunks": 508, + "tokens": 1430, + "ndcg5": 0.8243610443449215, + "ndcg10": 0.8699498574344997, + "p50_ms": 0.7442709757015109, + "p90_ms": 2.529487496940419, + "p95_ms": 2.6409500424051653, + "p99_ms": 2.875323643092997, + "index_ms": 99.80416699545458, + "by_category": { + "architecture": 0.7546946133749226, + "semantic": 0.9444444444444444, + "symbol": 0.8769765845238192 + } + }, + { + "repo": "chi", + "language": "go", + "mode": "semble-bm25", + "chunks": 508, + "tokens": 1479, + "ndcg5": 0.7760793639877802, + "ndcg10": 0.8080264799872139, + "p50_ms": 0.6144794751890004, + "p90_ms": 1.5702581906225532, + "p95_ms": 1.7293794488068663, + "p99_ms": 2.032375902635976, + "index_ms": 99.80416699545458, + "by_category": { + "architecture": 0.713769522104489, + "semantic": 0.847881083730162, + "symbol": 0.8769765845238192 + } + }, + { + "repo": "chi", + "language": "go", + "mode": "semble-semantic", + "chunks": 508, + "tokens": 1469, + "ndcg5": 0.784846039730822, + "ndcg10": 0.8083343638645248, + "p50_ms": 0.46122950152494013, + "p90_ms": 1.6213713912293315, + "p95_ms": 1.6317733039613813, + "p99_ms": 1.6600218589883298, + "index_ms": 99.80416699545458, + "by_category": { + "architecture": 0.6710499225459361, + "semantic": 0.8769765845238192, + "symbol": 0.8769765845238192 + } + }, + { + "repo": "chi", + "language": "go", + "mode": "unranked-auto", + "chunks": 508, + "tokens": 1474, + "ndcg5": 0.7750943573700484, + "ndcg10": 0.8107164279936492, + "p50_ms": 0.2502089773770422, + "p90_ms": 0.27464129962027073, + "p95_ms": 0.2769606886431575, + "p99_ms": 0.28509209398180246, + "index_ms": 99.80416699545458, + "by_category": { + "architecture": 0.6500599397782295, + "semantic": 0.8957340346272057, + "symbol": 0.8769765845238192 + } + }, + { + "repo": "chi", + "language": "go", + "mode": "unranked-balanced", + "chunks": 508, + "tokens": 1494, + "ndcg5": 0.7750943573700484, + "ndcg10": 0.8107164279936492, + "p50_ms": 0.2539374982006848, + "p90_ms": 0.2853876387234777, + "p95_ms": 0.2890044852392748, + "p99_ms": 0.2894008910516277, + "index_ms": 99.80416699545458, + "by_category": { + "architecture": 0.6500599397782295, + "semantic": 0.8957340346272057, + "symbol": 0.8769765845238192 + } + }, + { + "repo": "chi", + "language": "go", + "mode": "unranked-bm25", + "chunks": 508, + "tokens": 1509, + "ndcg5": 0.7849992763046483, + "ndcg10": 0.8062004809920114, + "p50_ms": 0.23893750039860606, + "p90_ms": 0.27545817429199815, + "p95_ms": 0.28824815235566337, + "p99_ms": 0.32771604077424843, + "index_ms": 99.80416699545458, + "by_category": { + "architecture": 0.6438160572464576, + "semantic": 0.8908647289784444, + "symbol": 0.8769765845238192 + } + }, + { + "repo": "chi", + "language": "go", + "mode": "unranked-semantic", + "chunks": 508, + "tokens": 1440, + "ndcg5": 0.7600354175668508, + "ndcg10": 0.7922699964769496, + "p50_ms": 0.2390210283920169, + "p90_ms": 0.26015459443442523, + "p95_ms": 0.2783089614240452, + "p99_ms": 0.29462896694894875, + "index_ms": 99.80416699545458, + "by_category": { + "architecture": 0.6635363114393633, + "semantic": 0.8088491186609905, + "symbol": 1.0 + } + }, + { + "repo": "circe", + "language": "scala", + "mode": "semble-auto", + "chunks": 379, + "tokens": 1505, + "ndcg5": 0.8979792795602554, + "ndcg10": 0.8979792795602554, + "p50_ms": 1.249917026143521, + "p90_ms": 3.1432000454515214, + "p95_ms": 3.784804214956239, + "p99_ms": 3.8301944418344647, + "index_ms": 85.11912496760488, + "by_category": { + "architecture": 0.7261859507142916, + "semantic": 0.9482433234612176, + "symbol": 1.0 + } + }, + { + "repo": "circe", + "language": "scala", + "mode": "semble-balanced", + "chunks": 379, + "tokens": 1507, + "ndcg5": 0.8979792795602554, + "ndcg10": 0.8979792795602554, + "p50_ms": 0.6219169590622187, + "p90_ms": 3.2576918136328454, + "p95_ms": 3.9886881131678815, + "p99_ms": 4.094737628474832, + "index_ms": 85.11912496760488, + "by_category": { + "architecture": 0.7261859507142916, + "semantic": 0.9482433234612176, + "symbol": 1.0 + } + }, + { + "repo": "circe", + "language": "scala", + "mode": "semble-bm25", + "chunks": 379, + "tokens": 1666, + "ndcg5": 0.8956727663582104, + "ndcg10": 0.8956727663582104, + "p50_ms": 0.49462501192465425, + "p90_ms": 2.883267006836831, + "p95_ms": 3.2033419876825064, + "p99_ms": 3.2118019682820886, + "index_ms": 85.11912496760488, + "by_category": { + "architecture": 0.8, + "semantic": 0.9107075055278181, + "symbol": 1.0 + } + }, + { + "repo": "circe", + "language": "scala", + "mode": "semble-semantic", + "chunks": 379, + "tokens": 1478, + "ndcg5": 0.8397050301241697, + "ndcg10": 0.8572488897732926, + "p50_ms": 0.46962499618530273, + "p90_ms": 2.981116424780339, + "p95_ms": 2.9899791174102575, + "p99_ms": 3.0348286277148873, + "index_ms": 85.11912496760488, + "by_category": { + "architecture": 0.7928526173809582, + "semantic": 0.8475878017079789, + "symbol": 1.0 + } + }, + { + "repo": "circe", + "language": "scala", + "mode": "unranked-auto", + "chunks": 379, + "tokens": 1527, + "ndcg5": 0.8226805133458662, + "ndcg10": 0.8378944515731235, + "p50_ms": 0.3019159776158631, + "p90_ms": 0.4179582116194069, + "p95_ms": 0.4668548121117055, + "p99_ms": 0.47690454637631774, + "index_ms": 85.11912496760488, + "by_category": { + "architecture": 0.6261859507142915, + "semantic": 0.8899149842107171, + "symbol": 1.0 + } + }, + { + "repo": "circe", + "language": "scala", + "mode": "unranked-balanced", + "chunks": 379, + "tokens": 1528, + "ndcg5": 0.8226805133458662, + "ndcg10": 0.8378944515731235, + "p50_ms": 0.26879203505814075, + "p90_ms": 0.29924981063231826, + "p95_ms": 0.3398002940230069, + "p99_ms": 0.42536007938906556, + "index_ms": 85.11912496760488, + "by_category": { + "architecture": 0.6261859507142915, + "semantic": 0.8899149842107171, + "symbol": 1.0 + } + }, + { + "repo": "circe", + "language": "scala", + "mode": "unranked-bm25", + "chunks": 379, + "tokens": 1628, + "ndcg5": 0.7973006481251294, + "ndcg10": 0.7973006481251294, + "p50_ms": 0.2636670251376927, + "p90_ms": 0.2774503780528903, + "p95_ms": 0.2865422109607607, + "p99_ms": 0.289841202320531, + "index_ms": 85.11912496760488, + "by_category": { + "architecture": 0.6035565121611999, + "semantic": 0.8300845230519507, + "symbol": 1.0 + } + }, + { + "repo": "circe", + "language": "scala", + "mode": "unranked-semantic", + "chunks": 379, + "tokens": 1545, + "ndcg5": 0.7373718647644338, + "ndcg10": 0.7373718647644338, + "p50_ms": 0.2602919703349471, + "p90_ms": 0.2793917781673372, + "p95_ms": 0.28105470701120794, + "p99_ms": 0.2817437413614243, + "index_ms": 85.11912496760488, + "by_category": { + "architecture": 0.48969182377587844, + "semantic": 0.7783278465131683, + "symbol": 1.0 + } + }, + { + "repo": "click", + "language": "python", + "mode": "semble-auto", + "chunks": 610, + "tokens": 1884, + "ndcg5": 1.0, + "ndcg10": 1.0, + "p50_ms": 0.4994999908376485, + "p90_ms": 2.3017923056613654, + "p95_ms": 2.4099399015540257, + "p99_ms": 2.605987995048053, + "index_ms": 86.24508301727474, + "by_category": { + "architecture": 1.0, + "semantic": 1.0, + "symbol": 1.0 + } + }, + { + "repo": "click", + "language": "python", + "mode": "semble-balanced", + "chunks": 610, + "tokens": 1857, + "ndcg5": 1.0, + "ndcg10": 1.0, + "p50_ms": 0.49847899936139584, + "p90_ms": 2.2167538176290695, + "p95_ms": 2.3762431257637218, + "p99_ms": 2.537015802809037, + "index_ms": 86.24508301727474, + "by_category": { + "architecture": 1.0, + "semantic": 1.0, + "symbol": 1.0 + } + }, + { + "repo": "click", + "language": "python", + "mode": "semble-bm25", + "chunks": 610, + "tokens": 1896, + "ndcg5": 0.9590790148145552, + "ndcg10": 0.9590790148145552, + "p50_ms": 0.38839547778479755, + "p90_ms": 1.4742458995897323, + "p95_ms": 1.5516276762355121, + "p99_ms": 1.8733919190708543, + "index_ms": 86.24508301727474, + "by_category": { + "architecture": 0.986620131524698, + "semantic": 0.9261859507142916, + "symbol": 1.0 + } + }, + { + "repo": "click", + "language": "python", + "mode": "semble-semantic", + "chunks": 610, + "tokens": 1615, + "ndcg5": 0.9446394630357187, + "ndcg10": 0.9446394630357187, + "p50_ms": 0.3676459891721606, + "p90_ms": 1.7381460871547463, + "p95_ms": 1.9220309739466757, + "p99_ms": 1.9839397713076323, + "index_ms": 86.24508301727474, + "by_category": { + "architecture": 0.9384882922619097, + "semantic": 0.9630929753571458, + "symbol": 0.9077324383928644 + } + }, + { + "repo": "click", + "language": "python", + "mode": "unranked-auto", + "chunks": 610, + "tokens": 1875, + "ndcg5": 0.9437503349954188, + "ndcg10": 0.952612299334621, + "p50_ms": 0.24947948986664414, + "p90_ms": 0.26419172645546496, + "p95_ms": 0.26843785017263144, + "p99_ms": 0.27255475462879986, + "index_ms": 86.24508301727474, + "by_category": { + "architecture": 0.965064413258251, + "semantic": 0.9630929753571458, + "symbol": 0.9077324383928644 + } + }, + { + "repo": "click", + "language": "python", + "mode": "unranked-balanced", + "chunks": 610, + "tokens": 1883, + "ndcg5": 0.9372038473168459, + "ndcg10": 0.9460658116560481, + "p50_ms": 0.24824999854899943, + "p90_ms": 0.26046697166748345, + "p95_ms": 0.26245653280057013, + "p99_ms": 0.2687576797325164, + "index_ms": 86.24508301727474, + "by_category": { + "architecture": 0.965064413258251, + "semantic": 0.9630929753571458, + "symbol": 0.875 + } + }, + { + "repo": "click", + "language": "python", + "mode": "unranked-bm25", + "chunks": 610, + "tokens": 1977, + "ndcg5": 0.9047210925935836, + "ndcg10": 0.9213877592602502, + "p50_ms": 0.24037549155764282, + "p90_ms": 0.2628246962558478, + "p95_ms": 0.2661895676283166, + "p99_ms": 0.2671707182889804, + "index_ms": 86.24508301727474, + "by_category": { + "architecture": 0.9105820163833688, + "semantic": 0.896426308690479, + "symbol": 1.0 + } + }, + { + "repo": "click", + "language": "python", + "mode": "unranked-semantic", + "chunks": 610, + "tokens": 1708, + "ndcg5": 0.8004072534454754, + "ndcg10": 0.8326708541167708, + "p50_ms": 0.24212501011788845, + "p90_ms": 0.25990839349105954, + "p95_ms": 0.2658880810486153, + "p99_ms": 0.2667104179272428, + "index_ms": 86.24508301727474, + "by_category": { + "architecture": 0.9180241781515845, + "semantic": 0.764527201342591, + "symbol": 0.875 + } + }, + { + "repo": "cobra", + "language": "go", + "mode": "semble-auto", + "chunks": 780, + "tokens": 2530, + "ndcg5": 0.9709860394574094, + "ndcg10": 0.9709860394574094, + "p50_ms": 0.7444580260198563, + "p90_ms": 4.801692295586691, + "p95_ms": 5.247285758377986, + "p99_ms": 6.670290777692569, + "index_ms": 147.35962502891198, + "by_category": { + "architecture": 1.0, + "semantic": 0.947247344468017, + "symbol": 1.0 + } + }, + { + "repo": "cobra", + "language": "go", + "mode": "semble-balanced", + "chunks": 780, + "tokens": 2707, + "ndcg5": 0.9525325271359822, + "ndcg10": 0.9525325271359822, + "p50_ms": 0.7229379843920469, + "p90_ms": 4.412566381506622, + "p95_ms": 4.730421135900544, + "p99_ms": 6.297984210541469, + "index_ms": 147.35962502891198, + "by_category": { + "architecture": 1.0, + "semantic": 0.947247344468017, + "symbol": 0.8769765845238192 + } + }, + { + "repo": "cobra", + "language": "go", + "mode": "semble-bm25", + "chunks": 780, + "tokens": 2954, + "ndcg5": 0.9775325271359823, + "ndcg10": 0.9775325271359823, + "p50_ms": 0.4600000102072954, + "p90_ms": 2.3775585810653883, + "p95_ms": 3.5909017460653567, + "p99_ms": 4.452646782738155, + "index_ms": 147.35962502891198, + "by_category": { + "architecture": 1.0, + "semantic": 0.9591500493381495, + "symbol": 1.0 + } + }, + { + "repo": "cobra", + "language": "go", + "mode": "semble-semantic", + "chunks": 780, + "tokens": 2569, + "ndcg5": 0.8603753963003303, + "ndcg10": 0.8603753963003303, + "p50_ms": 0.5893335037399083, + "p90_ms": 3.722441691206769, + "p95_ms": 4.408256578608417, + "p99_ms": 5.230417700367978, + "index_ms": 147.35962502891198, + "by_category": { + "architecture": 0.8769765845238192, + "semantic": 0.8586953108057904, + "symbol": 0.8333333333333334 + } + }, + { + "repo": "cobra", + "language": "go", + "mode": "unranked-auto", + "chunks": 780, + "tokens": 1837, + "ndcg5": 0.8346268032608155, + "ndcg10": 0.8512934699274822, + "p50_ms": 0.26337496819905937, + "p90_ms": 0.30196249717846513, + "p95_ms": 0.3095125255640596, + "p99_ms": 0.30970248277299106, + "index_ms": 147.35962502891198, + "by_category": { + "architecture": 0.7384460930122322, + "semantic": 0.9058420988095264, + "symbol": 0.8769765845238192 + } + }, + { + "repo": "cobra", + "language": "go", + "mode": "unranked-balanced", + "chunks": 780, + "tokens": 1840, + "ndcg5": 0.8346268032608155, + "ndcg10": 0.8512934699274822, + "p50_ms": 0.2616040001157671, + "p90_ms": 0.2908589143771678, + "p95_ms": 0.304239263641648, + "p99_ms": 0.31611424346920103, + "index_ms": 147.35962502891198, + "by_category": { + "architecture": 0.7384460930122322, + "semantic": 0.9058420988095264, + "symbol": 0.8769765845238192 + } + }, + { + "repo": "cobra", + "language": "go", + "mode": "unranked-bm25", + "chunks": 780, + "tokens": 1580, + "ndcg5": 0.7485399635247794, + "ndcg10": 0.7629932048406737, + "p50_ms": 0.25366651243530214, + "p90_ms": 0.26903687394224113, + "p95_ms": 0.29014789033681154, + "p99_ms": 0.3037959663197398, + "index_ms": 147.35962502891198, + "by_category": { + "architecture": 0.7584873889101339, + "semantic": 0.7343645463437464, + "symbol": 0.8769765845238192 + } + }, + { + "repo": "cobra", + "language": "go", + "mode": "unranked-semantic", + "chunks": 780, + "tokens": 1709, + "ndcg5": 0.8306128427182248, + "ndcg10": 0.848423202073626, + "p50_ms": 0.24635446607135236, + "p90_ms": 0.27457078685984015, + "p95_ms": 0.28536838362924755, + "p99_ms": 0.303006445756182, + "index_ms": 147.35962502891198, + "by_category": { + "architecture": 0.7478561567799132, + "semantic": 0.947247344468017, + "symbol": 0.6872021038816168 + } + }, + { + "repo": "commons-lang", + "language": "java", + "mode": "semble-auto", + "chunks": 6049, + "tokens": 1591, + "ndcg5": 0.8511907952337322, + "ndcg10": 0.8829368269797638, + "p50_ms": 0.834333011880517, + "p90_ms": 10.867083037737757, + "p95_ms": 18.60316703096032, + "p99_ms": 18.644399847835302, + "index_ms": 824.2972499574535, + "by_category": { + "architecture": 0.8710490642551528, + "semantic": 0.8520375838435416, + "symbol": 1.0 + } + }, + { + "repo": "commons-lang", + "language": "java", + "mode": "semble-balanced", + "chunks": 6049, + "tokens": 1590, + "ndcg5": 0.8511907952337322, + "ndcg10": 0.8829368269797638, + "p50_ms": 0.828833959531039, + "p90_ms": 10.737999982666224, + "p95_ms": 17.611667048186064, + "p99_ms": 18.42570062726736, + "index_ms": 824.2972499574535, + "by_category": { + "architecture": 0.8710490642551528, + "semantic": 0.8520375838435416, + "symbol": 1.0 + } + }, + { + "repo": "commons-lang", + "language": "java", + "mode": "semble-bm25", + "chunks": 6049, + "tokens": 1814, + "ndcg5": 0.8574255454038016, + "ndcg10": 0.8574255454038016, + "p50_ms": 0.6651669973507524, + "p90_ms": 9.387208032421768, + "p95_ms": 18.257917021401227, + "p99_ms": 19.0302497940138, + "index_ms": 824.2972499574535, + "by_category": { + "architecture": 0.8710490642551528, + "semantic": 0.8137706614795981, + "symbol": 1.0 + } + }, + { + "repo": "commons-lang", + "language": "java", + "mode": "semble-semantic", + "chunks": 6049, + "tokens": 1579, + "ndcg5": 0.9351940391657602, + "ndcg10": 0.9351940391657602, + "p50_ms": 0.5509999464266002, + "p90_ms": 10.073125013150275, + "p95_ms": 17.503625014796853, + "p99_ms": 18.16442497074604, + "index_ms": 824.2972499574535, + "by_category": { + "architecture": 0.9590717717793499, + "semantic": 0.9115613933673511, + "symbol": 1.0 + } + }, + { + "repo": "commons-lang", + "language": "java", + "mode": "unranked-auto", + "chunks": 6049, + "tokens": 1719, + "ndcg5": 0.8059836782203819, + "ndcg10": 0.8229459252255259, + "p50_ms": 0.32724998891353607, + "p90_ms": 0.373208022210747, + "p95_ms": 0.4045420209877193, + "p99_ms": 0.4200084018521011, + "index_ms": 824.2972499574535, + "by_category": { + "architecture": 0.8442893273045486, + "semantic": 0.7941476210179241, + "symbol": 0.9077324383928644 + } + }, + { + "repo": "commons-lang", + "language": "java", + "mode": "unranked-balanced", + "chunks": 6049, + "tokens": 1726, + "ndcg5": 0.8059836782203819, + "ndcg10": 0.8229459252255259, + "p50_ms": 0.3357079694978893, + "p90_ms": 0.37395802792161703, + "p95_ms": 0.3900409792549908, + "p99_ms": 0.41364176431670785, + "index_ms": 824.2972499574535, + "by_category": { + "architecture": 0.8442893273045486, + "semantic": 0.7941476210179241, + "symbol": 0.9077324383928644 + } + }, + { + "repo": "commons-lang", + "language": "java", + "mode": "unranked-bm25", + "chunks": 6049, + "tokens": 1628, + "ndcg5": 0.7250192403395216, + "ndcg10": 0.7250192403395216, + "p50_ms": 0.3251250018365681, + "p90_ms": 0.36316702608019114, + "p95_ms": 0.38062501698732376, + "p99_ms": 0.38722497411072254, + "index_ms": 824.2972499574535, + "by_category": { + "architecture": 0.8442893273045486, + "semantic": 0.647257593688918, + "symbol": 0.9077324383928644 + } + }, + { + "repo": "commons-lang", + "language": "java", + "mode": "unranked-semantic", + "chunks": 6049, + "tokens": 1695, + "ndcg5": 0.7380104439618131, + "ndcg10": 0.778518233789874, + "p50_ms": 0.329792033880949, + "p90_ms": 0.735500012524426, + "p95_ms": 0.8149169734679163, + "p99_ms": 0.8716497919522226, + "index_ms": 824.2972499574535, + "by_category": { + "architecture": 0.9355245321275764, + "semantic": 0.7842442721712661, + "symbol": 0.6407223757017241 + } + }, + { + "repo": "curl", + "language": "c", + "mode": "semble-auto", + "chunks": 8901, + "tokens": 1519, + "ndcg5": 0.6504072534454753, + "ndcg10": 0.7051303483604954, + "p50_ms": 1.1114375083707273, + "p90_ms": 1.7466672870796176, + "p95_ms": 1.8686982133658605, + "p99_ms": 2.1047732193255793, + "index_ms": 1377.2673750063404, + "by_category": { + "architecture": 0.6284680374280802, + "semantic": 0.7678540573051987 + } + }, + { + "repo": "curl", + "language": "c", + "mode": "semble-balanced", + "chunks": 8901, + "tokens": 1519, + "ndcg5": 0.6504072534454753, + "ndcg10": 0.7051303483604954, + "p50_ms": 0.9586249943822622, + "p90_ms": 1.1803127999883145, + "p95_ms": 1.2059666012646633, + "p99_ms": 1.3365597050869835, + "index_ms": 1377.2673750063404, + "by_category": { + "architecture": 0.6284680374280802, + "semantic": 0.7678540573051987 + } + }, + { + "repo": "curl", + "language": "c", + "mode": "semble-bm25", + "chunks": 8901, + "tokens": 1542, + "ndcg5": 0.6233120839842696, + "ndcg10": 0.7125621052909232, + "p50_ms": 0.7271039939951152, + "p90_ms": 1.3074039015918972, + "p95_ms": 1.3846273970557377, + "p99_ms": 1.4543254574527962, + "index_ms": 1377.2673750063404, + "by_category": { + "architecture": 0.6653225997935128, + "semantic": 0.7512126097888043 + } + }, + { + "repo": "curl", + "language": "c", + "mode": "semble-semantic", + "chunks": 8901, + "tokens": 1425, + "ndcg5": 0.6777389759882029, + "ndcg10": 0.6866009403274053, + "p50_ms": 1.3129794970154762, + "p90_ms": 1.797008397988975, + "p95_ms": 1.8258922907989474, + "p99_ms": 2.1611784433480348, + "index_ms": 1377.2673750063404, + "by_category": { + "architecture": 0.5890380276083014, + "semantic": 0.7664251416430358 + } + }, + { + "repo": "curl", + "language": "c", + "mode": "unranked-auto", + "chunks": 8901, + "tokens": 1547, + "ndcg5": 0.5511859507142916, + "ndcg10": 0.5787729892218665, + "p50_ms": 0.7855210278648883, + "p90_ms": 0.9732077829539776, + "p95_ms": 0.9944069141056389, + "p99_ms": 1.020214994205162, + "index_ms": 1377.2673750063404, + "by_category": { + "architecture": 0.4091856137469951, + "semantic": 0.7175262964285793 + } + }, + { + "repo": "curl", + "language": "c", + "mode": "unranked-balanced", + "chunks": 8901, + "tokens": 1547, + "ndcg5": 0.5511859507142916, + "ndcg10": 0.5787729892218665, + "p50_ms": 0.45252099516801536, + "p90_ms": 0.5801996856462212, + "p95_ms": 0.6325833091977984, + "p99_ms": 0.6433494680095464, + "index_ms": 1377.2673750063404, + "by_category": { + "architecture": 0.4091856137469951, + "semantic": 0.7175262964285793 + } + }, + { + "repo": "curl", + "language": "c", + "mode": "unranked-bm25", + "chunks": 8901, + "tokens": 1546, + "ndcg5": 0.4672387636026077, + "ndcg10": 0.4822902633858067, + "p50_ms": 0.8058124803937972, + "p90_ms": 0.9716081025544554, + "p95_ms": 0.9821902582189068, + "p99_ms": 1.0028052161214873, + "index_ms": 1377.2673750063404, + "by_category": { + "architecture": 0.4348998571087512, + "semantic": 0.5210642321579431 + } + }, + { + "repo": "curl", + "language": "c", + "mode": "unranked-semantic", + "chunks": 8901, + "tokens": 1486, + "ndcg5": 0.5490192160352625, + "ndcg10": 0.5834962420573302, + "p50_ms": 0.8818540081847459, + "p90_ms": 1.097870495868847, + "p95_ms": 1.2702392588835212, + "p99_ms": 1.508214235072955, + "index_ms": 1377.2673750063404, + "by_category": { + "architecture": 0.42738433581764884, + "semantic": 0.7112241653443423 + } + }, + { + "repo": "dapper", + "language": "csharp", + "mode": "semble-auto", + "chunks": 798, + "tokens": 1342, + "ndcg5": 0.8074356157188728, + "ndcg10": 0.8419126417409405, + "p50_ms": 2.4211664858739823, + "p90_ms": 5.021188076352703, + "p95_ms": 6.894139529322278, + "p99_ms": 10.63926070055458, + "index_ms": 164.17187498882413, + "by_category": { + "architecture": 0.5877789735348331, + "semantic": 0.8836259185138062, + "symbol": 1.0 + } + }, + { + "repo": "dapper", + "language": "csharp", + "mode": "semble-balanced", + "chunks": 798, + "tokens": 1335, + "ndcg5": 0.8074356157188728, + "ndcg10": 0.8419126417409405, + "p50_ms": 2.352229494135827, + "p90_ms": 3.5164791974239074, + "p95_ms": 5.044887863914486, + "p99_ms": 8.836844767793075, + "index_ms": 164.17187498882413, + "by_category": { + "architecture": 0.5877789735348331, + "semantic": 0.8836259185138062, + "symbol": 1.0 + } + }, + { + "repo": "dapper", + "language": "csharp", + "mode": "semble-bm25", + "chunks": 798, + "tokens": 1490, + "ndcg5": 0.8846268032608154, + "ndcg10": 0.9012934699274823, + "p50_ms": 1.5976664726622403, + "p90_ms": 2.2515713935717963, + "p95_ms": 3.8449809682788367, + "p99_ms": 8.110829792567522, + "index_ms": 164.17187498882413, + "by_category": { + "architecture": 0.8576691395183482, + "semantic": 0.8919379108058652, + "symbol": 1.0 + } + }, + { + "repo": "dapper", + "language": "csharp", + "mode": "semble-semantic", + "chunks": 798, + "tokens": 1301, + "ndcg5": 0.7696394630357186, + "ndcg10": 0.8191679888409855, + "p50_ms": 1.812583504943177, + "p90_ms": 2.5637790909968334, + "p95_ms": 3.8657369354041338, + "p99_ms": 8.36411461757961, + "index_ms": 164.17187498882413, + "by_category": { + "architecture": 0.5720417340858652, + "semantic": 0.8534763723443267, + "symbol": 1.0 + } + }, + { + "repo": "dapper", + "language": "csharp", + "mode": "unranked-auto", + "chunks": 798, + "tokens": 1568, + "ndcg5": 0.8011859507142916, + "ndcg10": 0.834519284047625, + "p50_ms": 0.2589794748928398, + "p90_ms": 0.28234561905264854, + "p95_ms": 0.292708957567811, + "p99_ms": 0.2946081757545471, + "index_ms": 164.17187498882413, + "by_category": { + "architecture": 0.4821315434523954, + "semantic": 0.9432199620879166, + "symbol": 0.8333333333333334 + } + }, + { + "repo": "dapper", + "language": "csharp", + "mode": "unranked-balanced", + "chunks": 798, + "tokens": 1563, + "ndcg5": 0.7955285910760186, + "ndcg10": 0.8288619244093521, + "p50_ms": 0.2560210123192519, + "p90_ms": 0.28059971518814564, + "p95_ms": 0.28203336987644434, + "p99_ms": 0.2844402613118291, + "index_ms": 164.17187498882413, + "by_category": { + "architecture": 0.4821315434523954, + "semantic": 0.9432199620879166, + "symbol": 0.7956176024115139 + } + }, + { + "repo": "dapper", + "language": "csharp", + "mode": "unranked-bm25", + "chunks": 798, + "tokens": 1509, + "ndcg5": 0.8727197786179612, + "ndcg10": 0.8727197786179612, + "p50_ms": 0.2549170167185366, + "p90_ms": 0.2688086999114603, + "p95_ms": 0.2736339229159057, + "p99_ms": 0.29339318862184877, + "index_ms": 164.17187498882413, + "by_category": { + "architecture": 0.9077324383928644, + "semantic": 0.8609643127089469, + "symbol": 0.8769765845238192 + } + }, + { + "repo": "dapper", + "language": "csharp", + "mode": "unranked-semantic", + "chunks": 798, + "tokens": 1495, + "ndcg5": 0.7105159313011155, + "ndcg10": 0.7283262906565166, + "p50_ms": 0.25470799300819635, + "p90_ms": 0.27137529687024653, + "p95_ms": 0.2721916534937918, + "p99_ms": 0.2732046996243298, + "index_ms": 164.17187498882413, + "by_category": { + "architecture": 0.42313401630407704, + "semantic": 0.8648507687955819, + "symbol": 0.5436432511904858 + } + }, + { + "repo": "ecto", + "language": "elixir", + "mode": "semble-auto", + "chunks": 1431, + "tokens": 3662, + "ndcg5": 0.9025638060198661, + "ndcg10": 0.9025638060198661, + "p50_ms": 0.5029579624533653, + "p90_ms": 6.243708624970167, + "p95_ms": 6.879370915703474, + "p99_ms": 7.437341385520995, + "index_ms": 249.66974998824298, + "by_category": { + "architecture": 1.0, + "semantic": 0.8960547934136506, + "symbol": 0.8333333333333334 + } + }, + { + "repo": "ecto", + "language": "elixir", + "mode": "semble-balanced", + "chunks": 1431, + "tokens": 3662, + "ndcg5": 0.9025638060198661, + "ndcg10": 0.9025638060198661, + "p50_ms": 0.5008750013075769, + "p90_ms": 6.10896700527519, + "p95_ms": 6.671741884201763, + "p99_ms": 7.327481135725975, + "index_ms": 249.66974998824298, + "by_category": { + "architecture": 1.0, + "semantic": 0.8960547934136506, + "symbol": 0.8333333333333334 + } + }, + { + "repo": "ecto", + "language": "elixir", + "mode": "semble-bm25", + "chunks": 1431, + "tokens": 4169, + "ndcg5": 0.8890941716165459, + "ndcg10": 0.9066380312656687, + "p50_ms": 0.41149999015033245, + "p90_ms": 5.8072832063771775, + "p95_ms": 6.507616076851261, + "p99_ms": 7.272256828146055, + "index_ms": 249.66974998824298, + "by_category": { + "architecture": 1.0, + "semantic": 0.8635478918498236, + "symbol": 1.0 + } + }, + { + "repo": "ecto", + "language": "elixir", + "mode": "semble-semantic", + "chunks": 1431, + "tokens": 2370, + "ndcg5": 0.8531746645274898, + "ndcg10": 0.8531746645274898, + "p50_ms": 0.4039170453324914, + "p90_ms": 5.6575582013465455, + "p95_ms": 6.402542261639607, + "p99_ms": 7.1591420855838805, + "index_ms": 249.66974998824298, + "by_category": { + "architecture": 1.0, + "semantic": 0.785409125078639, + "symbol": 1.0 + } + }, + { + "repo": "ecto", + "language": "elixir", + "mode": "unranked-auto", + "chunks": 1431, + "tokens": 4349, + "ndcg5": 0.8627783821428617, + "ndcg10": 0.8627783821428617, + "p50_ms": 0.27429196052253246, + "p90_ms": 0.2917086239904165, + "p95_ms": 0.294791697524488, + "p99_ms": 0.29779191594570875, + "index_ms": 249.66974998824298, + "by_category": { + "architecture": 0.8769765845238192, + "semantic": 0.8562253656593429, + "symbol": 0.8769765845238192 + } + }, + { + "repo": "ecto", + "language": "elixir", + "mode": "unranked-balanced", + "chunks": 1431, + "tokens": 4352, + "ndcg5": 0.8222743485941949, + "ndcg10": 0.8222743485941949, + "p50_ms": 0.2702090423554182, + "p90_ms": 0.2851081779226661, + "p95_ms": 0.2925493870861828, + "p99_ms": 0.303409902844578, + "index_ms": 249.66974998824298, + "by_category": { + "architecture": 0.8769765845238192, + "semantic": 0.8562253656593429, + "symbol": 0.6204510387155954 + } + }, + { + "repo": "ecto", + "language": "elixir", + "mode": "unranked-bm25", + "chunks": 1431, + "tokens": 5285, + "ndcg5": 0.8785545297482268, + "ndcg10": 0.8973022764381225, + "p50_ms": 0.2662090118974447, + "p90_ms": 0.280674803070724, + "p95_ms": 0.28480858309194446, + "p99_ms": 0.2867281227372587, + "index_ms": 249.66974998824298, + "by_category": { + "architecture": 0.8769765845238192, + "semantic": 0.8782933460579133, + "symbol": 1.0 + } + }, + { + "repo": "ecto", + "language": "elixir", + "mode": "unranked-semantic", + "chunks": 1431, + "tokens": 2832, + "ndcg5": 0.5430278613319967, + "ndcg10": 0.6086769142137795, + "p50_ms": 0.26408297708258033, + "p90_ms": 0.2791664330288768, + "p95_ms": 0.2819916408043355, + "p99_ms": 0.28553110896609724, + "index_ms": 249.66974998824298, + "by_category": { + "architecture": 0.8333333333333334, + "semantic": 0.6367675096061839, + "symbol": 0.26229458172713843 + } + }, + { + "repo": "exposed", + "language": "kotlin", + "mode": "semble-auto", + "chunks": 1408, + "tokens": 1644, + "ndcg5": 0.5893422178385317, + "ndcg10": 0.6477744920631955, + "p50_ms": 0.770041486248374, + "p90_ms": 4.920616693561898, + "p95_ms": 5.429579454357736, + "p99_ms": 5.506148710264824, + "index_ms": 213.9325410244055, + "by_category": { + "architecture": 0.42657598789683987, + "semantic": 0.6196972769695278, + "symbol": 1.0 + } + }, + { + "repo": "exposed", + "language": "kotlin", + "mode": "semble-balanced", + "chunks": 1408, + "tokens": 1644, + "ndcg5": 0.5893422178385317, + "ndcg10": 0.6477744920631955, + "p50_ms": 0.7556670170743018, + "p90_ms": 4.940874996827916, + "p95_ms": 5.517512472579256, + "p99_ms": 5.544302455382422, + "index_ms": 213.9325410244055, + "by_category": { + "architecture": 0.42657598789683987, + "semantic": 0.6196972769695278, + "symbol": 1.0 + } + }, + { + "repo": "exposed", + "language": "kotlin", + "mode": "semble-bm25", + "chunks": 1408, + "tokens": 1622, + "ndcg5": 0.64857880242896, + "ndcg10": 0.6788052875841408, + "p50_ms": 0.5406875279732049, + "p90_ms": 3.5325578122865426, + "p95_ms": 4.668302708887495, + "p99_ms": 4.760927728493698, + "index_ms": 213.9325410244055, + "by_category": { + "architecture": 0.4733315266297818, + "semantic": 0.6540079408423908, + "symbol": 1.0 + } + }, + { + "repo": "exposed", + "language": "kotlin", + "mode": "semble-semantic", + "chunks": 1408, + "tokens": 1582, + "ndcg5": 0.6976726396135717, + "ndcg10": 0.7065346039527741, + "p50_ms": 0.5757499893661588, + "p90_ms": 4.486537474440411, + "p95_ms": 4.612868756521493, + "p99_ms": 5.057373757008462, + "index_ms": 213.9325410244055, + "by_category": { + "architecture": 0.47689218602446437, + "semantic": 0.6928582514987207, + "symbol": 1.0 + } + }, + { + "repo": "exposed", + "language": "kotlin", + "mode": "unranked-auto", + "chunks": 1408, + "tokens": 1568, + "ndcg5": 0.4, + "ndcg10": 0.4651300908732604, + "p50_ms": 0.2747704857029021, + "p90_ms": 0.29830479179508984, + "p95_ms": 0.3052086685784161, + "p99_ms": 0.3115417552180588, + "index_ms": 213.9325410244055, + "by_category": { + "architecture": 0.10515495892857625, + "semantic": 0.5479765759924108, + "symbol": 0.4384882922619096 + } + }, + { + "repo": "exposed", + "language": "kotlin", + "mode": "unranked-balanced", + "chunks": 1408, + "tokens": 1568, + "ndcg5": 0.4, + "ndcg10": 0.4651300908732604, + "p50_ms": 0.2750209823716432, + "p90_ms": 0.29539622482843697, + "p95_ms": 0.2998360287165269, + "p99_ms": 0.30176723084878176, + "index_ms": 213.9325410244055, + "by_category": { + "architecture": 0.10515495892857625, + "semantic": 0.5479765759924108, + "symbol": 0.4384882922619096 + } + }, + { + "repo": "exposed", + "language": "kotlin", + "mode": "unranked-bm25", + "chunks": 1408, + "tokens": 1525, + "ndcg5": 0.41786120695511875, + "ndcg10": 0.508360279921817, + "p50_ms": 0.27164595667272806, + "p90_ms": 0.2914797223638743, + "p95_ms": 0.29450767615344375, + "p99_ms": 0.30350153509061784, + "index_ms": 213.9325410244055, + "by_category": { + "architecture": 0.21509067114197, + "semantic": 0.5872073992390321, + "symbol": 0.4336766652213271 + } + }, + { + "repo": "exposed", + "language": "kotlin", + "mode": "unranked-semantic", + "chunks": 1408, + "tokens": 1599, + "ndcg5": 0.28770711884305794, + "ndcg10": 0.34654084960576537, + "p50_ms": 0.2669795067049563, + "p90_ms": 0.2909755683504045, + "p95_ms": 0.2946687425719574, + "p99_ms": 0.31433373747859145, + "index_ms": 213.9325410244055, + "by_category": { + "architecture": 0.10034333188799373, + "semantic": 0.4173502322335651, + "symbol": 0.26229458172713843 + } + }, + { + "repo": "express", + "language": "javascript", + "mode": "semble-auto", + "chunks": 102, + "tokens": 1528, + "ndcg5": 0.9382034061286294, + "ndcg10": 0.9491237779695378, + "p50_ms": 0.39587498758919537, + "p90_ms": 0.43227091082371777, + "p95_ms": 0.5357089452445518, + "p99_ms": 1.763108205050228, + "index_ms": 20.20970801822841, + "by_category": { + "architecture": 0.9260679370558221, + "semantic": 0.95, + "symbol": 1.0 + } + }, + { + "repo": "express", + "language": "javascript", + "mode": "semble-balanced", + "chunks": 102, + "tokens": 1528, + "ndcg5": 0.9382034061286294, + "ndcg10": 0.9491237779695378, + "p50_ms": 0.38760449388064444, + "p90_ms": 0.4313332901801914, + "p95_ms": 0.5204017565120023, + "p99_ms": 1.743146755034102, + "index_ms": 20.20970801822841, + "by_category": { + "architecture": 0.9260679370558221, + "semantic": 0.95, + "symbol": 1.0 + } + }, + { + "repo": "express", + "language": "javascript", + "mode": "semble-bm25", + "chunks": 102, + "tokens": 1536, + "ndcg5": 0.8368598856384647, + "ndcg10": 0.8822572835014408, + "p50_ms": 0.3447500057518482, + "p90_ms": 0.37592851440422237, + "p95_ms": 0.44037435436621397, + "p99_ms": 1.3840412464924141, + "index_ms": 20.20970801822841, + "by_category": { + "architecture": 0.8302589052422117, + "semantic": 0.8833333333333332, + "symbol": 1.0 + } + }, + { + "repo": "express", + "language": "javascript", + "mode": "semble-semantic", + "chunks": 102, + "tokens": 1500, + "ndcg5": 0.9815464876785729, + "ndcg10": 0.9815464876785729, + "p50_ms": 0.34206247073598206, + "p90_ms": 0.375049706781283, + "p95_ms": 0.43804791930597364, + "p99_ms": 1.3659760117297977, + "index_ms": 20.20970801822841, + "by_category": { + "architecture": 1.0, + "semantic": 0.9630929753571458, + "symbol": 1.0 + } + }, + { + "repo": "express", + "language": "javascript", + "mode": "unranked-auto", + "chunks": 102, + "tokens": 1555, + "ndcg5": 0.864389356842921, + "ndcg10": 0.864389356842921, + "p50_ms": 0.2339795173611492, + "p90_ms": 0.24577562580816448, + "p95_ms": 0.24877500254660845, + "p99_ms": 0.25105501990765333, + "index_ms": 20.20970801822841, + "by_category": { + "architecture": 0.8948668746532273, + "semantic": 0.8761859507142915, + "symbol": 0.7539531690476383 + } + }, + { + "repo": "express", + "language": "javascript", + "mode": "unranked-balanced", + "chunks": 102, + "tokens": 1555, + "ndcg5": 0.864389356842921, + "ndcg10": 0.864389356842921, + "p50_ms": 0.2326664689462632, + "p90_ms": 0.24735056213103235, + "p95_ms": 0.2475396206136793, + "p99_ms": 0.24814152042381465, + "index_ms": 20.20970801822841, + "by_category": { + "architecture": 0.8948668746532273, + "semantic": 0.8761859507142915, + "symbol": 0.7539531690476383 + } + }, + { + "repo": "express", + "language": "javascript", + "mode": "unranked-bm25", + "chunks": 102, + "tokens": 1518, + "ndcg5": 0.8152784848832211, + "ndcg10": 0.8152784848832211, + "p50_ms": 0.22695798543281853, + "p90_ms": 0.24711698642931879, + "p95_ms": 0.24790654133539647, + "p99_ms": 0.24964769778307527, + "index_ms": 20.20970801822841, + "by_category": { + "architecture": 0.8234383032246558, + "semantic": 0.8279642067948915, + "symbol": 0.7539531690476383 + } + }, + { + "repo": "express", + "language": "javascript", + "mode": "unranked-semantic", + "chunks": 102, + "tokens": 1578, + "ndcg5": 0.899814948079167, + "ndcg10": 0.899814948079167, + "p50_ms": 0.22922951029613614, + "p90_ms": 0.24404589203186333, + "p95_ms": 0.24676522880326957, + "p99_ms": 0.24762021319475025, + "index_ms": 20.20970801822841, + "by_category": { + "architecture": 0.9595224133763464, + "semantic": 0.9017782560805999, + "symbol": 0.7539531690476383 + } + }, + { + "repo": "fastapi", + "language": "python", + "mode": "semble-auto", + "chunks": 1188, + "tokens": 1584, + "ndcg5": 0.7136739118980615, + "ndcg10": 0.7840277229562979, + "p50_ms": 0.5162910092622042, + "p90_ms": 2.2483000240754336, + "p95_ms": 2.4075229273876184, + "p99_ms": 3.1450717820553096, + "index_ms": 143.18399998592213, + "by_category": { + "architecture": 0.7310666719569522, + "semantic": 0.6953859694284116, + "symbol": 1.0 + } + }, + { + "repo": "fastapi", + "language": "python", + "mode": "semble-balanced", + "chunks": 1188, + "tokens": 1583, + "ndcg5": 0.7136739118980615, + "ndcg10": 0.7840277229562979, + "p50_ms": 0.5780209903605282, + "p90_ms": 2.4413579900283366, + "p95_ms": 2.5762246892554694, + "p99_ms": 3.477078526630065, + "index_ms": 143.18399998592213, + "by_category": { + "architecture": 0.7310666719569522, + "semantic": 0.6953859694284116, + "symbol": 1.0 + } + }, + { + "repo": "fastapi", + "language": "python", + "mode": "semble-bm25", + "chunks": 1188, + "tokens": 1564, + "ndcg5": 0.7523591926215158, + "ndcg10": 0.7892513512328503, + "p50_ms": 0.44520795927383006, + "p90_ms": 2.0085702533833683, + "p95_ms": 2.101173539995216, + "p99_ms": 2.2051346785156056, + "index_ms": 143.18399998592213, + "by_category": { + "architecture": 0.7271831598521447, + "semantic": 0.7118431132114993, + "symbol": 1.0 + } + }, + { + "repo": "fastapi", + "language": "python", + "mode": "semble-semantic", + "chunks": 1188, + "tokens": 1535, + "ndcg5": 0.6696119414476212, + "ndcg10": 0.7094931171092411, + "p50_ms": 0.39381199167110026, + "p90_ms": 1.9529993936885148, + "p95_ms": 1.9739083509193733, + "p99_ms": 2.0580152893671766, + "index_ms": 143.18399998592213, + "by_category": { + "architecture": 0.7243438271970604, + "semantic": 0.5632885448799927, + "symbol": 0.9226294385530917 + } + }, + { + "repo": "fastapi", + "language": "python", + "mode": "unranked-auto", + "chunks": 1188, + "tokens": 1647, + "ndcg5": 0.5149070016167425, + "ndcg10": 0.5445740262145695, + "p50_ms": 0.24999998277053237, + "p90_ms": 0.28218309744261205, + "p95_ms": 0.2838860440533608, + "p99_ms": 0.29607719159685075, + "index_ms": 143.18399998592213, + "by_category": { + "architecture": 0.5659637887513967, + "semantic": 0.5351277110817697, + "symbol": 0.5297424628754913 + } + }, + { + "repo": "fastapi", + "language": "python", + "mode": "unranked-balanced", + "chunks": 1188, + "tokens": 1651, + "ndcg5": 0.4855517014801122, + "ndcg10": 0.5330290854333404, + "p50_ms": 0.2528335025999695, + "p90_ms": 0.2894463774282485, + "p95_ms": 0.3028934501344338, + "p99_ms": 0.32091226719785476, + "index_ms": 143.18399998592213, + "by_category": { + "architecture": 0.5659637887513967, + "semantic": 0.5351277110817697, + "symbol": 0.4835626997505746 + } + }, + { + "repo": "fastapi", + "language": "python", + "mode": "unranked-bm25", + "chunks": 1188, + "tokens": 1491, + "ndcg5": 0.5634975608952068, + "ndcg10": 0.574417932736115, + "p50_ms": 0.2576660190243274, + "p90_ms": 0.29538391972891986, + "p95_ms": 0.3082246636040509, + "p99_ms": 0.3255777456797659, + "index_ms": 143.18399998592213, + "by_category": { + "architecture": 0.5171390097406843, + "semantic": 0.5485148303220536, + "symbol": 0.6960533887922159 + } + }, + { + "repo": "fastapi", + "language": "python", + "mode": "unranked-semantic", + "chunks": 1188, + "tokens": 1617, + "ndcg5": 0.4260597552042144, + "ndcg10": 0.5015666183922681, + "p50_ms": 0.26733303093351424, + "p90_ms": 0.32921252422966063, + "p95_ms": 0.3371166530996561, + "p99_ms": 0.34268973395228386, + "index_ms": 143.18399998592213, + "by_category": { + "architecture": 0.6563857735477362, + "semantic": 0.46330437724864537, + "symbol": 0.34603938700440867 + } + }, + { + "repo": "flask", + "language": "python", + "mode": "semble-auto", + "chunks": 558, + "tokens": 1834, + "ndcg5": 0.859874490822509, + "ndcg10": 0.8878534827452375, + "p50_ms": 0.4599580424837768, + "p90_ms": 1.6612079925835133, + "p95_ms": 2.7781249955296516, + "p99_ms": 2.808391395956278, + "index_ms": 77.17279100324959, + "by_category": { + "architecture": 0.8304380615499182, + "semantic": 0.9289735921850091, + "symbol": 0.8710490642551528 + } + }, + { + "repo": "flask", + "language": "python", + "mode": "semble-balanced", + "chunks": 558, + "tokens": 1821, + "ndcg5": 0.859874490822509, + "ndcg10": 0.8878534827452375, + "p50_ms": 0.4414590075612068, + "p90_ms": 1.623291987925768, + "p95_ms": 2.708582964260131, + "p99_ms": 2.745782968122512, + "index_ms": 77.17279100324959, + "by_category": { + "architecture": 0.8304380615499182, + "semantic": 0.9289735921850091, + "symbol": 0.8710490642551528 + } + }, + { + "repo": "flask", + "language": "python", + "mode": "semble-bm25", + "chunks": 558, + "tokens": 1732, + "ndcg5": 0.7674469765022351, + "ndcg10": 0.7974584658952637, + "p50_ms": 0.343833991792053, + "p90_ms": 1.501500024460256, + "p95_ms": 1.8142500193789601, + "p99_ms": 2.4132500169798736, + "index_ms": 77.17279100324959, + "by_category": { + "architecture": 0.6672851725542853, + "semantic": 0.8602258530140983, + "symbol": 0.8710490642551528 + } + }, + { + "repo": "flask", + "language": "python", + "mode": "semble-semantic", + "chunks": 558, + "tokens": 1637, + "ndcg5": 0.8164058453186723, + "ndcg10": 0.8443848372414009, + "p50_ms": 0.3376660170033574, + "p90_ms": 1.5465410542674363, + "p95_ms": 1.98787497356534, + "p99_ms": 2.3881413973867898, + "index_ms": 77.17279100324959, + "by_category": { + "architecture": 0.8478663329217984, + "semantic": 0.8684491186618104, + "symbol": 0.748025648778972 + } + }, + { + "repo": "flask", + "language": "python", + "mode": "unranked-auto", + "chunks": 558, + "tokens": 1902, + "ndcg5": 0.5747446581065243, + "ndcg10": 0.6365429182546349, + "p50_ms": 0.22154097678139806, + "p90_ms": 0.2775000175461173, + "p95_ms": 0.28437498258426785, + "p99_ms": 0.28814218239858747, + "index_ms": 77.17279100324959, + "by_category": { + "architecture": 0.6323902616299613, + "semantic": 0.6542319938213995, + "symbol": 0.581372506634071 + } + }, + { + "repo": "flask", + "language": "python", + "mode": "unranked-balanced", + "chunks": 558, + "tokens": 1888, + "ndcg5": 0.5923194317459787, + "ndcg10": 0.6303062360100276, + "p50_ms": 0.21895801182836294, + "p90_ms": 0.2842079848051071, + "p95_ms": 0.2864169655367732, + "p99_ms": 0.28964977245777845, + "index_ms": 77.17279100324959, + "by_category": { + "architecture": 0.6323902616299613, + "semantic": 0.6542319938213995, + "symbol": 0.5377157309218195 + } + }, + { + "repo": "flask", + "language": "python", + "mode": "unranked-bm25", + "chunks": 558, + "tokens": 1824, + "ndcg5": 0.5440872029041707, + "ndcg10": 0.5899716180610393, + "p50_ms": 0.2157079870812595, + "p90_ms": 0.2750420244410634, + "p95_ms": 0.2852919860742986, + "p99_ms": 0.28632477624341846, + "index_ms": 77.17279100324959, + "by_category": { + "architecture": 0.47663719781595015, + "semantic": 0.6535083035047858, + "symbol": 0.621450752005843 + } + }, + { + "repo": "flask", + "language": "python", + "mode": "unranked-semantic", + "chunks": 558, + "tokens": 1761, + "ndcg5": 0.6057176297988317, + "ndcg10": 0.6604837513049181, + "p50_ms": 0.2192090032622218, + "p90_ms": 0.28266600565984845, + "p95_ms": 0.28775003738701344, + "p99_ms": 0.2877828199416399, + "index_ms": 77.17279100324959, + "by_category": { + "architecture": 0.7595979144169523, + "semantic": 0.6022146642666754, + "symbol": 0.6428706898503957 + } + }, + { + "repo": "fmtlib", + "language": "cpp", + "mode": "semble-auto", + "chunks": 966, + "tokens": 1392, + "ndcg5": 0.9161732909393884, + "ndcg10": 0.9161732909393884, + "p50_ms": 0.48360449727624655, + "p90_ms": 0.7497497135773324, + "p95_ms": 2.2252020746236667, + "p99_ms": 2.347974019940011, + "index_ms": 313.424667052459, + "by_category": { + "architecture": 0.8769765845238192, + "semantic": 0.9329718794032037, + "symbol": 0.8769765845238192 + } + }, + { + "repo": "fmtlib", + "language": "cpp", + "mode": "semble-balanced", + "chunks": 966, + "tokens": 1393, + "ndcg5": 0.9161732909393884, + "ndcg10": 0.9161732909393884, + "p50_ms": 0.47708299825899303, + "p90_ms": 0.6976756209041947, + "p95_ms": 2.2028917068382725, + "p99_ms": 2.32664555485826, + "index_ms": 313.424667052459, + "by_category": { + "architecture": 0.8769765845238192, + "semantic": 0.9329718794032037, + "symbol": 0.8769765845238192 + } + }, + { + "repo": "fmtlib", + "language": "cpp", + "mode": "semble-bm25", + "chunks": 966, + "tokens": 1499, + "ndcg5": 0.9530803155822426, + "ndcg10": 0.9530803155822426, + "p50_ms": 0.392541493056342, + "p90_ms": 0.5311249813530611, + "p95_ms": 1.5905083680991083, + "p99_ms": 1.61913528223522, + "index_ms": 313.424667052459, + "by_category": { + "architecture": 1.0, + "semantic": 0.9329718794032037, + "symbol": 1.0 + } + }, + { + "repo": "fmtlib", + "language": "cpp", + "mode": "semble-semantic", + "chunks": 966, + "tokens": 1314, + "ndcg5": 0.9096268032608155, + "ndcg10": 0.9096268032608155, + "p50_ms": 0.3846664621960372, + "p90_ms": 0.5233044736087338, + "p95_ms": 1.4876399422064424, + "p99_ms": 1.5233280090615153, + "index_ms": 313.424667052459, + "by_category": { + "architecture": 0.8769765845238192, + "semantic": 0.9236197541480996, + "symbol": 0.8769765845238192 + } + }, + { + "repo": "fmtlib", + "language": "cpp", + "mode": "unranked-auto", + "chunks": 966, + "tokens": 1464, + "ndcg5": 0.748904389350578, + "ndcg10": 0.7822377226839113, + "p50_ms": 0.2686465159058571, + "p90_ms": 0.2775121887680143, + "p95_ms": 0.2896146470448002, + "p99_ms": 0.29515654023271054, + "index_ms": 313.424667052459, + "by_category": { + "architecture": 0.7956176024115139, + "semantic": 0.7803263080793109, + "symbol": 0.7777777777777777 + } + }, + { + "repo": "fmtlib", + "language": "cpp", + "mode": "unranked-balanced", + "chunks": 966, + "tokens": 1464, + "ndcg5": 0.748904389350578, + "ndcg10": 0.7822377226839113, + "p50_ms": 0.26352101122029126, + "p90_ms": 0.2807044831570238, + "p95_ms": 0.28121071227360517, + "p99_ms": 0.2837749576428905, + "index_ms": 313.424667052459, + "by_category": { + "architecture": 0.7956176024115139, + "semantic": 0.7803263080793109, + "symbol": 0.7777777777777777 + } + }, + { + "repo": "fmtlib", + "language": "cpp", + "mode": "unranked-bm25", + "chunks": 966, + "tokens": 1507, + "ndcg5": 0.6408764682653968, + "ndcg10": 0.7235666486327041, + "p50_ms": 0.2600204898044467, + "p90_ms": 0.2764039149042219, + "p95_ms": 0.27705033717211336, + "p99_ms": 0.2771764592034742, + "index_ms": 313.424667052459, + "by_category": { + "architecture": 0.785402395702674, + "semantic": 0.6999757791971665, + "symbol": 0.7718216255952429 + } + }, + { + "repo": "fmtlib", + "language": "cpp", + "mode": "unranked-semantic", + "chunks": 966, + "tokens": 1442, + "ndcg5": 0.7911732909393884, + "ndcg10": 0.8062247907225875, + "p50_ms": 0.25689598987810314, + "p90_ms": 0.27092499076388776, + "p95_ms": 0.27820834366139024, + "p99_ms": 0.27947527065407485, + "index_ms": 313.424667052459, + "by_category": { + "architecture": 0.8333333333333334, + "semantic": 0.7946068438894106, + "symbol": 0.8333333333333334 + } + }, + { + "repo": "gin", + "language": "go", + "mode": "semble-auto", + "chunks": 1171, + "tokens": 1541, + "ndcg5": 0.8740805312254514, + "ndcg10": 0.8898537750647378, + "p50_ms": 0.5613749963231385, + "p90_ms": 2.2010374872479597, + "p95_ms": 2.797158318571747, + "p99_ms": 3.3714652294293037, + "index_ms": 190.66474999999627, + "by_category": { + "architecture": 0.8729222796953519, + "semantic": 0.8690492566475132, + "symbol": 1.0 + } + }, + { + "repo": "gin", + "language": "go", + "mode": "semble-balanced", + "chunks": 1171, + "tokens": 1536, + "ndcg5": 0.8740805312254514, + "ndcg10": 0.8898537750647378, + "p50_ms": 0.5643539770971984, + "p90_ms": 2.1894618694204846, + "p95_ms": 2.7548187586944555, + "p99_ms": 3.3562637155409893, + "index_ms": 190.66474999999627, + "by_category": { + "architecture": 0.8729222796953519, + "semantic": 0.8690492566475132, + "symbol": 1.0 + } + }, + { + "repo": "gin", + "language": "go", + "mode": "semble-bm25", + "chunks": 1171, + "tokens": 1534, + "ndcg5": 0.8619840587485633, + "ndcg10": 0.8619840587485633, + "p50_ms": 0.4372289986349642, + "p90_ms": 1.6434958321042363, + "p95_ms": 1.934624690329657, + "p99_ms": 2.837758500827475, + "index_ms": 190.66474999999627, + "by_category": { + "architecture": 0.9159340381057252, + "semantic": 0.7949160860306287, + "symbol": 1.0 + } + }, + { + "repo": "gin", + "language": "go", + "mode": "semble-semantic", + "chunks": 1171, + "tokens": 1478, + "ndcg5": 0.7241497556074997, + "ndcg10": 0.7600005469427045, + "p50_ms": 0.42383300024084747, + "p90_ms": 1.6506499960087246, + "p95_ms": 2.149822952924297, + "p99_ms": 2.9280318075325327, + "index_ms": 190.66474999999627, + "by_category": { + "architecture": 0.746085697019908, + "semantic": 0.7021360687940581, + "symbol": 1.0 + } + }, + { + "repo": "gin", + "language": "go", + "mode": "unranked-auto", + "chunks": 1171, + "tokens": 1515, + "ndcg5": 0.6479643384849536, + "ndcg10": 0.6766950696812629, + "p50_ms": 0.26060399250127375, + "p90_ms": 0.2744829864241183, + "p95_ms": 0.28769132622983307, + "p99_ms": 0.2969382255105302, + "index_ms": 190.66474999999627, + "by_category": { + "architecture": 0.5838234503511545, + "semantic": 0.6909349212222671, + "symbol": 0.8102255193577976 + } + }, + { + "repo": "gin", + "language": "go", + "mode": "unranked-balanced", + "chunks": 1171, + "tokens": 1515, + "ndcg5": 0.645773150943011, + "ndcg10": 0.6745038821393203, + "p50_ms": 0.25402099709026515, + "p90_ms": 0.2728708030190319, + "p95_ms": 0.2781871648039669, + "p99_ms": 0.2911710541229695, + "index_ms": 190.66474999999627, + "by_category": { + "architecture": 0.5838234503511545, + "semantic": 0.6909349212222671, + "symbol": 0.7956176024115139 + } + }, + { + "repo": "gin", + "language": "go", + "mode": "unranked-bm25", + "chunks": 1171, + "tokens": 1512, + "ndcg5": 0.4965883462802331, + "ndcg10": 0.5221169772861021, + "p50_ms": 0.2543330192565918, + "p90_ms": 0.2701749908737838, + "p95_ms": 0.2716374961892143, + "p99_ms": 0.28702752490062267, + "index_ms": 190.66474999999627, + "by_category": { + "architecture": 0.3475478663504797, + "semantic": 0.5205565994588824, + "symbol": 0.8769765845238192 + } + }, + { + "repo": "gin", + "language": "go", + "mode": "unranked-semantic", + "chunks": 1171, + "tokens": 1540, + "ndcg5": 0.7059158166984061, + "ndcg10": 0.7151446015373845, + "p50_ms": 0.2494369982741773, + "p90_ms": 0.26556665543466806, + "p95_ms": 0.2684877661522478, + "p99_ms": 0.28666393714956934, + "index_ms": 190.66474999999627, + "by_category": { + "architecture": 0.6131402517906418, + "semantic": 0.7488361557062997, + "symbol": 0.7956176024115139 + } + }, + { + "repo": "gson", + "language": "java", + "mode": "semble-auto", + "chunks": 2881, + "tokens": 2139, + "ndcg5": 0.8605604482211635, + "ndcg10": 0.8920725120838435, + "p50_ms": 0.8031460165511817, + "p90_ms": 4.082362190820278, + "p95_ms": 4.931987519375982, + "p99_ms": 6.539197503589091, + "index_ms": 385.844458010979, + "by_category": { + "architecture": 0.6776301220263528, + "semantic": 0.9130929753571457, + "symbol": 1.0 + } + }, + { + "repo": "gson", + "language": "java", + "mode": "semble-balanced", + "chunks": 2881, + "tokens": 2167, + "ndcg5": 0.8605604482211635, + "ndcg10": 0.8920725120838435, + "p50_ms": 0.7855830190237612, + "p90_ms": 4.0085329965222645, + "p95_ms": 4.9640517390798795, + "p99_ms": 6.430376734351737, + "index_ms": 385.844458010979, + "by_category": { + "architecture": 0.6776301220263528, + "semantic": 0.9130929753571457, + "symbol": 1.0 + } + }, + { + "repo": "gson", + "language": "java", + "mode": "semble-bm25", + "chunks": 2881, + "tokens": 2348, + "ndcg5": 0.7853784522318564, + "ndcg10": 0.8136883212894614, + "p50_ms": 0.563562469324097, + "p90_ms": 3.5906496690586223, + "p95_ms": 4.365070807398297, + "p99_ms": 4.7556469385744995, + "index_ms": 385.844458010979, + "by_category": { + "architecture": 0.5557789707673965, + "semantic": 0.8050650542719644, + "symbol": 1.0 + } + }, + { + "repo": "gson", + "language": "java", + "mode": "semble-semantic", + "chunks": 2881, + "tokens": 1733, + "ndcg5": 0.691170668855047, + "ndcg10": 0.7277277755820014, + "p50_ms": 0.5668124940712005, + "p90_ms": 3.6297625047154725, + "p95_ms": 3.7139771215152, + "p99_ms": 4.171529005980118, + "index_ms": 385.844458010979, + "by_category": { + "architecture": 0.2837516284937633, + "semantic": 0.7419548997664978, + "symbol": 1.0 + } + }, + { + "repo": "gson", + "language": "java", + "mode": "unranked-auto", + "chunks": 2881, + "tokens": 2046, + "ndcg5": 0.4743631777136435, + "ndcg10": 0.5492112750367848, + "p50_ms": 0.28852096875198185, + "p90_ms": 0.31769548659212893, + "p95_ms": 0.3410350764170289, + "p99_ms": 0.355507037602365, + "index_ms": 385.844458010979, + "by_category": { + "architecture": 0.6005048438681956, + "semantic": 0.521956981795408, + "symbol": 0.560439384551472 + } + }, + { + "repo": "gson", + "language": "java", + "mode": "unranked-balanced", + "chunks": 2881, + "tokens": 2048, + "ndcg5": 0.47217199017170086, + "ndcg10": 0.5319685877116431, + "p50_ms": 0.28533299337141216, + "p90_ms": 0.2975040988530964, + "p95_ms": 0.3268764354288578, + "p99_ms": 0.34527529962360853, + "index_ms": 385.844458010979, + "by_category": { + "architecture": 0.6005048438681956, + "semantic": 0.521956981795408, + "symbol": 0.5029637601343332 + } + }, + { + "repo": "gson", + "language": "java", + "mode": "unranked-bm25", + "chunks": 2881, + "tokens": 1875, + "ndcg5": 0.32588912804029996, + "ndcg10": 0.414301807830717, + "p50_ms": 0.27666648384183645, + "p90_ms": 0.3081285860389471, + "p95_ms": 0.32106699072755873, + "p99_ms": 0.33474700641818345, + "index_ms": 385.844458010979, + "by_category": { + "architecture": 0.42796366518113454, + "semantic": 0.2767334108765917, + "symbol": 0.6344745645206479 + } + }, + { + "repo": "gson", + "language": "java", + "mode": "unranked-semantic", + "chunks": 2881, + "tokens": 1541, + "ndcg5": 0.4047210925935835, + "ndcg10": 0.4391981186156513, + "p50_ms": 0.27558300644159317, + "p90_ms": 0.2882336149923504, + "p95_ms": 0.31234338821377605, + "p99_ms": 0.327701477217488, + "index_ms": 385.844458010979, + "by_category": { + "architecture": 0.311019236584229, + "semantic": 0.503988542597611, + "symbol": 0.4166666666666667 + } + }, + { + "repo": "guzzle", + "language": "php", + "mode": "semble-auto", + "chunks": 386, + "tokens": 1526, + "ndcg5": 0.9232405187664462, + "ndcg10": 0.9232405187664462, + "p50_ms": 0.5341669893823564, + "p90_ms": 2.018270618282259, + "p95_ms": 2.1774714812636375, + "p99_ms": 2.1947606839239597, + "index_ms": 52.57558298762888, + "by_category": { + "architecture": 1.0, + "semantic": 0.8819084904099174, + "symbol": 1.0 + } + }, + { + "repo": "guzzle", + "language": "php", + "mode": "semble-balanced", + "chunks": 386, + "tokens": 1519, + "ndcg5": 0.9232405187664462, + "ndcg10": 0.9232405187664462, + "p50_ms": 0.5379580252338201, + "p90_ms": 2.0320415671449155, + "p95_ms": 2.168611902743578, + "p99_ms": 2.2038896195590496, + "index_ms": 52.57558298762888, + "by_category": { + "architecture": 1.0, + "semantic": 0.8819084904099174, + "symbol": 1.0 + } + }, + { + "repo": "guzzle", + "language": "php", + "mode": "semble-bm25", + "chunks": 386, + "tokens": 1466, + "ndcg5": 0.9538319645532836, + "ndcg10": 0.9538319645532836, + "p50_ms": 0.42243750067427754, + "p90_ms": 1.4202452264726164, + "p95_ms": 1.5957683965098115, + "p99_ms": 1.7562864848878232, + "index_ms": 52.57558298762888, + "by_category": { + "architecture": 1.0, + "semantic": 0.9289722531588978, + "symbol": 1.0 + } + }, + { + "repo": "guzzle", + "language": "php", + "mode": "semble-semantic", + "chunks": 386, + "tokens": 1413, + "ndcg5": 0.9200497724127199, + "ndcg10": 0.9309701442536282, + "p50_ms": 0.422958517447114, + "p90_ms": 1.702654198743403, + "p95_ms": 1.7357377539156007, + "p99_ms": 2.0636139385169368, + "index_ms": 52.57558298762888, + "by_category": { + "architecture": 1.0, + "semantic": 0.8938002219286587, + "symbol": 1.0 + } + }, + { + "repo": "guzzle", + "language": "php", + "mode": "unranked-auto", + "chunks": 386, + "tokens": 1511, + "ndcg5": 0.7138660212595743, + "ndcg10": 0.7350055129799069, + "p50_ms": 0.2429999876767397, + "p90_ms": 0.2607880684081465, + "p95_ms": 0.26175410603173077, + "p99_ms": 0.26561721810139716, + "index_ms": 52.57558298762888, + "by_category": { + "architecture": 0.8161531146418005, + "semantic": 0.7015939355462522, + "symbol": 0.7827324383928644 + } + }, + { + "repo": "guzzle", + "language": "php", + "mode": "unranked-balanced", + "chunks": 386, + "tokens": 1511, + "ndcg5": 0.7204125089381472, + "ndcg10": 0.7415520006584797, + "p50_ms": 0.24397901142947376, + "p90_ms": 0.25603329413570464, + "p95_ms": 0.2599100727820769, + "p99_ms": 0.2629819914000109, + "index_ms": 52.57558298762888, + "by_category": { + "architecture": 0.8161531146418005, + "semantic": 0.7015939355462522, + "symbol": 0.8154648767857288 + } + }, + { + "repo": "guzzle", + "language": "php", + "mode": "unranked-bm25", + "chunks": 386, + "tokens": 1454, + "ndcg5": 0.7173793323602942, + "ndcg10": 0.7695838318606214, + "p50_ms": 0.23491651518270373, + "p90_ms": 0.25307920295745134, + "p95_ms": 0.2584940637461841, + "p99_ms": 0.2615652303211391, + "index_ms": 52.57558298762888, + "by_category": { + "architecture": 0.8710490642551528, + "semantic": 0.7137330341882284, + "symbol": 0.875 + } + }, + { + "repo": "guzzle", + "language": "php", + "mode": "unranked-semantic", + "chunks": 386, + "tokens": 1513, + "ndcg5": 0.6110336155180744, + "ndcg10": 0.6212527353974988, + "p50_ms": 0.2371664741076529, + "p90_ms": 0.25131641887128353, + "p95_ms": 0.25472948618698865, + "p99_ms": 0.25757950788829476, + "index_ms": 52.57558298762888, + "by_category": { + "architecture": 0.836048356303169, + "semantic": 0.5320699722338823, + "symbol": 0.75 + } + }, + { + "repo": "http4s", + "language": "scala", + "mode": "semble-auto", + "chunks": 1849, + "tokens": 1482, + "ndcg5": 0.9437503349954188, + "ndcg10": 0.9546707068363268, + "p50_ms": 2.557937492383644, + "p90_ms": 3.4832291596103464, + "p95_ms": 3.91564193123486, + "p99_ms": 4.8481612018076685, + "index_ms": 334.6527910325676, + "by_category": { + "architecture": 0.9438515431945408, + "semantic": 0.9472756790816368, + "symbol": 1.0 + } + }, + { + "repo": "http4s", + "language": "scala", + "mode": "semble-balanced", + "chunks": 1849, + "tokens": 1469, + "ndcg5": 0.9437503349954188, + "ndcg10": 0.9546707068363268, + "p50_ms": 2.5810830120462924, + "p90_ms": 3.507399983936921, + "p95_ms": 3.948468732414768, + "p99_ms": 4.801093743881209, + "index_ms": 334.6527910325676, + "by_category": { + "architecture": 0.9438515431945408, + "semantic": 0.9472756790816368, + "symbol": 1.0 + } + }, + { + "repo": "http4s", + "language": "scala", + "mode": "semble-bm25", + "chunks": 1849, + "tokens": 1631, + "ndcg5": 0.9087376752205156, + "ndcg10": 0.9189567950999399, + "p50_ms": 1.8350210157223046, + "p90_ms": 2.9884500370826577, + "p95_ms": 3.1531249609543015, + "p99_ms": 3.4058249829104166, + "index_ms": 334.6527910325676, + "by_category": { + "architecture": 0.9391765301179814, + "semantic": 0.897257593688918, + "symbol": 1.0 + } + }, + { + "repo": "http4s", + "language": "scala", + "mode": "semble-semantic", + "chunks": 1849, + "tokens": 1409, + "ndcg5": 0.9775325271359823, + "ndcg10": 0.9775325271359823, + "p50_ms": 1.8694584723562002, + "p90_ms": 3.062003495870158, + "p95_ms": 3.214655659394339, + "p99_ms": 3.676198326284065, + "index_ms": 334.6527910325676, + "by_category": { + "architecture": 0.9732402630493958, + "semantic": 0.9736378395408184, + "symbol": 1.0 + } + }, + { + "repo": "http4s", + "language": "scala", + "mode": "unranked-auto", + "chunks": 1849, + "tokens": 1431, + "ndcg5": 0.7802715031241851, + "ndcg10": 0.8113914111067462, + "p50_ms": 0.2652915136422962, + "p90_ms": 0.31954971491359174, + "p95_ms": 0.32214581733569503, + "p99_ms": 0.34906276268884534, + "index_ms": 334.6527910325676, + "by_category": { + "architecture": 0.8710490642551528, + "semantic": 0.7845536625570005, + "symbol": 0.8769765845238192 + } + }, + { + "repo": "http4s", + "language": "scala", + "mode": "unranked-balanced", + "chunks": 1849, + "tokens": 1433, + "ndcg5": 0.7802715031241851, + "ndcg10": 0.8113914111067462, + "p50_ms": 0.26227100170217454, + "p90_ms": 0.3063746960833669, + "p95_ms": 0.3281521436292678, + "p99_ms": 0.3456640394870191, + "index_ms": 334.6527910325676, + "by_category": { + "architecture": 0.8710490642551528, + "semantic": 0.7845536625570005, + "symbol": 0.8769765845238192 + } + }, + { + "repo": "http4s", + "language": "scala", + "mode": "unranked-bm25", + "chunks": 1849, + "tokens": 1465, + "ndcg5": 0.7437503349954186, + "ndcg10": 0.7739768201505995, + "p50_ms": 0.25599999935366213, + "p90_ms": 0.2938798279501498, + "p95_ms": 0.31090861593838787, + "p99_ms": 0.34118175215553487, + "index_ms": 334.6527910325676, + "by_category": { + "architecture": 0.8710490642551528, + "semantic": 0.7311042469053625, + "symbol": 0.8769765845238192 + } + }, + { + "repo": "http4s", + "language": "scala", + "mode": "unranked-semantic", + "chunks": 1849, + "tokens": 1391, + "ndcg5": 0.7468280284933199, + "ndcg10": 0.7776527721158054, + "p50_ms": 0.2575624966993928, + "p90_ms": 0.29163722647354007, + "p95_ms": 0.3067083423957229, + "p99_ms": 0.3402752941474318, + "index_ms": 334.6527910325676, + "by_category": { + "architecture": 0.6812571030506751, + "semantic": 0.7863774380831489, + "symbol": 0.8333333333333334 + } + }, + { + "repo": "httpx", + "language": "python", + "mode": "semble-auto", + "chunks": 488, + "tokens": 1558, + "ndcg5": 0.8955062293473781, + "ndcg10": 0.9042955482416433, + "p50_ms": 0.46845898032188416, + "p90_ms": 2.09349999204278, + "p95_ms": 2.340042032301426, + "p99_ms": 2.3921420332044363, + "index_ms": 60.83579198457301, + "by_category": { + "architecture": 0.8795597859882119, + "semantic": 0.9113731397317297, + "symbol": 0.9261859507142916 + } + }, + { + "repo": "httpx", + "language": "python", + "mode": "semble-balanced", + "chunks": 488, + "tokens": 1560, + "ndcg5": 0.8955062293473781, + "ndcg10": 0.9042955482416433, + "p50_ms": 0.46504195779561996, + "p90_ms": 2.0884170080535114, + "p95_ms": 2.3309169919230044, + "p99_ms": 2.3678170167841017, + "index_ms": 60.83579198457301, + "by_category": { + "architecture": 0.8795597859882119, + "semantic": 0.9113731397317297, + "symbol": 0.9261859507142916 + } + }, + { + "repo": "httpx", + "language": "python", + "mode": "semble-bm25", + "chunks": 488, + "tokens": 1568, + "ndcg5": 0.8062355796019366, + "ndcg10": 0.8234648645335373, + "p50_ms": 0.3773749922402203, + "p90_ms": 1.6286250320263207, + "p95_ms": 1.6811669920571148, + "p99_ms": 2.0879669697023933, + "index_ms": 60.83579198457301, + "by_category": { + "architecture": 0.6610527108209145, + "semantic": 0.8927181584318247, + "symbol": 0.9261859507142916 + } + }, + { + "repo": "httpx", + "language": "python", + "mode": "semble-semantic", + "chunks": 488, + "tokens": 1510, + "ndcg5": 0.8560146564451269, + "ndcg10": 0.8560146564451269, + "p50_ms": 0.3697079955600202, + "p90_ms": 1.6349590150639415, + "p95_ms": 1.7031659954227507, + "p99_ms": 2.046899555716664, + "index_ms": 60.83579198457301, + "by_category": { + "architecture": 0.8252778468929375, + "semantic": 0.8409370115028494, + "symbol": 0.9261859507142916 + } + }, + { + "repo": "httpx", + "language": "python", + "mode": "unranked-auto", + "chunks": 488, + "tokens": 1696, + "ndcg5": 0.8047382392984483, + "ndcg10": 0.8135275581927134, + "p50_ms": 0.23795804008841515, + "p90_ms": 0.2603750326670706, + "p95_ms": 0.2644589985720813, + "p99_ms": 0.28872539987787604, + "index_ms": 60.83579198457301, + "by_category": { + "architecture": 0.7608370192414584, + "semantic": 0.8474766481983685, + "symbol": 0.8261859507142916 + } + }, + { + "repo": "httpx", + "language": "python", + "mode": "unranked-balanced", + "chunks": 488, + "tokens": 1697, + "ndcg5": 0.8285477631079721, + "ndcg10": 0.8373370820022372, + "p50_ms": 0.22674998035654426, + "p90_ms": 0.2556249964982271, + "p95_ms": 0.25758298579603434, + "p99_ms": 0.25874937418848276, + "index_ms": 60.83579198457301, + "by_category": { + "architecture": 0.7608370192414584, + "semantic": 0.8474766481983685, + "symbol": 0.9261859507142916 + } + }, + { + "repo": "httpx", + "language": "python", + "mode": "unranked-bm25", + "chunks": 488, + "tokens": 1668, + "ndcg5": 0.6912016347811178, + "ndcg10": 0.7084309197127185, + "p50_ms": 0.23329199757426977, + "p90_ms": 0.32699998700991273, + "p95_ms": 0.3691249876283109, + "p99_ms": 0.44789140811190015, + "index_ms": 60.83579198457301, + "by_category": { + "architecture": 0.5827857957686192, + "semantic": 0.7629857983903735, + "symbol": 0.7861353116146785 + } + }, + { + "repo": "httpx", + "language": "python", + "mode": "unranked-semantic", + "chunks": 488, + "tokens": 1582, + "ndcg5": 0.7603176330214843, + "ndcg10": 0.7603176330214843, + "p50_ms": 0.27295900508761406, + "p90_ms": 0.5920410039834678, + "p95_ms": 0.6438330165110528, + "p99_ms": 0.703566602896899, + "index_ms": 60.83579198457301, + "by_category": { + "architecture": 0.6325522654368135, + "semantic": 0.8230971868691133, + "symbol": 0.8261859507142916 + } + }, + { + "repo": "jackson-databind", + "language": "java", + "mode": "semble-auto", + "chunks": 8975, + "tokens": 1451, + "ndcg5": 0.6688795406139487, + "ndcg10": 0.7085306436511664, + "p50_ms": 1.0541249939706177, + "p90_ms": 7.330003927927464, + "p95_ms": 7.513154461048544, + "p99_ms": 8.905664491467176, + "index_ms": 1573.3569579897448, + "by_category": { + "architecture": 0.666564092985102, + "semantic": 0.5511752325288629, + "symbol": 0.9795358858896749 + } + }, + { + "repo": "jackson-databind", + "language": "java", + "mode": "semble-balanced", + "chunks": 8975, + "tokens": 1451, + "ndcg5": 0.6556761344853193, + "ndcg10": 0.7049985577034003, + "p50_ms": 1.0850830003619194, + "p90_ms": 6.409637496108191, + "p95_ms": 7.861506240442397, + "p99_ms": 9.493701215833422, + "index_ms": 1573.3569579897448, + "by_category": { + "architecture": 0.666564092985102, + "semantic": 0.5511752325288629, + "symbol": 0.9677622660637882 + } + }, + { + "repo": "jackson-databind", + "language": "java", + "mode": "semble-bm25", + "chunks": 8975, + "tokens": 1683, + "ndcg5": 0.6878563885187226, + "ndcg10": 0.697527708699586, + "p50_ms": 0.7450625125784427, + "p90_ms": 5.376591288950296, + "p95_ms": 6.332480622222648, + "p99_ms": 7.625462542055173, + "index_ms": 1573.3569579897448, + "by_category": { + "architecture": 0.5990536526463072, + "semantic": 0.5505873234177985, + "symbol": 1.0 + } + }, + { + "repo": "jackson-databind", + "language": "java", + "mode": "semble-semantic", + "chunks": 8975, + "tokens": 1415, + "ndcg5": 0.6811305269351839, + "ndcg10": 0.6920508987760922, + "p50_ms": 0.8089164912234992, + "p90_ms": 6.307162792654709, + "p95_ms": 7.218939569429495, + "p99_ms": 8.321920706075614, + "index_ms": 1573.3569579897448, + "by_category": { + "architecture": 0.4030687082608802, + "semantic": 0.6902808046057758, + "symbol": 0.9355245321275764 + } + }, + { + "repo": "jackson-databind", + "language": "java", + "mode": "unranked-auto", + "chunks": 8975, + "tokens": 1632, + "ndcg5": 0.43715933039679766, + "ndcg10": 0.4816633054323935, + "p50_ms": 0.3838124976027757, + "p90_ms": 0.4987671796698123, + "p95_ms": 0.5028868617955595, + "p99_ms": 0.5233437579590827, + "index_ms": 1573.3569579897448, + "by_category": { + "architecture": 0.5538115959075618, + "semantic": 0.38150350789296295, + "symbol": 0.5717794263455654 + } + }, + { + "repo": "jackson-databind", + "language": "java", + "mode": "unranked-balanced", + "chunks": 8975, + "tokens": 1651, + "ndcg5": 0.3993631777136435, + "ndcg10": 0.46167751210464036, + "p50_ms": 0.37781198625452816, + "p90_ms": 0.48341667861677706, + "p95_ms": 0.4908607283141464, + "p99_ms": 0.49570574308745563, + "index_ms": 1573.3569579897448, + "by_category": { + "architecture": 0.5538115959075618, + "semantic": 0.38150350789296295, + "symbol": 0.5051601152530552 + } + }, + { + "repo": "jackson-databind", + "language": "java", + "mode": "unranked-bm25", + "chunks": 8975, + "tokens": 1735, + "ndcg5": 0.5587538279832204, + "ndcg10": 0.5587538279832204, + "p50_ms": 0.38295850390568376, + "p90_ms": 0.4696997115388513, + "p95_ms": 0.4907041962724179, + "p99_ms": 0.5204080359544605, + "index_ms": 1573.3569579897448, + "by_category": { + "architecture": 0.5476927096222367, + "semantic": 0.4658373405795898, + "symbol": 0.7073461577228194 + } + }, + { + "repo": "jackson-databind", + "language": "java", + "mode": "unranked-semantic", + "chunks": 8975, + "tokens": 1479, + "ndcg5": 0.34961152140157087, + "ndcg10": 0.37825582192088386, + "p50_ms": 0.37964549846947193, + "p90_ms": 0.47017891774885356, + "p95_ms": 0.4867211420787499, + "p99_ms": 0.5198442557593808, + "index_ms": 1573.3569579897448, + "by_category": { + "architecture": 0.2828873999106688, + "semantic": 0.5683047374153594, + "symbol": 0.17265613368768315 + } + }, + { + "repo": "kotlinx-coroutines", + "language": "kotlin", + "mode": "semble-auto", + "chunks": 1569, + "tokens": 4462, + "ndcg5": 0.9074356157188728, + "ndcg10": 0.9074356157188728, + "p50_ms": 1.5235835162457079, + "p90_ms": 6.382792012300343, + "p95_ms": 7.202704480732793, + "p99_ms": 9.063374492689032, + "index_ms": 268.4546670061536, + "by_category": { + "architecture": 0.8333333333333334, + "semantic": 0.9034794510269613, + "symbol": 1.0 + } + }, + { + "repo": "kotlinx-coroutines", + "language": "kotlin", + "mode": "semble-balanced", + "chunks": 1569, + "tokens": 4465, + "ndcg5": 0.9074356157188728, + "ndcg10": 0.9074356157188728, + "p50_ms": 1.4926879957783967, + "p90_ms": 6.317874742671848, + "p95_ms": 7.217041708645413, + "p99_ms": 9.13097556272987, + "index_ms": 268.4546670061536, + "by_category": { + "architecture": 0.8333333333333334, + "semantic": 0.9034794510269613, + "symbol": 1.0 + } + }, + { + "repo": "kotlinx-coroutines", + "language": "kotlin", + "mode": "semble-bm25", + "chunks": 1569, + "tokens": 4525, + "ndcg5": 0.8911732909393884, + "ndcg10": 0.8911732909393884, + "p50_ms": 1.3267920003272593, + "p90_ms": 5.914792284602302, + "p95_ms": 7.746579489321448, + "p99_ms": 7.9789494891883805, + "index_ms": 268.4546670061536, + "by_category": { + "architecture": 0.8769765845238192, + "semantic": 0.8708954332297364, + "symbol": 1.0 + } + }, + { + "repo": "kotlinx-coroutines", + "language": "kotlin", + "mode": "semble-semantic", + "chunks": 1569, + "tokens": 3836, + "ndcg5": 0.8583247437591728, + "ndcg10": 0.8583247437591728, + "p50_ms": 1.3353539980016649, + "p90_ms": 4.909278807463124, + "p95_ms": 5.275345160043802, + "p99_ms": 8.69236900180112, + "index_ms": 268.4546670061536, + "by_category": { + "architecture": 0.8333333333333334, + "semantic": 0.833321062513104, + "symbol": 1.0 + } + }, + { + "repo": "kotlinx-coroutines", + "language": "kotlin", + "mode": "unranked-auto", + "chunks": 1569, + "tokens": 4760, + "ndcg5": 0.6920624189796882, + "ndcg10": 0.7098727783350893, + "p50_ms": 0.27462499565444887, + "p90_ms": 0.287174730328843, + "p95_ms": 0.2926229644799605, + "p99_ms": 0.29449179128278047, + "index_ms": 268.4546670061536, + "by_category": { + "architecture": 0.7103099178571526, + "semantic": 0.7190375580807379, + "symbol": 0.6666666666666666 + } + }, + { + "repo": "kotlinx-coroutines", + "language": "kotlin", + "mode": "unranked-balanced", + "chunks": 1569, + "tokens": 4774, + "ndcg5": 0.6920624189796882, + "ndcg10": 0.7098727783350893, + "p50_ms": 0.2730000123847276, + "p90_ms": 0.29574998188763857, + "p95_ms": 0.2959208330139518, + "p99_ms": 0.29851777013391256, + "index_ms": 268.4546670061536, + "by_category": { + "architecture": 0.7103099178571526, + "semantic": 0.7190375580807379, + "symbol": 0.6666666666666666 + } + }, + { + "repo": "kotlinx-coroutines", + "language": "kotlin", + "mode": "unranked-bm25", + "chunks": 1569, + "tokens": 4232, + "ndcg5": 0.6920624189796882, + "ndcg10": 0.6920624189796882, + "p50_ms": 0.2684789942577481, + "p90_ms": 0.2831588906701654, + "p95_ms": 0.2961038873763755, + "p99_ms": 0.31162079016212374, + "index_ms": 268.4546670061536, + "by_category": { + "architecture": 0.7539531690476383, + "semantic": 0.7106042227770993, + "symbol": 0.5436432511904858 + } + }, + { + "repo": "kotlinx-coroutines", + "language": "kotlin", + "mode": "unranked-semantic", + "chunks": 1569, + "tokens": 4608, + "ndcg5": 0.7605159313011154, + "ndcg10": 0.7605159313011154, + "p50_ms": 0.26966651785187423, + "p90_ms": 0.2854712132830173, + "p95_ms": 0.28606728883460164, + "p99_ms": 0.2877134666778147, + "index_ms": 268.4546670061536, + "by_category": { + "architecture": 0.8102255193577976, + "semantic": 0.7699744334249224, + "symbol": 0.6666666666666666 + } + }, + { + "repo": "ktor", + "language": "kotlin", + "mode": "semble-auto", + "chunks": 761, + "tokens": 2127, + "ndcg5": 0.8281680939691519, + "ndcg10": 0.8524483785913294, + "p50_ms": 0.7606045110151172, + "p90_ms": 3.8706622086465368, + "p95_ms": 4.513377047260292, + "p99_ms": 4.627408218220808, + "index_ms": 115.6385830254294, + "by_category": { + "architecture": 0.7987170106064461, + "semantic": 0.7797069282259236, + "symbol": 1.0 + } + }, + { + "repo": "ktor", + "language": "kotlin", + "mode": "semble-balanced", + "chunks": 761, + "tokens": 2012, + "ndcg5": 0.8281680939691519, + "ndcg10": 0.8524483785913294, + "p50_ms": 0.7600625103805214, + "p90_ms": 3.8275422703009108, + "p95_ms": 4.372910724487156, + "p99_ms": 4.716715754475444, + "index_ms": 115.6385830254294, + "by_category": { + "architecture": 0.7987170106064461, + "semantic": 0.7797069282259236, + "symbol": 1.0 + } + }, + { + "repo": "ktor", + "language": "kotlin", + "mode": "semble-bm25", + "chunks": 761, + "tokens": 2831, + "ndcg5": 0.7365399767327141, + "ndcg10": 0.767242684753733, + "p50_ms": 0.5339579947758466, + "p90_ms": 3.2300246763043114, + "p95_ms": 3.754591691540554, + "p99_ms": 3.852885566884652, + "index_ms": 115.6385830254294, + "by_category": { + "architecture": 0.7412775583289012, + "semantic": 0.5937015409674787, + "symbol": 1.0 + } + }, + { + "repo": "ktor", + "language": "kotlin", + "mode": "semble-semantic", + "chunks": 761, + "tokens": 1702, + "ndcg5": 0.8759795285115507, + "ndcg10": 0.8856508486924142, + "p50_ms": 0.5312079738359898, + "p90_ms": 3.1812794855795805, + "p95_ms": 3.4219794848468155, + "p99_ms": 3.5768294695299114, + "index_ms": 115.6385830254294, + "by_category": { + "architecture": 0.8651569245615428, + "semantic": 0.8081312145596405, + "symbol": 1.0 + } + }, + { + "repo": "ktor", + "language": "kotlin", + "mode": "unranked-auto", + "chunks": 761, + "tokens": 1999, + "ndcg5": 0.5872915919468538, + "ndcg10": 0.6508635193707744, + "p50_ms": 0.26235452969558537, + "p90_ms": 0.28650807798840106, + "p95_ms": 0.28903193306177855, + "p99_ms": 0.3034400148317218, + "index_ms": 115.6385830254294, + "by_category": { + "architecture": 0.5207978312225714, + "semantic": 0.5630012872548705, + "symbol": 0.9051127596788988 + } + }, + { + "repo": "ktor", + "language": "kotlin", + "mode": "unranked-balanced", + "chunks": 761, + "tokens": 1988, + "ndcg5": 0.5851004044049112, + "ndcg10": 0.6486723318288318, + "p50_ms": 0.2570835058577359, + "p90_ms": 0.2844714268576354, + "p95_ms": 0.2877975784940645, + "p99_ms": 0.30245954112615436, + "index_ms": 115.6385830254294, + "by_category": { + "architecture": 0.5207978312225714, + "semantic": 0.5630012872548705, + "symbol": 0.8978088012057569 + } + }, + { + "repo": "ktor", + "language": "kotlin", + "mode": "unranked-bm25", + "chunks": 761, + "tokens": 2805, + "ndcg5": 0.4539393393932413, + "ndcg10": 0.5127796305292629, + "p50_ms": 0.25595849729143083, + "p90_ms": 0.2774628112092614, + "p95_ms": 0.2814145584125072, + "p99_ms": 0.30101571814157063, + "index_ms": 115.6385830254294, + "by_category": { + "architecture": 0.37236839230391844, + "semantic": 0.28829773012662435, + "symbol": 0.9384882922619097 + } + }, + { + "repo": "ktor", + "language": "kotlin", + "mode": "unranked-semantic", + "chunks": 761, + "tokens": 1550, + "ndcg5": 0.5486892332004437, + "ndcg10": 0.6021427843608157, + "p50_ms": 0.2539169800002128, + "p90_ms": 0.27683748048730195, + "p95_ms": 0.2840167027898133, + "p99_ms": 0.29567053774371743, + "index_ms": 115.6385830254294, + "by_category": { + "architecture": 0.5902351351448782, + "semantic": 0.5605818286460981, + "symbol": 0.66452282344658 + } + }, + { + "repo": "laravel-framework", + "language": "php", + "mode": "semble-auto", + "chunks": 11681, + "tokens": 1402, + "ndcg5": 0.7625639617579317, + "ndcg10": 0.7880925927638007, + "p50_ms": 0.8429375011473894, + "p90_ms": 5.351812543813139, + "p95_ms": 5.494512527366169, + "p99_ms": 5.756902490393259, + "index_ms": 1608.2279999973252, + "by_category": { + "architecture": 0.7280655251168765, + "semantic": 0.7816070473207135, + "symbol": 0.9077324383928644 + } + }, + { + "repo": "laravel-framework", + "language": "php", + "mode": "semble-balanced", + "chunks": 11681, + "tokens": 1393, + "ndcg5": 0.7625639617579317, + "ndcg10": 0.7880925927638007, + "p50_ms": 0.7865625084377825, + "p90_ms": 5.326387780951336, + "p95_ms": 5.451556266052648, + "p99_ms": 5.665211234008893, + "index_ms": 1608.2279999973252, + "by_category": { + "architecture": 0.7280655251168765, + "semantic": 0.7816070473207135, + "symbol": 0.9077324383928644 + } + }, + { + "repo": "laravel-framework", + "language": "php", + "mode": "semble-bm25", + "chunks": 11681, + "tokens": 1454, + "ndcg5": 0.7669877978303391, + "ndcg10": 0.7779081696712472, + "p50_ms": 0.6130209949333221, + "p90_ms": 4.642958415206523, + "p95_ms": 5.093194395885804, + "p99_ms": 5.581272456911392, + "index_ms": 1608.2279999973252, + "by_category": { + "architecture": 0.6964295285583392, + "semantic": 0.7835807711050126, + "symbol": 0.9077324383928644 + } + }, + { + "repo": "laravel-framework", + "language": "php", + "mode": "semble-semantic", + "chunks": 11681, + "tokens": 1349, + "ndcg5": 0.6267551360919394, + "ndcg10": 0.653427459786543, + "p50_ms": 0.6307499716058373, + "p90_ms": 5.042595945997164, + "p95_ms": 5.225023571983911, + "p99_ms": 5.573704715934581, + "index_ms": 1608.2279999973252, + "by_category": { + "architecture": 0.5223798710259674, + "semantic": 0.6833367323784634, + "symbol": 0.8154648767857289 + } + }, + { + "repo": "laravel-framework", + "language": "php", + "mode": "unranked-auto", + "chunks": 11681, + "tokens": 1471, + "ndcg5": 0.34030599367766057, + "ndcg10": 0.4473687398168463, + "p50_ms": 0.3834374947473407, + "p90_ms": 0.4675955802667886, + "p95_ms": 0.47185267321765423, + "p99_ms": 0.4835369437932968, + "index_ms": 1608.2279999973252, + "by_category": { + "architecture": 0.564375576997099, + "semantic": 0.4119645478932605, + "symbol": 0.32226620657947197 + } + }, + { + "repo": "laravel-framework", + "language": "php", + "mode": "unranked-balanced", + "chunks": 11681, + "tokens": 1471, + "ndcg5": 0.34030599367766057, + "ndcg10": 0.4473687398168463, + "p50_ms": 0.38418747135438025, + "p90_ms": 0.4454291716683657, + "p95_ms": 0.5014691007090732, + "p99_ms": 0.5079610453685746, + "index_ms": 1608.2279999973252, + "by_category": { + "architecture": 0.564375576997099, + "semantic": 0.4119645478932605, + "symbol": 0.32226620657947197 + } + }, + { + "repo": "laravel-framework", + "language": "php", + "mode": "unranked-bm25", + "chunks": 11681, + "tokens": 1442, + "ndcg5": 0.41538891565470487, + "ndcg10": 0.4789534828143237, + "p50_ms": 0.3782915009651333, + "p90_ms": 0.42717502801679086, + "p95_ms": 0.49176048778463155, + "p99_ms": 0.5090192990610376, + "index_ms": 1608.2279999973252, + "by_category": { + "architecture": 0.7005416846445313, + "semantic": 0.2763772289862482, + "symbol": 0.5469707007246307 + } + }, + { + "repo": "laravel-framework", + "language": "php", + "mode": "unranked-semantic", + "chunks": 11681, + "tokens": 1381, + "ndcg5": 0.21806765580733928, + "ndcg10": 0.2913023620392444, + "p50_ms": 0.3748750314116478, + "p90_ms": 0.4494502907618881, + "p95_ms": 0.4788167017977685, + "p99_ms": 0.493130557006225, + "index_ms": 1608.2279999973252, + "by_category": { + "architecture": 0.4191290359358238, + "semantic": 0.21023822102601344, + "symbol": 0.25 + } + }, + { + "repo": "lazy.nvim", + "language": "lua", + "mode": "semble-auto", + "chunks": 596, + "tokens": 1547, + "ndcg5": 0.744298123441679, + "ndcg10": 0.7621084827970801, + "p50_ms": 0.586728478083387, + "p90_ms": 0.6351209303829819, + "p95_ms": 0.6712527014315128, + "p99_ms": 0.6909169442951679, + "index_ms": 100.65325000323355, + "by_category": { + "architecture": 0.8382026389820946, + "semantic": 0.6998496277366136 + } + }, + { + "repo": "lazy.nvim", + "language": "lua", + "mode": "semble-balanced", + "chunks": 596, + "tokens": 1547, + "ndcg5": 0.744298123441679, + "ndcg10": 0.7621084827970801, + "p50_ms": 0.5777080077677965, + "p90_ms": 0.6030628050211817, + "p95_ms": 0.620416650781408, + "p99_ms": 0.6419497646857053, + "index_ms": 100.65325000323355, + "by_category": { + "architecture": 0.8382026389820946, + "semantic": 0.6998496277366136 + } + }, + { + "repo": "lazy.nvim", + "language": "lua", + "mode": "semble-bm25", + "chunks": 596, + "tokens": 1534, + "ndcg5": 0.6875006699908374, + "ndcg10": 0.7164071526226262, + "p50_ms": 0.4409374960232526, + "p90_ms": 0.45800829539075494, + "p95_ms": 0.4630913812434301, + "p99_ms": 0.47081826895009726, + "index_ms": 100.65325000323355, + "by_category": { + "architecture": 0.8476118227349181, + "semantic": 0.6090578770762055 + } + }, + { + "repo": "lazy.nvim", + "language": "lua", + "mode": "semble-semantic", + "chunks": 596, + "tokens": 1489, + "ndcg5": 0.7372697611682815, + "ndcg10": 0.7756718125454544, + "p50_ms": 0.44410399277694523, + "p90_ms": 0.4678916768170893, + "p95_ms": 0.4730169603135437, + "p99_ms": 0.4802370152901858, + "index_ms": 100.65325000323355, + "by_category": { + "architecture": 0.91680837985909, + "semantic": 0.6601964392888433 + } + }, + { + "repo": "lazy.nvim", + "language": "lua", + "mode": "unranked-auto", + "chunks": 596, + "tokens": 1508, + "ndcg5": 0.59593890045302, + "ndcg10": 0.6521372572323081, + "p50_ms": 0.26825000531971455, + "p90_ms": 0.27418749523349106, + "p95_ms": 0.27630838158074766, + "p99_ms": 0.2828952606068924, + "index_ms": 100.65325000323355, + "by_category": { + "architecture": 0.7149853068731437, + "semantic": 0.600716125707988 + } + }, + { + "repo": "lazy.nvim", + "language": "lua", + "mode": "unranked-balanced", + "chunks": 596, + "tokens": 1508, + "ndcg5": 0.59593890045302, + "ndcg10": 0.6521372572323081, + "p50_ms": 0.2626874775160104, + "p90_ms": 0.27859940310008824, + "p95_ms": 0.2798166504362598, + "p99_ms": 0.2903297246666625, + "index_ms": 100.65325000323355, + "by_category": { + "architecture": 0.7149853068731437, + "semantic": 0.600716125707988 + } + }, + { + "repo": "lazy.nvim", + "language": "lua", + "mode": "unranked-bm25", + "chunks": 596, + "tokens": 1510, + "ndcg5": 0.46682888807279116, + "ndcg10": 0.5375654909846069, + "p50_ms": 0.26056249043904245, + "p90_ms": 0.2712043933570385, + "p95_ms": 0.2730785374296829, + "p99_ms": 0.27934929297771305, + "index_ms": 100.65325000323355, + "by_category": { + "architecture": 0.6006556892647142, + "semantic": 0.4859462378463371 + } + }, + { + "repo": "lazy.nvim", + "language": "lua", + "mode": "unranked-semantic", + "chunks": 596, + "tokens": 1470, + "ndcg5": 0.56315626712424, + "ndcg10": 0.620769885079751, + "p50_ms": 0.26302103651687503, + "p90_ms": 0.2797125023789704, + "p95_ms": 0.28517086466308683, + "p99_ms": 0.2877677575452253, + "index_ms": 100.65325000323355, + "by_category": { + "architecture": 0.7866192858804019, + "semantic": 0.4850749207883092 + } + }, + { + "repo": "libuv", + "language": "c", + "mode": "semble-auto", + "chunks": 2638, + "tokens": 1541, + "ndcg5": 0.5674229672679589, + "ndcg10": 0.623763115451377, + "p50_ms": 0.5708130192942917, + "p90_ms": 0.6952380586881192, + "p95_ms": 0.7371291780145839, + "p99_ms": 0.7713930081808938, + "index_ms": 393.6415830394253, + "by_category": { + "architecture": 0.5, + "semantic": 0.6097367949459744, + "symbol": 1.0 + } + }, + { + "repo": "libuv", + "language": "c", + "mode": "semble-balanced", + "chunks": 2638, + "tokens": 1541, + "ndcg5": 0.5674229672679589, + "ndcg10": 0.623763115451377, + "p50_ms": 0.568813004065305, + "p90_ms": 0.6609619013033807, + "p95_ms": 0.6672437302768232, + "p99_ms": 0.7393487263470887, + "index_ms": 393.6415830394253, + "by_category": { + "architecture": 0.5, + "semantic": 0.6097367949459744, + "symbol": 1.0 + } + }, + { + "repo": "libuv", + "language": "c", + "mode": "semble-bm25", + "chunks": 2638, + "tokens": 1558, + "ndcg5": 0.656603694232095, + "ndcg10": 0.6767528509119816, + "p50_ms": 0.43827100307680666, + "p90_ms": 0.5065163073595613, + "p95_ms": 0.521855620900169, + "p99_ms": 0.5247375380713493, + "index_ms": 393.6415830394253, + "by_category": { + "architecture": 0.43067655807339306, + "semantic": 0.6724655811203466, + "symbol": 1.0 + } + }, + { + "repo": "libuv", + "language": "c", + "mode": "semble-semantic", + "chunks": 2638, + "tokens": 1465, + "ndcg5": 0.5288922003888519, + "ndcg10": 0.5657836439628798, + "p50_ms": 0.42845847201533616, + "p90_ms": 0.4890709009487182, + "p95_ms": 0.4907673253910616, + "p99_ms": 0.5068534804740921, + "index_ms": 393.6415830394253, + "by_category": { + "architecture": 0.5, + "semantic": 0.5730929377365331, + "symbol": 0.5 + } + }, + { + "repo": "libuv", + "language": "c", + "mode": "unranked-auto", + "chunks": 2638, + "tokens": 1502, + "ndcg5": 0.3949118594962764, + "ndcg10": 0.4354992559350116, + "p50_ms": 0.27233350556343794, + "p90_ms": 0.29319998575374484, + "p95_ms": 0.299112472566776, + "p99_ms": 0.31032246944960207, + "index_ms": 393.6415830394253, + "by_category": { + "architecture": 0.6309297535714575, + "semantic": 0.421058631396043, + "symbol": 0.5 + } + }, + { + "repo": "libuv", + "language": "c", + "mode": "unranked-balanced", + "chunks": 2638, + "tokens": 1502, + "ndcg5": 0.3949118594962764, + "ndcg10": 0.4354992559350116, + "p50_ms": 0.27229150873608887, + "p90_ms": 0.304674991639331, + "p95_ms": 0.30873331998009235, + "p99_ms": 0.31227946805302054, + "index_ms": 393.6415830394253, + "by_category": { + "architecture": 0.6309297535714575, + "semantic": 0.421058631396043, + "symbol": 0.5 + } + }, + { + "repo": "libuv", + "language": "c", + "mode": "unranked-bm25", + "chunks": 2638, + "tokens": 1544, + "ndcg5": 0.3600344905534755, + "ndcg10": 0.49873206556011385, + "p50_ms": 0.26854148018173873, + "p90_ms": 0.2870628144592047, + "p95_ms": 0.29994786600582307, + "p99_ms": 0.33905595890246326, + "index_ms": 393.6415830394253, + "by_category": { + "architecture": 0.3562071871080222, + "semantic": 0.5145681631659018, + "symbol": 0.3562071871080222 + } + }, + { + "repo": "libuv", + "language": "c", + "mode": "unranked-semantic", + "chunks": 2638, + "tokens": 1509, + "ndcg5": 0.3810807658464532, + "ndcg10": 0.4155801703480549, + "p50_ms": 0.2657709992490709, + "p90_ms": 0.2894584438763559, + "p95_ms": 0.2990693988977, + "p99_ms": 0.3140466887271032, + "index_ms": 393.6415830394253, + "by_category": { + "architecture": 1.0, + "semantic": 0.3894763006276176, + "symbol": 0.3010299956639812 + } + }, + { + "repo": "messagepack-csharp", + "language": "csharp", + "mode": "semble-auto", + "chunks": 2179, + "tokens": 1478, + "ndcg5": 0.8810595925213971, + "ndcg10": 0.8810595925213971, + "p50_ms": 2.29714551824145, + "p90_ms": 3.8096701842732728, + "p95_ms": 3.838303845259361, + "p99_ms": 3.9822607528185467, + "index_ms": 357.34845796832815, + "by_category": { + "architecture": 0.7098603945740938, + "semantic": 0.8646389191257297, + "symbol": 1.0 + } + }, + { + "repo": "messagepack-csharp", + "language": "csharp", + "mode": "semble-balanced", + "chunks": 2179, + "tokens": 1453, + "ndcg5": 0.8810595925213971, + "ndcg10": 0.8810595925213971, + "p50_ms": 2.0927084842696786, + "p90_ms": 3.6393703310750425, + "p95_ms": 3.6978798336349428, + "p99_ms": 3.875275978352874, + "index_ms": 357.34845796832815, + "by_category": { + "architecture": 0.7098603945740938, + "semantic": 0.8646389191257297, + "symbol": 1.0 + } + }, + { + "repo": "messagepack-csharp", + "language": "csharp", + "mode": "semble-bm25", + "chunks": 2179, + "tokens": 1607, + "ndcg5": 0.7719825028945294, + "ndcg10": 0.7829028747354376, + "p50_ms": 1.6800420125946403, + "p90_ms": 2.4900419753976166, + "p95_ms": 2.6051670225569983, + "p99_ms": 3.1067669932963318, + "index_ms": 357.34845796832815, + "by_category": { + "architecture": 0.5144622537786347, + "semantic": 0.7333564977326905, + "symbol": 1.0 + } + }, + { + "repo": "messagepack-csharp", + "language": "csharp", + "mode": "semble-semantic", + "chunks": 2179, + "tokens": 1345, + "ndcg5": 0.8432406238070287, + "ndcg10": 0.8524694086460072, + "p50_ms": 1.8133539997506887, + "p90_ms": 2.525713091017679, + "p95_ms": 2.96443336410448, + "p99_ms": 3.3738202578388146, + "index_ms": 357.34845796832815, + "by_category": { + "architecture": 0.46950052509920975, + "semantic": 0.9079317858359227, + "symbol": 1.0 + } + }, + { + "repo": "messagepack-csharp", + "language": "csharp", + "mode": "unranked-auto", + "chunks": 2179, + "tokens": 1512, + "ndcg5": 0.715759137440029, + "ndcg10": 0.748620996578629, + "p50_ms": 0.287979026325047, + "p90_ms": 0.3148371877614409, + "p95_ms": 0.3163041692459956, + "p99_ms": 0.32092724286485463, + "index_ms": 357.34845796832815, + "by_category": { + "architecture": 0.48937363154746544, + "semantic": 0.809246539071705, + "symbol": 0.818815221962482 + } + }, + { + "repo": "messagepack-csharp", + "language": "csharp", + "mode": "unranked-balanced", + "chunks": 2179, + "tokens": 1515, + "ndcg5": 0.6692253095363593, + "ndcg10": 0.7028089127310467, + "p50_ms": 0.2865000278688967, + "p90_ms": 0.31596311018802226, + "p95_ms": 0.31721249979455024, + "p99_ms": 0.3185425099218264, + "index_ms": 357.34845796832815, + "by_category": { + "architecture": 0.48937363154746544, + "semantic": 0.809246539071705, + "symbol": 0.6879235538265327 + } + }, + { + "repo": "messagepack-csharp", + "language": "csharp", + "mode": "unranked-bm25", + "chunks": 2179, + "tokens": 1497, + "ndcg5": 0.7192308756365361, + "ndcg10": 0.7192308756365361, + "p50_ms": 0.27718799537979066, + "p90_ms": 0.3140291664749384, + "p95_ms": 0.33141491294372827, + "p99_ms": 0.33518295793328434, + "index_ms": 357.34845796832815, + "by_category": { + "architecture": 0.37531645883547177, + "semantic": 0.7218332077949214, + "symbol": 0.9124075438906488 + } + }, + { + "repo": "messagepack-csharp", + "language": "csharp", + "mode": "unranked-semantic", + "chunks": 2179, + "tokens": 1423, + "ndcg5": 0.5160559601140204, + "ndcg10": 0.5625464573825504, + "p50_ms": 0.2805620024446398, + "p90_ms": 0.30490453355014324, + "p95_ms": 0.3168211376760155, + "p99_ms": 0.3286642220336944, + "index_ms": 357.34845796832815, + "by_category": { + "architecture": 0.43697025143484197, + "semantic": 0.75965690301813, + "symbol": 0.3808765735354953 + } + }, + { + "repo": "mini.nvim", + "language": "lua", + "mode": "semble-auto", + "chunks": 4381, + "tokens": 1771, + "ndcg5": 0.9815464876785729, + "ndcg10": 0.9815464876785729, + "p50_ms": 0.6537919980473816, + "p90_ms": 0.7268291665241123, + "p95_ms": 0.7739898574072868, + "p99_ms": 0.830197992036119, + "index_ms": 809.8628330044448, + "by_category": { + "architecture": 1.0, + "semantic": 0.9769331095982161 + } + }, + { + "repo": "mini.nvim", + "language": "lua", + "mode": "semble-balanced", + "chunks": 4381, + "tokens": 1771, + "ndcg5": 0.9815464876785729, + "ndcg10": 0.9815464876785729, + "p50_ms": 1.0507084953133017, + "p90_ms": 1.2092375254724175, + "p95_ms": 1.2616125080967322, + "p99_ms": 1.3070224999682978, + "index_ms": 809.8628330044448, + "by_category": { + "architecture": 1.0, + "semantic": 0.9769331095982161 + } + }, + { + "repo": "mini.nvim", + "language": "lua", + "mode": "semble-bm25", + "chunks": 4381, + "tokens": 1757, + "ndcg5": 0.9815464876785729, + "ndcg10": 0.9815464876785729, + "p50_ms": 0.9073960245586932, + "p90_ms": 1.1127788922749462, + "p95_ms": 1.236283639445901, + "p99_ms": 1.2741567008197308, + "index_ms": 809.8628330044448, + "by_category": { + "architecture": 1.0, + "semantic": 0.9769331095982161 + } + }, + { + "repo": "mini.nvim", + "language": "lua", + "mode": "semble-semantic", + "chunks": 4381, + "tokens": 1690, + "ndcg5": 0.9815464876785729, + "ndcg10": 0.9815464876785729, + "p50_ms": 0.9938334987964481, + "p90_ms": 1.1698201997205615, + "p95_ms": 1.2150017690146342, + "p99_ms": 1.282167567987926, + "index_ms": 809.8628330044448, + "by_category": { + "architecture": 1.0, + "semantic": 0.9769331095982161 + } + }, + { + "repo": "mini.nvim", + "language": "lua", + "mode": "unranked-auto", + "chunks": 4381, + "tokens": 1726, + "ndcg5": 0.9565464876785729, + "ndcg10": 0.9565464876785729, + "p50_ms": 0.701145501807332, + "p90_ms": 0.8325958217028528, + "p95_ms": 0.8369851129828021, + "p99_ms": 0.8734970208024605, + "index_ms": 809.8628330044448, + "by_category": { + "architecture": 1.0, + "semantic": 0.9456831095982161 + } + }, + { + "repo": "mini.nvim", + "language": "lua", + "mode": "unranked-balanced", + "chunks": 4381, + "tokens": 1726, + "ndcg5": 0.9565464876785729, + "ndcg10": 0.9565464876785729, + "p50_ms": 0.7675624801777303, + "p90_ms": 0.9353752830065787, + "p95_ms": 0.9428937250049785, + "p99_ms": 0.9773787442827596, + "index_ms": 809.8628330044448, + "by_category": { + "architecture": 1.0, + "semantic": 0.9456831095982161 + } + }, + { + "repo": "mini.nvim", + "language": "lua", + "mode": "unranked-bm25", + "chunks": 4381, + "tokens": 1716, + "ndcg5": 0.8411732909393883, + "ndcg10": 0.8562247907225874, + "p50_ms": 0.7524164975620806, + "p90_ms": 0.8702249848283827, + "p95_ms": 0.8785770187387244, + "p99_ms": 0.940548235666938, + "index_ms": 809.8628330044448, + "by_category": { + "architecture": 0.9077324383928644, + "semantic": 0.8433478788050182 + } + }, + { + "repo": "mini.nvim", + "language": "lua", + "mode": "unranked-semantic", + "chunks": 4381, + "tokens": 1697, + "ndcg5": 0.9315464876785728, + "ndcg10": 0.949356847033974, + "p50_ms": 0.7624789723195136, + "p90_ms": 0.8987207955215127, + "p95_ms": 0.9449288481846452, + "p99_ms": 0.9672857588157058, + "index_ms": 809.8628330044448, + "by_category": { + "architecture": 1.0, + "semantic": 0.9366960587924675 + } + }, + { + "repo": "model2vec", + "language": "python", + "mode": "semble-auto", + "chunks": 203, + "tokens": 1629, + "ndcg5": 0.7164400093840788, + "ndcg10": 0.7375795011044112, + "p50_ms": 0.49564550863578916, + "p90_ms": 2.007016603602097, + "p95_ms": 2.291755611076951, + "p99_ms": 2.4280174821615215, + "index_ms": 36.53137502260506, + "by_category": { + "architecture": 0.6905456754532798, + "semantic": 0.7286840540343809, + "symbol": 0.8769765845238192 + } + }, + { + "repo": "model2vec", + "language": "python", + "mode": "semble-balanced", + "chunks": 203, + "tokens": 1641, + "ndcg5": 0.7164400093840788, + "ndcg10": 0.7375795011044112, + "p50_ms": 0.4849580000154674, + "p90_ms": 1.984549697954208, + "p95_ms": 2.2820541926193982, + "p99_ms": 2.413978014374152, + "index_ms": 36.53137502260506, + "by_category": { + "architecture": 0.6905456754532798, + "semantic": 0.7286840540343809, + "symbol": 0.8769765845238192 + } + }, + { + "repo": "model2vec", + "language": "python", + "mode": "semble-bm25", + "chunks": 203, + "tokens": 1734, + "ndcg5": 0.6259493638758683, + "ndcg10": 0.6669792956564887, + "p50_ms": 0.39537501288577914, + "p90_ms": 1.6562208824325355, + "p95_ms": 1.7936902004294097, + "p99_ms": 1.8010044819675386, + "index_ms": 36.53137502260506, + "by_category": { + "architecture": 0.6355310663031651, + "semantic": 0.625993869543616, + "symbol": 0.8769765845238192 + } + }, + { + "repo": "model2vec", + "language": "python", + "mode": "semble-semantic", + "chunks": 203, + "tokens": 1456, + "ndcg5": 0.7536469126324504, + "ndcg10": 0.7754876563142669, + "p50_ms": 0.386063038604334, + "p90_ms": 1.614500320283696, + "p95_ms": 1.644677101285197, + "p99_ms": 1.8354690336855126, + "index_ms": 36.53137502260506, + "by_category": { + "architecture": 0.7852311714872304, + "semantic": 0.7513134925874725, + "symbol": 0.8333333333333334 + } + }, + { + "repo": "model2vec", + "language": "python", + "mode": "unranked-auto", + "chunks": 203, + "tokens": 1741, + "ndcg5": 0.6811161480013384, + "ndcg10": 0.7146504735358594, + "p50_ms": 0.2363119856454432, + "p90_ms": 0.2636701799929142, + "p95_ms": 0.2642351028043777, + "p99_ms": 0.2722470148000866, + "index_ms": 36.53137502260506, + "by_category": { + "architecture": 0.8334586364906239, + "semantic": 0.6196939508139907, + "symbol": 0.7539531690476383 + } + }, + { + "repo": "model2vec", + "language": "python", + "mode": "unranked-balanced", + "chunks": 203, + "tokens": 1743, + "ndcg5": 0.6745696603227656, + "ndcg10": 0.7081039858572866, + "p50_ms": 0.23518750094808638, + "p90_ms": 0.25733300717547536, + "p95_ms": 0.2613642835058272, + "p99_ms": 0.2713400893844664, + "index_ms": 36.53137502260506, + "by_category": { + "architecture": 0.8334586364906239, + "semantic": 0.6196939508139907, + "symbol": 0.7103099178571526 + } + }, + { + "repo": "model2vec", + "language": "python", + "mode": "unranked-bm25", + "chunks": 203, + "tokens": 1697, + "ndcg5": 0.5272370444436654, + "ndcg10": 0.5761169323198614, + "p50_ms": 0.23264551418833435, + "p90_ms": 0.2548420161474496, + "p95_ms": 0.25642537511885166, + "p99_ms": 0.25655224919319153, + "index_ms": 36.53137502260506, + "by_category": { + "architecture": 0.6260293103290382, + "semantic": 0.4878273966951044, + "symbol": 0.7539531690476383 + } + }, + { + "repo": "model2vec", + "language": "python", + "mode": "unranked-semantic", + "chunks": 203, + "tokens": 1571, + "ndcg5": 0.6270969095331637, + "ndcg10": 0.667837758234822, + "p50_ms": 0.2325630048289895, + "p90_ms": 0.2553585625719279, + "p95_ms": 0.2594142482848838, + "p99_ms": 0.26064923556987196, + "index_ms": 36.53137502260506, + "by_category": { + "architecture": 0.7530157999912123, + "semantic": 0.6198791757523414, + "symbol": 0.6289509357448472 + } + }, + { + "repo": "monolog", + "language": "php", + "mode": "semble-auto", + "chunks": 774, + "tokens": 1443, + "ndcg5": 0.860832844681721, + "ndcg10": 0.8801754850434484, + "p50_ms": 0.683707999996841, + "p90_ms": 2.3662707943003625, + "p95_ms": 2.461547637358308, + "p99_ms": 2.6244095340371127, + "index_ms": 98.36754103889689, + "by_category": { + "architecture": 0.6980725016207014, + "semantic": 0.9113147192765458, + "symbol": 1.0 + } + }, + { + "repo": "monolog", + "language": "php", + "mode": "semble-balanced", + "chunks": 774, + "tokens": 1430, + "ndcg5": 0.860832844681721, + "ndcg10": 0.8801754850434484, + "p50_ms": 0.6920415034983307, + "p90_ms": 2.407645888160914, + "p95_ms": 2.461217303061858, + "p99_ms": 2.554443470435217, + "index_ms": 98.36754103889689, + "by_category": { + "architecture": 0.6980725016207014, + "semantic": 0.9113147192765458, + "symbol": 1.0 + } + }, + { + "repo": "monolog", + "language": "php", + "mode": "semble-bm25", + "chunks": 774, + "tokens": 1448, + "ndcg5": 0.8914902043199942, + "ndcg10": 0.9017093241994184, + "p50_ms": 0.5145419854670763, + "p90_ms": 1.6141336236614734, + "p95_ms": 1.6271642176434398, + "p99_ms": 1.7442992469295857, + "index_ms": 98.36754103889689, + "by_category": { + "architecture": 0.7842078582445822, + "semantic": 0.9113147192765458, + "symbol": 1.0 + } + }, + { + "repo": "monolog", + "language": "php", + "mode": "semble-semantic", + "chunks": 774, + "tokens": 1361, + "ndcg5": 0.8021911875419427, + "ndcg10": 0.8223403442218291, + "p50_ms": 0.5147289775777608, + "p90_ms": 1.642111915862188, + "p95_ms": 1.7570812779013072, + "p99_ms": 2.2637162648607037, + "index_ms": 98.36754103889689, + "by_category": { + "architecture": 0.48059662671954645, + "semantic": 0.9043823750838851, + "symbol": 1.0 + } + }, + { + "repo": "monolog", + "language": "php", + "mode": "unranked-auto", + "chunks": 774, + "tokens": 1426, + "ndcg5": 0.6622671390839401, + "ndcg10": 0.7039358344288715, + "p50_ms": 0.25772899971343577, + "p90_ms": 0.2719548239838332, + "p95_ms": 0.27566907519940287, + "p99_ms": 0.2814002177910879, + "index_ms": 98.36754103889689, + "by_category": { + "architecture": 0.537168306424291, + "semantic": 0.7892875156455978, + "symbol": 0.7 + } + }, + { + "repo": "monolog", + "language": "php", + "mode": "unranked-balanced", + "chunks": 774, + "tokens": 1412, + "ndcg5": 0.6622671390839401, + "ndcg10": 0.7039358344288715, + "p50_ms": 0.25752096553333104, + "p90_ms": 0.2681580255739391, + "p95_ms": 0.27231840358581394, + "p99_ms": 0.27399727317970246, + "index_ms": 98.36754103889689, + "by_category": { + "architecture": 0.537168306424291, + "semantic": 0.7892875156455978, + "symbol": 0.7 + } + }, + { + "repo": "monolog", + "language": "php", + "mode": "unranked-bm25", + "chunks": 774, + "tokens": 1460, + "ndcg5": 0.7507040761194669, + "ndcg10": 0.7595660404586693, + "p50_ms": 0.2515210071578622, + "p90_ms": 0.26897083735093474, + "p95_ms": 0.2704455197090283, + "p99_ms": 0.279755502124317, + "index_ms": 98.36754103889689, + "by_category": { + "architecture": 0.6108909204244194, + "semantic": 0.8005936453479832, + "symbol": 0.8261859507142916 + } + }, + { + "repo": "monolog", + "language": "php", + "mode": "unranked-semantic", + "chunks": 774, + "tokens": 1365, + "ndcg5": 0.4568433103525645, + "ndcg10": 0.4823719413584334, + "p50_ms": 0.2557294792495668, + "p90_ms": 0.26772969285957515, + "p95_ms": 0.2716639544814825, + "p99_ms": 0.277332765981555, + "index_ms": 98.36754103889689, + "by_category": { + "architecture": 0.16163380807110106, + "semantic": 0.6577410279670249, + "symbol": 0.452371901428583 + } + }, + { + "repo": "newtonsoft-json", + "language": "csharp", + "mode": "semble-auto", + "chunks": 4297, + "tokens": 1433, + "ndcg5": 0.8590790148145551, + "ndcg10": 0.8915189253205081, + "p50_ms": 4.806187527719885, + "p90_ms": 6.573358748573811, + "p95_ms": 7.043000665726141, + "p99_ms": 7.965133736142887, + "index_ms": 603.6213340121321, + "by_category": { + "architecture": 0.9251084237866075, + "semantic": 0.8436116330627744, + "symbol": 1.0 + } + }, + { + "repo": "newtonsoft-json", + "language": "csharp", + "mode": "semble-balanced", + "chunks": 4297, + "tokens": 1439, + "ndcg5": 0.8590790148145551, + "ndcg10": 0.8915189253205081, + "p50_ms": 4.775812005391344, + "p90_ms": 6.581988092511892, + "p95_ms": 6.883558275876568, + "p99_ms": 7.804044460644944, + "index_ms": 603.6213340121321, + "by_category": { + "architecture": 0.9251084237866075, + "semantic": 0.8436116330627744, + "symbol": 1.0 + } + }, + { + "repo": "newtonsoft-json", + "language": "csharp", + "mode": "semble-bm25", + "chunks": 4297, + "tokens": 1511, + "ndcg5": 0.8340790148145552, + "ndcg10": 0.8696997335253573, + "p50_ms": 3.494395496090874, + "p90_ms": 5.479437159374357, + "p95_ms": 6.166350012063049, + "p99_ms": 6.504169974359683, + "index_ms": 603.6213340121321, + "by_category": { + "architecture": 0.9251084237866075, + "semantic": 0.8039403752534094, + "symbol": 1.0 + } + }, + { + "repo": "newtonsoft-json", + "language": "csharp", + "mode": "semble-semantic", + "chunks": 4297, + "tokens": 1391, + "ndcg5": 0.8565464876785729, + "ndcg10": 0.874356847033974, + "p50_ms": 4.185228986898437, + "p90_ms": 5.625662184320392, + "p95_ms": 6.665102043189109, + "p99_ms": 7.54255323205143, + "index_ms": 603.6213340121321, + "by_category": { + "architecture": 0.9166666666666666, + "semantic": 0.81701244915268, + "symbol": 1.0 + } + }, + { + "repo": "newtonsoft-json", + "language": "csharp", + "mode": "unranked-auto", + "chunks": 4297, + "tokens": 1532, + "ndcg5": 0.7078428691643481, + "ndcg10": 0.7402827796703013, + "p50_ms": 0.32781201298348606, + "p90_ms": 0.36056655226275325, + "p95_ms": 0.36608064838219434, + "p99_ms": 0.3723833436379209, + "index_ms": 603.6213340121321, + "by_category": { + "architecture": 0.745401359302383, + "semantic": 0.7223995118506533, + "symbol": 0.7956176024115139 + } + }, + { + "repo": "newtonsoft-json", + "language": "csharp", + "mode": "unranked-balanced", + "chunks": 4297, + "tokens": 1547, + "ndcg5": 0.670046716481194, + "ndcg10": 0.7202969863425482, + "p50_ms": 0.3243330284021795, + "p90_ms": 0.35315825371071696, + "p95_ms": 0.35694133839569986, + "p99_ms": 0.35858831251971424, + "index_ms": 603.6213340121321, + "by_category": { + "architecture": 0.745401359302383, + "semantic": 0.7223995118506533, + "symbol": 0.6623789802264932 + } + }, + { + "repo": "newtonsoft-json", + "language": "csharp", + "mode": "unranked-bm25", + "chunks": 4297, + "tokens": 1521, + "ndcg5": 0.6274823322000669, + "ndcg10": 0.6695226120235361, + "p50_ms": 0.3172919969074428, + "p90_ms": 0.3504128137137741, + "p95_ms": 0.3538625140208751, + "p99_ms": 0.35557248280383646, + "index_ms": 603.6213340121321, + "by_category": { + "architecture": 0.6226339121022391, + "semantic": 0.6504226152597534, + "symbol": 0.8333333333333334 + } + }, + { + "repo": "newtonsoft-json", + "language": "csharp", + "mode": "unranked-semantic", + "chunks": 4297, + "tokens": 1507, + "ndcg5": 0.41359624688335794, + "ndcg10": 0.41359624688335794, + "p50_ms": 0.32031250884756446, + "p90_ms": 0.3487993613816798, + "p95_ms": 0.3533833398250863, + "p99_ms": 0.35350946185644716, + "index_ms": 603.6213340121321, + "by_category": { + "architecture": 0.4153804782863739, + "semantic": 0.43289631883117413, + "symbol": 0.3392608536019997 + } + }, + { + "repo": "nlohmann-json", + "language": "cpp", + "mode": "semble-auto", + "chunks": 1599, + "tokens": 1696, + "ndcg5": 0.8481244099790952, + "ndcg10": 0.8481244099790952, + "p50_ms": 0.5933954671490937, + "p90_ms": 2.6995211082976316, + "p95_ms": 3.074639235273935, + "p99_ms": 3.1362942402483895, + "index_ms": 229.59254099987447, + "by_category": { + "architecture": 0.7769670264708678, + "semantic": 0.8523719014285831, + "symbol": 1.0 + } + }, + { + "repo": "nlohmann-json", + "language": "cpp", + "mode": "semble-balanced", + "chunks": 1599, + "tokens": 1701, + "ndcg5": 0.8481244099790952, + "ndcg10": 0.8481244099790952, + "p50_ms": 0.597562495386228, + "p90_ms": 2.677628485253081, + "p95_ms": 3.007047262508422, + "p99_ms": 3.094542243052274, + "index_ms": 229.59254099987447, + "by_category": { + "architecture": 0.7769670264708678, + "semantic": 0.8523719014285831, + "symbol": 1.0 + } + }, + { + "repo": "nlohmann-json", + "language": "cpp", + "mode": "semble-bm25", + "chunks": 1599, + "tokens": 1646, + "ndcg5": 0.8488813031188188, + "ndcg10": 0.8591004229982431, + "p50_ms": 0.46766651212237775, + "p90_ms": 1.856571092503146, + "p95_ms": 2.23421010596212, + "p99_ms": 2.258942012558691, + "index_ms": 229.59254099987447, + "by_category": { + "architecture": 0.8369346630252996, + "semantic": 0.8323465818787765, + "symbol": 1.0 + } + }, + { + "repo": "nlohmann-json", + "language": "cpp", + "mode": "semble-semantic", + "chunks": 1599, + "tokens": 1600, + "ndcg5": 0.8039393393932415, + "ndcg10": 0.8326700705895507, + "p50_ms": 0.4259164852555841, + "p90_ms": 1.740738103399054, + "p95_ms": 2.389083334128373, + "p99_ms": 2.834950284450314, + "index_ms": 229.59254099987447, + "by_category": { + "architecture": 0.7567420320975702, + "semantic": 0.8356207187108022, + "symbol": 1.0 + } + }, + { + "repo": "nlohmann-json", + "language": "cpp", + "mode": "unranked-auto", + "chunks": 1599, + "tokens": 1728, + "ndcg5": 0.6537874476332355, + "ndcg10": 0.7218211776693584, + "p50_ms": 0.27568748919293284, + "p90_ms": 0.31279611866921186, + "p95_ms": 0.3156851162202656, + "p99_ms": 0.324837036896497, + "index_ms": 229.59254099987447, + "by_category": { + "architecture": 0.7134765628872772, + "semantic": 0.7011411055102833, + "symbol": 0.8102255193577976 + } + }, + { + "repo": "nlohmann-json", + "language": "cpp", + "mode": "unranked-balanced", + "chunks": 1599, + "tokens": 1727, + "ndcg5": 0.6515962600912928, + "ndcg10": 0.7196299901274157, + "p50_ms": 0.27520902222022414, + "p90_ms": 0.30020028352737427, + "p95_ms": 0.30461041314993054, + "p99_ms": 0.3157884703250602, + "index_ms": 229.59254099987447, + "by_category": { + "architecture": 0.7134765628872772, + "semantic": 0.7011411055102833, + "symbol": 0.7956176024115139 + } + }, + { + "repo": "nlohmann-json", + "language": "cpp", + "mode": "unranked-bm25", + "chunks": 1599, + "tokens": 1685, + "ndcg5": 0.747648213400138, + "ndcg10": 0.7687877051204705, + "p50_ms": 0.2685205254238099, + "p90_ms": 0.2936868986580521, + "p95_ms": 0.29749374371021986, + "p99_ms": 0.30119874980300665, + "index_ms": 229.59254099987447, + "by_category": { + "architecture": 0.7706324288023386, + "semantic": 0.6981327100793038, + "symbol": 1.0 + } + }, + { + "repo": "nlohmann-json", + "language": "cpp", + "mode": "unranked-semantic", + "chunks": 1599, + "tokens": 1701, + "ndcg5": 0.5594448072715084, + "ndcg10": 0.5965497136970236, + "p50_ms": 0.2689170360099524, + "p90_ms": 0.29460471705533564, + "p95_ms": 0.3001410368597135, + "p99_ms": 0.31306179880630225, + "index_ms": 229.59254099987447, + "by_category": { + "architecture": 0.63184391945669, + "semantic": 0.5508086837743642, + "symbol": 0.6666666666666666 + } + }, + { + "repo": "nvm", + "language": "bash", + "mode": "semble-auto", + "chunks": 309, + "tokens": 1620, + "ndcg5": 1.0, + "ndcg10": 1.0, + "p50_ms": 0.4722914891317487, + "p90_ms": 0.5002838850487024, + "p95_ms": 0.5040600663051009, + "p99_ms": 0.5170120159164071, + "index_ms": 56.07916601002216, + "by_category": { + "architecture": 1.0, + "semantic": 1.0 + } + }, + { + "repo": "nvm", + "language": "bash", + "mode": "semble-balanced", + "chunks": 309, + "tokens": 1620, + "ndcg5": 1.0, + "ndcg10": 1.0, + "p50_ms": 0.4690205096267164, + "p90_ms": 0.5042826989665627, + "p95_ms": 0.5084746691863984, + "p99_ms": 0.5220285186078399, + "index_ms": 56.07916601002216, + "by_category": { + "architecture": 1.0, + "semantic": 1.0 + } + }, + { + "repo": "nvm", + "language": "bash", + "mode": "semble-bm25", + "chunks": 309, + "tokens": 1648, + "ndcg5": 1.0, + "ndcg10": 1.0, + "p50_ms": 0.38595852674916387, + "p90_ms": 0.3954582614824176, + "p95_ms": 0.39630592218600214, + "p99_ms": 0.3977939940523356, + "index_ms": 56.07916601002216, + "by_category": { + "architecture": 1.0, + "semantic": 1.0 + } + }, + { + "repo": "nvm", + "language": "bash", + "mode": "semble-semantic", + "chunks": 309, + "tokens": 1500, + "ndcg5": 1.0, + "ndcg10": 1.0, + "p50_ms": 0.3744375135283917, + "p90_ms": 0.39036691887304187, + "p95_ms": 0.393503496889025, + "p99_ms": 0.39673427352681756, + "index_ms": 56.07916601002216, + "by_category": { + "architecture": 1.0, + "semantic": 1.0 + } + }, + { + "repo": "nvm", + "language": "bash", + "mode": "unranked-auto", + "chunks": 309, + "tokens": 1619, + "ndcg5": 0.9630929753571458, + "ndcg10": 0.9630929753571458, + "p50_ms": 0.27316651539877057, + "p90_ms": 0.2779964590445161, + "p95_ms": 0.2795027452521026, + "p99_ms": 0.28016772819682956, + "index_ms": 56.07916601002216, + "by_category": { + "architecture": 0.8154648767857288, + "semantic": 0.9794960974206366 + } + }, + { + "repo": "nvm", + "language": "bash", + "mode": "unranked-balanced", + "chunks": 309, + "tokens": 1619, + "ndcg5": 0.9630929753571458, + "ndcg10": 0.9630929753571458, + "p50_ms": 0.26720852474682033, + "p90_ms": 0.27709155692718923, + "p95_ms": 0.27798057126346976, + "p99_ms": 0.2789625182049349, + "index_ms": 56.07916601002216, + "by_category": { + "architecture": 0.8154648767857288, + "semantic": 0.9794960974206366 + } + }, + { + "repo": "nvm", + "language": "bash", + "mode": "unranked-bm25", + "chunks": 309, + "tokens": 1703, + "ndcg5": 0.9161732909393884, + "ndcg10": 0.9161732909393884, + "p50_ms": 0.2658125013113022, + "p90_ms": 0.2739829651545733, + "p95_ms": 0.27433091890998185, + "p99_ms": 0.2761997899506241, + "index_ms": 56.07916601002216, + "by_category": { + "architecture": 0.8154648767857288, + "semantic": 0.9273631147342394 + } + }, + { + "repo": "nvm", + "language": "bash", + "mode": "unranked-semantic", + "chunks": 309, + "tokens": 1544, + "ndcg5": 0.9380929753571458, + "ndcg10": 0.9380929753571458, + "p50_ms": 0.2652500115800649, + "p90_ms": 0.27499201823957264, + "p95_ms": 0.27916699764318764, + "p99_ms": 0.28106701211072505, + "index_ms": 56.07916601002216, + "by_category": { + "architecture": 0.8154648767857288, + "semantic": 0.9517183196428588 + } + }, + { + "repo": "pandoc", + "language": "haskell", + "mode": "semble-auto", + "chunks": 6178, + "tokens": 1393, + "ndcg5": 0.6773719014285831, + "ndcg10": 0.7090900678784489, + "p50_ms": 0.8944795117713511, + "p90_ms": 7.555441727163272, + "p95_ms": 8.78980656270869, + "p99_ms": 8.852727698395029, + "index_ms": 1005.650290986523, + "by_category": { + "architecture": 0.6696315434523955, + "semantic": 0.735395750829151 + } + }, + { + "repo": "pandoc", + "language": "haskell", + "mode": "semble-balanced", + "chunks": 6178, + "tokens": 1393, + "ndcg5": 0.6773719014285831, + "ndcg10": 0.7090900678784489, + "p50_ms": 0.8437500218860805, + "p90_ms": 7.3049621714744735, + "p95_ms": 8.564097911585122, + "p99_ms": 8.738486769143492, + "index_ms": 1005.650290986523, + "by_category": { + "architecture": 0.6696315434523955, + "semantic": 0.735395750829151 + } + }, + { + "repo": "pandoc", + "language": "haskell", + "mode": "semble-bm25", + "chunks": 6178, + "tokens": 1534, + "ndcg5": 0.6954395572359224, + "ndcg10": 0.6954395572359224, + "p50_ms": 0.5651460087392479, + "p90_ms": 6.19260797975585, + "p95_ms": 7.54140802891925, + "p99_ms": 7.656548817176372, + "index_ms": 1005.650290986523, + "by_category": { + "architecture": 0.6654332273484708, + "semantic": 0.71544377716089 + } + }, + { + "repo": "pandoc", + "language": "haskell", + "mode": "semble-semantic", + "chunks": 6178, + "tokens": 1350, + "ndcg5": 0.5942536065216308, + "ndcg10": 0.5942536065216308, + "p50_ms": 0.5863124679308385, + "p90_ms": 7.097775314468892, + "p95_ms": 8.043491683201864, + "p99_ms": 8.056664733448997, + "index_ms": 1005.650290986523, + "by_category": { + "architecture": 0.6029332273484708, + "semantic": 0.5884671926370709 + } + }, + { + "repo": "pandoc", + "language": "haskell", + "mode": "unranked-auto", + "chunks": 6178, + "tokens": 1469, + "ndcg5": 0.5885962468833579, + "ndcg10": 0.5885962468833579, + "p50_ms": 0.35389550612308085, + "p90_ms": 0.388916878728196, + "p95_ms": 0.40072642150335014, + "p99_ms": 0.409245298942551, + "index_ms": 1005.650290986523, + "by_category": { + "architecture": 0.6029332273484708, + "semantic": 0.5790382599066161 + } + }, + { + "repo": "pandoc", + "language": "haskell", + "mode": "unranked-balanced", + "chunks": 6178, + "tokens": 1469, + "ndcg5": 0.5885962468833579, + "ndcg10": 0.5885962468833579, + "p50_ms": 0.3552499692887068, + "p90_ms": 0.38047918933443725, + "p95_ms": 0.383454529219307, + "p99_ms": 0.39732454635668546, + "index_ms": 1005.650290986523, + "by_category": { + "architecture": 0.6029332273484708, + "semantic": 0.5790382599066161 + } + }, + { + "repo": "pandoc", + "language": "haskell", + "mode": "unranked-bm25", + "chunks": 6178, + "tokens": 1516, + "ndcg5": 0.5076420645710934, + "ndcg10": 0.538466808193579, + "p50_ms": 0.3413124941289425, + "p90_ms": 0.37877526483498514, + "p95_ms": 0.3936062741558999, + "p99_ms": 0.39902124903164804, + "index_ms": 1005.650290986523, + "by_category": { + "architecture": 0.4830875994788797, + "semantic": 0.575386280670045 + } + }, + { + "repo": "pandoc", + "language": "haskell", + "mode": "unranked-semantic", + "chunks": 6178, + "tokens": 1364, + "ndcg5": 0.37461414348591215, + "ndcg10": 0.41023486219671434, + "p50_ms": 0.3353954525664449, + "p90_ms": 0.38741668104194105, + "p95_ms": 0.3920066053979099, + "p99_ms": 0.40476850932464004, + "index_ms": 1005.650290986523, + "by_category": { + "architecture": 0.5358604681476768, + "semantic": 0.326484458229406 + } + }, + { + "repo": "phoenix", + "language": "elixir", + "mode": "semble-auto", + "chunks": 6865, + "tokens": 5361, + "ndcg5": 0.8854455694098825, + "ndcg10": 0.8854455694098825, + "p50_ms": 0.6355830118991435, + "p90_ms": 4.351741378195583, + "p95_ms": 5.740849993890151, + "p99_ms": 5.776670015184209, + "index_ms": 1025.31012496911, + "by_category": { + "architecture": 0.8624288341269906, + "semantic": 0.8659437588064073, + "symbol": 1.0 + } + }, + { + "repo": "phoenix", + "language": "elixir", + "mode": "semble-balanced", + "chunks": 6865, + "tokens": 5366, + "ndcg5": 0.8854455694098825, + "ndcg10": 0.8854455694098825, + "p50_ms": 0.6312080076895654, + "p90_ms": 4.333266569301485, + "p95_ms": 5.603703903034329, + "p99_ms": 5.717974388971925, + "index_ms": 1025.31012496911, + "by_category": { + "architecture": 0.8624288341269906, + "semantic": 0.8659437588064073, + "symbol": 1.0 + } + }, + { + "repo": "phoenix", + "language": "elixir", + "mode": "semble-bm25", + "chunks": 6865, + "tokens": 4542, + "ndcg5": 0.8231092011832086, + "ndcg10": 0.8606046945630005, + "p50_ms": 0.5173749523237348, + "p90_ms": 3.642349608708172, + "p95_ms": 4.7338119125925, + "p99_ms": 4.905862377490848, + "index_ms": 1025.31012496911, + "by_category": { + "architecture": 0.8182613617797254, + "semantic": 0.8553052772399258, + "symbol": 1.0 + } + }, + { + "repo": "phoenix", + "language": "elixir", + "mode": "semble-semantic", + "chunks": 6865, + "tokens": 3622, + "ndcg5": 0.8743293081244472, + "ndcg10": 0.8743293081244472, + "p50_ms": 0.5045419675298035, + "p90_ms": 4.006766411475836, + "p95_ms": 4.965195793192832, + "p99_ms": 5.495805551763624, + "index_ms": 1025.31012496911, + "by_category": { + "architecture": 0.8401784790716501, + "semantic": 0.8643786489599493, + "symbol": 1.0 + } + }, + { + "repo": "phoenix", + "language": "elixir", + "mode": "unranked-auto", + "chunks": 6865, + "tokens": 4859, + "ndcg5": 0.5987768978694179, + "ndcg10": 0.6282816128534446, + "p50_ms": 0.3017919952981174, + "p90_ms": 0.35390000557526946, + "p95_ms": 0.3658459463622421, + "p99_ms": 0.3683364193420857, + "index_ms": 1025.31012496911, + "by_category": { + "architecture": 0.6060381256302664, + "semantic": 0.7298596808041466, + "symbol": 0.4579965826380071 + } + }, + { + "repo": "phoenix", + "language": "elixir", + "mode": "unranked-balanced", + "chunks": 6865, + "tokens": 4857, + "ndcg5": 0.5882372560010987, + "ndcg10": 0.6177419709851254, + "p50_ms": 0.3074160194955766, + "p90_ms": 0.3602755721658468, + "p95_ms": 0.3625335870310664, + "p99_ms": 0.36967312451452017, + "index_ms": 1025.31012496911, + "by_category": { + "architecture": 0.6060381256302664, + "semantic": 0.7298596808041466, + "symbol": 0.3912455174719856 + } + }, + { + "repo": "phoenix", + "language": "elixir", + "mode": "unranked-bm25", + "chunks": 6865, + "tokens": 4857, + "ndcg5": 0.6006726545763182, + "ndcg10": 0.655712007605233, + "p50_ms": 0.29820797499269247, + "p90_ms": 0.35342498449608684, + "p95_ms": 0.35519998637028033, + "p99_ms": 0.3665399888996035, + "index_ms": 1025.31012496911, + "by_category": { + "architecture": 0.6424278537087322, + "semantic": 0.7124481343827828, + "symbol": 0.5631801734804518 + } + }, + { + "repo": "phoenix", + "language": "elixir", + "mode": "unranked-semantic", + "chunks": 6865, + "tokens": 4300, + "ndcg5": 0.5702795750586281, + "ndcg10": 0.6332802478784917, + "p50_ms": 0.2947499742731452, + "p90_ms": 0.35214999224990606, + "p95_ms": 0.35276245325803757, + "p99_ms": 0.3528524748980999, + "index_ms": 1025.31012496911, + "by_category": { + "architecture": 0.6153017228750899, + "semantic": 0.7598130814661574, + "symbol": 0.3919725445174765 + } + }, + { + "repo": "plug", + "language": "elixir", + "mode": "semble-auto", + "chunks": 507, + "tokens": 4239, + "ndcg5": 0.9261859507142916, + "ndcg10": 0.9261859507142916, + "p50_ms": 0.4831039987038821, + "p90_ms": 2.620854234555737, + "p95_ms": 3.2976274087559436, + "p99_ms": 3.960125515004619, + "index_ms": 118.93754097400233, + "by_category": { + "architecture": 1.0, + "semantic": 0.8945513581632737, + "symbol": 1.0 + } + }, + { + "repo": "plug", + "language": "elixir", + "mode": "semble-balanced", + "chunks": 507, + "tokens": 4204, + "ndcg5": 0.9261859507142916, + "ndcg10": 0.9261859507142916, + "p50_ms": 0.47816650476306677, + "p90_ms": 2.572649420471863, + "p95_ms": 3.287602111231536, + "p99_ms": 3.914254044648259, + "index_ms": 118.93754097400233, + "by_category": { + "architecture": 1.0, + "semantic": 0.8945513581632737, + "symbol": 1.0 + } + }, + { + "repo": "plug", + "language": "elixir", + "mode": "semble-bm25", + "chunks": 507, + "tokens": 4486, + "ndcg5": 0.9261859507142916, + "ndcg10": 0.9261859507142916, + "p50_ms": 0.402812467655167, + "p90_ms": 1.5990298241376886, + "p95_ms": 2.2072961408412097, + "p99_ms": 3.5259592224610947, + "index_ms": 118.93754097400233, + "by_category": { + "architecture": 1.0, + "semantic": 0.8945513581632737, + "symbol": 1.0 + } + }, + { + "repo": "plug", + "language": "elixir", + "mode": "semble-semantic", + "chunks": 507, + "tokens": 3105, + "ndcg5": 0.8977197786179613, + "ndcg10": 0.8977197786179613, + "p50_ms": 0.3916664863936603, + "p90_ms": 2.0050878112670047, + "p95_ms": 2.813597911153921, + "p99_ms": 3.6567867739358904, + "index_ms": 118.93754097400233, + "by_category": { + "architecture": 1.0, + "semantic": 0.8538853980256589, + "symbol": 1.0 + } + }, + { + "repo": "plug", + "language": "elixir", + "mode": "unranked-auto", + "chunks": 507, + "tokens": 3026, + "ndcg5": 0.7880929753571457, + "ndcg10": 0.8038662191964322, + "p50_ms": 0.2501244889572263, + "p90_ms": 0.26470902375876904, + "p95_ms": 0.26837144687306136, + "p99_ms": 0.2727406838675961, + "index_ms": 118.93754097400233, + "by_category": { + "architecture": 0.8333333333333334, + "semantic": 0.781885330739799, + "symbol": 0.8769765845238192 + } + }, + { + "repo": "plug", + "language": "elixir", + "mode": "unranked-balanced", + "chunks": 507, + "tokens": 3117, + "ndcg5": 0.7815464876785729, + "ndcg10": 0.7973197315178593, + "p50_ms": 0.24181249318644404, + "p90_ms": 0.2595830010250211, + "p95_ms": 0.2605205023428425, + "p99_ms": 0.27477052237372845, + "index_ms": 118.93754097400233, + "by_category": { + "architecture": 0.8333333333333334, + "semantic": 0.781885330739799, + "symbol": 0.8333333333333334 + } + }, + { + "repo": "plug", + "language": "elixir", + "mode": "unranked-bm25", + "chunks": 507, + "tokens": 3317, + "ndcg5": 0.8315464876785729, + "ndcg10": 0.8473197315178593, + "p50_ms": 0.23960403632372618, + "p90_ms": 0.2569410076830536, + "p95_ms": 0.2588222676422447, + "p99_ms": 0.2668980590533465, + "index_ms": 118.93754097400233, + "by_category": { + "architecture": 0.8333333333333334, + "semantic": 0.8533139021683704, + "symbol": 0.8333333333333334 + } + }, + { + "repo": "plug", + "language": "elixir", + "mode": "unranked-semantic", + "chunks": 507, + "tokens": 2620, + "ndcg5": 0.7496141434859123, + "ndcg10": 0.78409116950798, + "p50_ms": 0.239791494095698, + "p90_ms": 0.25448299711570144, + "p95_ms": 0.2549664000980556, + "p99_ms": 0.2588933031074703, + "index_ms": 118.93754097400233, + "by_category": { + "architecture": 0.8102255193577976, + "semantic": 0.7585869341796249, + "symbol": 0.8769765845238192 + } + }, + { + "repo": "pydantic", + "language": "python", + "mode": "semble-auto", + "chunks": 2868, + "tokens": 1849, + "ndcg5": 0.7530776934979013, + "ndcg10": 0.78378040151892, + "p50_ms": 1.2483334867283702, + "p90_ms": 6.651749997399747, + "p95_ms": 7.105504150968048, + "p99_ms": 10.120867190416897, + "index_ms": 433.8992079719901, + "by_category": { + "architecture": 0.6492397538295632, + "semantic": 0.7663662191964322, + "symbol": 1.0 + } + }, + { + "repo": "pydantic", + "language": "python", + "mode": "semble-balanced", + "chunks": 2868, + "tokens": 1871, + "ndcg5": 0.7530776934979013, + "ndcg10": 0.78378040151892, + "p50_ms": 1.2302710092626512, + "p90_ms": 6.029454187955709, + "p95_ms": 7.460329501191157, + "p99_ms": 10.405899501638482, + "index_ms": 433.8992079719901, + "by_category": { + "architecture": 0.6492397538295632, + "semantic": 0.7663662191964322, + "symbol": 1.0 + } + }, + { + "repo": "pydantic", + "language": "python", + "mode": "semble-bm25", + "chunks": 2868, + "tokens": 1833, + "ndcg5": 0.7636381417190647, + "ndcg10": 0.7854788854008812, + "p50_ms": 1.0941250075120479, + "p90_ms": 4.193357680924239, + "p95_ms": 5.971155880251903, + "p99_ms": 9.98506398056633, + "index_ms": 433.8992079719901, + "by_category": { + "architecture": 0.6791845424653482, + "semantic": 0.7444107388450233, + "symbol": 1.0 + } + }, + { + "repo": "pydantic", + "language": "python", + "mode": "semble-semantic", + "chunks": 2868, + "tokens": 1708, + "ndcg5": 0.7226294385530917, + "ndcg10": 0.7522964631509186, + "p50_ms": 0.9076254791580141, + "p90_ms": 3.8237961009144805, + "p95_ms": 5.287233021226714, + "p99_ms": 8.002713039168151, + "index_ms": 433.8992079719901, + "by_category": { + "architecture": 0.5894583534100267, + "semantic": 0.7399650986435234, + "symbol": 1.0 + } + }, + { + "repo": "pydantic", + "language": "python", + "mode": "unranked-auto", + "chunks": 2868, + "tokens": 1959, + "ndcg5": 0.5504705452125698, + "ndcg10": 0.5850729432083507, + "p50_ms": 0.2931254857685417, + "p90_ms": 0.32207888434641063, + "p95_ms": 0.3255378775065765, + "p99_ms": 0.3273747646017, + "index_ms": 433.8992079719901, + "by_category": { + "architecture": 0.4625275678750919, + "semantic": 0.6204707361301713, + "symbol": 0.7 + } + }, + { + "repo": "pydantic", + "language": "python", + "mode": "unranked-balanced", + "chunks": 2868, + "tokens": 1949, + "ndcg5": 0.5263596732528697, + "ndcg10": 0.5609620712486507, + "p50_ms": 0.29075000202283263, + "p90_ms": 0.3107201831880957, + "p95_ms": 0.31385797483380884, + "p99_ms": 0.31993797689210624, + "index_ms": 433.8992079719901, + "by_category": { + "architecture": 0.4625275678750919, + "semantic": 0.6204707361301713, + "symbol": 0.6035565121611999 + } + }, + { + "repo": "pydantic", + "language": "python", + "mode": "unranked-bm25", + "chunks": 2868, + "tokens": 1985, + "ndcg5": 0.5192981234416789, + "ndcg10": 0.5782437735490106, + "p50_ms": 0.2851664903573692, + "p90_ms": 0.32175891101360327, + "p95_ms": 0.3460455016465859, + "p99_ms": 0.4347754694754256, + "index_ms": 433.8992079719901, + "by_category": { + "architecture": 0.47637481069172644, + "semantic": 0.5749152553208338, + "symbol": 0.7261859507142916 + } + }, + { + "repo": "pydantic", + "language": "python", + "mode": "unranked-semantic", + "chunks": 2868, + "tokens": 1732, + "ndcg5": 0.39470537688715435, + "ndcg10": 0.4565307377996012, + "p50_ms": 0.2808120334520936, + "p90_ms": 0.3040003008209169, + "p95_ms": 0.3150541306240484, + "p99_ms": 0.31777718977537006, + "index_ms": 433.8992079719901, + "by_category": { + "architecture": 0.4662954460817987, + "semantic": 0.4487931083027581, + "symbol": 0.45524035339947344 + } + }, + { + "repo": "rack", + "language": "ruby", + "mode": "semble-auto", + "chunks": 483, + "tokens": 1435, + "ndcg5": 0.8902066119047684, + "ndcg10": 0.8902066119047684, + "p50_ms": 0.5274995055515319, + "p90_ms": 2.776945504592732, + "p95_ms": 3.509901682264169, + "p99_ms": 4.729113166104068, + "index_ms": 66.37025001691654, + "by_category": { + "architecture": 1.0, + "semantic": 0.8023719014285831, + "symbol": 1.0 + } + }, + { + "repo": "rack", + "language": "ruby", + "mode": "semble-balanced", + "chunks": 483, + "tokens": 1435, + "ndcg5": 0.8902066119047684, + "ndcg10": 0.8902066119047684, + "p50_ms": 0.5105205054860562, + "p90_ms": 2.7846167155075823, + "p95_ms": 3.51546609890647, + "p99_ms": 4.658093206817282, + "index_ms": 66.37025001691654, + "by_category": { + "architecture": 1.0, + "semantic": 0.8023719014285831, + "symbol": 1.0 + } + }, + { + "repo": "rack", + "language": "ruby", + "mode": "semble-bm25", + "chunks": 483, + "tokens": 1481, + "ndcg5": 0.8551549589285763, + "ndcg10": 0.8551549589285763, + "p50_ms": 0.4137505020480603, + "p90_ms": 2.3690998030360793, + "p95_ms": 2.874772259383461, + "p99_ms": 3.5172872635303047, + "index_ms": 66.37025001691654, + "by_category": { + "architecture": 1.0, + "semantic": 0.7392789260714372, + "symbol": 1.0 + } + }, + { + "repo": "rack", + "language": "ruby", + "mode": "semble-semantic", + "chunks": 483, + "tokens": 1378, + "ndcg5": 0.9517183196428588, + "ndcg10": 0.9517183196428588, + "p50_ms": 0.40160349453799427, + "p90_ms": 2.5162454927340154, + "p95_ms": 2.911326836328952, + "p99_ms": 3.626998958643524, + "index_ms": 66.37025001691654, + "by_category": { + "architecture": 1.0, + "semantic": 0.9130929753571457, + "symbol": 1.0 + } + }, + { + "repo": "rack", + "language": "ruby", + "mode": "unranked-auto", + "chunks": 483, + "tokens": 1532, + "ndcg5": 0.8440297814009061, + "ndcg10": 0.8440297814009061, + "p50_ms": 0.23654202232137322, + "p90_ms": 0.2696545037906617, + "p95_ms": 0.2738981624133885, + "p99_ms": 0.2800460741855204, + "index_ms": 66.37025001691654, + "by_category": { + "architecture": 1.0, + "semantic": 0.85, + "symbol": 0.7385072130432617 + } + }, + { + "repo": "rack", + "language": "ruby", + "mode": "unranked-balanced", + "chunks": 483, + "tokens": 1530, + "ndcg5": 0.8620990311558888, + "ndcg10": 0.8620990311558888, + "p50_ms": 0.2609375223983079, + "p90_ms": 0.3062669944483787, + "p95_ms": 0.36182935582473835, + "p99_ms": 0.5557986558414991, + "index_ms": 66.37025001691654, + "by_category": { + "architecture": 1.0, + "semantic": 0.85, + "symbol": 0.8035565121611998 + } + }, + { + "repo": "rack", + "language": "ruby", + "mode": "unranked-bm25", + "chunks": 483, + "tokens": 1551, + "ndcg5": 0.772479742668919, + "ndcg10": 0.7909982611874375, + "p50_ms": 0.26675048866309226, + "p90_ms": 0.39899560506455606, + "p95_ms": 0.5175999685889107, + "p99_ms": 0.7005199760897081, + "index_ms": 66.37025001691654, + "by_category": { + "architecture": 1.0, + "semantic": 0.7851115894139332, + "symbol": 0.6773705614469083 + } + }, + { + "repo": "rack", + "language": "ruby", + "mode": "unranked-semantic", + "chunks": 483, + "tokens": 1512, + "ndcg5": 0.7783369683624579, + "ndcg10": 0.7943961253801184, + "p50_ms": 0.24741649394854903, + "p90_ms": 0.2648957190103829, + "p95_ms": 0.2754106913926079, + "p99_ms": 0.31844856741372485, + "index_ms": 66.37025001691654, + "by_category": { + "architecture": 0.8333333333333334, + "semantic": 0.8350671137962739, + "symbol": 0.6896918237758785 + } + }, + { + "repo": "rails", + "language": "ruby", + "mode": "semble-auto", + "chunks": 870, + "tokens": 1403, + "ndcg5": 0.8273505311466753, + "ndcg10": 0.8540441468268698, + "p50_ms": 0.5834170151501894, + "p90_ms": 3.1330499798059477, + "p95_ms": 4.053427075268701, + "p99_ms": 4.253719035768881, + "index_ms": 152.69820898538455, + "by_category": { + "architecture": 0.845500808108813, + "semantic": 0.8494849471938838, + "symbol": 0.8812452583002233 + } + }, + { + "repo": "rails", + "language": "ruby", + "mode": "semble-balanced", + "chunks": 870, + "tokens": 1398, + "ndcg5": 0.8313644916892657, + "ndcg10": 0.8580581073694604, + "p50_ms": 0.5835415213368833, + "p90_ms": 3.083837183658035, + "p95_ms": 4.009674987173639, + "p99_ms": 4.213734992663376, + "index_ms": 152.69820898538455, + "by_category": { + "architecture": 0.845500808108813, + "semantic": 0.8494849471938838, + "symbol": 0.9013150610131764 + } + }, + { + "repo": "rails", + "language": "ruby", + "mode": "semble-bm25", + "chunks": 870, + "tokens": 1442, + "ndcg5": 0.8054753636489659, + "ndcg10": 0.8314677273676765, + "p50_ms": 0.44416700256988406, + "p90_ms": 2.43836640729569, + "p95_ms": 2.969883673358709, + "p99_ms": 3.263876738492399, + "index_ms": 152.69820898538455, + "by_category": { + "architecture": 0.8341462304650082, + "semantic": 0.7431882405743243, + "symbol": 0.9799301972870469 + } + }, + { + "repo": "rails", + "language": "ruby", + "mode": "semble-semantic", + "chunks": 870, + "tokens": 1335, + "ndcg5": 0.8213932929028847, + "ndcg10": 0.8488749724391494, + "p50_ms": 0.440500007243827, + "p90_ms": 2.686262468341739, + "p95_ms": 3.2619521196465944, + "p99_ms": 3.5975240485277022, + "index_ms": 152.69820898538455, + "by_category": { + "architecture": 0.853754323524531, + "semantic": 0.8553052772399258, + "symbol": 0.8266433990956823 + } + }, + { + "repo": "rails", + "language": "ruby", + "mode": "unranked-auto", + "chunks": 870, + "tokens": 1535, + "ndcg5": 0.5955918828431129, + "ndcg10": 0.6195053469655144, + "p50_ms": 0.25243801064789295, + "p90_ms": 0.25961632491089404, + "p95_ms": 0.2611743373563513, + "p99_ms": 0.2632012701360509, + "index_ms": 152.69820898538455, + "by_category": { + "architecture": 0.6549997640160055, + "semantic": 0.5642489014344912, + "symbol": 0.6363416882812003 + } + }, + { + "repo": "rails", + "language": "ruby", + "mode": "unranked-balanced", + "chunks": 870, + "tokens": 1533, + "ndcg5": 0.6021383705216858, + "ndcg10": 0.6268611904857484, + "p50_ms": 0.24964648764580488, + "p90_ms": 0.25569169083610177, + "p95_ms": 0.26980034017469734, + "p99_ms": 0.2699264622060582, + "index_ms": 152.69820898538455, + "by_category": { + "architecture": 0.6549997640160055, + "semantic": 0.5642489014344912, + "symbol": 0.6731209058823706 + } + }, + { + "repo": "rails", + "language": "ruby", + "mode": "unranked-bm25", + "chunks": 870, + "tokens": 1538, + "ndcg5": 0.5478302093894449, + "ndcg10": 0.5862322607666178, + "p50_ms": 0.24535402189940214, + "p90_ms": 0.2540165849495679, + "p95_ms": 0.25756186805665493, + "p99_ms": 0.25787878781557083, + "index_ms": 152.69820898538455, + "by_category": { + "architecture": 0.5459057908293498, + "semantic": 0.6248556782734316, + "symbol": 0.6093758374885467 + } + }, + { + "repo": "rails", + "language": "ruby", + "mode": "unranked-semantic", + "chunks": 870, + "tokens": 1475, + "ndcg5": 0.5237385461239759, + "ndcg10": 0.5685016189967601, + "p50_ms": 0.2447709848638624, + "p90_ms": 0.2552458143327385, + "p95_ms": 0.26665387267712504, + "p99_ms": 0.269630768452771, + "index_ms": 152.69820898538455, + "by_category": { + "architecture": 0.5630433321005218, + "semantic": 0.5508867410154318, + "symbol": 0.6116088009806209 + } + }, + { + "repo": "redis", + "language": "c", + "mode": "semble-auto", + "chunks": 12306, + "tokens": 1786, + "ndcg5": 0.9224229559439696, + "ndcg10": 0.9224229559439696, + "p50_ms": 0.7576454663649201, + "p90_ms": 0.8519128023181111, + "p95_ms": 0.8795832953182982, + "p99_ms": 0.9587494883453472, + "index_ms": 1945.187957957387, + "by_category": { + "architecture": 0.9430676558073394, + "semantic": 0.9017782560805999 + } + }, + { + "repo": "redis", + "language": "c", + "mode": "semble-balanced", + "chunks": 12306, + "tokens": 1786, + "ndcg5": 0.9224229559439696, + "ndcg10": 0.9224229559439696, + "p50_ms": 0.7364375051110983, + "p90_ms": 0.820512481732294, + "p95_ms": 0.8363562461454421, + "p99_ms": 0.8873712143395095, + "index_ms": 1945.187957957387, + "by_category": { + "architecture": 0.9430676558073394, + "semantic": 0.9017782560805999 + } + }, + { + "repo": "redis", + "language": "c", + "mode": "semble-bm25", + "chunks": 12306, + "tokens": 1675, + "ndcg5": 0.9161732909393884, + "ndcg10": 0.9161732909393884, + "p50_ms": 0.6348540482576936, + "p90_ms": 0.6698830344248563, + "p95_ms": 0.6783851364161819, + "p99_ms": 0.6981770496349782, + "index_ms": 1945.187957957387, + "by_category": { + "architecture": 0.9061606311644852, + "semantic": 0.9261859507142916 + } + }, + { + "repo": "redis", + "language": "c", + "mode": "semble-semantic", + "chunks": 12306, + "tokens": 1699, + "ndcg5": 0.8408764682653966, + "ndcg10": 0.8566497121046833, + "p50_ms": 0.5915829970035702, + "p90_ms": 0.6629250186961144, + "p95_ms": 0.6932125252205878, + "p99_ms": 0.7192425022367388, + "index_ms": 1945.187957957387, + "by_category": { + "architecture": 0.9386852807234541, + "semantic": 0.7746141434859122 + } + }, + { + "repo": "redis", + "language": "c", + "mode": "unranked-auto", + "chunks": 12306, + "tokens": 1802, + "ndcg5": 0.7820497592047851, + "ndcg10": 0.7820497592047851, + "p50_ms": 0.48849949962459505, + "p90_ms": 0.5246205779258162, + "p95_ms": 0.5346152116544545, + "p99_ms": 0.5445894249714911, + "index_ms": 1945.187957957387, + "by_category": { + "architecture": 0.7948459118879392, + "semantic": 0.7692536065216309 + } + }, + { + "repo": "redis", + "language": "c", + "mode": "unranked-balanced", + "chunks": 12306, + "tokens": 1802, + "ndcg5": 0.7820497592047851, + "ndcg10": 0.7820497592047851, + "p50_ms": 0.4921250219922513, + "p90_ms": 0.5275753210298717, + "p95_ms": 0.5321458273101599, + "p99_ms": 0.5970627965871244, + "index_ms": 1945.187957957387, + "by_category": { + "architecture": 0.7948459118879392, + "semantic": 0.7692536065216309 + } + }, + { + "repo": "redis", + "language": "c", + "mode": "unranked-bm25", + "chunks": 12306, + "tokens": 1725, + "ndcg5": 0.6758891280403, + "ndcg10": 0.7419126417409406, + "p50_ms": 0.4796250141225755, + "p90_ms": 0.5254371848423034, + "p95_ms": 0.5331228952854872, + "p99_ms": 0.5672909505665301, + "index_ms": 1945.187957957387, + "by_category": { + "architecture": 0.6707323081247354, + "semantic": 0.8130929753571458 + } + }, + { + "repo": "redis", + "language": "c", + "mode": "unranked-semantic", + "chunks": 12306, + "tokens": 1635, + "ndcg5": 0.6658764682653968, + "ndcg10": 0.701497186976199, + "p50_ms": 0.4610834876075387, + "p90_ms": 0.5408579891081899, + "p95_ms": 0.5496747384313494, + "p99_ms": 0.577668520854786, + "index_ms": 1945.187957957387, + "by_category": { + "architecture": 0.7930676558073393, + "semantic": 0.6099267181450585 + } + }, + { + "repo": "redux", + "language": "javascript", + "mode": "semble-auto", + "chunks": 87, + "tokens": 1683, + "ndcg5": 0.8928612069551187, + "ndcg10": 0.9175840269191813, + "p50_ms": 0.4566874704323709, + "p90_ms": 2.103492320748046, + "p95_ms": 2.301446118508467, + "p99_ms": 3.647089228616094, + "index_ms": 16.9183750404045, + "by_category": { + "architecture": 0.9032867981913646, + "semantic": 0.8813288610261599, + "symbol": 1.0 + } + }, + { + "repo": "redux", + "language": "javascript", + "mode": "semble-balanced", + "chunks": 87, + "tokens": 1692, + "ndcg5": 0.8928612069551187, + "ndcg10": 0.9175840269191813, + "p50_ms": 0.45599997974932194, + "p90_ms": 2.0960586436558515, + "p95_ms": 2.2736476268619312, + "p99_ms": 3.5514295194298007, + "index_ms": 16.9183750404045, + "by_category": { + "architecture": 0.9032867981913646, + "semantic": 0.8813288610261599, + "symbol": 1.0 + } + }, + { + "repo": "redux", + "language": "javascript", + "mode": "semble-bm25", + "chunks": 87, + "tokens": 1744, + "ndcg5": 0.880657359638273, + "ndcg10": 0.8951106009541674, + "p50_ms": 0.39568747160956264, + "p90_ms": 1.7201878072228283, + "p95_ms": 1.8221062899101537, + "p99_ms": 2.9409212491009367, + "index_ms": 16.9183750404045, + "by_category": { + "architecture": 0.8710490642551528, + "semantic": 0.852879737061381, + "symbol": 1.0 + } + }, + { + "repo": "redux", + "language": "javascript", + "mode": "semble-semantic", + "chunks": 87, + "tokens": 1534, + "ndcg5": 0.8981746049150108, + "ndcg10": 0.9148412715816774, + "p50_ms": 0.4000210028607398, + "p90_ms": 1.7467999714426696, + "p95_ms": 1.8536021118052313, + "p99_ms": 2.871654017362742, + "index_ms": 16.9183750404045, + "by_category": { + "architecture": 0.9105820163833688, + "semantic": 0.8703703703703703, + "symbol": 1.0 + } + }, + { + "repo": "redux", + "language": "javascript", + "mode": "unranked-auto", + "chunks": 87, + "tokens": 1725, + "ndcg5": 0.7846268032608154, + "ndcg10": 0.8191038292828831, + "p50_ms": 0.22504152730107307, + "p90_ms": 0.25022439076565206, + "p95_ms": 0.25183538964483887, + "p99_ms": 0.26833348732907325, + "index_ms": 16.9183750404045, + "by_category": { + "architecture": 0.7384460930122322, + "semantic": 0.8464518549167708, + "symbol": 0.8666666666666666 + } + }, + { + "repo": "redux", + "language": "javascript", + "mode": "unranked-balanced", + "chunks": 87, + "tokens": 1738, + "ndcg5": 0.7846268032608154, + "ndcg10": 0.8191038292828831, + "p50_ms": 0.2159790019504726, + "p90_ms": 0.2505871932953596, + "p95_ms": 0.25370628281962126, + "p99_ms": 0.26634126843418926, + "index_ms": 16.9183750404045, + "by_category": { + "architecture": 0.7384460930122322, + "semantic": 0.8464518549167708, + "symbol": 0.8666666666666666 + } + }, + { + "repo": "redux", + "language": "javascript", + "mode": "unranked-bm25", + "chunks": 87, + "tokens": 1755, + "ndcg5": 0.7280803155822425, + "ndcg10": 0.7625573416043103, + "p50_ms": 0.21675002062693238, + "p90_ms": 0.24310406297445297, + "p95_ms": 0.24645566008985045, + "p99_ms": 0.2660583239048719, + "index_ms": 16.9183750404045, + "by_category": { + "architecture": 0.7384460930122322, + "semantic": 0.7207929934088311, + "symbol": 0.8666666666666666 + } + }, + { + "repo": "redux", + "language": "javascript", + "mode": "unranked-semantic", + "chunks": 87, + "tokens": 1647, + "ndcg5": 0.8196394630357187, + "ndcg10": 0.8465252495818095, + "p50_ms": 0.21612548152916133, + "p90_ms": 0.24479557760059834, + "p95_ms": 0.2535964973503724, + "p99_ms": 0.2727856702404096, + "index_ms": 16.9183750404045, + "by_category": { + "architecture": 0.8058853585266572, + "semantic": 0.8624288341269906, + "symbol": 0.8666666666666666 + } + }, + { + "repo": "requests", + "language": "python", + "mode": "semble-auto", + "chunks": 316, + "tokens": 1634, + "ndcg5": 0.9673793323602942, + "ndcg10": 0.9673793323602942, + "p50_ms": 0.44543802505359054, + "p90_ms": 2.2950789018068463, + "p95_ms": 2.3537461733212695, + "p99_ms": 2.40814927208703, + "index_ms": 48.215540999080986, + "by_category": { + "architecture": 0.9770630826137678, + "semantic": 0.9385181336136884, + "symbol": 1.0 + } + }, + { + "repo": "requests", + "language": "python", + "mode": "semble-balanced", + "chunks": 316, + "tokens": 1641, + "ndcg5": 0.9673793323602942, + "ndcg10": 0.9673793323602942, + "p50_ms": 0.44422902283258736, + "p90_ms": 2.301287231966853, + "p95_ms": 2.3381082952255383, + "p99_ms": 2.3910544655518606, + "index_ms": 48.215540999080986, + "by_category": { + "architecture": 0.9770630826137678, + "semantic": 0.9385181336136884, + "symbol": 1.0 + } + }, + { + "repo": "requests", + "language": "python", + "mode": "semble-bm25", + "chunks": 316, + "tokens": 1743, + "ndcg5": 0.9322536197295659, + "ndcg10": 0.9419249399104294, + "p50_ms": 0.36595851997844875, + "p90_ms": 1.7489455116447061, + "p95_ms": 1.799197620130144, + "p99_ms": 1.863639519433491, + "index_ms": 48.215540999080986, + "by_category": { + "architecture": 0.908174877014625, + "semantic": 0.9351593323882765, + "symbol": 1.0 + } + }, + { + "repo": "requests", + "language": "python", + "mode": "semble-semantic", + "chunks": 316, + "tokens": 1518, + "ndcg5": 0.9556102206338835, + "ndcg10": 0.9556102206338835, + "p50_ms": 0.36039549740962684, + "p90_ms": 1.7500165791716427, + "p95_ms": 1.799507718533278, + "p99_ms": 2.173301568254828, + "index_ms": 48.215540999080986, + "by_category": { + "architecture": 1.0, + "semantic": 0.9351593323882765, + "symbol": 0.9261859507142916 + } + }, + { + "repo": "requests", + "language": "python", + "mode": "unranked-auto", + "chunks": 316, + "tokens": 1667, + "ndcg5": 0.7283771382562342, + "ndcg10": 0.7913931058263046, + "p50_ms": 0.23622901062481105, + "p90_ms": 0.252324971370399, + "p95_ms": 0.2590312680695206, + "p99_ms": 0.2652062487322837, + "index_ms": 48.215540999080986, + "by_category": { + "architecture": 0.8504524661486083, + "semantic": 0.8189036293372587, + "symbol": 0.664693163757553 + } + }, + { + "repo": "requests", + "language": "python", + "mode": "unranked-balanced", + "chunks": 316, + "tokens": 1672, + "ndcg5": 0.7468306505776613, + "ndcg10": 0.8098466181477317, + "p50_ms": 0.2275625301990658, + "p90_ms": 0.25188326835632324, + "p95_ms": 0.255074689630419, + "p99_ms": 0.2682485361583531, + "index_ms": 48.215540999080986, + "by_category": { + "architecture": 0.8504524661486083, + "semantic": 0.8189036293372587, + "symbol": 0.7385072130432616 + } + }, + { + "repo": "requests", + "language": "python", + "mode": "unranked-bm25", + "chunks": 316, + "tokens": 1708, + "ndcg5": 0.5898240924333563, + "ndcg10": 0.6443308622107146, + "p50_ms": 0.2228330122306943, + "p90_ms": 0.2550837118178606, + "p95_ms": 0.2570235199527815, + "p99_ms": 0.26370471401605755, + "index_ms": 48.215540999080986, + "by_category": { + "architecture": 0.6240240057008644, + "semantic": 0.6548508920449158, + "symbol": 0.6559284135897828 + } + }, + { + "repo": "requests", + "language": "python", + "mode": "unranked-semantic", + "chunks": 316, + "tokens": 1575, + "ndcg5": 0.7661287740193402, + "ndcg10": 0.7661287740193402, + "p50_ms": 0.22922901553101838, + "p90_ms": 0.2648538793437183, + "p95_ms": 0.2674565155757591, + "p99_ms": 0.2699576778104529, + "index_ms": 48.215540999080986, + "by_category": { + "architecture": 0.9358072203885207, + "semantic": 0.6512898282527885, + "symbol": 0.7123212623289701 + } + }, + { + "repo": "serde", + "language": "rust", + "mode": "semble-auto", + "chunks": 2187, + "tokens": 1624, + "ndcg5": 0.7172688678319089, + "ndcg10": 0.7269401880127725, + "p50_ms": 0.8439169905614108, + "p90_ms": 3.6172835971228796, + "p95_ms": 4.856005948386155, + "p99_ms": 5.183534777606837, + "index_ms": 295.99183396203443, + "by_category": { + "architecture": 0.7885962468833579, + "semantic": 0.7007303620904127, + "symbol": 0.6121147797198481 + } + }, + { + "repo": "serde", + "language": "rust", + "mode": "semble-balanced", + "chunks": 2187, + "tokens": 1612, + "ndcg5": 0.7013923995665121, + "ndcg10": 0.7288740791027768, + "p50_ms": 0.840603985125199, + "p90_ms": 3.6039380996953714, + "p95_ms": 4.848266640328802, + "p99_ms": 5.125919713755137, + "index_ms": 295.99183396203443, + "by_category": { + "architecture": 0.7885962468833579, + "semantic": 0.7007303620904127, + "symbol": 0.62178423516987 + } + }, + { + "repo": "serde", + "language": "rust", + "mode": "semble-bm25", + "chunks": 2187, + "tokens": 1631, + "ndcg5": 0.6920624189796883, + "ndcg10": 0.7017337391605518, + "p50_ms": 0.5289789987728, + "p90_ms": 2.5495118869002926, + "p95_ms": 3.8155166432261467, + "p99_ms": 3.842369709163904, + "index_ms": 295.99183396203443, + "by_category": { + "architecture": 0.8324356157188728, + "semantic": 0.5436432511904858, + "symbol": 0.6121147797198481 + } + }, + { + "repo": "serde", + "language": "rust", + "mode": "semble-semantic", + "chunks": 2187, + "tokens": 1184, + "ndcg5": 0.604266266296534, + "ndcg10": 0.6209329329632007, + "p50_ms": 0.6135204748716205, + "p90_ms": 2.5571295060217394, + "p95_ms": 3.4215586580103263, + "p99_ms": 3.638411710853688, + "index_ms": 295.99183396203443, + "by_category": { + "architecture": 0.5561606311644851, + "semantic": 0.7043537656746177, + "symbol": 0.6577324383928644 + } + }, + { + "repo": "serde", + "language": "rust", + "mode": "unranked-auto", + "chunks": 2187, + "tokens": 1566, + "ndcg5": 0.44523176840202705, + "ndcg10": 0.49662573095211576, + "p50_ms": 0.2861045068129897, + "p90_ms": 0.30892079812474554, + "p95_ms": 0.34441848401911557, + "p99_ms": 0.3476173209492117, + "index_ms": 295.99183396203443, + "by_category": { + "architecture": 0.5260842555148564, + "semantic": 0.5, + "symbol": 0.41791801597343775 + } + }, + { + "repo": "serde", + "language": "rust", + "mode": "unranked-balanced", + "chunks": 2187, + "tokens": 1546, + "ndcg5": 0.4645744087637541, + "ndcg10": 0.5001951274745564, + "p50_ms": 0.2762294898275286, + "p90_ms": 0.3229797992389649, + "p95_ms": 0.33610656682867557, + "p99_ms": 0.34468768688384444, + "index_ms": 295.99183396203443, + "by_category": { + "architecture": 0.5260842555148564, + "semantic": 0.5, + "symbol": 0.43576499858564094 + } + }, + { + "repo": "serde", + "language": "rust", + "mode": "unranked-bm25", + "chunks": 2187, + "tokens": 1669, + "ndcg5": 0.43898210339744576, + "ndcg10": 0.4725657065921333, + "p50_ms": 0.27733301976695657, + "p90_ms": 0.3142381436191499, + "p95_ms": 0.3254333481891081, + "p99_ms": 0.337720270617865, + "index_ms": 295.99183396203443, + "by_category": { + "architecture": 0.6118066694250938, + "semantic": 0.4384882922619096, + "symbol": 0.1755794210050676 + } + }, + { + "repo": "serde", + "language": "rust", + "mode": "unranked-semantic", + "chunks": 2187, + "tokens": 1392, + "ndcg5": 0.4065464876785729, + "ndcg10": 0.506325283345194, + "p50_ms": 0.2825625124387443, + "p90_ms": 0.30475449166260665, + "p95_ms": 0.33403987472411245, + "p99_ms": 0.33970793418120593, + "index_ms": 295.99183396203443, + "by_category": { + "architecture": 0.4346777703213359, + "semantic": 0.6359108127976215, + "symbol": 0.4910657717261977 + } + }, + { + "repo": "sinatra", + "language": "ruby", + "mode": "semble-auto", + "chunks": 134, + "tokens": 1898, + "ndcg5": 0.9815464876785729, + "ndcg10": 0.9815464876785729, + "p50_ms": 0.4774164699483663, + "p90_ms": 4.001133394194767, + "p95_ms": 4.284504806855695, + "p99_ms": 5.940100970910859, + "index_ms": 22.023458033800125, + "by_category": { + "architecture": 0.9261859507142916, + "semantic": 1.0, + "symbol": 1.0 + } + }, + { + "repo": "sinatra", + "language": "ruby", + "mode": "semble-balanced", + "chunks": 134, + "tokens": 1903, + "ndcg5": 0.9815464876785729, + "ndcg10": 0.9815464876785729, + "p50_ms": 0.48204147606156766, + "p90_ms": 4.0289465337991714, + "p95_ms": 4.194792298949325, + "p99_ms": 5.89655846066307, + "index_ms": 22.023458033800125, + "by_category": { + "architecture": 0.9261859507142916, + "semantic": 1.0, + "symbol": 1.0 + } + }, + { + "repo": "sinatra", + "language": "ruby", + "mode": "semble-bm25", + "chunks": 134, + "tokens": 2164, + "ndcg5": 0.9403945936705721, + "ndcg10": 0.9403945936705721, + "p50_ms": 0.39554148679599166, + "p90_ms": 2.9138538346160203, + "p95_ms": 3.0656639224616833, + "p99_ms": 5.353232812485654, + "index_ms": 22.023458033800125, + "by_category": { + "architecture": 0.7615783746822885, + "semantic": 1.0, + "symbol": 1.0 + } + }, + { + "repo": "sinatra", + "language": "ruby", + "mode": "semble-semantic", + "chunks": 134, + "tokens": 1583, + "ndcg5": 0.9530803155822426, + "ndcg10": 0.9530803155822426, + "p50_ms": 0.39412500336766243, + "p90_ms": 2.736908307997511, + "p95_ms": 3.661258003558034, + "p99_ms": 5.805218004970808, + "index_ms": 22.023458033800125, + "by_category": { + "architecture": 0.9261859507142916, + "semantic": 0.9367418397859326, + "symbol": 1.0 + } + }, + { + "repo": "sinatra", + "language": "ruby", + "mode": "unranked-auto", + "chunks": 134, + "tokens": 1547, + "ndcg5": 0.8711479713895818, + "ndcg10": 0.8711479713895818, + "p50_ms": 0.24075002875179052, + "p90_ms": 0.26057500508613884, + "p95_ms": 0.2640625258209184, + "p99_ms": 0.2726125024491921, + "index_ms": 22.023458033800125, + "by_category": { + "architecture": 0.8123212623289702, + "semantic": 0.881186284230377, + "symbol": 0.9051127596788988 + } + }, + { + "repo": "sinatra", + "language": "ruby", + "mode": "unranked-balanced", + "chunks": 134, + "tokens": 1545, + "ndcg5": 0.8496141434859122, + "ndcg10": 0.8646656432691111, + "p50_ms": 0.23035399499349296, + "p90_ms": 0.260107999201864, + "p95_ms": 0.26121629925910383, + "p99_ms": 0.27464323036838323, + "index_ms": 22.023458033800125, + "by_category": { + "architecture": 0.8123212623289702, + "semantic": 0.881186284230377, + "symbol": 0.8835049992773302 + } + }, + { + "repo": "sinatra", + "language": "ruby", + "mode": "unranked-bm25", + "chunks": 134, + "tokens": 1777, + "ndcg5": 0.8087376752205155, + "ndcg10": 0.8087376752205155, + "p50_ms": 0.229645986109972, + "p90_ms": 0.25326727190986276, + "p95_ms": 0.2612274285638705, + "p99_ms": 0.2716454799519852, + "index_ms": 22.023458033800125, + "by_category": { + "architecture": 0.6488153892673832, + "semantic": 0.8256307286748215, + "symbol": 0.9166666666666666 + } + }, + { + "repo": "sinatra", + "language": "ruby", + "mode": "unranked-semantic", + "chunks": 134, + "tokens": 1527, + "ndcg5": 0.8639821033974456, + "ndcg10": 0.8749024752383538, + "p50_ms": 0.22947898833081126, + "p90_ms": 0.2577792329248041, + "p95_ms": 0.2650586568051949, + "p99_ms": 0.26911177032161504, + "index_ms": 22.023458033800125, + "by_category": { + "architecture": 0.9261859507142916, + "semantic": 0.9179843896825461, + "symbol": 0.7675433740087843 + } + }, + { + "repo": "snapkit", + "language": "swift", + "mode": "semble-auto", + "chunks": 201, + "tokens": 1525, + "ndcg5": 0.7847362671174445, + "ndcg10": 0.8004579900088665, + "p50_ms": 1.7811250290833414, + "p90_ms": 3.8507332210429017, + "p95_ms": 5.2506166044622615, + "p99_ms": 6.757856886833903, + "index_ms": 31.86104103224352, + "by_category": { + "architecture": 0.7080343955788911, + "semantic": 0.754550097625513, + "symbol": 1.0 + } + }, + { + "repo": "snapkit", + "language": "swift", + "mode": "semble-balanced", + "chunks": 201, + "tokens": 1517, + "ndcg5": 0.7847362671174445, + "ndcg10": 0.8004579900088665, + "p50_ms": 1.7449170118197799, + "p90_ms": 3.797966602724046, + "p95_ms": 5.20758359925821, + "p99_ms": 6.742183922324326, + "index_ms": 31.86104103224352, + "by_category": { + "architecture": 0.7080343955788911, + "semantic": 0.754550097625513, + "symbol": 1.0 + } + }, + { + "repo": "snapkit", + "language": "swift", + "mode": "semble-bm25", + "chunks": 201, + "tokens": 1605, + "ndcg5": 0.7400994047442282, + "ndcg10": 0.796855471781194, + "p50_ms": 1.4397079939953983, + "p90_ms": 2.8064915561117236, + "p95_ms": 3.439849975984542, + "p99_ms": 4.052570003550499, + "index_ms": 31.86104103224352, + "by_category": { + "architecture": 0.8102255943731342, + "semantic": 0.7040634785765884, + "symbol": 1.0 + } + }, + { + "repo": "snapkit", + "language": "swift", + "mode": "semble-semantic", + "chunks": 201, + "tokens": 1382, + "ndcg5": 0.7039169946005486, + "ndcg10": 0.7438737201760983, + "p50_ms": 1.487916975747794, + "p90_ms": 2.666758641134948, + "p95_ms": 3.1200252240523683, + "p99_ms": 3.55800503399223, + "index_ms": 31.86104103224352, + "by_category": { + "architecture": 0.6105182098612075, + "semantic": 0.6912576761008079, + "symbol": 1.0 + } + }, + { + "repo": "snapkit", + "language": "swift", + "mode": "unranked-auto", + "chunks": 201, + "tokens": 1532, + "ndcg5": 0.687427736143569, + "ndcg10": 0.7042283082065048, + "p50_ms": 0.271082972176373, + "p90_ms": 0.2803750103339553, + "p95_ms": 0.2846082206815481, + "p99_ms": 0.2896880730986595, + "index_ms": 31.86104103224352, + "by_category": { + "architecture": 0.4391589745403695, + "semantic": 0.7437944756417137, + "symbol": 0.8769765845238192 + } + }, + { + "repo": "snapkit", + "language": "swift", + "mode": "unranked-balanced", + "chunks": 201, + "tokens": 1509, + "ndcg5": 0.7158177550996108, + "ndcg10": 0.7326183271625465, + "p50_ms": 0.26779196923598647, + "p90_ms": 0.27955780969932675, + "p95_ms": 0.2815414103679359, + "p99_ms": 0.2836418873630464, + "index_ms": 31.86104103224352, + "by_category": { + "architecture": 0.4391589745403695, + "semantic": 0.7437944756417137, + "symbol": 1.0 + } + }, + { + "repo": "snapkit", + "language": "swift", + "mode": "unranked-bm25", + "chunks": 201, + "tokens": 1595, + "ndcg5": 0.6828875669686268, + "ndcg10": 0.7102881198230901, + "p50_ms": 0.2639580052345991, + "p90_ms": 0.2776834066025913, + "p95_ms": 0.2800500253215432, + "p99_ms": 0.2825100300833583, + "index_ms": 31.86104103224352, + "by_category": { + "architecture": 0.5004219451139623, + "semantic": 0.7287928526838322, + "symbol": 0.8769765845238192 + } + }, + { + "repo": "snapkit", + "language": "swift", + "mode": "unranked-semantic", + "chunks": 201, + "tokens": 1458, + "ndcg5": 0.49238829922624816, + "ndcg10": 0.5566115537853153, + "p50_ms": 0.26470801094546914, + "p90_ms": 0.27853280771523714, + "p95_ms": 0.2813913859426975, + "p99_ms": 0.28361183591187, + "index_ms": 31.86104103224352, + "by_category": { + "architecture": 0.40876479517697223, + "semantic": 0.6090734826884373, + "symbol": 0.5820471449530406 + } + }, + { + "repo": "starlette", + "language": "python", + "mode": "semble-auto", + "chunks": 419, + "tokens": 1487, + "ndcg5": 0.9631568755014722, + "ndcg10": 0.9631568755014722, + "p50_ms": 0.48337504267692566, + "p90_ms": 1.9716250244528055, + "p95_ms": 2.3925419664010406, + "p99_ms": 2.5564084062352777, + "index_ms": 55.946209002286196, + "by_category": { + "architecture": 0.9296631259573561, + "semantic": 1.0, + "symbol": 1.0 + } + }, + { + "repo": "starlette", + "language": "python", + "mode": "semble-balanced", + "chunks": 419, + "tokens": 1496, + "ndcg5": 0.9631568755014722, + "ndcg10": 0.9631568755014722, + "p50_ms": 0.49291597679257393, + "p90_ms": 1.9775839755311608, + "p95_ms": 2.487791993189603, + "p99_ms": 2.6102920179255307, + "index_ms": 55.946209002286196, + "by_category": { + "architecture": 0.9296631259573561, + "semantic": 1.0, + "symbol": 1.0 + } + }, + { + "repo": "starlette", + "language": "python", + "mode": "semble-bm25", + "chunks": 419, + "tokens": 1548, + "ndcg5": 0.9777556181863641, + "ndcg10": 0.9777556181863641, + "p50_ms": 0.38604100700467825, + "p90_ms": 1.5623339568264782, + "p95_ms": 1.6178750083781779, + "p99_ms": 1.9773422391153874, + "index_ms": 55.946209002286196, + "by_category": { + "architecture": 0.9575334529012405, + "semantic": 1.0, + "symbol": 1.0 + } + }, + { + "repo": "starlette", + "language": "python", + "mode": "semble-semantic", + "chunks": 419, + "tokens": 1456, + "ndcg5": 0.9322234158453572, + "ndcg10": 0.9322234158453572, + "p50_ms": 0.3821250284090638, + "p90_ms": 1.5755000058561563, + "p95_ms": 1.6242919955402613, + "p99_ms": 1.9450919702649119, + "index_ms": 55.946209002286196, + "by_category": { + "architecture": 0.8706083393411361, + "semantic": 1.0, + "symbol": 1.0 + } + }, + { + "repo": "starlette", + "language": "python", + "mode": "unranked-auto", + "chunks": 419, + "tokens": 1538, + "ndcg5": 0.777567321333282, + "ndcg10": 0.777567321333282, + "p50_ms": 0.24295801995322108, + "p90_ms": 0.2609160146676004, + "p95_ms": 0.270417018327862, + "p99_ms": 0.2749170293100178, + "index_ms": 55.946209002286196, + "by_category": { + "architecture": 0.7980239277526614, + "semantic": 0.8839441578296375, + "symbol": 0.6261859507142915 + } + }, + { + "repo": "starlette", + "language": "python", + "mode": "unranked-balanced", + "chunks": 419, + "tokens": 1545, + "ndcg5": 0.7537577975237582, + "ndcg10": 0.7537577975237582, + "p50_ms": 0.23937498917803168, + "p90_ms": 0.2596249687485397, + "p95_ms": 0.2597499988041818, + "p99_ms": 0.2873164019547403, + "index_ms": 55.946209002286196, + "by_category": { + "architecture": 0.7980239277526614, + "semantic": 0.8839441578296375, + "symbol": 0.5261859507142915 + } + }, + { + "repo": "starlette", + "language": "python", + "mode": "unranked-bm25", + "chunks": 419, + "tokens": 1511, + "ndcg5": 0.7950344255862363, + "ndcg10": 0.8034743916235718, + "p50_ms": 0.23754098219797015, + "p90_ms": 0.26229204377159476, + "p95_ms": 0.2637919969856739, + "p99_ms": 0.26872558519244194, + "index_ms": 55.946209002286196, + "by_category": { + "architecture": 0.8385681736106864, + "semantic": 0.8035565121611998, + "symbol": 0.7261859507142916 + } + }, + { + "repo": "starlette", + "language": "python", + "mode": "unranked-semantic", + "chunks": 419, + "tokens": 1525, + "ndcg5": 0.6156126599075094, + "ndcg10": 0.6396799167290554, + "p50_ms": 0.23866695119068027, + "p90_ms": 0.2627079957164824, + "p95_ms": 0.2643340267241001, + "p99_ms": 0.2827331889420748, + "index_ms": 55.946209002286196, + "by_category": { + "architecture": 0.6414602669065557, + "semantic": 0.7754430630676099, + "symbol": 0.5 + } + }, + { + "repo": "telescope.nvim", + "language": "lua", + "mode": "semble-auto", + "chunks": 1053, + "tokens": 1652, + "ndcg5": 0.76266260192301, + "ndcg10": 0.76266260192301, + "p50_ms": 0.5643539770971984, + "p90_ms": 0.6339249899610877, + "p95_ms": 0.7360146002611159, + "p99_ms": 2.1612365095643304, + "index_ms": 136.19391695829108, + "by_category": { + "architecture": 0.7954904433608064, + "semantic": 0.7449860719180426 + } + }, + { + "repo": "telescope.nvim", + "language": "lua", + "mode": "semble-balanced", + "chunks": 1053, + "tokens": 1652, + "ndcg5": 0.76266260192301, + "ndcg10": 0.76266260192301, + "p50_ms": 0.5678960005752742, + "p90_ms": 0.6167868676129729, + "p95_ms": 0.7372312276856987, + "p99_ms": 2.1752462728181827, + "index_ms": 136.19391695829108, + "by_category": { + "architecture": 0.7954904433608064, + "semantic": 0.7449860719180426 + } + }, + { + "repo": "telescope.nvim", + "language": "lua", + "mode": "semble-bm25", + "chunks": 1053, + "tokens": 1672, + "ndcg5": 0.7817057975264041, + "ndcg10": 0.7817057975264041, + "p50_ms": 0.44414601870812476, + "p90_ms": 0.4673496703617275, + "p95_ms": 0.5540999758522968, + "p99_ms": 1.6481199895497394, + "index_ms": 136.19391695829108, + "by_category": { + "architecture": 0.8587004753197685, + "semantic": 0.7402471248684386 + } + }, + { + "repo": "telescope.nvim", + "language": "lua", + "mode": "semble-semantic", + "chunks": 1053, + "tokens": 1495, + "ndcg5": 0.7143505179387404, + "ndcg10": 0.7301237617780268, + "p50_ms": 0.42747947736643255, + "p90_ms": 0.45800802181474864, + "p95_ms": 0.5307538900524387, + "p99_ms": 1.4928507898002847, + "index_ms": 136.19391695829108, + "by_category": { + "architecture": 0.7277820419369281, + "semantic": 0.7313846878463106 + } + }, + { + "repo": "telescope.nvim", + "language": "lua", + "mode": "unranked-auto", + "chunks": 1053, + "tokens": 1562, + "ndcg5": 0.6511414337942434, + "ndcg10": 0.6847250369889311, + "p50_ms": 0.2734790032263845, + "p90_ms": 0.282225024420768, + "p95_ms": 0.2859062980860472, + "p99_ms": 0.2939812932163477, + "index_ms": 136.19391695829108, + "by_category": { + "architecture": 0.5793671491721605, + "semantic": 0.7414562073518073 + } + }, + { + "repo": "telescope.nvim", + "language": "lua", + "mode": "unranked-balanced", + "chunks": 1053, + "tokens": 1562, + "ndcg5": 0.6511414337942434, + "ndcg10": 0.6847250369889311, + "p50_ms": 0.27593798586167395, + "p90_ms": 0.2903952030465007, + "p95_ms": 0.29473715694621205, + "p99_ms": 0.2951802429743111, + "index_ms": 136.19391695829108, + "by_category": { + "architecture": 0.5793671491721605, + "semantic": 0.7414562073518073 + } + }, + { + "repo": "telescope.nvim", + "language": "lua", + "mode": "unranked-bm25", + "chunks": 1053, + "tokens": 1620, + "ndcg5": 0.6639821033974458, + "ndcg10": 0.7237297490821369, + "p50_ms": 0.2740210038609803, + "p90_ms": 0.287745805690065, + "p95_ms": 0.2913518081186339, + "p99_ms": 0.2954375691479072, + "index_ms": 136.19391695829108, + "by_category": { + "architecture": 0.7273193149747181, + "semantic": 0.7217969059092084 + } + }, + { + "repo": "telescope.nvim", + "language": "lua", + "mode": "unranked-semantic", + "chunks": 1053, + "tokens": 1500, + "ndcg5": 0.5040353574739782, + "ndcg10": 0.5563227428514471, + "p50_ms": 0.276542006758973, + "p90_ms": 0.29030859004706144, + "p95_ms": 0.2961205376777798, + "p99_ms": 0.29669129406102, + "index_ms": 136.19391695829108, + "by_category": { + "architecture": 0.3621292579800159, + "semantic": 0.6608884654745253 + } + }, + { + "repo": "tokio", + "language": "rust", + "mode": "semble-auto", + "chunks": 5338, + "tokens": 1756, + "ndcg5": 0.9266433990956824, + "ndcg10": 0.9368625189751066, + "p50_ms": 0.7768750074319541, + "p90_ms": 3.645766072440895, + "p95_ms": 5.359621124807747, + "p99_ms": 7.336824222002175, + "index_ms": 795.08541704854, + "by_category": { + "architecture": 0.8029215983923241, + "semantic": 0.9899650986435234, + "symbol": 1.0 + } + }, + { + "repo": "tokio", + "language": "rust", + "mode": "semble-balanced", + "chunks": 5338, + "tokens": 1748, + "ndcg5": 0.9266433990956824, + "ndcg10": 0.9368625189751066, + "p50_ms": 1.453999982913956, + "p90_ms": 4.772045527352021, + "p95_ms": 5.53280177700799, + "p99_ms": 5.849627530551515, + "index_ms": 795.08541704854, + "by_category": { + "architecture": 0.8029215983923241, + "semantic": 0.9899650986435234, + "symbol": 1.0 + } + }, + { + "repo": "tokio", + "language": "rust", + "mode": "semble-bm25", + "chunks": 5338, + "tokens": 1773, + "ndcg5": 0.9132034061286294, + "ndcg10": 0.934342897848962, + "p50_ms": 0.8378959901165217, + "p90_ms": 4.1608041909057665, + "p95_ms": 5.435889915679582, + "p99_ms": 6.512777992174959, + "index_ms": 795.08541704854, + "by_category": { + "architecture": 0.8833341916241165, + "semantic": 0.9233566009043177, + "symbol": 1.0 + } + }, + { + "repo": "tokio", + "language": "rust", + "mode": "semble-semantic", + "chunks": 5338, + "tokens": 1681, + "ndcg5": 0.9169720789148187, + "ndcg10": 0.9364199836332215, + "p50_ms": 1.2504789920058101, + "p90_ms": 4.918117320630699, + "p95_ms": 5.205954480334187, + "p99_ms": 6.635324481758286, + "index_ms": 795.08541704854, + "by_category": { + "architecture": 0.9018116803850807, + "semantic": 0.9146911987942431, + "symbol": 1.0 + } + }, + { + "repo": "tokio", + "language": "rust", + "mode": "unranked-auto", + "chunks": 5338, + "tokens": 1794, + "ndcg5": 0.7042662662965341, + "ndcg10": 0.7329969974928435, + "p50_ms": 0.7999794906936586, + "p90_ms": 0.9208666800986975, + "p95_ms": 0.9472024074057117, + "p99_ms": 0.9724404831649736, + "index_ms": 795.08541704854, + "by_category": { + "architecture": 0.5466689580771691, + "semantic": 0.8615986575892967, + "symbol": 0.7478561567799132 + } + }, + { + "repo": "tokio", + "language": "rust", + "mode": "unranked-balanced", + "chunks": 5338, + "tokens": 1799, + "ndcg5": 0.7042662662965341, + "ndcg10": 0.731853304804109, + "p50_ms": 0.7899370102677494, + "p90_ms": 0.9419250069186091, + "p95_ms": 0.9530416835332289, + "p99_ms": 0.9612747404025868, + "index_ms": 795.08541704854, + "by_category": { + "architecture": 0.5466689580771691, + "semantic": 0.8615986575892967, + "symbol": 0.7440438478174651 + } + }, + { + "repo": "tokio", + "language": "rust", + "mode": "unranked-bm25", + "chunks": 5338, + "tokens": 1750, + "ndcg5": 0.7161732909393883, + "ndcg10": 0.7469841028405844, + "p50_ms": 0.7497500337194651, + "p90_ms": 0.8845499716699123, + "p95_ms": 0.9036812407430261, + "p99_ms": 0.9482362482231109, + "index_ms": 795.08541704854, + "by_category": { + "architecture": 0.6519723067392391, + "semantic": 0.854414018894993, + "symbol": 0.6987560108693848 + } + }, + { + "repo": "tokio", + "language": "rust", + "mode": "unranked-semantic", + "chunks": 5338, + "tokens": 1715, + "ndcg5": 0.6298585716628425, + "ndcg10": 0.6704387024519106, + "p50_ms": 0.559791486011818, + "p90_ms": 0.7467667164746673, + "p95_ms": 0.8383294683881104, + "p99_ms": 0.8597994991578162, + "index_ms": 795.08541704854, + "by_category": { + "architecture": 0.6362548942179891, + "semantic": 0.6449106168118548, + "symbol": 0.7386599582059065 + } + }, + { + "repo": "trpc", + "language": "typescript", + "mode": "semble-auto", + "chunks": 690, + "tokens": 1522, + "ndcg5": 0.8273505311466753, + "ndcg10": 0.8491912748284915, + "p50_ms": 2.0867914718110114, + "p90_ms": 3.0229665921069686, + "p95_ms": 3.32287437922787, + "p99_ms": 4.158242083503863, + "index_ms": 107.28829097934067, + "by_category": { + "architecture": 0.7767199752358612, + "semantic": 0.8242950992404203, + "symbol": 1.0 + } + }, + { + "repo": "trpc", + "language": "typescript", + "mode": "semble-balanced", + "chunks": 690, + "tokens": 1515, + "ndcg5": 0.8273505311466753, + "ndcg10": 0.8491912748284915, + "p50_ms": 2.079041500110179, + "p90_ms": 2.974587498465553, + "p95_ms": 3.0527353956131273, + "p99_ms": 4.091813493869266, + "index_ms": 107.28829097934067, + "by_category": { + "architecture": 0.7767199752358612, + "semantic": 0.8242950992404203, + "symbol": 1.0 + } + }, + { + "repo": "trpc", + "language": "typescript", + "mode": "semble-bm25", + "chunks": 690, + "tokens": 1521, + "ndcg5": 0.7928612069551187, + "ndcg10": 0.8216136113546089, + "p50_ms": 1.5086659986991435, + "p90_ms": 2.0512208284344524, + "p95_ms": 2.1317643084330493, + "p99_ms": 3.0351200600853177, + "index_ms": 107.28829097934067, + "by_category": { + "architecture": 0.8101659157298811, + "semantic": 0.7072778430361615, + "symbol": 1.0 + } + }, + { + "repo": "trpc", + "language": "typescript", + "mode": "semble-semantic", + "chunks": 690, + "tokens": 1434, + "ndcg5": 0.8154461285881622, + "ndcg10": 0.8360378206099339, + "p50_ms": 1.492770534241572, + "p90_ms": 1.97151672327891, + "p95_ms": 2.058789916918614, + "p99_ms": 2.8700580057920875, + "index_ms": 107.28829097934067, + "by_category": { + "architecture": 0.7096165826699786, + "semantic": 0.8634033929769788, + "symbol": 1.0 + } + }, + { + "repo": "trpc", + "language": "typescript", + "mode": "unranked-auto", + "chunks": 690, + "tokens": 1480, + "ndcg5": 0.5394997876144049, + "ndcg10": 0.6032754521483804, + "p50_ms": 0.27587448130361736, + "p90_ms": 0.28907941305078566, + "p95_ms": 0.29195561655797064, + "p99_ms": 0.2963575453031808, + "index_ms": 107.28829097934067, + "by_category": { + "architecture": 0.3728436687664891, + "semantic": 0.6752217865325244, + "symbol": 0.8712414374216044 + } + }, + { + "repo": "trpc", + "language": "typescript", + "mode": "unranked-balanced", + "chunks": 690, + "tokens": 1468, + "ndcg5": 0.5394997876144049, + "ndcg10": 0.5999183341088737, + "p50_ms": 0.27064551250077784, + "p90_ms": 0.2861506422050297, + "p95_ms": 0.2901354746427387, + "p99_ms": 0.290294288424775, + "index_ms": 107.28829097934067, + "by_category": { + "architecture": 0.3728436687664891, + "semantic": 0.6752217865325244, + "symbol": 0.8578129652635775 + } + }, + { + "repo": "trpc", + "language": "typescript", + "mode": "unranked-bm25", + "chunks": 690, + "tokens": 1453, + "ndcg5": 0.5944085542131627, + "ndcg10": 0.6036373390521412, + "p50_ms": 0.2637499710544944, + "p90_ms": 0.2829043718520552, + "p95_ms": 0.2885556983528659, + "p99_ms": 0.29067836178001016, + "index_ms": 107.28829097934067, + "by_category": { + "architecture": 0.36102138553290686, + "semantic": 0.5977965281113669, + "symbol": 1.0 + } + }, + { + "repo": "trpc", + "language": "typescript", + "mode": "unranked-semantic", + "chunks": 690, + "tokens": 1439, + "ndcg5": 0.5427206381974323, + "ndcg10": 0.5791696410442096, + "p50_ms": 0.2639165031723678, + "p90_ms": 0.29573721694760025, + "p95_ms": 0.2961125399451703, + "p99_ms": 0.29782250872813165, + "index_ms": 107.28829097934067, + "by_category": { + "architecture": 0.4006646625954206, + "semantic": 0.7308880018539874, + "symbol": 0.6523719014285831 + } + }, + { + "repo": "vapor", + "language": "swift", + "mode": "semble-auto", + "chunks": 1485, + "tokens": 1336, + "ndcg5": 0.7240165939509212, + "ndcg10": 0.7653556218129065, + "p50_ms": 0.6523959746118635, + "p90_ms": 2.6620993972755973, + "p95_ms": 3.0039166478672996, + "p99_ms": 3.0615497269900516, + "index_ms": 251.12970802001655, + "by_category": { + "architecture": 0.49091435506199194, + "semantic": 0.7738835265051539, + "symbol": 1.0 + } + }, + { + "repo": "vapor", + "language": "swift", + "mode": "semble-balanced", + "chunks": 1485, + "tokens": 1343, + "ndcg5": 0.7240165939509212, + "ndcg10": 0.7653556218129065, + "p50_ms": 0.6377919926308095, + "p90_ms": 2.6228580099996184, + "p95_ms": 2.8142726456280798, + "p99_ms": 3.0356545455288138, + "index_ms": 251.12970802001655, + "by_category": { + "architecture": 0.49091435506199194, + "semantic": 0.7738835265051539, + "symbol": 1.0 + } + }, + { + "repo": "vapor", + "language": "swift", + "mode": "semble-bm25", + "chunks": 1485, + "tokens": 1335, + "ndcg5": 0.7424753504410309, + "ndcg10": 0.7987931201449151, + "p50_ms": 0.49777046660892665, + "p90_ms": 2.1169088839087635, + "p95_ms": 2.1972371701849625, + "p99_ms": 2.2109802247723565, + "index_ms": 251.12970802001655, + "by_category": { + "architecture": 0.5880032971355754, + "semantic": 0.800846607963684, + "symbol": 1.0 + } + }, + { + "repo": "vapor", + "language": "swift", + "mode": "semble-semantic", + "chunks": 1485, + "tokens": 1380, + "ndcg5": 0.6680676558073393, + "ndcg10": 0.7037313397069137, + "p50_ms": 0.49441700684837997, + "p90_ms": 2.1597288956400007, + "p95_ms": 2.185952381114476, + "p99_ms": 2.4448904668679456, + "index_ms": 251.12970802001655, + "by_category": { + "architecture": 0.5029637601343332, + "semantic": 0.6832668224096622, + "symbol": 1.0 + } + }, + { + "repo": "vapor", + "language": "swift", + "mode": "unranked-auto", + "chunks": 1485, + "tokens": 1406, + "ndcg5": 0.4613147192765458, + "ndcg10": 0.534701581071343, + "p50_ms": 0.2679579774849117, + "p90_ms": 0.2939371741376817, + "p95_ms": 0.2958541561383754, + "p99_ms": 0.31073726364411414, + "index_ms": 251.12970802001655, + "by_category": { + "architecture": 0.06152523225985606, + "semantic": 0.7066400751569702, + "symbol": 0.20549829081656998 + } + }, + { + "repo": "vapor", + "language": "swift", + "mode": "unranked-balanced", + "chunks": 1485, + "tokens": 1406, + "ndcg5": 0.4613147192765458, + "ndcg10": 0.533381578547951, + "p50_ms": 0.2656459982972592, + "p90_ms": 0.28009997331537306, + "p95_ms": 0.29054789629299194, + "p99_ms": 0.2988759760046378, + "index_ms": 251.12970802001655, + "by_category": { + "architecture": 0.06152523225985606, + "semantic": 0.7066400751569702, + "symbol": 0.19669827399395637 + } + }, + { + "repo": "vapor", + "language": "swift", + "mode": "unranked-bm25", + "chunks": 1485, + "tokens": 1377, + "ndcg5": 0.4559541823122644, + "ndcg10": 0.5243828131373733, + "p50_ms": 0.2611459931358695, + "p90_ms": 0.27401248225942254, + "p95_ms": 0.2832187252352014, + "p99_ms": 0.2846437582047656, + "index_ms": 251.12970802001655, + "by_category": { + "architecture": 0.3948585655931894, + "semantic": 0.6191940883550416, + "symbol": 0.21145444299910485 + } + }, + { + "repo": "vapor", + "language": "swift", + "mode": "unranked-semantic", + "chunks": 1485, + "tokens": 1435, + "ndcg5": 0.34598603945740936, + "ndcg10": 0.4519050929078607, + "p50_ms": 0.26302100741304457, + "p90_ms": 0.272774655604735, + "p95_ms": 0.27597082662396133, + "p99_ms": 0.2812277839984745, + "index_ms": 251.12970802001655, + "by_category": { + "architecture": 0.0, + "semantic": 0.6034290740125245, + "symbol": 0.19669827399395637 + } + }, + { + "repo": "vitest", + "language": "typescript", + "mode": "semble-auto", + "chunks": 2065, + "tokens": 1473, + "ndcg5": 0.7161732909393883, + "ndcg10": 0.7339836502947895, + "p50_ms": 0.697499985108152, + "p90_ms": 3.062258393038065, + "p95_ms": 3.7555777205852805, + "p99_ms": 5.127281951135953, + "index_ms": 312.4053750070743, + "by_category": { + "architecture": 0.7529637601343332, + "semantic": 0.6510809495536172, + "symbol": 1.0 + } + }, + { + "repo": "vitest", + "language": "typescript", + "mode": "semble-balanced", + "chunks": 2065, + "tokens": 1475, + "ndcg5": 0.7161732909393883, + "ndcg10": 0.7339836502947895, + "p50_ms": 0.6792085187043995, + "p90_ms": 3.0707792262546727, + "p95_ms": 3.7945149088045595, + "p99_ms": 5.146902959677388, + "index_ms": 312.4053750070743, + "by_category": { + "architecture": 0.7529637601343332, + "semantic": 0.6510809495536172, + "symbol": 1.0 + } + }, + { + "repo": "vitest", + "language": "typescript", + "mode": "semble-bm25", + "chunks": 2065, + "tokens": 1454, + "ndcg5": 0.6865154901128991, + "ndcg10": 0.7124109416185442, + "p50_ms": 0.4954579926561564, + "p90_ms": 2.2662290837615737, + "p95_ms": 2.751313947374002, + "p99_ms": 3.5781628184486167, + "index_ms": 312.4053750070743, + "by_category": { + "architecture": 0.7251859823565555, + "semantic": 0.6270093580210502, + "symbol": 1.0 + } + }, + { + "repo": "vitest", + "language": "typescript", + "mode": "semble-semantic", + "chunks": 2065, + "tokens": 1355, + "ndcg5": 0.6246141434859122, + "ndcg10": 0.6500587075060622, + "p50_ms": 0.513791514094919, + "p90_ms": 2.149916568305344, + "p95_ms": 2.579388962476515, + "p99_ms": 3.492877829703501, + "index_ms": 312.4053750070743, + "by_category": { + "architecture": 0.7192441461309548, + "semantic": 0.6532462975759559, + "symbol": 0.5 + } + }, + { + "repo": "vitest", + "language": "typescript", + "mode": "unranked-auto", + "chunks": 2065, + "tokens": 1527, + "ndcg5": 0.44176559630569673, + "ndcg10": 0.515073060478902, + "p50_ms": 0.27995850541628897, + "p90_ms": 0.3081291855778545, + "p95_ms": 0.3110086254309863, + "p99_ms": 0.32000173232518137, + "index_ms": 312.4053750070743, + "by_category": { + "architecture": 0.5959914285832172, + "semantic": 0.4295920580071579, + "symbol": 0.6666666666666666 + } + }, + { + "repo": "vitest", + "language": "typescript", + "mode": "unranked-balanced", + "chunks": 2065, + "tokens": 1528, + "ndcg5": 0.44176559630569673, + "ndcg10": 0.515073060478902, + "p50_ms": 0.27972899260930717, + "p90_ms": 0.31157557968981564, + "p95_ms": 0.3160396183375269, + "p99_ms": 0.3166415181476623, + "index_ms": 312.4053750070743, + "by_category": { + "architecture": 0.5959914285832172, + "semantic": 0.4295920580071579, + "symbol": 0.6666666666666666 + } + }, + { + "repo": "vitest", + "language": "typescript", + "mode": "unranked-bm25", + "chunks": 2065, + "tokens": 1525, + "ndcg5": 0.41051593130111536, + "ndcg10": 0.4589007644176478, + "p50_ms": 0.27479149866849184, + "p90_ms": 0.29625442693941295, + "p95_ms": 0.30887019529473037, + "p99_ms": 0.3176740085473284, + "index_ms": 312.4053750070743, + "by_category": { + "architecture": 0.4608821804281287, + "semantic": 0.40115656416219847, + "symbol": 0.6666666666666666 + } + }, + { + "repo": "vitest", + "language": "typescript", + "mode": "unranked-semantic", + "chunks": 2065, + "tokens": 1444, + "ndcg5": 0.47242295594396955, + "ndcg10": 0.486876197259864, + "p50_ms": 0.26966651785187423, + "p90_ms": 0.29967533191666007, + "p95_ms": 0.3016832924913615, + "p99_ms": 0.31396946287713945, + "index_ms": 312.4053750070743, + "by_category": { + "architecture": 0.5102677186074751, + "semantic": 0.425083421232039, + "symbol": 0.6666666666666666 + } + }, + { + "repo": "xmonad", + "language": "haskell", + "mode": "semble-auto", + "chunks": 241, + "tokens": 1695, + "ndcg5": 0.8486778764412228, + "ndcg10": 0.8486778764412228, + "p50_ms": 0.469812483061105, + "p90_ms": 2.6101124938577414, + "p95_ms": 2.621154219377786, + "p99_ms": 2.688098077196628, + "index_ms": 44.014332990627736, + "by_category": { + "architecture": 0.8476276829510184, + "semantic": 0.8493780054346924 + } + }, + { + "repo": "xmonad", + "language": "haskell", + "mode": "semble-balanced", + "chunks": 241, + "tokens": 1695, + "ndcg5": 0.8486778764412228, + "ndcg10": 0.8486778764412228, + "p50_ms": 0.4678125260397792, + "p90_ms": 2.55977496271953, + "p95_ms": 2.5989958288846537, + "p99_ms": 2.644532783306204, + "index_ms": 44.014332990627736, + "by_category": { + "architecture": 0.8476276829510184, + "semantic": 0.8493780054346924 + } + }, + { + "repo": "xmonad", + "language": "haskell", + "mode": "semble-bm25", + "chunks": 241, + "tokens": 1682, + "ndcg5": 0.7833046797020383, + "ndcg10": 0.7833046797020383, + "p50_ms": 0.3964375355280936, + "p90_ms": 1.9518749613780528, + "p95_ms": 1.9601833715569228, + "p99_ms": 1.9667702948208898, + "index_ms": 44.014332990627736, + "by_category": { + "architecture": 0.771726340540315, + "semantic": 0.7910235724765203 + } + }, + { + "repo": "xmonad", + "language": "haskell", + "mode": "semble-semantic", + "chunks": 241, + "tokens": 1624, + "ndcg5": 0.9252968226739917, + "ndcg10": 0.9355159425534157, + "p50_ms": 0.37824950413778424, + "p90_ms": 2.1070871967822313, + "p95_ms": 2.1718896430684254, + "p99_ms": 2.2671115113189444, + "index_ms": 44.014332990627736, + "by_category": { + "architecture": 0.9310574179906753, + "semantic": 0.9384882922619097 + } + }, + { + "repo": "xmonad", + "language": "haskell", + "mode": "unranked-auto", + "chunks": 241, + "tokens": 1676, + "ndcg5": 0.7026944590681548, + "ndcg10": 0.7171477003840491, + "p50_ms": 0.2579584834165871, + "p90_ms": 0.2750958257820457, + "p95_ms": 0.2794017840642482, + "p99_ms": 0.28044676990248263, + "index_ms": 44.014332990627736, + "by_category": { + "architecture": 0.7916110721445924, + "semantic": 0.6675054525436869 + } + }, + { + "repo": "xmonad", + "language": "haskell", + "mode": "unranked-balanced", + "chunks": 241, + "tokens": 1676, + "ndcg5": 0.7026944590681548, + "ndcg10": 0.7171477003840491, + "p50_ms": 0.26175001403316855, + "p90_ms": 0.27860409463755786, + "p95_ms": 0.28121189097873867, + "p99_ms": 0.28950957930646837, + "index_ms": 44.014332990627736, + "by_category": { + "architecture": 0.7916110721445924, + "semantic": 0.6675054525436869 + } + }, + { + "repo": "xmonad", + "language": "haskell", + "mode": "unranked-bm25", + "chunks": 241, + "tokens": 1680, + "ndcg5": 0.6215281498878024, + "ndcg10": 0.6628300247329578, + "p50_ms": 0.25581251247785985, + "p90_ms": 0.2732953173108399, + "p95_ms": 0.2756193949608132, + "p99_ms": 0.276157459593378, + "index_ms": 44.014332990627736, + "by_category": { + "architecture": 0.7151294216308947, + "semantic": 0.6279637601343332 + } + }, + { + "repo": "xmonad", + "language": "haskell", + "mode": "unranked-semantic", + "chunks": 241, + "tokens": 1653, + "ndcg5": 0.6707350399282392, + "ndcg10": 0.7024532063781049, + "p50_ms": 0.25539600756019354, + "p90_ms": 0.2785165735986084, + "p95_ms": 0.2790160302538425, + "p99_ms": 0.2843359822873026, + "index_ms": 44.014332990627736, + "by_category": { + "architecture": 0.6670743896594975, + "semantic": 0.7260390841905099 + } + }, + { + "repo": "zig", + "language": "zig", + "mode": "semble-auto", + "chunks": 26017, + "tokens": 1484, + "ndcg5": 0.8880929753571458, + "ndcg10": 0.8880929753571458, + "p50_ms": 1.2695624900516123, + "p90_ms": 13.261537818470972, + "p95_ms": 14.836283356999052, + "p99_ms": 20.03139028383884, + "index_ms": 3904.361000051722, + "by_category": { + "architecture": 0.7103099178571526, + "semantic": 0.9194664560924387 + } + }, + { + "repo": "zig", + "language": "zig", + "mode": "semble-balanced", + "chunks": 26017, + "tokens": 1484, + "ndcg5": 0.8880929753571458, + "ndcg10": 0.8880929753571458, + "p50_ms": 1.2242920056451112, + "p90_ms": 13.033599726622926, + "p95_ms": 14.74475827417337, + "p99_ms": 19.838784475577988, + "index_ms": 3904.361000051722, + "by_category": { + "architecture": 0.7103099178571526, + "semantic": 0.9194664560924387 + } + }, + { + "repo": "zig", + "language": "zig", + "mode": "semble-bm25", + "chunks": 26017, + "tokens": 1591, + "ndcg5": 0.8846268032608154, + "ndcg10": 0.8846268032608154, + "p50_ms": 1.01583352079615, + "p90_ms": 11.856687196996065, + "p95_ms": 12.94111041061115, + "p99_ms": 18.77388847002293, + "index_ms": 3904.361000051722, + "by_category": { + "architecture": 0.7103099178571526, + "semantic": 0.9153886065673442 + } + }, + { + "repo": "zig", + "language": "zig", + "mode": "semble-semantic", + "chunks": 26017, + "tokens": 1500, + "ndcg5": 0.7542662662965341, + "ndcg10": 0.7709329329632008, + "p50_ms": 1.848854502895847, + "p90_ms": 12.611729773925623, + "p95_ms": 16.59558156388813, + "p99_ms": 27.35208352445623, + "index_ms": 3904.361000051722, + "by_category": { + "architecture": 0.5872865023809717, + "semantic": 0.8033411265953587 + } + }, + { + "repo": "zig", + "language": "zig", + "mode": "unranked-auto", + "chunks": 26017, + "tokens": 1552, + "ndcg5": 0.7170624189796883, + "ndcg10": 0.7170624189796883, + "p50_ms": 1.237646007211879, + "p90_ms": 1.5404542209580543, + "p95_ms": 1.6611378174275162, + "p99_ms": 1.8772947881370778, + "index_ms": 3904.361000051722, + "by_category": { + "architecture": 0.37697658452381916, + "semantic": 0.7770775662366063 + } + }, + { + "repo": "zig", + "language": "zig", + "mode": "unranked-balanced", + "chunks": 26017, + "tokens": 1552, + "ndcg5": 0.7170624189796883, + "ndcg10": 0.7170624189796883, + "p50_ms": 1.292937493417412, + "p90_ms": 1.4592006336897612, + "p95_ms": 1.4729353948496282, + "p99_ms": 1.5631534787826238, + "index_ms": 3904.361000051722, + "by_category": { + "architecture": 0.37697658452381916, + "semantic": 0.7770775662366063 + } + }, + { + "repo": "zig", + "language": "zig", + "mode": "unranked-bm25", + "chunks": 26017, + "tokens": 1549, + "ndcg5": 0.7955285910760186, + "ndcg10": 0.8099818323919129, + "p50_ms": 1.4671875105705112, + "p90_ms": 1.6519160009920597, + "p95_ms": 1.6659618500852957, + "p99_ms": 1.6742587863700464, + "index_ms": 3904.361000051722, + "by_category": { + "architecture": 0.4356157957079623, + "semantic": 0.8760464271008455 + } + }, + { + "repo": "zig", + "language": "zig", + "mode": "unranked-semantic", + "chunks": 26017, + "tokens": 1511, + "ndcg5": 0.5894853749236579, + "ndcg10": 0.6052586187629444, + "p50_ms": 1.4346249809022993, + "p90_ms": 1.5622667036950588, + "p95_ms": 1.5807523828698322, + "p99_ms": 1.5858504822244868, + "index_ms": 3904.361000051722, + "by_category": { + "architecture": 0.47689218602446437, + "semantic": 0.6279115186579702 + } + }, + { + "repo": "zig-clap", + "language": "zig", + "mode": "semble-auto", + "chunks": 195, + "tokens": 1606, + "ndcg5": 0.9380929753571458, + "ndcg10": 0.9380929753571458, + "p50_ms": 0.5421875102911144, + "p90_ms": 2.476728794863448, + "p95_ms": 2.7971597475698218, + "p99_ms": 2.808464767294936, + "index_ms": 32.68433402990922, + "by_category": { + "architecture": 1.0, + "semantic": 0.9312144170634953 + } + }, + { + "repo": "zig-clap", + "language": "zig", + "mode": "semble-balanced", + "chunks": 195, + "tokens": 1606, + "ndcg5": 0.9380929753571458, + "ndcg10": 0.9380929753571458, + "p50_ms": 0.5207500071264803, + "p90_ms": 2.4292585963848983, + "p95_ms": 2.764203879632987, + "p99_ms": 2.7812407939927652, + "index_ms": 32.68433402990922, + "by_category": { + "architecture": 1.0, + "semantic": 0.9312144170634953 + } + }, + { + "repo": "zig-clap", + "language": "zig", + "mode": "semble-bm25", + "chunks": 195, + "tokens": 1635, + "ndcg5": 0.9346268032608155, + "ndcg10": 0.9346268032608155, + "p50_ms": 0.42400002712383866, + "p90_ms": 1.7774173058569431, + "p95_ms": 1.7966107290703803, + "p99_ms": 1.8375557532999665, + "index_ms": 32.68433402990922, + "by_category": { + "architecture": 1.0, + "semantic": 0.9273631147342394 + } + }, + { + "repo": "zig-clap", + "language": "zig", + "mode": "semble-semantic", + "chunks": 195, + "tokens": 1595, + "ndcg5": 0.9465338279036697, + "ndcg10": 0.9465338279036697, + "p50_ms": 0.42727100662887096, + "p90_ms": 1.8510164169128986, + "p95_ms": 1.945389885804616, + "p99_ms": 1.9696779985679314, + "index_ms": 32.68433402990922, + "by_category": { + "architecture": 1.0, + "semantic": 0.9405931421151885 + } + }, + { + "repo": "zig-clap", + "language": "zig", + "mode": "unranked-auto", + "chunks": 195, + "tokens": 1563, + "ndcg5": 0.6692536065216309, + "ndcg10": 0.7037306325436987, + "p50_ms": 0.26802101638168097, + "p90_ms": 0.2794672967866063, + "p95_ms": 0.27986905188299716, + "p99_ms": 0.28484023292548954, + "index_ms": 32.68433402990922, + "by_category": { + "architecture": 0.5654648767857288, + "semantic": 0.7190934942945842 + } + }, + { + "repo": "zig-clap", + "language": "zig", + "mode": "unranked-balanced", + "chunks": 195, + "tokens": 1563, + "ndcg5": 0.6692536065216309, + "ndcg10": 0.7037306325436987, + "p50_ms": 0.261478970060125, + "p90_ms": 0.27394156204536557, + "p95_ms": 0.27496182592585683, + "p99_ms": 0.2813587919808924, + "index_ms": 32.68433402990922, + "by_category": { + "architecture": 0.5654648767857288, + "semantic": 0.7190934942945842 + } + }, + { + "repo": "zig-clap", + "language": "zig", + "mode": "unranked-bm25", + "chunks": 195, + "tokens": 1629, + "ndcg5": 0.8380929753571458, + "ndcg10": 0.8547596420238124, + "p50_ms": 0.25733347865752876, + "p90_ms": 0.2683333295863122, + "p95_ms": 0.2750851272139698, + "p99_ms": 0.2808170218486339, + "index_ms": 32.68433402990922, + "by_category": { + "architecture": 1.0, + "semantic": 0.8386218244709027 + } + }, + { + "repo": "zig-clap", + "language": "zig", + "mode": "unranked-semantic", + "chunks": 195, + "tokens": 1605, + "ndcg5": 0.6770750787545915, + "ndcg10": 0.7078998223770769, + "p50_ms": 0.2542085130698979, + "p90_ms": 0.27404643478803337, + "p95_ms": 0.280483064125292, + "p99_ms": 0.2808638190617785, + "index_ms": 32.68433402990922, + "by_category": { + "architecture": 0.6577324383928644, + "semantic": 0.7134739761531006 + } + }, + { + "repo": "zls", + "language": "zig", + "mode": "semble-auto", + "chunks": 2617, + "tokens": 1548, + "ndcg5": 0.8380929753571458, + "ndcg10": 0.870954834495746, + "p50_ms": 0.5997920234221965, + "p90_ms": 0.6979336787480862, + "p95_ms": 1.0003651899751307, + "p99_ms": 4.415139405755321, + "index_ms": 315.31595799606293, + "by_category": { + "architecture": 0.8081767403960004, + "semantic": 0.9047584236263781 + } + }, + { + "repo": "zls", + "language": "zig", + "mode": "semble-balanced", + "chunks": 2617, + "tokens": 1548, + "ndcg5": 0.8380929753571458, + "ndcg10": 0.870954834495746, + "p50_ms": 0.5980625282973051, + "p90_ms": 0.683024973841384, + "p95_ms": 0.9297937504015894, + "p99_ms": 4.318158782552922, + "index_ms": 315.31595799606293, + "by_category": { + "architecture": 0.8081767403960004, + "semantic": 0.9047584236263781 + } + }, + { + "repo": "zls", + "language": "zig", + "mode": "semble-bm25", + "chunks": 2617, + "tokens": 1515, + "ndcg5": 0.825, + "ndcg10": 0.825, + "p50_ms": 0.4542914975900203, + "p90_ms": 0.5132711259648204, + "p95_ms": 0.716812186874452, + "p99_ms": 3.642495251260693, + "index_ms": 315.31595799606293, + "by_category": { + "architecture": 0.7142857142857143, + "semantic": 0.8846153846153846 + } + }, + { + "repo": "zls", + "language": "zig", + "mode": "semble-semantic", + "chunks": 2617, + "tokens": 1490, + "ndcg5": 0.8880929753571458, + "ndcg10": 0.8880929753571458, + "p50_ms": 0.46647951239719987, + "p90_ms": 0.4950378031935543, + "p95_ms": 0.733793794643137, + "p99_ms": 4.124058752786363, + "index_ms": 315.31595799606293, + "by_category": { + "architecture": 0.7857142857142857, + "semantic": 0.9432199620879166 + } + }, + { + "repo": "zls", + "language": "zig", + "mode": "unranked-auto", + "chunks": 2617, + "tokens": 1450, + "ndcg5": 0.5992012120245696, + "ndcg10": 0.6170115713799708, + "p50_ms": 0.2898545062635094, + "p90_ms": 0.3180955071002245, + "p95_ms": 0.32401012431364506, + "p99_ms": 0.3267020400380716, + "index_ms": 315.31595799606293, + "by_category": { + "architecture": 0.5552646867477916, + "semantic": 0.6502598938742209 + } + }, + { + "repo": "zls", + "language": "zig", + "mode": "unranked-balanced", + "chunks": 2617, + "tokens": 1450, + "ndcg5": 0.5992012120245696, + "ndcg10": 0.6170115713799708, + "p50_ms": 0.2868125156965107, + "p90_ms": 0.305299402680248, + "p95_ms": 0.30939587159082294, + "p99_ms": 0.31921277521178126, + "index_ms": 315.31595799606293, + "by_category": { + "architecture": 0.5552646867477916, + "semantic": 0.6502598938742209 + } + }, + { + "repo": "zls", + "language": "zig", + "mode": "unranked-bm25", + "chunks": 2617, + "tokens": 1460, + "ndcg5": 0.5955285910760186, + "ndcg10": 0.6133389504314197, + "p50_ms": 0.2872915065381676, + "p90_ms": 0.3090913232881576, + "p95_ms": 0.3136702085612342, + "p99_ms": 0.31373404373880476, + "index_ms": 315.31595799606293, + "by_category": { + "architecture": 0.5552646867477916, + "semantic": 0.6446097077995272 + } + }, + { + "repo": "zls", + "language": "zig", + "mode": "unranked-semantic", + "chunks": 2617, + "tokens": 1433, + "ndcg5": 0.6570497592047851, + "ndcg10": 0.6748601185601862, + "p50_ms": 0.2792919985949993, + "p90_ms": 0.2999664400704205, + "p95_ms": 0.304744066670537, + "p99_ms": 0.307815233245492, + "index_ms": 315.31595799606293, + "by_category": { + "architecture": 0.5615252225819133, + "semantic": 0.7358866010100253 + } + }, + { + "repo": "zod", + "language": "typescript", + "mode": "semble-auto", + "chunks": 3576, + "tokens": 1416, + "ndcg5": 0.599740727020101, + "ndcg10": 0.64272096242225, + "p50_ms": 2.5552705046720803, + "p90_ms": 3.7707750278059393, + "p95_ms": 3.9783354091923697, + "p99_ms": 5.283033450832589, + "index_ms": 565.7179160043597, + "by_category": { + "architecture": 0.5263150610131765, + "semantic": 0.6629407116016026, + "symbol": 0.7103099178571526 + } + }, + { + "repo": "zod", + "language": "typescript", + "mode": "semble-balanced", + "chunks": 3576, + "tokens": 1412, + "ndcg5": 0.6181942393415281, + "ndcg10": 0.6611744747436771, + "p50_ms": 2.5573955208528787, + "p90_ms": 3.6917493736837064, + "p95_ms": 3.988850020687097, + "p99_ms": 5.261470022960564, + "index_ms": 565.7179160043597, + "by_category": { + "architecture": 0.5263150610131765, + "semantic": 0.6629407116016026, + "symbol": 0.8333333333333334 + } + }, + { + "repo": "zod", + "language": "typescript", + "mode": "semble-bm25", + "chunks": 3576, + "tokens": 1430, + "ndcg5": 0.6517228436254816, + "ndcg10": 0.6626432154663898, + "p50_ms": 1.885500008938834, + "p90_ms": 2.678379084682092, + "p95_ms": 2.818926438339987, + "p99_ms": 3.757685276796108, + "index_ms": 565.7179160043597, + "by_category": { + "architecture": 0.5810890392971821, + "semantic": 0.6767367998898162, + "symbol": 0.7103099178571526 + } + }, + { + "repo": "zod", + "language": "typescript", + "mode": "semble-semantic", + "chunks": 3576, + "tokens": 1401, + "ndcg5": 0.39741162656171475, + "ndcg10": 0.4584299971096315, + "p50_ms": 1.8544165068306029, + "p90_ms": 2.7369665971491486, + "p95_ms": 2.8345243219519043, + "p99_ms": 3.7576712790178117, + "index_ms": 565.7179160043597, + "by_category": { + "architecture": 0.20968643164645015, + "semantic": 0.4821729156893829, + "symbol": 0.6872021038816168 + } + }, + { + "repo": "zod", + "language": "typescript", + "mode": "unranked-auto", + "chunks": 3576, + "tokens": 1436, + "ndcg5": 0.41710693589973635, + "ndcg10": 0.4441460800941159, + "p50_ms": 0.32564601860940456, + "p90_ms": 0.3599505871534348, + "p95_ms": 0.37942298222333193, + "p99_ms": 0.38585179951041937, + "index_ms": 565.7179160043597, + "by_category": { + "architecture": 0.41569519587268783, + "semantic": 0.3914777742169315, + "symbol": 0.7103099178571526 + } + }, + { + "repo": "zod", + "language": "typescript", + "mode": "unranked-balanced", + "chunks": 3576, + "tokens": 1430, + "ndcg5": 0.40519991125688215, + "ndcg10": 0.4322390554512617, + "p50_ms": 0.31556200701743364, + "p90_ms": 0.3829660068731755, + "p95_ms": 0.384276409749873, + "p99_ms": 0.39735527767334133, + "index_ms": 565.7179160043597, + "by_category": { + "architecture": 0.41569519587268783, + "semantic": 0.3914777742169315, + "symbol": 0.6309297535714575 + } + }, + { + "repo": "zod", + "language": "typescript", + "mode": "unranked-bm25", + "chunks": 3576, + "tokens": 1469, + "ndcg5": 0.39618418820942136, + "ndcg10": 0.431018024172731, + "p50_ms": 0.31520798802375793, + "p90_ms": 0.358679483179003, + "p95_ms": 0.3726899070898071, + "p99_ms": 0.37873796944040805, + "index_ms": 565.7179160043597, + "by_category": { + "architecture": 0.4077324383928644, + "semantic": 0.3824344745443972, + "symbol": 0.6725941869353331 + } + }, + { + "repo": "zod", + "language": "typescript", + "mode": "unranked-semantic", + "chunks": 3576, + "tokens": 1434, + "ndcg5": 0.268065033722998, + "ndcg10": 0.31140402408426815, + "p50_ms": 0.31764552113600075, + "p90_ms": 0.36839612293988466, + "p95_ms": 0.38568092277273536, + "p99_ms": 0.39476896403357387, + "index_ms": 565.7179160043597, + "by_category": { + "architecture": 0.09671320180863541, + "semantic": 0.3590377786180665, + "symbol": 0.3912455174719856 + } + } + ] +} diff --git a/benchmarks/results/semble-hybrid-5fbcd035d594.json b/benchmarks/results/semble-hybrid-5fbcd035d594.json new file mode 100644 index 00000000..bbaf6788 --- /dev/null +++ b/benchmarks/results/semble-hybrid-5fbcd035d594.json @@ -0,0 +1,1396 @@ +{ + "tool": "semble-hybrid", + "model": "minishlab/potion-code-16M-v2", + "summary": { + "ndcg10": 0.8546, + "tokens": 1779.0, + "p50_ms": 1.041, + "p90_ms": 3.628, + "p95_ms": 4.313, + "p99_ms": 5.207, + "index_ms": 479.8, + "by_category": { + "architecture": 0.8076, + "semantic": 0.8466, + "symbol": 0.9537 + } + }, + "by_language": { + "bash": { + "repos": 3, + "tokens": 1490.0, + "ndcg10": 0.8396, + "p50_ms": 0.606, + "p90_ms": 0.705, + "p95_ms": 0.723, + "p99_ms": 0.753, + "index_ms": 119.6 + }, + "c": { + "repos": 3, + "tokens": 1615.0, + "ndcg10": 0.7504, + "p50_ms": 0.738, + "p90_ms": 0.848, + "p95_ms": 0.881, + "p99_ms": 0.921, + "index_ms": 1251.5 + }, + "cpp": { + "repos": 3, + "tokens": 1530.0, + "ndcg10": 0.8814, + "p50_ms": 0.969, + "p90_ms": 6.049, + "p95_ms": 6.806, + "p99_ms": 7.198, + "index_ms": 1599.5 + }, + "csharp": { + "repos": 3, + "tokens": 1418.0, + "ndcg10": 0.8715, + "p50_ms": 3.279, + "p90_ms": 4.864, + "p95_ms": 5.454, + "p99_ms": 7.171, + "index_ms": 432.8 + }, + "elixir": { + "repos": 3, + "tokens": 4421.0, + "ndcg10": 0.9047, + "p50_ms": 0.799, + "p90_ms": 4.606, + "p95_ms": 5.651, + "p99_ms": 5.991, + "index_ms": 571.3 + }, + "go": { + "repos": 3, + "tokens": 1836.0, + "ndcg10": 0.9103, + "p50_ms": 0.604, + "p90_ms": 2.852, + "p95_ms": 3.24, + "p99_ms": 3.909, + "index_ms": 149.7 + }, + "haskell": { + "repos": 3, + "tokens": 1693.0, + "ndcg10": 0.7754, + "p50_ms": 1.665, + "p90_ms": 6.133, + "p95_ms": 8.398, + "p99_ms": 10.746, + "index_ms": 464.4 + }, + "java": { + "repos": 3, + "tokens": 1727.0, + "ndcg10": 0.8278, + "p50_ms": 0.942, + "p90_ms": 7.184, + "p95_ms": 10.226, + "p99_ms": 11.493, + "index_ms": 822.4 + }, + "javascript": { + "repos": 3, + "tokens": 1535.0, + "ndcg10": 0.9111, + "p50_ms": 0.572, + "p90_ms": 1.662, + "p95_ms": 1.777, + "p99_ms": 2.536, + "index_ms": 33.3 + }, + "kotlin": { + "repos": 3, + "tokens": 2744.0, + "ndcg10": 0.8026, + "p50_ms": 1.309, + "p90_ms": 5.664, + "p95_ms": 6.319, + "p99_ms": 7.148, + "index_ms": 255.9 + }, + "lua": { + "repos": 3, + "tokens": 1657.0, + "ndcg10": 0.8354, + "p50_ms": 0.869, + "p90_ms": 1.04, + "p95_ms": 1.282, + "p99_ms": 1.744, + "index_ms": 376.5 + }, + "php": { + "repos": 3, + "tokens": 1457.0, + "ndcg10": 0.8638, + "p50_ms": 0.889, + "p90_ms": 3.778, + "p95_ms": 3.862, + "p99_ms": 4.171, + "index_ms": 646.1 + }, + "python": { + "repos": 9, + "tokens": 1669.0, + "ndcg10": 0.8759, + "p50_ms": 0.596, + "p90_ms": 2.607, + "p95_ms": 3.154, + "p99_ms": 3.811, + "index_ms": 149.4 + }, + "ruby": { + "repos": 3, + "tokens": 1579.0, + "ndcg10": 0.9086, + "p50_ms": 0.59, + "p90_ms": 3.49, + "p95_ms": 4.058, + "p99_ms": 5.11, + "index_ms": 110.8 + }, + "rust": { + "repos": 3, + "tokens": 1679.0, + "ndcg10": 0.8263, + "p50_ms": 0.774, + "p90_ms": 3.417, + "p95_ms": 4.039, + "p99_ms": 4.918, + "index_ms": 481.3 + }, + "scala": { + "repos": 3, + "tokens": 1488.0, + "ndcg10": 0.9213, + "p50_ms": 1.51, + "p90_ms": 4.015, + "p95_ms": 4.44, + "p99_ms": 5.15, + "index_ms": 369.5 + }, + "swift": { + "repos": 3, + "tokens": 1465.0, + "ndcg10": 0.8478, + "p50_ms": 1.382, + "p90_ms": 3.308, + "p95_ms": 4.096, + "p99_ms": 5.054, + "index_ms": 176.1 + }, + "typescript": { + "repos": 3, + "tokens": 1470.0, + "ndcg10": 0.742, + "p50_ms": 1.767, + "p90_ms": 3.278, + "p95_ms": 3.589, + "p99_ms": 4.795, + "index_ms": 340.9 + }, + "zig": { + "repos": 3, + "tokens": 1546.0, + "ndcg10": 0.899, + "p50_ms": 0.804, + "p90_ms": 5.472, + "p95_ms": 6.264, + "p99_ms": 9.1, + "index_ms": 1426.2 + } + }, + "repos": [ + { + "repo": "abseil-cpp", + "language": "cpp", + "mode": "auto", + "chunks": 16813, + "tokens": 1502, + "ndcg5": 0.8621567083124564, + "ndcg10": 0.8799670676678575, + "p50_ms": 1.7526880255900323, + "p90_ms": 13.881383393891157, + "p95_ms": 14.589813144993968, + "p99_ms": 15.301995413028633, + "index_ms": 4207.161833008286, + "by_category": { + "architecture": 1.0, + "semantic": 0.8732894235571433, + "symbol": 0.8333333333333334 + } + }, + { + "repo": "aeson", + "language": "haskell", + "mode": "auto", + "chunks": 788, + "tokens": 1990, + "ndcg5": 0.7586215664331644, + "ndcg10": 0.768292886614028, + "p50_ms": 3.663708019303158, + "p90_ms": 8.337291935458786, + "p95_ms": 13.823436852544551, + "p99_ms": 20.433853762224306, + "index_ms": 204.5704170013778, + "by_category": { + "architecture": 0.6297424628754914, + "semantic": 0.81447636119354 + } + }, + { + "repo": "aiohttp", + "language": "python", + "mode": "auto", + "chunks": 1469, + "tokens": 1565, + "ndcg5": 0.8223865779223699, + "ndcg10": 0.8550948458996863, + "p50_ms": 0.5794999888166785, + "p90_ms": 2.9695829725824296, + "p95_ms": 3.466750029474497, + "p99_ms": 5.741750029847028, + "index_ms": 255.20258396863937, + "by_category": { + "architecture": 0.8634033929769788, + "semantic": 0.7681297792282846, + "symbol": 1.0 + } + }, + { + "repo": "alamofire", + "language": "swift", + "mode": "auto", + "chunks": 1302, + "tokens": 1533, + "ndcg5": 0.9775325271359823, + "ndcg10": 0.9775325271359823, + "p50_ms": 0.6334164936561137, + "p90_ms": 2.959575294516982, + "p95_ms": 3.9210437447763993, + "p99_ms": 5.261208752635864, + "index_ms": 240.66512496210635, + "by_category": { + "architecture": 1.0, + "semantic": 0.9591500493381495, + "symbol": 1.0 + } + }, + { + "repo": "axios", + "language": "javascript", + "mode": "auto", + "chunks": 299, + "tokens": 1395, + "ndcg5": 0.855657359638273, + "ndcg10": 0.866577731479181, + "p50_ms": 0.5914374778512865, + "p90_ms": 2.0009624888189137, + "p95_ms": 2.064710351987742, + "p99_ms": 2.1154084446607158, + "index_ms": 50.15324999112636, + "by_category": { + "architecture": 0.6276256451932702, + "semantic": 0.91038071151303, + "symbol": 1.0 + } + }, + { + "repo": "axum", + "language": "rust", + "mode": "auto", + "chunks": 1034, + "tokens": 1657, + "ndcg5": 0.7815464876785729, + "ndcg10": 0.8151300908732605, + "p50_ms": 0.7179584936238825, + "p90_ms": 3.0852708034217358, + "p95_ms": 3.2074392598588033, + "p99_ms": 4.480154240736736, + "index_ms": 162.87987504620105, + "by_category": { + "architecture": 0.8, + "semantic": 0.8288662191964322, + "symbol": 0.7854023957026741 + } + }, + { + "repo": "bash-it", + "language": "bash", + "mode": "auto", + "chunks": 1319, + "tokens": 1344, + "ndcg5": 0.6056128427182248, + "ndcg10": 0.6525424269728923, + "p50_ms": 0.7743125024717301, + "p90_ms": 0.9397041110787541, + "p95_ms": 0.9524014283670112, + "p99_ms": 0.9635802818229422, + "index_ms": 273.1900000362657, + "by_category": { + "architecture": 0.8154648767857288, + "semantic": 0.6044855749416566, + "symbol": 0.7123212623289701 + } + }, + { + "repo": "bats-core", + "language": "bash", + "mode": "auto", + "chunks": 97, + "tokens": 1505, + "ndcg5": 0.8661732909393883, + "ndcg10": 0.8661732909393883, + "p50_ms": 0.5432705220300704, + "p90_ms": 0.6024329864885658, + "p95_ms": 0.6189205741975458, + "p99_ms": 0.6848512974102049, + "index_ms": 17.051709000952542, + "by_category": { + "architecture": 0.7230866159492644, + "semantic": 0.9432199620879166 + } + }, + { + "repo": "cats", + "language": "scala", + "mode": "auto", + "chunks": 2393, + "tokens": 1478, + "ndcg5": 0.9113147192765458, + "ndcg10": 0.9113147192765458, + "p50_ms": 1.3327919878065586, + "p90_ms": 5.130912212189287, + "p95_ms": 5.44916877988726, + "p99_ms": 6.53093376662582, + "index_ms": 643.6321250512265, + "by_category": { + "architecture": 0.8065735963827292, + "semantic": 0.9, + "symbol": 1.0 + } + }, + { + "repo": "chi", + "language": "go", + "mode": "auto", + "chunks": 508, + "tokens": 1436, + "ndcg5": 0.8243610443449215, + "ndcg10": 0.8699498574344997, + "p50_ms": 0.6651250005234033, + "p90_ms": 2.168428699951619, + "p95_ms": 2.288968756329268, + "p99_ms": 2.3143273289315403, + "index_ms": 89.88929202314466, + "by_category": { + "architecture": 0.7546946133749226, + "semantic": 0.9444444444444444, + "symbol": 0.8769765845238192 + } + }, + { + "repo": "circe", + "language": "scala", + "mode": "auto", + "chunks": 379, + "tokens": 1505, + "ndcg5": 0.8979792795602554, + "ndcg10": 0.8979792795602554, + "p50_ms": 0.6027499912306666, + "p90_ms": 3.3372414298355575, + "p95_ms": 3.960562491556629, + "p99_ms": 4.059112508548424, + "index_ms": 85.14983300119638, + "by_category": { + "architecture": 0.7261859507142916, + "semantic": 0.9482433234612176, + "symbol": 1.0 + } + }, + { + "repo": "click", + "language": "python", + "mode": "auto", + "chunks": 610, + "tokens": 1884, + "ndcg5": 1.0, + "ndcg10": 1.0, + "p50_ms": 0.5098544934298843, + "p90_ms": 2.3656041943468153, + "p95_ms": 2.4095919856335968, + "p99_ms": 2.516752005321905, + "index_ms": 90.06562503054738, + "by_category": { + "architecture": 1.0, + "semantic": 1.0, + "symbol": 1.0 + } + }, + { + "repo": "cobra", + "language": "go", + "mode": "auto", + "chunks": 780, + "tokens": 2530, + "ndcg5": 0.9709860394574094, + "ndcg10": 0.9709860394574094, + "p50_ms": 0.5635619745589793, + "p90_ms": 4.03701193863526, + "p95_ms": 4.373191643389875, + "p99_ms": 5.844804695225318, + "index_ms": 135.8876249869354, + "by_category": { + "architecture": 1.0, + "semantic": 0.947247344468017, + "symbol": 1.0 + } + }, + { + "repo": "commons-lang", + "language": "java", + "mode": "auto", + "chunks": 6049, + "tokens": 1591, + "ndcg5": 0.8511907952337322, + "ndcg10": 0.8829368269797638, + "p50_ms": 0.7947499980218709, + "p90_ms": 10.509624960832298, + "p95_ms": 17.51479197992012, + "p99_ms": 18.58152480563149, + "index_ms": 822.6494999835268, + "by_category": { + "architecture": 0.8710490642551528, + "semantic": 0.8520375838435416, + "symbol": 1.0 + } + }, + { + "repo": "curl", + "language": "c", + "mode": "auto", + "chunks": 8901, + "tokens": 1519, + "ndcg5": 0.6504072534454753, + "ndcg10": 0.7051303483604954, + "p50_ms": 0.8543125004507601, + "p90_ms": 1.015158073278144, + "p95_ms": 1.0407214344013482, + "p99_ms": 1.0770106932613999, + "index_ms": 1327.4122920120135, + "by_category": { + "architecture": 0.6284680374280802, + "semantic": 0.7678540573051987 + } + }, + { + "repo": "dapper", + "language": "csharp", + "mode": "auto", + "chunks": 798, + "tokens": 1344, + "ndcg5": 0.8074356157188728, + "ndcg10": 0.8419126417409405, + "p50_ms": 2.360020502237603, + "p90_ms": 3.6472162755671906, + "p95_ms": 5.068613949697468, + "p99_ms": 8.751922806259238, + "index_ms": 155.168124998454, + "by_category": { + "architecture": 0.5877789735348331, + "semantic": 0.8836259185138062, + "symbol": 1.0 + } + }, + { + "repo": "ecto", + "language": "elixir", + "mode": "auto", + "chunks": 1431, + "tokens": 3662, + "ndcg5": 0.9025638060198661, + "ndcg10": 0.9025638060198661, + "p50_ms": 1.1147920158691704, + "p90_ms": 6.514775217510759, + "p95_ms": 7.448228873545303, + "p99_ms": 7.635579371126369, + "index_ms": 412.6155839767307, + "by_category": { + "architecture": 1.0, + "semantic": 0.8960547934136506, + "symbol": 0.8333333333333334 + } + }, + { + "repo": "exposed", + "language": "kotlin", + "mode": "auto", + "chunks": 1408, + "tokens": 1644, + "ndcg5": 0.5893422178385317, + "ndcg10": 0.6477744920631955, + "p50_ms": 1.6316460096277297, + "p90_ms": 6.7635625367984185, + "p95_ms": 7.309789565624669, + "p99_ms": 7.754991488764062, + "index_ms": 377.5974999880418, + "by_category": { + "architecture": 0.42657598789683987, + "semantic": 0.6196972769695278, + "symbol": 1.0 + } + }, + { + "repo": "express", + "language": "javascript", + "mode": "auto", + "chunks": 102, + "tokens": 1528, + "ndcg5": 0.9382034061286294, + "ndcg10": 0.9491237779695378, + "p50_ms": 0.663729035295546, + "p90_ms": 0.8905459253583103, + "p95_ms": 0.9586714761098856, + "p99_ms": 1.9195006525842457, + "index_ms": 21.771583007648587, + "by_category": { + "architecture": 0.9260679370558221, + "semantic": 0.95, + "symbol": 1.0 + } + }, + { + "repo": "fastapi", + "language": "python", + "mode": "auto", + "chunks": 1188, + "tokens": 1584, + "ndcg5": 0.7136739118980615, + "ndcg10": 0.7840277229562979, + "p50_ms": 0.5334164889063686, + "p90_ms": 2.254699741024524, + "p95_ms": 2.328877107356676, + "p99_ms": 3.2010090077528717, + "index_ms": 154.194499948062, + "by_category": { + "architecture": 0.7310666719569522, + "semantic": 0.6953859694284116, + "symbol": 1.0 + } + }, + { + "repo": "flask", + "language": "python", + "mode": "auto", + "chunks": 558, + "tokens": 1834, + "ndcg5": 0.859874490822509, + "ndcg10": 0.8878534827452375, + "p50_ms": 0.4380409955047071, + "p90_ms": 1.713958045002073, + "p95_ms": 2.7712919982150197, + "p99_ms": 2.7855911990627646, + "index_ms": 78.47445900551975, + "by_category": { + "architecture": 0.8304380615499182, + "semantic": 0.9289735921850091, + "symbol": 0.8710490642551528 + } + }, + { + "repo": "fmtlib", + "language": "cpp", + "mode": "auto", + "chunks": 966, + "tokens": 1392, + "ndcg5": 0.9161732909393884, + "ndcg10": 0.9161732909393884, + "p50_ms": 0.48914598301053047, + "p90_ms": 0.7529535912908637, + "p95_ms": 2.260754516464658, + "p99_ms": 2.3627844970906153, + "index_ms": 334.8659170442261, + "by_category": { + "architecture": 0.8769765845238192, + "semantic": 0.9329718794032037, + "symbol": 0.8769765845238192 + } + }, + { + "repo": "gin", + "language": "go", + "mode": "auto", + "chunks": 1171, + "tokens": 1541, + "ndcg5": 0.8740805312254514, + "ndcg10": 0.8898537750647378, + "p50_ms": 0.5839170189574361, + "p90_ms": 2.3495130939409146, + "p95_ms": 3.0585916334530343, + "p99_ms": 3.569184711086563, + "index_ms": 223.218499973882, + "by_category": { + "architecture": 0.8729222796953519, + "semantic": 0.8690492566475132, + "symbol": 1.0 + } + }, + { + "repo": "gson", + "language": "java", + "mode": "auto", + "chunks": 2881, + "tokens": 2139, + "ndcg5": 0.8605604482211635, + "ndcg10": 0.8920725120838435, + "p50_ms": 1.0307710035704076, + "p90_ms": 4.743503878125923, + "p95_ms": 5.786673241527752, + "p99_ms": 7.016068246448411, + "index_ms": 458.01270799711347, + "by_category": { + "architecture": 0.6776301220263528, + "semantic": 0.9130929753571457, + "symbol": 1.0 + } + }, + { + "repo": "guzzle", + "language": "php", + "mode": "auto", + "chunks": 386, + "tokens": 1526, + "ndcg5": 0.9232405187664462, + "ndcg10": 0.9232405187664462, + "p50_ms": 0.5680625326931477, + "p90_ms": 2.135829790495336, + "p95_ms": 2.2471295204013586, + "p99_ms": 2.3662594985216856, + "index_ms": 74.19937499798834, + "by_category": { + "architecture": 1.0, + "semantic": 0.8819084904099174, + "symbol": 1.0 + } + }, + { + "repo": "http4s", + "language": "scala", + "mode": "auto", + "chunks": 1849, + "tokens": 1482, + "ndcg5": 0.9437503349954188, + "ndcg10": 0.9546707068363268, + "p50_ms": 2.5930414849426597, + "p90_ms": 3.5755964054260407, + "p95_ms": 3.911455927300268, + "p99_ms": 4.8595247924095, + "index_ms": 379.65545896440744, + "by_category": { + "architecture": 0.9438515431945408, + "semantic": 0.9472756790816368, + "symbol": 1.0 + } + }, + { + "repo": "httpx", + "language": "python", + "mode": "auto", + "chunks": 488, + "tokens": 1558, + "ndcg5": 0.8955062293473781, + "ndcg10": 0.9042955482416433, + "p50_ms": 0.4811669932678342, + "p90_ms": 2.2815829725004733, + "p95_ms": 2.402417012490332, + "p99_ms": 2.4541834136471152, + "index_ms": 65.8357500215061, + "by_category": { + "architecture": 0.8795597859882119, + "semantic": 0.9113731397317297, + "symbol": 0.9261859507142916 + } + }, + { + "repo": "jackson-databind", + "language": "java", + "mode": "auto", + "chunks": 8975, + "tokens": 1451, + "ndcg5": 0.6688795406139487, + "ndcg10": 0.7085306436511664, + "p50_ms": 1.000250515062362, + "p90_ms": 6.300116574857386, + "p95_ms": 7.375599376973698, + "p99_ms": 8.881286257528698, + "index_ms": 1186.53941701632, + "by_category": { + "architecture": 0.666564092985102, + "semantic": 0.5511752325288629, + "symbol": 0.9795358858896749 + } + }, + { + "repo": "kotlinx-coroutines", + "language": "kotlin", + "mode": "auto", + "chunks": 1569, + "tokens": 4462, + "ndcg5": 0.9074356157188728, + "ndcg10": 0.9074356157188728, + "p50_ms": 1.5668539854232222, + "p90_ms": 6.381162232719363, + "p95_ms": 7.307158369803803, + "p99_ms": 9.11076526273973, + "index_ms": 259.2735840007663, + "by_category": { + "architecture": 0.8333333333333334, + "semantic": 0.9034794510269613, + "symbol": 1.0 + } + }, + { + "repo": "ktor", + "language": "kotlin", + "mode": "auto", + "chunks": 761, + "tokens": 2127, + "ndcg5": 0.8281680939691519, + "ndcg10": 0.8524483785913294, + "p50_ms": 0.7287914922926575, + "p90_ms": 3.846758318832145, + "p95_ms": 4.3394767883000895, + "p99_ms": 4.57726176304277, + "index_ms": 130.96566597232595, + "by_category": { + "architecture": 0.7987170106064461, + "semantic": 0.7797069282259236, + "symbol": 1.0 + } + }, + { + "repo": "laravel-framework", + "language": "php", + "mode": "auto", + "chunks": 11681, + "tokens": 1402, + "ndcg5": 0.7625639617579317, + "ndcg10": 0.7880925927638007, + "p50_ms": 0.8507290040142834, + "p90_ms": 5.549595801858231, + "p95_ms": 5.636016343487427, + "p99_ms": 5.97890326171182, + "index_ms": 1678.6730839521624, + "by_category": { + "architecture": 0.7280655251168765, + "semantic": 0.7816070473207135, + "symbol": 0.9077324383928644 + } + }, + { + "repo": "lazy.nvim", + "language": "lua", + "mode": "auto", + "chunks": 596, + "tokens": 1547, + "ndcg5": 0.744298123441679, + "ndcg10": 0.7621084827970801, + "p50_ms": 0.5817295168526471, + "p90_ms": 0.630675331922248, + "p95_ms": 0.6502062227809802, + "p99_ms": 0.6552412390010431, + "index_ms": 103.8897079997696, + "by_category": { + "architecture": 0.8382026389820946, + "semantic": 0.6998496277366136 + } + }, + { + "repo": "libuv", + "language": "c", + "mode": "auto", + "chunks": 2638, + "tokens": 1541, + "ndcg5": 0.5674229672679589, + "ndcg10": 0.623763115451377, + "p50_ms": 0.5790619761683047, + "p90_ms": 0.6679451675154269, + "p95_ms": 0.6775225745514036, + "p99_ms": 0.7507044961676, + "index_ms": 402.6727920281701, + "by_category": { + "architecture": 0.5, + "semantic": 0.6097367949459744, + "symbol": 1.0 + } + }, + { + "repo": "messagepack-csharp", + "language": "csharp", + "mode": "auto", + "chunks": 2179, + "tokens": 1478, + "ndcg5": 0.8810595925213971, + "ndcg10": 0.8810595925213971, + "p50_ms": 2.3172505025286227, + "p90_ms": 3.6841539025772367, + "p95_ms": 3.9044732780894265, + "p99_ms": 4.097228219616227, + "index_ms": 409.44316698005423, + "by_category": { + "architecture": 0.7098603945740938, + "semantic": 0.8646389191257297, + "symbol": 1.0 + } + }, + { + "repo": "mini.nvim", + "language": "lua", + "mode": "auto", + "chunks": 4381, + "tokens": 1771, + "ndcg5": 0.9815464876785729, + "ndcg10": 0.9815464876785729, + "p50_ms": 0.9321670222561806, + "p90_ms": 1.1523789202328771, + "p95_ms": 1.1881648882990705, + "p99_ms": 1.3154329889221115, + "index_ms": 814.8073750198819, + "by_category": { + "architecture": 1.0, + "semantic": 0.9769331095982161 + } + }, + { + "repo": "model2vec", + "language": "python", + "mode": "auto", + "chunks": 203, + "tokens": 1629, + "ndcg5": 0.7164400093840788, + "ndcg10": 0.7375795011044112, + "p50_ms": 0.5062084819655865, + "p90_ms": 2.5323625130113214, + "p95_ms": 2.5463957950705662, + "p99_ms": 2.7443119633244346, + "index_ms": 32.80008299043402, + "by_category": { + "architecture": 0.6905456754532798, + "semantic": 0.7286840540343809, + "symbol": 0.8769765845238192 + } + }, + { + "repo": "monolog", + "language": "php", + "mode": "auto", + "chunks": 774, + "tokens": 1443, + "ndcg5": 0.860832844681721, + "ndcg10": 0.8801754850434484, + "p50_ms": 1.2487914937082678, + "p90_ms": 3.648586879717186, + "p95_ms": 3.7043083313619722, + "p99_ms": 4.1687944502336896, + "index_ms": 185.3044999879785, + "by_category": { + "architecture": 0.6980725016207014, + "semantic": 0.9113147192765458, + "symbol": 1.0 + } + }, + { + "repo": "newtonsoft-json", + "language": "csharp", + "mode": "auto", + "chunks": 4297, + "tokens": 1433, + "ndcg5": 0.8590790148145551, + "ndcg10": 0.8915189253205081, + "p50_ms": 5.16056249034591, + "p90_ms": 7.2598044003825635, + "p95_ms": 7.387713954085485, + "p99_ms": 8.663342791842295, + "index_ms": 733.9182089781389, + "by_category": { + "architecture": 0.9251084237866075, + "semantic": 0.8436116330627744, + "symbol": 1.0 + } + }, + { + "repo": "nlohmann-json", + "language": "cpp", + "mode": "auto", + "chunks": 1599, + "tokens": 1696, + "ndcg5": 0.8481244099790952, + "ndcg10": 0.8481244099790952, + "p50_ms": 0.6665000109933317, + "p90_ms": 3.511941694887355, + "p95_ms": 3.5688940319232647, + "p99_ms": 3.929545178543776, + "index_ms": 256.57229201169685, + "by_category": { + "architecture": 0.7769670264708678, + "semantic": 0.8523719014285831, + "symbol": 1.0 + } + }, + { + "repo": "nvm", + "language": "bash", + "mode": "auto", + "chunks": 309, + "tokens": 1620, + "ndcg5": 1.0, + "ndcg10": 1.0, + "p50_ms": 0.4998125077690929, + "p90_ms": 0.5733835801947862, + "p95_ms": 0.5965954973362386, + "p99_ms": 0.6119854818098247, + "index_ms": 68.47487500635907, + "by_category": { + "architecture": 1.0, + "semantic": 1.0 + } + }, + { + "repo": "pandoc", + "language": "haskell", + "mode": "auto", + "chunks": 6178, + "tokens": 1393, + "ndcg5": 0.6773719014285831, + "ndcg10": 0.7090900678784489, + "p50_ms": 0.8531459898222238, + "p90_ms": 7.523442286765205, + "p95_ms": 8.806319054565392, + "p99_ms": 9.13923021114897, + "index_ms": 1145.9452910348773, + "by_category": { + "architecture": 0.6696315434523955, + "semantic": 0.735395750829151 + } + }, + { + "repo": "phoenix", + "language": "elixir", + "mode": "auto", + "chunks": 6865, + "tokens": 5361, + "ndcg5": 0.8854455694098825, + "ndcg10": 0.8854455694098825, + "p50_ms": 0.781457987613976, + "p90_ms": 4.610308003611862, + "p95_ms": 6.045558111509308, + "p99_ms": 6.116478861076757, + "index_ms": 1164.4802920054644, + "by_category": { + "architecture": 0.8624288341269906, + "semantic": 0.8659437588064073, + "symbol": 1.0 + } + }, + { + "repo": "plug", + "language": "elixir", + "mode": "auto", + "chunks": 507, + "tokens": 4239, + "ndcg5": 0.9261859507142916, + "ndcg10": 0.9261859507142916, + "p50_ms": 0.49947950174100697, + "p90_ms": 2.694241679273547, + "p95_ms": 3.458739919005894, + "p99_ms": 4.221747994306496, + "index_ms": 136.7432500119321, + "by_category": { + "architecture": 1.0, + "semantic": 0.8945513581632737, + "symbol": 1.0 + } + }, + { + "repo": "pydantic", + "language": "python", + "mode": "auto", + "chunks": 2868, + "tokens": 1849, + "ndcg5": 0.7530776934979013, + "ndcg10": 0.78378040151892, + "p50_ms": 1.0196665243711323, + "p90_ms": 4.770950292004276, + "p95_ms": 6.936606267117897, + "p99_ms": 9.27522126876283, + "index_ms": 477.22445795079693, + "by_category": { + "architecture": 0.6492397538295632, + "semantic": 0.7663662191964322, + "symbol": 1.0 + } + }, + { + "repo": "rack", + "language": "ruby", + "mode": "auto", + "chunks": 483, + "tokens": 1435, + "ndcg5": 0.8902066119047684, + "ndcg10": 0.8902066119047684, + "p50_ms": 0.6012289959471673, + "p90_ms": 3.0958375136833642, + "p95_ms": 3.865587498876264, + "p99_ms": 4.999317502952178, + "index_ms": 138.96166702033952, + "by_category": { + "architecture": 1.0, + "semantic": 0.8023719014285831, + "symbol": 1.0 + } + }, + { + "repo": "rails", + "language": "ruby", + "mode": "auto", + "chunks": 870, + "tokens": 1403, + "ndcg5": 0.8273505311466753, + "ndcg10": 0.8540441468268698, + "p50_ms": 0.6396459939423949, + "p90_ms": 3.2507497351616634, + "p95_ms": 4.030539593077265, + "p99_ms": 4.249641519854777, + "index_ms": 169.2257919930853, + "by_category": { + "architecture": 0.845500808108813, + "semantic": 0.8494849471938838, + "symbol": 0.8812452583002233 + } + }, + { + "repo": "redis", + "language": "c", + "mode": "auto", + "chunks": 12306, + "tokens": 1786, + "ndcg5": 0.9224229559439696, + "ndcg10": 0.9224229559439696, + "p50_ms": 0.7808124937582761, + "p90_ms": 0.8621535962447525, + "p95_ms": 0.9256482182536274, + "p99_ms": 0.9348632220644504, + "index_ms": 2024.4683329947293, + "by_category": { + "architecture": 0.9430676558073394, + "semantic": 0.9017782560805999 + } + }, + { + "repo": "redux", + "language": "javascript", + "mode": "auto", + "chunks": 87, + "tokens": 1683, + "ndcg5": 0.8928612069551187, + "ndcg10": 0.9175840269191813, + "p50_ms": 0.46075001591816545, + "p90_ms": 2.095695788739249, + "p95_ms": 2.307785104494543, + "p99_ms": 3.5724570299498715, + "index_ms": 27.94062497559935, + "by_category": { + "architecture": 0.9032867981913646, + "semantic": 0.8813288610261599, + "symbol": 1.0 + } + }, + { + "repo": "requests", + "language": "python", + "mode": "auto", + "chunks": 316, + "tokens": 1634, + "ndcg5": 0.9673793323602942, + "ndcg10": 0.9673793323602942, + "p50_ms": 0.44291652739048004, + "p90_ms": 2.298195817274973, + "p95_ms": 2.3587038565892726, + "p99_ms": 2.413740750635043, + "index_ms": 41.20962502202019, + "by_category": { + "architecture": 0.9770630826137678, + "semantic": 0.9385181336136884, + "symbol": 1.0 + } + }, + { + "repo": "serde", + "language": "rust", + "mode": "auto", + "chunks": 2187, + "tokens": 1624, + "ndcg5": 0.7172688678319089, + "ndcg10": 0.7269401880127725, + "p50_ms": 0.8781040087342262, + "p90_ms": 3.714495163876565, + "p95_ms": 4.889345497940667, + "p99_ms": 5.260035489336587, + "index_ms": 325.0218339962885, + "by_category": { + "architecture": 0.7885962468833579, + "semantic": 0.7007303620904127, + "symbol": 0.6121147797198481 + } + }, + { + "repo": "sinatra", + "language": "ruby", + "mode": "auto", + "chunks": 134, + "tokens": 1898, + "ndcg5": 0.9815464876785729, + "ndcg10": 0.9815464876785729, + "p50_ms": 0.5304580263327807, + "p90_ms": 4.123438138049096, + "p95_ms": 4.278952087042855, + "p99_ms": 6.081324036931616, + "index_ms": 24.14183400105685, + "by_category": { + "architecture": 0.9261859507142916, + "semantic": 1.0, + "symbol": 1.0 + } + }, + { + "repo": "snapkit", + "language": "swift", + "mode": "auto", + "chunks": 201, + "tokens": 1525, + "ndcg5": 0.7847362671174445, + "ndcg10": 0.8004579900088665, + "p50_ms": 2.8180420049466193, + "p90_ms": 4.073533427435905, + "p95_ms": 5.320816824678327, + "p99_ms": 6.802896980661896, + "index_ms": 35.99470801418647, + "by_category": { + "architecture": 0.7080343955788911, + "semantic": 0.754550097625513, + "symbol": 1.0 + } + }, + { + "repo": "starlette", + "language": "python", + "mode": "auto", + "chunks": 419, + "tokens": 1487, + "ndcg5": 0.9631568755014722, + "ndcg10": 0.9631568755014722, + "p50_ms": 0.8499170071445405, + "p90_ms": 2.2796669509261847, + "p95_ms": 3.16437502624467, + "p99_ms": 3.1703750253655016, + "index_ms": 149.57887504715472, + "by_category": { + "architecture": 0.9296631259573561, + "semantic": 1.0, + "symbol": 1.0 + } + }, + { + "repo": "telescope.nvim", + "language": "lua", + "mode": "auto", + "chunks": 1053, + "tokens": 1652, + "ndcg5": 0.76266260192301, + "ndcg10": 0.76266260192301, + "p50_ms": 1.0919579945039004, + "p90_ms": 1.3382704753894368, + "p95_ms": 2.007522564963439, + "p99_ms": 3.262504505109971, + "index_ms": 210.71416599443182, + "by_category": { + "architecture": 0.7954904433608064, + "semantic": 0.7449860719180426 + } + }, + { + "repo": "tokio", + "language": "rust", + "mode": "auto", + "chunks": 5338, + "tokens": 1756, + "ndcg5": 0.9266433990956824, + "ndcg10": 0.9368625189751066, + "p50_ms": 0.7257084944285452, + "p90_ms": 3.4515457809902736, + "p95_ms": 4.020210067392328, + "p99_ms": 5.013942008954471, + "index_ms": 955.91958303703, + "by_category": { + "architecture": 0.8029215983923241, + "semantic": 0.9899650986435234, + "symbol": 1.0 + } + }, + { + "repo": "trpc", + "language": "typescript", + "mode": "auto", + "chunks": 690, + "tokens": 1522, + "ndcg5": 0.8273505311466753, + "ndcg10": 0.8491912748284915, + "p50_ms": 2.048791473498568, + "p90_ms": 2.955091994954273, + "p95_ms": 3.06426078605, + "p99_ms": 4.015685797785407, + "index_ms": 110.92791601549834, + "by_category": { + "architecture": 0.7767199752358612, + "semantic": 0.8242950992404203, + "symbol": 1.0 + } + }, + { + "repo": "vapor", + "language": "swift", + "mode": "auto", + "chunks": 1485, + "tokens": 1336, + "ndcg5": 0.7240165939509212, + "ndcg10": 0.7653556218129065, + "p50_ms": 0.6931665120646358, + "p90_ms": 2.8913002985063945, + "p95_ms": 3.046577054192312, + "p99_ms": 3.0971481843153015, + "index_ms": 251.65366602595896, + "by_category": { + "architecture": 0.49091435506199194, + "semantic": 0.7738835265051539, + "symbol": 1.0 + } + }, + { + "repo": "vitest", + "language": "typescript", + "mode": "auto", + "chunks": 2065, + "tokens": 1473, + "ndcg5": 0.7161732909393883, + "ndcg10": 0.7339836502947895, + "p50_ms": 0.6920830346643925, + "p90_ms": 3.063187189400197, + "p95_ms": 3.7006374914199127, + "p99_ms": 5.065027484670279, + "index_ms": 313.1591249839403, + "by_category": { + "architecture": 0.7529637601343332, + "semantic": 0.6510809495536172, + "symbol": 1.0 + } + }, + { + "repo": "xmonad", + "language": "haskell", + "mode": "auto", + "chunks": 241, + "tokens": 1695, + "ndcg5": 0.8486778764412228, + "ndcg10": 0.8486778764412228, + "p50_ms": 0.4794579872395843, + "p90_ms": 2.5380583887454122, + "p95_ms": 2.5632860691985115, + "p99_ms": 2.6651572360424325, + "index_ms": 42.54212498199195, + "by_category": { + "architecture": 0.8476276829510184, + "semantic": 0.8493780054346924 + } + }, + { + "repo": "zig", + "language": "zig", + "mode": "auto", + "chunks": 26017, + "tokens": 1484, + "ndcg5": 0.8880929753571458, + "ndcg10": 0.8880929753571458, + "p50_ms": 1.277417002711445, + "p90_ms": 13.241121434839444, + "p95_ms": 15.012389258481567, + "p99_ms": 20.093844220973544, + "index_ms": 3897.3217079765163, + "by_category": { + "architecture": 0.7103099178571526, + "semantic": 0.9194664560924387 + } + }, + { + "repo": "zig-clap", + "language": "zig", + "mode": "auto", + "chunks": 195, + "tokens": 1606, + "ndcg5": 0.9380929753571458, + "ndcg10": 0.9380929753571458, + "p50_ms": 0.5296875024214387, + "p90_ms": 2.477704192278907, + "p95_ms": 2.7749378205044195, + "p99_ms": 2.784753972082399, + "index_ms": 33.42574997805059, + "by_category": { + "architecture": 1.0, + "semantic": 0.9312144170634953 + } + }, + { + "repo": "zls", + "language": "zig", + "mode": "auto", + "chunks": 2617, + "tokens": 1548, + "ndcg5": 0.8380929753571458, + "ndcg10": 0.870954834495746, + "p50_ms": 0.604875007411465, + "p90_ms": 0.6965708918869497, + "p95_ms": 1.0052047698991404, + "p99_ms": 4.421340935514303, + "index_ms": 347.77879202738404, + "by_category": { + "architecture": 0.8081767403960004, + "semantic": 0.9047584236263781 + } + }, + { + "repo": "zod", + "language": "typescript", + "mode": "auto", + "chunks": 3576, + "tokens": 1416, + "ndcg5": 0.599740727020101, + "ndcg10": 0.64272096242225, + "p50_ms": 2.559228480095044, + "p90_ms": 3.814778884407133, + "p95_ms": 4.003367052064278, + "p99_ms": 5.304107003030365, + "index_ms": 598.6725420225412, + "by_category": { + "architecture": 0.5263150610131765, + "semantic": 0.6629407116016026, + "symbol": 0.7103099178571526 + } + } + ] +} diff --git a/benchmarks/results/speed-5fbcd035d594.json b/benchmarks/results/speed-5fbcd035d594.json new file mode 100644 index 00000000..e3290b3b --- /dev/null +++ b/benchmarks/results/speed-5fbcd035d594.json @@ -0,0 +1,2152 @@ +{ + "repos": [ + "nvm", + "libuv", + "nlohmann-json", + "messagepack-csharp", + "phoenix", + "gin", + "aeson", + "gson", + "axios", + "ktor", + "telescope.nvim", + "monolog", + "flask", + "rack", + "axum", + "http4s", + "alamofire", + "trpc", + "zls" + ], + "summary": { + "semble": { + "avg_index_ms": 240.3, + "avg_p50_ms": 0.69, + "avg_p90_ms": 3.5, + "avg_p95_ms": 4.23, + "avg_p99_ms": 6.27 + } + }, + "results": [ + { + "repo": "nvm", + "language": "bash", + "tool": "semble", + "index_ms": 74.3, + "p50_ms": 0.49, + "p90_ms": 0.54, + "p95_ms": 0.58, + "p99_ms": 0.61, + "latencies_ms": [ + 0.957, + 0.539, + 0.483, + 0.475, + 0.467, + 0.503, + 0.464, + 0.459, + 0.455, + 0.46, + 0.55, + 0.509, + 0.499, + 0.485, + 0.492, + 0.61, + 0.592, + 0.534, + 0.529, + 0.511, + 0.571, + 0.535, + 0.531, + 0.524, + 0.529, + 0.463, + 0.462, + 0.474, + 0.459, + 0.442, + 0.501, + 0.487, + 0.481, + 0.478, + 0.499, + 0.596, + 0.523, + 0.503, + 0.481, + 0.473, + 0.526, + 0.518, + 0.522, + 0.535, + 0.525, + 0.463, + 0.441, + 0.436, + 0.427, + 0.44, + 0.523, + 0.497, + 0.495, + 0.493, + 0.505, + 0.531, + 0.502, + 0.507, + 0.486, + 0.48, + 0.48, + 0.474, + 0.467, + 0.465, + 0.457, + 0.533, + 0.536, + 0.524, + 0.517, + 0.513, + 0.503, + 0.476, + 0.465, + 0.464, + 0.454, + 0.568, + 0.584, + 0.53, + 0.515, + 0.474, + 0.464, + 0.452, + 0.451, + 0.441, + 0.456, + 0.502, + 0.501, + 0.485, + 0.47, + 0.472, + 0.474, + 0.451, + 0.445, + 0.456, + 0.465, + 0.502, + 0.59, + 0.534, + 0.515, + 0.48 + ] + }, + { + "repo": "libuv", + "language": "c", + "tool": "semble", + "index_ms": 363.1, + "p50_ms": 0.6, + "p90_ms": 0.7, + "p95_ms": 0.72, + "p99_ms": 0.8, + "latencies_ms": [ + 1.212, + 0.775, + 0.725, + 0.731, + 0.722, + 0.638, + 0.6, + 0.651, + 0.66, + 0.603, + 0.643, + 0.575, + 0.586, + 0.574, + 0.58, + 0.793, + 0.714, + 0.703, + 0.687, + 0.699, + 0.655, + 0.602, + 0.631, + 0.631, + 0.576, + 0.577, + 0.515, + 0.503, + 0.518, + 0.504, + 0.536, + 0.495, + 0.484, + 0.481, + 0.493, + 0.668, + 0.626, + 0.627, + 0.637, + 0.615, + 0.669, + 0.603, + 0.546, + 0.516, + 0.521, + 0.548, + 0.508, + 0.507, + 0.519, + 0.521, + 0.641, + 0.586, + 0.579, + 0.57, + 0.574, + 0.687, + 0.654, + 0.609, + 0.713, + 0.651, + 0.716, + 0.668, + 0.683, + 0.669, + 0.675, + 0.69, + 0.615, + 0.603, + 0.609, + 0.598, + 0.718, + 0.655, + 0.656, + 0.703, + 0.68, + 0.522, + 0.484, + 0.461, + 0.46, + 0.471, + 0.702, + 0.666, + 0.651, + 0.654, + 0.661, + 0.638, + 0.582, + 0.576, + 0.578, + 0.587, + 0.666, + 0.597, + 0.586, + 0.562, + 0.564, + 0.533, + 0.507, + 0.483, + 0.484, + 0.493 + ] + }, + { + "repo": "nlohmann-json", + "language": "cpp", + "tool": "semble", + "index_ms": 221.2, + "p50_ms": 0.63, + "p90_ms": 3.05, + "p95_ms": 3.11, + "p99_ms": 3.78, + "latencies_ms": [ + 1.057, + 0.699, + 0.665, + 0.654, + 0.669, + 0.519, + 0.464, + 0.469, + 0.457, + 0.455, + 0.499, + 0.463, + 0.452, + 0.458, + 0.451, + 0.629, + 0.608, + 0.561, + 0.544, + 0.542, + 0.738, + 0.691, + 4.474, + 0.794, + 0.716, + 0.676, + 0.614, + 0.63, + 0.649, + 0.628, + 0.611, + 0.568, + 0.553, + 0.576, + 0.554, + 0.643, + 0.619, + 0.628, + 0.649, + 0.608, + 0.676, + 0.618, + 0.599, + 0.581, + 0.581, + 0.688, + 0.619, + 0.619, + 0.605, + 0.616, + 0.681, + 0.664, + 0.639, + 0.614, + 0.676, + 0.649, + 0.569, + 0.543, + 0.542, + 0.537, + 0.783, + 0.737, + 0.727, + 0.778, + 0.756, + 3.774, + 3.078, + 3.152, + 3.103, + 3.195, + 0.638, + 0.576, + 0.573, + 0.596, + 0.571, + 0.566, + 0.526, + 0.516, + 0.522, + 0.507, + 0.635, + 0.59, + 0.644, + 0.583, + 0.58, + 2.846, + 2.439, + 2.482, + 2.351, + 2.365, + 3.437, + 3.067, + 3.058, + 3.048, + 3.093, + 3.046, + 2.636, + 2.667, + 2.703, + 2.569 + ] + }, + { + "repo": "messagepack-csharp", + "language": "csharp", + "tool": "semble", + "index_ms": 333.6, + "p50_ms": 2.25, + "p90_ms": 3.68, + "p95_ms": 3.8, + "p99_ms": 4.1, + "latencies_ms": [ + 4.023, + 3.621, + 3.517, + 3.678, + 3.619, + 1.102, + 0.94, + 0.868, + 0.88, + 0.873, + 0.944, + 0.796, + 0.782, + 0.783, + 0.775, + 0.869, + 0.798, + 0.781, + 0.773, + 0.761, + 0.735, + 0.691, + 0.67, + 0.7, + 0.674, + 0.861, + 0.819, + 0.811, + 0.791, + 0.824, + 0.891, + 0.793, + 0.724, + 0.757, + 0.748, + 0.808, + 0.76, + 0.765, + 0.75, + 0.743, + 3.337, + 3.153, + 3.259, + 3.271, + 3.175, + 0.731, + 0.678, + 0.67, + 0.665, + 0.662, + 3.508, + 3.343, + 3.438, + 3.414, + 3.248, + 4.097, + 3.577, + 3.526, + 3.639, + 3.559, + 4.232, + 3.892, + 3.783, + 3.931, + 3.773, + 2.42, + 1.977, + 2.016, + 2.019, + 1.982, + 2.689, + 2.288, + 2.31, + 2.289, + 2.215, + 3.439, + 3.054, + 2.882, + 2.911, + 3.007, + 2.775, + 2.354, + 2.382, + 2.441, + 2.352, + 3.734, + 3.773, + 3.577, + 3.661, + 3.793, + 0.756, + 0.699, + 0.682, + 0.687, + 0.699, + 3.316, + 2.92, + 2.812, + 2.86, + 2.876 + ] + }, + { + "repo": "phoenix", + "language": "elixir", + "tool": "semble", + "index_ms": 1068.3, + "p50_ms": 0.68, + "p90_ms": 5.13, + "p95_ms": 5.71, + "p99_ms": 6.27, + "latencies_ms": [ + 1.354, + 0.873, + 0.783, + 0.762, + 0.752, + 0.805, + 0.723, + 0.656, + 0.643, + 0.621, + 0.736, + 0.63, + 0.615, + 0.641, + 0.618, + 0.63, + 0.601, + 0.594, + 0.565, + 0.562, + 0.642, + 0.673, + 0.682, + 0.637, + 0.583, + 6.395, + 5.542, + 5.731, + 5.644, + 5.731, + 0.623, + 0.549, + 0.524, + 0.498, + 0.517, + 0.669, + 0.6, + 0.596, + 0.6, + 0.58, + 0.804, + 0.74, + 0.724, + 0.757, + 0.754, + 0.689, + 0.628, + 0.63, + 0.604, + 0.6, + 0.7, + 0.644, + 0.635, + 0.646, + 0.647, + 0.655, + 0.621, + 0.586, + 0.584, + 0.668, + 0.767, + 0.662, + 0.641, + 0.611, + 0.593, + 0.717, + 0.667, + 0.663, + 0.639, + 0.641, + 0.769, + 0.724, + 0.715, + 0.723, + 0.772, + 0.787, + 0.746, + 0.713, + 0.717, + 0.723, + 4.508, + 3.949, + 4.002, + 3.995, + 4.0, + 6.262, + 5.648, + 5.698, + 5.777, + 5.625, + 4.472, + 3.982, + 4.007, + 4.108, + 3.996 + ] + }, + { + "repo": "gin", + "language": "go", + "tool": "semble", + "index_ms": 186.6, + "p50_ms": 0.58, + "p90_ms": 2.37, + "p95_ms": 3.24, + "p99_ms": 3.59, + "latencies_ms": [ + 0.88, + 0.546, + 0.523, + 0.534, + 0.607, + 0.487, + 0.406, + 0.383, + 0.388, + 0.397, + 0.555, + 0.497, + 0.491, + 0.498, + 0.493, + 0.625, + 0.568, + 0.57, + 0.573, + 0.572, + 0.614, + 0.548, + 0.528, + 0.598, + 0.612, + 3.232, + 2.757, + 2.771, + 2.75, + 2.74, + 0.58, + 0.542, + 0.524, + 0.539, + 0.528, + 0.524, + 0.469, + 0.459, + 0.454, + 0.509, + 0.467, + 0.43, + 0.425, + 0.424, + 0.43, + 0.648, + 0.588, + 0.57, + 0.568, + 0.563, + 0.514, + 0.493, + 0.472, + 0.466, + 0.461, + 0.686, + 0.623, + 0.616, + 0.687, + 0.669, + 0.626, + 0.582, + 0.571, + 0.569, + 0.579, + 0.583, + 0.571, + 0.561, + 0.582, + 0.555, + 0.598, + 0.584, + 0.564, + 0.56, + 0.561, + 0.72, + 0.631, + 0.601, + 0.602, + 0.608, + 2.326, + 1.913, + 1.935, + 2.018, + 1.961, + 3.893, + 3.589, + 3.407, + 3.497, + 3.55, + 2.055, + 2.144, + 2.164, + 2.16, + 2.104, + 0.643, + 0.594, + 0.596, + 0.594, + 0.606 + ] + }, + { + "repo": "aeson", + "language": "haskell", + "tool": "semble", + "index_ms": 136.8, + "p50_ms": 2.95, + "p90_ms": 6.7, + "p95_ms": 11.01, + "p99_ms": 19.61, + "latencies_ms": [ + 0.926, + 0.617, + 0.585, + 0.58, + 0.577, + 3.693, + 3.298, + 3.204, + 3.142, + 3.127, + 3.351, + 2.933, + 2.818, + 2.808, + 2.869, + 3.353, + 2.975, + 3.029, + 2.955, + 2.947, + 3.473, + 3.077, + 2.945, + 3.033, + 3.038, + 3.625, + 3.285, + 3.402, + 3.162, + 3.346, + 3.261, + 2.717, + 2.717, + 2.85, + 2.848, + 20.169, + 19.222, + 19.236, + 19.607, + 19.42, + 3.707, + 3.245, + 3.2, + 3.149, + 3.096, + 4.056, + 3.546, + 3.665, + 3.648, + 3.469, + 3.354, + 3.006, + 2.923, + 2.862, + 2.947, + 0.585, + 0.551, + 0.617, + 0.562, + 0.549, + 0.592, + 0.553, + 0.539, + 0.551, + 0.551, + 0.612, + 0.593, + 0.582, + 0.604, + 0.571, + 3.255, + 2.716, + 2.812, + 2.883, + 2.806, + 0.597, + 0.591, + 0.577, + 0.582, + 0.594, + 10.575, + 9.827, + 9.857, + 9.79, + 9.976, + 6.36, + 5.377, + 5.459, + 5.396, + 5.432, + 0.537, + 0.497, + 0.501, + 0.493, + 0.501, + 3.28, + 2.979, + 2.836, + 2.901, + 2.935 + ] + }, + { + "repo": "gson", + "language": "java", + "tool": "semble", + "index_ms": 378.0, + "p50_ms": 0.84, + "p90_ms": 4.45, + "p95_ms": 5.46, + "p99_ms": 7.0, + "latencies_ms": [ + 8.419, + 6.962, + 6.74, + 6.779, + 6.985, + 1.052, + 0.84, + 0.821, + 0.855, + 0.846, + 1.042, + 0.835, + 0.835, + 0.822, + 0.824, + 4.426, + 3.948, + 3.947, + 3.831, + 3.941, + 2.952, + 2.466, + 2.526, + 2.507, + 2.458, + 0.841, + 0.721, + 0.708, + 0.711, + 0.711, + 0.783, + 0.692, + 0.68, + 0.705, + 0.716, + 0.743, + 0.626, + 0.578, + 0.592, + 0.573, + 0.696, + 0.653, + 0.637, + 0.667, + 0.637, + 0.643, + 0.583, + 0.569, + 0.587, + 0.585, + 0.977, + 0.9, + 0.845, + 0.824, + 0.831, + 0.849, + 0.794, + 0.78, + 0.778, + 0.77, + 0.788, + 0.74, + 0.745, + 0.771, + 0.756, + 0.76, + 0.704, + 0.694, + 0.699, + 0.71, + 0.844, + 0.769, + 0.747, + 0.77, + 0.752, + 0.899, + 0.911, + 0.864, + 0.812, + 0.766, + 4.384, + 3.847, + 3.715, + 3.814, + 3.777, + 2.872, + 2.47, + 2.545, + 2.471, + 2.459, + 5.39, + 4.685, + 4.921, + 4.724, + 4.913, + 3.005, + 2.731, + 2.757, + 2.811, + 2.859 + ] + }, + { + "repo": "axios", + "language": "javascript", + "tool": "semble", + "index_ms": 42.5, + "p50_ms": 0.63, + "p90_ms": 2.01, + "p95_ms": 2.1, + "p99_ms": 2.53, + "latencies_ms": [ + 0.993, + 0.742, + 0.781, + 0.667, + 0.605, + 0.526, + 0.475, + 0.456, + 0.511, + 0.458, + 0.601, + 0.686, + 0.673, + 0.587, + 0.567, + 0.639, + 0.557, + 0.563, + 0.553, + 0.541, + 2.529, + 2.025, + 1.999, + 1.959, + 1.938, + 1.963, + 1.535, + 1.52, + 1.447, + 1.394, + 2.567, + 2.085, + 2.059, + 2.059, + 2.062, + 0.648, + 0.602, + 0.545, + 0.512, + 0.516, + 0.597, + 0.547, + 0.536, + 0.525, + 0.516, + 0.629, + 0.569, + 0.553, + 0.563, + 0.559, + 0.551, + 0.537, + 0.524, + 0.596, + 0.622, + 0.665, + 0.603, + 0.564, + 0.668, + 0.571, + 0.682, + 0.648, + 0.643, + 0.619, + 0.613, + 0.61, + 0.58, + 0.567, + 0.572, + 0.567, + 0.523, + 0.502, + 0.502, + 0.51, + 0.504, + 0.62, + 0.579, + 0.567, + 0.573, + 0.766, + 0.784, + 0.703, + 0.681, + 0.662, + 0.787, + 2.479, + 1.973, + 1.967, + 1.921, + 2.0, + 2.488, + 1.892, + 1.881, + 1.895, + 1.842, + 2.458, + 2.007, + 1.956, + 1.991, + 1.985 + ] + }, + { + "repo": "ktor", + "language": "kotlin", + "tool": "semble", + "index_ms": 117.8, + "p50_ms": 0.81, + "p90_ms": 4.24, + "p95_ms": 4.58, + "p99_ms": 4.91, + "latencies_ms": [ + 5.586, + 4.736, + 4.691, + 4.743, + 4.905, + 1.064, + 0.795, + 0.781, + 0.774, + 0.742, + 0.736, + 0.659, + 0.628, + 0.623, + 0.616, + 0.881, + 0.841, + 0.889, + 0.82, + 0.784, + 0.735, + 0.687, + 0.67, + 0.666, + 0.664, + 0.715, + 0.652, + 0.649, + 0.642, + 0.648, + 0.666, + 0.595, + 0.581, + 0.573, + 0.572, + 0.576, + 0.527, + 0.527, + 0.52, + 0.522, + 2.408, + 2.316, + 2.308, + 2.312, + 2.323, + 0.668, + 0.606, + 0.595, + 0.595, + 0.592, + 0.72, + 0.658, + 0.658, + 0.717, + 0.694, + 3.939, + 3.936, + 3.937, + 4.031, + 4.034, + 4.222, + 3.668, + 3.716, + 3.699, + 3.783, + 2.804, + 2.387, + 2.372, + 2.397, + 2.362, + 3.912, + 3.315, + 3.306, + 3.352, + 3.384, + 2.619, + 2.222, + 2.274, + 2.223, + 2.183, + 4.578, + 4.432, + 4.407, + 4.54, + 4.417, + 0.784, + 0.805, + 0.771, + 0.766, + 0.742, + 0.575, + 0.544, + 0.534, + 0.524, + 0.539, + 2.75, + 2.279, + 2.231, + 2.218, + 2.25 + ] + }, + { + "repo": "telescope.nvim", + "language": "lua", + "tool": "semble", + "index_ms": 140.7, + "p50_ms": 0.59, + "p90_ms": 0.71, + "p95_ms": 2.3, + "p99_ms": 2.56, + "latencies_ms": [ + 1.043, + 0.686, + 0.639, + 0.638, + 0.639, + 0.594, + 0.534, + 0.538, + 0.531, + 0.529, + 0.679, + 0.594, + 0.595, + 0.593, + 0.593, + 0.717, + 0.611, + 0.585, + 0.574, + 0.584, + 0.66, + 0.618, + 0.622, + 0.598, + 0.603, + 0.616, + 0.559, + 0.565, + 0.589, + 0.553, + 0.578, + 0.599, + 0.583, + 0.564, + 0.516, + 0.585, + 0.578, + 0.562, + 0.551, + 0.543, + 0.577, + 0.548, + 0.531, + 0.532, + 0.538, + 3.102, + 2.397, + 2.48, + 2.479, + 2.553, + 0.668, + 0.595, + 0.583, + 0.569, + 0.586, + 0.59, + 0.557, + 0.55, + 0.553, + 0.556, + 0.57, + 0.539, + 0.537, + 0.547, + 0.532, + 0.655, + 0.689, + 0.699, + 0.65, + 0.625, + 0.583, + 0.578, + 0.551, + 0.561, + 0.595, + 0.629, + 0.564, + 0.565, + 0.549, + 0.537, + 0.608, + 0.595, + 0.578, + 2.294, + 0.783, + 0.709, + 0.615, + 1.439, + 0.668, + 0.57, + 0.58, + 0.526, + 0.531, + 0.528, + 0.523, + 0.667, + 0.607, + 0.599, + 0.585, + 0.596 + ] + }, + { + "repo": "monolog", + "language": "php", + "tool": "semble", + "index_ms": 97.1, + "p50_ms": 0.71, + "p90_ms": 2.43, + "p95_ms": 2.64, + "p99_ms": 2.87, + "latencies_ms": [ + 0.968, + 0.654, + 0.654, + 0.589, + 0.567, + 0.713, + 0.628, + 0.625, + 0.625, + 0.612, + 0.752, + 0.66, + 0.654, + 0.63, + 0.628, + 0.588, + 0.523, + 0.531, + 0.517, + 0.717, + 3.216, + 2.617, + 2.59, + 2.668, + 2.513, + 2.66, + 2.214, + 2.268, + 2.15, + 2.216, + 2.867, + 2.452, + 2.368, + 2.344, + 2.409, + 0.735, + 0.677, + 0.68, + 0.699, + 0.669, + 0.784, + 0.739, + 0.707, + 0.712, + 0.704, + 0.717, + 0.658, + 0.672, + 0.673, + 0.686, + 0.687, + 0.649, + 0.674, + 0.667, + 0.633, + 0.632, + 0.605, + 0.605, + 0.614, + 0.617, + 0.657, + 0.597, + 0.598, + 0.59, + 0.575, + 0.646, + 0.602, + 0.603, + 0.603, + 0.654, + 0.766, + 0.71, + 0.697, + 0.709, + 0.736, + 0.775, + 0.728, + 0.711, + 0.718, + 0.716, + 0.787, + 0.761, + 0.774, + 0.776, + 0.74, + 0.817, + 0.794, + 0.762, + 0.743, + 0.757, + 2.807, + 2.433, + 2.343, + 2.384, + 2.41, + 2.636, + 2.05, + 2.128, + 2.14, + 2.122 + ] + }, + { + "repo": "flask", + "language": "python", + "tool": "semble", + "index_ms": 69.4, + "p50_ms": 0.55, + "p90_ms": 3.12, + "p95_ms": 3.81, + "p99_ms": 5.08, + "latencies_ms": [ + 0.86, + 0.516, + 0.483, + 0.46, + 0.445, + 0.45, + 0.389, + 0.407, + 0.393, + 0.417, + 0.402, + 0.367, + 0.384, + 0.359, + 0.35, + 0.383, + 0.355, + 0.363, + 0.364, + 0.362, + 0.429, + 0.405, + 0.393, + 0.46, + 0.46, + 0.397, + 0.374, + 0.352, + 0.351, + 0.358, + 0.456, + 0.431, + 0.416, + 0.407, + 0.41, + 0.425, + 0.389, + 0.396, + 0.395, + 0.408, + 0.417, + 0.386, + 0.384, + 0.377, + 0.382, + 0.491, + 0.496, + 0.58, + 0.487, + 0.464, + 0.596, + 1.166, + 1.147, + 0.817, + 1.056, + 1.147, + 1.064, + 0.981, + 0.803, + 0.797, + 0.905, + 0.872, + 0.663, + 0.651, + 0.824, + 0.832, + 0.692, + 0.637, + 0.654, + 0.608, + 0.823, + 0.658, + 0.595, + 0.929, + 1.054, + 4.045, + 3.046, + 3.223, + 3.272, + 3.495, + 3.167, + 2.819, + 2.466, + 2.669, + 2.492, + 5.421, + 3.863, + 4.913, + 4.343, + 5.083, + 0.96, + 0.702, + 0.68, + 0.622, + 0.607, + 0.552, + 0.526, + 0.505, + 0.544, + 0.513, + 0.981, + 3.607, + 1.114, + 0.966, + 0.94 + ] + }, + { + "repo": "rack", + "language": "ruby", + "tool": "semble", + "index_ms": 171.5, + "p50_ms": 0.57, + "p90_ms": 3.32, + "p95_ms": 4.67, + "p99_ms": 5.49, + "latencies_ms": [ + 0.973, + 1.149, + 0.735, + 0.697, + 0.56, + 0.965, + 0.946, + 1.03, + 0.977, + 0.851, + 0.614, + 0.587, + 0.468, + 0.45, + 0.693, + 0.564, + 0.507, + 0.5, + 0.502, + 0.497, + 3.56, + 2.801, + 2.978, + 2.941, + 2.98, + 6.292, + 5.392, + 5.124, + 5.113, + 5.083, + 4.172, + 3.346, + 3.322, + 3.272, + 3.229, + 0.581, + 0.524, + 0.495, + 0.484, + 0.492, + 0.552, + 0.522, + 0.612, + 0.548, + 0.499, + 0.543, + 0.505, + 0.493, + 0.487, + 0.485, + 0.503, + 0.459, + 0.464, + 0.472, + 0.459, + 0.555, + 0.514, + 0.519, + 0.518, + 0.515, + 0.554, + 0.542, + 0.519, + 0.514, + 0.523, + 0.464, + 0.44, + 0.441, + 0.57, + 0.531, + 0.519, + 0.492, + 0.479, + 0.473, + 0.48, + 0.604, + 0.575, + 0.631, + 0.674, + 0.609, + 3.306, + 2.77, + 2.667, + 2.624, + 3.129, + 3.909, + 2.983, + 2.805, + 3.104, + 3.181 + ] + }, + { + "repo": "axum", + "language": "rust", + "tool": "semble", + "index_ms": 220.3, + "p50_ms": 1.35, + "p90_ms": 4.35, + "p95_ms": 5.09, + "p99_ms": 6.19, + "latencies_ms": [ + 1.518, + 1.458, + 1.198, + 1.138, + 1.259, + 1.469, + 1.115, + 1.096, + 1.372, + 1.173, + 6.881, + 6.044, + 5.996, + 6.182, + 5.908, + 1.117, + 0.955, + 1.039, + 1.098, + 1.005, + 1.157, + 1.076, + 0.945, + 0.901, + 0.897, + 1.007, + 0.947, + 0.943, + 1.1, + 1.12, + 4.741, + 4.335, + 4.188, + 4.253, + 4.171, + 1.398, + 1.324, + 1.373, + 1.19, + 1.198, + 1.247, + 1.068, + 1.043, + 1.03, + 1.153, + 1.403, + 1.28, + 1.118, + 1.305, + 1.249, + 1.377, + 1.327, + 1.096, + 1.03, + 1.05, + 3.148, + 3.547, + 3.35, + 3.272, + 3.139, + 5.052, + 4.293, + 4.51, + 4.456, + 4.639, + 4.316, + 3.567, + 3.701, + 3.643, + 3.572, + 1.223, + 1.134, + 1.28, + 1.197, + 1.071, + 1.251, + 1.554, + 1.195, + 1.173, + 1.41, + 1.007, + 0.924, + 0.873, + 0.862, + 0.853, + 3.121, + 3.27, + 3.286, + 3.053, + 3.169, + 3.449, + 3.924, + 3.425, + 3.213, + 3.134, + 3.454, + 3.588, + 3.593, + 3.473, + 3.663 + ] + }, + { + "repo": "http4s", + "language": "scala", + "tool": "semble", + "index_ms": 378.0, + "p50_ms": 2.54, + "p90_ms": 3.88, + "p95_ms": 4.59, + "p99_ms": 5.1, + "latencies_ms": [ + 4.568, + 3.459, + 3.494, + 3.432, + 3.45, + 0.755, + 0.615, + 0.579, + 0.547, + 0.54, + 3.693, + 3.292, + 3.263, + 3.308, + 3.311, + 3.2, + 2.836, + 2.755, + 2.828, + 2.822, + 0.756, + 0.618, + 0.618, + 0.618, + 0.605, + 0.749, + 0.6, + 0.59, + 0.65, + 0.614, + 5.41, + 5.025, + 5.005, + 5.099, + 5.077, + 0.646, + 0.559, + 0.55, + 0.547, + 0.542, + 0.686, + 0.607, + 0.596, + 0.589, + 0.597, + 0.777, + 0.722, + 0.707, + 0.709, + 0.701, + 4.018, + 3.355, + 3.442, + 3.457, + 3.347, + 0.707, + 0.646, + 0.634, + 0.639, + 0.665, + 3.866, + 3.339, + 4.387, + 3.399, + 3.41, + 0.817, + 0.725, + 0.71, + 0.701, + 0.692, + 4.229, + 3.881, + 3.899, + 3.809, + 3.852, + 0.642, + 0.586, + 0.624, + 0.601, + 0.573, + 3.807, + 3.445, + 3.496, + 3.587, + 3.507, + 2.501, + 2.409, + 2.443, + 2.471, + 2.461, + 2.675, + 2.672, + 2.661, + 2.587, + 2.621, + 3.797, + 3.258, + 3.258, + 3.358, + 3.289 + ] + }, + { + "repo": "alamofire", + "language": "swift", + "tool": "semble", + "index_ms": 160.8, + "p50_ms": 0.6, + "p90_ms": 3.17, + "p95_ms": 4.14, + "p99_ms": 5.46, + "latencies_ms": [ + 0.85, + 0.577, + 0.537, + 0.514, + 0.521, + 0.664, + 0.597, + 0.597, + 0.594, + 0.592, + 0.588, + 0.537, + 0.529, + 0.525, + 0.518, + 0.663, + 0.589, + 0.561, + 0.561, + 0.559, + 0.59, + 0.552, + 0.546, + 0.534, + 0.553, + 0.536, + 0.496, + 0.484, + 0.51, + 0.488, + 0.569, + 0.515, + 0.503, + 0.569, + 0.581, + 0.621, + 0.591, + 0.555, + 0.554, + 0.564, + 0.609, + 0.58, + 0.572, + 0.584, + 0.574, + 0.67, + 0.612, + 0.593, + 0.592, + 0.592, + 4.083, + 3.691, + 3.644, + 3.498, + 3.594, + 0.68, + 0.676, + 0.682, + 0.621, + 0.59, + 5.905, + 5.309, + 5.349, + 5.244, + 5.458, + 3.135, + 2.709, + 2.742, + 2.89, + 2.929, + 2.521, + 2.07, + 2.079, + 2.048, + 1.989, + 2.278, + 1.855, + 1.875, + 1.821, + 1.767, + 2.95, + 2.477, + 2.5, + 2.441, + 2.463, + 0.613, + 0.565, + 0.552, + 0.547, + 0.544, + 0.626, + 0.618, + 0.589, + 0.561, + 0.55, + 2.904, + 2.586, + 2.641, + 2.587, + 2.53 + ] + }, + { + "repo": "trpc", + "language": "typescript", + "tool": "semble", + "index_ms": 91.9, + "p50_ms": 2.11, + "p90_ms": 2.92, + "p95_ms": 3.86, + "p99_ms": 4.32, + "latencies_ms": [ + 3.85, + 2.877, + 2.946, + 2.952, + 2.845, + 0.761, + 0.687, + 0.699, + 0.674, + 0.681, + 0.858, + 0.791, + 0.786, + 0.855, + 0.837, + 2.652, + 2.68, + 2.603, + 2.659, + 2.784, + 0.844, + 0.734, + 0.735, + 0.736, + 0.738, + 0.79, + 0.732, + 0.707, + 0.693, + 0.689, + 0.672, + 0.611, + 0.599, + 0.595, + 0.611, + 0.835, + 0.77, + 0.749, + 0.757, + 0.746, + 2.45, + 2.332, + 2.349, + 2.335, + 2.343, + 2.675, + 2.715, + 2.747, + 2.721, + 2.654, + 2.763, + 2.74, + 2.684, + 2.764, + 2.813, + 0.73, + 0.678, + 0.71, + 0.718, + 0.672, + 2.509, + 2.175, + 2.112, + 2.19, + 2.118, + 2.333, + 1.967, + 1.979, + 1.969, + 1.882, + 2.522, + 2.123, + 2.141, + 2.073, + 2.018, + 2.259, + 1.828, + 1.847, + 1.81, + 1.772, + 2.266, + 1.858, + 1.847, + 1.886, + 1.786, + 4.319, + 4.281, + 4.236, + 4.344, + 4.127, + 2.945, + 2.924, + 2.829, + 2.902, + 2.923, + 2.649, + 2.541, + 2.626, + 2.631, + 2.608 + ] + }, + { + "repo": "zls", + "language": "zig", + "tool": "semble", + "index_ms": 313.9, + "p50_ms": 0.62, + "p90_ms": 0.74, + "p95_ms": 1.34, + "p99_ms": 5.29, + "latencies_ms": [ + 1.136, + 0.779, + 0.725, + 0.714, + 0.784, + 5.741, + 5.283, + 5.258, + 5.234, + 5.279, + 0.625, + 0.556, + 0.548, + 0.534, + 0.535, + 0.713, + 0.629, + 0.605, + 0.594, + 0.6, + 0.722, + 0.689, + 0.685, + 0.69, + 0.668, + 0.711, + 0.663, + 0.66, + 0.657, + 0.673, + 0.6, + 0.546, + 0.534, + 0.53, + 0.54, + 0.597, + 0.557, + 0.551, + 0.57, + 0.55, + 0.651, + 0.599, + 0.589, + 0.579, + 0.593, + 0.768, + 0.718, + 0.877, + 0.741, + 0.691, + 0.683, + 0.627, + 0.616, + 0.631, + 0.632, + 0.695, + 0.643, + 0.624, + 0.621, + 0.627, + 0.662, + 0.645, + 0.69, + 0.631, + 0.612, + 0.639, + 0.588, + 0.58, + 0.571, + 0.571, + 0.573, + 0.537, + 0.522, + 0.54, + 0.511, + 0.545, + 0.517, + 0.503, + 0.503, + 0.569, + 0.637, + 0.56, + 0.542, + 0.541, + 0.539, + 0.651, + 0.603, + 0.602, + 0.595, + 0.589, + 0.637, + 0.597, + 0.588, + 0.581, + 0.595, + 0.694, + 0.667, + 0.677, + 0.617, + 0.608 + ] + } + ] +} diff --git a/benchmarks/results/token-efficiency-5fbcd035d594.json b/benchmarks/results/token-efficiency-5fbcd035d594.json new file mode 100644 index 00000000..6ef4f851 --- /dev/null +++ b/benchmarks/results/token-efficiency-5fbcd035d594.json @@ -0,0 +1,323 @@ +{ + "tool": "token-efficiency", + "tokenizer": "cl100k_base", + "budgets": [ + 500, + 1000, + 2000, + 4000, + 8000, + 16000, + 32000 + ], + "n_queries": 1251, + "recall_at_budget": { + "semble": { + "500": 0.8432, + "1000": 0.9209, + "2000": 0.9632, + "4000": 0.9888, + "8000": 0.9952, + "16000": 0.9956, + "32000": 0.9956 + }, + "grep+read": { + "500": 0.0, + "1000": 0.0072, + "2000": 0.0228, + "4000": 0.042, + "8000": 0.0763, + "16000": 0.1003, + "32000": 0.1251 + }, + "grep-kw+read": { + "500": 0.0008, + "1000": 0.008, + "2000": 0.0368, + "4000": 0.0871, + "8000": 0.2114, + "16000": 0.3724, + "32000": 0.5837 + } + }, + "first_hit_reduction": { + "grep+read": { + "n_paired": 189.0, + "median_semble_tokens": 136.0, + "median_other_tokens": 7474.0, + "median_reduction": 0.9837, + "mean_reduction": 0.9665, + "semble_better_pct": 1.0, + "expected_cost_at_cap": 29726.5276, + "expected_cost_cap": 32000.0, + "expected_cost_ratio_vs_semble": 84.1753 + }, + "grep-kw+read": { + "n_paired": 1218.0, + "median_semble_tokens": 168.0, + "median_other_tokens": 19634.0, + "median_reduction": 0.9906, + "mean_reduction": 0.9768, + "semble_better_pct": 1.0, + "expected_cost_at_cap": 47199.2958, + "expected_cost_cap": 32000.0, + "expected_cost_ratio_vs_semble": 133.6522 + }, + "semble": { + "expected_cost_at_cap": 353.1503, + "expected_cost_cap": 32000.0 + } + }, + "plot": { + "budgets": [ + 100, + 114, + 130, + 149, + 170, + 194, + 222, + 253, + 289, + 331, + 378, + 431, + 493, + 563, + 643, + 735, + 839, + 959, + 1095, + 1251, + 1429, + 1633, + 1865, + 2131, + 2434, + 2780, + 3176, + 3628, + 4144, + 4734, + 5407, + 6176, + 7055, + 8059, + 9205, + 10515, + 12011, + 13720, + 15672, + 17902, + 20448, + 23358, + 26681, + 30476, + 34812, + 39765, + 45422, + 51884, + 59265, + 67697, + 77328, + 88329, + 100895, + 115249, + 131645, + 150374, + 171766, + 196203, + 224116, + 255999 + ], + "recall": { + "semble": [ + 0.1508, + 0.186, + 0.2375, + 0.3299, + 0.4684, + 0.5879, + 0.6551, + 0.6855, + 0.7182, + 0.7497, + 0.792, + 0.8228, + 0.8412, + 0.862, + 0.8809, + 0.8913, + 0.9065, + 0.9169, + 0.9301, + 0.94, + 0.946, + 0.9515, + 0.9598, + 0.966, + 0.9708, + 0.9752, + 0.9808, + 0.984, + 0.99, + 0.9904, + 0.9932, + 0.994, + 0.9944, + 0.9952, + 0.9952, + 0.9956, + 0.9956, + 0.9956, + 0.9956, + 0.9956, + 0.9956, + 0.9956, + 0.9956, + 0.9956, + 0.9956, + 0.9956, + 0.9956, + 0.9956, + 0.9956, + 0.9956, + 0.9956, + 0.9956, + 0.9956, + 0.9956, + 0.9956, + 0.9956, + 0.9956, + 0.9956, + 0.9956, + 0.9956 + ], + "grep+read": [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0008, + 0.0032, + 0.0056, + 0.0064, + 0.0072, + 0.008, + 0.0124, + 0.014, + 0.0172, + 0.022, + 0.026, + 0.0288, + 0.0328, + 0.0336, + 0.038, + 0.0428, + 0.0476, + 0.0516, + 0.0568, + 0.0659, + 0.0771, + 0.0795, + 0.0819, + 0.0895, + 0.0959, + 0.0987, + 0.1063, + 0.1131, + 0.1187, + 0.1211, + 0.1251, + 0.1267, + 0.1295, + 0.1363, + 0.1379, + 0.1411, + 0.1427, + 0.1427, + 0.1459, + 0.1471, + 0.1483, + 0.1483, + 0.1483, + 0.1483, + 0.1491, + 0.1491, + 0.1491 + ], + "grep-kw+read": [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0008, + 0.0008, + 0.0008, + 0.0008, + 0.0008, + 0.0008, + 0.0016, + 0.0032, + 0.0052, + 0.0068, + 0.008, + 0.0096, + 0.0156, + 0.0172, + 0.0248, + 0.0344, + 0.0428, + 0.0508, + 0.0604, + 0.0643, + 0.0795, + 0.0915, + 0.1163, + 0.1319, + 0.1547, + 0.1823, + 0.2126, + 0.2349, + 0.2697, + 0.2988, + 0.3304, + 0.3608, + 0.4243, + 0.4631, + 0.5043, + 0.5425, + 0.5769, + 0.6005, + 0.6319, + 0.6743, + 0.7264, + 0.7531, + 0.7819, + 0.8082, + 0.832, + 0.8512, + 0.877, + 0.8945, + 0.9089, + 0.9173, + 0.9233, + 0.9313, + 0.9373 + ] + } + } +} diff --git a/pyproject.toml b/pyproject.toml index d760a684..5e845098 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -26,7 +26,7 @@ classifiers = [ "Natural Language :: English", ] dependencies = [ - "model2vec>=0.4.0,<0.10", + "model2vec>=0.10.0,<0.11", "vicinity>=0.4.4,<0.5", "numpy>=1.24.0", "pathspec>=0.12", @@ -147,4 +147,5 @@ exclude_also = ["if __name__ == .__main__.:", "@abstractmethod"] exclude-newer = "1 week" [tool.uv.exclude-newer-package] +model2vec = false semble-grammars = false diff --git a/src/semble/index/create.py b/src/semble/index/create.py index 4c354779..129ec8e6 100644 --- a/src/semble/index/create.py +++ b/src/semble/index/create.py @@ -40,6 +40,8 @@ _MIN_FILES_FOR_PROCESSES = 200 # Files handed to the workers but not yet consumed by the main process. _MAX_FILES_IN_FLIGHT = 256 +# New chunks are embedded together once this many have accumulated. +_EMBED_BATCH_SIZE = 1024 # A file's chunks and their BM25 tokens. _ChunkedFile = tuple[list[Chunk], list[list[str]]] @@ -177,9 +179,10 @@ def create_index_from_path( chunks: list[Chunk] = [] chunk_ids: list[str] = [] - vector_parts: list[EmbeddingMatrix] = [] manifest: dict[str, FileManifestEntry] = {} - embedding_parts: list[tuple[int, int, int]] = [] + reused_parts: list[tuple[int, EmbeddingMatrix]] = [] + embedded_parts: list[tuple[list[int], EmbeddingMatrix]] = [] + new_rows: list[int] = [] skipped_large: list[str] = [] @@ -201,23 +204,25 @@ def create_index_from_path( miniters=1, # tqdm's adaptive miniters stalls the bar when fast files are followed by slow ones disable=not show_progress_bar, ): + start = len(chunks) if previous is not None and previous_entry is not None and previous_entry.mtime_ns == mtime_ns: file_chunks = previous.chunks[previous_entry.start : previous_entry.end] - vector_parts.append(previous.vectors[previous_entry.start : previous_entry.end]) + reused_parts.append((start, previous.vectors[previous_entry.start : previous_entry.end])) else: result = next(chunked) if result is None: continue file_chunks, file_tokens = result _reindex_file(bm25_index, indexed_path, file_tokens, previous_entry) + new_rows.extend(range(start, start + len(file_chunks))) - embedding_parts.append((len(vector_parts), len(chunks), len(file_chunks))) - vector_parts.append(embed_chunks(model, file_chunks)) - - start = len(chunks) chunks.extend(file_chunks) chunk_ids.extend(make_chunk_id(indexed_path, slot) for slot in range(len(file_chunks))) manifest[indexed_path] = FileManifestEntry(mtime_ns=mtime_ns, start=start, count=len(file_chunks)) + if len(new_rows) >= _EMBED_BATCH_SIZE: + embedded_parts.append((new_rows, embed_chunks(model, [chunks[row] for row in new_rows]))) + new_rows = [] + embedded_parts.append((new_rows, embed_chunks(model, [chunks[row] for row in new_rows]))) for indexed_path in previous_manifest.keys() - manifest.keys(): _reindex_file(bm25_index, indexed_path, [], previous_manifest[indexed_path]) @@ -229,10 +234,12 @@ def create_index_from_path( if previous is not None and _has_same_vector_layout(manifest, previous_manifest): embeddings = previous.vectors - for vector_part, start, count in embedding_parts: - embeddings[start : start + count] = vector_parts[vector_part] else: - embeddings = np.vstack(vector_parts) + embeddings = np.empty((len(chunks), model.dim), dtype=np.float32) + for start, vectors in reused_parts: + embeddings[start : start + len(vectors)] = vectors + for rows, vectors in embedded_parts: + embeddings[rows] = vectors bm25_index.set_doc_order(chunk_ids) semantic_index = SelectableBasicBackend(embeddings, BasicArgs()) diff --git a/tests/index/test_create.py b/tests/index/test_create.py index 15b0936f..a9feed76 100644 --- a/tests/index/test_create.py +++ b/tests/index/test_create.py @@ -1,5 +1,6 @@ import multiprocessing import threading +import zlib from pathlib import Path from types import SimpleNamespace from typing import Any @@ -26,10 +27,20 @@ def _write_files(root: Path, files: dict[str, str]) -> None: @pytest.mark.parametrize("in_processes", [False, True]) +@pytest.mark.parametrize(("embed_batch_size", "encode_calls"), [(1024, 1), (3, 2)]) def test_incremental_reindex_reuses_updates_and_prunes( - mock_model: Any, tmp_path: Path, monkeypatch: pytest.MonkeyPatch, in_processes: bool + mock_model: Any, + tmp_path: Path, + monkeypatch: pytest.MonkeyPatch, + in_processes: bool, + embed_batch_size: int, + encode_calls: int, ) -> None: """One incremental pass reuses unchanged vectors, re-embeds changes, and keeps BM25 slots current.""" + monkeypatch.setattr(create, "_EMBED_BATCH_SIZE", embed_batch_size) + mock_model.encode.side_effect = lambda texts, **_: np.eye(mock_model.dim, dtype=np.float32)[ + [zlib.crc32(text.encode()) % mock_model.dim for text in texts] + ] if in_processes: monkeypatch.setattr(create, "_MIN_FILES_FOR_PROCESSES", 0) monkeypatch.setenv("SEMBLE_INDEX_WORKERS", "2") @@ -45,7 +56,7 @@ def test_incremental_reindex_reuses_updates_and_prunes( bm25_before, semantic_before, chunks_before, manifest_before = create_index_from_path( tmp_path, mock_model, display_root=tmp_path ) - assert mock_model.encode.call_count == 4 # once per file, no second full pass + assert mock_model.encode.call_count == encode_calls a_entry = manifest_before["a.py"] b_entry = manifest_before["b.py"] a_vectors_before = semantic_before.vectors[a_entry.start : a_entry.end].copy() @@ -74,7 +85,7 @@ def test_incremental_reindex_reuses_updates_and_prunes( (tmp_path / "c.py").unlink() (tmp_path / "emptying.py").write_text(" " * 128) _write_files(tmp_path, {"d.py": "def brand_new_term():\n return 4\n"}) - bm25_after, semantic_after, _, manifest_after = create_index_from_path( + bm25_after, semantic_after, chunks_after, manifest_after = create_index_from_path( tmp_path, mock_model, display_root=tmp_path, previous=previous ) @@ -85,6 +96,7 @@ def test_incremental_reindex_reuses_updates_and_prunes( b_vectors_before, semantic_after.vectors[b_entry_after.start : b_entry_after.end], ) + np.testing.assert_array_equal(semantic_after.vectors, mock_model.encode([chunk.content for chunk in chunks_after])) assert "c.py" not in manifest_after assert "d.py" in manifest_after assert manifest_after["emptying.py"].count == 0 diff --git a/uv.lock b/uv.lock index f089a0ed..d50ade69 100644 --- a/uv.lock +++ b/uv.lock @@ -16,6 +16,7 @@ exclude-newer = "0001-01-01T00:00:00Z" # This has no effect and is included for exclude-newer-span = "P1W" [options.exclude-newer-package] +model2vec = false semble-grammars = false [[package]] @@ -1881,9 +1882,10 @@ wheels = [ [[package]] name = "model2vec" -version = "0.9.0" +version = "0.10.0" source = { registry = "https://pypi.org/simple" } dependencies = [ + { name = "huggingface-hub" }, { name = "jinja2" }, { name = "joblib" }, { name = "numpy", version = "2.2.6", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.11'" }, @@ -1893,9 +1895,9 @@ dependencies = [ { name = "tokenizers" }, { name = "tqdm" }, ] -sdist = { url = "https://files.pythonhosted.org/packages/fd/e5/118c4a8af078ff97d9228718fbcd7d4f7e9cae93c3af59652d6f3407010a/model2vec-0.9.0.tar.gz", hash = "sha256:f50229cea128c9db5cfa7b2173478294be3c84e5d3d7fb8487ebd7af285383ab", size = 4590528, upload-time = "2026-08-12T14:24:38.524Z" } +sdist = { url = "https://files.pythonhosted.org/packages/04/48/2ceace43798ca7cb8c32c50c7956d7446d7f90a3b3f65b201fb7fd87344c/model2vec-0.10.0.tar.gz", hash = "sha256:47119ef1f917b4da7a0010bf10c587c923639a6af4ea51cb2d424fc329d09d2f", size = 4622501, upload-time = "2026-10-11T10:47:59.266Z" } wheels = [ - { url = "https://files.pythonhosted.org/packages/af/ea/80246465cafa36a6c8c8ac767778423940e5b826fa57c10ebd957b570c1c/model2vec-0.9.0-py3-none-any.whl", hash = "sha256:8bcf3258d5668678739c13226a562d39eeaeb8fcac9b142bc4aceef8800d5b9d", size = 59865, upload-time = "2026-08-12T14:24:36.626Z" }, + { url = "https://files.pythonhosted.org/packages/84/c5/2b1b5648195bfcada5de6a1164d560c968ded8fc68f2ddd6118b21cb26b7/model2vec-0.10.0-py3-none-any.whl", hash = "sha256:a39c40de4f82c5e0e50c14bbb87a935c1c62094f3f0f0d8fde8e24f4a4f35b16", size = 89504, upload-time = "2026-10-11T10:47:57.679Z" }, ] [[package]] @@ -3866,7 +3868,7 @@ requires-dist = [ { name = "einops", marker = "extra == 'benchmark'", specifier = ">=0.8.2" }, { name = "matplotlib", marker = "extra == 'benchmark'", specifier = ">=3.7" }, { name = "mcp", marker = "extra == 'mcp'", specifier = ">=2.0,<3.0" }, - { name = "model2vec", specifier = ">=0.4.0,<0.10" }, + { name = "model2vec", specifier = ">=0.10.0,<0.11" }, { name = "mypy", marker = "extra == 'dev'", specifier = ">=1.0" }, { name = "numpy", specifier = ">=1.24.0" }, { name = "numpy", marker = "extra == 'benchmark'", specifier = ">=1.24.0" },