Enhance Pagination Logic for Get All Submodels and Submodel Descriptors

[mm-kgi]

Overview

This PR enhances the pagination flow for the following endpoints:

• GET /submodels
• GET /submodel-descriptors

The current implementation retrieves all Shell metadata from the Plugin, generates all Submodels/Submodel Descriptors, and only then applies pagination. This can result in unnecessary metadata retrieval and Submodel generation for large datasets.

This change introduces an improved pagination flow that performs pagination earlier in the processing pipeline while preserving the existing logical ordering and ensuring that no Submodels are skipped or duplicated.

Endpoints Covered

The same pagination approach is applied consistently to:

• GET /submodels
• GET /submodel-descriptors

This ensures both endpoints follow the same cursor and ordering semantics.

Edge Cases

The implementation considers:

• Cursor pointing to a Submodel in the middle of a Shell.
• Shells containing different numbers of Submodels.
• Partially processed Shells.
• Multiple Shell Templates.
• Cursor pointing to the last Submodel of a Shell.
• Transition from one Shell to the next.
• Ensuring no duplicate or skipped Submodels across pages.

Validation

The implementation should be validated against scenarios including:

• Cursor within the first Shell.
• Cursor at the last Submodel of a Shell.
• Cursor across multiple Shells.
• Multiple Shell Templates.
• Different numbers of Submodels per Shell.
• Small and large page sizes.
• First page without a cursor.
• Subsequent pages using the returned cursor.
• Ensuring consecutive pages contain no duplicates or missing Submodels.

Summary

This PR improves pagination by moving pagination closer to the metadata retrieval stage while retaining Submodel-level pagination semantics.

The solution ensures that:

• Plugin metadata retrieval is paginated.
• submodelId and aasId cursor handling is supported.
• Ordering is preserved.
• Submodels are neither skipped nor duplicated.
• Multiple Shell Templates are supported.
• Both /submodels and /submodel-descriptors follow consistent pagination behavior.
• Large datasets can be processed more efficiently.

[github-actions]

Dependency Review

✅ No vulnerabilities or license issues or OpenSSF Scorecard issues found.

Scanned Files

None

[Copilot]

Pull request overview

Moves submodel pagination closer to shell metadata retrieval for repository and registry endpoints.

Changes:

• Adds composite submodel/AAS cursors and shared pagination state.
• Propagates pagination through plugin metadata APIs and parallelizes requests.
• Updates configuration, tests, and expected responses.

Reviewed changes

Copilot reviewed 26 out of 26 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
SubmodelRepository/TestData/GetAllSubmodels_Expected.json	Updates expected IDs and formatting.
SubmodelRepository/TestData/GetAllSubmodels_ByIdShort_Expected.json	Updates filtered response IDs.
SubmodelRepository/TestData/GetAllSubmodels_By_SemanticId_Expected.json	Updates semantic-filter response IDs.
SubmodelRegistry/TestData/GetAllSubmodelDescriptors_WithLimitOne_Expected.json	Updates composite cursor.
SubmodelRegistry/TestData/GetAllSubmodelDescriptors_WithCursor_Expected.json	Updates cursor-page expectations.
TemplateManagementConfig.cs	Raises default concurrency limit.
PluginDataProvider.cs	Adds paginated metadata requests and parallel data requests.
PluginDataHandler.cs	Propagates metadata pagination.
CachedGetRequestClient.cs	Adds framework imports.
SubmodelPaginationCursor.cs	Defines composite cursor encoding.
appsettings.development.json	Raises development concurrency limits.
SubmodelRepositoryService.cs	Implements early submodel pagination.
SubmodelDescriptorService.cs	Implements early descriptor pagination.
SubmodelPagination.cs	Adds shared pagination state.
IPluginDataProvider.cs	Extends provider pagination contract.
IPluginDataHandler.cs	Extends handler pagination contract.
AssetIdSearchService.cs	Adapts handler invocation.
AasRepositoryService.cs	Adapts handler invocation.
PluginDataProviderTests.cs	Updates provider tests.
PluginDataHandlerTests.cs	Updates handler tests.
CachedGetRequestClientTests.cs	Normalizes source encoding.
SubmodelRepositoryServiceTests.cs	Adds pagination-boundary tests.
SubmodelDescriptorServiceTests.cs	Adds composite-cursor tests.
AssetIdSearchServiceTests.cs	Updates pagination arguments.
AasRepositoryServiceTests.cs	Updates handler expectations.
SubmodelDescriptorControllerTests.cs	Updates repository mocks.

Suppressed comments (1)

source/AAS.TwinEngine.DataEngine/ApplicationLogic/Services/Shared/SubmodelPagination.cs:34

• This no longer preserves the previous global Distinct() behavior: a submodel referenced by multiple shells is appended once per shell, so /submodels and /submodel-descriptors can return duplicates within or across pages. The prior regression test for shared submodel IDs was removed rather than adapted. Track IDs already emitted across the shell stream (with cursor state that remains valid across pages) before adding them.

        for (var i = startIndex; i < submodelIds.Count; i++)
        {
            CollectedIds.Add(submodelIds[i]);
            LastCollectedSubmodelId = submodelIds[i];

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[sonarqubecloud]

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

[github-actions]

Test & Coverage Report

Test Results Summary

Metric	Count
✅ Passed	1689
❌ Failed	0
⏭️ Skipped	0

View Detailed Test Results

---

Code Coverage

Unit Tests Coverage

Package	Line Rate	Branch Rate	Complexity	Health
AAS.TwinEngine.DataEngine	92%	78%	2402	✔
Summary	92% (3816 / 4154)	78% (1580 / 2037)	2402	✔

Minimum allowed line rate is 80%

Module Tests Coverage

Package	Line Rate	Branch Rate	Complexity	Health
AAS.TwinEngine.DataEngine	62%	42%	2402	➖
Summary	62% (2584 / 4154)	42% (863 / 2037)	2402	➖