Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion application/single_app/config.py
Original file line number Diff line number Diff line change
Expand Up @@ -97,7 +97,7 @@
EXECUTOR_TYPE = 'thread'
EXECUTOR_MAX_WORKERS = 30
SESSION_TYPE = 'filesystem'
VERSION = "0.250.071"
VERSION = "0.250.072"
IS_DEVELOPMENT = is_development_env_enabled()

SESSION_COOKIE_SAMESITE = os.getenv('SESSION_COOKIE_SAMESITE', 'Lax')
Expand Down
79 changes: 70 additions & 9 deletions application/single_app/functions_data_management.py
Original file line number Diff line number Diff line change
Expand Up @@ -270,7 +270,7 @@
{"name": "public_workspaces", "container_attr": "cosmos_public_workspaces_container", "container_name_attr": "cosmos_public_workspaces_container_name", "partition_key_path": "/id", "id_field": "id"},
{"name": "public_documents", "container_attr": "cosmos_public_documents_container", "container_name_attr": "cosmos_public_documents_container_name", "partition_key_path": "/id", "filter_field": "public_workspace_id", "documents": True},
{"name": "public_workspace_identities", "container_attr": "cosmos_public_workspace_identities_container", "container_name_attr": "cosmos_public_workspace_identities_container_name", "partition_key_path": "/public_workspace_id", "filter_field": "public_workspace_id"},
{"name": "public_prompts", "container_attr": "cosmos_public_prompts_container", "container_name_attr": "cosmos_public_prompts_container_name", "partition_key_path": "/id", "filter_field": "public_workspace_id"},
{"name": "public_prompts", "container_attr": "cosmos_public_prompts_container", "container_name_attr": "cosmos_public_prompts_container_name", "partition_key_path": "/id", "filter_fields": ["public_id", "public_workspace_id"]},

Check warning on line 273 in application/single_app/functions_data_management.py

View workflow job for this annotation

GitHub Actions / malicious-pr-security-review

Important - Changed line contains AI, plugin, agent, or workspace boundary marker. Recommendation%3A Check whether prompts, chat history, uploaded documents, embeddings, citations, settings, or identity can cross a new boundary.

Check warning on line 273 in application/single_app/functions_data_management.py

View workflow job for this annotation

GitHub Actions / malicious-pr-security-review

Important - Changed line contains secret or sensitive data source marker. Recommendation%3A Pair this source with any nearby network, logging, serialization, or process execution sink before approving.
],
}

Expand Down Expand Up @@ -2189,6 +2189,44 @@
return restore_warnings, migration_state


def _get_selected_scope_filter_fields(container_definition):
"""Return the trusted ownership fields used for selected-scope Cosmos reads."""

Check warning on line 2193 in application/single_app/functions_data_management.py

View workflow job for this annotation

GitHub Actions / malicious-pr-security-review

Important - Changed line contains secret or sensitive data source marker. Recommendation%3A Pair this source with any nearby network, logging, serialization, or process execution sink before approving.
configured_fields = container_definition.get("filter_fields")
if configured_fields is None:
configured_fields = [container_definition.get("filter_field")]
elif isinstance(configured_fields, str):
configured_fields = [configured_fields]
elif not isinstance(configured_fields, (list, tuple)):
raise DataManagementSettingsValidationError(
"Migration container filter fields must be a string, list, or tuple."
)

filter_fields = []
for field_name in configured_fields:
if not field_name:
continue
if not isinstance(field_name, str) or not re.fullmatch(
r"[A-Za-z_][A-Za-z0-9_]*",
field_name,
):
raise DataManagementSettingsValidationError(
"Migration container filter fields must be valid Cosmos property names."

Check warning on line 2213 in application/single_app/functions_data_management.py

View workflow job for this annotation

GitHub Actions / malicious-pr-security-review

Important - Changed line contains secret or sensitive data source marker. Recommendation%3A Pair this source with any nearby network, logging, serialization, or process execution sink before approving.
)
filter_fields.append(field_name)
return filter_fields


def _build_selected_scope_filter_clause(filter_fields):
"""Build the static Cosmos ownership predicate for one selected scope ID."""

Check warning on line 2220 in application/single_app/functions_data_management.py

View workflow job for this annotation

GitHub Actions / malicious-pr-security-review

Important - Changed line contains secret or sensitive data source marker. Recommendation%3A Pair this source with any nearby network, logging, serialization, or process execution sink before approving.
filter_clauses = [
f"c.{field_name} = @selected_id"
for field_name in filter_fields
]
if len(filter_clauses) == 1:
return filter_clauses[0]
return f"({' OR '.join(filter_clauses)})"


def _iter_selected_cosmos_records(
container_definition,
selection,
Expand Down Expand Up @@ -2255,11 +2293,13 @@
yield prepare_item(item)
return

filter_field = container_definition.get("filter_field")
if not filter_field:
filter_fields = _get_selected_scope_filter_fields(container_definition)
if not filter_fields:
return
filter_clause = _build_selected_scope_filter_clause(filter_fields)
seen_identities = set() if len(filter_fields) > 1 else None
for selected_id in ids:
query = f"SELECT * FROM c WHERE c.{filter_field} = @selected_id"
query = f"SELECT * FROM c WHERE {filter_clause}"
parameters = [{"name": "@selected_id", "value": selected_id}]
if source_start_epoch is not None:
query += " AND c._ts >= @source_start_epoch"
Expand All @@ -2272,6 +2312,15 @@
parameters=parameters,
enable_cross_partition_query=True,
):
if seen_identities is not None:
item_identity = _get_cosmos_document_identity(

Check warning on line 2316 in application/single_app/functions_data_management.py

View workflow job for this annotation

GitHub Actions / malicious-pr-security-review

Important - Changed line contains secret or sensitive data source marker. Recommendation%3A Pair this source with any nearby network, logging, serialization, or process execution sink before approving.
item,
container_definition["partition_key_path"],
)
if item_identity and item_identity in seen_identities:
continue
if item_identity:
seen_identities.add(item_identity)
yield prepare_item(item)


Expand Down Expand Up @@ -4728,15 +4777,27 @@
except (CosmosResourceNotFoundError, ResourceNotFoundError):
continue
return
filter_field = container_definition.get("filter_field")
if not filter_field:
filter_fields = _get_selected_scope_filter_fields(container_definition)
if not filter_fields:
return
filter_clause = _build_selected_scope_filter_clause(filter_fields)
seen_identities = set() if len(filter_fields) > 1 else None
for selected_id in selection.get("ids") or []:
yield from target_container.query_items(
query=f"SELECT * FROM c WHERE c.{filter_field} = @selected_id",
for item in target_container.query_items(
query=f"SELECT * FROM c WHERE {filter_clause}",
parameters=[{"name": "@selected_id", "value": selected_id}],
enable_cross_partition_query=True,
)
):
if seen_identities is not None:
item_identity = _get_cosmos_document_identity(

Check warning on line 4792 in application/single_app/functions_data_management.py

View workflow job for this annotation

GitHub Actions / malicious-pr-security-review

Important - Changed line contains secret or sensitive data source marker. Recommendation%3A Pair this source with any nearby network, logging, serialization, or process execution sink before approving.
item,
container_definition["partition_key_path"],
)
if item_identity and item_identity in seen_identities:
continue
if item_identity:
seen_identities.add(item_identity)
yield item


def _delete_target_cosmos_record(target_container, document, partition_key_path):
Expand Down
43 changes: 43 additions & 0 deletions docs/explanation/fixes/PUBLIC_WORKSPACE_PROMPT_MIGRATION_FIX.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
# Public Workspace Prompt Migration Fix (v0.250.072)

Check warning on line 1 in docs/explanation/fixes/PUBLIC_WORKSPACE_PROMPT_MIGRATION_FIX.md

View workflow job for this annotation

GitHub Actions / malicious-pr-security-review

Important - Changed line contains AI, plugin, agent, or workspace boundary marker. Recommendation%3A Check whether prompts, chat history, uploaded documents, embeddings, citations, settings, or identity can cross a new boundary.

Fixed in version: **0.250.072**

Related issue: [#1033](https://github.com/microsoft/simplechat/issues/1033)

Check warning on line 5 in docs/explanation/fixes/PUBLIC_WORKSPACE_PROMPT_MIGRATION_FIX.md

View workflow job for this annotation

GitHub Actions / malicious-pr-security-review

Important - Changed line contains external connection or remote asset marker. Recommendation%3A Review whether changed code can send prompts, files, credentials, cookies, settings, logs, or user data to a new sink.

Related config.py version update: `VERSION = "0.250.072"`

## Issue Description

Selected public-workspace Data Management migrations omitted current public prompt records. Current prompts are stored in `public_prompts` with their workspace ownership in `public_id`, while selected-scope migration used only the legacy `public_workspace_id` ownership field.

Check warning on line 11 in docs/explanation/fixes/PUBLIC_WORKSPACE_PROMPT_MIGRATION_FIX.md

View workflow job for this annotation

GitHub Actions / malicious-pr-security-review

Important - Changed line contains AI, plugin, agent, or workspace boundary marker. Recommendation%3A Check whether prompts, chat history, uploaded documents, embeddings, citations, settings, or identity can cross a new boundary.

## Root Cause Analysis

The selected-scope Cosmos iterator generated one equality filter from each container definition's `filter_field`. The `public_prompts` definition still named `public_workspace_id`, so current prompts with only `public_id` never reached the migration copy path. All-workspaces migration did not use that selected-scope filter and was unaffected.

## Technical Details

### Files Modified

- `application/single_app/functions_data_management.py`
- `application/single_app/config.py`
- `functional_tests/test_data_management_public_prompt_migration.py`
- Data Management and version-contract functional tests
- `docs/explanation/release_notes.md`

### Code Changes Summary

- Added a compatibility-aware `filter_fields` contract for selected-scope Cosmos migration definitions while preserving existing single `filter_field` definitions.
- Configured `public_prompts` to match current `public_id` ownership first and legacy `public_workspace_id` ownership second.
- Builds a parameterized OR predicate for selected public-workspace prompt reads and deduplicates transitional records that contain both ownership fields by Cosmos document identity.
- Applied the same selected-scope matching and deduplication to destination reconciliation scans.
- Left all-workspaces migration on its existing unfiltered container-read path.

## Impact Analysis

Selected migrations now copy current and legacy prompts owned by selected public workspaces, exclude prompts owned by unselected workspaces, and count each copied prompt once. Existing user, group, and all-workspace migration behavior remains unchanged.

## Validation

- Focused regression coverage verifies current, legacy, transitional, unselected, and all-workspace public-prompt cases.
- The focused test asserts source-read and copied artifact counts for selected and all-workspace migrations.
- Relevant Data Management Cosmos migration, reconciliation, orchestration, and security/version checks are run with this change.
1 change: 1 addition & 0 deletions docs/explanation/fixes/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ order: 120
category: Version History
---

- [Public Workspace Prompt Migration Fix](PUBLIC_WORKSPACE_PROMPT_MIGRATION_FIX.md)
- [Azure OpenAI Model Discovery Identity Fix](v0.250.001/AZURE_OPENAI_MODEL_DISCOVERY_IDENTITY_FIX.md)
- [CosmosClient Import Binding CodeQL Fix](COSMOSCLIENT_IMPORT_BINDING_CODEQL_FIX.md)
- [Conversation Cache Invalidation Authorization Fix](CONVERSATION_CACHE_INVALIDATION_AUTHORIZATION_FIX.md)
Expand Down
9 changes: 9 additions & 0 deletions docs/explanation/release_notes.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,15 @@

For feature-focused and fix-focused drill-downs by version, see [Features by Version](/explanation/features/) and [Fixes by Version](/explanation/fixes/).

### **(v0.250.072)**

#### Bug Fixes

* **Selected Public Workspace Prompt Migration**
* Selected public-workspace Data Management migrations now copy current prompts owned through `public_id` while retaining compatibility with legacy `public_workspace_id` records.
* Prompts outside the selected workspaces remain excluded, transitional records migrate once, and copied prompt artifact counts are accurate. All-workspaces migration behavior is unchanged.
* (Ref: microsoft/simplechat#1033, `functions_data_management.py`, `test_data_management_public_prompt_migration.py`)

### **(v0.250.071)**

#### New Features
Expand Down
Loading
Loading