Skip to content

Allow admins to customize Content Safety violation messages with Markdown support #989

Closed

Description

@paullizer

Summary

Admins should be able to customize the user-facing language shown when a chat message is blocked by Content Safety. The customized message should support Markdown so organizations can provide clearer policy language, links to internal acceptable-use policies, escalation instructions, or localized wording.

Today, the blocked message text is hard-coded in the chat backend when Azure Content Safety blocks a user message. The chat UI already renders safety messages through the existing Markdown + DOMPurify path, so this feature should expose the message as an admin-configurable setting while preserving sanitized rendering.

User Value

Organizations often need their own compliance, legal, HR, or security wording for safety violations. A configurable message lets admins:

  • Match the organization's tone and policy language.
  • Add links to internal acceptable-use or responsible AI policies.
  • Explain what the user should do next.
  • Support environment-specific or localized wording without code changes.

Proposed Behavior

Add an Admin Settings field under the existing Content Safety section that lets admins configure the blocked-message template.

The template should:

  • Support Markdown.
  • Have a safe default that matches or improves the current behavior.
  • Allow approved placeholders for violation context, such as:
    • {reasons}
    • {triggered_categories}
    • {blocklist_matches}
    • {violation_link}
  • Be used when a user message is blocked by Content Safety in chat.
  • Fall back to the default message if the admin-provided template is blank or invalid.
  • Render safely in the chat UI using the existing local Markdown and sanitization assets.

Acceptance Criteria

  • Admin Settings includes a Content Safety violation message template field in the existing Content Safety section.
  • The setting is persisted with the rest of app settings and has a documented default value.
  • When Content Safety blocks a user message, the persisted safety chat message uses the configured template instead of hard-coded copy.
  • Markdown in the configured message renders in the chat UI for both newly blocked messages and previously loaded safety messages.
  • Rendered Markdown is sanitized before insertion into the DOM.
  • The implementation does not add CDN-hosted JavaScript or frontend runtime dependencies.
  • Placeholder values are escaped/sanitized and cannot inject unsafe HTML or scripts.
  • Blank or malformed templates fall back to the default violation message.
  • Functional test coverage verifies default behavior, customized Markdown behavior, placeholder substitution, and fallback behavior.
  • Admin/settings documentation is updated to describe the template and supported placeholders.

Notes

Relevant implementation areas:

  • Backend hard-coded blocked message: application/single_app/route_backend_chats.py
  • Safety message persistence helper: application/single_app/route_backend_chats.py
  • Default settings: application/single_app/functions_settings.py
  • Admin Content Safety settings UI: application/single_app/templates/admin_settings.html
  • Admin settings JS behavior: application/single_app/static/js/admin/admin_settings.js
  • Safety message rendering path: application/single_app/static/js/chat/chat-messages.js

Current behavior builds a blocked message like:

  • Your message was blocked by Content Safety.
  • Reason
  • Triggered categories
  • Blocklist matches

The existing chat UI already routes safety messages through marked and DOMPurify, so this feature should reuse that local sanitized Markdown rendering path rather than introducing new frontend assets.

Open Questions

  • Should admins be able to preview the rendered Markdown in Admin Settings?
  • Should placeholders be required to preserve violation details, or should admins be allowed to hide category/blocklist details from end users?
  • Should the message support multiple templates in the future, such as separate templates for category severity blocks vs. blocklist matches?
  • Should there be a maximum template length?

Activity

  1. moved this from Pending Evaluation to In progress in Simple Chat Roadmapon Jul 27, 2026
  2. paullizer commented on Jul 27, 2026

    @paullizer
    CollaboratorAuthor

    Implementation is ready in PR #1079: #1079

    Completed:

    • Added a Markdown-enabled Safety Violation Message editor in Admin Settings with the existing local SimpleMDE toolbar.
    • Added an Include Trigger Information setting that controls whether reasons, categories/severities, and blocklist matches are appended.
    • Routed both normal and streaming Content Safety blocks through one shared formatter, while preserving the previous detailed default behavior.
    • Added seeded defaults, functional regression coverage, feature documentation, release notes, and version 0.250.060.

    Design decision:

    • The request is implemented with an explicit trigger-information toggle rather than dynamic Markdown placeholders. This lets administrators include or omit details without interpolating untrusted trigger content into their message template.

    Validation:

    • functional_tests/test_content_safety_violation_message.py: 5 passed.
    • Swagger and route-policy validation: passed (12 route-policy checks).
    • Changed-line XSS and broken-access-control guardrails: passed.
    • Python compilation and Git whitespace checks: passed.

    Follow-up:

    • Browser automation could not complete the admin-only visual check because the shared browser session received 403 for /admin/settings. The PR remains draft pending a quick toolbar confirmation in an admin-authenticated browser.
  3. moved this from In progress to In review in Simple Chat Roadmapon Jul 27, 2026
  4. paullizer commented on Jul 27, 2026

    @paullizer
    CollaboratorAuthor

    Final validation is complete in PR #1079: #1079

    Live administrator validation completed:

    • The Safety Violation Message field renders the existing 12-control local SimpleMDE Markdown toolbar.
    • Markdown-only edits enable Save Settings, synchronize to the submitted form, and persist after a full page reload.
    • A policy-blocked chat rendered the configured Markdown message with bold formatting, omitted trigger information while the toggle was disabled, and retained the Safety Violations link.
    • The original local Admin Settings message and trigger-information setting were restored after testing.

    Follow-up fix included:

    • d84eb597 stabilizes the hidden Safety-tab editor by refreshing CodeMirror after layout becomes visible and synchronizes editor changes with the Admin Settings form.
    • Final application version: 0.250.061.

    Final automated validation:

    • functional_tests/test_content_safety_violation_message.py: 5 passed.
    • Swagger and route-policy checks: passed.
    • Changed-line XSS and broken-access-control guardrails: passed.
    • Python compilation, JavaScript editor-script syntax, and whitespace checks: passed.

    The PR is now ready for review.

  5. paullizer commented on Aug 19, 2026

    @paullizer
    CollaboratorAuthor

    Shipped in v0.250.061 via #1079.

    Release note entries:

    • v0.250.061: Configurable Content Safety Violation Messages

    Full notes: https://microsoft.github.io/simplechat/explanation/release_notes/

    Closing as completed. This issue stayed open after the fix merged because pull requests target Development while the repository default branch is main, so GitHub never applied the Fixes # keyword automatically.

  6. moved this from In review to Released in Simple Chat Roadmapon Aug 19, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Labels

enhancementNew feature or request

Type

No type

Projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions