[gregrholden]

Description:

DEV-14680: Mechanism for running request through Guardrails
This branch includes a new services file (guardrails.py) in the llm application. It provides a class that creates the requisite boto3 client to send queries to Bedrock Guardrails. A new response status code has been added to the filter-search endpoint to account for the added Guardrails behavior: Status 503 if Guardrails is unavailable or if the client is misconfigured in the app. With the async version of the endpoint in place, Guardrails error codes are now packaged in a JsonResponse with an integer value passed to a "status" attribute instead of as a DRF Response object. This should not be an issue, just an update in how it's packaged on the backend.

The filter-search endpoint now also sends a Status 400 response if Guardrails intervenes and blocks the query.
This PR will require coordination with Ops and the Frontend team.

Technical Details:

IMPORTANT: If the Guardrails service is unavailable/unreachable, the Filter Search endpoint will cease functioning since it will block the query and return a 503 status code.
The llm/v2/views/filter_search.py endpoint was updated with a new function to handle passing queries to Bedrock Guardrails and returning its assessments. The endpoint can now send a 503 status if there is a misconfiguration in the app/client or if Guardrails is unavailable/unreachable. The endpoint will now send a 400 status if Guardrails intervenes.

A new file at llm/services/guardrails.py defines the BedrockGuardrailService class that:

• Retrieves the Guardrail version from the Guardrail specified by Guardrail ID in the env vars.
• Caches the version for 5 minutes.
• Instantiates the boto3 client for Bedrock Guardrails.
• Runs the Guardrail assessment and returns a GuardrailAssessment object prior to running the query against the LLM.
• If Guardrails intervenes, the endpoint receives the intervention message and blocks the query from reaching the LLM, sending a 400 status code to the frontend, with details of the reason for being blocked logged by Django.

This change requires coordination with Ops to provision:

• Environment variables in the ECS task definition (via terraform):
  -- AWS_BEDROCK_GUARDRAIL_ID (can be either the Guardrail ID or the Guardrail ARN, either works)

• The ECS task definition will also need to include permissions to use the bedrock:ApplyGuardrail API.

Guardrail status codes are no longer wrapped in Response objects. They are packaged in JsonResponse objects with a custom "status" attribute containing an integer value of 400 or 503, depending on the error.
Frontend team needs to be made aware that the filter-search endpoint can now also distribute 503 status codes if Guardrails or the app are not configured to communicate.
Frontend team needs to also be made aware that should Guardrails intervene, they will receive a 400 status code.
The API contract has been updated with these changes.

Requirements for PR Merge:

1. Unit & integration tests updated
2. API documentation updated (examples listed below)
   1. API Contracts
   2. API UI
   3. Comments
3. Data validation completed (examples listed below)
   1. Does this work well with the current frontend? Or is the frontend aware of a needed change?
   2. Is performance impacted in the changes (e.g., API, pipeline, downloads, etc.)?
   3. Is the expected data returned with the expected format?
4. Appropriate Operations ticket(s) created
5. Jira Ticket(s)
   1. DEV-14680

Explain N/A in above checklist:

[gregrholden]

@sethstoudenmier @zachflanders-frb This is ready for another review. According to Ops, Guardrails is in place in NPC DEV so we can try testing it there before merging if desired.

[zachflanders-frb]

I updated the view to return a search_error event if a guardrail is triggered in order to adhere to our api contract. I updated the api contract to make the search_id optional since this error is raised before the session is created so the search_id would be null.