feat(secops): add Detection Engineering Agent MCP tools and coverage evaluation skill - #305
Open
dandye wants to merge 9 commits into
Open
feat(secops): add Detection Engineering Agent MCP tools and coverage evaluation skill#305dandye wants to merge 9 commits into
dandye wants to merge 9 commits into
Conversation
…evaluation skill Implement the official Google SecOps Agentic Detection Engineering (ADE) toolset and end-to-end coverage evaluation workflow: - MCP Tools (`server/secops/secops_mcp/tools/detection_agent.py`): * `generate_threat_detection_opportunity`: Generates structured Threat Detection Opportunities (TDOs) from threat descriptions and log types. * `generate_synthetic_events`: Simulates attacker behavior by synthesizing raw logs and UDM events for given TDOs. * `evaluate_rule_coverage_long_running`: Initiates Chronicle LRO (`:evaluateRuleCoverageLongRunning`) to test synthetic events against active rulesets without committing mock logs to storage. * `get_operation`: Polls long-running operation status for asynchronous evaluation results. * `generate_rules`: Synthesizes candidate YARA-L 2.0 detection rules to close identified coverage gaps. * Exported in `server/secops/secops_mcp/tools/__init__.py`. - Skill & Documentation: * Packaged `detection-engineering-coverage-evaluation` skill in `extensions/google-secops/skills/detection-coverage/SKILL.md` with prompt-injection defense, 8-step lifecycle, and explicit user-confirmation safeguards before deploying rules. * Documented tool mapping in `extensions/google-secops/TOOL_MAPPING.md` and `GEMINI.md`. * Updated Detection Engineer persona documentation in `docs/personas/detection_engineer.md`. - Testing & Safeguards: * Added unit test suite covering API calling conventions, payload serialization, error handling, and parameter aliases in `server/secops/tests/test_secops_detection_agent_unit.py`. * Verified against live Google SecOps tenant with Application Default Credentials (ADC). * Added `.envrc` and `.gcloud/` to `.gitignore` to prevent credential leakage. TAG=agy CONV=b434fecb-6fd3-4841-b909-49d23ef70fa3
…ntation Document Google SecOps Agentic Detection Engineering (ADE) APIs and workflows: - Created `docs/agentic_detection_engineering.md` detailing the 8-stage lifecycle (TDO generation, synthetic telemetry simulation, sandboxed LRO rule coverage evaluation, gap analysis, and candidate YARA-L 2.0 rule synthesis). - Embedded SecOps UI configuration for Synthetic Data Visibility (`docs/img/synthetic_data_visibility.png`). - Updated `docs/servers/secops_mcp.md` with all 5 ADE MCP tools (`generate_threat_detection_opportunity`, `generate_synthetic_events`, `evaluate_rule_coverage_long_running`, `get_operation`, `generate_rules`) and Example 9 walkthrough. - Updated `docs/google_secops_extension.md` to register Skill 6 (`detection-engineering-coverage-evaluation`). - Updated `docs/toc.md` and `docs/index.md` navigation and quick links. TAG=agy CONV=b434fecb-6fd3-4841-b909-49d23ef70fa3
…et_rule tool - Add threat_description, threatDescription, log_types, logTypes parameter aliases in generate_threat_detection_opportunity. - Add plural parameter aliases (threat_detection_opportunities, tdo), raw dictionary unwrapping, and multi-TDO batching in generate_synthetic_events. - Add tdo_events, tdoEvents, opportunity_events, opportunityEvents aliases and raw wrapper dictionary handling in evaluate_rule_coverage_long_running. - Add operation_name, operationName aliases and endpoint normalization with auto-prefixed operations/ in get_operation. - Add threat_detection_opportunities, tdo, background_context, backgroundContext aliases, raw wrapper dictionary unwrapping, and multi-TDO batching in generate_rules. - Register get_rule tool alias in security_rules.py for full parity with Remote OneMCP server tool definitions. - Expand unit test suite in test_secops_detection_agent_unit.py with 7 new comprehensive test scenarios covering aliases, batching, and get_rule. TAG=agy CONV=b434fecb-6fd3-4841-b909-49d23ef70fa3
- Update evaluate_rule_coverage_long_running docs to show exclude_composite_coverage defaults to True. - Fix TOOL_MAPPING link in docs/google_secops_extension.md to point to GitHub file location. TAG=agy CONV=b434fecb-6fd3-4841-b909-49d23ef70fa3
…xpand wrapper dict unwrapping - Synthesize threat_detection_opportunity_events in single-TDO and multi-TDO paths of generate_synthetic_events to prevent KeyError in downstream tools. - Coerce string log_types to single-element list in generate_threat_detection_opportunity and _clean_tdo. - Unwrap tdo_events, tdoEvents, opportunity_events, opportunityEvents wrapper dictionaries in evaluate_rule_coverage_long_running. - Add warning logs when invalid TDOs are skipped during multi-TDO batching. - Update unit test suite to verify single-TDO tdo_events synthesis, string log_types coercion, and tdo_events wrapper key unwrapping. TAG=agy CONV=b434fecb-6fd3-4841-b909-49d23ef70fa3
- Fix create_security_rule typo to match create_rule tool name. - Clarify activation step instructions. TAG=agy CONV=b434fecb-6fd3-4841-b909-49d23ef70fa3
docs(secops): add Agentic Detection Engineering guide and tool documentation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Implements support for the newly launched Google SecOps Agentic Detection Engineering (ADE) APIs in
secops-mcpand introduces the canonicaldetection-engineering-coverage-evaluationskill for Antigravity and Gemini CLI agents.This enables security teams and autonomous agents to automate the end-to-end detection engineering lifecycle: ingesting threat intelligence, generating Threat Detection Opportunities (TDOs), simulating attacker behavior with synthetic UDM events, running isolated rule coverage evaluations via Chronicle Long-Running Operations (LRO), synthesizing candidate YARA-L 2.0 rules for identified coverage gaps, and deploying verified detections with explicit human-in-the-loop authorization.
Reference documentation: Agentic Detection Engineering in Google SecOps
Changes
1. New Detection Engineering MCP Tools (
server/secops/secops_mcp/tools/detection_agent.py)Adds 5 new tools to the SecOps FastMCP toolset:
generate_threat_detection_opportunity: Generates structured Threat Detection Opportunities (TDOs) from natural-language threat intel descriptions and target log types.generate_synthetic_events: Simulates attacker activity by synthesizing realistic raw logs, UDM event records, and JSON-encoded UDM strings for specific TDOs.evaluate_rule_coverage_long_running: Initiates a Chronicle LRO (:evaluateRuleCoverageLongRunning) to evaluate synthetic events against active rulesets in a safe sandbox simulation without writing mock data to production storage. Supports optional composite coverage exclusion.get_operation: Polls standard Google Long-Running Operations (LRO) until completion, surfacing progress metadata and final evaluation responses.generate_rules: Synthesizes production-ready candidate YARA-L 2.0 detection rules specifically tailored to close uncovered TDOs and coverage gaps.All tools are registered in FastMCP and exported via
server/secops/secops_mcp/tools/__init__.py.2. Detection Engineering Coverage Evaluation Skill (
extensions/google-secops/skills/detection-coverage/)Adds the
detection-engineering-coverage-evaluationskill (available via.agent/skills/detection-coverage/SKILL.md):3. Persona & Tool Mapping Documentation
docs/personas/detection_engineer.md: Updated Detection Engineer persona documentation to detail ADE tools, workflows, and sample execution flows.extensions/google-secops/TOOL_MAPPING.md: Added the Detection Engineering tool category mapping.extensions/google-secops/GEMINI.md: Added Skill 6 (detection-engineering-coverage-evaluation) to the extensions registry.4. Git Hygiene & Secret Isolation
.envrcand.gcloud/to.gitignoreto prevent credential and local configuration leakage when usingdirenvand local Application Default Credentials (ADC).Verification & Testing
Unit Tests
server/secops/tests/test_secops_detection_agent_unit.pycontaining 12 async unit tests.tdo_events,threat_detection_opportunities,opportunity_events), JSON serialization, and error handling for all 5 tools.uv run --with pytest --with pytest-asyncio pytest server/secops/tests/test_secops_detection_agent_unit.py # 12 passed in 0.65sLive Tenant Verification
Verified end-to-end execution against a live Google SecOps tenant using ADC:
generate_threat_detection_opportunity: Successfully extracted TDOs (t01,t02) for PowerShell-based C2 communication.generate_synthetic_events: Successfully generated 8 synthetic UDM events simulating suspicious PowerShell process launches and outbound network connections.evaluate_rule_coverage_long_running: Initiated LRO (operations/dea-bkFXS0...) with synthetic events.get_operation: Polled operation status to retrieve coverage results.generate_rules: Successfully synthesized candidate YARA-L 2.0 rule (ttp_suspicious_powershell_network_connection) closing the identified gap.TAG=agy
CONV=b434fecb-6fd3-4841-b909-49d23ef70fa3