[UI-REWRITE] Add tool preview tab - #53
Conversation
Signed-off-by: Pratik Gandhi <gandhipratik203@gmail.com>
Signed-off-by: Pratik Gandhi <gandhipratik203@gmail.com>
|
Depends on IBM/mcp-context-forge#5629 :( |
marekdano
left a comment
There was a problem hiding this comment.
Findings
1. Silent data loss in Try It form
File: src/components/tools/ToolTryItTab.tsx:34
Category: High
The reset effect depends on selectedTool.inputSchema by object reference, so any unrelated tools-list update silently wipes the user's in-progress arguments, headers, and preview result.
Failure scenario: User opens the Try It tab, fills in arguments, gets a preview result, then adds a tag via the always-visible sidebar control (handleAddToolTag in src/pages/Tools.tsx replaces the tool with a freshly-fetched object). ToolTryItTab doesn't remount (same tool id), but its effect fires because inputSchema is a new object reference, resetting args to defaults and discarding headers/preview — without the user touching the Try It form.
2. Content-Type header can be overridden
File: src/components/tools/ToolHeadersEditor.tsx:23
Category: Medium
DENIED_HEADERS blocks auth-sensitive headers but not content-type or x-requested-with, letting a user-entered header override the JSON Content-Type that client.ts always sets on preview requests.
Failure scenario: User adds a header named Content-Type with an arbitrary value in the Try It headers editor; it isn't rejected, gets spread into extraHeaders in requestWithMeta, and can duplicate/override the Content-Type header sent with the JSON preview body, breaking backend parsing.
3. ".." tool name misroutes requests
File: src/api/tools.ts:89
Category: Medium
TOOL_NAME_PATTERN permits a tool name of exactly "..", which survives encodeURIComponent and gets collapsed by URL path normalization, misrouting the
preview request to a different endpoint.
Failure scenario: toolsApi.preview('..') builds path /tools/preview/..; new URL() resolution in getRequestUrl normalizes this to /api/tools/, so the POST (with the preview request body) hits the tools list endpoint instead of failing validation as an invalid name.
4. Nested required-field logic ignores optional parent
File: src/components/tools/ToolArgumentsForm.tsx:33
Category: Medium
Nested object fields one level deep are always marked required from their own schema's required list, ignoring whether the parent object itself is optional.
Failure scenario: A schema declares an optional owner object containing a required email sub-property, with owner absent from the top-level required array. The generated form still forces the user to fill owner.email, blocking the Preview button even though omitting owner entirely should be valid.
5. NaN bypasses "number" type validation
File: src/components/tools/ToolArgumentsForm.tsx:118
Category: Low
validateToolArguments checks "number" fields with typeof current !== "number", which does not catch NaN, unlike the "integer" branch which correctly uses Number.isInteger.
Failure scenario: A "number" field ends up with value NaN (e.g. via paste or a non-sanitizing input path); no validation error is shown, and JSON.stringify silently turns NaN into null in the preview request body with no indication to the user that their input was dropped.
6. Duplicated header-editor logic
File: src/components/tools/ToolHeadersEditor.tsx:1
Category: Low
ToolHeadersEditor re-implements the add/remove/update key-value row pattern already present in src/components/mcp-servers/CustomHeadersAuth.tsx instead of sharing a common component.
Impact: Not a runtime bug — future header-editing fixes (e.g. tightening name validation, accessibility tweaks) must be duplicated across both files, risking drift between the two nearly-identical editors.
Signed-off-by: Pratik Gandhi <gandhipratik203@gmail.com>
|
Thanks for the detailed review. Addressed items 1-5 in
For item 6, I kept the header editor local to this PR because preview passthrough headers have different validation/security behavior than MCP server auth headers. |
Signed-off-by: Pratik Gandhi <gandhipratik203@gmail.com>
gcgoncalves
left a comment
There was a problem hiding this comment.
Solid. Particularly, well done with using Vite config for setting the feature flag. 👏
marekdano
left a comment
There was a problem hiding this comment.
Nice work addressing items 1–5, and thanks for the explanation on 6. Two things worth fixing before VITE_ENABLE_TOOL_PREVIEW is flipped on by default:
1. Raw-JSON editor clobbers input while typing
File: src/components/tools/ToolArgumentsForm.tsx:184 (raw JSON fallback for complex schemas)
Category: High
The textarea's onChange calls onChange(parsed) on every valid keystroke, which updates the parent value. The useEffect watching value (line ~146) then calls setRawJson(JSON.stringify(value, null, 2)), re-formatting and overwriting the textarea the instant the JSON becomes valid.
Failure scenario: Typing compact JSON like {"query":"cloudflare"} character by character — the moment it parses successfully, the field gets reformatted to indented multi-line JSON mid-keystroke, resetting the cursor and corrupting further typing. Makes the raw-JSON editor effectively unusable for anything beyond a single paste.
2. Tool name still not trimmed before use
File: src/api/tools.ts:96-101
Category: Low
validateToolName computes trimmed to check for ./.., but still runs TOOL_NAME_PATTERN.test(name) and returns the untrimmed name. A name with incidental leading/trailing whitespace (the pattern allows spaces) is sent untrimmed into the preview URL.
Failure scenario: toolsApi.preview(" search_issues ") builds /tools/preview/%20search_issues%20, which won't match the backend's tool lookup - a 404 on an otherwise-valid tool. Fix: use trimmed in both the pattern test and the return.
Not blocking merge since the feature is behind the flag and already gated on the backend dependency, but flagging so No. 1 in particular gets fixed before the flag goes live.
Summary
PR split plan
This PR is the first slice of issue #5630: it adds the mocked Try it workflow. Result rendering and live invocation are left for follow-up PRs.
Scope
Closes IBM/mcp-context-forge#6316
Refs IBM/mcp-context-forge#5630
This is PR 1 of the split. Full content-block rendering remains in IBM/mcp-context-forge#6317, and live invocation/spec-aware snippets remain in IBM/mcp-context-forge#6318.
Temporary flag removal is tracked in IBM/mcp-context-forge#6322.
Tests
npm run testnpm run lintnpx tsc --noEmit -p tsconfig.app.jsonnpm run format:checknpm run e2e -- e2e/tools.spec.tsExplanatory diagrams
PR file/folder tree
Before vs after
Manual verification
Manual test steps
Setup
Save the mock script from the next collapsible at the repo root as
tool-preview-manual.mjs.Two terminals:
Terminal B opens a Chrome for Testing window with
/auth/session,/api/rbac/my/permissions,/api/tools,/api/gateways, and/api/tools/preview/search_issuesmocked. Ctrl-C in terminal B to close. Do everything in that window, in the tab it opens.Steps
1. Open More options for github-server -> View details.
Expect: the details drawer opens with tabs Try it and Definition. Try it is selected by default.
2. Check the Try it tab.
Expect: Tool preview is visible, the
Search repository issuesdescription is shown, and the Read-only badge appears fromreadOnlyHint.3. Leave
queryempty.Expect: Preview is disabled because
queryis required by the input schema.4. Fill
querywithcloudflareandlimitwith5.Expect: Preview becomes enabled.
5. Click Add header. Enter
X-Api-Keyas the header name andteam-aas the value, then click Preview.Expect: result shows Preview 200, Resolved arguments, and Raw preview response.
6. Look at terminal B.
Expect: the preview request body is
{ "arguments": { "query": "cloudflare", "limit": 5 } }, and the logged passthrough headers includex-api-key: team-a.7. Change
X-Api-KeytoX-Tenant-Id, then click Re-run.Expect: the logged passthrough headers include
x-tenant-id: team-a.8. Change
X-Tenant-IdtoAuthorization.Expect: the header input turns red, the inline warning says
This header is not forwardable from the web UI., and Re-run is disabled.9. Click Definition.
Expect: the existing tools table is visible, including row actions like schema/edit/delete/toggle. This confirms the old details-table surface moved under the Definition tab rather than disappearing.
Teardown
Ctrl-C both terminals. If :5173 is stuck:
Mock script (tool-preview-manual.mjs)
Save at the repo root. Requires
@playwright/test, already a dev dependency; runnpx playwright install chromiumif the browser is missing.Manual test results
Run against
feat/6316-tool-preview-draweratdec90ff, branched frommainat414b714.github-serverquery=cloudflare,limit=5X-Api-Key=team-a{ "arguments": { "query": "cloudflare", "limit": 5 } }; header includesx-api-key: team-aX-Tenant-Id=team-ax-tenant-id: team-aAuthorizationObserved terminal output for allowed headers:
Scope of this verification: all backend responses are mocked. This covers frontend wiring only: drawer tabs, schema argument form, passthrough-header filtering, preview request construction, and basic preview result rendering. It does not verify a live backend preview endpoint. Full content-block rendering is tracked by IBM/mcp-context-forge#6317; live invocation/spec-aware snippets are tracked by IBM/mcp-context-forge#6318.