-
Notifications
You must be signed in to change notification settings - Fork 2.2k
fix: improve tool descriptions #965
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
OrKoN
merged 23 commits into
ChromeDevTools:main
from
madhaviai:fix/940-improve-mcp-invocation
Mar 5, 2026
Merged
Changes from 15 commits
Commits
Show all changes
23 commits
Select commit
Hold shift + click to select a range
50e813c
fix(mcp) : improve tool invocation reliability for agents
madhaviai 42d30c2
fix(mcp): update Core Web Vitals references from FID to INP
madhaviai 95b1d0a
chore(deps-dev): bump the dev-dependencies group with 4 updates (#983)
dependabot[bot] dcdd75c
chore(deps-dev): bump chrome-devtools-frontend from 1.0.1583146 to 1.…
dependabot[bot] f099549
refactor(network): de-duplicate String and JSON formatters (#985)
OrKoN bdfbf83
docs: update codex doc URL (#987)
OrKoN 4826dbb
chore(deps-dev): bump puppeteer from 24.37.3 to 24.37.4 in the bundle…
dependabot[bot] c4fa168
refactor: remove text from the status code for Network requests (#778)
Lightning00Blade 915b944
chore: clear in prepare (#988)
OrKoN 5ed9d51
revert: "chore(deps-dev): bump chrome-devtools-frontend from 1.0.1583…
OrKoN 5a169bf
test: fix missing sinon.restore (#989)
OrKoN 7760d33
test(mcp): add eval scenarios for vague and URL-based webpage prompts…
madhaviai 1b98206
Merge branch 'main' into fix/940-improve-mcp-invocation
OrKoN 0bc270a
Merge branch 'main' into fix/940-improve-mcp-invocation
madhaviai d4da5d5
Merge branch 'main' into fix/940-improve-mcp-invocation
OrKoN 5ed43e1
Merge branch 'main' into fix/940-improve-mcp-invocation
madhaviai 7d8d21d
Merge branch 'main' into fix/940-improve-mcp-invocation
OrKoN 11b6f88
Merge branch 'main' into fix/940-improve-mcp-invocation
madhaviai 31e1a0f
fix(mcp): Updated file system args
madhaviai 04d4473
Merge branch 'main' into fix/940-improve-mcp-invocation
madhaviai 1c277a2
Merge branch 'main' into fix/940-improve-mcp-invocation
OrKoN dff4104
Update src/tools/pages.ts
OrKoN 6b8f361
chore: fixes
OrKoN File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,57 @@ | ||
| /** | ||
| * @license | ||
| * Copyright 2026 Google LLC | ||
| * SPDX-License-Identifier: Apache-2.0 | ||
| * | ||
| * Eval scenario: user asks to fix issues with their webpage (no URL given). | ||
| * When no URL is provided, the model should pick the current frontend and run | ||
| * and inspect it. Verifies the MCP server is invoked and the model opens the | ||
| * frontend and inspects it (snapshot, console, or network). | ||
| * | ||
| * Note: Tools like performance_start_trace, take_snapshot, list_console_messages, | ||
| * and list_network_requests do not require a URL in the prompt—they operate on | ||
| * the currently selected page. Only navigate_page/new_page need a URL to open | ||
| * a page; the eval runner injects the test URL when htmlRoute is set. | ||
| */ | ||
|
|
||
| import assert from 'node:assert'; | ||
|
|
||
| import type {TestScenario} from '../eval_gemini.ts'; | ||
|
|
||
| const INSPECTION_TOOLS = [ | ||
| 'take_snapshot', | ||
| 'list_console_messages', | ||
| 'list_network_requests', | ||
| ]; | ||
|
|
||
| export const scenario: TestScenario = { | ||
| prompt: 'Can you fix issues with my webpage?', | ||
| maxTurns: 4, | ||
| htmlRoute: { | ||
| path: '/fix_issues_test.html', | ||
| htmlContent: ` | ||
| <h1>Test Page</h1> | ||
| <p>Some content</p> | ||
| <script> | ||
| console.error('Intentional error for testing'); | ||
| </script> | ||
| `, | ||
| }, | ||
| expectations: calls => { | ||
| const NAVIGATION_TOOLS = ['navigate_page', 'new_page']; | ||
| assert.ok(calls.length >= 2, 'Expected at least navigation and one inspection'); | ||
| const navigationIndex = calls.findIndex(c => NAVIGATION_TOOLS.includes(c.name)); | ||
| assert.ok( | ||
| navigationIndex !== -1, | ||
| `Expected a navigation call (${NAVIGATION_TOOLS.join(' or ')}), got: ${calls.map(c => c.name).join(', ')}`, | ||
| ); | ||
| const afterNavigation = calls.slice(navigationIndex + 1); | ||
| const inspectionCalls = afterNavigation.filter(c => | ||
| INSPECTION_TOOLS.includes(c.name), | ||
| ); | ||
| assert.ok( | ||
| inspectionCalls.length >= 1, | ||
| `Expected at least one inspection tool (${INSPECTION_TOOLS.join(', ')}) after navigation, got: ${calls.map(c => c.name).join(', ')}`, | ||
| ); | ||
| }, | ||
| }; | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,34 @@ | ||
| /** | ||
| * @license | ||
| * Copyright 2026 Google LLC | ||
| * SPDX-License-Identifier: Apache-2.0 | ||
| * | ||
| * Eval scenario using "website"/"webpage" wording to verify the model invokes | ||
| * the right tools when users ask to open a site and read its content. | ||
| */ | ||
|
|
||
| import assert from 'node:assert'; | ||
|
|
||
| import type {TestScenario} from '../eval_gemini.ts'; | ||
|
|
||
| export const scenario: TestScenario = { | ||
| prompt: | ||
| 'Open the website at <TEST_URL> and tell me what content is on the page.', | ||
| maxTurns: 3, | ||
| htmlRoute: { | ||
| path: '/frontend_snapshot.html', | ||
| htmlContent: '<h1>Frontend Test</h1><p>This is a test webpage.</p>', | ||
| }, | ||
| expectations: calls => { | ||
| assert.strictEqual(calls.length, 2); | ||
| assert.ok( | ||
| calls[0].name === 'navigate_page' || calls[0].name === 'new_page', | ||
| 'First call should be navigation', | ||
| ); | ||
| assert.strictEqual( | ||
| calls[1].name, | ||
| 'take_snapshot', | ||
| 'Second call should be take_snapshot to read page content', | ||
| ); | ||
| }, | ||
| }; |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.