--- name: "playwright-pro" description: "Production-grade Playwright testing toolkit. Use when the user mentions Playwright tests, end-to-end testing, browser automation, fixing flaky tests, test migration, CI/CD testing, or test suites. Generate tests, fix flaky failures, migrate from Cypress/Selenium, sync with TestRail, run on BrowserStack. 55 templates, 3 agents, smart reporting." --- # Playwright Pro Production-grade Playwright testing toolkit for AI coding agents. ## Available Commands When installed as a Claude Code plugin, these are available as `/pw:` commands: | Command | What it does | |---|---| | `/pw:init` | Set up Playwright — detects framework, generates config, CI, first test | | `/pw:generate ` | Generate tests from user story, URL, or component | | `/pw:review` | Review tests for anti-patterns and coverage gaps | | `/pw:fix ` | Diagnose and fix failing or flaky tests | | `/pw:migrate` | Migrate from Cypress or Selenium to Playwright | | `/pw:coverage` | Analyze what's tested vs. what's missing | | `/pw:testrail` | Sync with TestRail — read cases, push results | | `/pw:browserstack` | Run on BrowserStack, pull cross-browser reports | | `/pw:report` | Generate test report in your preferred format | ## Quick Start Workflow The recommended sequence for most projects: ``` 1. /pw:init → scaffolds config, CI pipeline, and a first smoke test 2. /pw:generate → generates tests from your spec or URL 3. /pw:review → validates quality and flags anti-patterns ← always run after generate 4. /pw:fix → diagnoses and repairs any failing/flaky tests ← run when CI turns red ``` **Validation checkpoints:** - After `/pw:generate` — always run `/pw:review` before committing; it catches locator anti-patterns and missing assertions automatically. - After `/pw:fix` — re-run the full suite locally (`npx playwright test`) to confirm the fix doesn't introduce regressions. - After `/pw:migrate` — run `/pw:coverage` to confirm parity with the old suite before decommissioning Cypress/Selenium tests. ### Example: Generate → Review → Fix ```bash # 1. Generate tests from a user story /pw:generate "As a user I can log in with email and password" # Generated: tests/auth/login.spec.ts # → Playwright Pro creates the file using the auth template. # 2. Review the generated tests /pw:review tests/auth/login.spec.ts # → Flags: one test used page.locator('input[type=password]') — suggests getByLabel('Password') # → Fix applied automatically. # 3. Run locally to confirm npx playwright test tests/auth/login.spec.ts --headed # 4. If a test is flaky in CI, diagnose it /pw:fix tests/auth/login.spec.ts # → Identifies missing web-first assertion; replaces waitForTimeout(2000) with expect(locator).toBeVisible() ``` ## Golden Rules 1. `getByRole()` over CSS/XPath — resilient to markup changes 2. Never `page.waitForTimeout()` — use web-first assertions 3. `expect(locator)` auto-retries; `expect(await locator.textContent())` does not 4. Isolate every test — no shared state between tests 5. `baseURL` in config — zero hardcoded URLs 6. Retries: `2` in CI, `0` locally 7. Traces: `'on-first-retry'` — rich debugging without slowdown 8. Fixtures over globals — `test.extend()` for shared state 9. One behavior per test — multiple related assertions are fine 10. Mock external services only — never mock your own app ## Locator Priority ``` 1. getByRole() — buttons, links, headings, form elements 2. getByLabel() — form fields with labels 3. getByText() — non-interactive text 4. getByPlaceholder() — inputs with placeholder 5. getByTestId() — when no semantic option exists 6. page.locator() — CSS/XPath as last resort ``` ## What's Included - **9 skills** with detailed step-by-step instructions - **3 specialized agents**: test-architect, test-debugger, migration-planner - **55 test templates**: auth, CRUD, checkout, search, forms, dashboard, settings, onboarding, notifications, API, accessibility - **2 MCP servers** (TypeScript): TestRail and BrowserStack integrations - **Smart hooks**: auto-validate test quality, auto-detect Playwright projects - **6 reference docs**: golden rules, locators, assertions, fixtures, pitfalls, flaky tests - **Migration guides**: Cypress and Selenium mapping tables ## Integration Setup ### TestRail (Optional) ```bash export TESTRAIL_URL="https://your-instance.testrail.io" export TESTRAIL_USER="your@email.com" export TESTRAIL_API_KEY="your-api-key" ``` ### BrowserStack (Optional) ```bash export BROWSERSTACK_USERNAME="your-username" export BROWSERSTACK_ACCESS_KEY="your-access-key" ``` ## Quick Reference See `reference/` directory for: - `golden-rules.md` — The 10 non-negotiable rules - `locators.md` — Complete locator priority with cheat sheet - `assertions.md` — Web-first assertions reference - `fixtures.md` — Custom fixtures and storageState patterns - `common-pitfalls.md` — Top 10 mistakes and fixes - `flaky-tests.md` — Diagnosis commands and quick fixes See `templates/README.md` for the full template index. FILE:CLAUDE.md # Playwright Pro — Agent Context You are working in a project with the Playwright Pro plugin installed. Follow these rules for all test-related work. ## Golden Rules (Non-Negotiable) 1. **`getByRole()` over CSS/XPath** — resilient to markup changes, mirrors how users see the page 2. **Never `page.waitForTimeout()`** — use `expect(locator).toBeVisible()` or `page.waitForURL()` 3. **Web-first assertions** — `expect(locator)` auto-retries; `expect(await locator.textContent())` does not 4. **Isolate every test** — no shared state, no execution-order dependencies 5. **`baseURL` in config** — zero hardcoded URLs in tests 6. **Retries: `2` in CI, `0` locally** — surface flakiness where it matters 7. **Traces: `'on-first-retry'`** — rich debugging without CI slowdown 8. **Fixtures over globals** — share state via `test.extend()`, not module-level variables 9. **One behavior per test** — multiple related `expect()` calls are fine 10. **Mock external services only** — never mock your own app ## Locator Priority Always use the first option that works: ```typescript page.getByRole('button', { name: 'Submit' }) // 1. Role (default) page.getByLabel('Email address') // 2. Label (form fields) page.getByText('Welcome back') // 3. Text (non-interactive) page.getByPlaceholder('Search...') // 4. Placeholder page.getByAltText('Company logo') // 5. Alt text (images) page.getByTitle('Close dialog') // 6. Title attribute page.getByTestId('checkout-summary') // 7. Test ID (last semantic) page.locator('.legacy-widget') // 8. CSS (last resort) ``` ## How to Use This Plugin ### Generating Tests When generating tests, always: 1. Use the `Explore` subagent to scan the project structure first 2. Check `playwright.config.ts` for `testDir`, `baseURL`, and project settings 3. Load relevant templates from `templates/` directory 4. Match the project's language (check for `tsconfig.json` → TypeScript, else JavaScript) 5. Place tests in the configured `testDir` (default: `tests/` or `e2e/`) 6. Include a descriptive test name that explains the behavior being verified ### Reviewing Tests When reviewing, check against: 1. All 10 golden rules above 2. The anti-patterns in `skills/review/anti-patterns.md` 3. Missing edge cases (empty state, error state, loading state) 4. Proper use of fixtures for shared setup ### Fixing Flaky Tests When fixing flaky tests: 1. Categorize first: timing, isolation, environment, or infrastructure 2. Use `npx playwright test --repeat-each=10` to reproduce 3. Use `--trace=on` for every attempt 4. Apply the targeted fix from `skills/fix/flaky-taxonomy.md` ### Using Built-in Commands Leverage Claude Code's built-in capabilities: - **Large migrations**: Use `/batch` for parallel file-by-file conversion - **Post-generation cleanup**: Use `/simplify` after generating a test suite - **Debugging sessions**: Use `/debug` alongside `/pw:fix` for trace analysis - **Code review**: Use `/review` for general code quality, `/pw:review` for Playwright-specific ### Integrations - **TestRail**: Configured via `TESTRAIL_URL`, `TESTRAIL_USER`, `TESTRAIL_API_KEY` env vars - **BrowserStack**: Configured via `BROWSERSTACK_USERNAME`, `BROWSERSTACK_ACCESS_KEY` env vars - Both are optional. The plugin works fully without them. ## File Conventions - Test files: `*.spec.ts` or `*.spec.js` - Page objects: `*.page.ts` in a `pages/` directory - Fixtures: `fixtures.ts` or `fixtures/` directory - Test data: `test-data/` directory with JSON/factory files FILE:README.md # Playwright Pro > Production-grade Playwright testing toolkit for AI coding agents. Generate tests, fix flaky failures, migrate from Cypress/Selenium, sync with TestRail, run on BrowserStack — all from your AI agent. ## Install ```bash # Claude Code plugin claude plugin install pw@claude-skills # Or load directly claude --plugin-dir ./engineering-team/playwright-pro ``` ## Commands | Command | What it does | |---|---| | `/pw:init` | Set up Playwright in your project — detects framework, generates config, CI, first test | | `/pw:generate ` | Generate tests from a user story, URL, or component name | | `/pw:review` | Review existing tests for anti-patterns and coverage gaps | | `/pw:fix ` | Diagnose and fix a failing or flaky test | | `/pw:migrate` | Migrate from Cypress or Selenium to Playwright | | `/pw:coverage` | Analyze what's tested vs. what's missing | | `/pw:testrail` | Sync with TestRail — read cases, push results, create runs | | `/pw:browserstack` | Run tests on BrowserStack, pull cross-browser reports | | `/pw:report` | Generate a test report in your preferred format | ## Quick Start ```bash # In Claude Code: /pw:init # Set up Playwright /pw:generate "user can log in" # Generate your first test # Tests are auto-validated by hooks — no extra steps ``` ## What's Inside ### 9 Skills Slash commands that turn natural language into production-ready Playwright tests. Each skill leverages Claude Code's built-in capabilities (`/batch` for parallel work, `Explore` for codebase analysis, `/debug` for trace inspection). ### 3 Specialized Agents - **test-architect** — Plans test strategy for complex applications - **test-debugger** — Diagnoses flaky tests using a systematic taxonomy - **migration-planner** — Creates file-by-file migration plans from Cypress/Selenium ### 55 Test Templates Ready-to-use, parametrizable templates covering: | Category | Count | Examples | |---|---|---| | Authentication | 8 | Login, logout, SSO, MFA, password reset, RBAC | | CRUD | 6 | Create, read, update, delete, bulk ops | | Checkout | 6 | Cart, payment, coupon, order history | | Search | 5 | Basic search, filters, sorting, pagination | | Forms | 6 | Multi-step, validation, file upload | | Dashboard | 5 | Data loading, charts, export | | Settings | 4 | Profile, password, notifications | | Onboarding | 4 | Registration, email verify, welcome tour | | Notifications | 3 | In-app, toast, notification center | | API | 5 | REST CRUD, GraphQL, error handling | | Accessibility | 3 | Keyboard nav, screen reader, contrast | ### 2 MCP Integrations - **TestRail** — Read test cases, create runs, push pass/fail results - **BrowserStack** — Trigger cross-browser runs, pull session reports with video/screenshots ### Smart Hooks - Auto-validates test quality when you write `*.spec.ts` files - Auto-detects Playwright projects on session start - Zero configuration required ## Integrations Setup ### TestRail (Optional) Set environment variables: ```bash export TESTRAIL_URL="https://your-instance.testrail.io" export TESTRAIL_USER="your@email.com" export TESTRAIL_API_KEY="your-api-key" ``` Then use `/pw:testrail` to sync test cases and push results. ### BrowserStack (Optional) ```bash export BROWSERSTACK_USERNAME="your-username" export BROWSERSTACK_ACCESS_KEY="your-access-key" ``` Then use `/pw:browserstack` to run tests across browsers. ## Works With | Agent | How | |---|---| | **Claude Code** | Full plugin — slash commands, MCP tools, hooks, agents | | **Codex CLI** | Copy `CLAUDE.md` to your project root as `AGENTS.md` | | **OpenClaw** | Use as a skill with `SKILL.md` entry point | ## Built-in Command Integration Playwright Pro doesn't reinvent what your AI agent already does. It orchestrates built-in capabilities: - `/pw:generate` uses Claude's `Explore` subagent to understand your codebase before generating tests - `/pw:migrate` uses `/batch` for parallel file-by-file conversion on large test suites - `/pw:fix` uses `/debug` for trace analysis alongside Playwright-specific diagnostics - `/pw:review` extends `/review` with Playwright anti-pattern detection ## Reference Based on battle-tested patterns from production test suites. Includes curated guidance on: - Locator strategies and priority hierarchy - Assertion patterns and auto-retry behavior - Fixture architecture and composition - Common pitfalls (top 20, ranked by frequency) - Flaky test diagnosis taxonomy ## License MIT FILE:agents/migration-planner.md --- name: migration-planner description: >- Analyzes Cypress or Selenium test suites and creates a file-by-file migration plan. Invoked by /pw:migrate before conversion starts. allowed-tools: - Read - Grep - Glob - LS --- # Migration Planner Agent You are a test migration specialist. Your job is to analyze an existing Cypress or Selenium test suite and create a detailed, ordered migration plan. ## Planning Protocol ### Step 1: Detect Source Framework Scan the project: **Cypress indicators:** - `cypress/` directory - `cypress.config.ts` or `cypress.config.js` - `@cypress` packages in `package.json` - `.cy.ts` or `.cy.js` test files **Selenium indicators:** - `selenium-webdriver` in dependencies - `webdriver` or `wdio` in dependencies - Test files importing `selenium-webdriver` - `chromedriver` or `geckodriver` in dependencies - Python files importing `selenium` ### Step 2: Inventory All Test Files List every test file with: - File path - Number of tests (count `it()`, `test()`, or test methods) - Dependencies (custom commands, page objects, fixtures) - Complexity (simple/medium/complex based on lines and patterns) ``` ## Test Inventory | # | File | Tests | Dependencies | Complexity | |---|---|---|---|---| | 1 | cypress/e2e/login.cy.ts | 5 | login command | Simple | | 2 | cypress/e2e/checkout.cy.ts | 12 | api helpers, fixtures | Complex | | 3 | cypress/e2e/search.cy.ts | 8 | none | Medium | ``` ### Step 3: Map Dependencies Identify shared resources that need migration: **Custom commands** (`cypress/support/commands.ts`): - List each command and what it does - Map to Playwright equivalent (fixture, helper function, or page object) **Fixtures** (`cypress/fixtures/`): - List data files - Plan: copy to `test-data/` with any format adjustments **Plugins** (`cypress/plugins/`): - List plugin functionality - Map to Playwright config options or fixtures **Page Objects** (if used): - List page object files - Plan: convert API calls (minimal structural change) **Support files** (`cypress/support/`): - List setup/teardown logic - Map to `playwright.config.ts` or `fixtures/` ### Step 4: Determine Migration Order Order files by dependency graph: 1. **Shared resources first**: custom commands → fixtures, page objects → helpers 2. **Simple tests next**: files with no dependencies, few tests 3. **Complex tests last**: files with many dependencies, custom commands ``` ## Migration Order ### Phase 1: Foundation (do first) 1. Convert custom commands → fixtures.ts 2. Copy fixtures → test-data/ 3. Convert page objects (API changes only) ### Phase 2: Simple Tests (quick wins) 4. login.cy.ts → auth/login.spec.ts (5 tests, ~15 min) 5. about.cy.ts → static/about.spec.ts (2 tests, ~5 min) ### Phase 3: Complex Tests 6. checkout.cy.ts → checkout/checkout.spec.ts (12 tests, ~45 min) 7. search.cy.ts → search/search.spec.ts (8 tests, ~30 min) ``` ### Step 5: Estimate Effort | Complexity | Time per test | Notes | |---|---|---| | Simple | 2-3 min | Direct API mapping | | Medium | 5-10 min | Needs locator upgrade | | Complex | 10-20 min | Custom commands, plugins, complex flows | ### Step 6: Identify Risks Flag tests that may need manual intervention: - Tests using Cypress-only features (`cy.origin()`, `cy.session()`) - Tests with complex `cy.intercept()` patterns - Tests relying on Cypress retry-ability semantics - Tests using Cypress plugins with no Playwright equivalent ### Step 7: Return Plan Return the complete migration plan to `/pw:migrate` for execution. FILE:agents/test-architect.md --- name: test-architect description: >- Plans test strategy for complex applications. Invoked by /pw:generate and /pw:coverage when the app has multiple routes, complex state, or requires a structured test plan before writing tests. allowed-tools: - Read - Grep - Glob - LS --- # Test Architect Agent You are a test architecture specialist. Your job is to analyze an application's structure and create a comprehensive test plan before any tests are written. ## Your Responsibilities 1. **Map the application surface**: routes, components, API endpoints, user flows 2. **Identify critical paths**: the flows that, if broken, cause revenue loss or user churn 3. **Design test structure**: folder organization, fixture strategy, data management 4. **Prioritize**: which tests deliver the most confidence per effort 5. **Select patterns**: which template or approach fits each test scenario ## How You Work You are a read-only agent. You analyze and plan — you do not write test files. ### Step 1: Scan the Codebase - Read route definitions (Next.js `app/`, React Router, Vue Router, Angular routes) - Read `package.json` for framework and dependencies - Check for existing tests and their patterns - Identify state management (Redux, Zustand, Pinia, etc.) - Check for API layer (REST, GraphQL, tRPC) ### Step 2: Catalog Testable Surfaces Create a structured inventory: ``` ## Application Surface ### Pages (by priority) 1. /login — Auth entry point [CRITICAL] 2. /dashboard — Main user view [CRITICAL] 3. /settings — User preferences [HIGH] 4. /admin — Admin panel [HIGH] 5. /about — Static page [LOW] ### Interactive Components 1. SearchBar — complex state, debounced API calls 2. DataTable — sorting, filtering, pagination 3. FileUploader — drag-drop, progress, error handling ### API Endpoints 1. POST /api/auth/login — authentication 2. GET /api/users — user list with pagination 3. PUT /api/users/:id — user update ### User Flows (multi-page) 1. Registration → Email Verify → Onboarding → Dashboard 2. Search → Filter → Select → Add to Cart → Checkout → Confirm ``` ### Step 3: Design Test Plan ``` ## Test Plan ### Folder Structure e2e/ ├── auth/ # Authentication tests ├── dashboard/ # Dashboard tests ├── checkout/ # Checkout flow tests ├── fixtures/ # Shared fixtures ├── pages/ # Page object models └── test-data/ # Test data files ### Fixture Strategy - Auth fixture: shared `storageState` for logged-in tests - API fixture: request context for data seeding - Data fixture: factory functions for test entities ### Test Distribution | Area | Tests | Template | Effort | |---|---|---|---| | Auth | 8 | auth/* | 1h | | Dashboard | 6 | dashboard/* | 1h | | Checkout | 10 | checkout/* | 2h | | Search | 5 | search/* | 45m | | Settings | 4 | settings/* | 30m | | API | 5 | api/* | 45m | ### Priority Order 1. Auth (blocks everything else) 2. Core user flow (the main thing users do) 3. Payment/checkout (revenue-critical) 4. Everything else ``` ### Step 4: Return Plan Return the complete plan to the calling skill. Do not write files. FILE:agents/test-debugger.md --- name: test-debugger description: >- Diagnoses flaky or failing Playwright tests using systematic taxonomy. Invoked by /pw:fix when a test needs deep analysis including running tests, reading traces, and identifying root causes. allowed-tools: - Read - Grep - Glob - LS - Bash --- # Test Debugger Agent You are a Playwright test debugging specialist. Your job is to systematically diagnose why a test fails or behaves flakily, identify the root cause category, and return a specific fix. ## Debugging Protocol ### Step 1: Read the Test Read the test file and understand: - What behavior it's testing - Which pages/URLs it visits - Which locators it uses - Which assertions it makes - Any setup/teardown (fixtures, beforeEach) ### Step 2: Run the Test Run it multiple ways to classify the failure: ```bash # Single run — get the error npx playwright test --grep "" --reporter=list 2>&1 # Burn-in — expose timing issues npx playwright test --grep "" --repeat-each=10 --reporter=list 2>&1 # Isolation check — expose state leaks npx playwright test --grep "" --workers=1 --reporter=list 2>&1 # Full suite — expose interaction npx playwright test --reporter=list 2>&1 ``` ### Step 3: Capture Trace ```bash npx playwright test --grep "" --trace=on --retries=0 2>&1 ``` Read the trace output for: - Network requests that failed or were slow - Elements that weren't visible when expected - Navigation timing issues - Console errors ### Step 4: Classify | Category | Evidence | |---|---| | **Timing/Async** | Fails on `--repeat-each=10`; error mentions timeout or element not found intermittently | | **Test Isolation** | Passes alone (`--workers=1 --grep`), fails in full suite | | **Environment** | Passes locally, fails in CI (check viewport, fonts, timezone) | | **Infrastructure** | Random crash errors, OOM, browser process killed | ### Step 5: Identify Specific Cause Common root causes per category: **Timing:** - Missing `await` on a Playwright call - `waitForTimeout()` that's too short - Clicking before element is actionable - Asserting before data loads - Animation interference **Isolation:** - Global variable shared between tests - Database not cleaned between tests - localStorage/cookies leaking - Test creates data with non-unique identifier **Environment:** - Different viewport size in CI - Font rendering differences affect screenshots - Timezone affects date assertions - Network latency in CI is higher **Infrastructure:** - Browser runs out of memory with too many workers - File system race condition - DNS resolution failure ### Step 6: Return Diagnosis Return to the calling skill: ``` ## Diagnosis **Category:** Timing/Async **Root Cause:** Missing await on line 23 — `page.goto('/dashboard')` runs without waiting, so the assertion on line 24 runs before navigation completes. **Evidence:** Fails 3/10 times on `--repeat-each=10`. Trace shows assertion firing before navigation response received. ## Fix Line 23: Add `await` before `page.goto('/dashboard')` ## Verification After fix: 10/10 passes on `--repeat-each=10` ``` FILE:hooks/detect-playwright.sh #!/usr/bin/env bash # Session start hook: detects if the project uses Playwright. # Outputs context hint for Claude if playwright.config exists. set -euo pipefail # Check for Playwright config in current directory or common locations PW_CONFIG="" for config in playwright.config.ts playwright.config.js playwright.config.mjs; do if [[ -f "$config" ]]; then PW_CONFIG="$config" break fi done if [[ -z "$PW_CONFIG" ]]; then exit 0 fi # Count existing test files TEST_COUNT=$(find . -name "*.spec.ts" -o -name "*.spec.js" -o -name "*.test.ts" -o -name "*.test.js" 2>/dev/null | grep -v node_modules | wc -l | tr -d ' ') echo "🎭 Playwright detected ($PW_CONFIG) — $TEST_COUNT test files found. Use /pw: commands for testing workflows." FILE:hooks/hooks.json { "hooks": { "PostToolUse": [ { "matcher": "Write|Edit", "hooks": [ { "type": "command", "command": "bash ${CLAUDE_PLUGIN_ROOT}/hooks/validate-test.sh" } ] } ], "SessionStart": [ { "hooks": [ { "type": "command", "command": "bash ${CLAUDE_PLUGIN_ROOT}/hooks/detect-playwright.sh" } ] } ] } } FILE:hooks/validate-test.sh #!/usr/bin/env bash # Post-write hook: validates Playwright test files for common anti-patterns. # Runs silently — only outputs warnings if issues found. # Input: JSON on stdin with tool_input.file_path set -euo pipefail # Read the file path from stdin JSON INPUT=$(cat) FILE_PATH=$(echo "$INPUT" | python3 -c " import sys, json try: data = json.load(sys.stdin) print(data.get('tool_input', {}).get('file_path', '')) except: print('') " 2>/dev/null || echo "") # Only check .spec.ts and .spec.js files if [[ ! "$FILE_PATH" =~ \.(spec|test)\.(ts|js|mjs)$ ]]; then exit 0 fi # Check if file exists if [[ ! -f "$FILE_PATH" ]]; then exit 0 fi WARNINGS="" # Check for waitForTimeout if grep -n 'waitForTimeout' "$FILE_PATH" >/dev/null 2>&1; then LINES=$(grep -n 'waitForTimeout' "$FILE_PATH" | head -3) WARNINGS="${WARNINGS}\n⚠️ waitForTimeout() found — use web-first assertions instead:\n${LINES}\n" fi # Check for non-web-first assertions if grep -n 'expect(await ' "$FILE_PATH" >/dev/null 2>&1; then LINES=$(grep -n 'expect(await ' "$FILE_PATH" | head -3) WARNINGS="${WARNINGS}\n⚠️ Non-web-first assertion — use expect(locator) instead:\n${LINES}\n" fi # Check for hardcoded localhost URLs if grep -n "http://localhost\|https://localhost\|http://127.0.0.1" "$FILE_PATH" >/dev/null 2>&1; then LINES=$(grep -n "http://localhost\|https://localhost\|http://127.0.0.1" "$FILE_PATH" | head -3) WARNINGS="${WARNINGS}\n⚠️ Hardcoded URL — use baseURL from config:\n${LINES}\n" fi # Check for page.$() usage if grep -n 'page\.\$(' "$FILE_PATH" >/dev/null 2>&1; then LINES=$(grep -n 'page\.\$(' "$FILE_PATH" | head -3) WARNINGS="${WARNINGS}\n⚠️ page.\$() is deprecated — use page.locator() or getByRole():\n${LINES}\n" fi # Output warnings if any found if [[ -n "$WARNINGS" ]]; then echo -e "\n🎭 Playwright Pro — Test Validation${WARNINGS}" fi FILE:integrations/browserstack-mcp/package.json { "name": "@pw/browserstack-mcp", "version": "1.0.0", "description": "MCP server for BrowserStack integration with Playwright Pro", "type": "module", "main": "src/index.ts", "scripts": { "start": "tsx src/index.ts", "build": "tsc" }, "dependencies": { "@modelcontextprotocol/sdk": "^1.0.0" }, "devDependencies": { "tsx": "^4.0.0", "typescript": "^5.0.0" } } FILE:integrations/browserstack-mcp/src/client.ts import type { BrowserStackConfig, BrowserStackPlan, BrowserStackBrowser, BrowserStackBuild, BrowserStackSession, BrowserStackSessionUpdate, } from './types.js'; export class BrowserStackClient { private readonly baseUrl = 'https://api.browserstack.com'; private readonly headers: Record; constructor(config: BrowserStackConfig) { const auth = Buffer.from(`${config.username}:${config.accessKey}`).toString('base64'); this.headers = { Authorization: `Basic ${auth}`, 'Content-Type': 'application/json', }; } private async request( method: string, endpoint: string, body?: unknown, ): Promise { const url = `${this.baseUrl}${endpoint}`; const options: RequestInit = { method, headers: this.headers, }; if (body) { options.body = JSON.stringify(body); } const response = await fetch(url, options); if (!response.ok) { const errorText = await response.text(); throw new Error( `BrowserStack API error ${response.status}: ${errorText}`, ); } return response.json() as Promise; } async getPlan(): Promise { return this.request('GET', '/automate/plan.json'); } async getBrowsers(): Promise { return this.request('GET', '/automate/browsers.json'); } async getBuilds(limit?: number, status?: string): Promise { let endpoint = '/automate/builds.json'; const params: string[] = []; if (limit) params.push(`limit=${limit}`); if (status) params.push(`status=${status}`); if (params.length > 0) endpoint += `?${params.join('&')}`; return this.request('GET', endpoint); } async getSessions(buildId: string, limit?: number): Promise { let endpoint = `/automate/builds/${buildId}/sessions.json`; if (limit) endpoint += `?limit=${limit}`; return this.request('GET', endpoint); } async getSession(sessionId: string): Promise { return this.request( 'GET', `/automate/sessions/${sessionId}.json`, ); } async updateSession( sessionId: string, update: BrowserStackSessionUpdate, ): Promise { return this.request( 'PUT', `/automate/sessions/${sessionId}.json`, update, ); } async getSessionLogs(sessionId: string): Promise { const url = `${this.baseUrl}/automate/sessions/${sessionId}/logs`; const response = await fetch(url, { headers: this.headers }); if (!response.ok) { throw new Error(`BrowserStack logs error ${response.status}`); } return response.text(); } } FILE:integrations/browserstack-mcp/src/index.ts #!/usr/bin/env npx tsx import { Server } from '@modelcontextprotocol/sdk/server/index.js'; import { StdioServerTransport } from '@modelcontextprotocol/sdk/server/stdio.js'; import { CallToolRequestSchema, ListToolsRequestSchema, } from '@modelcontextprotocol/sdk/types.js'; import { BrowserStackClient } from './client.js'; import type { BrowserStackSessionUpdate } from './types.js'; const config = { username: process.env.BROWSERSTACK_USERNAME ?? '', accessKey: process.env.BROWSERSTACK_ACCESS_KEY ?? '', }; if (!config.username || !config.accessKey) { console.error( 'Missing BrowserStack configuration. Set BROWSERSTACK_USERNAME and BROWSERSTACK_ACCESS_KEY.', ); process.exit(1); } const client = new BrowserStackClient(config); const server = new Server( { name: 'pw-browserstack', version: '1.0.0' }, { capabilities: { tools: {} } }, ); server.setRequestHandler(ListToolsRequestSchema, async () => ({ tools: [ { name: 'browserstack_get_plan', description: 'Get BrowserStack Automate plan details including parallel session limits', inputSchema: { type: 'object', properties: {} }, }, { name: 'browserstack_get_browsers', description: 'List all available browser and OS combinations for Playwright testing', inputSchema: { type: 'object', properties: {} }, }, { name: 'browserstack_get_builds', description: 'List recent test builds with status', inputSchema: { type: 'object', properties: { limit: { type: 'number', description: 'Max builds to return (default 10)' }, status: { type: 'string', enum: ['running', 'done', 'failed', 'timeout'], description: 'Filter by status', }, }, }, }, { name: 'browserstack_get_sessions', description: 'List test sessions within a build', inputSchema: { type: 'object', properties: { build_id: { type: 'string', description: 'Build hashed ID' }, limit: { type: 'number', description: 'Max sessions to return' }, }, required: ['build_id'], }, }, { name: 'browserstack_get_session', description: 'Get detailed session info including video URL, logs, and screenshots', inputSchema: { type: 'object', properties: { session_id: { type: 'string', description: 'Session hashed ID' }, }, required: ['session_id'], }, }, { name: 'browserstack_update_session', description: 'Update session status (mark as passed/failed) and name', inputSchema: { type: 'object', properties: { session_id: { type: 'string', description: 'Session hashed ID' }, status: { type: 'string', enum: ['passed', 'failed'], description: 'Test result status', }, name: { type: 'string', description: 'Updated session name' }, reason: { type: 'string', description: 'Reason for failure' }, }, required: ['session_id'], }, }, { name: 'browserstack_get_logs', description: 'Get text logs for a specific test session', inputSchema: { type: 'object', properties: { session_id: { type: 'string', description: 'Session hashed ID' }, }, required: ['session_id'], }, }, ], })); server.setRequestHandler(CallToolRequestSchema, async (request) => { const { name, arguments: args } = request.params; try { switch (name) { case 'browserstack_get_plan': { const plan = await client.getPlan(); return { content: [{ type: 'text', text: JSON.stringify(plan, null, 2) }] }; } case 'browserstack_get_browsers': { const browsers = await client.getBrowsers(); const playwrightBrowsers = browsers.filter( (b) => ['chrome', 'firefox', 'playwright-chromium', 'playwright-firefox', 'playwright-webkit'].includes( b.browser?.toLowerCase() ?? '', ) || b.browser?.toLowerCase().includes('playwright'), ); const summary = playwrightBrowsers.length > 0 ? playwrightBrowsers : browsers.slice(0, 50); return { content: [{ type: 'text', text: JSON.stringify(summary, null, 2) }] }; } case 'browserstack_get_builds': { const builds = await client.getBuilds( (args?.limit as number) ?? 10, args?.status as string | undefined, ); return { content: [{ type: 'text', text: JSON.stringify(builds, null, 2) }] }; } case 'browserstack_get_sessions': { const sessions = await client.getSessions( args!.build_id as string, args?.limit as number | undefined, ); return { content: [{ type: 'text', text: JSON.stringify(sessions, null, 2) }] }; } case 'browserstack_get_session': { const session = await client.getSession(args!.session_id as string); return { content: [{ type: 'text', text: JSON.stringify(session, null, 2) }] }; } case 'browserstack_update_session': { const update: BrowserStackSessionUpdate = {}; if (args?.status) update.status = args.status as 'passed' | 'failed'; if (args?.name) update.name = args.name as string; if (args?.reason) update.reason = args.reason as string; const updated = await client.updateSession(args!.session_id as string, update); return { content: [{ type: 'text', text: JSON.stringify(updated, null, 2) }] }; } case 'browserstack_get_logs': { const logs = await client.getSessionLogs(args!.session_id as string); return { content: [{ type: 'text', text: logs }] }; } default: return { content: [{ type: 'text', text: `Unknown tool: ${name}` }], isError: true }; } } catch (error) { const message = error instanceof Error ? error.message : String(error); return { content: [{ type: 'text', text: `Error: ${message}` }], isError: true }; } }); async function main() { const transport = new StdioServerTransport(); await server.connect(transport); } main().catch(console.error); FILE:integrations/browserstack-mcp/src/types.ts export interface BrowserStackConfig { username: string; accessKey: string; } export interface BrowserStackPlan { automate_plan: string; parallel_sessions_running: number; team_parallel_sessions_max_allowed: number; parallel_sessions_max_allowed: number; queued_sessions: number; queued_sessions_max_allowed: number; } export interface BrowserStackBrowser { os: string; os_version: string; browser: string; browser_version: string; device: string | null; real_mobile: boolean | null; } export interface BrowserStackBuild { automation_build: { name: string; hashed_id: string; duration: number; status: string; build_tag: string | null; }; } export interface BrowserStackSession { automation_session: { name: string; duration: number; os: string; os_version: string; browser_version: string; browser: string; device: string | null; status: string; hashed_id: string; reason: string; build_name: string; project_name: string; logs: string; browser_url: string; public_url: string; video_url: string; browser_console_logs_url: string; har_logs_url: string; }; } export interface BrowserStackSessionUpdate { name?: string; status?: 'passed' | 'failed'; reason?: string; } FILE:integrations/browserstack-mcp/tsconfig.json { "compilerOptions": { "target": "ES2022", "module": "ESNext", "moduleResolution": "bundler", "esModuleInterop": true, "strict": true, "outDir": "dist", "rootDir": "src", "declaration": true, "skipLibCheck": true }, "include": ["src/**/*"] } FILE:integrations/testrail-mcp/package.json { "name": "@pw/testrail-mcp", "version": "1.0.0", "description": "MCP server for TestRail integration with Playwright Pro", "type": "module", "main": "src/index.ts", "scripts": { "start": "tsx src/index.ts", "build": "tsc" }, "dependencies": { "@modelcontextprotocol/sdk": "^1.0.0" }, "devDependencies": { "tsx": "^4.0.0", "typescript": "^5.0.0" } } FILE:integrations/testrail-mcp/src/client.ts import type { TestRailConfig, TestRailProject, TestRailSuite, TestRailCase, TestRailCasePayload, TestRailRun, TestRailRunPayload, TestRailResult, TestRailResultPayload, } from './types.js'; export class TestRailClient { private readonly baseUrl: string; private readonly headers: Record; constructor(config: TestRailConfig) { this.baseUrl = config.url.replace(/\/+$/, ''); const auth = Buffer.from(`${config.user}:${config.apiKey}`).toString('base64'); this.headers = { Authorization: `Basic ${auth}`, 'Content-Type': 'application/json', }; } private async request( method: string, endpoint: string, body?: unknown, ): Promise { const url = `${this.baseUrl}/index.php?/api/v2/${endpoint}`; const options: RequestInit = { method, headers: this.headers, }; if (body) { options.body = JSON.stringify(body); } const response = await fetch(url, options); if (!response.ok) { const errorText = await response.text(); throw new Error( `TestRail API error ${response.status}: ${errorText}`, ); } return response.json() as Promise; } async getProjects(): Promise { const result = await this.request<{ projects: TestRailProject[] }>( 'GET', 'get_projects', ); return result.projects ?? result as unknown as TestRailProject[]; } async getSuites(projectId: number): Promise { return this.request('GET', `get_suites/${projectId}`); } async getCases( projectId: number, suiteId?: number, sectionId?: number, limit?: number, offset?: number, filter?: string, ): Promise { let endpoint = `get_cases/${projectId}`; const params: string[] = []; if (suiteId) params.push(`suite_id=${suiteId}`); if (sectionId) params.push(`section_id=${sectionId}`); if (limit) params.push(`limit=${limit}`); if (offset) params.push(`offset=${offset}`); if (filter) params.push(`filter=${encodeURIComponent(filter)}`); if (params.length > 0) endpoint += `&${params.join('&')}`; const result = await this.request<{ cases: TestRailCase[] }>( 'GET', endpoint, ); return result.cases ?? result as unknown as TestRailCase[]; } async addCase( sectionId: number, payload: TestRailCasePayload, ): Promise { return this.request( 'POST', `add_case/${sectionId}`, payload, ); } async updateCase( caseId: number, payload: Partial, ): Promise { return this.request( 'POST', `update_case/${caseId}`, payload, ); } async addRun( projectId: number, payload: TestRailRunPayload, ): Promise { return this.request( 'POST', `add_run/${projectId}`, payload, ); } async addResultForCase( runId: number, caseId: number, payload: TestRailResultPayload, ): Promise { return this.request( 'POST', `add_result_for_case/${runId}/${caseId}`, payload, ); } async getResultsForCase( runId: number, caseId: number, limit?: number, ): Promise { let endpoint = `get_results_for_case/${runId}/${caseId}`; if (limit) endpoint += `&limit=${limit}`; const result = await this.request<{ results: TestRailResult[] }>( 'GET', endpoint, ); return result.results ?? result as unknown as TestRailResult[]; } } FILE:integrations/testrail-mcp/src/index.ts #!/usr/bin/env npx tsx import { Server } from '@modelcontextprotocol/sdk/server/index.js'; import { StdioServerTransport } from '@modelcontextprotocol/sdk/server/stdio.js'; import { CallToolRequestSchema, ListToolsRequestSchema, } from '@modelcontextprotocol/sdk/types.js'; import { TestRailClient } from './client.js'; import type { TestRailCasePayload, TestRailRunPayload, TestRailResultPayload } from './types.js'; const config = { url: process.env.TESTRAIL_URL ?? '', user: process.env.TESTRAIL_USER ?? '', apiKey: process.env.TESTRAIL_API_KEY ?? '', }; if (!config.url || !config.user || !config.apiKey) { console.error( 'Missing TestRail configuration. Set TESTRAIL_URL, TESTRAIL_USER, and TESTRAIL_API_KEY.', ); process.exit(1); } const client = new TestRailClient(config); const server = new Server( { name: 'pw-testrail', version: '1.0.0' }, { capabilities: { tools: {} } }, ); server.setRequestHandler(ListToolsRequestSchema, async () => ({ tools: [ { name: 'testrail_get_projects', description: 'List all TestRail projects', inputSchema: { type: 'object', properties: {} }, }, { name: 'testrail_get_suites', description: 'List test suites in a project', inputSchema: { type: 'object', properties: { project_id: { type: 'number', description: 'Project ID' }, }, required: ['project_id'], }, }, { name: 'testrail_get_cases', description: 'Get test cases from a project. Supports filtering by suite, section, and search text.', inputSchema: { type: 'object', properties: { project_id: { type: 'number', description: 'Project ID' }, suite_id: { type: 'number', description: 'Suite ID (optional)' }, section_id: { type: 'number', description: 'Section ID (optional)' }, limit: { type: 'number', description: 'Max results (default 250)' }, offset: { type: 'number', description: 'Offset for pagination' }, filter: { type: 'string', description: 'Search text filter' }, }, required: ['project_id'], }, }, { name: 'testrail_add_case', description: 'Create a new test case in a section', inputSchema: { type: 'object', properties: { section_id: { type: 'number', description: 'Section ID to add the case to' }, title: { type: 'string', description: 'Test case title' }, template_id: { type: 'number', description: 'Template ID (2 = Test Case Steps)' }, priority_id: { type: 'number', description: 'Priority (1=Low, 2=Medium, 3=High, 4=Critical)' }, custom_preconds: { type: 'string', description: 'Preconditions text' }, custom_steps_separated: { type: 'array', items: { type: 'object', properties: { content: { type: 'string', description: 'Step action' }, expected: { type: 'string', description: 'Expected result' }, }, }, description: 'Test steps with expected results', }, }, required: ['section_id', 'title'], }, }, { name: 'testrail_update_case', description: 'Update an existing test case', inputSchema: { type: 'object', properties: { case_id: { type: 'number', description: 'Case ID to update' }, title: { type: 'string', description: 'Updated title' }, custom_preconds: { type: 'string', description: 'Updated preconditions' }, custom_steps_separated: { type: 'array', items: { type: 'object', properties: { content: { type: 'string' }, expected: { type: 'string' }, }, }, description: 'Updated test steps', }, }, required: ['case_id'], }, }, { name: 'testrail_add_run', description: 'Create a new test run in a project', inputSchema: { type: 'object', properties: { project_id: { type: 'number', description: 'Project ID' }, name: { type: 'string', description: 'Run name' }, description: { type: 'string', description: 'Run description' }, suite_id: { type: 'number', description: 'Suite ID' }, include_all: { type: 'boolean', description: 'Include all cases (default true)' }, case_ids: { type: 'array', items: { type: 'number' }, description: 'Specific case IDs to include (if include_all is false)', }, }, required: ['project_id', 'name'], }, }, { name: 'testrail_add_result', description: 'Add a test result for a specific case in a run', inputSchema: { type: 'object', properties: { run_id: { type: 'number', description: 'Run ID' }, case_id: { type: 'number', description: 'Case ID' }, status_id: { type: 'number', description: 'Status: 1=Passed, 2=Blocked, 3=Untested, 4=Retest, 5=Failed', }, comment: { type: 'string', description: 'Result comment or error message' }, elapsed: { type: 'string', description: 'Time spent (e.g., "30s", "1m 45s")' }, defects: { type: 'string', description: 'Defect IDs (comma-separated)' }, }, required: ['run_id', 'case_id', 'status_id'], }, }, { name: 'testrail_get_results', description: 'Get historical results for a test case in a run', inputSchema: { type: 'object', properties: { run_id: { type: 'number', description: 'Run ID' }, case_id: { type: 'number', description: 'Case ID' }, limit: { type: 'number', description: 'Max results to return' }, }, required: ['run_id', 'case_id'], }, }, ], })); server.setRequestHandler(CallToolRequestSchema, async (request) => { const { name, arguments: args } = request.params; try { switch (name) { case 'testrail_get_projects': { const projects = await client.getProjects(); return { content: [{ type: 'text', text: JSON.stringify(projects, null, 2) }] }; } case 'testrail_get_suites': { const suites = await client.getSuites(args!.project_id as number); return { content: [{ type: 'text', text: JSON.stringify(suites, null, 2) }] }; } case 'testrail_get_cases': { const cases = await client.getCases( args!.project_id as number, args?.suite_id as number | undefined, args?.section_id as number | undefined, args?.limit as number | undefined, args?.offset as number | undefined, args?.filter as string | undefined, ); return { content: [{ type: 'text', text: JSON.stringify(cases, null, 2) }] }; } case 'testrail_add_case': { const payload: TestRailCasePayload = { title: args!.title as string, template_id: args?.template_id as number | undefined, priority_id: args?.priority_id as number | undefined, custom_preconds: args?.custom_preconds as string | undefined, custom_steps_separated: args?.custom_steps_separated as TestRailCasePayload['custom_steps_separated'], }; const newCase = await client.addCase(args!.section_id as number, payload); return { content: [{ type: 'text', text: JSON.stringify(newCase, null, 2) }] }; } case 'testrail_update_case': { const updatePayload: Partial = {}; if (args?.title) updatePayload.title = args.title as string; if (args?.custom_preconds) updatePayload.custom_preconds = args.custom_preconds as string; if (args?.custom_steps_separated) { updatePayload.custom_steps_separated = args.custom_steps_separated as TestRailCasePayload['custom_steps_separated']; } const updated = await client.updateCase(args!.case_id as number, updatePayload); return { content: [{ type: 'text', text: JSON.stringify(updated, null, 2) }] }; } case 'testrail_add_run': { const runPayload: TestRailRunPayload = { name: args!.name as string, description: args?.description as string | undefined, suite_id: args?.suite_id as number | undefined, include_all: (args?.include_all as boolean) ?? true, case_ids: args?.case_ids as number[] | undefined, }; const run = await client.addRun(args!.project_id as number, runPayload); return { content: [{ type: 'text', text: JSON.stringify(run, null, 2) }] }; } case 'testrail_add_result': { const resultPayload: TestRailResultPayload = { status_id: args!.status_id as number, comment: args?.comment as string | undefined, elapsed: args?.elapsed as string | undefined, defects: args?.defects as string | undefined, }; const result = await client.addResultForCase( args!.run_id as number, args!.case_id as number, resultPayload, ); return { content: [{ type: 'text', text: JSON.stringify(result, null, 2) }] }; } case 'testrail_get_results': { const results = await client.getResultsForCase( args!.run_id as number, args!.case_id as number, args?.limit as number | undefined, ); return { content: [{ type: 'text', text: JSON.stringify(results, null, 2) }] }; } default: return { content: [{ type: 'text', text: `Unknown tool: ${name}` }], isError: true }; } } catch (error) { const message = error instanceof Error ? error.message : String(error); return { content: [{ type: 'text', text: `Error: ${message}` }], isError: true }; } }); async function main() { const transport = new StdioServerTransport(); await server.connect(transport); } main().catch(console.error); FILE:integrations/testrail-mcp/src/types.ts export interface TestRailConfig { url: string; user: string; apiKey: string; } export interface TestRailProject { id: number; name: string; announcement: string; is_completed: boolean; suite_mode: number; url: string; } export interface TestRailSuite { id: number; name: string; description: string | null; project_id: number; url: string; } export interface TestRailSection { id: number; suite_id: number; name: string; description: string | null; parent_id: number | null; depth: number; } export interface TestRailCaseStep { content: string; expected: string; } export interface TestRailCase { id: number; title: string; section_id: number; template_id: number; type_id: number; priority_id: number; estimate: string | null; refs: string | null; custom_preconds: string | null; custom_steps_separated: TestRailCaseStep[] | null; custom_steps: string | null; custom_expected: string | null; } export interface TestRailRun { id: number; suite_id: number; name: string; description: string | null; assignedto_id: number | null; include_all: boolean; is_completed: boolean; passed_count: number; failed_count: number; untested_count: number; url: string; } export interface TestRailResult { id: number; test_id: number; status_id: number; comment: string | null; created_on: number; elapsed: string | null; defects: string | null; } export interface TestRailResultPayload { status_id: number; comment?: string; elapsed?: string; defects?: string; } export interface TestRailRunPayload { suite_id?: number; name: string; description?: string; assignedto_id?: number; include_all?: boolean; case_ids?: number[]; refs?: string; } export interface TestRailCasePayload { title: string; template_id?: number; type_id?: number; priority_id?: number; estimate?: string; refs?: string; custom_preconds?: string; custom_steps_separated?: TestRailCaseStep[]; custom_steps?: string; custom_expected?: string; } FILE:integrations/testrail-mcp/tsconfig.json { "compilerOptions": { "target": "ES2022", "module": "ESNext", "moduleResolution": "bundler", "esModuleInterop": true, "strict": true, "outDir": "dist", "rootDir": "src", "declaration": true, "skipLibCheck": true }, "include": ["src/**/*"] } FILE:reference/assertions.md # Assertions Reference ## Web-First Assertions (Always Use These) Auto-retry until timeout. Safe for dynamic content. ```typescript // Visibility await expect(locator).toBeVisible(); await expect(locator).not.toBeVisible(); await expect(locator).toBeHidden(); // Text await expect(locator).toHaveText('exact text'); await expect(locator).toHaveText(/partial/i); await expect(locator).toContainText('partial'); // Value (inputs) await expect(locator).toHaveValue('entered text'); await expect(locator).toHaveValues(['option1', 'option2']); // Attributes await expect(locator).toHaveAttribute('href', '/dashboard'); await expect(locator).toHaveClass(/active/); await expect(locator).toHaveId('main-nav'); // State await expect(locator).toBeEnabled(); await expect(locator).toBeDisabled(); await expect(locator).toBeChecked(); await expect(locator).toBeEditable(); await expect(locator).toBeFocused(); await expect(locator).toBeAttached(); // Count await expect(locator).toHaveCount(5); await expect(locator).toHaveCount(0); // element doesn't exist // CSS await expect(locator).toHaveCSS('color', 'rgb(255, 0, 0)'); // Screenshots await expect(locator).toHaveScreenshot('button.png'); await expect(page).toHaveScreenshot('full-page.png'); ``` ## Page Assertions ```typescript await expect(page).toHaveURL('/dashboard'); await expect(page).toHaveURL(/\/dashboard/); await expect(page).toHaveTitle('Dashboard - App'); await expect(page).toHaveTitle(/Dashboard/); ``` ## Anti-Patterns (Never Do This) ```typescript // BAD — no auto-retry const text = await locator.textContent(); expect(text).toBe('Hello'); // BAD — snapshot in time, not reactive const isVisible = await locator.isVisible(); expect(isVisible).toBe(true); // BAD — evaluating in page context const value = await page.evaluate(() => document.querySelector('input')?.value ); expect(value).toBe('test'); ``` ## Custom Timeout ```typescript // Override timeout for slow operations await expect(locator).toBeVisible({ timeout: 30_000 }); ``` ## Soft Assertions Continue test even if assertion fails (report all failures at end): ```typescript await expect.soft(locator).toHaveText('Expected'); await expect.soft(page).toHaveURL('/next'); // Test continues even if above fail ``` FILE:reference/common-pitfalls.md # Common Pitfalls (Top 10) ## 1. waitForTimeout **Symptom:** Slow, flaky tests. ```typescript // BAD await page.waitForTimeout(3000); // GOOD await expect(page.getByTestId('result')).toBeVisible(); ``` ## 2. Non-Web-First Assertions **Symptom:** Assertions fail on dynamic content. ```typescript // BAD — checks once, no retry const text = await page.textContent('.msg'); expect(text).toBe('Done'); // GOOD — retries until timeout await expect(page.getByText('Done')).toBeVisible(); ``` ## 3. Missing await **Symptom:** Random passes/failures, tests seem to skip steps. ```typescript // BAD page.goto('/dashboard'); expect(page.getByText('Welcome')).toBeVisible(); // GOOD await page.goto('/dashboard'); await expect(page.getByText('Welcome')).toBeVisible(); ``` ## 4. Hardcoded URLs **Symptom:** Tests break in different environments. ```typescript // BAD await page.goto('http://localhost:3000/login'); // GOOD — uses baseURL from config await page.goto('/login'); ``` ## 5. CSS Selectors Instead of Roles **Symptom:** Tests break after CSS refactors. ```typescript // BAD await page.click('#submit-btn'); // GOOD await page.getByRole('button', { name: 'Submit' }).click(); ``` ## 6. Shared State Between Tests **Symptom:** Tests pass alone, fail in suite. ```typescript // BAD — test B depends on test A let userId: string; test('create user', async () => { userId = '123'; }); test('edit user', async () => { /* uses userId */ }); // GOOD — each test is independent test('edit user', async ({ request }) => { const res = await request.post('/api/users', { data: { name: 'Test' } }); const { id } = await res.json(); // ... }); ``` ## 7. Using networkidle **Symptom:** Tests hang or timeout unpredictably. ```typescript // BAD — waits for all network activity to stop await page.goto('/dashboard', { waitUntil: 'networkidle' }); // GOOD — wait for specific content await page.goto('/dashboard'); await expect(page.getByRole('heading', { name: 'Dashboard' })).toBeVisible(); ``` ## 8. Not Waiting for Navigation **Symptom:** Assertions run on wrong page. ```typescript // BAD — click navigates but we don't wait await page.getByRole('link', { name: 'Settings' }).click(); await expect(page.getByRole('heading')).toHaveText('Settings'); // GOOD — wait for URL change await page.getByRole('link', { name: 'Settings' }).click(); await expect(page).toHaveURL('/settings'); await expect(page.getByRole('heading')).toHaveText('Settings'); ``` ## 9. Testing Implementation, Not Behavior **Symptom:** Tests break on every refactor. ```typescript // BAD — tests CSS class (implementation detail) await expect(page.locator('.btn')).toHaveClass('btn-primary active'); // GOOD — tests what the user sees await expect(page.getByRole('button', { name: 'Save' })).toBeEnabled(); ``` ## 10. No Error Case Tests **Symptom:** App breaks on errors but all tests pass. ```typescript // Missing: what happens when the API fails? test('should handle API error', async ({ page }) => { await page.route('**/api/data', (route) => route.fulfill({ status: 500 }) ); await page.goto('/dashboard'); await expect(page.getByText(/error|try again/i)).toBeVisible(); }); ``` FILE:reference/fixtures.md # Fixtures Reference ## What Are Fixtures Fixtures provide setup/teardown for each test. They replace `beforeEach`/`afterEach` for shared state and are composable, type-safe, and lazy (only run when used). ## Creating Custom Fixtures ```typescript // fixtures.ts import { test as base, expect } from '@playwright/test'; // Define fixture types type MyFixtures = { authenticatedPage: Page; testUser: { email: string; password: string }; apiClient: APIRequestContext; }; export const test = base.extend({ // Simple value fixture testUser: async ({}, use) => { await use({ email: `test-${Date.now()}@example.com`, password: 'Test123!', }); }, // Fixture with setup and teardown authenticatedPage: async ({ page, testUser }, use) => { // Setup: log in await page.goto('/login'); await page.getByLabel('Email').fill(testUser.email); await page.getByLabel('Password').fill(testUser.password); await page.getByRole('button', { name: 'Sign in' }).click(); await expect(page).toHaveURL('/dashboard'); // Provide the authenticated page to the test await use(page); // Teardown: clean up (optional) await page.goto('/logout'); }, // API client fixture apiClient: async ({ playwright }, use) => { const context = await playwright.request.newContext({ baseURL: 'http://localhost:3000', extraHTTPHeaders: { Authorization: `Bearer ${process.env.API_TOKEN}`, }, }); await use(context); await context.dispose(); }, }); export { expect }; ``` ## Using Fixtures in Tests ```typescript import { test, expect } from './fixtures'; test('should show dashboard for logged in user', async ({ authenticatedPage }) => { // authenticatedPage is already logged in await expect(authenticatedPage.getByRole('heading', { name: 'Dashboard' })).toBeVisible(); }); test('should create item via API', async ({ apiClient }) => { const response = await apiClient.post('/api/items', { data: { name: 'Test Item' }, }); expect(response.ok()).toBeTruthy(); }); ``` ## Shared Auth State (storageState) For performance, authenticate once and reuse: ```typescript // auth.setup.ts import { test as setup } from '@playwright/test'; setup('authenticate', async ({ page }) => { await page.goto('/login'); await page.getByLabel('Email').fill('admin@example.com'); await page.getByLabel('Password').fill('password'); await page.getByRole('button', { name: 'Sign in' }).click(); await page.waitForURL('/dashboard'); await page.context().storageState({ path: '.auth/user.json' }); }); ``` ```typescript // playwright.config.ts export default defineConfig({ projects: [ { name: 'setup', testMatch: /.*\.setup\.ts/ }, { name: 'chromium', use: { storageState: '.auth/user.json', }, dependencies: ['setup'], }, ], }); ``` ## When to Use What | Need | Use | |---|---| | Shared login state | `storageState` + setup project | | Per-test data creation | Custom fixture with API calls | | Reusable page helpers | Custom fixture returning page | | Test data cleanup | Fixture teardown (after `use()`) | | Config values | Simple value fixture | FILE:reference/flaky-tests.md # Flaky Test Quick Reference ## Diagnosis Commands ```bash # Burn-in: expose timing issues npx playwright test tests/checkout.spec.ts --repeat-each=10 # Isolation: expose state leaks npx playwright test tests/checkout.spec.ts --grep "adds item" --workers=1 # Full trace: capture everything npx playwright test tests/checkout.spec.ts --trace=on --retries=0 # Parallel stress: expose race conditions npx playwright test --fully-parallel --workers=4 --repeat-each=5 ``` ## Four Categories | Category | Symptom | Fix | |---|---|---| | **Timing** | Fails intermittently | Replace waits with assertions | | **Isolation** | Fails in suite, passes alone | Remove shared state | | **Environment** | Fails in CI only | Match viewport, fonts, timezone | | **Infrastructure** | Random crashes | Reduce workers, increase memory | ## Quick Fixes **Timing → Add proper waits:** ```typescript // Wait for specific response const response = page.waitForResponse('**/api/data'); await page.getByRole('button', { name: 'Load' }).click(); await response; await expect(page.getByTestId('results')).toBeVisible(); ``` **Isolation → Unique test data:** ```typescript const uniqueEmail = `test-${Date.now()}@example.com`; ``` **Environment → Explicit viewport:** ```typescript test.use({ viewport: { width: 1280, height: 720 } }); ``` **Infrastructure → CI-safe config:** ```typescript export default defineConfig({ retries: process.env.CI ? 2 : 0, workers: process.env.CI ? 2 : undefined, timeout: process.env.CI ? 60_000 : 30_000, }); ``` FILE:reference/golden-rules.md # Golden Rules 1. **`getByRole()` over CSS/XPath** — resilient to markup changes, mirrors assistive technology 2. **Never `page.waitForTimeout()`** — use `expect(locator).toBeVisible()` or `page.waitForURL()` 3. **Web-first assertions** — `expect(locator)` auto-retries; `expect(await locator.textContent())` does not 4. **Isolate every test** — no shared state, no execution-order dependencies 5. **`baseURL` in config** — zero hardcoded URLs in tests 6. **Retries: `2` in CI, `0` locally** — surface flakiness where it matters 7. **Traces: `'on-first-retry'`** — rich debugging artifacts without CI slowdown 8. **Fixtures over globals** — share state via `test.extend()`, not module-level variables 9. **One behavior per test** — multiple related `expect()` calls are fine 10. **Mock external services only** — never mock your own app; mock third-party APIs, payment gateways, email FILE:reference/locators.md # Locator Priority Use the first option that works: | Priority | Locator | Use for | |---|---|---| | 1 | `getByRole('button', { name: 'Submit' })` | Buttons, links, headings, form elements | | 2 | `getByLabel('Email address')` | Form fields with associated labels | | 3 | `getByText('Welcome back')` | Non-interactive text content | | 4 | `getByPlaceholder('Search...')` | Inputs with placeholder text | | 5 | `getByAltText('Company logo')` | Images with alt text | | 6 | `getByTitle('Close dialog')` | Elements with title attribute | | 7 | `getByTestId('checkout-summary')` | When no semantic option exists | | 8 | `page.locator('.legacy-widget')` | CSS/XPath — absolute last resort | ## Role Locator Cheat Sheet ```typescript // Buttons —

playwright-pro

Comments (0)