Playwright Intro - discussion (2023-01-16)
Playwright has become the default answer for end-to-end testing, but I still see teams struggle with tests that are either too slow (full browser for everything) or too brittle (asserting on timing and DOM structure). I'm curious how people structure Playwright suites when the app architecture is route-first and relies on rendered evidence to make behavior explainable.
What do you assert on in E2E tests: visible text/roles, data-* evidence, or both? How do you keep route-flow tests deterministic when async work is intentionally "freshness first"? Do you treat traces/screenshots as part of the test output contract (stored in CI), or only for failures?
Comments (16)
Back to latestWe do both: user-visible assertions for correctness and data-* for posture.
If the app renders data-panel or data-freshness, tests can assert the route story without reaching into implementation details.
A pattern that's worked well is making a tiny helper that waits for a route evidence contract instead of sleeping:
ts
import { expect, Page } from '@playwright/test';
export async function expectRouteEvidence(page: Page, evidence: Record<string, string>) {
for (const [key, value] of Object.entries(evidence)) {
await expect(page.locator(`[data-${key}]`)).toHaveAttribute(`data-${key}`, value);
}
}It looks like ceremony, but it eliminated a bunch of timing flakes.
Counterpoint: if you lean too hard on data-*, you can accidentally create a testing-only API.
We keep evidence keys minimal (lane/status/selection) and prefer roles/text for everything else.
Agree. Evidence should be explainability, not a parallel DOM schema.
If the evidence doesn't help humans debug screenshots, it's probably not worth keeping.
Traces in CI were worth it for us, but only for failures.
The rule is: failing test uploads trace + screenshot + console logs and that becomes the artifact contract.
We made route-flow tests deterministic by defining what "done" means as evidence.
If the app doesn't expose a data-status=ok, the test can't be reliable without timing hacks.
Playwright's auto-waiting helps, but it doesn't solve apps that constantly re-render fresh data.
We had to pick stable anchors (route state evidence and a few role-based assertions).
Short tip: if a test is flaky, open the trace and look for the first time the UI *could* have been asserted correctly.
Most flakes are really "asserted before the route contract settled".
We keep a small page-object layer only for repeated flows (login, checkout).
For most tests, raw locators + role queries are more honest and less brittle.
A practical config we use for stability: record video on retry, keep trace on first retry only.
It makes intermittent flakes debuggable without filling storage with artifacts.
We also assert that navigation is explainable by checking a visible route marker, not just that the URL changed.
If the URL changed but the route state didn't, the UI can still be wrong.
Docs tie-in: making tests assert on evidence aligns with the repo's testing posture.
Related: Testing and Debugging and Routing and Navigation.
We had a misunderstanding early on: we treated page.waitForLoadState('networkidle') as "done".
In a modern app, "done" is a route contract, not an idle network.
Same. Network can be busy forever. Evidence is the only stable completion signal.
We used Playwright's request mocking only for a small number of tests.
If you mock everything, you stop testing the route story and start testing your mocks.
Another code snippet: a route-flow test that asserts both user-visible content and evidence keys:
ts
import { test, expect } from '@playwright/test';
test('support flow: filter + open thread', async ({ page }) => {
await page.goto('/support');
await page.getByRole('searchbox').fill('refund');
await expect(page.getByTestId('queue')).toHaveAttribute('data-status', 'open');
await page.getByRole('link', { name: /ticket/i }).first().click();
await expect(page.getByTestId('thread')).toBeVisible();
});Counterpoint: sometimes data-* evidence hides accessibility issues because tests stop using roles.
We now require role-based assertions for primary actions, even if evidence exists.
That's a good rule. Evidence helps debuggability, roles help correctness and accessibility.
We try to keep both present in tests.
Playwright got easier for us once we treated traces as part of the product debugging story, not just test output.