[Deep Dive #12] Testing Signals - discussion
Do testing signals (snapshots, render logs, and route-flow runs that are intentionally visible) actually reduce flakiness, or does it turn into a second UI surface you have to maintain?
What do you treat as a stable test signal (snapshots, render logs, render-tree inspection, route-flow runs)? How do you keep signals from leaking into product UI, while still being visible in screenshots and tests? And what's your cutoff for when a signal becomes too noisy to be useful?
Comments (10)
Back to latestIs this basically adding data-testid everywhere, or something different?
I'm not sure what makes a signal better than normal assertions.
Different. It's less tagging everything and more rendering the contract.
If the app can render a snapshot and keep a render log, tests can assert behavior without mocking internals.
Ok, so it's making runtime decisions observable, not just adding selectors.
Signals reduced flake for us because they moved assertions from timing to state.
Instead of waiting for a spinner to disappear, we assert on a stable snapshot/log signal and move on.
This pairs nicely with Testing and Debugging and the render-visible framing from Performance Audit Trails (Deep Dive #6).
A concrete example: route-flow runs + snapshots give you a stable baseline without tying tests to DOM shape.
ts
import { createSnapshot, runRouteFlow } from "react";
runRouteFlow("/docs/getting-started");
createSnapshot("route");The UI can change, but the contract signals stay stable unless you intentionally change behavior.
Counterpoint: signals can become a crutch where you stop testing user-visible behavior.
We require at least one real UI assertion per flow (text, focus, navigation) and then we use signals to disambiguate timing.
Agree. Signals should explain, not replace.
If signals say things are correct but the user sees broken UI, you've just moved the bug.
We kept signals from leaking by rendering them in a dedicated debug surface (snapshot + render log) that is visually subtle but present in screenshots.
The best signal is one you already want in prod for support.
If support benefits, it's less likely to get deleted as test-only clutter.
We hit a case where tests were flaky because we were asserting after a re-render that didn't always happen.
Once we added a render trail log keyed by surface, we could assert deterministically: form rendered 3 times after commit.
Signals are only good if they stay stable. Version them like contracts and keep them boring.
Noisy signals are the same as no signals. We keep ours to a handful: nav, fetch, guard, contractVersion.
If you can't explain a behavior from a screenshot + a snapshot/log signal, you'll end up writing flaky waits. Signals are the antidote.