PromptSharp › Prompt Library › Dev & Engineering › Flaky test investigation: separate the race condition from the environment

Flaky test investigation: separate the race condition from the environment

A test fails one run in twenty and the team has started re-running CI until it passes. Find out whether the flake is in the code, the test, or the environment.

The prompt — copy and run it

You are a senior engineer investigating an intermittently failing test. I will paste the test, the code under test, and failure logs from several runs, ideally with passing runs for comparison.

Produce:

A) FAILURE PATTERN — what differs between failing and passing runs in the logs I pasted: timing, ordering, environment, data, parallelism.

B) HYPOTHESES — ranked causes from the classic flake families (shared state between tests, timing assumptions, ordering of unordered collections, time zone or clock, network or external service, resource limits), each citing the log or code line that supports it.

C) DISCRIMINATING EXPERIMENTS — for the top hypotheses, the run that would confirm or rule it out: run in isolation, run with a fixed seed, run serially, loop the test many times, freeze the clock.

D) FIX DIRECTION — once confirmed, whether the fix belongs in the code, the test, or the environment, and why masking it with a retry is the wrong default.

Inputs: [PASTE THE TEST] · [PASTE CODE UNDER TEST] · [PASTE FAILING + PASSING RUN LOGS] · [CI SETUP: PARALLELISM, RUNNERS, SERVICES]

Rules: Do not invent log lines or causes the evidence does not support — label speculation as unconfirmed. Verify a hypothesis with a reproducing run before changing any code. Keep secrets, internal hostnames, and proprietary code out of consumer AI tools beyond what I pasted.

Why this prompt works

Flaky tests erode trust in the whole suite, and the usual response, re-running until green, hides real race conditions that later appear in production. Comparing failing and passing runs, ranking causes from known flake families, and requiring a reproducing experiment before a fix turns a nuisance into a diagnosable bug. Deciding where the fix belongs prevents the reflexive retry decorator.

Frequently asked

Related Dev & Engineering prompts