Smoke Testing vs Regression Testing: When to Run Each

Smoke testing is a quick check that a new build is not fundamentally broken: can you log in, load the main page and reach the core workflow? Regression testing is a wider check that recent changes have not broken features that already worked. Run smoke tests first, on every build, in minutes; run regression tests before a release, over the journeys that matter.

If you would rather have both built and gated in your pipeline for you, see how RAITHub would test this below, or the QA and test automation overview.

What is smoke testing?

Smoke testing is a shallow, broad pass that answers one question: is this build stable enough to test further? The name comes from hardware: power the board on and see if it smokes. In software, a smoke test touches the handful of things that, if broken, make everything else pointless, such as the app starting, sign-in working, and the home screen and one core action loading.

A smoke test is fast and runs often, usually on every build or every pull request. If it fails, you stop and fix the build before anyone spends time on deeper testing. It is a gate, not a thorough examination.

What is regression testing?

Regression testing checks that code you changed has not broken code you did not. A "regression" is a feature that used to work and now does not, usually because a change somewhere else had a side effect nobody expected. Regression testing re-runs the checks over existing features, not just the new one, so those side effects are caught before users meet them.

Because it is wider and deeper than a smoke test, regression testing takes longer, which is why teams automate the bulk of it. Running it manually on every release does not scale; for why this matters after every deploy, see regression testing after every deploy.

Smoke testing vs regression testing, side by side

Factor	Smoke testing	Regression testing
Question it answers	Is this build broken?	Did this change break anything that worked before?
Depth	Shallow and broad: core paths only	Wide and deep: existing features across the product
When to run	On every build or pull request, first	Before a release, after merges, after a bug fix
How long	Seconds to a few minutes	Minutes to hours, depending on coverage
Usually	Automated, a small fixed set	Automated in bulk, with some manual checks
On failure	Stop; the build is not worth testing	Find which change caused the regression, then fix
Also called	Build verification test, sanity gate	Non-regression testing

They are not rivals. A smoke test is the gate you run first; regression tests are the body of work you run once the gate passes. A healthy pipeline has both.

When should you run a smoke test?

• On every new build, before any deeper testing, so a broken build is caught in minutes rather than wasting a test cycle.
• After a deployment to staging or production, as a post-deploy check that the live system came up and the critical paths respond.
• Before a demo or an investor review, as a last-minute confidence check. The wider pass belongs earlier; see testing your app before an investor demo.

When should you run regression tests?

• Before every release, so a change made this sprint has not broken a feature shipped last sprint.
• After a bug fix, because a fix is itself a change and can reintroduce or create a regression.
• After a dependency upgrade or a refactor, where the change is wide and the blast radius is hard to predict.
• Continuously, if you ship often. Teams releasing weekly automate regression into CI so it runs on every merge; see a SaaS release process for weekly releases.

How do smoke and regression tests fit a CI pipeline?

They sit at different stages of the same pipeline, fastest and least costly first. A simple order looks like this:

1. Static checks: type checking and linting, which catch whole classes of error before anything runs.
2. Unit and integration tests: fast, close to the code, run on every push.
3. Smoke test: a small end-to-end set over the core paths, run on every pull request. If it fails, the build stops here.
4. Regression suite: the wider end-to-end coverage, run before a merge to the release branch or on a schedule.

This mirrors the test pyramid: many fast tests at the bottom, fewer slow ones at the top. For how to size each layer, see the test pyramid for a SaaS. A minimal smoke test in Playwright is often just a few checks:

import { test, expect } from '@playwright/test'

// Smoke test: the build is usable if these pass.
test('home page loads', async ({ page }) => {
  await page.goto('/')
  await expect(page.getByRole('heading', { level: 1 })).toBeVisible()
})

test('a signed-in user reaches the dashboard', async ({ page }) => {
  await page.goto('/login')
  await page.getByLabel('Email').fill('demo@example.com')
  await page.getByLabel('Password').fill('correct-horse')
  await page.getByRole('button', { name: 'Sign in' }).click()
  await expect(page).toHaveURL(/\/dashboard/)
})

The regression suite is the same shape, only much larger, and it runs later in the pipeline because it costs more time.

Buy, build or hire?

You can cover smoke and regression testing with a tool, a freelancer, a hire or a managed service.

Route	What it costs on the market	Choose this when
A tool or SaaS testing platform	Cloud browser and device testing from $29 to $39 a month per user (BrowserStack pricing)	Your developers write and own the tests and only need browsers, devices or a runner
Freelancers or crowdtesting	Upwork median $35 an hour for QA engineers (Upwork)	A one-off regression sweep before a big release
An in-house QA hire	US median wage $104,300 in May 2025, before benefits (BLS)	Testing is continuous and you can lead a QA function
A managed QAaaS team	Quoted per scope: a monthly plan or a dedicated team	You ship often and want smoke and regression gates built and kept green for you

If you want to do it yourself first: a useful smoke suite is a few hours of work for someone who knows your stack and Playwright; a real regression suite is an ongoing project, not an afternoon. The main risk of doing it alone is a flaky suite that fails at random and gets ignored, which is worse than no suite; see how to fix flaky end-to-end tests.

Why RAITHub for smoke and regression testing?

• Gates that block, not just report. RAITHub builds the smoke and regression layers into your CI so a failing change cannot reach users.
• Counted suites. PropDesk runs 1,024 automated tests and Sundor Skin runs 530+, including a suite that tries to read other buyers' data. TheSkinProof, the founder's own venture, runs 750+.
• Tests you keep. The suite and CI configuration live in your repository, with IP assigned to you and an NDA as standard.

When you don't need us for this

• When your developers already maintain a green smoke and regression suite and only need more devices. Buy a device cloud.
• When you need one regression pass before a single launch, not an ongoing gate. A pre-launch QA audit or a freelancer fits better.
• When you want testers placed under your own managers. RAITHub does not offer staff augmentation.

How RAITHub would test this

• Scope: name the core paths for the smoke gate and the wider journeys for regression, plus the browsers and devices in range.
• Smoke first: a small, fast end-to-end set on every pull request, set to block a merge when it fails.
• Regression next: a wider suite that runs before release or on a schedule, with the flaky tests repaired or removed so the signal stays trustworthy.
• Reporting: a regular report of what was caught and what changed, and a review of what repeated manual check should become an automated test.

Timeline: a monthly QA plan or dedicated team runs month to month; a one-off hardening pass is fixed in scope first. You receive: the test suite and CI configuration in your repository, a handover document, full IP and an NDA. Next step: a free 15-minute audit, then a written fixed quote. RAITHub publishes no rates.

To set up smoke and regression gates, tell RAITHub about your release process.

Frequently asked questions

What is the difference between smoke and regression testing?

Smoke testing is a quick, shallow check that a build is not fundamentally broken, run first. Regression testing is a wider, deeper check that recent changes have not broken features that already worked, run before a release.

Which do you run first, smoke or regression?

Smoke testing, always. If the smoke test fails, the build is not worth deeper testing, so you stop and fix it before spending time on the regression suite.

Is smoke testing the same as sanity testing?

They overlap. Smoke testing is a broad check across the core paths of a new build; sanity testing is a narrow check that one specific fix or feature works. Teams often use the terms loosely for the same quick gate.

Should smoke and regression tests be automated?

Yes, for the bulk of them. A smoke test runs on every build, and a regression suite runs on every release, so manual repetition does not scale. Automation keeps both fast enough to gate a pipeline.

How long should a smoke test take?

Seconds to a few minutes. If a smoke test grows past a few minutes it has usually turned into a small regression suite, and the core-path checks should be split back out so the gate stays fast.

Can I run regression testing without a full test suite?

You can do it manually, but it does not scale past a small product. Each release re-tests every feature, so without automation the cost grows with the product. Most teams automate regression early for that reason.