Manual vs Automated Testing: What to Automate and What to Keep Human

Automate any check you will run again on the next release: regression of core journeys, API contracts, permissions, calculations and anything run across many browsers. Keep people on what needs judgement: new features, exploratory testing, usability, visual polish and accessibility with a screen reader. Most products need both, with automation as the safety net and manual testing as the search party.

If you would rather have the split designed and run for you, see how RAITHub would test this below.

What is the difference between manual and automated testing?

Manual testing is a person using the software and deciding whether it behaves well. Automated testing is code that drives the software and checks a fixed expected result, usually on every change in a CI pipeline.

The difference is not speed alone. An automated test can only check what someone thought to write down. A person notices what nobody wrote down: a confusing label, a slow screen, a total that looks wrong. That is why the two are partners, not rivals.

Industry surveys reflect that. Katalon's State of Quality Report 2025, a survey of about 1,400 QA professionals, found that up to 82% of testers still use manual testing in their day-to-day work, and that regression testing is the most automated activity.

Which tests should you automate?

Automate a check when it is repeated, deterministic and expensive to miss.

• Regression of core journeys. Sign-up, login, checkout, the main workflow. If it broke once, it can break again.
• Permissions. "User B cannot read user A's invoice" is a yes or no answer, and it must hold on every release.
• API contracts. Status codes, response shapes and validation rules; see the API testing guide.
• Calculations. Prices, taxes, discounts and date logic, with many input combinations.
• Cross-browser smoke checks. The same journey in Chromium, Firefox and WebKit costs one script, not three people.
• Performance budgets. Load and latency thresholds that fail the build when crossed.

Martin Fowler's Practical Test Pyramid adds the shape: many fast unit tests, fewer service or API tests, and a small number of end-to-end UI tests. The test pyramid for a SaaS turns that into numbers.

Which tests should stay manual?

Keep a person on anything where the expected result is a judgement, or where the feature is still changing.

• Exploratory testing. Chartered sessions that hunt for what scripts miss; see the exploratory testing guide.
• New features in their first releases. Automating a screen that will change three times this month means rewriting the test three times.
• Usability and copy. Is it clear? Would a customer understand the error message? No assertion answers that.
• Visual polish. Screenshot comparison tools help, but someone has to decide whether a change is a bug or a redesign.
• Accessibility with assistive technology. Automated scanners catch some WCAG failures; a keyboard and screen-reader pass catches the rest.
• User acceptance. Whether the product fits the business is for its users to judge; see user acceptance testing.
• One-off checks. A data migration you run once, verified once.

How do you decide, check by check?

Ask four questions of each check. If most answers are in the left column, automate it.

Question	Points to automation	Points to manual
How often will it run?	Every release or every change	Once, or rarely
Is the expected result fixed?	Yes: a value, a status, a permission	No: it needs judgement or taste
Is the feature stable?	Shipped and settled	Still changing this month
What does missing it cost?	Money, data or trust	A cosmetic fix next week
How many combinations?	Many browsers, roles or inputs	One path, one setup

What does an automated check look like?

A minimal end-to-end regression test in Playwright, checking that a signed-in user can reach their dashboard. It uses role-based locators, which Playwright's best practices recommend because they match how users and assistive technology see the page.

import { test, expect } from '@playwright/test'

test('a returning user can sign in and see the dashboard', async ({ page }) => {
  await page.goto('/login')
  await page.getByLabel('Email').fill('qa-user@example.com')
  await page.getByLabel('Password').fill(process.env.QA_USER_PASSWORD ?? '')
  await page.getByRole('button', { name: 'Sign in' }).click()

  await expect(page).toHaveURL(/\/dashboard/)
  await expect(page.getByRole('heading', { name: 'Dashboard' })).toBeVisible()
})

Written once, this runs on every pull request. A person running the same steps by hand would pay for them every release. A person would also never notice anything else, though: that the dashboard took eight seconds, or that the welcome copy says "undefined". That is the trade in one example.

What are the costs and risks of each approach?

Automation costs more up front and less per run. Manual testing costs little to start and the same every time.

• Automation's hidden cost is maintenance. UI changes break selectors, and slow or unstable tests become flaky. Google's testing blog found that larger tests are more likely to be flaky than small ones, which is one reason to keep end-to-end tests few. The flaky e2e tests fix covers the repairs.
• Automation's other risk is false confidence. A green pipeline only says the written checks passed.
• Manual testing's hidden cost is repetition. Every release pays for the same regression again, and people get tired and skip steps.
• Manual testing's other risk is memory. If cases are not written down, coverage leaves with the tester.

Rate data reflects the up-front gap: automation QA runs 20 to 50% above manual QA in every region, according to QA Madness's 2026 rates. The manual testing cost guide has the full ranges.

How should a small SaaS split its testing?

A workable starting split for a product releasing weekly:

1. Automate the money and data paths first. Sign-up, login, payment, the main workflow, and the cross-tenant permission checks, gated in CI.
2. Add unit and API tests as code changes. Cheap, fast and stable; most bugs should be caught here.
3. Keep a short manual regression list for what is hard to automate: emails, file uploads, payments in production, third-party redirects.
4. Spend real human time on exploratory sessions for each new feature before it ships.
5. Move checks from manual to automated once a feature stops changing.

The QA set-up for an early-stage SaaS goes deeper on sequencing.

Buy, build or hire?

Option	What you get	Choose this when	Watch out for
A tool or SaaS testing platform	Low-code recorders, test management or a device cloud	Your developers will own the tests and need tooling	Recorded tests break easily and still need someone to maintain them
Freelancers or crowdtesting	Manual passes or scripted tests by the hour, or a crowd per test cycle	A burst of manual coverage before a release	Automation written by a short-term freelancer often outlives anyone who understands it
An in-house QA hire	One person doing both manual and automation work	You release often and can keep a tester busy full time	Few people are strong at both; one hire is also a single point of failure
A managed QAaaS team	Manual, exploratory and automated testing planned and run together, billed monthly or as an audit	You want the split designed and owned for you	Insist that tests live in your repository and run in your CI

Why RAITHub for this?

• Both halves in one service. QA as a service covers manual and exploratory testing, automation, API and performance testing, so the split follows your risk, not a vendor's specialism.
• Automation measured in real repositories. 750+ tests on TheSkinProof (the founder's own venture), 1,024 on PropDesk and 530+ on Sundor Skin, gated in CI.
• Tests stay yours. They live in your repository with IP assigned to you.

When to use a tool instead

• Your developers already write tests and only need a runner and a CI job. Use Playwright or Cypress directly; Playwright vs Cypress compares them.
• The product is a prototype you may throw away. A short manual checklist is enough until it settles.

How RAITHub would test this

• Scope: map your journeys and rank them by risk; mark each check automate or keep manual using the table above; automate the top journeys and permission checks in CI; run exploratory sessions on new features; keep a short manual release list.
• How you buy it: a monthly QA plan, a fixed-price one-off audit of your current testing, or a dedicated QA team that RAITHub manages and bills monthly. No staff augmentation.
• Timeline: set out in the fixed written quote; automation builds up release by release from the highest-risk journey down.
• What you receive: automated tests and CI configuration in your repository, the manual test list, session notes and bug reports, handover docs, IP assigned and an NDA as standard. See manual testing and QA test automation.
• Next step: a free 15-minute audit call, then a fixed written quote.

Pick QA / Reliability on the contact form to start.

Last reviewed: 7 October 2026.

Frequently asked questions

Can automated testing replace manual testing completely?

No. Automation checks what someone wrote down in advance. People still need to explore new features, judge usability and copy, and test with assistive technology. Automation replaces repeated manual regression, not human judgement.

What percentage of tests should be automated?

There is no fixed figure. A practical rule is to automate every check you run on each release and keep humans on new features and exploratory work. For most products, regression of core journeys, permissions and API contracts should be fully automated.

When is manual testing better than automation?

For new or changing features, usability, visual polish, accessibility with a screen reader, one-off checks such as a data migration, and user acceptance testing, where the users themselves decide.

Is test automation worth it for a startup?

For the journeys that make money or hold data, yes, as soon as you release more than occasionally. Start small: a handful of end-to-end tests on the core paths, gated in CI, and unit tests on business logic.

Which tool should we use for automated web testing?

Playwright and Cypress are common choices for end-to-end web tests. Pick one your developers will maintain, run it in CI on every pull request, and keep the end-to-end layer small.

Does RAITHub do both manual and automated testing?

Yes. RAITHub's QA as a service covers manual and exploratory testing, UAT support, automation, API and performance testing, sold as a monthly plan, a one-off audit or a dedicated managed team.