Softwr

Developer Tools · head to head

Atlantis vs Testsigma

Atlantis logo

Atlantis

Developer Tools

Runs Terraform plan and apply from pull request comments, self-hosted and free

From
Free
Rated
-
Full reviewWebsite
Testsigma logo

Testsigma

Testing

Natural language test automation for web, mobile and API with a self hosted open source edition

From
Free
Rated
-
Full reviewWebsite

The short version

  • Each has a real cost: Atlantis there is no company, no service level agreement and no paid support at any price, and the six maintainers are volunteers with day jobs, which some risk committees will not accept for a component holding production cloud credentials.; Testsigma the natural language layer is a fixed vocabulary rather than real English, so testers spend the first weeks learning which phrasings the parser accepts, which is learning a language after all.
  • They diverge on capability: Atlantis covers Pull request plans, Testsigma covers Natural language authoring.
  • Prices and features above were last checked on 31 August 2026.

Where they differ

Only the attributes on which Atlantis and Testsigma actually diverge.

Attributes where Atlantis and Testsigma differ
AttributeAtlantisTestsigma
Pricing modelOpen source, no licence feePer user per month
PlatformsLinux, Docker, Kubernetes, Self-hostedWeb, iOS, Android, Linux
CategoryDeveloper ToolsTesting

Identical on both: starting price (Free), free tier (Yes), user rating (Not yet rated).

What each one covers

Drawn from each product's published feature list. An absence here means we hold no record of it - not that the product lacks it.

Only in Atlantis

  • Pull request plans
  • Comment driven apply
  • Workspace locking
  • Policy checking
  • Multi-platform webhooks
  • Custom workflows

Only in Testsigma

  • Natural language authoring
  • Unified web, mobile and API tests
  • Self hosted community edition
  • Cloud device and browser grid
  • Data driven runs
  • Custom functions

What people use each for

The jobs each tool is most often brought in to do.

Atlantis

  • Making Terraform review meaningful by putting the actual plan output in front of the approvernot Testsigma
  • Removing local applies and the credential sprawl that comes with every engineer holding production keysnot Testsigma
  • Getting pull request driven infrastructure without paying a per-user subscription for a hosted platformnot Testsigma
  • Enforcing Conftest policies as a blocking check before an apply can runnot Testsigma

Testsigma

  • A team maintaining separate Selenium and Appium suites that wants one place for web and native mobile coveragenot Atlantis
  • Organisations that must keep test data and execution on premises for regulatory reasonsnot Atlantis
  • QA groups with manual testers being asked to automate without first learning Javanot Atlantis
  • Evaluating low code test automation properly before committing budget, by running the free edition against the real applicationnot Atlantis

Where each one falls short

Documented limitations, not opinions. Every one is a constraint you would hit in normal use.

Atlantis

  • There is no company, no service level agreement and no paid support at any price, and the six maintainers are volunteers with day jobs, which some risk committees will not accept for a component holding production cloud credentials.
  • The security surface is inherently sharp, because it is an internet-reachable webhook endpoint that executes applies with privileged credentials, gated by pull request comment authorisation that is coarse next to a real policy engine.
  • It is stateful, holding a working directory and lock database on local disk, so high availability and horizontal scaling are awkward and you own upgrades, webhook plumbing, secret rotation and the host itself.
  • It lacks the features that distinguish the commercial alternatives, with no native drift detection, no cost estimation, no managed state interface and no policy engine beyond the Conftest integration.
  • Terragrunt and monorepo layouts are not first-class, requiring hand-written custom workflows and repository configuration that becomes a growing maintenance burden as the number of repositories increases.

Testsigma

  • The natural language layer is a fixed vocabulary rather than real English, so testers spend the first weeks learning which phrasings the parser accepts, which is learning a language after all.
  • Anything conditional or arithmetic falls through to custom Java or JavaScript, and that code lives in the platform editor rather than your repository, where it gets no code review and no version history you control.
  • The community edition lags the cloud product and is not the version the vendor demonstrates, so a self hosted evaluation can understate or overstate what you would actually buy.
  • Cloud execution is billed separately from authoring seats, so a suite that grows from two hundred to two thousand tests changes the bill without anyone changing the plan.
  • Support and engineering are concentrated in Indian time zones, which slows incident turnaround for teams working in the Americas.

Pricing, plan by plan

Atlantis

Free
  • AtlantisFree
    • Apache-2.0, no usage limits and no seat count
    • No commercial edition and no paid support tier
    • Community support through GitHub and the CNCF Slack channel

Testsigma

Free
  • Community Edition$undefined/month
    • Self hosted, open source, no licence fee
    • You supply the infrastructure and the browser or device grid
  • Cloud$undefined/month
    • Priced per authoring user with additional charges for parallel cloud executions and device minutes
    • Published tiers exist but the figures that matter at team scale come from sales

Which should you pick?

Choose Atlantis if

  • You need pull request plans.
  • You want to start without paying.
  • You work on Linux, Docker, Kubernetes, Self-hosted.
  • You also want comment driven apply.
Atlantis review

Choose Testsigma if

  • You need natural language authoring.
  • You want to start without paying.
  • You work on Web, iOS, Android, Linux.
  • You also want unified web, mobile and api tests.
Testsigma review

Questions people ask

Is Atlantis or Testsigma better?
Neither clearly leads. Atlantis starts at Free and Testsigma at Free, and user ratings are close enough to be indistinguishable. Choose on capability and platform support.
Which is cheaper, Atlantis or Testsigma?
Atlantis starts at Free and Testsigma at Free.
Does Atlantis or Testsigma run on more platforms?
Atlantis runs on Linux, Docker, Kubernetes, Self-hosted. Testsigma runs on Web, iOS, Android, Linux.
Can I use Atlantis for free?
Both have a free tier, so you can try either at no cost before committing.
What is Atlantis best used for?
Atlantis is most often used for making terraform review meaningful by putting the actual plan output in front of the approver, removing local applies and the credential sprawl that comes with every engineer holding production keys, getting pull request driven infrastructure without paying a per-user subscription for a hosted platform, enforcing conftest policies as a blocking check before an apply can run. Of those, making terraform review meaningful by putting the actual plan output in front of the approver and removing local applies and the credential sprawl that comes with every engineer holding production keys are not what Testsigma is typically brought in for.
What can Atlantis do that Testsigma cannot?
Atlantis covers Pull request plans, Comment driven apply, Workspace locking, Policy checking. Testsigma covers Natural language authoring, Unified web, mobile and API tests, Self hosted community edition, Cloud device and browser grid.

Answered from the vendors’ own pages

Atlantis: Is there a company behind Atlantis?

No. It runs as a series of LF Projects under the Linux Foundation and is maintained by volunteers. There is no vendor to buy support from and no service level agreement.

Testsigma: Is the open source edition genuinely usable in production?

Yes for self hosted execution, though it trails the cloud version on newer features and you maintain the grid yourself.

Atlantis: Is it a CNCF project?

No. It uses a channel in the CNCF Slack, which is often misread as membership, but the project sits under LF Projects rather than the CNCF.

Testsigma: Do testers need to know Selenium?

Not to write ordinary steps. They will need someone who does as soon as the suite needs custom functions or awkward waits.

Atlantis: Does it work with OpenTofu?

Yes, there is a dedicated integration guide alongside Terraform, and it also has configuration for working with hosted Terraform backends.

Testsigma: Can it test native iOS and Android applications?

Yes, through Appium underneath, on the hosted device grid or your own devices.

Atlantis: What does it not do that a paid platform does?

Drift detection, cost estimation, a managed state and run interface, and a real policy engine. Atlantis does pull request plan and apply well and stops there.

Share

Related pages

More on Atlantis

More on Testsigma

Keep looking

Other head to heads