Softwr

Testing · head to head

Testsigma vs Windmill

Testsigma logo

Testsigma

Testing

Natural language test automation for web, mobile and API with a self hosted open source edition

From
Free
Free plan
Yes
Windmill logo

Windmill

Automation Integration

Open-source developer platform for building scripts, workflows, and internal apps

From
Free
Free plan
Yes

The short version

  • Testsigma is free; Windmill starts at $120/mo.
  • Only Testsigma lists natural language authoring; only Windmill lists script-to-workflow engine.
  • Where Testsigma falls short: support and engineering are concentrated in Indian time zones, which slows incident turnaround for teams working in the Americas.
  • Where Windmill falls short: free self-hosted tier caps SSO users at 10 and workspaces at 3, requiring Enterprise for larger teams.
  • Prices and features on this page were last checked on 20 November 2026.

How do Testsigma and Windmill differ?

Testsigma and Windmill differ on pricing model (Per user per month against Open-source), platforms and category. Only the attributes on which Testsigma and Windmill actually diverge.

Attributes where Testsigma and Windmill differ
AttributeTestsigmaWindmill
Pricing modelPer user per monthOpen-source
PlatformsWeb, iOS, Android, LinuxWeb, API, Linux, Windows, macOS
CategoryTestingAutomation Integration

Identical on both: starting price (Free), free tier (Yes).

Which features does each one have?

They share no listed features; 6 appear only for Testsigma and 6 only for Windmill. Drawn from each product's published feature list. An absence here means we hold no record of it - not that the product lacks it.

Only in Testsigma

  • Natural language authoring
  • Unified web, mobile and API tests
  • Self hosted community edition
  • Cloud device and browser grid
  • Data driven runs
  • Custom functions

Only in Windmill

  • Script-to-workflow engine
  • Auto-generated UIs
  • Self-hosted deployment
  • Git sync
  • SSO support
  • Audit logs

What is each one used for?

Windmill is most often brought in for turning ad-hoc scripts into scheduled jobs and APIs. The jobs each tool is most often brought in to do.

Testsigma

  • A team maintaining separate Selenium and Appium suites that wants one place for web and native mobile coveragenot Windmill
  • Organisations that must keep test data and execution on premises for regulatory reasonsnot Windmill
  • QA groups with manual testers being asked to automate without first learning Javanot Windmill
  • Evaluating low code test automation properly before committing budget, by running the free edition against the real applicationnot Windmill

Windmill

  • Turning ad-hoc scripts into scheduled jobs and APIsnot Testsigma
  • Building internal tools from existing Python/TypeScript codenot Testsigma
  • Self-hosted automation for engineering teamsnot Testsigma
  • Orchestrating multi-language workflowsnot Testsigma

Where do Testsigma and Windmill fall short?

Windmill's most cited limit: free self-hosted tier caps SSO users at 10 and workspaces at 3, requiring Enterprise for larger teams. Documented limitations, not opinions. Every one is a constraint you would hit in normal use.

Testsigma

  • The natural language layer is a fixed vocabulary rather than real English, so testers spend the first weeks learning which phrasings the parser accepts, which is learning a language after all
  • Anything conditional or arithmetic falls through to custom Java or JavaScript, and that code lives in the platform editor rather than your repository, where it gets no code review and no version history you control
  • The community edition lags the cloud product and is not the version the vendor demonstrates, so a self hosted evaluation can understate or overstate what you would actually buy
  • Cloud execution is billed separately from authoring seats, so a suite that grows from two hundred to two thousand tests changes the bill without anyone changing the plan
  • Support and engineering are concentrated in Indian time zones, which slows incident turnaround for teams working in the Americas

Windmill

  • Free self-hosted tier caps SSO users at 10 and workspaces at 3, requiring Enterprise for larger teams
  • Compute unit and seat-based pricing (developer vs. operator seats, worker compute units) adds pricing complexity beyond a flat monthly fee
  • Email triggers are capped at 100/day on the free tier, limiting event-driven use cases without upgrading
  • Smaller ecosystem of pre-built integrations compared to more established automation tools like n8n

How much do Testsigma and Windmill cost?

Testsigma is free to use; Windmill has a free plan, with paid plans from $120/mo. Plan by plan below, from each vendor's own pricing page.

Testsigma

Free
  • Community EditionOn request
    • Self hosted, open source, no licence fee
    • You supply the infrastructure and the browser or device grid
  • CloudOn request
    • Priced per authoring user with additional charges for parallel cloud executions and device minutes
    • Published tiers exist but the figures that matter at team scale come from sales
  • ProOn request
  • EnterpriseOn request

Windmill

Free
  • Free / Open-sourceFree
    • Unlimited executions
    • Self-hosted deployment
    • Max 10 users with SSO
  • Enterprise$120/mo
    • Unlimited executions and workspaces
    • Uncapped SSO seats
    • 24/7 priority support (3-hour response)

Which should you pick?

Choose Testsigma if

  • You need natural language authoring.
  • You want to start without paying.
  • You work on Web, iOS, Android, Linux.
  • You also want unified web, mobile and api tests.

Choose Windmill if

  • You need script-to-workflow engine.
  • You want to start without paying.
  • You work on Web, API, Linux, Windows, macOS.
  • You also want auto-generated uis.

Questions people ask

Is Testsigma or Windmill better?
Neither clearly leads. Testsigma starts at Free and Windmill at Free, and user ratings are close enough to be indistinguishable. Choose on capability and platform support.
Which is cheaper, Testsigma or Windmill?
Testsigma starts at Free and Windmill at Free.
Does Testsigma or Windmill run on more platforms?
Testsigma runs on Web, iOS, Android, Linux. Windmill runs on Web, API, Linux, Windows, macOS.
Can I use Testsigma for free?
Both have a free tier, so you can try either at no cost before committing.
What is Testsigma best used for?
Testsigma is most often used for a team maintaining separate selenium and appium suites that wants one place for web and native mobile coverage, organisations that must keep test data and execution on premises for regulatory reasons, qa groups with manual testers being asked to automate without first learning java, evaluating low code test automation properly before committing budget, by running the free edition against the real application. Of those, a team maintaining separate selenium and appium suites that wants one place for web and native mobile coverage and organisations that must keep test data and execution on premises for regulatory reasons are not what Windmill is typically brought in for.
What can Testsigma do that Windmill cannot?
Testsigma covers Natural language authoring, Unified web, mobile and API tests, Self hosted community edition, Cloud device and browser grid. Windmill covers Script-to-workflow engine, Auto-generated UIs, Self-hosted deployment, Git sync.

Answered from the vendors’ own pages

Testsigma: Is the open source edition genuinely usable in production?

Yes for self hosted execution, though it trails the cloud version on newer features and you maintain the grid yourself.

Windmill: What does Windmill cost?

Windmill's Free and Open-source edition is $0/month for self-hosted use with unlimited executions, while Enterprise starts at $120/month with unlimited workspaces and uncapped SSO seats, priced further by developer/operator seats and compute units.

Source
Testsigma: Do testers need to know Selenium?

Not to write ordinary steps. They will need someone who does as soon as the suite needs custom functions or awkward waits.

Windmill: Is there a free plan and what are its limits?

The free self-hosted plan supports unlimited executions but caps SSO at 10 users, workspaces at 3, groups/permissions at 4, and email triggers at 100 per day.

Source
Testsigma: Can it test native iOS and Android applications?

Yes, through Appium underneath, on the hosted device grid or your own devices.

Windmill: How is usage metered on Windmill Enterprise?

Enterprise usage is metered through seats (developer seats at $20/month, operator seats at $10/month) and compute units, e.g. $100/month for 2 standard-worker compute units.

Source
Windmill: What happens to a self-hosted Windmill deployment if the company shuts down?

The pricing page addresses this directly as an FAQ topic, noting that self-hosted applications built on the open-source edition can continue running independently of Windmill the company.

Source
Share

Related pages

Other head to heads