Testing · head to head
Testsigma vs Windmill

Testsigma
Testing
Natural language test automation for web, mobile and API with a self hosted open source edition
- From
- Free
- Free plan
- Yes

Windmill
Automation Integration
Open-source developer platform for building scripts, workflows, and internal apps
- From
- Free
- Free plan
- Yes
The short version
- Testsigma is free; Windmill starts at $120/mo.
- Only Testsigma lists natural language authoring; only Windmill lists script-to-workflow engine.
- Where Testsigma falls short: support and engineering are concentrated in Indian time zones, which slows incident turnaround for teams working in the Americas.
- Where Windmill falls short: free self-hosted tier caps SSO users at 10 and workspaces at 3, requiring Enterprise for larger teams.
- Prices and features on this page were last checked on 20 November 2026.
How do Testsigma and Windmill differ?
Testsigma and Windmill differ on pricing model (Per user per month against Open-source), platforms and category. Only the attributes on which Testsigma and Windmill actually diverge.
Identical on both: starting price (Free), free tier (Yes).
Which features does each one have?
They share no listed features; 6 appear only for Testsigma and 6 only for Windmill. Drawn from each product's published feature list. An absence here means we hold no record of it - not that the product lacks it.
Only in Testsigma
- Natural language authoring
- Unified web, mobile and API tests
- Self hosted community edition
- Cloud device and browser grid
- Data driven runs
- Custom functions
Only in Windmill
- Script-to-workflow engine
- Auto-generated UIs
- Self-hosted deployment
- Git sync
- SSO support
- Audit logs
What is each one used for?
Windmill is most often brought in for turning ad-hoc scripts into scheduled jobs and APIs. The jobs each tool is most often brought in to do.
Testsigma
- A team maintaining separate Selenium and Appium suites that wants one place for web and native mobile coveragenot Windmill
- Organisations that must keep test data and execution on premises for regulatory reasonsnot Windmill
- QA groups with manual testers being asked to automate without first learning Javanot Windmill
- Evaluating low code test automation properly before committing budget, by running the free edition against the real applicationnot Windmill
Windmill
- Turning ad-hoc scripts into scheduled jobs and APIsnot Testsigma
- Building internal tools from existing Python/TypeScript codenot Testsigma
- Self-hosted automation for engineering teamsnot Testsigma
- Orchestrating multi-language workflowsnot Testsigma
Where do Testsigma and Windmill fall short?
Windmill's most cited limit: free self-hosted tier caps SSO users at 10 and workspaces at 3, requiring Enterprise for larger teams. Documented limitations, not opinions. Every one is a constraint you would hit in normal use.
Testsigma
- The natural language layer is a fixed vocabulary rather than real English, so testers spend the first weeks learning which phrasings the parser accepts, which is learning a language after all
- Anything conditional or arithmetic falls through to custom Java or JavaScript, and that code lives in the platform editor rather than your repository, where it gets no code review and no version history you control
- The community edition lags the cloud product and is not the version the vendor demonstrates, so a self hosted evaluation can understate or overstate what you would actually buy
- Cloud execution is billed separately from authoring seats, so a suite that grows from two hundred to two thousand tests changes the bill without anyone changing the plan
- Support and engineering are concentrated in Indian time zones, which slows incident turnaround for teams working in the Americas
Windmill
- Free self-hosted tier caps SSO users at 10 and workspaces at 3, requiring Enterprise for larger teams
- Compute unit and seat-based pricing (developer vs. operator seats, worker compute units) adds pricing complexity beyond a flat monthly fee
- Email triggers are capped at 100/day on the free tier, limiting event-driven use cases without upgrading
- Smaller ecosystem of pre-built integrations compared to more established automation tools like n8n
How much do Testsigma and Windmill cost?
Testsigma is free to use; Windmill has a free plan, with paid plans from $120/mo. Plan by plan below, from each vendor's own pricing page.
Testsigma
Free- Community EditionOn request
- Self hosted, open source, no licence fee
- You supply the infrastructure and the browser or device grid
- CloudOn request
- Priced per authoring user with additional charges for parallel cloud executions and device minutes
- Published tiers exist but the figures that matter at team scale come from sales
- ProOn request
- EnterpriseOn request
Windmill
Free- Free / Open-sourceFree
- Unlimited executions
- Self-hosted deployment
- Max 10 users with SSO
- Enterprise$120/mo
- Unlimited executions and workspaces
- Uncapped SSO seats
- 24/7 priority support (3-hour response)
Which should you pick?
Choose Testsigma if
- You need natural language authoring.
- You want to start without paying.
- You work on Web, iOS, Android, Linux.
- You also want unified web, mobile and api tests.
Choose Windmill if
- You need script-to-workflow engine.
- You want to start without paying.
- You work on Web, API, Linux, Windows, macOS.
- You also want auto-generated uis.
Questions people ask
- Is Testsigma or Windmill better?
- Neither clearly leads. Testsigma starts at Free and Windmill at Free, and user ratings are close enough to be indistinguishable. Choose on capability and platform support.
- Which is cheaper, Testsigma or Windmill?
- Testsigma starts at Free and Windmill at Free.
- Does Testsigma or Windmill run on more platforms?
- Testsigma runs on Web, iOS, Android, Linux. Windmill runs on Web, API, Linux, Windows, macOS.
- Can I use Testsigma for free?
- Both have a free tier, so you can try either at no cost before committing.
- What is Testsigma best used for?
- Testsigma is most often used for a team maintaining separate selenium and appium suites that wants one place for web and native mobile coverage, organisations that must keep test data and execution on premises for regulatory reasons, qa groups with manual testers being asked to automate without first learning java, evaluating low code test automation properly before committing budget, by running the free edition against the real application. Of those, a team maintaining separate selenium and appium suites that wants one place for web and native mobile coverage and organisations that must keep test data and execution on premises for regulatory reasons are not what Windmill is typically brought in for.
- What can Testsigma do that Windmill cannot?
- Testsigma covers Natural language authoring, Unified web, mobile and API tests, Self hosted community edition, Cloud device and browser grid. Windmill covers Script-to-workflow engine, Auto-generated UIs, Self-hosted deployment, Git sync.
Answered from the vendors’ own pages
Testsigma: Is the open source edition genuinely usable in production?
Yes for self hosted execution, though it trails the cloud version on newer features and you maintain the grid yourself.
Windmill: What does Windmill cost?
Windmill's Free and Open-source edition is $0/month for self-hosted use with unlimited executions, while Enterprise starts at $120/month with unlimited workspaces and uncapped SSO seats, priced further by developer/operator seats and compute units.
SourceTestsigma: Do testers need to know Selenium?
Not to write ordinary steps. They will need someone who does as soon as the suite needs custom functions or awkward waits.
Windmill: Is there a free plan and what are its limits?
The free self-hosted plan supports unlimited executions but caps SSO at 10 users, workspaces at 3, groups/permissions at 4, and email triggers at 100 per day.
SourceTestsigma: Can it test native iOS and Android applications?
Yes, through Appium underneath, on the hosted device grid or your own devices.
Windmill: How is usage metered on Windmill Enterprise?
Enterprise usage is metered through seats (developer seats at $20/month, operator seats at $10/month) and compute units, e.g. $100/month for 2 standard-worker compute units.
SourceWindmill: What happens to a self-hosted Windmill deployment if the company shuts down?
The pricing page addresses this directly as an FAQ topic, noting that self-hosted applications built on the open-source edition can continue running independently of Windmill the company.
SourceRelated pages
Other head to heads
- Testsigma vs Katalon
- Testsigma vs mabl
- Testsigma vs Ranorex
- Testsigma vs Robot Framework
- Testsigma vs QA Wolf
- Testsigma vs Sauce Labs
- Testsigma vs Selenium
- Testsigma vs BrowserStack
- Testsigma vs Appium
- Testsigma vs Insomnia
- Testsigma vs Detox
- Testsigma vs Xray Test Management
- Testsigma vs GrowthBook
- Testsigma vs Puppeteer
- Testsigma vs TestRail
- Testsigma vs Kameleoon
- Testsigma vs PractiTest
- Testsigma vs WebdriverIO
- Testsigma vs n8n
- Testsigma vs Temporal
- Testsigma vs Pipedream
- Testsigma vs Trigger.dev
- Testsigma vs Hatchet
- Testsigma vs Mage AI
- Testsigma vs Prefect
- Testsigma vs Nango
- Testsigma vs Airbyte
- Testsigma vs Meltano
- Testsigma vs Flowable
- Testsigma vs RudderStack
- Testsigma vs Nintex
- Testsigma vs Camunda
- Testsigma vs Dify
- Testsigma vs Hevo Data
- Testsigma vs SnapLogic
- Testsigma vs Hightouch
- Windmill vs Katalon
- Windmill vs mabl
- Windmill vs Ranorex
- Windmill vs Robot Framework
- Windmill vs QA Wolf
- Windmill vs Sauce Labs
- Windmill vs Selenium
- Windmill vs BrowserStack
- Windmill vs Appium
- Windmill vs Insomnia
- Windmill vs Detox
- Windmill vs Xray Test Management
- Windmill vs GrowthBook
- Windmill vs Puppeteer
- Windmill vs TestRail
- Windmill vs Kameleoon
- Windmill vs PractiTest
- Windmill vs WebdriverIO
- Windmill vs n8n
- Windmill vs Temporal
- Windmill vs Pipedream
- Windmill vs Trigger.dev
- Windmill vs Hatchet
- Windmill vs Mage AI
- Windmill vs Prefect
- Windmill vs Nango
- Windmill vs Airbyte
- Windmill vs Meltano
- Windmill vs Flowable
- Windmill vs RudderStack
- Windmill vs Nintex
- Windmill vs Camunda
- Windmill vs Dify
- Windmill vs Hevo Data
- Windmill vs SnapLogic
- Windmill vs Hightouch
