Softwr

Testing · head to head

Cucumber vs Umami

Cucumber logo

Cucumber

Testing

Behaviour-driven development testing framework that returned to community ownership after nine years under SmartBear

From
Free
Rated
-
Full reviewWebsite
Umami logo

Umami

Marketing

Open source, cookie-free web analytics you can self-host for the cost of a database

From
Free
Rated
-
Full reviewWebsite

The short version

  • Each has a real cost: Cucumber gherkin's plain-language layer adds a real maintenance burden, keeping feature files, step definitions and implementation code synchronised across a large suite is a known source of drift and duplicated effort.; Umami there is no cohort analysis, retention curve, session replay or multi-touch attribution, so a marketing team that needs to justify channel spend will still buy a second tool and Umami becomes a supplementary dashboard.
  • They diverge on capability: Cucumber covers Gherkin syntax, Umami covers Cookie-free tracking.
  • Prices and features above were last checked on 1 September 2026.

Where they differ

Only the attributes on which Cucumber and Umami actually diverge.

Attributes where Cucumber and Umami differ
AttributeCucumberUmami
Pricing modelOpen source, no licence feePer month by event count
PlatformsLinux, macOS, WindowsWeb, Linux
CategoryTestingMarketing

Identical on both: starting price (Free), free tier (Yes), user rating (Not yet rated).

What each one covers

Drawn from each product's published feature list. An absence here means we hold no record of it - not that the product lacks it.

Only in Cucumber

  • Gherkin syntax
  • Step definition binding
  • Multi-language support
  • Living documentation
  • Tagging and scenario outlines
  • Reporting plugins

Only in Umami

  • Cookie-free tracking
  • Self-hosting
  • Small tracking script
  • Custom events
  • Funnels and goals
  • Multi-site and teams
  • Public dashboards
  • API access

What people use each for

The jobs each tool is most often brought in to do.

Cucumber

  • A team practising behaviour-driven development that wants shared, readable specifications between product owners and engineersnot Umami
  • An organisation wanting living documentation that stays synchronised with actual test coverage rather than a separate specification documentnot Umami
  • A QA team migrating acceptance tests from a proprietary BDD tool to an open source, community-governed alternativenot Umami
  • A Java, JavaScript or Ruby project needing a mature BDD framework rather than building Gherkin-style parsing from scratchnot Umami

Umami

  • A developer who wants traffic numbers on a personal or client site without adding a consent bannernot Cucumber
  • An organisation whose legal or procurement team has ruled out sending visitor data to a US advertising companynot Cucumber
  • A high-traffic site where hosted analytics priced by pageview band has become expensive and self-hosting removes the ceiling entirelynot Cucumber
  • A team that wants analytics data in a database they already control so they can query and join it themselvesnot Cucumber

Where each one falls short

Documented limitations, not opinions. Every one is a constraint you would hit in normal use.

Cucumber

  • Gherkin's plain-language layer adds a real maintenance burden, keeping feature files, step definitions and implementation code synchronised across a large suite is a known source of drift and duplicated effort.
  • Ownership has moved twice in under a decade, from independent to SmartBear-owned to independent community ownership again, which should temper confidence in any specific long-range roadmap claim.
  • Non-technical stakeholders who are meant to read and write Gherkin scenarios often do not engage with them in practice, leaving the readability benefit unrealised while the maintenance cost remains.
  • Step definition reuse across a large feature file set can become tangled, with ambiguous step matching causing hard-to-debug failures as a suite grows.
  • As a community-owned project without a corporate backer's support infrastructure, enterprise buyers wanting a formal support contract have no vendor to buy one from, unlike some competing BDD tools still sold commercially.

Umami

  • There is no cohort analysis, retention curve, session replay or multi-touch attribution, so a marketing team that needs to justify channel spend will still buy a second tool and Umami becomes a supplementary dashboard.
  • Self-hosting is only free in licence terms; you own the database, the upgrades, the backups and the query performance at scale, and a busy site's events table grows quickly enough to need real attention.
  • Because it stores no persistent identifier, it cannot follow a user across devices or sessions in any meaningful way, which makes returning-visitor and lifetime-value questions unanswerable by design rather than by omission.
  • Cloud overage is billed per event beyond the plan allowance, so a traffic spike or a badly instrumented custom event can produce a bill materially above the $20 or $200 headline before anyone notices.
  • It is maintained by a small team, so the pace of new capability is modest and enterprise expectations such as SSO, granular role permissions and formal support agreements are limited compared with commercial analytics vendors.

Pricing, plan by plan

Cucumber

Free
  • CucumberFree
    • Full functionality across Cucumber-JVM, Cucumber-JS, Cucumber-Ruby and other implementations
    • Community support via GitHub
    • Returned to independent community ownership in December 2024 after nine years connected to SmartBear

Umami

Free
  • Self-hostedFree
    • MIT licensed, no licence fee
    • No event limit imposed by the software
    • Unlimited websites and users
  • Cloud HobbyFree
    • Up to 100,000 events per month
    • 3 websites
    • 6 months data retention
  • Cloud Pro$20/month
    • Up to 1,000,000 events per month
    • Unlimited websites
    • Unlimited team members
  • Cloud Business$200/month
    • Up to 10,000,000 events per month
    • All Pro features
    • Higher volume overage rate

Which should you pick?

Choose Cucumber if

  • You need gherkin syntax.
  • You want to start without paying.
  • You work on Linux, macOS, Windows.
  • You also want step definition binding.
Cucumber review

Choose Umami if

  • You need cookie-free tracking.
  • You want to start without paying.
  • You work on Web, Linux.
  • You also want self-hosting.
Umami review

Questions people ask

Is Cucumber or Umami better?
Neither clearly leads. Cucumber starts at Free and Umami at Free, and user ratings are close enough to be indistinguishable. Choose on capability and platform support.
Which is cheaper, Cucumber or Umami?
Cucumber starts at Free and Umami at Free.
Does Cucumber or Umami run on more platforms?
Cucumber runs on Linux, macOS, Windows. Umami runs on Web, Linux.
Can I use Cucumber for free?
Both have a free tier, so you can try either at no cost before committing.
What is Cucumber best used for?
Cucumber is most often used for a team practising behaviour-driven development that wants shared, readable specifications between product owners and engineers, an organisation wanting living documentation that stays synchronised with actual test coverage rather than a separate specification document, a qa team migrating acceptance tests from a proprietary bdd tool to an open source, community-governed alternative, a java, javascript or ruby project needing a mature bdd framework rather than building gherkin-style parsing from scratch. Of those, a team practising behaviour-driven development that wants shared, readable specifications between product owners and engineers and an organisation wanting living documentation that stays synchronised with actual test coverage rather than a separate specification document are not what Umami is typically brought in for.
What can Cucumber do that Umami cannot?
Cucumber covers Gherkin syntax, Step definition binding, Multi-language support, Living documentation. Umami covers Cookie-free tracking, Self-hosting, Small tracking script, Custom events.

Answered from the vendors’ own pages

Cucumber: Is Cucumber owned by SmartBear?

No, not currently. SmartBear acquired Cucumber Ltd in 2019 but the open source project returned to independent community ownership in December 2024.

Umami: Is Umami really free?

Self-hosted, yes; it is MIT licensed with no event cap. You pay for the server and database. Umami Cloud has a free tier at 100,000 events a month and paid tiers from $20.

Cucumber: Is Cucumber free?

Yes, entirely open source with no licence fee across all its language implementations.

Umami: Do I need a cookie consent banner?

For Umami analytics specifically, generally no, because it sets no cookies and stores no personal data. Other tags on your site may still require one.

Cucumber: Does Cucumber replace Selenium or Appium?

No, Cucumber provides the Gherkin specification layer and step definition binding; the actual browser or mobile automation still needs a driver such as Selenium or Appium underneath.

Umami: Can it replace Google Analytics?

For traffic, referrers and event counts, yes. For attribution modelling, audiences and advertising integration, no.

Umami: What does it need to run?

A container and a PostgreSQL or MySQL database. Most people deploy it on infrastructure they already pay for.

Share

Related pages

More on Cucumber

More on Umami

Keep looking

Other head to heads