Log Management · head to head
ELK Stack vs Fluentd
The short version
- Each has a real cost: ELK Stack the free Basic self managed subscription excludes machine learning, alerting, single sign on via SAML or OpenID Connect, LDAP and Active Directory authentication, field and document level security, searchable snapshots and cross cluster replication; Fluentd fluentd needs more than 60 MB of memory at runtime, against roughly 450 KB for Fluent Bit
- They diverge on capability: ELK Stack covers Full-text search, Fluentd covers Log collection.
Where they differ
Only the attributes on which ELK Stack and Fluentd actually diverge.
Identical on both: starting price (Free), pricing model (open-source), free tier (Yes), platforms (Web, Api), user rating (Not yet rated), category (Log Management), founded (2011).
What each one covers
Drawn from each product's published feature list. An absence here means we hold no record of it - not that the product lacks it.
Only in ELK Stack
- Full-text search
- Data visualization
- Log aggregation
- Time-series analytics
Only in Fluentd
- Log collection
- Data parsing
- Filtering and buffering
- Event routing
Both cover
- API
- Webhooks
- REST
- Web support
- Api support
What people use each for
The jobs each tool is most often brought in to do.
ELK Stack
- Centralised log search and analytics over Elasticsearchnot Fluentd
- Dashboards and visualisation of machine data in Kibananot Fluentd
- Self managing a search and observability cluster on your own hardwarenot Fluentd
Fluentd
- Unified log collection and routing from many sources to many destinationsnot ELK Stack
- Parsing and transforming log records before forwardingnot ELK Stack
- Aggregating logs from containers into Elasticsearch, S3 or a SIEMnot ELK Stack
Where each one falls short
Documented limitations, not opinions. Every one is a constraint you would hit in normal use.
ELK Stack
- The free Basic self managed subscription excludes machine learning, alerting, single sign on via SAML or OpenID Connect, LDAP and Active Directory authentication, field and document level security, searchable snapshots and cross cluster replication
- Cross cluster operations require the same subscription tier on every cluster involved
- The 99.95 percent monthly uptime SLA applies only to the Platinum and Enterprise subscription tiers
- Three deployment models are priced on three different bases: resource based for Cloud Hosted, usage based for Serverless and licence based on nodes and RAM for self managed
- Elastic Cloud Serverless does not yet support traffic filtering, cross project search or bring your own key
- The pricing page publishes no rate for any tier
Fluentd
- Fluentd needs more than 60 MB of memory at runtime, against roughly 450 KB for Fluent Bit
- Fluentd is built as a Ruby gem and depends on other gems, so a Ruby runtime is required
- Its capability comes from over 1,000 external plugins rather than built in functionality, so each added input or output is a separate dependency
- The Fluent Bit documentation states that cloud providers have switched from Fluentd to Fluent Bit for performance and compatibility and calls Fluent Bit the next generation solution
Pricing, plan by plan
ELK Stack
Free- FreeFree
- Full-text search
- Data visualization
- Log aggregation
Fluentd
Free- FreeFree
- Log collection
- Data parsing
- Filtering and buffering
Which should you pick?
Choose ELK Stack if
- You need full-text search.
- You want to start without paying.
- You work on Web, Api.
- You also want data visualization.
Choose Fluentd if
- You need log collection.
- You want to start without paying.
- You work on Web, Api.
- You also want data parsing.
Questions people ask
- Is ELK Stack or Fluentd better?
- Neither clearly leads. ELK Stack starts at Free and Fluentd at Free, and user ratings are close enough to be indistinguishable. Choose on capability and platform support.
- Which is cheaper, ELK Stack or Fluentd?
- ELK Stack starts at Free and Fluentd at Free.
- Does ELK Stack or Fluentd run on more platforms?
- Both run on Web, Api, so platform support will not decide this one for you.
- Can I use ELK Stack for free?
- Both have a free tier, so you can try either at no cost before committing.
- What is ELK Stack best used for?
- ELK Stack is most often used for centralised log search and analytics over elasticsearch, dashboards and visualisation of machine data in kibana, self managing a search and observability cluster on your own hardware. Of those, centralised log search and analytics over elasticsearch and dashboards and visualisation of machine data in kibana are not what Fluentd is typically brought in for.
- What can ELK Stack do that Fluentd cannot?
- ELK Stack covers Full-text search, Data visualization, Log aggregation, Time-series analytics. Fluentd covers Log collection, Data parsing, Filtering and buffering, Event routing. Both handle API, Webhooks, REST, Web support.