Scalyr’s cloud native architecture is optimized to deliver the best ratio of scale / performance / cost. Any system can optimize one attribute, two is more difficult, all three is Scalyr. Keep your event data as long as you like, spike data as much as you need, and slam queries as fast as you can. Scalyr can handle it.
Keep data as long as you like, spike data as much as you need, and slam queries as fast as you can. Scalyr can handle it, and more.
Gestalt Tightly integrated, finely tuned, and elegant in it’s powerful simplicity, Scalyr's gestalt is a specialized event data cloud for fast and powerful analytics at scale. Many individual aspects of Scalyr’s architecture can be found elsewhere, but the combination yields optimizations for messy, high-dimensional, high-cardinality, and spiky event data and its use cases.
Clocked at over 200TB/day per customer and able to scale to petabytes, the ingestion pipeline makes data available in less than a second.
- Scalyr accepts structured, semi-structured and unstructured data from any digital system or service.
- Parsers extract structured fields from event logs in real time.
- Scalyr supports any event data, including cloud infrastructure, containerized applications, traditional servers, IoT endpoints, and more.
- Stream data from a broad range of shippers, queues, agents, distributed stream processing, and APIs.
STORAGE AND ANALYTICS
Tightly integrated, finely tuned, and elegant in it’s powerful simplicity, the gestalt is a specialized event data cloud for fast and powerful analytics at scale. Many individual aspects of Scalyr’s architecture can be found elsewhere, but the combination yields optimizations for messy, high-dimensional, high-cardinality, and spiky event data and its use cases. Scalyr handles all the maintenance, and delivers continual improvements in scale, features, and performance. It’s our mission and our passion.
Incoming data is parsed into a sleek columnar database. There is no need to create, store or update indexes because there are no indexes. Queries execute by scanning only the relevant columns, yielding rapid results whether the query matches ten records or ten billion. Free-text search is supported within a column or across the entire raw log.
Data is encrypted and retained in cloud object storage, providing durability, low cost, and the massive bandwidth needed for high-performance queries. Recently-used data is cached using a combination of RAM and SSD, while older data is fetched on the fly by leveraging the full network bandwidth of every computing node simultaneously. The result? Data is never “cold” and can be retained indefinitely.
Perform streaming analytics and continuously maintain results on tens of thousands of simple metrics and complex summaries to power real time dashboards and alerts. Unlike traditional time series, the Summary Service supports complex queries, unlimited data dimensions, and populates queries retroactively using the detailed event data in the column store. The effect is to allow new metrics to be created for free and on the fly, with full historical context.
Scalyr’s query engine supports ad-hoc queries, complex analysis, joins, pipe expressions, and more. Combined with the efficient columnar storage, intelligent caching, massive object-store bandwidth, and our secret sauce, the query engine completes 96% of queries in less than one second.
TWO WAYS TO CONSUME EVENT DATA
There are two ways to consume the event data analytics: through Scalyr’s UI or using APIs to power your application/service.
- Full-featured log analytics experience for incident management, root cause analysis, dashboards, alerting, and observability use cases.
- Use a mouse to save searches; refine searches via facets, specific/change time ranges, pivot from logs to metric views and back, compare search results over previous time intervals, and break down a search by attribute.
- Includes a powerful query language to do complex analysis, joins, pipe expressions, and more.
Your Service’s UI
- Scalyr’s Event Data Cloud can power custom applications, services, or solutions that require fast and affordable event data analytics.
- Standard RESTful APIs are available, and Scalyr will work with you to support your application or service’s architecture.
- Systems that use Elasticsearch, or other indexing solutions, and struggle to deliver analytics on high-cardinality, high-dimensionality event data will benefit from using the Event Data Cloud. The event data cloud compares well against open source on cost, scale, and performance.
Here are some of the attributes that set Scalyr apart.
Separating storage from compute, Scalyr combines dynamic scaling with the massive bandwidth of cloud object storage to maximize performance while minimizing costs. Capacity can be added or removed on demand, and there is no need to rebalance nodes, manage storage, or allocate compute resources.
Scalyr’s query engine uses horizontal scheduling, devoting the entire cluster – every CPU core on every compute node – to one query at a time. This simple raw horsepower – the same approach used by Google web search – provides unprecedented performance, as well as greatly increasing resource utilization and thus cost efficiency. Sophisticated prioritization schedules interactive queries above batch jobs and interleaves large historical queries with realtime analysis.
Most architectures are designed to operate in multiple small clusters, one per organization, team, or use case. Even if a shared cluster is used, most systems process queries in parallel. The result is that only a small amount of hardware is available for each query and performance suffers.
Economies of scale
Horizontal scheduling with massive multi-tenancy creates a network effect – as more customers ingest more data into Scalyr, it becomes both faster and more affordable for everyone. Scalyr believes that scale should be a benefit that drives speed and value, not a problem that degrades performance. That’s why it’s in our name.
Thrives on chaos
Managing event data is messy. There are conflicting schemas, or no schema at all, thousands of dimensions, and spikey data volumes and usage patterns. There are real time queries, historical queries, dashboards, ad-hoc exploration, and thousands of alerting rules, all competing for the same resources. Scalyr’s service manages event data and all the operational tasks associated with them for customers all over the world at scale, and with great performance and pricing.
The Scalyr Ecosystem
Scalyr integrates with the most-used engineering tools, our pluggable architecture allows us to add yours if something you need is missing.
If you want to join Scalyr’s ecosystem or explore a partnershipContact Us