Show HN: Okapi – a metrics engine based on open data formats
Show HN (score: 7)Description
Most metrics engines today use proprietary formats to store data and don’t use disaggregated storage and compute. Okapi changes that by leveraging open data formats and integrating with existing data lakes. This makes it possible to use standard OLAP tools like Snowflake, Databricks, DuckDB or even Jupyter / Polars to run analysis workflows (such as anomaly detection) while avoiding vendor lock-in in two ways - you can bring your own workflows and have a swappable compute engine. Disaggregation also reduces Ops burden of maintaining your own storage and the compute engine can be scaled up and down on demand.
Not all data can reside in a data-lake/object store though - this doesn’t work for recent data. To ease realtime queries Okapi first writes all metrics data to an in memory store and reads on recent data are served from this store. Metrics are rolled up as they arrive which helps ease memory pressure. Metrics are held in-memory for a configurable retention period after which it gets shipped out to object storage/datalake (currently only Parquet export is supported). This allows fast reads on recent data while offloading query-processing for older data. On benchmarks queries on in-memory data finish in under a millisecond while having write throughput of ~280k samples per second. On a real deployment, there’d be network delays so YMMV.
Okapi it is still early — feedback, critiques, and contributions welcome. Cheers !
More from Show
Show HN: Mist – a lightweight, self-hosted PaaS
Show HN: Mist – a lightweight, self-hosted PaaS Hi HN,<p>We’re building Mist, a lightweight, self-hosted PaaS for running and managing applications on your own servers.<p>We started Mist because we wanted a middle ground between raw VPS management and heavy, all-in-one platforms. Existing PaaS solutions often felt too complex or resource-intensive for small servers, homelabs, and side projects. We wanted something that keeps the PaaS experience but stays simple and predictable.<p>Our goals with Mist are: - A simple PaaS to deploy and manage apps on your own infrastructure - HTTPS, routing, and app access handled automatically - Low resource usage so it runs well on small VPSes - Self-hosted and transparent, with minimal magic<p>Mist focuses on being an opinionated but lightweight layer on top of your server. It doesn’t try to hide everything behind abstractions, but it does aim to remove the repetitive operational work that comes with managing apps by hand.<p>Mist is still early, and this is where we really need help. We’re actively looking for:<p>- Users who want a simple self-hosted PaaS and can share real-world feedback<p>- Contributors who want to help shape the core features, architecture, and UX<p>Website: <a href="https://trymist.cloud" rel="nofollow">https://trymist.cloud</a><p>Repo: <a href="https://github.com/corecollectives/mist" rel="nofollow">https://github.com/corecollectives/mist</a>
Show HN: Marten – Elegant Go web framework (nothing in the way)
Show HN: Marten – Elegant Go web framework (nothing in the way)
Show HN: I vibecoded an ARM64 operating system that boots on real hardware
Show HN: I vibecoded an ARM64 operating system that boots on real hardware VibeOS is a retro operating system that boots on qemu and Pi Zero 2W. I built this with Claude code in about 4 weeks.
Show HN: Seapie – a Python debugger where breakpoints drop into a REPL
Show HN: Seapie – a Python debugger where breakpoints drop into a REPL
No other tools from this source yet.