🛠️ Hacker News Tools

[Other] Show HN: I built a local data lake for AI powered data engineering and analytics I got tired of the overhead required to run even a simple data analysis - cloud setup, ETL pipelines, orchestration, cost monitoring - so I built a fully local data-stack&#x2F;IDE where I can write SQL&#x2F;Py, run it, see results, and iterate quickly and interactively.<p>You get data lake like catalog, zero-ETL, lineage, versioning, and analytics running entirely on your machine. You can import from a database, webpage, CSV, etc. and query in natural language or do your own work in SQL&#x2F;Pyspark. Connect to local models like Gemma or cloud LLMs like Claude for querying and analysis. You don’t have to setup local LLMs, it comes built in.<p>This is completely free. No cloud account required.<p>Downloading the software - <a href="https:&#x2F;&#x2F;getnile.ai&#x2F;downloads" rel="nofollow">https:&#x2F;&#x2F;getnile.ai&#x2F;downloads</a><p>Watch a demo - <a href="https:&#x2F;&#x2F;www.youtube.com&#x2F;watch?v=C6qSFLylryk" rel="nofollow">https:&#x2F;&#x2F;www.youtube.com&#x2F;watch?v=C6qSFLylryk</a><p>Check the code repo - <a href="https:&#x2F;&#x2F;github.com&#x2F;NileData&#x2F;local" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;NileData&#x2F;local</a><p>This is still early and I&#x27;d genuinely love your feedback on what&#x27;s broken, what&#x27;s missing, and if you find this useful for your data and analytics work.

[Other] USB for Software Developers: An introduction to writing userspace USB drivers

[IDE/Editor] Expanding Swift's IDE Support

[Other] Show HN: 500k+ events/sec transformations for ClickHouse ingestion Hi HN! We are Ashish and Armend, founders of GlassFlow.<p>Over the last year, we worked with teams running high-throughput pipelines into self-hosted ClickHouse. Mostly for observability and real-time analytics.<p>A question that came repeatedly was: What happens when throughput grows?<p>Usually, things work fine at 10k events&#x2F;sec, but we started seeing backpressure and errors at &gt;100k.<p>When the throughput per pipeline stops scaling, then adding more CPU&#x2F;memory doesn’t help because often parts of the pipeline are not parallelized or are bottlenecked by state handling.<p>At this point, engineers usually scale by adding more pipeline instances.<p>That works but comes with some trade-offs: - You have to split the workload (e.g., multiple pipelines reading from the same source) - Transformation logic gets duplicated across pipelines - Stateful logic becomes harder to manage and keep consistent - Debugging and changes get more difficult because the data flow is fragmented<p>Another challenge arises when working with high-cardinality keys like user IDs, session IDs, or request IDs, and when you need to handle longer time windows (24h or more). The state grows quickly and many systems rely on in-memory state, which makes it expensive and harder to recover from failures.<p>We wanted to solve this problem and rebuild our approach at GlassFlow.<p>Instead of scaling by adding more pipelines, we scale within a single pipeline by using replicas. Each replica consumes, processes, and writes independently, and the workload is distributed across them.<p>In the benchmarks we’re sharing, this scales to 500k+ events&#x2F;sec while still running stateful transformations and writing into ClickHouse.<p>A few things we think are interesting: - Scaling is close to linear as you add replicas - Works with stateful transformations (not just stateless ingestion) - State is backed by a file-based KV store instead of relying purely on memory - The ClickHouse sink is optimized for batching to avoid small inserts - The product is built with Go<p>Full write-up + benchmarks: <a href="https:&#x2F;&#x2F;www.glassflow.dev&#x2F;blog&#x2F;glassflow-now-scales-to-500k-events-per-sec" rel="nofollow">https:&#x2F;&#x2F;www.glassflow.dev&#x2F;blog&#x2F;glassflow-now-scales-to-500k-...</a><p>Repo: <a href="https:&#x2F;&#x2F;github.com&#x2F;glassflow&#x2F;clickhouse-etl" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;glassflow&#x2F;clickhouse-etl</a><p>Happy to answer questions about the design or trade-offs.

[Other] Show HN: TUI-use: Let AI agents control interactive terminal programs

[Other] Show HN: BAREmail ʕ·ᴥ·ʔ – minimalist Gmail client for bad WiFi I&#x27;ve been frustrated one too many times by terrible airplane wifi and not being able to load Gmail or Superhuman when all I want to do is get a few simple text-only emails out the door.<p>These clients have become pretty bloated with the assumption you&#x27;ve always got great bandwidth.<p>So I vibe coded BAREMAIL. It&#x27;s open source, has no backend, and you can just set it up for yourself. Takes ~3 mins to setup API access via Google Cloud Platform (thanks for making this not super easy Google!)<p>I tried to maintain nice design and some important keyboard shortcuts without getting to overBEARing.

[Other] Show HN: We fingerprinted 178 AI models' writing styles and similarity clusters We have a dataset of 3,095 standardized AI responses across 43 prompts. From each response, we extract a 32-dimension stylometric fingerprint (lexical richness, sentence structure, punctuation habits, formatting patterns, discourse markers).<p>Some findings:<p>- 9 clone clusters (&gt;90% cosine similarity on z-normalized feature vectors) - Mistral Large 2 and Large 3 2512 score 84.8% on a composite metric combining 5 independent signals - Gemini 2.5 Flash Lite writes 78% like Claude 3 Opus. Costs 185x less - Meta has the strongest provider &quot;house style&quot; (37.5x distinctiveness ratio) - &quot;Satirical fake news&quot; is the prompt that causes the most writing convergence across all models - &quot;Count letters&quot; causes the most divergence<p>The composite clone score combines: prompt-controlled head-to-head similarity, per-feature Pearson correlation across challenges, response length correlation, cross-prompt consistency, and aggregate cosine similarity.<p>Tech: stylometric extraction in Node.js, z-score normalization, cosine similarity for aggregate, Pearson correlation for per-feature tracking. Analysis script is ~1400 lines.

[Code Quality] A security scanner as fast as a linter – written in Rust

[API/SDK] Show HN: Skrun – Deploy any agent skill as an API

[Other] MegaTrain: Full Precision Training of 100B+ Parameter LLMs on a Single GPU

[Other] The Git Commands I Run Before Reading Any Code

[Other] We moved Railway's frontend off Next.js. Builds went from 10+ mins to under two

[Other] Xilem – An experimental Rust native UI framework

[Other] Show HN: Mo – checks GitHub PRs against decisions approved in Slack Built this after a recurring frustration at our agency: the team would agree on something in Slack (&quot;only admins can export users&quot;), someone would open a PR two weeks later that quietly broke it, and nobody caught it until QA or after deploy.<p>Mo watches a Slack channel for decisions. When someone tags @mo to approve something, it stores it. When a PR opens, Mo checks the diff against the approved decisions and flags conflicts before merge.<p>It doesn&#x27;t review code quality. It only cares whether the code matches what the team actually agreed to.<p>Would love feedback, especially from anyone who&#x27;s been burned by this exact problem.<p>Try it here: <a href="https:&#x2F;&#x2F;hey-mo.io" rel="nofollow">https:&#x2F;&#x2F;hey-mo.io</a>

[Other] S3 Files <a href="https:&#x2F;&#x2F;aws.amazon.com&#x2F;blogs&#x2F;aws&#x2F;launching-s3-files-making-s3-buckets-accessible-as-file-systems&#x2F;" rel="nofollow">https:&#x2F;&#x2F;aws.amazon.com&#x2F;blogs&#x2F;aws&#x2F;launching-s3-files-making-s...</a>

[Other] Show HN: Gemma 4 Multimodal Fine-Tuner for Apple Silicon About six months ago, I started working on a project to fine-tune Whisper locally on my M2 Ultra Mac Studio with a limited compute budget. I got into it. The problem I had at the time was I had 15,000 hours of audio data in Google Cloud Storage, and there was no way I could fit all the audio onto my local machine, so I built a system to stream data from my GCS to my machine during training.<p>Gemma 3n came out, so I added that. Kinda went nuts, tbh.<p>Then I put it on the shelf.<p>When Gemma 4 came out a few days ago, I dusted it off, cleaned it up, broke out the Gemma part from the Whisper fine-tuning and added support for Gemma 4.<p>I&#x27;m presenting it for you here today to play with, fork and improve upon.<p>One thing I have learned so far: It&#x27;s very easy to OOM when you fine-tune on longer sequences! My local Mac Studio has 64GB RAM, so I run out of memory constantly.<p>Anywho, given how much interest there is in Gemma 4, and frankly, the fact that you can&#x27;t really do audio fine-tuning with MLX, that&#x27;s really the reason this exists (in addition to my personal interest). I would have preferred to use MLX and not have had to make this, but here we are. Welcome to my little side quest.<p>And so I made this. I hope you have as much fun using it as I had fun making it.<p>-Matt

[Other] Building a framework-agnostic Ruby gem (and making sure it doesn't break)

[Other] Tailslayer: Library for reducing tail latency in RAM reads

[Other] Show HN: Marimo pair – Reactive Python notebooks as environments for agents Hi HN! We&#x27;re excited to share marimo pair [1] [2], a toolkit that drops AI agents into a running marimo notebook [3] session. This lets agents use marimo as working memory and a reactive Python runtime, while also making it easy for humans and agents to collaborate on computational research and data work.<p>GitHub repo: <a href="https:&#x2F;&#x2F;github.com&#x2F;marimo-team&#x2F;marimo-pair" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;marimo-team&#x2F;marimo-pair</a><p>Demo: <a href="https:&#x2F;&#x2F;www.youtube.com&#x2F;watch?v=6uaqtchDnoc" rel="nofollow">https:&#x2F;&#x2F;www.youtube.com&#x2F;watch?v=6uaqtchDnoc</a><p>marimo pair is implemented as an agent skill. Connect your agent of choice to a running notebook with:<p>&#x2F;marimo-pair pair with me on my_notebook.py<p>The agent can do anything a human can do with marimo and more. For example, it can obtain feedback by running code in an ephemeral scratchpad (inspect variables, run code against the program state, read outputs). If it wants to persist state, the agent can add cells, delete them, and install packages (marimo records these actions in the associated notebook, which is just a Python file). The agent can even manipulate marimo&#x27;s user interface — for fun, try asking your agent to greet you from within a pair session.<p>The agent effects all actions by running Python code in the marimo kernel. Under the hood, the marimo pair skill explains how to discover and create marimo sessions, and how to control them using a semi-private interface we call code mode.<p>Code mode lets models treat marimo as a REPL that extends their context windows, similar to recursive language models (RLMs). But unlike traditional REPLs, the marimo &quot;REPL&quot; incrementally builds a reproducible Python program, because marimo notebooks are dataflow graphs with well-defined execution semantics. As it uses code mode, the agent is kept on track by marimo&#x27;s guardrails, which include the elimination of hidden state: run a cell and dependent cells are run automatically, delete a cell and its variables are scrubbed from memory.<p>By giving models full control over a stateful reactive programming environment, rather than a collection of ephemeral scripts, marimo pair makes agents active participants in research and data work. In our early experimentation [4], we&#x27;ve found that marimo pair accelerates data exploration, makes it easy to steer agents while testing research hypotheses, and can serve as a backend for RLMs, yielding a notebook as an executable trace of how the model answered a query. We even use marimo pair to find and fix bugs in itself and marimo [5]. In these examples the notebook is not only a computational substrate but also a canvas for collaboration between humans and agents, and an executable, literate artifact comprised of prose, code, and visuals.<p>marimo pair is early and experimental. We would love your thoughts.<p>[1] <a href="https:&#x2F;&#x2F;github.com&#x2F;marimo-team&#x2F;marimo-pair" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;marimo-team&#x2F;marimo-pair</a><p>[2] <a href="https:&#x2F;&#x2F;marimo.io&#x2F;blog&#x2F;marimo-pair" rel="nofollow">https:&#x2F;&#x2F;marimo.io&#x2F;blog&#x2F;marimo-pair</a><p>[3] <a href="https:&#x2F;&#x2F;github.com&#x2F;marimo-team&#x2F;marimo" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;marimo-team&#x2F;marimo</a><p>[4] <a href="https:&#x2F;&#x2F;www.youtube.com&#x2F;watch?v=VKvjPJeNRPk" rel="nofollow">https:&#x2F;&#x2F;www.youtube.com&#x2F;watch?v=VKvjPJeNRPk</a><p>[5] <a href="https:&#x2F;&#x2F;github.com&#x2F;manzt&#x2F;dotfiles&#x2F;blob&#x2F;main&#x2F;.claude&#x2F;skills&#x2F;marimo-dev&#x2F;SKILL.md" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;manzt&#x2F;dotfiles&#x2F;blob&#x2F;main&#x2F;.claude&#x2F;skills&#x2F;m...</a>

[Other] Mysteries of Dropbox: Testing of a Distributed Sync Service (2016) [pdf]