Show HN: Ducktape – a tiny HTTP/2 wrapper around DuckDB's Appender API
Show HN (score: 7)Description
We were adding MotherDuck as a destination and the first version just used DuckDB’s Go driver directly. It worked great on my machine… until we wired it into our Transfer service (https://github.com/artie-labs/transfer).
Because the driver requires CGO, our cross-compiles to amd64 and arm64 started failing, we lost our easy static binaries, and our Docker images had to pull in C toolchains and system libraries just to support one dependency. We tried isolating the CGO bits in a separate module, but it still caused CI failures and forced us to rewrite chunks of our build pipeline. At that point it was clear we didn’t want CGO anywhere near our main service.
So I built ducktape: a tiny standalone microservice that wraps DuckDB’s Appender API behind HTTP/2 streams. Clients stream NDJSON over HTTP/2, and ducktape appends directly into DuckDB on the other side. No CGO in the main codebase, and we keep our cross-platform, pure-Go build story.
The overhead was surprisingly low in benchmarks: ~757 MiB/sec over HTTP/2 vs ~848 MiB/sec in-process — about 90% of native performance but over the network.
ducktape is open source and MIT licensed: https://github.com/artie-labs/ducktape
I’d love feedback, especially if you’ve tackled CGO isolation differently or have ideas to squeeze out more performance!
More from Show
Show HN: Marten – Elegant Go web framework (nothing in the way)
Show HN: Marten – Elegant Go web framework (nothing in the way)
Show HN: I vibecoded an ARM64 operating system that boots on real hardware
Show HN: I vibecoded an ARM64 operating system that boots on real hardware VibeOS is a retro operating system that boots on qemu and Pi Zero 2W. I built this with Claude code in about 4 weeks.
Show HN: Seapie – a Python debugger where breakpoints drop into a REPL
Show HN: Seapie – a Python debugger where breakpoints drop into a REPL
Show HN: Memory Graph – Interactive Python execution and memory visualizer
Show HN: Memory Graph – Interactive Python execution and memory visualizer Hello everybody,<p>I built Memory Graph to help students (and myself) build a correct mental model of Python references, mutability, and copying, and to make debugging data structures less “print-driven”. It’s inspired by Python Tutor, but focuses on clearer graphs and on running locally in many different environments and debuggers. The Memory Graph Web Debugger quickly turns your web browser into a Python debugger where the whole program state is visualized in each step, clearly showing aliasing and the structure of the data, giving insight that is hard to get with just printing.<p>Some interesting examples:<p>- Recursion, decimal to binary conversion: <a href="https://memory-graph.com/#codeurl=https://raw.githubusercontent.com/bterwijn/memory_graph/refs/heads/main/src/binary_convert.py&timestep=1.0&play" rel="nofollow">https://memory-graph.com/#codeurl=https://raw.githubusercont...</a><p>- Binary Tree data structure: <a href="https://memory-graph.com/#codeurl=https://raw.githubusercontent.com/bterwijn/memory_graph/refs/heads/main/src/bin_tree.py&timestep=0.2&play" rel="nofollow">https://memory-graph.com/#codeurl=https://raw.githubusercont...</a><p>- Cocktail Shaker Sort: <a href="https://memory-graph.com/#codeurl=https://raw.githubusercontent.com/bterwijn/memory_graph/refs/heads/main/src/cocktail_sort.py&breakpoints=16,45&continues=1&timestep=0.2&play" rel="nofollow">https://memory-graph.com/#codeurl=https://raw.githubusercont...</a><p>Here you can find some difficult Python Data Model exercises for use in education where Memory Graph helps explain the correct answer:<p><pre><code> https://github.com/bterwijn/memory_graph_videos/blob/main/exercises/exercises.md</code></pre>
No other tools from this source yet.