Show HN: We made our own inference engine for Apple Silicon

Hacker News (score: 60)

Found: July 15, 2025

ID: 317

Description

Other

Show HN: We made our own inference engine for Apple Silicon We wrote our inference engine on Rust, it is faster than llama cpp in all of the use cases. Your feedback is very welcomed. Written from scratch with idea that you can add support of any kernel and platform.

More from Hacker

Show HN: Sping – An HTTP/TCP latency tool that's easy on the eye

Show HN: Sping – An HTTP/TCP latency tool that's easy on the eye I've frequently found myself using [nvitop](<a href="https://github.com/XuehaiPan/nvitop" rel="nofollow">https://github.com/XuehaiPan/nvitop</a>) to diagnose GPU/CPU contention issues.The two best things about it are:- It's easy to install if I can access pip in the container- It makes a compelling screenshot (which helps me communicate with coworkers.)With those two lessons in mind: Here is Sping!Purpose: Help observe and diagnose latency issues at layer 4+ (TCP/HTTP/HTTPS)Two good things about it:- It's easy to install if you have pip. (Available at [service-ping-sping](<a href="https://pypi.org/project/service-ping-sping/" rel="nofollow">https://pypi.org/project/service-ping-sping/</a>) on PyPi)- It makes a compelling screenshot.Not sure if this is the kind of thing that anyone else would be interested in. But I've enjoyed making it and intend to keep using it.

The Chan-Zuckerbergs stopped funding social causes

No other tools from this source yet.