Show HN: Inworld Runtime – A C++ graph-based runtime for production AI apps
Show HN (score: 6)Description
We built it to solve the common problem we and our customers had: engineers spend more time on AI ops and plumbing than on actual feature development. This was often due to the challenge of using Python for I/O-bound, high-concurrency workloads and complexity maintaining pipelines with streams that use always-changing ML models.
Our solution is a high-performance runtime written in C++ with the core idea of defining AI logic as graphs. For instance, a basic voice-to-voice agent consists of STT → LLM → TTS nodes, while the connecting edges stream data and enforce conditions. This graph engine is portable (Linux, Windows, macOS) and can run on-device.
We built a few key features on top of this C++ core:
- Extensions. Runtime architecture decouples graph definition from implementation. If a pre-built component doesn't exist, you can register your own custom node/code and reuse it in any graph without writing any glue code.
- Routers. You can dynamically select models/settings on the per-node basis depending on the traffic as well as configure policies for fallbacks and retries to get the app ready for production.
- The Portal. A web-based control plane UI to deploy graphs, push config changes instantly, run A/B tests on live traffic, and monitor your app with logs, traces, and metrics.
- Unified API. Use our optimized models or route to providers like OpenAI, Anthropic, and Google through a single, consistent interface and one API key.
We have a Node.js SDK out now, with Python, Unity, Unreal, and native C++ coming soon. We plan to open-source the SDKs, starting with Node.js.
The docs are here: https://docs.inworld.ai/docs/runtime/overview
We're eager for feedback from fellow engineers and builders. What do you think?
More from Show
Show HN: Ragnerock, an AI data analysis tool
Show HN: Ragnerock, an AI data analysis tool Hi HN, I’m Matt Mahowald, and together with my cofounder John, we’re launching the public beta of Ragnerock today.<p>As a data scientist, you spend the majority of your time wrangling data. Even though you might have a set of techniques and tricks you like to use, how exactly you treat a particular source of data tends to be fairly bespoke, so you end up writing custom logic each time.<p>Ragnerock was born from the observation that modern LLMs can be used to automate a lot of the grunt work involved in this process, while still allowing for fully customizable pipelines. What’s more, by leveraging techniques like constrained decoding, it’s possible to provide a unified query interface regardless of the data source - bridging raw data sources like text and images with your existing structured data living in your databases.<p>Ragnerock has four main components:<p>- A workflow designer that lets you build LLM-driven data processing and analysis pipelines<p>- A job orchestration layer that runs those workflows<p>- A query interface which lets you inspect the results of those workflows with plain SQL<p>- A notebook system which is 100% API-compatible with Jupyter and runs on your existing kernels, so you can easily pull data into your existing environments and analyses<p>Ragnerock also supports bring-your-own AI (OpenAI, Anthropic, and Google APIs), databases, and blob storage, so you can join with your existing datasets and have all outputs flow to your data lake. We’re particularly excited about our web crawling feature, which allows you to scrape websites and trigger workflows on updates: for example, you might point Ragnerock at your favorite blog and run a workflow to assess posts for topics and sentiment.<p>You can try it out at <a href="https://www.ragnerock.com" rel="nofollow">https://www.ragnerock.com</a> ; no credit card needed and the first 20 hours of compute are free. It’s an early-stage product so we’re especially interested in feedback.<p>Happy to answer any questions - John and I will be around in the comments today.
Show HN: SyncVibe – Code with friends in the terminal, each with your own AI
Show HN: SyncVibe – Code with friends in the terminal, each with your own AI
No other tools from this source yet.