Hacker Remix

Show HN: DBOS TypeScript – Lightweight Durable Execution Built on Postgres

75 points by KraftyOne 1 day ago | 45 comments

Hi HN - Peter from DBOS here with my co-founder Qian (qianl_cs)

Today we want to share our TypeScript library for lightweight durable execution. We’ve been working on it since last year and recently released v2.0 with a ton of new features and major API overhaul.

https://github.com/dbos-inc/dbos-transact-ts

Durable execution means persisting the execution state of your program while it runs, so if it is ever interrupted or crashes, it automatically resumes from where it left off.

Durable execution is useful for a lot of things:

- Orchestrating long-running or business-critical workflows so they seamlessly recover from any failure.

- Running reliable background jobs with no timeouts.

- Processing incoming events (e.g. from Kafka) exactly once

- Running a fault-tolerant distributed task queue

- Running a reliable cron scheduler

- Operating an AI agent, or anything that connects to an unreliable or non-deterministic API.

What’s unique about DBOS’s take on durable execution (compared to, say, Temporal) is that it’s implemented in a lightweight library that’s totally backed by Postgres. All you have to do to use DBOS is “npm install” it and annotate your program with decorators. The decorators store your program’s execution state in Postgres as it runs and recover it if it crashes. There are no other dependencies you have to manage, no separate workflow server–just your program and Postgres.

One big advantage of this approach is that you can add DBOS to ANY TypeScript application–it’s just a library. For example, you can use DBOS to add reliable background jobs or cron scheduling or queues to your Next.js app with no external dependencies except Postgres.

Also, because it’s all in Postgres, you get all the tooling you’re familiar with: backups, GUIs, CLI tools–it all just works.

Want to try DBOS out? Initialize a starter app with:

    npx @dbos-inc/create -t dbos-node-starter

Then build and start your app with:

    npm install
    npm run build
    npm run start

Also check out the docs: https://docs.dbos.dev/

We'd love to hear what you think! We’ll be in the comments for the rest of the day to answer any questions you may have.

e12e 2 hours ago

Interesting idea. It seems like zodb (https://zodb.org) might enable some similar things for python - by simply being an object database?

Is it possible to mix typescript and python steps?

CMCDragonkai 1 day ago

Could you genericise the requirement in postgresql and provide a storage interface we could plug into? I think I have a use for this in Polykey (https://GitHub.com/MatrixAI/Polykey) but we use rocksdb (transactional key value embedded db).

KraftyOne 1 day ago

That's definitely worth considering! The core algorithms can work with any data store. That said, we're focused on Postgres right now because of its incredible support and popularity.

CMCDragonkai 1 day ago

You could imagine this working well for cloudflare workers - especially with time limits on execution. (Or with even aws compute market)

CMCDragonkai 1 day ago

Also this reminds me of orthogonal persistence https://wiki.c2.com/?TransparentPersistence

Did you do literature research of Smalltalk?

qianli_cs 1 day ago

Hello! I'm a co-founder at DBOS here and I'm happy to answer any questions :)

sarahdellysse 1 day ago

Hi there, I think I might have found a typo in your example class in the github README. In the class's `workflow` method, shouldn't we be `await`-ing those steps?

qianli_cs 1 day ago

Nice catch. Fixing it :)

nahuel0x 1 day ago

Can you change the workflow code for a running workflow that already advanced some steps? What support DBOS have for workflow evolution?

KraftyOne 1 day ago

It's not recommended--the assumed model is that every workflow finishes on the code version it started. This is managed automatically in our hosted version (DBOS Cloud) and there's an API for self-hosting: https://docs.dbos.dev/typescript/tutorials/development/self-...

That said, we know sometimes you have to do surgery on a long-running workflow, and we're looking at adding better tooling for it. It's completely doable because all the state is stored in Postgres tables (https://docs.dbos.dev/explanations/system-tables).

ilove196884 1 day ago

I know this this might sound scripted or can be considered cliche but what is the use case for DBOS.

qianli_cs 1 day ago

The main use case is to build reliable programs. For example, orchestrating long-running workflows, running cron jobs, and orchestrating AI agents with human-in-the-loop.

DBOS makes external asynchronous API calls reliable and crashproof, without needing to rely on an external orchestration service.

peterkelly 1 day ago

How do you persist execution state? Does it hook into the Python interpreter to capture referenced variables/data structures etc, so they are available when the state needs to be restored?

KraftyOne 1 day ago

That work is done by the decorators! They wrap around your functions and store the execution state of your workflows in Postgres, specifically:

- Which workflows are executing

- What their inputs were

- Which steps have completed

- What their outputs were

Here's a reference for the Postgres tables DBOS uses to manage that state: https://docs.dbos.dev/explanations/system-tables

CMCDragonkai 1 day ago

All of this seems it would fit any transactional key value structure.

swyx 1 day ago

> What’s unique about DBOS’s take on durable execution (compared to, say, Temporal) is that it’s implemented in a lightweight library that’s totally backed by Postgres. All you have to do to use DBOS is “npm install” it and annotate your program with decorators. The decorators store your program’s execution state in Postgres as it runs and recover it if it crashes. There are no other dependencies you have to manage, no separate workflow server–just your program and Postgres.

this is good until you the postgres server fills up with load and need to scale up/fan out work to a bunch of workers? how do you handle that?

(disclosure, former temporal employee, but also no hate meant, i'm all for making more good orcehstration choices)

KraftyOne 1 day ago

That's a really good question! Because DBOS is backed by Postgres, it scales as well as Postgres does, so 10K+ steps per second with a large database server. That's good for most workloads. Past that, you can split your workload into multiple services or shard it. Past that, you've probably outscaled any Postgres-based solution (very few services need this scale).

The big advantages of using Postgres are:

1. Simpler architecturally, as there are no external dependencies.

2. You have complete control over your execution state, as it's all on tables on your Postgres server (docs for those tables: https://docs.dbos.dev/explanations/system-tables#system-tabl...)

reissbaker 1 day ago

Unaffiliated with DBOS but I agree that Postgres will scale much further than most startups will ever need! Even Meta still runs MySQL under the hood (albeit with a very thick layer of custom ORM).