Hacker Remix

Ask HN: Easiest and hardest concurrency models to use correctly?

10 points by pkkm 2 weeks ago | 3 comments

There's a lot of options for concurrency. In Python alone, you can use threads with shared memory, threads with queues, processes with queues, concurrent.futures, asyncio, trio, or AnyIO. Java now has a preview of structured concurrency and virtual threads in addition to regular threads. There's also the CSP model of Go, as well as the actor model and supervision trees of Erlang/OTP. Software transactional memory seems to be popular in the purely functional world but rare outside of it.

I'm curious about your experiences with these concurrency models - which are easy to get right, and which are endless sources of bugs? I'm particularly interested in use cases where there are lots of long-running connections and correctness matters more than performance: chat servers, MMORPG servers, IoT central systems, etc.

atomicnature 1 week ago

Go's CSP model works well, if you first take the time to study it up a bit. The only drawback with the docs is they don't focus on correctness and potential concurrency bugs sufficiently. In the very least the stdlib could mark whether particular functions are goroutine safe or not. If you stay guarded for such things, you should be able to get a lot done with very few mistakes.

I have limited experience with Rust, but the ownership model and the borrow checker help with avoiding concurrency bugs as well. And personally for me, Rust slowed down the speed at which I could proceed solving the problem at hand. If you have time on your hands or you're very fluent with it, rust may give better results.

chubot 1 week ago

If you're using Python, I would consider pure asyncio + worker processes ...

asyncio gives you the "straight-line" style of code like Go, and good timeout support, which is necessary for robustness

asyncio.Queue gives you backpressure, just like Go

---

That's for using one core, and then if you need to use all cores, use separate processes. (Threads won't utilize all your cores)

And maybe hand-roll the worker processes the the async child process - I think anything that automagically uses the "pickle" module is bad ...

This way you're using 1 or 2 concurrency models, not 3!

---

I think Go is probably more foolproof because you don't have to make as many of these choices. (It multiplexes goroutines across OS threads, etc.) But there are still reasons to use Python, e.g. if you application revolves around C bindings to some important service, or you just like using Python

And Python's type system is actually more expressive than Go now ...

ahoka 1 week ago

I think the worst is “reactive”.