Hacker Remix

OrbStack: The fast, light, and easy way to run Docker containers and Linux

307 points by rpgbr 8 months ago | 156 comments

jchw 8 months ago

I don't generally prefer to work on macOS, but if I wind up using macOS to do work, I often find myself working a lot on things in virtual machines and containers.

Using Docker Desktop to compile Envoy using the standard Docker build process took somewhere in the ball park of 3 to 4 hours depending on my luck. OrbStack, on the other hand, brought it down to a bit under an hour, much closer to inline with a fresh compilation natively. Needless to say, the kinds of performance benefits I was seeing with OrbStack were game changers, and absolutely justify the cost.

Even if Docker Desktop improves to match the performance, OrbStack brings basically the whole WSL2 + Docker experience to macOS, while Docker just brings the usual Docker experience. If you get the value of WSL2 on Windows, you'll probably understand the value of OrbStack on macOS.

Sure, macOS is a UNIX environment, so a lot of the same software as Linux does run natively. However, a lot of Linux technologies don't really map to Darwin, so if you're working on Linux stuff on your macOS machine, there are plenty of use cases for virtual machines (case in point, Docker itself) not to mention simply being able to test software and build processes on Linux. The tight integration that OrbStack gives you is far better than just using Parallels or VMware. I have licenses for both at varying versions, but they're largely collecting dust on macOS, as now I basically only ever use traditional virtual machine products on macOS for the purpose of running Windows VMs.

I'm sure some people don't have any use for this: their Docker performance is fine, they don't need Linux for anything else, etc. However, for me, it's one of those things that makes macOS much more usable for development work.

magnio 8 months ago

Funny how WSL2 makes Windows much more usable than macOS for development. None of the free options (colima, multipass, etc) I've tried on macOS are as smooth, though OrbStack might be it.

I have also moved towards using devcontainers for my projects whenever I can, so that I can spin up my environment on whatever machine I have, or connect to a remote one if the machine doesn't allow it.

majormajor 8 months ago

I've never found working on WSL2 to be quite as smooth as working on Ubuntu or Fedora directly. I don't really understand why I'd keep Windows in the loop there if I was on non-Mac hardward.

And I've also found WSL2 less smooth than just working on Mac natively w/o containers. Containers are a necessary evil for testing certain types of things locally, but even the free tools for working with them on Mac seem fine, though Orbstack's gui is very nice.

(Is there a similar GUI for Linux container management? I've just been running shell commands for years now...)

Instead of moving more towards containers I've just been moving towards simpler, easier-to-set-up-on-Linux-or-Mac toolchains. But I don't have Windows as a target anyway, so that removes one huge need for containers.

justin_oaks 8 months ago

I've used Portainer, which works ok. It's web-based and is easy enough to run as a container itself.

My preferred UI for managing containers is Lazydocker. It's a terminal UI, so I can run it on servers too.

For the most part I just use the command line on Linux, but when I need to go through a large list of containers, images, or volumes to clean up, lazydocker is much better than the command line.

jchw 8 months ago

> None of the free options (colima, multipass, etc) I've tried on macOS are as smooth, though OrbStack might be it.

Yes, I am generally not terribly impressed by colima. Of course, it's great to have as an option, but in practice I ran into issues trying to use it in various places. One issue that I am sure isn't a huge deal to most users is that as far as I could tell, colima did not support IPv6.

I didn't try multipass, but I did try Podman Desktop. It had its niceities but largely was behind even Docker Desktop.

If you really miss WSL2 on macOS, you might genuinely find OrbStack enticing. Then again, it's not free, and obviously, I don't want to give anyone false hope. For "home" use, I just run desktop Linux, using native containers and libvirt for everything. If I had to pay for a decent development experience on my personal machines, I would definitely struggle to justify a subscription charge even if it was good. On the flip side, it's easy to budget OrbStack into the equation for professional use. For your employer it's virtually a no-brainer.

gigatexal 8 months ago

Lima ssh and you have WSL more or less. What are people missing?

jchw 8 months ago

For me, the primary draw of OrbStack is that it is very fast, which matters for me, as I wind up doing a lot of compute-bound things in containers. The fact that it provides a great dev experience similar to WSL2 is just a bonus.

gigatexal 8 months ago

what's the magic sauce that makes it so fast? custom vm engine? using apple's HVF? firecracker vm from AWS?

jchw 8 months ago

I think the main bottleneck in most macOS virtualization solutions winds up being I/O related. Docker Desktop and OrbStack both have custom solutions for bidirectional filesystem bridging and network integration, but for me OrbStack is much faster. OrbStack also can choose between using Apple Rosetta and qemu usermode for running Intel software on Apple Silicon.

talldayo 8 months ago

Imagine paying a subscription service to use something slower than QEMU. Yikes...

jchw 8 months ago

OK, I'll try to imagine that.

pjmlp 8 months ago

Only because it is a Linux VM, and people insist on using Linux specific stuff instead of UNIX, to the point younger generations have no clue about the difference.

Even the BSDs and Solaris/Illumos have add to add Linux translation layers.

Sad state where POSIX hardly matters for portable UNIX code.

talldayo 8 months ago

> Sad state where POSIX hardly matters for portable UNIX code.

Given the current state of POSIX applications, I would actually argue that the BSD/Linux hegemony we enjoy is the best possible outcome. The only people that are mad are the people paying for UNIX and expecting to get something better for it. Those people should have learned their lesson in the 90s, I have no empathy for POSIX apologists in 2024.

The only "sad state" is one where everyday people don't have access to free software. Mac users have always paid a time premium and a performance premium for access to normal development features, this ignorance of MacOS is a pattern that persists since the 90s. Of course nobody is bending over backwards to test portability with a proprietary OS.

pjmlp 8 months ago

In what concerns headless software probably, as they hardly managed anywhere else.

unilynx 8 months ago

To mirror the sibling comment, where's the POSIX container/zone/vm whatever specification? If the BSDs and Linux can agree on a meaningful subset, macOS might actually follow

pjmlp 8 months ago

There isn't any in POSIX, then again, it isn't as if we now need containers for every executable for any magical reason.

Also, just like in the good old days, it isn't hard to have something dealing with HP-UX Vaults, Aix logical partitions, Solaris/Ilumnos Zones, BSD jails, macOS Virtualiztion Framework,....

saagarjha 8 months ago

Just listing technologies that sound kind of similar isn’t enough to actually answer the problems people want solved. The “good old days” were basically just people crying about being unable to have any of the features we have now because they don’t match up or differ in subtly different ways.

pjmlp 8 months ago

Best way to solve problems is not to have them in first place, like getting a Linux laptop for doing Linux work.

jchw 8 months ago

It's harder and harder to use Linux at work outside of bigger tech companies these days. Security standards like SOC2 seem fairly difficult to satisfy for Linux workstations without serious compromises. This is a damn shame because there are approaches to secure Linux workstations that seem pretty powerful but security standards now are prescriptive about what you must do to secure your systems, and for Linux that's going to mean paying for some subscription software that most likely only supports a couple of distros, and if you're lucky, they might support kernels from the _current_ decade.

I used Linux workstations for most of my entire career, at nearly every job. Seems like around 2018 something changed and now I'm going to have to fight to get a desktop that I feel vaguely productive under for every single job I get going forward.

FpUser 8 months ago

Words of wisdom. I do not really have any dev related problems with WSL2 either. Normally I develop and debug on Windows and deploy to Linux as my code compiles and works natively on both. It is mostly C++ backends lately so I suspect I am in tiny minority.

saagarjha 8 months ago

I like my laptop though.

pjmlp 8 months ago

Then use it as Apple decides it is in our best interest to do so, :)

unilynx 8 months ago

I was responding to 'people insist on using Linux specific stuff instead of UNIX'. As far as I can tell there is no way to do containers without doing highly platform specific stuff. It would be very useful if the platforms worked towards a common 'more than chroot' thing.

As far as not really needing it, it's not like computers themselves are anywhere near the bottom of Maslow's pyramid, but that doesn't make them any less useful

AYBABTME 8 months ago

Can you make containers in Darwin?

shepherdjerred 8 months ago

https://darwin-containers.github.io/

AYBABTME 8 months ago

Seems like it's not quite what would qualify as a container for many, but a nice effort.

> rund doesn’t offer the usual level of container isolation that is achievable on other OSes due to limited Darwin kernel API.

https://github.com/darwin-containers/rund?tab=readme-ov-file...

pjmlp 8 months ago

Yes, the macOS way, with Virtualization Framework.

nyrikki 8 months ago

The insane stability of the Linux ABI is partially what makes containers useful.

The fact that containers can reliably depend on the ABI contract, thus placing almost any clib they wish they want inside the container is fairly unique.

That extreme stability of that contract is awesome for namespace decoupling. Unfortunately Apple and Microsoft do not have such stable interfaces.

Remember containers are just namespaces.

pjmlp 8 months ago

Only in the context of Linux containers, not in general, starting with HP-UX Vaults on UNIX land.

zamalek 8 months ago

Virtualization is not containerization. Linux has namespaces, BSD has jails, and even Windows has Windows containers (thought doubt anyone actually uses them). If that's the MacOS way, then the MacOS way must be incompetence.

pxc 8 months ago

Besides the way Apple puts a hard limit on the number of those you can spin up, don't they also virtualize hardware and run their own kernels? That's just not the kind of virtualization that containers are.

AYBABTME 8 months ago

The virtualization layer breaks many "container" expectations, I wouldn't call this containers without big caveats. Same as firecracker VMs may give some of the ergonomics of containers but come with a lot of limitations.

madeofpalk 8 months ago

> Funny how WSL2 makes Windows much more usable than macOS for development

As long as you use VS Code. Using another editor through the network share isnt great and runs into all sorts of other compatibility issues otherwise. I've also ran into a bunch of networking quirks with WSL2 + Docker that were frustrating to sort out.

WSL2 makes *nix development on Windows great, but I would still much prefer to just be in a native environment.

haberman 8 months ago

I have been happily using OrbStack for a while now, and I've had nothing but good experiences. The UI is polished and responsive, the containers have great performance and nice integration with the host, and overall the product seems to be constantly pushing itself to be even better.

I admit my greatest confusion about this software is how a product that appears to be a one-man show so quickly became more compelling than the well-funded incumbent (Docker Desktop). This is even more impressive considering that the developer appears to be a college student.

Hats off, this is amazing work.

kdrag0n 8 months ago

Love to hear that. We're actually a small team at OrbStack now!

saagarjha 8 months ago

I’ll let the actual developer respond but OrbStack has several people working on it now.

marvin-hansen 8 months ago

I switched to Orbstack about 2 weeks ago after having read about it here on HN.

I develop a cloud native system entirely writen in Rust. All my own containers are build without Docker thanks to rules oci in Bazel. However, for integration testing, I'm using internal tools that fire up, say a database container and run the tests all from within Bazel to leverage test caching and parallelization.

For a while, i was struggling to get around Dockers slow startup time on Mac. My CI server uses Firecracker VM's to isolate OCI containers so it's really only a docker on Mac issue.

My main take away:

- I am so close to delete Docker permanently. There is no comparison, not even close. All integration tests run so much faster.

- Especially parallel container starts a noticable faster.

- I've developed custom docker utils for testing and, believe me, the official Docker API is a humongous pile of garbage that I ended up re-implementing everything by wrapping the Docker command line. To nobody's surprise, even the custom docker utils work way faster and more reliable with OrbStack.

- Zero issues. I am still a little bit puzzled that OrbStack basically runs bug-free no matter what I throw at it. Take it as a compliment.

What I would like to see:

- A Ressource monitor or at least some graph that plots CPU and memory usage. In some rare cases the application in the container runs close to the limit probably because a query takes too long, a process got stuck or whatever. Stuff just happenens. Point is, having an eye on ressource usage helps to spot those corner cases early on.

For me, OrbStack is a clear win and a clear keeper. Well done Orb team and I wish you guys all the success in the world.

oarmstrong 8 months ago

> My CI server uses Firecracker VM's to isolate OCI containers

Is this something you built yourself? I've been looking for a CI tool that uses Firecracker but never found anything, I started building something myself but it never really got finished. Would love to drop that project and use something off the shelf.

aayushshah15 8 months ago

I'm obviously biased here but this is what we do at blacksmith dot sh. We run you GitHub Actions on consumer grade desktop CPUs with high single core performance, all inside ephemeral Firecracker VMs. Give us a shot!

marvin-hansen 8 months ago

BuildBuddy. Google it.

It's totally next level. My build is 70 crates, hundreds of unit tests, integration tests, multi platform docker images for two platforms, and everything is done in under 2 minutes, if it's slow(!). If I hit only an incremental change, build is completed within 30 seconds.

The future is now!

rfoo 8 months ago

I'm in a similar position but I need to make sure I run distro kernel (because that's part of integration) instead of whatever OrbStack shipped.

In the end I just run a Linux VM and run everything inside. Zero issues by definition.

I'd actually love to use OrbStack Machines cause it feels much nicer than UTM, but, well, I can't run OrbStack's patched Linux kernel :(

princevegeta89 8 months ago

I've been using Colima which has been great, and much better than Docker Desktop which sucked ass for me.

With Colima, file mounting and sharing caused reliability and permission issues for me though I've applied some workarounds with success. To avoid this mess, I'd much rather move to a VM though. I used VMWare Fusion and UTM but I still had the struggles with file sharing between host and the guest.

So I took a lot of steps back and I'm currently running a Lima VM with headless Ubuntu and things are great so far. For Vscode we got the remote SSH plugin and then there is the Jetbrains Gateway as well.

I'm sharing my experiences for people in similar shoes to try these out, if that helps!

totetsu 8 months ago

I did the same thing. Docker Desktop for Macos kept going into resource saving mode and then not responding to anything after some time, so I tried Orbstack after seeing it here.

KingMob 8 months ago

OrbStack is great in a lot of ways, and I universally prefer it over Docker for Mac.

That being said, it wasn't always been smooth sailing. Under the hood, OrbStack uses an 8TB sparse disk image, which doesn't play nice with most backup software.

https://github.com/orbstack/orbstack/issues/29

It caused me problems with Backblaze, but the Github issues for this show that it also breaks all sorts of backup software, including tarsnap, Druva inSync, Carbon Cloner, iDrive, Carbonite, and even Time Machine itself when formatted with HFS+, apparently.

The official position for a year was "won't fix", because it's an Apple technology, and backup software should support that. While technically correct, realistically, sparse image backup support was not very widespread at the time. (I have no idea about now, since I gave up trying to back up my Orbstack image with my whole disk backup.)

I like Orbstack, but I wish the devs had moved to exclude the disk image from backups immediately, instead of arguing with people about it for a year first.

All that being said, I do still like OrbStack a lot, and I hope to never see a repeat of this problem and how it was handled.

nwienert 8 months ago

The first reply on the issue you linked seems incredibly professional and well handled, and even recommends excluding the file from backups, I can't see a single issue there.

KingMob 8 months ago

Being polite is not quite the same thing as being handled professionally, and definitely not the same thing as handling it correctly.

Telling people to exclude the file from backup came too late for many. E.g., Time Machine users with older disks formatted with HFS+ would find their drives crashed/corrupted/wiped, and lost all their backups. Only afterwards would they start googling to see what happened. (Even now, the relevant FAQ still says "Time Machine supports them, so your backups will not be affected" which is not always correct.)

From the time the issue was opened, to the time they said they admitted they were wrong and excluded the Orbstack image from backups by default, was 13 months. Even if other solutions were on the table, the professional thing to do would have been to exclude the images ASAP, so customers weren't at risk of data loss, and then work on alternatives afterwards.

ignoramous 8 months ago

> I like Orbstack, but I wish the devs...

devs? afaik, it is just one teenager, Danny Lin (he might be 20 by now, though).

kdrag0n 8 months ago

A small team now :)

(not back then though)