Hacker Remix

The Shepherd 1.0.0 released

175 points by sharpshadow 5 days ago | 56 comments

davexunit 5 days ago

Shepherd is a wonderful little service manager. The project has been integrating the good features of systemd (socket activation, timers, etc.) while retaining the hackability of using a fully featured programming language for configuration. Been happily using it with Guix for over a decade!

NLnet has funded a really cool project to turn Shepherd into a distributed system that enables a fleet of Shepherds to cooperate. https://nlnet.nl/project/DistributedShepherd/

paroneayea 5 days ago

Also a note since there are a number of comments on here about this being a rival for Systemd: Shepherd precedes Systemd by quite a bit! Shepherd was previously known as "dmd", the "daemon for managing daemons", which was made in 2003! So in that sense, if anything was imitating anything (and not claiming anything was), it would be the reverse :)

skulk 5 days ago

You've been using Guix for over a decade? Is there somewhere I can read more about your experience? I think Guix is a cooler version of nix but I've been put off by the lack of packages and smaller community.

xelxebar 5 days ago

Not OP, but I've been daily driving Guix for about 5 years now quite happily. The biggest paint point with vanilla Guix is probably a lack of non-libre packages, especially the linux-libre kernel which strips out blobs. However, the nonguix [1] channel handily fills that hole, and there is first-class support for running a Nix daemon service [0] to cover anything not in nonguix.

Guix documentation is top notch, and it's really nice having all that available locally as texinfo pages. For writing packages yourself, I've found the Guix source to be eminently more greppable than Nix, which eases the dev onramp.

Nix's daemon is definitely more performant than Guix's, so equivalent operations will usually be quite a bit slower in Guix, but on the whole I'd say that Guix's more cohesive design makes the system more pleasant to work with.

Happy to help if you have any specific questions!

[0] https://guix.gnu.org/manual/en/guix.html#Nix-service

[1] https://gitlab.com/nonguix/nonguix/

skulk 4 days ago

(I know I could just RTFM but since you offered!) Does it do something like NixOS for declaring modules as functions and resolving the configuration as a fixed point of all module function applications (roughly, I don't think I'm describing it well)? This is a beautifully elegant way of defining configuration-with-options using lazy evaluation but it often leads to headaches like infinite recursion. Since Guile is not a lazy language I wonder how it achieves this.

qazxcvbnm 4 days ago

I don’t know how Guile does this, but note that finding a fixed point does not generally require a lazy language; you just can’t do it naively — you can still take a bunch of functions that act on a config and ‘while (changing) keep applying’ — equivalently, the Y combinator can’t be evaluated strictly to get your fixed points, but the Z combinator can.

Pay08 4 days ago

In all honesty, I don't know what that means, but the way Guix does configuration is with the Lisp equivalent of a builder pattern, which ends up looking quite like a key-value store.

(foo-configuration (bar-property baz-value) ...)

worik 5 days ago

> Shepherd is a wonderful little service manager

I believe you

But what practical use is it?

I cannot tell.

doublerabbit 5 days ago

> practical use

service manager.

Where by you can create a service of your own application in userland without needing to be PID 1 to execute the service. With all the bells and whistles of monitoring and trigger.

tmtvl 5 days ago

Eh, I'd say systemd still has far better dmcrypt support. The bad (I used to think it was terrible, but I tried installing Debian today, which lowered the bar even further) support for full-disk encryption is what keeps me off Guix, as a matter of fact.

davexunit 5 days ago

okay

danudey 5 days ago

Taking a quick look at this, the first thing I notice is that the configuration file syntax is obtuse and nigh-illegible unless you already know what things are, and even then it's cumbersome and looks frustrating to write.

e.g. instead of systemd's simple:

    Exec=/path/to/ntpd -n -c /etc/ntpd.conf -u ntpd -g

We have this mess:

    #:start (make-forkexec-constructor
           (list "…/bin/ntpd"
                 "-n" "-c" "/…/…-ntpd.conf" "-u" "ntpd" "-g")
           #:log-file "/var/log/ntpd.log")

They say you don't need to be an expert on guile scheme to build a service file, but it does seem as though you need to know things like what `'(ntpd)` means, which I assumed was some kind of identifier literal, and it seems to be what's called a 'data literal' in scheme, but I had to google to discover that since the actual guile scheme documentation didn't seem to explain that syntax at all.

I get that there are benefits to being able to do more advanced scripting in a service file, but this seems like a classic example of programmers writing things for programmers and not for users (or at least, not for users who don't 'get it').

Going to chalk this up as another 'GNU reimplemented something badly because of NIH syndrome' incident and move on.

uludag 5 days ago

I think this may be just due to unfamiliarity. Like, if that mess you shared was written as follows:

  {
    start: makeForkexecConstructor(["…/bin/ntpd", "-n", "-c", "/…/…-ntpd.conf", "-u", "ntpd", "-g"]),
    logFile: "/var/log/ntpd.log"
  }

this wouldn't bat an eye. Even though I've never used Guile, just familiarity with Elisp makes that example pretty straightforward. Lisp-adjacent people would all pretty quickly grok this I presume.

And I think it's also hard to claim that one is better than the other. I personally have gotten my feet wet with Guix and I would love the time to become more familiar it.

yencabulator 5 days ago

I would definitely bat an eye at "make fork exec constructor", regardless of what syntax sugar you sprinkle on that abomination.

ebilgenius 5 days ago

While I agree it's hard to claim one is outright better than the other and there are a lot of factors that go into it, I do feel it's important to toss my chip in as someone who's not familiar with Lisp, Scheme, or any other functional language that I find it's syntax and layout particularly difficult to parse, let alone get started with compared to say, the INI-inspired systemd conf files. I can detail why but the mere fact that I'd have to learn an entire functional programming language (no matter how "easy/simple" it's claimed to be) to be able to competently edit a service file is a huge immediate turn-off for me.

Pay08 4 days ago

This is very basic Scheme syntax (plus Guile's keyword argument extension), nothing advanced and definitely not all of Scheme.

rmgk 4 days ago

I don’t think this is about lisp syntax.

The systemd variant uses one of the universal DSLs for key-value pairs (key=value) and the universal DSL for calling programs (program name, space separated list of arguments).

The latter is even the same syntax that lisp uses for functions calls – thus I would argue the systemd config file looks more like a lisp program than the Guix version does.

As a person that has seen a reasonable amount of sexpression, this is what I would not bat an eye at:

(start /path/to/ntpd -n -c /etc/ntpd.conf -u ntpd -g)

Pay08 4 days ago

It is also strictly less powerful because of that, often needing to launch bash scripts instead of the programs directly.

rmgk 3 days ago

I understand this as systemds config being less powerful than using Guile?

If so, I fully agree, on the config being less powerful.

Maybe let me add to my original argument, because it does not seem to make it’s point well.

I think that it would be feasible (and worthwhile) to simplify the configuration structure while keeping Guile syntax, and have the complaint be a non issue. As opposed to keeping the semantic structure and just changing the syntax as the comment I replied to proposed.

bmacho 4 days ago

> and the universal DSL for calling programs (program name, space separated list of arguments).

Yeah, don't do that. Both the program name, and the arguments can and do contain spaces. Do instead what every other languages do, that is, use a list of strings to invoke programs.

> The latter is even the same syntax that lisp uses for functions calls

No it's not.

rmgk 3 days ago

The way that this syntax (see bash, zsh, fish, etc) usually work is that these lists are separated by spaces. You can still quote arguments if they contain spaces (or escape the spaces).

Choosing space for a very common thing in your language often makes sense, as you reduce the amount of visual elements. Thus space is often used to apply arguments to functions. Lisp and Haskell are common examples Though, you could argue that in Lisp it only is an application if it is within parenthesis.

bmacho 3 days ago

> The way that this syntax (see bash, zsh, fish, etc) usually work is that these lists are separated by spaces.

And programming languages that have variables, string type, and function calls do the opposite. In C, Python, JavaScript etc, in order to start a program you make a function call with a list of strings.

> Choosing space for a very common thing in your language often makes sense,

Not interpreting space separated things as strings but as keywords (built ins, variable names, literals) makes more sense for bigger programs.

kazinator 3 days ago

Lisp looks like it uses spaces, but that's only when we look at certain kinds of tokens that are terminated by whitespace, like numbers and symbols.

So (ls -l .foo) is a three element list of symbols in Common Lisp or Scheme, sure.

Symbols are not strings though; we don't necessarily want to use symbols this way. For one thing, they get interned. That means they stick around forever, or until explicitly uninterned. The symbols are memorized so that the next time they appear, the same object is returned (almost always implemented as a pointer to the same symbol).

String tokens use mandatory double quotes: ("ls" "-l" "

.foo")

packetlost 5 days ago

Eh, I'm a Schemer (not Guile though) but the above is dramatically less readable and, at a glance, understandable than the equivalent systemd .target file. The problem with Lisps (but especially Schemes) is there's magic everywhere that makes no sense without having a sizeable context of the program you're working in.

Bootvis 5 days ago

IMO in this example there seems too be to little magic.

blueflow 5 days ago

I vaguely learned lisp years ago and i already prefer the guile scheme over systemd - guile correctly takes an argument vector instead of a command string which needs to be split using some obtuse shell quoting rules.

Having the shell taken out of the equation is what Lennart promised but never delivered.

n8henrie 5 days ago

Interesting take. As a hobbyist, I get special characters and escapes wrong in systemd all the time, because they try to look like regular shell but aren't. And systemd-escape (or whatever it's called) just makes an ugly mess that makes it hard for me to read.

I don't know lisp, but the idea of having each argument be part of a list (kind of like launchd's xml format) resonates with me.

teddyh 5 days ago

> the actual guile scheme documentation didn't seem to explain that syntax at all.

  (quote data)
  'data

Quoting is used to obtain a literal symbol (instead of a variable reference), a literal list (instead of a function call), or a literal vector. ' is simply a shorthand for a quote form. For example,

  'x                   ⇒ x
  '(1 2 3)             ⇒ (1 2 3)
  '#(1 (2 3) 4)        ⇒ #(1 (2 3) 4)
  (quote x)            ⇒ x
  (quote (1 2 3))      ⇒ (1 2 3)
  (quote #(1 (2 3) 4)) ⇒ #(1 (2 3) 4)

Note that an application must not attempt to modify literal lists or vectors obtained from a quote form, since they may be in read-only memory.

— <https://www.gnu.org/software/guile/manual/html_node/Expressi...>

mmstr 5 days ago

For those that don't know, GNU Shepherd is an init system (like systemd) written in Guile (a LISP), the unit files are written with Guile too.

kkfx 5 days ago

My main remark is: please do consider DESKTOP more than HPC etc, Guix System have a great potential to surpass NixOS for good but can't until the desktop will be the main driver, because before working on a system we enjoy it personally, and personally means desktops and homeservers...