Hacker Remix

Parallel ./configure

143 points by brooke2k 15 hours ago | 116 comments

iforgotpassword 7 hours ago

The other issue is that people seem to just copy configure/autotools scripts over from older or other projects because either they are lazy or don't understand them enough to do it themselves. The result is that even with relatively modern code bases that only target something like x86, arm and maybe mips and only gcc/clang, you still get checks for the size of an int, or which header is needed for printf, or whether long long exists.... And then the entire code base never checks the generated macros in a single place, uses int64_t and never checks for stint.h in the configure script...

IshKebab 5 hours ago

I don't think it's fair to say "because they are lazy or don't understand". Who would want to understand that mess? It isn't a virtue.

A fairer criticism would be that they have no sense to use a more sane build system. CMake is a mess but even that is faaaaar saner than autotools, and probably more popular at this point.

xiaoyu2006 2 hours ago

Autotools use M4 to meta-program a bash script that meta-programs a bunch of C(++) sources and generates C(++) sources that utilizes meta-programming for different configurations; after which the meta-programmed script, again, meta-programs monolithic makefiles.

This is peak engineering.

krior 1 hour ago

Sounds like a headache. Is there a nice Python lib to generate all this M4-mumbo-jumbo?

knorker 16 minutes ago

autotools is the worst, except for all the others.

I'd like to think of myself as reasonable, so I'll just say that reasonable people may disagree with your assertion that cmake is in any way at all better than autotools.

rollcat 4 hours ago

This.

Simple projects: just use plain C. This is dwm, the window manager that spawned a thousand forks. No ./configure in sight: <https://git.suckless.org/dwm/files.html>

If you run into platform-specific stuff, just write a ./configure in simple and plain shell: <https://git.suckless.org/utmp/file/configure.html>. Even if you keep adding more stuff, it shouldn't take more than 100ms.

If you're doing something really complex (like say, writing a compiler), take the approach from Plan 9 / Go. Make a conditionally included header file that takes care of platform differences for you. Check the $GOARCH/u.h files here:

<https://go.googlesource.com/go/+/refs/heads/release-branch.g...>

(There are also some simple OS-specific checks: <https://go.googlesource.com/go/+/refs/heads/release-branch.g...>)

This is the reference Go compiler; it can target any platform, from any host (modulo CGO); later versions are also self-hosting and reproducible.

epcoa 5 hours ago

> either they are lazy or don't understand them enough to do it themselves.

Meh, I used to keep printed copies of autotools manuals. I sympathize with all of these people and acknowledge they are likely the sane ones.

rbanffy 6 hours ago

It’s always wise to be specific about the sizes you want for your variables. You don’t want your ancient 64-bit code to act differently on your grandkids 128-bit laptops. Unless, of course, you want to let the compiler decide whether to leverage higher precision types that become available after you retire.

codys 10 hours ago

I did something like the system described in this article a few years back. [1]

Instead of splitting the "configure" and "make" steps though, I chose to instead fold much of the "configure" step into the "make".

To clarify, this article describes a system where `./configure` runs a bunch of compilations in parallel, then `make` does stuff depending on those compilations.

If one is willing to restrict what the configure can detect/do to writing to header files (rather than affecting variables examined/used in a Makefile), then instead one can have `./configure` generate a `Makefile` (or in my case, a ninja file), and then have the "run the compiler to see what defines to set" and "run compiler to build the executable" can be run in a single `make` or `ninja` invocation.

The simple way here results in _almost_ the same behavior: all the "configure"-like stuff running and then all the "build" stuff running. But if one is a bit more careful/clever and doesn't depend on the entire "config.h" for every "<real source>.c" compilation, then one can start to interleave the work perceived as "configuration" with that seen as "build". (I did not get that fancy)

[1]: https://github.com/codyps/cninja/tree/master/config_h

tavianator 9 hours ago

Nice! I used to do something similar, don't remember exactly why I had to switch but the two step process did become necessary at some point.

Just from a quick peek at that repo, nowadays you can write

#if __has_attribute(cold)

and avoid the configure test entirely. Probably wasn't a thing 10 years ago though :)

o11c 9 hours ago

The problem is that the various `__has_foo` aren't actually reliable in practice - they don't tell you if the attribute, builtin, include, etc. actually works the way it's supposed to without bugs, or if it includes a particular feature (accepts a new optional argument, or allows new values for an existing argument, etc.).

aaronmdjones 3 hours ago

    #if __has_attribute(cold)

You should use double underscores on attribute names to avoid conflicts with macros (user-defined macros beginning with double underscores are forbidden, as identifiers beginning with double underscores are reserved).

    #if __has_attribute(__cold__)
    #  warning "This works too"
    #endif

    static void __attribute__((__cold__))
    foo(void)
    {
        // This works too
    }

codys 9 hours ago

yep. C's really come a long way with the special operators for checking if attributes exist, if builtins exist, if headers exist, etc.

Covers a very large part of what is needed, making fewer and fewer things need to end up in configure scripts. I think most of what's left is checking for items (types, functions) existence and their shape, as you were doing :). I can dream about getting a nice special operator to check for fields/functions, would let us remove even more from configure time, but I suspect we won't because that requires type resolution and none of the existing special operators do that.

mikepurvis 9 hours ago

You still need a configure step for the "where are my deps" part of it, though both autotools and CMake would be way faster if all they were doing was finding, and not any testing.

throwaway81523 9 hours ago

GNU Parallel seems like another convenient approach.

fmajid 3 hours ago

It has no concept of dependencies between tasks, or doing a topological sort prior to running the task queue. GNU Make's parallel mode (-j) has that.

creatonez 12 hours ago

Noticed an easter egg in this article. The text below "I'm sorry, but in the year 2025, this is ridiculous:" is animated entirely without Javascript or .gif files. It's pure CSS.

This is how it was done: https://github.com/tavianator/tavianator.com/blob/cf0e4ef26d...

o11c 12 hours ago

Unfortunately it forgets to HTML-escape the <wchar.h> etc.

tavianator 10 hours ago

Whoops! Forgot to do that when I switched from a ``` block to raw html

epistasis 13 hours ago

I've spent a fair amount of time over the past decades to make autotools work on my projects, and I've never felt like it was a good use of time.

It's likely that C will continue to be used by everyone for decades to come, but I know that I'll personally never start a new project in C again.

I'm still glad that there's some sort of push to make autotools suck less for legacy projects.

eqvinox 4 hours ago

To extend on sibling comments:

autoconf is in no way, shape or form an "official" build system associated with C. It is a GNU creation and certainly popular, but not to a "monopoly" degree, and it's share is declining. (plain make & meson & cmake being popular alternatives)

monkeyelite 12 hours ago

You can use make without configure. If needed, you can also write your own configure instead of using auto tools.

Creating a make file is about 10 lines and is the lowest friction for me to get programming of any environment. Familiarity is part of that.

viraptor 12 hours ago

It's a bit of a balance once you get bigger dependencies. A generic autoconf is annoying to write, but rarely an issue when packaging for a distro. Most issues I've had to fix in nixpkgs were for custom builds unfortunately.

But if you don't plan to distribute things widely (or have no deps).. Whatever, just do what works for you.

edoceo 12 hours ago

Write your own configure? For an internal project, where much is under domain control, sure. But for the 1000s of projects trying to multi-plarform and/or support flavours/versions - oh gosh.

monkeyelite 12 hours ago

It depends on how much platform specific stuff you are trying to use. Also in 2025 most packages are tailored for the operating system by packagers - not the original authors.

Autotools is going to check every config from the past 50 years.

charcircuit 4 hours ago

>Also in 2025 most packages are tailored for the operating system by packagers - not the original authors.

No? Most operating systems don't have a separate packager. They have the developer package the application.

monkeyelite 3 hours ago

Yes? Each operating system is very different and almost every package has patches or separate install scripts.

tidwall 13 hours ago

I've stopped using autotools for new projects. Just a Makefile, and the -j flag for concurrency.

psyclobe 11 hours ago

cmake ftw

JCWasmx86 9 hours ago

Or meson is a serious alternative to cmake (Even better than cmake imho)

torarnv 6 hours ago

CMake also does sequential configuration AFAIK. Is there any work to improve on that somewhere?

OskarS 4 hours ago

Meson and cmake in my experience are both MUCH faster though. It’s much less of an issue with these systems than with autotools.

aldanor 11 hours ago

You mean cargo build

yjftsjthsd-h 11 hours ago

... can cargo build things that aren't rust? If yes, that's really cool. If no, then it's not really in the same problem domain.

kouteiheika 9 hours ago

No it can't.

It can build a Rust program (build.rs) which builds things that aren't Rust, but that's an entirely different use case (building non-Rust library to use inside of Rust programs).

crabbone 2 hours ago

There's GprBuild (Ada tool) that can build C (not sure about C++). It also has more elaborate configuration structure, but I didn't use it extensively to tell what exactly and how exactly does it do it. In combination with Alire it can also manage dependencies Cargo-style.

touisteur 1 hour ago

Got it to build C++, CUDA and IIRC SYCL too.

malkia 10 hours ago

cmake uses configure, or configure-like too!

ahartmetz 7 hours ago

Same concept, but completely different implementation.