Today I am pleased to present, at long last, an initial public release of my Gales Linux project.
The goods
- gales-linux-20191201.tar.gz: repository archive
gales-linux-20191201.tar.gz-sums.asc: signed checksums thereof- gales-linux-20191201.tar.gz.asc: more simply, a direct signature of the tarball
To get started you may wish to examine the files:
BUILD, a lengthy recipe for bootstrapping the system from source.INSTALL, listing the steps to install the result on a machine. Presently quite terse; some experience with manual Linux installation such as in Gentoo will likely be needed as well as some clarifications.PORTS, explaining how to get started working with gports and turn the rather minimal base system into something more comfortable.gports/gales-util/gales-mirror-sync, with which you can download the much larger collection of sources referenced from this repository, both base and ports.(i) (ii)
Mirror sync is designed to work with minimal dependencies: you should be able to copy the script from the repository and run it on an existing Unix-like system, setting the DST variable to a fresh download directory and possibly replacing the "wget" call with "curl" or your preferred download tool. HTML scraping is neither required nor supported. If you'd rather not use my script you could parse the manifest yourself. (Just don't go putting something that blindly re-downloads the whole thing in a cron job, m'kay?) The hashes in the manifest act as a first pass of error detection in storage and transit, while canonical ones are in the signed repository.
The full mirror presently weighs around 440MB. Distribution is quite skew, with the top 10 file sizes:
$ awk '{print $2,$1}' Manifest.sha512 | sort -nr | head
93192404 linux-4.9.tar.xz
82935453 gcc-4.7.4.tar.bz2
76156220 gcc-6.4.0.tar.xz
22887305 db-4.8.30.tar.gz
22716802 binutils-2.24.tar.bz2
12495628 Python-2.7.13.tar.xz
12465748 php-5.6.34.tar.xz
11961692 perl-5.26.0.tar.xz
11570420 perl-5.24.2.tar.xz
9377313 bash-4.4.tar.gz
You'll need to supply the Linux kernel patch of your choice, or the whole thing if you need something other than 4.9.
The ports collection consists of:
acpica autoconf automake bash bc bison bzip2 cl-hyperspec clisp dash db flex gales-util gcc64 git gnupg less libevent libressl libusb links m4 man-pages man-pages-posix mandoc ncurses nginx ocaml openssh patch pciutils perl php56 py-setuptools python python-docs qmail readline redis sbcl sqlite sqlite-doc tmux ucspi-tcp vim xz zlibNote that some of these may be trimmed down or configured in more minimal ways than you're accustomed to, either due to compatibility problems with musl or static linking or as a general precaution. I'm not opposed to re-enabling features upon reasonable demonstration of safety, or you can certainly edit the builds to suit your own needs. Things I consider high priorities for porting effort include:
apache emacs gdb ghc gnat hdparm iptables mysql sdparm shred smartctl strace texinfo X11Obligatory disclaimer: Like any operating system targeting the "modern" hardware and software cocktail, Gales Linux contains large amounts of toxic and hazardous materials both known and unknown. While I have striven to make prudent and security-conscious choices, I am not attempting to keep up with the "penetrate and patch" rat-race in its many third-party components. Be careful out there.
Next steps
As my goal for now has been to release what I have without too much further fiddling, a few obstacles remain for a proper V genesis of the repository.(iii) What I've noted so far are some executable scripts in the repository that would either need to be invoked differently or somehow "chmod"ed before use, and one patch (to bin86) that includes a control character in its context lines. Also some clutter for potential removal is a number of old-style "context diffs" that I reground to unified format for BusyBox compatibility but preserved the originals for the sake of preservation (e.g. in bash and vim ports).
- I've fixed the previously noted subdirectory flaw. [^]
- Present mirror IP:
198.199.79.106 welshcomputing.com[^] - If such a direct conversion is even a sensible way to go; for instance, including externally maintained items as tarballs by hash reference is not in keeping with V principles, but taking ownership of the whole mess will be a larger project. [^]
Gales definitely looks very interesting! I have some questions though:
1. While I understand the reasoning for stack-limit, did you consider backporting the -fstack-clash-check instead?
2. Nice job with gcc patches, though what I find missing is the fix to a bug that AFAIK reduced the efficiency of stack protector of amd64 (this was mentioned somewhere by grsecurity people, can't find the link). Patch from my local machine (really old gentoo):
diff -Naur gcc-4.9.4.orig/gcc/config/i386/i386.c gcc-4.9.4/gcc/config/i386/i386.c
--- gcc-4.9.4.orig/gcc/config/i386/i386.c 2016-08-01 18:03:41.000000000 +0200
+++ gcc-4.9.4/gcc/config/i386/i386.c 2017-04-29 21:23:58.164465317 +0200
@@ -26860,7 +26860,7 @@
!= TYPE_MAIN_VARIANT (va_list_type_node)))
&& TYPE_SIZE (type)
&& TREE_CODE (TYPE_SIZE (type)) == INTEGER_CST
- && (TREE_INT_CST_LOW (TYPE_SIZE (type)) >= 16
+ && (TREE_INT_CST_LOW (TYPE_SIZE (type)) >= 128
|| TREE_INT_CST_HIGH (TYPE_SIZE (type))) && align < 128)
return 128;
}
Also, did you check the patches from musl-cross-make?
3. Just curious, why ocaml and ghc?
Comment by bvt — 2019-12-03 @ 23:38
Thanks for the scrutiny bvt! I'll have to dig a bit and get back on your questions. I have not done an exhaustive search for gcc patches (there's a ton of both not-backported and never-closed bugs on their tracker for example). Fixing the 'stack clash' thing for good would certainly be preferable to magic-number-based warnings, though those have been enlightening as far as how many things - even multithreaded - assume infinite stack without a care in the world. I don't presently recall what I found about -fstack-clash-check itself, though I was suspicious that they came up with some new optional thing targeted to the latest brand-name for something that should never have been possible to begin with.
Comment by Jacob Welsh — 2019-12-04 @ 18:02
Re stack clash, some more notes (not trying to set a direction here, just to put out some datapoints):
- The set of -fstack-clash-protection patches backported to gcc 4.8.6 is available in CentOS SRPMs. I don't have an access to any system with -fstack-clash-protection, so never looked at the code it generates, can't comment on the quality.
- The -fstack-clash IIRC is a thing coded for GNAT (IIRC, ensure that there always is at least one free&mapped-in page of stack) in such a way that it is useless for all other purposes.
- Recent musl has an ability to get pthreads stack size from PT_GNU_STACK header, which may be useful to avoid defaulting to large stacks for all applications.
Comment by bvt — 2019-12-04 @ 20:09
[...] sure that's going anywhere either, since meanwhile TMSR-OS management changed and there seems to be an alternative out there, different in some respects, and currently in need of further [...]
Pingback by Work plan for (the rest of) M12 2019 « The Tar Pit — 2019-12-07 @ 16:16
@Jacob Welsh
It's quite a pleasure to see Gales published, congrats!
@ bvt
Thanks for raising this, I'm make sure it's revisited.
Thanks for sharing the patch.
Comment by Robinson Dorion — 2019-12-09 @ 15:59
Updated to provide direct signature by reader request, thank you Diana Coman for pointing out what amounted to a pointless inconvenience.
@bvt: the Ada-specific one by my recollection is -fstack-check and Felker noted that it's actually worse than useless elsewhere, as it can jump the guard page if its assumptions don't hold. And you wouldn't know it's thus intended from its description in the manual.
Missed this... I like OCaml, learned it before Lisp and still miss the static typing, though perhaps Ada will satisfy once I get around to it. "Unison" is a pretty nice tool that uses it. I was at one point learning Haskell and among its dependents there's XMonad.
Comment by Jacob Welsh — 2019-12-13 @ 06:14
For the record, I have a preliminary install almost ready and I doubt it's going to be the last. :D
One quick question: is there any documentation for the service facility? So far I've been restarting daemons by manually killing the underlying process and having "supervise" automatically restart it. Is this standard procedure, or am I just blindly spinning around?
Comment by spyked — 2020-01-27 @ 16:36
Cool! Enlightenment is at http://cr.yp.to/daemontools.html (it's djb: expect ultra-terse but thorough). Really those docs ought to ship with the system too. For the particular question: "svc -t /service/foo".
Comment by Jacob Welsh — 2020-01-29 @ 00:13
Thank you! Indeed, including the documentation wouldn't hurt.
For the record, I had absolutely no idea DJB made a daemon manager. I kinda like the ultra-terse-but-thorough style, TBH.
Comment by spyked — 2020-01-29 @ 08:45
[...] previous writings to speak for themselves; more precisely, JFW's introduction to Gales Linux; the initial release; and Bvt's installation report are so far the foremost pieces on the subject, as far as I'm [...]
Pingback by A journey through the Gales installation process « The Tar Pit — 2020-01-31 @ 15:55
As spyked discovering the "tar --sort" hack reminded me to post:
Comment by Jacob Welsh — 2020-01-31 @ 22:40
One patch I'd like to point out for particular scrutiny by the cryptologically inclined is gports/libressl/libressl-2.5.4-entropy.patch (in the tarball here). As it reads in the header:
To perhaps clarify: the first point applies if you have a newer kernel with the "getrandom" syscall; the intent is that a process should block if there isn't adequate entropy at early bootup, rather than ignoring the problem. Once the pool is first initialized there's no further blocking, as with /dev/urandom.
The second point applies to older kernels lacking getrandom.
Comment by Jacob Welsh — 2020-01-31 @ 23:10
[...] Gales is the thing to break if you want to make Jacob cross2 and otherwise a fully static linux distribution of delightfully small size and clear setup that works reportedly ~everywhere, from Panama to the tar pit of many virtual machines and the backtrace of various network installations. [...]
Pingback by Thinkpad in Gales « Ossa Sepia — 2020-02-07 @ 15:11
[...] having already taken on the publication of a Linux distribution with similar static linking and deterministic bootstrapping goals, but going further in providing a [...]
Pingback by Build system overhaul for bitcoind « Fixpoint — 2020-04-28 @ 21:19
[...] Unix-like OS. I used a fresh, minimalist install of Gales Linux, which comes with the Gnu C Compiler (GCC) 4.7.4 ; musl C Library ; Busybox userland ; and Linux [...]
Pingback by GBW-NODE : Gales Bitcoin Wallet Node verified acquisition, build, install and run in 21ish short, simple steps. « Dorion Mode — 2020-07-01 @ 20:29
Adding the trackback from bvt's installation report attempted initially from Debian 9 / GCC 5. It appears the failure of GCC to build its own older versions hasn't been fixed as of Ubuntu 20.x (probably GCC 10 or some such).
Comment by Jacob Welsh — 2020-11-27 @ 18:36