An as-yet little publicized project that's been cooking at JWRD for a few years now is what we've been calling the Private Enterprise Computing Platform or PECP. It aims to provide a turnkey, top-to-bottom solution(i) for independent-minded shops to deploy self-hosted infrastructure for their data storage and communication needs. For starters this includes internal file sharing and backups, email, public website, and customer-facing and administrative database interface ("portal").
On the server side it's become a primary driver for our ongoing research and development in hardware and on Gales Linux. On the client side, the idea is to meet the market where it's at; this means mostly Windows and Mac, perhaps the occasional mainstream Linux system, and rather dreadful security conditions. It's about offering people an attainable first step towards taking more ownership and becoming better stewards of data, both their own and their customers'.(ii)
With this backdrop, let's zoom in now to the email piece. The contemporary architecture at least among open systems consists of a number of major components, which include: a Mail Transfer Agent to send and receive; sometimes a separate Delivery Agent for things like filtering; a Mail User Agent (client) to read and compose messages; and generally some Access Protocol to connect the MUA to the MTA's file system.
A popular setup these days is for the MUA to be a Web application, due to the client portability this provides, and we're going with this as our initial solution for the same reason. This means there's also a Web server, a scripting language, and possibly a Relational Database Management System involved in supporting the User Agent. The demands placed on the access protocol are also augmented by this "cloud-shaped" mode where the MUA is a largely stateless frontend to a server which stores the whole pile of accumulated mail, archived or otherwise. Finally, to exchange mail with unaffiliated external providers, a dependable Domain Name System server is required along with a Registrar to represent your ownership interests before the ever so grand and exalted Registry of your Top-Level Domain.
The biggest gaps in JWRD's software repertoire, then, were the access protocol and parts of the user agent stack. We started out looking at servers implementing the Internet Mail Access Protocol (IMAP), because an early survey of user agent software confirmed that most options there were going to demand it.
To give the best flavor, what follows is the series of progress reports we wrote for our pilot client, lightly edited to remove unrelated parts. Footnotes are from the originals except where noted as updates. The series documents the work and the readjustment of plans as it unfolded, turning from the simple job we'd anticipated of picking a product and configuring it to fit our system, into a veritable saga for our times. At many points the grass looked greener not just on some particular other side but on just about any conceivable other side too; but we persevered valiantly and/or stubbornly and here we are now, still without a working product but having at least the IMAP piece seemingly well under control. I intend to follow up shortly with releasing the resulting code as discussed.
From: Robinson Dorion, CEO, JWRD Computing
Date: 2022-08-25
With getting email up and running as the highest priority, we've decided to focus on that software first, rather than resuming with MySQL which may or may not end up being needed per se for email. Jacob reviewed my research on the IMAP server options and did some of his own, and will start digging into the code.
The question came up of whether we really need IMAP at all since there is at least one application that works directly from the filesystem (SqWebMail). We decided to stick with the plan for reasons of security (keeping the data more separated from the web server) and flexibility (perhaps to offer direct IMAP service if needed later).
I also did some research on the Horde webmail front end and next I'll dig into SquirrelMail.
Jacob is planning to fly up to Maine this Friday for a 2 week trip, most of which will be spent at the family cabin in Boothbay Harbor. He's bringing a laptop and some work may move forward, e.g. preparing the next release of Gales Linux, but more likely than not it'll be a proper vacation. [...]
From: Robinson Dorion, CEO, JWRD Computing
Date: 2022-09-14
Jacob had a solid 2 week break in Maine. While mostly unplugged and recharging, he reviewed the MySQL work in preparation for the next article in the series. Also, he reviewed and organized the Gales Linux work from the past couple years in preparation for a new release.(iii)
Getting email up and running remains the highest priority and deciding and porting the IMAP software is the proximate target.
[...]
From: Robinson Dorion, CEO, JWRD Computing
Date: 2022-09-20
[...]
IMAP research progressed last week with the options expanding to Dovecot, Courier, Cyrus and UW IMAP, then UW being eliminated because of its security record, design, and unmaintained status. A potentially key differentiating feature that came up was metadata caching or indexing, especially given the need for first-class webmail support.
The goal for this week is to further the research. This will include cracking open the documentation and code, analyzing the build systems and attempting builds. Optimistically, this will yield one of them being ported to Gales Linux by this time next week.
In parallel, I'll be continuing the research on the Webmail options.
From: Robinson Dorion, CEO, JWRD Computing
Date: 2022-09-27
IMAP server research has progressed, with thorough surveys of Courier, Cyrus and Dovecot conducted. This week, we're on the to build and porting stage for Dovecot.
Courier was ruled out because it lacks message indexing - either headers or contents - which means loading large folders will likely be painfully slow, especially in the webmail context. This left Cyrus and Dovecot, both of which provide indexing for headers as well as optionally contents, allowing for fast full-text search.
While some might argue Cyrus is still alive, to our eyes it's more of a zombie. It's supported by Fastmail, but some of the longer standing users have complained that Fastmail has hijacked the project from its Carnegie Mellon University roots. A couple other problems :
1) their mailing list essentially went inaccessible without prior discussion as of October 2020 ;
2) their long-standing builtin full-text indexer, 'squat', was broken and left unfixed for 2 years ;
3) their SQL authentication support is not builtin but requires a separate package implementing the Pluggable Authentication Modules (PAM) interface, which thus far is unsupported in Gales Linux.
On the dovecot side, the original developer from 2002, one Timo Sirainen, is still heavily involved, as part of the "Open Xchange" company. By some accounts, the project is quite well supported, "Confirmed. Timo (and other users) doesn't leave any (non trivial) question unanswered, wiki is full of useful info, important bugs are fixed quick (follow mercurial repo) and new versions are released often - dovecot is one of best supported open source projects I know."
While there have been some security vulnerabilities found in recent years, this could simply be indicative of wider usage leading to more attention from researchers.
In support of the view that there are more ready solutions to common problems in the Dovecot world, there are multiple documented options for migrating from competing servers ; whereas for the same topic on the Cyrus side there is only "Oooh... I'll see if I can dig it out."
While Dovecot supports PAM, it also allows various SQL DBs as separate drivers and thus will likely better fit the Gales environment.
To conclude, we're moving forward with the porting of Dovecot to Gales. There will likely be the usual sorts of hurdles on the software side; besides that, Jacob's power supply may come under threat if Hurricane Ian makes landfall from the Gulf. Nonetheless, if things go well, the port could be completed this week.
From: Robinson Dorion, CEO, JWRD Computing
Date: 2022-10-03
The gport for the IMAP server Dovecot progressed, but isn't quite finished.
Upstream uses a tool called "libtool", which is commonly used in the wider "autoconf"/"automake" (aka autoshit) tool set from GNU. We detest all of this tooling for the unnecessary complexity and opacity it imports into the building of projects. However, Rome wasn't built in a day and fixing all the braindamage in the extant software world won't be either. Consequentially, autoconf and automake already exist as gports. On this pass, we realized that prior attempts to de-libtoolize packages essentially was more like streamlining the build systems of autoconf projects while staying in the autoconf world and when stated at such, doesn't sound so smart.
If the actual goal is to get the best fit for Gales without caring about other systems, what's needed is to cut all the autoshit, not to spend time surgically removing only parts of it. Likewise, if the goal is to get it working quickly, then leave the whole spittoon alone. Jacob conducted a review of the current gports tree and found the shipped configure scripts of 9 projects demanding libtool. 3 gports have "kill libtool" patches. To date, none of the kill libtool work resulted in the complete removal of autoconf.
Furthermore, the key step in removing autoconf in our experience has been to run the original configure so as to get the comprehensive picture of what it did. The cleanest way to do that, is by generating configure from source. So not having libtool at hand increases the costs of that process. Thus, we decided to create a libtool gport, which Jacob completed.
The build system required further patching because it assumed some "automake" library code for detecting the "iconv" implementation is present in the environment, making for a spurious dependency on GNU Gettext. Gales uses the musl C library which provides the POSIX standard variant of "iconv", making such detection code pointless. Jacob bypassed this nonsense with 16 lines of code replacing their 293.
The steps remaining in the Dovecot build are :
1) run the configure script ;
2) look through various options ;
3) decide what to enable and disable ;
4) try building ;
5) fix what's broken ;(iv)
6) finish packaging ;
7) test ;
8) deploy.
The light at the end of the IMAP tunnel is growing bigger and the odds of it being daylight are increasing.
From: Robinson Dorion, CEO, JWRD Computing
Date: 2022-10-10
The gport for the IMAP server Dovecot progressed to uncover their build system isn't as reliable as we thought it was and that the next step is to try to replace it with something simpler.
Using the new libtool gport, Jacob regenerated the configuration script and worked through the options to decide what to enable. He noticed an anomaly in the autoconf probing of GCC hardening options enabled by default in Gales. Looking into it, he determined it's not a problem on our current hardware architecture (amd64) and noted the path of adjustments needed if we were to change to architectures such as i386 or 32-bit powerpc.
Anomaly noted, he compiled Dovecot which completed without errors. However, the Dovecot plugins were not linked into any of the static binaries. Looking closer at the autoconf output, it falsely detected support for loadable modules. He worked around this with an autoconf stub that returned the correct value to the configure script. After the next configure run, the build failed due to the codebase being inconsistent under said configuration.
At this point, he took to the Dovecot mailing list to report the error and ask for clarification. Their response was effectually : we don't test the code under that configuration because we don't use it in that way.(v) Furthermore, there won't be effort from us to fix the error, but we'd consider merging fixes if you do the work.
Given that they've left us to solve the problem on our own, we've decided to take the path that will work best for us in the long run by trying to cut out the autoshit all together. This path will require some deeper study of the plugin system to learn how they are invoked. We figure it's better to invest time in studying the layout and linkage of the codebase and cleaning out autoconf altogether than wading through the swamps they inhabit and fixing broken build system they publish.(vi) Why subsidize what we view as poor choices ?
The end result of the proposed autoconf clean up will be extracting the config.h file and redoing the build system as a short, digestible makefile and in the long run an IMAP server that carries less deployment overhead.
Recalibrating expectations, the goal for this week is to work through the statically linked build. Following that, there will be finishing the packaging, working out the configuration files, testing and deployment.
From: Robinson Dorion, CEO, JWRD Computing
Date: 2022-10-17
As foreshadowed in last week's report, JWRD-izing the Dovecot build system requires study of the layout and linkage of the codebase and Jacob made solid progress on that front this week. Traversing down this path uncovered that dynamic linking assumptions go deeper than simply the autoconf-based build system. The code itself, at least in some places, is built around dynamic linking.(vii)
Jacob started with the plugin interface, which didn't provide enough information so the next step was to study the modules API. 'Module' is used more generally than 'plugin' and includes things like authentication backends and indeed SQL drivers. Dovecot builds many binaries for different sub-components and we need to know which of those the plugins need to be linked into. Furthermore, we skimmed at a high level the list of plugins to determine which are of actual interest to us.
The previously noted Full-Text Search Indexing (fts) is the most interesting. From prior reconnaissance, we'd want either fts-squat, which they've deprecated, or the new fts-flatcurve, which pulls in a "xapian" dependency. We didn't yet find good reason for the deprecation of fts-squat, but that doesn't mean there wasn't one. One potential issue is its inability to shrink, i.e. the index files won't shrink when you delete messages or move to another mailbox/folder, but that may not require any changes to the code and could be mitigated with an occasional maintenance script for when the disk is getting too full.
In studying the Modules API, Jacob produced a file-level map of what calls Dovecot's Modules API and what executables each one ultimately gets linked into. It made for a strong exercise in learning to decode what all is going on in the Makefile.am files and a closer view of what all is in codebase generally.(viii) The map provides a complete if not necessarily concise or optimal answer to what the plugins can possibly be called from. After trying the exercise, the result wasn't as valuable as hoped since ~all binaries turned out potentially affected, necessitating a deeper inspection of each usage.
Jacob has studied 5 of the approximately 12 parts of the code which use the module_* functions so far and of the 5, very little changes are needed. The plan is to spend a day studying the remaining 7. After this exercise, we see 3 possible routes :
1) Don't make code changes. Deploy on the cluster and systematically test. Fix if issues arise.
2) Take the plugins on a case by case basis and repair minimum code to make them work statically for the few we need and care about. The approach is to bundle all desired plugins into the big libdovecot.a and replace all the dlsym calls with static calls. Then, any executable that invokes them gets the right files linked via the normal workings of the linker.
3) Write a generic replacement for the modules functions that works statically. This will include a type of dynamic dispatch table.
One encouraging finding is that dynamic symbol lookup is pretty limited compared to module loading generally ; in particular, per Jacob's earlier map, it's only used by lib-fs/fs-api.c, config/config-parser.c and config/doveconf.c. Meaning we've perhaps already dispensed with them under the first 5 areas studied.
Pigeonhole, which adds support for the Sieve language, came up as a possible external plugin of note that we haven't been considering yet. It provides mail filtering capabilities. Robinson will be looking to see which, if any, of the Webmail clients under consideration support the Sieve language. Pigeonhole is external to the core Dovecot source code tree, which requires being loaded as a plugin if you use it. The approach would be to get both tarballs and build in same tree in order to static link it.
Tangentially related, pursuing this path made the prior response received from upstream Dovecot developers worse than perceived on first pass ; they gave an invite to do the work on our own dime without any warning with regard to how hard they had made it to pursue static linking. Furthermore, if they're saying they only support their code when used in the ways that they currently use it... well, necessarily there are no bugs in that usage, otherwise they wouldn't be able to use it. In other words, the supported portion of dovecot is necessarily a subset of the bug-free portion, so no bugs can be reported.
While we climbed to new heights last week, it uncovered the mountain top is still further than previously estimated. The goal for this week is to finish the plugin research, decide which path to take, and implement some or all of the path taken. Finishing the gport, configuration and testing will still remain next week.
From: Robinson Dorion, CEO, JWRD Computing
Date: 2022-10-24
As foreshadowed in last week's report, the JWRD-izing of Dovecot continued with Jacob completing the survey of the parts of the code which load modules. 5 parts were studied last week and we reported there were approximately 12 total; the final figure came to 14 parts. Much of the fine grained details were shared in the #jwrd IRC log in real time and Jacob is working on an article to more clearly structure the findings. As far as the path to take, we've decided to try the approach of making a deep cut of the broken code, by replacing each necessary instance of module loading with static, hard-wired calls to each module's initialization routine (option #2 in last week's report), with the reasoning that static modules and a clean build system will be the best payoff for the work invested.
In terms of the Sieve programming language for filtering mentioned last week, I researched that both webmail options under consideration, Squirrel Mail and Horde, support Sieve functionality. This means that maintaining the option of supporting the Pigeonhole plugin may be worthwhile.
[...]
The goals for this week are to publish the article on the research of the module and plugin systems, implement the static calls and round out the research on the Webmail.
From: Robinson Dorion, CEO, JWRD Computing
Date: 2022-10-31
Work progressed in porting Dovecot last week. Jacob's first step was producing an article to clearly structure the findings from the prior work. With this firm ground to stand on, we decided to try replacing each necessary instance of module loading with static, hard-wired calls to each module's initialization routine,(ix) instead of implementing a more generic, static modules API.(x) Further complications then revealed that the shortest path would have to involve some degree of both anyway, so the outcome is a hybrid between the two. This work is tentatively complete for the four required instances, with cleanup of no longer functional code from the remaining instances still to do.
After this work is polished off, Jacob will work on producing a simplified Makefile to replace the build system.
On the webmail front, I rounded out the research, which Jacob will review this week. At present, Horde is the winner over Squirrel Mail. One of its dependencies is the Apache webserver ; so the port for Apache will need to be completed prior to the email service going live. Apache is also needed for the marketing website and admin/customer portal.
The goal for this week is to finish the Dovecot source code and build system changes and mark the Dovecot gporting complete.
From: Jacob Welsh, CTO, JWRD Computing
Date: 2022-11-14
As per the usual pattern, Dovecot porting work has proceeded steadily but with ever more obstacles arising along the way. "When going through hell, keep going", as the saying goes, and we have persevered, with the result of multiple improvements to the reliability and maintainability of the codebase, including removal of the unwanted "oauth2" feature and even the entire "dcrypt" subsystem. While we're still not done, the end is in sight more clearly than ever.
The first step was completion of all previously discussed actions pertaining to the dynamic module system, converting each desired piece either to the new static module initialization framework or else to build-time options not requiring the "modules" overhead, then cleaning out all the obsolete code. We ended up with six categories of module loading preserved out of the original 14. This concluded nearly four weeks of work, with the result that the desired modules and plugins can now work for the first time ever with a fully statically linked set of Dovecot binaries, and thus it can all run on Gales without compromising on the design principles of the system.
To make this a reality though still required revisiting the build system, which we had put aside to focus on the problems in the code itself. The next step there was to look at the autoconf-generated "config.h" file, which contains a full listing of the available code-level configuration switches and shows their proposed setting for the particular system. By auditing it, we learn whether the automated tools made the correct decisions for the system, and can then save a permanent version of it to cut those tools out of the loop altogether and vastly simplify the build process.
The difficulties were both quantitative, with over 300 config switches to sift through, and qualitative, with a few of them revealing deep troubles in the code as we worked to reduce the list to a more manageable size by removing unnecessary options. The thorniest one involved maintaining year 2038 safety (the new Y2K problem that's coming up for 32-bit Unix timestamp fields).
The investigation was productive even outside of the goal of replacing the build system, because it did turn up one case of incorrect autoconf decision due to bad test code that didn't work as intended on musl libc. We also learned that Dovecot has trouble when the system clock steps backwards; fortunately, time synchronization software usually aims for gradual adjustment rather than sudden stepping of the clock, but we may need to keep this in mind.
With the Dovecot-specific internals of config.h dusted, the build system rewrite moves into more familiar territory. The process is to convert the recursive nest of Automake files to a simple Makefile; this is underway, with a first milestone reached of building the central library "libdovecot.a" from 341 source files, as well as building 76 internal test programs. What remains is to continue this conversion to cover the real payload programs.
Then, we translate all the accumulated work from the git repository into patches or similar for the gports environment (perhaps issuing a whole new Dovecot release with "tar" file), then finally finish the gport and move on to testing. A preliminary build on Gales using the new system will be the goal for this week.
From: Jacob Welsh, CTO, JWRD Computing
Date: 2022-11-22
Recalling from last week:
With the Dovecot-specific internals of config.h dusted, the build system rewrite moves into more familiar territory. The process is to convert the recursive nest of Automake files to a simple Makefile; this is underway, with a first milestone reached of building the central library "libdovecot.a" from 341 source files, as well as building 76 internal test programs. What remains is to continue this conversion to cover the real payload programs.
This conversion is complete, after just a few wrinkles that needed ironing out.(xi) An additional piece not previously considered but now also complete was a similar conversion of build and install recipes for the documentation and example configuration files that ship with the package.
To summarize the build system rewrite in numbers: 6287 lines of Makefile.am, 3442 lines of m4 and 907 lines of configure.ac have been replaced by 2036 lines of Makefile and 686 lines of config.h. Besides the improvement in brevity, the new files have a far more regular and centralized structure, gathering all related things in one place.(xii) The downside is that porting to a very different system or changing the enabled feature set will require another pass through the config.h settings, rather than having them deduced automatically. However, we achieved the goal of one version that should work for any Gales platform; and otherwise, the magnitude of the reconfiguration task was reduced by the previous work on sanity-checking and removing unnecessary options.
Then, we translate all the accumulated work from the git repository into patches or similar for the gports environment (perhaps issuing a whole new Dovecot release with "tar" file), then finally finish the gport and move on to testing. A preliminary build on Gales using the new system will be the goal for this week.
This remains incomplete. I figured it made more sense to focus on the basic quality assurance first, by running the test suite, rather than issuing a release only to have to go back and patch it.(xiii)
This required first redoing a previous change to the somewhat messy random number generator interface code, so as to untangle it while still supporting a wider variety of systems (in particular, those lacking the getrandom() Linux system call which while nice in theory was only recently introduced). With this done I was able to run the tests in the Gentoo/musl environment I'd been working in.
The test run revealed three failures out of a total of 115 test programs (and many more individual test cases). At least that's better shape than MySQL was in, but the failures need investigating so that will remain the focus for now. With luck, they will be bad test cases or at least easy fixes, and the release could be wrapped up this week. But as luck isn't much to bank on, I'll be happy to at least get the failures diagnosed and documented.
From: Jacob Welsh, CTO, JWRD Computing
Date: 2022-11-28
The three dovecot test failures identified last week all proved to be due to faulty or unportable test code.
1. lib-charset/test-charset assumed support for the obscure UTF-7 encoding in the system's iconv interface (routines for character set conversion). To be on the safe side, I looked into how iconv was used in dovecot, did some data analysis regarding what character encodings are found in practice in the wild, and checked which ones are implemented by iconv as found in musl on Gales. The result is that pretty much everything within reason is covered, which means the software should be able to display and search messages with non-Latin/ASCII text (diacritics, Asian scripts, emoji etc.) at least on a "best effort" basis.
We don't yet know if it makes sense from a business standpoint to commit to full support of this usage. From the philosophically principled standpoint, we hold that in order for computer text to have meaning, there can be only one character set and it must be fixed, much like in order to read and write you must know the alphabet and it's neither acceptable nor useful to have that changing underneath you. The straightforward interpretation of this in practice would be to support ASCII only; but the requirements of the customers are not yet fully known.
In any case, there's not a shred of evidence of a need for UTF-7, so I removed those parts of the affected test cases.
2. lib-program-client/test-program-client-net assumed in-kernel support for IPv6: the supposed next-generation Internet Protocol, which 24 years after its inception still finds itself widely unloved across the industry, a non-solution to a non-problem. Since we use custom-configured kernels to minimize bloat and attack surface, we don't include it and all related system calls will fail, returning errors not expected by the test code. As before, I simply removed the IPv6 portion of the test, as I couldn't find a ready way to make it conditional on system support.
3. The file_cache_errors test in lib/test-lib was checking that system errors were handled properly; one such error produced the string "Out of memory" instead of "Cannot allocate memory" which the test was wired to expect. That's a difference between musl and glibc at the human-level interface, which isn't standardized or something that programmers are supposed to depend on, meaning that while perhaps well-intended, the test code was broken. I reduced its scope, having it check only that some error was reported rather than what exactly the report read.
While working on #3 I was frustrated by the slow pace of iteration, as it took over a minute each time to run the many test cases in the overall test-lib program before getting to the problematic one. Closer inspection revealed that most of the time was taken up by just a few gnarly tests. One such was deliberately wasting time and system resources in order to test builtin CPU limiting capabilities. Another several were testing various varieties of base64 encoding and decoding by using loops to feed large numbers of randomly generated samples through the code.(xiv) Paring back the demands of these cases (without removing them altogether) got the execution time of the program from 117 down to 11 seconds, and the overall test suite from over 6 minutes down to 4m27s (all on the same reference machine).
With a clean bill of health from the test suite, I turned to look at compiler warnings, which led to a few code tweaks: mostly minor cleanups to my own recent additions, then one to existing code: conservatively zeroing a pointer that was flagged as possibly used uninitialized.(xv)
I then moved toward a new dovecot release by updating its NEWS file to summarize all my changes, rebranding to "JWRD Dovecot", and bumping the version number from 2.3.19 to 2.4.0. Issuing a release candidate tarball allowed completion of a preliminary gport. As expected, there were some minor problems that turned up from building and testing the code in the new environment, but after another couple iterations it was building and fully passing the test suite there too.
That means we came out ahead of target for once this week. Call it luck, or perhaps we're improving on estimation.
The next steps here are: to publish the release, along with the series of patches leading up to it, for a full account of what all was done; study the documentation and example configurations, developing a suitable configuration for the PECP; and deploy an instance to start live testing. Since we don't yet have the webmail stack to plug in, our thinking is for Robinson to set up a Thunderbird instance as a desktop IMAP client with temporary credentials, load some sample data onto the server and put it through the paces.
- From a metal rack, up through uninterruptible power supply, servers, routers and switches, to operating system, middleware and end-user applications, with full engineering support, active monitoring and backups. Bring your own braindead ISP connection. [^]
- Consequently, mobile devices can play too, to the extent they're sufficiently capable, but are not the primary focus; we're not going to somehow make up for the many deficiencies that were built into them, oftentimes quite by design, the most significant being the total opposition to self-ownership. [^]
- Update: neither of these has happened yet. I don't even quite recall what that organizing was but I suppose it's still waiting for me somewhere. [^]
- Update: at this point I imagined the build was the only thing broken. I mean, the program itself works, right? I had even run it myself, for years, on Debian. [^]
- It's reasonable to conclude the static build worked at some time, because they have specific code to support the no-modules case. [^]
- Update: at this point I still imagined there was some usable underlying code and it was just the twisted build system standing in the way of linking it up properly. Like, you can be a sharp programmer within the confines of your preferred language and environment and just get clumsy when it comes to the gritty details of deploying on diverse real-world systems - can't you ? [^]
- One specific example is the ability to call functions by looking up their names as strings at runtime. [^]
- An http client & server ?! And here we were mentally knocking cyrus for throwing in such kitchen sinks ! [^]
- Option 2 in the week 41 report. [^]
- Option 3 in the week 41 report. [^]
- Three cases came up of duplicate definitions of global variables, preventing certain files from coexisting in the central libdovecot.a despite their being shared by multiple programs. In one case it was simplest to just link the problem file separately into its two consumers, while in the other two I was able to solve the problem at the root, removing the supposed need for duplicate definitions by moving some code around. Further detail is in the #jwrd log. [^]
- For example, it's now trivial to find the full list of programs being built, as each of the four categories of program (user-facing, internal, automated test and manual test) is defined by a single list in the single Makefile, where before you'd have to piece it together by scanning through 101 separate, noisy and nonuniformly structured "Makefile.am"s from all the subdirectories. [^]
- Update: I guess I should have at least put out patches for what I had done so far, what's the harm. At the very least it would have made for more digestible chunks. The actual mental process was probably more like, "writing stuff up for publication is hard and the rewards, if any, are distant, whereas trying stuff out is fun and rewarding now - and it doesn't count as laziness because it's productive too! [^]
- This is known as fuzz testing and it has its place, but there are smarter ways to go about it and in any case it doesn't belong in a general-purpose test suite meant for running early and often. [^]
- In C programming, a pointer is a variable that holds a memory address, thereby *pointing* somewhere else: usually to some other variable or data structure, but since it can hold literally any possible address, using an incorrectly set pointer can cause havoc ranging from program crashes to data corruption to ownage by an attacker. Due to various properties of the language (mainly the focus on execution speed at all costs, but "poverty" is not an altogether unfair way to put it), the system can't be built to accurately detect such problems, but it can warn about certain risky patterns. In this case, the use turned out to be safe on closer inspection, but since it wasn't obvious, it makes sense to stay on the safe side and humor the warning. This meant explicitly initializing the pointer to a safe default of zero/null which at least ensures the program will abort immediately without further damage if the pointer is used before being set properly. [^]
[...] subject as promised is the initial release of JWRD Dovecot, a full-featured, efficiently and statically compiled IMAP [...]
Pingback by JWRD Dovecot initial release, aka version 2.4.0 « Fixpoint — 2023-06-26 @ 04:40