Fixpoint

Upon receiving a resounding "meh" from the people code monkeys formerly considered "upstream" for the Dovecot IMAP server, we gained the clarity that we were indeed stuck with cleaning up yet another mess, to the degree it was to be cleaned up at all ; and we'd have to choose this degree carefully because holy hell there's still so much else to be done.

To bring some order to the discussion, let's start with an attempt at some previously missing definitions:

In Dovecot terminology, a module is one or more compilation units (.c files) compiled into a single shared object (.so file; contrary to appearances the suffix is not configurable), to be imported into process address space at startup, by the dynamic linker via dlopen.

The term plugin also comes up frequently, and seems to be more of a governance term than a technical one, referring to any collection of code built into any collection of modules - or perhaps even new executables - covering a particular class of optional features, and having some degree of independence in origin or maintenance from the main Dovecot codebase. That said, a number of plugins are currently bundled in the tree and covered in the documentation. Plugin support requires module support, but isn't the only thing that does.

Thus, our problem is that some number of possibly important features, which contributed to bringing Dovecot to the forefront of our search in the first place, currently require module support which currently assumes dynamic linking.

At first I'd been considering only the first layer of the problem, namely the build system, charitably proceeding on the hunch that the rot was contained to that layer. That's how it usually goes with onions after all, right? Perhaps it simply wasn't set up to link the modules in static library form (.a files) into the required executables so all I'd have to do was track down what those were and do some Makefile fiddling and we'd be set. Hey, since modules worked fine in static configuration for the Linux kernel, Python, PHP etc,⁽ⁱ⁾ with only that weirdo SBCL falling flat on its face thus far, how bad could it be?

Pretty bad, it turns out, as you've no doubt guessed by now. Seeing as a module compiles to a library, and a library is a collection of named procedures and variables without a main program entry point, if there's no code to refer to any of those procedures and variables (symbols) then they can't possibly have any effect.⁽ⁱⁱ⁾ And just as the build system was written without a hint of support for statically linking modules, so the code was written without a hint of support for statically initializing them.

Since I'd achieved a working build without module support, we at least had the option of leaving it out and doing without plugins. But without documentation especially of what all the internal, non-plugin modules were, we'd be proceeding blind to what missing pieces might come to bite us down the line.

As I'd learned from fixing the static build failure, most parts of Dovecot that act on modules - loading, unloading, initializing, deinitializing,⁽ⁱⁱⁱ⁾ and looking up symbols - don't call dlopen/dlsym directly but go through a "modules API", found at src/lib/module-dir.c. Thus my next step was to scan the codebase for all uses of those functions and draw up a complete map of what calls them directly and what those get linked into at the file level; that is, identifying all executables that could possibly call them, without fully tracing the call chains. While this was certainly helpful, the list of executables turned out to be pretty much everything so it wasn't enough.

I sketched out a next guiding question, roughly, to narrow the search to just the parts of the code that load modules so as to look at what they're doing and what modules they might load.^(iv) What soon came to light was that there are several different species of modules, corresponding loosely to the different places in the code that load them. Some of these can be seen just by looking at the build system because they're installed to a particular subdirectory of the top-level module installation path ($(prefix)/lib/dovecot aka moduledir aka pkglibdir), while others are more implicit.

With the looking done, what became sorely needed was to organize the findings into some consistent structure for easy reference, so as to finally make some decisions and keep track of progress without getting lost in the forest. Let us proceed to enumerate each of those load points, thereby labeling each species or type of module, and in some cases even listing each possible instance.

1.

• File: lib-fs/fs-api.c
• Function: fs_class_try_load_plugin
• Search path: moduledir
• Name list or pattern: libfs_<driver>.so
• Name parameter source: caller, perhaps config file
• Module type: Filesystem driver
• Plugin modules: libfs_compress, libfs_crypt, libfs_mail_crypt
• Non-plugin modules:
• Calling executables: possibly many
• Notes: Most drivers are builtin (fs_classes_init); modules are only used for plugin-supplied ones. Unclear if there's any difference between "class" and "driver". Uses module_get_symbol to load fs_class_<driver> so as to fs_class_register() it.
• Proposed action: Remove load point, fs-compress and mail-crypt plugins.

2.

• File: auth/main.c
• Function: main_preinit
• Search path: moduledir/auth
• Name list or pattern: *.so
• Name parameter source: n/a
• Module type: Authentication
• Plugin modules: lib20_auth_var_expand_crypt
• Non-plugin modules: libauthdb_imap, libmech_gssapi, libauthdb_ldap, libauthdb_lua
• Calling executables: libexec/auth
• Notes: The var-expand-crypt plugin also builds its same source file into a distinct but identical non-"auth" module: lib20_var_expand_crypt. gssapi, ldap and lua modules may be initialized specially via auth/passdb.c (passdbs_init). There's another load point at auth_module_load, called by passdb_preinit, userdb_preinit and mech_register_init, which seems entirely redundant with this one, loading specific modules that would be already covered by the blanket loading here.
• Proposed action: Remove load point, var-expand-crypt plugin, and non-plugin modules.

3.

• File: config/config-parser.c
• Function: config_parse_load_modules
• Search path: moduledir/settings
• Name list or pattern: *.so
• Name parameter source: n/a
• Module type: Settings
• Plugin modules: pigeonhole (external)
• Non-plugin modules:
• Calling executables: bin/doveconf, libexec/config
• Notes: Uses module_get_symbol_quiet to load: <modname>_set_roots, <modname>_service_settings_array, <modname>_service_settings.
• Proposed action: Remove load point, leaving a stub where the relevant pigeonhole module(s) would be statically initialized if we integrate it.

4.

• File: dict/main.c
• Function: main_init
• Search path: moduledir/dict
• Name list or pattern: *.so
• Name parameter source: n/a
• Module type: Dictionary
• Plugin modules:
• Non-plugin modules: libdict_ldap (if LDAP_PLUGIN)
• Calling executables: libexec/dict
• Notes: libdict_ldap already has the option of being builtin.
• Proposed action: Remove load point; in the build system and ifdefs for libdict_ldap, remove the plugin option so it's either builtin or absent.

5.

• File: lib-ssl-iostream/iostream-ssl.c
• Function: ssl_module_load
• Search path: moduledir
• Name list or pattern: libssl_iostream_openssl.so
• Name parameter source: n/a
• Module type: singular
• Plugin modules:
• Non-plugin modules: libssl_iostream_openssl (if BUILD_OPENSSL and HAVE_SSL)
• Calling executables: possibly many
• Notes:
• Proposed action: Remove load point and statically call ssl_iostream_openssl_init (conditional on HAVE_SSL as it already is).

6.

• File: login-common/main.c
• Function: login_load_modules
• Search path: configurable via global_login_settings->login_plugin_dir, default moduledir/login
• Name list or pattern: configurable list via global_login_settings->login_plugins, default none
• Name parameter source: n/a
• Module type: Login process
• Plugin modules:
• Non-plugin modules:
• Calling executables: libexec/imap-login, libexec/imap-urlauth-login, libexec/pop3-login, libexec/submission-login
• Notes:
• Proposed action: Remove load point.

7.

• File: old-stats/main.c
• Function: main_preinit
• Search path: moduledir/old-stats
• Name list or pattern: *.so
• Name parameter source: n/a
• Module type: Old statistics process
• Plugin modules: libold_stats_mail
• Non-plugin modules: libstats_auth
• Calling executables: libexec/old-stats
• Notes: Not to be confused with imap-old-stats which appears to be an ordinary Mail plugin.
• Proposed action: Look at the modules and decide whether to always include or fully remove.

8.

• File: doveadm/doveadm-mail.c
• Function: doveadm_mail_init_finish
• Search path: configurable via doveadm_settings->mail_plugin_dir, default moduledir
• Name list or pattern: configurable via doveadm_settings->mail_plugins, default none
• Name parameter source: n/a
• Module type: Mail or perhaps a Mail Administration subtype?
• Plugin modules: unknown, could be many
• Non-plugin modules: unknown
• Calling executables: bin/doveadm
• Notes:
• Proposed action: Replace with a static list of <modname>_init calls based on what plugins are enabled at compile time. Alternatively, retain the module loader code, adding an internally wired table of the <modname>_init functions so that all plugins get compiled and linked but only the user-specified ones initialized.

9.

• File: doveadm/doveadm-pw.c
• Function: cmd_pw
• Search path: moduledir/auth
• Name list or pattern: *.so
• Name parameter source: n/a
• Module type: Authentication
• Plugin modules: lib20_auth_var_expand_crypt
• Non-plugin modules: libauthdb_imap, libmech_gssapi, libauthdb_ldap, libauthdb_lua
• Calling executables: bin/doveadm
• Notes: It would seem that auth modules can add functionality both to the "auth" daemon process (handling authentication requests) and to doveadm (perhaps configuring users & passwords).
• Proposed action: Remove load point, var-expand-crypt plugin, and non-plugin modules.

10.

• File: doveadm/doveadm-util.c
• Function: doveadm_load_modules
• Search path: moduledir/doveadm
• Name list or pattern: *.so
• Name parameter source: n/a
• Module type: Administration
• Plugin modules: lib10_doveadm_acl_plugin, lib10_doveadm_quota_plugin, lib20_doveadm_fts_lucene_plugin (if BUILD_LUCENE), lib20_doveadm_fts_plugin, libdoveadm_mail_crypt_plugin
• Non-plugin modules:
• Calling executables: bin/doveadm, libexec/doveadm-server
• Notes:
• Proposed action: Remove load point and statically call <modname>_init for all such modules, each conditional on being enabled at compile time. Remove fts-lucene plugin.

11.

• File: lib-storage/mail-user.c
• Function: mail_user_try_load_class_plugin
• Search path: user->set->mail_plugin_dir, default moduledir
• Name list or pattern: <name>.so
• Name parameter source: storage driver, perhaps parsed from configured mailbox path
• Module type: Storage driver
• Plugin modules: none?
• Non-plugin modules: none?
• Calling executables: libexec/imap etc.
• Notes: Seems to be a vestige of modular mailbox formats which are now all builtin.
• Proposed action: Remove load point.

12.

• File: lmtp/lmtp-client.c
• Function: client_load_modules
• Search path: client->lmtp_set->mail_plugin_dir, default moduledir
• Name list or pattern: client->lmtp_set->mail_plugins, default none
• Name parameter source: n/a
• Module type: Mail or perhaps LMTP?
• Plugin modules: unknown, could be many
• Non-plugin modules: unknown
• Calling executables: libexec/lmtp
• Notes: Can one module extend both mail (imap etc) and lmtp processes, or are these properly regarded as distinct module types?
• Proposed action: If we do any sort of customized delivery, we'd be using the LDA, not LMTP; I'm pretty sure qmail doesn't speak the latter, and why should it. So... remove the whole LMTP tree??

13.

• File: lib-dcrypt/dcrypt.c
• Function: dcrypt_initialize
• Search path: moduledir
• Name list or pattern: libdcrypt_<backend>.so
• Name parameter source: "openssl" only, as default and also set explicitly by some callers.
• Module type: singular
• Plugin modules:
• Non-plugin modules: libdcrypt_openssl (if BUILD_DCRYPT_OPENSSL)
• Calling executables: bin/doveadm, various (via auth/db-oauth2.c and plugins mail-crypt, var-expand-crypt)
• Notes: Used to be switchable with gnutls.
• Proposed action: Similar to #5, remove load point and statically call dcrypt_openssl_init (conditional on DCRYPT_BUILD_OPENSSL? but that's a make variable, not a preprocessor one).

14.

• File: lib-storage/mail-storage-service.c
• Function: mail_storage_service_load_modules
• Search path: user_set->mail_plugin_dir, default moduledir
• Name list or pattern: user_set->mail_plugins, default none
• Name parameter source: n/a
• Module type: Mail
• Plugin modules: many
• Non-plugin modules: unknown
• Calling executables: bin/doveadm, libexec/doveadm-server, libexec/imap-urlauth-worker, libexec/imap, libexec/indexer-worker, libexec/dovecot-lda, libexec/lmtp, libexec/pop3, libexec/submission, libexec/script-login, libexec/quota-status (from quota plugin), possibly more via mail-crypt plugin.
• Notes: Presumed to be the "main entrance", where the chosen mail plugins get loaded by most executables that do such loading.
• Proposed action: See #8.

In trying to track down the various "unknowns" above for non-plugin modules (which at this point look most likely to be simply "none"), a troublesome question came up that I can't yet account for. The SQL drivers, which as noted can be configured as either builtins or modules following the libdriver_<dbname> pattern, didn't come up under any of these load points; yet there's no indication in the docs that they'd have to be loaded explicitly by listing them in a mail_plugins setting. The proposed action would be like #4, to remove the plugin option so they're either builtin or absent, but why did the exhaustive search fail to discover their existence?

As for what's the minimum work that must be done now to get the desired modules working, many of the "remove load point" items could be postponed as cleanups for another day, although I don't expect them to be too difficult anyway. The SSL parts could be left broken as they are. Looking at #8 and #14, it begins to appear that generic replacement of the modules API isn't really an alternative as much as an additional layer on top of what's required here anyway. For instance, such a replacement would need to reproduce the concept of "loading all modules of a given type i.e. from a given directory or naming pattern", in a context where there are no such filesystem directories to obtain a list of .so file names to scan. Thus I'm inclined more than ever not to bother.

1. The jury's out on Perl; it seems fine so far but I do see some modules with .a files in /gales/pkg/perl/arch/auto/ and I lack the familiarity to readily determine if they're truly working. [^]
2. Except for memory corruption attacks causing control to jump into library code unexpectedly, thus broadening the attacker's options for escalation, true, there is that "advantage", this is C after all. That risk is worsened by shared libraries, broad ones like libc especially : because they're shared between processes they have to be immutable, thus they have to be mapped into each process address space in full, unlike static libraries where only the compilation units actually referenced by a given executable need be linked into it. But that's OK, see, because shared libraries allow for address space scrambling, and we know how content scrambling worked out for DVDs, right? [^]
3. I have no idea why include code for unloading or deinitializing modules, in the proper order and everything. Surely you could simply restart any running daemons to apply a change to the module configuration, and don't you have to do that anyway? [^]
4. I removed module_dir_init from my search list upon finding it takes an array of loaded module objects, not their names, so tracing it would not be as informative as to which modules those are. [^]