{
  "$type": "site.standard.document",
  "bskyPostRef": {
    "cid": "bafyreicnl5by7rbon42krggmbn4lnq2nopvjwfdrhsektgpk2truhn6lmm",
    "uri": "at://did:plc:pi6woz4d47bkuws673w2il2r/app.bsky.feed.post/3mmfmn5aiz4q2"
  },
  "path": "/t/botan-bindings-devlog/6855?page=11#post_213",
  "publishedAt": "2026-05-21T23:11:34.000Z",
  "site": "https://discourse.haskell.org",
  "tags": [
    "last update",
    "swiss cheese model"
  ],
  "textContent": "# Supporting multiple versions of Botan\n\nThe last update ended with a brief foray into importing botan’s `BOTAN_HAS_<feature>` support macros, so we can eg determine what algorithms are available in a given installation - this update continues that.\n\n# Problems!\n\nA quick recap, adding this support isn’t trivial - there are a few issues that combine and really get in the way of what should have been an easy import-some-C-macros-and-use-them.\n\nThe first, and core, issue is that we can’t actually import any botan macros for use with `CPP` and conditional compilation, (which, as the intended use, would have been really nice):\n\n  * There is an indented `#define`, which is (according to GHC’s strict C parser) not legal, and so if we use eg `{-# LANGUAGE CPP, CApiFFI #-}` and then `import <botan/build.h>`, it throws an error because of the indented define\n  * This is because `build.h` is actually technically a C++ header, but it gets parsed as a C header because of the `.h` extension.\n  * This shouldn’t be a problem, we only should need to `import <botan/ffi.h>`, which genuinely _IS_ a C header, except…\n  * `ffi.h` imports `build.h`, which was written _to be usable as a C header_ and this works when _compiling C with a C++ compiler and C++ compilers do allow indented defines_ , but we are compiling with a strict C compiler, so we can’t even `import <botan/ffi.h>` to get access to `BOTAN_VERSION_MAJOR/MINOR/PATCH` macros for conditional compilation _that way_.\n\n\n\nThe second issue is how botan intends for you to use the macros:\n\n  * When a feature or algorithm is present, the relevant `BOTAN_HAS_<feature>` macro is defined\n  * The macro is defined with a constant value of the DATESTAMP of the release date of the relevant version that the feature was introduced. Eg, today’s date would be `20260521`.\n  * This is being changed to using a VERSION_CODE, which is the same thing except with version numbers instead of dates.\n  * There are macros for this, in `botan/version.h`, which… _is not a C header_ , and it also throws errors if imported\n  * This is fine, we can just `foreign import capi unsafe` it\n  * Except, if the macro isn’t defined, trying to `foreign import capi unsafe` it will… throw an error because of a missing symbol!\n  * That’s easy to fix, we can just define the macro as `0` if its missing, right?\n    * And don’t really care about DATESTAMP vs VERSION_CODE just that the value is `> 0`\n  * Except, we can’t check if the macro is defined yet _in order to do that_ , _because that requires being able to import it_ - a Catch 22\n  * In other words, we can’t CPP-import the macro to check if its defined, but if it isn’t defined, it will crash when we try to FFI-import it.\n\n\n\nThe third issue is retconning missing defines in older versions of botan:\n\n  * As part of a big push for improving the FFI, over the course of `3.0-3.12`, every new feature has come with a `BOTAN_HAS_<feature>` macro to indicate support.\n  * As part of that push, botan has also added multiple NEW defines for PRE-EXISTING features, so that they can be optionally disabled - note that they were not optional features before…\n  * A side effect of this, is that if we do add the new macros to support newer botan versions, it causes old versions of the library to crash because those symbols are predictably missing.\n  * Another side effect is that a missing `BOTAN_HAS_` macro can now mean one thing for one version of botan, and different thing for another version\n  * More specifically, a missing support macro means that the feature IS PRESENT for PRE-EXISTING FEATURES IN OLD VERSIONS, but means that the feature IS MISSING for NEW FEATURES or PRE-EXISTING FEATURES IN NEW VERSIONS\n  * Still with me?\n  * A concrete example is KDF (Key-Derivation Functions), a feature which has been present since before `3.0`, but has since had a new `BOTAN_HAS_KDF` macro added, in `3.9`.\n  * This means that for `3.0-3.8`, the `BOTAN_HAS_KDF` macro is always missing, because the feature is always present, because it cannot be turned off, but for `3.9+`, if `BOTAN_HAS_KDF` is missing, it means that the feature _is missing because it has been turned off_.\n  * We can handle his if we can tell what version we are running - hence, solving the first two issues solves this one\n\n\n\nBasically, we can’t use botan’s support macros how they are intended to be used, because it fails the swiss cheese model because these few holes lined up perfectly, to make fixing this one of the most annoying things I’ve had to deal with in a very long time. None of this would be a problem if it weren’t for that single indented `#define` - crazy, right?\n\n# Solutions!\n\nWe can actually handle this if we can get access to the version and support macros, except I’ve considered and / or tried a lot of different things to various levels\n\n  * A C header shim didn’t work, because it has the exact same problems.\n  * I could parse the `build.h` file myself to generate a fixed header with just the support macros defined but that sounds fragile\n  * Can we force the compiler to treat it as a C++ header despite the `.h` extension? This also sounds like a big hammer\n  * Can we parse the results of `botan --version` just to get the version code? This might not detect the right installation, eg when testing against a local install vs a global install\n\n\n\nI am not that much of a C++ guy, I prefer my C. So it took me a while to realize:\n\n  * Can I make a C++ header shim? By naming it with a `*.hpp` extension?\n  * If I do, we can handle _everything_ inside of it.\n\n\n\nAnd, fingers crossed, initial testing shows that it works! I’m still finalizing _precisely_ how it goes about defining macros for pre-existing features, but roughly something like this\n\n\n    #ifndef BOTAN_MACRO_SHIM_H\n    #define BOTAN_MACRO_SHIM_H\n\n    #include <botan/ffi.h>\n    #include <botan/build.h>\n\n    // Copied from <botan/version.h>, which cannot be imported properly\n    #define BOTAN_VERSION_CODE_FOR(a, b, c) (((a) << 16) | ((b) << 8) | (c))\n    #define BOTAN_VERSION_CODE BOTAN_VERSION_CODE_FOR(BOTAN_VERSION_MAJOR, BOTAN_VERSION_MINOR, BOTAN_VERSION_PATCH)\n\n    /*\n    Backwards-compatible macros for older versions\n    */\n\n    #if BOTAN_VERSION_CODE < BOTAN_VERSION_CODE_FOR(3,9,0)\n\n    // Macro is always missing <3.9, so we just define it as the current version code\n    #define BOTAN_HAS_KDF BOTAN_VERSION_CODE\n\n    // Alternative: Set it if if any *algorithm* is supported\n    // However, the KDF feature still technically exists and is supported,\n    // even if no algorithms are present\n    #if defined(BOTAN_HAS_KDF1) || defined(BOTAN_HAS_KDF2) || defined(BOTAN_HAS_HKDF) || ...\n    #define BOTAN_HAS_KDF BOTAN_VERSION_CODE // Or min code of all algorithms\n    #endif\n\n    #endif // End of < 3.9\n\n    /*\n    Convert undefined macros as 0\n    */\n\n    // We have to do this, for *every support macro*...\n    #ifndef BOTAN_HAS_KDF\n    #define BOTAN_HAS_KDF 0\n    #endif\n\n    // Alternative:\n    // Additionally convert any `> 0` to `1` so we can import as `Bool` instead of as a `CInt`\n    #if defined(BOTAN_HAS_KDF) && BOTAN_HAS_KDF > 0\n    #define BOTAN_HAS_KDF 1\n    #else\n    #define BOTAN_HAS_KDF 0\n    #endif\n\n    #endif // END OF BOTAN_MACRO_SHIM_H\n\n\n\n_THIS_ seems to be working… instead of importing the support macros from `build.h`, I can import them from `shim.hpp` instead, where they have already been fixed to avoid any missing symbols!\n\nOf course, now I have to do this for some 300-ish support macros - oooooh boy. It is going to take a little while because I can’t just auto this, I have to test each and every one. Oh, and its not documented when each macro was added!\n\n# Testing multiple versions\n\nIf I have to run through a bunch of different versions testing things to see when they were added, I’m going to make it easier for everyone.\n\nYou need 2 things to automate switching from a global `botan` install to a local one:\n\n  * A local `botan` installation of the requisite version to point to\n  * A `cabal.project.local` to do the pointing\n\n\n\nHitch in the plans - Botan requires that you set up local installs using `./configure.py`, but also this is where and how you do the pointing. What to do, what to do…\n\nSo anyway, I started scripting.\n\nLong story short, I have a glorious bash script that clones the randombit repo, checks out all official versions, builds all of them, and generates a `cabal.project.local` for each of them that can be automatically copied to the root. Now I can test any of the 12 versions almost painlessly - it still requires exiting GHC and re-building, but that’s way better than what it was like before.\n\nI’ll need to tidy up the script but this should make local builds easier for everyone, by allowing anyone to use any version or distribution of botan that they need - and the best part is, I needed to do this anyway to support `WASM`, which _requires_ a local build, so building for web is that much closer!\n\n# Back to work\n\nNow I begin the tedious task of doing this for every support macro - all 300 of them. But it will be worth it to be able to build against any 3.x version, not just the default distribution but even ones that enable and disable various features (which, is required for WASM!).\n\nThat’s it for today’s update!",
  "title": "Botan bindings devlog"
}