{
  "$type": "com.whtwnd.blog.entry",
  "theme": "github-light",
  "title": "Learning curve of a complex system",
  "content": "# Learning curve of a complex system\n\nAlternative title:\n\n# A rant on CMake by an inexperienced programmer\n\nA while ago, I wrote an article about writing an add-on for fcitx5. I chose\nCMake for the build system because that was what the tutorial article I was\nfollowing used. So when I decided to make one of the module (Input method\nengine) into a library, I used CMake. The problem was that an add-on was a\nshared library. I have to link the IME library to the shared library, and CMake\nrefuses to spit out the correct flags for ld to link the libraries properly.\n\nI set IME to a shared library, and seems like a shared library cannot be linked\nto a shared library. After searching deep in Stack Overflow and CMake forums, I\nfinally learned that I have to set `-DCMAKE_POSITION_INDEPENDENT_CODE=ON` and\nmake IME a static library.\n\nMy thought during the hour of struggle was: I hate CMake. But when I think\ndeeper, it is not really a CMake problem.\n\n## When we are forced to use complex systems\n\nThere are times that we are forced to use systems we aren't familiar with, be\nit CMake, ffmpeg[^1] or React[^2], it feels exceptionally difficult when we\ndon't know the tool enough. Remember the early days of learning to program,\neverything doesn't seem to make sense. You keep banging your head against the\nwall, trying to resolve all the errors you encounter.\n\nThere were a few times that I simply wanted to concatenate a few video segments\ninto one. I don't have video editing applications installed, and ffmpeg was all\nI had. Concatenating videos is a frequently asked question on the internet,\nexcept that video segments have different resolutions and aspect ratios, and\nsome of them even doesn't have an audio channel. I couldn't find someone\nalready asking the question with such specific situation. ChatGPT couldn't spit\nout the correct command[^3]. So I spent half an hour staring at the red error\nmessages, trying different combinations of options and arguments, until I\nfinally got it working[^4].\n\nI don't want to learn ffmpeg. There are simply too many options and filters to\nmemorize. I still couldn't figure out how to correctly map channels to refer\nto them later.\n\nThe number of features of a system scales with its complexity. Simple tools\nlike wc only have a few flags, and more complicated ones like sed has so\nmany commands that you probably would only use `s//` and `//d`. Not to mention\nthat the man page of ffmpeg have 2800 lines on a 81 character wide terminal\nwindow (250 lines for sed).\n\n## Do I need all these features?\n\nThere was a time that I wrote my Makefile to build my project. It has its own\nway to build libraries, handle dependencies, look for headers, etc. As long as\nI follow a certain convention to create modules and setup vendor libraries,\neverything just works when I type `make` in the terminal. It is probably more\ndifficult to write a Makefile than CMakeFile when there are target dependencies\nand second expansions of variables, but I would argue my Makefile is more easy\nto use than CMake. I wrote the Makefile once and used it for 2 years.\n\nOf course there are lots of things that my Makefile cannot do. It cannot\nautomatically search for installed libraries. It cannot clone a GitHub repo to\nuse as a dependency. It cannot install additional files to specific locations.\nIt cannot produce a library.\n\nBecause I didn't need these features.\n\nTry to remember how often you install a complex tool just to use one feature.\nOr you bundled the whole universe into your JavaScript project just to use one\nfunction. Most of the time we don't fully utilize the tools we use. Why should\nwe install things that we don't use?\n\nThe so-called \"UNIX Philosophy\" have several rules. It went through a view\nrevisions when it is summarized by different people at different times. All\nthose stuff isn't really important in this case. The idea I want to focus on\nis:\n\n> Write programs that do one thing and do it well.\n\nUncle Bob wrote in the book \"Clean Code\":\n\n> Functions should do one thing. They should do it well. They should do it\n> only.\n\nWhat is \"one thing\"?\n\n## It is an abstraction problem\n\nThere is really no correct answer to this. I like to think about it in terms of\nabstraction. Building a project is \"one thing\". Compiling a library is \"one\nthing\". Compiling a module in the library is \"one thing\". Compiling a file is\n\"one thing\". Parsing a file is \"one thing\". Programs and functions are different\nforms of abstractions anyways.\n\nWhen your goal is just to \"build the project\", CMake may be a good abstraction\nbecause it handles all the configuration, compilation and linking for you\n(given that you have written the CMakeFile correctly). However when you are\nthinking one level deeper, you can notice that there are at least 3 things\nCMake is doing. And when you think one more level deeper, the compiler (gcc or\nclang) is doing a whole bunch of things to compile your project.\n\nCompiling is one thing because I don't want to care about parsing the language,\nbuilding the AST, performing analysis, optimizing, spitting out assembly, etc.\nConfiguration is one thing because I don't know what it does. Linking is one\nthing because yet again I don't know how it works. I don't care how\nconfiguration and linking work because 99% of the time it just works, and I\nexpect them to work given the long history of the software.\n\nIt is all about the level of abstraction you care about.\n\n## I don't want to learn the thing\n\nIn the case of CMake, I don't really want to learn the tool because\n\n1. I am not using the tool daily for every C/C++ project\n2. It _seems_ too complicated\n3. I am not interested in the tool\n4. I just want to get the job done\n\nRemember, I use CMake because that is what the tutorial told me to use, and it\nworked well until I demanded more from the build system. My naive understanding\nof CMake caused me to think that isolating IME as a standalone CMake project is\na good choice. I've seen and heard people building libraries with CMake. It is\nthe classic \"surely it won't be that hard\" pitfall.\n\nBut then the problem is, how much should we learn about the tool we use?\n\nFor git of course you need to know how to commit changes, how branches work,\nhow to merge branches and how to resolve conflicts. If you are constantly\ndealing with JSON in the terminal, you may probably want to learn jq to get out\ndata, do simple operations on it, and use them to construct new JSON. If you\nare using JavaScript, I'd expect you to at least heard of the word \"event loop\"\nbecause it is such a fundamental concept in JS.\n\nHow about CMake?\n\nI know I have to create a build directory, cd into it, call `cmake ..` with\nsome options, and then run `make`[^5]. I also know I can\nspecify release build with `-DCMAKE_BUILD_TYPE=Release`. For the CMakeFile, I\nknow some basic structure, adding dependencies, setting target, and adding\nsource files. That is all I know about CMake. These knowledge can usually help\nme set up a working build system. We always learn just enough for us to do the\njob.\n\nI believe it applies to most people on Earth unless you are really really\ninterested in the piece of tool. Therefore tools should only contain just the\nset of features to serve its purpose. Of course there are always edge cases\nthat a tool needs to cover. I think tools should either generalize the problem\nto also cover the edge cases, meaning that the same simple interface should\nautomatically handle all those cases, or straight up not supporting it.\n\nBuilding projects are complicated. Imagine those multimillion dollar Silicon\nValley companies having an entire team dedicated just to build the thing. I\nrecognize that it is impossible to generalize \"building\" with a simple\ninterface. But the companies simply have their own build system. Is CMake being\ntoo greedy to cover all different situations to serve everyone's needs? Is it\nworth to provide that much feature at the cost of making the learning curve\nsteeper. How many times have you encountered a problem, you knew CMake can\nsolve it, it is just hidden behind some unknown feature?\n\n## I don't actually know what I am looking for\n\nProbably the biggest reason why I spent an hour looking for the solution is\nbecause I don't even know what I am looking for. I wanted to link a library to\nanother library, combining them as one. I thought it is a straightforward thing\nto do. The error `ld` gave at the last stage of building just confuses me.\n\nAs a normal programmer would do, I copied the error message, which was trivial\n`ld` complaining not being able to resolve symbols, added the word CMake, and\npasted to Google search. No working results. I tweaked the search query and\ncontinued searching.\n\nObviously I didn't know that I cannot just link a shared library to another\nshared library. Most of us don't know what they are looking for when they\nencounter an error message first time because we don't know what contributes to\nthe error. In my case I don't know setting IME module to shared library\ncontributes to the error, so I kept searching without mentioning it.\n\nI know this is how we learn in programming. It just sucks because I expected\nCMake, the de facto standard of build tool in C/C++ land[^6], cannot handle\nthis situation out of the box.\n\n## Conclusion\n\nIt is all about inexperience and skill issue. I know, I know. I remember the\nearly days of me learning pointer, serializing the raw pointer into a binary\nfile and wondering why I can't get the object back from the file. I can't keep\nbut wonder, why are complex tools so difficult to learn?\n\n[^1]: ffmpeg should have been a GUI application.\n[^2]: Not a web dev.\n[^3]: It is impossible to get ffmpeg working properly.\n[^4]: Or sometimes straight up giving up.\n[^5]: Who uses ninja?\n[^6]: CMake is also known for the hatred towards it.\n",
  "createdAt": "2024-11-23T08:47:59.121Z",
  "visibility": "public"
}