Koodihommia

No recap of lessons learned this year. Instead, here is a collection of some of the best music from 2021. Proud to say that I succeeded in my goal of listening to more Finnish records! Shoutout to Radio Helsinki and its feature programs for keeping me on that path. Albums and EPs Handshaking - Garden Clogs Seeing Fedja Kamari perform live was one of the most memorable experiences this summer. The song "Näin kai taas lankesin" even in a formidable seven-and-a-half minute length, is a masterwork in tension-building and incidentally my most listened tune of the year. Richard Dawson, Circle - Henki Dawsons falsetto started to catch my ear while listening to the Finnish indie Spotify list. Slowly it began to grow on me. The rough, unpolished sound of the Circle with its unbridled energy makes me grin. Ichiko Aoba - Windswept Adan The soothing voice of Aoba's gives me the chills. Her impressive upper range combined with a dreamy flute produces a powerful emotional experience. T...

The Advent of Code in Elixir continues to be a humbling experience year after year. One of my weaknesses in programming puzzles and, in general, is a lack of intuition on which common collection operations to use. Or, to be more precise, since I don't know them intimately, I don't know when to apply them in code. It is way too often I get feedback on a code review on  why do you do this thing with a plain map or reduce while you could use this and that helper function . Often, one can replace a custom function with a more expressive composition of well-known "stock" functions. I think a recap is in order. I enumerate in this post the most useful ones for me - or at least those I get the most feedback from. These are indeed useful even in the basic CRUD app!  I took the names from Scala and Elixir standard libraries and checked that functions with similar names exist in Lodash and Ramda JS external libraries Chunk/Group/Split/Partition Make a list of lists based on a f...

I don't know much about photography and much less about image formats. The little I know comes mostly from web development. There I use images as semi-static assets, and I know how to work around the potential problems.  That experience was of limited use when I worked on end-user-created images on a React Native app I'm working on. I needed a crash course into JPEGs.  We use react-native-camera to take photos. Snapping one has an elusive option called photo quality . It is an integer in a range between zero and one, and in the example code, it was set to 0.5. Zero point five . What does that mean? Is that good? 0.5 does not feel that good. To me, a modern iPhone is more than 0.5? The default value is 1 though, is that better? I wanted to find out what turning that mystery knob entails. Spoiler alert: the JPEG quality is clearly explained in the JPEG Wikipedia article, but obviously, I needed to jump through some hoops to understand that. I want to go through some generic th...

I'm a fan of Kent C. Dodds. He has an excellent blog , he holds office hours on Youtube  and he manages an active Discord community focused on helping developers make maintainable React apps. I especially appreciate the way he thinks about state management in React apps. It does not have to be so reliant on external libraries. React itself is a library for managing state, so why not depend on its built-in abstractions. On a React Native app I'm building, most (> 95%) of the state management is indeed done with the React primitives: useStates, Contexts, and Refs. It is a big pile of hook compositions, and that's the way I like it. If you are familiar with React and can read its documentation, you can understand the code. There are a handful of places where using the primitives fall short, though. In these spots, plain old React hooks would require excessive glue code and would ultimately be confusing. Also, skipping needless renders in components such as forms can be tric...

I write a lot of stupid and useless unit tests. I confess that I'm aiming towards the elusive 100% test coverage. Sure, I'll never get there, and I realize it is not even a desirable goal. After an assessed threshold, we start to get diminishing returns. Why do I keep writing wrong tests then? Which unit tests should I leave out? My case is that it is tricky to write a large codebase with only valuable tests. I dare say that some (minority) of the essential tests might have outright bad practices where they end up testing how well I can mock stuff. The value of the test is not predetermined; hence I keep writing a lot of seemingly poor ones. There is an argument that high test coverage does not strongly reduce bugs and increase maintainability. That is likely true. Nonetheless, it is not so apparent which tests will avoid the most bugs or are the most useful in refactoring in the long term. The cliché is that requirements change, sometimes towards unknown directions.  It is mor...

VS code feels slow. All of a sudden, saving took forever in a TS project. The first suspect was a recently added WallabyJS test runner extension. So I disabled all plugins but still, saving took several seconds. My head then turned towards TS. Arrogantly I blamed some dependency but I had not a clue which one. Github TS wiki has some tips and tricks on what to do, including instructions to run a profiler. Turns out, a new profiling tool was introduced not too long ago in TS 4.1.  The output looks like this. Compiling the slowest source file takes more than 7 seconds! There is one massively slow checkSourceFile call which, as the name says, compiles a single file. To my embarrassment, it was in my code 😱 The profiling output is rather hard to decipher. The UI shows which TS internal functions are called and references a separate JSON file containing the type names. In my case, the slow function(s) included some weird single character type names, so it was not much help. The ...

I tried out the Encore go backend framework. I had no particular project to use it for; it was more that I wanted to do some go programming. Here are my two cents about it. Encore uses a simple package-based structure to build simplistic services. It has the usual niceties with hot code reload, minimal boilerplate, easy authentication management, etc. An endpoint is defined by an annotated function with a set of defined parameters. Encore provides also a runtime platform with one command deploy (git push actually) which is rather cool. Perhaps the most opinionated feature to me was the transparent integration to PostgreSQL. If a service has a migration file with some DB calls, the run or the deploy command automatically creates a database for the service and runs the migrations. The cloud console is pretty neat Would I use Encore in an actual project? Maybe not. Firstly, it's still in beta. If I'd start a plain old rest project with Postgres, sure, it takes away the boilerplat...

SerenityOS is a cool project to implement a 90's style retro operating system. I ran into it when some commenter in Hacker news recommended watching the  Andreas Klings Youtube series  on the development.  I've been obsessively watching his content for months now. In the videos, Andreas screencasts his work on the various features he is implementing. They are a fascinating watch. He is a master software craftsman. There is plenty to learn from his sessions, even if you would not be interested in C++ or system development. I've been doing minor hacks here and there, mainly to the UI components, since they are the easiest to approach for me. I'm not planning to contribute much to the project, but I think I'll keep watching the videos and reading the Discord channel actively.  I have almost completed one feature, though. I extended the screenshot utility to capture gifs as well. I found an open-source library that creates gifs out of PNGs. It sorta works; the colors a...

I created a simple "about me" website in the most ridiculous possible way I could imagine.  https://jompanakumpana.fi/ I did it with kaboom.js , a simple library by  replit . It's best explained by just trying it out. Trust me, it is a fun way to spend an afternoon or two!

In an app I'm building, I use SVGs in certain list items. The SVGs are custom status icons and thus are pretty simple. They contain trivial path and circle elements, have one color without any fancy gradients. The list feels a bit janky though, especially on my crappiest Android test device. My intuition was already when I added the SVGs that this might happen. Android is not the most performant with vector graphics, especially when using react-native-svg. I did some profiling with the Flipper mobile app debugging tool. My gut feeling was confirmed. The component which is the most prominently visible in the flame graph is the SVG, being almost ten times slower to render than the text components.  Performance of the SVG component Since the SVGs are so simple, the solution to this performance problem is to convert the set to a font. We use a ton of icon fonts, and they are almost as fast to render as "normal" text. I used IcoMoon for the conversion. As expected, IcoMoon c...

A couple of podcasts I have listened to recently have flamed Java. It is a good "kusitolppa", a seriously uncool technology.  I'm pleased about not having to write Java anymore. Yet, I'll risk career suicide and take a stance to defend it against some of the low blows. After all, it was my bread and butter for quite many years. I get it; writing Java is not in fashion. It is likely a wrong choice for a modern startup. The language is full of mysterious cargo-cultish traditions, is verbose, has annotations on top of annotations and you have to use an editor which eats half of your memory (the other half is used by the Java process you are working on). Traditional Java and JDKs are ill-fitted for short-running processes like lambdas; it needs time to warm to get up to speed and needs more than one core to work efficiently. It has horrible dependency management tools by today's standards. Running a JVM process requires all kinds of weird flags, its defaults are unlik...

React native 0.64 by default has "inline requires"  enabled. We can now easily squeeze some milliseconds out of the app startup time. Here are a couple of examples of how to do it with React Navigation . If some navigation paths are behind feature flags, you can import them conditionally at the navigator level, skipping potentially unused code to be loaded on startup while preventing users' access to unreleased features. You can also create lazy loading screens that will require the actual implementation only when it has received a navigation event.

We used a Google "distroless" image as the base image on a go application deployment in a project a while ago. I never looked into what distroless really means, but I had a guess. I'm also curious to look under the hood of a docker image if they really are just merged archives. Recently there was a "Docker without docker" blog post in Hacker news about how simple docker (or rather, OCI) image format is. Spoiler: an image is tar archives on top of tar archives. The post has also a detailed explanation of how to pull images from a container registry. I shamelessly took the image pull script they shared and modified it a bit to pull the static distroless image from the Google container registry. The image is only one layer, and here is how it looks like after I extracted it and removed all the root level empty directories with all directories with at least one file expanded. It looks pretty empty to me, and I suppose that's th...

I learned from my brilliant colleagues a nice trick that left me once again wondering why on earth I discover these things only now. It's funny to run into something literally everyone else knows, and somehow, I miss. Say you have a file that you want to download on some trusted server. You may have a version of the file downloaded, but it may be somehow corrupted or partially downloaded. You could verify the file integrity by comparing the local and remote file hashes, and that is indeed a proper way to do it.  If you trust the remote, you can use plain old HTTPS to get the headers only with a HEAD request and get the content length. Just compare the result to local, and there you have it, a naive yet adequate method to check if you need to redownload a file. Note that this does not validate that the file contents are equal. 

Of all the horrific code I have written, by far, the most harmful and destructive have involved too early abstraction. Everyone knows the issues stemming from the wrong abstraction. Even copy-paste code would have been a better choice than creating some seemingly fancy, say, adapter pattern implementation. To be honest, I think copy-paste has a worse reputation than it should. A given project might have started with best practices, module structure, libraries of a prior project/16k star primer. That can be good! Some best practices naturally transcend projects. We should bring in the lessons learned but the timing matters. Upon a closer look, the project might indeed have agreeable abstractions. Still, those could be hollow and bring only extra LOCs and solve the issues faced in the previous projects, which have never occurred in the current one. Upon recently starting two soon-to-be large projects, I learned to leave some wisdom from previous lives behind. While deep in the coding zen...

A common thing I see in an enterprise system is that when an end-user does some action, say add a user, the underlying web of subsystems adds the user to multiple databases in separate transactions. Each of these transactions may happen in varying order and, even worse, can fail, leaving the system in an inconsistent state. A better way could be to write the user data to some main database and then other subsystems like search indexes, pull/push the data to other interested parties, thus eliminating the need for multiple end-user originating boundary transactions. That's the theory part; how about a technical solution. The idea of this post came from the koodia pinnan alla podcast about event-driven systems and CDC . One of the discussion topics in the show is emitting events from Postgres transaction logs.  I built an utterly simple change emitter and reader using Postgres with the wal2json transaction decoding plugin and a custom go event parser. I'll stick to the boring ...

Virtually all of the audio plugin development is done in low-level languages like C++ to keep the illusion of real-time as close to being true as possible. Recently though, ROLI released a new programming language for audio processing. It's called  SOUL , a less convoluted way of designing audio processing software. I took a look at it and in no time, created this simple detuned sinewave synth. Sure, it is pretty much based on the examples but it took only a couple of hours to implement. If I would have done this in C++, a week would not have been enough. SOUL provides abstractions for inputs and outputs, lots of ready-made functions for waveforms, phases, and time-to-frequency domain conversions. It's pretty new though and examples, editor support, and proper documentation is still lacking. I plan to keep an eye on the development and will return with some more demanding audio processing later on.  You can try the example on the browser by copy-pasting it to the browser play...

Back in the day, everyone was using Winamp. It is a music player with the user interface of a mixing console. The bloody thing had an equalizer on the front page! As an amateur music producer, I know that EQ is a powerful tool but making changes that sound good is difficult, to say the least. Mixing engineers spend considerable effort with the artist to balance the frequency ranges to arrive at the desired musical outcome. I bet the 15-year-old me butchered a lot of songs with the thing. Now we are using Spotify, a player with basically a search bar and a play button. I ran into something called Ousterhout's Law on the  Operating Systems: Three Easy Pieces book . Here is a quote from the book TIP: AVOID VOO-DOO CONSTANTS (OUSTERHOUT’S LAW) Avoiding voo-doo constants is a good idea whenever possible. Unfortunately, as in the example above, it is often difficult. One could try to make the system learn a good value, but that too is not straightforward. The frequent result: a configura...

The previous blog post about the memory structure left me thinking about where the memory is allocated. Why did one of the variables stay in the stack and one go to the heap?  I stumbled upon this wonderful presentation on the very subject. Turns out that the go compiler can tell me where the variables are allocated. You just need to give it a couple of GC flags. Let's take another look at the example program I used in the blog post. Looks like everything escapes to the heap when building with my mac! It most likely has something to do with the println() debug command taking a interface as its argument instead of a known type. Source: https://www.youtube.com/watch?v=ZMZpH4yT7M0

In a previous blog post, I took a look at how to enumerate all the syscalls and even their arguments using tools such as  eBPF . That left me pondering and craving to learn more about how memory is mapped and what do simple variables look like in the memory. What is behind all those memory addresses you can see in the stack traces? I do have an intuitive sense of that. Sure, I have seen blog posts and talks about the topic, taken a look at heap dumps in a hunt for memory leaks but I wonder does it make any sense to look at the memory in a language/runtime agnostic manner. Probably not, but hey, could be exciting. To find out, I created a simple program that simply prints out the contents of a few variables  I try to make the outputs depend on the runtime environment to avoid any unexpected compiler optimizations. I want to make sure the memory will be allocated at runtime. I run the code in my trusty Digitalocean VM with "no hang-up" and attach the GNU Project debugger (GDB) ...

 I built a simple MIDI looper with the Teensy 3.2 development board  It's a simple device. It has two MIDI inputs, one serving as a THRU port, mostly for forwarding sync messages to the connected device. The other input port is for connecting the instrument. The output port binds the loop for sending and receiving notes. The first version soldered to a stripboard. The left and center MIDI connectors are the inputs connected to the Teensy (header pins visible) through  an optocoupler circuit Teensy is the perfect fit for this kind of project, mostly due to its tiny dimensions. It works with the same development setup as Arduino, and virtually all libraries that work with Arduino work with Teensy. It even includes a MIDI library, which came in handy. The code is trivial; the gist is that we read the input tempo, start and stop messages from the THRU port and merge the tempo message with incoming note data. The note data quantizes to the nearest 8th sync message. Up to date...

Besides the object geometry information, IFC files may contain properties for the IFC objects. The properties can be, for example, some predefined dimension information such as an object volume or a choice of material. Some of the properties are predefined in the IFC standards, but custom ones can be added. IFC files can be massive and resource-intensive to process, so in some cases, it helps to separate the object properties from the geometry data. IfcOpenShell  is a toolset for processing IFC files. It is written mostly in C++ but also provides a Python interface. To read an IFC file >>> ifc_file = ifcopenshell.open("model.ifc") Fetch all objects of type IfcSlab >>> slab = ifc_file.by_type("IfcSlab")[1] Get the list of properties >>> slab.IsDefinedBy (#145075=IfcRelDefinesByType('2_fok0__fAcBZmMlQcYwie',#1,$,$,(#27,#59),#145074), #145140=IfcRelDefinesByProperties('3U2LyORgXC2f_hWf6I16C1',#1,$,$,(#27,#59),#145141), #145142...