Koodihommia

Stable diffusion magic created with DreamStudio This autumn, the Wunderdog Wundernut programming puzzle includes figuring out why the repository has a sand-colored PNG and which Harry Potter character to submit on a form. My submission to the previous puzzle was not very high-brow. I'm not saying my solution to the new one is more elegant, but at least I think I used a proper (read boring) tool. On the upside, I got to learn a new cipher type (affine) and use an OCR library for the first time ever! https://github.com/jjylik/wd-notrot13

I recently read the popular post about Redis architecture . It had a chapter about forking - the process of how they create a disk backup of the in-memory database contents. The chapter addresses some same system programming topics as the linux-insides book I've been glancing over recently. As the article points out, the POSIX fork system call creates a duplicate of the calling process but does not copy the memory pages. If the parent or any child process access a memory page, it points to the read-only shared memory until the process changes any values of that page. The calling process then gets a writable copy of the accessed memory pages (copy-on-write). DALL·E 2 did not know how to draw a go gopher but close enough! Only single-threaded applications support fork , but I still decided to try it out on a go app. Go runtime does not support fork due to go programs being multithreaded and other reasons I don't even try to understand. There is a ForkExec call in the standa...

I haven't soldered, reset any ESP32 chips, nor tinkered with breadboards this summer (yet). Instead, I wrote something spartan for my personal use. A simple tool to create Spotify playlists from Radio Helsinki programs. There is not too much to write home or blog about. I keep doing these small projects to keep the golang-fu up. I spent the most time researching stuff around what I needed to do, for example, how to create session cookies in go with the plain old standard library. If you want to create Spotify playlists from Radio Helsinki programs, visit this scary-looking URL  https://radiohelsinki-to-spotify.apps.jompanakumpana.fi/ Repo:  https://github.com/jjylik/radiohelsinki-to-spotify

I tried out the Encore go backend framework. I had no particular project to use it for; it was more that I wanted to do some go programming. Here are my two cents about it. Encore uses a simple package-based structure to build simplistic services. It has the usual niceties with hot code reload, minimal boilerplate, easy authentication management, etc. An endpoint is defined by an annotated function with a set of defined parameters. Encore provides also a runtime platform with one command deploy (git push actually) which is rather cool. Perhaps the most opinionated feature to me was the transparent integration to PostgreSQL. If a service has a migration file with some DB calls, the run or the deploy command automatically creates a database for the service and runs the migrations. The cloud console is pretty neat Would I use Encore in an actual project? Maybe not. Firstly, it's still in beta. If I'd start a plain old rest project with Postgres, sure, it takes away the boilerplat...

A common thing I see in an enterprise system is that when an end-user does some action, say add a user, the underlying web of subsystems adds the user to multiple databases in separate transactions. Each of these transactions may happen in varying order and, even worse, can fail, leaving the system in an inconsistent state. A better way could be to write the user data to some main database and then other subsystems like search indexes, pull/push the data to other interested parties, thus eliminating the need for multiple end-user originating boundary transactions. That's the theory part; how about a technical solution. The idea of this post came from the koodia pinnan alla podcast about event-driven systems and CDC . One of the discussion topics in the show is emitting events from Postgres transaction logs.  I built an utterly simple change emitter and reader using Postgres with the wal2json transaction decoding plugin and a custom go event parser. I'll stick to the boring ...

The previous blog post about the memory structure left me thinking about where the memory is allocated. Why did one of the variables stay in the stack and one go to the heap?  I stumbled upon this wonderful presentation on the very subject. Turns out that the go compiler can tell me where the variables are allocated. You just need to give it a couple of GC flags. Let's take another look at the example program I used in the blog post. Looks like everything escapes to the heap when building with my mac! It most likely has something to do with the println() debug command taking a interface as its argument instead of a known type. Source: https://www.youtube.com/watch?v=ZMZpH4yT7M0

In a previous blog post, I took a look at how to enumerate all the syscalls and even their arguments using tools such as  eBPF . That left me pondering and craving to learn more about how memory is mapped and what do simple variables look like in the memory. What is behind all those memory addresses you can see in the stack traces? I do have an intuitive sense of that. Sure, I have seen blog posts and talks about the topic, taken a look at heap dumps in a hunt for memory leaks but I wonder does it make any sense to look at the memory in a language/runtime agnostic manner. Probably not, but hey, could be exciting. To find out, I created a simple program that simply prints out the contents of a few variables  I try to make the outputs depend on the runtime environment to avoid any unexpected compiler optimizations. I want to make sure the memory will be allocated at runtime. I run the code in my trusty Digitalocean VM with "no hang-up" and attach the GNU Project debugger (GDB) ...

In my ongoing quest to understand more how GNU Linux works under the hood, I took upon myself a courageous assignment to figure out what a simple go program does from a kernel's point of view - which syscalls are called and what data is passed down there Here is the program So a simple hello world, which prints out the famous words and creates an empty file. To figure out which syscalls are triggered, I decided to use trace-cmd package. It is a frontend to ftrace which helps for example in filtering out the syscalls related to a single binary or a PID. After some intense googling, watching a couple of youtube videos , and fiddling around, I ran this command. sudo trace-cmd record -p function_graph --max-graph-depth 3 -e syscalls -g do_syscall_64 -F ./gohello It records a list of kernel functions practically in the order they are called with max depth and some filtering. It shows only syscalls and not, for example, system interrupts that clutter the output and are not in this exer...

Goroutine is an abstraction of go runtime for parallelism and concurrency. Go on runtime has its scheduler, which distributes goroutines to OS threads. The scheduler is split for each native OS thread scheduler, and each scheduler can steal work from others. I have done quite a lot of actor programming and know a little about different concurrency models. Go's goroutines can be compared to green threads or Erlang VM's processes, although the go community avoids making such comparisons. The best part of go concurrency is that it is not based on relatively inefficient OS threads. Creating a thread and switching between them is an expensive operation both in CPU cycles and memory terms. Goroutines should communicate with each other by messaging and not sharing memory. The Effective Go guide states that  Do not communicate by sharing memory; instead, share memory by communicating.  A Gouroutine can use a message channel to communicate with other goroutines. Sim...