13. October 2022

Safe Navigation in Python

In a chain of method calls, what happens when one of the calls returns a null? The next chained call will throw some kind of reference error. But what if you don’t want to deal with a reference error? What if you want the null value to be the answer? That’s where a safe navigation operator is useful.

more

21. September 2022

A Tale From Microservice Hell

Ages ago I was working as a contractor on a data engineering team, working on a knowledge graph platform that was built on a service architecture.

more

17. July 2022

Running a shell command for each entry in a PCollection

I was preparing for something I needed to do at work with beam, namely running a shell command for something that isn’t possible to run natively in the beam runtime, and did not find much documentation for it. It’s relatively straight-forward but I have not “blogged” in a while so here goes.

more

22. May 2022

Transferring Musical Likes Between Services

After using spotify for many years, I decided to try switching my music service a few months ago. I was finding that any time I put on shuffle, spotify would eventually direct me back to funk and soul music from the seventies. I generally love those styles of music, but for a service that is supposed to provide discovery, it wasn’t really working for me.

more

03. May 2022

Doing some biological modeling

At my new job our focus is to uncover scientific insights to accelerate and improve scientific research. I’m a data engineer by trade, but the science side of the business bleeds in to what I do on a fairly regular basis. Without the scientific and research background that many of my peers have, I feel the need to do some self improvement in those areas. So I was happy to find a series of videos on youtube titled Computational Biology with Python (Modeling Gene Networks) created by Mike Saint Antoine

more

12. April 2022

Diabolical Answers In Wordle

I’ve blogged several times about wordle before. It’s a great game for someone like me who loves words. I don’t think I’m a great wordle player, but I love thinking about the intricacies of the game. One bugaboo in wordle, which I’ll call “diabolical answers” is when you reach a point where you have four letters correct and there are many possible answers. The worst of these would have more than six possible words, meaning you aren’t guaranteed a win even though you’re almost done.

more

14. March 2022

Messing Around In Beam, π Day, And More Wordle

Apache Beam is a distributed programming framework, mostly designed as a counterpart to the DataFlow service in Google Cloud Platform. In the past I’ve done a fair bit of work on pipelines in Spark and with a service architecture, but I’ll be needing Beam for my new job, so Ive been playing around a bit with that.

more