17. July 2022

Running a shell command for each entry in a PCollection

I was preparing for something I needed to do at work with beam, namely running a shell command for something that isn’t possible to run natively in the beam runtime, and did not find much documentation for it. It’s relatively straight-forward but I have not “blogged” in a while so here goes.


22. May 2022

Transferring Musical Likes Between Services

After using spotify for many years, I decided to try switching my music service a few months ago. I was finding that any time I put on shuffle, spotify would eventually direct me back to funk and soul music from the seventies. I generally love those styles of music, but for a service that is supposed to provide discovery, it wasn’t really working for me.


03. May 2022

Doing some biological modeling

At my new job our focus is to uncover scientific insights to accelerate and improve scientific research. I’m a data engineer by trade, but the science side of the business bleeds in to what I do on a fairly regular basis. Without the scientific and research background that many of my peers have, I feel the need to do some self improvement in those areas. So I was happy to find a series of videos on youtube titled Computational Biology with Python (Modeling Gene Networks) created by Mike Saint Antoine