I learned recently that EKS Clusters will always grant admin privileges to the IAM entity that created the cluster: When you create an Amazon EKS cluster, the AWS Identity and Access Management (IAM) entity user or role, such as a federated user that creates the cluster, is automatically granted system:masters permissions in the cluster’s role-based access control (RBAC) configuration in the Amazon EKS control plane. This IAM entity doesn’t appear in any visible configuration, so make sure to keep track of which IAM entity originally created the cluster. Read more...

I just spent many hours over the course of several days figuring out how to set up a VPN between AWS and Digital Ocean with strongSwan. I had to piece together various blog and forum posts to get this to work. Why might you want a Site-to-Site VPN between Digital Ocean and AWS? First, let’s go over why a VPN connection between AWS and Digital Ocean might be useful for you. Read more...

Table of Contents Background Our Stack GitHub Actions Exporter Making Metrics Useful with GHA Supervisor Alerting on Builds that could use Smaller Runners Monitoring the GitHub Actions Queue Synthetic Tests Background I led the migration from Jenkins to GitHub Actions (GHA) at Venmo. We were required to run CI/CD pipelines on our infrastructure. One pain point I had with GitHub Actions self-hosted runners was the complete lack of ability to monitor anything related to GHA on the runners. Read more...

Volumes and bind mounts let you share files between the host machine and container so that you can persist data even after the container is stopped. If you’re running Docker on Linux, you have a third option: tmpfs mounts. When you create a container with a tmpfs mount, the container can create files outside the container’s writable layer. As opposed to volumes and bind mounts, a tmpfs mount is temporary, and only persisted in the host memory. Read more...

The remediation of the recent log4j security vulnerability has solidified an idea in my mind. Services and infrastructure must be regularly deployed to be considered healthy long-term. I’ve read through several Reddit threads where people in companies with a strong devops culture had an easier time doing the dependency update and deploy as opposed to others. The issue people tend to have were updating services that have had infrequent and/or manual deploys. Read more...

Hey everybody, thanks for signing up for my email list. I’m going to try to send something out at least weekly. This first newsletter I want to highlight a very useful Kubernetes feature that I started leveraging recently: port forwarding. You can bind a port from a kubernetes cluster to your development machine you can hit private endpoints on your kubernetes cluster. In the past week I’ve used it to access private endpoints and leverage a datadog agent that was on a cluster but not on my local development machine. Read more...

Table of Contents Background Gotchas with Actions on GitHub Enterprise Server Caching isn’t available GitHub Enterprise Server is behind GitHub Enterprise Cloud Using Public GitHub.com Actions Dockerhub pull rate limiting General Limitations No dropdowns for manually triggered jobs Self-hosted runner default labels You can’t restart a single job of a workflow Slow log output You can’t have actions that call other actions Metrics and observability Workflow YAML Syntax can confusing Closing Background I’ve been leading a team at my dayjob to migrate every engineering team to GitHub Actions (GHA) Read more...

Say you’re a developer working on a feature, and you want to get early feedback from other teammates. How would you go about it? Most companies don’t have a good process for this, and it ends up crippling their ability to ship features and iterate. The process at companies without review app environments: An engineer works on a UI feature that she wants to get some early feedback on how it looks and feels. Read more...

The latest version of macOS (Catalina) includes zsh by default instead of bash. I’ve been using zsh for many years and along the way I’ve started using a handful of tools that make me more productive in my terminal. asdf (version manager) asdf is a plugin-based version manager. For example, instead of using nvm to manage node versions and rvm to manage ruby versions, you can use asdf to manage everything in one tool. Read more...

When you first start learning Clojure, you’ll eventually come across the loop/recur construct that enables you to essentially do tail recursion. Why does Clojure need to have loop/recur when you can simply use a function and call it again? Let’s say you’re writing a tail-recursive (iterative) function that calculates the nth fibonacci number. Since we’re recursing from the tail position, what’s the difference between calling the function by its name: Read more...