Container

Be careful making thread-aware syscalls in Go: lock the thread

Oct 20, 2025 · 8 min read · Go Container Linux ·

Share on:

I recently shipped a bug that caused around 0.5% of container workloads to fail to start for one internal customer. It is an interesting mix of Linux namespaces, Go concurrency, and making syscalls in Go. This post walks through the bug and its fix. The need to run a program in its own network and mount namespace …

Mysterious Image Pull Failures: "401 Unauthorized" and "Not Found" After Migrating Containerd to v2

Oct 12, 2025 · 7 min read · container AWS ·

Share on:

Early this year, we migrated containerd from v1.7 to v2.0.5. However, we quickly noticed image pulls from Amazon Elastic Container Registry (ECR) began failing for both public and private ECR repositories. For example: 1# public ECR 2FATA[0031] failed to resolve reference …

x509: certificate signed by unknown authority? Maybe the cert pool is empty

Apr 15, 2025 · 6 min read · Linux Container SELinux Bottlerocket ·

Share on:

I recently worked on getting amazon-ssm-agent to run inside containers on Bottlerocket. During that process, I ran into a TLS issue connecting to amazonaws.com. The root cause turned out be interesting and we'll walk through it in this post. Running amazon-ssm-agent in a container: why and how? To enable sessions …

Missing Container Disk I/O Stats with cgroup v1 on Kernel 6.1

Nov 9, 2024 · 4 min read · Linux Container ·

Share on:

As Amazon Linux 2 (AL2) approaches its End of Life on June 30, 2025, we have started migrating our container platform from AL2 to Bottlerocket. The migration encountered a few speed bumps. In this post, we'll examine one of them: missing container disk I/O stats. Why are container I/O dashboards blank? Since …