Practical Engineering
open-menu closeme
Engineering
github linkedin rss
  • Be careful making thread-aware syscalls in Go: lock the thread

    calendar Oct 20, 2025 · 8 min read · Go Container Linux  ·
    Share on: twitter copy

    I recently shipped a bug that caused around 0.5% of container workloads to fail to start for one internal customer. It is an interesting mix of Linux namespaces, Go concurrency, and making syscalls in Go. This post walks through the bug and its fix. The need to run a program in its own network and mount namespace …


    Read More
  • Mysterious Image Pull Failures: "401 Unauthorized" and "Not Found" After Migrating Containerd to v2

    calendar Oct 12, 2025 · 7 min read · container AWS  ·
    Share on: twitter copy

    Early this year, we migrated containerd from v1.7 to v2.0.5. However, we quickly noticed image pulls from Amazon Elastic Container Registry (ECR) began failing for both public and private ECR repositories. For example: 1# public ECR 2FATA[0031] failed to resolve reference …


    Read More
  • x509: certificate signed by unknown authority? Maybe the cert pool is empty

    calendar Apr 15, 2025 · 6 min read · Linux Container SELinux Bottlerocket  ·
    Share on: twitter copy

    I recently worked on getting amazon-ssm-agent to run inside containers on Bottlerocket. During that process, I ran into a TLS issue connecting to amazonaws.com. The root cause turned out be interesting and we'll walk through it in this post. Running amazon-ssm-agent in a container: why and how? To enable sessions …


    Read More
  • Missing Container Disk I/O Stats with cgroup v1 on Kernel 6.1

    calendar Nov 9, 2024 · 4 min read · Linux Container  ·
    Share on: twitter copy

    As Amazon Linux 2 (AL2) approaches its End of Life on June 30, 2025, we have started migrating our container platform from AL2 to Bottlerocket. The migration encountered a few speed bumps. In this post, we'll examine one of them: missing container disk I/O stats. Why are container I/O dashboards blank? Since …


    Read More

Peng Zhang

Software Engineer

Recent Posts

  • Be careful making thread-aware syscalls in Go: lock the thread
  • Mysterious Image Pull Failures: "401 Unauthorized" and "Not Found" After Migrating Containerd to v2
  • EC2 IMDS is Unstable During Early Boot: Always Retry
  • Who Modified My Program in Bottlerocket?
  • Introducing bottlerocket-extra-kit: Essential debugging tools for Bottlerocket
  • Tips for Building Bottlerocket AMIs
  • Working Knowledge of Linux Memory: Concepts
  • Detect and fix rare cases where the primary ENI does not serve default traffic

Tags

GO 17 LINUX 15 ALGORITHMS 8 INTERVIEW 7 BOTTLEROCKET 4 CONTAINER 4 GUIDE 3 DISTRIBUTED-SYSTEM 2 SELINUX 2 WEB 2 AWS 1 COMPUTER-ARCHITECTURE 1 CONCURRENCY 1 CRYPTOGRAPHY 1
All Tags
ALGORITHMS8 AWS1 BOTTLEROCKET4 COMPUTER-ARCHITECTURE1 CONCURRENCY1 CONTAINER4 CRYPTOGRAPHY1 DATABASES1 DISTRIBUTED-SYSTEM2 EC21 GO17 GUIDE3 INTERVIEW7 LINUX15 SELINUX2 SHELL1 TESTING1 WEB2
[A~Z][0~9]
Peng Zhang

Copyright 2022-  PENG ZHANG. All Rights Reserved

to-top