Gotchas of defer in Go

Sep 10, 2023 · 6 min read · Go ·

Share on:

A defer statement invokes a function just before the surrounding function returns. Multiple defers within a function are executed in reverse order of their calls, following the Last In First Out (LIFO) principle. Defer is commonly used to ensure resource cleanup, regardless of the function's success or failure. For instance,

 1func (h *Handler) Handle(ctx context.Context) {
 2  // Use defer to catch and log panics. This prevents web server crash.
 3  defer func() {
 4    if r := recover(); r != nil {
 5      fmt.Println("Recovered", r)
 6    }
 7  }()
 8
 9  h.mu.Lock()
10  // Use defer to release lock, no matter what happens to Handle().
11  defer h.mu.Unlock()
12
13  // ...
14}

However, defer has nuances that lead to tricky bugs if not handled carefully. Those bugs are difficult to catch, partly because defer executes code out of the top-to-bottom program order.

Do not defer in loop

The following code executes multiple SQL queries. It might seem that rows.Close() is executed after each query. However, rows.Close() is only executed after completion of all queries. The reason is because defer executes just before the surrounding function returns, not when the surrounding blocks exits.

 1func foo(db *sql.DB, n int) {
 2  for i := 0; i < n; i++ {
 3    fmt.Printf("query %d, ", i)
 4    rows, err := db.Query("SELECT 1;")
 5    if err != nil {
 6      fmt.Println("Error querying the database:", err)
 7      return
 8    } else {
 9      fmt.Println("success")
10    }
11    defer rows.Close()
12
13    // Use rows ....
14  }
15}

To observe the bug in action, follow these steps. You can download full code here.

Step 1. Start a local PostgreSQL server and limit the server to allow a maximum of four connections.

1docker run --rm --name=pg1 -p 15432:5432 -e POSTGRES_PASSWORD=password postgres -c max_connections=4

Step 2. Query PostgreSQL five times using the foo method above.

 1func main() {
 2  connStr := "postgres://postgres:password@localhost:15432/postgres?sslmode=disable"
 3  db, err := sql.Open("postgres", connStr)
 4  if err != nil {
 5    fmt.Println("Error opening database connection:", err)
 6    return
 7  }
 8  defer db.Close()
 9
10  foo(db, 5)
11}

The following output shows that the first four queries succeeded but the fifth query failed because the we run out of postgres connections.

1query 0, success
2query 1, success
3query 2, success
4query 3, success
5query 4, Error querying the database: pq: sorry, too many clients already

The fix is simple: wrap the defer statements in a function. Luckily, the gocritic linter detects the defer-in-loop bug.

 1func fooFixed(db *sql.DB, n int) {
 2  for i := 0; i < n; i++ {
 3    func() {  // create an anonymous function.
 4      fmt.Printf("query %d, ", i)
 5      rows, err := db.Query("SELECT 1;")
 6      if err != nil {
 7        fmt.Println("Error querying the database:", err)
 8        return
 9      } else {
10        fmt.Println("success")
11      }
12      defer rows.Close()
13    }() // and immediately call it.
14  }
15}

Function arguments and method receivers are evaluated immediately.

Given the function foo below, can you predict the line printed when calling foo(10)?

1func foo(i int) (err error) {
2  defer handleError(err)
3  err = fmt.Errorf("error kind %d", i)
4  return err
5}
6
7func handleError(err error) {
8  log.Println(err)
9}

The line printed is "Nil". This is because the arguments to the deferred function are evaluated at the time of the defer statement, not when the deferred function is executed. The fix involves using a closure to capture the arguments.

1func fooFixed(i int) (err error) {
2  defer func() { // capture value using closure
3    handleError(err)
4  }()
5  err = fmt.Errorf("error kind %d", i)
6  return err
7}

Method receivers in defer statements are also evaluated at the time of defer statement. For example.

 1func foo() {
 2  c := Cat{name: "Tomcat"}
 3  defer c.Hi()  // output: "Bye Garfield"
 4  defer c.Bye() // output: "Hello Tomcat"
 5  c.name = "Garfield"
 6}
 7
 8type Cat struct {
 9  name string
10}
11
12func (c Cat) Hi() {
13  fmt.Printf("Hello %s\n", c.name)
14}
15
16func (c *Cat) Bye() { // Note the receiver is pointer type. 
17  fmt.Printf("Bye %s\n", c.name)
18}

Defer run on panics but not on os.Exit()

Whether the panic is a run-time panic (e.g., divide-by-zero) or an explicit panic(), the goroutine stops executing remaining statements, proceeds to execute deferred statements, and finally returns control to the caller. It's important to note that os.Exit() terminates the entire process, skipping deferred statements. Because log.Fatal uses os.Exit, it also skips deferred statement. Defer and panic are goroutine level mechanisms, while os.Exit is a process level mechanism. Luckily, the gocritic linter detects the defer-after-exit bug.

 1func foo(i int) {
 2  defer fmt.Println("foo() in defer")
 3
 4  switch i {
 5  case 0:
 6    os.Exit(0)
 7  case 1:
 8    os.Exit(1)
 9  case 2:
10    log.Fatal("fatal")
11  default:
12    panic(i)
13  }
14}

Handle errors from defer

If the deferred function encounters an error and we want to return the error to the caller, we need to be careful. Explicitly returning a value in a defer statement does not compile. The only way to return a value from a defer is through a named return variable. Can you spot the bug in the following code?

 1func main() {
 2  filename := "./a.txt"
 3  f, err := os.Open(filename)
 4  if err != nil {
 5    fmt.Println(err)
 6    return
 7  }
 8  fmt.Printf("file %s opened successfully\n", filename)
 9  err = writeHelloWorld(f)
10  if err != nil {
11    fmt.Printf("write failed %v\n", err)
12  } else {
13    fmt.Println(`successfully write "hello world" to file`)
14  }
15}
16
17func writeHelloWorld(f *os.File) (err error) {
18  defer func() {
19    err = f.Close() // The err here overrides the err from f.WriteString
20  }()
21
22  _, err = f.WriteString("hello world!\n")
23  return err
24}

If we create an empty file a.txt and run the above code, the output is

1file ./a.txt opened successfully
2successfully write "hello world" to file

It might seem that "hello world!" was successfully written to the file. However, the file remains empty. Why does this happen? This is because os.Open opens the file in read only mode, causing WriteString to fail with the error bad file descriptor. However, this error is overridden by the nil error within the defer.

How do we reconcile errors that occur outside of defer with errors inside defer? It depends on the use case. I typically give priority to errors occurring outside defer. For example,

 1func writeHelloWorldFixed(f *os.File) (err error) {
 2  defer func() {
 3    errCloseFile := f.Close()
 4    if err == nil && errCloseFile != nil {
 5      err = errCloseFile
 6    } else if err != nil && errCloseFile != nil {
 7      fmt.Printf("failed to close file %v", errCloseFile)
 8    }
 9  }()
10
11  _, err = f.WriteString("hello world!\n")
12  return err
13}

Gotchas of defer in Go

Do not defer in loop

Function arguments and method receivers are evaluated immediately.

Defer run on panics but not on os.Exit()

Handle errors from defer

Further reading