Gotchas of defer in Go

Sep 10, 2023 · 6 min read · Go ·

Share on:

A defer statement invokes a function just before the surrounding function returns. Multiple defers within a function are executed in reverse order of their calls, Last In First Out (LIFO). Defer is commonly used to ensure resource cleanup, regardless of the function's success or failure. For instance,

 1func (h *Handler) Handle(ctx context.Context) {
 2  // Use defer to catch and log panics. This prevents web server crash.
 3  defer func() {
 4    if r := recover(); r != nil {
 5      fmt.Println("Recovered", r)
 6    }
 7  }()
 8
 9  h.mu.Lock()
10  // Use defer to release lock, no matter what happens to Handle().
11  defer h.mu.Unlock()
12
13  // ...
14}

However, defer has nuances that could lead to tricky bugs. Those bugs are difficult to catch, partly because defer executes code in the top-to-bottom program order as programmers read them. In this post, we'll look at a few gothcas.

Do not defer in loop

The following code executes multiple SQL queries. It might seem that rows.Close() is executed after each query. However, rows.Close() is only executed after completion of all queries. That is because defer executes just before the surrounding function returns, not when the surrounding blocks exits.

 1func foo(db *sql.DB, n int) {
 2  for i := 0; i < n; i++ {
 3    fmt.Printf("query %d, ", i)
 4    rows, err := db.Query("SELECT 1;")
 5    if err != nil {
 6      fmt.Println("Error querying the database:", err)
 7      return
 8    } else {
 9      fmt.Println("success")
10    }
11    defer rows.Close()
12
13    // Use rows ....
14  }
15}

To observe the bug in action, download full code here and follow steps below.

Step 1. Start a local PostgreSQL server and limit the server to allow a maximum of four connections.

1docker run --rm --name=pg1 -p 15432:5432 -e POSTGRES_PASSWORD=password postgres -c max_connections=4

Step 2. Query PostgreSQL five times using the foo method above.

 1func main() {
 2  connStr := "postgres://postgres:password@localhost:15432/postgres?sslmode=disable"
 3  db, err := sql.Open("postgres", connStr)
 4  if err != nil {
 5    fmt.Println("Error opening database connection:", err)
 6    return
 7  }
 8  defer db.Close()
 9
10  foo(db, 5)
11}

The following output shows that the first four queries succeeded but the fifth query failed because the we run out of postgres connections.

1query 0, success
2query 1, success
3query 2, success
4query 3, success
5query 4, Error querying the database: pq: sorry, too many clients already

To fix, we need to wrap the defer statements in a function. The gocritic linter detects the defer-in-loop bug.

 1func fooFixed(db *sql.DB, n int) {
 2  for i := 0; i < n; i++ {
 3    func() {  // create an anonymous function.
 4      fmt.Printf("query %d, ", i)
 5      rows, err := db.Query("SELECT 1;")
 6      if err != nil {
 7        fmt.Println("Error querying the database:", err)
 8        return
 9      } else {
10        fmt.Println("success")
11      }
12      defer rows.Close()
13    }() // and immediately call it.
14  }
15}

Function arguments and method receivers are evaluated immediately.

Given the function foo below, can you predict the line printed when calling foo(10)?

1func foo(i int) (err error) {
2  defer handleError(err)
3  err = fmt.Errorf("error kind %d", i)
4  return err
5}
6
7func handleError(err error) {
8  log.Println(err)
9}

The line printed is "Nil". This is because the arguments to the deferred function are evaluated at the time of the defer statement, not when the deferred function is executed. To fix, use closure to capture arguments.

1func fooFixed(i int) (err error) {
2  defer func() { // capture value using closure
3    handleError(err)
4  }()
5  err = fmt.Errorf("error kind %d", i)
6  return err
7}

Method receivers in defer statements are also evaluated at the time of defer statement. For example.

 1func foo() {
 2  c := Cat{name: "Tomcat"}
 3  defer c.Hi()  // output: "Hello Tomcat" because `c` is evaluated and its name is "Tomcat" at the evaluation time.
 4  defer c.Bye() // output: "Bye Garfield" because `&c` is evaluated. 
 5  c.name = "Garfield"
 6}
 7
 8type Cat struct {
 9  name string
10}
11
12func (c Cat) Hi() {
13  fmt.Printf("Hello %s\n", c.name)
14}
15
16func (c *Cat) Bye() { // Note the receiver is pointer type. 
17  fmt.Printf("Bye %s\n", c.name)
18}

Defer run on panics but not on os.Exit()

Whether the panic is a run-time panic (e.g., divide-by-zero) or an explicit call of panic(), the goroutine stops executing remaining statements, proceeds to execute deferred statements, and finally returns control to the caller. On the other hand, os.Exit() terminates the entire process, skipping deferred statements. Fatal logs like log.Fatal use os.Exit, so they skip deferred statement. Defer and panic are goroutine level mechanisms, while os.Exit is a process level mechanism. The gocritic linter detects the defer-after-exit bug.

 1func foo(i int) {
 2  defer fmt.Println("foo() in defer")
 3
 4  switch i {
 5  case 0:
 6    os.Exit(0)
 7  case 1:
 8    log.Fatal("fatal")
 9  default:
10    panic(i)
11  }
12}

Handle errors from defer

The deferred function may encounter errors. Since defer function cannot return value, the only way to effectively return values is through named returns. Can you spot the bug in the following code?

 1func main() {
 2  filename := "./a.txt"
 3  f, err := os.Open(filename)
 4  if err != nil {
 5    fmt.Println(err)
 6    return
 7  }
 8  fmt.Printf("file %s opened successfully\n", filename)
 9  err = writeHelloWorld(f)
10  if err != nil {
11    fmt.Printf("write failed %v\n", err)
12  } else {
13    fmt.Println(`successfully write "hello world" to file`)
14  }
15}
16
17func writeHelloWorld(f *os.File) (err error) {
18  defer func() {
19    err = f.Close() // The err here overrides the err from f.WriteString
20  }()
21
22  _, err = f.WriteString("hello world!\n")
23  return err
24}

If we create an empty file a.txt and run the above code, the output is

1file ./a.txt opened successfully
2successfully write "hello world" to file

It might seem that "hello world!" was successfully written to the file. However, the file remains empty. Why does this happen? This is because os.Open opens the file in read only mode, causing WriteString to fail with the error bad file descriptor. However, this error is overridden by the nil error within the defer.

How do we reconcile errors that occur outside of defer with errors inside defer? It depends on the use case. I typically give priority to errors occurring outside defer. For example,

 1func writeHelloWorldFixed(f *os.File) (err error) {
 2  defer func() {
 3    errCloseFile := f.Close()
 4    if err == nil && errCloseFile != nil {
 5      err = errCloseFile
 6    } else if err != nil && errCloseFile != nil {
 7      fmt.Printf("failed to close file %v", errCloseFile)
 8    }
 9  }()
10
11  _, err = f.WriteString("hello world!\n")
12  return err
13}

Gotchas of defer in Go

Do not defer in loop

Function arguments and method receivers are evaluated immediately.

Defer run on panics but not on os.Exit()

Handle errors from defer

Further reading