Te-Yuan Liu
Home Blog
Go
May 15, 2026 · 5916 words · 30 mins read

Go is a statically typed, compiled programming language. It has fast compilation and concurrency support via goroutines and channels. It uses a garbage collector to manage the heap memory.

Here we Go

File

A Go file contains expressions and statements to perform some data processing.

We use snake_case for Go file names because of compatibility across case-sensitive OS like Linux and case-insensitive OS like Windows and MacOS. Also, the Go compiler uses underscores for special file suffixes to control the build process. For example, the Go compiler sees *_test.go as test file and *_linux.go as file that can only be compiled on Linux.

We use kebab-case for compiled binaries to be consistent with other shell tools.

Package

A package is a collection of Go files inside the same directory. Variables, constants, functions, and types defined under the same package are visible across all Go files in the package.

We use lowercase single word for package names so the user service package becomes userservice.

Module

A module is a collection of packages. It is like a project or repository. A module contains all packages inside its root directory (the directory that has the go.mod), including those in the subdirectories, except any subdirectory that contains another go.mod, which therefore defines another module.

Module initialization

We use go init mod <MODULE_PATH> to initialize a module where <MODULE_PATH> is the path to our module on a repository platform like GitHub. We usually use a GitHub repository to host a module so the module path is github.com/<ORGANIZATION>/<REPOSITORY>. The module path also defines the import path prefix for all packages within the module.

Running the command generates the go.mod file that stores the module path and the Go version we are using.

go.mod 

module <MODULE_PATH>
go <VERSION>

First Go program

We can create our first Go program by making a here_we_go.go file. Later on during the compile process, the Go compiler first converts this Go file into an assembly-like internal representation and then compiles the internal representation into a binary.

In order to let the compile succeed, there must be a Go file (here_we_go.go in our case here) that not only belongs to the main package but also includes a main function such that the compiler can locate the entrypoint of the binary.

here_we_go.go 
// Declare this file as part of the main package.
package main

// Import the built-in fmt package for input/output.
import (
    "fmt"
)

// Declare the main function in the main package.
func main() {
    // Print a line saying "Here we Go!".
    fmt.Println("Here we Go!")
}

Expression and statement

A Go file consists of expressions and statements for the computer to read, parse, and execute instructions. Here is a comparison between the expression and statement.

\ExpressionStatement
PurposeTo produce dataTo execute an instruction
Value returningYesNo
Examples5; a + b; len(array)x = 5; if ... else ...; for ...; return value; import "fmt"; i++

Compile and run

We can use the go build command to instruct the Go compiler to compile the working directory's Go files into a binary.

# Generate a binary named after the working directory.
go build

# Generate a binary with the specified name.
go build -o <BINARY_NAME>

And then we can run the produced binary.

# Run the binary, assuming its name is here-we-go.
./here-we-go

Furthermore, we can combine the compile and run into one command go run.

# Generate an ephemeral binary go-buildxxx inside the temporary directory /tmp, running it, and deletes the binary after execution.
go run here_we_go.go

# This also works as long as the working directory contains the Go file that includes the main package's main function.
go run .

If we want to install the binary so we can run it anywhere, export the Go install path and install the binary.

# Locate the Go install path.
go list -f '{{.Target}}'

# Export the path.
export PATH=$PATH:<GO_INSTALL_PATH>

# Compile and install the binary to the Go install path.
go install

Panic stack trace

If a panic occurs and crashes the program, Go prints the top of the call stack first, and then all the way to the program entrypoint, following a reverse chronological order (error -> main). The concept behind it is giving the most important log for debugging first.

Variable

Variable declaration

The var statement declares a list of variables. It can be at package or function level.

Go uses the var <VARIABLE_NAME> <VARIABLE_TYPE> declaration format for variables. This makes variable declaration easier to understand when comparing with C, especially for pointers.

// Declare an integer pointer p in C.
int *p

// Declare an integer pointer p in Go.
var p *int

Variables declared without an initial value are given their zero value.

We can declare a variable via the explicit way or its shorter version. Note that the shorter version is only available within a function so we must use the explicit way at the package level.

The shorter version is more commonly used because it is more concise. When using the shorter version, Go can infer the variable type automatically from its initial value. When the variable initial value is an untyped numeric constant, the variable type is decided by Go based on the precision of the constant.

// Explicit declaration
var i int = 10  // The int type here can be omitted as Go can infer it from the initial value.

// Shorter version
i := 10

Variable naming convention

The Go community has the naming convention of PascalCase for exported variables, functions, and camelCase for unexported variables, functions. For example, Pi is exported from the math package and can be accessed via math.Pi.

Primitive variable type

Variable type conversion

We can use the T(v) expression to convert a variable v to the type T.

// Explicit declaration
var i int = 2
var f float64 = float64(i)

// Shorter version
i := 2
f := float64(i)

String

String concatenation

Array

An array is a contiguous chunk of memory that has a fixed length, so the length is part of its type.

If the compiler thinks that an array is only accessed within a specific scope and its memory consumption is small (less than 64KB), it is stored on the stack and freed once out-of-scope.

Otherwise, it is allocated on the heap, which is managed by Go's garbage collector. The Go garbage collector reclaims memory when it sees that an object is no longer reachable by any active part of the program.

Array declaration

// Declare an integer array with 2 in length. By default the elements are set to 0.
var a [2]int

// Declare an integer array with an array literal.
var a = [2]int{1, 2}

// Shorter version
a := [2]int{1, 2}

// Declare an integer array with an array literal with length-inferring.
var a = [...]int{1, 2}

// Shorter version
a := [...]int{1, 2}

Slice

A slice is dynamically-sized, flexible view into the elements of an array. When created, it is allocated on the stack.

It contains:

Because a slice is lightweight and super cheap to copy, we usually pass it around by value.

If any slice still holds a pointer to its underlying array, the entire array remains in memory. Therefore, the way to release the array's memory is to set the slice to nil.

If our slice is using a small portion of the underlying array, we can use the below methods to cut down memory cost.

Slice declaration

A slice literal is like an array literal without the length. During the slice literal declaration, Go creates the underlying array and builds a slice that references it.

We can use the built-in println function to learn a slice's information.

// Declare an integer slice. By default its pointer = nil and length = capacity = 0.
var s []int

// Declare via a slice literal.
var s = []int{1, 2}

// Shorter version
s := []int{1, 2}

// Declare a slice with make so it has 2 in length and 4 in capacity.
// Under the hood, it creates an array with 4 in length and sets the slice's pointer to point to the array's first element.
s = make([]int, 2, 4)

// Shorter version
s := make([]int, 2, 4)

// Print the slice.
println(s)          // Print [2/4]0xa00001a120, where 2 for length, 4 for capacity, 0xa00001a120 for the pointer.
fmt.Println(len(s)) // Print 2 for length
fmt.Println(cap(s)) // Print 4 for capacity

// Declare a slice from an existing array
var a [2]int

// The following statements are equivalent.
s := a[:]
s := a[0:]
s := a[:2]
s := a[0:2]

Slice appending

We can append one or multiple elements or even another slice to the end of a slice. If the slice's capacity is exceeded, meaning the underlying array doesn't have the space to accommodate all elements, Go creates another array (usually doubling the array length), copying the elements into it, and updates the slice to use the new array.

If we know the exact capacity needed, or even just an estimate, we can preallocate the slice capacity and save the work for multiple array recreations.

s1 := []int{1}          // s1 = {1}
s1 = append(s1, 2, 3)   // s1 = {1, 2, 3}
s2 := []int{4}          // s2 = {4}
s1 = append(s1, s2...)  // s1 = {1, 2, 3, 4}

Byte slice generation

String slice concatenation

Heap

We can use the container/heap package to implement a min heap. It requires us to implement the Len, Less, Swap, Push, and Pop function for the container/heap package to use.

type IntHeap []int

func (h IntHeap) Len() int {
    return len(h)
}

func (h IntHeap) Less(i, j int) bool {
    return h[i] < h[j]
}

func (h IntHeap) Swap(i, j int) {
    h[i], h[j] = h[j], h[i]
}

func (h *IntHeap) Push(x any) {
    *h = append(*h, x.(int))
}

func (h *IntHeap) Pop() any {
    n := len(*h)
    last := (*h)[n - 1]
    *h = (*h)[: n - 1]
    return last
}

func main() {
    h := &IntHeap{}
    heap.Init(h)
    heap.Push(h, 1)
    s := heap.Pop(h)
}

Map

A map is used to store key-value pairs. Its zero value is nil and has no keys or the ability to accept new keys.

Map declaration

// Declare an integer-to-string map. This is meaningless because it's a nil map and cannot accept new keys.
var m map[int]string

// Declare via a map literal.
var m = map[int]string{1: "one"}

// Shorter version
m := map[int]string{1: "one"}

// Declare a map with make.
m := make(map[int]string)

When using a map literal to declare a map, we can omit the top-level value type if it is a type name.

type Vertex struct {
    X, Y int
}

// Explicit value type
m := map[string]Vertex{
    "Go": Vertex{1, 2},
}

// Shorter version
m := map[string]Vertex{
    "Go": {1, 2},
}

Map operation

// Insert or update a key-value pair.
m[key] = value

// Retrieve the value of a key.
value := m[key]

// Delete a key-value pair
delete(m, key)

// Test a key's existence.
value, ok := m[key]
// If the key is in m, ok is true.
// Otherwise, the value is the value type's zero value, and ok is false.

Pointer

A pointer is a variable that stores a memory address, usually the memory address of another variable. With a pointer, we can work on a large struct directly inside different functions without the need to copy and pass the object around.

Address operator

We can use the address operator & to get the memory address of a variable.

i := 2
p := &i

Dereference operator

We can use the dereference operator * to access the pointed-to variable.

i := 2
p := &i
*p = 4  // Now i has value 4.

Constant

Constant declaration

Go uses the const <CONSTANT_NAME> <CONSTANT_TYPE> declaration format for constants. Note that we cannot use the shorter version := to declare a constant.

Numeric constants are high-precision values. An untyped constant takes the type needed by its context.

Print

We often use the Print, Println, and Printf function from the built-in fmt package.

Flow control

Loop

For loop

A for loop has 3 components separated by semicolons.

  1. An initial statement is executed before the first iteration.
  2. A condition expression is evaluated before every iteration.
  3. A post statement is executed at the end of every iteration.
for i := 0; i < 10; i++ {
    fmt.Println(i)
}

i := 0
for i < 10 {
    fmt.Println(i)
    i++
}

// Declare a for loop that runs forever.
for {
    fmt.Println("Here we Go!")
}

For-range loop

A for-range loop provides a concise way to iterate over a range, string, array, slice, or map. The basic syntax assigns iteration values to one or two variables, followed by the range keyword and then the collection.

We can even use this to read a channel. Note that if we want to handle different values received from the channel separately, consider to use the goroutine select.

// Loop over the index.
for index := range collection

// Loop over both the index and value.
for index, value := range collection

// Loop over the value.
for _, value := range collection

// Loop over the value received from a channel.
for value := range channel
Collection typeFirst valueSecond value (optional)
IntegerIndex (int)Not applicable
StringIndex (int)Rune (Unicode code point)
Array or SliceIndex (int)Element copy
MapKeyValue copy
ChannelElementNot applicable

If and else

An if statement has 2 components separated by a semicolon.

  1. An initial statement is executed first. Its variables stay in the scope of the if and else.
  2. A condition expression is then evaluated.
if check := true; check == true {
    fmt.Println("Check is true.")
} else if check == false {
    fmt.Println("Check is false.")
} else {
    fmt.Println("Panic: check is not boolean.")
}

Switch

A switch statement is a shorter way for flow control. It evaluates cases from top to bottom, and runs the first case whose value is equal to the condition expression.

import (
    "fmt"
    "runtime"
)

func main() {
    fmt.Print("Go runs on ")
    switch os := runtime.GOOS; os {
    case "linux":
        fmt.Println("Linux.")
    case "darwin":
        fmt.Println("macOS.")
    case "windows":
        fmt.Println("Windows.")
    default:
        fmt.Printf("%s.\n", os)
    }
}

Switch without condition is the same as switch true and is a clean way to write long if-then-else chains.

Defer

A defer statement evaluates its function arguments immediately but postpones the function execution until the defer statement's surrounding function returns.

Deferred function calls are pushed onto a stack and follows the Last-In-First-Out (LIFO) execution order.

Function

Function declaration

Go uses the func <FUNCTION_NAME>(<FUNCTION_PARAMETER>) <RETURN_TYPE> declaration format for functions. This is also easier to understand comparing with C.

// Declare a function f that takes an integer parameter and returns an integer in C.
int f(int p)

// Declare a function f that takes an integer parameter and returns an integer in Go.
func f(p int) int

Function parameter

When two or more consecutive named function parameters share a type, we can omit the type from all but the last.

// Declare a function add that takes two integer parameters and returns an integer.
func add(x int, y int) int

// Shorter version
func add(x, y int) int

Function return value

When a function returns a value, it copies the value from the variable within the function and passes it to the outer world.

func main() {
    value := returnValue()
	fmt.Printf("value stored at %p\n", &value) // memory address 2
}

func returnValue() bool {
	value := true
	fmt.Printf("value stored at %p\n", &value) // memory address 1
	return value
}

The same holds for pointer returning. But note that value now lives on the heap instead of stack for access across different frames on the call stack.

func main() {
	pointer := returnPointer()
	fmt.Printf("pointer pointing to %p\n", pointer) // memory address 1
	fmt.Printf("pointer stored at %p\n", &pointer)  // memory address 3
}

func returnPointer() *bool {
	value := true
	pointer := &value
	fmt.Printf("pointer pointing to %p\n", pointer) // memory address 1
	fmt.Printf("pointer stored at %p\n", &pointer) // memory address 2
	return pointer
}

A function usually returns a value when the returned value is small or immutable. On the other hand, it returns a pointer when the returned value is large, to avoid costly copying, or a resource like API client, database connection, or a file, that shouldn't be copied.

A function's return values may be named. If so, they are treated as variables defined at the top of the function. These return value names should be used to document the meaning of the return values. A return statement without arguments returns the named return values.

We generally don't use named return values as they can decrease readability in longer functions.

func getXY() (x, y int) {
    x = 1
    y = 2
    return
}

Function value

Functions are values and can be passed around just like other values.

func compute(fn func(float64, float64) float64) float64 {
    return fn(3, 4)
}

func main() {
    hypot := func(x, y float64) float64 {
        return math.Sqrt(x * x + y * y)
    }
    fmt.Println(compute(hypot))
}

Function closure

A closure is a function value that references variables from outside its body. Each closure is bound to its own external variables.

// fibonacciFactory is a function that returns
// a fibonacci function, which is a closure, that returns
// the next number in the Fibonacci series.
func fibonacciFactory() func() int {
    i, pp, p := -1, 0, 1
    fibonacci := func() int {
        i += 1
        if i == 0 {
            return 0
        } else if i == 1 {
            return 1
        } else {
            c := pp + p
            pp = p
            p = c
            return c
        }
    }
    return fibonacci
}

func main() {
    f := fibonacciFactory()
    for i := 0; i < 10; i++ {
        fmt.Println(f())
    }
}

Method

A method is a function with a special receiver argument. We use the receiver argument to attach the function to a type, usually a struct. Note that the receiver type can't be a pointer and must be defined in the same package so we cannot declare a method with a receiver whose type is int.

There are 2 kinds of receivers, the value receiver and the pointer receiver.

Value receiver method

Pointer receiver method

Generic function

A generic function can handle the same parameter of different types using type parameters.

// Declare a function that takes in a T-typed slice `s` and a T-typed variable `x`. The T type can be any type that fulfills the built-in comparable constraint.
// The comparable constraint ensures that we can use a comparison operator like `==` on values of that type.
func getIndex[T comparable](s []T, x T) int

Function error handling

Go addresses function error with an explicit and imperative approach. In general, a function is expected to return a pair of values. The first one is the result value and the other one is the error value. Go expects us to check and handle the error using if err != nil {} for every function call.

Here are some tips for function error handling tailored for the code structure in this post.

Type

Type declaration

Go uses the type <TYPE_NAME> <UNDERLYING_TYPE> declaration format for types.

Struct

A struct is a collection of fields (variables).

Struct declaration

A struct literal denotes a newly allocated struct value by listing the values of its fields.

type Vertex struct {
    X int
    Y int
}

func main() {
    v1 := Vertex{}      // By default X = 0 and Y = 0
    v2 := Vertex{1, 2}  // X = 1 and Y = 2
    v3 := Vertex{X: 3}  // X = 3 and Y = 0
}

Struct field access

Struct fields are accessible via a dot. If we have a struct pointer, using a dot does the dereferencing automatically.

type Vertex struct {
    X int
    Y int
}

func main() {
    v := Vertex{1, 2}
    v.X = 3
    p := &Vertex{4, 5}
    p.Y = 6 // Same as (*p).Y = 6
}

Empty struct type

Empty struct type, struct{}, is a type that takes zero bytes of memory, and is the smallest possible data type in Go. Its type value is struct{}{}.

It allows creating a set from a map with empty struct value type, or a channel purely for signaling.

Interface

An Interface defines a set of method signatures for other types to implement, achieving polymorphism (flexibility). An interface value can hold any concrete type value as long as that concrete type implements those methods.

Under the hood, an interface value is a header that contains a type pointer and a data pointer. The type pointer points to a concrete type, and the data pointer points to a value of that concrete type.

Calling a method on an interface value effectively executes the same-named method of its concrete type value. If the interface value's data pointer is nil, calling the method will result in a nil pointer dereference runtime error. Therefore, it's a good practice to write code to gracefully handle nil receiver method call.

type I interface {
    M()
}

func describe(i I) {
    fmt.Printf("(%v, %T)\n", i, i)
}

type T1 struct {
    S string
}

func (t T1) M() {
    fmt.Println(t.S)
}

type T2 struct {
    S string
}

func (t *T2) M() {
    fmt.Println(t.S)
}

func main() {
    var i I
    describe(i) // (nil, nil)
    i.M()       // nil pointer dereference runtime error

    i = T1{"Here we Go!"}
    describe(i) // ({"Here we Go!"}, main.T1)
    i.M()       // "Here we Go!"

    i = &T2{"Here we Go!"}
    describe(i) // (&{"Here we Go!"}, *main.T2)
    i.M()       // "Here we Go!"
}

Empty interface type

An interface type that specifies no methods is an empty interface type, interface{}, or any. An empty interface type value, interface{}{}, or any{}, can hold any concrete type value and is used to handle the concrete type value that is unknown at compile time but figured out at runtime.

When we use an any type parameter in a function, Go has to box it and place it on the heap, pressuring Garbage Collection (GC).

Heavier GC load causes higher response latency because GC needs longer execution freezes to clean up the heap memory.

If possible, we should change to use generic type parameters because the types can be resolved at compile time, leading to zero heap allocation per function call.

Interface concrete type assertion

An interface concrete type assertion checks if the interface value holds a value of the specified concrete type. If so, it returns the concrete type value and a true ok value. Otherwise, it returns the zero value of the specified concrete type and a false ok value.

Interface type switch

An interface type switch is a switch statement that uses types as cases, rather than values as cases.

func do(i any) {
    switch v := i.(type) {
    case int:
        fmt.Printf("Twice %v is %v\n", v, v*2)
    case string:
        fmt.Printf("%q is %v bytes long\n", v, len(v))
    default:
        fmt.Printf("I don't know about type %T!\n", v)
    }
}

Error interface

The built-in error type is an interface that requires an Error() string method. we can use errors.New() or fmt.Errorf() to quickly build an error value, or we can develop a custom error type.

When implementing the method for a custom error type, we should use fmt.Sprintf on the error value's internal fields or the type-converted error value, rather than passing the error value back into the fmt function. This makes sure we don't create a recursion that leads to infinite looping.

Any function call, including arithmetic operation, file read/write, and network request, may fail unexpectedly. Therefore, Go makes functions return two values: the result, and an error. This forces the caller to explicitly handle the potential failure immediately.

type MyError struct {
    When time.Time
    What string
}

func (e *MyError) Error() string {
    return fmt.Sprintf("at %v, %s",
        e.When, e.What)
}

func run() error {
    return &MyError{
        time.Now(),
        "it didn't work",
    }
}

func main() {
    if err := run(); err != nil {
        fmt.Println(err)
    }
}

IO Reader interface

The io.Reader interface has a Read method that populates the given byte slice with data and returns the number of bytes populated and an error value. It returns an io.EOF error when the stream ends.

func (T) Read(b []byte) (n int, err error)

Image interface

type Image interface {
    ColorModel() color.Model
    Bounds() Rectangle
    At(x, y int) color.Color
}

Generic type

A generic value can hold any type value.

// List represents a singly-linked list that holds any type value.
type List[T any] struct {
    next *List[T]
    val  T
}

Concurrency

Goroutine

A goroutine is a lightweight (2 KB memory) thread managed by the Go runtime, whereas an OS thread (8 MB memory) is managed by the OS.

Given a small number of OS threads (usually the number of logical cores), the Go runtime can schedule thousands of goroutines to run on these few threads.

If a goroutine blocks because of waiting for I/O, the Go runtime scheduler swaps another runnable goroutine onto the thread.

We can start running a function f(x) inside a goroutine using go f(x), The evaluation of f and x happens in the current goroutine and the execution of f(x) happens in the new goroutine.

When a goroutine panics, it crashes the entire process.

Channel

Goroutines shared the same process memory so memory access must be synchronized. This is why we use channels.

A channel is where goroutines synchronize data with each other.

Channel declaration

We can use make to declare a channel.

ch := make(chan int)

Channel operation

We can send and receive elements through a channel with the channel operator <-.

If we have one sender and that sender has no more elements to send, then it should call close() to shutdown the channel.

If we have multiple senders, we should use a coordinator goroutine to wait for all senders to finish via sync.WaitGroup and then close the channel.

When the passed-in function of sync.WaitGroup.Go() panics, it does not call sync.WaitGroup.Done() but re-panics to prevent the main goroutine from proceeding.

Only a single sender or a coordinator goroutine should close the channel because sending to a closed channel causes a panic but receiving from a closed channel doesn't cause a panic.

A receiver can test whether a channel has been closed by assigning a second parameter ok to the receive expression. If ok is false then the channel is closed.

A receiver can use a for-range loop for i := range c to receive elements from the c channel repeatedly until it is closed. Note that this blocks the receiver goroutine until the channel is closed. This is one of the occasions where we need to close the channel to let the the receiver goroutine be garbage collected. Other than this, we usually don't need to close a channel.

ch <- y         // Send the value of y to channel ch.
x := <-ch       // Receive an element from channel ch and use it to create x.
close(ch)       // A sender closes the ch channel.
z, ok := <-ch   // A receiver checks if the ch channel is closed via the ok variable.

Channel buffer

A buffer is a channel's internal queue. By default the buffer length is 0, so a send blocks a goroutine until the element it's sending is received by another goroutine. Same for the read, a read blocks a goroutine until another goroutine sends an element to the channel for it to receive.

We can use the second argument of make to set the buffer length, and use cap(ch) for total buffer size (capacity), len(ch) for current buffered element count.

A send doesn't block a goroutine if the number of elements in the channel is less than the buffer length. Otherwise, the send blocks the goroutine. A read works in a similar way.

Generally, we don't want the sender to be blocked by a send so we will set the buffer capacity to the total number of elements or at least an estimate.

Goroutine worker pool

Although goroutine is cheap, it is not free. Creating one goroutine per job can grow the number of concurrent goroutines indefinitely. Therefore, we often adopt the worker pool pattern to limit the number of goroutines.

We use a coordinator goroutine to close the out channel and thus unblock the main function.

type Result struct {
    JobID int
    WorkerID int
    Value int
    Error error
}

func main() {
    in := make(chan int, 100)
    out := make(chan Result, cap(in))

    numWorkers := runtime.NumCPU() // Use CPU count for CPU-bound job and file descriptor count for IO-bound job.
    var wg sync.WaitGroup
    for workerID := range numWorkers {
        wg.Go(func() {
            worker(workerID, in, out)
        })
    }

    for n := range 100 {
        in <- n
    }
    close(in)

    go func() {
        wg.Wait()
        close(out)
    }()

    for o := range out {
        if o.Error != nil {
            fmt.Println(o.Error.Error())
        }
        fmt.Println(o.Value)
    }
    fmt.Println("finished!")
}

func worker(workerID int, in <-chan int, out chan<- int) {
    for i := range in {
        if i % 2 == 0 {
            out <- Result{
                JobID: i,
                WorkerID: workerID,
                Value: i * i,
                Error: nil,
            }
        } else {
            out <- Result{
                JobID: i,
                WorkerID: workerID,
                Error: errors.New("remainder is 1"),
            }
        }
    }
}

Goroutine pipeline

We can create a pipeline with goroutines.

func process1(in <-chan int) <-chan int {
    out := make(chan int, cap(in))
    go func () {
        defer close(out)
        for num := range in {
            out <- num
        }
    }()
    return out
}

func process2(in <-chan int) <-chan int {
    out := make(chan int, cap(in))
    go func () {
        defer close(out)
        for num := range in {
            out <- num
        }
    }()
    return out
}

out1 := process1(in)
out2 := process2(out1)

Goroutine select

Goroutine select is used for cases like signal handling alongside receiving values from a channel. The select statement pauses the current goroutine until it can execute one of its communication cases. If multiple are ready, it randomly choose one of them to execute.

We can specify a default case for it to send or receive without blocking when other cases are not ready yet.

If the select statement is in the main function, we can label the outer loop and break that loop when the result channel is closed.

func main() {
    ctx, cancel := context.WithTimeout(context.Background(), 4 * time.Second)
	defer cancel()

    results := make(chan int)
    var wg sync.WaitGroup

    for i := range 3 {
        wg.Go(func() {
            multiplyBy10(i, ctx, results)
        })
    }

    go func() {
        wg.Wait()
        close(results)
    }()

    for result := results {
        fmt.Printf("Received result: %d\n", result)
    }
}

func multiplyBy10(i int, ctx context.Context, results chan int) {
    select {
    case <- time.After(2 * time.Second):
        results <- i * 10
    case <- ctx.Done():
        fmt.Printf("Stopped due to context cancellation: %v", ctx.Err())
    default:
        time.Sleep(500 * time.Millisecond)
    }
}

Goroutine profiler

Go's built-in profiler pprof can start a http server and let we inspect how many goroutines are running and which line of code started them. We can set it up with the below steps.

  1. Add import _ "net/http/pprof".
  2. Register the handler function to an existing HTTP ServeMux with http.HandleFunc("/debug/pprof/", pprof.Index). Otherwise, run it in a side server with go func() {http.ListenAndServe("localhost:6060", nil)}().
  3. Visit http://localhost:6060/debug/pprof/goroutine?debug=1.

Mutex

We use Mutual-exclusion (Mutex) to ensure the exclusive access of a variable by one goroutine at a time. Go provides the sync.Mutex type and its methods, Lock, and Unlock, to achieve this.

var mu sync.Mutex

go func() {
    mu.Lock()
    defer mu.Unlock()
    process()
}()

Mutex deadlock

A Mutex deadlock is a situation that each one of the two goroutines is waiting for the resource held by the other one.

var (
    mu1 sync.Mutex
    mu2 sync.Mutex
)

go func() {
    mu1.Lock()
    defer mu1.Unlock()
    process()
    mu2.Lock()
    defer mu2.Unlock()
}

go func() {
    mu2.Lock()
    defer mu2.Unlock()
    process()
    mu1.Lock()
    defer mu1.Unlock()
}

One way to prevent the deadlock from happening is to establish a global lock ordering. If every goroutine has to lock mu1 first and then mu2, no deadlock would happen.

Mutex blocking

When our code makes an I/O call and writes the result to a shared resource via mutex, we should only lock the mutex during the write, such that other goroutines are not blocked by the I/O call.

Semaphore

A semaphore is a signaling counter that allows a limited number of goroutines to access a resource. It is useful for rate-limiting API calls or database connections.

Mutex is like a semaphore with size one.

type Semaphore chan struct{}

func NewSemaphore(n int) Semaphore {
    return make(Semaphore, n)
}

func (s Semaphore) Acquire() {
    s <- struct{}{}
}

func (s Semaphore) Release() {
    <-s
}

func main() {
    numConnection := 10
    s := NewSemaphore(numConnection)

    var wg sync.WaitGroup
    for i := range numConnection {
        wg.Go(func() {
            s.Acquire()
            defer s.Release()
            worker(i)
        })
    }
    go func() {
        wg.Wait()
        close(out)
    }()
}

Context

Contexts propagate request-scoped data, cancellation and timeout signals across service calls, preventing resource leaks and allowing for graceful shutdowns.

If we use one of the context with cause types like context.WithCancelCause(), we use context.Cause() for context cancel visibility, otherwise we use ctx.Err() given the context ctx. However, we till have to be aware of nested timeouts and the complexity brought by cross-service propagation.

func work() {
    ctx, cancel := context.WithTimeout(context.Background(), 2 * time.Second) // The timer of ctx starts ticking here.
    defer cancel()

    select {
    case <- time.after(1 * time.Second):
        fmt.Println("Finished work successfully.")
    case <- ctx.Done():
        fmt.Println("Timed out:", ctx.Err())
    }
}

Code structure

The goal of this code structure is flexibility with minimum abstraction. It divides the system into 3 layers, handler, service, and infrastructure.

Here is an application for user registration, login, and logout.

/cmd
    /main
        main.go
/internal
    /server
        /rest.go
    /domain
        /user.go
        /port.go
    /database
        /postgresql.go

Here are the 3 layers.

At last, cmd/main/main.go creates infrastructure interface values, service structs, and then the handler, injecting dependencies, and starts the handler.

Dependency management

Logging

Configuration

Validation

Dependency injection

Command Line Interface (CLI)

Database

Networking

Testing

Downloading remote modules

In each Go file, we use the import keyword to import packages from both local module and remote modules. To download a remote module and record its version in our go.mod. We use the go mod tidy command to add the missing module and also remove unneeded modules.

If two different modules have packages with the same name, we have to use alias for at least one of the package to avoid conflicting import.

import (
    "example.com/module1/x"
    m2x "example.com/module2/x" // Alias example.com/module2/x to m2x.
)

Removing all downloaded modules

Use go clean -modcache to remove all downloaded modules.

Memory management

Common memory bugs

Production best practice

Error handling

Asynchronous processing

Memory

Adoption challenge

References

←Previous Next→
Copyright © Te-Yuan Liu