Go: Preventing Data Races and Ensuring Thread-Safe Access
What happens if multiple goroutines modify the same data structure? Sadly, nothing good. Let's learn more about it.
Concurrent Modification
So far, our goroutines haven't gotten in each other's way. They've used channels to exchange data, which is safe. But what happens if several goroutines try to access the same object at the same time? Let's find out.
Let's write a program that counts word frequencies:
func main() {
// generate creates 100 words, each 3 letters long,
// and sends them to the channel.
in := generate(100, 3)
var wg sync.WaitGroup
wg.Add(2)
// count reads words from the input channel
// and counts how often each one appears.
count := func(counter map[string]int) {
defer wg.Done()
for word := range in {
counter[word]++
}
}
counter := map[string]int{}
go count(counter)
go count(counter)
wg.Wait()
fmt.Println(counter)
}
fatal error: concurrent map writes
goroutine 1 [sync.WaitGroup.Wait]:
sync.runtime_SemacquireWaitGroup(0x140000021c0?)
goroutine 34 [chan send]:
main.generate.func1()
goroutine 35 [running]:
internal/runtime/maps.fatal({0x104b4039e?, 0x14000038a08?})
goroutine 36 [runnable]:
internal/runtime/maps.newTable(0x104b78340, 0x80, 0x0, 0x0)
What is generate:
// generate creates nWords words, each wordLen letters long,
// and sends them to the channel.
func generate(nWords, wordLen int) <-chan string {
out := make(chan string)
go func() {
defer close(out)
for ; nWords > 0; nWords-- {
out <- randomWord(wordLen)
}
}()
return out
}
// randomWord returns a random word with n letters.
func randomWord(n int) string {
const vowels = "eaiou"
const consonants = "rtnslcdpm"
chars := make([]byte, n)
for i := 0; i < n; i += 2 {
chars[i] = consonants[rand.IntN(len(consonants))]
}
for i := 1; i < n; i += 2 {
chars[i] = vowels[rand.IntN(len(vowels))]
}
return string(chars)
}
generate() generates words and sends them to the in channel. main() creates an empty map called counter and passes it to two count() goroutines. count() reads from the in channel and fills the map with word counts. In the end, counter should contain the frequency of each word.
Let's run it:
map[cec:1 ... nol:2 not:3 ... tut:1]
And once again, just in case:
fatal error: concurrent map writes
goroutine 1 [sync.WaitGroup.Wait]:
sync.runtime_SemacquireWaitGroup(0x140000021c0?)
goroutine 34 [chan send]:
main.generate.func1()
goroutine 35 [running]:
internal/runtime/maps.fatal({0x104b4039e?, 0x14000038a08?})
goroutine 36 [runnable]:
internal/runtime/maps.newTable(0x104b78340, 0x80, 0x0, 0x0)
Panic!
Go doesn't let multiple goroutines write to a map at the same time. At first, this might seem odd. Here's the only operation that the count() goroutine does with the map:
counter[word]++
Looks like an atomic action. Why not perform it from multiple goroutines?
The problem is that the action only seems atomic. The operation "increase the key value in the map" actually involves several smaller steps. If one goroutine does some of these steps and another goroutine does the rest, the map can get messed up. That's what the runtime is warning us about.
Data Race
When multiple goroutines access the same variable at the same time, and at least one of them changes it, it's called a data race. Concurrent map modification in the previous section is an example of a data race.
A data race doesn't always cause a runtime panic (the map example in the previous section is a nice exception: Go's map implementation has built-in runtime checks that can catch some data races). That's why Go provides a special tool called the race detector. You can turn it on with the race flag, which works with the test, run, build, and install commands.
To use Go's race detector, you'll need to install gcc, the C compiler.
For example, take this program:
func main() {
var total int
var wg sync.WaitGroup
wg.Go(func() {
total++
})
wg.Go(func() {
total++
})
wg.Wait()
fmt.Println(total)
}
2
At first glance, it seems to work correctly. But actually, it has a data race:
go run -race race.go
==================
WARNING: DATA RACE
Read at 0x00c000112038 by goroutine 6:
main.main.func1()
race.go:16 +0x74
Previous write at 0x00c000112038 by goroutine 7:
main.main.func2()
race.go:21 +0x84
Goroutine 6 (running) created at:
main.main()
race.go:14 +0x104
Goroutine 7 (finished) created at:
main.main()
race.go:19 +0x1a4
==================
2
Found 1 data race(s)
If you're wondering why a data race is a problem for a simple operation like
total++— we'll cover it later in the chapter on atomic operations.
Channels, on the other hand, are safe for concurrent reading and writing, and they don't cause data races:
func main() {
ch := make(chan int, 2)
var wg sync.WaitGroup
wg.Go(func() {
ch <- 1
})
wg.Go(func() {
ch <- 1
})
wg.Wait()
fmt.Println(<-ch + <-ch)
}
2
Data races are dangerous because they're hard to spot. Your program might work fine a hundred times, but on the hundred and first try, it could give the wrong result. Always check your code with a race detector.
Sequential Modification
You can often rewrite a program to avoid concurrent modifications. Here is a possible approach for our word frequency program:
- Each
count()goroutine counts frequencies in its own map. - A separate
merge()function goes through the frequency maps and builds the final map.
func main() {
// generate creates 100 words, each 3 letters long,
// and sends them to the channel.
in := generate(100, 3)
var wg sync.WaitGroup
wg.Add(2)
// count reads words from the input channel
// and counts how often each one appears.
count := func(counters []map[string]int, idx int) {
defer wg.Done()
counter := map[string]int{}
for word := range in {
counter[word]++
}
counters[idx] = counter
}
counters := make([]map[string]int, 2)
go count(counters, 0)
go count(counters, 1)
wg.Wait()
// merge combines frequency maps.
counter := merge(counters...)
fmt.Println(counter)
}
// merge combines frequency maps into a single map.
func merge(counters ...map[string]int) map[string]int {
merged := map[string]int{}
for _, counter := range counters {
for word, freq := range counter {
merged[word] += freq
}
}
return merged
}
map[cec:1 ... nol:2 not:3 ... tut:1]
Now each count() goroutine works with its own map, so there are no concurrent modifications. After all goroutines finish, merge() combines the results into a single map. This approach avoids data races, but it requires more memory and an extra merge step.
Mutex
Sometimes you can't avoid concurrent modifications. In such cases, you need to synchronize access to shared data. A mutex (short for "mutual exclusion") is a synchronization primitive that ensures only one goroutine can access a piece of code at a time.
Let's fix our word frequency program using a mutex:
func main() {
// generate creates 100 words, each 3 letters long,
// and sends them to the channel.
in := generate(100, 3)
var wg sync.WaitGroup
wg.Add(2)
// count reads words from the input channel
// and counts how often each one appears.
count := func(lock *sync.Mutex, counter map[string]int) {
defer wg.Done()
for word := range in {
lock.Lock() // (2)
counter[word]++
lock.Unlock() // (3)
}
}
var lock sync.Mutex // (1)
counter := map[string]int{}
go count(&lock, counter)
go count(&lock, counter)
wg.Wait()
fmt.Println(counter)
}
map[cec:1 ... nol:2 not:3 ... tut:1]
We created the lock mutex ➊ and used it to protect access to the shared counter map ➋ ➌. This way, the count() goroutines don't cause data races, and the final counter[word] value is correct.
Here's how a mutex works:
Lock()acquires the mutex. If another goroutine already has it,Lock()blocks until the mutex becomes available.Unlock()releases the mutex, allowing other goroutines to acquire it.
The code between Lock() and Unlock() is called a critical section. Only one goroutine can execute the critical section at a time.
Important notes about mutexes:
- Always unlock a mutex after locking it. Use
deferto ensure unlocking happens even if the code panics. - Don't lock a mutex twice in the same goroutine without unlocking it first — this will cause a deadlock. Go's mutexes are not reentrant. This makes things harder for people who like to use mutexes in recursive functions (which isn't a great idea anyway).
- Like a wait group, a mutex has internal state, so you should only pass it as a pointer.
Read-Write Mutex
A regular mutex doesn't distinguish between read and write access: if one goroutine locks the mutex, others can't access the protected code. This isn't always necessary.
Here's the situation:
- One
writergoroutine writes data. - Four
readergoroutines read that same data.
var wg sync.WaitGroup
wg.Add(5)
var lock sync.Mutex
// writer fills in the word frequency map.
writer := func(counter map[string]int, nWrites int) {
defer wg.Done()
for ; nWrites > 0; nWrites-- {
word := randomWord(3)
lock.Lock()
counter[word]++
time.Sleep(time.Millisecond)
lock.Unlock()
}
}
// reader looks up random words in the frequency map.
reader := func(counter map[string]int, nReads int) {
defer wg.Done()
for ; nReads > 0; nReads-- {
word := randomWord(3)
lock.Lock()
_ = counter[word]
time.Sleep(time.Millisecond)
lock.Unlock()
}
}
start := time.Now()
counter := map[string]int{}
go writer(counter, 100)
go reader(counter, 100)
go reader(counter, 100)
go reader(counter, 100)
go reader(counter, 100)
wg.Wait()
fmt.Println("Took", time.Since(start))
Took 500ms
Even though we started 4 reader goroutines, they run sequentially because of the mutex. This isn't really necessary. It makes sense for readers to wait while the writer is updating the map. But why can't the readers run in parallel? They're not changing any data.
The sync package includes a sync.RWMutex that separates readers and writers. It provides two sets of methods:
Lock/Unlocklock and unlock the mutex for both reading and writing.RLock/RUnlocklock and unlock the mutex for reading only.
Here's how it works:
- If a goroutine locks the mutex with
Lock(), other goroutines will be blocked if they try to useLock()orRLock(). - If a goroutine locks the mutex with
RLock(), other goroutines can also lock it withRLock()without being blocked. - If at least one goroutine has locked the mutex with
RLock(), other goroutines will be blocked if they try to useLock().
This creates a "single writer, multiple readers" setup. Let's verify it:
var wg sync.WaitGroup
wg.Add(5)
var lock sync.RWMutex // (1)
// writer fills in the word frequency map.
writer := func(counter map[string]int, nWrites int) {
// Not changed.
defer wg.Done()
for ; nWrites > 0; nWrites-- {
word := randomWord(3)
lock.Lock()
counter[word]++
time.Sleep(time.Millisecond)
lock.Unlock()
}
}
// reader looks up random words in the frequency map.
reader := func(counter map[string]int, nReads int) {
defer wg.Done()
for ; nReads > 0; nReads-- {
word := randomWord(3)
lock.RLock() // (2)
_ = counter[word]
time.Sleep(time.Millisecond)
lock.RUnlock() // (3)
}
}
start := time.Now()
counter := map[string]int{}
go writer(counter, 100)
go reader(counter, 100)
go reader(counter, 100)
go reader(counter, 100)
go reader(counter, 100)
wg.Wait()
fmt.Println("Took", time.Since(start))
Took 200ms
The mutex type ➊ has changed, so have the locking ➋ and unlocking ➌ methods in the reader. Now, readers run concurrently, but they always wait while the writer updates the map. That's exactly what we need!
Channel as Mutex
Let's go back to the program that counts word frequencies:
func main() {
// generate creates 100 words, each 3 letters long,
// and sends them to the channel.
in := generate(100, 3)
var wg sync.WaitGroup
wg.Add(2)
// count reads words from the input channel
// and counts how often each one appears.
count := func(lock *sync.Mutex, counter map[string]int) {
defer wg.Done()
for word := range in {
lock.Lock() // (2)
counter[word]++
lock.Unlock() // (3)
}
}
var lock sync.Mutex // (1)
counter := map[string]int{}
go count(&lock, counter)
go count(&lock, counter)
wg.Wait()
fmt.Println(counter)
}
We created the lock mutex ➊ and used it to protect access to the shared counter map ➋ ➌. This way, the count() goroutines don't cause data races, and the final counter[word] value is correct.
We can also use a channel instead of a mutex to protect shared data:
type token struct{}
func main() {
// generate creates 100 words, each 3 letters long,
// and sends them to the channel.
in := generate(100, 3)
var wg sync.WaitGroup
wg.Add(2)
// count reads words from the input channel
// and counts how often each one appears.
count := func(lock chan token, counter map[string]int) {
defer wg.Done()
for word := range in {
lock <- token{} // (2)
counter[word]++
<-lock // (3)
}
}
lock := make(chan token, 1) // (1)
counter := map[string]int{}
go count(lock, counter)
go count(lock, counter)
wg.Wait()
fmt.Println(counter)
}
map[cec:1 ... nol:2 not:3 ... tut:1]
We created a lock channel with a one-element buffer ➊ and used it to protect access to the shared counter map ➋ ➌.
Two count() goroutines run concurrently. However, in each loop iteration, only one of them can put a token into the lock channel (like locking a mutex), update the counter, and take the token back out (like unlocking a mutex). So, even though the goroutines run in parallel, changes to the map happen sequentially.
As a result, the count() goroutines don't cause data races, and the final counter[word] value is correct — just like when we used a mutex.
Go's channels are a versatile concurrency tool. Often, you can use a channel instead of lower-level synchronization primitives. Sometimes, using a channel is unnecessary, as in the example above. Other times, however, it makes your code simpler and helps prevent mistakes. You'll see this idea come up again throughout the book.
Summary
Now you know how to safely change shared data from multiple goroutines using mutexes. Be careful not to overuse them — it's easy to make mistakes and cause data races or deadlocks.
Key points to remember:
- Data races occur when multiple goroutines access the same variable concurrently, and at least one modifies it.
- Use the race detector (
go run -race) to detect data races in your code. - Avoid concurrent modifications when possible by using separate data structures per goroutine.
- Use mutexes to protect shared data when concurrent modifications are unavoidable.
- Consider RWMutex when you have multiple readers and fewer writers.
- Channels can serve as mutexes in some scenarios, though mutexes are usually more appropriate.
Safe concurrent programming requires careful attention to data access patterns and proper synchronization.
