Leaking memory due to using the wrong map in Golang?

As you know, map is a built-in data type in Golang and it is a set of key/value pairs.
It sounds quite simple, when needed, add the record to the map, if you don’t need it anymore, delete it.
Wow, it’s so simple, but it still has a buildin, how can it lead to memory leaks, this is my mind set when I first used Go
Without further ado, I will give an example.

1. Situation

Here is a sample code that proves the map in golang eats memory without releasing it

1. First we will allocate memory, initialize the empty map m
2. Then loop 1 million times to attach the elelemt to m
3. Finally remove all elements in m and run GC

After each of the above steps, we will log the size of the heap memory using runtime.MemStats (MemStats records statistics about the memory allocator).
The last line runtime.KeepAlive(m) keeps the reference to map m from being collected by the GC.
Ok, let’s run the program, let’s predict what the heap memory size is!!

Woww, is this result the same as your prediction?

After allocating memory for map m, the heap size is the smallest – 0 MB. Then the heap size increases rapidly when 1 million elements are added to map m. In the end, even though the GC collected all the elements removed from the map, the heap size was still 293 MB. The heap memory size has decreased but not as we would like to 0 MB, right? So what is the reason?

2. What is the cause?

Before we find out why, we need to know how map works in golang.

Basically, Map in Go is a pointer to runtime.hmap, this struct hmap contains a lot of fields, of which B field represents the number of buckets currently in the map. Each of these buckets is a pointer to an array (each array has only 8 elements, when it runs out, create a new array and link to the previous array) containing the elements of the map. So when we add an element to the map, we are adding an element to the bucket. Same as when deleting.
I will leave the link here for those who want to learn more about this part: https://dave.cheney.net/2018/05/29/how-the-go-runtime-implements-maps-efficiently-without- generics

But one thing to note here is that when adding an element to the map, the number of buckets will be allocated and increased, the element is added to the bucket, but when we delete the element in the map, we only delete the element in the bucket. but will not delete the previously allocated number of buckets.

As in the book 100 Mistakes In Golang on page 88, the author gives a bucket number of 262,144 buckets when adding 1 million elements, and when removing, the number of buckets is still 262,144 (This we can use profiling in golang to see the number of buckets).

=> Conclusion: The number of buckets in the map only increases, not decreases. And this is the reason why the heap size is not reduced according to our expectations. Because the GC only collects the elements that have been deleted in the bucket, it does not affect the map.

3. Is this really the problem?

In my opinion, the fact that these buckets are not deleted when the element is deleted is not necessarily bad. Good or bad depends on the situation.

If our map contains elements without a CUD then of course there will not be this problem.
If the map contains elements that often have to be inserted and deleted, of course, this problem will arise. But in this situation there is also good and bad. When we delete elements and then add new elements regularly, the buckets in the map remain the same, the system will not need to re-allocate the buckets, thereby increasing the performance a little for the system :v. As for the bad side, you see, memory will not decrease exactly as we expected, devs without experience in this array are constantly trying to figure out why Mem keeps increasing but decreasing a little, then again asked by the boss =)

In fact, this case is very easy to happen, for example, when your system uses map to cache user data, on big promotional days like 1/1, millions of users visit the website to shop. But after a few days, you still see the number of Mem of the server at a high level without decreasing, so the boss presses your head again to deduct your salary :v.

A few months ago, I had a similar case, I worked as a freelancer for a company in the field of live streaming, entertainment content, online television. They use local cache to store user data and all information about each user’s viewing category (this data to vcl). After running load testing, the server’s memory increases like a rocket, not decreasing at all. My boss at that time must have been stuck, so kicked that task for me, told me to research why mem didn’t decrease =)). I was a fresh graduate at that time, so I didn’t know anything. Sit and study for a month without any report. You can guess what the result will be =)

4. Solution

The easiest solution is to reset the service, but we can’t reset it ourselves, we have to ask the devops to say “Hello, please reset the service X for me”. It’s okay to meet an easy devops, but if you meet a difficult guy, it’s okay to move. But resetting once may be okay, but resetting many times in a month, I think I’m going to quit my job =))

The next solution is to recreate that map. Suppose we run a goroutine, every 1 hour it copies all the elements in the old map and adds it to the new map, then replace the old map and it’s done. But this also has a disadvantage, after copying all elements to the new map, our mem still exists the old map until the next GC collects it.

Another solution is that we store the pointer to the data, not the data directly in the map. It does not solve the problem of the number of buckets not decreasing, but it will reduce the size of the bucket.

5. Summary

We need to be careful when using maps in golang if we don’t want the days of debugging without a way out :v. You should consider whether this situation is possible to use the map, and if it is used, there will be problems when the number of elements in the map grows. And finally, keep in mind: “Go map can only grow in size, there is not automated strategy to shrink it.”

P/s:
This is the first time I write a knowledge sharing article, and my experience is still quite small, mainly in books, it is impossible to avoid mistakes. But I always have the desire to share the knowledge that I know with everyone, especially gopher :v. So I really want to have the feedback and sharing of readers so that the next articles I have more motivation to write better. Thanks for everyone!!!!

6. References

https://dave.cheney.net/2018/05/29/how-the-go-runtime-implements-maps-efficiently-without-generics
https://teivah.medium.com/maps-and-memory-leaks-in-go-a85ebe6e7e69