Optimize Ruby performance

Tram Ho

We all know that Ruby is one of my favorite languages ​​because it simplifies the software development process, but Ruby is often associated with the concept of slow execution.

It's true that the Ruby 1.8 release released in 2003 is really slow. But since then, Ruby developers have been actively improving the performance of the language. Ruby 1.9 has added a virtual machine to make code execution faster. Ruby 2.0 has optimized memory manager to help develop large web applications quickly. And finally, thanks to the hard work of Koichi Sasada, Ruby has made great improvements to the garbage collector in versions 2.1 and 2.2. These efforts are ongoing and performance becomes one of the first goals for each new version.

This series of articles will follow the Ruby Performance Optimization that gives us a better overview of the performance issues in Ruby, why Ruby is slow and how to optimize when using Ruby for software development. .

What makes Ruby fast

When building an application that takes too long to load, or if the client responds slowly to performance, we begin the optimization process. And then, we will be wondering what to do to make the code run faster?

As a Ruby developer, when asked, I'm sure the majority of developers will answer that I don't know? when asked like that. Because we simply think we have written the code effectively, and what we do after that is bypass the optimization process, pushing all into the cache and scale. Because we don't know how to improve, why, in thinking, it's always hard to optimize optimization. But, in most cases, just a few tools to support or fix a few lines of code, the code we have run much faster.

In addition, cache and scale cannot be used forever to improve performance. Cache and scale will also have limits. Suppose, when we want to speed up the system, we will scale, split multiple services into multiple servers, use load balance, provide multiple servers for one app. However, as the system expands, or adds new features, we keep adding servers, so the cost will increase. Server cache is also limited. If anything is pushed into the cache, then the server cache will be full. From here, we need to approach the performance problem solution in a different direction.

What makes Ruby code slow?

To find out how to make Ruby code fast, first of all, we need to clarify what makes Ruby code slow

Usually, the first thought is that the code uses too complex algorithms: extra nested loops, calculations, and arrangements. And what do you need to do to fix this problem? You only need to evaluate, find out where it is slowing down, and use a more optimal algorithm. Keep repeating this process until it runs fast. This usually seems correct, but does not work well for Ruby code. Complex algorithms can cause part of the performance problem. But with Ruby, there's another bigger cause.

VD with a simple program to generate data for CSV files

We will temporarily run it with Ruby versions 1.8.7, 1.9.3, 2.0, 2.1, and 2.2, each with different performance characteristics. Ruby 1.8 is considered to be the slowest and oldest, with various interpreting and implementation architectures. Ruby 1.9.3 and 2.0 have the same release timeline with similar performance. Ruby 2.1 and 2.2 are versions developed to improve performance.

We have the following result:

We see that Ruby 2.1 and 2.2 have improved a lot, but the reality is still slow. 10000 records that take more than 2 seconds.

Let's find out, first of all, about algorithms. The problem we use is the O (nm) algorithm, no problem. So what can we optimize? Try turning off the garbage collection function by adding the GC.disable command

The results were surprising:

We can clearly see, the time is mostly running GC (garbage collection), while executing code in different versions is almost equivalent. Although the version of Ruby 2.1 has been improved GC time, but taking more than 50% of the execution time is still too big.

So what happens to Ruby GC, is it because the code is using too much memory? Or is Ruby GC itself too slow? The answer is both. Excessive use of memory is the essence of Ruby. Because it's Ruby's design language. Everything is an object, meaning the program needs a memory large enough to represent all the data as Ruby objects. In addition, GC running slowly is also a long-standing problem of Ruby. It uses an algorithm that is considered to be the slowest among garbage collection algorithms, it uses the idea of ​​mark-and-sweep, stop-the-world (meaning to mark and clean, along with stopping everything). . In addition, when the GC runs, the application must also stop. That leads to why the application takes a long time to freeze during the run.

Ruby 2.1 and 2.2 are improved over previous versions mainly due to the improved GC set.

So from above we have a question, why does GC take so much time. Specifically, what did GC do? We know the more memory we consume, the longer GC will run to complete the cleanup process. So is it because we need to initialize too much memory, we can know this by showing the memory size before and when running the benchmark. By printing out the process's RSS, it will display the process's memory in RAM.

We get the result:

As you can see, the original data was 1GB, but it took more than 2 GB of memory to process 1 GB of data. This is strange, why does the program need 2 GB instead of 1 GB of memory? How to fix this? Is there a way for the program to use less memory? These questions will be answered in the next section.

summary

  • Memory usage and GC (garbage collector) are the main causes of Ruby being slow.
  • Ruby consumes significant memory.
  • GC in Ruby 2.1 and later is 5 times faster than previous versions.
  • The performance of all Ruby compiler versions is the same.

Memory optimization

Using large memory is what makes Ruby slow. Therefore, to optimize, we need to reduce the memory capacity. In this section, we will learn how to reduce the time it takes for the garbage processor to clear.

First, we have a question, why not always stop GC, why use it. That's a really good solution, because time is mostly spent by the GC running. However, clearing the GC will increase the memory consumption to its peak. At that time, the operating system will overflow memory or need to start cleaning up. Both results make performance a lot slower than for the Ruby GC suite to work.

Going back to the example above, we know we need 2 GB of memory to process 1 GB of data. So we show what memory is used for.

The CSV row we created in the block immediately saves the results to memory until we join them with the new line character. That is really the place where the extra 1 GB of memory is used.

So, we can rewrite the other way without storing the results immediately. We can iterate through the rows and continue iterating through the columns and store the results in the csv variable.

Although the code is quite bad, the results are really significant:

Just a few simple changes, we have overcome the shortcomings of GC. The program is optimized even faster than the original program without GC. And if you run the optimal version that has GC disabled, you'll find that GC time is only 10% of the total execution time.

By making simple changes, we have improved the performance by 2.5 to 10 times. So, let's take a closer look at the code and think about how much memory is used for each line and function called. At that time, we will know where the memory is created too much, or where it is not used effectively to rewrite the code accordingly. Very simple, right?

But in my experience, it is not necessary to optimize everything other than memory. According to the 80-20 principle of optimizing performance in Ruby, 80% of performance improvement comes from memory optimization, the remaining 20% ​​comes from other things.

Please review, think and rewrite. Maybe they should think more. If optimizing memory requires you to rethink what the code is for, then we really need to think about that.

summary

  • The 80-20 rule of performance optimization in RUby: 80% of performance optimization comes from memory optimization, so memory optimization is needed first.
  • A program with optimized memory will have the same performance as most new Ruby versions.

Building Mind-set performance

Optimizing Ruby requires more thinking about what the code is all about than finding bottlenecks with special tools. The main skill to learn is the right way of thinking about performance, or Mind-set, in optimizing Ruby performance.

When you write code, always think about the amount of memory used and the garbage collector required, by answering the following questions:

  • Ruby is the best tool to solve the problem

Ruby is a general-purpose programming language, but that doesn't mean it can solve all problems. There are things Ruby does not do well. For example, the prime number problem needs to process a huge amount of data. So it requires a lot of memory.

Either the task will work effectively in the database or in background processes written by other programming languages. For example, Twitter uses Ruby on Rails to write in the frontend and use Scala worker for the backend side. Or with statistical problems, calculations, analysis, it is better to use the language R.

  • How much memory does our code use?

The less memory you have to use for the code, the less GC will have to do. There are a few tricks to reduce memory consumption, such as processing data stream by line and avoiding usage across objects. We will learn more through the next sections.

  • What is the actual performance of the code

Once you are sure that the memory has been optimized, start learning about the algorithm the code is being used.

By asking these questions, you have begun to gain mind-set performance when coding Ruby.

In the next section, we will discuss the performance issues that are commonly encountered. Thanks for watching.

Share the news now

Source : Viblo