Use where
instead of select
When performing a lot of calculations, the less data is loaded into memory the better, always use SQL queries instead of object methods.
For example:
1 2 3 4 |
shop_ids <span class="token punctuation">.</span> map <span class="token keyword">do</span> <span class="token operator">|</span> shop_id <span class="token operator">|</span> products <span class="token punctuation">.</span> select <span class="token punctuation">{</span> <span class="token operator">|</span> p <span class="token operator">|</span> p <span class="token punctuation">.</span> shop_id <span class="token operator">==</span> shop_id <span class="token punctuation">}</span> <span class="token punctuation">.</span> first <span class="token keyword">end</span> |
The code will run faster if written like this:
1 2 3 4 |
shop_ids <span class="token punctuation">.</span> map <span class="token keyword">do</span> <span class="token operator">|</span> shop_id <span class="token operator">|</span> products <span class="token punctuation">.</span> <span class="token function">where</span> <span class="token punctuation">(</span> shop_id <span class="token punctuation">:</span> shop_id <span class="token punctuation">)</span> <span class="token punctuation">.</span> first <span class="token keyword">end</span> |
Use pluck
instead of map
If only a few values are interested in each row, pluck should be used instead of map
For example:
1 2 3 4 |
Order.where(number: 'R545612547').map &:id # Order Load (5.0ms) SELECT `orders`.* FROM `orders` WHERE `orders`.`number` = 'R545612547' ORDER BY orders.created_at DESC => [1] |
Map will load all data into memory first, then perform id, pluck faster because it does not have to load data first.
1 2 3 4 |
<span class="token constant">Order</span> <span class="token punctuation">.</span> <span class="token function">where</span> <span class="token punctuation">(</span> number <span class="token punctuation">:</span> <span class="token string">'R545612547'</span> <span class="token punctuation">)</span> <span class="token punctuation">.</span> pluck <span class="token symbol">:id</span> <span class="token comment"># SQL (0.8ms) SELECT `orders`.`id` FROM `orders` WHERE `orders`.`number` = 'R545612547' ORDER BY orders.created_at DESC</span> <span class="token operator">=</span> <span class="token operator">></span> <span class="token punctuation">[</span> <span class="token number">1</span> <span class="token punctuation">]</span> |
Use ActiveRecord::Calculations#sum
instead of Enumerable#sum
Normally when performing calculations sometimes we use Enumerable::sum
to sum, this is a common mistake because ActiveRecord::Calculations
gave us a way to calculate without loading a pile of data into the set. remember first. If you want to perform Rails calculations, ActiveRecord::Calculations
is the best option to use.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 |
<span class="token constant">Benchmark</span> <span class="token punctuation">.</span> ips <span class="token keyword">do</span> <span class="token operator">|</span> x <span class="token operator">|</span> x <span class="token punctuation">.</span> <span class="token function">report</span> <span class="token punctuation">(</span> <span class="token string">"SQL sum"</span> <span class="token punctuation">)</span> <span class="token keyword">do</span> <span class="token constant">Loan</span> <span class="token punctuation">.</span> <span class="token function">sum</span> <span class="token punctuation">(</span> <span class="token symbol">:balance</span> <span class="token punctuation">)</span> <span class="token keyword">end</span> x <span class="token punctuation">.</span> <span class="token function">report</span> <span class="token punctuation">(</span> <span class="token string">"Ruby sum"</span> <span class="token punctuation">)</span> <span class="token keyword">do</span> <span class="token constant">Loan</span> <span class="token punctuation">.</span> <span class="token function">sum</span> <span class="token punctuation">(</span> <span class="token operator">&</span> <span class="token symbol">:balance</span> <span class="token punctuation">)</span> <span class="token comment"># Same as: Loan.all.map { |loan| loan.balance }.sum</span> <span class="token keyword">end</span> x <span class="token punctuation">.</span> compare <span class="token operator">!</span> <span class="token keyword">end</span> <span class="token comment"># Comparison:</span> <span class="token comment"># SQL sum: 7.89 i/s</span> <span class="token comment"># Ruby sum: 0.03 i/s - 209.85x slower</span> |
Use ActiveRecord::Calculations#maximum
instead of Enumerable#max
As explained above, we should use ActiveRecord::Calculations
to improve the calculation
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 |
<span class="token constant">Benchmark</span> <span class="token punctuation">.</span> ips <span class="token keyword">do</span> <span class="token operator">|</span> x <span class="token operator">|</span> x <span class="token punctuation">.</span> <span class="token function">report</span> <span class="token punctuation">(</span> <span class="token string">"SQL max"</span> <span class="token punctuation">)</span> <span class="token keyword">do</span> <span class="token constant">Loan</span> <span class="token punctuation">.</span> <span class="token function">maximum</span> <span class="token punctuation">(</span> <span class="token symbol">:amount</span> <span class="token punctuation">)</span> <span class="token keyword">end</span> x <span class="token punctuation">.</span> <span class="token function">report</span> <span class="token punctuation">(</span> <span class="token string">"Ruby max"</span> <span class="token punctuation">)</span> <span class="token keyword">do</span> <span class="token constant">Loan</span> <span class="token punctuation">.</span> <span class="token function">pluck</span> <span class="token punctuation">(</span> <span class="token symbol">:amount</span> <span class="token punctuation">)</span> <span class="token punctuation">.</span> max <span class="token keyword">end</span> x <span class="token punctuation">.</span> compare <span class="token operator">!</span> <span class="token keyword">end</span> <span class="token comment"># Comparison:</span> <span class="token comment"># SQL max: 541.9 i/s</span> <span class="token comment"># Ruby max: 0.5 i/s - 1113.47x slower</span> |
Use ActiveRecord::Calculations#minimum
instead of Enumerable#min
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 |
<span class="token constant">Benchmark</span> <span class="token punctuation">.</span> ips <span class="token keyword">do</span> <span class="token operator">|</span> x <span class="token operator">|</span> x <span class="token punctuation">.</span> <span class="token function">report</span> <span class="token punctuation">(</span> <span class="token string">"SQL min"</span> <span class="token punctuation">)</span> <span class="token keyword">do</span> <span class="token constant">Loan</span> <span class="token punctuation">.</span> <span class="token function">minimum</span> <span class="token punctuation">(</span> <span class="token symbol">:amount</span> <span class="token punctuation">)</span> <span class="token keyword">end</span> x <span class="token punctuation">.</span> <span class="token function">report</span> <span class="token punctuation">(</span> <span class="token string">"Ruby min"</span> <span class="token punctuation">)</span> <span class="token keyword">do</span> <span class="token constant">Loan</span> <span class="token punctuation">.</span> <span class="token function">pluck</span> <span class="token punctuation">(</span> <span class="token symbol">:amount</span> <span class="token punctuation">)</span> <span class="token punctuation">.</span> min <span class="token keyword">end</span> x <span class="token punctuation">.</span> compare <span class="token operator">!</span> <span class="token keyword">end</span> <span class="token comment"># Comparison:</span> <span class="token comment"># SQL min: 533.3 i/s</span> <span class="token comment"># Ruby min: 0.5 i/s - 1017.21x slower</span> |
Use Model.find_each
instead of Model.all.each
A common mistake is to use ActiveRecord::Scoping::Named::ClassMethods#all
+ ActiveRecord::Result#each
to iterate over a table of thousands of records.
1 2 3 4 |
<span class="token constant">Product</span> <span class="token punctuation">.</span> all <span class="token punctuation">.</span> <span class="token keyword">each</span> <span class="token keyword">do</span> <span class="token operator">|</span> product <span class="token operator">|</span> product <span class="token punctuation">.</span> <span class="token function">update_column</span> <span class="token punctuation">(</span> <span class="token symbol">:stock</span> <span class="token punctuation">,</span> <span class="token number">50</span> <span class="token punctuation">)</span> <span class="token keyword">end</span> |
The all
method will load all data into memory first, which can lead to many memory problems. To be clear, the problem here is not the method all
but the number of records it loads.
To process, we load records into each batch (default is 1000 records):
1 2 3 4 |
<span class="token constant">Product</span> <span class="token punctuation">.</span> find_each <span class="token keyword">do</span> <span class="token operator">|</span> product <span class="token operator">|</span> product <span class="token punctuation">.</span> <span class="token function">update_column</span> <span class="token punctuation">(</span> <span class="token symbol">:stock</span> <span class="token punctuation">,</span> <span class="token number">50</span> <span class="token punctuation">)</span> <span class="token keyword">end</span> |
To customize the number of records:
1 2 3 4 |
<span class="token constant">Product</span> <span class="token punctuation">.</span> <span class="token function">find_each</span> <span class="token punctuation">(</span> batch_size <span class="token punctuation">:</span> <span class="token number">200</span> <span class="token punctuation">)</span> <span class="token keyword">do</span> <span class="token operator">|</span> product <span class="token operator">|</span> product <span class="token punctuation">.</span> <span class="token function">update_column</span> <span class="token punctuation">(</span> <span class="token symbol">:stock</span> <span class="token punctuation">,</span> <span class="token number">50</span> <span class="token punctuation">)</span> <span class="token keyword">end</span> |
You can specify the starting point of the batch, which is useful if there are multiple workers processing in the same queue, you can let one worker process records from id 1-5000, and the other worker handles records with id from 500 onwards.
1 2 3 4 |
<span class="token constant">Product</span> <span class="token punctuation">.</span> <span class="token function">find_each</span> <span class="token punctuation">(</span> batch_size <span class="token punctuation">:</span> <span class="token number">200</span> <span class="token punctuation">,</span> start <span class="token punctuation">:</span> <span class="token number">5000</span> <span class="token punctuation">)</span> <span class="token keyword">do</span> <span class="token operator">|</span> product <span class="token operator">|</span> product <span class="token punctuation">.</span> <span class="token function">update_column</span> <span class="token punctuation">(</span> <span class="token symbol">:stock</span> <span class="token punctuation">,</span> <span class="token number">50</span> <span class="token punctuation">)</span> <span class="token keyword">end</span> |
Reference : https://www.fastruby.io/blog/performance/rails/writing-fast-rails.html