Hi guys, it’s me again. Today I would like to introduce to you a very cool and useful technique that is Rate Limiting
1. What is Rate Limiting?
Rate limiting is simply understood as limiting (limit) the number of requests (requests) to the server. In fact, one must use a number of algorithms to ensure fast, accurate performance and less memory consumption. Suppose our system receives thousands of requests, but among them can only handle hundreds of requests / s, for example, and the rest of the requests fail (because the system CPU is overloaded and cannot handle it). ).
To solve this problem, the Rate Limiting mechanism was born. Its purpose is only to allow receiving a certain number of requests in 1 unit of time. If so, it will return an error response.
2. Benefits, practical applications
- Limiting DDOS (Distributed Denial of Service) attack
- Brute force password in the system (exhaustive scan)
- Limit system spam, limit the number of redundant requests to handle
More about DDOS attack can be found Here, Brute force password here
3. express-rate-limit
In NodeJS, we can easily create Rate Limiting with express-rate-limit.
Setting
Like any other library, you can install express-rate-limit with the command
$ npm install –save express-rate-limit
Usage
After creating a project. We can use express-rate-limit with a few lines like this:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 |
<span class="token keyword">const</span> express <span class="token operator">=</span> <span class="token function">require</span><span class="token punctuation">(</span><span class="token string">'express'</span><span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token keyword">const</span> rateLimit <span class="token operator">=</span> <span class="token function">require</span><span class="token punctuation">(</span><span class="token string">'express-rate-limit'</span><span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token keyword">const</span> port <span class="token operator">=</span> <span class="token number">3000</span><span class="token punctuation">;</span> <span class="token keyword">const</span> app <span class="token operator">=</span> <span class="token function">express</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token keyword">const</span> apiLimiter <span class="token operator">=</span> <span class="token function">rateLimit</span><span class="token punctuation">(</span><span class="token punctuation">{</span> windowMs<span class="token operator">:</span> <span class="token number">60</span> <span class="token operator">*</span> <span class="token number">1000</span><span class="token punctuation">,</span> <span class="token comment">// 1 minutes</span> max<span class="token operator">:</span> <span class="token number">2</span><span class="token punctuation">,</span> message<span class="token operator">:</span> <span class="token string">'Too many connection'</span><span class="token punctuation">,</span> <span class="token punctuation">}</span><span class="token punctuation">)</span><span class="token punctuation">;</span> app<span class="token punctuation">.</span><span class="token function">get</span><span class="token punctuation">(</span><span class="token string">'/'</span><span class="token punctuation">,</span> apiLimiter<span class="token punctuation">,</span> <span class="token punctuation">(</span><span class="token parameter">req<span class="token punctuation">,</span>res</span><span class="token punctuation">)</span> <span class="token operator">=></span> <span class="token punctuation">{</span> res<span class="token punctuation">.</span><span class="token function">send</span><span class="token punctuation">(</span><span class="token template-string"><span class="token template-punctuation string">`</span><span class="token string">  I'm CuaMotCang</span><span class="token template-punctuation string">`</span></span><span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token punctuation">}</span><span class="token punctuation">)</span> app<span class="token punctuation">.</span><span class="token function">listen</span><span class="token punctuation">(</span>port<span class="token punctuation">,</span> <span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token operator">=></span> <span class="token punctuation">{</span> console<span class="token punctuation">.</span><span class="token function">log</span><span class="token punctuation">(</span><span class="token string">'App is running'</span><span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token punctuation">}</span><span class="token punctuation">)</span> |
In the above example, I created Rate Limiting for each IP address that can only be accessed up to 2 times per minute. We can check with Postman
The first image is the result returned when we access the path for the first time. The second image is the result returned when accessing the 3rd time onwards within 1 minute compared to the first request. It’s that simple, isn’t it?
Rate limit options
- windowMS: Time of a cycle (in milliseconds)
- Max: The maximum amount of requests that can be sent in 1 cycle (eg: windowMS = 36000, max = 1000 => can send up to 1 hour can send up to 1000 requests)
- Message: Notice returned to the client when accessing the specified number of times
- Handler: A simpler way is to return the client the desired data type. I return object data to the client. 429 is the return value when the number of requests exceeds the limit.
1234567891011<span class="token keyword">const</span> apiLimiter <span class="token operator">=</span> <span class="token function">rateLimit</span><span class="token punctuation">(</span><span class="token punctuation">{</span>windowMs<span class="token operator">:</span> <span class="token number">60</span> <span class="token operator">*</span> <span class="token number">1000</span><span class="token punctuation">,</span> <span class="token comment">// 1 minutes</span>max<span class="token operator">:</span> <span class="token number">2</span><span class="token punctuation">,</span><span class="token function-variable function">handler</span><span class="token operator">:</span> <span class="token keyword">function</span> <span class="token punctuation">(</span><span class="token parameter">req<span class="token punctuation">,</span> res</span><span class="token punctuation">)</span> <span class="token punctuation">{</span>res<span class="token punctuation">.</span><span class="token function">status</span><span class="token punctuation">(</span><span class="token number">429</span><span class="token punctuation">)</span><span class="token punctuation">.</span><span class="token function">send</span><span class="token punctuation">(</span><span class="token punctuation">{</span>status<span class="token operator">:</span> <span class="token number">500</span><span class="token punctuation">,</span>message<span class="token operator">:</span> <span class="token string">'Too many requests!'</span><span class="token punctuation">,</span><span class="token punctuation">}</span><span class="token punctuation">)</span><span class="token punctuation">;</span><span class="token punctuation">}</span><span class="token punctuation">,</span><span class="token punctuation">}</span><span class="token punctuation">)</span><span class="token punctuation">;</span>
And this is the result: - Skip: Can be used as a white list. In the following example, I bypass the local address so that I can use this address to freely send requests without being blocked by Rate Limiting.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 |
<span class="token keyword">const</span> apiLimiter <span class="token operator">=</span> <span class="token function">rateLimit</span><span class="token punctuation">(</span><span class="token punctuation">{</span> windowMs<span class="token operator">:</span> <span class="token number">60</span> <span class="token operator">*</span> <span class="token number">1000</span><span class="token punctuation">,</span> <span class="token comment">// 1 minutes</span> max<span class="token operator">:</span> <span class="token number">2</span><span class="token punctuation">,</span> <span class="token function-variable function">handler</span><span class="token operator">:</span> <span class="token keyword">function</span> <span class="token punctuation">(</span><span class="token parameter">req<span class="token punctuation">,</span> res</span><span class="token punctuation">)</span> <span class="token punctuation">{</span> res<span class="token punctuation">.</span><span class="token function">status</span><span class="token punctuation">(</span><span class="token number">429</span><span class="token punctuation">)</span><span class="token punctuation">.</span><span class="token function">send</span><span class="token punctuation">(</span><span class="token punctuation">{</span> status<span class="token operator">:</span> <span class="token number">500</span><span class="token punctuation">,</span> message<span class="token operator">:</span> <span class="token string">'Too many requests!'</span><span class="token punctuation">,</span> <span class="token punctuation">}</span><span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token punctuation">}</span><span class="token punctuation">,</span> <span class="token function-variable function">skip</span><span class="token operator">:</span> <span class="token punctuation">(</span><span class="token parameter">req<span class="token punctuation">,</span> res</span><span class="token punctuation">)</span> <span class="token operator">=></span> <span class="token punctuation">{</span> <span class="token keyword">if</span><span class="token punctuation">(</span>req<span class="token punctuation">.</span>ip <span class="token operator">===</span> <span class="token string">'::ffff:127.0.0.1'</span><span class="token punctuation">)</span> <span class="token keyword">return</span> <span class="token boolean">true</span><span class="token punctuation">;</span> <span class="token keyword">return</span> <span class="token boolean">false</span><span class="token punctuation">;</span> <span class="token punctuation">}</span> <span class="token punctuation">}</span><span class="token punctuation">)</span><span class="token punctuation">;</span> |
There are also some other options you can see details here.
Script testing
To make it simpler to test the code with a large number of requests and a larger time, you can refer to the following script:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 |
<span class="token keyword">const</span> <span class="token constant">URL</span> <span class="token operator">=</span> <span class="token string">'http://localhost:3000/'</span><span class="token punctuation">;</span> <span class="token keyword">const</span> request <span class="token operator">=</span> <span class="token function">require</span><span class="token punctuation">(</span><span class="token string">'request'</span><span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token keyword">const</span> <span class="token function-variable function">req</span> <span class="token operator">=</span> <span class="token keyword">async</span> <span class="token punctuation">(</span><span class="token parameter">url</span><span class="token punctuation">)</span> <span class="token operator">=></span> <span class="token punctuation">{</span> <span class="token keyword">return</span> request<span class="token punctuation">.</span><span class="token function">get</span><span class="token punctuation">(</span><span class="token punctuation">{</span> headers<span class="token operator">:</span> <span class="token punctuation">{</span><span class="token string">'content-type'</span> <span class="token operator">:</span> <span class="token string">'application/x-www-form-urlencoded'</span><span class="token punctuation">}</span><span class="token punctuation">,</span> url<span class="token operator">:</span> url<span class="token punctuation">,</span> <span class="token punctuation">}</span><span class="token punctuation">,</span> <span class="token keyword">function</span><span class="token punctuation">(</span><span class="token parameter">error<span class="token punctuation">,</span> response<span class="token punctuation">,</span> body</span><span class="token punctuation">)</span><span class="token punctuation">{</span> console<span class="token punctuation">.</span><span class="token function">log</span><span class="token punctuation">(</span>body<span class="token punctuation">)</span> <span class="token punctuation">}</span><span class="token punctuation">)</span> <span class="token punctuation">}</span><span class="token punctuation">;</span> <span class="token function">setInterval</span><span class="token punctuation">(</span><span class="token keyword">async</span> <span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token operator">=></span> <span class="token punctuation">{</span> <span class="token keyword">await</span> <span class="token function">req</span><span class="token punctuation">(</span><span class="token constant">URL</span><span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token punctuation">}</span><span class="token punctuation">,</span> <span class="token number">1000</span><span class="token punctuation">)</span> |
This script will request to the address http://localhost:3000/ every 1 second. You can customize it according to your purpose.
Epilogue
Hope this article will be of some help to you. In the article there are many shortcomings, hope everyone can comment
References: