Today I will introduce to everyone about Eager Loading in Django. Surely anyone from other languages like Ruby or PHP switched to Python are too familiar with this keyword, right? But not everyone knows about it and I will join you to find out what Eager Loading in Django will be like. Let’s go!
What is Eager Loading?
Eager Loading is a concept that while accessing data, you will get the necessary data along with objects (objects) related to that data. It is the opposite of Lazy Loading when you only receive one data in a query and have to execute the query again when you want to retrieve objects related to it.
Talking about here is a bit confusing, right =)) because I also find it a bit confusing. But okay, let’s once again understand it through the following example:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 | <span class="token comment"># models.py</span> <span class="token keyword">from</span> django <span class="token punctuation">.</span> db <span class="token keyword">import</span> models <span class="token keyword">class</span> <span class="token class-name">Brand</span> <span class="token punctuation">(</span> models <span class="token punctuation">.</span> Model <span class="token punctuation">)</span> <span class="token punctuation">:</span> name <span class="token operator">=</span> models <span class="token punctuation">.</span> CharField <span class="token punctuation">(</span> max_length <span class="token operator">=</span> <span class="token number">100</span> <span class="token punctuation">)</span> <span class="token keyword">def</span> <span class="token function">__str__</span> <span class="token punctuation">(</span> self <span class="token punctuation">)</span> <span class="token punctuation">:</span> <span class="token keyword">return</span> self <span class="token punctuation">.</span> name <span class="token keyword">class</span> <span class="token class-name">Car</span> <span class="token punctuation">(</span> models <span class="token punctuation">.</span> Model <span class="token punctuation">)</span> <span class="token punctuation">:</span> brand <span class="token operator">=</span> models <span class="token punctuation">.</span> ForeignKey <span class="token punctuation">(</span> <span class="token string">'Brand'</span> <span class="token punctuation">,</span> db_column <span class="token operator">=</span> <span class="token string">'brand_id'</span> <span class="token punctuation">,</span> on_delete <span class="token operator">=</span> models <span class="token punctuation">.</span> CASCADE <span class="token punctuation">)</span> name <span class="token operator">=</span> models <span class="token punctuation">.</span> CharField <span class="token punctuation">(</span> max_length <span class="token operator">=</span> <span class="token number">100</span> <span class="token punctuation">)</span> model <span class="token operator">=</span> models <span class="token punctuation">.</span> CharField <span class="token punctuation">(</span> max_length <span class="token operator">=</span> <span class="token number">100</span> <span class="token punctuation">)</span> <span class="token keyword">def</span> <span class="token function">__str__</span> <span class="token punctuation">(</span> self <span class="token punctuation">)</span> <span class="token punctuation">:</span> <span class="token keyword">return</span> self <span class="token punctuation">.</span> name |
1 2 3 4 5 | <span class="token operator">>></span> <span class="token operator">></span> <span class="token keyword">from</span> models <span class="token keyword">import</span> Car <span class="token operator">>></span> <span class="token operator">></span> cars <span class="token operator">=</span> Car <span class="token punctuation">.</span> objects <span class="token punctuation">.</span> <span class="token builtin">all</span> <span class="token punctuation">(</span> <span class="token punctuation">)</span> <span class="token operator">>></span> <span class="token operator">></span> <span class="token keyword">for</span> car <span class="token keyword">in</span> cars <span class="token punctuation">:</span> <span class="token punctuation">.</span> <span class="token punctuation">.</span> <span class="token punctuation">.</span> <span class="token keyword">print</span> <span class="token punctuation">(</span> car <span class="token punctuation">.</span> brand <span class="token punctuation">.</span> name <span class="token punctuation">)</span> |
As can be easily seen in the above example we had to perform: 1 query to retrieve all cars and for each car it would be a query to get its brand name. If we have 10 cars, this loop will have to perform 11 queries.
In other words, we had to perform N + 1 queries. What if we have a few thousand, or even a few million cars? That’s a huge number …
But with Eager Loading , this problem is easily solved.
Solve the problem
For this problem, I will introduce to you 2 ways to solve it. Before learning what these two ways are, let’s try to run the following code:
1 2 3 4 5 6 | <span class="token comment"># Cách 1</span> <span class="token operator">>></span> <span class="token operator">></span> <span class="token keyword">from</span> models <span class="token keyword">import</span> Car <span class="token operator">>></span> <span class="token operator">></span> cars <span class="token operator">=</span> Car <span class="token punctuation">.</span> objects <span class="token punctuation">.</span> select_related <span class="token punctuation">(</span> <span class="token string">'brand'</span> <span class="token punctuation">)</span> <span class="token operator">>></span> <span class="token operator">></span> <span class="token keyword">for</span> car <span class="token keyword">in</span> cars <span class="token punctuation">:</span> <span class="token punctuation">.</span> <span class="token punctuation">.</span> <span class="token punctuation">.</span> <span class="token keyword">print</span> <span class="token punctuation">(</span> car <span class="token punctuation">.</span> brand <span class="token punctuation">.</span> name <span class="token punctuation">)</span> |
1 2 3 4 5 6 | <span class="token comment"># Cách 2</span> <span class="token operator">>></span> <span class="token operator">></span> <span class="token keyword">from</span> models <span class="token keyword">import</span> Car <span class="token operator">>></span> <span class="token operator">></span> cars <span class="token operator">=</span> Car <span class="token punctuation">.</span> objects <span class="token punctuation">.</span> prefetch_related <span class="token punctuation">(</span> <span class="token string">'brand'</span> <span class="token punctuation">)</span> <span class="token operator">>></span> <span class="token operator">></span> <span class="token keyword">for</span> car <span class="token keyword">in</span> cars <span class="token punctuation">:</span> <span class="token punctuation">.</span> <span class="token punctuation">.</span> <span class="token punctuation">.</span> <span class="token keyword">print</span> <span class="token punctuation">(</span> car <span class="token punctuation">.</span> brand <span class="token punctuation">.</span> name <span class="token punctuation">)</span> |
If we look at it, we can see that these 2 ways are not different and similar to the original writing style, right? But after running, let’s see what the number of queries that these two codes performed is
1 2 3 | <span class="token comment"># Cách 1</span> <span class="token keyword">SELECT</span> <span class="token punctuation">`</span> car <span class="token punctuation">`</span> <span class="token punctuation">.</span> <span class="token punctuation">`</span> id <span class="token punctuation">`</span> <span class="token punctuation">,</span> <span class="token punctuation">`</span> car <span class="token punctuation">`</span> <span class="token punctuation">.</span> <span class="token punctuation">`</span> brand_id <span class="token punctuation">`</span> <span class="token punctuation">,</span> <span class="token punctuation">`</span> car <span class="token punctuation">`</span> <span class="token punctuation">.</span> <span class="token punctuation">`</span> name <span class="token punctuation">`</span> <span class="token punctuation">,</span> <span class="token punctuation">`</span> car <span class="token punctuation">`</span> <span class="token punctuation">.</span> <span class="token punctuation">`</span> model <span class="token punctuation">`</span> <span class="token punctuation">,</span> <span class="token punctuation">`</span> brand <span class="token punctuation">`</span> <span class="token punctuation">.</span> <span class="token punctuation">`</span> id <span class="token punctuation">`</span> <span class="token punctuation">,</span> <span class="token punctuation">`</span> brand <span class="token punctuation">`</span> <span class="token punctuation">.</span> <span class="token punctuation">`</span> name <span class="token punctuation">`</span> <span class="token keyword">FROM</span> <span class="token punctuation">`</span> car <span class="token punctuation">`</span> <span class="token keyword">INNER</span> <span class="token keyword">JOIN</span> <span class="token punctuation">`</span> brand <span class="token punctuation">`</span> <span class="token keyword">ON</span> <span class="token punctuation">(</span> <span class="token punctuation">`</span> car <span class="token punctuation">`</span> <span class="token punctuation">.</span> <span class="token punctuation">`</span> brand_id <span class="token punctuation">`</span> <span class="token operator">=</span> <span class="token punctuation">`</span> brand <span class="token punctuation">`</span> <span class="token punctuation">.</span> <span class="token punctuation">`</span> id <span class="token punctuation">`</span> <span class="token punctuation">)</span> |
1 2 3 4 | <span class="token comment"># Cách 2</span> <span class="token keyword">SELECT</span> <span class="token punctuation">`</span> car <span class="token punctuation">`</span> <span class="token punctuation">.</span> <span class="token punctuation">`</span> id <span class="token punctuation">`</span> <span class="token punctuation">,</span> <span class="token punctuation">`</span> car <span class="token punctuation">`</span> <span class="token punctuation">.</span> <span class="token punctuation">`</span> brand_id <span class="token punctuation">`</span> <span class="token punctuation">,</span> <span class="token punctuation">`</span> car <span class="token punctuation">`</span> <span class="token punctuation">.</span> <span class="token punctuation">`</span> name <span class="token punctuation">`</span> <span class="token punctuation">,</span> <span class="token punctuation">`</span> car <span class="token punctuation">`</span> <span class="token punctuation">.</span> <span class="token punctuation">`</span> model <span class="token punctuation">`</span> <span class="token keyword">FROM</span> <span class="token punctuation">`</span> car <span class="token punctuation">`</span> <span class="token keyword">SELECT</span> <span class="token punctuation">`</span> brand <span class="token punctuation">`</span> <span class="token punctuation">.</span> <span class="token punctuation">`</span> id <span class="token punctuation">`</span> <span class="token punctuation">,</span> <span class="token punctuation">`</span> brand <span class="token punctuation">`</span> <span class="token punctuation">.</span> <span class="token punctuation">`</span> name <span class="token punctuation">`</span> <span class="token keyword">FROM</span> <span class="token punctuation">`</span> brand <span class="token punctuation">`</span> <span class="token keyword">WHERE</span> <span class="token punctuation">`</span> brand <span class="token punctuation">`</span> <span class="token punctuation">.</span> <span class="token punctuation">`</span> id <span class="token punctuation">`</span> <span class="token operator">IN</span> <span class="token punctuation">(</span> <span class="token number">1</span> <span class="token punctuation">,</span> <span class="token number">2</span> <span class="token punctuation">,</span> <span class="token number">3</span> <span class="token punctuation">,</span> <span class="token number">4</span> <span class="token punctuation">,</span> <span class="token number">5</span> <span class="token punctuation">)</span> |
At this point, we have realized the difference between the two ways and compared with the original writing, right. Through the above code, I want to introduce to you two ways of handling that is to use select_related and prefetch_related .
According to Django’s doc it is clear:
select_relatedfollow foreign key relationships, select objects related to it when querying by JOIN them.prefetch_relateddoes separate searches for each relationship and does * joining in python .
Sorry guys because I have to mark an asterisk (*) before the phrase joining in python . Because I do not know how to describe it correctly the message of this phrase.
Both select_related and prefetch_related were designed to reduce the number of queries, but they are implemented in different ways.
select_related works by creating a query join and including the fields of the relevant object in the SELECT statement . Also for this reason, select_related only takes one query that can receive all objects involved. However, there are a few select_related when you use select_related because it only takes 1 query that can retrieve all the related objects, so to avoid having too many large result sets from many joins. In a relationship, select_related is recommended in one-to-many, or one-to-one relationships.
prefetch_related , on the other hand, executes the queries separately for each relationship, and concatenates them. This allows it to pre-search and pre-filter objects in a many-to-many relationship using the WHERE IN clause , which is not possible by select_related .
The above is my share in solving the problem of N + 1 queries. Hope you guys continue to support me in the next articles.
Related links:
https://docs.djangoproject.com/en/3.0/ref/models/querysets/#select-related