Me, NuxtJS and the live stream of thousands of CCU

A little story about the optimal path of Server-Side Rendering application running with NuxtJS.

Of course, when I set the key to write this blog post, the application I was working on had not reached ten thousand CCUs. So if you read this far, you can say that you can catch the view title. Basically, now say what to share, but the alum application has 3 thousand CCUs, it does not have much credit. But I think I’m always telling the truth, so I’ll tell you guys from the beginning of this article to avoid disappointment.

If you find the story potentially attractive with only 3k CCUs, then ok, let’s continue reading.

Well, I’m not optimizing the livestream system, so don’t be mistaken, you won’t be disappointed. This is a web system that contains livestream only.

First things first

As usual, I’m Minh Monmen , a DevOps but also a little Frontend for it’s reputation as Fullstack. My HTML CSS ability is only moderate, which means I can do it, but it’s not beautiful. I often receive tasks related to architecture, project creation or project optimization during the process of frontend professionals. Here are some of the things I have to deal with in the process of making the frontend for the website livestream of thousands of CCUs (target of thousands of CCUs =))).

This knowledge can be directly applied to frameworks similar to NextJS (for React), for example, because the architecture in general is not much different.

OK yet? Start!

Starting from CSR and SSR

The first big problem when starting a frontend project is understanding how your frontend application will be deployed. In the past, I have witnessed a wrong architectural choice of choosing SPA (Single Page Application) with CSR (Client-Side Rendering) to make a web application, which has led to many bad situations later. how. So my first piece of advice for you is:

Always initialize your project with a framework that has both SSR (Server-Side Rendering) and CSR (Client-Side Rendering) capabilities. Or they often call it Universal application. For example, for VueJS code, don’t use Vue but use NuxtJS, if ReactJS code, don’t use React but use NextJS, …

Of course, if you have chosen to choose pure Vue client or pure React client, … then if there is a time when SSR capabilities are needed, there are many ways to patch it. But I think it’s best to just choose a guy with an architecture that supports SSR from the beginning to be sure. It’s better to have an excess than a lack because if it’s lacking, it’s very hard to cover it up, my friend.

For those who do not know what CSR and SSR are, you can see more about how these two mechanisms work in the video: https://www.youtube.com/watch?v=XwswPqIXYoI . Roughly speaking:

• CSR is that the client will load an HTML page with only code without content to the browser, then run JS to render the website content.

• SSR is a server that will render an HTML page with content , then the browser loads the HTML page with this content, it just needs to display it and make the content interactive.

• Traditional SSR will reload a new HTML page from the server every time the user turns the page, and SSR with NuxtJS or NextJS is often called Universal app , ie the app will combine SSR for first rendering and CSR for the user to switch pages on app.

The biggest reason why big SPA guys like VueJS or ReactJS still have to do SSR mode is optimizing for SEO and Social Sharing. Google bot, Bing bot, Facebook bot, … usually only crawl our page content from the HTML page, but we don’t have time to load both js and css to render. Although of course the Google bot is now very smart, it can render the whole CSR page and everything, but from the perspective of veteran SEO experts, it is still advisable to create conditions for the bot to have the content right from the page. HTML is the best.

Ok, then init project with SSR only.

So we have a newly created Nuxt JS project. I use a CSS guy familiar to all developers, Bootstrap and choose the rendering mode as Universal. Done.

The first problem: Extract CSS

No need to code anything. I run the project as Production:

Open the browser with the path http://localhost:3000 , Nuxt’s demo page shows up perfectly. But with the sixth sense of a DevOps person, I opened F12 and immediately saw an interesting detail:

Well, of course I don’t need to find any far-fetched reason for an empty project that HTML weighs 220KB because just opening the HTML content will see the culprit: CSS in the header .

The project’s CSS is embedded directly into the homepage HTML. Of course, there are many reasons for NuxtJS to do this by default, but for the projects that I or you often encounter, using a CSS library (like I’m using Bootstrap) is extremely common. So when NuxtJS stuffs all the CSS of the page into the header as above, our HTML page will be VERY HUGE.

Maybe you see 200KB for 1 HTML page it’s not big, but notice this is an EMPTY project. I have met many projects with default settings like this that have HTML pages weighing up to 1.5MB!!!

Although a Universal application only has the first time when the page loads is SSR, and then the user navigation will be handled on the client, but if the user presses F5 or Open in new tab , the SSR process is done again. But with the livestream application I am working on, user F5 is like a daily routine. Therefore, it is not possible to let the HTML page be as heavy as that.

Another reason is that HTML is often not cached comfortably on Proxy or CDN, but must be revalidated with Origin to see if the Origin side has changed anything about the content. So leaving the CSS in the header like this will cause the other CSS to not be cached, but have to be reloaded from Origin if the content of our website changes.

The fix is ​​very simple, just set the build.extractCSS = true property in the nuxt.config.js file.

This option will configure Nuxt (or specifically webpack) to separate your CSS into separate files. These files can be cached comfortably on Browser (for 1 user F5), Proxy or CDN (for multiple users accessing the same). It is not difficult, but because it is not set by default, many people do not notice. Let’s take a look at NuxtJS’s 2018 vote on this feature to see how many people notice it:

Result after extractCSS:

As you can see, extractCSS does not reduce the amount of data that the browser has to download the first time , but will have a great effect when we refresh the page. The HTML page now weighs only 7KB, the 200KB part has been separated into a separate css file and will be cached by the browser .

Cache common data on the server

In the process of optimizing a NuxtJS application of several other parties, I encountered a common situation that there were very common data, displayed on every page like list category , menu , page meta , … but still being fetched. from the Backend API during the SSR process. This increases the burden on the latency of the SSR rendered page and also increases the processing burden on the Node server (running NuxtJS) and the Backend server (running the API).

So what do we do with this case? Cache of course. Although for Backend API, it is very common to use memory cache (1 global variable) or distributed cache (redis). However, when switching to NuxtJS, it is a framework for the Frontend, so setting the cache server is often forgotten. To solve this problem and how to use the cache server during SSR, I will guide you.

Don’t think it’s easy, the hardest part of SSR is understanding where your code is running . There are many friends when the code does not understand this and eventually leads to the setting of the cache and finds it useless (because the code runs on the client), or the data is messed up between users (due to the code snippet). that runs on the server).

First install node-cache with yarn or npm , then create a file named server.cache.ts (I use ts) in the modules directory as follows:

By creating this module, we can be sure that the cache object is created only once when starting the NuxtJS server . It will then be injected into the ssrContext (the context is only available on the server).

Now we can use cache in the components as follows: (note that when using it, you must check process.server first)

Avoid memory leak when setInterval

A technique often used when wanting to refresh data in real time is short-polling , partly because it makes the code logic much simpler, and partly because it’s extremely easy to implement both on the server and the client side. Don’t laugh, one day you will realize that even though there are a lot of flaws, it’s just an undeniable fact that short-polling is so easy to do with extremely fast dev time, it’s already a great choice. bright already.

To do short-polling , it is definitely impossible not to mention the divine function setInterval already. But it’s also because of this setInterval guy that I also met many funny situations.

Let’s start with the following code:

Familiar right? This is just setting an interval to refresh the data on the page every 15 seconds. However, when you run it, you will find that every route through a few pages, loading the component back and forth, you will see that the call to getItems to the API is repeated many times continuously, not after 15 seconds. This is because you are having problems with memory leaks

The problem here is that when created is run SSR ie run on the server, the other interval will be run forever in the Node process . And you imagine when 1000 users also load the page, it also creates 1000 intervals on the same server . Cat! My Nuxt server has been running for a while, if it’s fast, it’s 1 day fast, but it’s slow for a few days, then it’s terrible RAM and then restarts itself (I run it with docker).

Ok ok, but code like this is too elementary, after a while google to fix it, you can also learn to clear the interval in beforeDestroy as follows, right:

Oh my friend, the first time I got it wrong, it’s a new SSR that I’m not familiar with, but if I search google and fix it like this, it’s wrong, then I doubt you =))). But the more dangerous thing about this google fix is ​​that I think it’s correct and I can’t check the results.

But hey, the mistake is over and I’ll also tell you how I did it so you guys don’t expect me to be like you:

Ok, the only change here is that I check the value of process.client to make sure that only when created is called under the client will the interval be initialized, and not on the server.

Save bandwidth when short-polling with Etag

Still this short-polling guy causing trouble. I have an API containing resources that need short-polling and the API also weighs a few dozen KB after gzip (let’s not mention gzip here because it’s too simple). It is worth mentioning here that when running the other livestream app, I found that quite a bit of my public bandwidth was occupied, but the user did not have much. I also tried looking at the resources that the browser has to load when the page loads, but it doesn’t seem to be too much. Then I suddenly noticed that my short-polling request was still diligently calling the server with a green status of 200 .

Okay, here’s the culprit. Although calling the API with a frequency of 15s / call, but my data takes about 1 minute to change, sometimes it takes longer, so every time I get the response of the API, it also wastes a lot of bandwidth. there.

Ok, so how to only get new data back if there is a change (without changing the code logic – very longuuuuuuuuuu)? Using a built-in HTTP feature, of course, is Etag / If-none-match .

Basically, the server will return an etag which is the signature generated from the response of that request on the first call. The browser will save this tag and the next time the request will transmit the same etag as the previous time. If the server generates a response that sees the same signature as the etag transmitted by the browser, it will return http code 304 Not Modified with an empty response for the browser to understand and use the old response.

Normally, you probably wouldn’t notice this case and didn’t touch it at all. Partly because if you use expressjs, there is already a built-in etag in the response. Partly because this whole logic is implemented by the browser and the client brother when calling ajax can only see the normal response. However, I’m using fastify to make an API, and this guy doesn’t have the etag feature enabled, so it takes up so much bandwidth.

Nginx also has Etag enabled by default for static resources , i.e. js, css, image, etc., but it doesn’t turn on with proxy_pass when calling the API to the backend, so you have to handle it yourself.

Of course, once discovered, then handling is just a simple matter:

Avoid re-render when API response is unchanged by Etag

A problem I also face when rendering the interface on the client is that when the data is refreshed (which I have completely reloaded), the UI will have to re-render quite a lot, in addition, in my logic code, I also have to handle quite a lot. to transform the API response into content displayed to the user. So when short-polling but the data does not change, the browser has to spend a lot of unnecessary processing resources, causing lag for the user.

I don’t code the frontend much, so I don’t understand the advanced techniques of anti-re-render when the data is unchanged. However, I have applied an extremely simple and easy way to avoid re-rendering if the data does not change, which is to take advantage of the etag header I just used above.

Note: Do not try to use status = 304 to check if the request has changed or not because in the client code will still receive the entire response from the previous 200 request including status, headers.

Voila, with an application that has a lot of realtime data but uses a simple architecture, I have saved a lot of the browser’s processing resources. The web experience is so smooth =)))

Separating CSR for user and SSR for crawler bot

I ran it again for a while, and I realized that the SSR for the user in my problem was not good . Why?

• Dynamic data on the page is a lot and depends on the user, so it is relatively impossible to cache the entire page with HTML (actually it can be done but it takes a lot of effort).
• The waiting time to load enough data for HTML is relatively long when combining responses from many APIs. I have optimized by dividing the main data, the secondary data (ie the main data is loaded SSR, the secondary data is placed in the <client-only> tag to load only on the client) but the total response time is still quite good. .
• The processing resources for the Nuxt SSR child are relatively large. Although it has been optimized quite well, when there are 3k CCUs in, the Nuxt still takes up the most resources on the server (3 Nuxt containers + 30% CPU per container). My API has been fully optimized, so it doesn’t take up much resources.

So how can I handle thousands of CCUs now? Currently, I’m only using 1 4CPU + 8GB RAM to run the entire frontend + backend, quite economical right? Of course, I can scale more resources because the Nuxts run without state, so I can scale as much as I want. However, I still think this 1 server is more than enough to handle thousands of CCUs and press F5, so it’s still optimal.

One thing worth noting is that users don’t like to see a spinning blank page . They have to see something and then wait. So the new facebook-style skeleton app games were born. That is, instead of waiting to display the article to the user, it gives a white block with a few fuzzy lines to load. In addition, doing SSR is only related to SEO + social media sharing. So why don’t I play CSR for the user and the bot responds with SSR on the back?

CSR will have the advantage that the initial html load time is extremely fast (because there is no content). After that, I was free to show skeleton or load to wait for API response. This experience helps keep users on the web more than having to wait a long time for a full HTML page. This is also the method that most of the big e-commerce giants, social networks, … are doing. However, to do this, you must have a little bit of infrastructure knowledge.

First, we will separate the Nuxt app build into two deployments. One guy will run the CSR form to serve static HTML, JS, and CSS. 1 guy will run the SSR form to render the HTML dynamically. To do this, I will separate 1 more client.nuxt.config.js file to configure the CSR deployment. The default nuxt.config.js will be used for SSR.

This file is actually just changed a bit, set ssr: false so that when the build comes out, nuxt will target CSR only. In addition, my project uses nuxtServerInit to load data into the store. When down to the client, it will not be able to run the code in nuxtServerInit, so I have to install nuxt-client-init module instead.

Continue to edit package.json to build separately for client and server:

Edit more Dockerfile to run server node with SSR and nginx with CSR (because if it’s CSR, no server is needed, nginx has a lot better load than Node server):

And here is the Dockerfile used to run SSR

Here is the box, the most important step is to divide the load so that the User will go to the CSR page, the Crawler bot will go to the SSR page. Now I will configure the nginx proxy in front as follows: