CDN Split Solution – Reduce data transfer costs and production latency

Tram Ho

In this article we will not discuss what a CDN is, how to use it. The article is suitable for high-load products, multiple origin configurations running on AWS, considering multiple local CDN providers because of cost and latency issues.

AWS edge locations available in Vietnam

First, this solution also stems from extremely useful information that at the end of August 2022, AWS has launched 02 new AWS edge locations in Hanoi and Ho Chi Minh. This edge location is expected to improve first byte latency by 30% for data in transit, providing a secure, reliable, high-performance connection to end-users in Vietnam. Moreover, 1TB of global bandwidth is free. In fact, it’s really good, the latency is smaller than some local CDNs currently. (Verified by engineers of Sapo, Hoang-phuc and other units).

Basic CDN Flow:

Reality

Technical aspects

  1. From a production perspective, using only one provider for a service is an SPF – single point of failure.
  2. Although CDN providers are always committed to support against DDoS attacks at this tier, there is still a potential risk of attack/system failure with partial or complete connection failure.
  3. With many origins such as media files on S3, processing thumb images in the compute cluster, css, … requires a very complex configuration each time on a CDN provider.
  4. Moreover, the latency from each provider to the origin is different, usually going to the internet with high latency. Especially when the cable breaks, it’s even more catastrophic.
  5. In fact, I also have a situation where the CDN cluster of a Vietnam local brand fails many times a month, leading to an error on the end-user side and at the same time having to call origin again, causing the cost of data transfer to skyrocket from a few dozen to a few hundred. dollars, even more. (1TB ~ $1000 from S3 Singapore to Vietnam).

Cost aspect

  1. Data transfer = Cloudfront bandwidth is 0.120 USD/GB for the first 10TB. (About 2700 VND, calculated at the price of 23000 VND/Dollar). 10TB costs about $ 1200, ie ~ 28,800,000 VND ( => Letting data run directly from origin to end-user costs the same but lower latency.
  2. AWS Cloudfront used over 10TB, somewhere working with sales will reduce to $ 0.085 / GB (About 1955 VND / GB). The following 10TB costs about $850, which is ~ 20,000,000 VND. How much will it be if the bandwidth per month is up to 50-100TB??
  3. The cost per 1GB of bandwidth with local brands in Vietnam is only about 100-500 VND/GB. The following 10TB will cost about 1,000,000 VND -> 5,000,000 VND
  4. The cost of clearing the cache, the cost of preventing attacks with AWS is very specific, but the local brand is usually free.

Separation solution

We will use the cloudfront endpoint as the source for the local CDN provider. Detail:

After splitting, we use the cloudfront endpoint as the source for all CDN providers which will bring benefits:

  1. Switch providers simply with one endpoint.
  2. Latency from AWS’s PoP to S3 is of course super low (following the AWS backbone) but it’s right in Hanoi and HCM. Then from PoP to local CDN PoPs, there is no need to worry. Testing this is easy to demonstrate.
  3. Greatly reduce S3 CDN and data transfer out costs. (Of course there is still less overhead from Cloudfront to local CDN provider)
  4. Get better attack handling support from the local CDN provider.
  5. The worst case scenario is that all the local CDN providers are bad, just CNAME the domain directly to cloudfront without having to adjust too much.

Suggested criteria for choosing a CDN Provider

  1. Is the infrastructure large enough, high load capacity, distributed near the end-user of the product?
  2. Is there support for anti-DDoS attacks?
  3. What is the SLA for the service?
  4. When there is an error, there is a 24/7 support channel with Hotline, which chat?
  5. Competitive costs (Most of the prices in Vietnam are good, the difference is not much, so cost is the final factor)
  6. The provider’s peering ability with Vietnamese carriers: Which carrier’s CDN peering has the highest percentage of user traffic? Even if you have peering with 10 carriers that only account for 30%, it will not be as effective as 3 carriers that account for 70% of traffic in Vietnam.
  7. Ability to take advantage of in-house development for different functions of the same provider: For example, the ability to combine AWS Vietnam with a local CDN is difficult to use together compared to using Cloudfront native from Aws itself?
  8. Convenient in payment, this section accountants and brothers and sisters prefer to pay together for the services of a supplier rather than separate bills and reconcile monthly.
  9. Security: The nodes/edges of CDN providers are also used as tools to support external attacks (for example, Anti-DDOS), so it is necessary to check if it is needed in the near future? Is it easy to upgrade?

—-Criteria 6, 7, 8, 9 received additional comments from Mr. Bao – Head of International Sales Department of FPT Telecom.

Looking forward to receiving more contributions to improve more from everyone.

Share the news now

Source : Viblo