Google Cloud and Alibaba, realtime video distribution

Google Cloud, Alibaba multi-cloud autoscaling

Project Goal

Build an auto scaling cloud architecture to distribute real-time FullHD video, around the globe with more than concurrent 500K viewers for corporate customers

Recap of the steps needed

  1. Build the Linux Debian backend based on ISPconfig, NGINX, PHP, MySQL to distribute https pages to Frontends
  2. Collect log in central logserver
  3. Build Auto-scaling, load balanced and synchronized frontends
  4. Setup Geo DNS to route users traffic to the proximity zone in multicloud
  5. Build China zone frontends on Alibaba cloud platform and deal with sync lag
  6. Setup Level3 and Akamai CDNs
  7. Create an automated stress test instances pool with Apache bench and siege to test infrastructure

Project tools and providers

Cloud Providers

We choose to build on top of Google Cloud Platform as the main global infrastructure, based on the comparison of the features, the network distribution, latency, cost efficiency and many other detailed tests.
Is possible to view the full comparison here: Cloud Platform Comparison

In the Chinese zone the main providers are blocked by the government, this obligated us to adopt a multicloud architecture
For now there isn't any competitor to the features and the network of Alibaba cloud, even if the platform still couldn't compete with Google Cloud outside of China


The webserver chosen is NGINX, well known for the speed, born to break the C10K problem
Still lack a lot of components present in Apache2 it's the best choice for speed
In these days as this solution it's used with small instances, in order to distribute the load horizontally on multiple computational units

ISPconfig, control panel, guarantee to the customer the ability and more easiness in the configuration of a virtual host, used a backend, distribute the web content to the frontends by a continuous sync

The central log server, based on rsyslog, make possible to save log from any instance even if it's auto created and quickly destroyed

Apache bench, Siege: stress test tools, used to simulate thousand of request per second to the webservers

Everything has been written to be portable, in a multicloud design, in order to have the same services, instance configuration and similar result on Google Platform GCP and Alibaba Cloud

Testing and production results

Based on the stress tests a dual core Google Cloud instances could handle more than 11K request per second
In production, with the scaling, during a live the infrastructure reached 500K requests per second
The Chinese zone speed up more than 10 times the page load, from 38,6s to 3,1s