Video streaming has become very popular and billion-dollar industry in recent years. Video streaming services like Netflix and YouTube are providing billions of videos against a few clicks seemingly. This article sheds light on this topic by covering each aspect.
How does a video streaming service (like Netflix) works? What technology is behind and what might you need to set up your own video streaming service?
This article could be helpful for students who need to understand the backend of video streaming and Netflix. They can consult it in writing a research paper.
- What is Streaming Video and How Does it Work?
- How does Netflix work?
- Technologies used in Netflix
- Storage in Netflix
- How does Netflix approve a video?
- Avoiding duplicate videos in Netflix
- Requirements to set up Your own Video Streaming service:
What is Streaming Video and How Does it Work?
A video streaming platform indulges in a few steps to get the video out for the public, yet there are two types of streams. One is recorded-and-uploaded known as Video On Demand (VOD) and the second one is live streaming video. Netflix creates hundreds of thousands of copies of one video and distributes it to data centers located in different parts of the world. These copies are of different viewpoints such as 480, 720p, and HD.
These video streaming services also categories the videos based on their niches like YouTube has a ‘playlist‘ feature, it’s called asset management.
CDN (Content Delivery Network) is used by video streaming platforms when a video is uploaded. CDN spreads the copies of videos around the world (in data centers) and when a video is tapped to play it plays the video from the closest data center.
How does Netflix work?
Netflix bases on 3 components
- Open Connect (CDN)
CDN has already been described. It’s a content delivery network that spreads the content to different data centers around the world.
The client is a device on which a Netflix video is being browsed and played, it could be a computer, cell phone, or a tablet.
The backend includes critical functions such as serving the original HD video according to the device and internet quality of the user, this process is also called transcoding. If the internet connection is fast, HD resolution will be served otherwise low quality.
As we mention that Netflix works on 2 clouds; AWS and Open Connect; This is how it divides the
Roles between AWS and Open Connect
AWS controls jobs from opening Netflix to playing video including user-management. Such as logins, homepage, showing recommendations to the user, user history, billing, and customer service. Clicking a video shifts the load from AWS to Open Connect.
Technologies used in Netflix
Netflix uses machine learning models based on Hadoop to manage user searches, display, location, user-devices, and feedback.
Programming Languages used in Netflix
According to ciodive.com Netflix mostly used Python, for its ”full content lifecycle,” including tasks like security automation and training machine learning models for its recommendation algorithms. Parts of Netflix are also coded in React JS.
ELB (Elastic Load Balancer):
It also incorporates a two-tier load-balancing scheme called ELB (Elastic Load Balancer), it’s an Amazon product. It brings traffic to the front-end from zones (data centers).
Then comes ZUUL which handles the inner heavy working such as entertaining the requests form devices, web server, network protocol, connection management, and proxy task.
ZUUL in Netflix is based on three types of filters, inbound is used to authenticate and route the request. The endpoint filters are used to respond between original and regional zones. Outbound filters do gzip or adding/deleting a custom header if a response is sent back to the origin.
Hystrix is used in Netflix to manage the micro-services. It controls the latency and fault-tolerance, if a microservice is failed to respond it will not cause other services to stop, they are able to keep on continuing.
Netflix also makes use of Microservices which is a collaborating architecture of APIs. Different APIs keep on performing different tasks and if they need data from others they call them, and the other API responds with required data.
Though these APIs are designed to cooperate with each other but also configured with the least dependency on others. Netflix ensures stateless services if a request being fulfilled by a server fails the request would be shifted to the other server.
Read this article to learn blogging platforms that can help you earn a great passive income.
Storage in Netflix
RAM being a volatile memory is unable to store a large volume of data thus Netflix incorporates SSD (Solid-state Drive) which is a fast and cheaper medium of data storage.
Netflix classified storage in 2 types:
- Smaller Storage
- Consistent/Growing Viewer Base
The smaller storage type stores data for live viewing history in uncompressed form and the Growing type is used for a big amount of previous watching records. This data is stored in a compressed format.
What database does Netflix use?
Netflix uses two types of databases MySQL and Cassandra. MySQL is used to manage operations like billing and user information while Cassandra handles a huge volume of data. The video watching activities, User Interface activities, error logs are recorded by Cassandra.
How does Netflix approve a video?
Netflix invests much time approving the video. It searches for advanced artifacts, color variations, or missing frames that may have been brought about by past transcoding attempts or data transmission issues. The video is discarded if any issues are found. After the video is approved, it’s supplied to the Netflix media pipeline.
Avoiding duplicate videos in Netflix
YouTube is a colossal video sharing framework. Individuals can upload videos and the number of uploading videos increases exponentially every day. YouTube holds duplicates of some videos and this is very casual on it. But on Netflix, they use a judicious algorithm. For instance, when a video goes to a framework, the algorithm consequently checks whether this video is as of now kept in the framework or not. On the off chance that the framework has this video, at that point, it should decline the duplicate copy.
Requirements to set up Your own Video Streaming service:
To create a video streaming service you need
- to know specific programming languages
- awareness of web technologies, servers
- expertise in cybersecurity,
- and a few third-party services
You need to be an expert in programming languages like NodeJS, ReactJS, or C#, while for mobile applications Kotlin for Android and Swift for iOS are required.
You also need to have data centers. There are a few on-promise as well as cloud services that are needed, one needs to create read services way more than write services it should have a ratio of 9:1. Capacity estimation is required to determine strategies for cache management.
You need a database, either SQL or NoSQL. Backup solutions and data sharding plans are also needed to be prepared. For content streaming, third party services such as AWS is needed.
The all-out expense of the development of a video stream like YouTube or Netflix relies upon the number of features one wants to add. The more features it has, the more costly it is.
So, for the start, it is smarter to include fundamental features just into the first app version, to decrease its expense. Extra features can be included later when the app and website will begin picking up prominence.
The video streaming service is managing a huge amount of data while providing a stopless service. It must be fault-tolerant as well as has a backup mechanism. This paper explains the types of technologies used in video streaming platforms. The aim of this research was to do the breakdown of tools and technologies at the backend of video streaming platforms.
Following knowledge resources were also consulted while creating this piece of information.
- https://humanizing.tech/how-a-video-platform-works-191b2a7d1e47 (Everett, 2016)
- https://medium.com/@narengowda/netflix-system-design-dbec30fede8d (Narendra, 2018)
- https://www.cleveroad.com/blog/cost-to-create-a-streaming-service-like-netflix (Sergey 2020)
- Molina, L.E. and Bhulai, S., 2018. Recommendation System for Netflix.