Netflix streaming now runs entirely from AWS

DoubleNNs

Netflix started a cloud-migration 7 years ago after a major fire in one of their data centers. And now they've just finished moving into AWS. Their entire streaming service is ran from AWS, although the actual content is cached using their own CDN.
However, their DVD rental service is still ran from a colo-ed DC.

Netflix finishes its massive migration to the Amazon cloud | Ars Technica

techfiend

I wonder where netflix does the encoding. I really doubt they upload a full BD or whatever the source is to AWS then pay for all the cycles to encode it. That would be a massive bill. They also seem store the video at AWS but serve from their own CDN for likely another massive savings, but maybe not. I wonder if AWS had any say in that decision.

DoubleNNs

Transfer bandwidth into AWS is free and the encoding is most likely done on AWS. Netflix has talked at length in the past about their video encoding, but I don't remember ever noting what type of infrastructure/applications they use. Most likely their own custom app/ built on EC2. If they use spot instances would save a ton of money. I think they encode every video once and then just store the output. So potentially, that might not be an expensive part of their business.

The thing I find pretty funny is that Netflix is arguably one of Amazon's biggest AWS customers. Yet Amazon Prime Video streaming is in direct competition with Netflix. You have to wonder how much Amazon learned from Netflix's deployment when designing their own Video streaming service. Also weird how even if everyone uses Netflix instead of Prime Video, Amazon still gets paid regardless (tho possibly less so in the future, since as part of DR Netflix is already set up to move their workloads to other cloud solutions at the press of a button).

techfiend

Wouldn't spot instances lead to frames not encoded due to lost instance? I'm sure they have a pretty sophisticated process but for efficient video encoding it requires read ahead to make predictions. I suppose you could send a group of frames to a spot instance and if it doesn't complete send the job to another spot instance.

DoubleNNs

techfiend wrote: »

I suppose you could send a group of frames to a spot instance and if it doesn't complete send the job to another spot instance.

Yeah, that's what I believe they probably do. Their whole infrastructure is supposed to have a crazy level of redundancy and HA. Their "Simian Army" randomly kills instances, simulates Availability Zones going down, and even entire Regions, as a test for the developers to make sure their applications won't be affected by outages/system issues. A spot instance going down, or even 100, probably wouldn't affect them too much. Especially if they do the encoding ahead of time and store the output -- their entire spot cluster doing the encoding going down would make the encoding take longer, but that'd just delay the release of an Orange is the New Black episode or something, instead of being a customer facing issue.
This is, however, 100% my speculation.

techfiend

OITNB delay would come with a lot of customer criticism but I'm sure they have plenty of time to get the encoding done. Likely weeks to do a days job.