Netflix streaming now runs entirely from AWS

DoubleNNsDoubleNNs Posts: 2,013Member ■■■■■□□□□□
Netflix started a cloud-migration 7 years ago after a major fire in one of their data centers. And now they've just finished moving into AWS. Their entire streaming service is ran from AWS, although the actual content is cached using their own CDN.
However, their DVD rental service is still ran from a colo-ed DC.

Netflix finishes its massive migration to the Amazon cloud | Ars Technica
Goals for 2018:
Certs: RHCSA, LFCS: Ubuntu, CNCF CKA, CNCF CKAD | AWS Certified DevOps Engineer, AWS Solutions Architect Pro, AWS Certified Security Specialist, GCP Professional Cloud Architect
Learn: Terraform, Kubernetes, Prometheus & Golang | Improve: Docker, Python Programming
To-do | In Progress | Completed

Comments

  • techfiendtechfiend Posts: 1,481Member ■■■■□□□□□□
    I wonder where netflix does the encoding. I really doubt they upload a full BD or whatever the source is to AWS then pay for all the cycles to encode it. That would be a massive bill. They also seem store the video at AWS but serve from their own CDN for likely another massive savings, but maybe not. I wonder if AWS had any say in that decision.
    2018 AWS Solutions Architect - Associate (Apr) 2017 VCAP6-DCV Deploy (Oct) 2016 Storage+ (Jan)
    2015 Start WGU (Feb) Net+ (Feb) Sec+ (Mar) Project+ (Apr) Other WGU (Jun) CCENT (Jul) CCNA (Aug) CCNA Security (Aug) MCP 2012 (Sep) MCSA 2012 (Oct) Linux+ (Nov) Capstone/BS (Nov) VCP6-DCV (Dec) ITILF (Dec)
  • DoubleNNsDoubleNNs Posts: 2,013Member ■■■■■□□□□□
    Transfer bandwidth into AWS is free and the encoding is most likely done on AWS. Netflix has talked at length in the past about their video encoding, but I don't remember ever noting what type of infrastructure/applications they use. Most likely their own custom app/ built on EC2. If they use spot instances would save a ton of money. I think they encode every video once and then just store the output. So potentially, that might not be an expensive part of their business.

    The thing I find pretty funny is that Netflix is arguably one of Amazon's biggest AWS customers. Yet Amazon Prime Video streaming is in direct competition with Netflix. You have to wonder how much Amazon learned from Netflix's deployment when designing their own Video streaming service. Also weird how even if everyone uses Netflix instead of Prime Video, Amazon still gets paid regardless icon_smile.gif (tho possibly less so in the future, since as part of DR Netflix is already set up to move their workloads to other cloud solutions at the press of a button).
    Goals for 2018:
    Certs: RHCSA, LFCS: Ubuntu, CNCF CKA, CNCF CKAD | AWS Certified DevOps Engineer, AWS Solutions Architect Pro, AWS Certified Security Specialist, GCP Professional Cloud Architect
    Learn: Terraform, Kubernetes, Prometheus & Golang | Improve: Docker, Python Programming
    To-do | In Progress | Completed
  • techfiendtechfiend Posts: 1,481Member ■■■■□□□□□□
    Wouldn't spot instances lead to frames not encoded due to lost instance? I'm sure they have a pretty sophisticated process but for efficient video encoding it requires read ahead to make predictions. I suppose you could send a group of frames to a spot instance and if it doesn't complete send the job to another spot instance.
    2018 AWS Solutions Architect - Associate (Apr) 2017 VCAP6-DCV Deploy (Oct) 2016 Storage+ (Jan)
    2015 Start WGU (Feb) Net+ (Feb) Sec+ (Mar) Project+ (Apr) Other WGU (Jun) CCENT (Jul) CCNA (Aug) CCNA Security (Aug) MCP 2012 (Sep) MCSA 2012 (Oct) Linux+ (Nov) Capstone/BS (Nov) VCP6-DCV (Dec) ITILF (Dec)
  • DoubleNNsDoubleNNs Posts: 2,013Member ■■■■■□□□□□
    techfiend wrote: »
    I suppose you could send a group of frames to a spot instance and if it doesn't complete send the job to another spot instance.

    Yeah, that's what I believe they probably do. Their whole infrastructure is supposed to have a crazy level of redundancy and HA. Their "Simian Army" randomly kills instances, simulates Availability Zones going down, and even entire Regions, as a test for the developers to make sure their applications won't be affected by outages/system issues. A spot instance going down, or even 100, probably wouldn't affect them too much. Especially if they do the encoding ahead of time and store the output -- their entire spot cluster doing the encoding going down would make the encoding take longer, but that'd just delay the release of an Orange is the New Black episode or something, instead of being a customer facing issue.
    This is, however, 100% my speculation.
    Goals for 2018:
    Certs: RHCSA, LFCS: Ubuntu, CNCF CKA, CNCF CKAD | AWS Certified DevOps Engineer, AWS Solutions Architect Pro, AWS Certified Security Specialist, GCP Professional Cloud Architect
    Learn: Terraform, Kubernetes, Prometheus & Golang | Improve: Docker, Python Programming
    To-do | In Progress | Completed
  • techfiendtechfiend Posts: 1,481Member ■■■■□□□□□□
    OITNB delay would come with a lot of customer criticism but I'm sure they have plenty of time to get the encoding done. Likely weeks to do a days job.
    2018 AWS Solutions Architect - Associate (Apr) 2017 VCAP6-DCV Deploy (Oct) 2016 Storage+ (Jan)
    2015 Start WGU (Feb) Net+ (Feb) Sec+ (Mar) Project+ (Apr) Other WGU (Jun) CCENT (Jul) CCNA (Aug) CCNA Security (Aug) MCP 2012 (Sep) MCSA 2012 (Oct) Linux+ (Nov) Capstone/BS (Nov) VCP6-DCV (Dec) ITILF (Dec)
Sign In or Register to comment.