AWS S3 Outage
Anyone having a bad day? Hope everyone's BCP plan is working as designed.
Comments
-
chrisone
Member Posts: 2,278 ■■■■■■■■■□
Azure for the win?
Certs: CISSP, EnCE, OSCP, CRTP, eCTHPv2, eCPPT, eCIR, LFCS, CEH, SPLK-1002, SC-200, SC-300, AZ-900, AZ-500, VHL:Advanced+
2023 Cert Goals: SC-100, eCPTX -
alias454
Member Posts: 648 ■■■■□□□□□□
Always plan for failure no matter how robust something claims to be.“I do not seek answers, but rather to understand the question.” -
bornwith
Member Posts: 21 ■■□□□□□□□□
Saw an article in which Amazon claimed it was human error. Apparently some dude or dudette as the case man shut down the wrong server for maintenance. It caused a cascade effect. Wonder if they still have s job? -
Kinet1c
Member Posts: 604 ■■■■□□□□□□
Anyone having a bad day? Hope everyone's BCP plan is working as designed.
Yup, we'd a bad day with this but not catastrophic. We've a few items on our to do list after our internal post mortem. It was a late night considering we're in the EU.
2018 Goals - Learn all the Hashicorp products
Luck is what happens when preparation meets opportunity -
Verities
Member Posts: 1,162
DatabaseHead wrote: »We are on Azure no complaints.
Historically, if you compare the amount of outages that Azure has experienced, you will see they occur more frequently with AWS. -
DatabaseHead
Member Posts: 2,754 ■■■■■■■■■■
We've been on Azure for over a year never experienced anything like this........Historically, if you compare the amount of outages that Azure has experienced, you will see they occur more frequently with AWS. -
Verities
Member Posts: 1,162
DatabaseHead wrote: »We've been on Azure for over a year never experienced anything like this........
Let's take a quick look at the history of Azure's outages:
Global DNS outage hits Microsoft Azure customers | ZDNet
Microsoft Azure hit with widening outages in Europe and India - GeekWire
Azure Nightmare: Customer Suffers 9-Day Intermittent Outage, Gets No Help From Microsoft - Page: 1 | CRN
https://www.bloomberg.com/news/videos/b/689903ff-6758-411a-b834-44c813fd81c9
https://www.forbes.com/sites/benkepes/2014/11/18/its-a-return-to-the-azure-alypse-microsoft-azure-suffers-widespread-outage/#595c99d7421d
Microsoft Azure had more downtime in 2014 than main cloud rivals -
DatabaseHead
Member Posts: 2,754 ■■■■■■■■■■
Let's take a quick look at the history of Azure's outages:
Global DNS outage hits Microsoft Azure customers | ZDNet
Microsoft Azure hit with widening outages in Europe and India - GeekWire
Azure Nightmare: Customer Suffers 9-Day Intermittent Outage, Gets No Help From Microsoft - Page: 1 | CRN
https://www.bloomberg.com/news/videos/b/689903ff-6758-411a-b834-44c813fd81c9
https://www.forbes.com/sites/benkepes/2014/11/18/its-a-return-to-the-azure-alypse-microsoft-azure-suffers-widespread-outage/#595c99d7421d
Microsoft Azure had more downtime in 2014 than main cloud rivals
Like I said WE never experienced an outage....... Completely satisfied with their service and most certainly never experienced an outage of this magnitude.
Amazon took a massive hit, they look like a joke at the moment and unreliable.
On a side note, the engineer, errr ex-engineer who screwed it up was released.....
http://gizmodo.com/amazon-says-one-engineers-simple-mistake-brought-the-in-1792907038 -
paul78
Member Posts: 3,016 ■■■■■■■■■■
The Amazon post-mortem if anyone is interested - https://aws.amazon.com/message/41926/
-
alias454
Member Posts: 648 ■■■■□□□□□□
DatabaseHead wrote: »On a side note, the engineer, errr ex-engineer who screwed it up was released.....
Amazon Says One Engineer's Simple Mistake Brought the Internet Down
I read that article and didn't see where it mentioned the engineer in question was released?“I do not seek answers, but rather to understand the question.” -
paul78
Member Posts: 3,016 ■■■■■■■■■■
I read that article and didn't see where it mentioned the engineer in question was released?
Yeah - I don't see a mention of it anywhere. It would be very unfortunate and a bit unusual if Amazon released the engineer for this type of mistake. And any disciplinary measures certainly wouldn't be something that Amazon announces or discloses anyways. -
EagerDinosaur
Member Posts: 114
From the accounts I've read, I'd say that the AWS internal processes were the main cause of the outage. Expecting engineers to always correctly type command line parameters on live systems isn't reasonable.
This hasn't discouraged me from using the cloud. The on-premises infrastructure at the large IT company I work for seems considerably less reliable than AWS. -
DatabaseHead
Member Posts: 2,754 ■■■■■■■■■■
My bad wrong link posted. I was following a twitter feed that mentioned he was let go, of course heresay so worthless nonetheless. Will be interesting to see what other news comes out in the next couple of days.And any disciplinary measures certainly wouldn't be something that Amazon announces or discloses anyways.
Well of course not, I was thinking along the lines of unnamed sources. But of course that would never happen.....
