So finally Gmail went down for 150 million people

binarysoul

BBC NEWS | Technology | Engineer error knocks out Gmail

And Google thought its infrastructure was unbreakable! I'm sure there are many red faces in Google now.

matradley

binarysoul wrote: »

BBC NEWS | Technology | Engineer error knocks out Gmail

And Google thought its infrastructure was unbreakable! I'm sure there are many red faces in Google now.

That is the first mistake any security analyst should make about any network.

Forsaken_GA

There is no such thing as an unbreakable infrastructure. Everything is susceptible to human error. I'd bet dollars to donuts I'm not the only one on this board who's had a slight miscalculation and caused a minor outage and then had to sit through a meeting with upper management about how I wasn't going to do that anymore.

binarysoul

Forsaken_GA wrote: »

There is no such thing as an unbreakable infrastructure. Everything is susceptible to human error. I'd bet dollars to donuts I'm not the only one on this board who's had a slight miscalculation and caused a minor outage and then had to sit through a meeting with upper management about how I wasn't going to do that anymore.

We're not talking about the mail server of a company going down. We're talking about 0.15 billion users! That's 5 times the population of Canada It wasn't 'human error', it was bad planning. I've been in the field long enough to appreciate human error and system glitches, but boy when you affect tens of millions people one cannot be very sympathetic and supportive of Google. By the way, I don't use Gmail.

Should people complain? Maybe not, after all Gmail is free. Good wake up call for those who use gmail for business purposes. I know I've come hard on Google, but I will also say it's doing a better job that $Microsoft

fommy

did the engineer in question get the sack from his mistake?

arwes

They should've never come out of beta. When there's an issue, you can always use the "it's just a beta" defense!

skrpune

I have to say I'm surprised and impressed that they admitted what happened. Everyone messes up sometimes, and for them to openly admit that it was human error is pretty...well, human of them!

Pash

skrpune wrote: »

I have to say I'm surprised and impressed that they admitted what happened. Everyone messes up sometimes, and for them to openly admit that it was human error is pretty...well, human of them!

exactly, and human error is very common in IT as we all know, miscalculation and bad preperation also occur, its about minimizing these issues to keep a good uptime record.

everybody seems to wan't to hate the success story that is google, bit of a shame really if you ask me.

wd40

binarysoul wrote: »

Should people complain? Maybe not, after all Gmail is free. Good wake up call for those who use gmail for business purposes.

Google & Gmail are not free, no users = no advertisement revenue.
They have to keep "Free custmers" happy or they will go to other sites.

++ on the using Google for business issue.

dynamik

So Gmail and your mom have a lot in common, right?

Sorry, I had to.

msteinhilber

dynamik wrote: »

So Gmail and your mom have a lot in common, right?

Sorry, I had to.

I'm confused.

Are you referring to the fact that gmail "went down" or that it "serves" nearly 150 million people?

Paul Boz

Gmail did not go down. Gmail's web interface did. If you are using POP or IMAP you did not notice a problem. Hopefully if you're using Gmail for corporate use you're using POP or IMAP. also, if you're using gmail for corporate use you should probably register a domain and do your own email. It's cheaper and more professional.

I'm pleased with the response that google had to the community. they eloquently owned the problem, explained what happened, and resolved it. It is impossible to guarantee complete up time. When I worked at an ISP our goal was 100% up time but that's impossible and we knew that. Real engineers pride themselves on fixing problems as quickly as possible because they know these things happen.

dynamik

msteinhilber wrote: »

Are you referring to the fact that gmail "went down" or that it "serves" nearly 150 million people?

All of the above.

And yes, POP/IMAP were fine as noted in another thread on the subject. I was still able to send/receive on my iPhone (I have gmail hosting the email for my domain), so it really wasn't that big of a deal.

Forsaken_GA

binarysoul wrote: »

We're not talking about the mail server of a company going down. We're talking about 0.15 billion users!

We're talking about the web interface to a service that went down for a bit because an engineer overestimated the ability of the other servers to handle the load. Not the first time it's happened. Won't be the last.

It wasn't 'human error', it was bad planning.

Right, because making incorrect calculations on your ability to handle capacity for maintenance windows is *certainly* not a human error. It must have taken a computer to do that. Nevermind the fact that a human somewhere had to sign off on it, thereby making it *their* error.

Knock off the semantic crap. Someone made a mistake that resulted in an outage. End of story. Happens every single day, no matter branch of IT you're in.

I've been in the field long enough to appreciate human error and system glitches, but boy when you affect tens of millions people one cannot be very sympathetic and supportive of Google. By the way, I don't use Gmail.

I can be quite sympathetic to Google. The only reason this is 'news' is *because* it's Google, it doesn't happen that often. Comcast has frequent outages on a regional basis. Do you hear about them that often? Same thing with Shaw, RoadRunner, and a plethora of other ISP's, hosting providers, transit providers, et al.

Ahriakin

The only engineer that doesn't make mistakes is the one with access to the logs....
History may be written by victors but it is edited by geeks ( (c) Me 2009 )

Lizano

Forsaken_GA wrote: »

. Comcast has frequent outages on a regional basis. Do you hear about them that often? Same thing with Shaw, RoadRunner, and a plethora of other ISP's, hosting providers, transit providers, et al.

+1 to that. Heck, back when I worked for a top netorking manafucturer, one of the engineers had the brilliant idea of running a debug command on the HQ Core router of a customer that we managed remotely, that rtr hosted 1000+ VPN conections...of course the router hung, and humilliatingly enough we had to call the customer and ask them to physically reboot the rtr because we had screwed up...

It happens...