It's Been One of Those Weeks!

the_Grinch

Awful week so far and I need to vent. This week has confirmed why I didn't want to do end-user support anymore. We have a hosted VOIP solution (it sucks) and I am the contact person for when something goes wrong with it (put it on the new guy who knows nothing about the phone system). So basically, something goes wrong I copy down the extension and time then e-mail it to the support desk. Now the issues have been getting worse and worse. Dropped calls, echoes, phones ringing and pick up no one is there. Users are getting pissed and when I get there to get the information I get yelled at and stared at when I say "I'll contact the company."

The worst part is that while dealing with this I am also the person who fixes any PC related issues. With the phones sucking I have to stop what I am doing and deal with the phones. Thus it takes all day to do something simple. On top of that, I have to help test, enter data, and train on Dynamics AX so that I can train salespeople on how to use it. The company dragged their feet with bringing me on (interviewed in Dec and was hired in June) and now the issues have just spiraled out of control. We're behind on the Dynamics stuff, thus it's costing more money because we have to keep the consultants for a longer period.

I bring up the money because it was money that was the sole basis for choosing the hosted VOIP. My direct boss wanted a Nortel system so we could control it and do what we have to when there are issues. They formed a committee and looked at what was available. Pricing came into play and it was going to run $50,000 to $100,000 for the Nortel system. Company didn't want to spend that and saw that the hosted VOIP would only cost $5,000. Now the VOIP company didn't do any testing on the network or even come in to take a look. It was a "yeah it will work fine, here's the router." Network is over 10 years old running routers and switches made by the business arm of home networking companies.

So we had a different company come in to get the network up to snuff (suppose to fix the VOIP issues we're having). Cost $30,000 for a couple of switches, a router, and the install (Cisco stuff thank god!). Now we're still having issues and we've probably lost another $30,000 or more in business because of the issues. Plus the inconvenience of dealing with phones that aren't working properly. They didn't go with the Nortel system because they didn't want to replace the switches (yeah we did that anyway!)

Vent over....for now!

msteinhilber

Are you sure you don't work for my company???

That is one of the most frustrating things, when organizations fail to do it right the first time around and it ends up being done twice (wrong way first, fixed up the second time - hopefullly) and usually at a cost higher than it would have been to do it right the first time - and that's not even counting the difficult to factor costs of decreased productivity due to the problems/downtime.

Is your boss who wanted the Nortel system a fighter? Or are they the type of manager that see's the value in doing it right and presents the facts but when push comes to shove they just roll over? Where I'm at now it's all too often roll over and hack it together and we'll all nag about it 24/7 but deal with it.

the_Grinch

My direct boss isn't a fighter, but he does get his point across. I really don't think any amount of fighting would have gotten it through. Had I been hired at the time, I might have been able to help a little more (my direct boss is a programmer by trade). But overall, I don't think it would have done much good. I had to fight to get them to buy the Cisco equipment as they didn't see the difference in Cisco equipment and business equipment from home networking companies besides the price ($5000 vs $600). I flat out told my other boss, this is a company and that we have that stuff currently and there are issues, why would you buy more of it? I did at least win that debate!

But back to the grind tomorrow...is it Friday yet?

kalebksp

That's rough man, phone systems can be a PITA if they're not setup correctly. On the bright side, maybe it's good thing they didn't invest in a Nortel systems, since Nortel went bankrupt. Out of curiosity, is your hosted PBX Fonality? They're quite popular, but I don't hear a lot of good things about them.

msteinhilber wrote: »

Are you sure you don't work for my company???

Haha! I was wondering if he took my position at the company I used to work for, except that there is no way they'd pay $5000 for a phone system. The only thing I miss from that place was tinkering with the phone system I built them (Asterisk based). Although I heard that one of the guys decided that the server was too loud and literally tore a fan out of it, while it was running, and that was the end of that.

Kaminsky

Is a very simple rule in I.T. You get what you pay for !

the_Grinch

At least I know I'm not alone! Nope not using Fonality.

adam.hughes

You know what they say "High tech, Low budget."

NightShade03

Hmm that's funny my company seems to be suffering from the complete opposite problem...we have tons of money and they keep buying tons of different things from different vendors and then implement them all at once!! Obviously this creates nothing but total chaos 24x7, maybe a smaller budget with less equipment would have been a blessing

the_Grinch

To add to the fun our T1 went down so no calls or internet. AT&T kept saying it wasn't them, so we called our network company to come out and paid a ton to find out it was AT&T. I have since requested that I be allowed to carry my flask with me at work!

blargoe

NightShade03 wrote: »

Hmm that's funny my company seems to be suffering from the complete opposite problem...we have tons of money and they keep buying tons of different things from different vendors and then implement them all at once!! Obviously this creates nothing but total chaos 24x7, maybe a smaller budget with less equipment would have been a blessing

I can top that... we HAD tons of money and kept buying things from different vendors to implement all at once... then we stopped having tons of money, cut our staff in half, cut back headcount all over the company, and expect the same level of service for all of these applications. Plus expecting new stuff to be implemented at the same rate as before. We have 400 or so employees and 135 servers. 1 server for every 3 employees... and still growing... HOW DOES THAT EVEN HAPPEN?????

Pardon the interruption... now back to your regularly scheduled rant in progress.

NightShade03

blargoe wrote: »

We have 400 or so employees and 135 servers. 1 server for every 3 employees... and still growing... HOW DOES THAT EVEN HAPPEN?????

Haha I understand the pain entirely although I can't explain how it works either...we have around 1000 users, 70 servers, 70 switches....5 IT staff (1 engineer, 4 techs)

I liked someone else's comment about carrying around a flask, although I prefer a cricket bat

Xcluziv

blargoe wrote: »

I can top that... we HAD tons of money and kept buying things from different vendors to implement all at once... then we stopped having tons of money, cut our staff in half, cut back headcount all over the company, and expect the same level of service for all of these applications. Plus expecting new stuff to be implemented at the same rate as before. We have 400 or so employees and 135 servers. 1 server for every 3 employees... and still growing... HOW DOES THAT EVEN HAPPEN?????

Pardon the interruption... now back to your regularly scheduled rant in progress.

there's just something wrong with 400 or so employees and 135 servers...WTF?!?!?

Xcluziv

the_Grinch wrote: »

To add to the fun our T1 went down so no calls or internet. AT&T kept saying it wasn't them, so we called our network company to come out and paid a ton to find out it was AT&T. I have since requested that I be allowed to carry my flask with me at work!

Duh, its def the ISP if the 'calls' and 'internet' are down...smh. And employers wonder how they drive people to drink.....because they do stupid stuff like this

*reaches in back pocket and hands Grinch his flask*

Forsaken_GA

NightShade03 wrote: »

Haha I understand the pain entirely although I can't explain how it works either...we have around 1000 users, 70 servers, 70 switches....5 IT staff (1 engineer, 4 techs)

I liked someone else's comment about carrying around a flask, although I prefer a cricket bat

You have no idea how much I would love your ratios. We have 10 guys to run a 24/7/365 support operation for thousands of servers. If it wasn't for automation, we'd be screwed.

NightShade03

HA! I'd trade you in a minute! I would prefer 1000's of servers vs 1000's of users, I prefer scripting & testing rather then helping people and the stupidest questions on the planet!

blargoe

Xcluziv wrote: »

there's just something wrong with 400 or so employees and 135 servers...WTF?!?!?

We are a child company of a Fortune 500 so we get very favorable pricing on MS licensing and Dell hardware, which help get us where we are... that and management being afraid to tell the business "no" or "let's think this through" when they demand something...

We support 5 datacenters with 2 AD's, 2 sites with full Exchange, OCS, and Enterprise Vault deployments, several SharePoint Servers (with additional development servers running in tandem), about 15 SQL Servers, Websense Enterprise at every site, servers for our phone system at every site, NetBackup at every site, a 6 server SCOM deployment, SAV/SEP servers everywhere, 8 Citrix servers, HA clusters for our conveyor systems at each location, and many departments have their "production" applications that may or may not have dedicated Windows instances depending on system requirements, history, and company politics. Then the standard stuff like file and print at each location. And EMC SAN's at two sites. Moving all of it to VMWare.

We have one systems admin/engineer/architect (me), and Jr. Admin, a Network Engineer, two desktop/helpdesk, one tech in our China office (15 servers and maybe 75 users), a manager and a director. We've gotten by on investing in training and having (for the most part) the right people in place, but even so we're just treading water... barely.

It's always one of those weeks where I work. I could have my own regularly updated thread on the crap that goes on here.

Xcluziv

blargoe wrote: »

We are a child company of a Fortune 500 so we get very favorable pricing on MS licensing and Dell hardware, which help get us where we are... that and management being afraid to tell the business "no" or "let's think this through" when they demand something...

We support 5 datacenters with 2 AD's, 2 sites with full Exchange, OCS, and Enterprise Vault deployments, several SharePoint Servers (with additional development servers running in tandem), about 15 SQL Servers, Websense Enterprise at every site, servers for our phone system at every site, NetBackup at every site, a 6 server SCOM deployment, SAV/SEP servers everywhere, 8 Citrix servers, HA clusters for our conveyor systems at each location, and many departments have their "production" applications that may or may not have dedicated Windows instances depending on system requirements, history, and company politics. Then the standard stuff like file and print at each location. And EMC SAN's at two sites. Moving all of it to VMWare.

We have one systems admin/engineer/architect (me), and Jr. Admin, a Network Engineer, two desktop/helpdesk, one tech in our China office (15 servers and maybe 75 users), a manager and a director. We've gotten by on investing in training and having (for the most part) the right people in place, but even so we're just treading water... barely.

It's always one of those weeks where I work. I could have my own regularly updated thread on the crap that goes on here.

Sounds like its never a dull moment at your job. Always some coonery or foolishness going on. Something always needing repairing or replacing.

Why the heck is management afraid to say the forbidden word.."no". Thats what they are there for, but i guess they are some pack rats and just want everything

blargoe

Xcluziv wrote: »

Sounds like its never a dull moment at your job. Always some coonery or foolishness going on. Something always needing repairing or replacing.

Why the heck is management afraid to say the forbidden word.."no". Thats what they are there for, but i guess they are some pack rats and just want everything

Don't get me wrong, I've played this to my benefit as much as possible, lots of great training and experience with enterprise level applications, for the most part we have support set everything to best practices spec. But you can only stuff so much... stuff... in a 10 gallon bag. Ya know?

blargoe

Sorry, didn't intend to hijack the thread.

Xcluziv

blargoe wrote: »

Sorry, didn't intend to hijack the thread.

I guess its all relative....people venting about problem at work

apena7

Xcluziv wrote: »

there's just something wrong with 400 or so employees and 135 servers...WTF?!?!?

I guess there's always the option for clustering....talk about redundancy.

Forsaken_GA

NightShade03 wrote: »

HA! I'd trade you in a minute! I would prefer 1000's of servers vs 1000's of users, I prefer scripting & testing rather then helping people and the stupidest questions on the planet!

Then you wouldn't want to trade with me. I work for a web hosting company, so I get thousands of users along with those thousands of servers It's very frustrating when you're dealing with a company who's employee's aren't on the same page and changes that one directs you to make breaks something else the other guys are working on.

To give you an idea of the caliber of intelligence I deal with on a weekly basis -

Initial ticket entry - Customer: "My sites don't load anymore, please fix it"
Support Response: "Your sites are no longer loading because your domain has expired. Once you renew your domain with your registrar, your sites will begin loading again within hours"
Customer Response: "I'm losing thousands of dollars with this downtime, you need to fix your DNS servers so my sites will load"
Support Response: "Sir, we are not your registrar, we have no control over whether or not your domains are expired or not, and it's not our DNS servers that are the problem. You will need to renew your domains with <insert registrar here, usually Godaddy> in order for your sites to load"
Customer Response: "Oh, ok."
<few hours pass>
Customer Response: "I renewed the domain and my sites still aren't loading"
<quickly review the whois record and doublecheck the root nameservers just to see which DNS servers are authoritative, just to find out they're not us>
Support Response: "That domain is not pointed at our DNS servers, nor is it pointed at an IP address on our network. It looks like the domain is hosted by <insert competitor here>. You will need to contact them in order to get your site loading."
Customer Response: "Oh yeah, I forgot I moved that site last week. Thx for help"

And so on.

Personally, I like our customers being fat and dumb. It may take a little pounding to get something through their skulls, but they'll generally start behaving intelligently after they've taken that pounding. The customers who know just enough buzzwords to sound like they know what they're talking about are the ones who drive me batty.

the_Grinch

Yeah a bit of whining on my part! I can say that in the 5 weeks I have been at my job, the first month was great. My real frustration is that there isn't anything I can do when the network or phones go down. I can call the respected companies and that is it....

Forsaken_GA

The truth of the matter is that there are very few companies which are well run, and not many that make it a habit to go back and correct problems when they crop up. Even if there's a better way to do things, you'll often be up against institutional inertia, where the only reason that something is done a certain way is because "that's how it's always been done". Fighting against that is an uphill battle, especially if it involves spending money to effect a change. We all have our little gripes about our jobs.

In Network Warrior by Gary Donahue, he has a maxim -

Network designs are based on Politics, Money, and The Right Way To Do It - In that order.

I'd go so far as to extend that to every facet of a company, not just network design. If you've been around for a bit, you know just how true that is. I used to try and be agnostic about company politics in the beginning of my career. Later on I found out to my dismay, that the choices are to either play the game, or let the game play you.

/usr

I used to try and be agnostic about company politics in the beginning of my career. Later on I found out to my dismay, that the choices are to either play the game, or let the game play you.

There couldn't be more truth in that. In between my current job, last job and all of the completely insane decisions I've seen our clients make, I've come to realize that company politics is the deciding factor nearly 100% of the time.

I've never been a fan of playing those types of games and I'm still coming to realize that, like you said, it's either play or be played.

Lizano

My experience is, the smaller the company, the easier it is to get things done. Of course every rule has it´s exceptions, but normally that´s what I come across...

mikedisd2

/usr wrote: »

I've never been a fan of playing those types of games and I'm still coming to realize that, like you said, it's either play or be played.

What did Ralph Nader say? "Turn on to politics, or politics will turn on you."

chmod

I've been a contractor, consultant and in-house IT and i can say most of the IT guys, lacks the management experience. We have to proactive not reactive and we need first to be able to handle ourselves and our tasks i mean we need to first learn how to be our own managers even if it sounds stupid, then we need to play the management game, whatever we do we have to see it as project and define a goal for our IT department then we need to play management game in a higher level, what i mean is that we need to learn how to define timelines/timeframes/deadlines for our tasks, learn what ROI is and know several ways to do things then show to management why they have to do things the right way with a brief and straightforward explanation then show the ROI, the risks, the dates and the cost/ROI co-relation and let them choose but this needs to be done in a proactive way.

Whining is not an option, if thing were done/chosen/implemented the wrong way we should prepare a plan to show different option in how to make things to work correctly with ROI,dates, costs but for this we need to be capable to design good solution that are cost effective and know the business very well to be able to find/design solutions to business needs.
Think that you are preparing a financial plan or a marketing campaign so you should should prepare and show everything in a very professional way. If they don't care or refuse you know that at least u tried it and you covered your back but it is important to be very aggressive and confident to be respected and gain their trust.

Off course if you don't know the different ways to fix/implement things or to design or if you you come up with a bunch of B/S thing will remain the same.