Whats the worst jury rigging you have ever come across in IT?

CodeBlox

Doesn't have to be something you did but maybe a former admin. You know, some situations require a quick fix and it can be tempting to jury rig a solution together.

LarryDaMan

I've always used the term jerry-rigging, but jury-rigging means the same. (Interesting. Thanks Merriam-Webster)

I can't think of any great examples. I've seen many messy cabling jobs before, and for years, guys used Fluke tools to find a troubled connection instead of ever fixing and labeling the terrible cabling.

jvrlopez

When I worked in a server room, we had a switch/distribution panel for fiber SC connectors that started off with a few nodes running here and there.

After a while, there were probably about 200 pairs running all over the place.

Port security was never configured so the techs and admins just plugged in whatever wherever they wanted. Around the time it finally became an issue and higher ups were complaining (a distinguished visitor came through and leadership thought it was bad that he 'could've' seen that), we were told to make it "look nice." Port security was enabled right around this time so everything was locked down in place.

It got so bad that door could not close without considerable force and we were advised against opening it unless we had to run a new connection.

We finally had to come in one weekend, configure port security, and reroute everything and tie it down with velcro and route it down the side of the rack.

kanecain

My "un-named" company has several sites that are all connected using static routes...no OSPF, no EGIRP, hell not even RIP. ALL static routes.

SteveO86

Once to get around a faulty modem, I had to create a kron job to clear the arp cache of the router every 2 minutes.

It was a temp stop-gap until the carrier could get out there to replace modem. (Debugged ARP to find the modem sending weird ARP responses, and clearing the ARP cache resolved the issue until the modem freaked out again, usually every 3+ minutes)....

---

Had to disable CEF one a 3825 router to resolve some weird packet loss issue. (It was a long time ago)

---

Put a GRE into a bridge group for some L2 services to work across a L3 WAN. (Not sure I would call the jury-rigging though). Temp solution until a proper L2 circuit could get installed.

--- All that pertains some of cisco or network device.

Qord

We had an old (out of warranty) server that needed the bios battery replaced. When the work was done, the little plastic thing that holds the battery in place was broken off. Someone took a pencil, placed it eraser-side down, and taped the hell out of it to keep the battery in place. Couldn't even put the side panel of the case back on, the pencil was sticking out too much.

Replacement part would have cost about $2, 20 minutes of my time, and about an hour of total downtime. It was decided that this was too expensive to do. This server has since been replaced.

olaHalo

End user- My Excel doesn't open
Me- Deploy entire Windows Image with Excel from my desk.

kanecain

olaHalo wrote: »

End user- My Excel doesn't open
Me- Deploy entire Windows Image with Excel from my desk.

I second that! Who has the time to figure some stuff out.

ajs1976

Windows NT Domain. None of the workstations were joined to the domain and the users had local and domain accounts with matching passwords.

alan2308

We replaced a guy at a company who refused to spend a penny. I give the guy credit, he had some creative solutions. My favorite was the edge of their network where he had 4 Linksys home routers in series. He had everything on the network statically addressed so that it would go out the right router and therefore be NAT'ed to the right address.

eansdad

I've seen standard Cat5 running between buildings, running from the building to the ground up the fence between the wood fence, painted same color as fence to the 2nd building and up the wall.

I've seen 50ft cables coupled together to make longer runs. I've seen a run go to a drop, loop into the other drop and continue to the switch. I've seen drops that are plugged into a drop in the room next door for their connection.

It a wonder why a single bad 3Com NJ220 4 port mini switch in 1 building can take down 1/3rd of our district.

We are supporting a very large iPad cluster f*ck. Solution to the Bonjour protocol issue...APs will be set to 2 separate SSIDs, 1 for M$ laptops and Android devices and the other for Apple products. The Apple side will be Vlan's together (20+ buildings with about the same in subnets) and the SSID will run 0 authentication and security. Sure it will work, sorta...the AppleTVs are still plugged into drops and the APs are still over worked (40-60 connection on a single AP. wireless G with the AP plugged into 100mb port, at any given time). District is paying an engineer to come up with and do this project instead of purchasing a new core and layer 3 switches/routers (which we need to do anyway) that can handle it. In the end, the cost of the engineer will be about the cost of the core. Then again these are the same people who brought in an engineer to find that 3Com NJ220 because our Admins couldn't figure it out and wouldn't ask for help. Turned out that 1 of the Admins had swapped that NJ220 the day before that part of the network went down. Never thought to double check it....


Long story short...I've seen some really bad jerry-rigging and I've got pics and a short video to prove it....lol

the_Grinch

In a school district I was in they promised one of the teachers a color printer. My boss said the pile of printers in one area were all in working order and to plug it in to make sure it powered up. Plugged it in, opened it, and everything looked good (it was marked "GOOD"). Have the school order the ink and let the teacher know we were going to install it. Ink comes, I place it in and install it in our office to test. The black ink shows as not available and I was very confused. Open it and find out that the plastic clip that holds the ink in place was broken so it wasn't setting right. My boss flips and says "we don't have money for the printer you better figure it out you said it was working". Go to the school nurse get Advil and a tongue depresser. Take the Advil and using a rubber band, super clue, and the tongue depresser fashion a clip to hold it in place. Printer worked for an entire year before they got the money to replace it.

At my first full time job they had a server that had all the labels for chemical products on it. Basically the user would open the share drive (housed on this server) and print the needed labels. My boss didn't know that the server wasn't being backed up and since it was at least 10 years old it finally died one day. Production came to a halt as they couldn't label the products and ship. The server being so old, getting parts for it was not going to happen and there was an issue of some kind with the hard drive connection. Thankfully, one of our MSP consultants was there working on a project and said he'd take a look. An hour later he had an old PC using the PSU to power the server and another old server with the right connection to get the drive back up. Literally three machines on the floor all interconnected, was the greatest thing I've ever saw.

RobertKaucher

A company that shall remain nameless had 12 sites spanning 2 states all connecting back to corporate headquarters using consumer Linksys routers but worse, the computer systems all used telnet (no encryption, no VPN) over custom ports to access their ERP/CRM software. Customer data was being sent over these connections but payment information was sent through another system. I never confirmed that the data being sent over the open Internet included credit card info but I know for a fact that regular customers could set up a method that they could just call in an order and payment would be done at the store and a driver would be sent out to deliver their order. So I suspect that credit card numbers were stored in those systems but I cannot be certain.

Another that I did not witness myself, was a consultancy that signed their customers up for free DropBox accounts to use as their application's message queuing infrastructure.

Cpl.Klinger

I ran 300' of RJ-11 phone line once (in 50' segments) to get our backup credit card machine working when our data network went out. Need I mention that this was run outside? In the middle of winter? I wrapped each coupler in plastic wrap from our restaurant, and then duct taped over the shrink wrap. It was one of those "I don't care what you do to make it work, just make it work" directives from my boss. Worked the three days it took to get everything up and running again.

bermovick

Just last week we had to replace someone else's switch with one of ours temporarily. They require PoE but our only spare PoE switch was a 3550. They had fiber from the switch to their main building.

The fiber from the switch to their patch panel used LC connectors. The 3550 takes an SC connector and we had no cables with LC on one end and SC on the other. The patch panels for our network take ST connects, and we DID have cables that were SC-to-ST and LC-to-ST. We ended up pulling out a small portion of an unused patch panel to bridge 2 cables, ending up with an SC-ST/ST-LC cable to take out along with the switch.

Too bad the 3550 apparently isn't 302.af compliant so it still didn't work...

jibbajabba

Does fixing a 3750G Stack with cable ties count

instant000

I got this one often in a manufacturing company: A ten foot cable used where a twenty footer would be more appropriate.

The result: A cable stretched to the greatest degree, in mid-air.

We had a barcoding solution, and for whatever reason, if the server-side app ran for more than a couple of days, they flipped out and the guns could no longer scan. We knew it was a driver issue of some sort, but the vendor didn't really fix it. The solution: write an expect script that would kill the processes, clear out the working directory, and relaunch the drivers. The drivers were each run under a different account. The script was placed in a job, so that it would restart the drivers, and then send the network admins an e-mail that showed the drivers status (so, for whatever reason if there was an issue, we could intervene).

I had another case where we had a remote office that connected back to our office via VPN. They would complain that they couldn't log in right away when their work day started. At the time, I didn't know as much as I did now, but I did know that the VPN required them to generate interesting traffic. I set up a job on a station at the site that was "always on" to ping the home site periodically throughout the working day schedule, starting at 30 minutes prior, and ending at 30 minutes after, to keep the tunnel up. Probably would have a more elegant solution nowadays.

The intercom system was a phone wired into the speakers. You dial that particular extension, and it connected through hardwiring of the phone receiver to the PA system. (Maybe this is standard practice, but it looked pretty rigged together, to me.)

Oooh, I almost forgot a really good one. International data lines (from U.S. to Mexico) are terribly expensive. The workaround: terminate the circuit in EaglePass, and set up a wireless bridge to shoot the signal across the border to Piedras Negras. Except for really severe weather, the wireless bridges wouldn't drop off. We had bigger issues with maintaining power to the bridges, or with the data circuit itself.

it_consultant

This isn't necessarily jerry-rigged but it was a head scratcher. My buddy took over a network from a guy that was into experimenting. The core switches were Pre-Dell Force 10s, he could never get STP to work right (and apparently never heard of LACP) so to solve the problem of ports being blocked, he implemented metro ring protocol in his datacenter.

redz

I stuck a netgear hub in a guard post between two of our offices because of attenuation. Had to run 75 feet of extension cords just to get it power. Of course, the amount of CAT5 splicing I was doing didn't help matters at all. Yeah, I don't miss Iraq.

ecoblue

At a law firm, the legal practice management software has a separate calendaring function that can be synced to Outlook (so that the data in Outlook can be synced to smartphones via Exchange Activesync).

Previously, the calendaring function to Outlook sync was a one-way manual export process. The attorneys were comfortable with it and it worked well. After the most recent upgrade, the calendaring function to Outlook sync now happens automatically and is a two-way sync.

Unfortunately, the sync function now imports all the calendar items from Outlook into the legal practice management software, including the items it just exported to Outlook! The vendor's solution? Run a script on the database server to clear the duplicate entries from the database after each sync...

--chris--

Sage software = jerry rigging to the huxtabillion power (made up that word for dramatic effect).

Its not really an IT thing (sorta, more like poor programming) but the last place I worked used Sage software. Sage would release a major update for one of its core suites of software every 4-5 years, and minor updates every 9-12 months (minor updates/major updates = same price). We would frequently find bugs where a certain function would just not work under a certain condition. It was definitely a bug, we could reproduce without fail every time. I started emailing support (which is about $1200/year) to let them what we ran into, and how to proceed with the reporting process and if this will be addressed in a future update.


I cant find the emails, its been a few years since I gave enough of a damn to bother emailing Sage but the gist of there response was:

"Can you perform that function (the one that caused the bug) in another fashion?" "If so, do it that way" "This will be patched shorty, and released in the next minor update" which cost about $3800. The actual amount varied, but that is an accurate median price.

Really? We are paying top dollar for a beta version of your software and you want us to pay you more money for a patched version?

blargoe

I worked at a company for which the entire datacenter was something of a jerry rig. About 125 "servers", some of them actual server class machines, some of them several year old HP workstation class computers, sitting on three tiers of shelves cobbled together by the maintenance/facilities people out of what I suppose was leftover cubicle desk materials. The room had proper battery backup and air handling, but the power to the server "racks" was provided by surge protectors that were daisy chained together and in most cases just lying on the floor in the cluster of spaghetti of power and network cables. You couldn't access the back of an server without accidentally disconnecting one of more other servers most of the time.

Same company, the manager ordered a single Vonage phone number as though we were a small business (we were a company with over 3000 employees at the time), and rigged something up on our PBX system to reroute calls to our plants that were not within our local calling zone to use the Vonage VOIP connection, to save a very little bit of money on our telecom bill each month

This particular company completely outsourced its datacenter a few months after I left.

ecoblue

Okay, I have another doozy. Today, I went into a shared wiring closet at a client's location and saw a rack-mounted switch that was "cradled" into position using a "hammock" constructed of twisted pair wiring!

boobobobobob

This isn't that bad but it's the only picture i have. Mounted with double sided tape and upside down. The switch in the rack had 20+ available ports

CodeBlox

Hahaaa! Some of these are really great! I'm really getting a kick out of the consumer grade switch taped there!.

--chris--

I can add one to the list, I just set it up today per strict instructions from my boss.

We use Avaya for phones here. We have run out of Win 7 Avaya licenses, but have plenty of Win XP licenses.

A user's PC is long overdue for an upgrade and it needs to just be tossed/recycled. I have been told not to spend a dime, we have a pc on a different floor not in use and I am to set him up with everything he needs on that PC except Avaya. He will simply remote in and do his work on that PC, while keeping his current Win XP online for the sole purpose of using Avaya.

Its not super rigged, but I thought it was a good work around to a problem lol.

pitviper

The best I’ve see was an “electrician” who never did low voltage wiring but was part of a build-out that was bundled in with the GC. He literally attached each cat6 cable to a cordless drill and used it to untwist each pair roughly 3-4 feet in length, BOTH ends. (and proceeded to terminate it that way).

Also had a legacy telco guy punch a hole in the wall, with a screwdriver, about 3 feet up, and run an extension cord into a closet for power.

Seen more than a few pieces of equipment “mounted” by tying patch cables around them too.

instant000

SteveO86 wrote: »

Put a GRE into a bridge group for some L2 services to work across a L3 WAN. (Not sure I would call the jury-rigging though). Temp solution until a proper L2 circuit could get installed.

Reading this made me think of this:

OTV Decoded - A Fancy GRE Tunnel

ecoblue

I am amazed by the number of users that really don't know how to use Outlook, often because they are unwilling to learn, or think they have found a "clever" way to use it. These users jury-rig some frightening solutions...

<sarcasm> Did you know that you can use the "Deleted Items" folder in Outlook to save important mail? It's so easy to file messages, all you have to do is press "Delete" or click the red "X". Why would you want to waste time learning how to make folders to save your mail?
</sarcasm?

This practice usually becomes known when the user empties their Deleted Items AND THEN realizes everything they meant to save in that folder is gone too. If they are using a PST file and haven't made a backup, then it somehow becomes Microsoft's fault when they can't get their mail back...

EDIT:
For my part, I do whatever I can to console the user and try to teach them how to work with Outlook in a better way.

Frostbite

I needed some cabling installed to be able to put an IP phone where there wasn't one before. The president of my company insisted on hiring his friend's electrical company to run the cable. In two locations, rather than installing wall jacks, they just ran the cable straight out of the wall with an RJ45 connector on the end and a wall plate with a hole in the center.

I've complained a few times to him to try to get them back out to fix it and so far, no luck. Could I fix it myself? Sure, but we paid them to do it and I refuse to fix their work.

PurpleIT

Frostbite wrote: »

I needed some cabling installed to be able to put an IP phone where there wasn't one before. The president of my company insisted on hiring his friend's electrical company to run the cable. In two locations, rather than installing wall jacks, they just ran the cable straight out of the wall with an RJ45 connector on the end and a wall plate with a hole in the center.

I've complained a few times to him to try to get them back out to fix it and so far, no luck. Could I fix it myself? Sure, but we paid them to do it and I refuse to fix their work.

That is a very common practice. Not a good one, but very common.

I have a bunch of those connected to a SOHO switch that is balanced on a pillar above the drop-ceiling. The rationale for doing it this way was it was SO much easier than making runs to each desk from the server room and having to deal with patch panels, etc.

I believe I am the only person who still knows exactly where this switch is...

(for the record, I am not the one who did it, I don't condone it, but they also won't let me fix it)