please save me from my IT manager

hugolucky

my manager has lost his mind and he wont listen to how futile the project he gave is, and i dont yet have enough experience with maintaining servers to lead him to the waters edge

we have 8 servers, all Windows Server 2003 Standard edition, he wants all files that are older than 2 years removed and archived, theres a half million files spread across 10s of thousands of directories, and he wants me to maintain the directory stucture in the archive, and too top it all off he wants it all burned to CDs

according to him they did not install something when the servers were originally setup, something that would have done this all for us, and he couldnt, for some odd reason, provide any details on what that something was, and hes resisting my advise to put in a purchase order for software to do this for us, there has to be something out there that will do this without breaking the agencys meager budget

please guyz show me where the watering hole is here, and ill do my best to lead his ass to it, i dont wanna spend the rest of my career within the madness of manual recursion, help

cisco_trooper

hugolucky wrote:

my manager has lost his mind and he wont listen to how futile the project he gave is, and i dont yet have enough experience with maintaining servers to lead him to the waters edge

we have 8 servers, all Windows Server 2003 Standard edition, he wants all files that are older than 2 years removed and archived, theres a half million files spread across 10s of thousands of directories, and he wants me to maintain the directory stucture in the archive, and too top it all off he wants it all burned to CDs

according to him they did not install something when the servers were originally setup, something that would have done this all for us, and he couldnt, for some odd reason, provide any details on what that something was, and hes resisting my advise to put in a purchase order for software to do this for us, there has to be something out there that will do this without breaking the agencys meager budget

please guyz show me where the watering hole is here, and ill do my best to lead his ass to it, i dont wanna spend the rest of my career within the madness of manual recursion, help

Sounds like a good time for you to learn some scripting brother. If a file has a property, you can access it programmatically in some way. I wrote a similar program in C# for archiving files that had not been touched in X number of days. Good luck brother. It is not as difficult as it sounds. Just tell your IT Manager he is going to have to provide you the latitude to learn some scripting/programming so that this project can be done correctly the first time. It will be worth his time and yours. As far as burning the stuff to CDs, the guy is an idiot. Invest in a backup system of some kind.

woodworm

You didn't say why he wants you to do this?

I'm guessing he's not that IT literate so you need to give him some workable options to solve his problem. He may just think you don't want to do it just because it's a cr*ppy job.

dynamik

So what happens if a file that's less than two years old is in the same directory as a file that's greater than two years old, and they're both related and/or dependent on one another?

liven

ACTIVE PERL TO THE RESUCE!!!!!


Perl is the system administrators BEST FRIEND!!!!


Yes it is more of a unix scripting language, however I have used many times on windows and it works very very well.


Plus you can google for code snipits that are used on linux/unix/bsd and convert them to windows. Typically the same code will work on across platforms.


It will kinda hard at first, but really not that bad. If you need any help ping me. I am not a perl guru, but I write lots of it.

Oh ya, it is free, there is a massive community of Perl developers out there as well.

liven

here is another option:

http://www.filetransit.com/view.php?id=19586


looks like it cost 30 bucks...

sprkymrk

You could use the free MS tool ROBOCOPY.

It allows you to copy files older/newer than a specified date/time:

/MAXAGE:n :: MAXimum file AGE - exclude files older than n days/date.
/MINAGE:n :: MINimum file AGE - exclude files newer than n days/date.
/MAXLAD:n :: MAXimum Last Access Date - exclude files unused since n.
/MINLAD:n :: MINimum Last Access Date - exclude files used since n.
(If n < 1900 then n = n days, else n = YYYYMMDD date).

You'll have to copy it to a temporary location first, since it probably won't fit on a CD and maintain the directory tree structure. You can map it to a remote computer and do what you want with it from there.

You would run it on each server something like this:

Assuming the main directory with the data on the source server is C:\Data, and the source server is named Server01, and you will archive the files to a remote computer named ArchiveServer:

robocopy C:\Data\*.* \\ArchiveServer\c$\Server01\Data\ /E /COPYALL /MINAGE:730

Be sure and test and double check first, but this should copy any files older than 730 days (2 years) and it will copy all directories, including empty ones (/E) and will also copy all file attributes and security (NTFS). If you are feeling really brave, and have thoughorly tested it, you can add the /MOV switch, which also delete the files from the source once you have copied them (use with caution!).

Anyway, I use robocopy a lot for situations like this. It's worth downloading and looking at.

hugolucky

cisco_trooper wrote:

Sounds like a good time for you to learn some scripting brother. If a file has a property, you can access it programmatically in some way. I wrote a similar program in C# for archiving files that had not been touched in X number of days. Good luck brother. It is not as difficult as it sounds. Just tell your IT Manager he is going to have to provide you the latitude to learn some scripting/programming so that this project can be done correctly the first time. It will be worth his time and yours. As far as burning the stuff to CDs, the guy is an idiot. Invest in a backup system of some kind.

yes... i could turn lemons into lemonade here if i can learn some scripting and get this done right the first time, impressing the boss without sucking his butt and learning a new skill at the same time, sounds like great advise, thanks!

as far the CDs hes getting ready to order a 100 pack for me to start with, it blows my mind that he wants it done this way, hes been talking iSCSI lately but also mentions its about 3 years away from being implemented at the agency, and from what ive seen so far theres alot of talk on improvements but nothing comes of it, heck i gotta have a flashlight at work to read part numbers on the toner/ink cartridges because theres no light on that side of the 'dungeon' i work in and he just wont listen when i tell him i have a hard time seeing and i need another light on that side of the room

ive decided that im gonna be on the 2 year plan there anyway, 2 more years experience for the resume and im gonna be gone, but after what ive seen in just 6 months im not sure if i can deal with another 18 months of living with their methodology and budget constraints

woodworm wrote:

You didn't say why he wants you to do this?

I'm guessing he's not that IT literate so you need to give him some workable options to solve his problem. He may just think you don't want to do it just because it's a cr*ppy job.

sorry, i thought it would go without saying why this needs to be done, were running out room on the servers and its also taking forever to do the nightly backups

it is a crappy job, but ive done crappy jobs before, the reason i dont wanna do it is because i know that what hes asking is not efficient, i might be a newbie and not know the right way, thats why im here asking you guyz, but i do know that what hes proposing cant be the right way

thanks for the reply

hugolucky

dynamik wrote:

So what happens if a file that's less than two years old is in the same directory as a file that's greater than two years old, and they're both related and/or dependent on one another?

hmmm, good question, ill have to bring that up monday, thanks

....

liven, thanks for the suggestions, i well check that out

....

sprkymrk wrote:

You could use the free MS tool ROBOCOPY.

It allows you to copy files older/newer than a specified date/time:

/MAXAGE:n :: MAXimum file AGE - exclude files older than n days/date.
/MINAGE:n :: MINimum file AGE - exclude files newer than n days/date.
/MAXLAD:n :: MAXimum Last Access Date - exclude files unused since n.
/MINLAD:n :: MINimum Last Access Date - exclude files used since n.
(If n < 1900 then n = n days, else n = YYYYMMDD date).

You'll have to copy it to a temporary location first, since it probably won't fit on a CD and maintain the directory tree structure. You can map it to a remote computer and do what you want with it from there.

You would run it on each server something like this:

Assuming the main directory with the data on the source server is C:\Data, and the source server is named Server01, and you will archive the files to a remote computer named ArchiveServer:

Code:
robocopy C:\Data\*.* \\ArchiveServer\c$\Server01\Data\ /E /COPYALL /MINAGE:730


Be sure and test and double check first, but this should copy any files older than 730 days (2 years) and it will copy all directories, including empty ones (/E) and will also copy all file attributes and security (NTFS). If you are feeling really brave, and have thoughorly tested it, you can add the /MOV switch, which also delete the files from the source once you have copied them (use with caution!).

Anyway, I use robocopy a lot for situations like this. It's worth downloading and looking at.

thank-you, i will check that out this afternoon, i just got vmware workstation installed and have a server 2k3 Vm i can test with this weekend, thank you very much

....

thanks everyone, and have a great weekend!

snadam

you might want to suggest to him that there are better solutions than storing to a CD; Such as DVD or even external hard drive (my personal choice). You can get a good 1TB drive for about $250. And if you utilize the eSATA or even USB 2.0, you should still be writing faster than to a CD.

undomiel

I'm just trying to imagine the organization and labeling scheme you would need for a massive amount of cds like that. External HD is definitely the way to go.

dynamik

undomiel wrote:

I'm just trying to imagine the organization and labeling scheme you would need for a massive amount of cds like that. External HD is definitely the way to go.

Depends on your needs. I regularly archive data to DVDs. I burn two copies, one for the safe, and one I send home with a manager. I just use numbers for labeling (i.e. DVD 32, DVD 33, etc.), and I use Advanced Disk Catalog to index all the discs. I can find pretty much any file within a matter of seconds. I also like that the data is distributed. If a disc is destroyed or misplaced, it's 1) a relatively small amount of data lost and 2) cheaply and easily duplicated from the manager's copy.

undomiel

From the description I was picturing something more like my workplace's environment in which we have about half a TB of data, which would be very impractical for back-up to CD, much less on any sort of routine basis.

Smallguy

sprkymrk wrote:

You could use the free MS tool ROBOCOPY.

It allows you to copy files older/newer than a specified date/time:

/MAXAGE:n :: MAXimum file AGE - exclude files older than n days/date.
/MINAGE:n :: MINimum file AGE - exclude files newer than n days/date.
/MAXLAD:n :: MAXimum Last Access Date - exclude files unused since n.
/MINLAD:n :: MINimum Last Access Date - exclude files used since n.
(If n < 1900 then n = n days, else n = YYYYMMDD date).

You'll have to copy it to a temporary location first, since it probably won't fit on a CD and maintain the directory tree structure. You can map it to a remote computer and do what you want with it from there.

You would run it on each server something like this:

Assuming the main directory with the data on the source server is C:\Data, and the source server is named Server01, and you will archive the files to a remote computer named ArchiveServer:

robocopy C:\Data\*.* \\ArchiveServer\c$\Server01\Data\ /E /COPYALL /MINAGE:730

Be sure and test and double check first, but this should copy any files older than 730 days (2 years) and it will copy all directories, including empty ones (/E) and will also copy all file attributes and security (NTFS). If you are feeling really brave, and have thoughorly tested it, you can add the /MOV switch, which also delete the files from the source once you have copied them (use with caution!).

Anyway, I use robocopy a lot for situations like this. It's worth downloading and looking at.

there is also a GIU available for robocopy for those woh prefer a GUI I downloaded it the other day from

http://technet.microsoft.com/en-us/magazine/cc160891.aspx

justin42279

I've used ROBOCOPY plenty of times. I find it very handy. I could understand why your IT Manager would want to do this. Space capacity is always an issue, but I would feel wary of moving files based on age rather than access date. At my organization, we have plenty of files created over 2 years ago and still used. I'd rather use access date to move files not accessed in a year. If for some chance, there are files that are accessed only once a year, its easy to just return them to the same location.

Pash

snadam wrote:

you might want to suggest to him that there are better solutions than storing to a CD; Such as DVD or even external hard drive (my personal choice). You can get a good 1TB drive for about $250. And if you utilize the eSATA or even USB 2.0, you should still be writing faster than to a CD.

Exactly. An eSATA drive will pay for itself after a while. If he eventually want's to send it offsite you can then get a tape drive and backup your archive drive. Or if thats not in budget get a DVD burner for cheap and do it that way. All the customers i do work for, not one uses CD as a backup media, even for temp storage of any data.

hugolucky

thanks everyone

snadam wrote:

you might want to suggest to him that there are better solutions than storing to a CD; Such as DVD or even external hard drive (my personal choice). You can get a good 1TB drive for about $250. And if you utilize the eSATA or even USB 2.0, you should still be writing faster than to a CD.

and...

undomiel wrote:

I'm just trying to imagine the organization and labeling scheme you would need for a massive amount of cds like that. External HD is definitely the way to go.

thanks, i realize this, but my manager is gonna resist anything that requires him to turn in a purchase request, i bought a 500GB maxtor onetouch external hd last week for use here at home and im almost tempted to donate it to the agency for this project, but...... no.... im not doing that

Smallguy wrote:

there is also a GIU available for robocopy for those woh prefer a GUI I downloaded it the other day from

yes, i downloaded it this morning after checking out sprkymrk's suggestion on robocop, im gonna be looking at the robocop tool this weekend, thanks

justin42279, dynamik, and Pash...... thanks for your input as well

blargoe

What backup software are you using? Many backup software vendors have built in the ability to archive to tape. I think in Backup Exec, for example, you can tell it to archive files older than a certain date and it will move the files to tape and delete them from the file system. Something like this would be much more efficient for you than trying to burn all that crap to CD.

hugolucky

blargoe wrote:

What backup software are you using? Many backup software vendors have built in the ability to archive to tape. I think in Backup Exec, for example, you can tell it to archive files older than a certain date and it will move the files to tape and delete them from the file system. Something like this would be much more efficient for you than trying to burn all that crap to CD.

i do not know, he would know that, ive only been there about 6 months and it took 4 of them before he would even let me switch the backup tapes at remote storage, he keeps alot of stuff to himself and if i asked it would probably only bring on another 'dont worry about it' comment from him

the other day he asked me to grab a laptop outta storage that had wireless cability, so I grabbed an HP with Centrino, he comes over and asks me if it has wireless, so i point to the sticker that says Centrino, he then starts to give me a lecture on how Centrino is about power conservation and has nothing to do with wireless, i told him Centrino technology does have power saving features AND is a wireless technology as well, he just put up his hand and walked away, so i configuered the damned thing and deployed it without speaking to him anymore on it

about a month ago he sends me out to a remote office to configure some wireless HP terminals to work with the AP he setup before i started, this is what he gives me....

password: 12345678909876543212345678
encryption: WEP

i told him, in so many words, that his password was lame and he should NOT have choosen WEP as the AP and clients all support WPA, he wouldnt listen, for crying out loud you have HIPPA info being submitted to the network.... are you mad?..... is what i wanted to say to him

im just gonna say it, my boss is somewhat of a dumbass, and it is very frustrating being dismissed constantly by him, but this project hes handed me is BS, and sometimes in life you know whats the wrong way, you may not know whats right, but you know whats wrong, and after reading the suggestions here im now even more sure of what i need to do, im gonna refuse if he insists, and if it comes to it at my next interview i will simply tell the truth about why i left, something tells not to worry about walking away from this if i have to

thanks for listening and letting me vent a bit folks

Ahriakin

BestPractice and actually required for HIPPAA....If you are backing up data (whether on disks or tape or whatever) ENCRYPT IT! With HIPAA classed data it's a legal binding, your boss doesn't have a say in this.

justin42279

Don't screw with the Hippa. It means business.

keatron

justin42279 wrote:

Don't screw with the Hippa. It means business.

Funny.

brad-

hugolucky wrote:

and too top it all off he wants it all burned to CDs

I'd say punch cards or transcribing the files onto paper would be a better solution. Then maybe you could set up a card catalog like at a library. DEWEY DECIMAL FTW!

snadam

If your boss is that ignorant/arrogant, and you NEED to comply with HIPPA, then you might want to escalate this to HIS superior. Because if something happens with HIPPA, and his boss asks whats the deal; I guarantee that he will make YOU his scapegoat. Ever hear the term '$hit rolls downhill'? Honestly, I would go to the big boss on Monday and let hi know whats up. If he as is bad as you say he is. There is obviously no compromise with him.

pash wrote:

Exactly. An eSATA drive will pay for itself after a while. If he eventually want's to send it offsite you can then get a tape drive and backup your archive drive. Or if thats not in budget get a DVD burner for cheap and do it that way. All the customers i do work for, not one uses CD as a backup media, even for temp storage of any data.

yep, thats exactly how we have it at our job.

hugolucky

snadam wrote:

If your boss is that ignorant/arrogant, and you NEED to comply with HIPPA, then you might want to escalate this to HIS superior. Because if something happens with HIPPA, and his boss asks whats the deal; I guarantee that he will make YOU his scapegoat. Ever hear the term '$hit rolls downhill'? Honestly, I would go to the big boss on Monday and let hi know whats up. If he as is bad as you say he is. There is obviously no compromise with him.

ya maybe i should have the HIPPA sit on him, he really is that ignorant

for example, all the agencies HP TravelMates in the 2xx series have issues with ACPI, Event Viewer logs errors constantly on those machines, and its because they were designed for Windows 98/Me, and when he installed XP on them he didnt update BIOS first

one day i had one of those HPs at my desk, i was gonna update the BIOS and then reinstall XP, so i asked him where all the floppies were at, he rifled through his desk and handed me a used floppy that said....

'Introducing Windows 95'

i threw it in the garbage and brought in one of my own

blargoe

Find another job.

I wouldn't normally say this to someone who understands the concept of learning what one can early on and then stepping into a better role elsewhere, but I can't say putting up with this crap is worth it.

shednik

blargoe wrote:

Find another job.

I wouldn't normally say this to someone who understands the concept of learning what one can early on and then stepping into a better role elsewhere, but I can't say putting up with this crap is worth it.

+1 lol this sounds like something my previous boss would have asked me to do...hence why I left and haven't looked back. I do feel bad for the people that are still left to deal with that place, but trust me once you have some solid experience and certs/degree you can find a good job at a good company.

Sepiraph

hugolucky wrote:

blargoe wrote:

What backup software are you using? Many backup software vendors have built in the ability to archive to tape. I think in Backup Exec, for example, you can tell it to archive files older than a certain date and it will move the files to tape and delete them from the file system. Something like this would be much more efficient for you than trying to burn all that crap to CD.

i do not know, he would know that, ive only been there about 6 months and it took 4 of them before he would even let me switch the backup tapes at remote storage, he keeps alot of stuff to himself and if i asked it would probably only bring on another 'dont worry about it' comment from him

the other day he asked me to grab a laptop outta storage that had wireless cability, so I grabbed an HP with Centrino, he comes over and asks me if it has wireless, so i point to the sticker that says Centrino, he then starts to give me a lecture on how Centrino is about power conservation and has nothing to do with wireless, i told him Centrino technology does have power saving features AND is a wireless technology as well, he just put up his hand and walked away, so i configuered the damned thing and deployed it without speaking to him anymore on it

about a month ago he sends me out to a remote office to configure some wireless HP terminals to work with the AP he setup before i started, this is what he gives me....

password: 12345678909876543212345678
encryption: WEP

i told him, in so many words, that his password was lame and he should NOT have choosen WEP as the AP and clients all support WPA, he wouldnt listen, for crying out loud you have HIPPA info being submitted to the network.... are you mad?..... is what i wanted to say to him

im just gonna say it, my boss is somewhat of a dumbass, and it is very frustrating being dismissed constantly by him, but this project hes handed me is BS, and sometimes in life you know whats the wrong way, you may not know whats right, but you know whats wrong, and after reading the suggestions here im now even more sure of what i need to do, im gonna refuse if he insists, and if it comes to it at my next interview i will simply tell the truth about why i left, something tells not to worry about walking away from this if i have to

thanks for listening and letting me vent a bit folks

Does your boss happen to have pointy hair as well?

And wouldn't HIPPA compliance required something like AES or at least WPA (not a USisn), WEP is a joke that can be hackable in less than 5 minutes. Anyway regarding your original post, CD is an obsolete technology for file archiving. Even DVD doesn't cut it when the price-performance of external HD is so much better. Regarding file archive program, perhaps your boss was thinking of something like Norton Ghost, which was fairly popular in the network environment.

nel

Hi,

just reading through the post and it def looks like you've got the crap end of the stick!

If your boss is that ignorant/arrogant, and you NEED to comply with HIPPA, then you might want to escalate this to HIS superior.

Be carefull if your going above some ones head because it can make your working life very uncomfortable if you intend staying there! its just a suggestion but you have to remember that he could make your life nasty for doing so.

I would suggest doing it safely and bring it to your bosses attention in an informal chat. then if he doesnt budge on it, go above him.

By reading the other posts it looks like the suggestions cover the best ways to complete this task. another suggestion is also plan for the aftermath of it - i.e. suggest ways to manage it better so your showing some initiative to your peers with things like DFS. try and design a better security policy and backup plan aswell and i would make all the configs and steps in documenation because ive worked with many people who are "power hungry" and keep lots of information to themselves - these are some of the worst type of guys to work with because alot of the time you never know what the hell has happened and why until something goes wrong!

hugolucky

nel wrote:

Hi,

just reading through the post and it def looks like you've got the crap end of the stick!

If your boss is that ignorant/arrogant, and you NEED to comply with HIPPA, then you might want to escalate this to HIS superior.

Be carefull if your going above some ones head because it can make your working life very uncomfortable if you intend staying there! its just a suggestion but you have to remember that he could make your life nasty for doing so.

I would suggest doing it safely and bring it to your bosses attention in an informal chat. then if he doesnt budge on it, go above him.

By reading the other posts it looks like the suggestions cover the best ways to complete this task. another suggestion is also plan for the aftermath of it - i.e. suggest ways to manage it better so your showing some initiative to your peers with things like DFS. try and design a better security policy and backup plan aswell and i would make all the configs and steps in documenation because ive worked with many people who are "power hungry" and keep lots of information to themselves - these are some of the worst type of guys to work with because alot of the time you never know what the hell has happened and why until something goes wrong!

actually, i wouldnt take this to my managers superiour, doing things that way has only gotten me labeled a troublemaker in other places, but i do realize that if the s*it hits the fans it could easily fly my way, and not his

a few weeks back i posted here about my concerns with WEP, and i got some good info from you folks, so i emailed my concerns along with that info to my manager, his reply was basically to not worry about it, his 'network guy' (who i have yet to even meet, hes some kinda mysterious vapor-expert) said that it was ok

so i configured the damned things the way he wanted...

... but i also saved the handwritten instructions that he gave me, and i also archived my emails to a .pst file that i have here

so anyway.... it didnt work, the clients are only grabbing APIPA addys, im not sure whats going on with DHCP out there and neither does he because he cant find his documentation on how the APs were configured, hes saying that he may have used MAC filtering but hes not sure, he didnt even know what brand the APs were, he told me Cisco but when i got there they were Belkins

so i tell him that i need to go back out there and set the APs back to default, and then login and configure them for WPA, and then configure the clients for WPA as well, its not rocket science for crying out loud!

he said no, he needs to talk to his mysterious 'network guy'



man am i glad i started this thread, i gotz months of frustration backed up and this is very cathertic, thanks all for the advise and for listening, time for coffee and feeding the guinea pigs!