Important HyperV Safety Tip

hypnotoadhypnotoad Banned Posts: 915
How to ruin your saturday night:

1. Bring up a new Hyper-V server using your Windows 2008 RTM disc. For some reason the Hyper-V update fails, so you are still running Beta Hyper-V (which was on RTM). You are unaware of this, because who really looks at all those updates anyway when bringing up a new server?

2. Move your existing VHD's to your new server.

3. Create new VM's using your existing VHD's and let them start up. Let them boot in the background while you get all your VM's going.

4. Say "WTF" as Hyper-V proceedes to corrupt all of the guest VHD's and totally ruin half your servers!

5. Spend all night restoring VHD's from backup.

The lesson: never start a guest created in "Full" using the "Beta" host. It will corrupt the guest's VHD!

Thanks, Microsoft.

Comments

  • JDMurrayJDMurray Admin Posts: 13,023 Admin
    It sounds like you've just written VMWare's next print ad! icon_lol.gif
  • darkerosxxdarkerosxx Banned Posts: 1,343
    hypnotoad wrote: »
    How to ruin your saturday night:

    1. Bring up a new Hyper-V server using your Windows 2008 RTM disc. For some reason the Hyper-V update fails, so you are still running Beta Hyper-V (which was on RTM). You are unaware of this, because who really looks at all those updates anyway when bringing up a new server?

    2. Move your existing VHD's to your new server.

    3. Create new VM's using your existing VHD's and let them start up. Let them boot in the background while you get all your VM's going.

    4. Say "WTF" as Hyper-V proceedes to corrupt all of the guest VHD's and totally ruin half your servers!

    5. Spend all night restoring VHD's from backup.

    The lesson: never start a guest created in "Full" using the "Beta" host. It will corrupt the guest's VHD!

    Thanks, Microsoft.

    Seconded. Happened to me when I first started with Hyper-V.
  • HeroPsychoHeroPsycho Inactive Imported Users Posts: 1,940
    JDMurray wrote: »
    It sounds like you've just written VMWare's next print ad! icon_lol.gif

    I was gonna say that I thought the best safety tip for HyperV is don't use it. icon_lol.gif
    Good luck to all!
  • JordusJordus Banned Posts: 336
    So because you declined to properly patch a new machine, you are going to try and save face by blaming Microsoft?

    icon_rolleyes.gif


    I think this simply should serve as a reminder to stick to best practices and do things by the book. (They are called "BEST" for a reason)
  • JDMurrayJDMurray Admin Posts: 13,023 Admin
    Jordus wrote: »
    So because you declined to properly patch a new machine, you are going to try and save face by blaming Microsoft?
    Most betas don't allow patches to be applied. The point was that it wasn't realized that the software was beta and, uh, Microsoft released the beta, so it's, ah huh, Microsoft's fault. Ummm, yeah, that's it. icon_rolleyes.gif

    Well, if that'd happened to me I'd be writing a blog article about it right now. ;)
  • dynamikdynamik Banned Posts: 12,312 ■■■■■■■■■□
    It's odd that it doesn't detect the VMs are newer. Just today, VMware Workstation refused to open a 6.52 VM on a 6.50 installation (I wasn't thinking it was such a nice feature at the time because I had a 400mb+ download on a slow connection icon_lol.gif).
  • HeroPsychoHeroPsycho Inactive Imported Users Posts: 1,940
    Jordus wrote: »
    So because you declined to properly patch a new machine, you are going to try and save face by blaming Microsoft?

    icon_rolleyes.gif


    I think this simply should serve as a reminder to stick to best practices and do things by the book. (They are called "BEST" for a reason)

    Honestly, and I like Microsoft, and defend them against a lot of criticism, but I do think including beta software on RTM media is asking for stuff like this to happen.
    Good luck to all!
  • msteinhilbermsteinhilber Member Posts: 1,480 ■■■■■■■■□□
    Haven't run into this myself yet, but I've began to work heavily with Hyper-V in our shop so I'll make sure I avoid the issue. Realistically though, if I update a server I always verify the updates applied correctly - you do after all get an error notification regardless of if you updates through a browser or through the command line on a core and/or Hyper-V bare metal install. Not sure how WSUS behaves upon a failed update (I imagine it just writes to a log rather than display an error) if you use that, we don't use it in our shop so I'm not sure.
  • SWMSWM Member Posts: 287
    And for another Hyper V issue, check this link...

    Recently had a Intel S5000 Dual Quad core server running Hyper V with 3 virtual server guests. Installed Intel Active System Console to monitor the server hardware. All was fine until we rebooted.

    None of our Virtual machines would start and all became inaccessible...

    Intel® System Management Software - TA-922 : Intel® IPMI Driver causes Virtual Machines to be inaccessible

    basically the Intel software screwed up the registry permissions and prevented Hyper V manager accessing ANY virtual machines or allowing them to start. icon_cry.gif

    Intel have now released a patch and an update, but it caused me many hours of downtime....

    But besides that, am happy with Hyper V, but would love to be able to use more that 4 cores with each guest OS, hopefully Server 2008 R2 may address that issue..
    Isn't Bill such a Great Guy!!!!
  • astorrsastorrs Member Posts: 3,139 ■■■■■■□□□□
    HeroPsycho wrote: »
    Honestly, and I like Microsoft, and defend them against a lot of criticism, but I do think including beta software on RTM media is asking for stuff like this to happen.
    +1

    Including the unreleased beta version of Hyper-V as an installable role with 2008 RTM was stupid. I understand why they wanted to do it, but it should have just sent you to a website to download it - which then could have been updated when Hyper-V RTM'd.

    Either that or you could just follow the best Hyper-V safety tip offered above... use VMware ESX. :p
  • bertiebbertieb Member Posts: 1,031 ■■■■■■□□□□
    astorrs wrote:
    +1

    Including the unreleased beta version of Hyper-V as an installable role with 2008 RTM was stupid. I understand why they wanted to do it, but it should have just sent you to a website to download it - which then could have been updated when Hyper-V RTM'd.

    Either that or you could just follow the best Hyper-V safety tip offered above... use VMware ESX. :p

    +1 to all this :D (Sorry, had to jump on the bandwagon here)

    Still, this probably makes the guys at VMWare chuckle a bit.
    The trouble with quotes on the internet is that you can never tell if they are genuine - Abraham Lincoln
  • hypnotoadhypnotoad Banned Posts: 915
    I didn't follow BP's because I was in a hurry and I guess Microsoft didn't follow BP's (in my opinion) when they released that Hyper-V Beta on the RTM, and then didn't write a feature to stop people from attempting to run the two versions on the same set of VHD's (especially since it corrupts the VHD).
  • JDMurrayJDMurray Admin Posts: 13,023 Admin
    bertieb wrote: »
    Still, this probably makes the guys at VMWare chuckle a bit.
    This thread is convincing me that my decision to settle on VMWare's ESXi was a very good one.
  • HeroPsychoHeroPsycho Inactive Imported Users Posts: 1,940
    bertieb wrote: »
    +1 to all this :D (Sorry, had to jump on the bandwagon here)

    Still, this probably makes the guys at VMWare chuckle a bit.

    Kinda like how Microsoft must have chuckled at the ESX 3.5 Update 3 licensing bug fiasco. icon_wink.gif

    Don't get me wrong, I like VMware virtualization products more than MS, but VMware ain't exactly perfect, either...
    Good luck to all!
  • bertiebbertieb Member Posts: 1,031 ■■■■■■□□□□
    HeroPsycho wrote:
    Kinda like how Microsoft must have chuckled at the ESX 3.5 Update 3 licensing bug fiasco. icon_wink.gif

    Don't get me wrong, I like VMware virtualization products more than MS, but VMware ain't exactly perfect, either...

    Yup I know. I just think whomever decided to let a beta version of Hyper-V ship on RTM media was really, really stupid but I suspect it was some marketing guy who had the final say rather than the technical teams. MS have some of their sizeable guns firmly pointed towards the VM market and this kinda thing doesn't do them any favours at all.

    Still, this all makes me chuckle if nothing else (though not at your plight hypnotoad - I feel your pain) :)
    The trouble with quotes on the internet is that you can never tell if they are genuine - Abraham Lincoln
  • darkerosxxdarkerosxx Banned Posts: 1,343
    The problem with all this isn't that you should update before using the application. Updates shouldn't "break" usability. The problem is when you install and begin using Hyper-V, then apply updates, it breaks usability unless you start over completely.

    This is no one's fault but Microsoft's and their update should have allowed VM's created in beta to be used after the update. At they very least, it should say "UPGRADING FROM HYPER-V INCLUDED IN THE INSTALL MEDIA WILL BREAK YOUR VIRTUAL MACHINES!" and require approval before continuing.

    Classic Microsoft, though, IMO, you shouldn't expect less. :)
  • HeroPsychoHeroPsycho Inactive Imported Users Posts: 1,940
    darkerosxx wrote: »
    Classic Microsoft, though, IMO, you shouldn't expect less. :)

    I disagree. Microsoft is usually better about this kind of thing. You can site mistakes like this by any major IT company. I wouldn't say it's "classic Microsoft".

    I remember in my newb days upgrading an ESX server from 2.5 to 3, and it blowing up the VMFS volume on the local storage, and all the VM's were lost as an example.

    And in Microsoft's defense in this case, it was BETA software that has the problem. Like I said, my issue is it's easy to assume that all software included on install media for W2K8 should be production grade code.
    Good luck to all!
  • astorrsastorrs Member Posts: 3,139 ■■■■■■□□□□
    HeroPsycho wrote: »
    I disagree. Microsoft is usually better about this kind of thing. You can site mistakes like this by any major IT company. I wouldn't say it's "classic Microsoft".

    I remember in my newb days upgrading an ESX server from 2.5 to 3, and it blowing up the VMFS volume on the local storage, and all the VM's were lost as an example.

    And in Microsoft's defense in this case, it was BETA software that has the problem. Like I said, my issue is it's easy to assume that all software included on install media for W2K8 should be production grade code.
    Yeah the VMFS-2 to VMFS-3 upgrade could be fun... ;)
  • dynamikdynamik Banned Posts: 12,312 ■■■■■■■■■□
    darkerosxx wrote: »
    This is no one's fault but Microsoft's and their update should have allowed VM's created in beta to be used after the update. At they very least, it should say "UPGRADING FROM HYPER-V INCLUDED IN THE INSTALL MEDIA WILL BREAK YOUR VIRTUAL MACHINES!" and require approval before continuing.

    Yea, you got it backwards. The beta software corrupted the newer VMs. There wasn't an update that broke them.
  • darkerosxxdarkerosxx Banned Posts: 1,343
    I installed the server and accepted usage of the beta Hyper-V, made my VM's, then updated several days later and this happened.
  • JordusJordus Banned Posts: 336
    darkerosxx wrote: »
    I installed the server and accepted usage of the beta Hyper-V, made my VM's, then updated several days later and this happened.

    I did this once, in a non production environment (first time i was messing with hyper-V) and didnt have an issue. icon_lol.gif
Sign In or Register to comment.