Important HyperV Safety Tip

hypnotoad

How to ruin your saturday night:

1. Bring up a new Hyper-V server using your Windows 2008 RTM disc. For some reason the Hyper-V update fails, so you are still running Beta Hyper-V (which was on RTM). You are unaware of this, because who really looks at all those updates anyway when bringing up a new server?

2. Move your existing VHD's to your new server.

3. Create new VM's using your existing VHD's and let them start up. Let them boot in the background while you get all your VM's going.

4. Say "WTF" as Hyper-V proceedes to corrupt all of the guest VHD's and totally ruin half your servers!

5. Spend all night restoring VHD's from backup.

The lesson: never start a guest created in "Full" using the "Beta" host. It will corrupt the guest's VHD!

Thanks, Microsoft.

JDMurray

It sounds like you've just written VMWare's next print ad!

darkerosxx

hypnotoad wrote: »

How to ruin your saturday night:

1. Bring up a new Hyper-V server using your Windows 2008 RTM disc. For some reason the Hyper-V update fails, so you are still running Beta Hyper-V (which was on RTM). You are unaware of this, because who really looks at all those updates anyway when bringing up a new server?

2. Move your existing VHD's to your new server.

3. Create new VM's using your existing VHD's and let them start up. Let them boot in the background while you get all your VM's going.

4. Say "WTF" as Hyper-V proceedes to corrupt all of the guest VHD's and totally ruin half your servers!

5. Spend all night restoring VHD's from backup.

The lesson: never start a guest created in "Full" using the "Beta" host. It will corrupt the guest's VHD!

Thanks, Microsoft.

Seconded. Happened to me when I first started with Hyper-V.

HeroPsycho

JDMurray wrote: »

It sounds like you've just written VMWare's next print ad!

I was gonna say that I thought the best safety tip for HyperV is don't use it.

Jordus

So because you declined to properly patch a new machine, you are going to try and save face by blaming Microsoft?




I think this simply should serve as a reminder to stick to best practices and do things by the book. (They are called "BEST" for a reason)

JDMurray

Jordus wrote: »

So because you declined to properly patch a new machine, you are going to try and save face by blaming Microsoft?

Most betas don't allow patches to be applied. The point was that it wasn't realized that the software was beta and, uh, Microsoft released the beta, so it's, ah huh, Microsoft's fault. Ummm, yeah, that's it.

Well, if that'd happened to me I'd be writing a blog article about it right now.

dynamik

It's odd that it doesn't detect the VMs are newer. Just today, VMware Workstation refused to open a 6.52 VM on a 6.50 installation (I wasn't thinking it was such a nice feature at the time because I had a 400mb+ download on a slow connection ).

HeroPsycho

Jordus wrote: »

So because you declined to properly patch a new machine, you are going to try and save face by blaming Microsoft?




I think this simply should serve as a reminder to stick to best practices and do things by the book. (They are called "BEST" for a reason)

Honestly, and I like Microsoft, and defend them against a lot of criticism, but I do think including beta software on RTM media is asking for stuff like this to happen.

msteinhilber

Haven't run into this myself yet, but I've began to work heavily with Hyper-V in our shop so I'll make sure I avoid the issue. Realistically though, if I update a server I always verify the updates applied correctly - you do after all get an error notification regardless of if you updates through a browser or through the command line on a core and/or Hyper-V bare metal install. Not sure how WSUS behaves upon a failed update (I imagine it just writes to a log rather than display an error) if you use that, we don't use it in our shop so I'm not sure.

SWM

And for another Hyper V issue, check this link...

Recently had a Intel S5000 Dual Quad core server running Hyper V with 3 virtual server guests. Installed Intel Active System Console to monitor the server hardware. All was fine until we rebooted.

None of our Virtual machines would start and all became inaccessible...

Intel® System Management Software - TA-922 : Intel® IPMI Driver causes Virtual Machines to be inaccessible

basically the Intel software screwed up the registry permissions and prevented Hyper V manager accessing ANY virtual machines or allowing them to start.

Intel have now released a patch and an update, but it caused me many hours of downtime....

But besides that, am happy with Hyper V, but would love to be able to use more that 4 cores with each guest OS, hopefully Server 2008 R2 may address that issue..

astorrs

HeroPsycho wrote: »

Honestly, and I like Microsoft, and defend them against a lot of criticism, but I do think including beta software on RTM media is asking for stuff like this to happen.

+1

Including the unreleased beta version of Hyper-V as an installable role with 2008 RTM was stupid. I understand why they wanted to do it, but it should have just sent you to a website to download it - which then could have been updated when Hyper-V RTM'd.

Either that or you could just follow the best Hyper-V safety tip offered above... use VMware ESX.

bertieb

astorrs wrote:

+1

Including the unreleased beta version of Hyper-V as an installable role with 2008 RTM was stupid. I understand why they wanted to do it, but it should have just sent you to a website to download it - which then could have been updated when Hyper-V RTM'd.

Either that or you could just follow the best Hyper-V safety tip offered above... use VMware ESX.

+1 to all this (Sorry, had to jump on the bandwagon here)

Still, this probably makes the guys at VMWare chuckle a bit.

hypnotoad

I didn't follow BP's because I was in a hurry and I guess Microsoft didn't follow BP's (in my opinion) when they released that Hyper-V Beta on the RTM, and then didn't write a feature to stop people from attempting to run the two versions on the same set of VHD's (especially since it corrupts the VHD).

JDMurray

bertieb wrote: »

Still, this probably makes the guys at VMWare chuckle a bit.

This thread is convincing me that my decision to settle on VMWare's ESXi was a very good one.

HeroPsycho

bertieb wrote: »

+1 to all this (Sorry, had to jump on the bandwagon here)

Still, this probably makes the guys at VMWare chuckle a bit.

Kinda like how Microsoft must have chuckled at the ESX 3.5 Update 3 licensing bug fiasco.

Don't get me wrong, I like VMware virtualization products more than MS, but VMware ain't exactly perfect, either...

bertieb

HeroPsycho wrote:

Kinda like how Microsoft must have chuckled at the ESX 3.5 Update 3 licensing bug fiasco.

Don't get me wrong, I like VMware virtualization products more than MS, but VMware ain't exactly perfect, either...

Yup I know. I just think whomever decided to let a beta version of Hyper-V ship on RTM media was really, really stupid but I suspect it was some marketing guy who had the final say rather than the technical teams. MS have some of their sizeable guns firmly pointed towards the VM market and this kinda thing doesn't do them any favours at all.

Still, this all makes me chuckle if nothing else (though not at your plight hypnotoad - I feel your pain)

darkerosxx

The problem with all this isn't that you should update before using the application. Updates shouldn't "break" usability. The problem is when you install and begin using Hyper-V, then apply updates, it breaks usability unless you start over completely.

This is no one's fault but Microsoft's and their update should have allowed VM's created in beta to be used after the update. At they very least, it should say "UPGRADING FROM HYPER-V INCLUDED IN THE INSTALL MEDIA WILL BREAK YOUR VIRTUAL MACHINES!" and require approval before continuing.

Classic Microsoft, though, IMO, you shouldn't expect less.

HeroPsycho

darkerosxx wrote: »

Classic Microsoft, though, IMO, you shouldn't expect less.

I disagree. Microsoft is usually better about this kind of thing. You can site mistakes like this by any major IT company. I wouldn't say it's "classic Microsoft".

I remember in my newb days upgrading an ESX server from 2.5 to 3, and it blowing up the VMFS volume on the local storage, and all the VM's were lost as an example.

And in Microsoft's defense in this case, it was BETA software that has the problem. Like I said, my issue is it's easy to assume that all software included on install media for W2K8 should be production grade code.

astorrs

HeroPsycho wrote: »

I disagree. Microsoft is usually better about this kind of thing. You can site mistakes like this by any major IT company. I wouldn't say it's "classic Microsoft".

I remember in my newb days upgrading an ESX server from 2.5 to 3, and it blowing up the VMFS volume on the local storage, and all the VM's were lost as an example.

And in Microsoft's defense in this case, it was BETA software that has the problem. Like I said, my issue is it's easy to assume that all software included on install media for W2K8 should be production grade code.

Yeah the VMFS-2 to VMFS-3 upgrade could be fun...

dynamik

darkerosxx wrote: »

This is no one's fault but Microsoft's and their update should have allowed VM's created in beta to be used after the update. At they very least, it should say "UPGRADING FROM HYPER-V INCLUDED IN THE INSTALL MEDIA WILL BREAK YOUR VIRTUAL MACHINES!" and require approval before continuing.

Yea, you got it backwards. The beta software corrupted the newer VMs. There wasn't an update that broke them.

darkerosxx

I installed the server and accepted usage of the beta Hyper-V, made my VM's, then updated several days later and this happened.

Jordus

darkerosxx wrote: »

I installed the server and accepted usage of the beta Hyper-V, made my VM's, then updated several days later and this happened.

I did this once, in a non production environment (first time i was messing with hyper-V) and didnt have an issue.