Issue migrating the physical NIC on a nested lab from standard switch to dvSwitch

ande0255

So I was trying to play around with dvSwitch settings the last day or so, and I am having a hell of a time figuring this one out or finding an answer on my own. I am having a hard time finding a way to move the only physical NIC available on my nest lab from the standard switch that was automatically created with ESXi configuration to a dvSwitch I recently set up.

When I first went to migrate the hosts, it threw errors about creating VMk ports on the dvSwitch before removing the physical adapter from the standard switch, so I made a couple of VMk ports for each host and added them to the dvSwitch. Now I have tried to 'Remove' the physical adapter from the standard switches which it appeared it did, however in the dvPort Group it shows the same management IP that was set on the standard switch (not the newly created VMk port IPs), and when I go to add the physical NIC as a dvUplink it gives the following error after asking if I'm sure I want to move it from vSwitch0 to my dvSwitch:

"Call "HostNetworkSystem.UpdateNetworkConfig" for object "networkSystem-34" on vCenter Server "vCenter.vmlab.lan" failed.
Network configuration change disconnected the host '192.168.23.129' from vCenter server and has been rolled back.
fault.NetworkDisruptedAndConfigRolledBack.summary"

Wondering if there is a way to move this back and forth, or if the restriction of one physical NIC in the nested lab is going to be a huge restriction when it comes to labbing networking concepts?

Also on a side note, I noticed that the old / original mgmt IP's from the standard switch are still responding to pings, but the new IP's I assigned to the VMk ports on the dvS are non-responsive, and I am wondering if this is because the physical NIC is not moving to the dvSwitch?

It's odd cause when I look at the standard switches it shows vmnic0 (physical nic) as 'unused', but I cannot get this thing to migrate over to the dvSwitch.

Thoughts?

Essendon

I'm trying to understand what you've done, so pardon me if we go back and forth a little here. First, what do you mean by a physical NIC in a nested lab. Are you really trying to migrate the pNIC that's on your pHost? Or are you trying to migrate the vNIC on your nested host? Here's what I'd do, to keep it simple.

- Shut your vHosts down.
- In your Web Client, give your vHosts a few NIC's, there should be one already. Put them all on the same network as the one already there. I have 8 vNIC's on all my vHosts.
- Power on your vHosts.
- You'll now see the new vNIC's show up as available adapters. Add one of them to the default vSwitch on the vHosts.
- Remove one of the vNIC's and add it to the vDS. Migrate your vmknic to the vDS.
- Then remove the remaining NIC on the vSwitch and add to the vDS.
- This should complete the migration process.

EDIT: I think the problem your running into is the same I ran into too. This happens in a nested lab. It doesnt let you migrate the Management Network to the vDS, but does let you move any other vmknic's over to the vDS. Anyhow, give the above a go and see if it works. If it doesnt, you'll know you cant migrate the Mgmt N/w over to the vDS. It wouldnt let me move it with nested 5.0, unsure whether you can migrate with nested 5.5. Try and let us know!

jibbajabba

Did you allow promiscuous mode on your physical portgroups / vSwitches ?

Either way, not sure I understand the problem. My lab has 4 physical hosts and all I have on there are multiple nested labs which are a mix of 5.0 and 5.5 labs. All nested labs are on their own dvSwitch with no standard switch to speak of ..

I just build another physical server which I will use for nested labs as well ... IF my SSD arrives on time I will install the lot during the day, if that is the case then I will make screenshots of the process, maybe that makes it understand better what is going on.

ande0255

@ Essendon - Thank you for the input, I will definitely check this out tomorrow on the lab and let you know!

jibbajabba wrote: »

Did you allow promiscuous mode on your physical portgroups / vSwitches ?

Either way, not sure I understand the problem. My lab has 4 physical hosts and all I have on there are multiple nested labs which are a mix of 5.0 and 5.5 labs. All nested labs are on their own dvSwitch with no standard switch to speak of ..

I just build another physical server which I will use for nested labs as well ... IF my SSD arrives on time I will install the lot during the day, if that is the case then I will make screenshots of the process, maybe that makes it understand better what is going on.

I guess it's not so much of an issue as my lab didn't stop functioning, but it will not seem to let me assign the one NIC on my laptop as the physical adapter on the dvSwitch, giving the error I posted above. It seems as though the physical NIC is bound to the standard switch, and I feel like I am missing a step in the removal / migration process of moving the physical NIC from the standard switch to the dvSwitch.

Essendon

I dont understand why you have to muck around with the pNIC? Dont worry about the pNIC at all, just ensure you have promiscuous mode enabled like jibba said. Then do what I suggested. DONT eff with the pNIC!

jibbajabba

Err no, no need to mess with the pNIC .. UNLESS you have your vCenter on a complete different setup.

No point at all ... We might just misunderstand your setup there ?

ande0255

From my understanding from what I read in the Master vSphere 5 book, a vSwitch can be associated to multiple physical NICs, but a physical NIC can only be bound to a single vSwitch.

So my understanding is if you have 5 pNICs and a single vSwitch, you can use all 5 pNICs for the vSwitch, but if you have 5 vSwitches and 1pNIC then only one of those vSwitches can use the pNIC as an Uplink.

^ Is that an accurate statement? This is what led me to believe to migrate my ESXi hosts to the dvS from the standard switch, I would need to remove the physical NIC from the standard switch and attach it to the dvS.

Thanks again for the input!

jibbajabba

A vSwitch can have no, or all your physical nics associated. But you cannot associate a single physical nic with multiple vSwitches.

So, back to your nested lab, what is it you are trying to do ?

Here is a nested lab I started on a new "PC" I built today (takes all bloody day to windows update 2008R2 servers with SQL).

1 Physical host with (currently) the minimum amount of VMs to get a vCenter up and running (well, my way anyway, you could just use an appliance I suppose)



When you check the network - you can see I only got a single pNic



So there is only one vSwitch with one uplink



Once I finished the vCenter I will create, on the same physical host, multiple vSphere ESXi hosts which will then be attached to the said vCenter. Each VIRTUAL host will have anything from 4-6 virtual nics. The vCenter will then have a Distributed Switch with all those uplinks.

That is the beauty in nested labs, you can simulate multiple nics even though your underlying host has just one nic, and in my case, a really bad one at that.

From this point on there is nothing more you need to do with the physical host - just leave it alone. As I mentioned before, only requirement really is that Promiscuous mode needs to be allowed / set to Accept



You can of course attach your only host to the vCenter you host on it - but that is really just useful if you need to change configs etc. and you want everything under one "roof".

Hope it makes sense. Oh and in general, in order to migrate to a DVS, you indeed need an available pnic

You basically

1. Attach host to distributed switch using unclaimed adapter
2. Migrate vmkernel to distributed switch
3. Migrate portgroups to distributed switch
4. Remove standard switch
5. Add now available pnic to distributed switch.

Above example is when you for example got one standard switch with two pnics.

jibbajabba

Actually that is more elegant and demonstrates it a lot better what I mean with "no or all"

Essendon

I reckon what he's confused by is "physical NIC". In a virtual lab, there arent any physical NIC's (except the ones in the physical host). Just replace the word physical with the word virtual in your head, and you'll be fine.

ande0255

I think you gentlemen are right actually, I haven't had time in front of my pc to verify it yet, but I bet I'm thinking about it incorrectly. Rep for you jabba and apparently I rep'd you too recently to leave any Essendon, but I appreciate both your guys help a ton in getting some of these concepts straight.

And the second illustration did really drive home what you were talking about Jabba, thanks for that!

I'll report back if I run into any more derps on this subject

jibbajabba

It is like in a 'real' world. Your PC has one network card. Yo can't connect it with an RJ45 cable to two switches at the same time.

Some concepts are trickier and some are easier than you think