Over the last week, I have being migrating hundreds of hosts from one vSphere to another to facilitate an architectural redesigned provided to my client by VMware PSO.
As part of the migration, the hosts were going to be reimaged and new network settings applied to them. Each host had two physical NICs of which both were used in a VMware Distributed switch (One pNIC for VM Network/Management, the other dedicated for vMotion)
This posed a problem that I still needed to remain control of the hosts while I removed them from vSphere. For those reading this that has ever tried to remove a host from vSphere you will find there are some hoops you need to jump through:
- The host must be in maintenance mode, or you get an error like this:
- The removal of the host will result in loss of access to VMs, Resource Pool, vApps that remain on the host as well as the admin/log data associated with the host.
- The host must be removed from the VMware Distributed Switch (vDS) before attempted removal of the host from vSphere.
The obvious solution if to remove the host from the vDS and then you can remove the host (Assuming you have placed it in maintenance mode as per the above error).
[Home] -> [Inventory] -> [Networking]
Opps.. That didn’t work… If you read the error message it is complaining that a VMKernel is still in use. That results in the source of this blog; you can’t remove the Management VMKernel otherwise you lose management access and you can’t migrate it (yet) because all your physical NICs are in use. Therefore graceful removal of the hosts from vSphere is like Catch 22.
So here is what I came up with:
- Evacuate the host of all VMs and vApps by placing the host into Maintenance Mode
- Remove all the VMKernel from the vDS by selecting ones NOT used for management traffic by going to the [Manage Virtual Adapters] of the [vSphere Distributed Switch] section of [Networking] , selecting the non-management VMKernel and selecting ‘remove’. Once there is only the VMKernel port marked for Management Traffic left, select [Close] to exit the [Manage Virtual Adapters] window.
- The next aim is to free up at least one physical uplink port, which is done via the [Manage Physical Adapters] option on the vDS. Select the uplink (In this case ‘vmnic1’) and selecting [remove], accept the warning, and select [ok] to exit the [Manage Physical Adapters] window.
- Select the [vSphere Standard Switch] window, select [Add Networking], select [Virtual Machine] and [Next]. You should select the pNIC that you just removed from the vDS and then [Next]
- At the next window, give the network and name, VLAN (If required in your network). The VLAN ID is not important as the aim is to create a new Standard Switch (vSS) not a working virtual machine network for live traffic to go down. Once this is completed, select [Next], read the summary and select [Finish]
- The [Networking] screen will look similar to this
- Return to the [vSphere Distributed Switch] window, and select [Manage Virtual Adapters]. Select the remaining VMKernel (The one marked as ‘Management Traffic: Enabled’) and select [Migrate]
- Follow the prompts, selecting the vSS switch you have just created, filling in a network label and a VLAN ID (This time it is important that this is correct!) and select [Finish]
- When the migration has completed, you can check that networking is now host-centric and not vSphere-centric by selecting [vSphere Standard Switch] and you can now see that the management traffic VMKernel has been migrated.
- The host can now be removed from the vDS via [Home] -> [Inventory] -> [Networking], and selecting the vDS, and the [Hosts] tab. Find the host you want to remove, right click on the host and select [Remove from vSphere Distributed Switch]
- When this has completed, return to the hosts view [Home] -> [Inventory] -> [Hosts and Clusters], right click on the host and select [Remove], read the warnings and select [Yes]
- The host will now be successful removed from vSphere and its management influence.
The host has now been removed from the vSphere Distributed Switch and vSphere. The host is now independent of vSphere management and be reconfigured, reimaged or decommissioned as required.
Thank you for the article, you mentioned “”””architectural redesigned””””; if possible can you share something about it so that I can check for the same in my environment & also it will be a learning.
Thanks
Thank you vaibhav for your kind comments!
I will attempt to write something in the near future about the redesign process that has taken place at my client’s site and the benefits it has brought (Within the confines of NDA).
can we remove host from vdist switch without vCenter server ??
Hi Mitra, vDS are a construct of vCenter with Enterprise Plus license. There isn’t a graceful way to remove a host directly without vCenter but you can reset the networking from the DCUI. This will result in a network loss for any VMs or features (eg. vMotion, iSCSI, etc) that are using the vDS. After reset you will just be on vSS. The vCenter will need to be cleaned up afterwards thus not ideal. If you can, use vCenter every time.
Thanks for quick reply !!!
Issue Description :
Our customer environment having 75 ESXi Host with 750 VMs , all esxi host are configured with vDvswitch, the problem is our vCenter was down because of DB corruption (we try after restoring last backup off the date 28th April -but no luck ) now we are planning to move to new vCenter server & add all esxi hosts in it- but for that purpose we must remove all the esxi host from vDvswitch from old vC & problem is vCenter server is down .
thats why i asked you for solution how to come out of this situation .. !!
Hi Mitra,
If you are planning to deploy a new VC then if you have a spare NIC — create a Standard Switch — migrate the VM’s and the vmkernel port on it — build a new VC — if you have backup of vds then import it — else create all vds — then migrate the esxi and vms one at a time to the new network.
Hi Christian,
do you have an idea how I can do this in vSphere 6.5?
I want to migrate the vmkernel mgmt port from a dvSwitch to a Standard Switch with the Host client, without having the vCenter!
I can migrate the VMs, but not the vmkernel…
Regards, Andre
Hi Andre,
I haven’t tried it for 6.5. I’ll try in my lab and get back to you. vDS are a construct of vCenter so what you suggest may not be possible without vCenter but I’ll take a look.
Chris
Simply perfect! Thanks
Hi, can you tell me if this process will work if the vCenter server is one of the Windows VM’s on the servers in question – which will be going into maintenance mode? I really just need to convert from distributed switch to standard – not remove the hosts from the vCenter. Can I do all of this by connecting directly to each host instead of to vCenter using vSphere client? I’m assuming vCenter will update it’s setting once taken back out of maintenance mode and pulls the updated settings from the hosts.
Assuming you have multiple hosts, you can use this technique to do this on one host, configure the host for vSS setup, then migrate vCenter onto the newly configured host (I don’t think you can migrate the vm network from vDS to vSS without downtime but I maybe wrong… So power down vCenter, connect to the vSS host, go to the same datastore that vCenter is stored and register it to inventory. Once registered, you can change the network on the vm, power on and reconnect to vCenter as normal. You can then complete the migration to vSS on the remaining hosts.
Please note this is untested but should work in theory. Please test throughly before doing so in production.
Hi Chris,
thanks for your reply.
I could do it susseccfully with vSphere 6 – in the old vSphere client I could migrate a mgmt vmkernel from a dvSwitch to a standard switch, without having the vCenter present.
But at vSphere 6.5 I found no way for that.
It would be very nice and helpful if I get a workaround for that.
Regards, Andre
Hi Chris,
I tested this today in another lab environment.
As I told in vSphere 6.0 I could migrate the vmkernel port from a dvswitch to a standard switch.
In vSphere 6.5 this is not possible in vSphere host client.
In KB1008127 I found the (hopefully right) solution for this – per CLI I can delete the existing vmkernel, after that recreate a new vmkernel for management at the standard switch with the same ip adress. So you are right, an online migration without mgmt downtime for the ESXi host is not possible.
After that I can manage the ESXi host, and the main important thing is, this is possible without a vcenter and without interrupting the network traffic for the virtual machines.
Maybe in a future version this could be managed a little bit more comfortable.
Regards,
Andre