Unable to Provision Machines using XenDesktop Setup Wizard – A case of “The object or item referred to could not be found” 3

Suppose you’ve have your environment for a bit of time. Your PVS targets are humming along happily and life is good. After a recent company expansion, you have planned to add capacity. You fire up the XenDesktop Setup Wizard in PVS to easily add additional machines. After running through the wizard, you step away for coffee, and when you return, you see the unthinkable: 0 devices created. 5 devices failed.

As anyone who has run into this scenario before, the XenDesktop setup wizard does not necessarily clean up all partially provisioned resources. So before you’re able to go and try again, you need to check a few things. Keep in mind that this article applies to XenApp and XenDesktop 7+ as they can both be provisioned using this wizard.

1. If you were creating a new machine catalog, was the catalog created? Did the machine get added to that catalog in the XenApp / XenDesktop site?

2. Was the machine added to PVS?

3. Did the machine get added to Active Directory?

4. Was the machine added to your ESXi host?

In terms of the provisioning process, the ESXi host is where the VM will be created first. The XenDesktop Setup Wizard will take your inputs and begin provisioning at vCenter. It is there that you see an error similar to the one below.

Strange times indeed. Since you are an excellent ninja Citrix admin, you have already enabled Continuous CDF Tracing in Provisioning Services 7.x, so you fire up CDFControl from the Citrix Supportability Pack and browse to the relevant section.

Here we see that the connection resources from XenDesktop are parsed and it has selected the network that we wanted, VMVLAN1, which is a distributed port group. However, a few lines below we see that not much more is known from the PVS side as this error is reported back to the PVS Wizard as simply an invalid operation/failure.

After some searching, a dependency showed up that shows that there are some cases where the Distributed Port Group “Key” – the unique identifier – can be different than the ID. Fire up PowerCLI, and execute the following:

Get-VDPortGroup | select name,id,key | fl

We see plainly from the original error that PVS is searching for dvportgroup-22, but the key for that port group is actually dvportgroup-562.

How can this happen?

It turns out that this scenario is actually common and affects more than just PVS and XenDesktop. There are reports that it has affected versions of VMware SRM at some point in the past. The issue arises when you migrate a Distributed vSwitch from one vCenter to another – something that in fact happened in this case. When that migration was done, you may typically choose to “Preserve original distributed switch and port group identifiers” in order to ease migration of VM NIC’s when migrating from one vCenter to another seamlessly. In these instances, the key can differ from the ID.

It would seem as though PVS parses the ID, then searches for the unique ID with that key, so when it executes the “reconfigure VM” step to assign the network, the job fails.

What are the workarounds?

Create new port groups and migrate all VM’s to the new port group, just keep in mind there may be availability concerns when doing this. Consult the vSphere documentation for more details.

Alternatively, if you don’t need to use BDM, you can use the Streamed Setup Wizard to provision a machine. Keep in mind your template will need a write cache drive attached and you will be unable to migrate to the BDM boot partition in a supported manner at a later time.

Conclusion

Even when everything is humming along nicely, there are things that may be out of your control (such as a vCenter upgrade) that could affect you. Keep an open mind and realize that technology has come to rely on APIs where you may not expect them to. The connection to vCenter is an important one, so keep it in your mind when planning upgrades as it could affect the availability of your Citrix stack.