PowerCLI 4.1 U1: Current state of affairs

UPDATE 10-JAN-11: To deal with the issues identified below, I published a beta version of the VMware vSphere PowerPack with PowerCLI 4.1 or later support.  You can read more about the beta and how to install it here.

Luc Dekens suggested that I share my concerns about upgrading to PowerCLI 4.1 U1 here so that the rest of the community could be aware of some of the details behind the issues between upgrading from PowerCLI 4.0 U1 to version 4.1 or 4.1 U1.  Ask and ye shall receive. Smile

Back in July 2010 I shared my concerns about upgrading to version 4.1 of the PowerCLI due to a number of breaking changes introduced in that release.  These changes were far reaching and have caused us challenges as we try to support the VMware community with our VMware PowerPack for PowerGUI®.  A little while later, Scott Herold sat down with members of the PowerCLI team and they agreed to put in temporary support for the old object model as part of their 4.1 U1 release, so going forward this seemed like it would give us a smooth upgrade path for our customers.  PowerCLI 4.1 U1 was released earlier this month, and it was supposed to address these issues while also including great new enhancements for the community.  Unfortunately we were not aware of the release schedule for the PowerCLI, nor were we provided early release versions to work with, so while we were working on our PowerGUI Pro and PowerGUI 2.3 releases which included (among many other things) the VMware PowerPack for the first time, we had no choice but to include a firm requirement in the installer for PowerCLI 4.0 U1.  This is why the VMware PowerPack will not install with PowerGUI Pro or PowerGUI 2.3 if you are using any version of the PowerCLI other than 4.0 U1.  As a result, the inclusion of the VMware PowerPack in PowerGUI has met mixed reviews because many community members expected it to work with PowerCLI 4.1.1 out of the box, however we had no knowledge of that release during our own release cycle so we simply could not build-in support for that release.

Now that PowerGUI Pro and PowerGUI 2.3 are released as well as VMware’s PowerCLI 4.1 U1 though, which is supposed to come with complete support for the old type names, I suspected that I would be able to manually upgrade my PowerCLI to the latest version and that my PowerPack would continue to function the way it did before, possibly having a few minor issues to deal with.  That suspicion has proved incorrect because of a number of problems.  Based on my initial testing, here are the issues that I see in PowerCLI 4.1 U1:

  1. PowerCLI 4.1 U1 still returns the same type hierarchies that were returned in PowerCLI 4.1, which are not compatible with the VMware PowerPack which is based on the 4.0 U1 types.  As indicated on the blog post titled “PowerCLI 4.1: A fork in the road”, these incompatibilities affect many community-driven content that is based on the PowerCLI, including the VMware PowerPack, the VMware Community PowerPack, books, blog posts, shared scripts on PoshCode, the VMware VI Toolkit Extensions module, etc.  I have tested multiple commands (Get-VMHost, Get-VM, Get-Cluster, Get-Datacenter) against multiple labs using PowerCLI 4.1 U1 and none of them show the old type names, so backward compatibility is still broken as far as I can tell.
  2. When you invoke a PowerCLI cmdlet like Get-VM, if you show all of the properties for those VMs you will now see warnings about properties that are to be deprecated in a future release.  The properties still work today, but the warning indicates they are obsolete and therefore they are not guaranteed to work tomorrow.  The trouble is you see these warnings even if you are not directly using those properties.  For example, if you invoke Get-VM and pipe that to Format-List * to see all properties, you’ll see warnings in the output before you see the results.  In this case the warnings are benign (unless there are other warnings mixed in that should get your attention), however it shows a disturbing trend that VMware has taken to changing PowerCLI object data such that scripts that work today are not guaranteed to work tomorrow unless they are updated when the existing properties are deprecated, causing multiple rifts between users are on the current release and those who don’t upgrade right away.  It means you will see multiple ways to do things in scripts over time that have different properties, and when you view those scripts you may or may not be aware of which is current so this is bound to cause some user confusion.  This also makes it difficult for anyone to consider upgrading if you use anything with scripts that are dependent on the PowerCLI (PowerPacks for example), because there is no guarantee with this approach that those will not break when you switch from one version to the next unless you are sure they have upgraded to support new releases as well where object properties may deprecate over time.  Blog content that normally could be published and left as is for a very long time may become out of date more quickly.  Books will have a much more limited lifetime if they don’t use the right properties.  And so on.

Breaking changes in any programming language are considered something to completely avoid unless those changes are absolutely necessary.  On those rare few occasions when they are necessary, there are mechanisms in place that allow you to reduce the impact of those breaking changes (side-by-side versions for example).  PowerShell has a rich Extended Type System that allows for type extensions so that you can evolve objects over time, using alias properties, script properties, code methods, etc. to allow you to continue supporting older versions while you upgrade your technology to new versions.  All of these together should give cmdlet authors enough flexibility that they don’t need to introduce breaking changes haphazardly like this.  Unfortunately I don’t feel that VMware gets this yet, but I continue to try to evangelize this and work with them so that this gets easier going forward.

If you have any questions about any of this, please don’t hesitate to ask.

Thanks,

Kirk out.

A letter of apology to the VMware PowerPack community

UPDATE 10-JAN-11: I just published a beta version of the VMware vSphere PowerPack with PowerCLI 4.1 or later support.  You can read more about the beta and how to install it here.

I have heard via email and Twitter that quite a number of you were disappointed when you upgraded PowerGUI® Pro or PowerGUI® to version 2.3 only to discover that the VMware PowerPack does not work with PowerCLI 4.1.1.  Instead the VMware PowerPack today has a firm requirement for PowerCLI 4.0 U1.  I apologize for the disappointment that you have experienced with this and I felt I should take a minute to talk about the situation we are facing and let you know what our plans are going forward.

When VMware released version 4.1 of the PowerCLI, they completely changed the object model that they were using in earlier releases.  This change came with many undesirable and unwelcome side effects, effectively breaking backwards compatibility with many scripts that are built on top of the PowerCLI cmdlets, and the VMware PowerPack was no exception to the effects of those breaking changes.  We have been working with VMware to ensure that they are aware of the ramifications of changes like this, but it is an uphill battle because while they have said that they were committed to providing backwards compatibility for the old type names back into the PowerCLI in 4.1.1, our testing shows that this backward compatibility is not there like it should be.  Further, you may now start to see many warnings from scripts that previously ran without warning or errors because VMware is planning to deprecate many of the properties that were there in previous versions.  These warnings are a concern because they show that VMware is continuing to make changes that will break older scripts going forward without much communication at this point about why they are making those changes.  I am working with many other MVPs and vExperts in the community to give feedback about why this is really not a good idea, but unfortunately we haven’t had any knowledge about the changes in these releases any earlier than you have, so we haven’t been in a position to work closely with VMware to help prevent complications such as the ones introduced in the 4.1 and 4.1.1 releases.

Now we are faced with a decision about what should be done going forward.  I cannot simply do an upgrade for the VMware PowerPack to require 4.1.1 because existing customers who are using PowerCLI 4.0 U1 and who may not be able to upgrade to 4.1.1 right away due to internal policies will have a broken PowerPack on their hands if their PowerPack is auto-updated.  I can however create a separate version of the VMware PowerPack that does not auto-update to support users who have upgraded to PowerCLI 4.1.1 and who want to use the VMware PowerPack.  This will take some time though because the changes are not straightforward, and if you want a version of the PowerPack that has been put through the property quality control channels we will have to retest everything to make sure it works with the new version, which is no small feat for a PowerPack of this size.  I may also be able to create a version that works with both PowerCLI 4.0 U1 and PowerCLI 4.1.1, however I’m not very optimistic about that at this time.  There is also a VMware Community PowerPack available that I do not manage, and it suffers the same challenges.  All of this adds up to a lot of things to consider and a lot more work than should be required when looking at the right way to upgrade this PowerPack (and those considerations apply the VMware Community PowerPack as well as I just mentioned since it is partially dependent on the VMware PowerPack as well as the PowerCLI).

What will most likely happen is that we will create a version of the VMware PowerPack that requires PowerCLI 4.1.1 or later, and that we will try to support two different versions of the PowerPack for a while until we eventually deprecate the older version.  That is not what I would like to do, but my hand is being forced here so that is the only realistic option I see in front of me.  I may be able to create a preliminary beta version of that PowerPack fairly quickly, however it will not have been put through a full test cycle and it will not address the warnings that appear from properties that are being deprecated – that will require much more time and effort.  I will of course notify you when this beta version is available.  Our plan is obviously to support every new release of snapins and modules that come out that are used with our PowerPacks, ideally the moment that those releases become publically available, but usually providing that support is much easier than this so please bear with us as we work through the messy situation we’re in right now.

In the meantime I would recommend that you continue to use PowerCLI 4.0 U1 with the VMware PowerPack and the VMware Community PowerPack because it works very well and is something we can support today.  At the same time you can of course put version 4.1.1 of the PowerCLI in a separate VM so that you can work with both versions for a while — we are talking about a virtualization solution after all…

Thank you for your patience and continued support while we work through these challenges together.

Sincerely,

Kirk Munro