VDI Performance – Part 1 – Gold Images

To build a house you have to start with a solid foundation, I hate that saying but it’s true. It applies to VDI just the same: to have an efficient VDI environment you MUST have consistent gold images. The term gold image is what I use to base all of my View Pools on. I have a 3 step process to building my gold images:

  1. Create OS Template
  2. Create Gold Image
  3. Install Core Company/Pool Specific Software

Before I start with building my Pool Specific Gold Images I start with a Base Operating System Template for vCenter, this allows me to clone for Gold Images, Dedicated VM’s or Test Boxes with all specific Tools, Agents and other items that must be installed for a View Environment. This is the most important step in the process because this truly is the foundation for all future Gold Images in my environment. Let’s get to work!

Create OS Template

Create New Virtual Machine

In vSphere I create a new virtual machine called “Win7Gold”, I select “Custom” configuration with the following specs:

  • Name your VM
  • Choose an Inventory Location
  • Choose Host/Cluster Location
  • Choose Resource Pool (If DRS is enabled)
  • Select a Storage Resouce (Local or Shared Storage)
  • Virtual Machine Version (since I am running ESXi 5.5 I choose the latest)
  • Guest OS Version (For this example I am using Windows 7 x64)
  • CPU Sockets/Cores – 2vCPU
  • Memory – 3GB (View Best Practices say 3GB minimum)
  • Network – Choose Network and Adapter – VMXNet3
  • SCSI Controller – LSI Logic SAS
  • Create a new virtual disk (unless importing from an existing disk)
  • HDD – 32GB Thin Provision (This is a template, no need to build out the whole disk)

vCenter will now build out the base VM, time to edit some hardware!

  • Remove Floppy (You can always add to a specific Gold Image later if needed)
  • Select “Options” tab and review the Boot Options
    • Select “Force BIOS Setup”
  • Power on VM and Open VM Console
  • VM will boot into BIOS, select Advanced Tab
  • Disable all of those ports (You can always add to a specific Gold Image later if needed)

BIOS Disable Peripherals

  • Save and Exit
  • Back in vCenter – Edit VM Settings and Mount OS Media ISO (Make sure to check “Connected” and “Connect at Power On” and install your Operating System

Win 7 Install

  • Once Windows has finished installing we need to install some VMware packages before anything else because of conflicts with .Net Framework and View Agent

VMwareInstalledPrograms

Next step is to optimize Windows for Horizon View, VMware has a great guide for optimizing including a script to run, I am a little more visual and discovered VMware Flings. Flings has a great application called VMware OS Optimization Tool that can automate a lot of the tweaking for your OS Template, so let’s use it to tweak!

FlingResults

Once the Fling is installed let’s run an analysis and optimize everything except: Windows Firewall (for VMware Blast), Windows Update (latest and greatest updates) and Windows Search. Notice how only 4 are left? Everyone’s tweaks will be different but there are recommended and optional ones you can review!

Once you have optimized with Flings uninstall the OS Optimization Tool and lets get Microsoft Updates installed.

WindowsUpdates

Once Windows gets all of it’s updates (I don’t install IE10) There is one last update that we need to install related to the VMXNet3 driver (KB2550978). After this you can go into Services and disable Windows Update. Once that is disabled we can delete the update leftovers from

C:\Windows\SoftwareDistribution\Download

WindowsFeatures

After Windows Updates are finished there are some Windows Features that we don’t need use, lets remove them (optional) :

  • Games
  • Windows DVD Maker
  • Windows Media Center
  • Internet Printing Client
  • Windows Fax and Scan
  • Tablet PC components
  • Windows Gadget Platform
  • XPS Services
  • XPS Viewer

Last item before Sysprep is to erase all event logs, you can do this by running this script in an elevated command prompt:

for /F "tokens=*" %1 in ('wevtutil.exe el') DO wevtutil.exe cl "%1"

Let’s get ready to Sysprep our OS Template and convert to a VM Template for our Gold Images.

Sysprep

Run sysprep.exe at choose Generalize, OOBE, Shutdown

ConvertToTemplate

Right Click on VM and Choose Template > Convert to Template. Now that we have converted our OS Template into a VM Template we can proceed to cloning our template to build a Gold Image for View Pools. What’s nice about building a VM Template as opposed to moving to a Gold Image to start with is that now I can deploy a quick Test machine or build out Dedicated VM’s without affecting any other images. Also if you use vCloud Director, it will require having images as Templates for provisioning purposes.

Create Gold Images

Now that we have our OS Template built we can clone it to make a Gold Image. We maintain several Gold Images based on scope of work or department. Eventually we will move away from so many Gold Images and start utilizing Horizon Mirage (v2 now supports VDI). Until then let’s build out a Gold Image for our Accounting Department.

In vCenter let’s clone our Win7Template to make AccountingGold.

DeployTemplate

Each of our View Pools has specific hardware requirements based on the software they use, so let’s make some changes to address some accounting software that we will install. I will enlarge the hard drive to 72GB and increase RAM to 4GB.

TweakHardware

Since we ran Sysprep on the OS Template VM we need to go through the basic Windows Install steps of the new Gold Image. Next join the machine to the domain and reboot. Some people say to join the domain last, but it has been my experience that most software packages are located on a central file server and it makes things easier to authenticate to AD and if you are trying to configure a DSN, SQL or AD specific application.

AdjustBestPerformance

I like to tweak the display properties at this point by going to System Properties > Advanced System Settings > Advanced > Performance > Settings > Adjust for Best Performance. You can also set the Windows Theme to Windows Classic.

Install Core Company/Pool Specific Software

From here the sky is the limit on what you want to install for your Gold Images, we typically install MS Office, Flash, Chrome, Java, etc. For all applications we install, we always do a first run to ensure everything installed correctly. If you install MS Office re-enable Windows Update service and update Office (Internet or WSUS since we are in the Domain now). Once complete, disable Windows Update service and delete updates from:

C:\Windows\SoftwareDistribution\Download

So let’s assume you have installed everything that you want and are ready to provision this image for use by a View Pool, there are several things you want to tweak to make the linked-clone process go smoothly.

SDelete is a slack utility for Windows that will remove any slack in Windows partitions, since we are running our VM’s in Thin Provision I like to make sure my Gold Images are as compact as possible when the Replica’s are produced. You will need to download a copy of SDelete here. To start let’s run a check disk for the system partition:

chkdsk /f

Now we can run sdelete from a run command. NOTE: this will take some time

sdelete -z

Now lets put the Host that our Gold Image is running on into SSH mode. Shutdown your Gold Image and open Putty or your preferred SSH client.

ls
cd vmfs
ls
cd volumes
ls
cd SPECIFIC DATASTORE (Case Sensitive)
ls
cd SPECIFIC VM (Case Sensitive)
ls
vmkfstools -K SPECIFIC VMDK FILE (Case Sensitive)

Hole Punching should take a while but will eventually finish.

Once that is complete, start your VM back up and run chkdsk /f one more time. Also be sure to disable SSH in vCenter. Clear the event logs one more time.

Now let’s build our shutdown script and we will be done with our Gold Image. We use this script to release the Windows KMS key, MS Office KMS Rearm, IP release and shutdown. Create a txt file on the desktop called “script.bat” right-click on bat file and edit, insert the following text and save:

"C:\Program Files (x86)\Common Files\microsoft shared\OfficeSoftwareProtectionPlatform\OSPPREARM"
 slmgr /ckms
 ipconfig /release
 ipconfig /flushdns
 shutdown -s -t 00

Run the script.bat file and your machine is now 100% ready to become a Parent Gold Image for a View Pool, you can take a snapshot and start provisioning a View Pool!

Snapshot

Up next in our 6 Part series is Part 2 – PCoIP Best Practices

VDI Performance Series Index

Part 1 – Gold Images

Part 2 – PCoIP Best Practices

Part 3 – Persistent vs Non-Persistent

Part 4 – Storage

Part 5 – End User Experience

Part 6 – Wrap Up and What’s Next

Advertisements

DFW VMUG User Conference Download

DFW Opening Slide Deck

DFW VMUG held it’s annual User Conference September 25 at the Irving Convention Center and it was a blast. It was great to see so many people that are a part of the VMware community come together and geek out. I even had some of my non-VM coworkers come and learn about some new software and hardware at the Vendor Crawl. It was cool for them to see the journey VMware is on.

13 - 3

Also got a chance to meet up with some of the DFW VMUG members (Nigel, John, Brad, Matt and Me) The keynotes were well prepared and the speakers for the breakout sessions had some great material to present. I couldn’t get to all of them but I attended 4 sessions and wanted to do a session download, we do this at our company anytime someone goes to a conference, it helps to keep everyone informed:

Evolving to the Software-Defined Data Center: EMC Integrations                     – Tommy Trudgen

Tommy Trudgen (vTexan) from EMC gave a great presentation showcasing the evolution of Virtualization and how EMC is helping push the boundaries of the Software Defined Data Center. He illustrated the highs and lows of EMC and how they capitalized emerging software and technologies to keep them moving forward and a leader in the storage market.

VSAN Technical Best Practices                                                                                           – Dan Gillcrist

By the time VSAN had been announced at VMworld, I wasn’t able to get signed up any sessions. I wasn’t going to waste time with Hands On Labs at VMworld so I was really pumped to do a deep dive on the subject. For those that aren’t familiar with VSAN it is VMware’s version of SAN independence and storage convergence. The best example I can give on convergence is Google’s Server Clusters, by storing spinning drives at the nodes instead of centrally storing in a SAN, they couple all the disks into logical arrays. By doing this, you can scale storage and performance on demand as opposed to sizing a SAN prior to implementation and then adding shelves to handle growth, be it expected or not.

That was the quick version, for a more detailed assessment of what VSAN is, check out Duncan’s article at Yellow Bricks, lets dive into some best practices for VSAN.

Automation Generation                                                                                                       – Nick Weaver

Automation is a concept that I am trying to learn more about, this session threw gas on the fire for me. PowerCLI and PowerShell is the start to making vCenter commands more automated, but it doesn’t automate processes, in comes Puppet Labs. Puppet Labs is an automation platform (Purchase and Open Source Models) that the vCloud and vSphere teams use to automate provisioning of Application stacks and other management intensive processes. VMware also makes available all of the Puppet Lab scripts that they use on GitHub.

Think about the last time you implemented a software solution, what was the first thing you did, white papers right? Our company implemented an ERP platform and our install process was over 200 steps just for the servers! Now think about spitting out some code and clicking run….yup! To play with Puppet Labs start with something simple like the vCenter Server Appliance and configure IP, DNS, hostname, etc. In very simple terms, that’s Puppet Labs, check them out and if you have any questions, give Nick Weaver a tweet, also check out another project works on: Project Zombie.

The Emerging Technologies That Will Change VDI                                                   – Mark Vaughn

If you are part of the SLED market (State, Local and Education) then you have probably heard of Presidio, they have implemented VDI solutions in 1/3 of the Nations Colleges that is no small feat. Mark Vaughn gave a compelling presentation of common misconceptions with VDI, areas to avoid, lessons learned and what we can expect from VDI in the near future and how VMware is leading the industry in virtualizing desktops and offering Desktops-as-a-Service (DaaS).

Troubleshooting Storage Performance                                                                         – Brad Pinkston

Storage performance is on every admin’s mind, unless you have a Flash array! Brad gave a high level overview of how storage has evolved with the releases of vSphere 5.5, VSAN, and vFlas Read Cache. I got some good nuggets of information when running esxtop and understanding averages vs peaks, here are some valuable slides on esxtop info.

Disk IO Latencies Disk IO Queues esxtop explained Storage Best PracticesOn top of those valuable slides, he also recommended some tools that can be used to diagnose storage problems and help isolate the problem:

All in all it was a great User Conference, this was my first and will not be my last. Up next VMworld Barcelona is going on right now and then later in October VMworld on the Road is coming also, check those out if you can!

VDI Performance Series – Let’s Do This

What is all this talk about VDI?

VDI is a virtual desktop infrastructure designed to provision desktops as a service to the end user. The desktops are stored in the datacenter typically using a Host – Shared Storage scenario. End Users can connect to these Virtual Desktops using Hardware thin/zero client endpoints or a variety of software clients on PC, Mac, iOS, or Android. You can do a P2V for existing machines or build out “Pools” of desktops for specific use cases or by departments.

VMware’s VDI solution is called Horizon View, which is now part of the Horizon Suite. There are various components that make up Horizon View, I’m not going to go into that here because we are focusing primarily on VDI performance.  I’ve included a diagram of what a typical VDI environment would look like.

VMware-View-Layout

Our company made the decision to deploy VMware’s VDI solution Horizon View (as of 5.2) about 18 months ago. The adoption rate has been incredible, we began with a small test group of 15 users, mainly administrative, task workers and some advanced workers. Our test group rapidly became our production group as we went from 15 to over 100 in less than 6 months.

Fast forward to today and we have almost 200 desktops deployed on View and there isn’t any sign of us slowing down, we have about 80% of our staff on VDI and the other remaining users begging for the “black box”. They just flip out when they can swtich from a Zero Client station to their iPad to a conference room and back again!

With fast growth comes scaling problems, and we have encountered our fair share of them. First it was battery backup, then it was inadequate cooling, now it is performance. VMware has some great articles about industry Best Practices when deploying View, but there is much more information out there, some from VMware and not.

My goal is to lay out the hurdles that we have overcome with VDI performance, the lessons I’ve learned and what we are planning on doing in the future.

Part 1 – Gold Images

Part 2 – PCoIP Best Practices

Part 3 – Persistent vs Non-Persistent

Part 4 – Storage

Part 5 – End User Experience

Part 6 – Wrap Up and What’s Next

Look for the 6 Parts over the next several weeks.

vSphere 5.5 Upgrade – Walkthrough and Notes

Since the announcement of vSphere 5.5 at VMworld, I have been waiting for the day to upgrade our environment. That day has finally come. We decided to go with the Simple Install since there isn’t anything custom when we install vCenter.

We have Horizon View in our environment, so the first step was to shutdown all workstations, linked-clones, etc and disable provisioning to not create additional clones. We shutdown all servers except our Firewall, Exchange, Primary File Server and vCenter servers (Apps and DB’s). Now that we have our environment in a controlled state, we are ready to start the upgrade process.  So lets get to it!

Download the vCenter 5.5 Install Package for Windows from VMware, mount the ISO or extract the contents to a temp location and select autorun.exe

Slide0Select the Simple Install Option, review the upgrade process and click Install to start the upgrade of Single Sign-On.

Slide1

It has already detected the previous version, Click Next

Slide2

Accept the End User License Agreement, who really reads that stuff! Click Next

Slide3

Make sure that the FQDN, IP is accurate. vCenter Windows Server is joined to the Domain and Value is correct, Click Next

Slide4

It has detected the previous instance of SSO so it will migrate the existing information, Click Next

Slide5

Choose the local default domain account credentials for SSO, Click Next

Slide6

Confirm the Site Name, Click Next

Slide7

Choose the install location, Click Next

Slide8

Here is an overview of the choices we made, time to Install

Slide10

It will take a little bit of time to install, when it completes it will proceed to start installing vSphere Web Client components.

 Slide3

vSphere Web Client begins to install and register with SSO. Next up is the Inventory Service.

Slide1

Pretty simply choice, keep your existing inventory or stay up all night and rebuild it. We chose to get some sleep tonight! Click Next

Slide2

It’s ready to install the Inventory Services, Click Install

Slide5

Install the components and watch the progress bar. It will immediately go into vCenter Upgrade next.

Slide1

Click Next

Slide2

Enter License, Click Next Slide3 \

Enter your SQL Database credentials for vCenter Database, Click Next

Slide4

It prompts you that vSphere Update Manager isn’t compatible with 5.5 (we will upgrade that after we are done), Click OK

Slide5

Upgrade the existing vCenter database, check the box, Click Next

Slide6

Choose Automatic, Click Next

Slide7

Unless you want a Domain Account to auto login, choose a separate login account for the vCenter Service Account.

Slide8

Validate your ports, you have the option to increase the ephemeral ports. We didn’t increase them because we will not hit 2000+ VM’s before vSphere 6 comes out, Click Next

Slide9

Everybody has a different take on this option, we chose Large so that we don’t have to hit an artificial limit on JVM Memory issues, to each his own, Click Next

Slide10

Time to Install, Click Install

Slide11

The database upgrade took a while…

Slide13

Progress!!!

Slide15

vCenter has finished the upgrade process, are we done? Not just yet!

Time to install the Windows vSphere Client, just select the Client install option and click Install.

Slide0

Next through the options since there isn’t any customization to apply.

Slide5

vSphere Client is updated, to sign in!

So at this point we went to fire up vCenter Windows Client to start upgrading Update Manager but ran into an error where our AD accounts wouldn’t authenticate to vCenter. After a little research (Google!) we found this article from VMware. We have a AD Admin group nested into the Local Admin group, upgraded vCenter doesn’t like this.

It was an easy fix, log in using local admin to vSphere Web Client, go to vCenter permissions and add Domain Admin group (users to manage vCenter), then you are back in business. Now it’s time to upgrade vSphere Update Manager so we can start upgrading ESXi Hosts.

Slide1

Choose vSphere Update Manager and Click Install

Slide2

In the Installation has detected our existing version of Update Manager, Click OK

Slide3

Click Next

Slide4

Allow Update Manager to download baselines, patches and others after install, Click Next

Slide5

Verify your vCenter Information, remember that vCenter account we entered earlier? Time to enter it again, it will validate and Click Next

Slide6

SQL Connection Info, Click Next

Slide7

Choose Yes I want to upgrade my VUM Database!, Click Next

Slide8

Verify Port Settings, Click Next

Slide9

Click Install

Slide10

Watch the progress!

Slide11

Installation is complete time to fire up the vSphere Windows Client to install the plugin.

Slide12

Login using your credentials or use Windows Credentials. Notice the warning in the new version, you MUST use Windows VIC to manage Update Manager, Site Recovery Manager or an isolated Host.

Now we need to load vSphere Update Manager into our vSphere Client so navigate to Plug-ins > Manage Plug-ins

Slide13

Choose Download and Install under the Status Column, It will fire up the Client Install

Slide15

Click Next

Slide16

Accept the End User License Agreement, Click Next

Slide17

Click Install

Slide18

Watch the progress!

Slide19

Click Finish

Slide20

Now we see that vSphere Update Manager has been installed and is “Enabled” in vSphere Client, time to start upgrading hosts!

First we need to create a Baseline to see which hosts are out of compliance. So let’s start there.

From the Home view select Update Manager from Solutions and Applications

Slide1

Create a new Baseline

Slide2

Choose a Name and select Host Upgrade for your baseline type, Click Next

Slide3

Choose you ESXi Image, if you do not see the ESXi image in the list, go to Admin View > ESXi Images tab to import the ESXi 5.5 image, Click Next

Slide4

 

Verify your settings and Click Finish, now we can run our baseline again the hosts we want to upgrade. For now we are only upgrading our Production Server Hosts because we are waiting on Teradici to update the Firmware of the APEX 2800 PCoIP Offload Cards for vSphere 5.5

 

Final Thoughts

So we simulated almost everything before we did the upgrade in Production, aside from the Nested AD Group, we didn’t hit any snags. I hope this walk through has been beneficial. If you have any questions or comments please post them.

vSphere 5.5 Upgrade – Ready, Aim, Fire!

I was hoping to have my DFW VMUG User Conference Download article finished today but I am waiting on my slide decks to put the final pieces together. So look for that article in the next few days, spoiler alert: the User Conference was awesome!

I want to upgrade vSphere 5.5 all night long!

Tonight we will be upgrading our vCenter environment to 5.5. As our team was going through our checklist I ran across some good tidbits of information, a lot of which came from my previous post. The two biggest items so far have been the which order to upgrade the components and to make sure there would be any hiccups during the upgrade. The component upgrade process we are going to follow is based on KB Article 2057795. Here is the upgrade order I plan to take:

5.5 Upgrade

So far we are aware of one issue going into the upgrade, there is a Single Sign-On issue related to AD authentication when running SSO on Windows 2012, you can find the reference here. I’m glad we caught this because we were planning to migrate our vCenter server onto 2012. There is a fix, but will wait to do the OS upgrade for another night.

I will be following up with my notes on how the upgrade process went.

Wish me luck!