Dan Griffin’s Blog » Software Testing with Virtual Machines

Dan Griffin's Blog

Comments on security, PKI, smart cards, cryptography, and entrepreneurship.

Software Testing with Virtual Machines

April 22, 2007

My company does most of its software development testing using virtual machines, since that approach reduces the cost of setting up (and tearing down) complex scenarios. The VM approach raises some interesting challenges too …

One recent complex test configuration demonstrates such a challenge. The config is an Active Directory environment supporting smart card logon, deployment via Identity Lifecycle Manager, and a peer-to-peer IPsec policy. That configuration requires at least three separate images (one for Domain Controller/CA/IIS/SQL/ILM, and two clients to test IPsec). We typically try to run all of the images on a single server box via VMware Server. In this case, even though both clients are Vista - which we’ve found to be a real resource hog when it comes to virtualization - a single server with decent RAM and fast disks can handle it (just don’t try to boot all of the images at the same time if you’re in a hurry). In fact, we’ve even got a Longhorn Server Beta 3 Escrow guest image running smoothly on a Vista host - how’s that for cutting edge!

In general, the VM configuration allows for kernel debugging of the guests via serial pipes, as well as restoring snapshots when something goes really wrong during testing. All goodness.

Smart card testing in that configuration continues to pose a problem, though. We’ve encountered two issues. First issue - and there’s really no way around this one - we primarily interact with the VMs by connecting to the host server via RDP (Remote Desktop Protocol/Terminal Services/TS). Then we view the guests via the VM console from within the remote session. This is especially convenient since it allows the developer to toggle between VM windows during testing, rather than toggle between TS client windows (I find the latter to be truly painful). However, if the host server happens to be in a different room, then there’s no convenient way to do smart card testing. Why? Well, in this case, the TS smart card redirection feature allows the server to see any smart card readers plugged into the TS client. But those devices are only visible on the server via the smart card subsystem. I.e. they don’t show up in the USB stack. Thus they’re invisible to the guest VMs.

There’s a work-around to this, but in essence it breaks the virtual machine scalability model, which is to run as few physical machines as possible. The work-around is to run one of the test VMs from either the developer workstation, or a secondary machine sitting next to it. This raises the second issue: using VMware on Vista hosts, we haven’t been able to get the guest OS to detect a USB smart card reader (not sure if we’ve tried any other type of USB device). Anyone else had any luck with this? The same scenario works using XP as the host.

Anyway, the developer machine is running Vista, since that’s most of the work we do. Therefore, the only option has been to run a separate XP test machine right next to it, and to run a Vista VM guest on the XP host. The interesting bit is that we’re still connecting to the host machine via RDP, just like in the first scenario above. The difference in this case is that the smart card reader is plugged into the VM host machine, instead of into the developer machine. Since the VM host machine is sitting right next to the developer, there’s no problem accomplishing device removal and insertion. And since the host is XP, the Vista VM manages to latch onto the USB device.

I recently read that there’s a VMware v6 Beta available which advertises full Vista support. Hopefully it fixes the USB issue; we haven’t tried it yet. The work-around doesn’t work so well if multiple smart-card-enabled Vista guests are required in parallel, since anything less than server-class hardware will severly drag under that load.

Links:

* What smart card redirection looks like under the covers - http://blogs.msdn.com/spatdsg/archive/2006/10/16/smartcard-logon-over-terminal-services-rdp-redirection.aspx. The architecture is slightly different in Vista; Microsoft doesn’t seem to have documented this anywhere, though.

* VMware 6.0 Beta - http://www.vmware.com/products/beta/ws/releasenotes_ws60_beta.html.

Permalink |