The process of configuring our IPsec demo (http://download.microsoft.com/download/f/1/2/f12dbbb5-d164-4e7c-b42d-aaca3efb85dc/IPsecPingSample.EXE) to run on a TechEd kiosk has been educational! Here’s what I’ve learned about configuring IPsec policy and about IPsec programming considerations for non-domain joined environments.

First, the version of the IPsecPing client code available above makes a call to WSASetSocketPeerTargetName with a Service Principal Name (SPN) based on the DNS name (specified via the command-line) of the target peer. Using an SPN forces the identity of the server to be verified by the underlying authentication protocol, e.g. Kerberos. However, between peers that lack trust (i.e. non-domain joined; or joined to domains that don’t share trust), Kerberos isn’t available. This was the case in my TechEd demo environment. Anticipating this, I configured shared-password-based IPsec policies. But I’d forgotten that the code was written to require checking the SPN.

Note – I’m told that certificate-based IPsec authentication also supports verification of an SPN (presumably via something resembling schannel name mapping), and that in that situation a Kerberos environment isn’t required. I haven’t confirmed that.

Anyway, to allow IPsecPing to work in my demo environment, I modified the code to only conditionally call WSASetSocketPeerTargetName (based on the presence of a command-line argument). Now I’m able to confirm IPsec-protected IPv4 connectivity between the un-trusted peers.

But IPsec-protected IPv6 connectivity wasn’t working! Why? Well, this one was easy – I was using an IPsec policy that only specified v4 addresses. Actually, I’d followed the directions on p. 1144 of the Vista Resource Kit book (http://download.microsoft.com/download/f/c/7/fc7d048b-b7a5-4add-be2c-baaee38091e3/9780735622838_WindowsVistaRK_ch27.pdf) and created a Connection Security rule via Windows Firewall. However, when creating an address-specific rule that way, it appears that the only way to provide an v6-compatible policy is to type in the v6 addresses. I’m just too lazy for that

So I disabled my WF Connection Security rule and instead created a new IPsec policy via the IP Security Policies snap-in. That approach is arguably more time-consuming than the above, but it allows policies to be specified via DNS name (which – note – can probably be spoofed). The resulting filter ruleset seems to apply to both v4 and v6 traffic. Running IPsecPing with the -6 switch confirmed this (I used "stcpserver.exe -v6", from the Vista SDK sample code, on the test peer in this case; note that the latter listens on port 27015).