SSL Cipher Order strikes again

Issue:

Lync Server 2010 would intermittently fail to communicate with AOL AIM clients via PIC.

But wait...

That's not all... At the same time I also noticed the edge failing to negotiate TLS with Federated Partners that had Entrust certificates (may affect other CAs, but I noticed Entrust in my eventlog)

Note that this would only be reproduciable if your Lync/OCS 2007 R2 Edge role is running Windows Server 2008 (x64) or Windows Server 2008 R2 (x64); not Windows Server 2003 (x64).

Resolution:

Modify the SSL cipher order so that the OCS/Lync Edge role will initially establish the SSL dialog using the TLS_RSA_WITH_RC4_128_MD5 cipher suite.

In order to change the cipher suite order, do the following on your Windows Server 2008 (x64) or Windows Server 2008 R2 Edge server (if the edge server is joined to a DMZ domain then the Group Policy on the Domain Contorller will need to be modified instead of the local machine):

1. Start -> Run -> regedit.exe -> OK
2. Within the Registry Editor, locate the following Key
HKEY_LOCAL_MACHINE\SOFTWARE\Policies\Microsoft\Cryptography\Configuration\SSL\00010002

3. Right-click the key and select "Export"
4. Select the Desktop and type in an a filename to export the registry key to.
5. Right-click the file on the desktop and select “Edit”
6. The contents of the file should look like this if you have not set the cipher suite order yet:

Windows Registry Editor Version 5.00

[HKEY_LOCAL_MACHINE\SOFTWARE\Policies\Microsoft\Cryptography\Configuration\SSL\00010002]

7. If you have set the cipher suite order before then there may be a value for the key that looks similar to this:

Windows Registry Editor Version 5.00

[HKEY_LOCAL_MACHINE\SOFTWARE\Policies\Microsoft\Cryptography\Configuration\SSL\00010002]
"Functions"="TLS_RSA_WITH_AES_128_CBC_SHA,TLS_RSA_WITH_AES_256_CBC_SHA,TLS_RSA_WITH_RC4_128_SHA,TLS_RSA_WITH_3DES_EDE_CBC_SHA,TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA_P256,TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA_P384,TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA_P521,TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA_P256,TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA_P384,TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA_P521,TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA_P256,TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA_P384,TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA_P521,TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA_P256,TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA_P384,TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA_P521,TLS_DHE_DSS_WITH_AES_128_CBC_SHA,TLS_DHE_DSS_WITH_AES_256_CBC_SHA,TLS_DHE_DSS_WITH_3DES_EDE_CBC_SHA,TLS_RSA_WITH_RC4_128_MD5,SSL_CK_RC4_128_WITH_MD5,SSL_CK_DES_192_EDE3_CBC_WITH_MD5,TLS_RSA_WITH_NULL_MD5,TLS_RSA_WITH_NULL_SHA"

(Note there may be some truncation at the end because Group Policy Editor has a limit of 1023 characters) Thanks to Alex Lewis for pointing this out.

8. The goal is to move TLS_RSA_WITH_RC4_128_MD5 to be at the front of the list. So, in your exported registry key file, find TLS_RSA_WITH_RC4_128_MD5, cut it, navigate to the beginning of your long SSL cipher string, and paste TLS_RSA_WITH_RC4_128_MD5.. (Note: if your string was truncated before you might want to use the example below instead) The new order should look like the following:

Windows Registry Editor Version 5.00

[HKEY_LOCAL_MACHINE\SOFTWARE\Policies\Microsoft\Cryptography\Configuration\SSL\00010002]
"Functions"="TLS_RSA_WITH_RC4_128_MD5,TLS_RSA_WITH_AES_128_CBC_SHA,TLS_RSA_WITH_AES_256_CBC_SHA,TLS_RSA_WITH_RC4_128_SHA,TLS_RSA_WITH_3DES_EDE_CBC_SHA,TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA_P256,TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA_P384,TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA_P521,TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA_P256,TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA_P384,TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA_P521,TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA_P256,TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA_P384,TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA_P521,TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA_P256,TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA_P384,TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA_P521,TLS_DHE_DSS_WITH_AES_128_CBC_SHA,TLS_DHE_DSS_WITH_AES_256_CBC_SHA,TLS_DHE_DSS_WITH_3DES_EDE_CBC_SHA,SSL_CK_RC4_128_WITH_MD5,SSL_CK_DES_192_EDE3_CBC_WITH_MD5,TLS_RSA_WITH_NULL_MD5,TLS_RSA_WITH_NULL_SHA"

9. Check the string for accuracy and make sure there are no line breaks within the SSL cipher string.
10. Save the file and then double-click the file to merge the changes with your current Registry.
11. To verify, in the Registry Editor, navigate to
[HKEY_LOCAL_MACHINE\SOFTWARE\Policies\Microsoft\Cryptography\Configuration\SSL\00010002]
12. For the changes to take effect, restart your Windows Server.

Another way to do this is also via PowerShell take a look at this site for an example (Thanks to John A Cook for sending this along):

http://derek858.blogspot.com/2010/06/powershell-command-to-change-windows.html