Solving "m_safeCertContext is an invalid handle."

I've recently been trying to get an application working in Azure App Service that acts as a client who calls out to another service via WCF with TransportWithMessageCredential mode for security and Certificate mode for authentication. I've been getting the following error:

m_safeCertContext is an invalid handle.

According to this blog post, this error gets thrown when the certificate isn't correctly imported or has incorrect trust (for any of many possible reasons). Some of those reasons can include incorrect passwords, but there are others as well, like what I was encountering: in Azure App Service, there's no local user signed on when your application is running. Because of that, you run afoul of a subtle issue with managing certificates: all of the constructors, by default, use the user certificate store to temporarily store the PrivateKey of any loaded X509Certificate2 objects. Therefore, on an Azure App Service application, unless you use the new X509Certificate2(certBytes, passwordString, X509KeyStorageFlags.MachineKeySet | X509KeyStorageFlags.Exportable) constructor with the MachineKeySet | Exportable flags, your certificate will not be correctly read and will SILENTLY FAIL!!

Converting a .cer and .pvk to a .pfx file

The Windows SDK comes with a built-in tool for performing this conversion, the details of which you can find here. The command you'll need looks something like the following :

pvk2pfx /pvk MyCert.pvk /pi inputpassword /pfx MyCert.pfx /po outputpassword /spc MyCert.cer

Journey to robust webservices: kickstarting use of certificates phase II: using certificates for client authentication

There are a number of means of authenticating users, and one of the most secure is via a certificate. This article on CodeProject provides a start. Unfortunately, the article doesn't really mention a few things:

1. Depending on how you ran the tools, you may have inadvertently run the commands it gives you as an administrator. I ran them in a console that I already had open that was running as administrator, so they were installed as administrator, and I also happened to have been running in Visual Studio as an administrator at the time, so everything happily worked. Then when I rebooted my machine this morning and was running as a normal user, nothing worked, and I wasted over an hour trying to figure out why. That was why. Regardless of how you generated and installed the keys, you should go and explicitly grant permissions on the key to the users you want to have access to them. On the Microsoft website, there is a bundle of WCF samples (which you can find here). Included in the samples is a tool called FindPrivateKey. Download the samples, compile the program, and use it to find the key that you just generated. You'll need a command similar to the following : "C:\Samples\WCFWFCardSpace\WCF\Tools\FindPrivateKey\CS\bin\FindPrivateKey.exe My LocalMachine -t "28 ce e3 2c 7e 05 3a 97 a0 b4 92 fd d5 b0 f9 de 0e 4c 2e 4b"" where the value in quotes is the thumbprint of the non-signing (client) key you generated in the instructions from the article. Once it spits out the location of the file, you'll need to go and alter its permissions to allow whatever user your server / client application is running under access to the file.
2. Depending on the binding you're using with WCF on the server side, you may need to have a certificate with full chain trust on the IIS server (or other?) in order to use the binding (*cough* basicHttpBinding and any other transport-only security bindings *cough*). With that in mind, after you've generated the certificates in the article, while you're in the certificate manager MMC snap-in, you'll also need to copy (not move) the IIS developer certificate used in IIS to the "Trusted Root Certification Authorities" store.