After downloading a new certificate from our SSL provider, I found I was unable to install this new certificate.
Windows returned the error “Failed to Remove Certificate”.
Using OpenSSL on a Linux box, I did the following:
openssl pkcs12 -export -in my.crt -inkey my.key -certfile my.bundle -out my.pfx
I was then able to import my new certificate and install it.