Client Certificate Validation in .NET core is too restrictive

[systems2000]

Description

For a few days we have struggled with calling some SOAP and REST api's with client certificate authentication.
Although we've set HttpClientHandler's ClientCertificates property, the server still rejected our requests with 403.
It soon turned out that the client certificates, although set are not send. We have found a workaround with
using SocketsHttpHandler (easy when using HttpClientHandler directly, not so easy with WCF Stack).
However - as it bothered me for a while why the certs are not sent I have checked .NET Core code, and I've found the
code responsible for client certificates validation:

The IsValidClientCertificate method checks the cert's Extensions checking whether there is an X509EnhancedKeyUsageExtension
(by the way - the name of the property is wrong, it should be ExtendedKeyUsageExtension) defined as Client Authentication.

If the extension is X509KeyUsageExtension (not Enhanced) it checks whethever it is valid for a digital signature - checking X509KeyUsageFlags for DigitalSignature flag.
For the client certificates it is required to have the extensions described above.
However, neither the RFC 5280 nor TLS v1.2 restricts the client certificate to have only these extensions and allows others when there is a strong justification. For our customer's it looks like they have one.
I our case the first extension usage our client certificate has is the "Key Encipherment" (20).
With this in place the first iteration of the .Extensions loop in IsValidClientCertificates method checks this extension (ku) for digital signature and returns false.
This causes the validation to return false and the client certificate is not sent.
We have checked how other technologies and tools behave in this situation - PostMan, curl, Java & ... .Net Framework 4.6 & 4.8.
They all send the certificate without any warning or complain - so I believe, .NET core also should.
I'll be glad to prepare Pull Request to resolve the issue.

Reproduction Steps

1. Generate client certificate with following key usage extensions (order matters)
   a) Key Encipherment
   b) Digital Signature
   c) Client Authentication (extended)
2. Prepare request with HttpClientHandler and set ClientCertificates property.
3. Call API protected with ClientCertificateAuthentication.

Expected behavior

Client certificate is send, API is called and request succedded.

Actual behavior

Client certificate is not send, API is called and error code 403 is returned.

Regression?

It works properly with checked .NET Framework versions (4.6 & 4.8).

Known Workarounds

Use SocketsHttpHandler and override LocalCerticateSelectionCallback to send the client certificate.

Configuration

.NET Core 3.1, .NET Core 5.0, .NET Core 6.0.

Other information

the code responsible for client certificates validation, see description.

Tagging subscribers to this area: @dotnet/ncl
See info in area-owners.md if you want to be subscribed.

Issue Details

---

Description

For a few days we have struggled with calling some SOAP and REST api's with client certificate authentication.
Although we've set HttpClientHandler's ClientCertificates property, the server still rejected our requests with 403.
It soon turned out that the client certificates, although set are not send. We have found a workaround with
using SocketsHttpHandler (easy when using HttpClientHandler directly, not so easy with WCF Stack).
However - as it bothered me for a while why the certs are not sent I have checked .NET Core code, and I've found the
code responsible for client certificates validation:

The IsValidClientCertificate method checks the cert's Extensions checking whether there is an X509EnhancedKeyUsageExtension
(by the way - the name of the property is wrong, it should be ExtendedKeyUsageExtension) defined as Client Authentication.

If the extension is X509KeyUsageExtension (not Enhanced) it checks whethever it is valid for a digital signature - checking X509KeyUsageFlags for DigitalSignature flag.
For the client certificates it is required to have the extensions described above.
However, neither the RFC 5280 nor TLS v1.2 restricts the client certificate to have only these extensions and allows others when there is a strong justification. For our customer's it looks like they have one.
I our case the first extension usage our client certificate has is the "Key Encipherment" (20).
With this in place the first iteration of the .Extensions loop in IsValidClientCertificates method checks this extension (ku) for digital signature and returns false.
This causes the validation to return false and the client certificate is not sent.
We have checked how other technologies and tools behave in this situation - PostMan, curl, Java & ... .Net Framework 4.6 & 4.8.
They all send the certificate without any warning or complain - so I believe, .NET core also should .
I'll be glad to prepare Pull Request to resolve the issue.

Reproduction Steps

1. Generate client certificate with following key usage extensions (order matters)
   a) Key Encipherment
   b) Digital Signature
   c) Client Authentication (extended)
2. Prepare request with HttpClientHandler and set ClientCertificates property.
3. Call API protected with ClientCertificateAuthentication.

Expected behavior

Client certificate is send, API get called and request succedded.

Actual behavior

Client certificate is not send, API get called and error code 403 is returned.

Regression?

It works properly with checked .NET Framework versions (4.6 & 4.8).

Known Workarounds

Use SocketsHttpHandler and override RemoteCertificateValidationCallback class to simple return true, and LocalCerticateSelectionCallback to send the client certificate.

Configuration

.NET Core 3.1, .NET Core 5.0, .NET Core 6.0.

Other information

the code responsible for client certificates validation, see description.

Author:	systems2000
Assignees:	-
Labels:	area-System.Net.Http
Milestone:	-

Tagging subscribers to this area: @dotnet/ncl, @bartonjs, @vcsjones
See info in area-owners.md if you want to be subscribed.

Issue Details

---

Description

For a few days we have struggled with calling some SOAP and REST api's with client certificate authentication.
Although we've set HttpClientHandler's ClientCertificates property, the server still rejected our requests with 403.
It soon turned out that the client certificates, although set are not send. We have found a workaround with
using SocketsHttpHandler (easy when using HttpClientHandler directly, not so easy with WCF Stack).
However - as it bothered me for a while why the certs are not sent I have checked .NET Core code, and I've found the
code responsible for client certificates validation:

The IsValidClientCertificate method checks the cert's Extensions checking whether there is an X509EnhancedKeyUsageExtension
(by the way - the name of the property is wrong, it should be ExtendedKeyUsageExtension) defined as Client Authentication.

If the extension is X509KeyUsageExtension (not Enhanced) it checks whethever it is valid for a digital signature - checking X509KeyUsageFlags for DigitalSignature flag.
For the client certificates it is required to have the extensions described above.
However, neither the RFC 5280 nor TLS v1.2 restricts the client certificate to have only these extensions and allows others when there is a strong justification. For our customer's it looks like they have one.
I our case the first extension usage our client certificate has is the "Key Encipherment" (20).
With this in place the first iteration of the .Extensions loop in IsValidClientCertificates method checks this extension (ku) for digital signature and returns false.
This causes the validation to return false and the client certificate is not sent.
We have checked how other technologies and tools behave in this situation - PostMan, curl, Java & ... .Net Framework 4.6 & 4.8.
They all send the certificate without any warning or complain - so I believe, .NET core also should .
I'll be glad to prepare Pull Request to resolve the issue.

Reproduction Steps

1. Generate client certificate with following key usage extensions (order matters)
   a) Key Encipherment
   b) Digital Signature
   c) Client Authentication (extended)
2. Prepare request with HttpClientHandler and set ClientCertificates property.
3. Call API protected with ClientCertificateAuthentication.

Expected behavior

Client certificate is send, API get called and request succedded.

Actual behavior

Client certificate is not send, API get called and error code 403 is returned.

Regression?

It works properly with checked .NET Framework versions (4.6 & 4.8).

Known Workarounds

Use SocketsHttpHandler and override RemoteCertificateValidationCallback class to simple return true, and LocalCerticateSelectionCallback to send the client certificate.

Configuration

.NET Core 3.1, .NET Core 5.0, .NET Core 6.0.

Other information

the code responsible for client certificates validation, see description.

Author:	systems2000
Assignees:	-
Labels:	area-System.Net.Security, untriaged
Milestone:	-

[wfurt]

Starting with .NET 8.0 you can use SslClientAuthenticationOptions.ClientCertificateContext.
That can additionally give you control over intermediate CA certificates.

[bartonjs]

From RFC 5280, section 4.2 (Certificate Extensions)

A certificate MUST NOT include more
than one instance of a particular extension. For example, a
certificate may contain only one authority key identifier extension
(Section 4.2.1.1).

If the certificate contains two distinct KeyUsage extensions, one of which has only KeyEncipherment, and one of which has only DigitalSignature, the certificate itself is "invalid" (per that clause in 5280), so various platforms treating them differently is more or less fine. Garbage in, garbage out.

While KeyEncipherment without DigitalSignature is valid for an RSA server cert with the legacy RSA key exchange-based ciphersuites, all modern ciphersuites are DigitalSignature as the purpose of the ciphersuite. For client certificates, the only role the certificate has is a) to exchange static Diffie Hellman parameters or b) as an identity certificate, proven by a signature. Since no one does (a), that means it's only for signatures. So now we pop up to RFC 5246 section 7.4.6 and see that all certificate types include "the certificate MUST allow the key to be used for signing...", which ultimately bounces back to RFC 5280 KU's and see that the DigitalSignature bit is required on all client certificates.

So, no, in my opinion .NET (nee Core) isn't being too restrictive. The certificate (if I understand correctly) is poorly formed and needs to be issued by a more standards-compliant CA/process.

[danmoseley]

Is there some way an app could tell that we consider the cert invalid (like some error callback)?

[rzikm]

Is there some way an app could tell that we consider the cert invalid (like some error callback)?

No error callback, but there is EventSource logging which can be used to diagnose such problems and the logs include a reason for rejecting the certificate.

As bartonjs said, .NET is not being too restrictive here. In .NET 8, you should be able to provide a SslStreamCertificateContext to SslOptions on the SocketsHttpHandler which would take precedence over filtering ClientCertifcates collection by the HttpClientHandler. Are you able to test this?

Known Workarounds
Use SocketsHttpHandler and override RemoteCertificateValidationCallback class to simple return true, and LocalCerticateSelectionCallback to send the client certificate.

I don't think you need to set the RemoteCertificateValidationCallback to workaround this, in fact setting overriding this to always return true has security implications.

This issue has been marked needs-author-action and may be missing some important information.

[systems2000]

Hello,
I appreciate bartonjs's remark on RFC - good point, I've missed the sentence.
The only thing that still bothers me is the fact, that in some situations (like ours) you can't do anything to get the certificate in the right form. The certificate is issued by 3-rd party organization (in this case it is governmental one) and you can only take and use it or not use their API at all. Thus, for me, the client certificate validation should be something that is easy to turn off.
When it comes to RemoteCertificateValidationCallback - yes, it is not required, in fact in our case it was related to the fact the server certificate on test-env API was expired (no comment at all ;-) ).
I'm affraid that the SslStreamCertificateContext to SslOptions on SocketsHttpHandler does not bring any help with WCF stack other than the workaround we've implemented with our own EndpointBehavior (using SocketsHttpHandler) which is a bit complicated and not the simple and easy one. Am I missing something here?

[rzikm]

Unfortunately, I am not familiar with WCF stack and how easy it is to configure TLS on the connections. @mconnew Could you please chime in?