Make HTTPS work when using proxy

[timotaularson]

This works by passing the Proxy-Authorization header to the proxy
and stripping it out of the headers sent on to the server.
Fixes #361

[mdsolabs]

Hi,
I still can't make it work... With my proxy that doesn't require authentication

julia 1.1.1
HTTP.jl#master

proxy with http works fine :
julia> HTTP.request("GET","http://httpbin.org/uuid",proxy="http://proxy:8080")
HTTP.Messages.Response:
"""
HTTP/1.1 200 OK
Access-Control-Allow-Credentials: true
Access-Control-Allow-Origin: *
Content-Type: application/json
Date: Mon, 20 May 2019 15:24:29 GMT
Referrer-Policy: no-referrer-when-downgrade
Server: nginx
X-Content-Type-Options: nosniff
X-Frame-Options: DENY
X-XSS-Protection: 1; mode=block
Content-Length: 53
Proxy-Connection: Keep-Alive
Connection: Keep-Alive
Age: 0

{
"uuid": "44107841-444a-4850-8130-addc4ab28b6f"
}
"""

But, with https it doesn't :
julia> HTTP.request("GET","https://httpbin.org/uuid",proxy="http://proxy:8080")
ERROR: HTTP.ExceptionRequest.StatusError(400, HTTP.Messages.Response:
"""
HTTP/1.1 400 Bad Request
Cache-Control: no-cache
Pragma: no-cache
Content-Type: text/html; charset=utf-8
Proxy-Connection: close
Connection: close
Content-Length: 755

""")
Stacktrace:
[1] #request#1 at /Users/mdsolabs/.julia/packages/HTTP/6kBsP/src/ExceptionRequest.jl:22 [inlined]
[2] (::getfield(HTTP, Symbol("#kw##request")))(::NamedTuple{(:iofunction, :proxy),Tuple{Nothing,String}}, ::typeof(HTTP.request), ::Type{HTTP.ExceptionRequest.ExceptionLayer{HTTP.ConnectionRequest.ConnectionPoolLayer{HTTP.StreamRequest.StreamLayer}}}, ::HTTP.URIs.URI, ::HTTP.Messages.Request, ::Array{UInt8,1}) at ./none:0
[3] (::getfield(Base, Symbol("###48#49#50")){ExponentialBackOff,getfield(HTTP.RetryRequest, Symbol("##2#3")){Bool,HTTP.Messages.Request},typeof(HTTP.request)})(::Base.Iterators.Pairs{Symbol,Union{Nothing, String},Tuple{Symbol,Symbol},NamedTuple{(:iofunction, :proxy),Tuple{Nothing,String}}}, ::Function, ::Type, ::Vararg{Any,N} where N) at ./error.jl:231
[4] ##48#51 at ./none:0 [inlined]
[5] #request#1 at /Users/mdsolabs/.julia/packages/HTTP/6kBsP/src/RetryRequest.jl:44 [inlined]
[6] #request at ./none:0 [inlined]
[7] #request#1(::VersionNumber, ::String, ::Nothing, ::Nothing, ::Base.Iterators.Pairs{Symbol,String,Tuple{Symbol},NamedTuple{(:proxy,),Tuple{String}}}, ::Function, ::Type{HTTP.MessageRequest.MessageLayer{HTTP.RetryRequest.RetryLayer{HTTP.ExceptionRequest.ExceptionLayer{HTTP.ConnectionRequest.ConnectionPoolLayer{HTTP.StreamRequest.StreamLayer}}}}}, ::String, ::HTTP.URIs.URI, ::Array{Pair{SubString{String},SubString{String}},1}, ::Array{UInt8,1}) at /Users/mdsolabs/.julia/packages/HTTP/6kBsP/src/MessageRequest.jl:47
[8] (::getfield(HTTP, Symbol("#kw##request")))(::NamedTuple{(:proxy,),Tuple{String}}, ::typeof(HTTP.request), ::Type{HTTP.MessageRequest.MessageLayer{HTTP.RetryRequest.RetryLayer{HTTP.ExceptionRequest.ExceptionLayer{HTTP.ConnectionRequest.ConnectionPoolLayer{HTTP.StreamRequest.StreamLayer}}}}}, ::String, ::HTTP.URIs.URI, ::Array{Pair{SubString{String},SubString{String}},1}, ::Array{UInt8,1}) at ./none:0
[9] #request#1(::Int64, ::Bool, ::Base.Iterators.Pairs{Symbol,String,Tuple{Symbol},NamedTuple{(:proxy,),Tuple{String}}}, ::Function, ::Type{HTTP.RedirectRequest.RedirectLayer{HTTP.MessageRequest.MessageLayer{HTTP.RetryRequest.RetryLayer{HTTP.ExceptionRequest.ExceptionLayer{HTTP.ConnectionRequest.ConnectionPoolLayer{HTTP.StreamRequest.StreamLayer}}}}}}, ::String, ::HTTP.URIs.URI, ::Array{Pair{SubString{String},SubString{String}},1}, ::Array{UInt8,1}) at /Users/mdsolabs/.julia/packages/HTTP/6kBsP/src/RedirectRequest.jl:24
[10] (::getfield(HTTP, Symbol("#kw##request")))(::NamedTuple{(:proxy,),Tuple{String}}, ::typeof(HTTP.request), ::Type{HTTP.RedirectRequest.RedirectLayer{HTTP.MessageRequest.MessageLayer{HTTP.RetryRequest.RetryLayer{HTTP.ExceptionRequest.ExceptionLayer{HTTP.ConnectionRequest.ConnectionPoolLayer{HTTP.StreamRequest.StreamLayer}}}}}}, ::String, ::HTTP.URIs.URI, ::Array{Pair{SubString{String},SubString{String}},1}, ::Array{UInt8,1}) at ./none:0
[11] #request#5(::Base.Iterators.Pairs{Symbol,String,Tuple{Symbol},NamedTuple{(:proxy,),Tuple{String}}}, ::Function, ::String, ::HTTP.URIs.URI, ::Array{Pair{SubString{String},SubString{String}},1}, ::Array{UInt8,1}) at /Users/mdsolabs/.julia/packages/HTTP/6kBsP/src/HTTP.jl:300
[12] #request at ./none:0 [inlined]
[13] #request#6 at /Users/mdsolabs/.julia/packages/HTTP/6kBsP/src/HTTP.jl:314 [inlined]
[14] #request at ./none:0 [inlined] (repeats 2 times)
[15] top-level scope at none:0

Any idea ?

[timotaularson]

I setup an unauthenticated squid proxy and used 64bit Julia 1.1.1 on 64bit Ubuntu 18.04.2 LTS Linux with HTTP.jl#master and used your two example code snippets:
HTTP.request("GET","http://httpbin.org/uuid",proxy="http://proxy:8080")
HTTP.request("GET","https://httpbin.org/uuid",proxy="http://proxy:8080")
[with the "proxy" argument modified to point at my local proxy IP address and port] and both worked.
I could not reproduce the error you are getting.
If you configure your web browser to use your proxy can it access the https url from your example?
Are there perhaps any more clues in your proxy's log files?

[mdsolabs]

Well, I think I just found how to make it work... But it is a workaround...
First of all, I don't have access to the proxy's log files.
But, with the help of Wireshark, I found something interesting...

With this call : HTTP.request("GET","https://httpbin.org/uuid",proxy="http://proxy:8080")
Wireshark tells me this :

CONNECT httpbin.org: HTTP/1.1

HTTP/1.1 400 Bad Request
Cache-Control: no-cache
Pragma: no-cache
Content-Type: text/html; charset=utf-8
Proxy-Connection: close
Connection: close
Content-Length: 755

<HTML><HEAD>
<TITLE>Request Error</TITLE>
</HEAD>
<BODY>
<FONT face="Helvetica">
<big><strong></strong></big><BR>
</FONT>
<blockquote>
<TABLE border=0 cellPadding=1 width="80%">
<TR><TD>
<FONT face="Helvetica">
<big>Request Error (invalid_request)</big>
<BR>
<BR>
</FONT>
</TD></TR>
<TR><TD>
<FONT face="Helvetica">
Your request could not be processed. Request could not be handled
</FONT>
</TD></TR>
<TR><TD>
<FONT face="Helvetica">
This could be caused by a misconfiguration, or possibly a malformed request.
</FONT>
</TD></TR>
<TR><TD>
<FONT face="Helvetica" SIZE=2>
<BR>
</FONT>
</TD></TR>
</TABLE>
</blockquote>
</FONT>
</BODY></HTML>

Do you notice the target in the CONNECT request ? No port after the :

So I tried this : HTTP.request("GET","https://httpbin.org:443/uuid",proxy="http://proxy:8080")

And this works ! The port is now there

CONNECT httpbin.org:443 HTTP/1.1

HTTP/1.1 200 Connection established

Hope it helps...

For now I am going to use the workaround I found...