[outages] Potential CloudFlare Issue

Jeremy Chadwick jdc at koitsu.org
Mon Oct 1 07:28:54 EDT 2018


Cloudflare is **extremely**... how to put this... "SSL-sensitive".  That
is to say: missing TLS SNI, using an "old" version of OpenSSL, using
specific ciphers or missing other TLS-related fields, will all cause
ambiguous-looking errors emitted from OpenSSL.  There are other
providers who behave similarly, but CF happens to be the most sensitive
(on a technical level) of the bunch.

I am not seeing any issues myself, but Cloudflare has a remarkably
complicated network which is also anycasted, so basically "figuring out"
where a particular problem lies is horribly, horribly complicated.

Here's proof that, at least from my geoloc (see signature), my own site
behind CF is working fine:

$ echo | openssl s_client -servername jdc.koitsu.org -connect jdc.koitsu.org:443
CONNECTED(00000004)
depth=3 C = SE, O = AddTrust AB, OU = AddTrust External TTP Network, CN = AddTrust External CA Root
verify return:1
depth=2 C = GB, ST = Greater Manchester, L = Salford, O = COMODO CA Limited, CN = COMODO ECC Certification Authority
verify return:1
depth=1 C = GB, ST = Greater Manchester, L = Salford, O = COMODO CA Limited, CN = COMODO ECC Domain Validation Secure Server CA 2
verify return:1
depth=0 OU = Domain Control Validated, OU = PositiveSSL Multi-Domain, CN = sni55709.cloudflaressl.com
verify return:1
---
Certificate chain
 0 s:/OU=Domain Control Validated/OU=PositiveSSL Multi-Domain/CN=sni55709.cloudflaressl.com
   i:/C=GB/ST=Greater Manchester/L=Salford/O=COMODO CA Limited/CN=COMODO ECC Domain Validation Secure Server CA 2
 1 s:/C=GB/ST=Greater Manchester/L=Salford/O=COMODO CA Limited/CN=COMODO ECC Domain Validation Secure Server CA 2
   i:/C=GB/ST=Greater Manchester/L=Salford/O=COMODO CA Limited/CN=COMODO ECC Certification Authority
 2 s:/C=GB/ST=Greater Manchester/L=Salford/O=COMODO CA Limited/CN=COMODO ECC Certification Authority
   i:/C=SE/O=AddTrust AB/OU=AddTrust External TTP Network/CN=AddTrust External CA Root
---
Server certificate
-----BEGIN CERTIFICATE-----
MIILdjCCCxygAwIBAgIQFwqmvi/5k4FqJok9S88aHTAKBggqhkjOPQQDAjCBkjEL
MAkGA1UEBhMCR0IxGzAZBgNVBAgTEkdyZWF0ZXIgTWFuY2hlc3RlcjEQMA4GA1UE
BxMHU2FsZm9yZDEaMBgGA1UEChMRQ09NT0RPIENBIExpbWl0ZWQxODA2BgNVBAMT
L0NPTU9ETyBFQ0MgRG9tYWluIFZhbGlkYXRpb24gU2VjdXJlIFNlcnZlciBDQSAy
MB4XDTE4MDkyMzAwMDAwMFoXDTE5MDQwMTIzNTk1OVowazEhMB8GA1UECxMYRG9t
YWluIENvbnRyb2wgVmFsaWRhdGVkMSEwHwYDVQQLExhQb3NpdGl2ZVNTTCBNdWx0
aS1Eb21haW4xIzAhBgNVBAMTGnNuaTU1NzA5LmNsb3VkZmxhcmVzc2wuY29tMFkw
EwYHKoZIzj0CAQYIKoZIzj0DAQcDQgAEpmM7b4kDjwbz/RtqPL7wHxszS/qJ7KAC
jnBm4i9o06mF61O8XbuNtDMIyFPBv1lT1pdcfsuXylxb8tBR3k5GQ6OCCXgwggl0
MB8GA1UdIwQYMBaAFEAJYWfwvINxT94SCCxv1NQrdj2WMB0GA1UdDgQWBBQPi4n4
iyuQH53Xn0Q06YuNXB/zJDAOBgNVHQ8BAf8EBAMCB4AwDAYDVR0TAQH/BAIwADAd
BgNVHSUEFjAUBggrBgEFBQcDAQYIKwYBBQUHAwIwTwYDVR0gBEgwRjA6BgsrBgEE
AbIxAQICBzArMCkGCCsGAQUFBwIBFh1odHRwczovL3NlY3VyZS5jb21vZG8uY29t
L0NQUzAIBgZngQwBAgEwVgYDVR0fBE8wTTBLoEmgR4ZFaHR0cDovL2NybC5jb21v
ZG9jYTQuY29tL0NPTU9ET0VDQ0RvbWFpblZhbGlkYXRpb25TZWN1cmVTZXJ2ZXJD
QTIuY3JsMIGIBggrBgEFBQcBAQR8MHowUQYIKwYBBQUHMAKGRWh0dHA6Ly9jcnQu
Y29tb2RvY2E0LmNvbS9DT01PRE9FQ0NEb21haW5WYWxpZGF0aW9uU2VjdXJlU2Vy
dmVyQ0EyLmNydDAlBggrBgEFBQcwAYYZaHR0cDovL29jc3AuY29tb2RvY2E0LmNv
bTCCBrcGA1UdEQSCBq4wggaqghpzbmk1NTcwOS5jbG91ZGZsYXJlc3NsLmNvbYIT
Ki40YXNlbmlvcnZpZGVvcy5nYYIQKi5hLXJlYWRpY2FyZC5tbIIRKi5hcmNhbXRh
bmtjZW4uY2aCESouYXJjYW10YW5rY2VuLnRrghIqLmF6ZXZpcmlueWdpZ3kudGuC
DiouYmJjcmkuY29tLmF1ghEqLmJsb29kZmxvd2Vycy50a4IWKi5ib2R5YnVpbGRp
bmdwb3JuLm5ldIIZKi5jYXJhbWVsaXplZHRob3VnaHRzLmNvbYISKi5jb21wbGV0
ZWNhcmdvLnJvggwqLmNvcmRhYW4udGuCDiouZWdndGFzdGljLnRrgg8qLmZhaWhp
bXV0dWEubWyCCyouZmFuYXFqLmdhghYqLmZsYXRlYXJ0aHRoZWF0cmUub3JnghAq
LmZ1YmFycmVkLnRvZGF5ggkqLmctc2QudGuCECouZ3ZpZGVvcHJpbnQuY2aCFCou
aW5jcmlzaXNyZWxpZWYub3JnggwqLml0ZmFuYXMubHSCGCoua2luZXRpY3JlbGF4
bWFzc2FnZS5yb4IMKi5rb2l0c3Uub3JnghQqLmx1bHpnYW1laGFja2luZy5jZoIS
Ki5sdW1lYWJhc21lbG9yLmV1ghQqLm1vbmljYWlhY292ZW5jby5yb4IPKi5tcGF5
cmV2aWV3LmNmghYqLm1zbWVkaWNhbHN5c3RlbXMuY29tgg4qLm15d2VibGl2ZS5y
dYIPKi5uLXBzdmJvb2tzLmdhgg8qLm5yZXZpZXdodGMuZ3GCESoucGhnaC1yZXZp
ZXdzLmNmgg8qLnBsaWN5bXVuYXMuY2aCESoucHJvZ3JhbWEyMDE1Lmx0gg4qLnJl
dmlld3JxZi5tbIIYKi5zYWxham9jdXJpYnVjdXJlc3RpLnJvgg8qLnNhcHNpbmsu
Y28udGiCEyouc2tpc2lsdmVyc3Rhci5jb22CEyouc2tpc2lsdmVyc3Rhci5za2mC
DyoudC1mdWNvYm9vay5jZoIRKi50ZXB6Y2Fjb25zbWUuZ3GCCSoudHlyby5yb4IN
Ki51bGtpcmVzZS50a4IYKi51bHRyYWZpbG1lc29ubGluZWhkLmdhghgqLnVsdHJh
ZmlsbWVzb25saW5laGQubWyCEyoudXFpY3VmZWt5amFjeWwudGuCESoud3VtaWdv
cWVjb3R5LnRrggkqLnlyZGYuY2aCETRhc2VuaW9ydmlkZW9zLmdhgg5hLXJlYWRp
Y2FyZC5tbIIPYXJjYW10YW5rY2VuLmNmgg9hcmNhbXRhbmtjZW4udGuCEGF6ZXZp
cmlueWdpZ3kudGuCDGJiY3JpLmNvbS5hdYIPYmxvb2RmbG93ZXJzLnRrghRib2R5
YnVpbGRpbmdwb3JuLm5ldIIXY2FyYW1lbGl6ZWR0aG91Z2h0cy5jb22CEGNvbXBs
ZXRlY2FyZ28ucm+CCmNvcmRhYW4udGuCDGVnZ3Rhc3RpYy50a4INZmFpaGltdXR1
YS5tbIIJZmFuYXFqLmdhghRmbGF0ZWFydGh0aGVhdHJlLm9yZ4IOZnViYXJyZWQu
dG9kYXmCB2ctc2QudGuCDmd2aWRlb3ByaW50LmNmghJpbmNyaXNpc3JlbGllZi5v
cmeCCml0ZmFuYXMubHSCFmtpbmV0aWNyZWxheG1hc3NhZ2Uucm+CCmtvaXRzdS5v
cmeCEmx1bHpnYW1laGFja2luZy5jZoIQbHVtZWFiYXNtZWxvci5ldYISbW9uaWNh
aWFjb3ZlbmNvLnJvgg1tcGF5cmV2aWV3LmNmghRtc21lZGljYWxzeXN0ZW1zLmNv
bYIMbXl3ZWJsaXZlLnJ1gg1uLXBzdmJvb2tzLmdhgg1ucmV2aWV3aHRjLmdxgg9w
aGdoLXJldmlld3MuY2aCDXBsaWN5bXVuYXMuY2aCD3Byb2dyYW1hMjAxNS5sdIIM
cmV2aWV3cnFmLm1sghZzYWxham9jdXJpYnVjdXJlc3RpLnJvgg1zYXBzaW5rLmNv
LnRoghFza2lzaWx2ZXJzdGFyLmNvbYIRc2tpc2lsdmVyc3Rhci5za2mCDXQtZnVj
b2Jvb2suY2aCD3RlcHpjYWNvbnNtZS5ncYIHdHlyby5yb4ILdWxraXJlc2UudGuC
FnVsdHJhZmlsbWVzb25saW5laGQuZ2GCFnVsdHJhZmlsbWVzb25saW5laGQubWyC
EXVxaWN1ZmVreWphY3lsLnRrgg93dW1pZ29xZWNvdHkudGuCB3lyZGYuY2YwggEE
BgorBgEEAdZ5AgQCBIH1BIHyAPAAdQDuS723dc5guuFCaR+r4Z5mow9+X7By2IMA
xHuJeqj9ywAAAWYFgG8dAAAEAwBGMEQCIAZDFkCREYFzcI7H/sxnyNou2WyBIyiu
KAqrp5s/FD8sAiARzq/SbXXv6Go31/v5zyy3MrlnQurwGr8Ccb+nM8mAiwB3AHR+
2oMxrTMQkSGcziVPQnDCv/1eQiAIxjc1eeYQe8xWAAABZgWAb1oAAAQDAEgwRgIh
ANdkJOyQYY+QfRAyhHnFR0HNgMllEFXHajPvVMcGjx3EAiEAtBp9jQerK50pMQv0
fvZXU2qrniVQfhN41LpEDL4XxA4wCgYIKoZIzj0EAwIDSAAwRQIgbcKkuwmnGNNf
N6/OtzIdcQ65qQvuW7eGjj8RgBchfOoCIQDmK3sSAO+DAvR8EY5atA8stKDNjFbl
kCBKydj56tZ+IA==
-----END CERTIFICATE-----
subject=/OU=Domain Control Validated/OU=PositiveSSL Multi-Domain/CN=sni55709.cloudflaressl.com
issuer=/C=GB/ST=Greater Manchester/L=Salford/O=COMODO CA Limited/CN=COMODO ECC Domain Validation Secure Server CA 2
---
No client certificate CA names sent
Peer signing digest: SHA256
Server Temp Key: ECDH, P-256, 256 bits
---
SSL handshake has read 5337 bytes and written 456 bytes
---
New, TLSv1/SSLv3, Cipher is ECDHE-ECDSA-AES128-GCM-SHA256
Server public key is 256 bit
Secure Renegotiation IS supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
SSL-Session:
    Protocol  : TLSv1.2
    Cipher    : ECDHE-ECDSA-AES128-GCM-SHA256
    Session-ID: D1BC2EDA60A0E06AA7AFFEF76A4FDEC65127A5968F43D4E8666C0B81D2AE36B0
    Session-ID-ctx:
    Master-Key: BA6BADA117F9803CD2C4BCE9915D0D8A1365B959F867F85ABDEE127F038D1795ACE1273DAA3DEE414B3EB716E1833AED
    Key-Arg   : None
    PSK identity: None
    PSK identity hint: None
    SRP username: None
    TLS session ticket lifetime hint: 64800 (seconds)
    TLS session ticket:
    0000 - 82 39 99 f6 9d cc 98 53-75 ec d7 ea 6e 4f 7f 2c   .9.....Su...nO.,
    0010 - 03 6a e0 e7 ba af dc c7-20 97 6f e1 41 a6 73 55   .j...... .o.A.sU
    0020 - 20 1d 4e 81 7d 95 8e 74-50 5b fd 29 ca ed 6e a4    .N.}..tP[.)..n.
    0030 - 54 79 2b 9e 72 51 19 00-58 96 9f c6 0b 78 22 6a   Ty+.rQ..X....x"j
    0040 - df 1e 1a a5 ee 38 17 39-be f4 bd ae 59 7c 0c 8e   .....8.9....Y|..
    0050 - b0 c7 41 02 1a af 1c dd-bf a0 b1 09 b5 ff 23 84   ..A...........#.
    0060 - f9 2a cf 19 a3 4b ac 82-2d b3 ba 23 a9 e7 25 8c   .*...K..-..#..%.
    0070 - ba a9 7d f0 8e 6e 81 3a-f3 bc 4c 76 2e 18 b6 69   ..}..n.:..Lv...i
    0080 - 09 ec 47 61 c2 71 96 f7-07 ed 06 8e e3 50 41 ea   ..Ga.q.......PA.
    0090 - d7 10 54 d3 3a 5d 24 6f-d6 5b 16 b4 af d3 e0 48   ..T.:]$o.[.....H
    00a0 - cd 47 fa 55 d8 2d ec 52-7c 42 c4 87 0c 9a a5 94   .G.U.-.R|B......

    Start Time: 1538393205
    Timeout   : 300 (sec)
    Verify return code: 0 (ok)
---
DONE

-- 
| Jeremy Chadwick                                 jdc at koitsu.org |
| UNIX Systems Administrator                      PGP 0x2A389531 |
| Making life hard for others since 1977.                        |

On Mon, Oct 01, 2018 at 06:40:16PM +0800, Chris via Outages wrote:
> Hi all,
> 
> Not sure if anyone else is experiencing this currently, but I have a report
> from multiple sites (in different countries) that are having issues
> accessing various CloudFlare hosted sites (the CloudFlare website itself is
> working fine though). The issue is only with HTTPS, HTTP seems to be working
> fine. Issue appears to be happening for both v4 and v6.
> 
> Eg.
> 
> me at jumpoff1 ~ $ openssl s_client -connect 104.24.114.156:443
> CONNECTED(00000003)
> 140186033568600:error:14077410:SSL routines:SSL23_GET_SERVER_HELLO:sslv3
> alert handshake failure:s23_clnt.c:802:
> ---
> no peer certificate available
> ---
> No client certificate CA names sent
> ---
> SSL handshake has read 7 bytes and written 258 bytes
> ---
> New, (NONE), Cipher is (NONE)
> Secure Renegotiation IS NOT supported
> Compression: NONE
> Expansion: NONE
> No ALPN negotiated
> SSL-Session:
>     Protocol  : TLSv1
>     Cipher    : 0000
>     Session-ID:
>     Session-ID-ctx:
>     Master-Key:
>     Key-Arg   : None
>     PSK identity: None
>     PSK identity hint: None
>     SRP username: None
>     Start Time: 1538388672
>     Timeout   : 300 (sec)
>     Verify return code: 0 (ok)
> ---
> 
> I can reproduce the issue on web based proxy services as well, eg.
> https://hidester.com/proxy/ (US Server), https://www.vpnbook.com/webproxy
> (US Server).
> 
> Not sure how widespread this is as some people can access things, curious to
> know what others see.
> _______________________________________________
> Outages mailing list
> Outages at outages.org
> https://puck.nether.net/mailman/listinfo/outages



More information about the Outages mailing list