[Outages-discussion] [EXTERNAL] Re: Question about the Google “sorry” page...

[Damian Menscher]

Two cautions regarding proxies:
  - as mentioned before, make sure you don't have an open proxy, which
might be abused
  - if you're proxying only some traffic (eg, for content filtering, etc),
then be sure all Google traffic gets proxied out the same IP.  We sometimes
see weirdness when some requests go through the proxy, but other requests
go directly from the (home) IP.  This can cause problems, for example the
captcha exemption may fail due to the IP mis-match.

Damian

On Sun, Jun 28, 2020 at 9:55 PM Chapman, Brad (NBCUniversal) <
Brad.Chapman at nbcuni.com> wrote:

> Interesting; thanks.
>
> Would you expect to see this behavior in an environment where a proxy
> server is used to funnel traffic to the Internet and clients have to use a
> PAC file or WPAD?
>
> —Sent from my iPhone
>
> On Jun 28, 2020, at 9:34 PM, Damian Menscher <damian at google.com> wrote:
>
> 
> Blocking occurs when automated searching is detected, not simply due to
> the total volume of requests from a single IP.  As such, there is no option
> for an exception.
>
> To "solve" this, we recommend you minimize the number of users sharing an
> IP.  The easiest method is with IPv6, since then each user can have their
> own /64 (our abuse systems don't look deeper than that).  If you're stuck
> with IPv4, separate your corporate-managed machines from the guest wifi
> (which is harder to control), and try to give different groups of users
> their own NAT IP (by building or floor, etc).  That way when there's a
> problem you'll have fewer users impacted, and a smaller list of suspects.
>
> If you want to start digging into the reasons why your IP might have been
> blocked, the most common reasons for getting blocked (mostly for websearch)
> include (in no particular order):
>   - malware that proxies abuse for criminals
>   - browser extensions that automate searching
>   - misconfigured browsers that have anomalous behavior
>   - corporate proxies that are open for abuse
>   - users installing "P2P VPN" software, which is also abused
>
> Damian
> --
> Damian Menscher :: Security Reliability Engineer :: Google :: AS15169
>
> On Sun, Jun 28, 2020 at 4:57 PM Chapman, Brad (NBCUniversal) <
> Brad.Chapman at nbcuni.com> wrote:
>
>> Greetings Outages-Discussion,
>>
>> I hope you are all having a pleasant Sunday afternoon / evening with no
>> P1 / SevA / 4-alarm fires caused by a violation of Read-only Friday.
>>
>> Given the number of sysadmins and telecom / network engineers on this
>> list, I am guessing that we have seen (or been asked to explain) the Google
>> “Sorry” page.
>>
>> Occasionally, our company gets a burst of calls about this issue, until
>> the lockout expires on Google’s side.   We manage >50,000 computers so even
>> short lockouts can generate dozens of calls.
>>
>> Has anyone ever approached Google’s NOC team to request an exemption from
>> the Sorry page for their busiest external IP addresses? Or, if not a
>> blanket exemption, to request an increase in the threshold before it is
>> tripped?
>>
>> Hope you’re all staying safe.
>>
>> Cheers,
>> Brad Chapman
>> NBCUniversal
>>
>> —Sent from my iPhone
>> _______________________________________________
>> Outages-discussion mailing list
>> Outages-discussion at outages.org
>> https://puck.nether.net/mailman/listinfo/outages-discussion
>> <https://urldefense.com/v3/__https://puck.nether.net/mailman/listinfo/outages-discussion__;!!PIZeeW5wscynRQ!-T5SokgIYLbWPeqRO4boP4fHxQbHaOHVW5G6FNDQ4sI2cVgFNtCDeAvOwaP5eN4PNg$>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://puck.nether.net/pipermail/outages-discussion/attachments/20200628/e388865e/attachment.htm>