[outages] Meta: outages outage

[Jared Mauch]

> On Oct 5, 2021, at 5:34 AM, Jürgen Botz via Outages <outages at outages.org> wrote:
> 
> OK, aside from the Facebook outage, and no doubt because of it, we effectively had an outages outage... messages to the lsit were massively
> delayed, causing lots of duplicate reports and general failure to have
> any meaningful confirmations, etc.
> 
> Why?  What happened?  Well, outages is hosted on a server
> (puck.nether.net) with a couple dozen other lists, and some of these lists probably have members with email addresses at facebook domains,
> and the SMTP server kept trying to look those up and having wait for
> timeouts?  Something like that, except that since there were no routes
> to facebook's nameservers there shouldn't have been any need to wait
> for timeouts.  Anyone here form nether.net actually take a look?
> 
> It's more than idle curiosity, I'd like make my mail servers more
> resilient to this kind of situation, and the outages list of all
> probably should be, too.


Sure, I’ve gone and tweaked a few more things, there’s a balancing act here between having a single system send out lots of concurrent mail to a server, eg:

60756540E50*   22760 Tue Oct  5 01:28:00  outages-bounces at outages.org
(host itchy.cerento.com[199.190.154.20] said: 451 Only one recipient at a time (in reply to RCPT TO command))
                                         REDACTED at cerento.com

Some systems are explicitly configured to not be optimized (see above, then imagine how many people in parallel @gmail might get a message)

And there are many people with old subscriptions or domains that don’t exist anymore which gets to be exciting when everything is going on at once:

          (connect to mail.bestii.com[172.96.180.81]:25: Connection timed out)
                                         redacted at bestii.com
     (connect to canadawebhosting.com[74.201.58.138]:25: Connection timed out)
                                         redacted at canadawebhosting.com
(connect to muhpanel.ironusmaidenus.com[51.15.246.204]:25: Connection refused)
                                         redacted at ironusmaidenus.com
        (connect to mail.mailhost4.com[198.50.245.163]:25: Connection refused)
                                         redacted at mailhost4.com
           (connect to mail.etsms.com[50.232.238.69]:25: Connection timed out)
                                         redacted at etsms.com
          (connect to keyedupmedia.com[204.11.56.48]:25: Connection timed out)
                                         redacted at keyedupmedia.com
              (connect to metacloud.com[72.52.10.14]:25: Connection timed out)
                                         redacted at metacloud.com
 (connect to smtp.naturalwireless.com[52.119.91.194]:25: Connection timed out)
                                         redacted at naturalwireless.com
                                         redacted at naturalwireless.com
(host s1.mail.pciwest.net[2604:2400:a::425] said: 450 4.7.1 Bad Attachment .com (in reply to end of DATA command))
                                         redacted at presys.com
(delivery temporarily suspended: connect to mail.talueee.com[159.69.230.243]:25: No route to host)
                                         redacted at talueee.com
(connect to mail-1.meridian-enviro.com[198.58.69.47]:25: Connection timed out)
                                         redacted at meridian-enviro.com
                                                          (reason unavailable)

(All of these are from the same message btw)

I was away for personal reasons yesterday so missed out on all the fun.  I’ll check how this message goes out and see if the tweaks helped.

- Jared