[rbak-nsp] SEOS 6.4.1.1 and BGP problem (also 6.3.x.y series)
Marcin Kuczera
marcin at leon.pl
Mon Oct 18 14:41:38 EDT 2010
Jeff Tantsura wrote:
> Hi Marcin ,
>
> Yes, it is, as per RFC4271 Aggregator should be set to the BGP
> Identifier of the speaker (router-id)
>
> "The IP address SHOULD be the same as the BGP Identifier of the speaker"
>
> So it is not a bug, if a BGP optional transitive attribute is recognized
> as such BGP implementation must check the content of it
Well, SHOULD is not equal to MUST.
If there was a MUST statement, than I would agree on checking this.
But I agree that this is not a bug, rather a nasty feature ;)
> On another side - take a look at those prefixes elsewhere - they are
> being aggregated by local AS with a valid IP address as Aggregator, your
> AS is the only (as far as I can check)AS having
> Aggregator = 0.0.0.0
>
> Some other ASs:
>
> aggregated by 24863 172.18.1.6
>
> AS path: 9002 15412 24863 ? Aggregator: 24863 172.18.1.6
>
>
> Could you please check with your upstreams - who's sending you such an
> update and why?
Look at the history of my messages about that problem.
I have attached errorous messages, they can be decoded and you can find
out updatates that contain aggregator = 0.0.0.0
I have received them from 2 full feed upstreams !
Sometimes some falty updated are gone and other appear...
I know that this should not be 0.0.0.0, but as other implementations do
not work properly, sometimes it happens...
> I'd be more than willing to help you (and anybody else affected) out if
> it is critical to your business
It is for all of us using full feed BGP.. You never know the moment when
one update message will drop down whole BGP session, I you get it from
other upstreams - you are blackouted.
So that's why it is so important.
For future - this sanity check may be usefull.
What redback could do is:
- rollback this feature
- add a command that will:
-- disable this feature
-- drop just this update not affecting whole session (and send info to log)
> Please work with Andrzej on it.
> As a remark - there's absolutely no reason not to test/deploy 6.4 - the
> quality is getting better with every release!
I really needed 6.4.x.y due to RouteServer feature (if peer AS is not
present as first in path).. but all full feed sessions were down..
> I'd really like to know if anyone else has a problem with this specific
> sanity check.
at the moment, you could get an access to our router upgraded to 6.4.1.1
to see what happens (one upstream will be asking as to shutdown session
due to alarms ;)
Within 2 weeks this "test" device will run under production with 6.2.1.4
Regards,
Marcin
> Please unicast me jefftant at gmail.com <mailto:jefftant at gmail.com> for
> further discussion.
>
>
> Cheers,
>
> Jeff
>
>
>
> On Wed, Oct 13, 2010 at 6:48 AM, Marcin Kuczera <marcin at leon.pl
> <mailto:marcin at leon.pl>> wrote:
>
>
> Hello,
>
> As I found, the most probable cause is aggregator = 0.0.0.0
>
> [bgp1]R1_SE100#show bgp route 82.116.64.0/19 <http://82.116.64.0/19>
> BGP ipv4 unicast routing table entry: 82.116.64.0/19
> <http://82.116.64.0/19>, version 1836308
> Paths: total 2, best path count 1, best peer 193.111.38.97
> Advertised to non peer-group peers: 2
> 91.195.159.30 91.195.159.138
>
> 15744 13293 3356 2116 39197
> Nexthop 212.106.159.142 (0), peer 212.106.159.142 (83.230.95.254), AS
> 15744
> Origin IGP, localpref 200, med 0, weight 100, external
> aggregator: 0.0.0.0, AS 39197
> Community: 13000:9001
>
>
> 24724 6453 2116 39197
> Nexthop 193.111.38.97 (0), peer 193.111.38.97 (193.111.37.1), AS
> 24724
> Origin IGP, localpref 210, med 0, weight 100, external, best
> aggregator: 0.0.0.0, AS 39197
>
> [bgp1]R1_SE100#
>
> (this update i.e. drops down all external sessions on other SEOS
> 6.4.1.1)
>
>
> More HEX dump below, this seems to be a very, very serious problem.
> Same problem with 6.3.x.y. series..
>
> Regards,
> Marcin
>
>
> [bgp1]R2_SE100#show bgp neighbor 212.106.159.142 malform update
> Dump logged malformed UPDATE messages for ??? (100 total entries):
> Oct 13 14:51:47 Malformed UPDATE msg (nbr 212.106.159.142, context
> 0x40080002, 97 bytes, repeated 0 times, reason: Invalid msg) -
> ffff ffff ffff ffff ffff ffff ffff ffff 0061 0200 0000 4240 0101 0040
> 021a 0206 0000 3d80 0000 33ed 0000 0d1c 0000 0ddd 0000 0844 0000 991d 4
> 003 04d4 6a9f 8ec0 0708 0000 991d 0000 0000 c008 0c21 3c03 e833 ed04
> 6033 ed13 9215 5274 4014 57fc 40
> Oct 13 14:52:24 Malformed UPDATE msg (nbr 193.111.38.25, context
> 0x40080002, 78 bytes, repeated 1 times, reason: Invalid msg) -
> ffff ffff ffff ffff ffff ffff ffff ffff 004e 0200 0000 2f40 0101 0040
> 0216 0205 0000 6094 0000 1935 0000 1d31 0000 1d32 0000 4445 4003 04c1 6
> f26 19c0 0708 0000 4445 0000 0000 187d 070c 187d 0705
> Oct 13 14:52:26 Malformed UPDATE msg (nbr 212.106.159.142, context
> 0x40080002, 97 bytes, repeated 0 times, reason: Invalid msg) -
> ffff ffff ffff ffff ffff ffff ffff ffff 0061 0200 0000 4240 0101 0040
> 021a 0206 0000 3d80 0000 33ed 0000 0d1c 0000 0ddd 0000 0844 0000 991d 4
> 003 04d4 6a9f 8ec0 0708 0000 991d 0000 0000 c008 0c21 3c03 e833 ed04
> 6033 ed13 9215 5274 4014 57fc 40
> Oct 13 14:52:36 Malformed UPDATE msg (nbr 195.66.73.253, context
> 0x40080002, 89 bytes, repeated 0 times, reason: Invalid msg) -
> ffff ffff ffff ffff ffff ffff ffff ffff 0059 0200 0000 3a40 0101 00c0
> 0708 0000 991d 0000 0000 5002 0012 0204 0000 6094 0000 1935 0000 0844 0
> 000 991d 4003 04c1 6f26 6180 0404 0000 0000 4005 0400 0000 d213 5274
> 4015 5274 40
> Oct 13 14:52:59 Malformed UPDATE msg (nbr 193.111.38.25, context
> 0x40080002, 78 bytes, repeated 0 times, reason: Invalid msg) -
> ffff ffff ffff ffff ffff ffff ffff ffff 004e 0200 0000 2f40 0101 0040
> 0216 0205 0000 6094 0000 1935 0000 1d31 0000 1d32 0000 4445 4003 04c1 6
> f26 19c0 0708 0000 4445 0000 0000 187d 070c 187d 0705
> Oct 13 14:53:03 Malformed UPDATE msg (nbr 212.106.159.142, context
> 0x40080002, 97 bytes, repeated 0 times, reason: Invalid msg) -
> ffff ffff ffff ffff ffff ffff ffff ffff 0061 0200 0000 4240 0101 0040
> 021a 0206 0000 3d80 0000 33ed 0000 0d1c 0000 0ddd 0000 0844 0000 991d 4
> 003 04d4 6a9f 8ec0 0708 0000 991d 0000 0000 c008 0c21 3c03 e833 ed04
> 6033 ed13 9215 5274 4014 57fc 40
> Oct 13 14:53:22 Malformed UPDATE msg (nbr 195.66.73.253, context
> 0x40080002, 89 bytes, repeated 0 times, reason: Invalid msg) -
> ffff ffff ffff ffff ffff ffff ffff ffff 0059 0200 0000 3a40 0101 00c0
> 0708 0000 991d 0000 0000 5002 0012 0204 0000 6094 0000 1935 0000 0844 0
> 000 991d 4003 04c1 6f26 6180 0404 0000 0000 4005 0400 0000 d213 5274
> 4015 5274 40
> Oct 13 14:53:35 Malformed UPDATE msg (nbr 193.111.38.25, context
> 0x40080002, 78 bytes, repeated 0 times, reason: Invalid msg) -
> ffff ffff ffff ffff ffff ffff ffff ffff 004e 0200 0000 2f40 0101 0040
> 0216 0205 0000 6094 0000 1935 0000 1d31 0000 1d32 0000 4445 4003 04c1 6
> f26 19c0 0708 0000 4445 0000 0000 187d 070c 187d 0705
> Oct 13 14:53:44 Malformed UPDATE msg (nbr 212.106.159.142, context
> 0x40080002, 97 bytes, repeated 0 times, reason: Invalid msg) -
> ffff ffff ffff ffff ffff ffff ffff ffff 0061 0200 0000 4240 0101 0040
> 021a 0206 0000 3d80 0000 33ed 0000 0d1c 0000 0ddd 0000 0844 0000 991d 4
> 003 04d4 6a9f 8ec0 0708 0000 991d 0000 0000 c008 0c21 3c03 e833 ed04
> 6033 ed13 9215 5274 4014 57fc 40
> Oct 13 14:54:04 Malformed UPDATE msg (nbr 195.66.73.253, context
> 0x40080002, 89 bytes, repeated 0 times, reason: Invalid msg) -
> ffff ffff ffff ffff ffff ffff ffff ffff 0059 0200 0000 3a40 0101 00c0
> 0708 0000 991d 0000 0000 5002 0012 0204 0000 6094 0000 1935 0000 0844 0
> 000 991d 4003 04c1 6f26 6180 0404 0000 0000 4005 0400 0000 d213 5274
> 4015 5274 40
> Oct 13 14:54:15 Malformed UPDATE msg (nbr 193.111.38.25, context
> 0x40080002, 78 bytes, repeated 0 times, reason: Invalid msg) -
> ffff ffff ffff ffff ffff ffff ffff ffff 004e 0200 0000 2f40 0101 0040
> 0216 0205 0000 6094 0000 1935 0000 1d31 0000 1d32 0000 4445 4003 04c1 6
> f26 19c0 0708 0000 4445 0000 0000 187d 070c 187d 0705
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> David Freedman wrote:
>
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> The only other problem I can potentially see are in the communities:
>
>
> c0 080c 0000 3b41 0000 51cc 220a 220a
>
>
> 0000 3b41 = 0:15169
> 0000 51cc = 0:20940
> 220a 220a = 8714:8714
>
> RFC1997 reserves communities 0x0000000 -> 0x0000FFFF , perhaps
> this is
> being checked and rejected?
>
> Dave.
>
>
> Marcin Kuczera wrote:
>
> David Freedman wrote:
> Well, I notice from this output the following:
>
> c0 0708 0000 611f 0000 0000
>
> c0 - optional transitive attribute follows
> 07 - AGGREGATOR
> 08 - of 8 bytes
> 0000611f - from AS 24863
> 00000000 - aggregated by 0.0.0.0
>
> According to RFC4271, sec 5.1.7:
>
> AGGREGATOR is an optional transitive attribute, which MAY
> be included
> in updates that are formed by aggregation (see Section
> 9.2.2.2). A
> BGP speaker that performs route aggregation MAY add the
> AGGREGATOR
> attribute, which SHALL contain its own AS number and IP
> address. The
> IP address SHOULD be the same as the BGP Identifier of the
> speaker.
>
> It is possible that SEOS considers 0.0.0.0 to be an invalid BGP
> identifier and therefore an impossible aggregator
>
> For reference, this update covers the following prefixes:
>
> 41.178.0.0/24 <http://41.178.0.0/24>
> 41.196.4.0/22 <http://41.196.4.0/22>
> 41.196.8.0/22 <http://41.196.8.0/22>
> 41.196.12.0/22 <http://41.196.12.0/22>
> 41.196.16.0/22 <http://41.196.16.0/22>
> 41.196.24.0/22 <http://41.196.24.0/22>
> 41.196.36.0/23 <http://41.196.36.0/23>
>
>
> with AS_PATH 24724 15412 24863
>
> Since none of my SEOS boxes carry a full table can another
> SEOS operator
> with a full table confirm if they have these prefixes (and
> if so, what
> it shows as aggregator)
>
> I can tell you in Cisco IOS, I accept these prefixes.
>
>
> Printouts below, it seems that all aggregators here are
> 0.0.0.0
> 6.4.1.1 has probably some accidental bug or some
> unfinished function is
> implemented here..
> Regards,
> Marcin
>
>
>
> [bgp1]RedBack_SE100#show bgp route 41.178.0.0/24
> <http://41.178.0.0/24>
> BGP ipv4 unicast routing table entry: 41.178.0.0/24
> <http://41.178.0.0/24>, version 4979776
> Paths: total 2, best path count 1, best peer 193.111.38.97
> Advertised to non peer-group peers: 2
> 91.195.159.30 91.195.159.138
> 15744 13293 3356 15412 24863
> Nexthop 212.106.159.142 (0), peer 212.106.159.142
> (83.230.95.254), AS
> 15744
> Origin IGP, localpref 200, med 0, weight 100, external
> aggregator: 0.0.0.0, AS 24863
> Community: 13000:9001
>
>
> 24724 15412 24863
> Nexthop 193.111.38.97 (0), peer 193.111.38.97
> (193.111.37.1), AS 24724
> Origin incomplete, localpref 200, med 0, weight 100,
> external, best
> aggregator: 0.0.0.0, AS 24863
> Community: 0:15169 0:20940 8714:8714
> [bgp1]RedBack_SE100#
> [bgp1]RedBack_SE100#show bgp route 41.196.4.0/22
> <http://41.196.4.0/22>
> BGP ipv4 unicast routing table entry: 41.196.4.0/22
> <http://41.196.4.0/22>, version 4979779
> Paths: total 2, best path count 1, best peer 193.111.38.97
> Advertised to non peer-group peers: 2
> 91.195.159.30 91.195.159.138
> 15744 13293 3356 15412 24863
> Nexthop 212.106.159.142 (0), peer 212.106.159.142
> (83.230.95.254), AS
> 15744
> Origin IGP, localpref 200, med 0, weight 100, external
> aggregator: 0.0.0.0, AS 24863
> Community: 13000:9001
>
>
> 24724 15412 24863
> Nexthop 193.111.38.97 (0), peer 193.111.38.97
> (193.111.37.1), AS 24724
> Origin incomplete, localpref 200, med 0, weight 100,
> external, best
> aggregator: 0.0.0.0, AS 24863
> Community: 0:15169 0:20940 8714:8714
> [bgp1]RedBack_SE100#
> [bgp1]RedBack_SE100#show bgp route 41.196.8.0/22
> <http://41.196.8.0/22>
> BGP ipv4 unicast routing table entry: 41.196.8.0/22
> <http://41.196.8.0/22>, version 4979773
> Paths: total 2, best path count 1, best peer 193.111.38.97
> Advertised to non peer-group peers: 2
> 91.195.159.30 91.195.159.138
> 15744 13293 3356 15412 24863
> Nexthop 212.106.159.142 (0), peer 212.106.159.142
> (83.230.95.254), AS
> 15744
> Origin IGP, localpref 200, med 0, weight 100, external
> aggregator: 0.0.0.0, AS 24863
> Community: 13000:9001
>
>
> 24724 15412 24863
> Nexthop 193.111.38.97 (0), peer 193.111.38.97
> (193.111.37.1), AS 24724
> Origin incomplete, localpref 200, med 0, weight 100,
> external, best
> aggregator: 0.0.0.0, AS 24863
> Community: 0:15169 0:20940 8714:8714
> [bgp1]RedBack_SE100#
> [bgp1]RedBack_SE100#show bgp route 41.196.12.0/22
> <http://41.196.12.0/22>
> BGP ipv4 unicast routing table entry: 41.196.12.0/22
> <http://41.196.12.0/22>, version 4979775
> Paths: total 2, best path count 1, best peer 193.111.38.97
> Advertised to non peer-group peers: 2
> 91.195.159.30 91.195.159.138
> 15744 13293 3356 15412 24863
> Nexthop 212.106.159.142 (0), peer 212.106.159.142
> (83.230.95.254), AS
> 15744
> Origin IGP, localpref 200, med 0, weight 100, external
> aggregator: 0.0.0.0, AS 24863
> Community: 13000:9001
>
>
> 24724 15412 24863
> Nexthop 193.111.38.97 (0), peer 193.111.38.97
> (193.111.37.1), AS 24724
> Origin incomplete, localpref 200, med 0, weight 100,
> external, best
> aggregator: 0.0.0.0, AS 24863
> Community: 0:15169 0:20940 8714:8714
> [bgp1]RedBack_SE100#
> [bgp1]RedBack_SE100#show bgp route 41.196.16.0/22
> <http://41.196.16.0/22>
> BGP ipv4 unicast routing table entry: 41.196.16.0/22
> <http://41.196.16.0/22>, version 4979767
> Paths: total 2, best path count 1, best peer 193.111.38.97
> Advertised to non peer-group peers: 2
> 91.195.159.30 91.195.159.138
> 15744 13293 3356 15412 24863
> Nexthop 212.106.159.142 (0), peer 212.106.159.142
> (83.230.95.254), AS
> 15744
> Origin IGP, localpref 200, med 0, weight 100, external
> aggregator: 0.0.0.0, AS 24863
> Community: 13000:9001
>
>
> 24724 15412 24863
> Nexthop 193.111.38.97 (0), peer 193.111.38.97
> (193.111.37.1), AS 24724
> Origin incomplete, localpref 200, med 0, weight 100,
> external, best
> aggregator: 0.0.0.0, AS 24863
> Community: 0:15169 0:20940 8714:8714
> [bgp1]RedBack_SE100#
> [bgp1]RedBack_SE100#show bgp route 41.196.24.0/22
> <http://41.196.24.0/22>
> BGP ipv4 unicast routing table entry: 41.196.24.0/22
> <http://41.196.24.0/22>, version 4979769
> Paths: total 2, best path count 1, best peer 193.111.38.97
> Advertised to non peer-group peers: 2
> 91.195.159.30 91.195.159.138
> 15744 13293 3356 15412 24863
> Nexthop 212.106.159.142 (0), peer 212.106.159.142
> (83.230.95.254), AS
> 15744
> Origin IGP, localpref 200, med 0, weight 100, external
> aggregator: 0.0.0.0, AS 24863
> Community: 13000:9001
>
>
> 24724 15412 24863
> Nexthop 193.111.38.97 (0), peer 193.111.38.97
> (193.111.37.1), AS 24724
> Origin incomplete, localpref 200, med 0, weight 100,
> external, best
> aggregator: 0.0.0.0, AS 24863
> Community: 0:15169 0:20940 8714:8714
> [bgp1]RedBack_SE100#
> [bgp1]RedBack_SE100#show bgp route 41.196.36.0/23
> <http://41.196.36.0/23>
> BGP ipv4 unicast routing table entry: 41.196.36.0/23
> <http://41.196.36.0/23>, version 4979766
> Paths: total 2, best path count 1, best peer 193.111.38.97
> Advertised to non peer-group peers: 2
> 91.195.159.30 91.195.159.138
> 15744 13293 3356 15412 24863
> Nexthop 212.106.159.142 (0), peer 212.106.159.142
> (83.230.95.254), AS
> 15744
> Origin IGP, localpref 200, med 0, weight 100, external
> aggregator: 0.0.0.0, AS 24863
> Community: 13000:9001
>
>
> 24724 15412 24863
> Nexthop 193.111.38.97 (0), peer 193.111.38.97
> (193.111.37.1), AS 24724
> Origin incomplete, localpref 200, med 0, weight 100,
> external, best
> aggregator: 0.0.0.0, AS 24863
> Community: 0:15169 0:20940 8714:8714
> [bgp1]RedBack_SE100#
>
>
>
>
>
>
>
>
> Dave
>
>
> Blake Willis wrote:
>
> On Sep 27, 2010, at 2:12 PM, Marcin Kuczera wrote:
>
> [bgp1]se100-test#show bgp neighbor
> 195.66.73.253 malform update
> Dump logged malformed UPDATE messages for
> ??? (1 total entries):
> Sep 27 14:09:48 Malformed UPDATE msg (nbr
> 195.66.73.253, context
> 0x40080002, 120 bytes, repeated 4221 times,
> reason: Invalid msg) -
> ffff ffff ffff ffff ffff ffff ffff ffff 0078
> 0200 0000 4540 0101 02c0
> 0708 0000 611f 0000 0000 5002 000e 0203 0000
> 6094 0000 3c34 0000 611f 4
> 003 04c1 6f26 6180 0404 0000 0000 4005 0400
> 0000 c8c0 080c 0000 3b41
> 0000 51cc 220a 220a 1829 b200 1629 c404 1629
> c408 1629 c40c 1629
> c410 162
> 9 c418 1729 c424
>
> Ouch. I think it's time to open a case with the
> TAC. Looks like we
> won't be testing 6.4 any time soon then...
>
> Is there any possibility to ignore malformed
> updates instead of
> dropping down whole session ?
>
> I don't think I've seen a knob like that from
> any vendor so far...
>
> -Blake
> _______________________________________________
> redback-nsp mailing list
> redback-nsp at puck.nether.net
> <mailto:redback-nsp at puck.nether.net>
> https://puck.nether.net/mailman/listinfo/redback-nsp
>
> 7
>
>
> - --
>
> David Freedman
> Group Network Engineering
>
> david.freedman at uk.clara.net <mailto:david.freedman at uk.clara.net>
> Tel +44 (0) 20 7685 8000
>
> Claranet Group
> 21 Southampton Row
> London - WC1B 5HA - UK
> http://www.claranet.com
>
> Company Registration: 3152737 - Place of registration: England
>
> All the information contained within this electronic message from
> Claranet Ltd is covered by the disclaimer at
> http://www.claranet.co.uk/disclaimer
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.9 (GNU/Linux)
> Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
>
> iEYEARECAAYFAkygxwgACgkQtFWeqpgEZrIAVQCgwCSQ6M/hXPMYQ/Z9CaPXOR2t
> VMUAoM1PwML/PIHCsle0T7C9pV6oJec+
> =Dnrb
> -----END PGP SIGNATURE-----
>
>
>
>
> _______________________________________________
> redback-nsp mailing list
> redback-nsp at puck.nether.net <mailto:redback-nsp at puck.nether.net>
> https://puck.nether.net/mailman/listinfo/redback-nsp
>
>
More information about the redback-nsp
mailing list