[f-nsp] packetloss B2P622 POS blades

Jeroen Oldenhof jeroen at cj2.nl
Tue Feb 10 15:19:19 EST 2009


Jonas Frey schreef:
> Jeroen,
>
> did you update the POS image on the card, too?
> The POS cards have their own firmware.
>   

Hi Jonas,

yes, the POS firmware matches the MGMT firmware. If this is not the case 
the POS blade won't even boot..

Best,
  - Jeroen



> On Tue, 2009-02-10 at 16:12, Jeroen Oldenhof wrote:
>   
>> Ronald Esveld schreef:
>>     
>>> Jeroen,
>>>
>>> Try hw bc, to see if anything is wrong with the hw buffers
>>>   
>>>       
>> Hi Ronald,
>>
>> Thanks for your suggestion. I pasted the output of 'hw bc' and also 'sh 
>> back' for reference below.
>> I notice that the WriteDrops counters are rising pretty fast (1-10 
>> p/second) on the B24E at rtr-A and the B2P622 on rtr-B..
>>
>> could this indicate a problem on these boards?
>>
>> Best,
>>   - Jeroen
>>
>>
>> rtr-A#hw bc
>> Slot      FreeDepth  WriteDrop    WriteIn   WriteOut     ReadIn    ReadOut
>>                                     UseIn     UseOut     FreeIn    FreeOut
>>  1            3983    2942481   10814209   16571486   16571486   16571486
>>                                 14751231    6051473    8993954    7662627
>>  2             900    8880735   11046211    5143640    5143640    5143640
>>                                  5143640    2165476   11046211   11046334
>>  4             910    3527799    4921759   11235329   11235329   11235329
>>                                 11235329    1393960    4921759    4921872
>>
>> rtr-A#hw back
>> Slot  Mod   FreeQ    DMADrop    BPDrop    WriteDrop     Last
>> -----------------------------------------------------------------------
>>  1   B4GMR4  3983       0          0    2942481     D:26 H:23M:24S:8
>>  2   B24E     900       0          0    8881143     D:26 H:23M:24S:8
>>  4   B2P622   910       0          0    3527799     D:26 H:23M:24S:8
>>
>>
>>
>>
>> rtr-B#hw bc
>> Slot      FreeDepth  WriteDrop    WriteIn   WriteOut     ReadIn    ReadOut
>>                                     UseIn     UseOut     FreeIn    FreeOut
>>  1            1939          7   10548638   14716992   14716992   14716992
>>                                 10526350    6357989    6357996    4878671
>>  2             916          0    8861748    8861771    8861771    8861771
>>                                  8861772    8861749    8861749    8861856
>>  3             900          0     815540     815672     815672     815672
>>                                   815672     815540     815540     815663
>>  4             910     315291   15082124   16210906   16210906   16210906
>>                                 16210906   14766833   15082124   15082237
>> rtr-B#sh back
>> Slot  Mod   FreeQ    DMADrop    BPDrop    WriteDrop     Last
>> -----------------------------------------------------------------------
>>  1   B8GMR   1939       0          0          7     D:1  H:15M:37S:5
>>  2   B8GC     916       0          0          0          NEVER
>>  3   B24E     900       0          0          0          NEVER
>>  4   B2P622   909       0          0     315580     D:1  H:15M:34S:5
>>
>>
>>     
>>> -----Oorspronkelijk bericht-----
>>> Van: foundry-nsp-bounces at puck.nether.net
>>> [mailto:foundry-nsp-bounces at puck.nether.net] Namens Jeroen Oldenhof
>>> Verzonden: dinsdag 10 februari 2009 14:29
>>> Aan: foundry-nsp at puck.nether.net
>>> Onderwerp: [f-nsp] packetloss B2P622 POS blades
>>>
>>> Hi!
>>>
>>> Has anyone experience with the B2P622 Foundry POS blades?
>>>
>>> We've deployed them around 9 months ago when we took our STM-1 in 
>>> production.. but since then we're constantly facing packetloss of about
>>> 1%.
>>>
>>> Our setup (hope it remains readable):
>>>                   +-------+    STM-1    +-------+
>>> Transits / <---+--+ RTR-A +=============+ RTR-B +--- <internal>
>>> IX's           |  +-------+             +-------+
>>>          +-----+-+
>>>          |monitor|
>>>          +-------+
>>>
>>> RTR-A&B: Foundry BI4000 w/MGMT4 ironcore running 07.8.04 (B2R07804).
>>> RTR-A is an edge-router facing several transits and exchanges. Through 
>>> the STM it is connected to RTR-B using iBGP.
>>> the monitor is used for smokeping and other management tasks.
>>>
>>> At first we performed eBGP routing to the ix's and transits on RTR-A. 
>>> This caused much packetloss, even when pinging from the monitor-box to 
>>> RTR-A directly. With no traffic packetloss is zero, but getting much 
>>> worse when the amount of traffic grows, around 4% at 60mbit. RTR-A was 
>>> also pulling around 20% CPU. I assume the POS interface has little or no
>>>
>>> CAM for layer 3, which makes it querying the CPU big time. This being a 
>>> resource consuming i guess some packets could be dropped there.
>>>
>>> We then moved all routing and BGP functionality to RTR-B, making RTR-A 
>>> simply a breakout box. Packetloss is reduced, but still around 1%.. and 
>>> the smokepings sill looks awful..
>>> We also swapped POS/FE/MGMT blades and ports and tried different 
>>> firmwares on both ends. No port-errors on both ethernet ports or POS 
>>> ports. The STM provider (TATA) reports no errors.
>>>
>>> On all paths outside the STM there is ZERO loss: from monitoring box to 
>>> several internet destinations, from and to several internal hosts past 
>>> the RTR.. so the STM and its interfaces are definately the problem.
>>>
>>> I figured out that some POS debugging can be done using 'dm console-on 4
>>>
>>> 2', 'dm cli 4 2 Q' returns some interesting commands.. but I can't find 
>>> a way to use them properly..
>>>
>>> so.. anyone on this list has any experience with these? Or encountered 
>>> simmillar issues?
>>>
>>> Thanks!
>>>
>>> Jeroen Oldenhof
>>>
>>> telnet at RTR-B# show pos 4/2
>>>
>>> POS4/2 is up, line protocol is up
>>>   No port name
>>>   Hardware is Packet over Sonet
>>>   Peer Internet address is 0.0.0.0
>>>   MTU 4470 bytes, encapsulation PPP, clock is line
>>>   Framing is SDH, BW 155000Kbit, CRC 32
>>>   Loopback not set, keepalive is set (10 sec), scramble disabled
>>>   LCP state is opened, IPCP state is init
>>>   300 second input rate: 50423416 bits/sec, 8213 packets/sec
>>>   300 second output rate: 12068536 bits/sec, 5802 packets/sec
>>>   2135136909 packets input, 11504550338524 bytes, 0 no buffer
>>>   Received 0 CRCs, 0 shorts, 0 giants, 0 alignments
>>>   1940413782 packets output, 2732631789028 bytes, 0 underruns
>>>   Line protocol is UP
>>>   Member of 5 L2 VLANs, port is tagged, port state is FORWARDING
>>>   STP configured to ON
>>>   Configured Path Trace String :
>>>   Received Path Trace String : RTR-A 4/2
>>>
>>> _______________________________________________
>>>
>>>       
>
>   



More information about the foundry-nsp mailing list