[j-nsp] iSCSI, fast write, slow read

[Michael Loftis]

I've fought a similar issue.  You'll want to check for MAC Pause
Frames (show int <blah> extensive, and whatever your OS might have,
maybe ethtool) - the drops themselves also get accounted for usually
in the QoS section on the EX and QFX, often on the ingress port and
NOT the egress port (as they drop on a full buffer for the egress)

It is VERY possible you're running into buffering issues on the
EX3300.  Any MAC pause frames received on the downstream ports can
very negatively effect upstream ports and cause drops further upstream
even to unrelated ports as the buffers fill.

On Fri, Jul 10, 2015 at 7:26 AM, Mike Williams <mike.williams at comodo.com> wrote:
> Hey all,
>
> Firstly, this could very much possibly not be related to any Juniper equipment at all. If so, I apologise in advance.
>
>
>
> So.
> iSCSI.
> 4 servers.
> Target has a 10Gbps Mellanox Connect-X3 Pro.
> Three initiators have 1Gbps I350s.
>
> Writing data, and only writing data, to all 3 initiators runs at ~2.5Gbps.
> Exactly what I'd hope would happen.
>
> However, reading data is pegged to ~1Gbps.
> In fact, "any" reading of data at all pegs the throughput at ~1Gbps.
> 100Mbps reading, 900Mbps writing. 500Mbps reading, 500Mbps writing.
>
> Not sure if this attachment will make it through, but attached is a graph illustrating.
> Before 11:20 was a bonnie++ run reading and writing at the same time.
> After that was ~80 minutes of pure writing.
> Finally pure reading.
>
>
> In the middle is an EX3300.
>
> Hardware inventory:
> Item             Version  Part number  Serial number     Description
> Chassis                                x      EX3300-48T
> Routing Engine 0 REV 14   750-034247   x      EX3300 48-Port
> FPC 0            REV 14   750-034247   x      EX3300 48-Port
>   CPU                     BUILTIN      BUILTIN           FPC CPU
>   PIC 0                   BUILTIN      BUILTIN           48x 10/100/1000 Base-T
>   PIC 1          REV 14   750-034247   x      4x GE/XE SFP+
>     Xcvr 0       REV 01   740-021308   x           SFP+-10G-SR
>     Xcvr 1       REV 01   740-021308   x           SFP+-10G-SR
> Power Supply 0                                           PS 100W AC
> Fan Tray                                                 Fan Tray
>
> Physical interface: xe-0/1/0
>     Laser bias current                        :  7.904 mA
>     Laser output power                        :  0.5820 mW / -2.35 dBm
>     Module temperature                        :  41 degrees C / 106 degrees F
>     Module voltage                            :  3.3420 V
>     Receiver signal average optical power     :  0.1781 mW / -7.49 dBm
>
>
> Juniper branded optics both ends, single-mode fiber connecting them.
>
> It has almost entirely no config at all.
> 12.3R6.6, and a 9216 MTU configured on all ports as the only config change.
>
>
> Now, the reason I'm writing here is I suspect the EX.
>
> Reading across a back-to-back 10Gbps connection (same Connect-X3 Pros) goes at the sort of lick you'd expect for a disk subsystem that can sustain >2Gbps of writes.
>
>
> Am I seeing some sort of buffering issue? When a 10Gbps machine sends traffic to 1Gbps machines.
> Or maybe a physical limitation of the 3300?
>
>
> Closest thing I could find was a discussion about Cisco switches.
> https://supportforums.cisco.com/discussion/12483511/performance-issue-10gbs-1gbs-6880-x6800-ia-1521sy0a
> I don't see any drops logged, and don't understand Cisco config.
>
>
> Thanks for any advice!
>
> --
> Mike Williams
> _______________________________________________
> juniper-nsp mailing list juniper-nsp at puck.nether.net
> https://puck.nether.net/mailman/listinfo/juniper-nsp



-- 

"Genius might be described as a supreme capacity for getting its possessors
into trouble of all kinds."
-- Samuel Butler