[j-nsp] port mirror to multiple ports on MX80 in inet6
Paul Vlaar
paul at vlaar.net
Fri Oct 26 04:39:05 EDT 2012
Clarke,
I'm not sure you understood my topology 100%, but let me summarize things:
This is what the topology looks like:
[ Cisco ]
x.x.158.194/30
|
x.x.158.193/30
ge-1/3/11
|
[ MX80 ] irb.100: x.x.158.1/26
/ \
ge-1/0/2 ge-1/3/0
x.x.158.13/26 x.x.158.5/26
host1 host2
I hope that diagram is readable, and I couldn't really fit this into the
drawing nicely, but some additional data:
- analyzer host is connected to ge-1/3/2
- the filter that calls port-mirror is only on ge-1/0/2
- ge-1/0/2 and ge-1/3/0 are in the same bridge domain.
So the Cisco isn't hanging off ge-1/0/2, but comes in at ge-1/3/11.
- echo request from .194 to .13 can't be seen on the analyzer, but the
reply *is* seen.
- echo request from .5 to .13 *can* be seen, and so can the reply.
So you're saying that when we come in on ge-1/3/11 via L3 and go out to
ge-1/0/2, because we traverse into the bridge domain and come from L3,
this is not being mirrored? Somehow I am reading your description as the
other way around though, but I can understand the logic behind this.
I could go for L3 mirroring for the irb.100 interface, but the whole
thing that started this thread for me was that I can't mirror to more
than one next-hop for inet6 (inet has next-hop-group available, but not
inet6), and so I started looking for an alternative way, such as via L2.
What I've done now is followed Chuck Anderson's example of just plugging
the analyzer output port into another switch where the 2 analyzer hosts
are also connected. And then setting the next-hop MAC address to an
unknown to keep the traffic flooding. However I don't use a real
switch, but just cross-connect to another port on the same MX80, which
is part of a virtual switch, and into which I've also plugged the 2
analyzer servers.
So I got what I want, which is L3 mirroring for both v4 and v6 to more
than one analyzer box, at the cost of an extra patch cable and 3 GE
ports on the MX80. Kind of a hack, but it works.
This is actually better since we're only interested in traffic dumping
for specific TCP/UDP port numbers, but it would still be nice to know
what exactly was going on with the L2 results.
~paul
On 24/10/12 8:30 PM, Clarke Morledge wrote:
> Paul,
>
> In your last example, assuming that your cisco router is hanging off
> your mirror source port, ge-1/0/2, it makes sense from my experience
> that your "x.x.158.13 > x.x.158.194: ICMP echo reply" shows up in your
> mirror output, as I mentioned earlier, but not the ICMP echo request in
> the other direction. The echo request enters your L2 configured port,
> but since it then crosses a subnet boundary by hitting your irb.100, the
> MX will not treat it as L2 any more for mirroring purposes.
>
> So if you do a Layer3 port mirror with irb.100 as your mirror source,
> you should be able to see the packet.
>
> Traffic coming out of the IRB egressing out the L2 mirror source port
> gets treated as L2, which is why the L2 mirror works in that direction.
> There is something about the way Integrated Routing and Bridging works
> that accounts for this, but I do not fully understand it.
>
> With respect to the vlan tag/un-tag, because you changed the vlan-id to
> 1000 in the bridge-domain, as the original packet had a vlan tag of 100,
> this changes the mirrored packet. It shows up on the mirror output as
> untagged because your "encapsulation ethernet-bridge" on the interface
> will not tag the packet.
>
> I use the "encapsulation flexible-ethernet-services" with
> "flexible-vlan-tagging" and I am able to change the vlan-id of the
> mirrored output if I need to do that.
>
> The other cases you describe have me scratching my head as to what is
> up, but I've seen other weird things with layer2 mirroring that do not
> make much sense to me. So as to why the behavior between x.x.158.13 and
> x.x.158.5 is reversed now is really puzzling, particularly since traffic
> in both directions should just be L2.
>
> It bugs me that the L2 port mirror examples in the web documentation are
> really poor. They have made some improvements recently, but Juniper
> really needs to step up and cover these different scenarios in detail.
> Typically, I need to set up a port mirror on the fly for a quick look,
> but unfortunately, I end up messing with JTAC for several weeks trying
> to get something to work that takes about 5 minutes on a Cisco platform.
>
> The flexibility of the Junos platform allows for some complex mirroring,
> which is great, but I have wasted a lot of time trying to get a handle
> on this port mirroring thing and still do not get it..... where I can
> afford it, I just say "Forget it, I'll stick with a tap".
>
> If you can make any better heads or tails out of this, I'd like to hear
> about it.
>
> Clarke Morledge
> College of William and Mary
> Information Technology - Network Engineering
> Jones Hall (Room 18)
> Williamsburg VA 23187
>
> On Tue, 23 Oct 2012, Paul Vlaar wrote:
>
>> On 23/10/12 10:59 PM, Clarke Morledge wrote:
>>> ---------------------------------------------------
>>> My question for you would be if you have an IRB interface associated
>>> with the bridge-domain that your mirror source port is in, and if the
>>> ICMP traffic coming into the router is hitting that IRB. If that is the
>>> case, the MX will not treat the traffic coming into your IRB interface
>>> via your "encapsulation ethernet-bridge" as Layer2 traffic in this
>>> context, so it will not get mirrored.
>>> -----------------------------------------------------
>>
>> There is indeed an IRB associated with the bridge-domain of the port to
>> be mirrored:
>>
>> mx80> show configuration bridge-domains VLAN100 routing-interface
>> routing-interface irb.100;
>>
>>> show configuration interfaces irb.100
>> family inet {
>> address x.x.158.1/26;
>> }
>>
>> A traceroute from another router that is one L3 hop away from the MX80,
>> to the IP address of the host connected to the interface that we're
>> doing the port mirror on:
>>
>> cisco#traceroute x.x.158.13
>> Type escape sequence to abort.
>> Tracing the route to 199.115.158.13
>> VRF info: (vrf in name/id, vrf out name/id)
>> 1 x.x.158.193 0 msec 0 msec 0 msec
>> 2 x.x.158.13 0 msec 0 msec 0 msec
>> cisco#
>>
>> x.x.158.193 is the address of the point to point link at the MX80, and
>> x.x.158.13 is the IP address of the mirrored host.
>>
>> So as far as I can see it's not hitting the irb.100 address, however it
>> is doing this on the return, as it's the default gateway out of the host
>> at x.x.158.13. But the return is where we catch the ICMP reply, so that
>> part works.
>>
>> To be complete here, this is the L3 interface where the traffic comes in
>> from the other router:
>>
>> mx80> show configuration interfaces ge-1/3/11
>> unit 0 {
>> family inet {
>> address x.x.158.193/30;
>> }
>> }
>>
>> And this is the FIB entry for the target host:
>>
>> mx80> show route forwarding-table destination x.x.158.13
>> Routing table: default.inet
>> Internet:
>> Destination Type RtRef Next hop Type Index NhRef Netif
>> x.x.158.13/32 dest 1 0:1b:21:84:d7:a6 ucst 768 4 ge-1/0/2.0
>>
>> Routing table: default-switch.inet
>> Internet:
>> Destination Type RtRef Next hop Type Index NhRef Netif
>> default perm 0 rjct 538 1
>>
>> Routing table: __master.anon__.inet
>> Internet:
>> Destination Type RtRef Next hop Type Index NhRef Netif
>> default perm 0 rjct 529 1
>>
>>
>>> ------------------------------------------------------
>>>
>>> With respect to the vlan tagging on the port mirror output interface,
>>> the L2 packet being mirrored will egress with the original vlan tag
>>> intact, no matter what vlan id you configure on the mirror destination
>>> interface.
>>>
>>> However, if you insert the "vlan-id" keyword into the "bridge-domain"
>>> configuration, you can manipulate the vlan tag that gets egressed out of
>>> your mirror destination port. But if the "vlan-id" in the bridge domain
>>> is the same as the vlan-id of the mirror destination port, the original
>>> packet vlan-id gets preserved on output.
>>
>> I've tried setting vlan-id on the bridge-domain for the analyzer port:
>>
>> [edit bridge-domains analyzers]
>> mx80# show
>> domain-type bridge;
>> vlan-id 1000;
>> interface ge-1/3/2.0;
>>
>> After I commit, when I ping from the host connected to the same bridge
>> domain as the mirrored port, where before I could see the ICMP request
>> go in as well, I now only see the reply:
>>
>> 17:22:26.067209 00:1b:21:84:d7:a6 > 00:1b:21:86:a5:22, ethertype IPv4
>> (0x0800), length 98: x.x.158.13 > x.x.158.5: ICMP echo reply, id 56599,
>> seq 2, length 64
>> 17:22:27.067850 00:1b:21:84:d7:a6 > 00:1b:21:86:a5:22, ethertype IPv4
>> (0x0800), length 98: x.x.158.13 > x.x.158.5: ICMP echo reply, id 56599,
>> seq 3, length 64
>>
>> And what's more it is untagged now.
>>
>> The same for pinging from the cisco router:
>>
>> 17:25:22.720228 00:1b:21:84:d7:a6 > 80:71:1f:c6:34:f0, ethertype IPv4
>> (0x0800), length 114: x.x.158.13 > x.x.158.194: ICMP echo reply, id 33,
>> seq 2, length 80
>>
>> (previously this was tagged as well)
>>
>> So perhaps a trick can be found here to make this work both ways...
>>
>>> I have not tested this, but my guess is that this might also apply to
>>> packets being mirrored that are untagged at the source.
>>>
>>> Port mirroring on this platform is enough to make your head spin.
>>
>> I unfortunately have to agree on that.
>>
>> ~paul
>>
>>
More information about the juniper-nsp
mailing list