[c-nsp] QinQ 4500X -> ME3600 and access(pop) multiple inner vlans

CiscoNSP List CiscoNSP_list at hotmail.com
Thu Feb 4 01:18:34 EST 2016


Ok - Just an update to this, had another "test" service setup, same outer, new inner (940 and 942)....as it was going to take a very long time to confirm the existing service was setup "correctly" on MS end...

And all is working (lol thankfully!).....no vfi needed, original config works(Just remote end must not have been setup correctly for the first test service)

 service instance 940 ethernet
  description description TEST_OUTER_940_Inner_942
  encapsulation dot1q 940 second-dot1q 942
  rewrite ingress tag pop 2 symmetric
  bridge-domain 942

interface Vlan942
 description TEST_OUTER_INNER_TAGs
 mtu 9100
 ip address 10.97.97.1 255.255.255.252
 no ip proxy-arp
end


#ping 10.97.97.2
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 10.97.97.2, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 1/1/4 ms

#sh mac address-table dyn bridge-domain 942
          Mac Address Table
-------------------------------------------

BD      Mac Address       Type        Ports
----    -----------       --------    -----
 942    5087.89fd.d320    DYNAMIC     Gi0/24+Efp940


Thanks again for all who assisted.


________________________________________
From: cisco-nsp <cisco-nsp-bounces at puck.nether.net> on behalf of CiscoNSP List <CiscoNSP_list at hotmail.com>
Sent: Thursday, 4 February 2016 4:12 PM
To: Erik Sundberg; cisco-nsp at puck.nether.net
Subject: Re: [c-nsp] QinQ 4500X -> ME3600 and access(pop) multiple      inner   vlans

It does - Thanks very much Erik - I will try your solution later today.

Just as a side note, my "current" config (i.e. not using vfi), the carrier is seeing traffic originating from us, but no return traffic from azure/MS....so we are currently trying to confirm that the Azure/MS side is configured (Could take a while :( )

List/Eric - Should the following work?  Or is the vfi config 100% needed as Eric has provided to get this to work?

service instance 940 ethernet
  description description TEST_Outer_940_Inner_941
  encapsulation dot1q 940 second-dot1q 941
  rewrite ingress tag pop 2 symmetric
  bridge-domain 941

interface Vlan941
 description TEST_OUTER_INNER_TAGs
 mtu 9100
 ip address xxx.xxx.xxx.xxx 255.255.255.252
no ip proxy-arp

________________________________________
From: Erik Sundberg <ESundberg at nitelusa.com>
Sent: Thursday, 4 February 2016 4:04 PM
To: CiscoNSP List; cisco-nsp at puck.nether.net
Subject: RE: [c-nsp] QinQ 4500X -> ME3600 and access(pop) multiple inner        vlans

Rememer you removed\popped off both vlan tags of 800 and 20 on the interface, then put the untagged frame in bridge group 820.  The bridge group could have been 300, the bridge group number has no assoication to the VLAN configuration on the interface.


Here is a step by step,

Step By Step Description                        Commands "semi-colon is a new line"
--------------------------------------------------------------------------------------------------------------------------
1. Create a Bridge Group                        l2 vfi TESTING manual;  vpn id 820;  bridge-domain 820
2. Go to the interface                          interface g0/0/11
3. Incoming frame Outter 800, inner 20          encapsulation dot1q 800 second-dot1q 20
4. Removed Both Taggs 800 and 20                rewrite ingress tag pop 2 symmetric
5. Put Unttagged Frame in Bridge Group 820      bridge-domain 820
        Global Config
6a. ASR920 Created L3 Routed Interface  bridge-group 820; interface bdi820
6b. ME3800 Create L3 Routed Interface           interface vlan 820
6b. ME3800 Connect Interface to BridgeGroup     xconnect vfi TESTING
7. Assigned IP Address to the Interface         ip address 192.168.0.2 255.255.255.0


Does this help???


-----Original Message-----
From: CiscoNSP List [mailto:CiscoNSP_list at hotmail.com]
Sent: Wednesday, February 03, 2016 4:47 PM
To: Erik Sundberg <ESundberg at nitelusa.com>; cisco-nsp at puck.nether.net
Subject: Re: [c-nsp] QinQ 4500X -> ME3600 and access(pop) multiple inner vlans

...and quick question (I hope) on the VFI config you tested in the lab..you stated outer vlan 800, inner vlan 20, but in your conf you are using vlan 820?  Is this a typo, or on purpose?  i.e. shouldnt the bridge domain be 20, vpn id be 20 and vlan int be 20?....Im not familiar at all with vfi's so could be completely wrong :)

Cheers:

"I Did lab this up on a ASR920, the commands should be fairly close, I noted the difference between the ASR920 and ME3800

Carrier Side: QinQ Interface Outer VLAN 800 InnerVLAN 20 IP 192.168.0.1/24

l2 vfi TESTING manual
 vpn id 820
 bridge-domain 820

interface GigabitEthernet0/0/11
 no ip address
 negotiation auto
 service instance 820 ethernet
  encapsulation dot1q 800 second-dot1q 20
  rewrite ingress tag pop 2 symmetric
  bridge-domain 820

!!! ON A ASR920
bridge-domain 820
interface BDI820
 ip vrf forwarding TESTING2
 ip address 192.168.0.2 255.255.255.0
 no shut

!!!!ON A ME3800
int vlan 820
  xconnect vfi TESTING
  ip address 192.168.0.2 255.255.255.0
  no shut"



________________________________________
From: cisco-nsp <cisco-nsp-bounces at puck.nether.net> on behalf of CiscoNSP List <CiscoNSP_list at hotmail.com>
Sent: Thursday, 4 February 2016 6:52 AM
To: Erik Sundberg; cisco-nsp at puck.nether.net
Subject: Re: [c-nsp] QinQ 4500X -> ME3600 and access(pop) multiple inner        vlans

Thanks for confirming Eric (Re the 4500X) - I have another question...4500X will receive frame from carrier with outer tag (vlan 940), and not care about inner tag....vlan 940 must be configured on this switch, which it is, and tagged on both trunk ports (To carrier, and to ME3600)....but what about "return" traffic?  i.e. we are popping the inner vlan 941 on the ME3600, and bringing that up in a vlan Int.....wont return traffic be "tagged" vlan 941, and therefore be dropped by our 4500X?

And you are saying the only way to do this is via VPLS (i.e. My current conf, with just vlan Int wont work)

4500X to Carrier (Simple trunk port)

interface TenGigabitEthernet1/1/11
 description CARRIER_X_AGG_SY3_SN
 switchport trunk allowed vlan 76,940
 switchport mode trunk
 switchport nonegotiate
 mtu 1998
 storm-control broadcast level 1.00
 storm-control action trap
 spanning-tree bpdufilter enable
 spanning-tree guard root


4500X to ME3600 (Again, simple trunk port....lot more vlans, as we do all cust links on the ME's as VRFs etc)

interface TenGigabitEthernet1/1/3
 description DOT1Q_TRUNK_TO_ME3600
 switchport trunk allowed vlan 5,109,135,143,144,147,158,183,221-223,228,229
 switchport trunk allowed vlan add 265-269,279,284-286,296,307,321,324-326,335
 switchport trunk allowed vlan add 338,339,357,396-398,412,413,463,466-468,576
 switchport trunk allowed vlan add 577,606,626,661,663-666,747,758,759,800-810
 switchport trunk allowed vlan add 823,829,832,835,836,854,864,865,873,881,899
 switchport trunk allowed vlan add 931,940,941,1035,1303  switchport mode trunk  switchport nonegotiate  mtu 9100  storm-control broadcast level 1.00  storm-control action trap  spanning-tree bpdufilter enable  spanning-tree guard root


ME3600 Int conf that connects to 4500X:

interface GigabitEthernet0/24
 description DOT1QTRUNK_TO_4500X
 switchport trunk allowed vlan none
 switchport mode trunk
 dampening
 mtu 9100
 load-interval 30
 storm-control broadcast level pps 2k
 storm-control multicast level pps 2k
 storm-control action trap
 service instance 940 ethernet
  description description TEST_Outer_940_Inner_941
  encapsulation dot1q 940 second-dot1q 941
  rewrite ingress tag pop 2 symmetric
  bridge-domain 941

interface Vlan941
 description TEST_OUTER_INNER_TAGs
 mtu 9100
 ip address xxx.xxx.xxx.xxx 255.255.255.252  no ip proxy-arp


ME3600 has both vlans configured, 4500 only has vlan 940 configured (Ive tried adding vlan 941, but it made no difference)

________________________________________
From: Erik Sundberg <ESundberg at nitelusa.com>
Sent: Wednesday, 3 February 2016 10:22 PM
To: CiscoNSP List; cisco-nsp at puck.nether.net
Subject: RE: QinQ 4500X -> ME3600 and access(pop) multiple inner vlans

A Catalyst Switch will only look at the first VLAN Tag(Outter), it doesn't care about the inner vlan tag and will forward the frame on. Just watch your MTU Size, because you lose 4btyes to the inner vlan tag.

A Good Example. http://blog.jhe.co/2009/11/dot1q-tunneling.html


Share the config for the following Ports, and I can check it for you.

Cisco 4500X Provider Port
Cisco 4500X Port to ME3800.
Cisco ME3800 Port Config




I Did lab this up on a ASR920, the commands should be fairly close, I noted the difference between the ASR920 and ME3800

Carrier Side: QinQ Interface Outer VLAN 800 InnerVLAN 20 IP 192.168.0.1/24

l2 vfi TESTING manual
 vpn id 820
 bridge-domain 820

interface GigabitEthernet0/0/11
 no ip address
 negotiation auto
 service instance 820 ethernet
  encapsulation dot1q 800 second-dot1q 20
  rewrite ingress tag pop 2 symmetric
  bridge-domain 820

!!! ON A ASR920
bridge-domain 820
interface BDI820
 ip vrf forwarding TESTING2
 ip address 192.168.0.2 255.255.255.0
 no shut

!!!!ON A ME3800
int vlan 820
  xconnect vfi TESTING
  ip address 192.168.0.2 255.255.255.0
  no shut


ASR920#ping vrf TESTING2 192.168.0.1
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 192.168.0.1, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 1/2/4 ms

ASR920#show mac-address-table bdomain 820

   Nile Mac Address Entries

   BD    mac addr        type     ports
   ----------------------------------------------------------------------------------------------
   820   0012.448e.8062  DYNAMIC  Gi0/0/11.Efp820


ASR920#sh ip arp vrf TESTING2
Protocol  Address          Age (min)  Hardware Addr   Type   Interface
Internet  192.168.0.1             2   0012.448e.8062  ARPA   BDI820
Internet  192.168.0.2             -   00f2.8bdd.603f  ARPA   BDI820


-----Original Message-----
From: CiscoNSP List [mailto:CiscoNSP_list at hotmail.com]
Sent: Wednesday, February 03, 2016 3:51 AM
To: Erik Sundberg <ESundberg at nitelusa.com>; cisco-nsp at puck.nether.net
Subject: Re: QinQ 4500X -> ME3600 and access(pop) multiple inner vlans


Before I go any further trying to get this to work, Im hoping someone can answer this, so Im not trying to make something work that simply wont, given the hardware currently in place.

Given the "AGG" switch is a 4500X, ie not a "Metro E" switch, and double tagged frames is "Metro E", will our 4500X be seeing this double tagged frame as potentially "mangled" and drop it?

I have a suspicion this might be the case....but Im hoping I am wrong :(

Cheers
________________________________________
From: Erik Sundberg <ESundberg at nitelusa.com>
Sent: Wednesday, 3 February 2016 6:32 PM
To: CiscoNSP List; cisco-nsp at puck.nether.net
Subject: RE: QinQ 4500X -> ME3600 and access(pop) multiple inner vlans

http://supportforums.cisco.com/sites/default/files/legacy/8/2/1/96128-ASR%209000%20Multiple%20Services%20onthe%20same%20port%20example.jpg

Look at the Routed VPLS/EoMPLS section

http://www.cisco.com/c/en/us/td/docs/switches/metro/me3600x_3800x/software/release/15-1_2_ey/configuration/guide/3800x3600xscg/swmpls.html#pgfId-1260366

What you are looking for a is a VPLS Routed Interface Configuration, just association the VPLS instance with the bridge group. I have never tried this, so I could be wrong, but it's worth a shot.

Try adding this. This creates' a VPLS Instance and associates it to the Bridge Group 941.
l2 vfi TESTING manual
 vpn id 941
 bridge-domain 941

Then under your VLAN Interface associate the VLAN Interface with the VPLS Instanace.
interface Vlan941
 xconnect vfi TESTING   <<<<<<<<<<<< ADD THIS
 description INNER_OUTER_TAG_TEST
 mtu 9100
 ip address xxx.xxx.xxx.xxx 255.255.255.252  no ip proxy-arp


Also watch your MTU Size's a mismatch can cause the VPLS instance to be down. You can check it with "show mpls l2transport vc vcid 941 detail"

This looks good

interface GigabitEthernet0/24   <- Connects to 4500X
service instance 940 ethernet
  description description Inner_outer_tag_test_Outer_940_Inner_941
  encapsulation dot1q 940 second-dot1q 941
  rewrite ingress tag pop 2 symmetric
  bridge-domain 941





-----Original Message-----
From: CiscoNSP List [mailto:CiscoNSP_list at hotmail.com]
Sent: Wednesday, February 03, 2016 12:06 AM
To: Erik Sundberg <ESundberg at nitelusa.com>; cisco-nsp at puck.nether.net
Subject: Re: QinQ 4500X -> ME3600 and access(pop) multiple inner vlans

Thanks Eric,

We have no visibility into the remote end, but I have setup the following on one of our ME's (Test service, that has supposedly been configured by carrier, and remote end)

Vlans are:

940 (outer)
941 (Inner)

Both vlans have been created on the ME, and only vlan 940 on the 4500X that connects to carrier:

ME3600 conf

interface GigabitEthernet0/24   <- Connects to 4500X
service instance 940 ethernet
  description description Inner_outer_tag_test_Outer_940_Inner_941
  encapsulation dot1q 940 second-dot1q 941
  rewrite ingress tag pop 2 symmetric
  bridge-domain 941

interface Vlan941
 description INNER_OUTER_TAG_TEST
 mtu 9100
 ip address xxx.xxx.xxx.xxx 255.255.255.252  no ip proxy-arp


Im unable to ping remote end, nor am I seeing any dynamic Macs for bridge domain 941 - Is there any additional commands I can run to "see" if we are indeed receiving the Outer and Inner Tags on the ME?

The only MAC I am learning on the 4500X is from the ME3600

#sh mac address-table dynamic vlan 940
Unicast Entries
 vlan     mac address     type        protocols               port
---------+---------------+--------+---------------------+---------------
---------+---------------+--------+---------------------+----------
 940      3462.882a.4640   dynamic ip,ipx,assigned,other TenGigabitEthernet1/1/3


cheers

________________________________________
From: Erik Sundberg <ESundberg at nitelusa.com>
Sent: Tuesday, 2 February 2016 10:10 PM
To: CiscoNSP List; cisco-nsp at puck.nether.net
Subject: RE: QinQ 4500X -> ME3600 and access(pop) multiple inner vlans

You are probably better using a service instance for each vlan. Then you can send each VLAN where ever.

service instance 800 ethernet
  description description LINK_TO_CARRIER_X_VIA_4500X
  encapsulation dot1q 800 second-dot1q 10
  rewrite ingress tag pop 2 symmetric
  bridge-domain 10

service instance 801 ethernet
  description description LINK_TO_CARRIER_X_VIA_4500X
  encapsulation dot1q 800 second-dot1q 20
  rewrite ingress tag pop 2 symmetric
  bridge-domain 20


--------------------------------------------------------------------------

Another way of doing it is the following

If you have two tag come in, only POP 1 Tag. Then your CTag are put in to the Bridge Domain.

Int G0/1
service instance 800 ethernet
  description LINK_TO_CARRIER_X_VIA_4500X
  encapsulation dot1q 800
  rewrite ingress tag pop 1 symmetric
  bridge-domain 800


Then on your Egress port you can set it as untagged, C Tags of 10,20, 30 will be in the bridge domain and will be passed on egress.

 Int G0/2
Description to 4500x
service instance 400 ethernet
  description 4500X
  encapsulation dot1q untagged
  bridge-domain 800


If you need to pull one CTag out for something else you can do that like so.

Int G0/1
service instance 804 ethernet
  description Go Somewhere else
  encapsulation dot1q 800 second-dot1q 40
  rewrite ingress tag pop 1 symmetric
  bridge-domain 40


-----Original Message-----
From: CiscoNSP List [mailto:CiscoNSP_list at hotmail.com]
Sent: Tuesday, February 02, 2016 4:01 AM
To: Erik Sundberg <ESundberg at nitelusa.com>; cisco-nsp at puck.nether.net
Subject: Re: QinQ 4500X -> ME3600 and access(pop) multiple inner vlans


Thanks for the quick reply - We use service instances quite a bit, but only popping first tag, then creating vlan int (for vrf), or PW....so fairly basic stuff :)

In this circumstance, where we would receive vlan 800 as outer tag, and we want to access inner vlans 10,20,30 how would this look under a service instance?

ie.  Something like?

 service instance 800 ethernet
  description description LINK_TO_CARRIER_X_VIA_4500X
  encapsulation dot1q 800 second-dot1q 10,20,30
  rewrite ingress tag pop 2 symmetric
  bridge-domain ?


or a separate service instance and pop inner vlans on each one?

 service instance 800 ethernet
  description description LINK_TO_CARRIER_X_VIA_4500X
  encapsulation dot1q 800 second-dot1q 10
  rewrite ingress tag pop 2 symmetric
  bridge-domain 10

service instance 801 ethernet
  description description LINK_TO_CARRIER_X_VIA_4500X
  encapsulation dot1q 800 second-dot1q 20
  rewrite ingress tag pop 2 symmetric
  bridge-domain 20

Cheers


________________________________________
From: Erik Sundberg <ESundberg at nitelusa.com>
Sent: Tuesday, 2 February 2016 8:40 PM
To: CiscoNSP List; cisco-nsp at puck.nether.net
Subject: RE: QinQ 4500X -> ME3600 and access(pop) multiple inner vlans

http://www.cisco.com/c/dam/en/us/td/docs/switches/metro/me3600x_3800x/software/design/guide/ME3600x_Design_Guide.pdf

http://www.cisco.com/c/dam/en/us/td/docs/switches/metro/me3600x_3800x/software/design/guide/CE2-0_certification_v1.pdf

http://d2zmdbbm9feqrf.cloudfront.net/2012/usa/pdf/BRKSPG-2209.pdf


You must do switchport commands
Services Instance is just a number, we keep it the same as the VLAN Id Under the Service Instance you specify the VLAN ID with endcapsulation dot1q x Rewrite ingress Tag POP 1 symmetric  -- This removes the first VLAN Tag on in incoming frame, if you do pop 2, it removes the 2 VLAN Tags.

First Example is VPLS with Bridge Domains. Bridge domain ID does not have to the same as the VLAN Id Second Example is EoMPLS XConnect

VPLS
------
l2 vfi  VPLS1 manual
 vpn id 41
 bridge-domain 41
 neighbor 1.2.3.4 encapsulation mpls

interface GigabitEthernet0/19
 switchport trunk allowed vlan none
 switchport mode trunk
 mtu 9180

!VPLS Example
 service instance 41 ethernet
  encapsulation dot1q 41
  rewrite ingress tag pop 1 symmetric
  bridge-domain 41

!EoMPLS Example
service instance 117 ethernet
  encapsulation dot1q 117
  rewrite ingress tag pop 1 symmetric
  xconnect 3.4.5.6 275 encapsulation mpls pw-class L2VPN
  xconnect <Remote router IP> <Unique ID in the AS>



I hope this helps.


-----Original Message-----
From: cisco-nsp [mailto:cisco-nsp-bounces at puck.nether.net] On Behalf Of CiscoNSP List
Sent: Tuesday, February 02, 2016 3:28 AM
To: cisco-nsp at puck.nether.net
Subject: [c-nsp] QinQ 4500X -> ME3600 and access(pop) multiple inner vlans

Hi Everyone,


We have an AGG port(Standard trunk port) to a carrier on a 4500X - Port has multiple customer vlans for p-t-p eth services.

A service they have released will allow us to connect to azure/office 365 via QinQ(Carrier doing QinQ, not us) - i.e. We agree to an outer vlan tag with the carrier, and they create QinQ tunnel to azure/office 365...then multiple inner vlan tags are agreed to between us/azure for various services over this QinQ tunnel.

My question is this:

With our current setup (i.e. 4500X, standard dot1q trunk), we would just tag the outer vlan for the carrier to use for the QinQ tunnel to azure...this is fine, but for us to be able to "access" the inner vlans, Im hoping we can trunk this outer vlan to an ME3600, and then pop each inner vlan, and use them as needed.....Is this possible? ie will the "inner" tags be maintained going through the 4500X, and if so, if someone could point me in the direction of ME3600 docco that details how to pop the individual inner vlans, it would be greatly appreciated.

Eg.

Carriers outer vlan is 800
Inner tags from azure are 10,20,30

Cheers


_______________________________________________
cisco-nsp mailing list  cisco-nsp at puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

________________________________

CONFIDENTIALITY NOTICE: This e-mail transmission, and any documents, files or previous e-mail messages attached to it may contain confidential information that is legally privileged. If you are not the intended recipient, or a person responsible for delivering it to the intended recipient, you are hereby notified that any disclosure, copying, distribution or use of any of the information contained in or attached to this transmission is STRICTLY PROHIBITED. If you have received this transmission in error please notify the sender immediately by replying to this e-mail. You must destroy the original transmission and its attachments without reading or saving in any manner. Thank you.

________________________________

CONFIDENTIALITY NOTICE: This e-mail transmission, and any documents, files or previous e-mail messages attached to it may contain confidential information that is legally privileged. If you are not the intended recipient, or a person responsible for delivering it to the intended recipient, you are hereby notified that any disclosure, copying, distribution or use of any of the information contained in or attached to this transmission is STRICTLY PROHIBITED. If you have received this transmission in error please notify the sender immediately by replying to this e-mail. You must destroy the original transmission and its attachments without reading or saving in any manner. Thank you.

________________________________

CONFIDENTIALITY NOTICE: This e-mail transmission, and any documents, files or previous e-mail messages attached to it may contain confidential information that is legally privileged. If you are not the intended recipient, or a person responsible for delivering it to the intended recipient, you are hereby notified that any disclosure, copying, distribution or use of any of the information contained in or attached to this transmission is STRICTLY PROHIBITED. If you have received this transmission in error please notify the sender immediately by replying to this e-mail. You must destroy the original transmission and its attachments without reading or saving in any manner. Thank you.

________________________________

CONFIDENTIALITY NOTICE: This e-mail transmission, and any documents, files or previous e-mail messages attached to it may contain confidential information that is legally privileged. If you are not the intended recipient, or a person responsible for delivering it to the intended recipient, you are hereby notified that any disclosure, copying, distribution or use of any of the information contained in or attached to this transmission is STRICTLY PROHIBITED. If you have received this transmission in error please notify the sender immediately by replying to this e-mail. You must destroy the original transmission and its attachments without reading or saving in any manner. Thank you.
_______________________________________________
cisco-nsp mailing list  cisco-nsp at puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

________________________________

CONFIDENTIALITY NOTICE: This e-mail transmission, and any documents, files or previous e-mail messages attached to it may contain confidential information that is legally privileged. If you are not the intended recipient, or a person responsible for delivering it to the intended recipient, you are hereby notified that any disclosure, copying, distribution or use of any of the information contained in or attached to this transmission is STRICTLY PROHIBITED. If you have received this transmission in error please notify the sender immediately by replying to this e-mail. You must destroy the original transmission and its attachments without reading or saving in any manner. Thank you.
_______________________________________________
cisco-nsp mailing list  cisco-nsp at puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/


More information about the cisco-nsp mailing list