Ethernet Flow Point: VLAN tag manipulation for the VPLS service

We continue one of my previous VPLS topology: this time we're going to take a look at what happens from the perspective of the service provider if the customer tags the ingress traffic with 802.1Q.

VPLS service without 802.1Q

I simplified the topology a little: now we have two PEs (CSR7 and CSR6 [we need CSR1000v and IOS-XE for the PEs, because regular IOS doesn't support service instances and VPLS configuration]), and R8 and R9 are customer devices. I configured R8 and R9 are in the same subnet: R8 has an IP address 192.168.1.8/24 and R9 is 192.168.1.9/24. At this point they send untagged traffic without 802.1Q, so they should be able to reach each other as if they were connected to the same switch. So let's quickly configure the VPLS service for R8 and R9 the same way as I did in the previous VPLS post:

CSR7(config)#int g3
CSR7(config-if)#no ip address
CSR7(config-if)#no shut
CSR7(config-if)#service instance 100 ethernet
CSR7(config-if-srv)#encapsulation ?
default catch-all unconfigured encapsulation
dot1ad 802.1ad - Provider Bridges
dot1q IEEE 802.1Q Virtual LAN or S-VLAN
priority-tagged Priority tagged EFP
untagged Untagged encapsulation

CSR7(config-if-srv)#encapsulation default
CSR7(config-if-srv)#bridge-domain 100
CSR7(config-if-srv)#exit
CSR7(config-if)#exit
CSR7(config)#bridge-domain 100
CSR7(config-bdomain)#exit
CSR7(config)#l2 vfi VPLS manual
CSR7(config-vfi)#vpn id 10
%LINEPROTO-5-UPDOWN: Line protocol on Interface pseudowire0, changed state to up
CSR7(config-vfi)#bridge-domain 100
%VFI-6-VFI_STATUS_CHANGED: Status of VFI VPLS changed from DOWN to UP
CSR7(config-vfi)#neighbor 150.6.6.6 encapsulation mpls
%LDP-5-NBRCHG: LDP Neighbor 150.6.6.6:0 (3) is UP

I'd like to highlight the encapsulation command here: if we issue this one under a subinterface, it encapsulates the traffic with 802.1Q VLAN tag. Under the service instance the encapsulation means a whole different thing: this command works like a "match" statement. encapsulation default basically means that the PE router takes everything on ingress (regardless of whether the traffic was tagged, untagged or even double-tagged), and sends to the remote PE encapsulated with the two MPLS label (transport + service label as we've seen in my previous post). If I were saying encapsulation untagged, then the router would drop all of the tagged traffic on ingress, and would only forward untagged traffic to the remote PE. Similarly encapsulation dot1q 10 would drop every frame besides frames tagged with VLAN 10.

Let's also configure the other PE (CSR6) as well using the same vpn id and the same encapsulation default under the service instance:

CSR6(config)#int g3
CSR6(config-if)#no ip addr
CSR6(config-if)#no shut
CSR6(config-if)#service instance 100 ethernet
CSR6(config-if-srv)#encapsulation default
CSR6(config-if-srv)#bridge-domain 100
CSR6(config-if-srv)#exit
CSR6(config-if)#exit
CSR6(config)#bridge-domain 100
CSR6(config-bdomain)#exit
CSR6(config)#l2 vfi VPLS manual
CSR6(config-vfi)#vpn id 10
CSR6(config-vfi)#bridge-domain 100
%VFI-6-VFI_STATUS_CHANGED: Status of VFI VPLS changed from DOWN to UP
CSR6(config-vfi)#neighbor 150.7.7.7 encapsulation mpls
%LDP-5-NBRCHG: LDP Neighbor 150.7.7.7:0 (3) is UP

Now a targeted LDP session is initiated between the loopback of CSR6 and CSR7 which provides the service label for the VPLS service:

Targeted LDP Label Mapping message between CSR6 and CSR6: CSR6 provides the service label (6000) for the VPLS service

Now we have connectivity between R8 and R9:

R9#ping 192.168.1.8
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 192.168.1.8, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 3/3/5 ms

PEs receiving 802.1Q traffic form the customer

There is noting new yet. But what happens if the customer starts tagging the traffic on R8 and R9 with 802.1Q? In this example R8 tags the traffic with VLAN 88 and R9 tags the traffic with VLAN 99:

R9(config-if)#int g0/0.99
R9(config-subif)#encapsulation dot1Q 99
R9(config-subif)#ip address 192.168.1.9 255.255.255.0
R9(config-subif)#no shut

R8(config)#int g0/0.88
R8(config-subif)#encapsulation dot1Q 88
R8(config-subif)#ip address 192.168.1.8 255.255.255.0
R8(config-subif)#no shut

VPLS Customer tagging the ingress traffic — Customer (R8 and R9) starts tagging the ingress traffic with 802.1Q

Now they can't reach each other, because the VLAN tag is different. The service provider still forwards the traffic tagged with the corresponding VLAN tag, but the customer routers (R8 and R9) drop the traffic because the VLAN tag doesn't match. For example if R9 tries to ping R8:

R9(config)#do ping 192.168.1.8
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 192.168.1.8, timeout is 2 seconds:
.....
Success rate is 0 percent (0/5)

The ping fails, but the service provider still forwards the frames tagged with VLAN 99 as we can see below. R8, the customer who drops the packet because the VLAN tag does not match. Also notice the service label (6000): this is what CSR6 sent to CSR7 in the previous pcap.

VLAN tag in the SP core — Frames tagged with VLAN 99 are forwarded to the remote site

Popping the customer tag

With the concept of Ethernet Flow Point (EFP) we have many solutions to rectify this issue as a service provider: we can change the VLAN tag, remove, push a new tag etc. So let's take a look a few examples: the customer still tags the traffic with VLAN 99 and VLAN 88 on the other site, we as a service provider have to find a solution and provide connectivity.

CSR7(config-if)#service instance 100 ethernet
CSR7(config-if-srv)#encapsulation dot1q 99
CSR7(config-if-srv)#rewrite ?
egress Egress Rewrite
ingress Ingress Rewrite

CSR7(config-if-srv)#rewrite ingress tag ?
pop Pop the tag
push Rewrite Operation of push
translate Translate Tag

CSR7(config-if-srv)#rewrite ingress tag pop ?
1 Pop the outermost tag
2 Pop two outermost tags

CSR7(config-if-srv)#rewrite ingress tag pop 1 ?
symmetric Tag egress packets as specified in encapsulation
<cr> <cr>

CSR7(config-if-srv)#rewrite ingress tag pop 1 symmetric

First we set the encapsulation to dot1q 99: again this works as a "match" statement, CSR7 expects to have traffic tagged with VLAN 99, untagged traffic, or traffic tagged with other VLAN numbers will be dropped. The second command is the rewrite ingress tag pop 1 symmetric. This command does the following: on ingress (ingress from the perspective of the service provider, so from CE to PE) we remove the topmost VLAN tag (pop 1). The optional symmetric keyword defines what we do on egress: on the egress (from PE to CE) we send the traffic tagged with the VLAN tag specified with the encapsulation command (so on egress we send traffic tagged with VLAN 99, that's why we need the encapsulation command). Note that if you've specified multiple VLANs with the encapsulation command (which is also possible) the symmetric keyword won't work: we need a single VLAN. Also notice how many options we have above: we can also do an egress action (alternatively, instead of the symmetric keyword we could also create a new egress action), we can remove, change, add new tags etc. We have many options to manipulate the VLAN tags.

And we do the same thing on the other side:

CSR6(config-if)#service instance 100 ethernet
CSR6(config-if-srv)#encapsulation dot1q 88
CSR6(config-if-srv)#rewrite ingress tag pop 1 symmetric

Now R8 and R9 can reach each other again:

So the PEs remove the VLAN tag when forwarding the customer traffic towards the core, and they push a new VLAN tag (specified with the encapsulation command) when forwarding the traffic egress towards the customer. This way we can provide connectivity for the customer routers (R8, and R9) even if they tag the traffic with different VLANs.

Changing the VLAN number

Now let's take a look at another example:

CSR7(config-if)#do show run int g3
Building configuration...

Current configuration : 234 bytes
!
interface GigabitEthernet3
no ip address
negotiation auto
no mop enabled
no mop sysid
service instance 100 ethernet
encapsulation dot1q 99
rewrite ingress tag translate 1-to-1 dot1q 88 symmetric
bridge-domain 100
!
end

CSR6(config-if)#do show run int g3
Building configuration...

Current configuration : 176 bytes
!
interface GigabitEthernet3
no ip address
negotiation auto
no mop enabled
no mop sysid
service instance 100 ethernet
encapsulation dot1q 88
bridge-domain 100
!
end

Now CSR7 changes (translate) the VLAN tag from 99 to 88 when forwarding towards CSR6, and it changes back to VLAN 99 when forwarding towards the customer (R9) because of the symmetric keyword again. CSR6 doesn't do any rewriting: it just sends traffic tagged with VLAN 88 towards CSR7.

Look at the pcap below: this is what CSR7 sends towards CSR6 (service label is 6000). The packets were captured between R2 and R4 (transport label is 4004) and the traffic is tagged with VLAN 88 because of the translate command above:

CSR7 translates to VLAN 99 — CSR7 changes the VLAN tag to 88 when forwarding the traffic to CSR6

So the customer routers can reach each other again:

R9(config-if)#do ping 192.168.1.8
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 192.168.1.8, timeout is 2 seconds:
.!!!!
Success rate is 80 percent (4/5), round-trip min/avg/max = 4/5/7 ms

Q-in-Q double tagging

Finally in the last example let's take a look how we can push a new VLAN tag (Q-in-Q) to the customer traffic:

CSR7(config-if-srv)#do show run int g3
Building configuration...

Current configuration : 242 bytes
!
interface GigabitEthernet3
no ip address
negotiation auto
no mop enabled
no mop sysid
service instance 100 ethernet
encapsulation dot1q 99
rewrite ingress tag translate 1-to-2 dot1q 1067 second-dot1q 88
bridge-domain 100
!
end

CSR6(config-if-srv)#do show run int g3
Building configuration...

Current configuration : 251 bytes
!
interface GigabitEthernet3
no ip address
negotiation auto
no mop enabled
no mop sysid
service instance 100 ethernet
encapsulation dot1q 88
rewrite ingress tag translate 1-to-1 dot1q 99
rewrite egress tag pop 1
bridge-domain 100
!
end

In this example CSR7 tags the traffic with two VLAN tags (Q-in-Q): it changes VLAN 99 to VLAN 88, and pushes an additional second tag (VLAN 1067). CSR6 removes the topmost (1067) VLAN tag (rewrite egress tag pop 1), and changes VLAN 88 to 99 while forwarding the customer traffic towards CSR7. So CSR6 doesn't do Q-in-Q in this example.

CSR7 send Q-in-Q traffic to CSR6 — CSR7 sends double-tagged frames to CSR6

In the pcap above we can see the traffic CSR7 sends to CSR6 (service label 6000), CSR7 sends the traffic with Q-in-Q encapsulation. The topmost tag (1067) is only relevant within the service provider core. Upon receiving the Q-in-Q frame, CSR6 removes the topmost tag (1067) and forwards the traffic to R8 with just a single VLAN tag (VLAN 88).