design of 802.1QinQ tunnels

m4rtin

In real life lets say last-mile provider provides Q-in-Q service and company ABC has a trunk port(NNI) with this last-mile provider. Company ABC would like to connect 10 different customers from 10 different physical locations to it's backbone using the last-mile provider network. Each customer should have a different VLAN in ABC network. Am I correct, that last-mile provider needs to provide 10 different metro VID's- one for each customer(one for each tunnel in other words)?

networker050184

Each location will typically have its own vlan.

m4rtin

networker050184 wrote: »

Each location will typically have its own vlan.

ok, that's what I thought as well However, how do they achieve this? I mean usually the setup of tunnel-port in last-mile providers edge switch should be something like this:

C3750#sho run int Fa1/0/1
Building configuration...

Current configuration : 158 bytes
!
interface FastEthernet1/0/1
 description -> WS-2924-XL
 switchport access vlan 777
 switchport mode dot1q-tunnel
 l2protocol-tunnel stp
 no cdp enable
end

C3750#

..and in ABC network the trunk NNI port facing the last-mile provider is usually configured like this:

WS-C2924-XL#show running-config interface Fa0/2
Building configuration...

Current configuration:
!
interface FastEthernet0/2
 description -> C3750
 switchport trunk encapsulation dot1q
 switchport trunk allowed vlan 1,888,1002-1005
 switchport mode trunk
end

WS-C2924-XL#

It's easy to understand, that ABC is able to add additional VLAN tag's to it's trunk port and those are encapsulated in last-mile providers network to VLAN 777. However, if ABC would like to add new VLAN to it's NNI port with last-mile provider, but this new VLAN is for another company and needs to be delivered to another physical location, I can't see any other possibility than making a new NNI with last-mile provider I mean is it possible to say somehow, that "VLAN 777 should encapsulate VLAN 888, VLAN 999 should encapsulate VLAN 833, VLAN 776 should encapsulate VLAN 882" etc?

networker050184

You are going to want a device that can push two tags.

m4rtin

networker050184 wrote: »

You are going to want a device that can push two tags.

Could you please explain this solution bit more thoroughly? Any examples or additional reading would be perfection itself:)

Panzer919

We would add 14 to the max mtu size to accommodate the additional vlan tags, then set the port as switchport mode dot1q-tunnel, turn on BPDU guard and Root guard.

the config would look like this from the providers side.


interface FastEthernet0/24
switchport access vlan 254 (you dont need to worry about what their VLAN tag)
switchport mode dot1q-tunnel


If you are looking at it from the company in question, it would just appear as a ethernet trunk. Although some services (cdp, VTP) wont work without additional configs.

they would carry the vlan through the network and setup another switch at site B and the config would look the same.

networker050184

If you just have to put the inside tag then you can use Paper919s method and let the network provider push theirs on top. If you need to push two tags before the network provider tag you can use an ethernid device or you can have a router terminating and pushing two tags if you have layer3 on the edge.

Sorry for the short answers. I'm on my phone in the airport traveling today.

Panzer919

networker050184 wrote: »

Paper919

Sorry for the short answers. I'm on my phone in the airport traveling today.

lol, gotta love autocorrect

Panzer919

m4rtin wrote: »

I mean is it possible to say somehow, that "VLAN 777 should encapsulate VLAN 888, VLAN 999 should encapsulate VLAN 833, VLAN 776 should encapsulate VLAN 882" etc?

you CAN do this but it wastes VLANS and from an ISP perspective we hated to waste vlans. We would usually just run the same vlan out to all your (the customers) sites and suggest that you put all your sites on different vlans if you wanted them separated out.

m4rtin

Panzer919, if I understood you correctly, you meant something like this:



So local-loop provider will transfer all the ABC data in VLAN 777 in its backbone via trunk ports and two switches before the end customer(customer X, customer Z and customer Y in this case) device, VLAN 777 is removed and traffic is directed to last switch of local-loop provider premises. In this last switch, the traffic is forwarded to access port and VLAN tag set by ABC is removed as well(882,888,999 in this example).

Did I understand you correctly?


networker050184,
did I understand you correctly, that there would be three VLAN tag's in the frame? Are there any advantages with this triple-tagged frames solution over the solution drawn above?

//drawn solution has at least one disadvantage- there might occur VLAN overlapping in the last switch of last-mile provider. Maybe it's possible to strip off both(outer VLAN set by last-mile provider and inner VLAN set by ABC) VLAN's in the port facing the end-customer(customer X, customer Z and customer Y)?

Panzer919

m4rtin wrote: »

Panzer919, if I understood you correctly, you meant something like this:



So local-loop provider will transfer all the ABC data in VLAN 777 in its backbone via trunk ports and two switches before the end customer(customer X, customer Z and customer Y in this case) device, VLAN 777 is removed and traffic is directed to last switch of local-loop provider premises. In this last switch, the traffic is forwarded to access port and VLAN tag set by ABC is removed as well(882,888,999 in this example).

Did I understand you correctly?

I got a lil confused by your response so I will say it a different way.

I am the customer, the ISP puts in a switch and says you can use port fa0/1. I plug my Ethernet cable in and setup my port for what ever vlan I am using. You can almost imagine that the ISP is not there and that what you are plugging into is your switch on the other side. You dont see what goes on in between.

I am the ISP, I assign my customer VLAN 400. VTP puts vlan 400 across my network. I then go the the customers site and configure the switchport as a trunk so that it carrys our management vlan 300 and the customers vlan 400. The interface facing the customer is setup for vlan 400. It then becomes transparent to the customer.

so a frame comes in to the customers switch and gets tagged with their internal vlan 10, once it reaches the ISP switch that frame is then tagged again with vlan 400. so (vlan400(vlan10[frame])) it travels across the network on vlan 400 to the other end(s) where the process is reversed. So once it reaches the CPE, device the ISP installed, it strips off the 400 tag, then when your switch see's it, all it see's is a frame with a vlan tag of 10.

does that clarify things a lil better?

m4rtin

Panzer919 wrote: »

I am the customer, the ISP puts in a switch and says you can use port fa0/1. I plug my Ethernet cable in and setup my port for what ever vlan I am using. You can almost imagine that the ISP is not there and that what you are plugging into is your switch on the other side. You dont see what goes on in between.

I understand that. ISP has configured an access port towards me and that's what I visualized on my drawing as well. All the traffic from ISP is untagged.

Panzer919 wrote: »

I am the ISP, I assign my customer VLAN 400. VTP puts vlan 400 across my network. I then go the the customers site and configure the switchport as a trunk so that it carrys our management vlan 300 and the customers vlan 400. The interface facing the customer is setup for vlan 400. It then becomes transparent to the customer.

Yes, that's what I thought as well: trunk port(allowed vlan 300,400) on access-switch facing the ISP distribution switch and access port on access-switch in VLAN 400 facing the customer.

Panzer919 wrote: »

Iso a frame comes in to the customers switch and gets tagged with their internal vlan 10, once it reaches the ISP switch that frame is then tagged again with vlan 400. so (vlan400(vlan10[frame])) it travels across the network on vlan 400 to the other end(s) where the process is reversed. So once it reaches the CPE, device the ISP installed, it strips off the 400 tag, then when your switch see's it, all it see's is a frame with a vlan tag of 10.

If it get's tagged with VLAN 400 right after accessing the ISP network, ISP needs to configure tunnel port facing the customer? If yes, then shouldn't customer configure trunk port(allowed vlan 10) facing the ISP equimpment not the access port?

Panzer919

m4rtin wrote: »

If it get's tagged with VLAN 400 right after accessing the ISP network, ISP needs to configure tunnel port facing the customer? If yes, then shouldn't customer configure trunk port(allowed vlan 10) facing the ISP equimpment not the access port?

I'm 99% sure that it is access, but it has been over a year since I've done it. Reason being is that the port facing the hubsite is the trunk (carrying the subs vlan and management vlan), facing the customer is an access port with the higher MTU. The ISP will never see the customers vlan 10, only traffic going through vlan 400.

networker050184

Most of the time from what I've seen the ISP isn't going to use a switch for this task. They will use something like an Accedian Ethernid as the demarc that just pushes 400 on anything that comes in.

I've only seen one instance where an ISP will use an actual ethernet switch as their demarc. In that case it was a tunnel port.

Panzer919

networker050184 wrote: »

Most of the time from what I've seen the ISP isn't going to use a switch for this task. They will use something like an Accedian Ethernid as the demarc that just pushes 400 on anything that comes in.

I've only seen one instance where an ISP will use an actual ethernet switch as their demarc. In that case it was a tunnel port.

We used Cisco ME3400's as a demarc, now the ISP is using some generic media converter that can do this. I'm pretty certain we used access towards the sub and trunk towards the HE. Not unless there are different ways this can be configured.

wolverene13

Panzer919 wrote: »

We used Cisco ME3400's as a demarc, now the ISP is using some generic media converter that can do this. I'm pretty certain we used access towards the sub and trunk towards the HE. Not unless there are different ways this can be configured.

We use 3400ME's also. If they have a 3400 at all of their sites, we provide one VLAN for the customer and they take care of all of the separation of networks into different VLANs on their end. If they don't have a 3400, we either do the QinQ at the port on the switch that is in the CO, or in some cases (like NC State Govt. or some of the casinos out in Vegas, we will actually trunk with the customer, so we provide them with several different VLANs (which is a real pain in the @$$).

wolverene13

Panzer919 wrote: »

I'm 99% sure that it is access, but it has been over a year since I've done it. Reason being is that the port facing the hubsite is the trunk (carrying the subs vlan and management vlan), facing the customer is an access port with the higher MTU. The ISP will never see the customers vlan 10, only traffic going through vlan 400.

It depends on what the customer equipment on the other side is set to. Remember, this is a tunnel so essentially the provider's network is transparent to the customer. It's as if the customer's equipment at each site is all connected to the same Layer 2 switch. Normally, the customer configures their port as a trunking port so that all of their devices are trunking with one another across the WAN.

wolverene13

m4rtin wrote: »

If it get's tagged with VLAN 400 right after accessing the ISP network, ISP needs to configure tunnel port facing the customer? If yes, then shouldn't customer configure trunk port(allowed vlan 10) facing the ISP equimpment not the access port?

Yes, that is correct.

Panzer919

wolverene13 wrote: »

It depends on what the customer equipment on the other side is set to. Remember, this is a tunnel so essentially the provider's network is transparent to the customer. It's as if the customer's equipment at each site is all connected to the same Layer 2 switch. Normally, the customer configures their port as a trunking port so that all of their devices are trunking with one another across the WAN.

correct, unless they just have a single remote site, then they may only use 1 vlan and treat it like an access port.