MPLS LAYER 3 VPN ON CISCO

How to Configure MPLS Layer 3 VPN?

MPLS operates in the middle of the data link layer (Layer 2) and the network layer (Layer 3) hence it is considered to be a Layer 2.5 protocol. It operates using the protocol called LDP (Label Distribution Protocol) which assigns labels ranging from 16 to 1,048,575 (0-15 reserved and cannot be used in Cisco routers) to IP prefixes/subnets in the routing table. LDP relies on the routing table in order for it to form its LIB (Label Information Base) and LFIB (Label Forwarding Information Base). LSR (Label Switch Routers) are routers in the middle of the Service provider network that uses label to perform routing. LER (Label Edge Routers) are routers that are entry and exit points of the network. They are generally the Provider Edge (PE) routers.

The three general operations of LDP when dealing with labeling packets are PUSH, SWAP and POP. PUSH means that the incoming packet has no label and has to assign a new label to it. SWAP is basically changing the label to a different label. POP on the other hand, is to remove the label. LERs are usually the ones who PUSH labels to packets, LSRs do SWAP of the labels and the penultimate hop routers (Adjacent LSRs to the LERs, in our case Routers R2 and R3) do the POP operation. The feature where LSRs POP labels before it sends out to LERs is called PHP (Penultimate Hop Popping) with Implicit-null.

The LDP router-id needs to be reachable through the global routing table in order for LDP to form neighborship. The router-id election for LDP is the highest IP address of any loopback interface. If no loopback interfaces exist, it will be the highest IP address of any physical interface that is operational.

Configure MPLS LDP in the Service Provider network
R1(config)# int fa0/0
R1(config-if)# mpls ip
R1(config-if)# exit
R1(config)# mpls label ?
protocol Set platform default label distribution protocol
range Label range
R1(config)# mpls label range ?
Minimum label value
R1(config)# mpls label range 1000 1999
% Label range changes will take effect at the next reload.
R1(config)# mpls ldp router-id lo0 force
R2(config)# int fa0/0
R2(config-if)# mpls ip
R2(config-if)# int f0/1
R2(config-if)# mpls ip
R2(config-if)# mpls label range 2000 2999

R3(config)# int fa0/0
R3(config-if)# mpls ip
R3(config-if)# int fa0/1
R3(config-if)# mpls ip
R3(config-if)# mpls label range 3000 3999

R4(config)# int fa0/0
R4(config-if)# mpls ip
R4(config-if)# mpls label range 4000 4999

The command “mpls ip” is required to form LDP neighbors. It is only configured in interfaces that are inside the service provider network. Any interfaces such as loopbacks or those facing the customer are not required to be configured because LDP is not required between customer and PE routers. Though the customer is connected to the MPLS network, it is a common practice for service providers not to make their network visible to the customer.

The “mpls label range” command in the routers sets the number of labels only. I configured it that way so it will be easier to explain later how LDP works. In the example configuration above, the number of labels that can be assigned for each router only amounts to 1000. If the network has more than 1000 prefixes, the rest of the prefixes will not be labeled and will be routed using IP.

The “mpls ldp router-id loopback0 force” command enforces the LDP to use the IP address of Loopback0 as its ID. The “force” keyword will tear down existing LDP sessions and clear all the current bindings and applies the changes to the LDP ID. If “force” is not used, the router will wait until the current interface of the LDP ID goes down before it applies the new LDP ID specified in the command.

The routers have been restarted. Let’s check the LDP neighborship. Let’s use two examples for brevity.

R1#sh mpls ldp neigh
Peer LDP Ident: 2.2.2.2:0; Local LDP Ident 1.1.1.1:0
TCP connection: 2.2.2.2.18805 - 1.1.1.1.646
State: Oper; Msgs sent/rcvd: 46/46; Downstream
Up time: 00:31:24
LDP discovery sources:
FastEthernet0/0, Src IP addr: 12.12.12.2
Addresses bound to peer LDP Ident:
12.12.12.2 2.2.2.2 23.23.23.2

R3#sh mpls ldp neigh
Peer LDP Ident: 2.2.2.2:0; Local LDP Ident 3.3.3.3:0
TCP connection: 2.2.2.2.646 - 3.3.3.3.42778
State: Oper; Msgs sent/rcvd: 67/66; Downstream
Up time: 00:49:52
LDP discovery sources:
FastEthernet0/1, Src IP addr: 23.23.23.2
Addresses bound to peer LDP Ident:
12.12.12.2 2.2.2.2 23.23.23.2
Peer LDP Ident: 4.4.4.4:0; Local LDP Ident 3.3.3.3:0
TCP connection: 4.4.4.4.59644 - 3.3.3.3.646
State: Oper; Msgs sent/rcvd: 67/67; Downstream
Up time: 00:49:51
LDP discovery sources:
FastEthernet0/0, Src IP addr: 34.34.34.4
Addresses bound to peer LDP Ident:
34.34.34.4 4.4.4.4

R1#show mpls forwarding-table
Local Outgoing Prefix Bytes tag Outgoing Next Hop
tag tag or VC or Tunnel Id switched interface
1000 Pop tag 2.2.2.2/32 0 Fa0/0 12.12.12.2
1001 Pop tag 23.23.23.0/24 0 Fa0/0 12.12.12.2
1002 2001 34.34.34.0/24 0 Fa0/0 12.12.12.2
1003 2002 3.3.3.3/32 0 Fa0/0 12.12.12.2
1004 2003 4.4.4.4/32 0 Fa0/0 12.12.12.2

As mentioned, the LDP ID will be the highest loopback IP address that is operational in the router. The LDP routers, before they form an LDP session, elect which router will be active and passive. The router chosen as active will initiate the LDP TCP connection. In our case, R2 initiated a connection using a random number which in this case is 18805, R1 responds back with the port 646, the TCP port that is assigned to LDP. The “Addresses bound to peer LDP Ident:” section specifies that the routes below are directly connected to the LDP neighbor. Directly connected routes to the neighbor by default will not have any label assigned in the LIB (Label Information Base).

The “show mpls forwarding-table” also called the LFIB, shows the actions which LDP will take when it receives a specific label. As you can see, it doesn’t put any labels to directly connected routes of its adjacent LDP neighbor which is R2.

Configure VRF in the Provider Edge (PE) Routers:

VRF (Virtual Routing and Forwarding) is comparable to a VLAN in a switch. VRF is used to create different routing tables that are separated from each other. Since one VRF can’t see what routes are in another VRF, the same IP prefix can exist in different VRFs. However, duplicate IP prefixes will have an issue when it comes to route-leaking between VRFs. 

R1(config)# ip vrf CUST-A
R1(config-vrf)# rd 65002:1
R1(config-vrf)# route
R1(config-vrf)# route-target import 65002:1
R1(config-vrf)# route-target export 65002:1

R4(config)# ip vrf CUST-A
R4(config-vrf)# rd 65002:1
R4(config-vrf)# route-target import 65002:1
R4(config-vrf)# route-target export 65002:1

Let’s apply the VRF into the interface facing the CE (customer edge) router.
R1(config-if)# ip vrf forwarding CUST-A
% Interface FastEthernet0/1 IP address 15.15.15.1 removed due to enabling VRF CUST-A
R1(config-if)# ip address 15.15.15.1 255.255.255.0

R4(config-if)# int fa0/1
R4(config-if)# ip vrf forwarding CUST-A
% Interface FastEthernet0/1 IP address 46.46.46.4 removed due to enabling VRF CUST-A
R4(config-if)# ip address 46.46.46.4 255.255.255.0

The VRF name is locally significant. It is not a transitive attribute that will be shared between routers. In fact, in an MPLS VPN network, as long as the RD (Route Distinguisher) and the RT (Route Target) values are configured correctly but the VRF names are different, the MPLS VPN service will work.

RD is what Multiprotocol BGP uses to distinguish and makes the route unique. The standard telco practice is to assign a unique RD for every customer. RT on the other hand, is an extended BGP community that marks, tags or classifies the prefix. The “export” keyword in the command means that the route will be marked and announced out with that value; “import” means put all the routes with that mark, into the VRF’s routing table specified above the command.

Configure BGP VPNv4 peering between R1 and R4:

VPNv4 is an address-family of Multiprotocol BGP. To explain it simply, VPNv4 is a collection of all routes from different VRFs that were marked with the extended community route-target. This is the address-family where route-leaking can be performed. Route-leaking is simply sharing a route from one VRF to another. Common application for this is, one company wants to connect to another company’s servers and they happen to be connected to the same MPLS provider. 

R1(config)# router bgp 65001
R1(config-router)# address-family vpnv4
R1(config-router-af)# neigh 4.4.4.4 activate

R1#sh run | inc router bgp | address-family vpnv4|neigh
router bgp 65001
bgp log-neighbor-changes
neighbor 4.4.4.4 remote-as 65001
neighbor 4.4.4.4 update-source Loopback0
neighbor 4.4.4.4 activate
neighbor 4.4.4.4 next-hop-self
address-family vpnv4
neighbor 4.4.4.4 activate
neighbor 4.4.4.4 send-community extended

R4(config-if)# router bgp 65001
R4(config-router)# address-family vpnv4
R4(config-router-af)# neigh 1.1.1.1 activate

R4#sh ip bgp vpnv4 all sum
BGP router identifier 4.4.4.4, local AS number 65001
BGP table version is 1, main routing table version 1

Neighbor V AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down State/PfxRcd
1.1.1.1 4 65001 116 116 1 0 0 00:01:09 0

In VPNv4 address-family configuration, you simply issue the neighbor statement and the keyword “activate”. The BGP peering configuration needs to be done outside the address-family. The router understands that VPNv4 peering needs to activate extended communities so it automatically configured the statement highlighted above. In regards to the VPNv4 BGP peering, we can’t see any prefixes for now since there is no peering yet between the PE s and CEs.

Configure Peering between PE routers R1 and R4 to customer routers CUST_A-R1 and CUST-A-R2. Announce Loopback 10 and 100 in the CE routers. Verify connectivity.

R1(config)# router bgp 65001
R1(config-router)# address-family vpnv4
R1(config-router-af)# address-family ipv4 vrf CUST-A
R1(config-router-af)# neighbor 15.15.15.5 remote-as 65002
R1(config-router-af)# neighbor 15.15.15.5 activate
R1(config-router-af)# neighbor 15.15.15.5 as-override

CUST_A-R1(config)# router bgp 65002
CUST_A-R1(config-router)# neighbor 15.15.15.1 remote-as 65001
CUST_A-R1(config-router)# network 5.5.5.5 mask 255.255.255.255
CUST_A-R1(config-router)# network 55.55.55.55 mask 255.255.255.255

R4(config)# router bgp 65001
R4(config-router)# address-family ipv4 vrf CUST-A
R4(config-router-af)# neighbor 46.46.46.6 remote-as 65002
R4(config-router-af)# neighbor 46.46.46.6 activate
R4(config-router-af)# neighbor 46.46.46.6 as-override

CUST_A-R2(config)# router bgp 65002
CUST_A-R2(config-router)# network 6.6.6.6 mask 255.255.255.255
CUST_A-R2(config-router)# network 66.66.66.66 mask 255.255.255.255
CUST_A-R2(config-router)# neighbor 46.46.46.4 remote-as 65001

The PE is configured with an “address-family ipv4 vrf” when peering with the CE routers. The “as-override” command replaces the AS of the route to circumvent the BGP loop prevention. BGP loop prevention blocks any route that it receives from an eBGP peer with its own AS (65002 in this case) inside it. The AS for the customer is 65002, but notice the output below, the PE’s replaced the AS to 65001 to enable communication between these two routers with the same AS inside an MPLS cloud. CUST_A-R2 is now able to see the CUST_A-R1 routes but with a different AS. Another way to do this is to configure a neighbor statement with “allowas-in” keyword.

R1#show ip bgp vpnv4 vrf CUST-A
BGP table version is 7, local router ID is 1.1.1.1
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal,
r RIB-failure, S Stale
Origin codes: i - IGP, e - EGP, ? - incomplete

Network Next Hop Metric LocPrf Weight Path
Route Distinguisher: 65002:1 (default for vrf CUST-A)
*> 5.5.5.5/32 15.15.15.5 0 0 65002 i
*>i6.6.6.6/32 4.4.4.4 0 100 0 65002 i
*> 55.55.55.55/32 15.15.15.5 0 0 65002 i

CUST_A-R2#sh ip bgp
BGP table version is 5, local router ID is 66.66.66.66
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal,
r RIB-failure, S Stale
Origin codes: i - IGP, e - EGP, ? - incomplete

Network Next Hop Metric LocPrf Weight Path
*> 5.5.5.5/32 46.46.46.4 0 65001 65001 i
*> 6.6.6.6/32 0.0.0.0 0 32768 i
*> 55.55.55.55/32 46.46.46.4 0 65001 65001 i
*> 66.66.66.66/32 0.0.0.0 0 32768 i
Lets check and verify connectivity.
CUST_A-R2#traceroute 55.55.55.55 source l100

Type escape sequence to abort.
Tracing the route to 55.55.55.55

1 46.46.46.4 24 msec 28 msec 20 msec
2 34.34.34.3 88 msec 92 msec 100 msec
3 23.23.23.2 108 msec 92 msec 80 msec
4 15.15.15.1 80 msec 68 msec 72 msec
5 15.15.15.5 88 msec 88 msec 80 msec

As we can see, there is a full reachability between the CE routers but the traceroute shows the path it took inside the service provider core network. This is not an advisable behavior, normally service provider from the customer any information about its core network.

Let’s configure a way to do that.
R1(config)# no mpls ip propagate-ttl
R4(config)# no mpls ip propagate-ttl
Let’s test that again.
CUST_A-R2#traceroute 55.55.55.55 source l100

Type escape sequence to abort.
Tracing the route to 55.55.55.55

1 46.46.46.4 28 msec 16 msec 20 msec
2 15.15.15.1 80 msec 80 msec 80 msec
3 15.15.15.5 108 msec 104 msec 96 msec

Now, the service provider network has been hidden through the “no mpls ip propagate-ttl” command.