vPC VXLAN EVPN Leaf and Spine – Part 8 – L3VNI Configuration – Multi Tenant Leaking

Published by Nick Carlton on

In the last part, we got a multiple tenant VRFs working, in this part, we will look at joining the tenants up via route leaking.

Lets remind ourselves of the topology:

Image

So, currently we have our two VRFs with separate routing tables for vlan 10 and vlan 20:

LEAF-1# show ip route vrf OVERLAY-TENANT1
IP Route Table for VRF "OVERLAY-TENANT1"

10.1.1.0/24, ubest/mbest: 1/0, attached
    *via 10.1.1.254, Vlan10, [0/0], 00:57:09, direct
10.1.1.1/32, ubest/mbest: 1/0, attached
    *via 10.1.1.1, Vlan10, [190/0], 00:07:33, hmm
10.1.1.2/32, ubest/mbest: 1/0
    *via 10.0.1.102%default, [200/0], 00:06:22, bgp-64500, internal, tag 64500, segid: 100999 tunnelid: 0xa000166 encap: VXLAN

10.1.1.3/32, ubest/mbest: 1/0
    *via 10.0.1.103%default, [200/0], 00:06:07, bgp-64500, internal, tag 64500, segid: 100999 tunnelid: 0xa000167 encap: VXLAN

10.1.1.254/32, ubest/mbest: 1/0, attached
    *via 10.1.1.254, Vlan10, [0/0], 00:57:09, local
LEAF-1# show ip route vrf OVERLAY-TENANT2
IP Route Table for VRF "OVERLAY-TENANT2"

10.2.1.0/24, ubest/mbest: 1/0, attached
    *via 10.2.1.254, Vlan20, [0/0], 00:07:00, direct
10.2.1.1/32, ubest/mbest: 1/0, attached
    *via 10.2.1.1, Vlan20, [190/0], 00:07:00, hmm
10.2.1.2/32, ubest/mbest: 1/0
    *via 10.0.1.102%default, [200/0], 00:06:30, bgp-64500, internal, tag 64500, segid: 100998 tunnelid: 0xa000166 encap: VXLAN

10.2.1.3/32, ubest/mbest: 1/0
    *via 10.0.1.103%default, [200/0], 00:06:11, bgp-64500, internal, tag 64500, segid: 100998 tunnelid: 0xa000167 encap: VXLAN

10.2.1.254/32, ubest/mbest: 1/0, attached
    *via 10.2.1.254, Vlan20, [0/0], 00:07:00, local

VRF Configuration

To do this, we should pick a border leaf that we want to use to complete the leaking, we can leak locally on all leaves, but this can clog up the routing table on the leaves.

In this case, because we are doing vPC, we have to pick a pair of leaves. We will use LEAF-3 and LEAF-4 as the border nodes.

We need to attract the other VTEPs to the border nodes using routes. Sometimes it makes sense to have default routes, however, we will use summaries in this example. Lets say 10.1.0.0/16 is reserved for Tenant1 and 10.2.0.0/16 is reserved for Tenant2.

Lets configure BGP on LEAF-3 and LEAF-4:

router bgp 64500
  vrf OVERLAY-TENANT1
    address-family ipv4 unicast
      aggregate-address 10.2.0.0/16 summary-only
  vrf OVERLAY-TENANT2
    address-family ipv4 unicast
      aggregate-address 10.1.0.0/16 summary-only

Now, this configuration won't don anything on its own, as BGP needs to be able to see routes within these summaries in order to advertise them, currently we are doing any leaking and therefore they won't be advertised.

In order to get these routes leaked, we need to configure VRF leaking on the border nodes:

vrf context OVERLAY-TENANT1
  address-family ipv4 unicast
    route-target import 64500:100998
    route-target import 64500:100998 evpn
vrf context OVERLAY-TENANT2
  address-family ipv4 unicast
    route-target import 64500:100999
    route-target import 64500:100999 evpn

The above is a little confusing if you look at it for the first time, lets run through what is happening.

Because in our main VRF configuration we have set route-target both auto and route-target both auto evpn. This auto creates the RTs for the VRF, in the case of this environment, the first part of the RT is the ASN and the second part is the VNI assigned to the VRF. Taking this into account, we get the below RTs:

vrf OVERLAY-TENANT1 - 64500:100999
vrf OVERLAY-TENANT2 - 64500:100998

Adding evpn to the end of the second route-target command, ensures that evpn routes are also pulled in with the same RT value. Just leaking the networks is not enough here as the evpn routes will point to a specific leaf where that host is.

Once this configuration is in place, we should see the leaking happening on LEAF-3 in the route tables:

LEAF-3# show ip route vrf OVERLAY-TENANT1
IP Route Table for VRF "OVERLAY-TENANT1"

10.1.0.0/16, ubest/mbest: 1/0
    *via Null0, [220/0], 00:00:43, bgp-64500, discard, tag 64500
10.1.1.0/24, ubest/mbest: 1/0, attached
    *via 10.1.1.254, Vlan10, [0/0], 00:58:32, direct
10.1.1.1/32, ubest/mbest: 1/0
    *via 10.0.1.101%default, [200/0], 00:15:16, bgp-64500, internal, tag 64500, segid: 100999 tunnelid: 0xa000165 encap: VXLAN

10.1.1.2/32, ubest/mbest: 1/0, attached
    *via 10.1.1.2, Vlan10, [190/0], 00:14:06, hmm
10.1.1.3/32, ubest/mbest: 1/0
    *via 10.0.1.103%default, [200/0], 00:13:51, bgp-64500, internal, tag 64500, segid: 100999 tunnelid: 0xa000167 encap: VXLAN

10.1.1.254/32, ubest/mbest: 1/0, attached
    *via 10.1.1.254, Vlan10, [0/0], 00:58:32, local
10.2.0.0/16, ubest/mbest: 1/0
    *via Null0, [220/0], 00:00:51, bgp-64500, discard, tag 64500
10.2.1.0/24, ubest/mbest: 1/0, attached
    *via 10.2.1.254%OVERLAY-TENANT2, Vlan20, [20/0], 00:00:54, bgp-64500, external, tag 64500
10.2.1.1/32, ubest/mbest: 1/0
    *via 10.0.1.101%default, [200/0], 00:00:54, bgp-64500, internal, tag 64500, segid: 100998 (Asymmetric) tunnelid: 0xa000165 encap: VXLAN

10.2.1.3/32, ubest/mbest: 1/0
    *via 10.0.1.103%default, [200/0], 00:00:54, bgp-64500, internal, tag 64500, segid: 100998 (Asymmetric) tunnelid: 0xa000167 encap: VXLAN

LEAF-3# show ip route vrf OVERLAY-TENANT2
IP Route Table for VRF "OVERLAY-TENANT2"

10.1.0.0/16, ubest/mbest: 1/0
    *via Null0, [220/0], 00:00:53, bgp-64500, discard, tag 64500
10.1.1.0/24, ubest/mbest: 1/0, attached
    *via 10.1.1.254%OVERLAY-TENANT1, Vlan10, [20/0], 00:00:56, bgp-64500, external, tag 64500
10.1.1.1/32, ubest/mbest: 1/0
    *via 10.0.1.101%default, [200/0], 00:00:56, bgp-64500, internal, tag 64500, segid: 100999 (Asymmetric) tunnelid: 0xa000165 encap: VXLAN

10.1.1.3/32, ubest/mbest: 1/0
    *via 10.0.1.103%default, [200/0], 00:00:56, bgp-64500, internal, tag 64500, segid: 100999 (Asymmetric) tunnelid: 0xa000167 encap: VXLAN

10.2.0.0/16, ubest/mbest: 1/0
    *via Null0, [220/0], 00:00:45, bgp-64500, discard, tag 64500
10.2.1.0/24, ubest/mbest: 1/0, attached
    *via 10.2.1.254, Vlan20, [0/0], 00:14:26, direct
10.2.1.1/32, ubest/mbest: 1/0
    *via 10.0.1.101%default, [200/0], 00:14:30, bgp-64500, internal, tag 64500, segid: 100998 tunnelid: 0xa000165 encap: VXLAN

10.2.1.2/32, ubest/mbest: 1/0, attached
    *via 10.2.1.2, Vlan20, [190/0], 00:14:00, hmm
10.2.1.3/32, ubest/mbest: 1/0
    *via 10.0.1.103%default, [200/0], 00:13:42, bgp-64500, internal, tag 64500, segid: 100998 tunnelid: 0xa000167 encap: VXLAN

10.2.1.254/32, ubest/mbest: 1/0, attached
    *via 10.2.1.254, Vlan20, [0/0], 00:14:26, local

This is a lot of information but we can see that 10.1 and 10.2 are now in the same routing table on the border leaves. Lets see what it looks like from the perspective or a normal leaf:

LEAF-2# show ip route vrf OVERLAY-TENANT1
IP Route Table for VRF "OVERLAY-TENANT1"

10.1.1.0/24, ubest/mbest: 1/0, attached
    *via 10.1.1.254, Vlan10, [0/0], 01:06:41, direct
10.1.1.1/32, ubest/mbest: 1/0, attached
    *via 10.1.1.1, Vlan10, [190/0], 00:17:05, hmm
10.1.1.2/32, ubest/mbest: 1/0
    *via 10.0.1.102%default, [200/0], 00:15:54, bgp-64500, internal, tag 64500, segid: 100999 tunnelid: 0xa000166 encap: VXLAN

10.1.1.3/32, ubest/mbest: 1/0
    *via 10.0.1.103%default, [200/0], 00:15:39, bgp-64500, internal, tag 64500, segid: 100999 tunnelid: 0xa000167 encap: VXLAN

10.1.1.254/32, ubest/mbest: 1/0, attached
    *via 10.1.1.254, Vlan10, [0/0], 01:06:41, local
10.2.0.0/16, ubest/mbest: 1/0
    *via 10.0.1.3%default, [200/0], 00:02:39, bgp-64500, internal, tag 64500, segid: 100999 tunnelid: 0xa000103 encap: VXLAN

LEAF-2# show ip route vrf OVERLAY-TENANT2
IP Route Table for VRF "OVERLAY-TENANT2"

10.1.0.0/16, ubest/mbest: 1/0
    *via 10.0.1.3%default, [200/0], 00:02:42, bgp-64500, internal, tag 64500, segid: 100998 tunnelid: 0xa000103 encap: VXLAN

10.2.1.0/24, ubest/mbest: 1/0, attached
    *via 10.2.1.254, Vlan20, [0/0], 00:16:17, direct
10.2.1.1/32, ubest/mbest: 1/0, attached
    *via 10.2.1.1, Vlan20, [190/0], 00:16:17, hmm
10.2.1.2/32, ubest/mbest: 1/0
    *via 10.0.1.102%default, [200/0], 00:15:50, bgp-64500, internal, tag 64500, segid: 100998 tunnelid: 0xa000166 encap: VXLAN

10.2.1.3/32, ubest/mbest: 1/0
    *via 10.0.1.103%default, [200/0], 00:15:32, bgp-64500, internal, tag 64500, segid: 100998 tunnelid: 0xa000167 encap: VXLAN

10.2.1.254/32, ubest/mbest: 1/0, attached
    *via 10.2.1.254, Vlan20, [0/0], 00:16:17, local

We can see all the local routes as we could before, and now we also see the /16 routes. However, these are only via a single path, and we have two border nodes, why is that?

Well, its BGP not adding in the second path because we haven't told it to! Lets allow the other path to populate in the routing table, this wants to be added on all leaves:

router bgp 64500
  vrf OVERLAY-TENANT1
    address-family ipv4 unicast
      maximum-paths ibgp 2
  vrf OVERLAY-TENANT2
    address-family ipv4 unicast
      maximum-paths ibgp 2

Now we see both paths:

LEAF-2# show ip route 10.2.0.0 vrf OVERLAY-TENANT1
IP Route Table for VRF "OVERLAY-TENANT1"

10.2.0.0/16, ubest/mbest: 2/0
    *via 10.0.1.3%default, [200/0], 00:00:30, bgp-64500, internal, tag 64500, segid: 100999 tunnelid: 0xa000103 encap: VXLAN
    *via 10.0.1.4%default, [200/0], 00:00:30, bgp-64500, internal, tag 64500, segid: 100999 tunnelid: 0xa000104 encap: VXLAN

LEAF-2# show ip route 10.1.0.0 vrf OVERLAY-TENANT2
IP Route Table for VRF "OVERLAY-TENANT2"

10.1.0.0/16, ubest/mbest: 2/0
    *via 10.0.1.3%default, [200/0], 00:00:36, bgp-64500, internal, tag 64500, segid: 100998 tunnelid: 0xa000103 encap: VXLAN
    *via 10.0.1.4%default, [200/0], 00:00:36, bgp-64500, internal, tag 64500, segid: 100998 tunnelid: 0xa000104 encap: VXLAN

We now also have conectivity between the two vlans again:

Image

In the next part we will look at allowing external access to other resources outside of the fabric.

Categories: CiscoCisco Nexus vPCVXLAN
Previous Post: vPC VXLAN EVPN Leaf and Spine - Part 7 - L3VNI Configuration - Multi Tenant

0 Comments

Leave a Reply

Avatar placeholder

Your email address will not be published. Required fields are marked *

Related Posts

Cisco

vPC VXLAN EVPN Leaf and Spine – Part 7 – L3VNI Configuration – Multi Tenant

In the last part, we got a single tenant VRF working, in this part, we will look at splitting vlan 20 off into its own separate Tenant to show a multi-tenant setup. Lets remind ourselves Read more…

Cisco

vPC VXLAN EVPN Leaf and Spine – Part 6 – L3VNI Configuration – Single Tenant

In this part we are going to look at the L3VNI config to get the hosts in the two vlans talking to each other. Lets remind ourselves of the topology: All of the configuration below Read more…

Cisco

vPC VXLAN EVPN Leaf and Spine – Part 5 – Anycast Gateways

In this part we are going to look at the anycast gateway setup of the topology. This will help get the servers off their local network. Lets remind ourselves of the topology: The layout itself Read more…