VXLAN EVPN Leaf and Spine – Part 7 – L3VNI Configuration – Multi Tenant Leaking

In the last part, we got a multiple tenant VRFs working, in this part, we will look at joining the tenants up via route leaking.

Lets remind ourselves of the topology:

So, currently we have our two VRFs with separate routing tables for vlan 10 and vlan 20:

leaf-1# show ip route vrf OVERLAY-TENANT1
IP Route Table for VRF OVERLAY-TENANT1

10.10.1.0/24, ubest/mbest: 1/0, attached
    *via 10.10.1.254, Vlan10, [0/0], 00:08:09, direct
10.10.1.1/32, ubest/mbest: 1/0, attached
    *via 10.10.1.1, Vlan10, [190/0], 00:06:02, hmm
10.10.1.3/32, ubest/mbest: 1/0
    *via 10.0.0.5%default, [200/0], 00:06:02, bgp-64500, internal, tag 64500, se
gid: 100999 tunnelid: 0xa000005 encap: VXLAN

10.10.1.254/32, ubest/mbest: 1/0, attached
    *via 10.10.1.254, Vlan10, [0/0], 00:08:09, local

leaf-1# show ip route vrf OVERLAY-TENANT2
IP Route Table for VRF OVERLAY-TENANT2

10.20.1.0/24, ubest/mbest: 1/0, attached
    *via 10.20.1.254, Vlan20, [0/0], 00:08:16, direct
10.20.1.1/32, ubest/mbest: 1/0, attached
    *via 10.20.1.1, Vlan20, [190/0], 00:05:37, hmm
10.20.1.3/32, ubest/mbest: 1/0
    *via 10.0.0.5%default, [200/0], 00:05:37, bgp-64500, internal, tag 64500, se
gid: 100998 tunnelid: 0xa000005 encap: VXLAN

10.20.1.254/32, ubest/mbest: 1/0, attached
    *via 10.20.1.254, Vlan20, [0/0], 00:08:16, local

VRF Configuration

To do this, we should pick a border leaf that we want to use to complete the leaking, we can leak locally on all leaves, but this can clog up the routing table on the leaves.

In this case, we will use leaf-3 as the border node. Lets add some static default routes into the two tenant VRFs on it:

vrf context OVERLAY-TENANT1
  ip route 0.0.0.0/0 Null0
exit
vrf context OVERLAY-TENANT2
  ip route 0.0.0.0/0 Null0
exit

Both VRFs now have a static default route pointed at null0, we need to add the route within the BGP process too on leaf-3:

router bgp 64500
  vrf OVERLAY-TENANT1
    address-family ipv4 unicast
      network 0.0.0.0/0
  vrf OVERLAY-TENANT2
    address-family ipv4 unicast
      network 0.0.0.0/0

We should now see these routes appear on the other leaves:

leaf-1# show ip route 0.0.0.0 vrf OVERLAY-TENANT1
IP Route Table for VRF OVERLAY-TENANT1

0.0.0.0/0, ubest/mbest: 1/0
    *via 10.0.0.5%default, [200/0], 00:11:55, bgp-64500, internal, tag 64500, se
gid: 100999 tunnelid: 0xa000005 encap: VXLAN

leaf-1# show ip route 0.0.0.0 vrf OVERLAY-TENANT2
IP Route Table for VRF OVERLAY-TENANT2

0.0.0.0/0, ubest/mbest: 1/0
    *via 10.0.0.5%default, [200/0], 00:11:49, bgp-64500, internal, tag 64500, se
gid: 100998 tunnelid: 0xa000005 encap: VXLAN

Currently, we still can't communicate between the Tenants because we haven't completed the VRF leaking on the border node:

vrf context OVERLAY-TENANT1
  address-family ipv4 unicast
    route-target import 64500:100998
    route-target import 64500:100998 evpn
vrf context OVERLAY-TENANT2
  address-family ipv4 unicast
    route-target import 64500:100999
    route-target import 64500:100999 evpn

The above is a little confusing if you look at it for the first time, lets run through what is happening.

Because in our main VRF configuration we have set route-target both auto and route-target both auto evpn. This auto creates the RTs for the VRF, in the case of this environment, the first part of the RT is the ASN and the second part is the VNI assigned to the VRF. Taking this into account, we get the below RTs:

vrf OVERLAY-TENANT1 - 64500:100999
vrf OVERLAY-TENANT2 - 64500:100998

Verification

We can confirm these are the correct values by looking at the BGP table:

leaf-3(config)# show bgp ipv4 unicast 10.10.1.0 vrf OVERLAY-TENANT1
BGP routing table information for VRF <strong>OVERLAY-TENANT1</strong>, address family IPv4 Unicast
BGP routing table entry for 10.10.1.0/24, version 21
Paths: (2 available, best #1)
Flags: (0x88080012) (high32 00000000) on xmit-list, is in urib, is backup urib r
oute, is in HW
  vpn: version 37, (0x00000000100002) on xmit-list

  Advertised path-id 1, VPN AF advertised path-id 1
  Path type: internal, path is valid, is best path, no labeled nexthop, in rib
             Imported from 10.0.0.3:3:[5]:[0]:[0]:[24]:[10.10.1.0]/224 
  AS-Path: NONE, path sourced internal to AS
    10.0.0.3 (metric 81) from 10.0.0.1 (10.0.0.1)
      Origin IGP, MED not set, localpref 100, weight 0
      Received label 100999
      Extcommunity: <strong>RT:64500:100999</strong> ENCAP:8 Router MAC:5203.e101.1b08
      Originator: 10.0.0.3 Cluster list: 10.0.0.1 

  Path type: internal, path is valid, not best reason: Router Id, no labeled nex
thop
             Imported from 10.0.0.4:3:[5]:[0]:[0]:[24]:[10.10.1.0]/224 
  AS-Path: NONE, path sourced internal to AS
    10.0.0.4 (metric 81) from 10.0.0.1 (10.0.0.1)
      Origin IGP, MED not set, localpref 100, weight 0
      Received label 100999
      Extcommunity: <strong>RT:64500:100999</strong> ENCAP:8 Router MAC:5218.5fb4.1b08
      Originator: 10.0.0.4 Cluster list: 10.0.0.1 

...

Adding evpn to the end of the second route-target command, ensures that evpn routes are also pulled in with the same RT value. Just leaking the networks is not enough here as the evpn routes will point to a specific leaf where that host is.

Once this configuration is in place, we should see the leaking happening on leaf-3 in the route tables:

leaf-3# show ip route vrf <strong>OVERLAY-TENANT1</strong>

0.0.0.0/0, ubest/mbest: 1/0
    *via Null0, [1/0], 00:30:12, static
10.10.1.0/24, ubest/mbest: 1/0, attached
    *via 10.10.1.254, Vlan10, [0/0], 00:54:13, direct
10.10.1.1/32, ubest/mbest: 1/0
    *via 10.0.0.3%default, [200/0], 00:40:44, bgp-64500, internal, tag 64500, se
gid: 100999 tunnelid: 0xa000003 encap: VXLAN

10.10.1.3/32, ubest/mbest: 1/0, attached
    *via 10.10.1.3, Vlan10, [190/0], 00:51:26, hmm
10.10.1.254/32, ubest/mbest: 1/0, attached
    *via 10.10.1.254, Vlan10, [0/0], 00:54:13, local
<strong>10.20.1.0/24</strong>, ubest/mbest: 1/0, attached
    *via 10.20.1.254%OVERLAY-TENANT2, Vlan20, [20/0], 00:01:30, bgp-64500, exter
nal, tag 64500
<strong>10.20.1.1/32</strong>, ubest/mbest: 1/0
    *via 10.0.0.3%default, [200/0], 00:40:44, bgp-64500, internal, tag 64500, se
gid: 100998 (Asymmetric) tunnelid: 0xa000003 encap: VXLAN

leaf-3# show ip route vrf <strong>OVERLAY-TENANT2</strong>

0.0.0.0/0, ubest/mbest: 1/0
    *via Null0, [1/0], 00:30:08, static
<strong>10.10.1.0/24</strong>, ubest/mbest: 1/0, attached
    *via 10.10.1.254%OVERLAY-TENANT1, Vlan10, [20/0], 00:01:40, bgp-64500, exter
nal, tag 64500
<strong>10.10.1.1/32</strong>, ubest/mbest: 1/0
    *via 10.0.0.3%default, [200/0], 00:40:29, bgp-64500, internal, tag 64500, se
gid: 100999 (Asymmetric) tunnelid: 0xa000003 encap: VXLAN

10.20.1.0/24, ubest/mbest: 1/0, attached
    *via 10.20.1.254, Vlan20, [0/0], 00:54:15, direct
10.20.1.1/32, ubest/mbest: 1/0
    *via 10.0.0.3%default, [200/0], 00:40:29, bgp-64500, internal, tag 64500, se
gid: 100998 tunnelid: 0xa000003 encap: VXLAN

10.20.1.3/32, ubest/mbest: 1/0, attached
    *via 10.20.1.3, Vlan20, [190/0], 00:50:58, hmm
10.20.1.254/32, ubest/mbest: 1/0, attached
    *via 10.20.1.254, Vlan20, [0/0], 00:54:15, local

We can see from the above that within each VRF, we see the other VRFs routes, this is as per the VRF leaking we configured.

In the next part we will look at allowing external access to other resources outside of the fabric.