VCF 4 Multi AZ Networking

In a VCF 4 multi AZ deployment, before stretching the cluster to AZ2, there's some manual tasks required for the T0 to peer with the ToR switches in AZ2. The process is well documented in the NSX-T Data Center Configuration for Availability Zone 2 for the Management Domain in Region A on the VVD6 documentation pages.

Although afterwards, we noticed the T0 Gateway, was learning routes from AZ1, and advertising them to the ToR in AZ2. The ToR switch in AZ2 had a more preferred path for those routes, so in theory traffic for those external networks should never be routed through the T0. Unless of course the aforementioned preferred path goes away! Check out What Is BGP? | BGP Routing Explained from Cloudflare for more info.

You can check which routes are being advertised to the neighbor by SSH'ing to each Edge Node, selecting the T0 SR VRF, and running the following command:

1en01(tier0_sr)> get bgp neighbor <BGP neighbor IP> advertised-routes

Asking smarter people than myself, it turns out the T0 Gateway uses BGP just like any other router, and a function of BGP is to take routes learnt from one ASN, and advertise that to other ASN's. It all made sense now because the ToRs in each AZ had their own Autonomous Number. So it was working as designed.

I'm not sure why this is something that might be needed for a T0, but in our case, we didn't see a need for this ever to happen, so be to on the safe I created a route map, and set the Community to "No Export"

To create this route map within NSX-T, edit the T0 Gateway.

Expand the ROUTING section, and next to Route Maps, click Set, or a number if there's existing route maps.

In the Set Route Maps window, click ADD ROUTE MAP. Add or update the following two route maps:

Apply and save the route maps, then expand the BGP section, and click on the BGP Neighbors link.

Edit the neighbors and click on the Route Filter. In the Set Route Filter window, edit the route filter, and under In Filter, click Configure. Select the appropriate route filter depending on if the neighbor is in AZ1 or AZ2.

After applying that, switch back to the Edge Nodes, and confirm it's only advertising networks defined within NSX-T.

1en01(tier0_sr)> get bgp neighbor <BGP neighbor IP> advertised-routes