VPS Route reduction project
Route reduction?
Reason is simple. I have three small VMs (NYC1BR1, LUX1BR1, AMS1BR1) which have only 2GB of RAM available. Each of these VMs receive full routes from the VPS provider, and they also had all the other routes from the other VMs. It doesn’t seem such a good idea to have these VMs, with a small amount of ram, storing 3 full IPv6 BGP tables, they’re already running somewhat low in RAM.
Considering I have a full mesh of tunnels for IPv6 connectivity among all my AS203528 routers, and I have no services directly connected to either of those three VMs, then the idea is to have those VMs only have transit routes received from that provider, plus my internal routes only. I’ll make sure that we never have the scenario of “IPv6 traffic traversing a VM that does not have a route for it”.
Adding a community to transit routes
I decided to add large community “203528:1:1” to all routes I receive via transit.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
set policy large-community-list VIA_TRANSIT rule 10 action 'permit'
set policy large-community-list VIA_TRANSIT rule 10 regex '203528:1:1'
set policy route-map IMPORT_FRC_AMS_002 rule 40 action 'deny'
set policy route-map IMPORT_FRC_AMS_002 rule 40 description 'Drop anything longer than /48'
set policy route-map IMPORT_FRC_AMS_002 rule 40 match ipv6 address prefix-list 'IPV6_SMALL_PREFIX'
set policy route-map IMPORT_FRC_AMS_002 rule 50 action 'deny'
set policy route-map IMPORT_FRC_AMS_002 rule 50 description 'Drop long AS paths'
set policy route-map IMPORT_FRC_AMS_002 rule 50 match as-path 'LONG_AS'
set policy route-map IMPORT_FRC_AMS_002 rule 60 action 'deny'
set policy route-map IMPORT_FRC_AMS_002 rule 60 description 'Drop Bogons'
set policy route-map IMPORT_FRC_AMS_002 rule 60 match ipv6 address prefix-list 'IPV6_BOGONS'
set policy route-map IMPORT_FRC_AMS_002 rule 70 action 'deny'
set policy route-map IMPORT_FRC_AMS_002 rule 70 description 'Drop my own routes'
set policy route-map IMPORT_FRC_AMS_002 rule 70 match ipv6 address prefix-list 'IPV6_MY_OWN_PREFIX'
set policy route-map IMPORT_FRC_AMS_002 rule 100 action 'deny'
set policy route-map IMPORT_FRC_AMS_002 rule 100 match rpki 'invalid'
set policy route-map IMPORT_FRC_AMS_002 rule 110 action 'permit'
set policy route-map IMPORT_FRC_AMS_002 rule 110 description 'Catch all'
set policy route-map IMPORT_FRC_AMS_002 rule 110 set large-community add '203528:1:1'
set policy route-map IMPORT_FRC_AMS_002 rule 110 set local-preference '90'
After deploying:
1
2
3
4
5
6
7
8
9
10
11
12
fabrizzio@AMS1BR1:~$ show bgp ipv6 2600::
BGP routing table entry for 2600::/48, version 32241838
Paths: (1 available, best #1, table default)
34872 1239
2a0c:b640:11::ffff from 2a0c:b640:11::ffff (10.192.255.193)
(fe80::cc82:f5ff:fe1c:5819) (used)
Origin IGP, metric 0, localpref 90, valid, external, bestpath-from-AS 34872, best (First path received), rpki validation-state: not found
Community: 65101:1082 65102:1000 65103:276 65104:150
Large Community: 34872:10:0 203528:1:1
AddPath ID: RX 0, TX-All 577767 TX-Best-Per-AS 0
Advertised to: 192.168.248.12 192.168.248.13 192.168.248.14 192.168.248.15 192.168.248.16 192.168.248.90 192.168.248.91
Last update: Thu Jun 29 16:39:23 2023
Not sending these routes to other Transit-only VMs.
For this I had to create a new route-map on each of my transit VMs to deny those routes that have transit community:
1
2
3
4
set policy route-map ibgp_ula_nh_transit rule 9 action 'deny'
set policy route-map ibgp_ula_nh_transit rule 9 match large-community large-community-list 'VIA_TRANSIT'
set policy route-map ibgp_ula_nh_transit rule 10 action 'permit'
set policy route-map ibgp_ula_nh_transit rule 10 set ipv6-next-hop global 'fc0e:8f02:21d0:ffff::10'
Compared to the old route-map:
1
2
set policy route-map ibgp_ula_nh rule 10 action 'permit'
set policy route-map ibgp_ula_nh rule 10 set ipv6-next-hop global 'fc0e:8f02:21d0:ffff::10'
And just configure that route-map as export on the iBGP sessions facing the other VPS VMs.
Blackhole prevention
Even though I have full mesh of tunnels for IPv6 connectivity between routers, and the IGP metrics I put there should make traffic to router X always use a tunnel directly to X (instead of taking a tunnel to Y and from then to X), I still want to be completely sure that no IPv6 traffic traffic not destined to the VPS VMs will be inspected by those.
For that purpose, on the three routers that are part of the IPv6 tunnel mesh but do not hold all the copies of routing tables, I enabled the IS-IS overload bit.
1
set protocols isis set-overload-bit
That way, tunnels terminating there will never be used for transit traffic to a different router (they are fully meshed for a reason).
Results
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
IPv6 Unicast Summary (VRF default):
BGP router identifier 192.168.248.10, local AS number 203528 vrf-id 0
BGP table version 36188466
RIB entries 348056, using 64 MiB of memory
Peers 12, using 8682 KiB of memory
Neighbor V AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down State/PfxRcd PfxSnt Desc
192.168.248.11 4 203528 3571763 9313096 0 0 0 03w0d01h 3 2179 To NYC1BR1
192.168.248.12 4 203528 2792352 11942257 0 0 0 01w6d08h 8 185807 To OSR1BR1
192.168.248.13 4 203528 2871855 11844769 0 0 0 01w6d08h 962 185807 To OSR1BR2
192.168.248.14 4 203528 873696 11984491 0 0 0 01w6d08h 81 185807 To OSR2BR1
192.168.248.15 4 203528 31803 11845179 0 0 0 01w6d08h 6 185807 To OSR2BR2
192.168.248.16 4 203528 30344 11749802 0 0 0 01w6d08h 4 185807 To OSR1BR3
192.168.248.18 4 203528 4230 16984 0 0 0 1d05h31m 3 2179 To LUX1BR1
192.168.248.90 4 203528 30315 11654475 0 0 0 03w0d01h 1 185807 To OSR1GLASS1
192.168.248.91 4 203528 30315 11654469 0 0 0 03w0d01h 1 185807 To OSR2GLASS1
2a0c:b640:11::ffff 4 34872 14567492 544268 0 0 0 02w4d21h 183628 1 Servperso Transit
2a0c:b641:701:0:a5:20:2409:1 4 202409 160597 60684 0 0 0 5d18h16m 1186 1 LocIX DUS RS1
2a0c:b641:701:0:a5:20:2409:2 4 202409 162836 60665 0 0 0 5d18h15m 990 1 LocIX DUS RS2
Looks much better now, compared to a router where I still hold multiple full tables:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
IPv6 Unicast Summary (VRF default):
BGP router identifier 192.168.248.16, local AS number 203528 vrf-id 0
BGP table version 27201097
RIB entries 348239, using 64 MiB of memory
Peers 9, using 6511 KiB of memory
Neighbor V AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down State/PfxRcd PfxSnt Desc
192.168.248.10 4 203528 8245023 19243 0 0 0 01w6d08h 185806 4 To AMS1BR1
192.168.248.11 4 203528 3314909 19243 0 0 0 01w6d08h 182993 4 To NYC1BR1
192.168.248.12 4 203528 19383 19248 0 0 0 01w6d08h 8 4 To OSR1BR1
192.168.248.13 4 203528 358776 19245 0 0 0 01w6d08h 970 4 To OSR1BR2
192.168.248.14 4 203528 576986 19242 0 0 0 01w6d08h 81 4 To OSR2BR1
192.168.248.15 4 203528 19415 19244 0 0 0 01w6d08h 6 4 To OSR2BR2
192.168.248.18 4 203528 1171984 10633 0 0 0 2d05h57m 183111 4 To LUX1BR1
192.168.248.90 4 203528 19242 19243 0 0 0 01w6d08h 1 4 To OSR1GLASS1
192.168.248.91 4 203528 19242 19245 0 0 0 01w6d08h 1 4 To OSR2GLASS1