VyOS Add path from single route reflector
My network AS203528 has eBGP Multihop peering to a couple of route collectors - both from BGP.tools and NLNOG
These peerings are off a couple of my routers OSR1GLASS1 and OSR2GLASS1 - in both Ostrava sites, and their only purpose is for these route collection peerings and for the Looking glass (Hyperglass). Being accessible by the Looking glass means they’re very firewalled - to prevent any lateral movement just in case they would be compromised by a vulnerability on Hyperglass.
I make heavy use of addpath on my network, every AS203528 router has this configured (tx-all) and each reflector has the same set of routes. (So 3x full routes from each transit plus all the IXP and peering routes)
1
2
3
4
5
Neighbor V AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down State/PfxRcd PfxSnt Desc
192.168.254.150 4 203528 63435172 70296 175726069 0 0 06w6d19h 785102 1 To OSR1RR1
192.168.254.151 4 203528 63382540 70296 175726069 0 0 06w6d19h 785102 1 To OSR1RR2
192.168.254.152 4 203528 63425417 70296 175726069 0 0 06w6d19h 785102 1 To OSR1RR3
192.168.254.153 4 203528 63980723 70249 175726069 0 0 02w5d04h 785102 1 To OSR2RR1
The problem is that if I enable addpath in tx-all mode in the eBGP peerings towards the route collectors, it will send 4 copies of each route (due to 4xRRs), and if I enable it as addpath-tx-per-as some information will be missing.
I’d rather only send routes from a single RR to those collectors - however I don’t want to drop the other three iBGP peerings to RRs to keep that redundancy (I could technically do it as these routers are not carrying any traffic other than to/from themselves).
First attempts
I had first tried couple of things - I thought either filtering by (RR) peer IP would work - it didn’t (Zero routes matched). I also tried filtering by ip route-source (being RR IP) - still no matches.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
fabrizzio@OSR1GLASS1# show policy route-map EXPORT_FROM_SINGLE_RR
rule 10 {
action permit
match {
peer 192.168.254.150
}
}
rule 11 {
action permit
match {
ip {
route-source {
prefix-list RR1
}
}
}
}
rule 20 {
action deny
}
[edit]
fabrizzio@OSR1GLASS1# show policy prefix-list RR1
rule 1 {
action permit
prefix 192.168.254.150/32
}
[edit]
1
2
Neighbor V AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down State/PfxRcd PfxSnt Desc
2a0c:2f07:9459::b7 4 212232 93753 59816781 175758122 0 0 00:33:36 0 0 BGP.Tools Peering
Dirty fix
AS203528 is unfortunately getting more and more workaround and quick/dirty fixes. Luckily it’s just my network :) If someone else had to manage this I would fear for their sanity :)
Before just disabling (at RR side) those three extra RR iBGP peerings I wanted to try to fix it the way I always fix things in AS203528.
For me what’s important is to only mess with the GLASS routers - I don’t want to touch the RRs in any way.
Currently, on OSR1GLASS1, the iBGP peering to each of the RRs is like this:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
set protocols bgp neighbor 192.168.254.150 address-family ipv4-unicast addpath-tx-all
set protocols bgp neighbor 192.168.254.150 address-family ipv4-unicast nexthop-self
set protocols bgp neighbor 192.168.254.150 address-family ipv4-unicast route-map import 'prevent_ibgp_blackholing'
set protocols bgp neighbor 192.168.254.150 address-family ipv4-unicast soft-reconfiguration inbound
set protocols bgp neighbor 192.168.254.150 address-family ipv6-unicast addpath-tx-all
set protocols bgp neighbor 192.168.254.150 address-family ipv6-unicast nexthop-self
set protocols bgp neighbor 192.168.254.150 address-family ipv6-unicast prefix-list import 'DENY-DEFAULTSv6'
set protocols bgp neighbor 192.168.254.150 address-family ipv6-unicast route-map export 'ibgp_ula_nh'
set protocols bgp neighbor 192.168.254.150 address-family ipv6-unicast route-map import 'prevent_ibgp_blackholing'
set protocols bgp neighbor 192.168.254.150 address-family ipv6-unicast soft-reconfiguration inbound
set protocols bgp neighbor 192.168.254.150 description 'To OSR1RR1'
set protocols bgp neighbor 192.168.254.150 remote-as 'internal'
set protocols bgp neighbor 192.168.254.150 timers connect '1'
set protocols bgp neighbor 192.168.254.150 update-source 'dum0'
I have the import map prevent_ibgp_blackholing for self-explanatory purpose
1
2
3
4
5
set policy route-map prevent_ibgp_blackholing rule 10 action 'deny'
set policy route-map prevent_ibgp_blackholing rule 10 match ip nexthop type 'blackhole'
set policy route-map prevent_ibgp_blackholing rule 20 action 'deny'
set policy route-map prevent_ibgp_blackholing rule 20 match ipv6 nexthop type 'blackhole'
set policy route-map prevent_ibgp_blackholing rule 30 action 'permit'
I’m just going to create a different prevent_ibgp_blackholing map per RR to map a different community for each RR
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
set policy route-map prevent_ibgp_blackholing_RR1 rule 10 action 'deny'
set policy route-map prevent_ibgp_blackholing_RR1 rule 10 match ip nexthop type 'blackhole'
set policy route-map prevent_ibgp_blackholing_RR1 rule 20 action 'deny'
set policy route-map prevent_ibgp_blackholing_RR1 rule 20 match ipv6 nexthop type 'blackhole'
set policy route-map prevent_ibgp_blackholing_RR1 rule 30 action 'permit'
set policy route-map prevent_ibgp_blackholing_RR1 rule 30 set large-community add 203528:999:1
set policy route-map prevent_ibgp_blackholing_RR2 rule 10 action 'deny'
set policy route-map prevent_ibgp_blackholing_RR2 rule 10 match ip nexthop type 'blackhole'
set policy route-map prevent_ibgp_blackholing_RR2 rule 20 action 'deny'
set policy route-map prevent_ibgp_blackholing_RR2 rule 20 match ipv6 nexthop type 'blackhole'
set policy route-map prevent_ibgp_blackholing_RR2 rule 30 action 'permit'
set policy route-map prevent_ibgp_blackholing_RR2 rule 30 set large-community add 203528:999:2
set policy route-map prevent_ibgp_blackholing_RR3 rule 10 action 'deny'
set policy route-map prevent_ibgp_blackholing_RR3 rule 10 match ip nexthop type 'blackhole'
set policy route-map prevent_ibgp_blackholing_RR3 rule 20 action 'deny'
set policy route-map prevent_ibgp_blackholing_RR3 rule 20 match ipv6 nexthop type 'blackhole'
set policy route-map prevent_ibgp_blackholing_RR3 rule 30 action 'permit'
set policy route-map prevent_ibgp_blackholing_RR3 rule 30 set large-community add 203528:999:3
set policy route-map prevent_ibgp_blackholing_RR4 rule 10 action 'deny'
set policy route-map prevent_ibgp_blackholing_RR4 rule 10 match ip nexthop type 'blackhole'
set policy route-map prevent_ibgp_blackholing_RR4 rule 20 action 'deny'
set policy route-map prevent_ibgp_blackholing_RR4 rule 20 match ipv6 nexthop type 'blackhole'
set policy route-map prevent_ibgp_blackholing_RR4 rule 30 action 'permit'
set policy route-map prevent_ibgp_blackholing_RR4 rule 30 set large-community add 203528:999:4
set policy large-community-list RR1 rule 1 action permit
set policy large-community-list RR1 rule 1 regex 203528:999:1
set policy large-community-list RR2 rule 1 action permit
set policy large-community-list RR2 rule 1 regex 203528:999:2
set policy large-community-list RR3 rule 1 action permit
set policy large-community-list RR3 rule 1 regex 203528:999:3
set policy large-community-list RR4 rule 1 action permit
set policy large-community-list RR4 rule 1 regex 203528:999:4
Then I can apply it per-RR
1
2
3
4
set protocols bgp neighbor 192.168.254.150 address-family ipv6-unicast route-map import prevent_ibgp_blackholing_RR1
set protocols bgp neighbor 192.168.254.151 address-family ipv6-unicast route-map import prevent_ibgp_blackholing_RR2
set protocols bgp neighbor 192.168.254.152 address-family ipv6-unicast route-map import prevent_ibgp_blackholing_RR3
set protocols bgp neighbor 192.168.254.153 address-family ipv6-unicast route-map import prevent_ibgp_blackholing_RR4
And I will change the export route map (for eBGP collectors) to filter by this community
1
2
3
set policy route-map EXPORT_FROM_SINGLE_RR rule 10 action 'permit'
set policy route-map EXPORT_FROM_SINGLE_RR rule 10 match large-community large-community-list RR1
set policy route-map EXPORT_FROM_SINGLE_RR rule 20 action 'deny'
Yay! This worked!
1
2
3
4
5
6
7
8
9
10
11
12
13
IPv6 Unicast Summary (VRF default):
BGP router identifier 192.168.248.90, local AS number 203528 vrf-id 0
BGP table version 176287252
RIB entries 394841, using 36 MiB of memory
Peers 6, using 121 KiB of memory
Neighbor V AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down State/PfxRcd PfxSnt Desc
192.168.254.150 4 203528 63480280 70353 176287252 0 0 06w6d20h 784950 1 To OSR1RR1
192.168.254.151 4 203528 63427527 70353 176287252 0 0 06w6d20h 784950 1 To OSR1RR2
192.168.254.152 4 203528 63470548 70353 176287252 0 0 06w6d20h 784950 1 To OSR1RR3
192.168.254.153 4 203528 64025775 70306 176287252 0 0 02w5d05h 784950 1 To OSR2RR1
2001:7b8:62b:1:0:d4ff:fe72:7848 4 199036 140596 62318519 176287252 0 0 00:02:39 0 784898 NLNOG Peering
2a0c:2f07:9459::b7 4 212232 93799 60353308 176287252 0 0 00:02:39 0 784898 BGP.Tools Peering
The small amount of differences in routes from iBGP RR peerings vs what’s sent out to collectors is that I avoid sending internal routes to them.