My network AS203528 has eBGP Multihop peering to a couple of route collectors - both from BGP.tools and NLNOG

These peerings are off a couple of my routers OSR1GLASS1 and OSR2GLASS1 - in both Ostrava sites, and their only purpose is for these route collection peerings and for the Looking glass (Hyperglass). Being accessible by the Looking glass means they're very firewalled - to prevent any lateral movement just in case they would be compromised by a vulnerability on Hyperglass.

I make heavy use of addpath on my network, every AS203528 router has this configured (tx-all) and each reflector has the same set of routes. (So 3x full routes from each transit plus all the IXP and peering routes)

1Neighbor                        V         AS   MsgRcvd   MsgSent   TblVer  InQ OutQ  Up/Down State/PfxRcd   PfxSnt Desc
2192.168.254.150                 4     203528  63435172     70296 175726069    0    0 06w6d19h       785102        1 To OSR1RR1
3192.168.254.151                 4     203528  63382540     70296 175726069    0    0 06w6d19h       785102        1 To OSR1RR2
4192.168.254.152                 4     203528  63425417     70296 175726069    0    0 06w6d19h       785102        1 To OSR1RR3
5192.168.254.153                 4     203528  63980723     70249 175726069    0    0 02w5d04h       785102        1 To OSR2RR1

The problem is that if I enable addpath in tx-all mode in the eBGP peerings towards the route collectors, it will send 4 copies of each route (due to 4xRRs), and if I enable it as addpath-tx-per-as some information will be missing.

I'd rather only send routes from a single RR to those collectors - however I don't want to drop the other three iBGP peerings to RRs to keep that redundancy (I could technically do it as these routers are not carrying any traffic other than to/from themselves).

First attempts

I had first tried couple of things - I thought either filtering by (RR) peer IP would work - it didn't (Zero routes matched). I also tried filtering by ip route-source (being RR IP) - still no matches.

 1fabrizzio@OSR1GLASS1# show  policy route-map EXPORT_FROM_SINGLE_RR
 2 rule 10 {
 3     action permit
 4     match {
 5         peer 192.168.254.150
 6     }
 7 }
 8 rule 11 {
 9     action permit
10     match {
11         ip {
12             route-source {
13                 prefix-list RR1
14             }
15         }
16     }
17 }
18 rule 20 {
19     action deny
20 }
21[edit]
22fabrizzio@OSR1GLASS1# show  policy prefix-list RR1
23 rule 1 {
24     action permit
25     prefix 192.168.254.150/32
26 }
27[edit]

1Neighbor                        V         AS   MsgRcvd   MsgSent   TblVer  InQ OutQ  Up/Down State/PfxRcd   PfxSnt Desc
22a0c:2f07:9459::b7              4     212232     93753  59816781 175758122    0    0 00:33:36            0        0 BGP.Tools Peering

Dirty fix

AS203528 is unfortunately getting more and more workaround and quick/dirty fixes. Luckily it's just my network :) If someone else had to manage this I would fear for their sanity :)

Before just disabling (at RR side) those three extra RR iBGP peerings I wanted to try to fix it the way I always fix things in AS203528.

The usual way

For me what's important is to only mess with the GLASS routers - I don't want to touch the RRs in any way.

Currently, on OSR1GLASS1, the iBGP peering to each of the RRs is like this:

 1set protocols bgp neighbor 192.168.254.150 address-family ipv4-unicast addpath-tx-all
 2set protocols bgp neighbor 192.168.254.150 address-family ipv4-unicast nexthop-self
 3set protocols bgp neighbor 192.168.254.150 address-family ipv4-unicast route-map import 'prevent_ibgp_blackholing'
 4set protocols bgp neighbor 192.168.254.150 address-family ipv4-unicast soft-reconfiguration inbound
 5set protocols bgp neighbor 192.168.254.150 address-family ipv6-unicast addpath-tx-all
 6set protocols bgp neighbor 192.168.254.150 address-family ipv6-unicast nexthop-self
 7set protocols bgp neighbor 192.168.254.150 address-family ipv6-unicast prefix-list import 'DENY-DEFAULTSv6'
 8set protocols bgp neighbor 192.168.254.150 address-family ipv6-unicast route-map export 'ibgp_ula_nh'
 9set protocols bgp neighbor 192.168.254.150 address-family ipv6-unicast route-map import 'prevent_ibgp_blackholing'
10set protocols bgp neighbor 192.168.254.150 address-family ipv6-unicast soft-reconfiguration inbound
11set protocols bgp neighbor 192.168.254.150 description 'To OSR1RR1'
12set protocols bgp neighbor 192.168.254.150 remote-as 'internal'
13set protocols bgp neighbor 192.168.254.150 timers connect '1'
14set protocols bgp neighbor 192.168.254.150 update-source 'dum0'

I have the import map prevent_ibgp_blackholing for self-explanatory purpose

1set policy route-map prevent_ibgp_blackholing rule 10 action 'deny'
2set policy route-map prevent_ibgp_blackholing rule 10 match ip nexthop type 'blackhole'
3set policy route-map prevent_ibgp_blackholing rule 20 action 'deny'
4set policy route-map prevent_ibgp_blackholing rule 20 match ipv6 nexthop type 'blackhole'
5set policy route-map prevent_ibgp_blackholing rule 30 action 'permit'

I'm just going to create a different prevent_ibgp_blackholing map per RR to map a different community for each RR

 1set policy route-map prevent_ibgp_blackholing_RR1 rule 10 action 'deny'
 2set policy route-map prevent_ibgp_blackholing_RR1 rule 10 match ip nexthop type 'blackhole'
 3set policy route-map prevent_ibgp_blackholing_RR1 rule 20 action 'deny'
 4set policy route-map prevent_ibgp_blackholing_RR1 rule 20 match ipv6 nexthop type 'blackhole'
 5set policy route-map prevent_ibgp_blackholing_RR1 rule 30 action 'permit'
 6set policy route-map prevent_ibgp_blackholing_RR1 rule 30 set large-community add 203528:999:1
 7
 8set policy route-map prevent_ibgp_blackholing_RR2 rule 10 action 'deny'
 9set policy route-map prevent_ibgp_blackholing_RR2 rule 10 match ip nexthop type 'blackhole'
10set policy route-map prevent_ibgp_blackholing_RR2 rule 20 action 'deny'
11set policy route-map prevent_ibgp_blackholing_RR2 rule 20 match ipv6 nexthop type 'blackhole'
12set policy route-map prevent_ibgp_blackholing_RR2 rule 30 action 'permit'
13set policy route-map prevent_ibgp_blackholing_RR2 rule 30 set large-community add 203528:999:2
14
15set policy route-map prevent_ibgp_blackholing_RR3 rule 10 action 'deny'
16set policy route-map prevent_ibgp_blackholing_RR3 rule 10 match ip nexthop type 'blackhole'
17set policy route-map prevent_ibgp_blackholing_RR3 rule 20 action 'deny'
18set policy route-map prevent_ibgp_blackholing_RR3 rule 20 match ipv6 nexthop type 'blackhole'
19set policy route-map prevent_ibgp_blackholing_RR3 rule 30 action 'permit'
20set policy route-map prevent_ibgp_blackholing_RR3 rule 30 set large-community add 203528:999:3
21
22set policy route-map prevent_ibgp_blackholing_RR4 rule 10 action 'deny'
23set policy route-map prevent_ibgp_blackholing_RR4 rule 10 match ip nexthop type 'blackhole'
24set policy route-map prevent_ibgp_blackholing_RR4 rule 20 action 'deny'
25set policy route-map prevent_ibgp_blackholing_RR4 rule 20 match ipv6 nexthop type 'blackhole'
26set policy route-map prevent_ibgp_blackholing_RR4 rule 30 action 'permit'
27set policy route-map prevent_ibgp_blackholing_RR4 rule 30 set large-community add 203528:999:4
28
29set policy large-community-list RR1 rule 1 action permit
30set policy large-community-list RR1 rule 1 regex 203528:999:1
31
32set policy large-community-list RR2 rule 1 action permit
33set policy large-community-list RR2 rule 1 regex 203528:999:2
34
35set policy large-community-list RR3 rule 1 action permit
36set policy large-community-list RR3 rule 1 regex 203528:999:3
37
38set policy large-community-list RR4 rule 1 action permit
39set policy large-community-list RR4 rule 1 regex 203528:999:4

Then I can apply it per-RR

1set protocols bgp neighbor 192.168.254.150 address-family ipv6-unicast route-map import prevent_ibgp_blackholing_RR1
2set protocols bgp neighbor 192.168.254.151 address-family ipv6-unicast route-map import prevent_ibgp_blackholing_RR2
3set protocols bgp neighbor 192.168.254.152 address-family ipv6-unicast route-map import prevent_ibgp_blackholing_RR3
4set protocols bgp neighbor 192.168.254.153 address-family ipv6-unicast route-map import prevent_ibgp_blackholing_RR4

And I will change the export route map (for eBGP collectors) to filter by this community

1set policy route-map EXPORT_FROM_SINGLE_RR rule 10 action 'permit'
2set policy route-map EXPORT_FROM_SINGLE_RR rule 10 match large-community large-community-list RR1
3set policy route-map EXPORT_FROM_SINGLE_RR rule 20 action 'deny'

Yay! This worked!

 1IPv6 Unicast Summary (VRF default):
 2BGP router identifier 192.168.248.90, local AS number 203528 vrf-id 0
 3BGP table version 176287252
 4RIB entries 394841, using 36 MiB of memory
 5Peers 6, using 121 KiB of memory
 6
 7Neighbor                        V         AS   MsgRcvd   MsgSent   TblVer  InQ OutQ  Up/Down State/PfxRcd   PfxSnt Desc
 8192.168.254.150                 4     203528  63480280     70353 176287252    0    0 06w6d20h       784950        1 To OSR1RR1
 9192.168.254.151                 4     203528  63427527     70353 176287252    0    0 06w6d20h       784950        1 To OSR1RR2
10192.168.254.152                 4     203528  63470548     70353 176287252    0    0 06w6d20h       784950        1 To OSR1RR3
11192.168.254.153                 4     203528  64025775     70306 176287252    0    0 02w5d05h       784950        1 To OSR2RR1
122001:7b8:62b:1:0:d4ff:fe72:7848 4     199036    140596  62318519 176287252    0    0 00:02:39            0   784898 NLNOG Peering
132a0c:2f07:9459::b7              4     212232     93799  60353308 176287252    0    0 00:02:39            0   784898 BGP.Tools Peering

The small amount of differences in routes from iBGP RR peerings vs what's sent out to collectors is that I avoid sending internal routes to them.