L2TPv3 Ethernet Tunnel VyOS to Mikrotik

I have been using powerline adapters to connect the bedroom TV, Odroid and IPTV box to my network. It worked fine, quite fast close to 1Gbit speeds, and can pass through VLANs without any problems.

That is, until one of the adapters started smelling like burnt plastic one random Sunday afternoon. I’ve decided it’s too much of a fire risk even if they honor the warranty and replace them. In the meantime there’s an Ethernet cable being squished by a door hinge.

I’ve purchased a Mikrotik hAP ax3 to use as the main access point for my flat - and the hAP ac3 that was the main AP so far, will be used as the other side of the wireless bridge. I’ve just installed the ac3 config into the ax3, then wiped the ac3.

Requirements

I like to keep the configs of the Mikrotiks as simple as possible. The main AP is connected to my core routers using BGP (I really don’t want the Mikrotik participating in IS-IS+iBGP for my own sanity). I don’t bring any VLANs to the main AP and I plan to keep it as it is, so some other device will have to be the tunnel head end.

The Mikrotik client (at the bedroom) needs to terminate some kind of L2/Ethernet tunnel, because I need to pass the IPTV VLAN as-is, as well as a few other VLANs from my network. Unfortunately it looks like the EoIP is not intercompatible with GRETAP, so I will have to try to use L2TPv3 tunnels (so they can have head-end on the VyOS kit)

BGP setup

I’ve just set up a secondary SSID on the main AP (one for 2.4GHz one for 5GHz), and made the client connect to both of these. I’ve set up IPv4/IPv6 addressing on both interfaces at both ends, and four BGP sessions (one per interface and per AFI, the sessions over 2.4GHz are configured to also prepend AS, as they’re supposed to be the backup).

Wireless trouble

One weird thing I ran into, is the fact that when putting the hAP ac3 into Station mode, the 5GHZ interface has some channel limitations beyond 5640MHz. My AP had chosen a channel beyond 5700MHz as I left the automatic selection - of course it didn’t link up. I had to limit the channel selection AP side to up to 5640MHz.

L2TPv3 tunnels

VERY IMPORTANT Due to VyOS bug T1080 - make sure that the VyOS router where the L2TPv3 tunnel is provisioned, always has a route to the tunnel destination even early on the boot process, meaning, don’t rely on routing protocols to provide a route to the tunnel destination!. Other you WILL have problems when you reboot the VyOS router, as the L2TPv3 interface will try to come up before your routing protocols do, it will fail and cause mayhem.

I sort this out by just adding some dummy interface (not advertised into BGP, and a less specific match so it’s not going to matter once routing protocols are up)

set interfaces dummy dum4 address '192.168.252.254/24'
set interfaces dummy dum4 description 'Bug T1080 Workaround'

Now the configuration I’m adding for the bridge, L2TPv3 tunnel and Ethernet interface.

fabrizzio@OSR1CR6# compare
[interfaces bridge]
+ br3 {
+     description "OSR1TV2 NYC VPN L2 VIA OSR1CR6 - OSR1M2"
+     enable-vlan
+     ipv6 {
+         address {
+             no-default-link-local
+         }
+     }
+     member {
+         interface eth14 {
+             allowed-vlan "100"
+             native-vlan "100"
+         }
+         interface l2tpeth1 {
+             allowed-vlan "100"
+             native-vlan "100"
+         }
+     }
+ }
[interfaces ethernet]
+ eth14 {
+     description "OSR1TV2 NYC VPN L2 VIA OSR1CR6 - OSR1M2"
+     mtu "1500"
+ }
[interfaces]
+ l2tpv3 l2tpeth1 {
+     description "OSR1TV2 NYC VPN L2 VIA OSR1CR6 - OSR1M2"
+     encapsulation "ip"
+     mtu "1500"
+     peer-session-id "200"
+     peer-tunnel-id "200"
+     remote "192.168.252.11"
+     session-id "200"
+     source-address "192.168.254.15"
+     tunnel-id "200"
+ }

Configuration on the Mikrotik side seemed quite straightforward (I didn’t read the manual!).

But when I used the Torch it’s not showing me the broadcast traffic I expected, instead some weird ethernet protocol 7a5a.

Reading around - you need to enable Use L2 Specific Sublayer on the Mikrotik tunnel endpoint.

After that, Torch looks alright, I can see the VRRP packets coming in :D

Mikrotik Bridge setup

This is the part I dreaded the most. I’ve set up a simple bridge, no STP, no VLAN filtering:

Then just added an Ethernet port, and the L2TPv3 tunnel to the bridge. Completely default configs.

Results

I’ve connected my laptop to that port on the Mikrotik and it grabbed an IP right away. I ran a speed test, with the below results.

I think it’s not bad, considering that the L2TPv3 tunnel is originating on an hAP ac3, traversing a 5GHz link to the hAP ax3, then through a 1GbE link, entering my VyOS core - it will traverse the 2x2.5GbE core to get to the VPN router, then the traffic will be tunneled again through the 2x2.5GbE core to AS203528, where it’s tunneled over Internet via Wireguard to a Linode at NYC, where it finally exits. Yay! Finally I can stream the TV service which rhymes with some plumbing thing.

I’ve just gotta repeat this for the 2nd VLAN (which allows access to my SMB server) and for the IPTV VLAN. I will just set up the L2TPv3 head-end for IPTV on a different VyOS router, on a different server, to ensure I can at least get some service on my TV in case one of my servers decide to release the magic smoke.