Network fun!
As it’s almost Australian Tax season (though I don’t know whether it’ll affect me much), I decided to plop the money down for some decent networking gear. Most of our house is wired terribly: as cheaply as possible, because we rent so investing in good infrastructure is just not a good use of funds.
This leaves us with a flat LAN, with switches all over the place. The switches are also as cheap as I could get… opp-shop routers that had Gigabit ports were my sole criteria, and our venerable D6200 doing WAP duties. It’s been bugging me for a while now that with an 8 year old on a Windows machine, malware is basically guaranteed to end up inside our LAN at some point. I have most of the gear to limit the damage that can do, and so I figured that the business justification is there, and that leads us to the above paragraph.
The cheapest stuff I could find that I knew would do the job was a couple of pieces from Ubiquiti networks: a UniFi switch (the smallest, self-powered unit they have) and a UniFi AP (I spent a little more for the one that had POE compatibility with the switch to avoid yet another wall-wart). I spent the weekend rearranging things, with only the faintest of a clue what I’m doing (networking is not my strong suit at all these days).
So I have a separate VLAN for the “home” stuff, and the old WLAN name is strapped to that VLAN also. The untagged LAN is limited to a couple of switch ports, and a separate WLAN for my Macbook. The router is VLAN aware, and connected to a trunk port. I had to tear a bunch of stuff out of dhcpd’s configuration to make it work correctly, and instead of using 10.0.0.0/8 (the 10 netblock is aesthetically pleasing to me), I switched to several /23s, though I’m probably going to switch to a couple of /22s instead, in the same netblock.
Why so much address space? Well, I’m not smart enough to be aware of any reason not to do it for starters (there are potential issues with things like broadcasts, but they don’t, to my knowledge, go away just by using smaller TCP/IP subnets), but also I find it eases the administration burden to have them clearly delineated. On the old arrangement, 10.0.0.0/24 was the “permanent” hosts, I’d assign static assignments in that netblock. 10.0.1.0/24 was dynamic DHCP. 10.0.2.0/24 was the “games” network, with them getting special treatment for QoS for non-HTTP traffic, as well as having carte blanche to create UPNP mappings for themselves only. There’s an additional netblock for VMs, but it causes more trouble than it’s worth.
I had originally planned to have each of these subnets on their own VLAN. Why not, right? Well, because it turns out that I have lots of stuff that needs to straddle the networks - primarily our fileserver. It has a Samba server on it, which each of the Windows machines has access to for backups. My MBP also needs to access it for Time Machine backups. Finally, it uses Plex, which the PS4 accesses directly. So that’s three VLANs it has to be present on.
I could just route between them, but unfortunately the UniFi switches are only L2 and the APU that is our network router is only good for about 300mbps intra-vlan routing in my testing. I suspect I could double that by using two separate hardware interfaces and two switch ports, but that brings its own problems (chances of routing loops, I’m under the impression), but it’s still disappointing and I feel like it would drive up network latency while a large copy is ongoing.
But there are further issues also: the Steam Link. I adore this thing, but it’s a “gaming” device (in that it’s on an unmanaged switch connected to the TV, so if there’s any network it’s going to be in, it’ll be in the same one as the PS4 - and before you ask, Wifi is not an acceptable solution in these cases). It apparently expects to be in the same broadcast domain as the Windows machines it streams from though, so that means it also belongs in the Windows group. If I had unlimited funds, this is trivially solvable with another managed switch (or when we buy our own house, a single really big managed switch and individual ethernet drops), but at the moment I’m stuck.
So it seemed to make the most sense to go back to a hybrid of VLANs and my original setup, where I use VLANs to segregate things that really need to be separate (work and home mainly, possibly a third wireless network with zero LAN privileges if we get into IoT at some point) and then just made-up network boundaries for my own convenience for the rest. One minor improvement is the above-mentioned VMs issue, by not having everything in 10.0.0.0/8 things like VMs and LXC suddenly start working on their own.
I managed to lock myself out of the router only twice, which is always nice. I don’t have an RS232 cable on anything but my desktop (several meters from the router in most situations), so after the second time I set up the second interface with static IPs in the 192.168.0.0/24 block so I could get back in if I locked myself out.
One other issue is DNS: “router” for example will mean different things depending on which network I’m in, and I don’t want to have to remember that. I already run a form of split-horizon DNS, in that my domain varies greatly depending on whether you’re inside or outside of my LAN, and splitting that again did not sound like fun. So what I did instead was have two different sets of subdomains with the relevant DNS records present:
router A 10.0.2.1
router.labs A 10.0.0.1
Then some magic in the DHCP configuration:
option domain-search "labs.home.fwaggle.org", "home.fwaggle.org", "fwaggle.org";
and DNS resolution mostly works correctly, ssh router works regardless of which machine I’m on. Edit: apparently this doesn’t work on Windows clients - thankfully there are no Windows machines on the work network, and the Home network has no concept of the work network.
On the whole I’m rather happy with it. I still have some firewall rules to write to actually split the networks up (more opportunities to lock myself out I guess?), and it’ll be nice if I get a switch that can do the L3 routing for me. At the moment I’ve been toying with having the file server be VLAN-aware as well… it does effectively punch a gaping hole between the networks, but given that my threat model is mainly “minecraft related malware” and not really a persistent attacker (I’m probably boned if they get a foothold in my network regardless, due to the amount of convenience we’ve set up) it probably doesn’t matter much.
I’m rather impressed by the Ubiquiti gear. It has a couple of tiny annoyances, such as the L2-only issue, the fact that the WAP panics and gives up if it can’t get an IP address, and that the interface really wants you to drink the kool-aid and buy their USG as well. Not entirely sure how I feel about the steaming pile of Java and MongoDB either. If I’d known more, there was probably gear I could have bought that would have done a better job (I’d imagine there’s something out there in the same price range that’d do L3 routing a well) but I like this stuff.
fwagglePublished:
Modified:
Filed under:
Location:
Horsham, VIC, AustraliaNavigation: Older Entry Newer Entry