WiFi – Somewhere Out There

Homelab Chronicles 15: Three VLANs to Rule Them All

I finally split out my IoT devices onto their own VLAN. That’s it that’s the post. See you all next time!

No, of course I gotta say more. I’m a wordy MFer, after all! But yes, I finally put the IoT stuff on its own network. I should’ve done it a long time ago. It’s been on my list of Homelab to-do’s for like 4-5yrs. A good chunk of not doing it was because I wasn’t sure how to implement it.

I had experimented with VLANs a couple years ago, then doing some basic firewalling to make sure the VLANs and the devices on them couldn’t communicate. I set up a couple VLANs and then created an “RFC 1918” firewall rule to disallow communications between VLANs and their associated subnets. I even have a couple ports tagged on my switches for network isolation purposes.

Lately at work, however, I’ve had some opportunities to play around with and troubleshoot some of the network and firewall issues. So with that better understanding of the Unifi Zone Firewall Policies, I finally pulled the trigger.

I guess I’ll start from the beginning, explaining the three main VLANs I have:

The Default/Main VLAN. This has my computers, consoles, phones, etc.
The Guest VLAN. This one is attached to my guest WiFi, naturally. Device isolation is enabled here.
The IoT VLAN. This is my smart plugs, WiFi bulbs, Google Nest devices, TVs, etc.

The reason for separating the IoT devices from everything else has to do with security. Some of these devices are cheap smart plugs and even smart bulbs. Some of these are from overseas companies I’ve never even heard of before. That’s on me for using these instead of stuff from a more reputable brand, but still. Who knows what any of these devices are doing or watching on the network. I’ve mentioned before I have some Google Nest devices. Are these snooping on my network, sending network usage patterns back to Google? Who knows.

Anyway, most of these IoT devices require WiFi. So I created a new WiFi SSID and attached the IoT VLAN to it. I don’t necessarily know if it increases security, but I also set it as a hidden network. At the very least, since I live in a sizable apartment complex, I didn’t want to further clutter up an already cluttered SSID list. That said, there are one or two devices that are wired, such as TVs. For those, I had to tag ports on the appropriate switches. That’s one area I would like to play with more: port tagging. However, since I’m using some of the more basic Unifi gear, there are some limitations.

Fom there, I merely had to place the networks in the correct zones. I created a new IoT zone and then moved the IoT VLAN into it. My Main network is in the out-of-the-box Internal zone. And when I created a Guest WiFi network, Unifi automatically placed it in the Hotspot zone, with the proper firewall rules to isolate the network (devices on the Guest network are also isolated from each other).

By default, user-created zones in Unifi are automatically isolated from other networks (though they’ll always at least have Internet access, which is in the External zone). So creating the IoT zone did most of the work for me. I didn’t have to create all these rules to isolate it from other zones. However, I wanted devices in the Internal zone to be able to communicate to devices in the IoT zone. Two reasons for this:

If I lose Internet access, I want some limited control over IoT devices. I don’t want to have to press the power button on my smart switches if I want to turn them on/off; I want to be able to use my phones and apps.
I have a Chromecast and a few other devices that I can cast media to. I moved those devices to the IoT VLAN, but of course my computers and phones will remain on the Main VLAN.

However, if a device is communicating “down” to devices in the the isolated IoT VLAN, there needs to be some ability to communicate “up” as well. If I ping from Main to IoT, that only works if IoT can respond back. But I don’t want to devices from IoT able to initiate and maintain connections to Main.

Luckily, Unifi makes this easy—though I had to ask around and play around a lot to figure this out.

One single rule in the “source Internal, destination IoT” part of the zone matrix can be made to allow this behavior. On the source side, I selected the Main (Default) network in the Internal zone to allow connections on any port. On the destination side, I have the IoT network in the IoT zone. The most important thing here is, on the source parameters, that checkbox to “Auto Allow Return Traffic.” When the rule is saved, that checkbox creates a corresponding partner rule in the “source IoT, destination Internal” part of the matrix to allow only traffic from IoT only if Internal initiated first.

Essentially, I can ping from Main to IoT, but I can’t ping from IoT to Main. This works because the “bottom” rule, or broadest firewall rule, in the IoT zone, disallows traffic to/from any other zones. Like I said before, that’s Unifi’s default behavior for new zones. My rule creates an exception to the policy.

I tested everything out afterwards, and the results were mixed. I disconnected my network from the Internet to see how different applications would react. From my cell phone (also only on WiFi), I could easily still control the smart plugs, which is exactly what I wanted.

But the media casting wasn’t as successful. The Chromecast seemed to accept casting from a computer via direct network connection, but Spotify on my phone didn’t quite work with my Google Nest speakers without Internet. Not ideal, but I don’t cast that much anyway. The main thing is controlling my lighting.

With this finally done, I feel like my basic network setup is essentially complete. There’s always more that I could tweak. For example, I have a printer. Maybe I put the printer on its own VLAN and allow any device in any network the ability to print to it. Right now, it sits in the Main VLAN with my computers. I don’t have that many guests, so it’s really NBD.

But for now, this is good. Time to move onto something else.

Homelab Chronicles 08: Investigating the Incident

Homelab

(Continued from Homelab Chronicles 07).

From the moment I woke up, through work, and into the afternoon, I was constantly monitoring my network. I have the Unifi app on my phone, so it was easy to see the list of clients connected to WiFi. Luckily, nothing unexpected connected.

Maybe the “Special Victims Unit” can investigate

At this point, I assumed my neighborly adversary (adversarial neighbor?) knew they had been caught. The WiFi network they had connected to had disappeared overnight, as did my similarly named “main” one. In its place, a new SSID would pop up on “Emily’s iPad” when they tried to connect, with a name that wasn’t mean, crude, or insulting, but one with a subtle message that basically says, “I see you and I know what you did.”

I forgot to mention that my main WLAN has always used WPA2/WPA3 for authenticating. I think there are ways to crack WPA2, but I’ll get into that in the future.

Once I got home, I jumped back onto the Unifi Controller to see what information I could glean. Having “fancy” Ubiquiti Unifi gear means the platform records and stores a lot more information than the average household router. I mentioned in the previous Chronicle that the Controller can tell me the device manufacturer by looking up MAC addresses. I can also see connection histories. With packet sniffing and traffic analysis tools, I can also see general traffic usage, i.e. where they were going.

So when did they first get on my network?

Unifi gives alerts when devices disconnect and connect. I silenced these alerts, because that’d be annoying for as many devices I have, but it records these nonetheless. It also shows the last 5 times a device connected, along with a duration. Most of the unauthorized devices appeared to have connected within the last 10-14 days. However, I did see one device with a recorded connection date around 20 days ago. It was connected for 13 days straight. It had “Amazon” in the hostname, so I’m assuming it’s some kind of smart home device that’s always on.

Sadly, because my server, and therefore Controller, was turned off to save on AC and electricity costs, there are large gaps in the 2-3 month history. It’s possible the devices were connected further back than 20 days ago. But that “Amazon” device only had two entries; 20 days ago and then overnight when I powercycled the WAP. So I’m assuming that nearly 3 weeks ago was when they first cracked the password.

Where did they go or what did they do?

Naturally, my next bit of curiosity was wanting to know what they were doing while connected. I needed to know if the adversaries were doing illegal things. Were they engaged in piracy? I don’t need a(nother) copyright strike on my ISP records! I hope to god they weren’t doing anything more illegal than that.

The Insights tab for the blocked devices showed me generally what they were doing. And it was mostly mundane, everyday stuff. Lots of streaming content from YouTube, Netflix, Hulu, Spotify, etc. Looking at that “Amazon” device, I could see traffic entries for Hulu and Amazon Video. Maybe it’s not a smart home device, but instead a Fire Stick or Fire Tablet. Interestingly, I deduced they have a child: I found a traffic entry on a blocked device for Roblox, the popular kids game. I’m more of a Minecraft guy, myself.

Traffic sources for a single device in Unifi Console. — *Roblox? Oof.*

Looking at Internet traffic overall, I could see there were other devices that were connected prior to my discovery. The only ones I outright blocked were those that happened to be connected to WiFi at the time. There was traffic to Xbox gaming services, which was tied to a device with an appropriate hostname: XboxOne. It looked like they downloaded a game or update/patch since it was a sizable 1.75GB download.

List of top Internet traffic sources in Unifi Console. — *Probably some Call of Duty “scrub noob.”*

But overall, traffic was pretty low. Certainly not enough for me to notice Internet speed degradation. Helps that I have gigabit fiber.

It doesn’t look like they were engaging in torrenting of pirated material, but at the same time, I’m not familiar with how that would look in Unifi. There isn’t a “Torrenting” category of traffic that popped up and I don’t know if that exists. But given the overall low data usage, it doesn’t seem that way.

Is this a crime?

I do want to point out that what “Emily” did is highly illegal. They hacked/cracked their way into my network. Every state in the US has laws on the books about this, as does the federal government, I’m sure. But not only did they engage in unauthorized access to a network, they also used my Internet connection, that I pay for. That’s theft of services. I didn’t authorize “Emily” and their family to be on my network, nor did I allow them to use my Internet connection.

“Unauthorized access” entails approaching, trespassing within, communicating with, storing data in, retrieving data from, or otherwise intercepting and changing computer resources without consent. These laws relate to these and other actions that interfere with computers, systems, programs or networks.
National Conference of State Legislatures – Computer Crime Statutes

But then who would I report this to? And who would investigate this? My city PD and the state have more important things to worry about. I think.

Speaking of worry, because of how they entered the network, by cracking into it, I’m still worried about my computers. While one of my cybersecurity buddies introduced me to SIEMs a few months ago and had me install Wazuh on my server, I only have it monitoring one computer, the one I use the most. I have other computers that are on almost all the time that I don’t use as much. More importantly, I’m not anywhere near proficient enough to be able to analyze all the logs that Wazuh is collecting. As a result, I still need to figure out what I want or need to do about my computers. Were they able to gain access to them? Upload and run malicious code?

On the other hand, someone online that I talked to about this did mention that it was odd that they didn’t attempt to obfuscate their device names. So maybe they’re just a “script kiddie” that for some reason can’t or doesn’t want to pay for their own Internet like an adult.

Regardless, it still has me worried. And that’s the rub. Even though this was entirely digital, it feels the same as if someone physically broke in and entered my home without me knowing. And then stayed there, hidden away, in the attic.

As an introvert and a sort of hermit (Look, it’s hot and humid as hell out there!), my home is my sanctuary. I’m sure that’s true for everyone, introvert or not. But because I spend so much time on my computers for work and for leisure, that too is my “home,” as sad and ridiculous as that may sound. Same concept though; in this world we live in, our “digital life” through our devices and everything stored on them is important to each of us. Find me a person that would be OK with someone “rifling” through their cell phone. Or OK with someone posting even a silly status on their Facebook or Twitter behind their back. Some people don’t even want their family or significant others to look through their phone, much less a stranger. My privacy has been invaded and my feeling of safety, shattered. It sounds dramatic, but it’s true.

I still have work to clean this up. And I have some ideas. But I’ll get to that on my next post, Soon™, which will wrap this ‘incident’ up.

—To be continued.

Homelab Chronicles 07: ALERT! – Unauthorized Access

Homelab

It’s been awhile since I’ve done anything with my Homelab. I’ve been busy with work, travel, and lounging around. There’ve even been extended periods over the last 2-3 months where my server has been completely turned off so I can save some money on electricity during the hot, hot summer. Plus, when it’s 100°F (37.7°C) outside and my AC is trying to keep things at a “cool” 78°F (25.6°C), the last thing I need is a server putting out even more heat.

But I was forced to take a look at things the other night when my Ubiquiti USG was making strange sounds. Fearing that it was going out, I wanted to look around to see how much a replacement would cost. I needed some information on my USG, so around midnight before going to bed, I booted up the server — it hosts an Ubuntu VM that itself hosts the Unifi Controller — and signed-in to the Unifi Controller via Web.

Almost immediately I was struck by how many clients were supposedly connected to the network: 34 devices.

Now, I’m a single guy with no kids, living in a 2-bedroom apartment. But I’m also an IT professional, a geek, and a gamer. I have several computers, cell phones, tablets, consoles, and such. I also have some smart home stuff like plugs, thermostat, cameras, etc. But the number of devices connected is pretty stable. Like 20-25.

So to see 34 clients was surprising.

I started with the list of wired connections. About 10 devices that I mostly recognized, even with just MAC addresses. Unifi has a neat feature where it’ll lookup MAC addresses to find manufacturer information. Anyway, all good there. So I went to the list of wireless connections.

At the very top of the list, I saw 10 devices that I didn’t recognize. One had a hostname of “Emilys-iPad.” I’m not an Emily. I don’t know an Emily. And I certainly don’t have an iPad named Emily…’s-iPad.

List of Devices in Unifi Console — *Who the hell is Emily and why does she have an iPad on my network?*

My heart started racing and I got jitters. Devices were on my WiFi network that were not mine. Devices that I didn’t authorize, by someone that I didn’t know. There were a couple Amazon devices, an LG device, and other hostnames I didn’t recognize. But I don’t have any Amazon devices, nor LG.

How long have these been on my network? Whose are these? But more importantly, how did they get on the network?

I didn’t spend much time answering those questions, as the situation needed to be dealt with. Instead of going to bed, I took a screenshot of the device list with hostnames and MAC addresses, and then immediately got to work.

To start, I disconnected and blocked all the devices from connecting to my WAP. I noticed that all the devices were connected to a secondary WLAN with a separate SSID; more on that in a second. I disabled and then deleted that WLAN. I then powercycled the USG and the Unifi WAP to make sure those devices were off the WLAN and wouldn’t be able to connect again. When it restarted, nothing was connected to that WLAN and only my devices were connected to the “main” WLAN. The threats were removed.

OK, so now about this WLAN. Some months ago, I whipped out my old Playstation Portable (PSP). I was feeling nostalgic and wanted to find some old games on the Playstation Store, so I needed to connect my PSP to the Internet. I have a modern WiFi 6 (802.11ax) Unifi AP. Unfortunately, the PSP, being so old, can only connect to 802.11b or 802.11g networks. I can’t remember the decision making process, but I eventually created a secondary WLAN, that was specifically for b/g devices. And of course I password protected it. However, since the PSP is old, I used the old-school WEP (Wireless Equivalent Privacy) as the password protocol.

Devices were on my WiFi network that were not mine. Devices that I didn’t authorize, by someone that I didn’t know.

Anyway, after I was finished with my PSP, I didn’t take the network down. “Never know when I might want to use it again,” I thought. So I left it up. Nothing was connected to it since. Since then, I’ve signed-in to the Unifi Controller a handful of times and never noticed anything other than my devices on my main WLAN. I honestly forgot that I even had it up. Until this happened.

With the threats neutralized, I could finally start doing some investigating. And my first question was obviously how they got on the network.

I’m assuming I password protected the WLAN. Because I’m not an idiot. Usually. But if it was only with WEP…well, there’s a reason why we’ve moved to WPA (WiFi Protected Access), WPA2, and WPA3.

According to Wikipedia, WEP was created in 1999. 23yrs ago. And over time, major vulnerabilities were found quickly. Without getting into the nitty-gritty, it’s not hard to crack a WEP password. There are programs out there online that are easy to find to sniff packets, analyze data, and eventually crack the password. Possibly in minutes.

That said…it’s not exactly something I’d expect my average neighbor to be doing. I’ve known about cracking WiFi passwords and “wardriving” for a long time. But even I’ve never done it.

I got a little nervous thinking about that. What kind of adversary is one of my neighbors? Are they also an IT person? Maybe a security professional?

And if they were on my network, what else did they see or even touch? In retrospect, it was dumb of me to do this, but I didn’t put that WLAN on a separate VLAN. I mean, why would I? I’m the only one connecting to it, with my one device. What that means is if anything connects to that b/g network, they’re on THE network. They can see my computers, my server, my consoles, my smart devices…everything.

Do I now have to wipe all my computers and VMs? I mean, some need it, but it’s still an undertaking to have to redo everything. It’d likely take a whole weekend and then some.

That led me down another path, concerning my “main” WLAN. Did I use the same password for that b/g network, too? If so, they’d know the password to my main WLAN, as well, which has a different, but similarly-styled SSID.

So I nuked my main WLAN and created an entirely new one with a new SSID and new complex password. I then had to reconnect my smart home devices.

At that point, it was already around 2:00am, and I had to go into the office in the morning. What started as me wanting to find some model information on my USG turned into DEFCON1 at home.

But with the unauthorized devices off the network, a new WLAN, and the important devices back online, I felt somewhat comfortable going to bed. The investigation would have to wait until I got home the next day.

—To be continued.

Homelab Chronicles 06 – “Hey Google…” “I’m Sorry, Something Went Wrong”

General, Homelab

I woke up early today, on a Saturday, to my alarm clock(s) going off. I was planning to go to a St. Patrick’s Day Parade and post-parade party with a friend. After turning off my phone alarm(s), I told my Google Nest Mini to stop the alarm that was blaring.

Unfortunately, it informed me that something went wrong. Though it did turn off. Usually when my Google Nest Mini has issues, it’s because WiFi messed up. So I stumbled out of bed, still half-asleep, to the guest bedroom, where the network “rack”—a small metal bookshelf—and the Unifi AP was at. My main 24-port switch had lights blinking. I looked up at the AP high up on the wall and saw the steady ring of blue light, indicating everything was working. OK, so not a WiFi problem, nor a network problem. Probably.

In the hallway, I passed by my Ecobee thermostat to turn the heat up a little and then noticed a “?” mark on the button for local weather. Ah, so I didn’t have Internet. Back in my room and I picked up my phone: 5G, instead of WiFi. On my computer, the Formula 1 livestream of the Bahrain track test, which I fell asleep to, had stopped. And reloading the page simply displayed a “No connection” error. I opened a command prompt and ran ipconfig /all and ping 8.8.8.8. The ping didn’t go anywhere, but I still had a proper internal IP in the subnet. Interesting. Guess the DHCP lease was still good.

Only one last place to check: the living room where the Google Fiber Jack and my Unifi Secure Gateway router were. Maybe there was a Fiber outage. Or maybe my cat had accidentally knocked the AC adapter off messing around in places he shouldn’t. Sunlight was streaming in from the balcony sliding door, making it hard to see the LED on the Jack. I covered the LED on the Fiber Jack with my hands as best as I could: it was blue. Which meant this wasn’t an outage. Uh oh. Only one other thing it could be.

Next to the Fiber Jack, surrounding my TV, I have some shelving with knickknacks and little bits of artwork. Hidden behind one art piece is my USG and an 8-port switch. I removed the art to see the devices. The switch was blinking normally. But on the USG, the console light was blinking with periodicity, while the WAN and LAN lights were out. Oh no, please don’t tell me the “magic smoke” escaped from the USG.

On closer inspection, it looked like the USG was trying to boot up repeatedly. It was even making a weird sound like a little yelp in time with the console LED going on and off. So I traced the power cable to the power strip and unplugged it, waited 15 seconds, and plugged it in again. Same thing happened. I really didn’t want to have to buy a new USG; they’re not terribly expensive, but they’re not inexpensive, either.

I tried plugging it into a different outlet on the power strip, but it kept quickly boot-looping. I then brought it to a different room and plugged it into a power outlet; no change. Great.

But then I noticed that there was a little green LED on the power brick. And it was flashing at the same frequency as the USG’s console light when plugged in. Hmm, maybe the power adapter went bad. I could deal with that, provided I had a spare lying around.

The Unifi power brick said “12V, 1 amp” for the output. So I started looking around. On my rack, I had an external HDD that was cold. I looked at its AC adapter and saw “12V, 2 amps.” That was promising, but could I use a 2 amp power supply on a device that only wants 1 amp? I looked online, via my phone, and the Internet said, “Yes.” Perfect.

I swapped the AC adapter on the USG. The little barrel connector that goes into the USG seemed to fit, if not just a smidge loose. Then I plugged it back into the wall.

It turned on and stayed on! Ha!

I brought it back to the shelf and reconnected everything. It took about 5 minutes for it to fully boot up. Afterwards, I went back to my computer and waited for an Internet connection to come back, and it did.

All in all, it was a 15-20 minute troubleshooting adventure. Not what I preferred to do straight out of bed on a Saturday morning, but it got fixed. I already ordered a new AC adapter from Amazon that should arrive in a few days.

Afterwards, I got ready and went to the parade. A bit nippy at about 25°F (about -3°C), but at least it was bright and sunny with barely any wind. I went to the party and had a couple beers. It definitely made up for the morning IT sesh.