Why does my internet suck?
Why does my Internet suck?
Early my career, working from home was really a novelty, so I didn’t really care too much about my Internet. The TV works, I could play online matches of Halo. Life was great! Though, I do not miss the hours I spent idling in traffic, or my existential thoughts around the decades of politics that were involved for my manager to navigate and acquire an entire row of end cubes with a corner view of the flatirons.
Now I worry about how since then my work has transitioned to full-time views looking out into the world from the 1940s stucco box of bricks of which I call home. These days, existence has become more and more dependent on that little black box provided by Xfinity, for a convenient price of $15 dollars per month. It’s one after one a hodgepodge of calendar calls on a variety of vendors (FaceTime, Meet, Zoom, Hangout, Huddles or MSTeams) all with their own little annoyances. Yes, that list is rank ordered in terms of quality I experience. Considering my reality, it’s kind of wonder at the considerable effort I put into trying to ignore my internet woes. I have mainly gone by just accepting persistence quality issues as I sit on the call after call where my video drops out or how my audio sounds like DJ cutting records. I have largely just assumed that my issues stem from some combo of buggy products and Comcast being a soulless conglomerate who couldn’t care less of their users experience. I ultimately categorized these experiences with the tune of the Frank Sinatra song “That’s Life” like I do with so many other things.
While I feel that I have an unusually high tolerance for life’s little inconveniences, my partner’s tolerance is unusually thin for these things. I do hit breaking point, but it is more so related to hearing a barrage of exasperated grievances as with our terrible Wi-Fi. As of recent its been doubly triggering, because my years of perceived schlepping on the couch with my laptop isn’t actually work, she too is now alongside me on the couch schlepping remotely with her job & masters program. So the occasional grievance has morphed into a full-blown catastrophe. As I reflect that I now have trigger phrases like “Our Internet Sucks”, “The Wi-Fi is garbage”.
I originally started network changes small. Mainly to not pay rental fees. So, I at least ditched the rental Comcast gear. Reviewed cable modem and Wi-Fi router which had reasonable price and a healthy amount of reviews. In 2019, I picked up a TP-Link AC2300 and NETGEAR CM600 this was the era where we had the cheapest Comcast Internet that was available, and I was blissfully ignorant. This lasted until about early 2022 where I replaced the TP-Link with the NETGEAR R6700AXS .
Around this time I also began to tinker more Raspberry Pi hobbies and in general our home devices started to grow(smart light, PiHole, AlgoTrading, & personal devices). In turn bumped up the bandwidth on Xfinity and bobs your uncle, right?
Not a chance! the complaints kept rolling, kicked off looking into getting fiber, but those leads didn’t go far. So in 2023 I maxed out the Xfinity bandwidth to Gig + speeds. Well, old modem wouldn’t handle speeds over a gig and regular usage didn’t come close to those speeds. So I picked up a new modem NETGEAR CM2000 to handle the additional capacity. I also like the concept that support for mesh might help deadspots. TP-Link AXE5400
The problem is my partner works at our dining table for the most part with a direct line of sight to the Wi-Fi router, which is probably only 12’ - 15’ away. So, I don’t think mesh will help her issues, but it might still solve my abysmal performance in my office. My desk in my office is probably only 15’ - 18’ feet away from the router, but there is a zigzag hallway, and a bathroom that block direct line of site. I really think stucco walls might have a metal lath that acts as a faraday cage, creating chaos for signals to reach that room.
The problem is the grievance bell is still ringing. At this point, our internet experience is like when bill & ted go to the future it’s all out of place. I’m streaming 4K, downloading docker images is a breeze, uploading is great, but it’s mainly the video conferencing, it’s still mainly shit.
Is it the Pi Hole? IDK its late 2023 figure i’ll drop the Pi Hole and pick up a Firewalla . It will cover all the stuff I was doing on with the Pi Hole. DNS over HTTPS with a private Cloudflare endpoint, it has ad blocking and I can configure VPNs (though i’m not sold on this unless I was hosting my own and letting my traffic exit with AWS), it’s got VLANS, and can have HomeBridge to manage all the IoT that i’ve picked up. Great! Was able to get all that running, but there still continues to be issues…
Well let’s figure out networking, all the backbone nics seems to be using 1GbE except for the Firewalla. There is some quirks and I think the agg to the Firewalla through the Wi-Fi modem is just not cutting it. Life at this point is more or less the same, except I’ve shelled out more money to have more or less the same experience that I was having back in 2019 (that’s not entirely true most internet experience is snappy. Except for Ad Heavy Sites such as Instagram (its really noticeable it’s a 50/50 crap shoot if content will load) The funny thing is if you switch to 5G it all loads quick. At this point my tiny home network looks something like this:
Well, maybe the Ethernet devices don’t help the Wi-Fi-fi router. Created a new plan to pick up a switch and put that into the mix. Ended up fetching a QNAP sw-m408-4c Pretty excited!! Eight 1GbE ports & four 10GbE. Now, I have enough ports to wire most my wired thing and hobbies. Now, I also get to negotiate true NBASE-T Ethernet 10 GbE with Mac Studio(😂 maybe one day my outbound network could absorb that). I also was able be able to configure dual 2.5GbE agg links to the Firewalla. This should be plenty enough to handle my network traffic.
BufferBloat
“Do your video or audio calls sometimes stutter? Does your web browsing slow down? Do video games lag? If so, bufferbloat may be to blame.
What Is Bufferbloat? Bufferbloat is a software issue with networking equipment that causes spikes in your Internet connection’s latency when a device on the network uploads or downloads files.”
https://www.bufferbloat.net/projects/bloat/wiki/Tests_for_Bufferbloat/ https://bufferbloat-and-beyond.net/ This sounds like what i’m experiencing.
Reporting Results
Mac Studio
With the Mac Studio, I’m able to show that the network and general bandwidth is not an issue. Even when the agg link is two 1GbE connection, speeds are decent enough to get a Good rating from Cloudflare across the board. Bumping the agg links to the 2.5GbE(Firewalla nic’s max) I’m finally able to get use the full bandwidth from my ISP.
10GbE -> 2GbE(Firewalla) -> Comcast 1M+GbE
Cloudflare speed test
10GbE -> 5GbE(Firewalla) -> Comcast 1+GbE
waveform bufferbloat test
Flent is a suite of tests we developed to diagnose bufferbloat and other connectivity problems. Because Flent has been tested to 40GigE, you can get a good feel for how the connection behaves while you tune your settings. In particular, Flent’s RRUL test shows download and upload speeds and latency in one set of charts.
Macbook
With the macbook I’m 100% using Wi-Fi so I know there’s going to be some extra overhead over ethernet
Macbook Pro Wifi (5G) -> 1Gbe (Nic) -> 2GbE(Firewalla) -> Comcast 1+GbE
waveform bufferbloat test
The only consistency on WiFi is how inconsistent it is
I ended up running back to back speed test test sitting on my couch while watching youtube through my TV(hardwired ethernet connection) & running the test from my macbook on the Wi-Fi. Again i’m only sitting about 6’ away from the router without any obstructions. This really shows the issue under a normal network load. I picked up that my Wi-Fi experience only worsens with moderate usage.
I continue to get warned about how bad my Wi-Fi is even google warns how bad my situation is.
Next Steps
Smart Queue with Firewalla
I probably need to spend some time and energy to setup Smart Queue Features with the Firewalla. They recently release Cake
Dave Täht, co-founder of the Bufferbloat Project, commented: “The FQ-Codel (RFC8290) and the newer CAKE packet scheduling/AQM algorithms are nearly universal on servers and clients. They give the “little guy” – the small packets, the first packets in a new connection to anywhere – a boost until the flow achieves parity with other flows from other sources. DNS, gaming traffic, VoIP, videoconferencing, any new flow, to anywhere, get a small boost. That’s it. After that, all network traffic gets treated equally. Big flows – from Netflix, Google, Comcast, your Mom, or to Timbuktu – all achieve parity, with minimal delay and buffering, at the worldwide variety of real round trip times.” 1
Firewalla has smart queue and with their latest release they have the option to use CAKE
| |
Network Segmentation
Separate devices on to their own VLAN segments, Block IoT traffic to the internet
Anything Else I can think of
Next steps could be to roll out any of the recommendations that BufferBloat article has on Wi-Fi hardware. Guess I pick something with openwrt Really makes me think, if Ubiquity products also encounter these issues. I’ll also need to read more up on https://wiki.stoplagging.com/ as things like SQM only help to the ~350Mb connection