{
  "$type": "site.standard.document",
  "description": "feat: monitor home internet with Raspberry Pi, InfluxDB, and Grafana",
  "path": "/posts/monitor-home-internet-raspberry-pi/",
  "publishedAt": "2024-12-02T00:00:00.000Z",
  "site": "https://read.ryancowl.es",
  "tags": [
    "Projects"
  ],
  "textContent": "I got mad at my ISP. Now you can get mad at yours with this one simple trick!\n\nI recently noticed a drop in performance with my home internet. Calls have been choppy, videos buffer a lot, and the overall quality of connection has noticeably decreased. I tried troubleshooting things on my local network to the best of my ability but the issues persisted. Even connecting directly to the router, I continued to see high latency#Packet-switchednetworks) and packet loss.\n\nSo, like most normal people, I decided to set up a Raspberry Pi to run network tests and collect data to better understand my network issues. I flashed a fresh image of Raspberry Pi OS onto an old 3 Model B+ and decided on a few tools to use for the job: speedtest-cli, mtr, InfluxDB, and Grafana.\n\nI’ll explain how and why I use each below. But the desired outcome is a dashboard where I can identify spikes or trends that indicate ongoing problems with my internet connection. I can share that information with my ISP in an attempt to diagnose and hopefully resolve the issue.\n\n  \n\nSet Up the RasPi\n\nWe’ll first need to prepare our Raspberry Pi. I used Raspberry Pi Imager to start fresh with a clean install of Raspberry Pi OS. If you use the imager, you can follow its prompts to set the hostname, enable SSH access, and configure additional settings. For this use case, make sure to enable SSH so you can login to the Pi remotely.\n\nAfter the Pi is up and running, you’ll need to connect it to your home network. If possible, I suggest connecting directly to your router via ethernet. By doing so, you can rule out issues related to wireless connectivity.\n\nOnce your Raspberry Pi is connected, you can login via SSH with  or , where  is the hostname or  is the IP assigned to your Raspberry Pi. If you are unsure what either of these are, you can run  [^1]. That command will return something like the following:\n\nFrom that response, you can pull the hostname () or the IP (). Then you can connect with .\n\n  \n\nInstall Packages\n\nOnce you’re logged in via SSH, you’ll need to install the tools we are going to use. We’ll use , which is included in Raspberry Pi OS, to test latency. We’ll use speedtest-cli to test upload and download speeds. And we’ll use mtr to check for packet loss.\n\nWe can store the results from each of these tools using InfluxDB, which is a time series database built to handle metrics, events, and logs. Finally, we’ll use Grafana to create a dashboard to display these results over time and tie everything together.\n\nThe Grafana package isn’t currently included in the default Raspberry Pi OS repositories. So we’ll need to add it ourselves:\n\nNow we can update our package list and install Grafana:\n\nWe’ve got everything we need installed now and we can start putting it all together.\n\n  \n\nSet Up InfluxDB\n\nLet’s start the InfluxDB service. This is what we’ll use to create the database that will store our test results. We can start the service and create a database like so:\n\n  \n\nSet Up Grafana\n\nWe’ll use Grafana to visualize and display our data. We’ve installed it above, now we need to start the service and configure it to use the InfluxDB database we just created:\n\nBy default, Grafana will use port . You can access the web UI via browser by hostname and port: . Or you can access via IP address and port like so: .\n\nThe default login credentials for Grafana are:\nUsername: \nPassword: \n\nUse those to login and you’ll be prompted to set a more secure password. After setting a new password, we can move on to the next step. We’ll return to Grafana a bit later to create the dashboards.\n\n  \n\nWrite the Tests\n\nWith everything in place ready to catch our data, we can write our tests. We’ll just use some simple bash scripts for each. Let’s start by writing a script to run our speedtests.\n\nTest Upload and Download Speeds\n\nRaspberry Pi OS ships with GNU nano and vi), so we can use either to create and edit our file. I’ll create and open a new file for our speedtest script with:\n\nIn this script we’ll run , capture the results as JSON (this is where we need jq), and then write them to our database. InfluxDB uses port  to run its HTTP service by default, so the script assumes we’re using the default settings. I’ve included some basic logging as well, which will help us troubleshoot issues later on if need be.\n\n\\[!WARNING\\] You’ll need to update the filepaths in these scripts (e.g. ) to match your directory structure.\n\nTo ensure you saved your file (it’s  then ) you can [^2] and check the output. That should give you the contents of the file we just created.\n\nWith our script created, we need to make it executable so we can run it:\n\nLet’s take it for a test drive with . It may take a minute, but you should see the results of the speedtest after running it manually. Later on, we’ll use cron to schedule it to run automatically.\n\nTest Latency\n\nThis might be overkill since  can capture . But I wanted to include a dedicated ping test for a couple reasons. A separate test allows me to separate concerns. And since the ping test alone will be lighter than a full speedtest, I can run it more frequently to establish a tighter baseline.\n\nLike our speedtest, let’s create a new file for our ping test with our editor of choice:\n\nOur script as written below will ping Google’s Public DNS at  ten times and capture the average round-trip time and jitter. Then it will write those results to the database.\n\n\\[!INFO\\] Remember to update the filepaths so they match your directory structure. You can also adjust the count and target IP address if you’d like[^3]\n\nAs with the above, we’ll make it executable with . Then we can double check the contents of the file with  and take it for a manual test drive with .\n\nTest for Packet Loss by Hop\n\n (My Traceroute) combines the functionality of  and  into a single network diagnostic tool. It allows you to see latency and packet loss for each hop) along the way to the destination whereas  only shows the packet loss to the final destination. This is helpful in determining where packet loss is occurring along the route. You can use  to view live results or print them to a report. In our case, we’ll do the latter and store what we need from the reports in our database. First, let’s again create a new script:\n\nIn the script, we’ll configure  to run and extract packet loss for each hop by IP address. We’ll use the  flag to output results as a report which we can then write to the database. Again, you can modify the parameters like count () and target IP address () if you’d like [^3].\n\nAnd once again, we can check the contents of the file with  and make it executable with . Then we can the script manually to test it with .\n\n  \n\nSchedule the Tests\n\nWith our scripts ready, we can use cron to schedule them to run at regular intervals.\n\n\\[!INFO\\] You can adjust the timing for each test. For example, I run  and  every 30 minutes and  every 15 minutes. Again, remember to change the  path to match your own.\n\nIf you’re unsure of the full path for your scripts, you can use  to find them. For example, we can find the path for our speedtest script like so:\n\n  \n\nConnect the Data\n\nWith our tests running and populating the database with their results, we can head back to Grafana to create a dashboard to visualize our findings.\n\nOpen Grafana again in your browser (e.g. ). First we need to connect Grafana to the datasource, which we created with InfluxDB earlier:\nOn the left, head over to Connections => Add new connection.\nSelect InfluxDB.\nIn the top right, click Add new data source.\n\nMost of the default values should suffice. We just need to add:\nURL: \nDatabase: \n\nNow click Save & test. You should receive confirmation that the datasource is working and measurements were found.\n\n  \n\nVisualize the Results\n\nWith our data source connected, it’s time to create the dashboard. In the top right, click Build a Dashboard and follow these steps:\nClick Add Visualization.\nSelect Influxdb as the data source.\nScroll down to Queries.\n\nYou’ll now need to build your query for each metric you want to visualize. For example, we can display results over time from our ping test by creating a panel with two queries. First, we’ll look at ping:\n\nAnd second, we’ll look at jitter:\n\nYou can customize the display options as much or as little as you’d like using the panel on the right. You can add a title, choose the graph style, and set units appropriately. When you’re happy with the look, you can save the panel and create two more for the  and  test results following a similar process. I landed on the following:\n\n  \n\nTroubleshooting\n\nIt took a little trial and error to get things running smoothly, but here are a few troubleshooting tips that helped me along the way.\n\nCheck the logs\n\nWe included basic logging for each of our scripts. Those logs are a good place to start to see if there are any errors. For example, we can check the speedtest logs with  to see if the script is being run and completed successfully.\n\nIf there there aren’t any errors in those logs then we can check the cron logs with . If all is well, we should see entries like the following:\n\nCheck File Permissions\n\nIf the scripts aren’t running as scheduled, check the file permissions to make sure they are executable. You can check the permissions for all shell scripts with [^4] like so:\n\nIf any scripts don’t have execute permissions to run via cron, you can add them like so (where  is the script in question):\n\nRun the Scripts Manually\n\nIf the permissions look good but tests still aren’t running, you can try testing the scripts manually. From the command line run  (where  is the script in question) and see if it completes successfully. If it does, you may need to check that the syntax is correct in your crontab. You can use  to list current cron jobs. You can make changes with  as needed and then restart cron with .\n\nIf you receive an error when running the scripts manually, we can rule out cron for now and instead use the error messages to diagnose the issue with script itself. Once the script runs manually without error, restart cron and see if the issue is resolved.\n\n  \n\nWhat’s Next?\n\nTo date, my network issues persist and my ISP has shown no interest whatsoever in my Grafana dashboard (I know, right?). But it feels good to have my frustration captured with pretty charts and numbers! Eventually my ISP admitted they are experiencing issues with connectivity in my area due to outdated infrastucture that will not be updated soon and thus suggested that I find a new provider. Perhaps not the ideal_ outcome, but hey, at least I’m ready to thoroughly test my new provider’s service.\n\n[^1]:  stands for Address Resolution Protocol and is a tool used in network administration to map IP addresses to MAC (Media Access Control) addresses of devices on a local network. When you run , it lists all the entries in the ARP table, showing the IP addresses and their corresponding MAC addresses.\n\n[^2]:  is a standard Unix utility) that reads files sequentially and outputs them to standard output. It’s handy to quickly check the contents of a file.\n\n[^3]: For our  and  tests we’re using  which is Google’s Public DNS. We can generally trust that to be reliable. You could also use Cloudflare’s DNS Resolver at  or any other reliable host for these tests.\n\n[^4]:  is a command to list computer files and directories in Unix and Unix-like operating systems.  itself lists files and directories, the  flag provides more detailed information, and  matches all files ending with  in the current directory.",
  "title": "Monitoring Home Internet Performance with a Raspberry Pi"
}