Lost in Space : Dodging millions of Ads, Malware and Trackers

Internet is littered with trackers, ads, malwares and miners yearning to get their bite of the information pie. This is a step by step guide to setup Pi-Hole and blocking site and content across the whole network at your home. Unlike the browser based extensions that can only block the content you pull up in the browsers, This solution provides a comprehensive coverage for all devices in your home network. You can use this to also control what sites can be accessed in your home network. Please note, unlike a firewall, this can ONLY block what is in the block list. There are plenty of lists out in the Wild Wild World, so compute power and sky are the limits for your block lists. My setup has a bit over 3 Million sites in block list.

pihole

Do keep in mind, the internet lives and thrives on the ad revenue. Many sites you enjoy depends on that to provide you that content for free. So, if you truly love a site, do enable their ads so that they can continue to run free.

The way Pi-Hole works is simple. We setup the Pi-hole to be the DNS server for the network. When Pi-Hole receives the DNS query, it checks it against the blocklist and responds with a 0.0.0.0 if it is blocked. If not, it forwards the request out to the DNS provider we setup in the configuration and relays the response to the original requestor.

In this guide, we will be setting up the PiHole on a Raspberry Pi and setup Magan and Cloudflared to do the DNS over HTTPS (DOH). One would need a basic understanding on the networking concepts and some linux skills to follow this guide.

Fair Warning: adding this protection to your network with a large block list as mine may break the Internet Experience as you know it, as several sites (Facebook or Youtube for example) will start behaving differently. Images not loading correctly and Youtube videos not playing anymore are common issues. During the initial days of operation, you would need to visit the query log and see what you need to whitelist or blacklist to tune the experience to a tolerable level. It may take a few weeks, but you will enjoy the benefits of quicker page loads and cleaner pages.

It is your responsibility to review periodically and update lists as needed.

Parts List

This is a list of the essential parts we need for the project. You can get all these as pre-built kits online and in-stores as well.

  1. A Raspberry Pi – preferably, get the latest and greatest. If you have a Microcenter near by, that may be your cheapest option. If not, look for the authorized sellers. Newer models have both ethernet and Wifi built in, so you could use that to connect to more than one IP addresses or even networks if you feel like it.
  2. Micro SD card – My personal choice is the Samsung EVO series. 32GB or above recommended.
  3. Micro USB Power supply – Raspberry Pi has official power supply available, but not necessary. as long as you can find a good micro USB power supply, that should work.
  4. Ethernet cable – Cat 5 or above should be fine.
  5. Raspberry pi case (Optional, but helps to keep it safe).
  6. Keyboard, Mouse and Monitor with an HDMI input for initial setup. I have a Logitech K400 that I use for this. You can also do this entirely headless should you chose to, by modifying the image and have it connect to the wifi.
  7. HDMI cable.

Prepare the Pi

  1. Download Raspbian from here.
    Use your favorite image flashing tool to write the image to the SD card. My preferred tool is Balena Etcher.
  2. Insert the SD card to the Pi.
  3. Connect your Raspberry Pi to the display, keyboard and Mouse. We plan to run this headless after, so this is temporary.
  4. Connect the network cable and plug the other end to an available network port in your home router.
  5. Connect the micro USB power and boot the device.

First boot of the Raspbian OS will resize the disk space to use the full disk space and will run you through the initial setup (Set the location, timezone, change default password, wifi network, updating the OS to the latest version etc. Reboot the Pi at the end of the OS update, when prompted.

Don’t forget to enable SSH from the Raspberry Pi Config utility (Menu > Preferences > Raspberry Pi Configuration > Interfaces). This is off by default and we would need this to manage things when we eventually run everything headless.

Once we get back on the network, find the IP address of the Raspberry Pi. there are multiple ways to find the IP. You can either open a terminal and run ifconfig or you can look at your router administration site and find the pi in the connected device list.

ifconfig | grep "10.1.10."
inet 10.1.10.109 netmask 255.255.255.0 broadcast 10.1.10.255

Now, we can switch back to your computer and do the remaining work via terminal. I would recommend leaving the peripherals connected for now, just in case we have to login on the pi locally to undo something we accidentally messed up.

ping 10.1.10.109 
PING 10.1.10.109 (10.1.10.109): 56 data bytes
64 bytes from 10.1.10.109: icmp_seq=0 ttl=63 time=3.286 ms

Open your terminal and ssh to your Pi and run the remaining steps from a remote terminal.

ssh pi@10.1.10.109

Rename the Pi

Let us rename the Pi to something more meaningful so that it makes sense.

Update the hostname file and change the name to whatever you like.

sudo vi /etc/hostname

Next, Update the hosts file and replace raspberrypi with piadmin01

sudo vi /etc/hosts

This will take effect next time when you reboot the machine.

Setup a static IP address

sudo vi /etc/dhcpcd.conf

Add the following at the end of the file. You may change the IP to what suits to your network. Just remember to use the appropriate IP for the rest of the steps.

interface eth0
static ip_address=10.1.10.2/24
static routers=10.1.10.1

Save the file and Restart your network interface or Pi itself for the changes to take effect. Now the Pi should be on the new IP address. You can run a simple ping test to make sure that is listening on the new IP address.

ping 10.1.10.109 
PING 10.1.10.109 (10.1.10.109): 56 data bytes
Request timeout for icmp_seq 0

ping 10.1.10.2
64 bytes from 10.1.10.2: icmp_seq=0 ttl=64 time=3.130 ms

SSH to the new IP address and continue the setup.

ssh pi@10.1.10.2

Setting up the DNS Over HTTPS (DOH) Services Locally

We will work on setting up two DOH proxy services locally and use them for all DNS resolutions. This way, we ensure that all of Pi-Hole’s resolutions happen via DOH and not through your local ISP. We will be setting up Magan for the Google’s DNS service and Argo-Tunnel for the Cloudflare.

Setup service account for the DOH services. This is not mandatory, but I prefer to keep it separate just for the sake of my own sanity. I also gave it a long random string as password so that I can forget it soon after I make it.

Magan’s native binary needs libjason-c3. The Go version can run without that.

sudo apt install libjson-c3
sudo adduser magan
sudo su magan

Download the binary for Magan. Native magan-armv7l is preferred. There are different flavors available based on what you are running this on.

cd
wget https://github.com/evuraan/Magan/raw/master/bin/magan-armv7l
chmod +x magan-armv7l

if you prefer the go version, you can get that too. Let us test and see if this works

./magan-armv7l -p 5678

Open another terminal and see if this works

dig @10.1.10.2 -p 5678 www.google.com

; <<>> DiG 9.10.6 <<>> @10.1.10.2 -p 5678 www.google.com
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 27578
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 0

;; QUESTION SECTION:
;www.google.com. IN A

;; ANSWER SECTION:
www.google.com. 299 IN A 172.217.15.68

;; Query time: 668 msec
;; SERVER: 10.1.10.2#5678(10.1.10.2)
;; WHEN: Thu May 09 00:10:14 EDT 2019
;; MSG SIZE rcvd: 62

Let us setup Magan to run as a service so that it takes care of itself.

sudo vi /etc/systemd/system/magan.service

Add the following service definition

[Unit]
Description=Magan DNS over HTTPS Server
Wants=network-online.target
After=network.target network-online.target

[Service]
ExecStart=/home/magan/magan-armv7l -p 5678
Restart=on-abort

[Install]
WantedBy=multi-user.target

Let us enable the service

sudo systemctl enable magan.service
sudo systemctl start magan.service

Check and see if the service started correctly

sudo systemctl status magan.service

Just to make sure that it is listening, let us check if we have ports up.

netstat -a | grep 5678
tcp6 0 0 [::]:5678 [::]:* LISTEN 
udp6 0 0 [::]:5678 [::]:*

Now that Magan is up and running, let us go ahead and setup Argo-tunnel for Cloudeflare

mkdir cloudflare && cd cloudflare
wget https://bin.equinox.io/c/VdrWdbjqyF/cloudflared-stable-linux-arm.tgz
tar -xvzf cloudflared-stable-linux-arm.tgz

Checking to see we got the latest version

./cloudflared --version
cloudflared version 2019.4.1 (built 2019-04-19-2152 UTC)

Now, Just like Magan, let us turn this into a service.

Create the Service definition file needed

sudo vi /etc/systemd/system/cloudflared.service

Add the following service definition.

[Unit]
Description=CloudFlare DNS over HTTPS Proxy
Wants=network-online.target
After=network.target network-online.target

[Service]
ExecStart=/home/magan/cloudflare/cloudflared proxy-dns --port 6789 --upstream https://1.1.1.1/.well-known/dns-query --upstream https://1.0.0.1/.well-known/dns-query
Restart=on-abort

[Install]
WantedBy=multi-user.target

Enable and get this to run.

sudo systemctl enable cloudflared.service


sudo systemctl start cloudflared.service

Check the status

sudo systemctl status cloudflared.service

Install and setup Pi-hole

Now that we have the DOH services up and running, let us setup Pi-Hole.

curl -sSL https://install.pi-hole.net | bash

Select the defaults for all the screens/options and wait for the installation to complete. You can note the password at the last screen if you like, but even if you miss it, no need to worry. We can always change it in the command line.

Once we have the Pi-Hole up and running, we should change the DNS service provider to the local providers we setup above. In the defaults, it uses the Google DNS servers.

sudo vi /etc/dnsmasq.d/01-pihole.conf

Comment out or remove the lines below in the file

server=8.8.8.8
server=8.8.4.4

Add the following lines after the lines above.

server=127.0.0.1#5678
server=127.0.0.1#6789

Disable the built in DNS outbounds for the Pi-Hole. Edit the SetupVars file and comment out the PIHOLE_DNS_1 and PIHOLE_DNS_2 entries.

sudo vi /etc/pihole/setupVars.conf

If you didn’t like the password for the web front end, you can set a new admin console password

pihole -a -p

Check for updates for Pi-Hole (just in case)

pihole -up

at this point, the PiHole should be working as expected. The default blocklists only have about 113,000 sites in it. There are several popular block lists available on the internet and I am currently consuming several lists, which has almost 3 Million sites in the blocklists. There are two ways to do this – using the front end or using the good old command line. /etc/pihole/adlist.list is the master list of the urls to the the blocklists. You can add the new ones to the end of this file and run pihole -g to reload the lists. If you need some lists, check this out.

You can bring up the admin console on the browser and login with the password you setup earlier.

http://10.1.10.2/admin/index.php

In the Admin site, do take a moment to explore and figure out the locations of key settings. Query log is where you can look and see what each client in your network is requesting at. At least for the initial few weeks, keep an eye and you may have some interesting insights into what your stuff is calling for. In my first few days, I found that Samsung devices call home a LOT. I blocked it as soon as I found that out and that still remains the most blocked item in my list.

Now that everything is done, give it a final reboot and check if it is working as expected.

$ dig @10.1.10.2 popads.net 
; <<>> DiG 9.10.6 <<>> @10.1.10.2 popads.net
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 13231
;; flags: qr aa rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1
;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;popads.net. IN A
;; ANSWER SECTION:

popads.net. 2 IN A 0.0.0.0

;; Query time: 51 msec

;; SERVER: 10.1.10.2#53(10.1.10.2)

;; WHEN: Fri May 10 09:13:33 EDT 2019

;; MSG SIZE  rcvd: 55

Success! See that it is giving 0.0.0.0 as the response, which is how it works.

Update the Router configuration

Find the network settings in your router and select the option to Manually Configure the DNS servers. Put the IP address of the our Pi-Hole server as the first DNS server. It may take a bit of time as the clients within the network pickup the changes and eventually all the clients will start using the new DNS servers.

Now, enjoy the ad-free sites. Feel free to ask questions in the comments.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s