Steve has written a great script to automate building an adblocking hosts file.
This is great for blocking adverts on a specific device, but I'd like to use it on my router to block all adverts at home without the fuss of updating the hosts file on each device individually: my girlfriend's laptop, her phone, two tablets, etc.
I've created this thread because I don't want to take over Steve's, which might be best remaining specific to Kubuntu.
My router runs OpenWrt, but the plan is to create something that would run on any Linux router (DD-WRT etc).
Steve's original script, which creates a host file from multiple sources, is here (call it gethosts.sh):
I wanted to re-enable google analytics for my website, which is on a server behind my router, so I have written another script that will allow you to re-enable certain hosts.
Call this one edit-hosts.sh
The script takes the product of Steve's gethosts.sh (which is the file ~/hosts-block), and removes any lines with terms that match a whitelist file that you create. The whitelist file should contain one phrase per line, and can be located anywhere: you tell edit-hosts.sh where it is when you call it (e.g. edit-hosts.sh ~/whitelist). The product is the file ~/hosts-block-less-whitelist.
The script works on Kubuntu,but I'm still working on editing it so it will work on OpenWrt and I've now figured out how to make it work with OpenWrt too. In fact, I've merged the two scripts into one and added some extra bits. Have a look in the next post for the new script.
This is great for blocking adverts on a specific device, but I'd like to use it on my router to block all adverts at home without the fuss of updating the hosts file on each device individually: my girlfriend's laptop, her phone, two tablets, etc.
I've created this thread because I don't want to take over Steve's, which might be best remaining specific to Kubuntu.
My router runs OpenWrt, but the plan is to create something that would run on any Linux router (DD-WRT etc).
Steve's original script, which creates a host file from multiple sources, is here (call it gethosts.sh):
Code:
#!/bin/bash # If this is our first run, save a copy of the system's original hosts file and set to read-only for safety if [ ! -f ~/hosts-system ] then echo "Saving copy of system's original hosts file..." cp /etc/hosts ~/hosts-system chmod 444 ~/hosts-system fi # Perform work in temporary files temphosts1=$(mktemp) temphosts2=$(mktemp) # Obtain various hosts files and merge into one echo "Downloading ad-blocking hosts files..." wget -nv -O - http://winhelp2002.mvps.org/hosts.txt >> $temphosts1 wget -nv -O - http://hosts-file.net/ad_servers.asp >> $temphosts1 wget -nv -O - http://someonewhocares.org/hosts/hosts >> $temphosts1 wget -nv -O - "http://pgl.yoyo.org/adservers/serverlist.php?hostformat=hosts&showintro=0&mimetype=plaintext" >> $temphosts1 # Do some work on the file: # 1. Remove MS-DOS carriage returns # 2. Delete all lines that don't begin with 127.0.0.1 # 3. Delete any lines containing the word localhost because we'll obtain that from the original hosts file # 4. Replace 127.0.0.1 with 0.0.0.0 because then we don't have to wait for the resolver to fail # 5. Scrunch extraneous spaces separating address from name into a single tab # 6. Delete any comments on lines # 7. Clean up leftover trailing blanks # Pass all this through sort with the unique flag to remove duplicates and save the result echo "Parsing, cleaning, de-duplicating, sorting..." sed -e 's/\r//' -e '/^127.0.0.1/!d' -e '/localhost/d' -e 's/127.0.0.1/0.0.0.0/' -e 's/ \+/\t/' -e 's/#.*$//' -e 's/[ \t]*$//' < $temphosts1 | sort -u > $temphosts2 # Combine system hosts with adblocks echo Merging with original system hosts... echo -e "\n# Ad blocking hosts generated "$(date) | cat ~/hosts-system - $temphosts2 > ~/hosts-block # Clean up temp files and remind user to copy new file echo "Cleaning up..." rm $temphosts1 $temphosts2 echo "Done." echo echo "Copy ad-blocking hosts file with this command:" echo " sudo cp ~/hosts-block /etc/hosts" echo echo "You can always restore your original hosts file with this command:" echo " sudo cp ~/hosts-system /etc/hosts" echo "so don't delete that file! (It's saved read-only for your protection.)" echo
Call this one edit-hosts.sh
Code:
#!/bin/bash
#Before calling this script, create a whitelist file containing phrases to allow, one phrase per line
if [ $# -ne 1 ]; then
echo "Usage: $0 whitelist_file_location"
exit
fi
INPUT_FILE=~/hosts-block
OUTPUT_FILE=~/hosts-block-less-whitelist
#first, remove empty lines from whitelist_file (or next step will throw an error)
sed '/^$/d' $1 > tt
mv tt $1
echo 'Removed empty lines from whitelist_file'
cp $INPUT_FILE $OUTPUT_FILE
#now, read lines from whitelist file and remove entries with matching content from OUTPUT_FILE
cat $1 | while read line; do
sed -e '/'$line'/d' $OUTPUT_FILE > tt
mv tt $OUTPUT_FILE
echo 'Removed any lines containing' $line
done
The script works on Kubuntu,
Comment