Script to automate building an adblocking hosts file

[SteveRiley]

Do any of you use AdAway on Android (GPLv3)? I think it does exactly this, pulls a list of websites from 4 different sources, merges them and adds them to the hosts file.

AdAway was my inspiration for writing the script. But rather than look at their source code, I wanted to write my own from scratch as a learning exercise. This was the first Bash shell script I ever wrote.

For some reason AdAway asks you to reboot afterwards, does this script take effect immediately?

I think that's because Android only reads its /etc/hosts file during boot. By default, "regular" Linux consults /etc/hosts before DNS every time an application performs a host name lookup. (*) No reboot is necessary. You might, though, need to flush your resolver cache. One of these should do it:

sudo service dnsmasq restart

sudo service networking force-reload

The reason for the difference in performance between this and browser based blockers must be the effort taken to reshape the page to remove the empty advert spaces, right? I think I'd miss that function, but will give this a try.

Depends on how the site's HTML is structured. You might see a frame with an error, or the frame might simply not appear. I've experienced both. My script definitely makes Phoronix a much more pleasing site to visit.

If you visit a site with pop-ups, do you still get a window, just nothing in it?

Dunno, I always enable popup blockers.


(*) Edit: Kubicle reminded me in a post below that this behavior can be changed.

[jlittle]

Theoretically, you can include "./" in $PATH and have your the system check the cwd for executables....

If you have a blank entry in your $PATH, including starting or ending with the separator colon, that means the cwd. I've done that for three decades. It's only an issue if the cwd is writable by people (or bots or software) you don't trust; we don't do that in a typical linux install.

Regards, John Little

[kubicle]

Wait, that would be so convenient. Why doesn't Linux do that?

Theoretically, you can include "./" in $PATH and have your the system check the cwd for executables, but as a general rule it is much better to have your binaries in $PATH than make $PATH dance around your binaries.

[Feathers McGraw]

This thread is excellent! A really good read, thanks to all contributors. This must be how a lot of free software projects get started!

Do any of you use AdAway on Android (GPLv3)? I think it does exactly this, pulls a list of websites from 4 different sources, merges them and adds them to the hosts file. For some reason AdAway asks you to reboot afterwards, does this script take effect immediately?

The reason for the difference in performance between this and browser based blockers must be the effort taken to reshape the page to remove the empty advert spaces, right? I think I'd miss that function, but will give this a try.

If you visit a site with pop-ups, do you still get a window, just nothing in it?

Feathers

[SteveRiley]

Instead of manually doing this, why not just have it in the script? It seems to do fine for me.

Feel free to modify the source as you see fit. My goal was to write a script that does one thing: create the new hosts file. The mechanism for replacing the existing one is, as you've seen, an exercise left for the reader

[oshunluvr]

Are you questioning how to run a weekly job? If yes, in crontab select every month and 1 day a week.

# Minute   Hour   Day of Month       Month          Day of Week        Command
# (0-59)  (0-23)     (1-31)    (1-12 or Jan-Dec)  (0-6 or Sun-Sat)

So; 0 0 * * 1 would run every Monday at midnight

The script outputs the file ~/hosts-block. Each time you run it, you'll need to manually replace your existing host file with this command:
Code:

sudo cp ~/hosts-block /etc/hosts

Instead of manually doing this, why not just have it in the script? It seems to do fine for me. Also, I was thinking of using KDE's Autostart instead of a weekly cron job, but it seems there are issues with it because there are no options to run weekly, as Windows does, unless someone knows of another option?

[SteveRiley]

Fixed that...somehow, URL BBCODES snuck into there.

[oshunluvr]

Cut and paste doesn't work 'cause the forum is truncating the fourth source link.

[SecretCode]

[SteveRiley]

Oh indeed. I'm not suggesting you did something wrong. I'm suggesting a slight improvement to Steve's instructions. ...

Alrighty... added an instruction to remind folks how to run it with ~/gethosts. And added the bit about ~/bin. (I didn't know this!)

[SecretCode]

Oh indeed. I'm not suggesting you did something wrong. I'm suggesting a slight improvement to Steve's instructions. ...

[Snowhog]

Save the text above into a file called ~/gethosts.

I followed Steve's instructions exactly as stated to above -- which saves the file in the users home directory.

[SecretCode]

Windows, which always checks the current subdirectory before following %PATH%

Wait, that would be so convenient. Why doesn't Linux do that?

Oh, I remember. Security!

[SteveRiley]

Right... unlike Windows, which always checks the current subdirectory before following %PATH%, Linux only follows $PATH. So

./gethosts

would work if your current working directory is the one containing the script. Alternately, if your script is in your home directory,

~/gethosts

will execute the script regardless of what your current working directory is.