Script to Block Ad Servers

General discussions about Little Snitch
MrBobbyG
Posts: 4
Joined: Fri Dec 25, 2015 1:47 am

Script to Block Ad Servers

Postby MrBobbyG » Fri Dec 25, 2015 11:31 pm

Here is a script to take the hosts from From http://pgl.yoyo.org/adservers/ and convert them to rules for Little Snitch.

It has options to grab the entire list or grab new entries past a certain date.

Just grab the output and copy/paste into Little Snitch.

The script is written to block access to any process, any port.
You can also delete those lines and it will only block Mail.
Or use the options to enter any process, port or protocol you want to block.

Save the script in your home directory, I called mine getadblock.sh.

Make sure to make it executable:

Code: Select all

chmod +x  getadblock.sh


Output looks like:

Code: Select all

# Ad server hostnames for the Little Snitch Mac OSX applications
#
# For more information about this list, see: http://pgl.yoyo.org/adservers/
# ----
# last updated:   Mon, 21 Dec 2015 16:15:24 GMT
# entries:        2443
# format:         little-snitch
# credits:        Peter Lowe - pgl@yoyo.org - http://pgl.yoyo.org/ - https://twitter.com/pgl
# this URL:       http://pgl.yoyo.org/adservers/serverlist.php?hostformat=one-line&showintro=1&mimetype=plaintext
# other formats:  http://pgl.yoyo.org/adservers/formats.php
# policy:         http://pgl.yoyo.org/adservers/policy.php
#


action: deny
direction: outgoing
process: any
owner: me
destination: adblockanalytics.com
port: any
protocol: any
help: From http://pgl.yoyo.org/adservers/


Best Wishes.



-------------

Code: Select all

#!/bin/sh

usage()
{

echo "Usage:      "${0##*/}" [-d day -m month -y year] [-p processname] [-P port] [-r protocol]

OPTIONS:
   -d   day
   -m   month
   -y   year
   -p   Name of OSX Process to block, default is all
   -P   Port # to block
   -r   Protocol o block (6 = IP)
   -?   List this help page"
}

DAY=""
MONTH=""
YEAR=""

#block mail only
PROCESS="/Applications/Mail.app/Contents/MacOS/Mail
via: /System/Library/Frameworks/WebKit.framework/Versions/A/XPCServices/com.apple.WebKit.WebContent.xpc/Contents/MacOS/com.apple.WebKit.WebContent"
PORT=80
PROTOCOL=6

#block any access
PROCESS="any"
PORT="any"
PROTOCOL="any"



while getopts “p:P:r:d:m:y:?” OPTION
do
     case $OPTION in
         p)
           PROCESS=$OPTARG
           ;;
        P)
           PORT=$OPTARG
           ;;
        r)
           PROTOCOL=$OPTARG
           ;;
        d)
           DAY=$OPTARG
           ;;
        m)
           MONTH=$OPTARG
           ;;
        y)
           if [ $OPTARG -lt 100 ]
           then
              let YEAR=$OPTARG+2000
           else
              YEAR=$OPTARG
           fi
           ;;
        ?)
           usage
           exit
           ;;
     esac
done

if [ -n "$DAY" ]
then
   STARTDATE=`date -j -f "%Y%m%d" "$YEAR$MONTH$DAY"`
   COMMENT="From http://pgl.yoyo.org/adservers/ - start date $STARTDATE"
else
   COMMENT="From http://pgl.yoyo.org/adservers/"
fi

RULES=`curl -fs "http://pgl.yoyo.org/adservers/serverlist.php?hostformat=one-line&showintro=0&startdate%5Bday%5D=$DAY&startdate%5Bmonth%5D=$MONTH&startdate%5Byear%5D=$YEAR&mimetype=plaintext" | grep -v \#`
DATE=`curl -fs "http://pgl.yoyo.org/adservers/serverlist.php?hostformat=hosts&showintro=0&startdate%5Bday%5D=&startdate%5Bmonth%5D=&startdate%5Byear%5D=" | grep "# last updated:"`
ENTRIES=`curl -fs "http://pgl.yoyo.org/adservers/serverlist.php?hostformat=hosts&showintro=0&startdate%5Bday%5D=&startdate%5Bmonth%5D=&startdate%5Byear%5D=" | grep "entries"`

echo "# Ad server hostnames for the Little Snitch Mac OSX applications
#
# For more information about this list, see: http://pgl.yoyo.org/adservers/
# ----
$DATE
$ENTRIES
# format:         little-snitch
# credits:        Peter Lowe - pgl@yoyo.org - http://pgl.yoyo.org/ - https://twitter.com/pgl
# this URL:       http://pgl.yoyo.org/adservers/serverlist.php?hostformat=one-line&showintro=1&mimetype=plaintext
# other formats:  http://pgl.yoyo.org/adservers/formats.php
# policy:         http://pgl.yoyo.org/adservers/policy.php
#"
if [ -n "$DAY" ]
then
   echo "# start date:    $STARTDATE"
fi
echo ""

for RULE in $(echo $RULES | tr "," "\n")
do
  echo "
action: deny
direction: outgoing
process: $PROCESS
owner: me
destination: $RULE
port: $PORT
protocol: $PROTOCOL
help: $COMMENT
"
done

exit

cyphunk
Posts: 1
Joined: Wed Mar 23, 2016 12:14 pm

Re: Script to Block Ad Servers

Postby cyphunk » Wed Mar 23, 2016 12:14 pm

This is exactly what I was looking for. Thanks

VIGUY
Posts: 1
Joined: Tue Apr 05, 2016 11:54 pm

Re: Script to Block Ad Servers

Postby VIGUY » Wed Apr 06, 2016 11:33 pm

Appreciate you putting that together- but getting an error after it's compiled and run.

./getadblock.sh
./getadblock.sh: line 38: syntax error near unexpected token `)'
./getadblock.sh: line 38: `         p)'


?? Any Ideas?

Cheers

MrBobbyG
Posts: 4
Joined: Fri Dec 25, 2015 1:47 am

Re: Script to Block Ad Servers

Postby MrBobbyG » Thu Apr 07, 2016 6:44 am

Go to http://pgl.yoyo.org/adservers/ instead and download the same file in a format that little snitch can consume.


Example:
http://pgl.yoyo.org/adservers/serverlis ... 5Byear%5D=

rassy
Posts: 2
Joined: Sat Jul 30, 2016 6:38 pm

Re: Script to Block Ad Servers

Postby rassy » Sat Jul 30, 2016 6:40 pm

Please can someone tell me exactly how to import this list into LS?
I'm not that command line savvy.
Thanks.

hand123
Posts: 5
Joined: Sun Oct 30, 2016 11:33 pm

Re: Script to Block Ad Servers

Postby hand123 » Sun Oct 30, 2016 11:34 pm

[quote="rassy"]Please can someone tell me exactly how to import this list into LS?
I'm not that command line savvy.
Thanks.[/quote]

I would like to know as well. Have you learned how? Thanks.

MrBobbyG
Posts: 4
Joined: Fri Dec 25, 2015 1:47 am

Re: Script to Block Ad Servers

Postby MrBobbyG » Sun Oct 30, 2016 11:55 pm

You don't need the command line to import the rules. Just open them up in a text editor. Copy them. Open up Rules in Little Snitch. Hit Paste.

hand123
Posts: 5
Joined: Sun Oct 30, 2016 11:33 pm

Re: Script to Block Ad Servers

Postby hand123 » Tue Nov 01, 2016 12:07 am

[quote="MrBobbyG"]You don't need the command line to import the rules. Just open them up in a text editor. Copy them. Open up Rules in Little Snitch. Hit Paste.[/quote]

Hello. Is there an easy way to add all of these ad blocking rules from this list all at hone time in the Little Snitch Configuration GUI? I only see the ability to add one at a time. If there are instructions posted somewhere where I can read them on my own, could you direct me to those instructions? Thanks.

MrBobbyG
Posts: 4
Joined: Fri Dec 25, 2015 1:47 am

Re: Script to Block Ad Servers

Postby MrBobbyG » Tue Nov 01, 2016 12:26 am

Open the file in a text editor. Copy all the rows. Paste into Little Snitch Rules.

Bob

hand123
Posts: 5
Joined: Sun Oct 30, 2016 11:33 pm

Re: Script to Block Ad Servers

Postby hand123 » Tue Nov 01, 2016 6:49 am

[quote="MrBobbyG"]Open the file in a text editor. Copy all the rows. Paste into Little Snitch Rules.

Bob[/quote]


Obviously I'm missing something. As I try to create a new rule in Little Snitch, I get the following dialog box to only enter one rule at a time. Do you know of a place where I can read instructions on bulk insert of rules from a text file? Thanks.

http://i95.photobucket.com/albums/l131/ ... w7jkq8.png

rassy
Posts: 2
Joined: Sat Jul 30, 2016 6:38 pm

Re: Script to Block Ad Servers

Postby rassy » Mon Nov 14, 2016 11:04 am

Done it!

Go to https://pgl.yoyo.org/as/serverlist.php? ... ttlesnitch

Download the list as a text file.
Crop the header if necessary.
Copy the whole list to the clipboard.
Open Little Snitch "Rules".
Don't do anything else except go to menu bar at the top of the screen, click "Edit>paste".
Wait a few seconds and voila! :)

Thanks for the help everyone.

hand123
Posts: 5
Joined: Sun Oct 30, 2016 11:33 pm

Re: Script to Block Ad Servers

Postby hand123 » Mon Nov 14, 2016 10:58 pm

Excellent. Thanks.


[quote="rassy"]Done it!

Go to https://pgl.yoyo.org/as/serverlist.php?hostformat=littlesnitch

Download the list as a text file.
Crop the header if necessary.
Copy the whole list to the clipboard.
Open Little Snitch "Rules".
Don't do anything else except go to menu bar at the top of the screen, click "Edit>paste".
Wait a few seconds and voila! :)

Thanks for the help everyone.[/quote]

expatmac
Posts: 1
Joined: Sat Nov 26, 2016 4:25 pm

Re: Script to Block Ad Servers

Postby expatmac » Sat Nov 26, 2016 4:27 pm

rassy wrote:Done it!


Perfect - thanks to all for this!

binaryanomaly
Posts: 2
Joined: Sun Mar 19, 2017 12:03 am

Re: Script to Block Ad Servers

Postby binaryanomaly » Sun Mar 19, 2017 12:09 am

rassy wrote:Done it!

Go to https://pgl.yoyo.org/as/serverlist.php? ... ttlesnitch

Download the list as a text file.
Crop the header if necessary.
Copy the whole list to the clipboard.
Open Little Snitch "Rules".
Don't do anything else except go to menu bar at the top of the screen, click "Edit>paste".
Wait a few seconds and voila! :)

Thanks for the help everyone.


This basically works, unfortunately there are a few downsides of the approach:
    - The list is quite aggressive, it breaks i.e. youtube completely
    - Having 2000+ rules for "Any Process" makes the UI sort of unusable to have an overview and efficiently manage all profiles/rules. Labels and folding support would be nice
    - The update process is not solved but manual

Would love to see better support from little snitch for integration of adblocking lists and auto updates of these.

hummingdrone
Posts: 11
Joined: Fri Jan 06, 2017 6:04 pm

Re: Script to Block Ad Servers

Postby hummingdrone » Thu May 04, 2017 12:29 pm

Yes that would be great.


Return to “Little Snitch General”

Who is online

Users browsing this forum: No registered users and 4 guests