Page 1 of 2

Script to Block Ad Servers

Posted: Fri Dec 25, 2015 11:31 pm
by MrBobbyG
Here is a script to take the hosts from From http://pgl.yoyo.org/adservers/ and convert them to rules for Little Snitch.

It has options to grab the entire list or grab new entries past a certain date.

Just grab the output and copy/paste into Little Snitch.

The script is written to block access to any process, any port.
You can also delete those lines and it will only block Mail.
Or use the options to enter any process, port or protocol you want to block.

Save the script in your home directory, I called mine getadblock.sh.

Make sure to make it executable:

Code: Select all

chmod +x  getadblock.sh


Output looks like:

Code: Select all

# Ad server hostnames for the Little Snitch Mac OSX applications
#
# For more information about this list, see: http://pgl.yoyo.org/adservers/
# ----
# last updated:   Mon, 21 Dec 2015 16:15:24 GMT
# entries:        2443
# format:         little-snitch
# credits:        Peter Lowe - pgl@yoyo.org - http://pgl.yoyo.org/ - https://twitter.com/pgl
# this URL:       http://pgl.yoyo.org/adservers/serverlist.php?hostformat=one-line&showintro=1&mimetype=plaintext
# other formats:  http://pgl.yoyo.org/adservers/formats.php
# policy:         http://pgl.yoyo.org/adservers/policy.php
#


action: deny
direction: outgoing
process: any
owner: me
destination: adblockanalytics.com
port: any
protocol: any
help: From http://pgl.yoyo.org/adservers/


Best Wishes.



-------------

Code: Select all

#!/bin/sh

usage()
{

echo "Usage:      "${0##*/}" [-d day -m month -y year] [-p processname] [-P port] [-r protocol]

OPTIONS:
   -d   day
   -m   month
   -y   year
   -p   Name of OSX Process to block, default is all
   -P   Port # to block
   -r   Protocol o block (6 = IP)
   -?   List this help page"
}

DAY=""
MONTH=""
YEAR=""

#block mail only
PROCESS="/Applications/Mail.app/Contents/MacOS/Mail
via: /System/Library/Frameworks/WebKit.framework/Versions/A/XPCServices/com.apple.WebKit.WebContent.xpc/Contents/MacOS/com.apple.WebKit.WebContent"
PORT=80
PROTOCOL=6

#block any access
PROCESS="any"
PORT="any"
PROTOCOL="any"



while getopts “p:P:r:d:m:y:?” OPTION
do
     case $OPTION in
         p)
           PROCESS=$OPTARG
           ;;
        P)
           PORT=$OPTARG
           ;;
        r)
           PROTOCOL=$OPTARG
           ;;
        d)
           DAY=$OPTARG
           ;;
        m)
           MONTH=$OPTARG
           ;;
        y)
           if [ $OPTARG -lt 100 ]
           then
              let YEAR=$OPTARG+2000
           else
              YEAR=$OPTARG
           fi
           ;;
        ?)
           usage
           exit
           ;;
     esac
done

if [ -n "$DAY" ]
then
   STARTDATE=`date -j -f "%Y%m%d" "$YEAR$MONTH$DAY"`
   COMMENT="From http://pgl.yoyo.org/adservers/ - start date $STARTDATE"
else
   COMMENT="From http://pgl.yoyo.org/adservers/"
fi

RULES=`curl -fs "http://pgl.yoyo.org/adservers/serverlist.php?hostformat=one-line&showintro=0&startdate%5Bday%5D=$DAY&startdate%5Bmonth%5D=$MONTH&startdate%5Byear%5D=$YEAR&mimetype=plaintext" | grep -v \#`
DATE=`curl -fs "http://pgl.yoyo.org/adservers/serverlist.php?hostformat=hosts&showintro=0&startdate%5Bday%5D=&startdate%5Bmonth%5D=&startdate%5Byear%5D=" | grep "# last updated:"`
ENTRIES=`curl -fs "http://pgl.yoyo.org/adservers/serverlist.php?hostformat=hosts&showintro=0&startdate%5Bday%5D=&startdate%5Bmonth%5D=&startdate%5Byear%5D=" | grep "entries"`

echo "# Ad server hostnames for the Little Snitch Mac OSX applications
#
# For more information about this list, see: http://pgl.yoyo.org/adservers/
# ----
$DATE
$ENTRIES
# format:         little-snitch
# credits:        Peter Lowe - pgl@yoyo.org - http://pgl.yoyo.org/ - https://twitter.com/pgl
# this URL:       http://pgl.yoyo.org/adservers/serverlist.php?hostformat=one-line&showintro=1&mimetype=plaintext
# other formats:  http://pgl.yoyo.org/adservers/formats.php
# policy:         http://pgl.yoyo.org/adservers/policy.php
#"
if [ -n "$DAY" ]
then
   echo "# start date:    $STARTDATE"
fi
echo ""

for RULE in $(echo $RULES | tr "," "\n")
do
  echo "
action: deny
direction: outgoing
process: $PROCESS
owner: me
destination: $RULE
port: $PORT
protocol: $PROTOCOL
help: $COMMENT
"
done

exit

Re: Script to Block Ad Servers

Posted: Wed Mar 23, 2016 12:14 pm
by cyphunk
This is exactly what I was looking for. Thanks

Re: Script to Block Ad Servers

Posted: Wed Apr 06, 2016 11:33 pm
by VIGUY
Appreciate you putting that together- but getting an error after it's compiled and run.

./getadblock.sh
./getadblock.sh: line 38: syntax error near unexpected token `)'
./getadblock.sh: line 38: `         p)'


?? Any Ideas?

Cheers

Re: Script to Block Ad Servers

Posted: Thu Apr 07, 2016 6:44 am
by MrBobbyG
Go to http://pgl.yoyo.org/adservers/ instead and download the same file in a format that little snitch can consume.


Example:
http://pgl.yoyo.org/adservers/serverlis ... 5Byear%5D=

Re: Script to Block Ad Servers

Posted: Sat Jul 30, 2016 6:40 pm
by rassy
Please can someone tell me exactly how to import this list into LS?
I'm not that command line savvy.
Thanks.

Re: Script to Block Ad Servers

Posted: Sun Oct 30, 2016 11:34 pm
by hand123
[quote="rassy"]Please can someone tell me exactly how to import this list into LS?
I'm not that command line savvy.
Thanks.[/quote]

I would like to know as well. Have you learned how? Thanks.

Re: Script to Block Ad Servers

Posted: Sun Oct 30, 2016 11:55 pm
by MrBobbyG
You don't need the command line to import the rules. Just open them up in a text editor. Copy them. Open up Rules in Little Snitch. Hit Paste.

Re: Script to Block Ad Servers

Posted: Tue Nov 01, 2016 12:07 am
by hand123
[quote="MrBobbyG"]You don't need the command line to import the rules. Just open them up in a text editor. Copy them. Open up Rules in Little Snitch. Hit Paste.[/quote]

Hello. Is there an easy way to add all of these ad blocking rules from this list all at hone time in the Little Snitch Configuration GUI? I only see the ability to add one at a time. If there are instructions posted somewhere where I can read them on my own, could you direct me to those instructions? Thanks.

Re: Script to Block Ad Servers

Posted: Tue Nov 01, 2016 12:26 am
by MrBobbyG
Open the file in a text editor. Copy all the rows. Paste into Little Snitch Rules.

Bob

Re: Script to Block Ad Servers

Posted: Tue Nov 01, 2016 6:49 am
by hand123
[quote="MrBobbyG"]Open the file in a text editor. Copy all the rows. Paste into Little Snitch Rules.

Bob[/quote]


Obviously I'm missing something. As I try to create a new rule in Little Snitch, I get the following dialog box to only enter one rule at a time. Do you know of a place where I can read instructions on bulk insert of rules from a text file? Thanks.

http://i95.photobucket.com/albums/l131/ ... w7jkq8.png

Re: Script to Block Ad Servers

Posted: Mon Nov 14, 2016 11:04 am
by rassy
Done it!

Go to https://pgl.yoyo.org/as/serverlist.php? ... ttlesnitch

Download the list as a text file.
Crop the header if necessary.
Copy the whole list to the clipboard.
Open Little Snitch "Rules".
Don't do anything else except go to menu bar at the top of the screen, click "Edit>paste".
Wait a few seconds and voila! :)

Thanks for the help everyone.

Re: Script to Block Ad Servers

Posted: Mon Nov 14, 2016 10:58 pm
by hand123
Excellent. Thanks.


[quote="rassy"]Done it!

Go to https://pgl.yoyo.org/as/serverlist.php?hostformat=littlesnitch

Download the list as a text file.
Crop the header if necessary.
Copy the whole list to the clipboard.
Open Little Snitch "Rules".
Don't do anything else except go to menu bar at the top of the screen, click "Edit>paste".
Wait a few seconds and voila! :)

Thanks for the help everyone.[/quote]

Re: Script to Block Ad Servers

Posted: Sat Nov 26, 2016 4:27 pm
by expatmac
rassy wrote:Done it!


Perfect - thanks to all for this!

Re: Script to Block Ad Servers

Posted: Sun Mar 19, 2017 12:09 am
by binaryanomaly
rassy wrote:Done it!

Go to https://pgl.yoyo.org/as/serverlist.php? ... ttlesnitch

Download the list as a text file.
Crop the header if necessary.
Copy the whole list to the clipboard.
Open Little Snitch "Rules".
Don't do anything else except go to menu bar at the top of the screen, click "Edit>paste".
Wait a few seconds and voila! :)

Thanks for the help everyone.


This basically works, unfortunately there are a few downsides of the approach:
    - The list is quite aggressive, it breaks i.e. youtube completely
    - Having 2000+ rules for "Any Process" makes the UI sort of unusable to have an overview and efficiently manage all profiles/rules. Labels and folding support would be nice
    - The update process is not solved but manual

Would love to see better support from little snitch for integration of adblocking lists and auto updates of these.

Re: Script to Block Ad Servers

Posted: Thu May 04, 2017 12:29 pm
by hummingdrone
Yes that would be great.