-
Notifications
You must be signed in to change notification settings - Fork 14
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Support for more than 65,536 IPs? #10
Comments
It certainly appears to be possible, but perhaps inadvisable. The limit of 65,536 seems to be a default value for Line 73 in 00dbe79
ufw-blocklist/ufw-blocklist-ipsum Line 68 in 00dbe79
You could change them from the default of
You'll want to update this line to load the level 1 list updates: ufw-blocklist/ufw-blocklist-ipsum Line 14 in 00dbe79
And perhaps these lines too, after creating Lines 29 to 31 in 00dbe79
It takes a while, but appears to work:
No idea if this is a good idea. Let the world know :) |
Although it's against the spirit of this repository to write state to disk, the hashing for these longer lists is now a significant performance hit. I added a "restore file" with the output of I also added a I don't expect this to be accepted as a pull request, so I've included the diff here in case that's helpful. diff --git a/after.init b/after.init
index 8d6df2b..d0cecec 100644
--- a/after.init
+++ b/after.init
@@ -26,9 +26,12 @@ set -e
export ipsetname=ufw-blocklist-ipsum
# seed file containing the list of IP addresses to be blocked, one per line
-# curl -sS -f --compressed 'https://raw.githubusercontent.com/stamparm/ipsum/master/levels/4.txt' > /etc/ipsum.4.txt
+# curl -sS -f --compressed 'https://raw.githubusercontent.com/stamparm/ipsum/master/levels/1.txt' > /etc/ipsum.1.txt
# ipset is updated daily by /etc/cron.daily/ufw-blocklist-ipsum
-export seedlist=/etc/ipsum.4.txt
+export seedlist=/etc/ipsum.1.txt
+
+# restore file containing the saved ipset hash for quicker loading
+export restorefile="/etc/${ipsetname}.ipset"
export IPSET_EXE="/sbin/ipset"
# check ipset exists and is executable
@@ -70,7 +73,7 @@ start)
fi
# create an empty ipset
- $IPSET_EXE create $ipsetname hash:net -exist
+ $IPSET_EXE create $ipsetname hash:net -exist maxelem 262144
$IPSET_EXE flush $ipsetname
## Insert firewall rules to take precedence, removing them and adding them back if they already existed
@@ -106,6 +109,12 @@ start)
iptables -A ufw-blocklist-forward -j DROP -m comment --comment "ufw-blocklist-forward"
iptables -I FORWARD -m set --match-set $ipsetname dst -j ufw-blocklist-forward
+ # check if blocklist restore file exists
+ if [ -f "$restorefile" ]; then
+ $IPSET_EXE restore -exist < "$restorefile"
+ exit 0
+ fi
+
# add members to the ipset
# start this in a subshell and then disown the job so we return quickly.
(
@@ -145,6 +154,24 @@ status)
# show the last 10 lines from the logs
journalctl | grep -i blocklist | tail
;;
+save)
+ if set_exists $ipsetname; then
+ $IPSET_EXE save -output save $ipsetname > "$restorefile"
+ fi
+ ;;
+restore)
+ # check that blocklist restore file exists
+ if [ ! -f "$restorefile" ]; then
+ echo "ufw after.init: $restorefile does not exist."
+ exit 1
+ fi
+
+ if set_exists $ipsetname; then
+ $IPSET_EXE flush $ipsetname
+ fi
+
+ $IPSET_EXE restore -exist < "$restorefile"
+ ;;
flush-all)
# flush sets created above. Use /etc/cron.daily/ufw-blocklist-ipsum to repopulate
$IPSET_EXE flush $ipsetname
@@ -163,6 +190,6 @@ flush-all)
;;
*)
echo "'$1' not supported"
- echo "Usage: /etc/ufw/after.init {start|stop|flush-all|status}"
+ echo "Usage: /etc/ufw/after.init {start|stop|flush-all|status|save|restore}"
;;
esac
diff --git a/ufw-blocklist-ipsum b/ufw-blocklist-ipsum
index 0efb978..368c6ef 100644
--- a/ufw-blocklist-ipsum
+++ b/ufw-blocklist-ipsum
@@ -11,7 +11,7 @@
# install this file into /etc/cron.daily/ufw-blocklist-ipsum
## URL must return a text file with one IP address per line
-ipsumurl='https://raw.githubusercontent.com/stamparm/ipsum/master/levels/3.txt'
+ipsumurl='https://raw.githubusercontent.com/stamparm/ipsum/master/levels/1.txt'
# reject the new list if there are fewer than minlen number of ip addresses
minlen=1000
@@ -19,6 +19,9 @@ ipsetname=ufw-blocklist-ipsum
ipset_exe=/usr/sbin/ipset
logger="/usr/bin/logger -t ${ipsetname}"
+## restore file containing the saved ipset hash for quicker loading
+restorefile="/etc/${ipsetname}.ipset"
+
## Check if ipsetname exists. exit if not - ie no set to update
ipsetstatus=$("${ipset_exe}" -t list "${ipsetname}" 2>/dev/null )
RET=$?
@@ -65,7 +68,7 @@ isvalidip () {
## create a temporary ipset
tmpsetname="$(mktemp -u | cut -f2 -d'.')-tmp"
-$ipset_exe -q create "$tmpsetname" hash:net
+$ipset_exe -q create "$tmpsetname" hash:net maxelem 262144
RET=$?
if [ $RET -ne 0 ]; then
$logger -s "error code $RET creating temporary ipset $tmpsetname"
@@ -109,3 +112,6 @@ if [ $RET -ne 0 ]; then
fi
$logger "finished updating $ipsetname. Old entry count: $ipsetcount New count: $cnt of $scrublistlen"
+
+## save the ipset to $restorefile
+$ipset_exe save -output save "$ipsetname" > "$restorefile" |
Hi, I have ufw-blocklist installed and it's working really well, thanks.
I'd like to use the Level 1 IPSUM blocklist (currently about 255K IP's) and I tried editing the script to achieve this, but it seems that only 65,536 IPs can be loaded?
Assuming i'm not doing something wrong, is that a limit that can be extended?
Cheers
The text was updated successfully, but these errors were encountered: