-
Notifications
You must be signed in to change notification settings - Fork 6
/
ipdbtools.1
322 lines (322 loc) · 12.4 KB
/
ipdbtools.1
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
.\" man file ipdbtools(1)
.\"
.\" Created by Dr. Rolf Jansen on 2016-07-10.
.\" Copyright © 2016-2018 Dr. Rolf Jansen. All rights reserved.
.\"
.\" Redistribution and use in source and binary forms, with or without modification,
.\" are permitted provided that the following conditions are met:
.\"
.\" 1. Redistributions of source code must retain the above copyright notice,
.\" this list of conditions and the following disclaimer.
.\"
.\" 2. Redistributions in binary form must reproduce the above copyright notice,
.\" this list of conditions and the following disclaimer in the documentation
.\" and/or other materials provided with the distribution.
.\"
.\" THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS
.\" OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY
.\" AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER
.\" OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
.\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
.\" DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER
.\" IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF
.\" THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
.\"
.Dd 2016-08-20
.Dt ipdbtools 1
.Os FreeBSD, Darwin
.Sh NAME
.Nm ipup
.Nm - ipdb
.Nm - ipdb-update.sh
.Nd Tools for generating IP based Geo-blocking and Geo-routing tables
in order to configure the system's firewall and/or routing facilities
.sp
.Sh SYNOPSIS
.Nm
.Op Fl h
.Op Fl r Ar bstfiles
.Ao Ar IP_address Ac
.sp
.Nm
.Op Fl h
.Fl t Ar CC:DD:.. | CC=nnnnn:DD=mmmmm:.. | \*q\*q
.Op Fl n Ar table_number
.Op Fl v Ar table_value
.Op Fl x Ar offset
.Op Fl p
.Op Fl 4
.Op Fl 6
.Op Fl r Ar bstfiles
.sp
.Nm
.Op Fl h
.Fl q Ar CC
.sp
.Nm ipdb
.Ao Ar outnamebase Ac Ao Ar datafile1 Ac Ao Ar datafile2 Ac Ao Ar datafile3 Ac ...
.sp
.Nm ipdb-update.sh
.Op Ao Ar ftp.RIR__mirror_name.net Ac
.sp
.Sh DESCRIPTION
In general, access control by the \fBfirewall\fP is established by selectors that can be attributed to incoming and outgoing IP packets, like
physical interfaces on which the packets are going, source and destination IP addresses, protocol types, port numbers, content types and
content, etc., and \fBrouting\fP is determined by destination IP addresses. The Geo-location would be just another selector, but this
information is not carried explicitly with IP packets, however, it can be obtained using the IP address as a key for looking-up the location
in an IP database. For example, the country to which a given IP address is delegated, can be obtained with the common Unix tool \fBwhois\fP(1).
.Pp
\fBwhois\fP does an online look-up in the IP databases of the 5 \fBR\fPegional \fBI\fPnternet \fBR\fPegistries (AFRINIC, APNIC, ARIN, LACNIC, RIPENCC),
and this is the most reliable way to obtain the country code for a given IP address, because the \fBRIR\fP's are the authorities for
internet number delegations. Unfortunately, online database look-up is by far too slow for even thinking about being utilized
on the firewall level, where IP packets need to be processed in a microsecond time scale. Therefore, a locally maintained
IP Geo-location database is indispensable in the given respect. The System's own routing and filtering tables can be configured
to do these tasks if there is a source of the appropriate data. The \fBipdbtools\fP(1) are designed to provide this data
and to assist managing and using it.
.Pp
The three tools in the package are:
.Bl -tag -width " ipdb-update.sh"
.It \ \ \fBipup\fP
A tool to utilize the IP Geo-location tables to look-up the country code belonging to an IP address or generate sorted lists of CIDR
compatible IP address/masklen pairs per country code, formatted as raw CIDR ranges or \fBipfw\fP(8) table construction directives.
.It \ \ \fBipdb\fP
A tool for consolidating the IP address ranges from the RIR delegation statistics files into binary sorted tables of IP ranges + country codes,
suitable for direct utilization by the \fBipup\fP look-up tool. IPv4 and IPv6 ranges are stored in separate files.
.It \ \ \fBipdb-update.sh\fP
A shell script to update the IP Geo-location tables by downloading the 5 RIR delegation statistics files from a \fBR\fPegional \fBI\fPnternet
\fBR\fPegistry mirror, and invoking \fBipdb\fP to generate the binary sorted tables. It is suitable for invocation by cron.
.El
.sp
.Sh Setting up the local IP Geo-location tables
The authoritative IP Geo-location information must be obtained from the 5 RIR's, and compiled into an optimized format, suitable for quickly
looking-up the country codes of given IP addresses. This information is present in so called delegation statistics files on the
ftp servers of each RIR, and APNIC, LACNIC and RIPENCC mirror the files of the other RIR's on their servers - as of the date
of this writing, ARIN and AFRINIC do not mirror current delegation statistics of the other RIR's.
.Pp
1) Choose one of the three useful mirror sites, depending on where you are located:
.Bl -tag -width " ftp.lacnic.net "
.It \ \ \fBftp.ripe.net\fP
RIPENCC -- Europe and Eurasia [default mirror]
.It \ \ \fBftp.apnic.net\fP
APNIC -- Asia Pacific
.It \ \ \fBftp.lacnic.net\fP
LACNIC -- Latin America and Caribbean
.El
.Pp
2) As user root execute the shell script \fBipdb-update.sh\fP with the chosen mirror as the parameter, for example ftp.apnic.net:
.Pp
# ipdb-update.sh ftp.apnic.net
.br
>>>>
.br
/usr/local/etc/ipdb/IPRanges/afrinic.md5 100% of 74 B 277 kBps 0s
.br
/usr/local/etc/ipdb/IPRanges/afrinic.dat 100% of 397 kB 1330 kBps 0s
.br
/usr/local/etc/ipdb/IPRanges/apnic.md5 100% of 73 B 264 kBps 0s
.br
/usr/local/etc/ipdb/IPRanges/apnic.dat 100% of 4045 kB 1259 kBps 4s
.br
/usr/local/etc/ipdb/IPRanges/arin.md5 100% of 67 B 246 kBps 0s
.br
/usr/local/etc/ipdb/IPRanges/arin.dat 100% of 8160 kB 1270 kBps 7s
.br
/usr/local/etc/ipdb/IPRanges/lacnic.md5 100% of 74 B 274 kBps 0s
.br
/usr/local/etc/ipdb/IPRanges/lacnic.dat 100% of 1870 kB 1271 kBps 2s
.br
/usr/local/etc/ipdb/IPRanges/ripencc.md5 100% of 74 B 270 kBps 0s
.br
/usr/local/etc/ipdb/IPRanges/ripencc.dat 100% of 10 MB 1258 kBps 9s
.br
ipdb v1.2b, Copyright © 2016-2018 Dr. Rolf Jansen
.br
Processing RIR data files ...
.sp
afrinic.dat apnic.dat arin.dat lacnic.dat ripencc.dat
.sp
Number of processed IP-Ranges = 113267
.sp
As shown above, this will download the delegation statistics data together with MD5 hashes for integrity checking into the directory
.Ar /usr/local/etc/ipdb/IPRanges/ .
Then the \fBipdb\fP tool will process the data files and generate two binary sorted table (.bst) files, one for the IPv4 ranges
.Ar /usr/local/etc/IPRanges/ipcc.bst.v4
and another one for the IPv6 ranges
.Ar /usr/local/etc/IPRanges/ipcc.bst.v6 .
.sp
.Sh USAGE AND OPTIONS
\fBQuering the local IP Geo-location tables\fP
.sp
Use the
.Nm
tool for the various queries:
.Bl -tag -width -indent
.It Fl h
Show the usage instructions.
.It Op Fl r Ar bstfiles
Base path to the binary sorted tables (.v4 and .v6) with the consolidated IP ranges which were generated by the \fBipdb\fP tool [default: \fI/usr/local/etc/ipdb/IPRanges/ipcc.bst\fP].
.sp
.It \fBFirst usage form\fP -- CC query:
.It Ao Ar IP_address Ac
IPv4 or IPv6 address for which the country code should be looked-up.
.sp
.It \fBSecond usage form\fP -- firewall and routing table generation:
.It Fl t Ar CC:DD:.. | CC=nnnnn:DD=mmmmm:.. | CC:DD=ooooo:EE;.. | \*q\*q
Output all IP address/masklen pairs belonging to the listed countries, given by 2 letter capital country codes, separated by colon. An empty CC
list (denoted by "") means any country code. A table value can be assigned per country code in the following manner:
.br
\ \ -t BR=10000:DE=10100:US:CA:AU=10200.
.br
In the case of no assignment, no value [0] or the global value defined by either the -v or the -x option is utilized.
.It Op Fl n Ar table_number
The ipfw table number between 0 and 65534 [default: 0].
.It Op Fl v Ar table_value
A global 32-bit unsigned value for all ipfw table entries [default: no value -> 0].
.It Op Fl x Ar offset
Decimal encode the given \fICC\fP and add it to the \fIoffset\fP for computing the table value:
.br
value = \fIoffset\fP + ((C1 - 'A')*26 + (C2 - 'A'))*10.
.It Op Fl p
Plain IP table generation, i.e. without ipfw table construction directives, and any -n, -v and -x flags are ignored in this mode.
.It Op Fl 4
Process only the \fIIPv4\fP address ranges.
.It Op Fl 6
Process only the \fIIPv6\fP address ranges.
.sp
.It \fBThird usage form\fP -- compute the encoded value of a country code:
.It Fl q Ar CC
The country code to be encoded (see -x flag above).
.El
.sp
.Sh EXAMPLES
Check whether the IP Geo-location tables are ready by looking-up some addresses using the
.Nm
tool:
.br
.sp
$ ipup 62.175.157.33
.br
\ \ \ 62.175.157.33 in 62.174.0.0 - 62.175.255.255 in ES
.br
.sp
$ ipup 141.33.17.2
.br
\ \ \ 141.33.17.2 in 141.12.0.0 - 141.80.255.255 in DE
.br
.sp
$ ipup 99.67.80.80
.br
\ \ \ 99.67.80.80 in 98.160.0.0 - 99.191.255.255 in US
.br
.sp
$ ipup 192.168.1.1
.br
\ \ \ 192.168.1.1 not found
.br
.sp
$ ipup 2001:0618:85a3:08d3:1319:8a2e:0370:7344
.br
\ \ \ 2001:0618:85a3:08d3:1319:8a2e:0370:7344 in 2001:618:0:0:0:0:0:0 - 2001:618:ffff:ffff:ffff:ffff:ffff:ffff in CH
.br
.sp
.Sh Firewall Examples
.Nm
can be used for Geo-blocking together with \fBipfw\fP(8). For this purpose,
.Nm
would generate tables of CIDR ranges for the selected country codes, and these tables can be directly piped into
\fBipfw\fP(8). The respective configuration script may contain something like:
.sp
\&.\&.\&.
.br
# Allow only web access from DE, BR, US:
.br
/usr/local/bin/ipup -t DE:BR:US -n 7 | /sbin/ipfw -q /dev/stdin
.br
/sbin/ipfw -q add 70 deny tcp from not table\e(7\e) to any 80,443 in recv em0 setup
.br
\&.\&.\&.
.sp
\fBOR\fP vice versa:
.sp
\&.\&.\&.
.br
# Deny web access from certain countries we don't like this week:
.br
/usr/local/bin/ipup -t TR:SA:RU:GB -n 66 | /sbin/ipfw -q /dev/stdin
.br
/sbin/ipfw -q add 70 allow tcp from not table\e(66\e) to any 80,443 in recv em0 setup
.br
\&.\&.\&.
.br
.sp
In the case of a different firewall facility, a plain table (without ipfw directives) can be generated using
.Nm
by specifying the \fB-p\fP flag. The table may be piped into a pre-processing command before being passed to the firewall utility:
.sp
# Output data in the format of some other fictional firewall:
.br
/usr/local/bin/ipup -t FR:ES:PT -x0 | awk '{print "add-filter", $4, $5}'
.sp
\fBOR\fP
.sp
/usr/local/bin/ipup -p -t US:CA | while read TABLE NUM ADD ADDR VAL; do myfirewall add filter $ADDR value $VAL; done
.sp
.br
.Sh Routing Example
.Nm
is well suited for manipulating the system's routing table by the way of the \fBroute\fP(8) utility:
.br
\&.\&.\&.
.br
# Force packets to Austria to take a different route:
.br
/usr/local/bin/ipup -p -t AT | while read LINE; do /sbin/route add $LINE $SOMEROUTER; done
.br
\&.\&.\&.
.br
.sp
.Sh Cronjob for keeping the IP Geo-location tables updated
\fBipdb-update.sh\fP may be executed by a weekly (perhaps daily) cronjob,
for this you might want to add the following entry to /etc/crontab:
.sp
\&.\&.\&.
.br
# Weekly update of the IP Geo-location tables
.br
\ \ 5 4 * * 6 root /usr/local/bin/ipdb-update.sh ftp.apnic.net > /dev/null 2>&1 && /fullpath/to/fw_or_router_reinit_script
.br
\&.\&.\&.
.br
.sp
.Sh FILES
.Bl -tag -width
.It Pa /usr/local/etc/IPRanges/
directory for maintaining the IP Geo-location tables
.It Pa /usr/local/etc/IPRanges/ipcc.bst.v4
binary (\fIuint32_t\fP) sorted table of IPv4 ranges and its country codes
.It Pa /usr/local/etc/IPRanges/ipcc.bst.v6
binary (\fIuint128t\fP) sorted table of IPv6 ranges and its country codes
.El
.sp
.Sh SEE ALSO
.Xr whois 1 ,
.Xr ipfw 8 ,
.Xr route 8
.sp
in Ports:
.Xr ip2cc 1 ,
.Xr IP::Country 3
.sp
.Sh AUTHOR
.An Dr. Rolf Jansen - Copyright © 2016 - all rights reserved.
.sp
.Sh IMPORTANT NOTE
Improper use of the \fBipdb tools\fP may result in erroneous IP tables, and firewalls
or routers may be rendered non-functional once configured with incorrect tables.
.Pp
In NO event shall the author and/or copyright owner be liable for ANY damages
resulting from ANY use of this software. Use the \fBipdb tools\fP at your own risk!
.sp
.Sh BUGS
The \fBipdb tools\fP have been carefully developed and tested. Anyway, the tools
are provided without any expressed or implied warrantee of being 100 % bug free.