Logscript is a Python daemon that parses logs and fires events based on rules.
Rules are stored in /etc/logscript/rules.d
They are json formatted as follows:
{
"name": "test",
"glob": "/var/log/test.log",
"regex": "^(?P<digit>\\d)$",
"occurences": 1,
"script": "helloworld",
"cache_key": "$digit"
}
name: A name for the rule, recorded in logs when triggered
glob: Glob match for log files the rule applies to
regex: Regex pattern to look for, standard JSON special characters (capturing groups permitted)
occurences: Number of times the rule must match before triggering
script: The script to run when the rule triggers
cache_key: A key to cache alerts by (using captured groups from the regex), cached alerts will not be triggered. Set to false to prevent caching.
Scripts are stored in /etc/logscript/scripts.d
They are Python scripts and are executed as such.
The filename is not relevant (although must be a .py Python file).
The 'script' referred to in a rule references the function name within the script file, and not the script filename itself.
Scripts are passed three variables:
rule: The full rule object
line: The log line that triggered the rule
match: The re match object (use match.group('groupname') to use capturing groups)
These can then be used as required within the script (for example, adding the log line to an incident ticket).
For the example rule given earlier, a function named helloworld must be defined in any script file as below:
def helloworld(rule, line, match):
print('hello world' + match.group('digit'))
This shows the 'digit' capturing group from the rule regex being used within the script.
A cached alert will not be triggered again until the cache is cleared.
Alerts can be cached based on the regex capturing groups.
For example, a rule has the following regex looking for a vulnerability in the logs:
(?PCVE.*)
To cache this, based on only the extracted vulnerability, a cache_key can be set as below:
$vulnerability
The vulnerability will then be in the cache, and alerts will not be triggered again (although will be logged).
Install dependencies
sudo apt install python3-sh
Then...
On Debian, just install the deb package:
sudo dpkg -i python3-logscript_0.0.1-1_all.deb
This will create a systemd service which can be used as follows:
systemctl start python3-logscript
systemctl stop python3-logscript
systemctl restart python3-logscript
On other systems, you'll need to put the source somewhere, create the following directories:
/etc/logscript/rules.d
/etc/logscript/scripts.d
Then run as follows:
python logscript
The program was packaged into a debian package as shown below:
python setup.py sdist
dh_make -p logscript_0.0.1 -f dist/logscript-0.0.1.tar.gz -e <email>
vi debian/changelog
vi debian/control
echo 'dist/logscript-0.0.1.tar.gz' > debian/source/include-binaries
dpkg-buildpackage
To make modifications:
- Extract the distribution archive
tar -xvzf dist/logscript-0.0.1.tar.gz
-
Make the necessary changes
-
Update debian/changelog with details of change
-
Rebuild the package
dpkg-buildpackage -b
This will generate a new debian package which can be installed.