LightningFilter is a high-speed traffic filtering mechanism that performs authentication, rate limiting, and duplicate detection. LightningFilter uses the DPDK framework, enabling high-speed packet processing.
This repository contains the open-source version, which offers at least the core functionalities of a LightningFilter. The closed-source version provides additional functionalities.
The software is licensed under BSD-3.
The license might change to Apache v2 if possible.
DPDK (BSD 3) https://www.dpdk.org/
json-parser (BSD 2) https://github.com/json-parser/json-parser
murmurhash (public domain) https://github.com/aappleby/smhasher/blob/master/src/MurmurHash3.cpp
hashdict (MIT licensed) https://github.com/exebook/hashdict.c
See docs/Installation.md for detailed information.
There are mainly two possibilities to build LightingFilter: with the help of a docker container or natively on a machine. Both of the options rely on Ubuntu 22.04.
To install dependencies, we use the script provided in usertools and then set the required environment variables:
./usertools/install_deps.sh
source dependencies/env_vars
To build LightningFilter we use CMake.
mkdir build
cd build
cmake ../
make
For the docker container build, install docker and add the user to the docker group.
Then run the docker.sh script to create the docker image and container that builds LightningFilter.
./docker.sh build <CMAKE_ARGS>
Note: Because CMake flags are cached, once set, any following build call uses them.
After compiling the application, the executable is in build/src/ and can be run as follows:
build/src/lf <EAL Parameters> -- <LF Parameters>
LightningFilter expects various parameters, which are divided into EAL and LF parameters. Script examples that run LightningFilter can be found in the test directory, e.g., in test/perfnet_ip.
DPDK defines the EAL parameters, which are described here. The application's help text describes the LF parameters.
E.g.:
build/src/lf --lcores (0-2)@(0-2),(3-7)@(3,7) --log-level lf:debug \
-- \
-p 0x1 --portmap "(0,0,o),(0,0,i)" --c lf_config.json
More info: docs/Parameters.md
The DPDK telemetry API provides information and statistics of a running DPDK instance. Also, LightningFilter collects additional statistics and exposes them to the same interface.
Launch interactive client script:
sudo ./usertools/lf-telemetry.py
When using a file prefix, the file prefix is set with the -f flag:
sudo ./usertools/lf-telemetry.py -f "file_prefix"
More info: docs/Metrics.md
LightningFilter provides an interface through a Unix socket during runtime, just as for the statistics.
Launch interactive client script:
sudo ./usertools/lf-ipc.py
When running LightningFilter with a file prefix, set the file prefix as follows:
sudo ./usertools/lf-ipc.py -f "file_prefix"
The script also allows running single commands without starting the interactive mode:
sudo ./usertools/lf-ipc.py --cmd=<command> {--params=<parameters>}
To develop on LightningFilter, fork the repository, and clone it onto your machine. If you want to contribute to the open-source repository, we recommend adding the open-source repository as the upstream for the main branch (open-source):
git checkout open-source
git remote add upstream [email protected]:netsec-ethz/lightning-filter.git
git fetch upstream
git branch --set-upstream-to upstream/open-source
git pull
To get quickly started with developing LightningFilter, we provide a Development Container setup (.devcontainer/devcontainer.json) with all required dependencies and some useful tools.
When using VS Code, just install the Visual Studio Code Dev Containers extension and open the project in a container (>Dev Container: Reopen in Container).
Alternatively, the developer container can also be created with the docker.sh script:
./docker.sh dev_image && ./docker.sh dev_up
(in the build directory)
make run_tests
The tests require additional packages:
sudo apt-get install bsdmainutils tmux
(in test/testnet_scion)
sudo ./integration_test.sh ../../build/src/lf ~/scion
Requires an appropriate build (see README in directory).
(in test/testnet_ip)
sudo ./integration_test.sh ../../build/src/lf
Requires an appropriate build (see README in directory).
To run all of the unit and integration tests with different settings (compilation configurations), run the script tests.sh.
For the performance tests, use the test configurations in the directories test/perfnet_scion and test/perfnet_ip.
The README.md, in the corresponding directories, provides additional information on the setup and required adjustments.
The docs directory contains a collection of documentation files. The following list provides an overview of them.
-
Installation Description of the installation process and provider scripts.
-
Functionality Description of the core functionalities
- Parameters Documentation on the application parameters.
- Configuration Documentation of configuration file.
- Metrics
- Monitoring Monitoring setup with Grafana
- IPC Inter process communication interface
- Features Additional features (Jumbo Frames, Checksum Offloading)
- Plugins Short description of the plugin system. -Control Traffic Overview of control traffic processing
-
Implementation
- Key Manager Overview of the keymanager module.
- Ratelimiter Overview of the ratelimiter module.
- Multi Core Multi core approach and distributor
- Code Quality Code style and format, as well as linter
- Optimizations Optimization approaches and effects
-
Specification
-
Troubleshooting
- Debug Debugging setup example (VSCode)
- Profiling Profiling setup example (Perf, VTune)
- Troubleshooting Common problems and solutions
The directory usertools contains tools and scripts that allow the user to interact more easily with the LightningFilter.