diff --git a/docs/yara_protection.md b/docs/yara_protection.md
new file mode 100644
index 00000000..af4cf5e2
--- /dev/null
+++ b/docs/yara_protection.md
@@ -0,0 +1,245 @@
+# Yara Protection
+
+The Yara Protection provides a more advanced way to build pattern matching for events.
+
+It is built above the existing tooling from .
+
+## Enabling the Protection
+
+First, make sure you add `--yara-rules` to your start command of draupnir.
+This argument takes a path to the place where it will load yara rule files
+(files ending in `.yara`) from.
+
+Then in your admin room:
+
+```
+!draupnir enable YaraDetection
+```
+
+## Setting up the policy list
+
+For some actions, you will need access to a policy list. Therefore, you can set
+a setting that includes the roomID of your policy list as a value:
+
+```
+!mjolnir config set YaraDetection.banPolicyList
+```
+
+Currently, it is only used for bans. However, this may be extended in the future.
+
+## Temporary disabling rules
+
+Rules can temporarily disabled by using tags.
+
+Then in matrix in your admin room you can use
+
+```
+!mjolnir config add YaraDetection.disabledTags
+```
+
+to add tags which should be disabled.
+
+Note that these are filtered _after_ the yara rules are executed currently.
+
+## Writing Matrix Compatible YARA rules
+
+### Available Actions
+
+Firstly, we have a few possible actions we can take with the YARA rules.
+These are all metadata in the YARA world, as shown in the next section.
+
+Possible Actions are:
+
+#### Notify
+
+This causes a notification that a rule has been matched in the admin room.
+
+If you also set `NotifcationText` it will additionally notify the user in the room.
+
+As an example:
+
+```yara
+rule TestRule : test_rule
+{
+ meta:
+ Author = "MTRNord"
+ Description = "Test Rule"
+ hash = "06fdc3d7d60da6b884fd69d7d1fd3c824ec417b2b7cdd40a7bb8c9fb72fb655b"
+ Action = "Notify"
+ strings:
+ $test_string = "Test" ascii nocase
+
+ condition:
+ $test_string
+}
+```
+
+This will cause a notification in the admin room if `Test` in any casing is matched in an event.
+
+### RedactAndNotify
+
+This behaves like the normal `Notify`, but also redacts the event that has been matched.
+
+### Kick and Ban
+
+Kick and Ban both take optional `Reason` metadata. This allows you to set, in the case of a kick, the reason field of the event, and in the case of a ban, it will set
+the reason for the policy list event.
+
+Both actions will additionally redact the event that matched.
+
+Bans can be reverted by using the regular unban flow.
+
+### Silence
+
+Silencing a user means it will first get their message redacted, and after that, it
+will not be kicked or banned, but the permissions to write will be removed for the user that sent the matched event.
+
+## Activated yara modules
+
+The yara integration is built on top of .
+This means that at the time of writing, the following yara modules are available:
+
+- `json` (Not available on windows)
+- `pe`
+- `elf`
+- `hash`
+- `math`
+- `time`
+- `console` (However this is not hooked to draupnir currently)
+- `string`
+- `lnk`
+
+// TODO: Verify if this is actually true
+
+## The json module
+
+Since the json module is a non-standard module, here is a short introduction to the available functions and how to use it in matrix context.
+
+As with other modules in yara you can import it by writing `import "json"` at
+the top of your yara file.
+
+The module brings the following conditional expressions:
+
+- `json.key_exists("")` - Allows you to check a key or nested key (delimited
+by dots. Escape dots which do not define levels as `\\.` for example `m\\.mentions`)
+- `json.value_exists("", "")` - Checks if a value exists. This can be
+a string, a float, an integer, or a regex expression.
+- `json.array_includes("", "")` - Checks if an array of strings
+contains the value.
+- `json.get_string_value("")` - Gets the value of a key if its a string
+- `json.get_integer_value("")` - Gets the value of a key if its an integer
+- `json.get_float_value("")` - Gets the value of a string if its a float
+
+### Example
+
+This example matches for "Test" in any casing if the event is of msgtype `m.text`
+
+```yara
+rule TestRule : test_rule
+{
+ meta:
+ Author = "MTRNord"
+ Description = "Test Rule"
+ hash = "06fdc3d7d60da6b884fd69d7d1fd3c824ec417b2b7cdd40a7bb8c9fb72fb655b"
+ Action = "Notify"
+ strings:
+ $test_string = "Test" ascii nocase
+
+ condition:
+ $test_string and json.value_exists("content.msgtype", "m.text")
+}
+```
+
+## Writing a YARA rule - A short guide
+
+_For more advanced concepts, please also read _
+
+Yara rules are a concept that is primarily used in the virus detection world.
+Hence, some things might be less useful for Matrix. We are going to focus
+on some simple matrix-specific concepts in this short guide.
+
+### What are YARA rules even?
+
+Yara rules are essentially a way to fingerprint data or more specifically match
+patterns in data. You can think about it as a more expressive way to write a boolean
+expression.
+
+### How are they structured
+
+Yara rules are similar to what you would call a function or method in languages.
+It always starts with `rule : ` and then has curly braces around
+the condition itself.
+
+After this basic structure you find up to 3 sections. `meta` which describes the
+rule itself and in case of Draupnir is used to add information for how Draupnir
+is supposed to evaluate the results of the rule, `strings` which define patterns
+which should be matched for. These can many different kinds like plain strings,
+byte patterns or regular expressions. Last but not least is the `condition` section,
+which is used to define the actual condition. Here you would tell it where the logical
+connections between multiple strings happen or if a string should be matched multiple times.
+
+### An example - Matching long audio messages
+
+As an example we are going to build a simple rule to match audio messages and then
+check their length. As a result we will tell Draupnir to issue a warning to the user.
+
+#### What do we need to check for?
+
+A quick look at gives us an overview what we need.
+
+First we do want to check its a `msgtype` wit the value `m.audio`.
+Then we also see there is a `duration` field. It is part of the AudioInfo which is under the `info` key.
+The duration is also defined to be in milliseconds.
+
+With this we can now start writing a condition.
+
+#### Metadata
+
+Before we go ahead of ourselfs though we want to define the metadata block.
+
+Some useful things to always include are:
+
+- `Author` - This is the person who wrote the rule. This helps to report bugs.
+- `Description` - A sentence or two describing what this rule will do if used.
+- `hash` - This is a hash for a file over at Virustotal which this rule will match.
+This is very useful if you want to use the since
+it will use that value to verify your rule is working.
+- `sharing` - This is useful if you intend to share rules with others. I suggest using this pattern:
+ as it is very
+easy to understand and easy to use. Keep in mind most of your rules likely should be marked as `TLP:RED`.
+When in doubt always go for `TLP:RED`.
+
+Additional to this metadata in Draupnir we also need an `Action`. This is used to make Draupnir act
+on a certain rule result. Without this it won't do anything when a rule matches.
+
+For our example we decided to go with the `Notify` value and a `NotificationText` since we
+want to tell the sender about a match.
+
+With this we have something like this:
+
+```yara
+rule match_audio_duration : event_type
+{
+ meta:
+ Author = "MTRNord"
+ Description = "Matches if the audio file is very long (longer than 1m) and notifies the user"
+ Action = "Notify"
+ NotificationText = "Hi, your media seems pretty long for this room. Consider sending audio files smaller than 1m of duration please."
+}
+```
+
+#### Matching the event data
+
+## Security Considerations
+
+Yara seems really powerful at first sight. However, please note the following
+very important things when starting to use it in your community:
+
+- Yara is NOT a virus scanner. Do not use it as one! Yara is designed as a pattern
+matching engine. This means that, while it can catch some silly things, it will not replace clamav or similar.
+- Be mindful about published rules. Rules can easily be used against you if they are public.
+A spammer can look at them, find their issues or even use existing tools to generate
+spam, which does avoid the rules entirely. Be ABSOLUTE SURE if you publish any of your rules.
+- Any message and media are being evaluated through yara. Make sure that your rules are
+as lightweight as possible. Try to test them locally using the yara command line.
+This yields various warnings about optimizing the rules. If you are not careful, you might Denial of Service your own protection.
diff --git a/package.json b/package.json
index cdc2226b..00b11b97 100644
--- a/package.json
+++ b/package.json
@@ -15,7 +15,7 @@
"lint": "eslint ./**/*.ts",
"start:dev": "yarn build && node --async-stack-traces lib/index.js",
"test": "ts-mocha --project ./tsconfig.json test/commands/**/*.ts",
- "test:integration": "NODE_ENV=harness ts-mocha --async-stack-traces --forbid-only --require test/integration/fixtures.ts --timeout 300000 --project ./tsconfig.json \"test/integration/**/*Test.ts\"",
+ "test:integration": "NODE_ENV=harness ts-mocha --async-stack-traces --forbid-only --require test/integration/fixtures.ts --timeout 300000 --project ./tsconfig.json \"test/integration/**/*Test.ts\" -- --yara-rules ./test/integration/yara_rules",
"test:integration:single": "NODE_ENV=harness npx ts-mocha --require test/integration/fixtures.ts --timeout 300000 --project ./tsconfig.json",
"test:appservice:integration": "NODE_ENV=harness ts-mocha --async-stack-traces --forbid-only --timeout 300000 --project ./tsconfig.json \"test/appservice/integration/**/*Test.ts\"",
"test:appservice:integration:single": "NODE_ENV=harness npx ts-mocha --timeout 300000 --project ./tsconfig.json",
diff --git a/src/protections/YaraDetection.ts b/src/protections/YaraDetection.ts
index a3450bcc..804ae562 100644
--- a/src/protections/YaraDetection.ts
+++ b/src/protections/YaraDetection.ts
@@ -152,19 +152,19 @@ export class YaraDetection extends Protection {
await mjolnir.client.redactEvent(roomId, event["event_id"]);
await mjolnir.client.kickUser(event["sender"], roomId, kickReason);
const eventPermalink = Permalinks.forEvent(roomId, event['event_id']);
- await mjolnir.managementRoomOutput.logMessage(LogLevel.WARN, this.name, `Yara matched for event ${eventPermalink} and kicked the User:\nScan ${result.identifier} found match: ${JSON.stringify(result.strings)}`);
+ await mjolnir.managementRoomOutput.logMessage(LogLevel.WARN, this.name, `YARA rule matched for event ${eventPermalink} and kicked the User:\nScan ${result.identifier} found match: ${JSON.stringify(result.strings)}`);
}
private async actionBan(mjolnir: Mjolnir, roomId: string, event: any, result: YaraRuleResult, ban_reason?: string) {
const eventPermalink = Permalinks.forEvent(roomId, event['event_id']);
if (!this.settings.banPolicyList.value) {
- await mjolnir.managementRoomOutput.logMessage(LogLevel.WARN, this.name, `Yara matched for event ${eventPermalink} but was unable to ban the user since there is no policy list for bans configured:\nScan ${result.identifier} found match: ${JSON.stringify(result.strings)}`);
+ await mjolnir.managementRoomOutput.logMessage(LogLevel.WARN, this.name, `YARA rule matched for event ${eventPermalink} but was unable to ban the user since there is no policy list for bans configured:\nScan ${result.identifier} found match: ${JSON.stringify(result.strings)}`);
}
await mjolnir.client.redactEvent(roomId, event["event_id"]);
await mjolnir.policyListManager.lists.find(list => list.roomId == this.settings.banPolicyList.value)?.banEntity(EntityType.RULE_USER, event["sender"], ban_reason ?? "Automatic ban using Yara Rule");
- await mjolnir.managementRoomOutput.logMessage(LogLevel.WARN, this.name, `Yara matched for event ${eventPermalink} and banned the User:\nScan ${result.identifier} found match: ${JSON.stringify(result.strings)}`);
+ await mjolnir.managementRoomOutput.logMessage(LogLevel.WARN, this.name, `YARA rule matched for event ${eventPermalink} and banned the User:\nScan ${result.identifier} found match: ${JSON.stringify(result.strings)}`);
}
private async actionSilence(mjolnir: Mjolnir, roomId: string, event: any, result: YaraRuleResult) {
@@ -188,7 +188,7 @@ export class YaraDetection extends Protection {
}
const eventPermalink = Permalinks.forEvent(roomId, event['event_id']);
- await mjolnir.managementRoomOutput.logMessage(LogLevel.WARN, this.name, `Yara matched for event ${eventPermalink} and silenced the User:\nScan ${result.identifier} found match: ${JSON.stringify(result.strings)}`);
+ await mjolnir.managementRoomOutput.logMessage(LogLevel.WARN, this.name, `YARA rule matched for event ${eventPermalink} and silenced the User:\nScan ${result.identifier} found match: ${JSON.stringify(result.strings)}`);
}
@@ -197,7 +197,7 @@ export class YaraDetection extends Protection {
*/
private async actionNotify(mjolnir: Mjolnir, roomId: string, event: any, result: YaraRuleResult, notificationText?: string) {
const eventPermalink = Permalinks.forEvent(roomId, event['event_id']);
- await mjolnir.managementRoomOutput.logMessage(LogLevel.WARN, this.name, `Yara matched for event ${eventPermalink}:\nScan ${result.identifier} found match: ${JSON.stringify(result.strings)}`);
+ await mjolnir.managementRoomOutput.logMessage(LogLevel.WARN, this.name, `YARA rule matched for event ${eventPermalink}:\nScan ${result.identifier} found match: ${JSON.stringify(result.strings)}`);
if (notificationText) {
const userPermalink = Permalinks.forUser(event['sender']);
await mjolnir.client.sendNotice(roomId, `${userPermalink}: ${notificationText}`);
diff --git a/test/integration/yaraRuleTest.ts b/test/integration/yaraRuleTest.ts
new file mode 100644
index 00000000..29800f7e
--- /dev/null
+++ b/test/integration/yaraRuleTest.ts
@@ -0,0 +1,84 @@
+import { strict as assert } from "assert";
+
+import { UserID } from "matrix-bot-sdk";
+import { Suite } from "mocha";
+import { Mjolnir } from "../../src/Mjolnir";
+import { createBanList, getFirstEventMatching } from "./commands/commandUtils";
+import { newTestUser } from "./clientHelper";
+import { YaraDetection } from "../../src/protections/YaraDetection";
+
+describe("Test: YaraDetection protection", function () {
+ beforeEach(async function () {
+ // Setup an instance of YaraDetection
+ this.yara_rules = new YaraDetection();
+ await this.mjolnir.protectionManager.registerProtection(this.yara_rules);
+ await this.mjolnir.protectionManager.enableProtection("YaraDetection");
+
+ // Setup a moderator.
+ this.moderator = await newTestUser(this.config.homeserverUrl, { name: { contains: "moderator" } });
+ await this.moderator.joinRoom(this.mjolnir.managementRoomId);
+
+ // Setup a bad_user.
+ this.bad_user = await newTestUser(this.config.homeserverUrl, { name: { contains: "bad_user" } });
+
+ // Setup a protected room
+ const mjolnirId = await this.mjolnir.client.getUserId();
+ this.protected_room = await this.moderator.createRoom({ invite: [mjolnirId, await this.bad_user.getUserId()] });
+ await this.mjolnir.client.joinRoom(this.protected_room);
+ await this.bad_user.joinRoom(this.protected_room);
+ await this.moderator.setUserPowerLevel(mjolnirId, this.protected_room, 100);
+ await this.mjolnir.addProtectedRoom(this.protected_room);
+
+ // Setup a policy list
+ const banList = await createBanList(this.mjolnir.managementRoomId, this.mjolnir.client, this.moderator);
+
+ const SETTINGS = {
+ banPolicyList: this.mjolnir.policyListManager.resolveListShortcode(banList).roomId,
+
+ };
+ for (let key of Object.keys(SETTINGS)) {
+ this.yara_rules.settings[key].setValue(SETTINGS[key]);
+ }
+ });
+
+ afterEach(async function () {
+ await this.moderator?.stop();
+ await this.bad_user?.stop();
+ });
+
+ it('Should notify about a match in the admin room', async function () {
+ // check for the warning
+ const event = await getFirstEventMatching({
+ matrix: this.mjolnir.matrixEmitter,
+ targetRoom: this.mjolnir.managementRoomId,
+ lookAfterEvent: async function () {
+ // Send a message that issues a warning
+ await this.bad_user.sendMessage(this.protected_room, "Test");
+ return undefined;
+ },
+ predicate: function (event: any): boolean {
+ return (event['content']?.['body'] ?? '').startsWith('⚠ | YARA rule matched for event')
+ }
+ })
+
+ assert.notStrictEqual(event, undefined)
+ });
+
+ it('Should notify the user in the room', async function () {
+ // check for the warning
+ const event = await getFirstEventMatching({
+ matrix: this.mjolnir.matrixEmitter,
+ targetRoom: this.protected_room,
+ lookAfterEvent: async function () {
+ // Send a message that issues a warning
+ await this.bad_user.sendMessage(this.protected_room, "Test notify user");
+ return undefined;
+ },
+ predicate: function (event: any): boolean {
+ return (event['content']?.['body'] ?? '').startsWith('Please don\'t')
+ }
+ })
+
+ assert.notStrictEqual(event, undefined)
+ });
+});
diff --git a/test/integration/yara_rules/test.yara b/test/integration/yara_rules/test.yara
new file mode 100644
index 00000000..5c9f6362
--- /dev/null
+++ b/test/integration/yara_rules/test.yara
@@ -0,0 +1,27 @@
+rule TestRule : test_rule
+{
+ meta:
+ Author = "MTRNord"
+ Description = "Test Rule"
+ hash = "06fdc3d7d60da6b884fd69d7d1fd3c824ec417b2b7cdd40a7bb8c9fb72fb655b"
+ Action = "Notify"
+ strings:
+ $test_string = "Test" ascii nocase
+
+ condition:
+ $test_string
+}
+
+rule NotifyUserRule : test_rule
+{
+ meta:
+ Author = "MTRNord"
+ Description = "NotifyUserRule"
+ Action = "Notify"
+ NotificationText = "Please don't"
+ strings:
+ $test_string = "Test notify user" ascii nocase
+
+ condition:
+ $test_string
+}