Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fixes for new firmware 8.2.0 of Pixiu (AB#23392) #26

Merged
merged 3 commits into from
Oct 21, 2024
Merged

Fixes for new firmware 8.2.0 of Pixiu (AB#23392) #26

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
44d7879
add tests
wpoely86 Aug 14, 2024
758567a
fixes for 8.2.0, part 1
wpoely86 Oct 20, 2024
127392e
fixes for 8.2.0, part 2
wpoely86 Oct 20, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 6 additions & 2 deletions files/pixiu
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,15 +6,19 @@ PIXIU_NGINX \s*%{IPORHOST:syslog_hostname} %{SYSLOGPROG} %{IPORHOST:clientip} (?

PIXIU_TOMCAT \s*%{IPORHOST:syslog_hostname} %{SYSLOGPROG} \[%{HTTPDATE:timestamp}\]\^%{IPORHOST:clientip}\^%{HTTPDUSER:auth}\^%{WORD:verb} %{NOTSPACE:request} HTTP/%{NUMBER:httpversion}\^%{GREEDYDATA:other}

PIXIU_ALARM_PREFIX \s*%{IPORHOST:syslog_hostname} %{SYSLOGPROG}: .+%{IPORHOST} (?<alarm_object>OceanStor-Distributed-Storage)\s+%{NUMBER} (?<alarm_id>0x[A-F0-9]+) (?<alarm_severity>%{WORD})\(%{NUMBER:alarm_severity_level}\):
PIXIU_ALARM_PREFIX \s*%{IPORHOST:syslog_hostname} %{SYSLOGPROG}: .+ (?<alarm_object>OceanStor-Distributed-Storage)\s+%{NUMBER} (?<alarm_id>0x[A-F0-9]+) (?<alarm_severity>%{WORD})\(%{NUMBER:alarm_severity_level}\):

PIXIU_ALARM_USER_TYPE user name, owner name %{USERNAME:username}

PIXIU_BYTES [0-9]+(?:K|M|G|T|P)?B

PIXIU_QUOTA_SIZE_TYPE (?:(?<used_files>%{NUMBER:int})|(?<used_space>%{PIXIU_BYTES}))

PIXIU_ALARM_COMMON The used (?<quota_used_type>%{WORD}) \(%{PIXIU_QUOTA_SIZE_TYPE}\) of (?:the )?quota \(type (?<quota_type>%{WORD}) quota(?:, %{PIXIU_ALARM_USER_TYPE})?\) of dtree \(name (?<dtree>%{NOTSPACE}), ID (?<dtree_id>%{NUMBER:int})\) at %{UNIXPATH:path} in namespace \(name (?<namespace>%{NOTSPACE}), ID (?<namespace_id>%{NUMBER:int})\)
PIXIU_ALARM_COMMON The used (?<quota_used_type>%{WORD}) \(%{PIXIU_QUOTA_SIZE_TYPE}\) of (?:the )?quota \(type (?:UNIX )?(?<quota_type>%{WORD}) quota(?:, %{PIXIU_ALARM_USER_TYPE})?\) of (%{PIXIU_ALARM_COMMON_OLD}|%{PIXIU_ALARM_COMMON_NEW})

PIXIU_ALARM_COMMON_OLD dtree \(name (?<dtree>%{NOTSPACE}), ID (?<dtree_id>%{NUMBER:int})\) at %{UNIXPATH:path} in namespace \(name (?<namespace>%{NOTSPACE}), ID (?<namespace_id>%{NUMBER:int})\)

PIXIU_ALARM_COMMON_NEW namespace \(name (?<namespace>%{NOTSPACE}), ID (?<namespace_id>%{NUMBER:int})\), dtree \(name (?<dtree>%{NOTSPACE}),\s?ID (?<dtree_id>%{NUMBER:int})\), or path %{UNIXPATH:path}\)?

PIXIU_USER_INODE_ALARM %{PIXIU_ALARM_PREFIX} %{PIXIU_ALARM_COMMON} reaches (?:or approaches )?(?:to )?the (:?(?<quota_limit_type>%{WORD}) (?:file|space) quantity quota of|(?:file|space) quantity (?<quota_limit_type>%{WORD}) quota of) \((?<quota_files_limit>%{NUMBER:int})\).*

Expand Down
73 changes: 67 additions & 6 deletions tests/data/pixiu
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -100,7 +100,27 @@ data = [
},
},
{
"raw": "<187>Aug 20 20:18:25 C4STO01-Node1 alarm[2613490]: <186>2023-08-20 20:18:25 DST 172.19.104.10 OceanStor-Distributed-Storage 259068 0xFEA6A000E Major(1): The used files (1501) of quota (type directory quota) of dtree (name test1, ID 4101) at / in namespace (name admin, ID 71) reaches to the file quantity soft quota of (1000).",
"raw": "<188>Oct 20 20:51:03 C4STO01-Node1 alarm[53019]: <188>2024-10-20 20:51:01 DST OceanStor-Distributed-Storage 2171540 0xFEA6A000B Warning(1): The used files (3000) of quota (type directory quota) of dtree (name test, ID 4101) at /test in namespace (name backups, ID 98) reaches or approaches to the file quantity hard quota of (3000). Data will fail to be written.",
"expected": {
"@source_host": "C4STO01-Node1",
"program": "alarm",
"alarm_severity": "Warning",
"alarm_object": "OceanStor-Distributed-Storage",
"alarm_id": "0xFEA6A000B",
"used_files": 3000,
"quota_type": "directory",
"quota_used_type": "files",
"dtree": "test",
"dtree_id": 4101,
"namespace": "backups",
"namespace_id": 98,
"quota_limit_type": "hard",
"quota_files_limit": 3000,
"path": "/test",
},
},
{
"raw": "<187>Aug 20 20:18:25 C4STO01-Node1 alarm[26190]: <186>2023-08-20 20:18:25 DST 172.19.104.10 OceanStor-Distributed-Storage 259068 0xFEA6A000E Major(1): The used files (1501) of quota (type directory quota) of dtree (name test1, ID 4101) at / in namespace (name admin, ID 71) reaches to the file quantity soft quota of (1000).",
"expected": {
"@source_host": "C4STO01-Node1",
"program": "alarm",
Expand Down Expand Up @@ -160,14 +180,14 @@ data = [
},
},
{
"raw": "<187>Aug 20 19:40:26 C4STO01-Node1 alarm[2613490]: <186>2023-08-20 19:32:24 DST 2023-08-20 19:40:26 DST 172.19.104.10 OceanStor-Distributed-Storage 259045 0xFEA6A000F Major(2): The used space (6638MB) of quota (type user quota, user name, owner name vsc10042) of dtree (name 100, ID 8193) at /100 in namespace (name user, ID 70) reaches to the space soft quota of (6144MB).",
"raw": "<188>Oct 20 20:35:17 C4STO01-Node1 alarm[53019]: <188>2024-10-20 20:35:17 DST OceanStor-Distributed-Storage 2171251 0xFEA6A000F Warning(1): The used space (8191MB) of the quota (type UNIX user quota, user name, owner name vsc10042) of namespace (name user, ID 70), dtree (name 100, ID 8193), or path /100 reaches the space soft quota of (6144MB).",
"expected": {
"@source_host": "C4STO01-Node1",
"program": "alarm",
"alarm_severity": "Major",
"alarm_severity": "Warning",
"alarm_object": "OceanStor-Distributed-Storage",
"alarm_id": "0xFEA6A000F",
"used_space": 6638000000,
"used_space": 8191000000,
"quota_used_type": "space",
"quota_type": "user",
"username": "vsc10042",
Expand All @@ -180,6 +200,27 @@ data = [
"path": "/100",
},
},
{
"raw": "<188>Oct 20 20:32:56 C4STO01-Node1 alarm[53019]: <188>2024-10-20 20:32:56 DST OceanStor-Distributed-Storage 2171198 0xFEA6A0011 Warning(1): The used space (8192MB) of the quota (type UNIX user quota, user name, owner name vsc10042) of namespace (name user, ID 70), dtree (name 100,ID 8193), or path /100) reaches or approaches to the hard space quota of (8192MB). The user or user group will fail to write data.",
"expected": {
"@source_host": "C4STO01-Node1",
"program": "alarm",
"alarm_severity": "Warning",
"alarm_object": "OceanStor-Distributed-Storage",
"alarm_id": "0xFEA6A0011",
"used_space": 8192000000,
"quota_used_type": "space",
"quota_type": "user",
"username": "vsc10042",
"dtree": "100",
"dtree_id": 8193,
"namespace": "user",
"namespace_id": 70,
"quota_limit_type": "hard",
"quota_space_limit": 8192000000,
"path": "/100",
},
},
{
"raw": "<190>Feb 27 12:04:21 HKSTO03-Node1 alarm[1266782]: <189>2024-02-27 12:04:21 172.19.96.130 OceanStor-Distributed-Storage 543842 0x200F002A0015 Informational(8): User (user name adm_wpoelmans) succeeded in logging in from source (172.18.124.113).",
"expected": {
Expand Down Expand Up @@ -345,8 +386,6 @@ data = [
},
},
{
# "raw": '<174>Oct 17 12:50:42 HKSTO03-Node1 nginx 127.0.0.1 - - [17/Oct/2022:12:50:42 +0200] "GET /dsware/service/cluster/storagepool/query*** HTTP/1.1" 200 "0.062" 924 "-" "Apache-HttpClient/5.1 (Java/1.8.0_322)" "-" 127.0.0.1:9527 200 "0.064" "********" "********" "********" "127.0.0.1" "application/json;charset=UTF-8" "-" "-" "gzip, x-gzip, deflate"',

"raw": '<174>Aug 14 09:15:16 C4STO01-Node1 nginx 127.0.0.1 [14/Aug/2024:09:15:16 +0200] "GET /api/v2/cluster/se*** HTTP/1.1" 200 "0.046" 3072 "Apache-HttpClient/5.1 (Java/1.8.0_372)" "-" 127.0.0.1:8099 200 "0.046" "********" "********" "********" "127.0.0.1" "application/json;charset=UTF-8" "-" ',
"expected": {
"@source_host": "C4STO01-Node1",
Expand All @@ -357,4 +396,26 @@ data = [
"response": 200,
},
},
{
"raw": '<174>Aug 14 16:00:24 C4STO01-Node1 nginx 134.184.143.132 [14/Aug/2024:16:00:05 +0200] "GET /api/v2/converged_service/nam*** HTTP/1.1" 200 "0.165" 48 "vsc-rest-client" "-" 127.0.0.1:8778 200 "0.166" "********" "None" "None" "" "-" "-" ',
"expected": {
"@source_host": "C4STO01-Node1",
"program": "nginx",
"clientip": "134.184.143.132",
"verb": "GET",
"request": "/api/v2/converged_service/nam***",
"response": 200,
},
},
{
"raw": "<174>Sep 11 11:43:47 C4STO01-Node1 nginx 172.18.124.214 [11/Sep/2024:11:43:42 +0200] \"GET /gui/api/v2/common/ala*** HTTP/1.1\" 200 \"0.132\" 59 \"Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:122.0) Gecko/20100101 Firefox/122.0\" \"-\" 127.0.0.1:8088 200 \"0.117\" \"None\" \"********\" \"********\" \"\" \"application/json; charset=UTF-8\" \"application/json, text/plain, */*\" ",
"expected": {
"@source_host": "C4STO01-Node1",
"program": "nginx",
"clientip": "172.18.124.214",
"verb": "GET",
"request": "/gui/api/v2/common/ala***",
"response": 200,
},
},
]