"top-ads for audit" and "top-domains for audit" very slow when total items is less than 10

[Tatsh]

Versions

• Pi-hole: 5.18.2
• AdminLTE: 5.21
• FTL: 5.25.1

Platform

• OS and version: Gentoo Linux

Expected behavior

>top-ads for audit should be the same speed regardless of the size of the list.

Actual behavior / bug

Using the Audit log page in Pi-hole Admin is extremely slow generally because the auto-refresh takes on average about 4.3 seconds.

Steps to reproduce

Steps to reproduce the behavior:

1. time echo '>top-ads for audit >quit' | nc -vv localhost 4711

Shell example:

 $ time nc -vv localhost 4711 <<< '>top-ads for audit >quit'
localhost [127.0.0.1] 4711 (?) open
0 2 amung.us
1 2 clarity.ms
2 2 1.cpm.ak-is2.net
 sent 25, rcvd 49

actual  0m4.334s
user    0m0.001s
sys     0m0.002s
CPU     0.08%

When there are 10 or more items it is near instant:

 $ time nc -vv localhost 4711 <<< '>top-domains for audit >quit'
localhost [127.0.0.1] 4711 (?) open
0 71 4.5.0.0.0.0.0.0.0.0.0.0.0.0.0.0.f.0.c.0.2.0.0.4.0.b.8.f.7.0.6.2.ip6.arpa
1 36 5.0.0.0.0.0.0.0.0.0.0.0.0.0.0.f.0.c.0.2.0.0.4.0.b.8.f.7.0.6.2.ip6.arpa
2 36 0.0.0.0.0.0.0.0.0.0.0.f.0.c.0.2.0.0.4.0.b.8.f.7.0.6.2.ip6.arpa
3 35 0.0.0.0.0.0.0.0.0.0.0.0.0.0.f.0.c.0.2.0.0.4.0.b.8.f.7.0.6.2.ip6.arpa
4 35 0.0.0.0.0.0.0.0.0.0.0.0.0.f.0.c.0.2.0.0.4.0.b.8.f.7.0.6.2.ip6.arpa
5 35 0.0.0.0.0.0.0.0.0.0.0.0.f.0.c.0.2.0.0.4.0.b.8.f.7.0.6.2.ip6.arpa
6 35 0.0.0.0.0.0.0.0.0.0.f.0.c.0.2.0.0.4.0.b.8.f.7.0.6.2.ip6.arpa
7 35 0.0.0.0.0.0.0.0.0.f.0.c.0.2.0.0.4.0.b.8.f.7.0.6.2.ip6.arpa
8 35 0.0.0.0.0.0.0.0.f.0.c.0.2.0.0.4.0.b.8.f.7.0.6.2.ip6.arpa
9 35 0.0.0.0.0.0.0.f.0.c.0.2.0.0.4.0.b.8.f.7.0.6.2.ip6.arpa
 sent 29, rcvd 690

actual  0m0.097s
user    0m0.000s
sys     0m0.003s
CPU     3.60%

[Tatsh]

The slowness seems to be inside unlock_shm(). The data is sent quickly (write()) and then these syscalls are repeated many times before >quit is processed.

[pid 3858812] 13:32:23.739339 fcntl(39</var/lib/pihole/gravity.db>, F_SETLK, {l_type=F_UNLCK, l_whence=SEEK_SET, l_start=0, l_len=0}) = 0
[pid 3858812] 13:32:23.739415 fcntl(39</var/lib/pihole/gravity.db>, F_SETLK, {l_type=F_RDLCK, l_whence=SEEK_SET, l_start=1073741824, l_len=1}) = 0
[pid 3858812] 13:32:23.739477 fcntl(39</var/lib/pihole/gravity.db>, F_SETLK, {l_type=F_RDLCK, l_whence=SEEK_SET, l_start=1073741826, l_len=510}) = 0
[pid 3858812] 13:32:23.739535 fcntl(39</var/lib/pihole/gravity.db>, F_SETLK, {l_type=F_UNLCK, l_whence=SEEK_SET, l_start=1073741824, l_len=1}) = 0

Full strace log: out.log.gz

[DL6ER]

Thanks for you input. Please note, however, that we are getting close to release Pi-hole v6.0 featuring a completely rebuilt API. There is no Telnet anymore.

[yubiuser]

... and no audit...

[NoGitHubForYou]

Interestingly, on a possible related note, if the audit log has 11 or more entries for blocked domains and I click audit on an entry, almost instantly the domain will disappear from the list and added to audit database. Otherwise 10 or less blocked domains in list, clicking the audit button on an entry will cause the list to take many seconds to update and sometimes cause the entire dns server to stop responding for a minute or longer. Forget trying to click audit button on multiple entries at once because the entire dns server will fall over and often require a restart.

[NoGitHubForYou]

... and no audit...

@yubiuser

Can you elaborate? Is audit log gone entirely or just through the api?

[yubiuser]

Is audit log gone entirely

Yes it is. There were likely only a few users and it was never fully implemented, hard to explain and caused more issues than is solved.

[NoGitHubForYou]

Is audit log gone entirely

Yes it is. There were likely only a few users and it was never fully implemented, hard to explain and caused more issues than is solved.

I used it, obviously, but like you said I was one of a few users who used it. Audit log was really handy for my block list project. I understand though. Thanks for your reply.

[github-actions]

This issue is stale because it has been open 30 days with no activity. Please comment or update this issue or it will be closed in 5 days.