Fix "xapian audit creates a bogus directory with no entries" bug

[brong]

This caused real production issues at Fastmail. We have a weekly cronjob running squatter -A to audit indexes. This creates a zero-record database, which when you later try to open it causes issues!

The fix is to check that we got all the expected Xapian glass files before considering a database to be "real", and so we don't include databases without terms.

This really is a multi-stage bug! Stage 1:

[72048] =====> Instance::_fork_command[1969] Running: "/usr/cyrus/bin/squatter" "-C" "/run/cass/0959120101/conf/imapd.conf" "-A"

C: 8 search fuzzy subject xyzzy
S: * SEARCH 1
S: 8 OK Completed (1 msgs in 0.000 secs)

Still works - but then we use compact_dbs to compact that empty database, and it produces a broken database!

But then we compact that one down, and:

[72048] =====> Instance::_fork_command[1969] Running: "/usr/cyrus/bin/squatter" "-C" "/run/cass/0959120101/conf/imapd.conf" "-z" "t2" "-t" "t1"

C: 9 search fuzzy subject xyzzy
S: * SEARCH
S: 9 OK Completed (0 msgs in 0.000 secs)

and in syslog:

Sep 01 20:03:46.836820 elg 1003450101/imap[72211]: command: 9 Search
Sep 01 20:03:46.837198 elg 1003450101/imap[72211]: xapian_wrapper: invalid db version in /run/cass/1003450101/search2/user/uuid/4/9/4942a15e-d3da-4845-bcc2-24967d476c5e/xapian.1

[rsto]

Ouch.. looks good to me with one nit.

[rsto]

Given that we expect the majority of users to have all these files the cost of calling malloc/free five times looks unnecessary expensive, given that we could achieve the same by using a single struct buf for the building the file names in the loop.

But since the cost of calling stat is likely to make the cost of malloc/free look small in comparison, there's no big need for this optimization.

[brong]

Yeah, I don't think that it's anywhere near important compared to all the other costs. Not only stat but the locking cost per user - we have only a few files per user.