Add new Apache or MIT license rule #3738

[vasily-pozdnyakov]

Fixes #3738

Tasks

• Reviewed contribution guidelines
• PR is descriptively titled 📑 and links the original issue above 🔗
• Commits are in uniquely-named feature branch and has no merge conflicts 📁

[vasily-pozdnyakov]

Hi, I am new to new rules creation and have a couple of questions:

1. What does scancode-reindex-licenses do? It does not introduce any visible changes (validation is clear, what about reindex?).
2. Why most of the rules are weak (without "{{}}")? If I understand correctly, it might bring a lot of false positives (let's say, there is a matching license notice for MIT, but with a different license name (Apache) - the scancode might detect it incorrectly), is it like that?

[AyanSinhaMahapatra]

@vasily-pozdnyakov welcome and thanks for the PR!

What does scancode-reindex-licenses do? It does not introduce any visible changes (validation is clear, what about reindex?).

When you get SCTK from a git checkout and install scancode to run locally (or get scancode from github releases or pip install) you have the license indexes at scancode-toolkit/src/licensedcode/data/cache/license_index/ which contains the license index (pre-built if you're downloading the release/via pip) that is used for license detection when you run scancode. Now if you are updating/adding new rule/licenses in scancode you want to run scancode-reindex-licenses so next time you run a scan locally these rules are in the index and are used in license detection, but this does no changes in the repository of rules.

Why most of the rules are weak (without "{{}}")? If I understand correctly, it might bring a lot of false positives (let's say, there is a matching license notice for MIT, but with a different license name (Apache) - the scancode might detect it incorrectly), is it like that?

That's mostly correct, this could (and does also) generate a lot of false positives in a lot of cases. See #2878

We are working on resolving this and I have a pending PR at #3254 testing out adding required phrases automatically and massively across all the rules from the following:

1. license names and other license keywords
2. from other instances of these required phrases added in other rules.
   This would improve the accuracy significantly for these rules and also make sure when we add a required phrase, same is marked across the same phrases present in all rules. There is just a bit of work remaining on testing this and verifying this works correctly across all the rules. But this addition of required phrases would be continuous, or a massive one-time effort otherwise.

[pombredanne]

Thanks. Just a minor but important nit wrt. to the license expression. The norm for MIT and Pache in the Rust ecosystem is to be a choice and buck2 and this new rules is not an exception.

We could also later add more rules for derived notices such as https://github.com/facebook/buck2?tab=readme-ov-file#license

[AyanSinhaMahapatra]

Thanks++ @vasily-pozdnyakov, merging!