Docs: add more information about parallel option

[jrfnl]

Repost from squizlabs/PHP_CodeSniffer#1732:

I've just been looking into the parallel running option a bit more and am left with some questions.

• When should you use this option ?
• Does it work equally well with phpcs as well as phpcbf ?
• What things should you take into consideration when determining the amount of parallel processes to use ?
• What can go wrong ? Are there any type of sniffs which would be incompatible with parallel processing ?
• Any thing else which behaves differently if parallel processing is turned on ?

It might be useful to add a section to the Wiki Advanced Usage page about this.

---

👉🏻 The thread in the original post contains lots of useful comments and would be a good starting point to start writing this up.

[mabar]

Best course of action would be to just automate it. Since the original thread phpstan and others started to use https://github.com/theofidry/cpu-core-counter for that

[jrfnl]

@mabar Nice idea and something which could be considered as a feature request some time in the future. Having said that, that doesn't negate the validity of the current issue which is about the lack of documentation for how things are now.

[mabar]

To answer your questions

When should you use this option ?

Ideally always, it speeds up analysis a lot

Does it work equally well with phpcs as well as phpcbf ?

It looks like it does. Even initial phpcbf run on large legacy codebases is quite fast.

What things should you take into consideration when determining the amount of parallel processes to use ?

If you don't plan to run the checks nonstop, it's fine to just use all cores. On huge codebases, it helps a lot. On small codebases using more cores than necessary slows down execution, but the speed gain is still massive and optimizing for each small package is just not worth it. Perhaps codesniffer itself may decide to use less cores than given available, based on overall amount of files and amount of cached and unchanged files.

Using --parallel=$(shell nproc || sysctl -n hw.logicalcpu || wmic cpu get NumberOfLogicalProcessors || echo 4) to use all cores worked for me just fine on Linux, Windows and Mac. It's practically the same thing that https://github.com/theofidry/cpu-core-counter does.
Note that the syntax using shell may be specific to Makefile

What can go wrong ? Are there any type of sniffs which would be incompatible with parallel processing ?

I am currently at exactly 200 sniffs from codesniffer and slevomat/coding-standard and never had issues related to parallel processing.

Any thing else which behaves differently if parallel processing is turned on ?

Error handling. I had some very long file, whose processing was stuck. I had to disable parallelism in order to identify the file. Sadly, I don't remember why exactly was it a problem.

[jrfnl]

When should you use this option ?

Ideally always, it speeds up analysis a lot

Caveat: it doesn't work on Windows.

What can go wrong ? Are there any type of sniffs which would be incompatible with parallel processing ?

I am currently at exactly 200 sniffs from codesniffer and slevomat/coding-standard and never had issues related to parallel processing.

Anecdotal evidence is not an answer. There are some sniffs which check things across a codebase - like the Generic.Classes.DuplicateClassName sniff. Those type of sniffs "remember" information about previously seen code and I can imagine that information cache will not work correctly when running in parallel. That's the kind of thing I'd like to see investigated to answer that question (and no, I haven't investigated this myself, if for no other reason than that I'm using Windows).

[mbomb007]

grpc/grpc#20250 (comment)

[Luc45]

As per #419, parallelization is silently disabled if the pcntl extension is not available, how would be the best way to document this?

I'll kick-off with this suggestion to the Configuration Options page. Maybe we can also have one section in Advanced Usage, and link there if it needs further explanation/details?

Enabling Parallelization

By default, PHP_CodeSniffer will run using a single thread. To take advantage of parallel processing, you can set the default number of threads by using the parallel configuration option.

phpcs --config-set parallel 4
or
phpcs --parallel 4

Note: Parallel processing requires the 'pcntl' PHP extension. If the extension is not available, PHP_CodeSniffer will default to single-thread execution."

[mbomb007]

I still think it'd be helpful if a message is printed to the user, even if only in verbose mode. I spent quite a while trying to figure out why it wasn't parallelized at one point, and having the message would have saved me the time of debugging it.

Documentation isn't enough, IMHO.

[jrfnl]

Documentation isn't enough, IMHO.

@mbomb007 Agreed. I'll be opening a proposal ticket for a --diagnose option in the foreseeable future. The rough outline of it is described in #419 (comment)
That should hopefully cover your concern.

[jrfnl]

@Luc45 Thank you for getting the docs started. I do think it needs to be fleshed out a little more.

• It should definitely mention the ability to set --parallel at runtime via the CLI argument.
• I think it may also be good to list the situations in which the parallel option has no effect/will not be respected (STDIN/interactive mode/high verbosity)
• And some short guidance on how to set the value might also help - i.e. the limit is the number of Cores available on a system (?).
• Probably also good to link the mention of the pcntl extension to the PHP manual ? This link seems appropriate: https://www.php.net/manual/en/pcntl.installation.php

[jrfnl]

While it doesn't directly address this issue, PR #447 is a proposal to improve the help screens, including improved information on the help screen about the --parallel option.

Reviews and testing of that PR would be appreciated.

[mbomb007]

Regarding making --parallel default/automatic, if it is then there should be a way to disable that. Also, it's not likely to bug very many people, but using --parallel does change the way progress is shown with -p. That should probably be documented, too.