test_examples uses ExampleRunner

[michaelbenayoun]

• Updates ExampleRunner to make it both usable for Neuron cache filling and testing examples
• Updates the tests/test_examples.py file
  • It is possible to run "tiny" versions of the models by setting the following flag RUN_TINY=1. It will run the example with the same model except that it contains much less blocks.
  • Each model, for a given task, is tested on different sharding strategies: no sharding, ZeRO-1, TP, TP + ZeRO-1 (when it makes sense)
  • Each model has a defined "coverage" associated to it: low, middle, high. By setting the flag COVERAGE it is possible to run test a subset of the models. The coverage of a model is defined by its uniqueness. Ideally running tests/test_examples.py with COVERAGE=high should cover most of the use cases.
• Checks that the training loss is decreasing
• Test example workflow is now triggered manually

In following PRs:

• Validation is fixed and performed, each evaluation score is compared to a reference value, either hardcoded manually or computed by running a similar training job with Transformers on GPU.
• Enable ZeRO-1 tests once Parallel cross entropy #222 is merged.

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint.

[JingyaHuang]

LGTM!! Would be great if we can have all green CIs from now!

[dacorvo]

LGTM !