…r into error_logging_callback

Add pytorch nightly and CUDA 12.1 support for composer docker images

What issue(s) does this change relate to?
Related to https://mosaicml.atlassian.net/browse/GRT-2305

Tests
docker image: mosaicml/ci-staging:72744756-794c-4390-94db-72c212dd5e00 (cuda 12.1, pytorch 2.1.0)

mcli connect temp-test-ZAVxMh
Python 3.10.12 (main, Jun  7 2023, 12:45:35) [GCC 9.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import torch
>>> print(torch.version)
<module 'torch.version' from '/usr/lib/python3/dist-packages/torch/version.py'>
>>> print(torch.__version__)
2.1.0.dev20230623+cu121
>>> print(torch.version.cuda)
12.1
Integration Test
@mvpatel2000 has validated that this trains on initial mpt-2 experiments and speeds up training by +7-8% from 0.25 MFU to 0.27 MFU

* fix autoresume with slashed directory

* Revert "fix autoresume with slashed directory"

This reverts commit 3dfb5f5.

revert

* fix

* fix precommit

* Update in_context_learning_evaluation.py

* Update in_context_learning_evaluation.py

* Update in_context_learning_evaluation.py

* add tests

Signed-off-by: Prithvi Kannan <[email protected]>
Co-authored-by: Evan Racah <[email protected]>
Co-authored-by: eracah <[email protected]>

Upstreams and generalizes the callback that logs generations to wandb from foundry to composer.

…2476)

Upgrade torch docker nightly version to 08-23-23 so that we get nccl version 0.18.3 which was merged on 08-18-23.

Test pytorch 2.1.0 docker images on ci/cd mosaicml#2469

Will be removed in v0.18.

* Update RTD build config with build.os
* Remove python.version

---------

Co-authored-by: Bandish Shah <[email protected]>

# What does this PR do?
Security vulnerability in `semver` seen due to node. This PR upgrades the node version to bump up semver from 7.5.1 to 7.5.2

# Tests
Action Run: https://github.com/mosaicml/composer/actions/runs/6017539089
Correct version of semver seen after upgrade: 
```
mosaicml#14 [pytorch_stage  7/24] RUN npm list -g semver --depth=1
mosaicml#14 2.223 /usr/lib
mosaicml#14 2.223 `-- [email protected]
mosaicml#14 2.223   `-- [email protected]
mosaicml#14 2.223 
mosaicml#14 DONE 2.4s
```

* Gating tying modules w/ FSDP

* Changing weight tying filtering to be less aggressive

* precommit formatting

* Removing min_params

* formatting?

* removing overlap with another commit

* add fix

* fix tests

* qwf

* dsfg

* add key

* remove short

* add map test

* remove comment

* filter warning

* simplify wrapping

* checkdown

* fix torchmetrics

* 300

* fix tests

* remove metric

* cleanup

* bug fixes

* fix lint

* fix lint

* fix test

* lint

* remove cuda

* fix tests

* fix ignore

* fix loading

* fix test

* save ckpt

---------

Co-authored-by: Mihir Patel <[email protected]>
Co-authored-by: Daniel King <[email protected]>
Co-authored-by: Your Name <[email protected]>

* Adding some fixes to FSDP tests

* Add filter warnings

…ch 1.13 (mosaicml#2497)

* skip metrics in state dict

* fix unit tests

* add peak memory stats

* fix tests

Bumps [gitpython](https://github.com/gitpython-developers/GitPython) from 3.1.31 to 3.1.34.
- [Release notes](https://github.com/gitpython-developers/GitPython/releases)
- [Changelog](https://github.com/gitpython-developers/GitPython/blob/main/CHANGES)
- [Commits](gitpython-developers/GitPython@3.1.31...3.1.34)

---
updated-dependencies:
- dependency-name: gitpython
  dependency-type: direct:development
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

The `torch_prof_remote_file_name` argument of `Profiler` is passed
as the `remote_file_name` argument of `TorchProfiler`, which supports
passing `None` to disable uploading trace files. Prior to this
commit, passing `None` to `Profiler` to do this whilst using a
static type checker led to a type error.

* no metric save

* fix docs

* checkdown

* fix tests

* filter warning

* move to device

* fix device gpu

* Update composer/core/state.py

Co-authored-by: Daniel King <[email protected]>

---------

Co-authored-by: Daniel King <[email protected]>

* Refactor mosaic_fsdp.py

* Format file

* Rename monkey patch function

* Fix import path

* Format files

* Fix version

* Refactor mosaic_fsdp.py

* Format file

* Rename monkey patch function

* Fix import path

* Format files

* Fix version

* Fix import path

* Monkey patch ChunkShardingSpec to dynamically detect sharding dim

* Format file

* Add non divisible functionality to ChunkShardingSpec

* Format file

* Format file

* add torch2

* add code

* tag more changes

* Update composer/trainer/mosaic_fsdp.py

Co-authored-by: Vitaliy Chiley <[email protected]>

* monkeypatch init

* raise pins

* add print

* more logs

* change if statements

* remove imports

* remove imports

* fix init

* fix versioning

* add hybrid shard

* checkdown

* revert hsdp

* add peak memory stats

* lint

* imports

* Update composer/trainer/mosaic_fsdp.py

Co-authored-by: Daniel King <[email protected]>

* fix wrap

* fix gate

* lint

* test

* change thresh

* import typing

* fix checks

* nuke pyright

* typo

* Update composer/trainer/mosaic_fsdp.py

Co-authored-by: Brian <[email protected]>

* Update composer/trainer/mosaic_fsdp.py

Co-authored-by: Brian <[email protected]>

* Update composer/trainer/mosaic_fsdp_utils.py

Co-authored-by: Brian <[email protected]>

* resolve comments

* add comments

* add comments

---------

Co-authored-by: Vitaliy Chiley <[email protected]>
Co-authored-by: Daniel King <[email protected]>
Co-authored-by: Brian <[email protected]>

Bumps [gitpython](https://github.com/gitpython-developers/GitPython) from 3.1.34 to 3.1.35.
- [Release notes](https://github.com/gitpython-developers/GitPython/releases)
- [Changelog](https://github.com/gitpython-developers/GitPython/blob/main/CHANGES)
- [Commits](gitpython-developers/GitPython@3.1.34...3.1.35)

---
updated-dependencies:
- dependency-name: gitpython
  dependency-type: direct:development
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

Bumps [pytest](https://github.com/pytest-dev/pytest) from 7.4.0 to 7.4.2.
- [Release notes](https://github.com/pytest-dev/pytest/releases)
- [Changelog](https://github.com/pytest-dev/pytest/blob/main/CHANGELOG.rst)
- [Commits](pytest-dev/pytest@7.4.0...7.4.2)

---
updated-dependencies:
- dependency-name: pytest
  dependency-type: direct:development
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

…ric agreggation (mosaicml#2531)

* fix github actions

* make gpu test

* add PR tests

* fix test

* remove pr daily

* remove pr daily

…r into error_logging_callback