Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Heterogeneous #684

Draft
wants to merge 146 commits into
base: main
Choose a base branch
from
Draft

Heterogeneous #684

wants to merge 146 commits into from

Conversation

XiaohanZhangCMU
Copy link
Member

Description of changes:

Issue #, if available:

Merge Checklist:

Put an x without space in the boxes that apply. If you are unsure about any checklist, please don't hesitate to ask. We are here to help! This is simply a reminder of what we are going to look for before merging your pull request.

General

  • I have read the contributor guidelines
  • This is a documentation change or typo fix. If so, skip the rest of this checklist.
  • I certify that the changes I am introducing will be backward compatible, and I have discussed concerns about this, if any, with the MosaicML team.
  • I have updated any necessary documentation, including README and API docs (if appropriate).

Tests

  • I ran pre-commit on my change. (check out the pre-commit section of prerequisites)
  • I have added tests that prove my fix is effective or that my feature works (if appropriate).
  • I ran the tests locally to make sure it pass. (check out testing)
  • I have added unit and/or integration tests as appropriate to ensure backward compatibility of the changes.

dependabot bot and others added 21 commits June 3, 2024 17:35
@dependabot
Bump pydantic from 2.7.1 to 2.7.2 (mosaicml#692)
f9871e6
@dependabot
Bump uvicorn from 0.29.0 to 0.30.1 (mosaicml#691)
4f338e1
@snarayan21
Make sure epoch_size is an int (mosaicml#693)
f5c57eb 
* typo

* epoch size int
@dependabot
Bump databricks-sdk from 0.27.1 to 0.28.0 (mosaicml#687)
55c9f85
@dependabot
Bump pytest from 8.2.1 to 8.2.2 (mosaicml#697)
f7c7a9a
@huxuan @snarayan21
fix: expand user path for Writer's output directory. (mosaicml#694)
8904102 
* fix: expand user path for Writer's output directory.

* Update streaming/base/format/base/writer.py

---------

Co-authored-by: Saaketh Narayan <[email protected]>
Co-authored-by: Saaketh Narayan <[email protected]>
@dependabot @snarayan21
Bump pydantic from 2.7.2 to 2.7.3 (mosaicml#696)
1af4365 
Bumps [pydantic](https://github.com/pydantic/pydantic) from 2.7.2 to 2.7.3.
- [Release notes](https://github.com/pydantic/pydantic/releases)
- [Changelog](https://github.com/pydantic/pydantic/blob/main/HISTORY.md)
- [Commits](pydantic/pydantic@v2.7.2...v2.7.3)

---
updated-dependencies:
- dependency-name: pydantic
  dependency-type: direct:development
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Saaketh Narayan <[email protected]>
@snarayan21
Fix edge cases with scalar or empty numpy array encoding (mosaicml#702)
be97343 
* typo

* wo
@snarayan21 @karan6181
Raise IndexError in Spanner object instead of ValueError (mosaicm…
dc8ac7b 
…l#701)

* typo

* woo

* woooo

---------

Co-authored-by: Karan Jariwala <[email protected]>
@snarayan21
Fix linting issues with numpy 2 (mosaicml#705)
ea4f0c3 
* lint

* lint

* isin
@dependabot
Bump pydantic from 2.7.3 to 2.7.4 (mosaicml#704)
2b53acb
@snarayan21 @mvpatel2000
Enable correct resumption from the end of an epoch (mosaicml#700)
a5b9eea 
* typo

* potensh

* tests

* tests

* Update streaming/base/partition/relaxed.py

Co-authored-by: Mihir Patel <[email protected]>

* Update streaming/base/partition/relaxed.py

Co-authored-by: Mihir Patel <[email protected]>

* ready

---------

Co-authored-by: Mihir Patel <[email protected]>
@snarayan21 @mvpatel2000
Fix drop_first checking in partitioning to account for world_size…
27d61d8 
… divisibility (mosaicml#706)

* typo

* potensh

* tests

* tests

* Update streaming/base/partition/relaxed.py

Co-authored-by: Mihir Patel <[email protected]>

* Update streaming/base/partition/relaxed.py

Co-authored-by: Mihir Patel <[email protected]>

* ready

* epoch_size_checks

* epoch_size_checks

* Update streaming/base/partition/__init__.py

Co-authored-by: Mihir Patel <[email protected]>

---------

Co-authored-by: Mihir Patel <[email protected]>
@Hprairie
fix convert imagenet (mosaicml#708)
eb61352
@dependabot
Bump pytest-split from 0.8.2 to 0.9.0 (mosaicml#710)
02db72d
@vanshcsingh
Remove duplicate dbfs: prefix from error message (mosaicml#712)
517dc6d
@bigning
a (mosaicml#713)
67ab85c
@snarayan21
Upgrade ci_testing, remove codeql (mosaicml#714)
7089eef 
* linting_codeql

* yo

* yo
@XiaohanZhangCMU
Wrap with nparray (mosaicml#719)
d9198ad
@dependabot @XiaohanZhangCMU
Bump pydantic from 2.7.4 to 2.8.2 (mosaicml#718)
6bbcf5a 
Bumps [pydantic](https://github.com/pydantic/pydantic) from 2.7.4 to 2.8.2.
- [Release notes](https://github.com/pydantic/pydantic/releases)
- [Changelog](https://github.com/pydantic/pydantic/blob/main/HISTORY.md)
- [Commits](pydantic/pydantic@v2.7.4...v2.8.2)

---
updated-dependencies:
- dependency-name: pydantic
  dependency-type: direct:development
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Xiaohan Zhang <[email protected]>
@dependabot @XiaohanZhangCMU
Bump databricks-sdk from 0.28.0 to 0.29.0 (mosaicml#715)
2f7defa 
Bumps [databricks-sdk](https://github.com/databricks/databricks-sdk-py) from 0.28.0 to 0.29.0.
- [Release notes](https://github.com/databricks/databricks-sdk-py/releases)
- [Changelog](https://github.com/databricks/databricks-sdk-py/blob/main/CHANGELOG.md)
- [Commits](databricks/databricks-sdk-py@v0.28.0...v0.29.0)

---
updated-dependencies:
- dependency-name: databricks-sdk
  dependency-type: direct:development
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Xiaohan Zhang <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@karan6181 karan6181 Awaiting requested review from karan6181 karan6181 will be requested when the pull request is marked ready for review karan6181 is a code owner

@bandish-shah bandish-shah Awaiting requested review from bandish-shah bandish-shah will be requested when the pull request is marked ready for review bandish-shah is a code owner

@snarayan21 snarayan21 Awaiting requested review from snarayan21 snarayan21 will be requested when the pull request is marked ready for review snarayan21 is a code owner

At least 1 approving review is required to merge this pull request.

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
10 participants
@XiaohanZhangCMU @snarayan21 @huxuan @Hprairie @vanshcsingh @bigning @orionw @mvpatel2000 @eitanturok @jaehwana2z