Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added IOMixin for reporter #74

Draft
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

Hrsj123
Copy link

@Hrsj123 Hrsj123 commented Feb 8, 2024

closes #48

I've tried to create a new IOMixin class that provides methods for opening, reading, writing, and closing files at a given path.

Please review the code and share your feedback on it.

@Hrsj123 Hrsj123 marked this pull request as draft February 8, 2024 18:07
@nicholasjng
Copy link
Collaborator

Thanks for the PR. I assume you crafted this from my suggestion in #48?

We rearranged some APIs on the tentative base reporter interface, which I merged yesterday in #71.

There, instead of open() and close(), we have an API set consisting of initialize/finalize (to acquire/release resources that are needed by a reporter during its object lifetime), read/write (for benchmark record IO), and batched versions thereof.

We decided to (temporarily) not go ahead with open() and close() because they are too specific to the file case.
FWIW, the Beam file-based sink, which was our main inspiration for the API set presented in #48, is directly tied to file IO as well.

That's not to say that this is useless, though. I'm wondering how we could use such an abstraction to abstractify file writes to whichever format we like, such as JSON, YAML, TOML, the usual suspects.

If you'd like to get involved in that, I would suggest the following:

  1. Pull in current main, and create a feature branch for a file reporter.
  2. Add a Python file in src/nnbench/reporter/, maybe file.py, which holds the file reporter.
  3. The constructor should take a result directory, which we save the records to.
  4. The initialize() method should create this directory, if not already existent.
  5. The read() method should open a single file in that directory, and parse the data with the given format, like e.g. JSON, TOML, YAML.
  6. The write() method should write the record to a file in that directory, just like you implemented here.
  7. Finalize can be empty, or optionally destroy the directory again if you'd like.

Once we have that, we can think about how to make this work across file formats. How does that sound?

@Hrsj123
Copy link
Author

Hrsj123 commented Feb 11, 2024

I got your point, I will try creating a generic file reporter.

@Hrsj123 Hrsj123 mentioned this pull request Feb 12, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add flexible base reporters for file IO, web, and DBs
2 participants