Bring in Rules.md and YAML schema #222
Open
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Expanded the directory structure requirements for CLOSED and OPEN submissions, detailing the necessary files and their organization within the submission structure.
Updated submission guidelines and directory structure requirements for OPEN and CLOSED benchmarks, including detailed validation checks and file requirements.
Remove all the text that carried over from the Submission_guidelines.md file, and start filling out the main structure of the sections of the new document.
Updated sections for validating training options and checkpointing options in the rules document.
Removed section on validating phases and its related content.
Reorganize the sections under 'Validating the Checkpointing Options' for clarity.
First drraft of a schema for the SystemDescription.yaml files created by submitters. This is in the format required by Yamale.
Add README for submission validation checker directory
|
MLCommons CLA bot All contributors have signed the MLCommons CLA ✍️ ✅ |
Contributor
Author
|
I plan to reformat the text to include a short title when I get a chance. If this PR has not yet been merged into Modernization when I'm ready to do that work, I'll do the work in my branch and do another PR to bring it into Modernization. |
Clarify the meaning of two rules.
Updated rules to clarify requirements and add description tokens for each section.
|
FYI: PR mlcommons/inference_policies#324 for proposed YAML for PSU data. We'll want to start working on integrating this into the YAML. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This brings in the latest version of the Rules.md file, and an early snapshot of the ongoing development of the YAML schema checker. Future Rules.md development should be done in the Modernization branch (ie: I'll stop working on it in my branch), but since @dslik is working on the Schema (based in my branch), he will need to decide how to get his work into the Modernization branch.