Skip to content
This repository was archived by the owner on Sep 9, 2025. It is now read-only.

Proposal for new Evaluation repo - #79

Merged
nathan-weinberg merged 2 commits into
instructlab:mainfrom
nathan-weinberg:new-eval
Jun 11, 2024
Merged

Proposal for new Evaluation repo#79
nathan-weinberg merged 2 commits into
instructlab:mainfrom
nathan-weinberg:new-eval

Conversation

@nathan-weinberg

Copy link
Copy Markdown
Contributor

Resolves #76

@mergify mergify Bot added the backend InstructLab Backend Services label Jun 6, 2024
@nathan-weinberg
nathan-weinberg force-pushed the new-eval branch 5 times, most recently from ec0937d to dfad7d9 Compare June 10, 2024 15:11
@nathan-weinberg nathan-weinberg changed the title [WIP] Proposal for new Evaluation repo Proposal for new Evaluation repo Jun 10, 2024
@nathan-weinberg
nathan-weinberg marked this pull request as ready for review June 10, 2024 15:11
@nathan-weinberg
nathan-weinberg force-pushed the new-eval branch 3 times, most recently from 2678fd5 to 88ed642 Compare June 10, 2024 17:08

@russellb russellb left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

one suggested addition

Comment thread docs/backend/eval-repo.md
@nathan-weinberg
nathan-weinberg requested a review from cdoern June 10, 2024 18:31

@russellb russellb left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm - going to leave this open a while longer to gather more ACKs

Signed-off-by: Nathan Weinberg <nweinber@redhat.com>
@danmcp

danmcp commented Jun 10, 2024

Copy link
Copy Markdown
Contributor

LGTM

Comment thread docs/evaluation/eval-repo.md
Comment thread docs/evaluation/eval-repo.md
Comment thread docs/evaluation/eval-repo.md Outdated
@alimaredia

Copy link
Copy Markdown
Contributor

Besides the paragraph I wanted removed, this proposal in fantastic! Thanks Nathan!

Co-authored-by: Ali Maredia <amaredia@redhat.com>
Signed-off-by: Nathan Weinberg <nweinber@redhat.com>
@nathan-weinberg
nathan-weinberg merged commit 3ee5f52 into instructlab:main Jun 11, 2024
@nathan-weinberg
nathan-weinberg deleted the new-eval branch June 11, 2024 01:40

@lhawthorn lhawthorn left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, nice work @nathan-weinberg

@lhawthorn

Copy link
Copy Markdown
Contributor

@nathan-weinberg I believe our intent is to have proposals for new repos approved by the Oversight Committee, though I admit that if that is true it could be made more clear in our governance documentation.

Arrived here via ping from @russellb to the oversight committee to review this proposal, so will let him as pinger and Oversight Committee chair opine on my statement above.

@nathan-weinberg

Copy link
Copy Markdown
Contributor Author

@lhawthorn thanks! I assumed the approval from @russellb as a member of the Oversight Committee fulfilled that requirement, is that not the case?

The repo is already created so it's a moot point now, but if more process is needed someone will need to flesh that out in the doc you linked for next time.

@alinaryan

Copy link
Copy Markdown
Contributor

LGTM

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

backend InstructLab Backend Services

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Create proposal for new evaluation library repo

6 participants