Skip to content
This repository was archived by the owner on Apr 23, 2026. It is now read-only.

enable local document sources to remove dependency on remote git fetching - #1382

Closed
relyt0925 wants to merge 1 commit into
instructlab:mainfrom
relyt0925:local-document-sources
Closed

enable local document sources to remove dependency on remote git fetching#1382
relyt0925 wants to merge 1 commit into
instructlab:mainfrom
relyt0925:local-document-sources

Conversation

@relyt0925

Copy link
Copy Markdown
Contributor

Currently in the generation process: if any knowledge document or grounded skill is being processed: there is a hard dependency on needing to fetch the documents from the remote git source. For potential private taxonomy sources: this requires the ilab cli git processes to be configured to talk to the private git sources storing the documents. This introduces the ability to add a local document directory in the knowledge or grounded skill directory that can hold all document contents. The SDG process can then read all content from this directory meaning if the content has already been prepulled there is no need to dynmaically reach out to the remote git repo

Checklist:

  • Commit Message Formatting: Commit titles and messages follow guidelines in the
    conventional commits.
  • Changelog updated with breaking and/or notable changes for the next minor release.
  • Documentation has been updated, if necessary.
  • Unit tests have been added, if necessary.
  • Integration tests have been added, if necessary.

@mergify mergify Bot added the needs-rebase This Pull Request needs to be rebased label Jun 17, 2024
@mergify

mergify Bot commented Jun 17, 2024

Copy link
Copy Markdown
Contributor

This pull request has merge conflicts that must be resolved before it can be
merged. @relyt0925 please rebase it. https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

@mergify mergify Bot added the ci-failure PR has at least one CI failure label Jun 17, 2024
@relyt0925
relyt0925 force-pushed the local-document-sources branch from cee38ab to b16ce35 Compare June 17, 2024 03:15
@mergify mergify Bot removed the needs-rebase This Pull Request needs to be rebased label Jun 17, 2024
@relyt0925
relyt0925 force-pushed the local-document-sources branch from b16ce35 to 378f85a Compare June 17, 2024 03:17
@mergify mergify Bot added ci-failure PR has at least one CI failure and removed ci-failure PR has at least one CI failure labels Jun 17, 2024
@relyt0925

Copy link
Copy Markdown
Contributor Author

/retest

@relyt0925
relyt0925 force-pushed the local-document-sources branch from 378f85a to eade27f Compare June 17, 2024 04:05
@mergify mergify Bot removed the ci-failure PR has at least one CI failure label Jun 17, 2024
Currently in the generation process: if any knowledge document or grounded skill is being processed: there is a hard dependency on needing to fetch the documents from the remote git source. For potential private taxonomy sources: this requires the ilab cli git processes to be configured to talk to the private git sources storing the documents. This introduces the ability to add a local document directory in the knowledge or grounded skill directory that can hold all document contents. The SDG process can then read all content from this directory meaning if the content has already been prepulled there is no need to dynmaically reach out to the remote git repo.

Signed-off-by: Tyler Lisowski <lisowski@us.ibm.com>
@relyt0925
relyt0925 force-pushed the local-document-sources branch from eade27f to 5354371 Compare June 17, 2024 04:21
@bjhargrave

Copy link
Copy Markdown
Contributor

I am not sure this is a good idea.

See some discussion around knowledge sources in:

#1160 (comment)

And any change in knowledge sources would need to be coordinated across the frontend, backend, schema, and taxonomy repos. So such changes should start with a design document in dev-docs to gain consensus on a design which can then be properly documented for users.

@relyt0925 relyt0925 closed this Jun 17, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants