Skip to content

quickstart.md

Latest commit

 

History

History
96 lines (69 loc) · 5.41 KB

quickstart.md

File metadata and controls

96 lines (69 loc) · 5.41 KB
Raw

Quickstart

This quickstart gives the smallest runnable CLI and Python examples and tells you what output to expect. For background and terminology, see Concepts.

One minimal run

Use the benchmark runner when you want a full pipeline driven by a YAML config, and use the Python snippet when you want to exercise the sampling API directly in code. [1][2][3][4]

Run the benchmark runner with the toy inductive config:

python -m bench.main --config bench/configs/experiments/toy_inductive.yaml

If your config uses environment placeholders (like ${MODSSC_OUTPUT_DIR}), export them first:

export MODSSC_OUTPUT_DIR=/tmp/modssc_runs
export MODSSC_DATASET_CACHE_DIR=/tmp/modssc_cache/datasets
export MODSSC_PREPROCESS_CACHE_DIR=/tmp/modssc_cache/preprocess

Or set one global cache root for all runtime caches:

export MODSSC_CACHE_ROOT=/tmp/modssc_cache

Run a quick Python API pass with the dataset loader and sampling plan:

from modssc.data_loader import load_dataset
from modssc.sampling import HoldoutSplitSpec, LabelingSpec, SamplingPlan, sample

ds = load_dataset("toy", download=True)
plan = SamplingPlan(
    split=HoldoutSplitSpec(test_fraction=0.0, val_fraction=0.2, stratify=True),
    labeling=LabelingSpec(mode="fraction", value=0.2, strategy="balanced"),
)
res, _ = sample(ds, plan=plan, seed=0, dataset_fingerprint=str(ds.meta["dataset_fingerprint"]))
print(res.stats)

The benchmark command and the toy config live in bench/main.py and bench/configs/experiments/toy_inductive.yaml. The sampling API is exposed through the public modssc.sampling package and built around plan plus storage primitives such as src/modssc/sampling/plan.py. [1][2][3][4]

What you should see

For the benchmark run, a timestamped folder is created under the configured run.output_dir (for example runs/) containing:

  • config.yaml (copied config)
  • run.json (metrics + metadata)
  • error.txt (only if the run fails)

These outputs are written by the bench runner and context utilities. [5][6][7][8]

For the Python snippet, you should see a stats dictionary printed to stdout. The stats structure is produced by modssc.sampling.stats.build_inductive_stats. [9]

Next steps

Troubleshooting

!!! warning If a dataset provider is missing, the loader raises an optional dependency error with a suggested pip install "modssc[extra]" command. [10]

!!! tip Use modssc doctor to see which CLI bricks are available and which extras are missing. [11]

If you want a symptom-driven route instead of a workflow page, use Common errors and where to go.

Sources
  1. bench/main.py
  2. bench/configs/experiments/toy_inductive.yaml
  3. src/modssc/sampling/api.py
  4. src/modssc/sampling/plan.py
  5. README.md
  6. bench/README.md
  7. bench/context.py
  8. bench/orchestrators/reporting.py
  9. src/modssc/sampling/stats.py
  10. src/modssc/data_loader/errors.py
  11. src/modssc/cli/app.py