Skip to content

How to attach more than 100 artifacts. #383

@lisb20

Description

@lisb20

Hi, thank you for the great work on this project — it’s been very helpful for my experiments.

While digging into the code, I noticed that there appears to be a hard limit of 100 artifacts being preserved in the evolving process. In the code (L400 process_parallel.py), I see the following comment and logic:

# Include artifacts for programs that might be selected
# IMPORTANT: This limits artifacts (execution outputs/errors) to first 100 programs only.
# This does NOT affect program code - all programs are fully serialized above.
# With max_artifact_bytes=20KB and population_size=1000, artifacts could be 20MB total,
# which would significantly slow worker process initialization. The limit of 100 keeps
# artifact data under 2MB while still providing execution context for recent programs.
# Workers can still evolve properly as they have access to ALL program code.
# for pid in list(self.database.programs.keys())[:100]:
for pid in list(self.database.programs.keys()):
    artifacts = self.database.get_artifacts(pid)
    if artifacts:
        snapshot["artifacts"][pid] = artifacts

I tried commenting out the [:100] slice (as shown above), expecting that artifacts for more than 100 programs would be retained. However, even after this change, the system still does not keep more than 100 artifacts.

This makes me wonder whether there is another implicit limit elsewhere (e.g., in the database layer, snapshot serialization, or artifact storage logic) that enforces the same constraint.

Could you please let me know:

Whether this 100-artifact limit is intentionally enforced in another part of the code, or if there is a recommended way to increase or disable this limit safely?

Thanks a lot for your time and for maintaining this project!

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions