Skip to content

Add Echo-Memory community pipeline for Wan 2.1 1.3B - #14471

Open
WayneJin0918 wants to merge 2 commits into
huggingface:mainfrom
WayneJin0918:echo-memory-community-pipeline
Open

Add Echo-Memory community pipeline for Wan 2.1 1.3B#14471
WayneJin0918 wants to merge 2 commits into
huggingface:mainfrom
WayneJin0918:echo-memory-community-pipeline

Conversation

@WayneJin0918

@WayneJin0918 WayneJin0918 commented Aug 13, 2026 •
edited
Loading

Copy link
Copy Markdown

What does this PR do?

Adds a GitHub community pipeline that overlays the released Echo-Memory context_k1 DiT fine-tune onto official Wan-AI/Wan2.1-T2V-1.3B-Diffusers.

This is a community overlay, not a new official Wan checkpoint. Extra action-MLP / SSM slots stay in the Echo-Memory research stack (arXiv:2606.09803).

Fixes #14482

Before submitting

  • Did you use an AI agent (Claude Code, Codex, Cursor, etc.) to help with this PR? If so:
    • Did you read the Coding with AI agents guide?
    • Did you run the self-review skill on the diff?
    • Did you share the final self-review notes in the PR description or a comment?
  • Did you read the contributor guideline?
  • Did you read our philosophy doc? (important for complex PRs)
  • Was this discussed/approved via a GitHub issue or the forum? Please add a link to it if that's the case.
  • Did you make sure to update the documentation with your changes? Here are the
    documentation guidelines, and
    here are tips on formatting docstrings.
  • Did you write any new necessary tests?
  • Are you the author (or part of the team) of the model/pipeline (only applicable for model/pipeline related PRs)?

Self-review notes

  • Community exception: this subclasses WanPipeline on purpose. .ai/pipelines.md says not to subclass sibling pipelines inside src/; examples/community is the documented place for overlays.
  • load_echo_memory_weights mutates self.transformer in place. That is the overlay. Call it once after from_pretrained.
  • The rename dict is copied from scripts/convert_wan_to_diffusers.py because that script is not importable. Offline check: 825 / 825 official Wan 1.3B transformer keys match after remap; 360 action/SSM extras are skipped.
  • No src/ changes. No GPU generation test in this PR; remap helpers were checked locally with dummy tensors and against the official Diffusers weight map.
  • Left from_echo_memory as a convenience constructor. Happy to drop it if reviewers prefer only load_*.

Test plan

  • Dummy-tensor remap: self_attn.q → attn1.to_q, norm2/norm3 swap, action_mlp / self_attn_with_action skipped
  • Official index overlap: converted context_k1 keys == Wan-AI/Wan2.1-T2V-1.3B-Diffusers transformer weight_map (825 / 825)
  • DiffusionPipeline.from_pretrained("Wan-AI/Wan2.1-T2V-1.3B-Diffusers", custom_pipeline="pipeline_echo_memory")
  • pipe.load_echo_memory_weights() or pipe.load_converted_echo_memory_weights()

Converted weights and the same pipeline class: Wayne-King/echo-memory-diffusers.

Who can review?

@asomoza (community pipelines)

@WayneJin0918 @cursoragent
Add an Echo-Memory community pipeline for Wan 2.1 1.3B.
4f80a4c 
Overlay the released context_k1 row onto official Diffusers Wan weights after remapping original DiffSynth/Wan keys. Extra action/SSM slots stay in the Echo-Memory research stack.

Co-authored-by: Cursor <cursoragent@cursor.com>
@github-actions github-actions Bot added examples size/M PR with diff < 200 LOC labels Aug 13, 2026
@gonzalolater

gonzalolater commented Aug 14, 2026

Copy link
Copy Markdown

ok

@github-actions

github-actions Bot commented Aug 14, 2026

Copy link
Copy Markdown
Contributor

Hi @WayneJin0918, thanks for the PR! It does not appear to link an issue it fixes. If this PR addresses an existing issue, please add a closing keyword (e.g. Fixes #1234) to the PR description so the issue is linked. See the contribution guide for more details. If this PR intentionally does not fix a tracked issue, a maintainer can add the no-issue-needed label to silence this reminder.

@WayneJin0918 WayneJin0918 mentioned this pull request Aug 15, 2026
Open
2 tasks
@WayneJin0918 @cursoragent
Use the Diffusers logger in the Echo-Memory community pipeline.
444cb8c 
Replace print logging and guard an empty transformer so the overlay matches community pipeline conventions.

Co-authored-by: Cursor <cursoragent@cursor.com>
@WayneJin0918

WayneJin0918 commented Aug 15, 2026

Copy link
Copy Markdown
Author

Thanks for the reminder. This PR is a community-pipeline addition rather than a bugfix, so I opened a tracking issue and linked it:

Fixes #14482

Also updated the description to the contributor template (AI-agent checklist + self-review notes) and switched the overlay helper from print to the Diffusers logger.

@github-actions github-actions Bot added size/L PR with diff > 200 LOC and removed size/M PR with diff < 200 LOC labels Aug 15, 2026
@WayneJin0918

WayneJin0918 commented Aug 15, 2026

Copy link
Copy Markdown
Author

Following the guidance on #14482: the usable path is the Hub repo, not this examples/community file.

Live pipeline: https://huggingface.co/Wayne-King/echo-memory-diffusers

I commented on the issue as an author and tagged @asomoza. Happy to close this PR if you prefer Hub / Modular only, or to keep it if you still want the GitHub community example.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

examples size/L PR with diff > 200 LOC

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

[Community pipeline] Echo-Memory overlay for Wan 2.1 1.3B

2 participants

@WayneJin0918 @gonzalolater