Skip to content

feat: localCheckpoints + docs - #662

Merged
SemyonSinchenko merged 3 commits into
graphframes:masterfrom
SemyonSinchenko:651-localCheckpoint
Jul 21, 2025
Merged

SemyonSinchenko merged 3 commits into
graphframes:masterfrom
SemyonSinchenko:651-localCheckpoint

Conversation

@SemyonSinchenko

Copy link
Copy Markdown
Collaborator

What changes were proposed in this pull request?

  • optional configuration to use localCheckpoint instead of checkpoint or parquet
  • updated tests
  • new documentation page with GraphFrames configurations
  • small updates and fixes in docs

Why are the changes needed?

Close #651

P.S. I will be honest, that I used AI-agent to generate configurations.md but I checked it manually and it looks OK for me.

@SemyonSinchenko

Copy link
Copy Markdown
Collaborator Author

I'm thinking about using local checkpoints as defaults starting from 1.0 release to avoid problems with Spark Connect, k8s, etc. where using persistent storage is not always possible or may be hard to configure.

Comment thread core/src/main/scala/org/graphframes/lib/ConnectedComponents.scala Outdated
…point

# Conflicts:
#	core/src/test/scala/org/graphframes/lib/PregelSuite.scala
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat: use localCheckpoint in iterative algorithms

2 participants