feat: refactor ConnectedComponents API - #803

Merged

SemyonSinchenko merged 31 commits into

graphframes:mainfrom

SemyonSinchenko:775-refactor-cc-api

Mar 11, 2026

Commits on Feb 28, 2026

Commits on Mar 5, 2026

refactor: remove GraphX implementation and maxIter from TwoPhase
SemyonSinchenko
committed
3f6c1d0
feat: add runAQE method for AQE-based two-phase connected components
SemyonSinchenko
committed
b97f19c
refactor: calcMinNbrSum object method, DecimalType(38,10), drop overflow
SemyonSinchenko
committed
44772b3
refactor: simplify calcMinNbrSum and remove unused variable
SemyonSinchenko
committed
44bfb4e
refactor: extract shared output generation to buildOutput method
SemyonSinchenko
committed
354ec7d
feat: add intermediate storage level and unpersist in GraphX CC
SemyonSinchenko
committed
6a8c0bf
feat: use runAQE for two-phase CC when broadcastThreshold is -1
SemyonSinchenko
committed
3d1972e
refactor: remove unused spark variable in TwoPhase
SemyonSinchenko
committed
d3d3230
feat: add isGraphPrepared to skip graph preparation in run and runAQE
SemyonSinchenko
committed
4cac662
refactor: add isGraphPrepared and checkpoint params to algorithm runs
SemyonSinchenko
committed
2321744
feat: add internal graph preparation control to ConnectedComponents
SemyonSinchenko
committed
2047d52
refactor: rename _isGraphPrepared and update setter
SemyonSinchenko
committed
4949d3c
docs: clarify setIsGraphPrepared docstring on algorithm prep steps
SemyonSinchenko
committed
43982af
docs: replace Connected Components section with comprehensive algorithm guide
SemyonSinchenko
committed
6d778b7
feat: rename graphframes to two_phase and add checkpointing to rand_cont
SemyonSinchenko
committed
075026b
test: add local checkpointing to RandomizedContractionSuite
SemyonSinchenko
committed
d2d2ecb
refactor: unpersist edges earlier and in finally block
SemyonSinchenko
committed
5bfaac1
test: add System.gc() to make test more robust
SemyonSinchenko
committed
7813237
test: clean up Spark checkpoint directory in RandomizedContractionSuite
SemyonSinchenko
committed
328f4fc
test(RandomizedContractionSuite): move checkpoint cleanup after unpersist
SemyonSinchenko
committed
cc3abbd
chore: check memory leaks for persisted only
SemyonSinchenko
committed
04ebe28
fix: make the test robust, not random
SemyonSinchenko
committed
a4aee16
chore: drop the redundant
SemyonSinchenko
committed
8562ee6

Commits on Mar 6, 2026

chor: naming
SemyonSinchenko
committed
54a7ae9

Commits on Mar 11, 2026

Merge remote-tracking branch 'graphframes/main' into 775-refactor-cc-api
SemyonSinchenko
committed
ab5aec2
fix: addressing James' comments
SemyonSinchenko
committed
6915dc2
fix: python docstrings
SemyonSinchenko
committed
ce062ff