Skip to content

doc: update README with details on nightly builds #622

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
SemyonSinchenko merged 1 commit into from
Jul 4, 2025
Merged

doc: update README with details on nightly builds #622

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
65 changes: 37 additions & 28 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,12 +3,13 @@
[![Scala CI](https://github.com/graphframes/graphframes/actions/workflows/scala-ci.yml/badge.svg)](https://github.com/graphframes/graphframes/actions/workflows/scala-ci.yml)
[![Python CI](https://github.com/graphframes/graphframes/actions/workflows/python-ci.yml/badge.svg)](https://github.com/graphframes/graphframes/actions/workflows/python-ci.yml)
[![pages-build-deployment](https://github.com/graphframes/graphframes/actions/workflows/pages/pages-build-deployment/badge.svg)](https://github.com/graphframes/graphframes/actions/workflows/pages/pages-build-deployment)
[![scala-central-publish](https://github.com/graphframes/graphframes/actions/workflows/scala-publish.yml/badge.svg)](https://github.com/graphframes/graphframes/actions/workflows/scala-publish.yml)

# GraphFrames: DataFrame-based Graphs

This is a package for graphs processing and analytics on scale. It is built on top of Apache Spark and relies on DataFrame abstraction. Users can write highly expressive queries by leveraging the DataFrame API, combined with a new API for network motif finding. The user also benefits from DataFrame performance optimizations within the Spark SQL engine. GraphFrames works in Java, Scala, and Python.

You can find user guide and API docs at https://graphframes.github.io/graphframes
You can find user guide and API docs at https://graphframes.io

## GraphFrames is Back!

Expand Down Expand Up @@ -172,25 +173,15 @@ libraryDependencies += "graphframes" % "graphframes" % "0.8.4-spark3.5-s_2.12"

## GraphFrames and Maven

GraphFrames is not on Maven Central Repository but we are going to restore it soon. For now use Spark Packages system to install the package: [https://spark-packages.org/package/graphframes/graphframes](https://spark-packages.org/package/graphframes/graphframes).

```xml
<dependencies>
<!-- list of dependencies -->
<dependency>
<groupId>graphframes</groupId>
<artifactId>graphframes</artifactId>
<version>0.8.4-spark3.5-s_2.12</version>
</dependency>
</dependencies>
<repositories>
<!-- list of other repositories -->
<repository>
<id>SparkPackagesRepo</id>
<url>https://repos.spark-packages.org/</url>
</repository>
</repositories>
```
Please see the section about nightly builds!

**WARNING!**

**=========================**

Due to governance problems and limitations, all the new releases of `GraphFrames` will be published to the Maven Central under the namespace `io.graphframes` (not `org.graphframes`)!

**=========================**

## GraphFrames Internals

Expand All @@ -199,22 +190,40 @@ Relational Queries, Dave et al. 2016](https://people.eecs.berkeley.edu/~matei/pa

## Building and running unit tests

To compile this project, run `build/sbt assembly` from the project home directory. This will also run the Scala unit tests.

To run the Python unit tests, run the `run-tests.sh` script from the `python/` directory. You will need to set `SPARK_HOME` to your local Spark installation directory.
To compile the core project, run `build/sbt package` from the project home directory.
To compile the Spark Connect Plugin, run `build/sbt connect/package`

## Release new version
## Spark version compatibility

Please see guide `dev/release_guide.md`.
This project is compatible with Spark 3.4+. Significant speed improvements have been made to DataFrames in recent versions of Spark, so you may see speedups from using the latest Spark version.

## Spark version compatibility
Nightly builds of GraphFrames:

This project is compatible with Spark 3.4+. Significant speed improvements have been made to DataFrames in recent versions of Spark, so you may see speedups from using the latest Spark version.
| Component | Spark 3.x (Scala 2.12) | Spark 3.x (Scala 2.13) | Spark 4.x (Scala 2.13) |
|---------------------|------------------------|------------------------|------------------------|
| graphframes | ✓ | ✓ | ✓ |
| graphframes-connect | ✓ | ✓ | ✓ |

## Contributing

GraphFrames is collaborative effort among UC Berkeley, MIT, Databricks and the open source community. We welcome open source contributions as well!
GraphFrames was made as collaborative effort among UC Berkeley, MIT, Databricks and the open source community. At the moment GraphFrames is maintained by the group of individual contributors.

See [contribution guide](./CONTRIBUTING.md)

## Releases:

See [release notes](https://github.com/graphframes/graphframes/releases).

## Nightly builds

GraphFrames project is publishing SNAPSHOTS (nightly builds) to the "Central Portal Snapshots."
Please read [this section](https://central.sonatype.org/publish/publish-portal-snapshots/#consuming-snapshot-releases-for-your-project) of the Sonatype documentation to check how can you use snapshots in your project.

GroupId: `io.graphframes`
ArtifactIds:
- `graphframes-spark3_2.12`
- `graphframes-spark3_2.13`
- `graphframes-connect-spark3_2.12`
- `graphframes-connect-spark3_2.13`
- `graphframes-spark4_2.13`
- `graphframes-connect-spark4_2.13`