Skip to content

feat: redesign python API #713

Description

@SemyonSinchenko

Is your feature request related to a problem? Please describe.
It is really hard to maintain due to two implementations.

Describe the solution you would like

  • All the logic is in the top-level graphframes.py instead of being duplicated in connect / classic
  • Naming convention should follow Python, not Scala (snake_case)
  • Pregel and AM should be top-level classes without details of the implementation
  • Connect / Classic should not contain logic, docs, etc., but should be just simple functions classes to proxy the py4j or protobuf

Component

  • Scala Core Internal
  • Scala API
  • Spark Connect Plugin
  • Infrastructure
  • PySpark Classic
  • PySpark Connect

Additional context
BREAKING CHANGES

I see it as a part of the planning 1.0 release.

Are you planning on creating a PR?

  • I'm willing to make a pull-request

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Projects

    No projects

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions