Commits

Breadcrumbs

History for

onmaster

User selector

All users

All time

Commit History

Commits on Dec 17, 2025

• [SPARK-41916][FOLLOW-UP] Remove unused imports

  Show description for 1413dbb
  zhengruifeng

  committed
  1413dbb

  Copy full SHA for 1413dbb

• [SPARK-41916][FOLLOW-UP] Restore python linter

  Show description for cebd4ab
  zhengruifeng

  committed
  cebd4ab

  Copy full SHA for cebd4ab

• [SPARK-41916][ML] Torch distributor: support multiple torchrun processes per task if task.gpu.amount > 1

  Show description for b9ac53d
  WeichenXu123

  committed
  b9ac53d

  Copy full SHA for b9ac53d

• [SPARK-54728][PYTHON][DOCS] Remove a wrong note in `dataframe.isEmpty`

  Show description for 10bc8bc
  zhengruifeng

  committed
  10bc8bc

  Copy full SHA for 10bc8bc

• [SPARK-54706][ML][CONNECT] Make DistributedLDAModel work with local file system

  Show description for 8a96d59
  zhengruifeng

  committed
  8a96d59

  Copy full SHA for 8a96d59

Commits on Dec 16, 2025

• [SPARK-54453][PYTHON][TEST] Add more coverage tests to conversion

  Show description for 31d996d
  

  gaogaotiantian

  authored and

  zhengruifeng

  committed
  31d996d

  Copy full SHA for 31d996d

• [SPARK-54589][PYTHON] Consolidate ArrowStreamAggPandasIterUDFSerializer into ArrowStreamAggPandasUDFSerializer

  Show description for 25c57d8
  

  Yicong-Huang

  authored and

  zhengruifeng

  committed
  25c57d8

  Copy full SHA for 25c57d8

• [SPARK-54702][INFRA] Recognize existing coverage object to install worker hooks

  Show description for 86fc601
  

  gaogaotiantian

  authored and

  zhengruifeng

  committed
  86fc601

  Copy full SHA for 86fc601

Commits on Dec 15, 2025

• [SPARK-54692][DOCS] Add python_worker_logs tvf doc to API reference

  Show description for 48c323a
  

  ueshin

  authored and

  zhengruifeng

  committed
  48c323a

  Copy full SHA for 48c323a

Commits on Dec 12, 2025

• [SPARK-53616][PYTHON] Introduce iterator API for pandas grouped agg UDF

  Show description for f601aa6
  

  Yicong-Huang

  authored and

  zhengruifeng

  committed
  f601aa6

  Copy full SHA for f601aa6

• [SPARK-54632][INFRA][FOLLOW-UP] Reenable ruff on our CI and lint-python

  Show description for 4c58ecb
  

  gaogaotiantian

  authored and

  HyukjinKwon

  committed
  4c58ecb

  Copy full SHA for 4c58ecb

Commits on Dec 11, 2025

• [SPARK-51920][SS][PYTHON] Fix type handling of namedTuple for transfromWithState

  Show description for ac5fce4
  

  zeruibao

  authored and

  HeartSaVioR

  committed
  ac5fce4

  Copy full SHA for ac5fce4

• [SPARK-54643][PYTHON][TESTS][FOLLOW-UP] Restore classic only workflows

  Show description for ca17514
  zhengruifeng

  committed
  ca17514

  Copy full SHA for ca17514

• [SPARK-52837][PYTHON][FOLLOWUP] Add `versionadded` to `TimeType` doc

  Show description for d80a0f3
  dongjoon-hyun

  committed
  d80a0f3

  Copy full SHA for d80a0f3

• Revert "[SPARK-54632][INFRA][FOLLOW-UP] Enable ruff on our CI and lint-python"

  Show description for 9daad92
  HyukjinKwon

  committed
  9daad92

  Copy full SHA for 9daad92

Commits on Dec 10, 2025

• [SPARK-54639][PYTHON] Avoid unnecessary Table creation in Arrow serializers

  Show description for 19a1da9
  

  Yicong-Huang

  authored and

  HyukjinKwon

  committed
  19a1da9

  Copy full SHA for 19a1da9

• [SPARK-54632][INFRA][FOLLOW-UP] Enable ruff on our CI and lint-python

  Show description for 19ec991
  

  gaogaotiantian

  authored and

  HyukjinKwon

  committed
  19ec991

  Copy full SHA for 19ec991

• [SPARK-54603][PYTHON][DOCS] Fix typo in contributing.rst

  Show description for 7e684b3
  

  Yicong-Huang

  authored and

  HyukjinKwon

  committed
  7e684b3

  Copy full SHA for 7e684b3

• [SPARK-54604][PYTHON][DOCS] Fix grammar issues in python_data_source.rst

  Show description for 179ade2
  

  Yicong-Huang

  authored and

  HyukjinKwon

  committed
  179ade2

  Copy full SHA for 179ade2

• [SPARK-54532][PYTHON] Add sqlstate for a set of PySparkException error classes

  Show description for 7d62da3
  

  khakhlyuk

  authored and

  HyukjinKwon

  committed
  7d62da3

  Copy full SHA for 7d62da3

• [SPARK-54653][DOCS] Add cross-session note to cache/persist public APIs

  Show description for 8065ba7
  

  yyanyy

  authored and

  gengliangwang

  committed
  8065ba7

  Copy full SHA for 8065ba7

• [SPARK-54340][PYTHON][FOLLOW-UP] Add link and examples for `run-with-viztracer`

  Show description for 4520c17
  

  zhengruifeng

  authored and

  dongjoon-hyun

  committed
  4520c17

  Copy full SHA for 4520c17

• [SPARK-54619][PYTHON] Add a sanity check for configuration numbers

  Show description for 0701855
  

  gaogaotiantian

  authored and

  dongjoon-hyun

  committed
  0701855

  Copy full SHA for 0701855

• [SPARK-54650][PYTHON][FOLLOW-UP] Apply `pandas.api.types.is_integer_dtype` to check the series dtype

  Show description for c09bdb9
  

  zhengruifeng

  authored and

  dongjoon-hyun

  committed
  c09bdb9

  Copy full SHA for c09bdb9

Commits on Dec 9, 2025

• [SPARK-54632][PYTHON] Add the option to use ruff for lint

  Show description for e09f2a3
  

  gaogaotiantian

  authored and

  HyukjinKwon

  committed
  e09f2a3

  Copy full SHA for e09f2a3

• Revert "[SPARK-54450][INFRA] Allow run-tests to accept unittest style string for test names"

  Show description for c9165e6
  HyukjinKwon

  committed
  c9165e6

  Copy full SHA for c9165e6

• Revert "[SPARK-54450][FOLLOWUP] Relax the restriction of run-tests"

  Show description for 87f6ff8
  HyukjinKwon

  committed
  87f6ff8

  Copy full SHA for 87f6ff8

• [SPARK-54644][PYTHON][SS][TESTS] Always stop the query before await it

  Show description for 01662e3
  

  gaogaotiantian

  authored and

  HyukjinKwon

  committed
  01662e3

  Copy full SHA for 01662e3

• [SPARK-54441][INFRA][FOLLOW-UP] Enforce new forked process to create its own coverage object

  Show description for 0d9d4ad
  

  gaogaotiantian

  authored and

  HyukjinKwon

  committed
  0d9d4ad

  Copy full SHA for 0d9d4ad

• [MINOR][TESTS] Remove some unnecessary imports in tests

  Show description for 52ec186
  

  gaogaotiantian

  authored and

  HyukjinKwon

  committed
  52ec186

  Copy full SHA for 52ec186

• [SPARK-54643][SDP][PYTHON][TESTS] Run missing pyspark pipelines tests in CI

  Show description for 0824934
  

  sryza

  authored and

  dongjoon-hyun

  committed
  0824934

  Copy full SHA for 0824934

• [SPARK-54650][PYTHON] Move int to decimal conversion into `_create_converter_from_pandas`

  Show description for a189a45
  zhengruifeng

  committed
  a189a45

  Copy full SHA for a189a45

• [SPARK-54617][PYTHON][SQL] Enable Arrow Grouped Iter Aggregate UDF registration for SQL

  Show description for 08c0783
  

  Yicong-Huang

  authored and

  zhengruifeng

  committed
  08c0783

  Copy full SHA for 08c0783

• [SPARK-54438][PYTHON] Consolidate ArrowStreamAggArrowIterUDFSerializer into ArrowStreamAggArrowUDFSerializer

  Show description for 3dffd12
  

  Yicong-Huang

  authored and

  zhengruifeng

  committed
  3dffd12

  Copy full SHA for 3dffd12

• [SPARK-54441][INFRA][FOLLOWUP] Save coverage even when an exception is raised

  Show description for 7631065
  

  gaogaotiantian

  authored and

  zhengruifeng

  committed
  7631065

  Copy full SHA for 7631065