The Apache Arrow team is pleased to announce the version 23 release of the Apache Arrow ADBC libraries. This release includes 41 resolved issues from 20 distinct contributors.

This is a release of the libraries, which are at version 23. The API specification is versioned separately and is at version 1.1.0.

The subcomponents are versioned independently:

• C/C++/GLib/Go/Python/Ruby: 1.11.0
• C#: 0.23.0
• Java: 0.23.0
• R: 0.23.0
• Rust: 0.23.0

The release notes below are not exhaustive and only expose selected highlights of the release. Many other bugfixes and improvements have been made: we refer you to the complete changelog.

Release Highlights

A breaking change has been made to the Rust APIs (pre-1.0): returned RecordBatchReaders are now type-erased and boxed for caller flexibility; this also fixes the returned reader lifetime accidentally being tied to input argument lifetimes (#3904).

A driver manager for Node.js is now available from NPM (#4046, #4091, #4116, #4125, etc.).

The C++ and Rust driver managers now support connection profiles (#3876, #3973, #4080, #4083 etc.). (Note that other bindings that use the C++ driver manager, including GLib/Ruby, Go, Java, Python, R, and so on, inherit this support.)

The Go APIs have added interfaces that always take a context.Context for consistency, and to make sure context like telemetry traces propagate properly (#4009).

The Python driver manager has added specific parameters for using connection profiles as well (#4078, #4118). Also, non-string option values are directly accepted for convenience (#4088). adbc_get_statistics has been added (#4129).

The JNI bindings (allowing use of C/C++/Go/Rust/etc. drivers from Java) now support more functions (GetObjects, GetInfo, ExecuteSchema, etc.) (#3966, #3972, #4056).

Packages are now being uploaded to Homebrew (#4131).

Python wheels now require manylinux_2_28, up from manylinux2010, following PyArrow (#4146). On macOS, macOS 12 is now the minimum version due to upgrading to Go 1.25+ (including on conda-forge, where the packages previously pinned Go 1.24 to avoid this).

The PostgreSQL driver tries to reconcile Arrow NA arrays with PostgreSQL types when binding (#4098). Also, a bug in conversion from Arrow decimals to PostgreSQL numerics has been fixed (#3787).

The SQLite driver now enables various optional features, like math functions (#4147).

Contributors

$ git shortlog --perl-regexp --author='^((?!dependabot\[bot\]).*)$' -sn apache-arrow-adbc-22..apache-arrow-adbc-23
    35	David Li
    12	Kent Wu
    10	Matt Topol
     8	eitsupi
     6	Bryce Mecum
     5	Bruce Irschick
     4	Mandukhai Alimaa
     3	Emil Sadek
     3	Tornike Gurgenidze
     2	Dewey Dunnington
     2	Felipe Oliveira Carvalho
     2	eric-wang-1990
     1	Curt Hagenlocher
     1	Ian Cook
     1	Madhavendra Rathore
     1	Mila Page
     1	Pavel Agafonov
     1	Roshan Banisetti
     1	davidhcoe
     1	oglego

Roadmap

We are working on the next revision of the API standard, focusing on missing features (primarily metadata/catalog data). We welcome anyone interested in contributing. Current progress can be found in the 1.2.0 specification milestone.

Getting Involved

We welcome questions and contributions from all interested. Issues can be filed on GitHub, and questions can be directed to GitHub or the Arrow mailing lists.

As you may have read in a previous blog post ¹, the Apache Arrow project recently turned 10 years old. We are grateful to everyone who helped us achieve this milestone, and we wanted to celebrate the community's accomplishments, by publishing our community highlights from 2025.

We were inspired by the research by Dr Cat Hicks et al ², who found that concrete evidence of progress and accomplishments is instrumental to motivation and collaboration in developer teams. We think the same should hold for open source.

New contributors

It has been great to see many new contributors joining the project in the past year, with over 300 such individuals observed across the main Apache Arrow language implementations.

Worth highlighting is alinaliBQ who has been very active on the C++ Flight SQL ODBC Driver work together with justing-bq.

AntoinePrv has done a huge amount of work on the C++ Parquet implementation and andishgar in the C++ Statistics area.

rmnskb got involved with PyArrow in EuroPython sprints and has contributed multiple PRs since then. On the same event paddyroddy also started with his first contribution and helped on the Python packaging side further on.

sdf-jkl, liamzwbao, friendlymatthew, and klion26 helped drive early Variant functionality in the Rust Parquet implementation and contributed a number of follow-up improvements.

jecsand838 drove major improvements to the Rust arrow-avro crate, work highlighted in the Introducing Arrow Avro blog post.

Release, Packaging and CI

A lot of work has been done around the Continuous Integration and Developer Tools area. Ensuring a project with the reach of Arrow is properly working requires validation on a huge matrix of operating systems, architectures, libraries, versions. Needless to say that maintenance work has tremendous importance for the health of the project and the positive contributor experience.

The most active contributors in the main repository are the ones contributing heavily on those areas while also providing the most review capacity. Shout out to kou and raulcd for taking such good care of the project and devoting countless hours so that everything runs smoothly.

Notable contributions worth mentioning are enhanced release automation and reproducible builds for sources, migrating remaining AppVeyor and Azure jobs to GitHub actions, improving dev experience with more pre-commit checks instead of custom made linting tools.

Moving some implementations out of the main repository (apache/arrow on GitHub) helped with easier releases and maintenance of the main repository and also of separate language implementations. The current apache/arrow repo now holds the format specification, C++ implementation together with all the bindings to it (Python, R, Ruby and C GLib). Other languages now live in their own apache/ repos namely apache/arrow-java, apache/arrow-js, apache/arrow-rs, apache/arrow-go, apache/arrow-nanoarrow, apache/arrow-dotnet and apache/arrow-swift.

Migration of infrastructure from Voltron Data

As Voltron Data has wound down its operations in 2025, the Arrow project had to migrate benchmarking infrastructure and nightly report from Voltron-managed services to an Arrow-managed AWS account. This work has been driven by rok.

Closing of Stale issues

thisisnic was working on closing of stale issues in the apache/arrow repository which helped surfacing important issues that were overlooked or forgotten.

Code contributions

C++ implementation

Community support for maintenance and development of the Acero C++ is continuing with multiple bigger contributions in 2025 done by pitrou and zanmato1984.

Many kernels have been moved from the integrated compute module into a separate, optional package for improvement of modularity and distribution size when optional compute functionality is not being used. The work has been done by raulcd.

Arrow C++ Parquet implementation

There have been multiple contributions to fix and improve fuzzing support for Parquet. Fuzzing work is led by pitrou who is also one of the most active members of the community guiding other developers and supporting us with abundant review capacity.

Multiple newer types have also been supported in the last year, namely: VARIANT, UUID, GEOMETRY and GEOGRAPHY contributed by neilechao and paleolimbot.

An important feature added has also been Content-Defined Chunking which improves deduplication of Parquet files with mostly identical contents, by choosing data page boundaries based on actual contents rather than a number of values ³. This work has been done by kszucs.

There have been improvements in the Parquet encryption support for most of the releases in the last year. These efforts have been driven mostly by EnricoMi, pitrou, adamreeve and kapoisu.

PyArrow

A lot of work has been put into adding type annotations. It all started in July at EuroPython sprints and the code is now ready to be reviewed and merged. Some more review capacity will be needed to get this over the finish line. The work has been championed by rok.

Rust

Arrow Rust community invested heavily in the Rust parquet reader for which they created several blog posts ⁴, ⁵. The work has been championed by alamb and etseidl.

Java

The biggest changes in apache/arrow-java for 2025 have been connected to Flight and Avro components plus Sphinx support due to the Java implementation being moved into a separate Apache repository. Contributors involved in the above are lidavidm and martin-traverse.

Go

There has been a lot of work related to new variant type in the Parquet implementation done in apache/arrow-go all by zeroshade.

Noticeable emphasis was also visible on performance-focused PRs leading to the addition of row seeking, bloom filter reading/writing, and reduction of allocations in the Parquet library along with significant optimization work in the compute.Take kernels. Shout out to pixelherodev and hamilton-earthscope for the emphasis they placed on improving performance.

Nanoarrow

Bigger work in nanoarrow include Decimal32/64 and ListView/LargeListView support, LZ4 and ZSTD decompression in the IPC reader, and broader packaging via Conan, Homebrew, and vcpkg. Contributors driving most above are paleolimbot and WillAyd.

Arrow Summit 25

One last thing to highlight would be our first Arrow Summit 25 that was held in Paris in October 2025. The event was a great success and it brought users, contributors and maintainers together. It definitely was a highlight of the year for many of us. Thanks to raulcd and pitrou for organizing the event.

Thank you!

We would like to thank every single contributor to Apache Arrow for being a part of this great community and project! Hope this blog post helps to validate all the work you have done and motivates us to continue collaborating and growing together!

The Notebooks with the analysis for this blog post can be found in ⁶.

Note not all language implementations are mentioned. Some due to being moved into a separate repository in 2025 resulting in missing information for large amount of merged pull requests. Others due to having lower number of bigger contributions in the past year.

The Apache Arrow team is pleased to announce the v19.0.0 release of Apache Arrow Java.

Changelog

What's Changed

Breaking Changes

• GH-774: Consoliate BitVectorHelper.getValidityBufferSize and BaseValueVector.getValidityBufferSizeFromCount by @rtadepalli in #775
• GH-586: Override fixedSizeBinary method for UnionMapWriter by @axreldable in #885
• GH-891: Add ExtensionTypeWriterFactory to TransferPair by @jhrotko in #892
• GH-948: Use buffer indexing for UUID vector by @jhrotko in #949
• GH-139: [Flight] Stop return null from MetadataAdapter.getAll(String) and getAllByte(String) by @axreldable in #1016

New Features and Enhancements

• GH-52: Make RangeEqualsVisitor of RunEndEncodedVector more efficient by @ViggoC in #761
• GH-765: Do not close/free imported BaseStruct objects by @pepijnve in #766
• GH-79: Move splitAndTransferValidityBuffer to BaseValueVector by @rtadepalli in #777
• GH-731: Avro adapter, output dictionary-encoded fields as enums by @martin-traverse in #779
• GH-725: Added ExtensionReader by @xxlaykxx in #726
• GH-882: Add support for loading native library from a user specified location by @pepijnve in #883
• GH-109: Implement Vector Validators for StringView by @ViggoC in #886
• GH-900: Fix gandiva groupId in arrow-bom by @XN137 in #901
• GH-762: Implement VectorAppender for RunEndEncodedVector by @ViggoC in #884
• GH-825: Add UUID canonical extension type by @jhrotko in #903
• GH-110: Flight SQL JDBC related StringView components implementation by @ViggoC in #905
• GH-863: [JDBC] Suppress benign exceptions from gRPC layer on ArrowFlightSqlClientHandler#close by @ennuite in #910
• GH-929: Add UUID support in JDBC driver by @xborder in #930
• GH-952: Add OAuth support by @xborder in #953
• GH-946: Add Variant extension type support by @tmater in #947
• GH-130: Fix AutoCloseables to work with @nullable structures by @axreldable in #1017
• GH-1038: Trim object memory for ArrowBuf by @lriggs in #1044
• GH-1061: Add codegen classifier jar for arrow-vector. by @lriggs in #1062
• GH-301: [Vector] Allow adding a vector at the end of VectorSchemaRoot by @axreldable in #1013
• GH-552: [Vector] Add absent methods to the UnionFixedSizeListWriter by @axreldable in #1052

Full Changelog: changelog

The Apache Arrow team is pleased to announce the v18.5.2 release of Apache Arrow Go. This patch release covers 16 commits from 6 distinct contributors.

Contributors

$ git shortlog -sn v18.5.1..v18.5.2
    11	Matt Topol
     2	daniel-adam-tfs
     1	Evan Todd
     1	Rusty Conover
     1	Stas Spiridonov
     1	William

Changelog

What's Changed

• chore: bump parquet-testing submodule by @zeroshade in #633
• fix(arrow/array): handle empty binary values correctly in BinaryBuilder by @zeroshade in #634
• test(arrow/array): add test to binary builder by @zeroshade in #636
• fix(parquet): decryption of V2 data pages by @daniel-adam-tfs in #596
• perf(arrow): Reduce the amount of allocated objects by @spiridonov in #645
• fix(parquet/file): regression with decompressing data by @zeroshade in #652
• fix(arrow/compute): take on record/array with nested struct by @zeroshade in #653
• fix(parquet/file): write large string values by @zeroshade in #655
• ci: ensure extra GC cycle for flaky tests by @zeroshade in #661
• fix(arrow/array): handle exponent notation for unmarshal int by @zeroshade in #662
• fix(flight/flightsql/driver): fix time.Time params by @etodd in #666
• fix(parquet): bss encoding and tests on big endian systems by @daniel-adam-tfs in #663
• fix(parquet/pqarrow): selective column reading of complex map column by @zeroshade in #668
• feat(arrow/ipc): support custom_metadata on RecordBatch messages by @rustyconover in #669
• feat: Support setting IPC options in FlightSQL call options by @peasee in #674
• chore(dev/release): embed hash of source tarball into email by @zeroshade in #675
• chore(arrow): bump PkgVersion to 18.5.2 by @zeroshade in #676

New Contributors

• @spiridonov made their first contribution in #645
• @etodd made their first contribution in #666
• @rustyconover made their first contribution in #669
• @peasee made their first contribution in #674

Full Changelog: https://github.com/apache/arrow-go/compare/v18.5.1...v18.5.2

The Apache Arrow team is pleased to announce the 0.8.0 release of Apache Arrow nanoarrow. This release consists of 28 resolved GitHub issues from 10 contributors.

Release Highlights

• Support for building String View arrays by buffer
• LZ4 decompression support in IPC reader
• Support for Conan
• Support for Hombrew

See the Changelog for a detailed list of contributions to this release.

Features

String Views By Buffer

The C library in general supports two methods for producing or consuming arrays: most users use the builder pattern (e.g., ArrowArrayAppendString()); however, the "build by buffer" pattern can be effective when using nanoarrow with a higher level runtime like C++, Rust, Python, or R, all of which have mechanisms to build buffers already. The C library supports this with ArrowArraySetBuffer(); however, there was no way to reserve and/or set variadic buffers for string view arrays. In nanoarrow 0.8.0, the array builder API fully supports both mechanisms for building string view arrays.

LZ4 Decompression Support

The Arrow IPC reader included in the nanoarrow C library supports most features of the Arrow IPC format; however, decompression support for the LZ4 codec was missing which made the library and its bindings unusable for some common use cases. In 0.8.0, decompression for the LZ4 codec was added to the C library.

Users of the C library will need to configure CMake with -DNANOARROW_IPC_WITH_LZ4=ON and -DNANOARROW_IPC=ON to use CMake-resolved LZ4; however, client libraries can also use an existing ZSTD or LZ4 implementation using callbacks just like in 0.7.0.

nanoarrow on Conan

The nanoarrow C library can now be installed using the Conan C/C++ Package Manager! CMake projects can now use find_package(nanoarrow) when using a Conan-enabled toolchain after adding the nanoarrow dependency to conanfile.txt.

Thanks to @wgtmac for contributing the recipe!

nanoarrow on Homebrew

The nanoarrow C library can now be installed using Homebrew!

brew install nanoarrow

CMake projects can then use find_package(nanoarrow) when using Homebrew-provided cmake and allows other vcpkg ports to use nanoarrow as a dependency.

Thanks to @ankane for contributing the formula!

Contributors

This release consists of contributions from 12 contributors in addition to the invaluable advice and support of the Apache Arrow community.

$ git shortlog -sn apache-arrow-nanoarrow-0.8.0.dev..apache-arrow-nanoarrow-0.8.0-rc0
    23  Dewey Dunnington
     2  Bryce Mecum
     2  Dirk Eddelbuettel
     1  Even Rouault
     1  Kevin Liu
     1  Michael Chirico
     1  Namit Kewat
     1  Nyall Dawson
     1  Sutou Kouhei
     1  William Ayd

The Apache Arrow team is pleased to announce the 23.0.1 release. It includes a security fix for the C++ IPC file reader, so be sure to read the relevant details below to see if you are affected.

Apart from that, 23.0.1 is mostly a bugfix release that includes 28 resolved issues on 29 distinct commits from 12 distinct contributors.

See the Install Page to learn how to get the libraries for your platform.

The release notes below are not exhaustive and only expose selected highlights of the release. Many other bugfixes and improvements have been made: we refer you to the complete changelog.

C++ notes

• Fix possible OOB write in buffered IO (GH-48311).

IPC

CVE-2026-25087: Use After Free vulnerability in IPC file reader

Fix a security issue can be triggered when reading an Arrow IPC file (but not an IPC stream) with pre-buffering enabled, if the IPC file contains data with variadic buffers (such as Binary View and String View data).

Pre-buffering is disabled by default, so your code is vulnerable only if it enables it explicitly by calling RecordBatchFileReader::PreBufferMetadata. Affected Arrow C++ versions are 15.0.0 through 23.0.0. The fix integrated in 23.0.1 can also be separately viewed at GH-48925.

See our separate announcement for further detail.

Other fixes

• Avoid memory blowup with excessive variadic buffer count in IPC (GH-48900).

Gandiva

• Fix passing CPU attributes to LLVM (GH-48160).
• Detect overflow in repeat() (GH-49159).

Parquet

• Avoid re-serializing footer for signature verification (GH-48858).

Python notes

• Added missing NOTICE.txt and LICENSE.txt to wheels (GH-48983).
• Some fixes for compatibility with newer Cython versions like (GH-48965), (GH-49156) and (GH-49138).

Ruby notes

• Fix a bug where Arrow::ExecutePlan nodes may be Garbage Collected (GH-48880).

R notes

• Bump C++20 for R build infrastructure (GH-48817) and fix some C++ 20 related compilation issues (GH-48973).

Other modules and languages

No general changes were made to the other libraries or languages.

The Apache Arrow project was officially established and had its first git commit on February 5th 2016, and we are therefore enthusiastic to announce its 10-year anniversary!

Looking back over these 10 years, the project has developed in many unforeseen ways and we believe to have delivered on our objective of providing agnostic, efficient, durable standards for the exchange of columnar data.

How it started

From the start, Arrow has been a joint effort between practitioners of various horizons looking to build common grounds to efficiently exchange columnar data between different libraries and systems. In this blog post, Julien Le Dem recalls how some of the founders of the Apache Parquet project participated in the early days of the Arrow design phase. The idea of Arrow as an in-memory format was meant to address the other half of the interoperability problem, the natural complement to Parquet as a persistent storage format.

Apache Arrow 0.1.0

The first Arrow release, numbered 0.1.0, was tagged on October 7th 2016. It already featured the main data types that are still the bread-and-butter of most Arrow datasets, as evidenced in this Flatbuffers declaration:

/// ----------------------------------------------------------------------
/// Top-level Type value, enabling extensible type-specific metadata. We can
/// add new logical types to Type without breaking backwards compatibility

union Type {
  Null,
  Int,
  FloatingPoint,
  Binary,
  Utf8,
  Bool,
  Decimal,
  Date,
  Time,
  Timestamp,
  Interval,
  List,
  Struct_,
  Union
}

The release announcement made the bold claim that "the metadata and physical data representation should be fairly stable as we have spent time finalizing the details". Does that promise hold? The short answer is: yes, almost! But let us analyse that in a bit more detail:

• the Columnar format, for the most part, has only seen additions of new datatypes since 2016. One single breaking change occurred: Union types cannot have a top-level validity bitmap anymore.

• the IPC format has seen several minor evolutions of its framing and metadata format; these evolutions are encoded in the MetadataVersion field which ensures that new readers can read data produced by old writers. The single breaking change is related to the same Union validity change mentioned above.

First cross-language integration tests

Arrow 0.1.0 had two implementations: C++ and Java, with bindings of the former to Python. There were also no integration tests to speak of, that is, no automated assessment that the two implementations were in sync (what could go wrong?).

Integration tests had to wait for November 2016 to be designed, and the first automated CI run probably occurred in December of the same year. Its results cannot be fetched anymore, so we can only assume the tests passed successfully. 🙂

From that moment, integration tests have grown to follow additions to the Arrow format, while ensuring that older data can still be read successfully. For example, the integration tests that are routinely checked against multiple implementations of Arrow have data files generated in 2019 by Arrow 0.14.1.

No breaking changes... almost

As mentioned above, at some point the Union type lost its top-level validity bitmap, breaking compatibility for the workloads that made use of this feature.

This change was proposed back in June 2020 and enacted shortly thereafter. It elicited no controversy and doesn't seem to have caused any significant discontent among users, signaling that the feature was probably not widely used (if at all).

Since then, there has been precisely zero breaking change in the Arrow Columnar and IPC formats.

Apache Arrow 1.0.0

We have been extremely cautious with version numbering and waited until July 2020 before finally switching away from 0.x version numbers. This was signalling to the world that Arrow had reached its "adult phase" of making formal compatibility promises, and that the Arrow formats were ready for wide consumption amongst the data ecosystem.

Apache Arrow, today

Describing the breadth of the Arrow ecosystem today would take a full-fledged article of its own, or perhaps even multiple Wikipedia pages. Our "powered by" page can give a small taste.

As for the Arrow project, we will merely refer you to our official documentation:

1. The various specifications that cater to multiple aspects of sharing Arrow data, such as in-process zero-copy sharing between producers and consumers that know nothing about each other, or executing database queries that efficiently return their results in the Arrow format.

2. The implementation status page that lists the implementations developed officially under the Apache Arrow umbrella (native software libraries for C, C++, C#, Go, Java, JavaScript, Julia, MATLAB, Python, R, Ruby, and Rust). But keep in mind that multiple third-party implementations exist in non-Apache projects, either open source or proprietary.

However, that is only a small part of the landscape. The Arrow project hosts several official subprojects, such as ADBC and nanoarrow. A notable success story is Apache DataFusion, which began as an Arrow subproject and later graduated to become an independent top-level project in the Apache Software Foundation, reflecting the maturity and impact of the technology.

Beyond these subprojects, many third-party efforts have adopted the Arrow formats for efficient interoperability. GeoArrow is an impressive example of how building on top of existing Arrow formats and implementations can enable groundbreaking efficiency improvements in a very non-trivial problem space.

It should also be noted that Arrow, as an in-memory columnar format, is often used hand in hand with Parquet for persistent storage; as a matter of fact, most official Parquet implementations are nowadays being developed within Arrow repositories (C++, Rust, Go).

Tomorrow

The Apache Arrow community is primarily driven by consensus, and the project does not have a formal roadmap. We will continue to welcome everyone who wishes to participate constructively. While the specifications are stable, they still welcome additions to cater for new use cases, as they have done in the past.

The Arrow implementations are actively maintained, gaining new features, bug fixes, and performance improvements. We encourage people to contribute to their implementation of choice, and to engage with us and the community.

Now and going forward, a large amount of Arrow-related progress is happening in the broader ecosystem of third-party tools and libraries. It is no longer possible for us to keep track of all the work being done in those areas, but we are proud to see that they are building on the same stable foundations that have been laid 10 years ago.

We are thrilled to announce the official publication of a Security Model for Apache Arrow.

The Arrow security model covers a core subset of the Arrow specifications: the Arrow Columnar Format, the Arrow C Data Interface and the Arrow IPC Format. It sets expectations and gives guidelines for handling data coming from untrusted sources.

The specifications covered by the Arrow security model are building blocks for all the other Arrow specifications, such as Flight and ADBC.

The ideas underlying the Arrow security model were informally shared between Arrow maintainers and have informed decisions for years, but they were left undocumented until now.

Implementation-specific security considerations, such as proper API usage and runtime safety guarantees, will later be covered in the documentation of the respective implementations.

The Apache Arrow team is pleased to announce the v18.5.1 release of Apache Arrow Go. This patch release covers 10 commits from 6 distinct contributors.

Contributors

$ git shortlog -sn v18.5.0..v18.5.1
     6	Matt Topol
     1	Alfonso Subiotto Marqués
     1	Arnold Wakim
     1	Bryce Mecum
     1	Rok Mihevc
     1	cai.zhang

Changelog

What's Changed

• fix(internal): fix assertion on undefined behavior by @amoeba in #602
• ci(benchmark): switch to new conbench instance by @rok in #593
• fix(flight): make StreamChunksFromReader ctx aware and cancellation-safe by @arnoldwakim in #615
• fix(parquet/variant): fix basic stringify by @zeroshade in #624
• fix(parquet/pqarrow): fix partial struct panic by @zeroshade in #630
• Flaky test fixes by @zeroshade in #629
• ipc: clear variadicCounts in recordEncoder.reset() by @asubiotto in #631
• fix(arrow/cdata): Handle errors to prevent panic by @xiaocai2333 in #614

New Contributors

• @rok made their first contribution in #593
• @asubiotto made their first contribution in #631
• @xiaocai2333 made their first contribution in #614

Full Changelog: https://github.com/apache/arrow-go/compare/v18.5.0...v18.5.1

The Apache Arrow team is pleased to announce the 23.0.0 release. This release covers over 3 months of development work and includes 336 resolved issues on 417 distinct commits from 71 distinct contributors. See the Install Page to learn how to get the libraries for your platform.

The release notes below are not exhaustive and only expose selected highlights of the release. Many other bugfixes and improvements have been made: we refer you to the complete changelog.

Community

As per our newly started tradition of rotating the PMC chair once a year Antoine Pitrou was elected as the new PMC chair and VP, succeeding Neal Richardson.

Thanks for your contributions and participation in the project!

Arrow Flight RPC Notes

An ODBC driver for Apache Arrow Flight SQL has been completed. Currently it is not packaged for release, but can be built from source.

C++ Notes

The C++ standard has been updated to C++ 20 GH-45885 and the minimum GCC to 8.

Some improvements to leverage C++ 20 GH-48592,

Compute

• Graceful error handling for decimal binary arithmetic and comparison instead of firing confusing assertions. GH-35957
• Fixed an issue where the MinMax kernel was emitting -inf/inf for all-NaN input. GH-46063
• Avoid ZeroCopyCastExec when casting between Binary offset types to avoid high overheads. GH-43660
• Enhanced type checking for hash join residual filter in Acero. GH-48268

Format

• Clarified that empty compressed buffers can omit the length header. GH-47918

Parquet

• A new setting to limit the number of rows written per page has been added. GH-47030
• A arrow::Result version of parquet::arrow::FileReader::Make() has been added. GH-44810
• Support for reading INT-encoded Decimal statistics as Arrow scalars. GH-47955

Several bug fixes including:

• Fixed invalid Parquet files written when dictionary encoded pages are large. GH-47973
• Fixed pre-1970 INT96 timestamps roundtrip. GH-48246
• Fixed potential crash when reading invalid Parquet data. GH-48308
• Added compatibility with non-compliant RLE streams. GH-47981
• Fixed Util & Level Conversion logic on big-endian systems. GH-48218

Encryption

• Simplified nested field encryption configuration. GH-41246
• Improved column encryption API. GH-48337
• Better fuzzing support for encrypted files. GH-48335

Miscellaneous C++ changes

• Added support for CUDA 13 GH-47677
• Drop support for gold linker GH-45484
• Leverage CMake 3.25 upgrade by reducing complexity and maintenance burden on our third party dependency management GH-48317, GH-48316, GH-48315, GH-48248, GH-48181, GH-48178, GH-48091, GH-48074

Linux Packaging Notes

Fixed a bug that the parquet-devel RPM package depends on parquet-glib-devel.

See also: GH-48044

CentOS 7 support has been dropped.

See also: GH-40735

MATLAB Notes

Added support for building against MATLAB R2025b GH-48154.

Python Notes

Compatibility notes

• Deprecated Array.format is removed GH-48102.
• Experimental tag has been removed for Arrow PyCapsule Interface GH-47975.
• PyWeakref_GetRef has replaced the use of PyWeakref_GET_OBJECT to support Python 3.15 GH-47823.

New features

• Bindings for scatter and inverse_permutationare added GH-48167.
• max_rows_per_page argument is now exposed in parquet.WriterProperties GH-48096.
• External key material and rotation is enabled for individual Parquet files GH-31869.

Other improvements

• Nested field encryption configuration has been simplified GH-41246.
• Reading INT-encoded Decimal statistics with StatisticsAsScalars is now supported GH-47955.
• Unsigned dictionary indices are now supported in pandas conversion GH-47022.
• Added code examples for compute functions min, max and min_max GH-48668.
• Add temporal unit checking in NumPyDtypeUnifier GH-48625
• Error message is improved when mixing numpy.datetime64 values with different units (e.g., datetime64[s] and datetime64[ms]) in a single array GH-48463.
• The source argument is now checked in pyarrow.parquet.read_table GH-47728.

Relevant bug fixes

• ipc.Message __repr__ has been corrected to use f-string GH-48608.
• Failures when reading parquet files written with non-compliant RLE encoders have been fixed in C++ with adding compatibility GH-47981.
• Memory usage is now reduced when using to_pandas() with many extension arrays columns GH-47861.
• Missing required argument error in FSSpecHandler delete_root_dir_contents has been fixed GH-47559.
• Invalid RecordBatch.from_struct_array batch for sliced arrays with offset zero has been fixed in the C++ GH-44318.

R Notes

Compatibility notes

• GCS have been turned off by default GH-48342.
• OpenSSL 1.x builds have been removed GH-45449

Relevant bug fixes

• Fixed a segfault that could be raised when concatenatig tables GH-47000.

Several Continuous integration fixes and minor bugs have also been added to the release for a full list check the release notes.

Ruby and C GLib Notes

All missing compute function options have been added. So we can use all compute functions from Ruby and C GLib. This is done by Sten Larsson.

Fixed size list array support has been added.

See also: GH-48362

Changing thread pool configuration support in Acero has been added. This is done by Sten Larsson.

Duration support has been added.

CSV writer support has been added.

See also: GH-48680

Ruby

Experimental Pure Ruby Apache Arrow reader implementation has been added as red-arrow-format gem.

See also: GH-48132

We'll add experimental writer implementation in the next release.

Arrow::Column#to_arrow{,_array,_chunked_array} have been added. They are for convenient.

See also: GH-48292

Auto Apache Arrow type detection in Arrow::Array.new has been improved for nested integer list case.

See also:

• GH-48478
• GH-48481

Arrow::FixedSizeListArray.new(data_type, values) support has been added.

See also: GH-48610

C GLib

We use Arrow-${MAJOR}.${MINOR}.{gir,typelib} not Arrow-1.0.{gir,typelib} for .gir and .typelib file names. It's for co-existent multiple C GLib versions in the same system.

See also: GH-48616

Java, JavaScript, Go, .NET, Swift and Rust Notes

The Java, JavaScript, Go, .NET, Swift and Rust projects have moved to separate repositories outside the main Arrow monorepo.

• For notes on the latest release of the Java implementation, see the latest Arrow Java changelog.
• For notes on the latest release of the JavaScript implementation, see the latest Arrow JavaScript changelog.
• For notes on the latest release of the Rust implementation see the latest Arrow Rust changelog.
• For notes on the latest release of the Go implementation, see the latest Arrow Go changelog.
• For notes on the latest release of the .NET implementation, see the latest Arrow .NET changelog.
• For notes on the latest release of the Swift implementation, see the latest Arrow Swift changelog.