Skip to content

Expand ORC writer capabilities #84

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
mrxiad wants to merge 2 commits into
base: main
Choose a base branch
from
Open

Expand ORC writer capabilities #84

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
206f628
feat: add writer compression and row indexes
May 6, 2026
b35f806
feat: expand ORC writer capabilities
May 6, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 7 additions & 7 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,16 +12,18 @@ See the [documentation](https://docs.rs/orc-rust/latest/orc_rust/) for examples

## Supported features

This crate currently only supports reading ORC files into Arrow arrays. Write support is planned
(see [Roadmap](#roadmap)). The below features listed relate only to reading ORC files.
At this time, we aim to support the [ORCv1](https://orc.apache.org/specification/ORCv1/) specification only.
This crate supports reading ORC files into Arrow arrays and writing flat Arrow
`RecordBatch`es to ORC files. At this time, we aim to support the
[ORCv1](https://orc.apache.org/specification/ORCv1/) specification only.

- Read synchronously & asynchronously (using Tokio)
- All compression types (Zlib, Snappy, Lzo, Lz4, Zstd)
- All ORC data types
- All encodings
- Rudimentary support for retrieving statistics
- Retrieving user metadata into Arrow schema metadata
- Write Arrow arrays synchronously, with an async writer API for async sinks
- Writer compression, row group indexes, bloom filters, and column statistics

## Roadmap

Expand All @@ -32,9 +34,8 @@ The following lists the rough roadmap for features to be implemented, from highe

- Performance enhancements
- Predicate pushdown
- Row indices
- Bloom filters
- Write from Arrow arrays
- Complete row index seek positions
- Nested type write support
- Encryption

A non-Arrow API interface is not planned at the moment. Feel free to raise an issue if there is such
Expand Down Expand Up @@ -120,4 +121,3 @@ To regenerate/update the [proto.rs](src/proto.rs) file, execute the [regen.sh](r
```shell
./regen.sh
```

Loading
Loading