Paimon 1.0 official release: this marks a milestone version of Paimon, a production-ready official release.
•
Alibaba Group's OpenLake project with StarRocks as the primary Lakehouse engine: many Paimon-specific features have been stably validated in internal versions and will be fully contributed to the community.
•
By 2025, we will achieve full feature and performance parity with Apache Iceberg, empowering open-source users to unlock the full potential of Streaming Lakehouse.
DLA Query Optimization
•
Metadata super optimization:
◦
Paimon metadata distributed plan
◦
Paimon manifest cache & background refresh
◦
Enhanced statistics collection
◦
End-to-end observability for metadata
•
Advanced query optimization
◦
Index optimization (e.g., Bitmap file index support)
◦
Improve complex predicate pushdown capabilities
◦
Optimizations for specific query patterns (e.g., count(*), select limit, topn)
◦
Point lookup optimization: Paimon supports record-level optimization
◦
Support primary key (PK)/foreign key (FK) constraints in Paimon tables
•
Asynchronous scan range distribution: support for All-At-Once and Phased scheduler
Fully Featured LakeHouse (Achieve parity with Iceberg/Polaris)
•
Performant and cost-effective writes: collaborate with compute engines to build end-to-end data processing pipelines purely based on the data lake
•
Full operation support: time-travel, management for tagging & branching, enable DDL, DML, procedures, and seamless table migration