AP

Data Version Control Tools

Apache Paimon Software Review

paimon.apache.org · checked 4 Oct 2026

Lake format for Flink and Spark workloads that mix streaming with batch processing.

How it plugs in

—Public API
NoOpen source
YesFree tier

Signal on Data Version Control Tools6.6#5 of 36

Apache Paimon is a table format for realtime lakehouse systems. It supports streaming and batch workloads, primary-key updates, append processing, transactions, time travel, and schema changes. Paimon runs through engines such as Flink and Spark, offers a Python client, and stores data on several filesystem services you provide.

Apache Paimon home page

Review

In a line

Apache Paimon provides a lake format for realtime lakehouse architectures. Flink and Spark can run both its streaming and batch operations.

What it does

Primary-key tables support substantial update workloads with configurable merge engines such as deduplication, partial updates, aggregation, and first-row updates. Streaming updates use an LSM structure, merge engines, and changelog producers.

Append tables cover batch and streaming workloads. They can merge small files automatically, compact data, and sort with Z-ordering. Transactions follow ACID semantics, while time travel, schema evolution, and metadata support very large datasets with many partitions.

The project lists blob, vector, and full-text storage, plus global indexing. Documented CDC pipelines cover MySQL, PostgreSQL, Kafka, MongoDB, and Pulsar.

How it connects

Paimon is open source and available through an API on Linux, macOS, Windows, and self-hosted setups. Its engine ecosystem includes Flink, Spark, Hive, Trino, Presto, StarRocks, and Doris, with supported reads, writes, and table operations varying by engine and version.

Filesystem documentation covers local files, HDFS, Aliyun OSS, S3, Tencent COS, Azure Storage, Huawei OBS, and Google Cloud Storage. Each engine needs a compatible connector or built-in integration plus catalog and warehouse access; distributed clusters must share storage.

PyPaimon includes catalog, table, Arrow, pandas, and command-line interfaces. Core table reads and writes run without a JVM or an active Flink or Spark cluster. Paimon can publish Iceberg metadata for Iceberg connectors to read existing files, while Paimon retains writing and maintenance. See Apache Iceberg for that access route.

Good fit

Paimon works for teams creating lakehouse tables that combine streaming changes with batch jobs. It fits workloads needing primary-key updates, append processing, time travel, schema evolution, or point-in-time rollback.

Python-based analytics and AI or multimodal work can use PyPaimon with Ray, PyTorch, and Pandas. The deployment bundle includes engine JARs, filesystem plugins, and a Java API for embedding Paimon in a Java application.

Watch for

Review the compatibility matrix before selecting an engine: supported versions and read, write, and table features differ across Flink, Spark, Hive, Trino, Presto, StarRocks, and Doris.

Engine integrations depend on catalog and warehouse access, while distributed processes must be able to use shared storage. Security and authorization are generally handled by the surrounding catalog, engine, services, operators, and storage settings.

Paimon offers table-level snapshots, dataset branching, and point-in-time rollback, but the facts do not describe a managed hosting option. Support is through the user mailing list and GitHub issue tracker.

Pricing

Apache Paimon is free. It is an open-source data lake platform, with connectors and storage plugins selected for your environment.

There is no paid tier listed.

Verdict

Choose Paimon if you need an open table format for continuously changing data and batch processing, and you can manage the engines, catalog, and storage around it. Its update mechanisms, CDC routes, historical reads, and schema changes support a broad table workflow.

Avoid committing before checking engine versions and operation support. Teams wanting a hosted service, rather than a self-managed format with bring-your-own storage, should look for a product whose facts explicitly include managed operations.

Our take

Apache Paimon is a practical choice for organizations assembling their own lakehouse stack around Flink, Spark, or another listed engine. It combines streaming updates, batch tables, CDC ingestion, transactions, schema changes, and historical reads in a free plan. The tradeoff is operational work: you supply storage and catalogs, select compatible connectors, and handle surrounding security controls. PyPaimon helps Python users work without a running cluster for core table access. Review the compatibility matrix and deployment needs before adopting it.

  • Free plan
  • Supports streaming and batch table processing
  • Primary-key merge options include deduplication and partial updates
  • Includes time travel, schema evolution, and ACID transactions
  • PyPaimon includes Python, Arrow, pandas, and CLI APIs
  • Engine integrations need catalog and warehouse access
  • Engine features and versions vary by integration
  • Distributed processing requires storage shared among participating processes
  • Security boundaries depend on surrounding services and configuration
  • Pick it ifTeams building self-managed realtime lakehouse tables; Streaming and batch workloads using supported engines; Python, AI, or multimodal table workflows
  • Skip it ifYou need a hosted service with storage and operations included; Your chosen engine or version is outside the compatibility matrix; You cannot provide shared storage for distributed processing

Pricing

Apache Paimon has a free plan. You choose the storage, catalog, connectors, and deployment components needed for your setup.

  • Apache PaimonOpen-source data lake platform; choose connectors and storage plugins for your environmentFree

As paimon.apache.org lists them, checked 4 Oct 2026.

Questions

Is Apache Paimon free?

Yes. Apache Paimon is listed with a free plan.

Which engines work with Paimon?

Flink, Spark, Hive, Trino, Presto, StarRocks, and Doris are listed integrations. Supported operations and engine versions vary.

Can Paimon ingest change data?

The project documents CDC pipelines for MySQL, PostgreSQL, Kafka, MongoDB, and Pulsar.

What storage can Paimon use?

Paimon documents local files, HDFS, Aliyun OSS, S3, Tencent COS, Azure Storage, Huawei OBS, and Google Cloud Storage.

Does Paimon have a Python client?

Yes. PyPaimon includes catalog, table, Arrow, pandas, and command-line APIs. Core reads and writes work without a JVM or an active Flink or Spark cluster.

Can other tools read Paimon data?

Paimon can publish Iceberg metadata, allowing applications to use Iceberg connectors for existing files. Paimon remains responsible for writing and maintenance.

Does Paimon support history and rollback?

Yes. It supports time travel, table-level snapshots, dataset branching, and point-in-time rollback.