Delta Lake Integrations

Use the following frameworks, Delta Sharing clients, managed services, and/or community integrations
for Delta Lake and Delta Sharing.

Frameworks

Use the following frameworks and languages including but not limited to Apache Flink, Apache Spark, Trino, and Rust.

Apache Druid

docs | source code
Druid
This connector allows Apache Druid to read from Delta Lake.

Apache Flink

docs | source code
Flinkstandalone
This connector allows Apache Flink to write to Delta Lake.

Apache Hive

docs | source code
Hivestandalone
This connector allows Apache Hive to read from Delta Lake.

Apache Pulsar

docs | source code
Pulsarcommunity
This connector allows Apache Pulsar to read from and write to Delta Lake.

Apache Spark™

docs | source code
Spark
This connector allows Apache Spark™ to read from and write to Delta Lake.

docs | source code
ClickHouse
ClickHouse is a column-oriented database that allows users to run SQL queries on Delta Lake tables. This provides a read-only integration with existing Delta Lake tables in Amazon S3.

Dagster

docs | source code
DagsterPython
Use the Delta Lake IO Manager to read from and write to Delta Lake in your Dagster orchestration pipelines.

Delta Rust API

docs | source code
RustPython
This library allows Rust (with Python bindings) low level access to Delta tables and is intended to be used with data processing frameworks like datafusion, ballista, rust-dataframe, vega, etc.

Delta Standalone

docs | source code
ScalaJavastandalone
This library allows Scala and Java-based projects (including Apache Flink, Apache Hive, Apache Beam, and PrestoDB) to read from and write to Delta Lake.

FINOS Legend

docs | source code
FINOSLegend
An extension to the FINOS Legend framework for Apache Spark™ / Delta Lake based environment, combining best of open data standards with open source technologiesThis connector allows Trino to read from and write to Delta Lake.

Hopsworks

docs | source code
HopsworksPython
Hopsworks Feature Store stores, manages, and serves feature data in Delta Lake.

Kafka Delta Ingest

docs | source code
KafkaRust
This project builds a highly efficient daemon for streaming data through Apache Kafka into Delta Lake.

PrestoDB

docs | source code
PrestoDBstandalone
This connector allows PrestoDB to read from Delta Lake.

SQL Delta Import

docs | source code
SQLJDBC
This utility is for importing data from a JDBC source into a Delta Lake table.

StarRocks

docs | source code
StarRocks
StarRocks, a Linux Foundation project, is a next-generation sub-second MPP OLAP database for full analytics scenarios, including multi-dimensional analytics, real-time analytics, and ad-hoc queries. StarRocks has the ability to read from Delta Lake.

Trino

docs | source code
Trino
This connector allows Trino to read from and write to Delta Lake.

Sharing

Use the following clients that integrate with Delta Sharing from C++ to Rust.

C++

docs | source code
C++Delta Sharingcommunity
This connector allows a C++ client to read from Delta Sharing endpoint.

Excel

docs
ExcelDelta Sharing
This connector allows a Excel client to read from Delta Sharing endpoint.

Go

source code
GoDelta Sharingcommunity
This connector allows a Go client to read from Delta Sharing endpoint.

Java

source code
JavaDelta Sharingcommunity
This connector allows a Java client to read from Delta Sharing endpoint.

Kotosiro Sharing

source code
KotosiroDelta SharingRust
A Minimalistic Rust Implementation of Delta Sharing Server.

MLflow

docs | source code
MLflowDelta Sharingcommunity
Perform model exchange via Delta Sharing and MLflow

node.js

docs | source code
node.jsDelta Sharingcommunity
This connector allows node.js to read from Delta Sharing endpoint.

Oracle

OracleDelta Sharing
This connector allows for Delta Sharing with Oracle Autonomous Database Data Studio.

Power BI

docs
PowerBIDelta Sharing
This connector allows Power BI to read from Delta Sharing endpoint.

R

source code
RDelta Sharingcommunity
This connector allows a R client to read from Delta Sharing endpoint.

Rust

docs | source code
RustDelta Sharingcommunity
This connector allows a Rust client to read from Delta Sharing endpoint.

Terminal

source code
TerminalDelta Sharingcommunity
Terminal application for browsing Delta Sharing Metadata.

Services

Use the managed services of your choice that integrate with Delta Lake.

Athena

docs
AthenaAWS
This utility allows Athena to natively read from Delta Lake starting with Athena SQL 3.0

AWS EMR

docs
AWSEMR
Starting with Amazon EMR release 6.9.0, you can use Apache Spark 3.x on Amazon EMR clusters with Delta Lake tables.

AWS Glue

docs
AWSGlue
AWS Glue 3.0 and later supports the Linux Foundation Delta Lake framework

aws-pandas-sdk

docs | source code
pandasAWSawswrangleraws-pandas-sdk
pandas on AWS - Easy integration with AWS services including optional dependency with Delta Lake

Azure Stream Analytics

docs
AzureASAStream Analytics
Azure Stream Analytics provides native write support for Delta Lake

BigQuery

docs
BigQuery’s native Delta Lake support enables seamless delivery of data for downstream applications.

Databricks

docs
DatabricksAzureGCPAWS
Delta Lake is included within Databricks allowing it to read from and write to Delta Lake.

Microsoft Fabric

docs
In order to achieve seamless data access across all compute engines in Microsoft Fabric, Delta Lake is chosen as the unified table format.

Power BI

docs | source code
PowerBIcommunity
This connector allows Power BI to read from Delta Lake.

Redshift

docs | source code
RedshiftAWSmanifest
This utility allows AWS Redshift to read from Delta Lake using a manifest file.

Snowflake (Beta)

docs
Snowflake
This preview allows Snowflake to read from Delta Lake via an external table.

Starburst

docs
StarburstAzureGCPAWS
The Starburst Delta Lake connector is an extended version of the Trino/Delta Lake connector with configuration and usage identical.

StarTree

docs
StarTreePinot
StarTree Cloud includes the Apache Pinot / Delta Lake connector.

Community

Try out the following community integrations with Delta Lake.