This repository contains the IO connectors for Apache Pulsar.
Connectors are packaged as NAR files and
can be deployed into any Pulsar installation by placing them in the connectors directory or
mounting them into the apachepulsar/pulsar Docker image.
| Connector | Description |
|---|---|
| Canal | MySQL binlog via Alibaba Canal |
| Debezium (MySQL, PostgreSQL, MongoDB, MSSQL, Oracle) | CDC via Debezium |
| DynamoDB | Amazon DynamoDB Streams |
| File | Local filesystem |
| Kafka | Apache Kafka |
| Kinesis | Amazon Kinesis Data Streams |
| MongoDB | MongoDB change streams |
| NSQ | NSQ messaging |
| RabbitMQ | RabbitMQ / AMQP |
| Connector | Description |
|---|---|
| Aerospike | Aerospike database |
| Alluxio | Alluxio distributed storage |
| Azure Data Explorer | Azure Data Explorer (Kusto) |
| Cassandra | Apache Cassandra |
| Elasticsearch / OpenSearch | Elasticsearch and OpenSearch |
| HBase | Apache HBase |
| HDFS3 | Hadoop HDFS |
| HTTP | HTTP endpoint |
| InfluxDB | InfluxDB time-series database |
| JDBC (PostgreSQL, MariaDB, ClickHouse, SQLite, OpenMLDB) | JDBC databases |
| Kafka | Apache Kafka |
| Kinesis | Amazon Kinesis Data Streams |
| MongoDB | MongoDB |
| Redis | Redis |
| Solr | Apache Solr |
| Connector | Description |
|---|---|
| Kafka Connect Adaptor | Run Kafka Connect connectors on Pulsar |
- JDK 17 or later — e.g. Eclipse Temurin or Amazon Corretto
Note: This project includes a Gradle Wrapper so no separate Gradle installation is needed. Use
./gradlewon Linux/macOS andgradlew.baton Windows.
Compile and assemble all modules:
./gradlew assembleNAR files are produced under each connector's build/libs/ directory.
Build a specific connector:
./gradlew :elastic-search:assembleBuild the distribution package containing all connector NARs:
./gradlew :distribution:pulsar-io-distribution:assembleCheck source code license headers:
./gradlew rat spotlessCheckAuto-fix license headers:
./gradlew spotlessApply# All unit tests
./gradlew test
# Specific connector
./gradlew :elastic-search:test
# Specific test class
./gradlew :elastic-search:test --tests "ElasticSearchSinkTests"Mount connector NARs into the Pulsar container:
docker run -v /path/to/connectors:/pulsar/connectors apachepulsar/pulsarCopy NAR files to the connectors/ directory of your Pulsar installation:
cp elastic-search/build/libs/pulsar-io-elastic-search-*.nar $PULSAR_HOME/connectors/This repository follows its own release cadence, independent from Apache Pulsar releases. All connectors are released together as a single release. The initial release version matches the Pulsar version at the time of the split.
Each release specifies which Pulsar versions it is compatible with. The pulsar-io-core
API has been stable for years, so connectors are generally compatible across Pulsar 4.x
releases.
Contributions are welcome! Please see CONTRIBUTING.md in the main Pulsar repository for guidelines.
Licensed under the Apache License, Version 2.0. See LICENSE for details.