Transcript

Hi, I'm Arjun, an engineer at Warp Stream. And I'm going to demo orbit. It's a tool integrated with the rest of warp scheme which can be used to replicate data and metadata from your source Kopka compatible clusters into your destination warp stream cluster. The replicated data never leaves your VPC. It can be used to create cost effective replicas or to migrate to warp stream. The cool thing about orbit though, is that it's offset preserving. This means that the records being replicated from the source to the destination are kept In Sync down to the offsets of the records. This really simplifies migration. Now let me show you a demo. Orbit is fully controllable from a single YAML file in the destination cluster, which can be edited through the workstream console or through Terraform API. In the file, specify the bootstrap URL of the source cluster. Specify the credentials for the source cluster. Specify the names of the topics you want to copy, and in this case I'm also copying the consumer group offsets. And that's it. Hit save. And deploy the pipeline. Orbit is now transparently copying the topic configurations and topic data from the source to the destination while reserving offsets. Because we're also copying consumer group offsets, you can point your coffee consumers to the destination and they'll start reading from the correct offsets. Orbit is observable through Prometheus endpoint or just through the warped stream console. Click on the Orbit source Cluster Offsets Consumer Group. This group is special. It tells you the life of every topic which is being replicated by orbit compared to the source cluster. In this case the lag is 0, which means that our topic which we were replicating is caught up with the source. Cool, let's migrate this topic over to the destination now. We can see that the topic is currently managed by Orbit. But because the topic is caught up with the source, we can actually cut over our producers from the source to the dust station. To do that, simply add the following fields to the orbit YAML for the topic and then hit save. And that's it. Now you can write to the topic using regular coffee clients and it'll just work. That's orbit.