85 results found
-
Move Karapace updates out of maintenance updates into IaC definitions
As a customer
I want to be able to select Karapace version updates in Terraform code instead of maintenance updates GUI
so that I can control when Karapace updates are applied from my IaC definition.
In addition, Karapace updates can introduce service interruptions for users because all consumer instances are terminated. Moving it out of maintenance updates means less user interruptions on often more important maintenance updates.4 votes -
Azure Data Lake Gen2 (ADLS) Kafka connector
Kafka Connector to allow for streaming data to Azure Data Lake Gen2 as a sink.
4 votes -
Schema subjects view with massive amounts of subjects
As a Avro schema administrator
I want to be able to quickly find an Avro schema subject in schema registry
so that I can view the schema, compare schema versions and change compatibility level.
In addition, we have 5-10k schema subjects in some of our projects, and there is no paging in the schema subject view - all subjects are loaded which makes the view very slow.Possible solutions might be to add paging, or simply limiting the number of subjects listed.
4 votes -
Include custom Kafka topic tags as Prometheus metric labels
The Aiven platform allows custom key/value tags to be added to resources such as Kafka topics. It would be useful to have these exposed as additional labels on metrics so that alerts can be triggered based on this metadata.
3 votes -
Flink HTTP API sink
As a developer I would like to push the output of a Flink operation to an HTTP API sink.
3 votes -
Kafka Connect GCS Sink: Support using field values to define bucket name or file name prefix
As data streaming architect
I want to be able to export records from Kafka to GCS and use values in the record to define the bucket or file name
so that I can organize data by those values to make them easier to find and process.Use case is a multi-user/multi-tenant application where user info is a value in the record. Need to be able to organize the output in object storage by that value somehow.
2 votes -
Kafka versioned updates
As a platform engineer
I want the service updates to be versioned and to be able to select a specific version (Kafka, Karapace, etc) to update to so that I can perform correct change promotion from non-prod to production (instead of being forced to always apply the latest) and so that if a new version is released while an update is in-flight it would not lead to different versions running across the brokers (as is currently the case, which we have experienced can lead to incompatibilities). Additionally, I would like it to be clear what version(s) is currently running in…2 votes -
Add OpenTelemetry API and SDK to enable traceability in Debezium connector
As IT Architect
I want to provide complete traceability within my microservice mesh where 2 microservice communicate via Outbox pattern via Debezium connector. In order to achieve this, Debezium connector requires some of the OpenTelemtry APIs are on the Kafka Connect classpath.
I want to have this feature so that I can see complete chain of interactions for specific request, observe what and where time is spent inside particular microservice and find out possible bottlenecks.2 votes -
Aiven API support for Kafka Connect Java clients
As an AIven developer user
I want to use the Java code generated from Aiven API to communicate with an Aiven managed KafkaConnect cluster
so that I can build micro services in Java platform to manage various connectors on the cluster
In addition, this gives more security for Aiven users and data while the standard REST API supports only BasicAuth.2 votes -
Kafka tiered storage with external S3 bucket
As a developer
I want to have the ability to use our own S3 bucket for storing Kafka tiered storage
so that I can access the data from S3 and query some data for debugging (without streaming all the data to Kafka)2 votes -
Connectivity check for MirrorMaker with external Kafka integration
Debugging failed connectivity between Aiven for Apache MirrorMaker and an external Kafka configured using an integration endpoint is very difficult today. No errors are surfaced in the integration endpoint configuration screen itself, we must wait for the replication flow to attempt to start and then dig out cryptic errors from MM2 logs.
Some kinds of errors (e.g. failure to build SSL keystores) are not even surfaced in MM2 logs and are only visible to Aiven operators.
Please consider adding a basic connectivity check to allow for quicker troubleshooting and iteration. This check should ensure that the network path between Aiven…
2 votes -
Add Datadog integration to Flink
As a data engineer
I want to use my integrate my existing Datadog subscription with Flink
so that I can store and monitor all metrics across my stack in a single location.
In addition, this functionality is already available on other services in Aiven2 votes -
Ease the setup of Kafka Connectors when using internal Aiven services
As a developer,
I want to have an easy setup of my Kafka connectors when they are using internal Aiven services such as Postgres or OpenSearch
so that I can save time, avoid mistakes, and enforce the strength of the platform.2 votes -
Improve support for Debezium SQL Server use cases
As an architect, I want to bring business logic out of the database and into a decoupled stream processing / event driven architecture framework. With SQL Server, I want to stream changes to Apache Kafka, using Debezium. This must support use cases where sensitivity classifications in SQL Server are used/required for things like PII.
2 votes -
Kafka rack awareness
Kafka consumer clients on Aiven for Kafka should be able to use client.rack configuration introduced in KIP-392
2 votes -
Kafka rate limits and quotas configuration via Terraform
As a Cloud platform engineer I need to provide the capability, to developers, to set quotas via Terraform when then set-up applications that will produce to/consume from a Kafka cluster.
2 votes -
Configure CPU threshold for alert mails in kafka
As a kafka user, I want to configure the threshold used to send CPU warning emails. Right now we are getting emails because the CPU exceeds 50%, which is irrelevant. We know we can configure the emails to go to another mail address but in that case we would loose all other technical mails as well.
It would also be good for us if we can turn off that single alert.2 votes -
MirrorMaker 2 connectivity for Kafka Brokers with private or internal hostnames
If an external Kafka service has brokers with hostnames that are not publicly resolvable, for example, where hostnames are registered in a corporate DNS server which is not accessible from outside, it is currently not possible for MM2 to successfully connect.
The workaround to use IP addresses is not sufficient, since after bootstrap the broker hostnames will be returned and subsequent connections will go there.
We will also likely see issues with TLS handshakes when brokers present certificates belonging to hostnames that the Aiven side does not understand.
The only current workaround is for the configuration of the external Kafka…
2 votes -
Group topics by type when displayed in Aiven Console
As a developer / operator,
I want to see Kafka topics grouped by type in Aiven Console
so that I can distinguish between topics created by Schema Registry, MirrorMaker2, Connect and directly in Kafka itself.
In addition, i want to apply filters and sorting by type.2 votes -
SAP Source connector for Aiven for Apache Kafka Connect
As a developer,
I want to be able to pull data from SAP systems,
so I could build my data pipelines.2 votes
- Don't see your idea?