31 results found
-
Support for "stretched" cluster across multi regions/clouds for Aiven for Apache Kafka
As a developer / SRE,
I want to have my Apache Kafka cluster to run across multiple regions and potentially even clouds,
so that I can ensure high availability setup and have near zero RTO and RPO in case of DR, i.e. network, region or provider failure.
In addition, I want to benefit from the geo-distributed cluster setup and read/write from/to the geographically closest broker to optimise my network latency and cost.50 votes -
Kafka fined grained ACLs
As an ops engineer,
I want to be able to declare fined grained ACLs
so that I avoid having to grant "admin" rights to users which only need "DeleteRecords" rights on specific topics.
The client uses Kafka Streams, which needs specific rights (https://docs.confluent.io/platform/current/streams/developer-guide/security.html#required-acl-setting-for-secure-ak-clusters) which are not covered by aiven predefined rights.
Currently, the "admin" is to broad for such access (I don't want the user to be able to create topics).6 votes -
Schema Validation on Apache Kafka broker side
As a developer,
I want to make sure schema is being validated not just on client but also on a broker side,
so that I can make sure all messages in the topic correspond to the same schema and does not contain any mixed schemas.17 votes -
Programmatic Apache Kafka Consumer (group) management
As a developer,
I want to programmatically manage my consumers and consumer groups,
so that I can see the status of them, perform CRUD operations, show members of a group, reset offsets and similar.
In addition, I want to be able to do so also in Aiven Console.18 votes -
Kafka versioned updates
As a platform engineer
I want the service updates to be versioned and to be able to select a specific version (Kafka, Karapace, etc) to update to so that I can perform correct change promotion from non-prod to production (instead of being forced to always apply the latest) and so that if a new version is released while an update is in-flight it would not lead to different versions running across the brokers (as is currently the case, which we have experienced can lead to incompatibilities). Additionally, I would like it to be clear what version(s) is currently running in…2 votes -
Support for public CA for SASL for Aiven for Apache Kafka
As developer,
I want to use public CA (i.e. Let's Encrypt),
so that I can connect to my Apache for Kafka service without installing any additional certificates.
In addition, I can trust publish authority issuing certificates instead of validating 3rd party certificates.14 votes -
Support for both allow and black lists in Kafka service
As a managed Kafka administrator
I want to have the ability to black-list a bunch of IP addresses I believe are suspicious from my Kafka service, so that I can prevent my service being disrupted by unexpected traffic
In addition, I would like to keep the current allow-list to be able to allow know IPs and a way to resolve a conflict between allow and block list where block list takes priority.1 vote -
Cluster leader balancing CPU vs Disk - can we choose?
As an OPS Engineer
I want to be able to balance the cluster based on CPU usage rather than Disk usage. The current algorithm focuses on Disk usage which is not optimum for our application.
Can we have an option to apply partition rebalancing based on CPU usage?1 vote -
Kafka tiered storage with external S3 bucket
As a developer
I want to have the ability to use our own S3 bucket for storing Kafka tiered storage
so that I can access the data from S3 and query some data for debugging (without streaming all the data to Kafka)2 votes -
Configurable maximum validity for service user certificates
As a developer / security engineer,
I want to be able to define lifetime of certificates (i.e. 30-90 days) used to authenticate my service users,
so that I can have certificate rotation policies in place to ensure compliance and security best practices.10 votes -
Custom Kafka Connectors on Aiven (Bring your own Connector)
As Developer, I want to utilize Custom Connectors on Aiven,
so that I can integrate our proprietary data systems and custom applications with Apache Kafka without having to manage the underlying infrastructure.In addition, this will allow developers to concentrate more on the building of business-critical applications instead of getting tied up with infrastructural tasks.
9 votesRoadmapped ·AdminMichael Tansini (Product Manager Data Streaming (Kafka and Flink), Aiven.io) responded
-
kafka_connect_connector_metrics availability over Prometheus
Our customer Jago wants to monitor the status of connectors and tasks but currently can't find relevant metrics to do so. They want to be able to monitor the status of connectors and tasks on a dashboard and also get notified whenever a connector has not been running for X minutes.
Jago has a connector running but can not find the metrics for kafka.connect:type=connector-metrics,connector=*.
The specific metrics they are looking for is the one related to the status of a connector. For example, in the customers current self-managed kafka connect, they have the following metrics. This is convenient because they…
1 vote -
Include custom Kafka topic tags as Prometheus metric labels
The Aiven platform allows custom key/value tags to be added to resources such as Kafka topics. It would be useful to have these exposed as additional labels on metrics so that alerts can be triggered based on this metadata.
3 votes -
Temporarily disable service user on Aiven for Apache Kafka
As a developer,
I want to temporarily disable service user,
so that I can test is service user still in use by any of the applications or temporarily ban abusive clients.10 votes -
Ability to scale Kafka cluster without upgrade
As a Kafka Administrator
I want to ahve ability to scalse Kafka clsuter without upgrade, that increses time of scale.
In production this can become critical and lead to downtime. For example last 2 times the scale lasted for 12 hours.1 vote -
Audit logging as self-service option
As a developer,
I want to enable audit logging for my service,
so that I can keep track of breadcrumbs left by all the changes made to a service to ensure compliance.8 votes -
Enforcing naming convention for Apache Kafka topics
As a SRE / Apache Kafka operator / developer,
I want to make sure all my topics adhere to a naming convention,
so that I can ensure consistency of my topics.
In addition, I can use naming convention to better identify, group, locate and categorise my topics.8 votes -
Capture and expose "last used" date for service users
As a developer / SRE,
I want to know when service users were last used,
so that I can know is service user still in use or was abandoned and could be removed.10 votes -
Add MaxTimestampSpec data to the topic/partition details from the Aiven API
As a developer,
I want to know latest offset and max timestamp for a given topic/partition,
so that I can identify when topic/partition was last time used and in case it is no longer in use I can cleanup stale topics/partitions.6 votes -
Support for cross-project and cross-vpc service integration for all services in streaming domain
As a developer,
I want to integrate my services in streaming domain, i.e. Apache Kafka, Apache Kafka Connect, Apache MirrorMaker2, etc with other services in any project and/or VPC,
so that I can build data streaming pipelines and benefit from the use use of aforementioned services.6 votes
- Don't see your idea?