91 results found
-
Field Level Encryption Support for Aiven Products
TL;DR;
As Aiven Customer
I want to be able to be able to encrypt any form of sensitive data (PII or PCI) so that I can manage sensitive data in a legally compliant and user-privacy respectful manner.Detailed description of the proposal:
Hi Aiven!
Hope all is well with you. I have a feature/service suggestion which I believe will make your existing product portfolio even stronger!It’s a thing I call “Aiven for Privacy FTW!” and it’s basically a standalone “field-level-encryption” service used for managing of PII and PCI fields/properties in the event payload in legally (eg. GDPR and/or CCPA)…
1 voteI will be closing this out, but the idea is valid. We are looking at building a proxy service for Kafka and this could be in part of the roadmap for that component as you have described. We suggest for those who want to use encryption on Kafka to do this on the producer and consumer sides as the data would be encrypted from end to end.
-
Configure CPU threshold for alert mails in kafka
As a kafka user, I want to configure the threshold used to send CPU warning emails. Right now we are getting emails because the CPU exceeds 50%, which is irrelevant. We know we can configure the emails to go to another mail address but in that case we would loose all other technical mails as well.
It would also be good for us if we can turn off that single alert.2 votes -
MirrorMaker 2 connectivity for Kafka Brokers with private or internal hostnames
If an external Kafka service has brokers with hostnames that are not publicly resolvable, for example, where hostnames are registered in a corporate DNS server which is not accessible from outside, it is currently not possible for MM2 to successfully connect.
The workaround to use IP addresses is not sufficient, since after bootstrap the broker hostnames will be returned and subsequent connections will go there.
We will also likely see issues with TLS handshakes when brokers present certificates belonging to hostnames that the Aiven side does not understand.
The only current workaround is for the configuration of the external Kafka…
2 votes -
Azure Data Lake Gen2 (ADLS) Kafka connector
Kafka Connector to allow for streaming data to Azure Data Lake Gen2 as a sink.
5 votes -
Flink HTTP API sink
As a developer I would like to push the output of a Flink operation to an HTTP API sink.
3 votes -
Schema references support for AVRO schema in Karapace
As data engineer, developer
I want to be able use schema references in AVRO schemas in Karapace
so that I can define and reuse complex data structures or types within other schemas8 votes -
Group topics by type when displayed in Aiven Console
As a developer / operator,
I want to see Kafka topics grouped by type in Aiven Console
so that I can distinguish between topics created by Schema Registry, MirrorMaker2, Connect and directly in Kafka itself.
In addition, i want to apply filters and sorting by type.2 votes -
SAP Source connector for Aiven for Apache Kafka Connect
As a developer,
I want to be able to pull data from SAP systems,
so I could build my data pipelines.4 votes -
Schema subjects view with massive amounts of subjects
As a Avro schema administrator
I want to be able to quickly find an Avro schema subject in schema registry
so that I can view the schema, compare schema versions and change compatibility level.
In addition, we have 5-10k schema subjects in some of our projects, and there is no paging in the schema subject view - all subjects are loaded which makes the view very slow.Possible solutions might be to add paging, or simply limiting the number of subjects listed.
4 votes -
Sending Apache Kafka metrics to Datadog
As Software Engineer at Wex
I want to send additional metrics to Datadog
so that I can send important metrics from Aiven's cluster as these ones:
IsrShrinksPerSec
IsrExpandsPerSec
ActiveControllerCount
OfflinePartitionsCount
TotalTimeMs
PurgatorySize
RequestsPerSec
Network bytes sent/received
BytesInPerSec/BytesOutPerSecAccording to Datadog's documentation, these metrics are considered highly significant.
https://www.datadoghq.com/blog/monitoring-kafka-performance-metrics/#kafka-emitted-metrics2 votes -
io.debezium.transforms.partitions.PartitionRouting
It would be great it the
io.debezium.transforms.partitions.PartitionRouting
SMT was available for use when configuring a Kafka Debezium source connector on the Aiven platform.1 vote -
Custom Kafka Connectors on Aiven (Bring your own Connector)
As Developer, I want to utilize Custom Connectors on Aiven,
so that I can integrate our proprietary data systems and custom applications with Apache Kafka without having to manage the underlying infrastructure.In addition, this will allow developers to concentrate more on the building of business-critical applications instead of getting tied up with infrastructural tasks.
12 votesRoadmapped ·AdminMichael Tansini (Product Manager Data Streaming (Kafka and Flink), Aiven.io) responded
-
Expose MirrorMaker 2 replication.policy.separator property
As a devops developer
I want to change the replication policy separator
so that I can use our existing topic separators without conflicting with MM2.
In addition, I find it extremely inconvenient to have to change our existing schema because we cannot configure the replication.policy.separator in MM2.1 voteWe can see it as useful configuration to expose. At the same time it will need to gather a bit more interest to be taken in to development.
Thanks!
-
Certificate file support for Kafka connectors
As a developer / security engineer,
I want to be able to authenticate my Apache Kafka connectors via mTLS,
so that I connect to my external services in a secure way.24 votesRoadmapped ·AdminMichael Tansini (Product Manager Data Streaming (Kafka and Flink), Aiven.io) responded
-
Karapace OAuth2 support
As a developer / security engineer,
I want to be able to authenticate (i.e. Azure AD) and authorize (OAuth) my service users using OAuth2,
so that I can manage all users in a central place and don't need to worry about missing something during on- and off- boarding.8 votesGathering Interest ·AdminMichael Tansini (Product Manager Data Streaming (Kafka and Flink), Aiven.io) responded
-
Add Oracle JDBC support for Aiven for Kafka Connect
As a developer,
I want to be able to connect to my Oracle database from Aiven for Apache Kafka Connect,
so that I can read and write data from/to Oracle DB and/or Apache Kafka to enable my use-cases.9 votes -
Support for "stretched" cluster across multi regions/clouds for Aiven for Apache Kafka
As a developer / SRE,
I want to have my Apache Kafka cluster to run across multiple regions and potentially even clouds,
so that I can ensure high availability setup and have near zero RTO and RPO in case of DR, i.e. network, region or provider failure.
In addition, I want to benefit from the geo-distributed cluster setup and read/write from/to the geographically closest broker to optimise my network latency and cost.52 votes -
Add MaxTimestampSpec data to the topic/partition details from the Aiven API
As a developer,
I want to know latest offset and max timestamp for a given topic/partition,
so that I can identify when topic/partition was last time used and in case it is no longer in use I can cleanup stale topics/partitions.6 votes -
Ability to choose Apache Kafka service user password length using Aiven Console
As a developer,
I want to be able to select password length and difficulty when creating a service user,
so that I can stay complaint with internal security and compliance rules.2 votes -
Send a user alert in case stale credentials are being detected in connector configuration for Aiven for Apache Kafka Connect.
As a developer,
I want to be notified when I have stale credentials in my connectors,
so that I can update them and make sure connection from a connector to a source or sink system could be successfully established.2 votes
- Don't see your idea?