94 results found

1. Improve support for Debezium SQL Server use cases

   As an architect, I want to bring business logic out of the database and into a decoupled stream processing / event driven architecture framework. With SQL Server, I want to stream changes to Apache Kafka, using Debezium. This must support use cases where sensitivity classifications in SQL Server are used/required for things like PII.

   2 votes

   Vote Vote Vote

   We're glad you're here

   Please sign in to leave feedback

   Signed in as (Sign out)

   Close

   Close

   Vote

   We’ll send you updates on this idea

   Roadmapped  ·  0 comments  ·  Aiven for Apache Kafka® Connect  ·  Delete…  ·  Admin →
   How important is this to you?

   Nice to have You must login first! Should have You must login first! Must have You must login first!

   We're glad you're here

   Please sign in to leave feedback

   Signed in as (Sign out)

   Close

   Close

2. Add external Delta Lake connector for Apache Flink

   As a developer, I need to be able to read and write from Databricks so that I can complement my existing data architecture

   2 votes

   Vote Vote Vote

   We're glad you're here

   Please sign in to leave feedback

   Signed in as (Sign out)

   Close

   Close

   Vote

   We’ll send you updates on this idea

   Gathering Interest  ·  0 comments  ·  Aiven for Apache Flink®  ·  Delete…  ·  Admin →
   How important is this to you?

   Nice to have You must login first! Should have You must login first! Must have You must login first!

   We're glad you're here

   Please sign in to leave feedback

   Signed in as (Sign out)

   Close

   Close

3. Auto-Scaling

   As a developer, I need Flink to be able to automatically increase compute resources to a self-defined threshold to seamlessly process significant increases in traffic in order to maintain performance. This increase in traffic would be for either a permanently higher workload, or for a temporary pre-defined amount of time, and would be triggered by a customer request, with subsequent additional billing impacts

   2 votes

   Vote Vote Vote

   We're glad you're here

   Please sign in to leave feedback

   Signed in as (Sign out)

   Close

   Close

   Vote

   We’ll send you updates on this idea

   Gathering Interest  ·  0 comments  ·  Aiven for Apache Flink®  ·  Delete…  ·  Admin →
   How important is this to you?

   Nice to have You must login first! Should have You must login first! Must have You must login first!

   We're glad you're here

   Please sign in to leave feedback

   Signed in as (Sign out)

   Close

   Close

4. Kafka rate limits and quotas configuration via Terraform

   As a Cloud platform engineer I need to provide the capability, to developers, to set quotas via Terraform when then set-up applications that will produce to/consume from a Kafka cluster.

   3 votes

   Vote Vote Vote

   We're glad you're here

   Please sign in to leave feedback

   Signed in as (Sign out)

   Close

   Close

   Vote

   We’ll send you updates on this idea

   Roadmapped  ·  0 comments  ·  Aiven for Apache Kafka®  ·  Delete…  ·  Admin →
   How important is this to you?

   Nice to have You must login first! Should have You must login first! Must have You must login first!

   We're glad you're here

   Please sign in to leave feedback

   Signed in as (Sign out)

   Close

   Close

5. Automated dynamic quota configuration

   As a Cloud platform engineer, I need to have an automated way to set up and update quota configurations on a cluster, taking into account changes in resources consumption patterns amongst producers and consumers.

   1 vote

   Vote Vote Vote

   We're glad you're here

   Please sign in to leave feedback

   Signed in as (Sign out)

   Close

   Close

   Vote

   We’ll send you updates on this idea

   Gathering Interest  ·  0 comments  ·  Aiven for Apache Kafka®  ·  Delete…  ·  Admin →
   How important is this to you?

   Nice to have You must login first! Should have You must login first! Must have You must login first!

   We're glad you're here

   Please sign in to leave feedback

   Signed in as (Sign out)

   Close

   Close

6. Visualization of client IDs that interact with a Kafka cluster

   As a Cloud platform engineer, I need to have a view of Kafka cluster resources consumption across all clients IDs. Such information is needed to configure quotas and to manage them.

   2 votes

   Vote Vote Vote

   We're glad you're here

   Please sign in to leave feedback

   Signed in as (Sign out)

   Close

   Close

   Vote

   We’ll send you updates on this idea

   Gathering Interest  ·  0 comments  ·  Aiven for Apache Kafka®  ·  Delete…  ·  Admin →
   How important is this to you?

   Nice to have You must login first! Should have You must login first! Must have You must login first!

   We're glad you're here

   Please sign in to leave feedback

   Signed in as (Sign out)

   Close

   Close

7. Field Level Encryption Support for Aiven Products

   TL;DR;
   As Aiven Customer
   I want to be able to be able to encrypt any form of sensitive data (PII or PCI) so that I can manage sensitive data in a legally compliant and user-privacy respectful manner.

   Detailed description of the proposal:
   Hi Aiven!
   Hope all is well with you. I have a feature/service suggestion which I believe will make your existing product portfolio even stronger!

   It’s a thing I call “Aiven for Privacy FTW!” and it’s basically a standalone “field-level-encryption” service used for managing of PII and PCI fields/properties in the event payload in legally (eg. GDPR and/or CCPA) and secure manner (ie. sensitive data encrypted in transit and at rest).

   I made a following high-level architecture/component diagram with aim to visualize how would “Aiven for Privacy FTW!” fit together with your products and within client context. See attached.

   After drawing and discussing this internally, I realized that it would be even better if this is a stand-alone service that can be instantiated and run fully within the client context and manage sensitive fields/properties even before event enters Aiven managed services. This way, the sensitivity will be fully managed and stay within the client/producer control and will not pollute your services with sensitive data.

   Why should you consider this? I’d like to claim that you’re in unique position in the industry with a portfolio of solutions that can be connected fully end-to-end from event producer to consumer and with this type of functionality you’ll be able to provide a turn-key solution to all your clients.

   Nothing here is new, tbh. There is no novelty here and others are doing/offering something similar already, but no one is giving a full package solution with privacy compliance being made a first class citizen. You could.

   Example of others doing stuff in this area:
   - Snowflake allows their customers to use external tokenizers to unlock/show sensitive data based on the access privilege: https://docs.snowflake.com/en/user-guide/security-column-ext-token-intro
   - MongoDB is offering their clients ability to encrypt sensitive data on the client side using: https://www.mongodb.com/docs/manual/core/csfle/
   - Conduktor provides encryption all the way down to field/property level: https://www.conduktor.io/encryption/

   In all these cases (at least MongoDB and Snowflake), “Aiven for Privacy FTW!” if it becomes a thing could be used if done as a stand-alone service. So, it could strengthen your position further in the industry.

   I hope you will consider this and feel free to contact me either directly or through our Aiven KAM, Hanna Behr, if you want to talk more about this.

   Thanks,
   Srdjan B. (Eng. Director, Sinch SaaS App Platform)

   TL;DR;
   As Aiven Customer
   I want to be able to be able to encrypt any form of sensitive data (PII or PCI) so that I can manage sensitive data in a legally compliant and user-privacy respectful manner.

   Detailed description of the proposal:
   Hi Aiven!
   Hope all is well with you. I have a feature/service suggestion which I believe will make your existing product portfolio even stronger!

   It’s a thing I call “Aiven for Privacy FTW!” and it’s basically a standalone “field-level-encryption” service used for managing of PII and PCI fields/properties in the event payload in legally (eg. GDPR and/or CCPA)… more

   •  

   1 vote

   Vote Vote Vote

   We're glad you're here

   Please sign in to leave feedback

   Signed in as (Sign out)

   Close

   Close

   Vote

   We’ll send you updates on this idea

   1 comment  ·  Other  ·  Delete…  ·  Admin →
   How important is this to you?

   Nice to have You must login first! Should have You must login first! Must have You must login first!

   We're glad you're here

   Please sign in to leave feedback

   Signed in as (Sign out)

   Close

   Close
   Shelved  ·  Jonah Kowall responded

   I will be closing this out, but the idea is valid. We are looking at building a proxy service for Kafka and this could be in part of the roadmap for that component as you have described. We suggest for those who want to use encryption on Kafka to do this on the producer and consumer sides as the data would be encrypted from end to end. 

8. Configure CPU threshold for alert mails in kafka

   As a kafka user, I want to configure the threshold used to send CPU warning emails. Right now we are getting emails because the CPU exceeds 50%, which is irrelevant. We know we can configure the emails to go to another mail address but in that case we would loose all other technical mails as well.
   It would also be good for us if we can turn off that single alert.

   2 votes

   Vote Vote Vote

   We're glad you're here

   Please sign in to leave feedback

   Signed in as (Sign out)

   Close

   Close

   Vote

   We’ll send you updates on this idea

   Gathering Interest  ·  0 comments  ·  Aiven for Apache Kafka®  ·  Delete…  ·  Admin →
   How important is this to you?

   Nice to have You must login first! Should have You must login first! Must have You must login first!

   We're glad you're here

   Please sign in to leave feedback

   Signed in as (Sign out)

   Close

   Close

9. MirrorMaker 2 connectivity for Kafka Brokers with private or internal hostnames

   If an external Kafka service has brokers with hostnames that are not publicly resolvable, for example, where hostnames are registered in a corporate DNS server which is not accessible from outside, it is currently not possible for MM2 to successfully connect.

   The workaround to use IP addresses is not sufficient, since after bootstrap the broker hostnames will be returned and subsequent connections will go there.

   We will also likely see issues with TLS handshakes when brokers present certificates belonging to hostnames that the Aiven side does not understand.

   The only current workaround is for the configuration of the external Kafka service to change so that extra listeners or certificates are added, but this is often very difficult to achieve in a production environment where change is tightly managed or where downtime is to be avoided.

   If an external Kafka service has brokers with hostnames that are not publicly resolvable, for example, where hostnames are registered in a corporate DNS server which is not accessible from outside, it is currently not possible for MM2 to successfully connect.

   The workaround to use IP addresses is not sufficient, since after bootstrap the broker hostnames will be returned and subsequent connections will go there.

   We will also likely see issues with TLS handshakes when brokers present certificates belonging to hostnames that the Aiven side does not understand.

   The only current workaround is for the configuration of the external Kafka… more

   2 votes

   Vote Vote Vote

   We're glad you're here

   Please sign in to leave feedback

   Signed in as (Sign out)

   Close

   Close

   Vote

   We’ll send you updates on this idea

   Gathering Interest  ·  0 comments  ·  Aiven for Apache Kafka® MirrorMaker 2  ·  Delete…  ·  Admin →
   How important is this to you?

   Nice to have You must login first! Should have You must login first! Must have You must login first!

   We're glad you're here

   Please sign in to leave feedback

   Signed in as (Sign out)

   Close

   Close

10. Azure Data Lake Gen2 (ADLS) Kafka connector

    Kafka Connector to allow for streaming data to Azure Data Lake Gen2 as a sink.

    5 votes

    Vote Vote Vote

    We're glad you're here

    Please sign in to leave feedback

    Signed in as (Sign out)

    Close

    Close

    Vote

    We’ll send you updates on this idea

    Gathering Interest  ·  0 comments  ·  Aiven for Apache Kafka® Connect  ·  Delete…  ·  Admin →
    How important is this to you?

    Nice to have You must login first! Should have You must login first! Must have You must login first!

    We're glad you're here

    Please sign in to leave feedback

    Signed in as (Sign out)

    Close

    Close

11. Flink HTTP API sink

    As a developer I would like to push the output of a Flink operation to an HTTP API sink.

    3 votes

    Vote Vote Vote

    We're glad you're here

    Please sign in to leave feedback

    Signed in as (Sign out)

    Close

    Close

    Vote

    We’ll send you updates on this idea

    Shelved  ·  0 comments  ·  Aiven for Apache Flink®  ·  Delete…  ·  Admin →
    How important is this to you?

    Nice to have You must login first! Should have You must login first! Must have You must login first!

    We're glad you're here

    Please sign in to leave feedback

    Signed in as (Sign out)

    Close

    Close

12. Schema references support for AVRO schema in Karapace

    As data engineer, developer
    I want to be able use schema references in AVRO schemas in Karapace
    so that I can define and reuse complex data structures or types within other schemas

    8 votes

    Vote Vote Vote

    We're glad you're here

    Please sign in to leave feedback

    Signed in as (Sign out)

    Close

    Close

    Vote

    We’ll send you updates on this idea

    Gathering Interest  ·  0 comments  ·  Karapace  ·  Delete…  ·  Admin →
    How important is this to you?

    Nice to have You must login first! Should have You must login first! Must have You must login first!

    We're glad you're here

    Please sign in to leave feedback

    Signed in as (Sign out)

    Close

    Close

13. Group topics by type when displayed in Aiven Console

    As a developer / operator,
    I want to see Kafka topics grouped by type in Aiven Console
    so that I can distinguish between topics created by Schema Registry, MirrorMaker2, Connect and directly in Kafka itself.
    In addition, i want to apply filters and sorting by type.

    2 votes

    Vote Vote Vote

    We're glad you're here

    Please sign in to leave feedback

    Signed in as (Sign out)

    Close

    Close

    Vote

    We’ll send you updates on this idea

    Gathering Interest  ·  2 comments  ·  Aiven for Apache Kafka®  ·  Delete…  ·  Admin →
    How important is this to you?

    Nice to have You must login first! Should have You must login first! Must have You must login first!

    We're glad you're here

    Please sign in to leave feedback

    Signed in as (Sign out)

    Close

    Close

14. SAP Source connector for Aiven for Apache Kafka Connect

    As a developer,
    I want to be able to pull data from SAP systems,
    so I could build my data pipelines.

    4 votes

    Vote Vote Vote

    We're glad you're here

    Please sign in to leave feedback

    Signed in as (Sign out)

    Close

    Close

    Vote

    We’ll send you updates on this idea

    Gathering Interest  ·  0 comments  ·  Aiven for Apache Kafka® Connect  ·  Delete…  ·  Admin →
    How important is this to you?

    Nice to have You must login first! Should have You must login first! Must have You must login first!

    We're glad you're here

    Please sign in to leave feedback

    Signed in as (Sign out)

    Close

    Close

15. Schema subjects view with massive amounts of subjects

    As a Avro schema administrator
    I want to be able to quickly find an Avro schema subject in schema registry
    so that I can view the schema, compare schema versions and change compatibility level.
    In addition, we have 5-10k schema subjects in some of our projects, and there is no paging in the schema subject view - all subjects are loaded which makes the view very slow.

    Possible solutions might be to add paging, or simply limiting the number of subjects listed.

    4 votes

    Vote Vote Vote

    We're glad you're here

    Please sign in to leave feedback

    Signed in as (Sign out)

    Close

    Close

    Vote

    We’ll send you updates on this idea

    Gathering Interest  ·  3 comments  ·  Karapace  ·  Delete…  ·  Admin →
    How important is this to you?

    Nice to have You must login first! Should have You must login first! Must have You must login first!

    We're glad you're here

    Please sign in to leave feedback

    Signed in as (Sign out)

    Close

    Close

16. Sending Apache Kafka metrics to Datadog

    As Software Engineer at Wex
    I want to send additional metrics to Datadog
    so that I can send important metrics from Aiven's cluster as these ones:
    IsrShrinksPerSec
    IsrExpandsPerSec
    ActiveControllerCount
    OfflinePartitionsCount
    TotalTimeMs
    PurgatorySize
    RequestsPerSec
    Network bytes sent/received
    BytesInPerSec/BytesOutPerSec

    According to Datadog's documentation, these metrics are considered highly significant.
    https://www.datadoghq.com/blog/monitoring-kafka-performance-metrics/#kafka-emitted-metrics

    2 votes

    Vote Vote Vote

    We're glad you're here

    Please sign in to leave feedback

    Signed in as (Sign out)

    Close

    Close

    Vote

    We’ll send you updates on this idea

    Shelved  ·  0 comments  ·  Aiven for Apache Kafka®  ·  Delete…  ·  Admin →
    How important is this to you?

    Nice to have You must login first! Should have You must login first! Must have You must login first!

    We're glad you're here

    Please sign in to leave feedback

    Signed in as (Sign out)

    Close

    Close

17. io.debezium.transforms.partitions.PartitionRouting

    It would be great it the io.debezium.transforms.partitions.PartitionRouting SMT was available for use when configuring a Kafka Debezium source connector on the Aiven platform.

    1 vote

    Vote Vote Vote

    We're glad you're here

    Please sign in to leave feedback

    Signed in as (Sign out)

    Close

    Close

    Vote

    We’ll send you updates on this idea

    Gathering Interest  ·  0 comments  ·  Aiven for Apache Kafka® Connect  ·  Delete…  ·  Admin →
    How important is this to you?

    Nice to have You must login first! Should have You must login first! Must have You must login first!

    We're glad you're here

    Please sign in to leave feedback

    Signed in as (Sign out)

    Close

    Close

18. Custom Kafka Connectors on Aiven (Bring your own Connector)

    As Developer, I want to utilize Custom Connectors on Aiven,
    so that I can integrate our proprietary data systems and custom applications with Apache Kafka without having to manage the underlying infrastructure.

    In addition, this will allow developers to concentrate more on the building of business-critical applications instead of getting tied up with infrastructural tasks.

    15 votes

    Vote Vote Vote

    We're glad you're here

    Please sign in to leave feedback

    Signed in as (Sign out)

    Close

    Close

    Vote

    We’ll send you updates on this idea

    0 comments  ·  Aiven for Apache Kafka®  ·  Delete…  ·  Admin →
    How important is this to you?

    Nice to have You must login first! Should have You must login first! Must have You must login first!

    We're glad you're here

    Please sign in to leave feedback

    Signed in as (Sign out)

    Close

    Close
    Roadmapped  ·  AdminMichael Tansini (Senior Product Manager, Aiven.io) responded

    
    

19. Expose MirrorMaker 2 replication.policy.separator property

    As a devops developer
    I want to change the replication policy separator
    so that I can use our existing topic separators without conflicting with MM2.
    In addition, I find it extremely inconvenient to have to change our existing schema because we cannot configure the replication.policy.separator in MM2.

    1 vote

    Vote Vote Vote

    We're glad you're here

    Please sign in to leave feedback

    Signed in as (Sign out)

    Close

    Close

    Vote

    We’ll send you updates on this idea

    0 comments  ·  Aiven for Apache Kafka®  ·  Delete…  ·  Admin →
    How important is this to you?

    Nice to have You must login first! Should have You must login first! Must have You must login first!

    We're glad you're here

    Please sign in to leave feedback

    Signed in as (Sign out)

    Close

    Close
    Gathering Interest  ·  Mykhailo Lieibenson responded

    We can see it as useful configuration to expose. At the same time it will need to gather a bit more interest to be taken in to development. 

    Thanks!

20. Certificate file support for Kafka connectors

    As a developer / security engineer,
    I want to be able to authenticate my Apache Kafka connectors via mTLS,
    so that I connect to my external services in a secure way.

    24 votes

    Vote Vote Vote

    We're glad you're here

    Please sign in to leave feedback

    Signed in as (Sign out)

    Close

    Close

    Vote

    We’ll send you updates on this idea

    0 comments  ·  Aiven for Apache Kafka® Connect  ·  Delete…  ·  Admin →
    How important is this to you?

    Nice to have You must login first! Should have You must login first! Must have You must login first!

    We're glad you're here

    Please sign in to leave feedback

    Signed in as (Sign out)

    Close

    Close
    Roadmapped  ·  AdminMichael Tansini (Senior Product Manager, Aiven.io) responded