12 results found

1. Tiered storage for OpenSearch

   As a developer or database administrator,
   I would like to utilise object storage in my Aiven for OpenSearch instance,
   so that I can store larger amounts of data at a lower cost than attached SSD or HDD.

   35 votes

   Vote Vote Vote

   We're glad you're here

   Please sign in to leave feedback

   Signed in as (Sign out)

   Close

   Close

   Vote

   We’ll send you updates on this idea

   1 comment  ·  Aiven for OpenSearch®  ·  Delete…  ·  Admin →
   How important is this to you?

   Nice to have You must login first! Should have You must login first! Must have You must login first!

   We're glad you're here

   Please sign in to leave feedback

   Signed in as (Sign out)

   Close

   Close
   Roadmapped  ·  AdminJohn Joseph Kennedy (Head of Databases, Aiven.io) responded

   This is being planned into the roadmap for later in the year. 

2. Opensearch backpressure mechanisms

   As an Opensearch administrator, I want to be able to limit the impact of heavy requests on my cluster.
   So that when some client applications make these requests, I can mitigate impacts for other client applications.

   An example would be to be able to use backpressure mechanism
   https://opensearch.org/docs/latest/tuning-your-cluster/availability-and-recovery/search-backpressure/

   13 votes

   Vote Vote Vote

   We're glad you're here

   Please sign in to leave feedback

   Signed in as (Sign out)

   Close

   Close

   Vote

   We’ll send you updates on this idea

   Roadmapped  ·  0 comments  ·  Aiven for OpenSearch®  ·  Delete…  ·  Admin →
   How important is this to you?

   Nice to have You must login first! Should have You must login first! Must have You must login first!

   We're glad you're here

   Please sign in to leave feedback

   Signed in as (Sign out)

   Close

   Close

3. make shard_indexing_pressure configurable

   Dear Aiven Community,

   As a DevOps Engineer,
   I want to be able to configure the shardindexingpressure settings directly within the Aiven platform and through the Aiven Terraform provider,
   so that I can better manage indexing loads on Opensearch clusters, optimize performance during high data throughput, and prevent potential bottlenecks.

   In addition, this configuration capability is essential for dynamically adjusting indexing pressure based on real-time data demands. Currently, trying to enable or modify shardindexingpressure settings results in a 403 error, indicating that the feature is not supported in Aiven's current Opensearch offerings. Enabling this feature would allow users to set parameters like softlimit, minlimit, and maxoutstandingrequests to improve operational efficiency and resource utilization.

   This idea is important to me because it supports critical DevOps principles such as infrastructure as code and automation, which are essential for scalable, efficient deployment and management of large-scale search environments. By allowing users to configure these settings, Aiven can enhance its service offerings, making it a more attractive option for enterprises requiring robust, adaptable search infrastructure.

   The benefits of enabling shardindexingpressure in Aiven Opensearch include:

   Improved Performance Management: Users gain the ability to finely tune their cluster's response to varying indexing loads, enhancing overall system responsiveness and stability.
   Greater Flexibility and Control: Configurability via both the Aiven console and Terraform promotes a more tailored approach to resource management, which is especially beneficial in environments with fluctuating data loads.
   Enhanced Resource Efficiency: Proper management of shard indexing pressure prevents overutilization of shard resources, facilitating better distribution and utilization of resources across the cluster.
   I urge the community to support this feature request, as it will significantly enhance the utility and manageability of Aiven's Opensearch service for all users facing similar challenges.

   Thank you for considering this enhancement.

   Best regards,

   Dear Aiven Community,

   As a DevOps Engineer,
   I want to be able to configure the shardindexingpressure settings directly within the Aiven platform and through the Aiven Terraform provider,
   so that I can better manage indexing loads on Opensearch clusters, optimize performance during high data throughput, and prevent potential bottlenecks.

   In addition, this configuration capability is essential for dynamically adjusting indexing pressure based on real-time data demands. Currently, trying to enable or modify shardindexingpressure settings results in a 403 error, indicating that the feature is not supported in Aiven's current Opensearch offerings. Enabling this feature would allow users to set parameters… more

   • 

   8 votes

   Vote Vote Vote

   We're glad you're here

   Please sign in to leave feedback

   Signed in as (Sign out)

   Close

   Close

   Vote

   We’ll send you updates on this idea

   Roadmapped  ·  0 comments  ·  Aiven for OpenSearch®  ·  Delete…  ·  Admin →
   How important is this to you?

   Nice to have You must login first! Should have You must login first! Must have You must login first!

   We're glad you're here

   Please sign in to leave feedback

   Signed in as (Sign out)

   Close

   Close

4. Support exactly-once semantics when connecting Kafka to ClickHouse

   As a developer using Kafka to stream messages to ClickHouse,
   I want to be sure that messages as delivered and ingested exactly once, as opposed to at-least-once, to ClickHouse to not have to deal with duplicated data that I have to deduplicate down the line. This removes both a cognitive and operational load from my data pipeline
   c.f. https://github.com/ClickHouse/clickhouse-kafka-connect

   The exactly-once semantics has been implemented in the newer version of the ClickHouse Kafka Connect Sink (https://github.com/ClickHouse/clickhouse-kafka-connect).

   4 votes

   Vote Vote Vote

   We're glad you're here

   Please sign in to leave feedback

   Signed in as (Sign out)

   Close

   Close

   Vote

   We’ll send you updates on this idea

   Roadmapped  ·  0 comments  ·  Aiven for ClickHouse®  ·  Delete…  ·  Admin →
   How important is this to you?

   Nice to have You must login first! Should have You must login first! Must have You must login first!

   We're glad you're here

   Please sign in to leave feedback

   Signed in as (Sign out)

   Close

   Close

5. Add support for the DeltaLake Engine to ClickHouse

   As a customer of Aiven, I would like to pull data from many disparate sources into my ClickHouse data warehouse to provide my users best-in-class analytics and performance. Adding support for DeltaLake would open up new sources from which I can seamlessly consume from.

   4 votes

   Vote Vote Vote

   We're glad you're here

   Please sign in to leave feedback

   Signed in as (Sign out)

   Close

   Close

   Vote

   We’ll send you updates on this idea

   Roadmapped  ·  0 comments  ·  Aiven for ClickHouse®  ·  Delete…  ·  Admin →
   How important is this to you?

   Nice to have You must login first! Should have You must login first! Must have You must login first!

   We're glad you're here

   Please sign in to leave feedback

   Signed in as (Sign out)

   Close

   Close

6. Option for ClickHouse Tiered Storage to use my own object storage

   As a user of Aiven for ClickHouse on AWS, Microsoft Azure or Google Cloud:
   I would like to leverage my own object storage account with Tiered Storage for Aiven for ClickHouse (that I am already using BYOC on).

   3 votes

   Vote Vote Vote

   We're glad you're here

   Please sign in to leave feedback

   Signed in as (Sign out)

   Close

   Close

   Vote

   We’ll send you updates on this idea

   Roadmapped  ·  0 comments  ·  Aiven for ClickHouse®  ·  Delete…  ·  Admin →
   How important is this to you?

   Nice to have You must login first! Should have You must login first! Must have You must login first!

   We're glad you're here

   Please sign in to leave feedback

   Signed in as (Sign out)

   Close

   Close

7. Support for Postgresql table engine

   As a developer with a current PostgreSQL instance,
   I would like to utilise the ClickHouse PostgreSQL Table engine,
   so that I can easily read from that external PostgreSQL instance and insert the data into my Aiven for ClickHouse instance.

   3 votes

   Vote Vote Vote

   We're glad you're here

   Please sign in to leave feedback

   Signed in as (Sign out)

   Close

   Close

   Vote

   We’ll send you updates on this idea

   Roadmapped  ·  2 comments  ·  Aiven for ClickHouse®  ·  Delete…  ·  Admin →
   How important is this to you?

   Nice to have You must login first! Should have You must login first! Must have You must login first!

   We're glad you're here

   Please sign in to leave feedback

   Signed in as (Sign out)

   Close

   Close

8. MongoDB functions and table engine.

   As someone running analytics with Clickhouse,
   I want to be able to create functions that read data from an external MongoDB instance, as well as create tables that span data in external MongoDB instances,
   so that I can enrich my analytics with data stored elsewhere, or easily migrate data from a MongoDB instance to my Aiven for Clickhouse instance.

   3 votes

   Vote Vote Vote

   We're glad you're here

   Please sign in to leave feedback

   Signed in as (Sign out)

   Close

   Close

   Vote

   We’ll send you updates on this idea

   Roadmapped  ·  1 comment  ·  Aiven for ClickHouse®  ·  Delete…  ·  Admin →
   How important is this to you?

   Nice to have You must login first! Should have You must login first! Must have You must login first!

   We're glad you're here

   Please sign in to leave feedback

   Signed in as (Sign out)

   Close

   Close

9. Support for searchable snapshots

   As an user of Opensearch

   I want to be able to store large amounts of immutable logs for lengthy periods of time

   so that I can support compliance and other regulatory requirements placed on me

   In addition, I need this to be provided in a cost efficient manner, leveraging technologies such as object storage. Given queries against this data are infrequent price far outweighs performance.

   2 votes

   Vote Vote Vote

   We're glad you're here

   Please sign in to leave feedback

   Signed in as (Sign out)

   Close

   Close

   Vote

   We’ll send you updates on this idea

   Roadmapped  ·  0 comments  ·  Aiven for OpenSearch®  ·  Delete…  ·  Admin →
   How important is this to you?

   Nice to have You must login first! Should have You must login first! Must have You must login first!

   We're glad you're here

   Please sign in to leave feedback

   Signed in as (Sign out)

   Close

   Close

10. Iceberg Table Engine

    As developer ina. company with a large amount of data stored in Iceberg,
    I want to have the ClickHouse Iceberg Table engine,
    so that I can effectively analyse the large amounts of data stored in iceberg, via Aiven for Clickhouse.

    2 votes

    Vote Vote Vote

    We're glad you're here

    Please sign in to leave feedback

    Signed in as (Sign out)

    Close

    Close

    Vote

    We’ll send you updates on this idea

    Roadmapped  ·  0 comments  ·  Aiven for ClickHouse®  ·  Delete…  ·  Admin →
    How important is this to you?

    Nice to have You must login first! Should have You must login first! Must have You must login first!

    We're glad you're here

    Please sign in to leave feedback

    Signed in as (Sign out)

    Close

    Close

11. MySQL Table engine

    As someone drunning analytics queries with Clickhouse against multiple data sources,
    I want to have the MySQL table engine enabled so that I can create tables in Clickhouse that span data in external MySQL instances,
    so that I can bring the value of external data sources into my analysis.

    2 votes

    Vote Vote Vote

    We're glad you're here

    Please sign in to leave feedback

    Signed in as (Sign out)

    Close

    Close

    Vote

    We’ll send you updates on this idea

    Roadmapped  ·  0 comments  ·  Aiven for ClickHouse®  ·  Delete…  ·  Admin →
    How important is this to you?

    Nice to have You must login first! Should have You must login first! Must have You must login first!

    We're glad you're here

    Please sign in to leave feedback

    Signed in as (Sign out)

    Close

    Close

12. Opensearch Large Shard Warning

    As an operator of Opensearch,
    I want to be alerted when my shards are outside of recommend best practices of 10-50GB / shard,
    so that I can avoid having overly large shard size cause performance problems for ingestion and query.

    In addition, please tell me how within the alert to split my shards if they do get too large.

    1 vote

    Vote Vote Vote

    We're glad you're here

    Please sign in to leave feedback

    Signed in as (Sign out)

    Close

    Close

    Vote

    We’ll send you updates on this idea

    0 comments  ·  Aiven for OpenSearch®  ·  Delete…  ·  Admin →
    How important is this to you?

    Nice to have You must login first! Should have You must login first! Must have You must login first!

    We're glad you're here

    Please sign in to leave feedback

    Signed in as (Sign out)

    Close

    Close
    Roadmapped  ·  AdminHoang Minh Vo (Product Manager, Aiven.io) responded

    Thanks Jason. We can roadmap this. WIll update the idea when we have more concrete timeline