48 results found
-
Allow customer to stick with one minor OpenSearch version for an extended period
As a DB admin
I want to stick w a minor OS version until I have done all my testing with the new minor versions to make sure it does not break anything
so that I can confidently upgrade my OS.1 vote -
Support and/or adopt PGSync to help users maintain real-time sync between Postgres and Opensearch
As an OpenSearch and Postgres user
I want to easily keep these databases in sync
so that I can provide low latency, high throughput full text search that doesn't impact my transactional database.Ever since search engines were invented, developers have struggled with keeping their system of records (typically relational databases like Oracle, Postgres, MySQL, etc) in sync with the search engine. Batch oriented processes are often not real-time enough resulting in poor user experiences where the search results don't return the same information as what's represented in the system of record. Event-oriented solutions such as writing updates from the…
1 vote -
Support externally hosted models in OpenSearch
As developer
I want to configure OpenSearch to generate embeddings using remote models on OpenAI, Cohere, Sagemaker, etc.
so that I can use the best available models on the market and simplify my codebase by letting OpenSearch generate these embeddingshttps://opensearch.org/docs/latest/ml-commons-plugin/remote-models/index/
1 vote -
Ability to change refresh interval for Index using Terraform
As a DBRE
I want to change the index refresh interval from a default value to some non-default value as a part of our latency optimization. With that, I would like this ability through IAC and control through our codebase.The idea here is to control this value from IAC and in more organized fashion.
1 vote -
Opensearch Backup to another Region
Provide a capability to backup opensearch to a secondary region in the even of hyperscaller regional outage allowing for customers to develop DR strategies meeting their RTO/RPO.
2 votes -
Simplified Data Visualization Dashboard
As a Data Analyst, I propose developing a user-friendly data visualization dashboard tailored for non-technical users. The goal is to make data insights accessible without requiring advanced analytics skills. This dashboard would feature drag and drop functionality, allowing users to easily customize charts and graphs. Pre-built templates for common metrics could help streamline the process, and real-time data updates would ensure users always see the most current information.
Incorporating interactive elements like clickable filters and tooltips would enhance usability, enabling users to explore data without feeling overwhelmed. By focusing on intuitive design and ease of use, this dashboard could empower…
1 vote -
Enable index rollup settings
There is a reported issue where incomplete results are returned when querying rollup indexes in OpenSearch. The issue was created upstream and found here:
https://github.com/opensearch-project/index-management/issues/903#issuecomment-2146610171
The solution suggested is to enable the plugin "plugins.rollup.search.searchalljobs" and local testing confirms that enabling this resolves the bug report, but access to the /_cluster/settings are currently not supported.
3 votes -
Restore from external snapshot
As a database administrator
I want to restore from external snapshot that isnt hosted on Aiven
so that I can migrate data from certain OpenSearch and Elasticsearch cluster (Aiven and Non-Aiven) to Aiven for OpenSearch3 votes -
make shard_indexing_pressure configurable
Dear Aiven Community,
As a DevOps Engineer,
I want to be able to configure the shardindexingpressure settings directly within the Aiven platform and through the Aiven Terraform provider,
so that I can better manage indexing loads on Opensearch clusters, optimize performance during high data throughput, and prevent potential bottlenecks.In addition, this configuration capability is essential for dynamically adjusting indexing pressure based on real-time data demands. Currently, trying to enable or modify shardindexingpressure settings results in a 403 error, indicating that the feature is not supported in Aiven's current Opensearch offerings. Enabling this feature would allow users to set parameters…
8 votes -
Opensearch Large Shard Warning
As an operator of Opensearch,
I want to be alerted when my shards are outside of recommend best practices of 10-50GB / shard,
so that I can avoid having overly large shard size cause performance problems for ingestion and query.In addition, please tell me how within the alert to split my shards if they do get too large.
1 voteThanks Jason. We can roadmap this. WIll update the idea when we have more concrete timeline
-
Make index snapshots possible for faster data restoration
As a SRE or operations engineer
I want to be able to create index snapshots manually or with a policy
so that I can easily and quickly restore partial data after an unintentional index corruption.
In addition, I understand this would require a shared storage between cluster nodes which means additional storage so it would make sense to make it a paid feature and/or include it in the tiered storage project.4 votesThanks for posting the idea, the idea is reasonable. I will update the status once we know more about plan with this idea
-
Support custom plugins in OpenSearch
As a developer
I want to use a custom plugin in OpenSearch
so that I can implement custom scoring or text analysis or use a community open source plugin.2 votesWe need to validate each plugins installed in Aiven for OpenSearch to ensure that the plugin wont cause any harm to our services.
-
Enable all cross cluster replication APIs
As an OpenSearch operator
I want to be able to pause, stop, and start cross cluster replication
so that I can use this feature to support failover in a disaster recovery scenario.Typically in Elasticsearch or Opensearch, CCR can be used to support a DR deployment by placing two clusters (a leader and follower) in separate regions. When the leader cluster becomes unavailable, applications/clients can failover to the follower cluster by stopping replication on the following cluster which makes it a regular index.
https://opensearch.org/docs/latest/tuning-your-cluster/replication-plugin/api/
1 voteCurrently we are controlling this process of start/pause etc so we can control replication process during the entire cluster's lifecycle (through all the node recycling, upgrading etc.) to ensure the stability of our services.
I put this to Gather interest, I am also aware of the failover capability mentioned in the ideas as well (we have different ideas ticket for that), this is something we can have a look and see if we need to expose all APIs if the main usecase is failover
-
OpenSearch compute-optimized and storage-optimized clusters
As an architect
I want to create right-sized clusters for my use case
so that I can get the most value.Currently, all the OpenSearch clusters have a 1:4 CPU:RAM ratio. High throughput application search use cases often have small data sets and can benefit from more relative CPU than RAM or disk (e.g. 1:2 CPU:RAM ratio). Logging use cases with large volumes of data may benefit from storage optimized instances with 1:8 CPU:RAM ratio with more disk.
1 vote -
Opensearch backpressure mechanisms
As an Opensearch administrator, I want to be able to limit the impact of heavy requests on my cluster.
So that when some client applications make these requests, I can mitigate impacts for other client applications.An example would be to be able to use backpressure mechanism
https://opensearch.org/docs/latest/tuning-your-cluster/availability-and-recovery/search-backpressure/13 votes -
Support two-zone OpenSearch clusters
As an OpenSearch architect
I want to provision the minimum amount of hardware I need to meet my requirements
so that I can optimize costs.Currently, OpenSearch on Aiven only supports 3 AZ deployments for production-grade plans. OpenSearch clusters with data nodes deployed across 2 AZs can be considered production-grade as long as you have master nodes across 3 AZs.
1 vote -
Support for searchable snapshots
As an user of Opensearch
I want to be able to store large amounts of immutable logs for lengthy periods of time
so that I can support compliance and other regulatory requirements placed on me
In addition, I need this to be provided in a cost efficient manner, leveraging technologies such as object storage. Given queries against this data are infrequent price far outweighs performance.
2 votes -
Opensearch ML Plugin cluster settings in UI
As Developer
I want to be able to leverage the new ML capabilities in OpenSearch (https://opensearch.org/docs/latest/ml-commons-plugin/)
so that I can use new features like Semantic search, leveraging external models etc.
There are various cluster settings that needs to be exposed to the end-users to enable these features.The most urgent is "onlyrunonmlnode" which is set to "true" in Aiven Clusters. This needs to be set to "false" to allow ML workloads to be executed on any node (until we have the capability to assign dedicated ML-nodes in the cluster)
There are more configs that…
5 votes -
Set BASE_URL for Opensearch dashboards
As Opensearch Administrator
I want to be able to set custom Base_URLs for my OpenSearch clusters
so that I can simplify usage for my customers when they have multiple OS services per group/client/customer.1 vote -
PPL & Observability SIEM features in OpenSearch
As a Security Analyst,
I want to utilise the Observability plugin & PPL in an efficient manner. Features such as tooltips and autocomplete would help a lot, as well as bug fixes and regular updates.
The syntax is not well nor widely understood, and there are lingering bugs, which for a user are very hard to duplicate across the microcosm of repositories which bundle into the suite.
Observabilitiy & PPL feels like a very promising place for OpenSearch to become more useful to a security operations team, where currently the capabilities are extremely limiting.
While OpenSearch Dashboards, with tenancy and…
2 votes
- Don't see your idea?