Query
The query nodes are responsible for taking a user query, fanning this out to all indexers and cache nodes that may have matching results based on service name and timestamp, and then aggregating the results before returning to the user.
System design
The query service is a stateless service that contains no data. It executes the following actions for each query:
Receives a query from the UI
Locates the relevant shards using the metadata from the cluster manager.
Performs a scatter-gather operation across the shards in different tiers (index and cache)
Returns the response to the UI.
The query service exposes an OpenSearch API over HTTP so existing UIs like Kibana/Grafana can query the data in Astra.
Search protocol
Search request path
For every search request:
Identify the partitions for that service in the time range.
Identify the snapshots generated by those partitions in that time range. For each snapshot pick a replica for that snapshot. If a live and non-live replica for a snapshot exists, prefer non-live snapshot.
Query the search nodes for the matched snapshots and get their URLs.
Query the URLs, merge their results and return the response to the user.
Basic operation
Build & deploy
Query nodes can safely be scaled up or down depending on query load, and require no special considerations around deployment speed or updating configurations.