Search Index, gRPC: SearchIndexService.List

Статья создана

Yandex Cloud

Обновлена 17 декабря 2024 г.

gRPC request
ListSearchIndicesRequest
ListSearchIndicesResponse
SearchIndex
ExpirationConfig
TextSearchIndex
ChunkingStrategy
StaticChunkingStrategy
VectorSearchIndex
HybridSearchIndex
CombinationStrategy
MeanCombinationStrategy
ReciprocalRankFusionCombinationStrategy

List search indexes in a specific folder.

gRPC request

rpc List (ListSearchIndicesRequest) returns (ListSearchIndicesResponse)

ListSearchIndicesRequest

{
  "folder_id": "string",
  "page_size": "int64",
  "page_token": "string"
}

Request message for listing search indexes in a specific folder.

Field	Description
folder_id	string Required field. Folder ID from which to list search indexes.
page_size	int64 Maximum number of threads to return per page.
page_token	string Token to retrieve the next page of results.

ListSearchIndicesResponse

{
  "indices": [
    {
      "id": "string",
      "folder_id": "string",
      "name": "string",
      "description": "string",
      "created_by": "string",
      "created_at": "google.protobuf.Timestamp",
      "updated_by": "string",
      "updated_at": "google.protobuf.Timestamp",
      "expiration_config": {
        "expiration_policy": "ExpirationPolicy",
        "ttl_days": "int64"
      },
      "expires_at": "google.protobuf.Timestamp",
      "labels": "map<string, string>",
      // Includes only one of the fields `text_search_index`, `vector_search_index`, `hybrid_search_index`
      "text_search_index": {
        "chunking_strategy": {
          // Includes only one of the fields `static_strategy`
          "static_strategy": {
            "max_chunk_size_tokens": "int64",
            "chunk_overlap_tokens": "int64"
          }
          // end of the list of possible fields
        }
      },
      "vector_search_index": {
        "doc_embedder_uri": "string",
        "query_embedder_uri": "string",
        "chunking_strategy": {
          // Includes only one of the fields `static_strategy`
          "static_strategy": {
            "max_chunk_size_tokens": "int64",
            "chunk_overlap_tokens": "int64"
          }
          // end of the list of possible fields
        }
      },
      "hybrid_search_index": {
        "text_search_index": {
          "chunking_strategy": {
            // Includes only one of the fields `static_strategy`
            "static_strategy": {
              "max_chunk_size_tokens": "int64",
              "chunk_overlap_tokens": "int64"
            }
            // end of the list of possible fields
          }
        },
        "vector_search_index": {
          "doc_embedder_uri": "string",
          "query_embedder_uri": "string",
          "chunking_strategy": {
            // Includes only one of the fields `static_strategy`
            "static_strategy": {
              "max_chunk_size_tokens": "int64",
              "chunk_overlap_tokens": "int64"
            }
            // end of the list of possible fields
          }
        },
        "chunking_strategy": {
          // Includes only one of the fields `static_strategy`
          "static_strategy": {
            "max_chunk_size_tokens": "int64",
            "chunk_overlap_tokens": "int64"
          }
          // end of the list of possible fields
        },
        "normalization_strategy": "NormalizationStrategy",
        "combination_strategy": {
          // Includes only one of the fields `mean_combination`, `rrf_combination`
          "mean_combination": {
            "mean_evaluation_technique": "MeanEvaluationTechnique",
            "weights": [
              "double"
            ]
          },
          "rrf_combination": {
            "k": "google.protobuf.Int64Value"
          }
          // end of the list of possible fields
        }
      }
      // end of the list of possible fields
    }
  ],
  "next_page_token": "string"
}

Response message for the list operation.

Field

Description

indices[]

SearchIndex

List of search indexes in the specified folder.

next_page_token

string

Token to retrieve the next page of results.

SearchIndex

Represents a search index used to store and query data, either using traditional keyword-based text search or vector-based search mechanisms.

Field	Description
id	string Unique identifier of the search index.
folder_id	string ID of the folder that the search index belongs to.
name	string Name of the search index.
description	string Description of the search index.
created_by	string Identifier of the subject who created this search index.
created_at	google.protobuf.Timestamp Timestamp representing when the search index was created.
updated_by	string Identifier of the subject who last updated this search index.
updated_at	google.protobuf.Timestamp Timestamp representing the last time this search index was updated.
expiration_config	ExpirationConfig Configuration for the expiration of the search index, defining when and how the search index will expire.
expires_at	google.protobuf.Timestamp Timestamp representing when the search index will expire.
labels	object (map<string, string>) Set of key-value pairs that can be used to organize and categorize the search index.
text_search_index	TextSearchIndex Keyword-based text search index configuration. This type of index is used for traditional text search, where documents are indexed based on their keywords. Includes only one of the fields `text_search_index`, `vector_search_index`, `hybrid_search_index`. Type of the search index. It can be either a traditional keyword-based text search or a vector-based search.
vector_search_index	VectorSearchIndex Vector-based search index configuration. This type is used for vector search, where documents are indexed using vector embeddings. Includes only one of the fields `text_search_index`, `vector_search_index`, `hybrid_search_index`. Type of the search index. It can be either a traditional keyword-based text search or a vector-based search.
hybrid_search_index	HybridSearchIndex Hybrid (vector-based + keyword-based) search index configuration This type is used for hybrid search, where documents are indexed using both keyword-based and vector-based search mechanisms. Includes only one of the fields `text_search_index`, `vector_search_index`, `hybrid_search_index`. Type of the search index. It can be either a traditional keyword-based text search or a vector-based search.

ExpirationConfig

Field

Description

expiration_policy

enum ExpirationPolicy

EXPIRATION_POLICY_UNSPECIFIED
STATIC
SINCE_LAST_ACTIVE

ttl_days

int64

TextSearchIndex

Defines the configuration for a traditional keyword-based text search index.

Field

Description

chunking_strategy

ChunkingStrategy

Chunking strategy used to split text into smaller chunks before indexing.
In the case of text search, tokens are individual text characters.

ChunkingStrategy

Defines a general strategy for chunking text into smaller segments.
Currently, only StaticChunkingStrategy is supported.

Field

Description

static_strategy

StaticChunkingStrategy

Includes only one of the fields static_strategy.

StaticChunkingStrategy

Defines a chunking strategy where chunks are created with a fixed maximum chunk size and an overlap between consecutive chunks.

Field

Description

max_chunk_size_tokens

int64

The maximum number of tokens allowed in a single chunk.
Constraints: must be within the range [100, 2048].
Default value: 800

chunk_overlap_tokens

int64

The number of tokens that should overlap between consecutive chunks.
This allows for some context from the previous chunk to be included in the next chunk.
Constraints: must be less than or equal to half of max_chunk_size_tokens.
Default value: 400

VectorSearchIndex

Defines the configuration for a vector-based search index. This type uses embeddings to represent documents and queries.

Field	Description
doc_embedder_uri	string The ID of the model to be used for obtaining document text embeddings.
query_embedder_uri	string The ID of the model to be used for obtaining query text embeddings.
chunking_strategy	ChunkingStrategy Chunking strategy used to split text into smaller chunks before indexing. In the case of vector search, tokens are produced by the tokenizer from the embedding model.

HybridSearchIndex

Defines the configuration for a hybrid (vector-based + keyword-based) search index. This type uses both embeddings and keyword-based search to represent documents and queries.

Field	Description
text_search_index	TextSearchIndex Configuration for a traditional keyword-based text search index.
vector_search_index	VectorSearchIndex Configuration for a vector-based search index.
chunking_strategy	ChunkingStrategy Common chunking strategy that applies to both text and vector search indexes. If provided, it overrides the individual chunking strategies in both `text_search_index` and `vector_search_index`. In this case, both text and vector search will use token-based chunking, where tokens are produced by the tokenizer of the embedding model.
normalization_strategy	enum NormalizationStrategy Normalization strategy for relevance scores from different indices. Default is MIN_MAX_STRATEGY `NORMALIZATION_STRATEGY_UNSPECIFIED` `MIN_MAX`: https://en.wikipedia.org/wiki/Feature_scaling#Rescaling_(min-max_normalization) `L2`: https://en.wikipedia.org/wiki/Cosine_similarity#L2-normalized_Euclidean_distance
combination_strategy	CombinationStrategy Combination strategy for merging rankings from different indices. Default is arithmetic mean

CombinationStrategy

Combination strategy for merging rankings from different indices

Field

Description

mean_combination

MeanCombinationStrategy

Includes only one of the fields mean_combination, rrf_combination.

rrf_combination

ReciprocalRankFusionCombinationStrategy

Includes only one of the fields mean_combination, rrf_combination.

MeanCombinationStrategy

Field

Description

mean_evaluation_technique

enum MeanEvaluationTechnique

Technique for averaging relevance scores from different indices. Default is ARITHMETIC

MEAN_EVALUATION_TECHNIQUE_UNSPECIFIED
ARITHMETIC: https://en.wikipedia.org/wiki/Arithmetic_mean
GEOMETRIC: https://en.wikipedia.org/wiki/Geometric_mean
HARMONIC: https://en.wikipedia.org/wiki/Harmonic_mean

weights[]

double

Weights used for evaluating the weighted mean of relevance scores. The sum of the values must equal 1.0
If not provided, all scores are given equal weight

ReciprocalRankFusionCombinationStrategy

https://plg.uwaterloo.ca/~gvcormac/cormacksigir09-rrf.pdf

Field

Description

google.protobuf.Int64Value

The parameter k for RRFscore. Default is 60

Search Index, gRPC: SearchIndexService.List

gRPC requestgRPC request

ListSearchIndicesRequestListSearchIndicesRequest

ListSearchIndicesResponseListSearchIndicesResponse

SearchIndexSearchIndex

ExpirationConfigExpirationConfig

TextSearchIndexTextSearchIndex

ChunkingStrategyChunkingStrategy

StaticChunkingStrategyStaticChunkingStrategy

VectorSearchIndexVectorSearchIndex

HybridSearchIndexHybridSearchIndex

CombinationStrategyCombinationStrategy

MeanCombinationStrategyMeanCombinationStrategy

ReciprocalRankFusionCombinationStrategyReciprocalRankFusionCombinationStrategy

Была ли статья полезна?

gRPC request

ListSearchIndicesRequest

ListSearchIndicesResponse

SearchIndex

ExpirationConfig

TextSearchIndex

ChunkingStrategy

StaticChunkingStrategy

VectorSearchIndex

HybridSearchIndex

CombinationStrategy

MeanCombinationStrategy

ReciprocalRankFusionCombinationStrategy