Skip to content

RCBC-489: Support for base64 encoded vector types #146

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
May 29, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion ext/couchbase
19 changes: 15 additions & 4 deletions lib/couchbase/search_options.rb
Original file line number Diff line number Diff line change
Expand Up @@ -1071,13 +1071,23 @@ class VectorQuery

# Constructs a +VectorQuery+ instance
#
# @param [String] vector_field_name the document field that contains the vector.
# @param [Array<Float>] vector_query the vector query to run.
# @overload initialize(vector_field_name, vector_query)
# @param [String] vector_field_name the document field that contains the vector.
# @param [Array<Float>] vector_query the vector query.
#
# @yieldparam [MatchPhraseQuery] self
# @overload initialize(vector_field_name, base64_vector_query)
# @param [String] vector_field_name the document field that contains the vector.
# @param [String] base64_vector_query the vector query represented as a base64-encoded sequence of little-endian IEEE 754 floats.
#
# @yieldparam [VectorQuery] self
def initialize(vector_field_name, vector_query)
@vector_field_name = vector_field_name
@vector_query = vector_query

if vector_query.respond_to?(:to_str)
@base64_vector_query = vector_query.to_str
else
@vector_query = vector_query
end

yield self if block_given?
end
Expand All @@ -1092,6 +1102,7 @@ def to_h
{
field: @vector_field_name,
vector: @vector_query,
vector_base64: @base64_vector_query,
k: num_candidates || 3,
boost: boost,
}.compact
Expand Down
9 changes: 9 additions & 0 deletions test/search_test.rb
Original file line number Diff line number Diff line change
Expand Up @@ -81,7 +81,7 @@
options.consistent_with(mutation_state)
options.limit = 100
attempts = 0
loop do

Check failure on line 84 in test/search_test.rb

View workflow job for this annotation

GitHub Actions / test_linux_x86_64 (7.2.4)

test/search_test.rb.test_simple_search

Couchbase::Error::AmbiguousTimeout: unable to perform search query for index "idx-default-82153": : ambiguous_timeout (13), context={"error":"13, ambiguous_timeout (13)","client_context_id":"a6ef45-932c-ee44-aa59-5a87b3725f80db","index_name":"idx-default-82153","query":"{\"query\":\"arthur\"}","parameters":"{\"ctl\":{\"consistency\":{\"level\":\"at_plus\",\"vectors\":{\"idx-default-82153\":{\"705/144982028730334\":60}}},\"timeout\":75000},\"explain\":false,\"query\":{\"query\":\"arthur\"},\"size\":100}","http_status":0,"http_body":"","last_dispatched_to":"172.17.0.2:8094","last_dispatched_from":"172.17.0.1:36082"}...

Check failure on line 84 in test/search_test.rb

View workflow job for this annotation

GitHub Actions / test_linux_x86_64 (7.0.5)

test/search_test.rb.test_simple_search

Couchbase::Error::InternalServerFailure: unable to perform search query for index "idx-default-84745": : internal_server_failure (5), context={"error":"5, internal_server_failure (5)","client_context_id":"2604b4-cd13-c24b-08c2-d5d29fe72e8dc7","index_name":"idx-default-84745","query":"{\"query\":\"arthur\"}","parameters":"{\"ctl\":{\"consistency\":{\"level\":\"at_plus\",\"vectors\":{\"idx-default-84745\":{\"220/183141746100843\":60}}},\"timeout\":75000},\"explain\":false,\"query\":{\"query\":\"arthur\"},\"size\":100}","http_status":503,"http_body":"server write time out.","last_dispatched_to":"172.17.0.4:8094","last_dispatched_from":"172.17.0.1:41048"}...
begin
break if attempts >= 30

Expand Down Expand Up @@ -115,7 +115,7 @@
options.limit = 100
attempts = 0
retry_delay = 0.5
loop do

Check failure on line 118 in test/search_test.rb

View workflow job for this annotation

GitHub Actions / test_linux_x86_64 (7.0.5)

test/search_test.rb.test_doc_id_search_query

Couchbase::Error::InternalServerFailure: unable to perform search query for index "idx-default-26529": : internal_server_failure (5), context={"error":"5, internal_server_failure (5)","client_context_id":"7c801d-bd42-5a41-6d53-3282dc8ac2372f","index_name":"idx-default-26529","query":"{\"ids\":[\"search_test_109_foo_5653314-8536896171\",\"search_test_109_bar_2473314-8536896171\"]}","parameters":"{\"ctl\":{\"consistency\":{\"level\":\"at_plus\",\"vectors\":{\"idx-default-26529\":{\"367/132368809725680\":58,\"548/189499885842538\":60}}},\"timeout\":75000},\"explain\":false,\"query\":{\"ids\":[\"search_test_109_foo_5653314-8536896171\",\"search_test_109_bar_2473314-8536896171\"]},\"size\":100}","http_status":503,"http_body":"server write time out.","last_dispatched_to":"172.17.0.4:8094","last_dispatched_from":"172.17.0.1:58914"}...
begin
break if attempts >= 20

Expand Down Expand Up @@ -237,6 +237,15 @@
assert_equal SearchQuery.match_none.to_json, enc_query
end

def test_vector_search_query_base64
base64_query = "aOeYBEXJ4kI="
enc_vector_query = VectorQuery.new("foo", base64_query).to_h

refute enc_vector_query.key?(:vector)
assert enc_vector_query.key?(:vector_base64)
assert_equal enc_vector_query[:vector_base64], base64_query
end

def test_vector_search_not_supported
skip("#{name}: Server supports vector search") if !env.protostellar? && env.server_version.supports_vector_search?

Expand Down
Loading