feat: integrate weaviate #581
Conversation
This reverts commit 3bd543f.
…/ragbits into 347-feat-integrate-weaviate
|
Trivy scanning results. Report Summary ┌──────────────────────┬──────┬─────────────────┬─────────┐
For OSS Maintainers: VEX NoticeIf you're an OSS maintainer and Trivy has detected vulnerabilities in your project that you believe are not actually exploitable, consider issuing a VEX (Vulnerability Exploitability eXchange) statement. To disable this notice, set the TRIVY_DISABLE_VEX_NOTICE environment variable. uv.lock (uv)Total: 16 (MEDIUM: 8, HIGH: 6, CRITICAL: 2) ┌──────────────┬────────────────┬──────────┬────────┬───────────────────┬───────────────┬──────────────────────────────────────────────────────────────┐ |
|
Code Coverage SummaryDiff against mainResults for commit: 2962e25 Minimum allowed coverage is ♻️ This comment has been updated with latest results |
| from weaviate import WeaviateAsyncClient | ||
| from weaviate.classes.config import Configure, VectorDistances | ||
| from weaviate.classes.query import Filter, MetadataQuery | ||
| from weaviate.collections.classes.filters import _Filters |
There was a problem hiding this comment.
It seems there is a public alias in weaviate.collections.classes.filters called FilterReturn - maybe we should use it instead?
There was a problem hiding this comment.
Changed to FilterReturn
| wvc.data.DataObject( | ||
| uuid=str(entry.id), | ||
| properties=self._flatten_metadata( | ||
| entry.model_dump(exclude={"id"}, exclude_none=True, mode="json") # type: ignore |
There was a problem hiding this comment.
I fixed all these mypy issues
|
|
||
| outputs_results = [] | ||
| for object_ in results.objects: | ||
| entry_raw = {"uuid": object_.uuid, "properties": self._unflatten_metadata(object_.properties)} # type: ignore |
packages/ragbits-core/pyproject.toml
Outdated
| @@ -89,6 +89,10 @@ hf = [ | |||
| s3 = [ | |||
| "boto3>=1.35.42,<2.0.0", | |||
| ] | |||
| weaviate = [ | |||
| "weaviate-client>=4.9.6", | |||
| "psycopg[binary,pool]>=3.2.6", | |||
There was a problem hiding this comment.
There was some error with protobuf after I installed weaviate and I had to add this package to make it work
There was a problem hiding this comment.
also caught my attention, do you remember what the error was?
There was a problem hiding this comment.
I think it was similar to this: ImportError: cannot import name 'runtime_version' from 'google.protobuf' but I don't remember exactly where I found the solution
There was a problem hiding this comment.
i can't reproduce it, i removed this dep and the example works fine
There was a problem hiding this comment.
I had some issues with env so maybe this issue resulted from something being broken in my dependencies
There was a problem hiding this comment.
Removed it and code works fine
| Enables the WeaviateVectorStore to be pickled and unpickled. | ||
| """ | ||
| # TODO: To be implemented. Required for Ray processing. | ||
| raise NotImplementedError |
There was a problem hiding this comment.
can we implement this? if there is a way to do it in a similar way to qdrant, i'd do it here
No description provided.