Adding support to Azure API #8
Conversation
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #8 +/- ##
===========================================
+ Coverage 60.98% 96.78% +35.80%
===========================================
Files 33 38 +5
Lines 2048 1246 -802
===========================================
- Hits 1249 1206 -43
+ Misses 799 40 -759 ☔ View full report in Codecov by Sentry. |
txt =
Error using [matlab.net.http.RequestMessage/sendOneRequest](matlab:matlab.lang.internal.introspective.errorDocCallback('matlab.net.http.RequestMessage/sendOneRequest', 'E:\jobarchive\Bdeepcore\2024_05_30_h08m36s03_job2628741_pass\matlab\toolbox\matlab\external\interfaces\webservices\http+matlab+net+http\RequestMessage.m', 1348)) ([line 1348](matlab: opentoline('E:\jobarchive\Bdeepcore\2024_05_30_h08m36s03_job2628741_pass\matlab\toolbox\matlab\external\interfaces\webservices\http+matlab+net+http\RequestMessage.m',1348,0))) Error in [matlab.net.http.RequestMessage/sendAfterChallenge](matlab:matlab.lang.internal.introspective.errorDocCallback('matlab.net.http.RequestMessage/sendAfterChallenge', 'E:\jobarchive\Bdeepcore\2024_05_30_h08m36s03_job2628741_pass\matlab\toolbox\matlab\external\interfaces\webservices\http+matlab+net+http\RequestMessage.m', 1681)) ([line 1681](matlab: opentoline('E:\jobarchive\Bdeepcore\2024_05_30_h08m36s03_job2628741_pass\matlab\toolbox\matlab\external\interfaces\webservices\http+matlab+net+http\RequestMessage.m',1681,0))) |
There was a problem hiding this comment.
Our +llms/internal functions have quite low coverage. I would be great to add more tests:
https://app.codecov.io/gh/matlab-deep-learning/llms-with-matlab/tree/AzureAPI/%2Bllms%2F%2Binternal
ollama sends a (correct) header of Content-Type: application/ndjson, so we need to switch from StreamConsumer to BinaryConsumer, or the response will never get dispatched. ollama uses a slightly different JSON format, need to branch based on presence of fields. Azure sends JSON broken up in several packets. We remember incomplete lines in the object and reassemble when the next response comes in. Azure sends "choices":[]. Handle by ignoring that line.
Removed the positional arguments from `azureChat` constructor, and replaced with NVPs that default to reading from the environment instead. Updated instructions to include those environment variables in `.env`.
| URL = "http://localhost:11434/api/chat"; % TODO: model parameter | ||
|
|
||
| % The JSON for StopSequences must have an array, and cannot say "stop": "foo". | ||
| % The easiest way to ensure that is to never pass in a scalar … |
There was a problem hiding this comment.
How about: "To ensure that the JSON for StopSequences has a non-scalar array (is a non-scalar array?), create a two-element array if a scalar is passed."
There was a problem hiding this comment.
No, in JSON, a one-element array is still an array and would be fine. MATLAB's jsonencode just doesn't create "stop": ["foo"] as we need it to. But we can make it produce "stop": ["foo","foo"] instead.
|
|
||
| ```matlab | ||
| mdl = openAIImages(ModelName="dall-e-3"); | ||
| images = generate(mdl,"Create a 3D avatar of a whimsical sushi on the beach. He is decorated with various sushi elements and is playfully interacting with the beach environment."); |
There was a problem hiding this comment.
This is delightful.
Having the test points in alphabetic order makes it easier to visually compare with the file system and verify all example files are being tested.
ollamaChat.m
Outdated
| @@ -61,11 +61,6 @@ | |||
| % | |||
| % SystemPrompt - System prompt. | |||
|
|
|||
| % Ollama model properties not exposed: | |||
| % repeat_last_n, repeat_penalty - could not find an example where they made a difference | |||
| % mirostat, mirostat_eta, mirostat_tau - looking for the best API design | |||
There was a problem hiding this comment.
Can we document this somewhere internal so we don't forget about it?
We had CI-only problems with the tfs_z=0 test files. Log the installed Ollama version to help locate differences between dev environment and CI run.
Ollama fixed reacting to Seed in 0.1.43, add a test point.
It is not yet clear what causes Ollama to not be reliable in these situations, but we do see differences. Will need to analyse separately and report upstream.
Closes #5