Skip to content

Commit 980ae08

Browse files
author
AWS
committed
Amazon Bedrock Runtime Update: This release introduces our latest bedrock runtime API, InvokeModelWithBidirectionalStream. The API supports both input and output streams and is supported by only HTTP2.0.
1 parent 43e67de commit 980ae08

File tree

2 files changed

+136
-5
lines changed

2 files changed

+136
-5
lines changed

‎.changes/next-release/feature-AmazonBedrockRuntime-43d956e.json

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
{
2+
"type": "feature",
3+
"category": "Amazon Bedrock Runtime",
4+
"contributor": "",
5+
"description": "This release introduces our latest bedrock runtime API, InvokeModelWithBidirectionalStream. The API supports both input and output streams and is supported by only HTTP2.0."
6+
}

‎services/bedrockruntime/src/main/resources/codegen-resources/service-2.json

Lines changed: 130 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -43,8 +43,8 @@
4343
"output":{"shape":"ConverseResponse"},
4444
"errors":[
4545
{"shape":"AccessDeniedException"},
46-
{"shape":"ResourceNotFoundException"},
4746
{"shape":"ThrottlingException"},
47+
{"shape":"ResourceNotFoundException"},
4848
{"shape":"ModelTimeoutException"},
4949
{"shape":"InternalServerException"},
5050
{"shape":"ServiceUnavailableException"},
@@ -65,8 +65,8 @@
6565
"output":{"shape":"ConverseStreamResponse"},
6666
"errors":[
6767
{"shape":"AccessDeniedException"},
68-
{"shape":"ResourceNotFoundException"},
6968
{"shape":"ThrottlingException"},
69+
{"shape":"ResourceNotFoundException"},
7070
{"shape":"ModelTimeoutException"},
7171
{"shape":"InternalServerException"},
7272
{"shape":"ServiceUnavailableException"},
@@ -110,12 +110,36 @@
110110
{"shape":"InternalServerException"},
111111
{"shape":"ServiceUnavailableException"},
112112
{"shape":"ValidationException"},
113-
{"shape":"ModelNotReadyException"},
114113
{"shape":"ServiceQuotaExceededException"},
114+
{"shape":"ModelNotReadyException"},
115115
{"shape":"ModelErrorException"}
116116
],
117117
"documentation":"<p>Invokes the specified Amazon Bedrock model to run inference using the prompt and inference parameters provided in the request body. You use model inference to generate text, images, and embeddings.</p> <p>For example code, see <i>Invoke model code examples</i> in the <i>Amazon Bedrock User Guide</i>. </p> <p>This operation requires permission for the <code>bedrock:InvokeModel</code> action.</p> <important> <p>To deny all inference access to resources that you specify in the modelId field, you need to deny access to the <code>bedrock:InvokeModel</code> and <code>bedrock:InvokeModelWithResponseStream</code> actions. Doing this also denies access to the resource through the Converse API actions (<a href=\"https://docs.aws.amazon.com/bedrock/latest/APIReference/API_runtime_Converse.html\">Converse</a> and <a href=\"https://docs.aws.amazon.com/bedrock/latest/APIReference/API_runtime_ConverseStream.html\">ConverseStream</a>). For more information see <a href=\"https://docs.aws.amazon.com/bedrock/latest/userguide/security_iam_id-based-policy-examples.html#security_iam_id-based-policy-examples-deny-inference\">Deny access for inference on specific models</a>. </p> </important> <p>For troubleshooting some of the common errors you might encounter when using the <code>InvokeModel</code> API, see <a href=\"https://docs.aws.amazon.com/bedrock/latest/userguide/troubleshooting-api-error-codes.html\">Troubleshooting Amazon Bedrock API Error Codes</a> in the Amazon Bedrock User Guide</p>"
118118
},
119+
"InvokeModelWithBidirectionalStream":{
120+
"name":"InvokeModelWithBidirectionalStream",
121+
"http":{
122+
"method":"POST",
123+
"requestUri":"/model/{modelId}/invoke-with-bidirectional-stream",
124+
"responseCode":200
125+
},
126+
"input":{"shape":"InvokeModelWithBidirectionalStreamRequest"},
127+
"output":{"shape":"InvokeModelWithBidirectionalStreamResponse"},
128+
"errors":[
129+
{"shape":"AccessDeniedException"},
130+
{"shape":"ResourceNotFoundException"},
131+
{"shape":"ThrottlingException"},
132+
{"shape":"ModelTimeoutException"},
133+
{"shape":"InternalServerException"},
134+
{"shape":"ServiceUnavailableException"},
135+
{"shape":"ModelStreamErrorException"},
136+
{"shape":"ValidationException"},
137+
{"shape":"ServiceQuotaExceededException"},
138+
{"shape":"ModelNotReadyException"},
139+
{"shape":"ModelErrorException"}
140+
],
141+
"documentation":"<p>Invoke the specified Amazon Bedrock model to run inference using the bidirectional stream. The response is returned in a stream that remains open for 8 minutes. A single session can contain multiple prompts and responses from the model. The prompts to the model are provided as audio files and the model's responses are spoken back to the user and transcribed.</p> <p>It is possible for users to interrupt the model's response with a new prompt, which will halt the response speech. The model will retain contextual awareness of the conversation while pivoting to respond to the new prompt.</p>"
142+
},
119143
"InvokeModelWithResponseStream":{
120144
"name":"InvokeModelWithResponseStream",
121145
"http":{
@@ -134,8 +158,8 @@
134158
{"shape":"ServiceUnavailableException"},
135159
{"shape":"ModelStreamErrorException"},
136160
{"shape":"ValidationException"},
137-
{"shape":"ModelNotReadyException"},
138161
{"shape":"ServiceQuotaExceededException"},
162+
{"shape":"ModelNotReadyException"},
139163
{"shape":"ModelErrorException"}
140164
],
141165
"documentation":"<p>Invoke the specified Amazon Bedrock model to run inference using the prompt and inference parameters provided in the request body. The response is returned in a stream.</p> <p>To see if a model supports streaming, call <a href=\"https://docs.aws.amazon.com/bedrock/latest/APIReference/API_GetFoundationModel.html\">GetFoundationModel</a> and check the <code>responseStreamingSupported</code> field in the response.</p> <note> <p>The CLI doesn't support streaming operations in Amazon Bedrock, including <code>InvokeModelWithResponseStream</code>.</p> </note> <p>For example code, see <i>Invoke model with streaming code example</i> in the <i>Amazon Bedrock User Guide</i>. </p> <p>This operation requires permissions to perform the <code>bedrock:InvokeModelWithResponseStream</code> action. </p> <important> <p>To deny all inference access to resources that you specify in the modelId field, you need to deny access to the <code>bedrock:InvokeModel</code> and <code>bedrock:InvokeModelWithResponseStream</code> actions. Doing this also denies access to the resource through the Converse API actions (<a href=\"https://docs.aws.amazon.com/bedrock/latest/APIReference/API_runtime_Converse.html\">Converse</a> and <a href=\"https://docs.aws.amazon.com/bedrock/latest/APIReference/API_runtime_ConverseStream.html\">ConverseStream</a>). For more information see <a href=\"https://docs.aws.amazon.com/bedrock/latest/userguide/security_iam_id-based-policy-examples.html#security_iam_id-based-policy-examples-deny-inference\">Deny access for inference on specific models</a>. </p> </important> <p>For troubleshooting some of the common errors you might encounter when using the <code>InvokeModelWithResponseStream</code> API, see <a href=\"https://docs.aws.amazon.com/bedrock/latest/userguide/troubleshooting-api-error-codes.html\">Troubleshooting Amazon Bedrock API Error Codes</a> in the Amazon Bedrock User Guide</p>"
@@ -168,8 +192,8 @@
168192
"output":{"shape":"StartAsyncInvokeResponse"},
169193
"errors":[
170194
{"shape":"AccessDeniedException"},
171-
{"shape":"ResourceNotFoundException"},
172195
{"shape":"ThrottlingException"},
196+
{"shape":"ResourceNotFoundException"},
173197
{"shape":"InternalServerException"},
174198
{"shape":"ServiceUnavailableException"},
175199
{"shape":"ValidationException"},
@@ -393,6 +417,30 @@
393417
},
394418
"documentation":"<p>The Model automatically decides if a tool should be called or whether to generate text instead. For example, <code>{\"auto\" : {}}</code>.</p>"
395419
},
420+
"BidirectionalInputPayloadPart":{
421+
"type":"structure",
422+
"members":{
423+
"bytes":{
424+
"shape":"PartBody",
425+
"documentation":"<p>The audio content for the bidirectional input.</p>"
426+
}
427+
},
428+
"documentation":"<p>Payload content for the bidirectional input. The input is an audio stream.</p>",
429+
"event":true,
430+
"sensitive":true
431+
},
432+
"BidirectionalOutputPayloadPart":{
433+
"type":"structure",
434+
"members":{
435+
"bytes":{
436+
"shape":"PartBody",
437+
"documentation":"<p>The speech output of the bidirectional stream.</p>"
438+
}
439+
},
440+
"documentation":"<p>Output from the bidirectional stream. The output is speech and a text transcription.</p>",
441+
"event":true,
442+
"sensitive":true
443+
},
396444
"Blob":{"type":"blob"},
397445
"Body":{
398446
"type":"blob",
@@ -2102,6 +2150,83 @@
21022150
},
21032151
"payload":"body"
21042152
},
2153+
"InvokeModelWithBidirectionalStreamInput":{
2154+
"type":"structure",
2155+
"members":{
2156+
"chunk":{
2157+
"shape":"BidirectionalInputPayloadPart",
2158+
"documentation":"<p>The audio chunk that is used as input for the invocation step.</p>"
2159+
}
2160+
},
2161+
"documentation":"<p>Payload content, the speech chunk, for the bidirectional input of the invocation step.</p>",
2162+
"eventstream":true
2163+
},
2164+
"InvokeModelWithBidirectionalStreamOutput":{
2165+
"type":"structure",
2166+
"members":{
2167+
"chunk":{
2168+
"shape":"BidirectionalOutputPayloadPart",
2169+
"documentation":"<p>The speech chunk that was provided as output from the invocation step.</p>"
2170+
},
2171+
"internalServerException":{
2172+
"shape":"InternalServerException",
2173+
"documentation":"<p>The request encountered an unknown internal error.</p>"
2174+
},
2175+
"modelStreamErrorException":{
2176+
"shape":"ModelStreamErrorException",
2177+
"documentation":"<p>The request encountered an error with the model stream.</p>"
2178+
},
2179+
"validationException":{
2180+
"shape":"ValidationException",
2181+
"documentation":"<p>The input fails to satisfy the constraints specified by an Amazon Web Services service.</p>"
2182+
},
2183+
"throttlingException":{
2184+
"shape":"ThrottlingException",
2185+
"documentation":"<p>The request was denied due to request throttling.</p>"
2186+
},
2187+
"modelTimeoutException":{
2188+
"shape":"ModelTimeoutException",
2189+
"documentation":"<p>The connection was closed because a request was not received within the timeout period.</p>"
2190+
},
2191+
"serviceUnavailableException":{
2192+
"shape":"ServiceUnavailableException",
2193+
"documentation":"<p>The request has failed due to a temporary failure of the server.</p>"
2194+
}
2195+
},
2196+
"documentation":"<p>Output from the bidirectional stream that was used for model invocation.</p>",
2197+
"eventstream":true
2198+
},
2199+
"InvokeModelWithBidirectionalStreamRequest":{
2200+
"type":"structure",
2201+
"required":[
2202+
"modelId",
2203+
"body"
2204+
],
2205+
"members":{
2206+
"modelId":{
2207+
"shape":"InvokeModelIdentifier",
2208+
"documentation":"<p>The model ID or ARN of the model ID to use. Currently, only <code>amazon.nova-sonic-v1:0</code> is supported.</p>",
2209+
"location":"uri",
2210+
"locationName":"modelId"
2211+
},
2212+
"body":{
2213+
"shape":"InvokeModelWithBidirectionalStreamInput",
2214+
"documentation":"<p>The prompt and inference parameters in the format specified in the <code>BidirectionalInputPayloadPart</code> in the header. You must provide the body in JSON format. To see the format and content of the request and response bodies for different models, refer to <a href=\"https://docs.aws.amazon.com/bedrock/latest/userguide/model-parameters.html\">Inference parameters</a>. For more information, see <a href=\"https://docs.aws.amazon.com/bedrock/latest/userguide/api-methods-run.html\">Run inference</a> in the Bedrock User Guide.</p>"
2215+
}
2216+
},
2217+
"payload":"body"
2218+
},
2219+
"InvokeModelWithBidirectionalStreamResponse":{
2220+
"type":"structure",
2221+
"required":["body"],
2222+
"members":{
2223+
"body":{
2224+
"shape":"InvokeModelWithBidirectionalStreamOutput",
2225+
"documentation":"<p>Streaming response from the model in the format specified by the <code>BidirectionalOutputPayloadPart</code> header.</p>"
2226+
}
2227+
},
2228+
"payload":"body"
2229+
},
21052230
"InvokeModelWithResponseStreamRequest":{
21062231
"type":"structure",
21072232
"required":["modelId"],

0 commit comments

Comments
 (0)