Skip to content

Commit 64a9730

Browse files
authored
doc: Add supported task types to schema builder omission (#4584)
* Add task types supported in HF_TASK metadata override documentation * Formatting changes * Minor update
1 parent 48faa9d commit 64a9730

File tree

1 file changed

+10
-5
lines changed

1 file changed

+10
-5
lines changed

src/sagemaker/serve/builder/model_builder.py

Lines changed: 10 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -141,6 +141,9 @@ class ModelBuilder(Triton, DJL, JumpStart, TGI, Transformers):
141141
The schema builder translates the input into bytes and converts the response
142142
into a stream. All translations between the server and the client are handled
143143
automatically with the specified input and output.
144+
The schema builder can be omitted for HuggingFace models with task types TextGeneration,
145+
TextClassification, and QuestionAnswering. Omitting SchemaBuilder is in
146+
beta for FillMask, and AutomaticSpeechRecognition use-cases.
144147
model (Optional[Union[object, str]): Model object (with ``predict`` method to perform
145148
inference) or a HuggingFace/JumpStart Model ID. Either ``model`` or ``inference_spec``
146149
is required for the model builder to build the artifact.
@@ -164,10 +167,11 @@ class ModelBuilder(Triton, DJL, JumpStart, TGI, Transformers):
164167
``TORCHSERVE``, ``MMS``, ``TENSORFLOW_SERVING``, ``DJL_SERVING``,
165168
``TRITON``, and``TGI``.
166169
model_metadata (Optional[Dict[str, Any]): Dictionary used to override model metadata.
167-
Currently, ``HF_TASK`` is overridable for HuggingFace model. ``MLFLOW_MODEL_PATH``
168-
is available for providing local path or s3 path to MLflow artifacts. However,
169-
``MLFLOW_MODEL_PATH`` is experimental and is not intended for production use
170-
at this moment.
170+
Currently, ``HF_TASK`` is overridable for HuggingFace model. HF_TASK should be set for
171+
new models without task metadata in the Hub, adding unsupported task types will throw
172+
an exception. ``MLFLOW_MODEL_PATH`` is available for providing local path or s3 path
173+
to MLflow artifacts. However, ``MLFLOW_MODEL_PATH`` is experimental and is not
174+
intended for production use at this moment.
171175
"""
172176

173177
model_path: Optional[str] = field(
@@ -268,7 +272,8 @@ class ModelBuilder(Triton, DJL, JumpStart, TGI, Transformers):
268272
default=None,
269273
metadata={
270274
"help": "Define the model metadata to override, currently supports `HF_TASK`, "
271-
"`MLFLOW_MODEL_PATH`"
275+
"`MLFLOW_MODEL_PATH`. HF_TASK should be set for new models without task metadata in "
276+
"the Hub, Adding unsupported task types will throw an exception"
272277
},
273278
)
274279

0 commit comments

Comments
 (0)