initial keras-hub support #986
Conversation
martin-gorner
requested review from
osanseviero,
SBrandeis,
gary149,
Wauplin,
julien-c and
pcuenca
as code owners
| # Possible tasks are CausalLM, TextToImage, ImageClassifier, ... | ||
| # full list here: https://keras.io/api/keras_hub/models/#api-documentation |
There was a problem hiding this comment.
do you know if there's a way to know which task/architecture to use, depending on what's inside the repo? (or a manual tag set by keras.push?)
There was a problem hiding this comment.
looking at https://huggingface.co/martin-gorner/llama-3.2-1B-pirate-instruct, it seems the architecture is pushed with the model card:
This model is related to a CausalLM task.
So yeah agree with Julien that if we can have this info in the metadata, it would be awesome so simplify the snippet
There was a problem hiding this comment.
I'm currently investigating this. Yes, this would be the best solution. I do see a text-generation "pipeline-tag" on models that I upload. This is probably what we should use. It won't be part of this PR though.
There was a problem hiding this comment.
😫 right now the pipeline-tag is only populated for CausalLM and TextClassifier models.
https://github.com/keras-team/keras-hub/blob/a45110ee4f04750cf43214b591847cffa1074310/keras_hub/src/utils/preset_utils.py#L320
There was a problem hiding this comment.
btw, the "example use" section on Kaggle is much more complete. For reference: https://www.kaggle.com/models/keras/gemma2
There was a problem hiding this comment.
btw, the "example use" section on Kaggle is much more complete. For reference: https://www.kaggle.com/models/keras/gemma2
I feel that this kind of completeness should be written in the model card directly. For example, https://huggingface.co/google/gemma-2-9b-keras model card could contain code snippets on how to generate and fit on data. What we try to achieve in "use this model" is to have a small, self-contained and personalized snippet to load a model
There was a problem hiding this comment.
+1 on making the snippet more complete; in general, we've been pushing for more complete, end-to-end snippets that people can just copy and run.
|
Not sure about the tests failing. It does not look like it's related to this PR, is it? |
|
(failing tests don't seem related no) |
Co-authored-by: Pedro Cuenca <[email protected]>
There was a problem hiding this comment.
Looks good to merge! :)
To sum-up, follow-up PRs will be about 1. parsing task.json for better code snippets 2. removing keras-nlp library entirely (in a few weeks?).
Failing CI doesn't seem related to this PR. Let's wait for another approval given the discussions in this PR :)
Co-authored-by: Julien Chaumond <[email protected]>
removing filter=false
Initial support for Keras-hub: