You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Update on "[llava][21/N] Add llava runner test binary and build script"
Add a `main.cpp` and CMakeLists.txt for llava runner. This runner takes in an image in the format of `.pt` (a serialized pytorch model) along with text prompt. It will generate text tokens in a way similar to llama runner.
Run `build.sh` to build the runner.
To serialize the image into a `.pt` file, run the following script:
```python
import torch
from torch import nn
copy = torch.tensor(resized)
m = nn.Module()
par = nn.Parameter(copy, requires_grad=False)
m.register_parameter("0",par)
tensors = torch.jit.script(m)
tensors.save("image.pt")
```
To run the runner, use the following command:
```
cmake-out/examples/models/llava/llava_main \
--tokenizer_path tokenizer.bin \
--model_path llava_kv_768.pte \
--prompt "\nWhat are the things I should be cautious about when I visit here?" \
--image_path image.pt \
--temperature 0
```
Differential Revision: [D61146432](https://www.internalfb.com/diff/D61146432)
[ghstack-poisoned]
0 commit comments