Making make install work better by default.
#2004
Conversation
| @@ -68,7 +68,7 @@ jobs: | |||
| ~/.cargo/git | |||
| - name: Install | |||
| run: | | |||
| make install | |||
| make install-cpu | |||
There was a problem hiding this comment.
Simple server tests are running on non accelerated hardwared, therefore cannot install the kernels.
| @@ -19,6 +19,10 @@ tokenizers = { version = "0.19.1", features = ["http"] } | |||
| hf-hub = { version = "0.3.1", features = ["tokio"] } | |||
|
|
|||
| [profile.release] | |||
| incremental = true | |||
There was a problem hiding this comment.
This speeds up devs by improving significantly build times.
| @@ -193,7 +193,7 @@ COPY --from=flash-att-builder /usr/src/flash-attention/csrc/layer_norm/build/lib | |||
| COPY --from=flash-att-builder /usr/src/flash-attention/csrc/rotary/build/lib.linux-x86_64-cpython-310 /opt/conda/lib/python3.10/site-packages | |||
|
|
|||
| # Copy build artifacts from flash attention v2 builder | |||
| COPY --from=flash-att-v2-builder /usr/src/flash-attention-v2/build/lib.linux-x86_64-cpython-310 /opt/conda/lib/python3.10/site-packages | |||
| COPY --from=flash-att-v2-builder /opt/conda/lib/python3.10/site-packages/flash_attn_2_cuda.cpython-310-x86_64-linux-gnu.so /opt/conda/lib/python3.10/site-packages | |||
There was a problem hiding this comment.
pip installing means the library is in this location instead.
| cd integration-tests && pip install -r requirements.txt | ||
| cd clients/python && pip install . | ||
| install-server-cpu: | ||
| cd server && make install-server |
There was a problem hiding this comment.
BUILD_EXTENSIONS is removed (it's not used anymore anyway, only for bloom and it's not even guaranteed to work on all hardware)
| @@ -10,18 +10,26 @@ unit-tests: | |||
|
|
|||
| gen-server: | |||
| # Compile protos | |||
| pip install grpcio-tools==1.51.1 mypy-protobuf==3.4.0 'types-protobuf>=3.20.4' --no-cache-dir | |||
| pip install grpcio-tools==1.62.2 mypy-protobuf==3.6.0 'types-protobuf' --no-cache-dir | |||
There was a problem hiding this comment.
In line with pyproject.toml
| cd flash-attention/csrc/rotary && python setup.py build | ||
| cd flash-attention/csrc/layer_norm && python setup.py build | ||
| build-flash-attention: | ||
| if [ ! -d 'flash-attention' ]; then \ |
There was a problem hiding this comment.
Very dumb protection for recompiling all the time, at least it doesn't fail when you rerun the command.
There was a problem hiding this comment.
Keeing build- variants only because of the dockerfiles.
| flash_att_v2_commit_rocm := 2554f490101742ccdc56620a938f847f61754be6 | ||
|
|
||
| build-flash-attention-v2-cuda: | ||
| pip install -U packaging wheel |
There was a problem hiding this comment.
Wheel is necessary and flash-attn doesn't respect pep517 (Dao-AILab/flash-attention#453) so nothing really better is doable I feel
What does this PR do?
Making
make installa much better sane default to start local dev environments.Fixes # (issue)
Before submitting
Pull Request section?
to it if that's the case.
documentation guidelines, and
here are tips on formatting docstrings.
Who can review?
Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.