Adding lm-eval 0.3.0 as an install req in llama2 (#2394)

Summary:
Pull Request resolved: #2394

Adding install reqs for enabling model evaluation in llama2.

The versions are intentional due to conflicts with ET install reqs

pip install datasets==2.14.0
pip install lm-eval==0.3.0

bypass-github-export-checks
bypass-github-pytorch-ci-checks
bypass-github-executorch-ci-checks

Reviewed By: jerryzh168

Differential Revision: D54839623

fbshipit-source-id: 0ccf588172a707de6c16c693179946b64c428465

Author: Jack-Khuu

examples/models/llama2/install_requirement_helper.py

@@ -0,0 +1,29 @@
+# Copyright (c) Meta Platforms, Inc. and affiliates.
+# All rights reserved.
+#
+# This source code is licensed under the BSD-style license found in the
+# LICENSE file in the root directory of this source tree.
+
+"""
+This removes the imported [examples] module from pip installing lm_eval due to
+colliding module name with ET package
+"""
+
+try:
+    # If the import fails, this means there's nothing to remove
+    import examples
+
+    try:
+        # If the import succeeds, this means it isn't using lm_eval's module
+        import examples.models
+    except:
+        print(
+            "Failed to import examples.models due to lm_eval conflict. Removing lm_eval examples module"
+        )
+        import shutil
+
+        examples_path = examples.__path__[0]
+        shutil.rmtree(examples_path)
+
+except:
+    pass

examples/models/llama2/install_requirements.sh

@@ -9,3 +9,14 @@
 # Install sentencepiece for llama tokenizer
 pip install snakeviz sentencepiece
 pip install torchao-nightly
+
+# Install datasets for HuggingFace dataloader
+# v2.14.0 is intentional to force lm-eval v0.3.0 compatibility
+pip install datasets==2.14.0
+
+# Install lm-eval for Model Evaluation with lm-evalution-harness
+# v0.3.0 is intentional
+pip install lm-eval==0.3.
+
+# Call the install helper for further setup
+python examples/models/llama2/install_requirement_helper.py