|
4 | 4 | "cell_type": "markdown",
|
5 | 5 | "metadata": {},
|
6 | 6 | "source": [
|
7 |
| - "## Function Calling with OpenAI Python Client" |
| 7 | + "# Functions\n", |
| 8 | + "\n", |
| 9 | + "The OpenAI compatbile web server in `llama-cpp-python` supports function calling.\n", |
| 10 | + "\n", |
| 11 | + "Function calling allows API clients to specify a schema that gives the model a format it should respond in.\n", |
| 12 | + "Function calling in `llama-cpp-python` works by combining models pretrained for function calling such as [`functionary`](https://huggingface.co/abetlen/functionary-7b-v1-GGUF) with constrained sampling to produce a response that is compatible with the schema.\n", |
| 13 | + "\n", |
| 14 | + "Note however that this improves but does not guarantee that the response will be compatible with the schema.\n", |
| 15 | + "\n", |
| 16 | + "## Requirements\n", |
| 17 | + "\n", |
| 18 | + "Before we begin you will need the following:\n", |
| 19 | + "\n", |
| 20 | + "- A running `llama-cpp-python` server with a function calling compatible model. [See here](https://llama-cpp-python.readthedocs.io/en/latest/server/#function-calling)\n", |
| 21 | + "- The OpenAI Python Client `pip install openai`\n", |
| 22 | + "- (Optional) The Instructor Python Library `pip install instructor`\n", |
| 23 | + "\n", |
| 24 | + "## Function Calling with OpenAI Python Client\n", |
| 25 | + "\n", |
| 26 | + "We'll start with a basic demo that only uses the OpenAI Python Client." |
8 | 27 | ]
|
9 | 28 | },
|
10 | 29 | {
|
|
27 | 46 | "\n",
|
28 | 47 | "client = openai.OpenAI(\n",
|
29 | 48 | " api_key = \"sk-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx\", # can be anything\n",
|
30 |
| - " base_url = \"http://100.64.159.73:8000/v1\"\n", |
| 49 | + " base_url = \"http://100.64.159.73:8000/v1\" # NOTE: Replace with IP address and port of your llama-cpp-python server\n", |
31 | 50 | ")\n",
|
32 | 51 | "\n",
|
33 | 52 | "# Example dummy function hard coded to return the same weather\n",
|
|
113 | 132 | "source": [
|
114 | 133 | "# Function Calling with Instructor\n",
|
115 | 134 | "\n",
|
116 |
| - "You'll need to install the [`instructor`](https://github.com/jxnl/instructor/) package to run this notebook. You can do so by running the following command in your terminal:\n", |
| 135 | + "The above example is a bit verbose and requires you to manually verify the schema.\n", |
| 136 | + "\n", |
| 137 | + "For our next examples we'll use the `instructor` library to simplify the process and accomplish a number of different tasks with function calling.\n", |
| 138 | + "\n", |
| 139 | + "You'll first need to install the [`instructor`](https://github.com/jxnl/instructor/).\n", |
| 140 | + "\n", |
| 141 | + "You can do so by running the following command in your terminal:\n", |
117 | 142 | "\n",
|
118 | 143 | "```bash\n",
|
119 | 144 | "pip install instructor\n",
|
120 | 145 | "```\n",
|
121 | 146 | "\n",
|
122 |
| - "We'll highlight a few basic examples taken from the [instructor cookbook](https://jxnl.github.io/instructor/)\n", |
| 147 | + "Below we'll go through a few basic examples taken directly from the [instructor cookbook](https://jxnl.github.io/instructor/)\n", |
123 | 148 | "\n",
|
124 | 149 | "## Basic Usage"
|
125 | 150 | ]
|
|
0 commit comments