restructure (#462)

Summary:

This diff is a preparation for bundled program documentation, including:
1. move bundled program documentation from tutorial/ to sdk/
2. remove the bento notebook, make its content directly in the .md file
3. elementary update to in line with new api

Differential Revision: D49550056

Author: Songhao Jia

docs/source/sdk-bundled-io.md

@@ -1,3 +1,360 @@
-# BundledIO
+# Bundled Program
 
-TBA
+## Introduction
+Bundled Program is a wrapper around the core ExecuTorch program designed to help users wrapping test cases and other related info with the models they deploy. Bundled Program is not necessarily a core part of the program and not needed for its execution but is more necessary for various other use-cases, especially for model correctness evaluation such as e2e testing during model bring-up etc.
+
+Overall procedure can be broken into two stages, and in each stage we are supporting:
+* **Emit stage**: Bundling test I/O cases as well as other useful info in key-value pairs along with the ExecuTorch program.
+* **Runtime stage**: Accessing, executing and verifying the bundled test cases during runtime.
+
+## Emit stage
+
+ This stage mainly focuses on the creation of a BundledProgram, and dump it out to the disk as a flatbuffer file. The main procedure is as follow:
+1. Create a model and emit its executorch program.
+2. Construct a BundledConfig to record all info need to be bundled.
+3. Generate BundledProgram using the emited model and BundledProgram
+4. Serialize the BundledProgram and dump it out to the disk.
+
+### Step 1: Create a model and emit its executorch program.
+
+This is not the part BunledProgram focusing on, so we just give left an example here without detailed APIs usage. Most of the example is borrowed from bundled_program/tests/common.py:
+
+```python
+
+import torch
+from executorch import exir
+from executorch.exir import ExecutorchBackendConfig
+from executorch.exir.passes import MemoryPlanningPass, ToOutVarPass
+
+
+class SampleModel(torch.nn.Module):
+    """An example model with multi-methods. Each method has multiple input and single output"""
+
+    def __init__(self) -> None:
+        super().__init__()
+        self.a: torch.Tensor = 3 * torch.ones(2, 2, dtype=torch.int32)
+        self.b: torch.Tensor = 2 * torch.ones(2, 2, dtype=torch.int32)
+
+    def encode(
+        self, x: torch.Tensor, q: torch.Tensor
+    ) -> torch.Tensor:
+        z = x.clone()
+        torch.mul(self.a, x, out=z)
+        y = x.clone()
+        torch.add(z, self.b, out=y)
+        torch.add(y, q, out=y)
+        return y
+
+    def decode(
+        self, x: torch.Tensor, q: torch.Tensor
+    ) -> torch.Tensor:
+        y = x * q
+        torch.add(y, self.b, out=y)
+        return y
+
+
+method_names = ["encode", "decode"]
+model = SampleModel()
+
+capture_inputs = {
+    m_name: (
+        (torch.rand(2, 2) - 0.5).to(dtype=torch.int32),
+        (torch.rand(2, 2) - 0.5).to(dtype=torch.int32),
+    )
+    for m_name in method_names
+}
+
+# Trace to FX Graph and emit the program
+program = (
+    exir.capture_multiple(model, capture_inputs)
+    .to_edge()
+    .to_executorch()
+    .program
+)
+
+```
+
+### Step 2: Construct BundledConfig
+
+BundledConfig is a class under `executorch/bundled_program/config.py` that contains all information needs to be bundled for model verification. Here's the constructor api to create BundledConfig:
+
+```python
+class BundledConfig:
+    def __init__(
+        self,
+        method_names: List[str],
+        inputs: List[List[Any]],
+        expected_outputs: List[List[Any]],
+    ) -> None:
+        """Contruct the config given inputs and expected outputs
+
+        Args:
+            method_names: All method names need to be verified in program.
+            inputs: All sets of input need to be test on for all methods. Each list
+                    of `inputs` is all sets which will be run on the method in the
+                    program with corresponding method name. Each set of any `inputs` element should
+                    contain all inputs required by eager_model with the same inference function
+                    as corresponding execution plan for one-time execution.
+
+            expected_outputs: Expected outputs for inputs sharing same index. The size of
+                    expected_outputs should be the same as the size of inputs and provided method_names.
+        """
+
+```
+
+Here's an example of creating a bundled program for SampleModel above:
+
+```python
+
+from executorch.bundled_program.config import BundledConfig
+
+# number of input sets needed to be verified
+n_input = 10
+
+# All Input sets need to be verified for all execution plans.
+inputs = [
+    # The below list is all inputs for a single execution plan (inference method).
+    [
+        # Each list below is a individual input set.
+        # The number of inputs, dtype and size of each input follow Program's spec.
+        [
+            (torch.rand(2, 2) - 0.5).to(dtype=torch.int32),
+            (torch.rand(2, 2) - 0.5).to(dtype=torch.int32),
+        ]
+        for _ in range(n_input)
+    ]
+    for _ in range(len(program.execution_plan))
+]
+
+# Expected outputs align with inputs.
+expected_outputs = [
+    [[getattr(model, m_name)(*x)] for x in inputs[i]]
+    for i, m_name in enumerate(method_names)
+]
+
+
+bundled_config = BundledConfig(
+    method_names, inputs, expected_outputs
+)
+
+```
+
+### Step 3: Generate BundledProgram
+
+To create BundledProgram, we provice `create_bundled_program` under `executorch/bundled_program/core.py` to generate BundledProgram by bundling the emitted executorch program with the bundled_config:
+
+```python
+
+def create_bundled_program(
+    program: Program,
+    bundled_config: BundledConfig,
+) -> BundledProgram:
+    """
+    Args:
+        program: The program to be bundled.
+        bundled_config: The config to be bundled.
+    """
+```
+
+Example:
+
+```python
+from executorch.bundled_program.core import create_bundled_program
+
+bundled_program = create_bundled_program(program, bundled_config)
+```
+
+### Step 4: Serialize BundledProgram to Flatbuffer.
+
+To serialize BundledProgram to make runtime APIs use it, we provide two APIs, both under `executorch/bundled_program/serialize/__init__.py`.
+
+Serialize BundledProgram to flatbuffer:
+
+```python
+def serialize_from_bundled_program_to_flatbuffer(
+    bundled_program: BundledProgram,
+) -> bytes
+```
+
+Deserialize flatbuffer to BundledProgram:
+
+```python
+def deserialize_from_flatbuffer_to_bundled_program(
+    flatbuffer: bytes
+) -> BundledProgram
+```
+
+Example:
+```python
+from executorch.bundled_program.serialize import (
+    serialize_from_bundled_program_to_flatbuffer,
+    deserialize_from_flatbuffer_to_bundled_program,
+)
+
+serialized_bundled_program = serialize_from_bundled_program_to_flatbuffer(bundled_program)
+regenerate_bundled_program = deserialize_from_flatbuffer_to_bundled_program(serialized_bundled_program)
+
+```
+
+## Runtime Stage
+This stage mainly focuses on executing the model with the bundled inputs and and comparing the model's output with the bundled expected output. We provide multiple APIs to handle the key parts of it.
+
+### Get executorch program ptr from BundledProgram buffer
+We need the pointer to executorch program to do the execution. To unify the process of loading and executing BundledProgram and Program flatbuffer, we create an API:
+ ```c++
+
+/**
+ * Finds the serialized ExecuTorch program data in the provided file data.
+ *
+ * The returned buffer is appropriate for constructing a
+ * torch::executor::Program.
+ *
+ * Calling this is only necessary if the file could be a bundled program. If the
+ * file will only contain an unwrapped ExecuTorch program, callers can construct
+ * torch::executor::Program with file_data directly.
+ *
+ * @param[in] file_data The contents of an ExecuTorch program or bundled program
+ *                      file.
+ * @param[in] file_data_len The length of file_data, in bytes.
+ * @param[out] out_program_data The serialized Program data, if found.
+ * @param[out] out_program_data_len The length of out_program_data, in bytes.
+ *
+ * @returns Error::Ok if the program was found, and
+ *     out_program_data/out_program_data_len point to the data. Other values
+ *     on failure.
+ */
+Error GetProgramData(
+    void* file_data,
+    size_t file_data_len,
+    const void** out_program_data,
+    size_t* out_program_data_len);
+```
+
+Here's an example of how to use the GetProgramData API:
+```c++
+  std::shared_ptr<char> buff_ptr;
+  size_t buff_len;
+
+// FILE_PATH here can be either BundledProgram or Program flatbuffer file.
+  Error status = torch::executor::util::read_file_content(
+      FILE_PATH, &buff_ptr, &buff_len);
+  ET_CHECK_MSG(
+      status == Error::Ok,
+      "read_file_content() failed with status 0x%" PRIx32,
+      status);
+
+  uint32_t prof_tok = EXECUTORCH_BEGIN_PROF("de-serialize model");
+
+  const void* program_ptr;
+  size_t program_len;
+  status = torch::executor::util::GetProgramData(
+      buff_ptr.get(), buff_len, &program_ptr, &program_len);
+  ET_CHECK_MSG(
+      status == Error::Ok,
+      "GetProgramData() failed with status 0x%" PRIx32,
+      status);
+```
+
+### Load bundled input to ExecutionPlan
+To execute the program on the bundled input, we need to load the bundled input into the ExecutionPlan. Here we provided an API called `torch::executor::util::LoadBundledInput`:
+
+```c++
+
+/**
+ * Load testset_idx-th bundled input of method_idx-th Method test in
+ * bundled_program_ptr to given Method.
+ *
+ * @param[in] method The Method to verify.
+ * @param[in] bundled_program_ptr The bundled program contains expected output.
+ * @param[in] method_name  The name of the Method being verified.
+ * @param[in] testset_idx  The index of input needs to be set into given Method.
+ *
+ * @returns Return Error::Ok if load successfully, or the error happens during
+ * execution.
+ */
+__ET_NODISCARD Error LoadBundledInput(
+    Method& method,
+    serialized_bundled_program* bundled_program_ptr,
+    MemoryAllocator* memory_allocator,
+    const char* method_name,
+    size_t testset_idx);
+```
+
+### Verify the plan's output.
+We call `torch::executor::util::VerifyResultWithBundledExpectedOutput` to verify the method's output with bundled expected outputs. Here's the details of this API:
+
+```c++
+/**
+ * Compare the Method's output with testset_idx-th bundled expected
+ * output in method_idx-th Method test.
+ *
+ * @param[in] method The Method to extract outputs from.
+ * @param[in] bundled_program_ptr The bundled program contains expected output.
+ * @param[in] method_name  The name of the Method being verified.
+ * @param[in] testset_idx  The index of expected output needs to be compared.
+ * @param[in] rtol Relative tolerance used for data comparsion.
+ * @param[in] atol Absolute tolerance used for data comparsion.
+ *
+ * @returns Return Error::Ok if two outputs match, or the error happens during
+ * execution.
+ */
+__ET_NODISCARD Error VerifyResultWithBundledExpectedOutput(
+    Method& method,
+    serialized_bundled_program* bundled_program_ptr,
+    MemoryAllocator* memory_allocator,
+    const char* method_name,
+    size_t testset_idx,
+    double rtol = 1e-5,
+    double atol = 1e-8);
+
+```
+
+### Example
+
+Here we provide an example about how to run the bundled program step by step. Most of the code are borrowed from "fbcode/executorch/sdk/runners/executor_runner.cpp" and please review that file if you need more info and context:
+
+```c++
+    // method_name is the name for the method we want to test
+    // memory_manager is the executor::MemoryManager variable for executor memory allocation.
+    // program is the executorch program.
+    Result<Method> method = program->load_method(method_name, &memory_manager);
+    EXECUTORCH_END_PROF(prof_tok);
+    ET_CHECK_MSG(
+        method.ok(),
+        "load_method() failed with status 0x%" PRIx32,
+        method.error());
+
+    // Load testset_idx-th input in the buffer to plan
+    status = torch::executor::util::LoadBundledInput(
+          *method,
+          program_data.bundled_program_data(),
+          &bundled_input_allocator,
+          method_name,
+          FLAGS_testset_idx);
+      ET_CHECK_MSG(
+          status == Error::Ok,
+          "LoadBundledInput failed with status 0x%" PRIx32,
+          status);
+
+    // Execute the plan
+    status = method->execute();
+    ET_CHECK_MSG(
+        status == Error::Ok,
+        "method->execute() failed with status 0x%" PRIx32,
+        status);
+
+    // Verify the result.
+    status = torch::executor::util::VerifyResultWithBundledExpectedOutput(
+          *method,
+          program_data.bundled_program_data(),
+          &bundled_input_allocator,
+          method_name,
+          FLAGS_testset_idx,
+          FLAGS_rtol,
+          FLAGS_atol);
+      ET_CHECK_MSG(
+          status == Error::Ok,
+          "Bundle verification failed with status 0x%" PRIx32,
+          status);
+
+```

docs/website/docs/tutorials/bundled_program.md

@@ -1,3 +1,5 @@
+DEPRECATED: This document is moving to //executorch/docs/source/sdk-bundled-io.md
+
 # Bundled Program
 
 ## Introduction