Delegate Integration with SDK Documentation (#643)

Summary:
Pull Request resolved: #643

^^^

Original Doc: https://docs.google.com/document/d/1v_oL_yXGF7UyPPJTe_btt8SeItivV58vaSrazyEQ37I/edit

Reviewed By: tarun292

Differential Revision: D49968371

fbshipit-source-id: 1776c4d347f94341731dc902e5c86d805580bdfb

Author: Jack-Khuu

docs/source/sdk-delegate-integration.md

@@ -1,3 +1,121 @@
 # Delegate Integration
 
-TBA
+[Delegate backends](compiler-delegate-and-partitioner.md) are a prominent component of on-device models due to their flexibility in defining behavior. A side effect of this flexibility is that it operates as an opaque transformation. This obfuscates rich associations and mutations that are valuable in post-processing.
+- For example, if two different operator fusions were to occur within a delegate, post processing wouldn’t be able to separate the two transformations.
+
+Specifically, it makes associating runtime information (such as profiling results) through delegated graphs difficult. Delegate Debug Identifiers provides a framework through which delegate authors can propagate this information and utilize it for post run analysis.
+
+The preparation is broken down into two stages:
+- **Ahead-of-time (AOT)**: Delegate authors generate a __Debug Handle Map__.
+- **Runtime**: Delegate authors log using the __Delegate Debug Identifiers__ registered AOT in the __Debug Handle Map__.
+
+## Ahead-of-Time
+Delegate authors propagate what transformations occur in a lowered backend by returning a **Debug Handle Map** from the backend implementation.
+
+### Generating a Debug Handle Map
+**Debug Handle Maps** communicate what transformations occurred in a backend by mapping **Delegate Debug Identifiers** to debug handles.
+
+**Delegate Debug Identifiers** are generated or user-provided identifiers for representing points of interest during runtime. Recall that debug handles are unique identifiers to operator instances in the model graph.
+
+For example:
+- **{ 0: (10, 11), 1: (11, 12) }:** Identifiers 0 and 1 in the runtime correspond to operators with the debug handles (10, 11) and (11, 12) respectively.
+- **{ “Fancy Fusion”: (11, 12, 15) }**: Identifier “Fancy Fusion” in the runtime corresponds to operators with debug handles (11, 12, 15).
+
+```{Note}
+Identifiers are a means of connecting runtime results to the model graph; the interpretation of the identifiers is defined by the delegate author.*
+```
+
+**Debug Handle Maps** are constructed through the use of **DelegateMappingBuilder** and returned as a part of `PreprocessResult`.
+
+```python
+class PreprocessResult:
+    processed_bytes: bytes = bytes()
+
+    debug_handle_map: Optional[
+        Union[Dict[int, Tuple[int]], Dict[str, Tuple[int]]]
+    ] = None
+```
+PreprocessResult is defined [here](https://github.com/pytorch/executorch/blob/main/exir/backend/backend_details.py).
+
+#### DelegateMappingBuilder
+`DelegateMappingBuilder` is a helper class for managing and constructing Debug Handle Maps. The result of the builder should be passed in when constructing PreprocessResult.
+
+`DelegateMappingBuilder` is defined [here](https://github.com/pytorch/executorch/blob/main/exir/backend/utils.py)
+
+A `DelegateMappingBuilder` instance can be constructed in one of 2 modes: manual identifiers or generated identifiers.
+
+```python
+# Manual Identifiers, Default
+builder = DelegateMappingBuilder(generated_identifiers=False)
+
+# Generated Identifiers
+builder = DelegateMappingBuilder(generated_identifiers=True)
+```
+
+With **manual identifiers**, users pass in a **Delegate Debug Identifier** when creating entries.
+With **generated identifiers**, the builder will auto-assign a **Delegate Debug Identifier**.
+
+To add an entry to the **Debug Handle Map**, use `insert_delegate_mapping_entry`. It takes `fx.Node(s)` to associate and an optional **Delegate Debug Identifier** (used for the manual identifiers). The identifier recorded is returned from the call.
+
+```python
+def insert_delegate_mapping_entry(
+    self,
+    nodes: Union[fx.Node, List[fx.Node]],
+    identifier: Optional[Union[int, str]] = None,
+) -> Union[int, str]:
+```
+
+To retrieve the **Debug Handle Map**, use `get_delegate_mapping`.
+```python
+def get_delegate_mapping(
+    self,
+) -> Union[Dict[int, Tuple[int]], Dict[str, Tuple[int]]]
+```
+
+A demo of the AOT mapping can be found [here](https://github.com/pytorch/executorch/blob/main/exir/backend/test/backend_with_delegate_mapping_demo.py)
+
+
+## Runtime
+Corresponding to the AOT map, the runtime then defines the functionality through which these events are logged.
+
+### Real-Time Logging
+
+ExecuTorch allows you to log in real time. **Real time Logging** is useful when timestamps are available as the execution occurs. It provides minimal overhead and is intuitive for authors to call.
+
+To log events in real-time (for example, explicitly denoting the profiling start and stop), `event_tracer_start_profiling_delegate` is used to create an `EventEntry` and `event_tracer_end_profiling_delegate` is used to conclude the `EventEntry` for the provided `EventTracer`.
+
+To start an `EventTracerEntry` using `event_tracer_start_profiling_delegate`, the **Delegate Debug Identifier** (provided AOT to the `debug_handle_map`) is passed as either the name or `delegate_debug_id` argument depending on the **Delegate Debug Identifier** type (str and int respectively)
+
+```c++
+EventTracerEntry event_tracer_start_profiling_delegate(
+    EventTracer* event_tracer,
+    const char* name,
+    DebugHandle delegate_debug_id)
+```
+
+To conclude an `EventTracerEntry`, `event_tracer_end_profiling_delegate` is simply provided the original `EventTracerEntry`.
+
+Optionally, additional runtime `metadata` can also be logged at this point.
+
+```c++
+void event_tracer_end_profiling_delegate(
+    EventTracer* event_tracer,
+    EventTracerEntry event_tracer_entry,
+    const char* metadata = nullptr)
+```
+
+### Post-Time Logging
+ExecuTorch also allows you to log in post time. Some runtime settings don't have access to timestamps while it is executing. **Post-Time Logging** enables authors to still be able to log these events.
+
+To log events in post (for example, logging start and end time simultaneously) `event_tracer_log_profiling_delegate` is called with a combination of the arguments used in the real-time logging API’s and timestamps.
+
+```c++
+void event_tracer_log_profiling_delegate(
+    EventTracer* event_tracer,
+    const char* name,
+    DebugHandle delegate_debug_id,
+    et_timestamp_t start_time,
+    et_timestamp_t end_time,
+    const char* metadata = nullptr)
+```
+A demo of the runtime code can be found [here](https://github.com/pytorch/executorch/blob/main/runtime/executor/test/test_backend_with_delegate_mapping.cpp).