Skip to content

Commit 4e0e340

Browse files
pytorchbotOlivia-liu
pytorchbot
and
Olivia-liu
authored
New URL for the Debug in Delegates documentation page (#5846)
New URL for the Delegates Debugging documentation page (#5769) Summary: Pull Request resolved: #5769 This diff is to rename the "sdk-delegate-integration" documentation page to "delegate-debugging". The title and subtitles are updated as well. Old URL: https://pytorch.org/executorch/main/sdk-delegate-integration.html New URL ("sdk" is removed): https://pytorch.org/executorch/main/delegate-debugging.html Design doc: https://docs.google.com/document/d/1l6DYTq9Kq6VrPohruRFP-qScZDj01W_g4zlKyvqKGF4/edit?usp=sharing <- This is where the new file name come from. Reviewed By: dbort Differential Revision: D63656566 fbshipit-source-id: 0cd9019f2071a50b2f2176a33f278992f17c75e8 (cherry picked from commit dc24983) Co-authored-by: Olivia Liu <[email protected]>
1 parent 18845b0 commit 4e0e340

File tree

5 files changed

+157
-154
lines changed

5 files changed

+157
-154
lines changed

‎docs/source/compiler-delegate-and-partitioner.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -133,7 +133,7 @@ Providing consistent debugging experience, be it for runtime failures or perform
133133

134134
Delegated program or subgraphs are opaque to ExecuTorch runtime and appear as a special `call_delegate` instruction, which asks corresponding backend to handle the execution of the subgraph or program. Due to the opaque nature of backend delgates, native Developer Tools does not have visibility into delegated program. Thus the debugging, functional or performance, experiences of delegated execution suffers significantly as compared to it's non-delegated counterpart.
135135

136-
In order to provide consistent debugging experience to users, regardless of the use of delegation for a model, Developer Tools provide an interface to correlate delegated (sub)graph to original (sub)graph. The Developer Tools do so via debug handles map which allows delegates to generate internal handles that can be associated with the original (sub)graph consumed by the delegate. Then at runtime, backend developer can report error or profiling information using the internal handle, which will be mapped to original (sub)graph using the debug handle map. For more information, please refer to [Developer Tools Delegate Integration](./sdk-delegate-integration).
136+
In order to provide consistent debugging experience to users, regardless of the use of delegation for a model, Developer Tools provide an interface to correlate delegated (sub)graph to original (sub)graph. The Developer Tools do so via debug handles map which allows delegates to generate internal handles that can be associated with the original (sub)graph consumed by the delegate. Then at runtime, backend developer can report error or profiling information using the internal handle, which will be mapped to original (sub)graph using the debug handle map. For more information, please refer to [Delegate Debugging](./delegate-debugging).
137137

138138
By leveraging the debug identifier, backend developer can embed the debug as part of the delegated blob
139139

‎docs/source/delegate-debugging.md

Lines changed: 152 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,152 @@
1+
# Delegate Debugging
2+
3+
[Delegate backends](compiler-delegate-and-partitioner.md) are a prominent component of on-device models due to their flexibility in defining behavior. A side effect of this flexibility is that it operates as an opaque transformation. This obfuscates rich associations and mutations that are valuable in post-processing.
4+
- For example, if two different operator fusions were to occur within a delegate, post processing wouldn’t be able to separate the two transformations.
5+
6+
Specifically, it makes associating runtime information (such as profiling results) through delegated graphs difficult. Delegate Debug Identifiers provides a framework through which delegate authors can propagate this information and utilize it for post run analysis.
7+
8+
The preparation is broken down into three stages:
9+
- **Ahead-of-time (AOT)**: Delegate authors generate a __Debug Handle Map__.
10+
- **Runtime**: Delegate authors log using the __Delegate Debug Identifiers__ registered AOT in the __Debug Handle Map__.
11+
- **Deserialization**: Delegate authors provide a parser for custom metadata in delegate events.
12+
13+
## Ahead-of-Time Integration
14+
Delegate authors propagate what transformations occur in a lowered backend by returning a **Debug Handle Map** from the backend implementation.
15+
16+
### Generating a Debug Handle Map
17+
**Debug Handle Maps** communicate what transformations occurred in a backend by mapping **Delegate Debug Identifiers** to debug handles.
18+
19+
**Delegate Debug Identifiers** are generated or user-provided identifiers for representing points of interest during runtime. Recall that debug handles are unique identifiers to operator instances in the model graph.
20+
21+
For example:
22+
- **{ 0: (10, 11), 1: (11, 12) }:** Identifiers 0 and 1 in the runtime correspond to operators with the debug handles (10, 11) and (11, 12) respectively.
23+
- **{ “fused_op_1_2_3”: (11, 12, 15) }**: Identifier “fused_op_1_2_3” in the runtime corresponds to operators with debug handles (11, 12, 15), and 11, 12, 15 corresponds to the op 1, op 2 and op 3.
24+
25+
```{Note}
26+
Identifiers are a means of connecting runtime results to the model graph; the interpretation of the identifiers is defined by the delegate author.
27+
```
28+
29+
**Debug Handle Maps** are constructed through the use of **DelegateMappingBuilder** and returned as a part of `PreprocessResult`.
30+
31+
```python
32+
class PreprocessResult:
33+
processed_bytes: bytes = bytes()
34+
35+
debug_handle_map: Optional[
36+
Union[Dict[int, Tuple[int]], Dict[str, Tuple[int]]]
37+
] = None
38+
```
39+
PreprocessResult is defined [here](https://github.com/pytorch/executorch/blob/main/exir/backend/backend_details.py).
40+
41+
#### DelegateMappingBuilder
42+
`DelegateMappingBuilder` is a helper class for managing and constructing Debug Handle Maps. The result of the builder should be passed in when constructing PreprocessResult.
43+
44+
`DelegateMappingBuilder` is defined [here](https://github.com/pytorch/executorch/blob/main/exir/backend/utils.py)
45+
46+
A `DelegateMappingBuilder` instance can be constructed in one of 2 modes: manual identifiers or generated identifiers.
47+
48+
```python
49+
# Manual Identifiers, Default
50+
builder = DelegateMappingBuilder(generated_identifiers=False)
51+
52+
# Generated Identifiers
53+
builder = DelegateMappingBuilder(generated_identifiers=True)
54+
```
55+
56+
With **manual identifiers**, users pass in a **Delegate Debug Identifier** when creating entries.
57+
With **generated identifiers**, the builder will auto-assign a **Delegate Debug Identifier**.
58+
59+
To add an entry to the **Debug Handle Map**, use `insert_delegate_mapping_entry`. It associates one of `fx.Node(s)` or debug handles(s) (sourced from node.meta["debug_handle"]) to an optional **Delegate Debug Identifier** (used for the manual identifiers). The identifier recorded is returned from the call.
60+
61+
```python
62+
def insert_delegate_mapping_entry(
63+
self,
64+
nodes: Optional[Union[Node, List[Node]]] = None,
65+
handles: Optional[Union[int, List[int]]] = None,
66+
identifier: Optional[Union[int, str]] = None,
67+
) -> Union[int, str]:
68+
```
69+
70+
To retrieve the **Debug Handle Map**, use `get_delegate_mapping`.
71+
```python
72+
def get_delegate_mapping(
73+
self,
74+
) -> Union[Dict[int, Tuple[int]], Dict[str, Tuple[int]]]
75+
```
76+
77+
A demo of the AOT mapping can be found [here](https://github.com/pytorch/executorch/blob/main/exir/backend/test/backend_with_delegate_mapping_demo.py)
78+
79+
80+
## Runtime Logging
81+
Corresponding to the AOT map, the runtime then defines the functionality through which these events are logged.
82+
83+
### Real-Time Logging
84+
85+
ExecuTorch allows you to log in real time. **Real time Logging** is useful when timestamps are available as the execution occurs. It provides minimal overhead and is intuitive for authors to call.
86+
87+
To log events in real-time (for example, explicitly denoting the profiling start and stop), `event_tracer_start_profiling_delegate` is used to create an `EventEntry` and `event_tracer_end_profiling_delegate` is used to conclude the `EventEntry` for the provided `EventTracer`.
88+
89+
To start an `EventTracerEntry` using `event_tracer_start_profiling_delegate`, the **Delegate Debug Identifier** (provided AOT to the `debug_handle_map`) is passed as either the name or `delegate_debug_id` argument depending on the **Delegate Debug Identifier** type (str and int respectively)
90+
91+
```c++
92+
EventTracerEntry event_tracer_start_profiling_delegate(
93+
EventTracer* event_tracer,
94+
const char* name,
95+
DebugHandle delegate_debug_id)
96+
```
97+
98+
To conclude an `EventTracerEntry`, `event_tracer_end_profiling_delegate` is simply provided the original `EventTracerEntry`.
99+
100+
Optionally, additional runtime `metadata` can also be logged at this point.
101+
102+
```c++
103+
void event_tracer_end_profiling_delegate(
104+
EventTracer* event_tracer,
105+
EventTracerEntry event_tracer_entry,
106+
const void* metadata = nullptr,
107+
size_t metadata_len = 0)
108+
```
109+
110+
### Post-Time Logging
111+
ExecuTorch also allows you to log in post time. Some runtime settings don't have access to timestamps while it is executing. **Post-Time Logging** enables authors to still be able to log these events.
112+
113+
To log events in post (for example, logging start and end time simultaneously) `event_tracer_log_profiling_delegate` is called with a combination of the arguments used in the real-time logging API’s and timestamps.
114+
115+
```c++
116+
void event_tracer_log_profiling_delegate(
117+
EventTracer* event_tracer,
118+
const char* name,
119+
DebugHandle delegate_debug_id,
120+
et_timestamp_t start_time,
121+
et_timestamp_t end_time,
122+
const void* metadata = nullptr,
123+
size_t metadata_len = 0)
124+
```
125+
A demo of the runtime code can be found [here](https://github.com/pytorch/executorch/blob/main/runtime/executor/test/test_backend_with_delegate_mapping.cpp).
126+
127+
128+
## Surfacing custom metadata from delegate events
129+
130+
As seen in the runtime logging API's above, users can log an array of bytes along with their delegate profiling event. We make this data available for users in post processing via the [Inspector API](./sdk-inspector.rst).
131+
132+
Users can pass a metadata parser when creating an instance of the Inspector. The parser is a callable that deserializes the data and returns a list of strings or a dictionary containing key-value pairs. The deserialized data is then added back to the corresponding event in the event block for user consumption. Here's an example of how to write this parser:
133+
134+
NOTE: The input to the deserializer is a list where each entry is a series of bytes (essentially each entry is an immutable bytearray). Users are expected to iterate over this list, deserialize each entry and then return it in the expected format which is either a list of strings, or a dict.
135+
136+
```python
137+
Inspector(
138+
etdump_path=etdump_path,
139+
# Optional
140+
etrecord=etrecord_path,
141+
# Optional, only needed if debugging was enabled.
142+
buffer_path=buffer_path,
143+
delegate_metadata_parser=parse_delegate_metadata
144+
)
145+
146+
147+
def parse_delegate_metadata(delegate_metadatas: List[bytes]) -> Union[List[str], Dict[str, Any]]:
148+
metadata_str = []
149+
for metadata_bytes in delegate_metadatas:
150+
metadata_str += str(metadata_bytes)
151+
return metadata_str
152+
```

‎docs/source/index.rst

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -208,7 +208,7 @@ Topics in this section will help you get started with ExecuTorch.
208208
sdk-debugging
209209
sdk-inspector
210210
memory-planning-inspection
211-
sdk-delegate-integration
211+
delegate-debugging
212212
devtools-tutorial
213213

214214
.. toctree::

‎docs/source/sdk-delegate-integration.md

Lines changed: 2 additions & 151 deletions
Original file line numberDiff line numberDiff line change
@@ -1,152 +1,3 @@
1-
# Developer Tools Delegate Integration
1+
# Debug in Delegates
22

3-
[Delegate backends](compiler-delegate-and-partitioner.md) are a prominent component of on-device models due to their flexibility in defining behavior. A side effect of this flexibility is that it operates as an opaque transformation. This obfuscates rich associations and mutations that are valuable in post-processing.
4-
- For example, if two different operator fusions were to occur within a delegate, post processing wouldn’t be able to separate the two transformations.
5-
6-
Specifically, it makes associating runtime information (such as profiling results) through delegated graphs difficult. Delegate Debug Identifiers provides a framework through which delegate authors can propagate this information and utilize it for post run analysis.
7-
8-
The preparation is broken down into three stages:
9-
- **Ahead-of-time (AOT)**: Delegate authors generate a __Debug Handle Map__.
10-
- **Runtime**: Delegate authors log using the __Delegate Debug Identifiers__ registered AOT in the __Debug Handle Map__.
11-
- **Deserialization**: Delegate authors provide a parser for custom metadata in delegate events.
12-
13-
## Ahead-of-Time
14-
Delegate authors propagate what transformations occur in a lowered backend by returning a **Debug Handle Map** from the backend implementation.
15-
16-
### Generating a Debug Handle Map
17-
**Debug Handle Maps** communicate what transformations occurred in a backend by mapping **Delegate Debug Identifiers** to debug handles.
18-
19-
**Delegate Debug Identifiers** are generated or user-provided identifiers for representing points of interest during runtime. Recall that debug handles are unique identifiers to operator instances in the model graph.
20-
21-
For example:
22-
- **{ 0: (10, 11), 1: (11, 12) }:** Identifiers 0 and 1 in the runtime correspond to operators with the debug handles (10, 11) and (11, 12) respectively.
23-
- **{ “fused_op_1_2_3”: (11, 12, 15) }**: Identifier “fused_op_1_2_3” in the runtime corresponds to operators with debug handles (11, 12, 15), and 11, 12, 15 corresponds to the op 1, op 2 and op 3.
24-
25-
```{Note}
26-
Identifiers are a means of connecting runtime results to the model graph; the interpretation of the identifiers is defined by the delegate author.
27-
```
28-
29-
**Debug Handle Maps** are constructed through the use of **DelegateMappingBuilder** and returned as a part of `PreprocessResult`.
30-
31-
```python
32-
class PreprocessResult:
33-
processed_bytes: bytes = bytes()
34-
35-
debug_handle_map: Optional[
36-
Union[Dict[int, Tuple[int]], Dict[str, Tuple[int]]]
37-
] = None
38-
```
39-
PreprocessResult is defined [here](https://github.com/pytorch/executorch/blob/main/exir/backend/backend_details.py).
40-
41-
#### DelegateMappingBuilder
42-
`DelegateMappingBuilder` is a helper class for managing and constructing Debug Handle Maps. The result of the builder should be passed in when constructing PreprocessResult.
43-
44-
`DelegateMappingBuilder` is defined [here](https://github.com/pytorch/executorch/blob/main/exir/backend/utils.py)
45-
46-
A `DelegateMappingBuilder` instance can be constructed in one of 2 modes: manual identifiers or generated identifiers.
47-
48-
```python
49-
# Manual Identifiers, Default
50-
builder = DelegateMappingBuilder(generated_identifiers=False)
51-
52-
# Generated Identifiers
53-
builder = DelegateMappingBuilder(generated_identifiers=True)
54-
```
55-
56-
With **manual identifiers**, users pass in a **Delegate Debug Identifier** when creating entries.
57-
With **generated identifiers**, the builder will auto-assign a **Delegate Debug Identifier**.
58-
59-
To add an entry to the **Debug Handle Map**, use `insert_delegate_mapping_entry`. It associates one of `fx.Node(s)` or debug handles(s) (sourced from node.meta["debug_handle"]) to an optional **Delegate Debug Identifier** (used for the manual identifiers). The identifier recorded is returned from the call.
60-
61-
```python
62-
def insert_delegate_mapping_entry(
63-
self,
64-
nodes: Optional[Union[Node, List[Node]]] = None,
65-
handles: Optional[Union[int, List[int]]] = None,
66-
identifier: Optional[Union[int, str]] = None,
67-
) -> Union[int, str]:
68-
```
69-
70-
To retrieve the **Debug Handle Map**, use `get_delegate_mapping`.
71-
```python
72-
def get_delegate_mapping(
73-
self,
74-
) -> Union[Dict[int, Tuple[int]], Dict[str, Tuple[int]]]
75-
```
76-
77-
A demo of the AOT mapping can be found [here](https://github.com/pytorch/executorch/blob/main/exir/backend/test/backend_with_delegate_mapping_demo.py)
78-
79-
80-
## Runtime
81-
Corresponding to the AOT map, the runtime then defines the functionality through which these events are logged.
82-
83-
### Real-Time Logging
84-
85-
ExecuTorch allows you to log in real time. **Real time Logging** is useful when timestamps are available as the execution occurs. It provides minimal overhead and is intuitive for authors to call.
86-
87-
To log events in real-time (for example, explicitly denoting the profiling start and stop), `event_tracer_start_profiling_delegate` is used to create an `EventEntry` and `event_tracer_end_profiling_delegate` is used to conclude the `EventEntry` for the provided `EventTracer`.
88-
89-
To start an `EventTracerEntry` using `event_tracer_start_profiling_delegate`, the **Delegate Debug Identifier** (provided AOT to the `debug_handle_map`) is passed as either the name or `delegate_debug_id` argument depending on the **Delegate Debug Identifier** type (str and int respectively)
90-
91-
```c++
92-
EventTracerEntry event_tracer_start_profiling_delegate(
93-
EventTracer* event_tracer,
94-
const char* name,
95-
DebugHandle delegate_debug_id)
96-
```
97-
98-
To conclude an `EventTracerEntry`, `event_tracer_end_profiling_delegate` is simply provided the original `EventTracerEntry`.
99-
100-
Optionally, additional runtime `metadata` can also be logged at this point.
101-
102-
```c++
103-
void event_tracer_end_profiling_delegate(
104-
EventTracer* event_tracer,
105-
EventTracerEntry event_tracer_entry,
106-
const void* metadata = nullptr,
107-
size_t metadata_len = 0)
108-
```
109-
110-
### Post-Time Logging
111-
ExecuTorch also allows you to log in post time. Some runtime settings don't have access to timestamps while it is executing. **Post-Time Logging** enables authors to still be able to log these events.
112-
113-
To log events in post (for example, logging start and end time simultaneously) `event_tracer_log_profiling_delegate` is called with a combination of the arguments used in the real-time logging API’s and timestamps.
114-
115-
```c++
116-
void event_tracer_log_profiling_delegate(
117-
EventTracer* event_tracer,
118-
const char* name,
119-
DebugHandle delegate_debug_id,
120-
et_timestamp_t start_time,
121-
et_timestamp_t end_time,
122-
const void* metadata = nullptr,
123-
size_t metadata_len = 0)
124-
```
125-
A demo of the runtime code can be found [here](https://github.com/pytorch/executorch/blob/main/runtime/executor/test/test_backend_with_delegate_mapping.cpp).
126-
127-
128-
## Surfacing custom metadata from delegate events
129-
130-
As seen in the runtime logging API's above, users can log an array of bytes along with their delegate profiling event. We make this data available for users in post processing via the [Inspector API](./sdk-inspector.rst).
131-
132-
Users can pass a metadata parser when creating an instance of the Inspector. The parser is a callable that deserializes the data and returns a list of strings or a dictionary containing key-value pairs. The deserialized data is then added back to the corresponding event in the event block for user consumption. Here's an example of how to write this parser:
133-
134-
NOTE: The input to the deserializer is a list where each entry is a series of bytes (essentially each entry is an immutable bytearray). Users are expected to iterate over this list, deserialize each entry and then return it in the expected format which is either a list of strings, or a dict.
135-
136-
```python
137-
Inspector(
138-
etdump_path=etdump_path,
139-
# Optional
140-
etrecord=etrecord_path,
141-
# Optional, only needed if debugging was enabled.
142-
buffer_path=buffer_path,
143-
delegate_metadata_parser=parse_delegate_metadata
144-
)
145-
146-
147-
def parse_delegate_metadata(delegate_metadatas: List[bytes]) -> Union[List[str], Dict[str, Any]]:
148-
metadata_str = []
149-
for metadata_bytes in delegate_metadatas:
150-
metadata_str += str(metadata_bytes)
151-
return metadata_str
152-
```
3+
Please update your link to <https://pytorch.org/executorch/main/delegate-debugging.html>. This URL will be deleted after v0.4.0.

‎docs/source/sdk-profiling.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@ Profiling in ExecuTorch gives users access to these runtime metrics:
44
- Model Load Time.
55
- Operator Level Execution Time.
66
- Delegate Execution Time.
7-
- If the delegate that the user is calling into has been integrated with the [Developer Tools](./sdk-delegate-integration.md), then users will also be able to access delegated operator execution time.
7+
- If the delegate that the user is calling into has been integrated with the [Developer Tools](./delegate-debugging.md), then users will also be able to access delegated operator execution time.
88
- End-to-end Inference Execution Time.
99

1010
One uniqe aspect of ExecuTorch Profiling is the ability to link every runtime executed operator back to the exact line of python code from which this operator originated. This capability enables users to easily identify hotspots in their model, source them back to the exact line of Python code, and optimize if chosen to.

0 commit comments

Comments
 (0)