Skip to content

Commit 812aa06

Browse files
authored
[Android benchmarking] Cool down the device before a run
This could reduce the variance. Example: 25.44 → 24.74 which is good variance.
1 parent b8c3ee9 commit 812aa06

File tree

2 files changed

+10
-0
lines changed

2 files changed

+10
-0
lines changed

extension/benchmark/android/benchmark/android-llm-device-farm-test-spec.yml.j2

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -109,6 +109,10 @@ phases:
109109
- |
110110
adb -s $DEVICEFARM_DEVICE_UDID shell am force-stop org.pytorch.minibench
111111

112+
adb -s $DEVICEFARM_DEVICE_UDID shell dumpsys deviceidle force-idle
113+
adb -s $DEVICEFARM_DEVICE_UDID shell dumpsys deviceidle unforce
114+
adb -s $DEVICEFARM_DEVICE_UDID shell sleep 10
115+
112116
if [ -n "$BIN_FOUND" ]; then
113117
adb -s $DEVICEFARM_DEVICE_UDID shell am start -W -n org.pytorch.minibench/.LlmBenchmarkActivity \
114118
--es "model_dir" "/data/local/tmp/minibench" \

extension/benchmark/android/benchmark/app/src/main/java/org/pytorch/minibench/BenchmarkActivity.java

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -95,6 +95,12 @@ protected void onPostExecute(Void aVoid) {
9595
new BenchmarkMetric(
9696
benchmarkModel,
9797
"avg_inference_latency(ms)",
98+
stats.latency.stream().mapToDouble(l -> l).average().orElse(0.0f),
99+
0.0f));
100+
results.add(
101+
new BenchmarkMetric(
102+
benchmarkModel,
103+
"trimmean_inference_latency(ms)",
98104
usedLatencyResults.stream().mapToDouble(l -> l).average().orElse(0.0f),
99105
0.0f));
100106
// Model load time

0 commit comments

Comments
 (0)