Enable RANGE definition in shader YAML for repetitive shader generation #4196

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Closed

Esteb37 wants to merge 4 commits into pytorch:main from Esteb37:export-D59588223

Contributor

Esteb37 commented Jul 10, 2024

Summary:
GPUInfo requires the compilation of a ton of shaders with small variations to emulate OpenCL's kernel compilation on runtime with strings. This diff enables a RANGE implementation to generate these shaders dynamically without typing each value out.

The format is

- RANGE: [start, end] // Inclusive
  SUFFIX: string // Optional

If a suffix is selected, the shaders will be called name_suffix_K for each value K from the range. If no suffix is selected, the shaders will be simply called name_K

Differential Revision: D59588223

pytorch-bot bot commented Jul 10, 2024 •

edited

Loading

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/4196

📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❗ 1 Active SEVs

There are 1 currently active SEVs. If your PR is affected, please view them below:

[PREEMPTIVE] CPU amd64 non-GPU instances migration to Linux AMZN2023

This comment was automatically generated by Dr. CI and updates every 15 minutes.

facebook-github-bot added the CLA Signed label

Contributor

facebook-github-bot commented Jul 10, 2024

This pull request was exported from Phabricator. Differential Revision: D59588223

facebook-github-bot added the fb-exported label

Contributor

facebook-github-bot commented Jul 11, 2024

This pull request was exported from Phabricator. Differential Revision: D59588223

Esteb37 force-pushed the export-D59588223 branch from 6bb9f29 to a5f8ee5 Compare

July 11, 2024 14:33

Esteb37 pushed a commit to Esteb37/executorch that referenced this pull request


          Enable RANGE definition in shader YAML for repetitive shader generati…

a5f8ee5

…on (pytorch#4196)

Summary:
Pull Request resolved: pytorch#4196

GPUInfo requires the compilation of a ton of shaders with small variations to emulate OpenCL's kernel compilation on runtime with strings. This diff enables a RANGE implementation to generate these shaders dynamically without typing each value out.

The format is

    - RANGE: [start, end] // Inclusive
      SUFFIX: string // Optional

If a suffix is selected, the shaders will be called `name_suffix_K	` for each value K from the range. If no suffix is selected, the shaders will be simply called `name_K	`

Differential Revision: D59588223

Contributor

facebook-github-bot commented Jul 11, 2024

This pull request was exported from Phabricator. Differential Revision: D59588223

Esteb37 pushed a commit to Esteb37/executorch that referenced this pull request


          Enable RANGE definition in shader YAML for repetitive shader generati…

60fb1ce

…on (pytorch#4196)

Summary:
Pull Request resolved: pytorch#4196

GPUInfo requires the compilation of a ton of shaders with small variations to emulate OpenCL's kernel compilation on runtime with strings. This diff enables a RANGE implementation to generate these shaders dynamically without typing each value out.

The format is

    - RANGE: [start, end] // Inclusive
      SUFFIX: string // Optional

If a suffix is selected, the shaders will be called `name_suffix_K	` for each value K from the range. If no suffix is selected, the shaders will be simply called `name_K	`

Differential Revision: D59588223

Esteb37 force-pushed the export-D59588223 branch from a5f8ee5 to 60fb1ce Compare

July 11, 2024 14:43

SS-JIA self-requested a review

July 11, 2024 15:42

SS-JIA approved these changes

View reviewed changes

Contributor

facebook-github-bot commented Jul 11, 2024

This pull request was exported from Phabricator. Differential Revision: D59588223

Esteb37 pushed a commit to Esteb37/executorch that referenced this pull request


          Enable RANGE definition in shader YAML for repetitive shader generati…

69216b1

…on (pytorch#4196)

Summary:
Pull Request resolved: pytorch#4196

GPUInfo requires the compilation of a ton of shaders with small variations to emulate OpenCL's kernel compilation on runtime with strings. This diff enables a RANGE implementation to generate these shaders dynamically without typing each value out.

The format is

    - RANGE: [start, end] // Inclusive
      SUFFIX: string // Optional

If a suffix is selected, the shaders will be called `name_suffix_K	` for each value K from the range. If no suffix is selected, the shaders will be simply called `name_K	`

Reviewed By: SS-JIA

Differential Revision: D59588223

Esteb37 force-pushed the export-D59588223 branch from 60fb1ce to 69216b1 Compare

July 11, 2024 16:14

estebanpadilla added 3 commits

July 11, 2024 09:17


          RegCount NITER calculation (pytorch#4159)

902c963

Summary:
Pull Request resolved: pytorch#4159

This adds an internal implementation of https://github.com/microsoft/ArchProbe.

This stack introduces a kernel that can be used to get the number of available registers on a mobile GPU by gradually increasing the number of accessed elements and detecting dramatic drops in performance. See [this paper ](https://www.microsoft.com/en-us/research/uploads/prod/2022/02/mobigpu_mobicom22_camera.pdf), page 4, for more information.

This first diff gets the number of iterations (NITER) that can run in 1000us, to be used in the following tests.

The kernel looks like the following for any K number of registers:

  float reg_data0 = float(niter) + 0;
  float reg_data1 = float(niter) + 1;
  ...
  float reg_dataK = float(niter) + K;

  int i = 0;
  for (; i < niter; ++i) {
    reg_data0 *= reg_dataK;
    reg_data1 *= reg_data0;
    reg_data2 *= reg_data1;
    ...
    reg_dataK *= reg_data(K-1);
  }

  i = i >> 31;

  buffer_out.data[0 * i] = reg_data0;
  buffer_out.data[1 * i] = reg_data1;
  ...
  buffer_out.data[K * i] = reg_dataK;

Differential Revision: D59405012

Reviewed By: SS-JIA


          RegCount max registers calculation (pytorch#4171)

e82560b

Summary:
Pull Request resolved: pytorch#4171

This project adds an internal implementation of https://github.com/microsoft/ArchProbe.

This stack introduces a kernel that can be used to get the number of available registers on a mobile GPU by gradually increasing the number of accessed elements and detecting dramatic drops in performance. See [this paper](https://www.microsoft.com/en-us/research/uploads/prod/2022/02/mobigpu_mobicom22_camera.pdf), page 4, for more information.

This diffs finds the number of registers in a single thread by increasing the number of registers and finding changes in latency. For a Galaxy S22, the latency graph looks like this.

 {F1750619092}

We can easily spot the moment where there is a spill into memory.

Differential Revision: D59494644

Reviewed By: SS-JIA


          RegCount concurrency calculation (pytorch#4173)

e0e9a9b

Summary:
Pull Request resolved: pytorch#4173

This project adds an internal implementation of https://github.com/microsoft/ArchProbe.

This stack introduces a kernel that can be used to get the number of available registers on a mobile GPU by gradually increasing the number of accessed elements and detecting dramatic drops in performance. See [this paper](https://www.microsoft.com/en-us/research/uploads/prod/2022/02/mobigpu_mobicom22_camera.pdf), page 4, for more information.

This diff finds the number of concurrency groups available for full registry usage, and half registry usage.

On a Samsung Galaxy S22, the latency graphs for full and half registry usage look like this:

Full:
{F1750677545}

Half:
 {F1750679467}

Differential Revision: D59497314

Reviewed By: SS-JIA

Contributor

facebook-github-bot commented Jul 11, 2024

This pull request was exported from Phabricator. Differential Revision: D59588223

Esteb37 force-pushed the export-D59588223 branch from 69216b1 to acc1adf Compare

July 11, 2024 16:21

Esteb37 pushed a commit to Esteb37/executorch that referenced this pull request


          Enable RANGE definition in shader YAML for repetitive shader generati…

acc1adf

…on (pytorch#4196)

Summary:
Pull Request resolved: pytorch#4196

GPUInfo requires the compilation of a ton of shaders with small variations to emulate OpenCL's kernel compilation on runtime with strings. This diff enables a RANGE implementation to generate these shaders dynamically without typing each value out.

The format is

    - RANGE: [start, end] // Inclusive
      SUFFIX: string // Optional

If a suffix is selected, the shaders will be called `name_suffix_K	` for each value K from the range. If no suffix is selected, the shaders will be simply called `name_K	`

Reviewed By: SS-JIA

Differential Revision: D59588223

Contributor

facebook-github-bot commented Jul 11, 2024

This pull request was exported from Phabricator. Differential Revision: D59588223

Esteb37 pushed a commit to Esteb37/executorch that referenced this pull request


          Enable RANGE definition in shader YAML for repetitive shader generati…

3d5c277

…on (pytorch#4196)

Summary:
Pull Request resolved: pytorch#4196

GPUInfo requires the compilation of a ton of shaders with small variations to emulate OpenCL's kernel compilation on runtime with strings. This diff enables a RANGE implementation to generate these shaders dynamically without typing each value out.

The format is

    - RANGE: [start, end] // Inclusive
      SUFFIX: string // Optional

If a suffix is selected, the shaders will be called `name_suffix_K	` for each value K from the range. If no suffix is selected, the shaders will be simply called `name_K	`

Reviewed By: SS-JIA

Differential Revision: D59588223

Esteb37 force-pushed the export-D59588223 branch from acc1adf to 3d5c277 Compare

July 11, 2024 16:44


          Enable RANGE definition in shader YAML for repetitive shader generati…

f474e9d

…on (pytorch#4196)

Summary:
Pull Request resolved: pytorch#4196

GPUInfo requires the compilation of a ton of shaders with small variations to emulate OpenCL's kernel compilation on runtime with strings. This diff enables a RANGE implementation to generate these shaders dynamically without typing each value out.

The format is

    - RANGE: [start, end] // Inclusive
      SUFFIX: string // Optional

If a suffix is selected, the shaders will be called `name_suffix_K	` for each value K from the range. If no suffix is selected, the shaders will be simply called `name_K	`

Reviewed By: SS-JIA

Differential Revision: D59588223

Esteb37 force-pushed the export-D59588223 branch from 3d5c277 to f474e9d Compare

July 11, 2024 16:51

facebook-github-bot closed this in

c698791

facebook-github-bot added the Merged label

Contributor

facebook-github-bot commented Jul 11, 2024

This pull request has been merged in c698791.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CLA Signed fb-exported Merged