Skip to content

Commit 2a2d757

Browse files
pytorchbotswolchok
pytorchbot
and
swolchok
authored
Bump pinned PT commit to 20250625 and mirror torch/standalone to ExecuTorch (#12068)
This PR was created by the merge bot to help merge the original PR into the main branch. ghstack PR number: #11968 by @swolchok ^ Please use this as the source of truth for the PR details, comments, and reviews ghstack PR base: https://github.com/pytorch/executorch/tree/gh/swolchok/475/base ghstack PR head: https://github.com/pytorch/executorch/tree/gh/swolchok/475/head Merge bot PR base: https://github.com/pytorch/executorch/tree/main Merge bot PR head: https://github.com/pytorch/executorch/tree/gh/swolchok/475/orig @diff-train-skip-merge Co-authored-by: Scott Wolchok <[email protected]>
1 parent b74c68d commit 2a2d757

File tree

12 files changed

+145
-96
lines changed

12 files changed

+145
-96
lines changed

‎.ci/docker/ci_commit_pins/pytorch.txt

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1 +1 @@
1-
5616fa4a68718ead203314a3467f7dd9547153ae
1+
9b498d3bb28b8e3411ce464dd2755c5b96d92c8f

‎.ci/docker/common/install_conda.sh

Lines changed: 6 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -13,7 +13,7 @@ source "$(dirname "${BASH_SOURCE[0]}")/utils.sh"
1313
install_miniconda() {
1414
BASE_URL="https://repo.anaconda.com/miniconda"
1515
CONDA_FILE="Miniconda3-py${PYTHON_VERSION//./}_${MINICONDA_VERSION}-Linux-x86_64.sh"
16-
if [[ $(uname -m) == "aarch64" ]]; then
16+
if [[ $(uname -m) == "aarch64" ]]; then
1717
CONDA_FILE="Miniconda3-py${PYTHON_VERSION//./}_${MINICONDA_VERSION}-Linux-aarch64.sh"
1818
fi
1919

@@ -71,4 +71,8 @@ fix_conda_ubuntu_libstdcxx() {
7171
install_miniconda
7272
install_python
7373
install_pip_dependencies
74-
fix_conda_ubuntu_libstdcxx
74+
# Hack breaks the job on aarch64 but is still necessary everywhere
75+
# else.
76+
if [ "$(uname -m)" != "aarch64" ]; then
77+
fix_conda_ubuntu_libstdcxx
78+
fi

‎.ci/docker/conda-env-ci.txt

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
cmake=3.26.4
1+
cmake=3.31.2
22
ninja=1.10.2
33
libuv
44
llvm-openmp

‎.ci/scripts/check_c10_sync.sh

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -12,3 +12,4 @@ pushd pytorch
1212
git checkout "$pytorch_pin"
1313
popd
1414
"$(dirname "${BASH_SOURCE[0]}")"/compare_dirs.sh runtime/core/portable_type/c10/c10 pytorch/c10
15+
"$(dirname "${BASH_SOURCE[0]}")"/compare_dirs.sh runtime/core/portable_type/c10/torch/standalone pytorch/torch/standalone

‎install_requirements.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -71,7 +71,7 @@ def python_is_compatible():
7171
#
7272
# NOTE: If you're changing, make the corresponding change in .ci/docker/ci_commit_pins/pytorch.txt
7373
# by picking the hash from the same date in https://hud.pytorch.org/hud/pytorch/pytorch/nightly/
74-
NIGHTLY_VERSION = "dev20250601"
74+
NIGHTLY_VERSION = "dev20250625"
7575

7676

7777
def install_requirements(use_pytorch_nightly):

‎runtime/core/portable_type/c10/README.md

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -11,3 +11,10 @@ would cause all headers in that directory to be includeable with
1111
`#include <foo.h>`. In particular, that includes
1212
`runtime/core/portable_type/complex.h`, which would shadow the C99
1313
`complex.h` standard header.
14+
15+
`torch/standalone` has been added as an extra "even more bottom of
16+
stack" directory in PyTorch, so we have to add it to our sync
17+
here. The extra "stutter" c10 directory causing `c10/torch/standlone`
18+
is unfortunately awkward; perhaps we can rename the top-level
19+
directory to `pytorch_embedded_mirror` when we have extra time to work
20+
through CI failures.

‎runtime/core/portable_type/c10/c10/macros/Export.h

Lines changed: 2 additions & 86 deletions
Original file line numberDiff line numberDiff line change
@@ -1,95 +1,11 @@
11
#ifndef C10_MACROS_EXPORT_H_
22
#define C10_MACROS_EXPORT_H_
33

4-
/* Header file to define the common scaffolding for exported symbols.
5-
*
6-
* Export is by itself a quite tricky situation to deal with, and if you are
7-
* hitting this file, make sure you start with the background here:
8-
* - Linux: https://gcc.gnu.org/wiki/Visibility
9-
* - Windows:
10-
* https://docs.microsoft.com/en-us/cpp/cpp/dllexport-dllimport?view=vs-2017
11-
*
12-
* Do NOT include this file directly. Instead, use c10/macros/Macros.h
13-
*/
14-
15-
// You do not need to edit this part of file unless you are changing the core
16-
// pytorch export abstractions.
17-
//
18-
// This part defines the C10 core export and import macros. This is controlled
19-
// by whether we are building shared libraries or not, which is determined
20-
// during build time and codified in c10/core/cmake_macros.h.
21-
// When the library is built as a shared lib, EXPORT and IMPORT will contain
22-
// visibility attributes. If it is being built as a static lib, then EXPORT
23-
// and IMPORT basically have no effect.
24-
25-
// As a rule of thumb, you should almost NEVER mix static and shared builds for
26-
// libraries that depend on c10. AKA, if c10 is built as a static library, we
27-
// recommend everything dependent on c10 to be built statically. If c10 is built
28-
// as a shared library, everything dependent on it should be built as shared. In
29-
// the PyTorch project, all native libraries shall use the macro
30-
// C10_BUILD_SHARED_LIB to check whether pytorch is building shared or static
31-
// libraries.
32-
33-
// For build systems that do not directly depend on CMake and directly build
34-
// from the source directory (such as Buck), one may not have a cmake_macros.h
35-
// file at all. In this case, the build system is responsible for providing
36-
// correct macro definitions corresponding to the cmake_macros.h.in file.
37-
//
38-
// In such scenarios, one should define the macro
39-
// C10_USING_CUSTOM_GENERATED_MACROS
40-
// to inform this header that it does not need to include the cmake_macros.h
41-
// file.
42-
434
#ifndef C10_USING_CUSTOM_GENERATED_MACROS
445
#include <c10/macros/cmake_macros.h>
456
#endif // C10_USING_CUSTOM_GENERATED_MACROS
467

47-
#ifdef _WIN32
48-
#define C10_HIDDEN
49-
#if defined(C10_BUILD_SHARED_LIBS)
50-
#define C10_EXPORT __declspec(dllexport)
51-
#define C10_IMPORT __declspec(dllimport)
52-
#else
53-
#define C10_EXPORT
54-
#define C10_IMPORT
55-
#endif
56-
#else // _WIN32
57-
#if defined(__GNUC__)
58-
#define C10_EXPORT __attribute__((__visibility__("default")))
59-
#define C10_HIDDEN __attribute__((__visibility__("hidden")))
60-
#else // defined(__GNUC__)
61-
#define C10_EXPORT
62-
#define C10_HIDDEN
63-
#endif // defined(__GNUC__)
64-
#define C10_IMPORT C10_EXPORT
65-
#endif // _WIN32
66-
67-
#ifdef NO_EXPORT
68-
#undef C10_EXPORT
69-
#define C10_EXPORT
70-
#endif
71-
72-
// Definition of an adaptive XX_API macro, that depends on whether you are
73-
// building the library itself or not, routes to XX_EXPORT and XX_IMPORT.
74-
// Basically, you will need to do this for each shared library that you are
75-
// building, and the instruction is as follows: assuming that you are building
76-
// a library called libawesome.so. You should:
77-
// (1) for your cmake target (usually done by "add_library(awesome, ...)"),
78-
// define a macro called AWESOME_BUILD_MAIN_LIB using
79-
// target_compile_options.
80-
// (2) define the AWESOME_API macro similar to the one below.
81-
// And in the source file of your awesome library, use AWESOME_API to
82-
// annotate public symbols.
83-
84-
// Here, for the C10 library, we will define the macro C10_API for both import
85-
// and export.
86-
87-
// This one is being used by libc10.so
88-
#ifdef C10_BUILD_MAIN_LIB
89-
#define C10_API C10_EXPORT
90-
#else
91-
#define C10_API C10_IMPORT
92-
#endif
8+
#include <torch/standalone/macros/Export.h>
939

9410
// This one is being used by libtorch.so
9511
#ifdef CAFFE2_BUILD_MAIN_LIB
@@ -159,4 +75,4 @@
15975
#define C10_API_ENUM
16076
#endif
16177

162-
#endif // C10_MACROS_MACROS_H_
78+
#endif // C10_MACROS_EXPORT_H_

‎runtime/core/portable_type/c10/c10/macros/Macros.h

Lines changed: 17 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -403,11 +403,24 @@ __host__ __device__
403403
#endif // __SYCL_DEVICE_ONLY__
404404
}
405405
#endif // NDEBUG
406-
// ROCm disable kernel assert by default
406+
// ROCm disables kernel assert by default for performance considerations.
407+
// Though ROCm supports __assert_fail, it uses kernel printf which has
408+
// a non-negligible performance impact even if the assert condition is
409+
// never triggered. We choose to use abort() instead which will still
410+
// terminate the application but without a more useful error message.
407411
#if !defined(C10_USE_ROCM_KERNEL_ASSERT) and defined(USE_ROCM)
408-
#define CUDA_KERNEL_ASSERT(cond)
409-
#define CUDA_KERNEL_ASSERT_MSG(cond, msg)
410-
#define SYCL_KERNEL_ASSERT(cond)
412+
#define CUDA_KERNEL_ASSERT(cond) \
413+
if C10_UNLIKELY (!(cond)) { \
414+
abort(); \
415+
}
416+
#define CUDA_KERNEL_ASSERT_MSG(cond, msg) \
417+
if C10_UNLIKELY (!(cond)) { \
418+
abort(); \
419+
}
420+
#define SYCL_KERNEL_ASSERT(cond) \
421+
if C10_UNLIKELY (!(cond)) { \
422+
abort(); \
423+
}
411424
#else
412425
#define CUDA_KERNEL_ASSERT(cond) \
413426
if (C10_UNLIKELY(!(cond))) { \

‎runtime/core/portable_type/c10/c10/targets.bzl

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -124,7 +124,9 @@ def define_common_targets():
124124
"//executorch/...",
125125
"@EXECUTORCH_CLIENTS",
126126
],
127-
deps = select({
127+
exported_deps = [
128+
"//executorch/runtime/core/portable_type/c10/torch/standalone:torch_standalone_headers",
129+
] + select({
128130
"DEFAULT": [],
129131
# Half-inl.h depends on vec_half.h from ATen, but only when building for x86.
130132
"ovr_config//cpu:x86_64": [

‎runtime/core/portable_type/c10/torch/standalone/TARGETS

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
load(":targets.bzl", "define_common_targets")
2+
3+
oncall("executorch")
4+
5+
define_common_targets()

‎runtime/core/portable_type/c10/torch/standalone/macros/Export.h

Lines changed: 87 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,87 @@
1+
#pragma once
2+
3+
/* Header file to define the common scaffolding for exported symbols.
4+
*
5+
* Export is by itself a quite tricky situation to deal with, and if you are
6+
* hitting this file, make sure you start with the background here:
7+
* - Linux: https://gcc.gnu.org/wiki/Visibility
8+
* - Windows:
9+
* https://docs.microsoft.com/en-us/cpp/cpp/dllexport-dllimport?view=vs-2017
10+
*
11+
* Do NOT include this file directly. Instead, use c10/macros/Macros.h
12+
*/
13+
14+
// You do not need to edit this part of file unless you are changing the core
15+
// pytorch export abstractions.
16+
//
17+
// This part defines the C10 core export and import macros. This is controlled
18+
// by whether we are building shared libraries or not, which is determined
19+
// during build time and codified in c10/core/cmake_macros.h.
20+
// When the library is built as a shared lib, EXPORT and IMPORT will contain
21+
// visibility attributes. If it is being built as a static lib, then EXPORT
22+
// and IMPORT basically have no effect.
23+
24+
// As a rule of thumb, you should almost NEVER mix static and shared builds for
25+
// libraries that depend on c10. AKA, if c10 is built as a static library, we
26+
// recommend everything dependent on c10 to be built statically. If c10 is built
27+
// as a shared library, everything dependent on it should be built as shared. In
28+
// the PyTorch project, all native libraries shall use the macro
29+
// C10_BUILD_SHARED_LIB to check whether pytorch is building shared or static
30+
// libraries.
31+
32+
// For build systems that do not directly depend on CMake and directly build
33+
// from the source directory (such as Buck), one may not have a cmake_macros.h
34+
// file at all. In this case, the build system is responsible for providing
35+
// correct macro definitions corresponding to the cmake_macros.h.in file.
36+
//
37+
// In such scenarios, one should define the macro
38+
// C10_USING_CUSTOM_GENERATED_MACROS
39+
// to inform this header that it does not need to include the cmake_macros.h
40+
// file.
41+
42+
#ifdef _WIN32
43+
#define C10_HIDDEN
44+
#if defined(C10_BUILD_SHARED_LIBS)
45+
#define C10_EXPORT __declspec(dllexport)
46+
#define C10_IMPORT __declspec(dllimport)
47+
#else
48+
#define C10_EXPORT
49+
#define C10_IMPORT
50+
#endif
51+
#else // _WIN32
52+
#if defined(__GNUC__)
53+
#define C10_EXPORT __attribute__((__visibility__("default")))
54+
#define C10_HIDDEN __attribute__((__visibility__("hidden")))
55+
#else // defined(__GNUC__)
56+
#define C10_EXPORT
57+
#define C10_HIDDEN
58+
#endif // defined(__GNUC__)
59+
#define C10_IMPORT C10_EXPORT
60+
#endif // _WIN32
61+
62+
#ifdef NO_EXPORT
63+
#undef C10_EXPORT
64+
#define C10_EXPORT
65+
#endif
66+
67+
// Definition of an adaptive XX_API macro, that depends on whether you are
68+
// building the library itself or not, routes to XX_EXPORT and XX_IMPORT.
69+
// Basically, you will need to do this for each shared library that you are
70+
// building, and the instruction is as follows: assuming that you are building
71+
// a library called libawesome.so. You should:
72+
// (1) for your cmake target (usually done by "add_library(awesome, ...)"),
73+
// define a macro called AWESOME_BUILD_MAIN_LIB using
74+
// target_compile_options.
75+
// (2) define the AWESOME_API macro similar to the one below.
76+
// And in the source file of your awesome library, use AWESOME_API to
77+
// annotate public symbols.
78+
79+
// Here, for the C10 library, we will define the macro C10_API for both import
80+
// and export.
81+
82+
// This one is being used by libc10.so
83+
#ifdef C10_BUILD_MAIN_LIB
84+
#define C10_API C10_EXPORT
85+
#else
86+
#define C10_API C10_IMPORT
87+
#endif

‎runtime/core/portable_type/c10/torch/standalone/targets.bzl

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,14 @@
1+
load("@fbsource//xplat/executorch/build:runtime_wrapper.bzl", "runtime")
2+
3+
def define_common_targets():
4+
"""Defines targets that should be shared between fbcode and xplat.
5+
6+
The directory containing this targets.bzl file should also contain both
7+
TARGETS and BUCK files that call this function.
8+
"""
9+
10+
runtime.cxx_library(
11+
name = "torch_standalone_headers",
12+
exported_headers = glob(["**/*.h"]),
13+
header_namespace = "torch/standalone",
14+
)

0 commit comments

Comments
 (0)