Skip to content

Commit c33d7df

Browse files
beccadaxbeccadax
committed
Prototype an SDK validation test
This test compares two API digests of large chunks of the installed SDK—one from the toolchain’s API digester, one from the just-built API digester—and flags any changes in behavior that would affect source compatibility.
1 parent 9b10ab3 commit c33d7df

File tree

2 files changed

+230
-0
lines changed

2 files changed

+230
-0
lines changed

‎validation-test/WholeSDK/Inputs/intentionally-changed-apis.txt

Lines changed: 192 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,192 @@
1+
/* Generic Signature Changes */
2+
3+
/* RawRepresentable Changes */
4+
5+
/* Removed Decls */
6+
7+
// FIXME: Cause unknown; possibly a change in headers in the clang resource dir.
8+
Var __GNUC_VA_LIST has been removed
9+
10+
/* Moved Decls */
11+
12+
/* Renamed Decls */
13+
14+
/* Type Changes */
15+
16+
/* Decl Attribute changes */
17+
18+
/* Fixed-layout Type Changes */
19+
20+
/* Protocol Conformance Change */
21+
22+
// SE-0331: These imported structs have pointer fields, so they became
23+
// non-Sendable once the Unsafe*Pointer types were no longer Sendable.
24+
Struct DIR has removed conformance to Sendable
25+
Struct FTS has removed conformance to Sendable
26+
Struct FTS.__Unnamed_union___Anonymous_field8 has removed conformance to Sendable
27+
Struct _RuneEntry has removed conformance to Sendable
28+
Struct _RuneLocale has removed conformance to Sendable
29+
Struct _RuneRange has removed conformance to Sendable
30+
Struct __Reply__host_lockgroup_info_t has removed conformance to Sendable
31+
Struct __Reply__host_processor_info_t has removed conformance to Sendable
32+
Struct __Reply__host_processor_sets_t has removed conformance to Sendable
33+
Struct __Reply__host_processors_t has removed conformance to Sendable
34+
Struct __Reply__host_virtual_physical_table_info_t has removed conformance to Sendable
35+
Struct __Reply__kext_request_t has removed conformance to Sendable
36+
Struct __Reply__kmod_control_t has removed conformance to Sendable
37+
Struct __Reply__kmod_get_info_t has removed conformance to Sendable
38+
Struct __Reply__mach_memory_info_t has removed conformance to Sendable
39+
Struct __Reply__mach_port_get_set_status_t has removed conformance to Sendable
40+
Struct __Reply__mach_port_names_t has removed conformance to Sendable
41+
Struct __Reply__mach_port_space_info_t has removed conformance to Sendable
42+
Struct __Reply__mach_ports_lookup_t has removed conformance to Sendable
43+
Struct __Reply__mach_vm_read_t has removed conformance to Sendable
44+
Struct __Reply__mach_vm_region_info_64_t has removed conformance to Sendable
45+
Struct __Reply__mach_vm_region_info_t has removed conformance to Sendable
46+
Struct __Reply__mach_zone_info_t has removed conformance to Sendable
47+
Struct __Reply__processor_set_tasks_t has removed conformance to Sendable
48+
Struct __Reply__processor_set_tasks_with_flavor_t has removed conformance to Sendable
49+
Struct __Reply__processor_set_threads_t has removed conformance to Sendable
50+
Struct __Reply__task_get_dyld_image_infos_t has removed conformance to Sendable
51+
Struct __Reply__task_get_emulation_vector_t has removed conformance to Sendable
52+
Struct __Reply__task_threads_t has removed conformance to Sendable
53+
Struct __Reply__task_zone_info_t has removed conformance to Sendable
54+
Struct __Reply__vm_mapped_pages_info_t has removed conformance to Sendable
55+
Struct __Reply__vm_read_t has removed conformance to Sendable
56+
Struct __Request__host_security_create_task_token_t has removed conformance to Sendable
57+
Struct __Request__kext_request_t has removed conformance to Sendable
58+
Struct __Request__kmod_control_t has removed conformance to Sendable
59+
Struct __Request__mach_port_construct_t has removed conformance to Sendable
60+
Struct __Request__mach_ports_register_t has removed conformance to Sendable
61+
Struct __Request__mach_vm_write_t has removed conformance to Sendable
62+
Struct __Request__task_create_t has removed conformance to Sendable
63+
Struct __Request__task_register_dyld_image_infos_t has removed conformance to Sendable
64+
Struct __Request__task_set_emulation_vector_t has removed conformance to Sendable
65+
Struct __Request__task_unregister_dyld_image_infos_t has removed conformance to Sendable
66+
Struct __Request__vm_write_t has removed conformance to Sendable
67+
Struct __darwin_pthread_handler_rec has removed conformance to Sendable
68+
Struct __darwin_sigaltstack has removed conformance to Sendable
69+
Struct __darwin_ucontext has removed conformance to Sendable
70+
Struct __darwin_ucontext64 has removed conformance to Sendable
71+
Struct __msfilterreq has removed conformance to Sendable
72+
Struct __nl_cat_d has removed conformance to Sendable
73+
Struct __sFILE has removed conformance to Sendable
74+
Struct __sbuf has removed conformance to Sendable
75+
Struct __shmid_ds_new has removed conformance to Sendable
76+
Struct __siginfo has removed conformance to Sendable
77+
Struct _ftsent has removed conformance to Sendable
78+
Struct _malloc_zone_t has removed conformance to Sendable
79+
Struct _opaque_pthread_t has removed conformance to Sendable
80+
Struct _pcred has removed conformance to Sendable
81+
Struct addrinfo has removed conformance to Sendable
82+
Struct aiocb has removed conformance to Sendable
83+
Struct au_arb_t has removed conformance to Sendable
84+
Struct au_arg32_t has removed conformance to Sendable
85+
Struct au_arg64_t has removed conformance to Sendable
86+
Struct au_cert_hash_t has removed conformance to Sendable
87+
Struct au_class_ent has removed conformance to Sendable
88+
Struct au_event_ent has removed conformance to Sendable
89+
Struct au_execarg_t has removed conformance to Sendable
90+
Struct au_execenv_t has removed conformance to Sendable
91+
Struct au_file_t has removed conformance to Sendable
92+
Struct au_identity_t has removed conformance to Sendable
93+
Struct au_invalid_t has removed conformance to Sendable
94+
Struct au_krb5_principal_t has removed conformance to Sendable
95+
Struct au_opaque_t has removed conformance to Sendable
96+
Struct au_path_t has removed conformance to Sendable
97+
Struct au_sdev_handle has removed conformance to Sendable
98+
Struct au_session has removed conformance to Sendable
99+
Struct au_text_t has removed conformance to Sendable
100+
Struct au_user_ent has removed conformance to Sendable
101+
Struct au_zonename_t has removed conformance to Sendable
102+
Struct bpf_program has removed conformance to Sendable
103+
Struct ctlname has removed conformance to Sendable
104+
Struct datum has removed conformance to Sendable
105+
Struct dk_error_description_t has removed conformance to Sendable
106+
Struct dk_format_capacities_t has removed conformance to Sendable
107+
Struct dk_provision_status_t has removed conformance to Sendable
108+
Struct dk_unmap_t has removed conformance to Sendable
109+
Struct dl_info has removed conformance to Sendable
110+
Struct entry has removed conformance to Sendable
111+
Struct eproc has removed conformance to Sendable
112+
Struct eventreq has removed conformance to Sendable
113+
Struct exception has removed conformance to Sendable
114+
Struct extern_proc has removed conformance to Sendable
115+
Struct fchecklv has removed conformance to Sendable
116+
Struct fsignatures has removed conformance to Sendable
117+
Struct fssearchblock has removed conformance to Sendable
118+
Struct fstab has removed conformance to Sendable
119+
Struct glob_t has removed conformance to Sendable
120+
Struct glob_t.__Unnamed_union___Anonymous_field5 has removed conformance to Sendable
121+
Struct gmonparam has removed conformance to Sendable
122+
Struct group has removed conformance to Sendable
123+
Struct hfs_mount_args has removed conformance to Sendable
124+
Struct hostent has removed conformance to Sendable
125+
Struct iconv_allocation_t has removed conformance to Sendable
126+
Struct iconv_fallbacks has removed conformance to Sendable
127+
Struct iconv_hooks has removed conformance to Sendable
128+
Struct if_clonereq has removed conformance to Sendable
129+
Struct if_nameindex has removed conformance to Sendable
130+
Struct ifaddrs has removed conformance to Sendable
131+
Struct ifdrv has removed conformance to Sendable
132+
Struct ifmaddrs has removed conformance to Sendable
133+
Struct ifmedia_description has removed conformance to Sendable
134+
Struct ifmediareq has removed conformance to Sendable
135+
Struct ifqueue has removed conformance to Sendable
136+
Struct iovec has removed conformance to Sendable
137+
Struct kevent has removed conformance to Sendable
138+
Struct kinfo_proc has removed conformance to Sendable
139+
Struct klist has removed conformance to Sendable
140+
Struct kmod_info has removed conformance to Sendable
141+
Struct kmod_reference has removed conformance to Sendable
142+
Struct lconv has removed conformance to Sendable
143+
Struct mach_msg_ool_descriptor_t has removed conformance to Sendable
144+
Struct mach_msg_ool_ports_descriptor_t has removed conformance to Sendable
145+
Struct mig_subsystem has removed conformance to Sendable
146+
Struct mig_symtab has removed conformance to Sendable
147+
Struct msg has removed conformance to Sendable
148+
Struct msgbuf has removed conformance to Sendable
149+
Struct msghdr has removed conformance to Sendable
150+
Struct ndrv_protocol_desc has removed conformance to Sendable
151+
Struct netent has removed conformance to Sendable
152+
Struct option has removed conformance to Sendable
153+
Struct passwd has removed conformance to Sendable
154+
Struct pipebuf has removed conformance to Sendable
155+
Struct port_obj_tentry has removed conformance to Sendable
156+
Struct protoent has removed conformance to Sendable
157+
Struct pseminfo has removed conformance to Sendable
158+
Struct pshminfo has removed conformance to Sendable
159+
Struct pstats has removed conformance to Sendable
160+
Struct qelem has removed conformance to Sendable
161+
Struct rb_node has removed conformance to Sendable
162+
Struct rb_tree has removed conformance to Sendable
163+
Struct rb_tree_ops_t has removed conformance to Sendable
164+
Struct regex_t has removed conformance to Sendable
165+
Struct routine_descriptor has removed conformance to Sendable
166+
Struct rpc_routine_descriptor has removed conformance to Sendable
167+
Struct rpc_signature has removed conformance to Sendable
168+
Struct rpc_subsystem has removed conformance to Sendable
169+
Struct rpcent has removed conformance to Sendable
170+
Struct rslvmulti_req has removed conformance to Sendable
171+
Struct rt_addrinfo has removed conformance to Sendable
172+
Struct sa_endpoints has removed conformance to Sendable
173+
Struct sbuf has removed conformance to Sendable
174+
Struct servent has removed conformance to Sendable
175+
Struct sf_hdtr has removed conformance to Sendable
176+
Struct sigevent has removed conformance to Sendable
177+
Struct sigstack has removed conformance to Sendable
178+
Struct tm has removed conformance to Sendable
179+
Struct tokenstr has removed conformance to Sendable
180+
Struct tokenstr.__Unnamed_union_tt has removed conformance to Sendable
181+
Struct uprof has removed conformance to Sendable
182+
Struct vfsidctl has removed conformance to Sendable
183+
Struct vfsstatfs has removed conformance to Sendable
184+
Struct vmspace has removed conformance to Sendable
185+
Struct wordexp_t has removed conformance to Sendable
186+
187+
/* Protocol Requirement Change */
188+
189+
/* Class Inheritance Change */
190+
191+
/* Others */
192+

‎validation-test/WholeSDK/api-digester.test

Lines changed: 38 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,38 @@
1+
// This test compares the behavior of the toolchain compiler to the just-built
2+
// compiler using the currently installed SDK. That means its behavior depends
3+
// on the installed toolchain compiler and SDK. If you have trouble reproducing
4+
// a failure locally, make sure you're using the same Xcode that was used when
5+
// the test failure occurred.
6+
7+
// Some failures will indicate regressions, but others will indicate intentional
8+
// changes to compiler behavior. If a failure is caused by an intentional
9+
// change, add it to the appropriate section of
10+
// 'Inputs/intentionally-changed-apis.txt'.
11+
12+
// REQUIRES: VENDOR=apple
13+
14+
// RUN: %empty-directory(%t)
15+
// RUN: %empty-directory(%t/module-cache)
16+
17+
// Generate a baseline digest using the toolchain API digester, capturing how the host toolchain imports APIs.
18+
// RUN: time env xcrun swift-api-digester -target %target-triple -dump-sdk -module-list-file %s -o %t/baseline.json -module-cache-path %t/module-cache -sdk %sdk -swift-version 5 -avoid-location -avoid-tool-args
19+
20+
// Generate a digest using the just-built API digester, capturing how the just-built compiler imports APIs.
21+
// RUN: time %api-digester -dump-sdk -module-list-file %s -o %t/result.json -module-cache-path %t/module-cache -sdk %sdk -swift-version 5 -avoid-location -avoid-tool-args
22+
23+
// Compare them.
24+
// RUN: time %api-digester -diagnose-sdk -module-list-file %s -input-paths %t/baseline.json -input-paths %t/result.json -module-cache-path %t/module-cache -sdk %sdk -swift-version 5 -avoid-location -avoid-tool-args -breakage-allowlist-path %S/Inputs/intentionally-changed-apis.txt
25+
26+
// FIXME: Forces the test to fail so we can see the output from `time`
27+
// RUN: false
28+
29+
// List of modules to test. Please keep it alphabetical.
30+
// Missing modules will not cause the test to fail.
31+
32+
AppKit
33+
Darwin
34+
Dispatch
35+
Foundation
36+
SwiftUI
37+
UIKit
38+
WatchKit

0 commit comments

Comments
 (0)