Skip to content

feat: Yamato jobs to trigger format & UPM project pack #489

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 48 commits into from
May 11, 2022
Merged
Show file tree
Hide file tree
Changes from 45 commits
Commits
Show all changes
48 commits
Select commit Hold shift + click to select a range
0d50df7
formatting yamato job and dotnet format project
fernando-cortez Feb 3, 2022
68ebecb
trigger job added to pack and test projects
fernando-cortez Feb 4, 2022
6649e26
Merge branch 'develop' into feature/code-formatting-job
fernando-cortez Feb 4, 2022
dc17400
Merge branch 'feature/code-formatting-job' into feature/yamato-trigge…
fernando-cortez Feb 4, 2022
c20fab3
reordered project versions so 2020.3 is tested first
fernando-cortez Feb 24, 2022
c2d9836
merge develop
fernando-cortez Feb 24, 2022
10105e3
removing legacy blocker on ubuntu
fernando-cortez Apr 25, 2022
bbb5712
mobile build & run tests added & triggered by PRs
fernando-cortez Apr 26, 2022
7c23f9d
adding reference to project metafile
fernando-cortez Apr 26, 2022
27a09eb
dependency file name fix
fernando-cortez Apr 26, 2022
516dd0b
reordering name
fernando-cortez Apr 26, 2022
5850345
android commands syntax fixed
fernando-cortez Apr 26, 2022
4b75de7
job name definition descriptor
fernando-cortez Apr 26, 2022
890d315
name for build file
fernando-cortez Apr 26, 2022
a2a8552
adding matching build jobs for running jobs
fernando-cortez Apr 26, 2022
6c2b551
reference to project metafile
fernando-cortez Apr 26, 2022
2f9525d
Merge branch 'develop' into feature/yamato-trigger-format-project-pack
fernando-cortez Apr 26, 2022
fa0ae17
Merge branch 'feature/yamato-trigger-format-project-pack' into featur…
fernando-cortez Apr 26, 2022
812e5eb
using project path as name
fernando-cortez Apr 26, 2022
cc8b256
updated build params
fernando-cortez Apr 27, 2022
4a0d753
burst compilation removed on android, ios utr url param
fernando-cortez Apr 27, 2022
d0f0e68
disable burst .py file
fernando-cortez Apr 27, 2022
f010420
shifting order of operations
fernando-cortez Apr 28, 2022
6e77e97
test: using ngo params
fernando-cortez Apr 28, 2022
c32ebe0
python script enabling/disabling burst fixed, cleanup
fernando-cortez May 2, 2022
a354c0a
Merge branch 'develop' into feature/yamato-trigger-format-project-pack
fernando-cortez May 2, 2022
f709dcb
Merge branch 'feature/yamato-trigger-format-project-pack' into featur…
fernando-cortez May 2, 2022
1bf0b7f
specifying test filter
fernando-cortez May 2, 2022
f75bcf4
adding testfilter to builds and runs
fernando-cortez May 2, 2022
44c9593
android image change
fernando-cortez May 2, 2022
c050492
utr command param update for playerconnection
fernando-cortez May 2, 2022
aa36010
utr version?
fernando-cortez May 3, 2022
7e7b7f8
project name addition
fernando-cortez May 3, 2022
fbe1a10
adding editor path
fernando-cortez May 3, 2022
cbac55a
testing building with utr 0.12.0 as well
fernando-cortez May 3, 2022
140e404
formatting test
fernando-cortez May 3, 2022
0023594
joined mobile build and run to single yml file
fernando-cortez May 4, 2022
6f4b998
format
fernando-cortez May 4, 2022
43ab6fd
Merge branch 'feature/yamato-trigger-format-project-pack-mobile' into…
fernando-cortez May 4, 2022
c475c5a
Merge branch 'develop' into feature/yamato-trigger-format-project-pack
fernando-cortez May 4, 2022
1fa8875
test editors 2021 & trunk
fernando-cortez May 4, 2022
9ad8c35
just 2021
fernando-cortez May 4, 2022
c1f50b3
adding testfilter for only bossroom tests
fernando-cortez May 4, 2022
0aa265a
removing testfilter on packing
fernando-cortez May 5, 2022
528b7d4
fix to hanging tests
fernando-cortez May 5, 2022
8976485
code source added
fernando-cortez May 6, 2022
46c18eb
Merge branch 'develop' into feature/yamato-trigger-format-project-pack
fernando-cortez May 10, 2022
9a24c10
formatting files from RNSM PR
fernando-cortez May 11, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
27 changes: 27 additions & 0 deletions .yamato/_triggers.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
{% metadata_file .yamato/project.metafile %}
---

# Run all relevant tasks when a pull request targeting specified
# branches is created or updated.
pull_request_trigger:
name: Pull Request Trigger (main, develop, & release branches)
dependencies:
- .yamato/project-standards.yml#standards_{{ projects.first.name }}
{% for project in projects -%}
{% for platform in test_platforms -%}
# desktop platforms
- .yamato/project-tests.yml#test_{{ project.name }}_{{ project.test_editors.first }}_{{ platform.name }}
{% endfor -%}
# iOS
- .yamato/mobile-build-and-run.yml#mobile_test_ios_{{ project.name }}_{{ project.test_editors.first }}
# Android
- .yamato/mobile-build-and-run.yml#mobile_test_android_{{ project.name }}_{{ project.test_editors.first }}
{% endfor -%}
triggers:
cancel_old_ci: true
pull_requests:
- targets:
only:
- "main"
- "develop"
- "/release\/.*/"
93 changes: 93 additions & 0 deletions .yamato/disable-burst-if-requested.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,93 @@
import argparse
import json
import os


args = None
platform_plugin_definition = None


def resolve_target(platform):
resolved_target = platform
if 'StandaloneWindows' in platform:
resolved_target = 'StandaloneWindows'
elif 'StandaloneLinux' in platform:
resolved_target = 'StandaloneLinux64'

return resolved_target


def create_config(settings_path, platform):
config_name = os.path.join(settings_path, 'BurstAotSettings_{}.json'.format(resolve_target(platform)))
monobehaviour = {
'm_Enabled': True,
'm_EditorHideFlags': 0,
'm_Name': "",
'm_EditorClassIdentifier': 'Unity.Burst.Editor:Unity.Burst.Editor:BurstPlatformAotSettings',
'EnableOptimisations': True,
'EnableSafetyChecks': False,
'EnableBurstCompilation': True
}

data = {'MonoBehaviour': monobehaviour}
with open(config_name, 'w') as f:
json.dump(data, f)
return config_name


def get_or_create_AOT_config(project_path, platform):
settings_path = os.path.join(project_path, 'ProjectSettings')
if not os.path.isdir(settings_path):
os.mkdir(settings_path)
config_names = [os.path.join(settings_path, filename) for filename in os.listdir(settings_path) if filename.startswith("BurstAotSettings_{}".format(resolve_target(platform)))]
if not config_names:
return [create_config(settings_path, platform)]
return config_names


def disable_AOT(project_path, platform):
config_names = get_or_create_AOT_config(project_path, platform)
for config_name in config_names:
set_AOT(config_name, False)


def enable_AOT(project_path, platform):
config_names = get_or_create_AOT_config(project_path, platform)
for config_name in config_names:
set_AOT(config_name, True)


def set_AOT(config_file, status):
config = None
with open(config_file, 'r') as f:
config = json.load(f)

assert config is not None, 'AOT settings not found; did the burst-enabled build finish successfully?'

config['MonoBehaviour']['EnableBurstCompilation'] = status
with open(config_file, 'w') as f:
json.dump(config, f)


def main():
enable_burst = os.environ.get('ENABLE_BURST_COMPILATION', 'true').strip().lower()
if enable_burst == 'true':
print('BURST COMPILATION: ENABLED')
elif enable_burst == 'false':
print('BURST COMPILATION: DISABLED')
disable_AOT(args.project_path, args.platform)
else:
sys.exit('BURST COMPILATION: unexpected value: {}'.format(enable_burst))


def parse_args():
global args
parser = argparse.ArgumentParser(description='This tool disables burst AOT compilation')
parser.add_argument('--project-path', help='Specify the location of the unity project.')
parser.add_argument('--platform', help="Platform to be used to run the build.")
args = parser.parse_args()


if __name__ == '__main__':
parse_args()
main()
127 changes: 127 additions & 0 deletions .yamato/mobile-build-and-run.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,127 @@
{% metadata_file .yamato/project.metafile %}
---

{% for project in projects -%}
{% for editor in project.test_editors -%}
Build_Player_With_Tests_iOS_{{ project.name }}_{{ editor }}:
name: build {{ project.name }} - {{ editor }} on iOS
agent:
type: Unity::VM::osx
image: mobile/macos-10.15-testing:stable
flavor: b1.large

commands:
- pip install unity-downloader-cli --index-url https://artifactory.prd.it.unity3d.com/artifactory/api/pypi/pypi/simple --upgrade
- unity-downloader-cli -c Editor -c iOS -u {{ editor }} --fast --wait
- curl -s https://artifactory.prd.it.unity3d.com/artifactory/unity-tools-local/utr-standalone/utr --output utr
- chmod +x ./utr
- ./utr --suite=playmode --platform=iOS --editor-location=.Editor --testproject={{ project.path }} --player-save-path=build/players --artifacts_path=build/logs --build-only --testfilter=Unity.Multiplayer.Samples.BossRoom.Tests.Runtime

artifacts:
players:
paths:
- "build/players/**"
logs:
paths:
- "build/logs/**"
{% endfor -%}
{% endfor -%}

{% for project in projects -%}
{% for editor in project.test_editors -%}
Build_Player_With_Tests_Android_{{ project.name }}_{{ editor }}:
name: build {{ project.name }} - {{ editor }} on Android
agent:
type: Unity::VM
# Any generic image can be used, no need to have Android tools in the image for building
# All Android tools will be downloaded by unity-downloader-cli
image: desktop/android-execution-r19:v0.1.1-860408
flavor: b1.xlarge

commands:
# Download unity-downloader-cli
- pip install unity-downloader-cli --index-url https://artifactory.prd.it.unity3d.com/artifactory/api/pypi/pypi/simple --upgrade
- curl -s https://artifactory.prd.it.unity3d.com/artifactory/unity-tools/utr-standalone/utr.bat --output utr.bat
- python .yamato/disable-burst-if-requested.py --project-path {{ project.path }} --platform Android
- unity-downloader-cli -c Editor -c Android -u {{ editor }} --fast --wait
# Build player(s)
- set UTR_VERSION=0.12.0
- ./utr.bat --suite=playmode --platform=Android --editor-location=.Editor --testproject={{ project.path }} --player-save-path=build/players --artifacts_path=build/logs --scripting-backend=mono --build-only --testfilter=Unity.Multiplayer.Samples.BossRoom.Tests.Runtime
artifacts:
players:
paths:
- "build/players/**"
logs:
paths:
- "build/logs/**"
variables:
CI: true
ENABLE_BURST_COMPILATION: False
{% endfor -%}
{% endfor -%}

# For every editor version, run iOS project tests without
# running package tests too since they are handled on their respective jobs
{% for project in projects -%}
{% for editor in project.test_editors -%}
mobile_test_ios_{{ project.name }}_{{ editor }}:
name: {{ project.name }} mobile project tests - {{ editor }} on iOS
agent:
type: Unity::mobile::iPhone
image: mobile/macos-10.15-testing:latest
flavor: b1.medium

# Skip repository cloning
skip_checkout: true

# Set a dependency on the build job
dependencies:
- .yamato/mobile-build-and-run.yml#Build_Player_With_Tests_iOS_{{ project.name }}_{{ editor }}

commands:
# Download standalone UnityTestRunner
- curl -s https://artifactory.prd.it.unity3d.com/artifactory/unity-tools-local/utr-standalone/utr --output utr
# Give UTR execution permissions
- chmod +x ./utr
# Run the test build on the device
- ./utr --suite=playmode --platform=iOS --player-load-path=build/players --artifacts_path=build/test-results --testfilter=Unity.Multiplayer.Samples.BossRoom.Tests.Runtime

artifacts:
logs:
paths:
- "build/test-results/**"
{% endfor -%}
{% endfor -%}

# For every editor version, run Android project tests without
# running package tests too since they are handled on their respective jobs
{% for project in projects -%}
{% for editor in project.test_editors -%}
mobile_test_android_{{ project.name }}_{{ editor }}:
name: {{ project.name }} mobile project tests - {{ editor }} on Android
agent:
type: Unity::mobile::shield
image: mobile/android-execution-r19:stable
flavor: b1.medium

# Skip repository cloning
skip_checkout: true
# Set a dependency on the build job
dependencies:
- .yamato/mobile-build-and-run.yml#Build_Player_With_Tests_Android_{{ project.name }}_{{ editor }}
commands:
# Download standalone UnityTestRunner
- curl -s https://artifactory.prd.it.unity3d.com/artifactory/unity-tools/utr-standalone/utr.bat --output utr.bat
- |
set ANDROID_DEVICE_CONNECTION=%BOKKEN_DEVICE_IP%
start %ANDROID_SDK_ROOT%\platform-tools\adb.exe connect %BOKKEN_DEVICE_IP%
start %ANDROID_SDK_ROOT%\platform-tools\adb.exe devices
set UTR_VERSION=0.12.0
./utr --artifacts_path=build/test-results --testproject={{ project.path }} --editor-location=.Editor --reruncount=2 --suite=playmode --platform=android --player-connection-ip=%BOKKEN_HOST_IP% --player-load-path=build/players --testfilter=Unity.Multiplayer.Samples.BossRoom.Tests.Runtime
# Set uploadable artifact paths
artifacts:
logs:
paths:
- "build/test-results/**"
{% endfor -%}
{% endfor -%}
17 changes: 17 additions & 0 deletions .yamato/project-pack.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
{% metadata_file .yamato/project.metafile %}
---
{% for project in projects -%}
pack_{{ project.name }}:
name: Pack {{ project.name }}
agent:
type: Unity::VM
image: package-ci/ubuntu:stable
flavor: b1.small
commands:
- npm install upm-ci-utils@stable -g --registry https://artifactory.prd.cds.internal.unity3d.com/artifactory/api/npm/upm-npm
- upm-ci project pack --project-path {{ project.path }}
artifacts:
packages:
paths:
- "upm-ci~/packages/**/*"
{% endfor -%}
29 changes: 29 additions & 0 deletions .yamato/project-tests.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
{% metadata_file .yamato/project.metafile %}
---

# For every platform and editor version, run its project tests without
# running package tests too since they are handled on their respective
# jobs
{% for project in projects -%}
{% for editor in project.test_editors -%}
{% for platform in test_platforms -%}
test_{{ project.name }}_{{ editor }}_{{ platform.name }}:
name : {{ project.name }} project tests - {{ editor }} on {{ platform.name }}
agent:
type: {{ platform.type }}
image: {{ platform.image }}
flavor: {{ platform.flavor}}
commands:
- npm install upm-ci-utils@stable -g --registry https://artifactory.prd.cds.internal.unity3d.com/artifactory/api/npm/upm-npm
- pip install unity-downloader-cli --index-url https://artifactory.prd.it.unity3d.com/artifactory/api/pypi/pypi/simple
- unity-downloader-cli -u {{ editor }} -c editor -w --fast
- upm-ci project test -u {{ editor }} --project-path {{ project.path }} --type project-tests --extra-utr-arg=--testfilter=Unity.Multiplayer.Samples.BossRoom.Tests.Runtime
artifacts:
logs:
paths:
- "upm-ci~/test-results/**/*"
dependencies:
- .yamato/project-pack.yml#pack_{{ project.name }}
{% endfor -%}
{% endfor -%}
{% endfor -%}
5 changes: 1 addition & 4 deletions .yamato/project.metafile
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,4 @@ projects:
- name: com.unity.multiplayer.samples.coop
path: Packages/com.unity.multiplayer.samples.coop
test_editors:
- 2021.1
- 2021.2
- 2020.3
- trunk
- 2021.3
10 changes: 5 additions & 5 deletions Assets/Tests/Runtime/HostAndDisconnectTest.cs
Original file line number Diff line number Diff line change
Expand Up @@ -41,14 +41,14 @@ IEnumerator WaitUntilMainMenuSceneIsLoaded()
// MainMenu is loaded as soon as Startup scene is launched, validate it is loaded
yield return new TestUtilities.WaitForSceneLoad(k_MainMenuSceneName);

yield return new WaitForEndOfFrame();
yield return null;
}

IEnumerator WaitUntilCharacterIsSelectedAndReady(int playerIndex)
{
yield return new TestUtilities.WaitForSceneLoad(k_CharSelectSceneName);

yield return new WaitForEndOfFrame();
yield return null;

// select a Character
var seatObjectName = $"PlayerSeat ({playerIndex})";
Expand All @@ -61,7 +61,7 @@ IEnumerator WaitUntilCharacterIsSelectedAndReady(int playerIndex)
uiCharSelectPlayerSeat.OnClicked();

// selecting a class will enable the "Ready" button, next frame it is selectable
yield return new WaitForEndOfFrame();
yield return null;

// hit ready
ClientCharSelectState.Instance.OnPlayerClickedReady();
Expand Down Expand Up @@ -128,13 +128,13 @@ public IEnumerator IP_HostAndDisconnect_Valid([ValueSource(nameof(s_PlayerIndice
// select "DIRECT IP" button
clientMainMenuState.OnDirectIPClicked();

yield return new WaitForEndOfFrame();
yield return null;

// select the "HOST" button
ipHostingUI.OnCreateClick();

// confirming hosting will initialize the hosting process; next frame the results will be ready
yield return new WaitForEndOfFrame();
yield return null;

// verify hosting is successful
Assert.That(m_NetworkManager.IsListening && m_NetworkManager.IsHost);
Expand Down