Check-in PSA related auto-generated files to support online compiler. #11685

jainvikas8 · 2019-10-14T15:27:11Z

This PR is to add support for an online compiler which is currently failing LPC55S69 (PSA target).

Support the online compiler build for PSA targets, some files are auto-generated (LPC55S69_NS and ARM_MUSCA_A1_NS) by the script from the offline build system. Since these files are identical for these V8-M targets it would be good to check them in to support online compiler as they are COMPONENT_SPE and services related.

Offline build system only for PSA related targets will be affected, as the files are checked in, it will use those files instead of auto-generating them.

ciarmcom · 2019-10-14T17:00:28Z

@jainvikas8, thank you for your changes.
@ashok-rao @maclobdell @ARMmbed/mbed-os-tools @ARMmbed/mbed-os-maintainers please review.

jainvikas8 · 2019-11-05T10:33:28Z

Needs a preceding PR merge #11775

Tested with:
mbed test --compile -m LPC55S69_NS -n *compliance* -t GCC_ARM --run
mbed test --compile -m LPC55S69_NS -n tests-psa-* -t ARMC6 --run

0xc0170 · 2019-11-05T15:03:08Z

Needs a preceding PR merge #11775

Merged, this can proceed?

jainvikas8 · 2019-11-05T18:36:59Z

Needs a preceding PR merge #11775

Merged, this can proceed?

Yes, please start CI.

mark-edgeworth

This will change both the online and offline (mbed-cli) builds. It looks like the change removes all of the auto generation in favor of checked-in files for both online and offline building (ie the generator will not now be used at all). Is this intentional. If so, can you add a note to the PR description to indicate that offline builds are affected too?

urutva · 2019-11-06T11:40:46Z

tools/build.py

@@ -44,7 +44,6 @@
 from tools.utils import argparse_dir_not_parent
 from tools.utils import NoValidToolchainException
 from tools.utils import print_end_warnings
-from tools.psa import generate_psa_sources, clean_psa_autogen
 from tools.resources import OsAndSpeResourceFilter


@jainvikas8 Do we still need OsAndSpeResourceFilter ?

I'm not sure about this, as the main objective is to not make drastic changes to the offline build system and only remove auto-generation of header files.

urutva · 2019-11-06T11:57:13Z

tools/build.py

-                                source_dirs=options.source_dir,
-                                ignore_paths=[options.build_dir]
-                            )
-
                        resource_filter = None
                        if target.is_PSA_secure_target:


PSA secure target is built by tools/psa/release.py. Can we remove this?

Same comment as #11685 (comment)

urutva · 2019-11-06T11:58:05Z

tools/make.py

-                    source_dirs=options.source_dir,
-                    ignore_paths=[options.build_dir]
-                )
-
            resource_filter = None
            if target.is_PSA_secure_target:


PSA secure target is built by tools/psa/release.py right. Can we remove this?

Same comment as #11685 (comment)

urutva · 2019-11-06T11:58:21Z

tools/project.py

-                generate_psa_sources(source_dirs=options.source_dir,
-                                     ignore_paths=[]
-                )
-
            resource_filter = None
            if target.is_PSA_secure_target:


PSA secure target is built by tools/psa/release.py right. Can we remove this?

Same comment as #11685 (comment)

urutva · 2019-11-06T12:00:37Z

tools/test.py

@@ -244,12 +240,6 @@ def main():
                if target.is_PSA_secure_target:


Will mbed test be run on secure targets? If not then can we remove this?

#11685 (comment)

urutva · 2019-11-06T12:06:54Z

@jainvikas8 The tools/psa/release.py should be updated to check in the auto-generated files in the respective locations. The tools/psa/release.py is run as part of every mbed-os release, updating it will ensure that auto-generated and template files are in sync in mbed-os.

mark-edgeworth · 2019-11-06T14:08:49Z

Thank you for adding the comment about offline. I've no further objections.

components/TARGET_PSA/services/inc/mbed_spm_partitions.h

components/TARGET_PSA/services/inc/autogen_sid.h

Patater

LGTM

0xc0170 · 2019-11-08T10:19:43Z

@jainvikas8 Can you update your first comment based on the latest PR template (it should specify PR type) - to be certain about the release version.

0xc0170 · 2019-11-08T10:19:52Z

Started CI

jainvikas8 · 2019-11-08T10:39:33Z

@jainvikas8 The tools/psa/release.py should be updated to check in the auto-generated files in the respective locations. The tools/psa/release.py is run as part of every mbed-os release, updating it will ensure that auto-generated and template files are in sync in mbed-os.

@0xc0170 I'm looking to resolve this above comment, so there will be tool/psa/release.py changes when any secure target is built, it will freshly generate those PSA autogenerate files as part of new mbed-os-release. This would save us time to maintain these files.
TARGET_PSA
--TARGET_MBED_SPM
--COMPONENT_SPE
psa_setup.c
--TARGET_TFM
--COMPONENT_SPE
--inc
tfm_partition_defs.inc
tfm_partition_list.inc
tfm_service_list.inc
tfm_spm_signal_defs.h
--services
--inc
autogen_sid.h
mbed_spm_partitions.h

mbed-ci · 2019-11-08T11:36:38Z

Test run: SUCCESS

Summary: 11 of 11 test jobs passed
Build number : 1
Build artifacts

To support online compiler build for PSA targets, some files are auto-generated (LPC55S69_NS and ARM_MUSCA_A1_NS) by the script from offline build system. Since these files are identical for these V8-M targets it would be good to check them in to support online compiler as they are COMPONENT_SPE and services related. In folder: components/TARGET_PSA/ --TARGET_MBED_SPM --COMPONENT_SPE psa_setup.c --TARGET_TFM --COMPONENT_SPE --inc tfm_partition_defs.inc tfm_partition_list.inc tfm_service_list.inc tfm_spm_signal_defs.h --services --inc autogen_sid.h mbed_spm_partitions.h Signed-off-by: Vikas Katariya <[email protected]>

Only generate PSA headers/source related to components and services when Secure build is initiated during compile time of PSA targets. Let the Non-secure build rely on the checked-in files already present. Signed-off-by: Vikas Katariya <[email protected]>

Since the offline build is made to auto-generate PSA related components and services for Secure targets, we can change the output directory to update the files in the respective locations. TARGET_PSA --TARGET_MBED_SPM --COMPONENT_SPE psa_setup.c --TARGET_TFM --COMPONENT_SPE --inc tfm_partition_defs.inc tfm_partition_list.inc tfm_service_list.inc tfm_spm_signal_defs.h --services --inc autogen_sid.h mbed_spm_partitions.h The release script is been modified to commit these files if there are any changes detected when `--commit` argument is passed. Cleaning of auto-generated is been removed as it uses the main directory for its operations, but PSA auto-generation will work if any of the service and application-based manifests are updated. Signed-off-by: Vikas Katariya <[email protected]>

Signed-off-by: Vikas Katariya <[email protected]>

jainvikas8 · 2019-11-08T15:49:21Z

@0xc0170 Please, can you start the CI. Thanks.

0xc0170 · 2019-11-11T14:47:30Z

CI started

mbed-ci · 2019-11-11T16:49:20Z

Test run: SUCCESS

Summary: 11 of 11 test jobs passed
Build number : 2
Build artifacts

jainvikas8 changed the title ~~[WIP] Removing the PSA_AUTOGEN dependencies to support online compiler~~ [WIP] Check-in PSA_AUTOGEN to support online compiler for PSA targets. Oct 14, 2019

ciarmcom requested review from ashok-rao, maclobdell and a team October 14, 2019 17:00

ciarmcom added the needs: review label Oct 14, 2019

0xc0170 added the do not merge label Oct 14, 2019

jainvikas8 force-pushed the IOTSEC-1246 branch from 3fe983b to 7451445 Compare October 17, 2019 14:34

jainvikas8 force-pushed the IOTSEC-1246 branch 5 times, most recently from a8df8dd to 56ef8f3 Compare November 4, 2019 15:18

jainvikas8 changed the title ~~[WIP] Check-in PSA_AUTOGEN to support online compiler for PSA targets.~~ [WIP] Check-in PSA related auto-generated files to support online compiler. Nov 4, 2019

jainvikas8 changed the title ~~[WIP] Check-in PSA related auto-generated files to support online compiler.~~ Check-in PSA related auto-generated files to support online compiler. Nov 5, 2019

0xc0170 added needs: preceding PR and removed do not merge labels Nov 5, 2019

0xc0170 removed the needs: preceding PR label Nov 5, 2019

mark-edgeworth suggested changes Nov 6, 2019

View reviewed changes

urutva suggested changes Nov 6, 2019

View reviewed changes

mark-edgeworth approved these changes Nov 6, 2019

View reviewed changes

Patater reviewed Nov 7, 2019

View reviewed changes

components/TARGET_PSA/services/inc/mbed_spm_partitions.h Show resolved Hide resolved

components/TARGET_PSA/services/inc/autogen_sid.h Show resolved Hide resolved

Patater approved these changes Nov 7, 2019

View reviewed changes

0xc0170 requested a review from urutva November 8, 2019 10:18

0xc0170 approved these changes Nov 8, 2019

View reviewed changes

0xc0170 added needs: CI release-version: 5.14.2 and removed needs: review labels Nov 8, 2019

0xc0170 added needs: work and removed needs: CI labels Nov 8, 2019

jainvikas8 added 4 commits November 8, 2019 14:57

Documentation update for tools/psa

cdb5ebd

Signed-off-by: Vikas Katariya <[email protected]>

jainvikas8 force-pushed the IOTSEC-1246 branch from 56ef8f3 to cdb5ebd Compare November 8, 2019 15:01

jainvikas8 requested review from 0xc0170, mark-edgeworth and Patater November 8, 2019 15:31

Patater approved these changes Nov 8, 2019

View reviewed changes

0xc0170 approved these changes Nov 11, 2019

View reviewed changes

0xc0170 added ready for merge and removed needs: work labels Nov 11, 2019

0xc0170 merged commit 78d26a7 into ARMmbed:master Nov 11, 2019

0xc0170 removed the ready for merge label Nov 11, 2019

		@@ -244,12 +240,6 @@ def main():
		if target.is_PSA_secure_target:

Check-in PSA related auto-generated files to support online compiler. #11685

Check-in PSA related auto-generated files to support online compiler. #11685

Uh oh!

Conversation

jainvikas8 commented Oct 14, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ciarmcom commented Oct 14, 2019

Uh oh!

jainvikas8 commented Nov 5, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

0xc0170 commented Nov 5, 2019

Uh oh!

jainvikas8 commented Nov 5, 2019

Uh oh!

mark-edgeworth left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

urutva commented Nov 6, 2019

Uh oh!

mark-edgeworth commented Nov 6, 2019

Uh oh!

Uh oh!

Uh oh!

Patater left a comment

Choose a reason for hiding this comment

Uh oh!

0xc0170 commented Nov 8, 2019

Uh oh!

0xc0170 commented Nov 8, 2019

Uh oh!

jainvikas8 commented Nov 8, 2019

Uh oh!

mbed-ci commented Nov 8, 2019

Uh oh!

jainvikas8 commented Nov 8, 2019

Uh oh!

0xc0170 commented Nov 11, 2019

Uh oh!

mbed-ci commented Nov 11, 2019

Uh oh!

Uh oh!

jainvikas8 commented Oct 14, 2019 •

edited

Loading

jainvikas8 commented Nov 5, 2019 •

edited

Loading