CMake: Fix cmake response files #13427

0xc0170 · 2020-08-13T16:31:37Z

Summary of changes

Testing this changes with Travis jobs, as there are implications.

Impact of changes

Migration actions required

Documentation

Pull request type

Patch update (Bug fix / Target update / Docs update / Test update / Refactor)
Feature update (New feature / Functionality change / New API)
Major update (Breaking change E.g. Return code change / API behaviour change)

Test results

No Tests required for this change (E.g docs only update)
Covered by existing mbed-os tests (Greentea or Unittest)
Tests / results supplied as part of this PR

Reviewers

ciarmcom · 2020-08-13T17:00:35Z

@0xc0170, thank you for your changes.
@ARMmbed/mbed-os-maintainers please review.

0xc0170 · 2020-08-14T08:40:10Z

I could reproduce ARMClang issue (different error) - I'll check why response files are causing build failures..

0xc0170 · 2020-08-14T09:37:57Z

Found it: https://developer.arm.com/documentation/101754/0614/armclang-Reference/armclang-Command-line-Options/-file?search=5eec717fe24a5e02d07b26d8 . ARMClang on windows using rsp files needs paths to be escaped 🤕

If they are not (what works on cmd), this happens ignoring nonexistent directory "..mbed-osconnectivitynanostacksal-stack-nanostacksource6LoWPANFragmentation"

@hugueskamba I am getting different error (windows related), please look at the --cpu error on MacOS, I could not reporduce locally so far.

0xc0170 · 2020-08-14T09:59:55Z

I created an issue for CMake https://gitlab.kitware.com/cmake/cmake/-/issues/21093

mergify · 2020-08-15T16:57:58Z

This PR cannot be merged due to conflicts. Please rebase to resolve them.

0xc0170 · 2020-08-17T08:28:31Z

@hugueskamba looks like our branch is broken for nrf52 after the latest rebase. I'll look at it now

0xc0170 · 2020-08-17T11:22:51Z

Rebased to get in nrf52 fix.

0xc0170 · 2020-08-17T14:03:24Z

The response files introduced couple of bugs to the system.

My findings so far on windows, using cmake 3.18.1, ninja 1.10.0, make 4.3, blinky example:

GCC ARM:

ninja: OK
make: fails (the application needs to request response files as well, not sure about why, I created cmake issue https://gitlab.kitware.com/cmake/cmake/-/issues/21104 . It can be fixed if an app adds response file set() to the cmake = workaround

ARMClang:

ninja: fails , escaped paths issue, created cmake issue https://gitlab.kitware.com/cmake/cmake/-/issues/21093
make: failing with error coming from assembly file `Error: A1067E: Output file specified as 'mbed-os/platform/source/TARGET_CORTEX_M/TOOLCHAIN_ARM/except.S'
, but it has already been specified as 'CMakeFiles/mbed-os.dir/platform/source/TARGET_CORTEX_M/TOOLCHAIN_ARM/except.o'

mergify · 2020-08-18T15:33:21Z

This PR cannot be merged due to conflicts. Please rebase to resolve them.

0xc0170 · 2020-08-21T14:29:02Z

@hugueskamba Please review.
We shall enable this to unblock windows users and fix any bugs that appear as a result. Do you still have problems compiling ARMClang?

hugueskamba

ARM toolchain does not build successfully with Make or Ninja.

This commit should not be merged until that is fixed because I work mostly with the ARM toolchain. Since we do not have CI check with the ARM toolchain , I ensure that it does not get broken. I also prefer it because it builds significantly much faster than GCC_ARM.

Ninja failure:

1 warning generated.
[405/846] Building ASM object mbed-os/CMakeFiles/mbed-os.dir/rtos/source/TARGET_CORTEX/rtx5/RTX/Source/TOOLCHAIN_ARM/TARGET_RTOS_M4_M7/irq_cm4f.o
FAILED: mbed-os/CMakeFiles/mbed-os.dir/rtos/source/TARGET_CORTEX/rtx5/RTX/Source/TOOLCHAIN_ARM/TARGET_RTOS_M4_M7/irq_cm4f.o 
"/Library/Application Support/Mbed Studio/mbed-studio-tools/ac6/bin/armasm"  @mbed-os/CMakeFiles/mbed-os.dir/rtos/source/TARGET_CORTEX/rtx5/RTX/Source/TOOLCHAIN_ARM/TARGET_RTOS_M4_M7/irq_cm4f.o.rsp -o mbed-os/CMakeFiles/mbed-os.dir/rtos/source/TARGET_CORTEX/rtx5/RTX/Source/TOOLCHAIN_ARM/TARGET_RTOS_M4_M7/irq_cm4f.o ../mbed-os/rtos/source/TARGET_CORTEX/rtx5/RTX/Source/TOOLCHAIN_ARM/TARGET_RTOS_M4_M7/irq_cm4f.S
Fatal error: A9912E: No --cpu selected
1 Error, 0 Warnings
[410/846] Building CXX object mbed-os/CMakeFiles/mbed-os.dir/rtos/source/TARGET_CORTEX/mbed_rtx_idle.o
ninja: build stopped: subcommand failed.

Make failure:

[ 80%] Building ASM object mbed-os/CMakeFiles/mbed-os.dir/platform/source/TARGET_CORTEX_M/TOOLCHAIN_ARM/except.o
Error: A1067E: Output file specified as '/Users/hugkam01/projects/mbed/examples/mbed-os-example-blinky/mbed-os/platform/source/TARGET_CORTEX_M/TOOLCHAIN_ARM/except.S', but it has already been specified as 'CMakeFiles/mbed-os.dir/platform/source/TARGET_CORTEX_M/TOOLCHAIN_ARM/except.o'
armclang: error: no input files
Fatal error: A1905U: Pre-processor step failed for '@CMakeFiles/mbed-os.dir/includes_ASM.rsp'
2 Errors, 0 Warnings
make[2]: *** [mbed-os/CMakeFiles/mbed-os.dir/platform/source/TARGET_CORTEX_M/TOOLCHAIN_ARM/except.o] Error 8
make[1]: *** [mbed-os/CMakeFiles/mbed-os.dir/all] Error 2
make: *** [all] Error 2

0xc0170 · 2020-09-03T13:38:43Z

Once #13540 and #13534 are in (they are required for this PR), this should be simple change to get in. Only limitation I know currently will be Ninja windows bug in CMake: https://gitlab.kitware.com/cmake/cmake/-/issues/21093

I rebased this one, to clean it up. Only one commit adding response files for now.

mergify · 2020-09-03T15:44:00Z

This PR cannot be merged due to conflicts. Please rebase to resolve them.

As Mbed OS is built as whole, we have long paths again. This is known issue with windows. To fix the paths, we need to use response files. Ninja is special, needs to be forced to use long paths.

0xc0170 · 2020-09-03T15:51:35Z

Once #13540 and #13534

Both merged, I rebased it. @hugueskamba can you test if this works with armclang6 (ninja, makefiles) ?

hugueskamba · 2020-09-03T17:02:04Z

Once #13540 and #13534

Both merged, I rebased it. @hugueskamba can you test if this works with armclang6 (ninja, makefiles) ?

Mac OS, ARMClang 6.13, cmake version 3.18.0
Ninja: Success
Make: Failed

Warning: L3912W: Option 'legacyalign' is deprecated.
Error: L6218E: Undefined symbol mbed_fault_context (referred from mbed-os/CMakeFiles/mbed-os.dir/platform/source/TARGET_CORTEX_M/TOOLCHAIN_ARM/except.o).
Finished: 0 information, 1 warning and 1 error messages.
make[2]: *** [mbed-os-example-blinky.elf] Error 1
make[1]: *** [CMakeFiles/mbed-os-example-blinky.dir/all] Error 2
make: *** [all] Error 2

Linux, ARMClang 6.13, cmake version 3.18.2
Ninja: Success
Make: Success

hugueskamba · 2020-09-03T17:25:16Z

Once #13540 and #13534

Both merged, I rebased it. @hugueskamba can you test if this works with armclang6 (ninja, makefiles) ?

Mac OS, ARMClang 6.13, cmake version 3.18.0
Ninja: Success
Make: Failed
Warning: L3912W: Option 'legacyalign' is deprecated.
Error: L6218E: Undefined symbol mbed_fault_context (referred from mbed-os/CMakeFiles/mbed-os.dir/platform/source/TARGET_CORTEX_M/TOOLCHAIN_ARM/except.o).
Finished: 0 information, 1 warning and 1 error messages.
make[2]: *** [mbed-os-example-blinky.elf] Error 1
make[1]: *** [CMakeFiles/mbed-os-example-blinky.dir/all] Error 2
make: *** [all] Error 2
Linux, ARMClang 6.13, cmake version 3.18.2
Ninja: Success
Make: Success

@0xc0170

I have updated CMake on Mac OS from 3.18.0 to 3.18.2 and here are the results:

Mac OS, ARMClang 6.13, cmake version 3.18.2
Ninja: Success
Make: Success

hugueskamba

I recommend we update the minimum required CMake in this PR to 3.18.2 keeping in mind that we will need to update it again once a newer version is released due to the bug we uncovered and fixed.

There are fixes in the latest version we need for ARMClang.

Pull request has been modified.

0xc0170 · 2020-09-04T07:37:38Z

💯 for 3.18.2 results - this is ready to get in ! I updated the requirements.

Note: we use 3.18.2 from now on (the reason is above or in the commit msg).

The only limitation we know at the moment: Ninja with ARMClang on windows not working: https://gitlab.kitware.com/cmake/cmake/-/issues/21093

hugueskamba · 2020-09-04T10:01:47Z

💯 for 3.18.2 results - this is ready to get in ! I updated the requirements.

Note: we use 3.18.2 from now on (the reason is above or in the commit msg).

The only limitation we know at the moment: Ninja with ARMClang on windows not working: https://gitlab.kitware.com/cmake/cmake/-/issues/21093

The bug you mentioned was reported with CMake version 3.18.1, do you experience the same failure with version 3.18.2?

0xc0170 · 2020-09-04T11:00:15Z

yes, 3.18.2 still contains the bug

mergify bot added release-version: feature branch do not merge labels Aug 13, 2020

ciarmcom requested a review from a team August 13, 2020 17:00

ciarmcom added the needs: review label Aug 13, 2020

hugueskamba force-pushed the feature-cmake branch from 25b0725 to 849b3f0 Compare August 15, 2020 16:57

mergify bot added needs: work and removed needs: review labels Aug 15, 2020

0xc0170 force-pushed the fix_cmake_response_files branch from ed355b0 to e20513e Compare August 17, 2020 07:51

0xc0170 force-pushed the fix_cmake_response_files branch from e20513e to 629d981 Compare August 17, 2020 11:22

0xc0170 added do not merge and removed do not merge labels Aug 17, 2020

hugueskamba force-pushed the feature-cmake branch from a57d101 to 129e697 Compare August 18, 2020 15:32

0xc0170 force-pushed the fix_cmake_response_files branch from 629d981 to a431cd9 Compare August 21, 2020 14:16

0xc0170 requested a review from hugueskamba August 21, 2020 14:17

0xc0170 removed the do not merge label Aug 21, 2020

0xc0170 added needs: review and removed needs: work labels Aug 21, 2020

0xc0170 changed the title ~~WIP: Fix cmake response files~~ CMake: Fix cmake response files Aug 21, 2020

hugueskamba previously requested changes Aug 21, 2020

View reviewed changes

mergify bot added the needs: work label Aug 21, 2020

hugueskamba force-pushed the feature-cmake branch from 1272a43 to 7f85097 Compare September 3, 2020 12:47

0xc0170 force-pushed the fix_cmake_response_files branch from 0aa9846 to fff832e Compare September 3, 2020 13:38

CMake: use response files

b16a2c2

As Mbed OS is built as whole, we have long paths again. This is known issue with windows. To fix the paths, we need to use response files. Ninja is special, needs to be forced to use long paths.

0xc0170 force-pushed the fix_cmake_response_files branch from fff832e to b16a2c2 Compare September 3, 2020 15:45

0xc0170 added needs: review and removed do not merge needs: work labels Sep 3, 2020

hugueskamba previously requested changes Sep 3, 2020

View reviewed changes

mergify bot added needs: work and removed needs: review labels Sep 3, 2020

CMake: 3.18.2 minimum version set

6ae34a6

There are fixes in the latest version we need for ARMClang.

0xc0170 added needs: review and removed needs: work labels Sep 4, 2020

hugueskamba approved these changes Sep 4, 2020

View reviewed changes

hugueskamba added ready for merge and removed needs: review labels Sep 4, 2020

mergify bot added the needs: CI label Sep 4, 2020

hugueskamba removed the needs: CI label Sep 4, 2020

hugueskamba merged commit d0280ec into ARMmbed:feature-cmake Sep 4, 2020

mergify bot removed the ready for merge label Sep 4, 2020

0xc0170 deleted the fix_cmake_response_files branch September 4, 2020 11:00

CMake: Fix cmake response files #13427

CMake: Fix cmake response files #13427

Uh oh!

Conversation

0xc0170 commented Aug 13, 2020 • edited by hugueskamba Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary of changes

Impact of changes

Migration actions required

Documentation

Pull request type

Test results

Reviewers

Uh oh!

ciarmcom commented Aug 13, 2020

Uh oh!

0xc0170 commented Aug 14, 2020

Uh oh!

0xc0170 commented Aug 14, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

0xc0170 commented Aug 14, 2020

Uh oh!

mergify bot commented Aug 15, 2020

Uh oh!

0xc0170 commented Aug 17, 2020

Uh oh!

0xc0170 commented Aug 17, 2020

Uh oh!

0xc0170 commented Aug 17, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

mergify bot commented Aug 18, 2020

Uh oh!

0xc0170 commented Aug 21, 2020

Uh oh!

hugueskamba left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

0xc0170 commented Sep 3, 2020

Uh oh!

mergify bot commented Sep 3, 2020

Uh oh!

0xc0170 commented Sep 3, 2020

Uh oh!

hugueskamba commented Sep 3, 2020

Uh oh!

hugueskamba commented Sep 3, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

hugueskamba left a comment

Choose a reason for hiding this comment

Uh oh!

0xc0170 commented Sep 4, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

hugueskamba commented Sep 4, 2020

Uh oh!

0xc0170 commented Sep 4, 2020

Uh oh!

Uh oh!

0xc0170 commented Aug 13, 2020 •

edited by hugueskamba

Loading

0xc0170 commented Aug 14, 2020 •

edited

Loading

0xc0170 commented Aug 17, 2020 •

edited

Loading

hugueskamba left a comment •

edited

Loading

hugueskamba commented Sep 3, 2020 •

edited

Loading

0xc0170 commented Sep 4, 2020 •

edited

Loading