Colorized the short Warnings/Errors generated by the toolchains

[theotherjimmy]

These commits add colorization to all of the short form Warning and Error messages generated by the compiler when the --color switch is provided on the command line. The default colorization will make Warnings Yellow and Errors Red, but can be changed by the user, in mbed_settings.py if they find yellow and red unreadable or ugly.
Affects these top level scripts:

• build.py
• make.py
• test.py

Therefore the mbed subcommands compile, and test are also affected.

[theotherjimmy]

cc @bogdanm @screamerbg @0xc0170 @sg- @rgrover

[screamerbg]

@theotherjimmy Don't like the colorama dependency in utils.py. Why not handle notify function/output and colorize from there?

[theotherjimmy]

I could. I assumed that it could be a more general purpose colorization function and used in more than just the toolchains.

[theotherjimmy]

Don't merge yet. New implementation coming.

[theotherjimmy]

Implementation will be: add option to options.py to add colorization; use that option to replace the notification function within the toolchains through the constructor.

[theotherjimmy]

The implementation is much better now, but still changes something in tools/toolchains/__init__.py, which I don't like.

[bogdanm]

I know that @screamerbg had some issues with colorama at some point. @screamerbg, is that still the case?

[bogdanm]

What did you test this on? Doesn't work in my Linux console, the ANSI escapes don't seem to work:

Building project blinky (K64F, GCC_ARM)
Compile: main.cpp
�[.31m[Error] main.cpp@4: 'ds' does not name a type
�[.0m�[.33m[Warning] main.cpp@17: 'rtos::Thread::Thread(void (*)(const void*), void*, osPriority, uint32_t, unsigned char*)[...]
�[.0m./main.cpp:4:1: error: 'ds' does not name a type

My TERM is set to xterm (and is fully capable of doing color output).

[bogdanm]

...and the same thing happens in Windows (cmd.exe). 👎

[theotherjimmy]

@bogdanm: I tested it Xterm (echo $TERM is xterm), URxvt (echo $TERM is rxvt-unicode), and windows cmd.exe (echo $TERM is $TERM 😄).
before Replace color definitions...:
URxvt gets color, xterm gets no color, cmd.exe gets no color. However, I do not see the escape codes.
After:
URxvt gets color, xterm gets color, cmd.exe still no color. I still don't see escape codes.

[PrzemekWirkus]

@theotherjimmy Would you be so kind and add meaningful description to this PR ? See here

[theotherjimmy]

@PrzemekWirkus Thank you for the friendly reminder. Please find a description of what this PR adds to our tools in the first comment.

[screamerbg]

Thanks, Jimmy.
+1

@bogdanm can you confirm that the latest version works for you?

[bogdanm]

Linux: it works partially. Warnings like the one below are shown in the proper color:

[Warning] RTX_CM_lib.h@681: implicit declaration of function 'atexit' [-Wimplicit-function-declaration]

Errors are more problematic. For the error below:

Compile: main.cpp
[Error] main.cpp@4: 'ds' does not name a type
[Warning] main.cpp@17: 'rtos::Thread::Thread(void (*)(const void*), void*, osPriority, uint32_t, unsigned char*)' is deprecated (declared at ./mbed-os/core/rtos/rtos/Thread.h:138): Thread-spawning constructors hide errors and may lead to complex program state when a thread is declared [-Wdeprecated-declarations]
[Error] main.cpp@20: 'led1' was not declared in this scope
./main.cpp:4:1: error: 'ds' does not name a type
 ds
 ^
./main.cpp: In function 'int main()':
./main.cpp:17:30: warning: 'rtos::Thread::Thread(void (*)(const void*), void*, osPriority, uint32_t, unsigned char*)' is deprecated (declared at ./mbed-os/core/rtos/rtos/Thread.h:138): Thread-spawning constructors hide errors and may lead to complex program state when a thread is declared [-Wdeprecated-declarations]
     Thread thread(led2_thread);
                              ^
./main.cpp:20:9: error: 'led1' was not declared in this scope
         led1 = !led1;
         ^
[ERROR] ./main.cpp:4:1: error: 'ds' does not name a type
 ds
 ^
./main.cpp: In function 'int main()':
./main.cpp:17:30: warning: 'rtos::Thread::Thread(void (*)(const void*), void*, osPriority, uint32_t, unsigned char*)' is deprecated (declared at ./mbed-os/core/rtos/rtos/Thread.h:138): Thread-spawning constructors hide errors and may lead to complex program state when a thread is declared [-Wdeprecated-declarations]
     Thread thread(led2_thread);
                              ^
./main.cpp:20:9: error: 'led1' was not declared in this scope
         led1 = !led1;
         ^

the first 4 lines are properly colorized, the others are shown in the regular color. Also, I have a linker error that's not being colorized at all.

I don't consider any of the above to be a blocker for merging this PR. We can improve it later.
I'm going to test in Windows and come back with the results in a separate comment.

[bogdanm]

I can see the exact same behaviour in Windows, which is a good thing.
Conclusion: not yet complete, but ready to be merged in as far as I'm concerned. 👍

[theotherjimmy]

@bogdanm, Yep, only the short form of the Warnings/Errors are colorized at the moment. It should be possible to build on this PR to make a more complete colorization in the future.