Feature/separate header from app (ARMmbed#98)

LiyouZhou · web-flow · commit 0d2873ab99ef · 2018-04-03T18:09:39.000+01:00
FEATURE
* Do not assume active header contiguous with app

Modify the README
* Info re hdr loc &amp; storage choices
* Add info re RoT
* and link to docs re firmware stoarge
API porting
* restore build badge and clean up

IOTUC-70
diff --git a/README.md b/README.md
@@ -1,4 +1,6 @@
 # mbed-bootloader
+ 
+[![Build Status](https://jenkins-internal.mbed.com/buildStatus/icon?job=ARMmbed/mbed-bootloader-internal/master)](https://jenkins-internal.mbed.com/job/ARMmbed/job/mbed-bootloader-internal/job/master/)
 
 Generic bootloader to be used in conjunction with [mbed-cloud-client](https://github.com/ARMmbed/mbed-cloud-client).
 
@@ -7,7 +9,7 @@ Generic bootloader to be used in conjunction with [mbed-cloud-client](https://gi
 1. Install `mbed-cli` https://github.com/ARMmbed/mbed-cli
 1. Run `mbed deploy` to pull in dependencies
 1. Compile by running `mbed compile -t GCC_ARM -m (K64F|NUCLEO_F429ZI|UBLOX_EVK_ODIN_W2) --profile=tiny.json`
-1. Use this [script](https://github.com/ARMmbed/mbed-cloud-client-example/blob/master/tools/combine_bootloader_with_app.py) to combine the bootloader with application `python tools/combine_bootloader_with_app.py -a {application.bin} -b {bootloader.bin} --app-offset {firmware_metadata_header_address+firmware_metadata_header_size} --header-offset {firmware_metadata_header_address} -o {combined.bin}`.
+1. Use this [script](https://github.com/ARMmbed/mbed-cloud-client-example/blob/master/tools/combine_bootloader_with_app.py) to combine the bootloader with application `python tools/combine_bootloader_with_app.py -a {application.bin} -b {bootloader.bin} --app-offset {application-start-address} --header-offset {firmware_metadata_header_address} -o {combined.bin}`.
 1. Flash `{combined.bin}` to device by drag and drop.
 
 ## Metadata Header
@@ -18,19 +20,37 @@ The firmware metadata header structure can be found [here](https://github.com/AR
 
 ## Configurations
 
-User **must** set in `mbed_app.json`:
+NOTE: All these configurations must be set the same in the mbed cloud client when compiling the corresponding application for successful update operation.
+
+### Active Application and Header
+
 1. `update-client.application-details`, Address at which the metadata header of the active firmware is written. **Must align to flash erase boundary**
 1. `application-start-address`, Address at which The application starts **Must align to vector table size boundary and flash write page boundary**. It is assumed the region between `update-client.application-details` and `application-start-address` contains only the header. MUST be the same as "target.mbed_app_start" in the application.
+
+If the `application-start-address` is set less than one erase sector after the `update-client.application-details`, the two regions will be erased together. Otherwise the two regions will be erased separately in which case `application-start-address` must also align to **flash erase boundary**.
+
+### Firmware Candidate Storage
+
+1. `MBED_CLOUD_CLIENT_UPDATE_STORAGE`, This need to be set in the "macros" section of `mbed_app.json`. Choices are ARM_UCP_FLASHIAP_BLOCKDEVICE and ARM_UCP_FLASHIAP. This determines whether the firmware is stored on a blockdevice or internal flash. If blockdevice is used `ARM_UC_USE_PAL_BLOCKDEVICE=1` must also be set. 
 1. `update-client.storage-address`, The address in sd block device or internal flash where the firmware candidates are stored. **Must align to flash erase boundary**
 1. `update-client.storage-size`, total size on the block device or internal flash reserved for firmware storage. It will be rounded up to align with flash erase sector size automatically.
 1. `update-client.storage-locations`, The number of slots in the firmware storage.
 1. `update-client.storage-page`, The write page size of the underlying storage.
 
-If you are using SOTP to provide the RoT, you must set the following:
-- "sotp-section-1-address", "sotp-section-1-size", "sotp-section-2-address", "sotp-section-2-size"
-The addresses **Must align to flash erase boundary**. The sizes must be full sector sized and at least 1k large.
+NOTE: See the [mbed cloud client documentation](https://cloud.mbed.com/docs/current/porting/update-k64f-port.html) for more information about storage options avaiable and porting to new platforms.
+
+### Device Secret Key
+
+The bootloader uses device secret key to authenticate anything that is stored on external storage. The update client must be able to obtain the same key as the bootlaoder. The key is derived from a device root of trust using the algorithm [here](https://github.com/ARMmbed/mbed-cloud-client/blob/master/update-client-hub/modules/common/source/arm_uc_crypto.c#L401).
+
+You may choose to use SOTP to generate and store device RoT. During first boot the mbed cloud client will generate a random number from an available entropy source and storge it in SOTP on internal flash. On subsequent boots, the RoT will be read from SOTP. To enable SOTP RoT, you must set the following:
+1. Macro `PAL_USE_INTERNAL_FLASH=1` and `PAL_INT_FLASH_NUM_SECTIONS=2` to indicate that 2 sectors are used for SOTP.
+1. Macro `ARM_UC_USE_SOTP=1` to tell bootloader to retrive RoT from SOTP.
+1. "sotp-section-1-address", "sotp-section-1-size", "sotp-section-2-address", "sotp-section-2-size". The addresses **Must align to flash erase boundary**. The sizes must be full sector sized and at least 1k large.
+
+Alternatively you can choose to use a custom device specific RoT by implementing the function `mbed_cloud_client_get_rot_128bit`. An example can be found [here](https://github.com/ARMmbed/mbed-bootloader-internal/blob/master/source/example_insecure_rot.c#L40).
 
-All these configurations must be set the same in the mbed cloud client when compiling the corresponding application for successful update operation.
+### MISC
 
 User **may** set in `mbed_app.json`:
 1. `MAX_COPY_RETRIES`, The number of retries after a failed copy attempt.
diff --git a/source/active_application.cpp b/source/active_application.cpp
@@ -180,58 +180,107 @@ int checkActiveApplication(arm_uc_firmware_details_t* details)
     return result;
 }
 
+int eraseSectorBySector(uint32_t addr, uint32_t size)
+{
+    tr_debug("Erasing from 0x%08" PRIX32 " to 0x%08" PRIX32,
+             (uint32_t) addr, (uint32_t) addr + size);
+
+    int result = -1;
+    uint32_t erase_address = addr;
+
+    /* Erase flash to make place for new application. Erasing sector by sector as some
+       platforms have varible sector sizes and mbed-os cannot deal with erasing multiple
+       sectors successfully in that case. https://github.com/ARMmbed/mbed-os/issues/6077 */
+    while (erase_address < (addr + size))
+    {
+        uint32_t sector_size = flash.get_sector_size(erase_address);
+        result = flash.erase(erase_address,
+                             sector_size);
+        if (result != 0)
+        {
+            tr_debug("Erasing from 0x%08" PRIX32 " to 0x%08" PRIX32 " failed with retval %i",
+                     erase_address, erase_address + sector_size, result);
+            break;
+        }
+        else
+        {
+            erase_address += sector_size;
+        }
+    }
+
+    return result;
+}
+
+uint32_t getSectorAlignedSize(uint32_t addr, uint32_t size)
+{
+    tr_debug("getSectorAlignedSize at 0x%08" PRIX32 " of size 0x%08" PRIX32,
+             (uint32_t) addr, (uint32_t) size);
+
+    /* Find the exact end sector boundary. Some platforms have different sector
+       sizes from sector to sector. Hence we count the sizes 1 sector at a time here */
+    uint32_t erase_address = addr;
+    while (erase_address < (addr + size))
+    {
+        erase_address += flash.get_sector_size(erase_address);
+    }
+
+    return erase_address - addr;
+}
+
 /**
  * Wipe the ACTIVE firmware region in the flash
  */
 bool eraseActiveFirmware(uint32_t firmwareSize)
 {
     tr_debug("eraseActiveFirmware");
 
-    /* Find the exact end sector boundary. Some platforms have different sector
-       sizes from sector to sector. Hence we count the sizes 1 sector at a time here */
-    uint32_t erase_address = FIRMWARE_METADATA_HEADER_ADDRESS;
-    uint32_t size_needed = FIRMWARE_METADATA_HEADER_SIZE + firmwareSize;
-    while (erase_address < (FIRMWARE_METADATA_HEADER_ADDRESS + size_needed))
+    uint32_t fw_metadata_hdr_size = getSectorAlignedSize(FIRMWARE_METADATA_HEADER_ADDRESS,
+                                                         ARM_UC_INTERNAL_HEADER_SIZE_V2);
+    uint32_t size_needed = 0;
+    uint32_t erase_start_addr = 0;
+    int result = 0;
+
+    if (((FIRMWARE_METADATA_HEADER_ADDRESS + fw_metadata_hdr_size) < \
+         (MBED_CONF_APP_APPLICATION_START_ADDRESS)) || \
+        (FIRMWARE_METADATA_HEADER_ADDRESS > MBED_CONF_APP_APPLICATION_START_ADDRESS))
     {
-        erase_address += flash.get_sector_size(erase_address);
+        /* header separate from app */
+        tr_debug("Erasing header separately from active application");
+
+        /* erase header section first */
+        result = eraseSectorBySector(FIRMWARE_METADATA_HEADER_ADDRESS, fw_metadata_hdr_size);
+
+        /* setup erase of the application region */
+        size_needed = firmwareSize;
+        erase_start_addr = MBED_CONF_APP_APPLICATION_START_ADDRESS;
+    }
+    else /* header contiguous with app */
+    {
+        /* setup erase of the header + application region */
+        size_needed = fw_metadata_hdr_size + firmwareSize;
+        erase_start_addr = FIRMWARE_METADATA_HEADER_ADDRESS;
     }
 
-    /* check that the erase will not exceed MBED_CONF_APP_MAX_APPLICATION_SIZE */
-    int result = -1;
-    if (erase_address < (MBED_CONF_APP_MAX_APPLICATION_SIZE + \
-                         MBED_CONF_APP_APPLICATION_START_ADDRESS))
+    if (result == 0)
     {
-        tr_debug("Erasing from 0x%08" PRIX32 " to 0x%08" PRIX32,
-                 (uint32_t) FIRMWARE_METADATA_HEADER_ADDRESS,
-                 (uint32_t) erase_address);
-
-        /* Erase flash to make place for new application. Erasing sector by sector as some
-           platforms have varible sector sizes and mbed-os cannot deal with erasing multiple
-           sectors successfully in that case. https://github.com/ARMmbed/mbed-os/issues/6077 */
-        erase_address = FIRMWARE_METADATA_HEADER_ADDRESS;
-        while (erase_address < (FIRMWARE_METADATA_HEADER_ADDRESS + size_needed))
+        uint32_t erase_end_addr = erase_start_addr + \
+                                  getSectorAlignedSize(erase_start_addr,
+                                                       size_needed);
+        uint32_t max_end_addr = MBED_CONF_APP_MAX_APPLICATION_SIZE + \
+                                MBED_CONF_APP_APPLICATION_START_ADDRESS;
+        /* check that the erase will not exceed MBED_CONF_APP_MAX_APPLICATION_SIZE */
+        if (erase_end_addr <= max_end_addr)
         {
-            uint32_t sector_size = flash.get_sector_size(erase_address);
-            result = flash.erase(erase_address,
-                                 sector_size);
-            if (result != 0)
-            {
-                tr_debug("Erasing from 0x%08" PRIX32 " to 0x%08" PRIX32 " failed with retval %i",
-                         erase_address, erase_address + sector_size, result);
-                break;
-            }
-            else
-            {
-                erase_address += sector_size;
-            }
+            result = eraseSectorBySector(erase_start_addr, size_needed);
+        }
+        else
+        {
+            result = -1;
+            tr_error("Firmware size 0x%" PRIX32 " rounded up to the nearest sector boundary 0x%" \
+                     PRIX32 " is larger than the maximum application size 0x%" PRIX32,
+                     firmwareSize, erase_end_addr - MBED_CONF_APP_APPLICATION_START_ADDRESS,
+                     MBED_CONF_APP_MAX_APPLICATION_SIZE);
         }
-    }
-    else
-    {
-        tr_error("Firmware size 0x%" PRIX32 " rounded up to the nearest sector boundary 0x%" \
-                 PRIX32 " is larger than the maximum application size 0x%" PRIX32,
-                 firmwareSize, erase_address - MBED_CONF_APP_APPLICATION_START_ADDRESS,
-                 MBED_CONF_APP_MAX_APPLICATION_SIZE);
     }
 
     return (result == 0);
@@ -249,16 +298,19 @@ bool writeActiveFirmwareHeader(arm_uc_firmware_details_t* details)
         const uint32_t pageSize = flash.get_page_size();
         const uint32_t programSize = (ARM_UC_INTERNAL_HEADER_SIZE_V2 + pageSize - 1)
                                      / pageSize * pageSize;
+        const uint32_t fw_metadata_hdr_size = \
+                            getSectorAlignedSize(FIRMWARE_METADATA_HEADER_ADDRESS,
+                                                 ARM_UC_INTERNAL_HEADER_SIZE_V2);
 
         /* coverity[no_escape] */
         MBED_BOOTLOADER_ASSERT((programSize <= BUFFER_SIZE),
                "Header program size %" PRIu32 " bigger than buffer %d\r\n",
                programSize, BUFFER_SIZE);
 
         /* coverity[no_escape] */
-        MBED_BOOTLOADER_ASSERT((programSize <= FIRMWARE_METADATA_HEADER_SIZE),
+        MBED_BOOTLOADER_ASSERT((programSize <= fw_metadata_hdr_size),
                "Header program size %" PRIu32 " bigger than expected header %d\r\n",
-               programSize, FIRMWARE_METADATA_HEADER_SIZE);
+               programSize, fw_metadata_hdr_size);
 
         /* pad buffer to 0xFF */
         memset(buffer_array, 0xFF, programSize);
diff --git a/source/bootloader_config.h b/source/bootloader_config.h
@@ -40,17 +40,4 @@
        "To use pre configured profiles: mbed compile --app-config configs/<config>.json"
 #endif
 
-/* FIRMWARE_METADATA_HEADER_SIZE */
-#if defined(MBED_CONF_APP_APPLICATION_START_ADDRESS) && \
-    defined(FIRMWARE_METADATA_HEADER_ADDRESS)
-#define FIRMWARE_METADATA_HEADER_SIZE \
-            (MBED_CONF_APP_APPLICATION_START_ADDRESS - \
-             FIRMWARE_METADATA_HEADER_ADDRESS)
-#endif
-
-#if !defined(FIRMWARE_METADATA_HEADER_SIZE)
-#error "configure application_start_address in mbed_app.json\n" \
-       "To use pre configured profiles: mbed compile --app-config configs/<config>.json"
-#endif
-
 #endif // BOOTLOADER_CONFIG_H
