Skip to content

SFDP: consolidation of SFDP parsing [2/5] #12426

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 9 commits into from
Feb 25, 2020
Merged

SFDP: consolidation of SFDP parsing [2/5] #12426

merged 9 commits into from
Feb 25, 2020

Conversation

VeijoPesonen
Copy link
Contributor

@VeijoPesonen VeijoPesonen commented Feb 13, 2020

Summary of changes

Depends on PR #12318(already merged) . That one should go in before you start reviewing this one. Purpose of this PR is to consolidate SFDP header and table parsing and make the QSPIF- and SPIFBlockDevices to use the same shared implementation. Before this PR almost the same code was found from both of the components.

Impact of changes

Migration actions required

Documentation


Pull request type

[X] Patch update (Bug fix / Target update / Docs update / Test update / Refactor)
[] Feature update (New feature / Functionality change / New API)
[] Major update (Breaking change E.g. Return code change / API behaviour change)

Test results

[] No Tests required for this change (E.g docs only update)
[] Covered by existing mbed-os tests (Greentea or Unittest)
[X] Tests / results supplied as part of this PR

SPIF compilation

mbed test --compile -m NRF52840_DK -n features-storage-* --app-config tools/test_configs/SPIFBlockDeviceAndHeapBlockDevice.json

QSPIF compilation

mbed test --compile -m NRF52840_DK -n features-storage-* --app-config tools/test_configs/QSPIFBlockDeviceAndHeapBlockDevice.json
mbedgt: test suite report:
| target                      | platform_name       | test suite                                                                           | result | elapsed_time (sec) | copy_method |
|-----------------------------|---------------------|--------------------------------------------------------------------------------------|--------|--------------------|-------------|
| CY8CPROTO_062_4343W-GCC_ARM | CY8CPROTO_062_4343W | features-storage-filesystem-littlefs-tests-filesystem-dirs                           | FAIL   | 130.41             | default     |
| CY8CPROTO_062_4343W-GCC_ARM | CY8CPROTO_062_4343W | features-storage-filesystem-littlefs-tests-filesystem-files                          | OK     | 30.77              | default     |
| CY8CPROTO_062_4343W-GCC_ARM | CY8CPROTO_062_4343W | features-storage-filesystem-littlefs-tests-filesystem-interspersed                   | OK     | 36.18              | default     |
| CY8CPROTO_062_4343W-GCC_ARM | CY8CPROTO_062_4343W | features-storage-filesystem-littlefs-tests-filesystem-seek                           | OK     | 227.55             | default     |
| CY8CPROTO_062_4343W-GCC_ARM | CY8CPROTO_062_4343W | features-storage-filesystem-littlefs-tests-filesystem_integration-format             | OK     | 24.04              | default     |
| CY8CPROTO_062_4343W-GCC_ARM | CY8CPROTO_062_4343W | features-storage-filesystem-littlefs-tests-filesystem_recovery-resilience            | OK     | 29.79              | default     |
| CY8CPROTO_062_4343W-GCC_ARM | CY8CPROTO_062_4343W | features-storage-filesystem-littlefs-tests-filesystem_recovery-resilience_functional | OK     | 77.05              | default     |
| CY8CPROTO_062_4343W-GCC_ARM | CY8CPROTO_062_4343W | features-storage-filesystem-littlefs-tests-filesystem_recovery-wear_leveling         | OK     | 225.19             | default     |
| CY8CPROTO_062_4343W-GCC_ARM | CY8CPROTO_062_4343W | features-storage-filesystem-littlefs-tests-filesystem_retarget-dirs                  | FAIL   | 131.65             | default     |
| CY8CPROTO_062_4343W-GCC_ARM | CY8CPROTO_062_4343W | features-storage-filesystem-littlefs-tests-filesystem_retarget-files                 | OK     | 26.51              | default     |
| CY8CPROTO_062_4343W-GCC_ARM | CY8CPROTO_062_4343W | features-storage-filesystem-littlefs-tests-filesystem_retarget-interspersed          | OK     | 30.27              | default     |
| CY8CPROTO_062_4343W-GCC_ARM | CY8CPROTO_062_4343W | features-storage-filesystem-littlefs-tests-filesystem_retarget-seek                  | OK     | 285.94             | default     |
| CY8CPROTO_062_4343W-GCC_ARM | CY8CPROTO_062_4343W | features-storage-tests-blockdevice-buffered_block_device                             | OK     | 16.65              | default     |
| CY8CPROTO_062_4343W-GCC_ARM | CY8CPROTO_062_4343W | features-storage-tests-blockdevice-flashsim_block_device                             | OK     | 14.15              | default     |
| CY8CPROTO_062_4343W-GCC_ARM | CY8CPROTO_062_4343W | features-storage-tests-blockdevice-general_block_device                              | OK     | 75.47              | default     |
| CY8CPROTO_062_4343W-GCC_ARM | CY8CPROTO_062_4343W | features-storage-tests-blockdevice-heap_block_device                                 | OK     | 20.85              | default     |
| CY8CPROTO_062_4343W-GCC_ARM | CY8CPROTO_062_4343W | features-storage-tests-blockdevice-mbr_block_device                                  | OK     | 17.49              | default     |
| CY8CPROTO_062_4343W-GCC_ARM | CY8CPROTO_062_4343W | features-storage-tests-blockdevice-util_block_device                                 | OK     | 16.05              | default     |
| CY8CPROTO_062_4343W-GCC_ARM | CY8CPROTO_062_4343W | features-storage-tests-kvstore-direct_access_devicekey_test                          | OK     | 18.38              | default     |
| CY8CPROTO_062_4343W-GCC_ARM | CY8CPROTO_062_4343W | features-storage-tests-kvstore-filesystemstore_tests                                 | OK     | 49.29              | default     |
| CY8CPROTO_062_4343W-GCC_ARM | CY8CPROTO_062_4343W | features-storage-tests-kvstore-general_tests_phase_1                                 | OK     | 113.07             | default     |
| CY8CPROTO_062_4343W-GCC_ARM | CY8CPROTO_062_4343W | features-storage-tests-kvstore-general_tests_phase_2                                 | OK     | 111.83             | default     |
| CY8CPROTO_062_4343W-GCC_ARM | CY8CPROTO_062_4343W | features-storage-tests-kvstore-securestore_whitebox                                  | OK     | 31.62              | default     |
| CY8CPROTO_062_4343W-GCC_ARM | CY8CPROTO_062_4343W | features-storage-tests-kvstore-static_tests                                          | OK     | 42.9               | default     |
| CY8CPROTO_062_4343W-GCC_ARM | CY8CPROTO_062_4343W | features-storage-tests-kvstore-tdbstore_whitebox                                     | OK     | 19.55              | default     |
| DISCO_L475VG_IOT01A-GCC_ARM | DISCO_L475VG_IOT01A | features-storage-filesystem-littlefs-tests-filesystem-dirs                           | OK     | 49.43              | default     |
| DISCO_L475VG_IOT01A-GCC_ARM | DISCO_L475VG_IOT01A | features-storage-filesystem-littlefs-tests-filesystem-files                          | OK     | 28.78              | default     |
| DISCO_L475VG_IOT01A-GCC_ARM | DISCO_L475VG_IOT01A | features-storage-filesystem-littlefs-tests-filesystem-interspersed                   | OK     | 15.01              | default     |
| DISCO_L475VG_IOT01A-GCC_ARM | DISCO_L475VG_IOT01A | features-storage-filesystem-littlefs-tests-filesystem-seek                           | OK     | 57.3               | default     |
| DISCO_L475VG_IOT01A-GCC_ARM | DISCO_L475VG_IOT01A | features-storage-filesystem-littlefs-tests-filesystem_integration-format             | OK     | 31.21              | default     |
| DISCO_L475VG_IOT01A-GCC_ARM | DISCO_L475VG_IOT01A | features-storage-filesystem-littlefs-tests-filesystem_recovery-resilience            | OK     | 29.88              | default     |
| DISCO_L475VG_IOT01A-GCC_ARM | DISCO_L475VG_IOT01A | features-storage-filesystem-littlefs-tests-filesystem_recovery-resilience_functional | OK     | 65.4               | default     |
| DISCO_L475VG_IOT01A-GCC_ARM | DISCO_L475VG_IOT01A | features-storage-filesystem-littlefs-tests-filesystem_recovery-wear_leveling         | OK     | 261.37             | default     |
| DISCO_L475VG_IOT01A-GCC_ARM | DISCO_L475VG_IOT01A | features-storage-filesystem-littlefs-tests-filesystem_retarget-dirs                  | OK     | 48.37              | default     |
| DISCO_L475VG_IOT01A-GCC_ARM | DISCO_L475VG_IOT01A | features-storage-filesystem-littlefs-tests-filesystem_retarget-files                 | OK     | 24.43              | default     |
| DISCO_L475VG_IOT01A-GCC_ARM | DISCO_L475VG_IOT01A | features-storage-filesystem-littlefs-tests-filesystem_retarget-interspersed          | OK     | 15.22              | default     |
| DISCO_L475VG_IOT01A-GCC_ARM | DISCO_L475VG_IOT01A | features-storage-filesystem-littlefs-tests-filesystem_retarget-seek                  | OK     | 61.91              | default     |
| DISCO_L475VG_IOT01A-GCC_ARM | DISCO_L475VG_IOT01A | features-storage-tests-blockdevice-buffered_block_device                             | OK     | 12.27              | default     |
| DISCO_L475VG_IOT01A-GCC_ARM | DISCO_L475VG_IOT01A | features-storage-tests-blockdevice-flashsim_block_device                             | OK     | 12.6               | default     |
| DISCO_L475VG_IOT01A-GCC_ARM | DISCO_L475VG_IOT01A | features-storage-tests-blockdevice-general_block_device                              | OK     | 38.41              | default     |
| DISCO_L475VG_IOT01A-GCC_ARM | DISCO_L475VG_IOT01A | features-storage-tests-blockdevice-heap_block_device                                 | OK     | 17.34              | default     |
| DISCO_L475VG_IOT01A-GCC_ARM | DISCO_L475VG_IOT01A | features-storage-tests-blockdevice-mbr_block_device                                  | OK     | 13.15              | default     |
| DISCO_L475VG_IOT01A-GCC_ARM | DISCO_L475VG_IOT01A | features-storage-tests-blockdevice-util_block_device                                 | OK     | 13.06              | default     |
| DISCO_L475VG_IOT01A-GCC_ARM | DISCO_L475VG_IOT01A | features-storage-tests-kvstore-direct_access_devicekey_test                          | OK     | 15.54              | default     |
| DISCO_L475VG_IOT01A-GCC_ARM | DISCO_L475VG_IOT01A | features-storage-tests-kvstore-static_tests                                          | OK     | 29.67              | default     |
| DISCO_L475VG_IOT01A-GCC_ARM | DISCO_L475VG_IOT01A | features-storage-tests-kvstore-tdbstore_whitebox                                     | OK     | 13.8               | default     |
| K82F-GCC_ARM                | K82F                | features-storage-filesystem-littlefs-tests-filesystem-dirs                           | OK     | 83.93              | default     |
| K82F-GCC_ARM                | K82F                | features-storage-filesystem-littlefs-tests-filesystem-files                          | OK     | 38.89              | default     |
| K82F-GCC_ARM                | K82F                | features-storage-filesystem-littlefs-tests-filesystem-interspersed                   | OK     | 19.03              | default     |
| K82F-GCC_ARM                | K82F                | features-storage-filesystem-littlefs-tests-filesystem-seek                           | OK     | 69.75              | default     |
| K82F-GCC_ARM                | K82F                | features-storage-filesystem-littlefs-tests-filesystem_integration-format             | OK     | 34.41              | default     |
| K82F-GCC_ARM                | K82F                | features-storage-filesystem-littlefs-tests-filesystem_recovery-resilience            | OK     | 33.21              | default     |
| K82F-GCC_ARM                | K82F                | features-storage-filesystem-littlefs-tests-filesystem_recovery-resilience_functional | OK     | 68.63              | default     |
| K82F-GCC_ARM                | K82F                | features-storage-filesystem-littlefs-tests-filesystem_recovery-wear_leveling         | OK     | 271.87             | default     |
| K82F-GCC_ARM                | K82F                | features-storage-filesystem-littlefs-tests-filesystem_retarget-dirs                  | OK     | 82.51              | default     |
| K82F-GCC_ARM                | K82F                | features-storage-filesystem-littlefs-tests-filesystem_retarget-files                 | OK     | 37.56              | default     |
| K82F-GCC_ARM                | K82F                | features-storage-filesystem-littlefs-tests-filesystem_retarget-interspersed          | OK     | 19.58              | default     |
| K82F-GCC_ARM                | K82F                | features-storage-filesystem-littlefs-tests-filesystem_retarget-seek                  | OK     | 80.7               | default     |
| K82F-GCC_ARM                | K82F                | features-storage-tests-blockdevice-buffered_block_device                             | OK     | 14.66              | default     |
| K82F-GCC_ARM                | K82F                | features-storage-tests-blockdevice-flashsim_block_device                             | OK     | 13.33              | default     |
| K82F-GCC_ARM                | K82F                | features-storage-tests-blockdevice-general_block_device                              | OK     | 48.28              | default     |
| K82F-GCC_ARM                | K82F                | features-storage-tests-blockdevice-heap_block_device                                 | OK     | 16.73              | default     |
| K82F-GCC_ARM                | K82F                | features-storage-tests-blockdevice-mbr_block_device                                  | OK     | 19.1               | default     |
| K82F-GCC_ARM                | K82F                | features-storage-tests-blockdevice-util_block_device                                 | OK     | 14.94              | default     |
| K82F-GCC_ARM                | K82F                | features-storage-tests-filesystem-general_filesystem                                 | OK     | 48.77              | default     |
| K82F-GCC_ARM                | K82F                | features-storage-tests-kvstore-direct_access_devicekey_test                          | OK     | 18.94              | default     |
| K82F-GCC_ARM                | K82F                | features-storage-tests-kvstore-static_tests                                          | OK     | 34.2               | default     |
| K82F-GCC_ARM                | K82F                | features-storage-tests-kvstore-tdbstore_whitebox                                     | OK     | 14.59              | default     |
| NRF52840_DK-GCC_ARM         | NRF52840_DK         | features-storage-filesystem-littlefs-tests-filesystem-dirs                           | OK     | 158.48             | default     |
| NRF52840_DK-GCC_ARM         | NRF52840_DK         | features-storage-filesystem-littlefs-tests-filesystem-files                          | OK     | 64.34              | default     |
| NRF52840_DK-GCC_ARM         | NRF52840_DK         | features-storage-filesystem-littlefs-tests-filesystem-interspersed                   | OK     | 25.26              | default     |
| NRF52840_DK-GCC_ARM         | NRF52840_DK         | features-storage-filesystem-littlefs-tests-filesystem-seek                           | OK     | 132.24             | default     |
| NRF52840_DK-GCC_ARM         | NRF52840_DK         | features-storage-filesystem-littlefs-tests-filesystem_integration-format             | OK     | 37.42              | default     |
| NRF52840_DK-GCC_ARM         | NRF52840_DK         | features-storage-filesystem-littlefs-tests-filesystem_recovery-resilience            | OK     | 55.69              | default     |
| NRF52840_DK-GCC_ARM         | NRF52840_DK         | features-storage-filesystem-littlefs-tests-filesystem_recovery-resilience_functional | OK     | 73.79              | default     |
| NRF52840_DK-GCC_ARM         | NRF52840_DK         | features-storage-filesystem-littlefs-tests-filesystem_recovery-wear_leveling         | OK     | 93.85              | default     |
| NRF52840_DK-GCC_ARM         | NRF52840_DK         | features-storage-filesystem-littlefs-tests-filesystem_retarget-dirs                  | OK     | 167.17             | default     |
| NRF52840_DK-GCC_ARM         | NRF52840_DK         | features-storage-filesystem-littlefs-tests-filesystem_retarget-files                 | OK     | 70.16              | default     |
| NRF52840_DK-GCC_ARM         | NRF52840_DK         | features-storage-filesystem-littlefs-tests-filesystem_retarget-interspersed          | OK     | 24.49              | default     |
| NRF52840_DK-GCC_ARM         | NRF52840_DK         | features-storage-filesystem-littlefs-tests-filesystem_retarget-seek                  | OK     | 163.8              | default     |
| NRF52840_DK-GCC_ARM         | NRF52840_DK         | features-storage-tests-blockdevice-buffered_block_device                             | OK     | 21.22              | default     |
| NRF52840_DK-GCC_ARM         | NRF52840_DK         | features-storage-tests-blockdevice-flashsim_block_device                             | OK     | 22.31              | default     |
| NRF52840_DK-GCC_ARM         | NRF52840_DK         | features-storage-tests-blockdevice-general_block_device                              | OK     | 64.3               | default     |
| NRF52840_DK-GCC_ARM         | NRF52840_DK         | features-storage-tests-blockdevice-heap_block_device                                 | OK     | 23.43              | default     |
| NRF52840_DK-GCC_ARM         | NRF52840_DK         | features-storage-tests-blockdevice-mbr_block_device                                  | OK     | 21.65              | default     |
| NRF52840_DK-GCC_ARM         | NRF52840_DK         | features-storage-tests-blockdevice-util_block_device                                 | OK     | 20.84              | default     |
| NRF52840_DK-GCC_ARM         | NRF52840_DK         | features-storage-tests-filesystem-general_filesystem                                 | OK     | 69.11              | default     |
| NRF52840_DK-GCC_ARM         | NRF52840_DK         | features-storage-tests-kvstore-direct_access_devicekey_test                          | OK     | 29.15              | default     |
| NRF52840_DK-GCC_ARM         | NRF52840_DK         | features-storage-tests-kvstore-static_tests                                          | OK     | 51.7               | default     |
| NRF52840_DK-GCC_ARM         | NRF52840_DK         | features-storage-tests-kvstore-tdbstore_whitebox                                     | OK     | 26.16              | default     |
mbedgt: test suite results: 2 FAIL / 88 OK

Reviewers

@SeppoTakalo
@michalpasztamobica


@VeijoPesonen
Copy link
Contributor Author

@ARMmbed/team-nxp for your information.

@ciarmcom ciarmcom requested review from a team February 13, 2020 14:00
@ciarmcom
Copy link
Member

@VeijoPesonen, thank you for your changes.
@ARMmbed/mbed-os-tools @ARMmbed/mbed-os-core @ARMmbed/mbed-os-storage @ARMmbed/mbed-os-maintainers please review.

Copy link
Contributor

@michalpasztamobica michalpasztamobica left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This all looks good and is a valuable code clean-up. The remarks I put in are minor and can also be included in the next stage of work.
I wonder about two things:

  1. Do the available greentea tests provide enough coverage to give us confidence that when they pass we haven't broken the feature? :) If not - would it be the right moment to add some unittests before the refactoring, to ensure the functionality stays the same (for error handling and corner cases)? Maybe they would also make the debugging easier in case some bug hides somewhere, as was the case in the previous PR. They are also more glitch-proof than tests running on real HW ;-)
  2. How much is the n from the PR's title? I appreciate splitting the work into smaller chunks as that also makes the review easier, I am just wondering what does the split look like.

} else {
tr_debug("SFDP erase types are not available - falling back to legacy 4k erase instruction");

// 0xFF indicates that the legacy 4k erase instruction is not supported
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[minor] Perhaps we could add an SFDP_ERASE_BITMASK_TYPE_4K_ERASE_UNSUPPORTED to replace the 0xff below and get rid of this comment?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed.

@VeijoPesonen
Copy link
Contributor Author

1. Do the available greentea tests provide enough coverage to give us confidence that when they pass we haven't broken the feature? :) If not - would it be the right moment to add some unittests before the refactoring, to ensure the functionality stays the same (for error handling and corner cases)? Maybe they would also make the debugging easier in case some bug hides somewhere, as was the case in the previous PR. They are also more glitch-proof than tests running on real HW ;-)

I had a discussion about this some time ago with @SeppoTakalo and the end result was that HW tests are good enough. I trust that the HW vendors get the SFDP standard more correct than I would by mimicking the structures in unit tests.

2. How much is the `n` from the PR's title? I appreciate splitting the work into smaller chunks as that also makes the review easier, I am just wondering what does the split look like.

I really don't know yet. Now I just try to rip out all the common parts from SPIF- and QSPIF-modules and try to fit those somehow under SFDP module. I'm more eager to do refactoring the code under SFDP module once I have all the parts there.

Copy link

@kyle-cypress kyle-cypress left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me in general. Just a couple of minor style and documentation comments.

mark-edgeworth
mark-edgeworth previously approved these changes Feb 17, 2020
@mergify mergify bot added needs: CI and removed needs: review labels Feb 17, 2020
SeppoTakalo
SeppoTakalo previously approved these changes Feb 17, 2020
@0xc0170
Copy link
Contributor

0xc0170 commented Feb 17, 2020

Ci started

@mergify mergify bot added needs: work and removed needs: CI labels Feb 17, 2020
@mbed-ci
Copy link

mbed-ci commented Feb 17, 2020

Test run: FAILED

Summary: 1 of 4 test jobs failed
Build number : 1
Build artifacts

Failed test jobs:

  • jenkins-ci/mbed-os-ci_build-IAR

@VeijoPesonen
Copy link
Contributor Author

From the logs

[ERROR] Internal error: [Front end]: assertion failed at: "/home/jenkins_ci01/slave_top/workspace/ArmReleaseBuild/core/translator/compiler_core/src/parser/edg/il.c", line 23626


Fatal error detected, aborting.

[mbed] Working path "/builds/ws/mbed-os-ci_build-IAR@3/examples/mbed-os-example-socket-stats" (program)

@0xc0170
Copy link
Contributor

0xc0170 commented Feb 18, 2020

Just noticed this in another PR, creating a new blocker as its in nightly and PRs as well

cc @ARMmbed/mbed-os-test

@0xc0170
Copy link
Contributor

0xc0170 commented Feb 18, 2020

I restarted the job

@mbed-ci
Copy link

mbed-ci commented Feb 18, 2020

Test run: FAILED

Summary: 1 of 4 test jobs failed
Build number : 2
Build artifacts

Failed test jobs:

  • jenkins-ci/mbed-os-ci_build-IAR

@0xc0170
Copy link
Contributor

0xc0170 commented Feb 18, 2020

CI restarted (the error was fixed)

@mbed-ci
Copy link

mbed-ci commented Feb 18, 2020

Test run: SUCCESS

Summary: 11 of 11 test jobs passed
Build number : 3
Build artifacts

@mergify mergify bot dismissed stale reviews from mark-edgeworth and SeppoTakalo February 20, 2020 07:47

Pull request has been modified.

@VeijoPesonen
Copy link
Contributor Author

Rebased, @0xc0170 would you please restart the tests if necessary.

@0xc0170
Copy link
Contributor

0xc0170 commented Feb 24, 2020

CI restarted

@mbed-ci
Copy link

mbed-ci commented Feb 24, 2020

Test run: SUCCESS

Summary: 11 of 11 test jobs passed
Build number : 4
Build artifacts

@0xc0170 0xc0170 merged commit 1c12083 into ARMmbed:master Feb 25, 2020
@VeijoPesonen VeijoPesonen deleted the sfdp_split_bptbl branch February 25, 2020 14:09
@VeijoPesonen VeijoPesonen changed the title SFDP: consolidation of SFDP parsing [2/n] SFDP: consolidation of SFDP parsing [2/5] Feb 28, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

8 participants