Skip to content

FAT: Add support for block sizes of 512-4096 bytes #3972

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Jun 3, 2017

Conversation

geky
Copy link
Contributor

@geky geky commented Mar 20, 2017

This is necessary for support of block devices with >512 byte blocks, such as most SPI flash parts. This is necessary for FAT support of SPI flash parts.

  • Enabled support of up to 4096 byte blocks
  • Added support for heap-backed buffers using _FS_HEAPBUF
    • Necessary to avoid stack overflows
    • Avoids over-aggresive allocations of _MAX_SS
  • Enabled _FS_TINY to further reduce memory footprint
    • Haven't found a downside for this yet except for possible thread contention

cc @simonqhughes

@geky geky force-pushed the fat-big-blocks branch 3 times, most recently from 971d19a to 15e38ac Compare March 20, 2017 22:04
@0xc0170
Copy link
Contributor

0xc0170 commented Mar 21, 2017

retest uvisor

@sg-
Copy link
Contributor

sg- commented Mar 22, 2017

Enabled _FS_TINY to further reduce memory footprint
Haven't found a downside for this yet except for possible thread contention

This seems pretty serious given its a multi-threaded OS by default, no?

@sg-
Copy link
Contributor

sg- commented Mar 22, 2017

How compatible are these changes if we upgrade to 0.12?

@sg-
Copy link
Contributor

sg- commented Mar 22, 2017

/morph test-nightly

@geky
Copy link
Contributor Author

geky commented Mar 22, 2017

Enabled _FS_TINY to further reduce memory footprint
Haven't found a downside for this yet except for possible thread contention

This seems pretty serious given its a multi-threaded OS by default, no?

There is appropriate locking, so it's still is thread safe.

The trade off is ram (extra 4kbyte per file) vs some speed, but the driver is already sharing a single spi resource that serializes any storage operations.

How compatible are these changes if we upgrade to 0.12?

It doesn't look like there have been many architectural changes, so it should be an easy update. Though I can look into upstreaming the HEAPBUF option.

Here's the chanfs changelog (we're on R0.11a):
http://elm-chan.org/fsw/ff/updates.txt

@mbed-bot
Copy link

Result: SUCCESS

Your command has finished executing! Here's what you wrote!

/morph test-nightly

Output

mbed Build Number: 1717

All builds and test passed!

@adbridge
Copy link
Contributor

@sg- are you happy with @geky comments ?

@geky geky mentioned this pull request Mar 30, 2017
@@ -146,6 +146,16 @@ DWORD get_fattime(void)
| (DWORD)(ptm->tm_sec/2 );
}

void *ff_memalloc(UINT size)
{
return malloc(size);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should this be new or is the OOM condition caught by the caller?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah yeah, you're right this should probably be new

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually I changed this to propogate the error up, it's used for allocating large file buffers, so it's not unreasonable to be propogated as an error code (complies to posix and matches existing use of ff_memalloc in the chanfs layer).

@sg-
Copy link
Contributor

sg- commented Apr 10, 2017

One last question around the performance of using _FS_TINY. Is there a speed impact? The comment is a bit unclear. Also I think trying to get the _FS_HEAPBUF mainlined would give a bit more confidence this is a reasonable change.

@0xc0170
Copy link
Contributor

0xc0170 commented Apr 21, 2017

@geky Update for this one?

@theotherjimmy
Copy link
Contributor

@geky could you address @sg-'s comments?

@adbridge
Copy link
Contributor

@geky bump

@geky geky force-pushed the fat-big-blocks branch from 15e38ac to 530ee2a Compare May 24, 2017 19:35
@geky
Copy link
Contributor Author

geky commented May 24, 2017

It should be updated per feedback.

In regards to speed change, here's a quick benchmark using the "basic" tests found in the sd-driver:
https://github.com/ARMmbed/sd-driver/blob/master/features/TESTS/filesystem/basic/basic.cpp

runtime
without heap blocks 35.484s
with heap blocks 35.451s

In the scale of block device operations there is no speed change.

There may be a speed change when multiple files are being written to from multiple threads, but we really aren't there yet. Memory consumptions seems to be the higher priority.

Copy link
Contributor

@sg- sg- left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM although there is an open issue (enhancement) to upgrade the filesystem. I hope the issue author will make the patch but still makes wonder how long this patch will last before getting overwritten by a copy paste upgrade if not accepted upstream. #4285

@0xc0170
Copy link
Contributor

0xc0170 commented May 25, 2017

/morph test-nightly

@mbed-bot
Copy link

Result: FAILURE

Your command has finished executing! Here's what you wrote!

/morph test-nightly

Output

mbed Build Number: 343

Test failed!

@geky
Copy link
Contributor Author

geky commented May 25, 2017

failure looks like udp loss issue fixed in #4369
/morph test-nightly

@sg-
Copy link
Contributor

sg- commented May 25, 2017

failure looks like udp loss issue fixed in #4369
Should we merge that before running this again?

@geky
Copy link
Contributor Author

geky commented May 25, 2017

The issue is sporadic, though I think the core team already killed the job to make way for higher priority prs.

@mbed-bot
Copy link

Result: ABORTED

Your command has finished executing! Here's what you wrote!

/morph test-nightly

Output

mbed Build Number: 347

Build failed!

@geky
Copy link
Contributor Author

geky commented May 25, 2017

But yeah, we should merge #4369 as soon as we can, currently test-nightly is a bit haywire

@sg-
Copy link
Contributor

sg- commented May 31, 2017

/morph test-nightly

@mbed-bot
Copy link

Result: FAILURE

Your command has finished executing! Here's what you wrote!

/morph test-nightly

Output

mbed Build Number: 382

Test failed!

@sg- sg- added the needs: CI label Jun 3, 2017
This is necessary for support of block devices with >512 byte
blocks, such as most SPI flash parts.

- Enabled support of up to 4096 byte blocks
- Added support for heap-backed buffers using _FS_HEAPBUF
  - Necessary to avoid stack overflows
  - Avoids over-aggresive allocations of _MAX_SS
- Enabled _FS_TINY to further reduce memory footprint
  - Haven't found a downside for this yet except for possible
    thread contention
@0xc0170
Copy link
Contributor

0xc0170 commented Jun 3, 2017

Rebased

@studavekar
Copy link
Contributor

/morph test

@mbed-bot
Copy link

mbed-bot commented Jun 3, 2017

Result: SUCCESS

Your command has finished executing! Here's what you wrote!

/morph test

Output

mbed Build Number: 427

All builds and test passed!

@sg- sg- merged commit ebeb776 into ARMmbed:master Jun 3, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants