add mcu family search & filter to downloads page

[durapensa]

Search based on MCU family is working, for boards with family entries in bootloaders.json

E.g., try 'atmel-samd' (83 boards vs 0), 'nrf52840' (25 boards vs 12) or 'esp32s2' (21 boards vs 4).

MCU search could be refined by adding specific MCU part numbers to bootloaders.json in a new field for mcu, for any/all boards (e.g. ATSAMD21 for the Circuit Playground Express, etc.). The same is true for any other component not covered by 'features'.

Filter is not working (no boards displayed), I suspect due to logic in shouldDisplayDownload(), 'Processor family' list contains extraneous entries (first & last, a problem with mcufamilyList generation?), plus layout has issues ('Sort By' is pushed to the bottom). Help! ;-)

Dropping the filter changes from downloads.html until its debugged would allow the MCU family search improvements to go live any time.

[FoamyGuy]

@durapensa thanks for working on this excellent new functionality!

I tested these changes locally. In my case both searching and filtering by mcufamily does seem to work properly.

For filtering:

leads to:

I do see what you mean about sort-by getting pushed down below manufactures. I took a few quick attempts at ideas to resolve that, but came up empty. I'll try to do some more research on css display: grid to see if I can come up with any meaningful suggestions.

At a high level I think we could stack features and mcufamily one on top of the other and then have sort-by stay in the right column. e.g.

But I don't quite know how to achieve this with CSS. If anyone a bit more familiar with CSS has any ideas I'd be happy to try them out.

[durapensa]

@FoamyGuy I like your layout idea. My CSS is rusty, but if no one else has a try this week I will.

The filter logic still needs some work. E.g. On the live site, choosing Manufacturer:Adafruit & Features:Display narrows down the 65 Adafruit boards to the correct 16 boards with displays, but with this PR all 37 Adafruit boards are showing. And when a Manufacturer & Processor family are both chosen, the results are additive rather than subtractitive.

The logic could keep chaining shouldDisplay = true & shouldDisplay = false in nested if/else statements, but I'm hoping for something more elegant.

[FoamyGuy]

Ah, I see.

I did figure out how to modify the page to make the grid stack features and mcufamily. All that is needed is adding an additional div around the divs with classes mcufamilies and features

<div>
  <div class="mcufamilies">
    <fieldset>
      <legend>Processor family</legend>
      <ul class="content"></ul>
    </fieldset>
  </div>
  <div class="features">
    <fieldset>
      <legend>Features</legend>
      <ul class="content"></ul>
    </fieldset>
  </div>
</div>

By inclduing both of those inside of a div it makes the outer div fit into the 2nd column within the grid, but puts each filter div one on top of the other inside of that column:

Maybe a bit of extra margin in between them could be in order, but this is a pretty good start, and turned out to be simpler than I was originally thinking. It doesn't require changing the CSS at all, just inserting the one extra layer of <div> in the html.

[durapensa]

Ooo, that looks great! And easy too. I'm still banging on the JavaScript filter logic; it could be adapted for the functionality described, but extending it will get hairier & hairier.

I'd like to ask for input from the maintainers of downloads.js: is it time to use a JavaScript library like list.js ("tiny, invisible and simple, yet powerful and incredibly fast")?

[jwcooper]

@durapensa I kind of prefer to continue to avoid 3rd party dependencies. I'd like to refactor and clean up the downloads.js file prior to pulling in more dependencies. It's grown quite a bit from when it was originally committed.

[durapensa]

@jwcooper A refactor is a great idea.

A question for the CircuitPython team is whether or not a new 'MCU family' filter and/or future filters are desirable. Knowing the answer to that question will help determine how to [re]structure the filter logic, and whether or not some minimalistic vanilla-javascript library (i.e. not loaded from a 3rd party, but a copy in assets/javascript) could be useful.

[makermelissa]

It's likely future filters will be desired.

[durapensa]

Filter/display should work correctly now - ready for more testing

[FoamyGuy]

I tested the version from the latest commit successfully.

These changes look good to me. The new MCU family filter and searching functionally appears to be working as intended.

[durapensa]

Probably more boards from families atmel-samd, stm, nrf52840, mimxrt10xx can be added to bootloaders.json (with/without bootloader_id entries, depending on existence in, e.g. adafruit/uf2-samdx1/tree/master/boards).

[durapensa]

This might be buried in a review comment, a new branch containing only bootloaders.json edits, with trailing commas removed & mcufamily additions, is at durapensa@1115b64. It covers all 234 boards displayed on circuitpython.org/downloads (checked by ticking all 'Processor family' boxes).

The revised bootloaders.json was generated from a python script, so there are a few alphabetical reorders & other json cleanups too.

It can be included in this PR, or a new PR opened.

[FoamyGuy]

These changes look good to me. I tested the latest commits successfully running locally..

I think we should leave the commas for a future PR, I'll reach out to the person that added this file initially and see if they have any insight into why it is that way or if there is any reason not to remove them so that it conforms to the JSON spec.

[durapensa]

Okay, good by me. Commas cleanup & board additions in #763

[durapensa]

Merged @dhalbert's commits, local testing is okay.

[durapensa]

Probably the guide by @makermelissa at https://learn.adafruit.com/how-to-add-a-new-board-to-the-circuitpython-org-website should be updated to reflect adding an entry with board_id & family to bootloaders.json (and optionally a value for bootloader_id if the board can be flashed/reflashed with a UF2 bootloader living in an adafruit repo).

[makermelissa]

Do new board require adding entries to bootloaders.json? If that's the case, I'd rather have the data in the board file to make adding new boards as easy as possible. I thought the bootloaders.json was something automatically handled by scripts in circuitpython.

[FoamyGuy]

I re-tested locally with the latest commits and everything looks and functions good to me.

@makermelissa adding info to bootloaders.json is a "soft requirement" perhaps. If it's omitted I believe it doesn't cause any serious errors that prevent the device page from working correctly. But it would cause the device not to show up under the appropriate MCU family filter / search that this PR adds and it would cause the bootloader section of the downloads page not to get included so it may be a bit tougher for folks to find information about updating the bootloader.

I'm not sure if bootloaders.json gets used for anything beyond those two functionalities at this time.

[durapensa]

bootloaders.json is a manually edited file, so far as I know; the only way to match up CircuitPython build board_id with the UF2 build board identifier is manually, that is, with build directories found in /adafruit/uf2-samdx1, /adafruit/Adafruit_nRF52_Bootloader, and /adafruit/tinyuf2 (stm, esp32s2, etc.).

Many boards that have UF2 bootloaders built for them however do not have bootloader_id entries in bootloaders.json, probably due to either simple omission or UF2 being ported & built sometime after the board was originally added.

Discovering bootloaders.json and its purpose in the first place was a manual process for me, looking through previous PRs for board adds.

Now that all boards' mcu families are in bootloaders.json, it should be easy to script moving that data to board.md files, but a mention of bootloaders.json and its purpose probably should still be made in the tutorial, for people adding boards that have a UF2-in-flash capability.

[makermelissa]

@makermelissa adding info to bootloaders.json is a "soft requirement" perhaps. If it's omitted I believe it doesn't cause any serious errors that prevent the device page from working correctly. But it would cause the device not to show up under the appropriate MCU family filter / search that this PR adds and it would cause the bootloader section of the downloads page not to get included so it may be a bit tougher for folks to find information about updating the bootloader.

I'm not sure if bootloaders.json gets used for anything beyond those two functionalities at this time.

Ok, I that will work for the time being, but I think I'd like to move this data to the board files at some point. That way if something gets messed up on the formatting and is overlooked, it doesn't affect all the boards. It will also make it so adding your board isn't any more scary than it was before. As it is, we get maybe only around 10% of users who add their boards to CP who will also add them to the website.

[dhalbert]

I did the bootloader PR in #446. I made it a separate file because it was not easy to generate the bootloader info automatically from the various bootloader repos, which don't have board names consistent with the CircuitPython names. Moving to some per-board data files would make sense, but it was more work at the time than a single file.

Also at the time there was also some urgency to provide a spot to download bootloaders because of some significant bootloader bugs.

[durapensa]

Moved to #766