Allow filtering Library Manager search by library name

[per1234]

Feature request

Library Manager currently contains over 4000 libraries and more are added every day. An overwhelming number of results may be returned from a search for common keywords (e.g., arduino-cli lib search servo). Arduino CLI does not provide any tools for filtering search results.

This becomes much more of a problem in Arduino IDE, where search is used for the sake of navigating the list in addition to browsing to discover new libraries.

It would be useful to be able to provide users with instructions for installing a library which leads them directly to a specific library rather than sifting through a list of libraries. The logical way to accomplish this would be by allowing the user to limit search keyword scope to the name field.

A fairly standard approach would be to support a "<qualifier>:<value>" style search syntax (e.g., arduino-cli lib search name:servo). Some evidence this is intuitive here. This would typically result in a search for libraries with "servo" in their name. That does reduce the number of search results, but it doesn't provide the ability to restrict the search to an exact name match (case insensitive). I'm not sure what would be the best way to provide that capability.

Even though the motivation for the capability is based on a use case most common to Arduino IDE, it seems to me that the best place to implement it is in Arduino CLI.

Environment

• CLI version: nightly-20211101 Commit: f7b22f7 Date: 2021-11-01T01:27:40Z
• OS and platform: Windows 10

Additional context

Originally requested at https://forum.arduino.cc/t/suggestion-improve-library-manager/847666

Additional requests

• Advanced search for libraries arduino-ide#633
• https://forum.arduino.cc/t/arduino-ide-2-1-0-is-now-available/1117237/7

Related

• Library search exact word enhancement arduino-ide#1915

[zvonler]

Thinking about the syntax here, how about:

• <qualifier>:<value> to match any results with <value> anywhere in their <qualifier> field
• <qualifier>=<value> to match only results with exactly <value> for their <qualifier> field
• <qualifier>~<regex> to match only results for which <regex> matches their <qualifier> field

The output of the "lib search" command suggests that the available qualifier names could be these:

Architecture
Author
Category
Dependencies
Maintainer
Name
Paragraph
Sentence
Types
Versions
Website

I think the search command should match qualifier names in a case-insensitive fashion, and accept unique prefixes, so e.g. arch=esp32 would be equivalent to specifying Architecture=esp32.

I know this has been sitting for a while, but does anyone have comments on the above?

[zvonler]

Hi @cmaglie, I see in your last commit to arduino/utils/search.go that the ':' character is preserved when other non-alphanumerics are removed from the search terms - does something downstream already use those colon characters, or was it possibly to make room for the suggestion in the original description of this ticket? I tried some library searches in the IDE but adding a ':' seemed to eliminate all matches, so I'm trying to figure out what's special about it.

[cmaglie]

Hi @zvonler!

Thinking about the syntax here, how about:

• <qualifier>:<value> to match any results with <value> anywhere in their <qualifier> field
• <qualifier>=<value> to match only results with exactly <value> for their <qualifier> field
• <qualifier>~<regex> to match only results for which <regex> matches their <qualifier> field

I had a chat with my colleagues about this, we like the qualifier:value and qualifier=value syntax, but not much the regex ~.
The concerns about the regexp are that it's difficult to parse, you don't really know when the regexp finishes. For example in the query:

name~Flash arch:script

you don't know if the search is:

• name matches Flash && arch equals script
  or:
• name matches Flash arch:script (that is still a valid regexp).

We also have to consider the case where we want to match multiple words separated by space, in this case we must introduce quoting like:

name:"Flash Storage"
"A multi-word search term" name:"Flash Strorage" "Another Multi-word term"

and, if we want to search the literal name:"Flash Storage" we should also consider quote-escaping:

"name:\"Flash Storage\""
"Punch lib" author:"Robert \"Rocky\" Balboa"

the ':' character is preserved when other non-alphanumerics are removed from the search terms

IIRC it has been preserved to allow searching for FQBN like arduino:avr:uno.

[zvonler]

Thanks for that, I will work on an implementation of the ':' and '=' features.

I'm fine without the regex support, but I will suggest that doing a sed-style approach, where the pattern is surrounded by a user-chosen character that is also expected to terminate the pattern, would alleviate some of those parsing concerns. That might look like name~/pat/, perhaps with support for backslash escaping of the delimiter character. Doing it that way also makes it natural to apply match modifiers after the pattern ending delimiter, e.g. arch~/arm/i for a case-insensitive regex.

[zvonler]

Opened this PR #2373 that does almost everything I suggested - it lacks the partial match support, e.g. "arch" for "architecture". I'm not sure how valuable that is, given that none of the qualifier names are really any longer than the likely strings the user is going to also be typing in.

[alessio-perugini]

Done in: #2373