New feature: mbed releases

README.md

@@ -163,9 +163,24 @@ Use `mbed ls` to list all the libraries imported to your program:
 
 ```
 $ cd mbed-os-program
-$ mbed ls -a
-mbed-os-program (mbed-os-program)
-`- mbed-os (https://github.com/ARMmbed/mbed-os#89962277c207)
+$ mbed ls
+mbed-os-program (no revision)
+`- mbed-os (#182bbd51bc8d, tags: latest, mbed-os-5.6.5)
+```
+
+Use `mbed releases` to list all releases in a program or library:
+
+```
+$ cd mbed-os
+$ mbed releases
+mbed-os (#182bbd51bc8d, tags: latest, mbed-os-5.6.5)
+  ...
+  * mbed-os-5.6.0 
+  * mbed-os-5.6.1 
+  * mbed-os-5.6.2 
+  * mbed-os-5.6.3 
+  * mbed-os-5.6.4 
+  * mbed-os-5.6.5  <- current
 ```
 
 <span class="notes">**Note**: If you want to start from an existing folder in your workspace, you can use `mbed new .`, which initializes an Mbed program, as well as a new Git or Mercurial repository in that folder. </span>
@@ -686,11 +701,11 @@ To push the changes in your local tree upstream, run `mbed publish`. `mbed publi
 Let's assume that the list of dependencies of your program (obtained by running `mbed ls`) looks like this:
 
 ```
-my-mbed-os-example (a5ac4bf2e468)
-|- mbed-os (5fea6e69ec1a)
-`- my-libs (e39199afa2da)
-   |- my-libs/iot-client (571cfef17dd0)
-   `- my-libs/test-framework (cd18b5a50df4)
+my-mbed-os-example (#a5ac4bf2e468)
+|- mbed-os (#182bbd51bc8d, tags: latest, mbed-os-5.6.5)
+`- my-libs (#e39199afa2da)
+   |- my-libs/iot-client (#571cfef17dd0)
+   `- my-libs/test-framework (#cd18b5a50df4)
 ```
 
 Let's assume that you make changes to `iot-client`. `mbed publish` detects the change on the leaf `iot-client` dependency and asks you to commit it. Then `mbed publish` detects that `my-libs` depends on `iot-client`, updates the `my-libs` dependency on `iot-client` to its latest version by updating the `iot-client.lib` file and asks you to commit it. This propagates up to `my-libs` and finally to your program, `my-mbed-os-example`.
@@ -741,6 +756,44 @@ The update command fails if there are changes in your program or library that `m
 
 ### Updating to an upstream version
 
+Before updating a program or a library, it's good to know the names of the stable releases, usually marked with a tag using a common format, such as `1.2`, `v1.0.1`, `r5.6`, `mbed-os-5.6` and so on. 
+
+You can find stable release versions of a program or a library using the `mbed releases` command:
+
+```
+$ cd mbed-os
+$ mbed releases
+mbed-os (#182bbd51bc8d, tags: latest, mbed-os-5.6.5)
+  ...
+  * mbed-os-5.6.0 
+  * mbed-os-5.6.1 
+  * mbed-os-5.6.2 
+  * mbed-os-5.6.3 
+  * mbed-os-5.6.4 
+  * mbed-os-5.6.5  <- current
+```
+
+You can also recursively list stable releases for your program and libraries using the `-r` switch, for example `mbed releases -r`.
+
+Lastly, you can list unstable releases, such as release candidates, alphas and betas by using the `-u` switch.
+
+```
+$ cd mbed-client
+$ mbed releases -u
+mbed-client (#31e5ce203cc0, tags: v3.0.0)
+  * mbed-os-5.0-rc1 
+  * mbed-os-5.0-rc2 
+  * r0.5-rc4 
+  ...
+  * v2.2.0 
+  * v2.2.1 
+  * v3.0.0  <- current
+```
+
+You can use the `-a` switch to print release and revision hash pairs.
+
+Mbed CLI recognizes stable release if the tags are in standard versioning format, such as `MAJOR[.MINOR][.PATCH][.BUILD]`, and optionally prefixed with either `v`, `r` or `mbed-os`. Unstable releases can be suffixed with any letter/number/hyphen/dot combination.
+
 #### Updating a program
 
 To update your program to another upstream version, go to the root folder of the program, and run:
@@ -751,6 +804,7 @@ $ mbed update [branch|tag|revision]
 
 This fetches new revisions from the remote repository, updating the program to the specified branch, tag or revision. If you don't specify any of these, then `mbed update` updates to the latest revision of the current branch. `mbed update` performs this series of actions recursively against all dependencies in the program tree.
 
+
 #### Updating a library
 
 You can change the working directory to a library folder and use `mbed update` to update that library and its dependencies to a different revision than the one referenced in the parent program or library. This allows you to experiment with different versions of libraries/dependencies in the program tree without having to change the parent program or library.
@@ -769,11 +823,11 @@ There are two main scenarios when updating:
 
 * Update with local uncommitted changes: *dirty* update.
 
-Run `mbed update [branch|revision|tag_name]`. You might have to commit or stash your changes if the source control tool (Git or Mercurial) throws an error that the update will overwrite local changes.
+Run `mbed update [branch|tag|revision]`. You might have to commit or stash your changes if the source control tool (Git or Mercurial) throws an error that the update will overwrite local changes.
 
 * Discard local uncommitted changes: *clean* update.
 
-Run `mbed update [branch|revision|tag_name] --clean`
+Run `mbed update [branch|tag|revision] --clean`
 
 Specifying a branch to `mbed update` will only check out that branch and won't automatically merge or fast-forward to the remote/upstream branch. You can run `mbed update` to merge (fast-forward) your local branch with the latest remote branch. On Git, you can do `git pull`.
 

circle.yml

@@ -7,13 +7,18 @@ test:
         - mbed config -G protocol ssh
         - cd .tests && mbed new new-test
         - cd .tests/new-test && mbed ls
+        - cd .tests/new-test && mbed releases -r
         - cd .tests/new-test && mbed compile --source=. --source=mbed-os/TESTS/integration/basic -j 0
         - cd .tests/new-test && mbed test --compile -n mbed-os-tests-integration-basic -j 0
         - cd .tests && mbed import https://developer.mbed.org/teams/Morpheus/code/mbed-Client-Morpheus-hg hg-test
+        - cd .tests/hg-test && mbed ls
+        - cd .tests/hg-test && mbed releases -r
         - cd .tests/hg-test && mbed update b02527cafcde8612ff051fea57e9975aca598807 --clean
         - cd .tests/hg-test && mbed update --clean
         - cd .tests/hg-test && mbed compile -j 0
         - cd .tests && mbed import https://developer.mbed.org/users/samux/code/USBSerial_HelloWorld bld-test
+        - cd .tests/bld-test && mbed ls
+        - cd .tests/bld-test && mbed releases -r
         - cd .tests/bld-test/mbed && mbed update 85 --clean
         - cd .tests/bld-test && mbed update --clean
         - cd .tests/bld-test && mbed compile -m LPC1768 -j 0

mbed/mbed.py

@@ -110,6 +110,11 @@
 # mbed sdk builds url
 regex_build_url = r'^(https?://([\w\-\.]*mbed\.(co\.uk|org|com))/(users|teams)/([\w\-]{1,32})/(repos|code)/([\w\-]+))/builds/?([\w\-]{6,40}|tip)?/?$'
 
+# match official release tags
+regex_rels_official = r'^(release|rel|mbed-os|[rv]+)?[.-]?\d+(\.\d+)*$'
+# match rc/beta/alpha release tags
+regex_rels_all = r'^(release|rel|mbed-os|[rv]+)?[.-]?\d+(\.\d+)*([a-z0-9.-]+)?$'
+
 # base url for all mbed related repos (used as sort of index)
 mbed_base_url = 'https://github.com/ARMmbed'
 # default mbed OS url
@@ -369,6 +374,9 @@ def getrev():
     def getbranch():
         return "default"
 
+    def gettags(rev=None):
+        return []
+
 
 # pylint: disable=no-self-argument, no-method-argument, no-member, no-self-use, unused-argument
 @scm('hg')
@@ -512,6 +520,15 @@ def getrev():
     def getbranch():
         return pquery([hg_cmd, 'branch']).strip() or ""
 
+    def gettags(rev=None):
+        tags = []
+        refs = pquery([hg_cmd, 'tags']).strip().splitlines() or []
+        for ref in refs:
+            m = re.match(r'^(.+?)\s+(\d+)\:([a-f0-9]+)$', ref)
+            if m and (not rev or m.group(1).startswith(rev)):
+                tags.append(m.group(1) if rev else [m.group(3), m.group(1)])
+        return tags
+
     def remoteid(url, rev=None):
         return pquery([hg_cmd, 'id', '--id', url] + (['-r', rev] if rev else [])).strip() or ""
 
@@ -657,7 +674,7 @@ def checkout(rev, clean=False):
             return
         info("Checkout \"%s\" in %s" % (rev, os.path.basename(os.getcwd())))
         branch = None
-        refs = Git.getrefs(rev)
+        refs = Git.getbranches(rev)
         for ref in refs: # re-associate with a local or remote branch (rev is the same)
             m = re.match(r'^(.*?)\/(.*?)$', ref)
             if m and m.group(2) != "HEAD": # matches origin/<branch> and isn't HEAD ref
@@ -774,17 +791,39 @@ def getbranch(rev='HEAD'):
             branch = "master"
         return branch if branch != "HEAD" else ""
 
-    # Finds refs (local or remote branches). Will match rev if specified
-    def getrefs(rev=None, ret_rev=False):
+    # Get all refs
+    def getrefs():
+        try:
+            return pquery([git_cmd, 'show-ref', '--dereference']).strip().splitlines()
+        except ProcessException:
+            return []
+
+    # Finds branches (local or remote). Will match rev if specified
+    def getbranches(rev=None, ret_rev=False):
         result = []
-        lines = pquery([git_cmd, 'show-ref']).strip().splitlines()
-        for line in lines:
-            m = re.match(r'^(.+)\s+(.+)$', line)
+        refs = Git.getrefs()
+        for ref in refs:
+            m = re.match(r'^(.+)\s+refs\/(heads|remotes)\/(.+)$', ref)
             if m and (not rev or m.group(1).startswith(rev)):
-                if re.match(r'refs\/(heads|remotes)\/', m.group(2)): # exclude tags
-                    result.append(m.group(1) if ret_rev else re.sub(r'refs\/(heads|remotes)\/', '', m.group(2)))
+                result.append(m.group(1) if ret_rev else m.group(3))
         return result
 
+    # Finds tags. Will match rev if specified
+    def gettags(rev=None):
+        tags = []
+        refs = Git.getrefs()
+        for ref in refs:
+            m = re.match(r'^(.+)\s+refs\/tags\/(.+)$', ref)
+            if m and (not rev or m.group(1).startswith(rev)):
+                t = m.group(2)
+                if re.match(r'^(.+)\^\{\}$', t): # detect tag "pointer"
+                    t = re.sub(r'\^\{\}$', '', t) # remove "pointer" chars, e.g. some-tag^{}
+                    for tag in tags:
+                        if tag[1] == t:
+                            tags.remove(tag)
+                tags.append(t if rev else [m.group(1), t])
+        return tags
+
     # Finds branches a rev belongs to
     def revbranches(rev):
         branches = []
@@ -2055,7 +2094,7 @@ def update(rev=None, clean=False, clean_files=False, clean_deps=False, ignore=Fa
 
 # Synch command
 @subcommand('sync',
-    help='Synchronize library references',
+    help='Synchronize library references\n\n',
     description=(
         "Synchronizes all library and dependency references (.lib files) in the\n"
         "current program or library.\n"
@@ -2123,21 +2162,63 @@ def sync(recursive=True, keep_refs=False, top=True):
         "View the dependency tree of the current program or library."))
 def list_(detailed=False, prefix='', p_path=None, ignore=False):
     repo = Repo.fromrepo()
+    revtags = repo.scm.gettags(repo.rev) if repo.rev else []
+    revstr = ('#' + repo.rev[:12] + (', tags: ' + ', '.join(revtags[0:2]) if len(revtags) else '')) if repo.rev else ''
 
-    print "%s (%s)" % (prefix + (relpath(p_path, repo.path) if p_path else repo.name), ((repo.url+('#'+str(repo.rev)[:12] if repo.rev else '') if detailed else str(repo.rev)[:12]) or 'no revision'))
+    print "%s (%s)" % (prefix + (relpath(p_path, repo.path) if p_path else repo.name), ((repo.url + ('#' + str(repo.rev)[:12] if repo.rev else '') if detailed else revstr) or 'no revision'))
 
     for i, lib in enumerate(sorted(repo.libs, key=lambda l: l.path)):
-        if prefix:
-            nprefix = prefix[:-3] + ('|  ' if prefix[-3] == '|' else '   ')
-        else:
-            nprefix = ''
+        nprefix = (prefix[:-3] + ('|  ' if prefix[-3] == '|' else '   ')) if prefix else ''
         nprefix += '|- ' if i < len(repo.libs)-1 else '`- '
 
         if lib.check_repo(ignore):
             with cd(lib.path):
                 list_(detailed, nprefix, repo.path, ignore=ignore)
 
 
+# Command release for cross-SCM release tags of repositories
+@subcommand('releases',
+    dict(name=['-a', '--all'], dest='detailed', action='store_true', help='Show revision hashes'),
+    dict(name=['-u', '--unstable'], dest='unstable', action='store_true', help='Show unstable releases well, e.g. release candidates, alphas, betas, etc'),
+    dict(name=['-r', '--recursive'], action='store_true', help='Show release tags for all libraries and sub-libraries as well'),
+    help='Show release tags',
+    description=(
+        "Show release tags for the current program or library."))
+def releases_(detailed=False, unstable=False, recursive=False, prefix='', p_path=None):
+    repo = Repo.fromrepo()
+    tags = repo.scm.gettags()
+    revtags = repo.scm.gettags(repo.rev)  if repo.rev else [] # associated tags with current commit
+    revstr = ('#' + repo.rev[:12] + (', tags: ' + ', '.join(revtags[0:2]) if len(revtags) else '')) if repo.rev else ''
+    regex_rels = regex_rels_all if unstable else regex_rels_official
+
+    # Generate list of tags
+    rels = []
+    for tag in tags:
+        if re.match(regex_rels, tag[1]):
+            rels.append(tag[1] + " %s%s" % ('#' + tag[0] if detailed else "", " <- current" if tag[1] in revtags else ""))
+
+    # Print header
+    print "%s (%s)" % (prefix + (relpath(p_path, repo.path) if p_path else repo.name), ((repo.url + ('#' + str(repo.rev)[:12] if repo.rev else '') if detailed else revstr) or 'no revision'))
+
+    # Print list of tags
+    rprefix = (prefix[:-3] + ('|  ' if prefix[-3] == '|' else '   ')) if recursive and prefix else ''
+    rprefix += '| ' if recursive and len(repo.libs) > 1 else '  '
+    if len(rels):
+        for rel in rels:
+            print rprefix + '* ' + rel
+    else:
+        print rprefix + 'No release tags detected'
+
+    if recursive:
+        for i, lib in enumerate(sorted(repo.libs, key=lambda l: l.path)):
+            nprefix = (prefix[:-3] + ('|  ' if prefix[-3] == '|' else '   ')) if prefix else ''
+            nprefix += '|- ' if i < len(repo.libs)-1 else '`- '
+
+            if lib.check_repo():
+                with cd(lib.path):
+                    releases_(detailed, unstable, recursive, nprefix, repo.path)
+
+
 # Command status for cross-SCM status of repositories
 @subcommand('status',
     dict(name=['-I', '--ignore'], action='store_true', help='Ignore errors related to missing libraries.'),