Adds support for symlinked libraries/repos #526 #574

screamerbg · 2017-12-02T15:14:12Z

Addresses #526. @sg- Please test on your setup.

theotherjimmy

Looks great. Thanks!

sg- · 2017-12-06T22:16:05Z

Seems to still have borked my repo

C:\dev\mbed-os-example-blinky>mklink /J mbed-os C:\dev\mbed-os
Junction created for mbed-os <<===>> C:\dev\mbed-os

C:\dev\mbed-os-example-blinky>mbed compile
Building project mbed-os-example-blinky (NRF52_DK, ARM)
Scan: .
Scan: mbed
Scan: env
Scan: FEATURE_BLE
Compile [  0.3%]: AnalogIn.cpp
Compile [  0.7%]: BusIn.cpp
Compile [  1.0%]: BusInOut.cpp
Compile [  1.3%]: main.cpp
Compile [  1.7%]: FlashIAP.cpp
...
Total Static RAM memory (data + bss): 9519 bytes
Total Flash memory (text + data): 58036 bytes

Image: .\BUILD\nrf52_dk\arm\mbed-os-example-blinky.hex

C:\dev\mbed-os-example-blinky>git status
On branch master
Your branch is behind 'origin/master' by 18 commits, and can be fast-forwarded.
  (use "git pull" to update your local branch)
Changes not staged for commit:
  (use "git add <file>..." to update what will be committed)
  (use "git checkout -- <file>..." to discard changes in working directory)

        modified:   main.cpp

Untracked files:
  (use "git add <file>..." to include in what will be committed)

        mbed-os/

no changes added to commit (use "git add" and/or "git commit -a")

C:\dev\mbed-os-example-blinky>mbed update
[mbed] Updating library "mbed-os-example-blinky" to latest revision in the current branch
[mbed] Updating reference "mbed-os-example-blinky" -> "https://github.com/armmbed/mbed-os-example-blinky/#50a0878e40fa6d3d05fb6e1bce1e5ee5537e742a"
[mbed] ERROR: Unpublished changes in "mbed-os" in "C:\dev\mbed-os-example-blinky\mbed-os".
[mbed] ERROR: Please publish them first using the "publish" command.
[mbed] ERROR: You can also use --clean-deps to discard all local commits and replace the library with the one included in this revision. WARNING: This action cannot be undone.
---

C:\dev\mbed-os-example-blinky>mbed update --clean-deps
[mbed] Updating library "mbed-os-example-blinky" to latest revision in the current branch
[mbed] Removing library "mbed-os" (changed URL). Will add from new URL.
[mbed] Adding library "mbed-os" from "https://github.com/ARMmbed/mbed-os" at rev #5f6572179d66

C:\dev\mbed-os-example-blinky>mbed-cli --version
1.2.2

C:\dev\mbed-os-example-blinky>cd ../

C:\dev>cd mbed-os

C:\dev\mbed-os>dir
 Volume in drive C is OSDisk
 Volume Serial Number is C23E-A3EA

 Directory of C:\dev\mbed-os

12/06/2017  03:51 PM    <DIR>          .
12/06/2017  03:51 PM    <DIR>          ..
               0 File(s)              0 bytes
               2 Dir(s)  387,041,562,624 bytes free

theotherjimmy · 2017-12-06T22:38:28Z

@sg- Did you use the branch this PR is proposing for that test? I can't tell from the log (It will also show up as 1.2.2).

sg- · 2017-12-06T23:20:24Z

Not sure but I think so. Here is what I did.

C:\dev\daplink>cd ..

C:\dev>cd mbed-cli

C:\dev\mbed-cli>git status
On branch screamer-f/symlinks-support
nothing to commit, working tree clean

C:\dev\mbed-cli>git log -2
commit c52ab12480ca84d58a1d2c43e7d67408a009d0d6
Merge: 001c68b b792281
Author: Sam Grove <[email protected]>
Date:   Tue Dec 5 10:03:00 2017 -0600

    Merge branch 'f/symlinks-support' of https://github.com/screamerbg/neo into screamer-f/symlinks-support

commit b79228158496bbf6cad0d2fd909b3d0927ef191a
Author: Mihail Stoyanov <[email protected]>
Date:   Sat Dec 2 15:12:44 2017 +0000

    Adds support for symlinked libraries/repos #526

C:\dev\mbed-cli>

and then ran

python setup.py install

sg-

Doesn't seem to work for me but maybe I did something wrong?

theotherjimmy · 2017-12-07T21:18:01Z

@screamerbg While your fixing that commit, you misspelled symlink as symlunk in the commit message.

This is a workaround to python's os.getcwd() which always returns the real path, not the logical one. Due to this os.getcwd() would never return the path inside a symlunk. This commit also introduces getcwd() implementation that uses _cwd.

screamerbg · 2017-12-07T22:43:47Z

@sg- I'm unable to reproduce this behavior.

D:\Work\testing>mkdir test

D:\Work\testing>cd test

D:\Work\testing\test>mbed import mbed-os-example-blinky
[mbed] Importing program "mbed-os-example-blinky" from "https://github.com/ARMmbed/mbed-os-example-blinky" at latest revision in the current branch
[mbed] Adding library "mbed-os" from "https://github.com/ARMmbed/mbed-os" at rev #5f6572179d66

D:\Work\testing\test>git clone https://github.com/ARMmbed/mbed-os
Cloning into 'mbed-os'...
remote: Counting objects: 158750, done.
remote: Total 158750 (delta 0), reused 1 (delta 0), pack-reused 158749R
Receiving objects: 100% (158750/158750), 180.59 MiB | 6.75 MiB/s, done.
Resolving deltas: 100% (107416/107416), done.
Checking connectivity... done.
Checking out files: 100% (12310/12310), done.

D:\Work\testing\test>cd mbed-os-example-blinky

D:\Work\testing\test\mbed-os-example-blinky>rm -fr mbed-os

D:\Work\testing\test\mbed-os-example-blinky>mklink /J mbed-os d:\Work\testing\test\mbed-os
Junction created for mbed-os <<===>> d:\Work\testing\test\mbed-os

D:\Work\testing\test\mbed-os-example-blinky>mbed update
[mbed] Updating program "mbed-os-example-blinky" to latest revision in the current branch
[mbed] Updating library "mbed-os" to rev #5f6572179d66

D:\Work\testing\test\mbed-os-example-blinky>cd ..

D:\Work\testing\test>cd mbed-os

D:\Work\testing\test\mbed-os>git log -1
WARNING: terminal is not fully functional
commit 5f6572179d66ce4c09d6517b659ac51133cc980d
Merge: 182bbd5 4ad5e42
Author: Martin Kojtal <[email protected]>
Date:   Mon Dec 4 11:55:10 2017 +0000

    Merge pull request #5634 from ARMmbed/release-candidate

    Release candidate for mbed-os-5.6.6

The symlinked mbed-os manual clone is updated correctly. This is using mbed CLI from this PR with symlink support.

@sg- could you provide from scratch command log so I can reproduce it? Thanks in advance.

screamerbg · 2017-12-07T22:47:00Z

Also I tried updating mbed-os-example-blinky to a specific branch and the symlinked mbed-os was updated just fine.

D:\Work\testing\test\mbed-os>cd ..

D:\Work\testing\test>cd mbed-os-example-blinky

D:\Work\testing\test\mbed-os-example-blinky>mbed update mbed-os-5.5.5
[mbed] Updating program "mbed-os-example-blinky" to branch mbed-os-5.5.5
[mbed] Updating library "mbed-os" to rev #db4be94693c3

D:\Work\testing\test\mbed-os-example-blinky>cd ..

D:\Work\testing\test>cd mbed-os

D:\Work\testing\test\mbed-os>git log -1
WARNING: terminal is not fully functional
commit db4be94693c3a873cfa0c025a5ad2e62b86a8474
Merge: 4c256f0 4e1ec4a
Author: Martin Kojtal <[email protected]>
Date:   Mon Aug 14 11:10:07 2017 +0100

    Merge pull request #4889 from ARMmbed/release-candidate

    Release candidate for mbed-os-5.5.5

sg- · 2017-12-12T06:04:47Z

Hmm. I deleted all mbed-cli versions again and pulled a your neo fork of mbed-cli and all seems ok now. Maybe the merge of the remote branch and local install was messed up. I had also force pulled the remote and verified the sha with your branch but dont have that in the logs and didn't post here (I might not have cleaned cli installs or not installed). Anyways, seems we have different terminal versions rm versus rmdir but it seems to work ok. Logs below.

C:\dev>pip uninstall mbed-cli
Uninstalling mbed-cli-1.2.2:
  c:\python27\lib\site-packages\mbed_cli-1.2.2-py2.7.egg
  c:\python27\scripts\mbed-cli-script.py
  c:\python27\scripts\mbed-cli.exe
  c:\python27\scripts\mbed-cli.exe.manifest
  c:\python27\scripts\mbed-script.py
  c:\python27\scripts\mbed.exe
  c:\python27\scripts\mbed.exe.manifest
Proceed (y/n)? y
  Successfully uninstalled mbed-cli-1.2.2
Traceback (most recent call last):
  File "c:\python27\lib\runpy.py", line 174, in _run_module_as_main
    "__main__", fname, loader, pkg_name)
  File "c:\python27\lib\runpy.py", line 72, in _run_code
    exec code in run_globals
  File "C:\Python27\Scripts\pip.exe\__main__.py", line 9, in <module>
  File "c:\python27\lib\site-packages\pip\__init__.py", line 233, in main
    return command.main(cmd_args)
  File "c:\python27\lib\site-packages\pip\basecommand.py", line 252, in main
    pip_version_check(session)
  File "c:\python27\lib\site-packages\pip\utils\outdated.py", line 102, in pip_version_check
    installed_version = get_installed_version("pip")
  File "c:\python27\lib\site-packages\pip\utils\__init__.py", line 838, in get_installed_version
    working_set = pkg_resources.WorkingSet()
  File "c:\python27\lib\site-packages\pip\_vendor\pkg_resources\__init__.py", line 644, in __init__
    self.add_entry(entry)
  File "c:\python27\lib\site-packages\pip\_vendor\pkg_resources\__init__.py", line 700, in add_entry
    for dist in find_distributions(entry, True):
  File "c:\python27\lib\site-packages\pip\_vendor\pkg_resources\__init__.py", line 1949, in find_eggs_in_zip
    if metadata.has_metadata('PKG-INFO'):
  File "c:\python27\lib\site-packages\pip\_vendor\pkg_resources\__init__.py", line 1463, in has_metadata
    return self.egg_info and self._has(self._fn(self.egg_info, name))
  File "c:\python27\lib\site-packages\pip\_vendor\pkg_resources\__init__.py", line 1823, in _has
    return zip_path in self.zipinfo or zip_path in self._index()
  File "c:\python27\lib\site-packages\pip\_vendor\pkg_resources\__init__.py", line 1703, in zipinfo
    return self._zip_manifests.load(self.loader.archive)
  File "c:\python27\lib\site-packages\pip\_vendor\pkg_resources\__init__.py", line 1643, in load
    mtime = os.stat(path).st_mtime
WindowsError: [Error 2] The system cannot find the file specified: 'c:\\python27\\lib\\site-packages\\mbed_cli-1.2.2-py2.7.egg'

C:\dev>pip uninstall mbed-cli
Cannot uninstall requirement mbed-cli, not installed

C:\dev>pip uninstall mbed-cli
Cannot uninstall requirement mbed-cli, not installed

C:\dev>mkdir temp

C:\dev>cd temp

C:\dev\temp>git clone https://github.com/screamerbg/neo
Cloning into 'neo'...
remote: Counting objects: 2290, done.
remote: Total 2290 (delta 0), reused 0 (delta 0), pack-reused 2290Receiving objects:  98% (2245/2290), 748.00 KiB | 282.00 KiB/s
Receiving objects: 100% (2290/2290), 761.08 KiB | 282.00 KiB/s, done.
Resolving deltas: 100% (1146/1146), done.

C:\dev\temp>cd neo

C:\dev\temp\neo>git checkout f/symlinks-support
Switched to a new branch 'f/symlinks-support'
Branch f/symlinks-support set up to track remote branch f/symlinks-support from origin.

C:\dev\temp\neo>git log
commit c6ab9bd87d5acbbc8bd75383ff65f4f2bc98d8dd
Author: Mihail Stoyanov <[email protected]>
Date:   Thu Dec 7 21:19:13 2017 +0000

    Make use of global variable _cwd to store the path cd(path) changes to.
    This is a workaround to python's os.getcwd() which always returns the real path, not the logical one.
    Due to this os.getcwd() would never return the path inside a symlunk.
    This commit also introduces getcwd() implementation that uses _cwd.

commit b79228158496bbf6cad0d2fd909b3d0927ef191a
Author: Mihail Stoyanov <[email protected]>
Date:   Sat Dec 2 15:12:44 2017 +0000

    Adds support for symlinked libraries/repos #526

commit 63c9daa87f1f28e82b7dfef518257490c54c70f0
Author: Mihail Stoyanov <[email protected]>
Date:   Mon Oct 2 16:51:34 2017 +0100

    Fix hardcoded 'python' call and use python_cmd var instead
    Also bump version to 1.2.2

commit c2f7b06cdf6c2803fc8e2c7de55c04be0b28b2f9
Author: Mihail Stoyanov <[email protected]>
Date:   Thu Sep 28 12:19:28 2017 +0100

    Release 1.2.1

commit fc5efdb463f9a8901450999e181663e13d52810d

C:\dev\temp\neo>python setup.py install
running install
running bdist_egg
running egg_info
creating mbed_cli.egg-info
writing mbed_cli.egg-info\PKG-INFO
writing top-level names to mbed_cli.egg-info\top_level.txt
writing dependency_links to mbed_cli.egg-info\dependency_links.txt
writing entry points to mbed_cli.egg-info\entry_points.txt
writing manifest file 'mbed_cli.egg-info\SOURCES.txt'
reading manifest file 'mbed_cli.egg-info\SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'mbed_cli.egg-info\SOURCES.txt'
installing library code to build\bdist.win32\egg
running install_lib
running build_py
creating build
creating build\lib
creating build\lib\mbed
copying mbed\mbed.py -> build\lib\mbed
copying mbed\__init__.py -> build\lib\mbed
creating build\bdist.win32
creating build\bdist.win32\egg
creating build\bdist.win32\egg\mbed
copying build\lib\mbed\mbed.py -> build\bdist.win32\egg\mbed
copying build\lib\mbed\__init__.py -> build\bdist.win32\egg\mbed
byte-compiling build\bdist.win32\egg\mbed\mbed.py to mbed.pyc
byte-compiling build\bdist.win32\egg\mbed\__init__.py to __init__.pyc
creating build\bdist.win32\egg\EGG-INFO
copying mbed_cli.egg-info\PKG-INFO -> build\bdist.win32\egg\EGG-INFO
copying mbed_cli.egg-info\SOURCES.txt -> build\bdist.win32\egg\EGG-INFO
copying mbed_cli.egg-info\dependency_links.txt -> build\bdist.win32\egg\EGG-INFO
copying mbed_cli.egg-info\entry_points.txt -> build\bdist.win32\egg\EGG-INFO
copying mbed_cli.egg-info\top_level.txt -> build\bdist.win32\egg\EGG-INFO
zip_safe flag not set; analyzing archive contents...
creating dist
creating 'dist\mbed_cli-1.2.2-py2.7.egg' and adding 'build\bdist.win32\egg' to it
removing 'build\bdist.win32\egg' (and everything under it)
Processing mbed_cli-1.2.2-py2.7.egg
Copying mbed_cli-1.2.2-py2.7.egg to c:\python27\lib\site-packages
Adding mbed-cli 1.2.2 to easy-install.pth file
Installing mbed-script.py script to C:\Python27\Scripts
Installing mbed.exe script to C:\Python27\Scripts
Installing mbed.exe.manifest script to C:\Python27\Scripts
Installing mbed-cli-script.py script to C:\Python27\Scripts
Installing mbed-cli.exe script to C:\Python27\Scripts
Installing mbed-cli.exe.manifest script to C:\Python27\Scripts

Installed c:\python27\lib\site-packages\mbed_cli-1.2.2-py2.7.egg
Processing dependencies for mbed-cli==1.2.2
Finished processing dependencies for mbed-cli==1.2.2

C:\dev\temp\neo>cd ..

C:\dev\temp>mbed import mbed-os-example-blinky
[mbed] Importing program "mbed-os-example-blinky" from "https://github.com/ARMmbed/mbed-os-example-blinky" at latest revision in the current branch
[mbed] Adding library "mbed-os" from "https://github.com/ARMmbed/mbed-os" at rev #5f6572179d66

C:\dev\temp>git clone https://github.com/ARMmbed/mbed-os
Cloning into 'mbed-os'...
remote: Counting objects: 158772, done.
remote: Total 158772 (delta 0), reused 1 (delta 0), pack-reused 158771
Receiving objects: 100% (158772/158772), 180.61 MiB | 419.00 KiB/s, done.
Resolving deltas: 100% (107432/107432), done.
Checking out files: 100% (12310/12310), done.

C:\dev\temp>cd mbed-os-example-blinky

C:\dev\temp\mbed-os-example-blinky>rm -fr mbed-os
'rm' is not recognized as an internal or external command,
operable program or batch file.

C:\dev\temp\mbed-os-example-blinky>del mbed-os
C:\dev\temp\mbed-os-example-blinky\mbed-os\*, Are you sure (Y/N)? Y

C:\dev\temp\mbed-os-example-blinky>rmdir mbed-os
The directory is not empty.

C:\dev\temp\mbed-os-example-blinky>rmdir mbed-os /s
mbed-os, Are you sure (Y/N)? Y

C:\dev\temp\mbed-os-example-blinky>mklink /J mbed-os C:\dev\temp\mbed-os
Junction created for mbed-os <<===>> C:\dev\temp\mbed-os

C:\dev\temp\mbed-os-example-blinky>git log -2
commit 50a0878e40fa6d3d05fb6e1bce1e5ee5537e742a
Merge: 38dba8b 8f2dd7e
Author: Martin Kojtal <[email protected]>
Date:   Mon Dec 4 14:20:21 2017 +0000

    Merge pull request #98 from adbridge/master

    Updating mbed-os to mbed-os-5.6.6

commit 8f2dd7e93bfec34fff9e97c06f7975d2659199e1
Author: adbridge <[email protected]>
Date:   Mon Dec 4 12:51:13 2017 +0000

    Updating mbed-os to mbed-os-5.6.6

C:\dev\temp\mbed-os-example-blinky>git status
On branch master
Your branch is up-to-date with 'origin/master'.
nothing to commit, working tree clean

C:\dev\temp\mbed-os-example-blinky>cd mbed-os

C:\dev\temp\mbed-os-example-blinky\mbed-os>git log -2
commit 2e1c2a1cdf168dd739e777165b21f6cd0509c188
Merge: 41591eb83 c6130306e
Author: Martin Kojtal <[email protected]>
Date:   Fri Dec 1 08:15:26 2017 +0000

    Merge pull request #5538 from geky/littlefs-staging

    Integrate littlefs into mbed OS

commit 41591eb83b11f167054fd1d5236680de88c5885f
Merge: 9f8ac6523 3252fa097
Author: Martin Kojtal <[email protected]>
Date:   Thu Nov 30 18:26:45 2017 +0000

    Merge pull request #5602 from artokin/nanostack_release_v704

    Nanostack release v704

C:\dev\temp\mbed-os-example-blinky\mbed-os>cd ..

C:\dev\temp\mbed-os-example-blinky>mbed update
[mbed] Updating program "mbed-os-example-blinky" to latest revision in the current branch
[mbed] Updating library "mbed-os" to rev #5f6572179d66

C:\dev\temp\mbed-os-example-blinky>cd mbed-os

C:\dev\temp\mbed-os-example-blinky\mbed-os>git log -2
commit 5f6572179d66ce4c09d6517b659ac51133cc980d
Merge: 182bbd51b 4ad5e42ca
Author: Martin Kojtal <[email protected]>
Date:   Mon Dec 4 11:55:10 2017 +0000

    Merge pull request #5634 from ARMmbed/release-candidate

    Release candidate for mbed-os-5.6.6

commit 4ad5e42ca7e320de7d8a130887ac6e21744c7a6a
Author: adbridge <[email protected]>
Date:   Fri Dec 1 17:01:47 2017 +0000

    Update Mbed version block for patch release

C:\dev\temp\mbed-os-example-blinky\mbed-os>git status
On branch mbed-os-5.6
Your branch is up-to-date with 'origin/mbed-os-5.6'.
nothing to commit, working tree clean

C:\dev\temp\mbed-os-example-blinky\mbed-os>

theotherjimmy · 2017-12-12T16:41:28Z

mbed/mbed.py

@@ -1389,6 +1398,8 @@ def update_tools(self, path):
            with cd(os.path.join(path, tools_dir)):
                try:
                    action("Updating the mbed 2.0 SDK tools...")
+                    print path
+                    print os.path.join(path, tools_dir)


These look like debug prints. If they are, could you remove them?

Thanks for spotting this.

Adds support for symlinked libraries/repos ARMmbed#526

b792281

theotherjimmy approved these changes Dec 4, 2017

View reviewed changes

screamerbg requested a review from sg- December 4, 2017 18:24

sg- reviewed Dec 6, 2017

View reviewed changes

screamerbg added the needs work label Dec 7, 2017

screamerbg force-pushed the f/symlinks-support branch 2 times, most recently from 5443c4d to 86e85cd Compare December 7, 2017 20:54

screamerbg force-pushed the f/symlinks-support branch from 86e85cd to c6ab9bd Compare December 7, 2017 21:19

sg- approved these changes Dec 12, 2017

View reviewed changes

theotherjimmy reviewed Dec 12, 2017

View reviewed changes

Remove prints

b664f2b

screamerbg removed the needs work label Dec 19, 2017

screamerbg merged commit 9aff07b into ARMmbed:master Dec 19, 2017

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Adds support for symlinked libraries/repos #526 #574

Adds support for symlinked libraries/repos #526 #574

Uh oh!

screamerbg commented Dec 2, 2017

Uh oh!

theotherjimmy left a comment

Uh oh!

sg- commented Dec 6, 2017 •

edited

Loading

Uh oh!

theotherjimmy commented Dec 6, 2017 •

edited

Loading

Uh oh!

sg- commented Dec 6, 2017

Uh oh!

sg- left a comment

Uh oh!

theotherjimmy commented Dec 7, 2017

Uh oh!

screamerbg commented Dec 7, 2017 •

edited

Loading

Uh oh!

screamerbg commented Dec 7, 2017 •

edited

Loading

Uh oh!

sg- commented Dec 12, 2017

Uh oh!

theotherjimmy Dec 12, 2017

Uh oh!

screamerbg Dec 19, 2017

Uh oh!

Uh oh!

Adds support for symlinked libraries/repos #526 #574

Adds support for symlinked libraries/repos #526 #574

Uh oh!

Conversation

screamerbg commented Dec 2, 2017

Uh oh!

theotherjimmy left a comment

Choose a reason for hiding this comment

Uh oh!

sg- commented Dec 6, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

theotherjimmy commented Dec 6, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

sg- commented Dec 6, 2017

Uh oh!

sg- left a comment

Choose a reason for hiding this comment

Uh oh!

theotherjimmy commented Dec 7, 2017

Uh oh!

screamerbg commented Dec 7, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

screamerbg commented Dec 7, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

sg- commented Dec 12, 2017

Uh oh!

theotherjimmy Dec 12, 2017

Choose a reason for hiding this comment

Uh oh!

screamerbg Dec 19, 2017

Choose a reason for hiding this comment

Uh oh!

Uh oh!

sg- commented Dec 6, 2017 •

edited

Loading

theotherjimmy commented Dec 6, 2017 •

edited

Loading

screamerbg commented Dec 7, 2017 •

edited

Loading

screamerbg commented Dec 7, 2017 •

edited

Loading