Installation failure on Julia ≥ 1.4 & Mac #46

mcabbott · 2020-04-21T08:58:57Z

I can't seem to make this work on Julia 1.4 or nightly, dev or add appears to install, but it freezes when I ask it to calculate anything:

(@v1.5) pkg> add IntelVectorMath
   Updating registry at `~/.julia/registries/General`
   Updating git-repo `https://github.com/JuliaRegistries/General.git`
┌ Warning: Some registries failed to update:
│     — /Users/me/.julia/registries/General — failed to fetch from repo
└ @ Pkg.Types /Users/julia/buildbot/worker/package_macos64/build/usr/share/julia/stdlib/v1.5/Pkg/src/Types.jl:1118
  Resolving package versions...
   Updating `~/.julia/environments/v1.5/Project.toml`
   c8ce9da6 + IntelVectorMath v0.4.0
   Updating `~/.julia/environments/v1.5/Manifest.toml`
   1d5cc7b8 + IntelOpenMP_jll v2018.0.3+0
   c8ce9da6 + IntelVectorMath v0.4.0
   856f044c + MKL_jll v2020.0.166+0

(@v1.5) pkg> build IntelVectorMath

julia> using IntelVectorMath
[ Info: Precompiling IntelVectorMath [c8ce9da6-5d36-5c03-b118-5a70151be7bc]

julia> pkgdir(IntelVectorMath)
"/Users/me/.julia/packages/IntelVectorMath/BuWXf"

julia> exp.(ones(2))
2-element Array{Float64,1}:
 2.718281828459045
 2.718281828459045

julia> IVM.exp(ones(2))
^CKilled: 9

$ cd /Users/me/.julia/packages/IntelVectorMath/BuWXf
$ ls
LICENSE.md   Project.toml README.md    benchmark    src          test

There seems to be no /deps/build.log file, and perhaps its failure to write here means something?

I can install it on Julia 1.3, on which I have MKL.jl installed. I can try other combinations of versions & MKL but haven't yet. (My 1.4 is the official download.) However in the deps/build.log which I think belongs to the 1.3 version, there are a bunch of errors printed:

$ ls
BuWXf r9uot uZB3Y
$ cd r9uot
$ ls
LICENSE.md   Project.toml README.md    benchmark    deps         src          test
$ cd deps
$ ls
build.jl  build.log usr
$ cat build.log
[ Info: Downloading https://github.com/JuliaPackaging/Yggdrasil/releases/download/MKL-v2019.0.117/MKL.v2019.0.117.x86_64-apple-darwin14.tar.gz to /Users/me/.julia/packages/IntelVectorMath/r9uot/deps/usr/downloads/MKL.v2019.0.117.x86_64-apple-darwin14.tar.gz...
ERROR: LoadError: Could not download https://github.com/JuliaPackaging/Yggdrasil/releases/download/MKL-v2019.0.117/MKL.v2019.0.117.x86_64-apple-darwin14.tar.gz to /Users/me/.julia/packages/IntelVectorMath/r9uot/deps/usr/downloads/MKL.v2019.0.117.x86_64-apple-darwin14.tar.gz:
TaskFailedException(Task (failed) @0x00000001069f1210)
Stacktrace:
 [1] error(::String) at /Applications/Julia-1.5.app/Contents/Resources/julia/lib/julia/sys.dylib:?
 [2] download(::String, ::String; verbose::Bool) at /Users/me/.julia/packages/BinaryProvider/kcGxO/src/PlatformEngines.jl:502
 [3] download_verify(::String, ::String, ::String; verbose::Bool, force::Bool, quiet_download::Bool) at /Users/me/.julia/packages/BinaryProvider/kcGxO/src/PlatformEngines.jl:571
 [4] install(::String, ::String; prefix::Prefix, tarball_path::String, force::Bool, ignore_platform::Bool, verbose::Bool) at /Users/me/.julia/packages/BinaryProvider/kcGxO/src/Prefix.jl:314
 [5] top-level scope at /Users/me/.julia/packages/IntelVectorMath/r9uot/deps/build.jl:51
 [6] include(::String) at /Applications/Julia-1.5.app/Contents/Resources/julia/lib/julia/sys.dylib:?
 [7] top-level scope at none:5
in expression starting at /Users/me/.julia/packages/IntelVectorMath/r9uot/deps/build.jl:49
caused by [exception 1]
TaskFailedException:
InterruptException:
Stacktrace:
 [1] poptaskref(::Base.InvasiveLinkedListSynchronized{Task}) at /Applications/Julia-1.5.app/Contents/Resources/julia/lib/julia/sys.dylib:?
 [2] wait(::Base.GenericCondition{Base.Threads.SpinLock}) at /Applications/Julia-1.5.app/Contents/Resources/julia/lib/julia/sys.dylib:?
 [3] wait_readnb(::Base.PipeEndpoint, ::Int64) at /Applications/Julia-1.5.app/Contents/Resources/julia/lib/julia/sys.dylib:?
 [4] eof at ./stream.jl:48 [inlined]
 [5] eof at ./io.jl:361 [inlined]
 [6] readuntil_many(::Pipe, ::Array{Char,1}) at /Users/me/.julia/packages/BinaryProvider/kcGxO/src/OutputCollector.jl:22
 [7] macro expansion at /Users/me/.julia/packages/BinaryProvider/kcGxO/src/OutputCollector.jl:49 [inlined]
 [8] (::BinaryProvider.var"#1#3"{Pipe,Base.GenericCondition{Base.AlwaysLockedST},Array{Tuple{Float64,String},1}})() at ./task.jl:358
Stacktrace:
 [1] wait at ./task.jl:267 [inlined]
 [2] fetch at ./task.jl:282 [inlined]
 [3] wait(::OutputCollector) at /Users/me/.julia/packages/BinaryProvider/kcGxO/src/OutputCollector.jl:155
 [4] download(::String, ::String; verbose::Bool) at /Users/me/.julia/packages/BinaryProvider/kcGxO/src/PlatformEngines.jl:495
 [5] download_verify(::String, ::String, ::String; verbose::Bool, force::Bool, quiet_download::Bool) at /Users/me/.julia/packages/BinaryProvider/kcGxO/src/PlatformEngines.jl:571
 [6] install(::String, ::String; prefix::Prefix, tarball_path::String, force::Bool, ignore_platform::Bool, verbose::Bool) at /Users/me/.julia/packages/BinaryProvider/kcGxO/src/Prefix.jl:314
 [7] top-level scope at /Users/me/.julia/packages/IntelVectorMath/r9uot/deps/build.jl:51
 [8] include(::String) at /Applications/Julia-1.5.app/Contents/Resources/julia/lib/julia/sys.dylib:?
 [9] top-level scope at none:5
AVX2 support detected, vml_avx2 selected

The text was updated successfully, but these errors were encountered:

KristofferC · 2020-04-21T09:06:52Z

I can't seem to install this on Julia 1.4 or nightly, dev or add leads me to it freezing when I ask it to calculate anything:

You can install it, it just hangs when you call a function. I can repro, and it seems like this happens on 1.4 but not 1.3?

KristofferC · 2020-04-21T09:11:21Z

Here is an MWE that hangs for me:

using MKL_jll

gamma1(x) = (x = copy(x); ccall((:vdTGamma, libmkl_rt), Cvoid, (Int32, Ptr{Float64}, Ptr{Float64}), length(x), x, x); x);
gamma1([1.0, 2.0])

mcabbott · 2020-04-21T09:16:09Z

You can install it, it just hangs when you call a function.

Yes, sorry, proofreading failure too. (Now fixed.)

Your example crashes immediately for me on 1.4, while mine freezes and needs to be ctrl-C-ed.

KristofferC · 2020-04-21T09:17:54Z

Can you try with 1.3? And with the nightly?

mcabbott · 2020-04-21T09:21:31Z

Your example also works on my 1.3 (which has MKL.jl), and also fails on my 1.5. Both are MKL_jll v2020.0.166+0. (1.5 is 1.5.0-DEV.585 (2020-04-07) Commit 16652a6655* (13 days old master) with some packagecompiler-ing of plots & revise, but 1.4 is the default downloaded binary. Neither has MKL.jl.)

(@v1.5) pkg> st MKL_jll
     Status `~/.julia/environments/v1.5/Project.toml`
   856f044c MKL_jll v2020.0.166+0

julia> using MKL_jll

julia> gamma1(x) = (x = copy(x); ccall((:vdTGamma, libmkl_rt), Cvoid, (Int32, Ptr{Float64}, Ptr{Float64}), length(x), x, x); x);

julia> gamma1([1.0, 2.0])

^CKilled: 9

KristofferC · 2020-04-21T09:44:33Z

Okay thanks. So there has been some regression here in 1.3 to 1.4. I tried 1.4-RC2 and it seems to work there but it failed on 1.4.0.

I noticed that it works when I use my standard installation of MKL. So I looked at the difference between the libraries and there was one file that had a different hash: libmkl_intel_thread.dylib). When I copied that file from my local installation to the one MKL_jll uses it works.

So my conclusion is that the libmkl_intel_thread.dylib that MKL_jll provides on mac is somehow failty.

KristofferC · 2020-04-21T09:50:05Z

Opened JuliaPackaging/Yggdrasil#915

KristofferC · 2020-04-21T10:51:25Z

Can you try run

pkg> add https://github.com/KristofferC/MKL_jll.jl

and try again?

mcabbott · 2020-04-21T11:02:26Z

OK, that appears to fix things on 1.4:

  Resolving package versions...
   Updating `~/.julia/environments/v1.4/Project.toml`
  [856f044c] + MKL_jll v2020.0.166+1 #master (https://github.com/KristofferC/MKL_jll.jl)
   Updating `~/.julia/environments/v1.4/Manifest.toml`
  [1d5cc7b8] + IntelOpenMP_jll v2018.0.3+0
  [856f044c] + MKL_jll v2020.0.166+1 #master (https://github.com/KristofferC/MKL_jll.jl)

julia> using MKL_jll
[ Info: Precompiling MKL_jll [856f044c-d86e-5d09-b602-aeab76dc8ba7]
Downloading artifact: MKL

Downloading artifact: MKL

julia> gamma1(x) = (x = copy(x); ccall((:vdTGamma, libmkl_rt), Cvoid, (Int32, Ptr{Float64}, Ptr{Float64}), length(x), x, x); x);

julia> gamma1([1.0, 2.0])
2-element Array{Float64,1}:
 1.0
 1.0

(@v1.4) pkg> add IntelVectorMath
  Resolving package versions...
   Updating `~/.julia/environments/v1.4/Project.toml`
  [c8ce9da6] + IntelVectorMath v0.4.0
   Updating `~/.julia/environments/v1.4/Manifest.toml`
  [c8ce9da6] + IntelVectorMath v0.4.0

(@v1.4) pkg> build IntelVectorMath

julia> using IntelVectorMath
[ Info: Precompiling IntelVectorMath [c8ce9da6-5d36-5c03-b118-5a70151be7bc]

julia> IVM.exp(ones(3))
3-element Array{Float64,1}:
 2.718281828459045
 2.718281828459045
 2.718281828459045

And on 1.5 too. Many thanks!

Crown421 · 2020-04-21T11:23:39Z

This was an issue for the last couple of months with the github actions CI for nightly, I mentioned it on the slack channel.
Thanks for finding a fix @KristofferC

KristofferC · 2020-04-24T05:46:29Z

This should be fixed with the new release of MKL_jll. We should make sure we run CI on 1.4.

mcabbott · 2020-04-24T20:07:05Z

Great, can confirm that MKL_jll v2020.0.166+1 solves the problem here. Thanks again for figuring this stuff out!

xuangu-fang · 2021-04-14T17:05:39Z

Can you try run
pkg> add https://github.com/KristofferC/MKL_jll.jl
and try again?

it helps to fix on 1.6 on win10

KristofferC mentioned this issue Apr 21, 2020

MKL_jll does not function properly on Mac in Julia 1.4 JuliaPackaging/Yggdrasil#915

Closed

mcabbott closed this as completed Apr 24, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Installation failure on Julia ≥ 1.4 & Mac #46

Installation failure on Julia ≥ 1.4 & Mac #46

mcabbott commented Apr 21, 2020 •

edited

Loading

KristofferC commented Apr 21, 2020

Uh oh!

KristofferC commented Apr 21, 2020

Uh oh!

mcabbott commented Apr 21, 2020 •

edited

Loading

Uh oh!

KristofferC commented Apr 21, 2020

Uh oh!

mcabbott commented Apr 21, 2020 •

edited

Loading

Uh oh!

KristofferC commented Apr 21, 2020 •

edited

Loading

Uh oh!

KristofferC commented Apr 21, 2020

Uh oh!

KristofferC commented Apr 21, 2020

Uh oh!

mcabbott commented Apr 21, 2020 •

edited

Loading

Uh oh!

Crown421 commented Apr 21, 2020

Uh oh!

KristofferC commented Apr 24, 2020

Uh oh!

mcabbott commented Apr 24, 2020

Uh oh!

xuangu-fang commented Apr 14, 2021

Uh oh!

Installation failure on Julia ≥ 1.4 & Mac #46

Installation failure on Julia ≥ 1.4 & Mac #46

Comments

mcabbott commented Apr 21, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

KristofferC commented Apr 21, 2020

Uh oh!

KristofferC commented Apr 21, 2020

Uh oh!

mcabbott commented Apr 21, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

KristofferC commented Apr 21, 2020

Uh oh!

mcabbott commented Apr 21, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

KristofferC commented Apr 21, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

KristofferC commented Apr 21, 2020

Uh oh!

KristofferC commented Apr 21, 2020

Uh oh!

mcabbott commented Apr 21, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Crown421 commented Apr 21, 2020

Uh oh!

KristofferC commented Apr 24, 2020

Uh oh!

mcabbott commented Apr 24, 2020

Uh oh!

xuangu-fang commented Apr 14, 2021

Uh oh!

mcabbott commented Apr 21, 2020 •

edited

Loading

mcabbott commented Apr 21, 2020 •

edited

Loading

mcabbott commented Apr 21, 2020 •

edited

Loading

KristofferC commented Apr 21, 2020 •

edited

Loading

mcabbott commented Apr 21, 2020 •

edited

Loading