Skip to content

Adding MKL_jll artifact #39

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 10 commits into from
Mar 7, 2020
Merged

Adding MKL_jll artifact #39

merged 10 commits into from
Mar 7, 2020

Conversation

Crown421
Copy link
Collaborator

This pull request add dependence on the MKL_jll artifact, and changes all references from the libmkl_vml_avx libraries to the libmkl_rt library.
This has the advantage that MKL chooses the correct instruction set on runtime, making this package and its precompile cache hardware independent.
(Thanks @KristofferC !)

Paradiso.jl currently checks for the existence of a local install of MKL and uses it if one is found. However, given that MKL_jll is now an explicit dependency, it will installed regardless. Given this, it might as well be used.
This will be revisited if it becomes useful to enable user choice for a specific version of MKL via a manual external install.

@Crown421
Copy link
Collaborator Author

This currently fails only on Mac with Julia 1.3, but not with nightly. According to the log IVM actually returns different results.
Will try tomorrow or so on my Mac to investigate.

@KristofferC
Copy link
Member

Paradiso.jl currently checks for the existence of a local install of MKL and uses it if one is found.
This will be revisited if it becomes useful to enable user choice for a specific version of MKL via a manual external install.

In Pardiso.jl I thought about removing that check and just tell people to use Override.toml instead (https://julialang.github.io/Pkg.jl/v1/artifacts/#Overriding-artifact-locations-1).

@Crown421
Copy link
Collaborator Author

Reading about this I think I agree on the use of the Override.toml.

@KristofferC
Copy link
Member

KristofferC commented Feb 29, 2020

This currently fails only on Mac with Julia 1.3, but not with nightly. According to the log IVM actually returns different results.

Yeah, something is weird, the result changes a lot based on the length of the input vector:

julia> IntelVectorMath.gamma!(copy(INPUT[1:200]))[1:5]
5-element Array{Float64,1}:
     1.5191049997264277e21
     1.3775142112900882e11
     1.1814039832680319
     1.0624313147244564
 15581.002689052646

julia> IntelVectorMath.gamma!(copy(INPUT[1:800]))[1:5]
5-element Array{Float64,1}:
 Inf
 Inf
  0.9233209669728436
  0.9676118954476614
 Inf

I don't understand why it passes on nightly...

@KristofferC
Copy link
Member

See JuliaPackaging/BinaryBuilder.jl#700 for some more info about the test failure.

@Crown421
Copy link
Collaborator Author

Crown421 commented Mar 1, 2020

I have added a warning the readme, and an __init__ function, that warns if it detects that SpecialFunctions is already loaded.

@@ -8,6 +8,12 @@ const IVM = IntelVectorMath
# import Base: .^, ./
include("setup.jl")

function __init__()
if Sys.isapple() && :SpecialFunctions in names(Main; imported = true)
@warn "It appears SpecialFunctions was loaded prior to this package, which currently on mac may lead to wrong results. For further details see github.com/JuliaMath/IntelVectorMath.jl"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This will not catch the case when someone gets SpecialFunctions loaded as a dependency, because then the name will not be available in Main.

To do it "properly", you need to look at Base.loaded_modules

So it would be something like

special_functions_uuid = Base.UUID("276daf66-3868-5448-9aa4-cd146d93841b")
if Sys.isapple() && haskey(Base.loaded_modules, Base.PkgId(special_functions_uuid, "SpecialFunctions"))
    ...
end

And in fact, the problem (as shown in JuliaPackaging/BinaryBuilder.jl#700) is not really SpecialFunctions but CompilerSupportLibraries_jll so maybe that is the package that should be looked if it is loaded.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would you in general say this is reasonable thing to do?

src/setup.jl Outdated
@@ -97,14 +87,14 @@ function def_binary_op(tin, tout, jlname, jlname!, mklname, broadcast)
$(isempty(exports) ? nothing : Expr(:export, exports...))
function ($jlname!)(out::Array{$tout,N}, A::Array{$tin,N}, B::Array{$tin,N}) where {N}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The N can be removed here.

src/setup.jl Outdated
@@ -97,14 +87,14 @@ function def_binary_op(tin, tout, jlname, jlname!, mklname, broadcast)
$(isempty(exports) ? nothing : Expr(:export, exports...))
function ($jlname!)(out::Array{$tout,N}, A::Array{$tin,N}, B::Array{$tin,N}) where {N}
size(out) == size(A) == size(B) || $(broadcast ? :(return broadcast!($jlname, out, A, B)) : :(throw(DimensionMismatch())))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is this broadcast business about? That seems odd to me, it doesn't even call VML? Just remove?

Copy link
Collaborator Author

@Crown421 Crown421 Mar 2, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

More legacy code, maybe something that was a thing before Julia 0.7? I will have a look

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It appears to have been some construct to maybe allow something like IVM.atan(rand(10,10), rand(100,1))
But guessing at the users intention here seems wrong, so instead now properly throwing a dimension mismatch error.

Copy link
Member

@aminya aminya left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tested on Windows 10, Julia 1.5.0-DEV74 with being already MKL.jl installed and works as expected.

@Crown421 Crown421 merged commit dd2732d into master Mar 7, 2020
@Crown421 Crown421 deleted the artifact branch March 7, 2020 13:47
@KristofferC
Copy link
Member

Nice :)

@aminya
Copy link
Member

aminya commented Mar 8, 2020

Thank you for the work!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants