Skip to content

Add a stable public interface for accessing the data underlying a BenchmarkResult #9

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Jan 22, 2021
Merged

Conversation

DilumAluthge
Copy link
Member

Fixes #8

The idea here is that we add a stable public interface for getting the raw data inside a BenchmarkResult. Then users can just take that raw data and use whatever plotting package they want.

@DilumAluthge DilumAluthge marked this pull request as ready for review January 21, 2021 09:13
@DilumAluthge DilumAluthge marked this pull request as draft January 21, 2021 09:13
@chriselrod
Copy link
Collaborator

Do you want to get this in before registering?
Or maybe change the plotting library?

@DilumAluthge
Copy link
Member Author

Let's get this in first. I'll do one last test locally and then I'll mark this as ready for review.

I think the default plotting library is good, no need to change it.

@DilumAluthge
Copy link
Member Author

Just as an example of how to use this feature. I ran the following code:

julia> import BLASBenchmarksCPU

julia> import StatsPlots

julia> benchmark_result = BLASBenchmarksCPU.runbench(Float64; sizes = [1, 2, 5, 10, 20, 50, 100, 200, 500, 1_000])
Progress: 100%|███████████████████████████████████████████████████| Time: 0:06:46
  Size:      (1000, 1000, 1000)
  BLIS:      68.47
  Gaius:     44.62
  MKL:       69.59
  Octavian:  61.52
  OpenBLAS:  66.8
  Tullio:    51.29
Bennchmark Result of Matrix{Float64}, threaded = true
60×4 DataFrame
 Row │ Size   Library   GFLOPS        Time
     │ Int64  Symbol    Float64       Float64
─────┼────────────────────────────────────────────
   11  BLIS       0.000117592  1.7008e-5
                                 
  601000  Tullio    51.2924       0.0389921

julia> df = BLASBenchmarksCPU.benchmark_result_df(benchmark_result)
60×4 DataFrame
 Row │ Size   Library   GFLOPS        Time
     │ Int64  Symbol    Float64       Float64
─────┼────────────────────────────────────────────
   11  BLIS       0.000117592  1.7008e-5
                                 
  601000  Tullio    51.2924       0.0389921

julia> df[!, :Size] = Float64.(df[!, :Size])
60-element Vector{Float64}:
    1.0
    
 1000.0

julia> df[!, :GFLOPS] = Float64.(df[!, :GFLOPS])
60-element Vector{Float64}:
  0.00011759172154280339
  
 51.29240051024655

julia> df[!, :Time] = Float64.(df[!, :Time])
60-element Vector{Float64}:
 1.7008e-5
 
 0.038992131

julia> StatsPlots.@df df StatsPlots.plot(
           :Size,
           :GFLOPS;
           group = :Library,
           legend = :bottomright,
       )

And I received as output this plot, which was generated by StatsPlots.jl:

plot

@DilumAluthge DilumAluthge marked this pull request as ready for review January 22, 2021 19:01
@DilumAluthge
Copy link
Member Author

This is ready to merge.

@chriselrod
Copy link
Collaborator

Looks like performance could be a lot better.

@chriselrod chriselrod merged commit 8c67f89 into JuliaLinearAlgebra:master Jan 22, 2021
@DilumAluthge DilumAluthge deleted the dpa/public-functions branch January 22, 2021 19:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Customize the colors used in the plot?
2 participants