Skip to content

Allows to trigger on labelling and clean-up the process #400

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 16 commits into from
Feb 10, 2022
Merged

Allows to trigger on labelling and clean-up the process #400

merged 16 commits into from
Feb 10, 2022

Conversation

theogf
Copy link
Member

@theogf theogf commented Nov 4, 2021 •
edited
Loading

Allows to trigger benchmarking on labelling, will cancel previous running jobs and will always reuse the same comment to upload the benchmarks

The in place workflow is from tkf/BenchmarkCI.jl#91
Note that the proposed method in the posted issue is not relevant anymore and the workflow used corresponds to what is currently in master for JET.jl

@github-actions
Copy link
Contributor

github-actions bot commented Nov 4, 2021

Benchmark result

Judge result

Benchmark Report for /home/runner/work/KernelFunctions.jl/KernelFunctions.jl

Job Properties

  • Time of benchmarks:
    • Target: 4 Nov 2021 - 14:04
    • Baseline: 4 Nov 2021 - 14:05
  • Package commits:
    • Target: a4d289
    • Baseline: 33d64d
  • Julia commits:
    • Target: ae8452
    • Baseline: ae8452
  • Julia command flags:
    • Target: None
    • Baseline: None
  • Environment variables:
    • Target: None
    • Baseline: None

Results

A ratio greater than 1.0 denotes a possible regression (marked with ❌), while a ratio less
than 1.0 denotes a possible improvement (marked with ✅). Only significant results - results
that indicate possible regressions or improvements - are shown below (thus, an empty table means that all
benchmark results remained invariant between builds).

ID time ratio memory ratio
["Exponential", "Vecs", "kernelmatrix_diagX"] 1.07 (5%) ❌ 1.00 (1%)
["SqExponential", "ColVecs", "kernelmatrix_diagX"] 1.08 (5%) ❌ 1.00 (1%)
["SqExponential", "RowVecs", "kernelmatrix_diagX"] 1.06 (5%) ❌ 1.00 (1%)
["SqExponential", "Vecs", "kernelmatrixXY"] 0.94 (5%) ✅ 1.00 (1%)
["SqExponential", "Vecs", "kernelmatrix_diagX"] 1.07 (5%) ❌ 1.00 (1%)

Benchmark Group List

Here's a list of all the benchmark groups executed by this job:

  • ["Exponential", "ColVecs"]
  • ["Exponential", "RowVecs"]
  • ["Exponential", "Vecs"]
  • ["SqExponential", "ColVecs"]
  • ["SqExponential", "RowVecs"]
  • ["SqExponential", "Vecs"]

Julia versioninfo

Target

Julia Version 1.6.3
Commit ae8452a9e0 (2021-09-23 17:34 UTC)
Platform Info:
  OS: Linux (x86_64-pc-linux-gnu)
      Ubuntu 20.04.3 LTS
  uname: Linux 5.11.0-1020-azure #21~20.04.1-Ubuntu SMP Mon Oct 11 18:54:28 UTC 2021 x86_64 x86_64
  CPU: Intel(R) Xeon(R) Platinum 8272CL CPU @ 2.60GHz: 
              speed         user         nice          sys         idle          irq
       #1  2593 MHz       1516 s          2 s        126 s       1078 s          0 s
       #2  2593 MHz        401 s          1 s        126 s       2215 s          0 s
       
  Memory: 6.788990020751953 GB (3416.5234375 MB free)
  Uptime: 277.0 sec
  Load Avg:  1.1  0.67  0.29
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-11.0.1 (ORCJIT, skylake-avx512)

Baseline

Julia Version 1.6.3
Commit ae8452a9e0 (2021-09-23 17:34 UTC)
Platform Info:
  OS: Linux (x86_64-pc-linux-gnu)
      Ubuntu 20.04.3 LTS
  uname: Linux 5.11.0-1020-azure #21~20.04.1-Ubuntu SMP Mon Oct 11 18:54:28 UTC 2021 x86_64 x86_64
  CPU: Intel(R) Xeon(R) Platinum 8272CL CPU @ 2.60GHz: 
              speed         user         nice          sys         idle          irq
       #1  2593 MHz       2114 s          2 s        136 s       1085 s          0 s
       #2  2593 MHz        411 s          1 s        128 s       2816 s          0 s
       
  Memory: 6.788990020751953 GB (3357.38671875 MB free)
  Uptime: 339.0 sec
  Load Avg:  1.03  0.73  0.34
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-11.0.1 (ORCJIT, skylake-avx512)

Target result

Benchmark Report for /home/runner/work/KernelFunctions.jl/KernelFunctions.jl

Job Properties

  • Time of benchmark: 4 Nov 2021 - 14:4
  • Package commit: a4d289
  • Julia commit: ae8452
  • Julia command flags: None
  • Environment variables: None

Results

Below is a table of this job's results, obtained by running the benchmarks.
The values listed in the ID column have the structure [parent_group, child_group, ..., key], and can be used to
index into the BaseBenchmarks suite to retrieve the corresponding benchmarks.
The percentages accompanying time and memory values in the below table are noise tolerances. The "true"
time/memory value for a given benchmark is expected to fall within this percentage of the reported value.
An empty cell means that the value was zero.

ID time GC time memory allocations
["Exponential", "ColVecs", "kernelmatrixX"] 5.550 μs (5%) 6.73 KiB (1%) 3
["Exponential", "ColVecs", "kernelmatrixXY"] 5.650 μs (5%) 6.97 KiB (1%) 4
["Exponential", "ColVecs", "kernelmatrix_diagX"] 219.382 ns (5%) 480 bytes (1%) 2
["Exponential", "ColVecs", "kernelmatrix_diagXY"] 386.212 ns (5%) 480 bytes (1%) 2
["Exponential", "RowVecs", "kernelmatrixX"] 1.933 μs (5%) 3.67 KiB (1%) 4
["Exponential", "RowVecs", "kernelmatrixXY"] 2.244 μs (5%) 5.59 KiB (1%) 6
["Exponential", "RowVecs", "kernelmatrix_diagX"] 141.580 ns (5%) 320 bytes (1%) 2
["Exponential", "RowVecs", "kernelmatrix_diagXY"] 328.879 ns (5%) 320 bytes (1%) 2
["Exponential", "Vecs", "kernelmatrixX"] 7.725 μs (5%) 6.59 KiB (1%) 4
["Exponential", "Vecs", "kernelmatrixXY"] 7.967 μs (5%) 6.59 KiB (1%) 4
["Exponential", "Vecs", "kernelmatrix_diagX"] 220.753 ns (5%) 480 bytes (1%) 2
["Exponential", "Vecs", "kernelmatrix_diagXY"] 535.644 ns (5%) 480 bytes (1%) 2
["SqExponential", "ColVecs", "kernelmatrixX"] 5.700 μs (5%) 6.73 KiB (1%) 3
["SqExponential", "ColVecs", "kernelmatrixXY"] 5.267 μs (5%) 6.97 KiB (1%) 4
["SqExponential", "ColVecs", "kernelmatrix_diagX"] 212.138 ns (5%) 480 bytes (1%) 2
["SqExponential", "ColVecs", "kernelmatrix_diagXY"] 360.770 ns (5%) 480 bytes (1%) 2
["SqExponential", "RowVecs", "kernelmatrixX"] 1.978 μs (5%) 3.67 KiB (1%) 4
["SqExponential", "RowVecs", "kernelmatrixXY"] 2.178 μs (5%) 5.59 KiB (1%) 6
["SqExponential", "RowVecs", "kernelmatrix_diagX"] 143.568 ns (5%) 320 bytes (1%) 2
["SqExponential", "RowVecs", "kernelmatrix_diagXY"] 292.255 ns (5%) 320 bytes (1%) 2
["SqExponential", "Vecs", "kernelmatrixX"] 7.450 μs (5%) 6.59 KiB (1%) 4
["SqExponential", "Vecs", "kernelmatrixXY"] 7.300 μs (5%) 6.59 KiB (1%) 4
["SqExponential", "Vecs", "kernelmatrix_diagX"] 211.968 ns (5%) 480 bytes (1%) 2
["SqExponential", "Vecs", "kernelmatrix_diagXY"] 497.415 ns (5%) 480 bytes (1%) 2

Benchmark Group List

Here's a list of all the benchmark groups executed by this job:

  • ["Exponential", "ColVecs"]
  • ["Exponential", "RowVecs"]
  • ["Exponential", "Vecs"]
  • ["SqExponential", "ColVecs"]
  • ["SqExponential", "RowVecs"]
  • ["SqExponential", "Vecs"]

Julia versioninfo

Julia Version 1.6.3
Commit ae8452a9e0 (2021-09-23 17:34 UTC)
Platform Info:
  OS: Linux (x86_64-pc-linux-gnu)
      Ubuntu 20.04.3 LTS
  uname: Linux 5.11.0-1020-azure #21~20.04.1-Ubuntu SMP Mon Oct 11 18:54:28 UTC 2021 x86_64 x86_64
  CPU: Intel(R) Xeon(R) Platinum 8272CL CPU @ 2.60GHz: 
              speed         user         nice          sys         idle          irq
       #1  2593 MHz       1516 s          2 s        126 s       1078 s          0 s
       #2  2593 MHz        401 s          1 s        126 s       2215 s          0 s
       
  Memory: 6.788990020751953 GB (3416.5234375 MB free)
  Uptime: 277.0 sec
  Load Avg:  1.1  0.67  0.29
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-11.0.1 (ORCJIT, skylake-avx512)

Baseline result

Benchmark Report for /home/runner/work/KernelFunctions.jl/KernelFunctions.jl

Job Properties

  • Time of benchmark: 4 Nov 2021 - 14:5
  • Package commit: 33d64d
  • Julia commit: ae8452
  • Julia command flags: None
  • Environment variables: None

Results

Below is a table of this job's results, obtained by running the benchmarks.
The values listed in the ID column have the structure [parent_group, child_group, ..., key], and can be used to
index into the BaseBenchmarks suite to retrieve the corresponding benchmarks.
The percentages accompanying time and memory values in the below table are noise tolerances. The "true"
time/memory value for a given benchmark is expected to fall within this percentage of the reported value.
An empty cell means that the value was zero.

ID time GC time memory allocations
["Exponential", "ColVecs", "kernelmatrixX"] 5.417 μs (5%) 6.73 KiB (1%) 3
["Exponential", "ColVecs", "kernelmatrixXY"] 5.450 μs (5%) 6.97 KiB (1%) 4
["Exponential", "ColVecs", "kernelmatrix_diagX"] 210.661 ns (5%) 480 bytes (1%) 2
["Exponential", "ColVecs", "kernelmatrix_diagXY"] 375.369 ns (5%) 480 bytes (1%) 2
["Exponential", "RowVecs", "kernelmatrixX"] 1.911 μs (5%) 3.67 KiB (1%) 4
["Exponential", "RowVecs", "kernelmatrixXY"] 2.222 μs (5%) 5.59 KiB (1%) 6
["Exponential", "RowVecs", "kernelmatrix_diagX"] 139.106 ns (5%) 320 bytes (1%) 2
["Exponential", "RowVecs", "kernelmatrix_diagXY"] 322.414 ns (5%) 320 bytes (1%) 2
["Exponential", "Vecs", "kernelmatrixX"] 7.650 μs (5%) 6.59 KiB (1%) 4
["Exponential", "Vecs", "kernelmatrixXY"] 7.867 μs (5%) 6.59 KiB (1%) 4
["Exponential", "Vecs", "kernelmatrix_diagX"] 207.117 ns (5%) 480 bytes (1%) 2
["Exponential", "Vecs", "kernelmatrix_diagXY"] 537.771 ns (5%) 480 bytes (1%) 2
["SqExponential", "ColVecs", "kernelmatrixX"] 5.433 μs (5%) 6.73 KiB (1%) 3
["SqExponential", "ColVecs", "kernelmatrixXY"] 5.167 μs (5%) 6.97 KiB (1%) 4
["SqExponential", "ColVecs", "kernelmatrix_diagX"] 196.070 ns (5%) 480 bytes (1%) 2
["SqExponential", "ColVecs", "kernelmatrix_diagXY"] 347.373 ns (5%) 480 bytes (1%) 2
["SqExponential", "RowVecs", "kernelmatrixX"] 1.944 μs (5%) 3.67 KiB (1%) 4
["SqExponential", "RowVecs", "kernelmatrixXY"] 2.144 μs (5%) 5.59 KiB (1%) 6
["SqExponential", "RowVecs", "kernelmatrix_diagX"] 134.991 ns (5%) 320 bytes (1%) 2
["SqExponential", "RowVecs", "kernelmatrix_diagXY"] 295.576 ns (5%) 320 bytes (1%) 2
["SqExponential", "Vecs", "kernelmatrixX"] 7.750 μs (5%) 6.59 KiB (1%) 4
["SqExponential", "Vecs", "kernelmatrixXY"] 7.775 μs (5%) 6.59 KiB (1%) 4
["SqExponential", "Vecs", "kernelmatrix_diagX"] 198.463 ns (5%) 480 bytes (1%) 2
["SqExponential", "Vecs", "kernelmatrix_diagXY"] 495.342 ns (5%) 480 bytes (1%) 2

Benchmark Group List

Here's a list of all the benchmark groups executed by this job:

  • ["Exponential", "ColVecs"]
  • ["Exponential", "RowVecs"]
  • ["Exponential", "Vecs"]
  • ["SqExponential", "ColVecs"]
  • ["SqExponential", "RowVecs"]
  • ["SqExponential", "Vecs"]

Julia versioninfo

Julia Version 1.6.3
Commit ae8452a9e0 (2021-09-23 17:34 UTC)
Platform Info:
  OS: Linux (x86_64-pc-linux-gnu)
      Ubuntu 20.04.3 LTS
  uname: Linux 5.11.0-1020-azure #21~20.04.1-Ubuntu SMP Mon Oct 11 18:54:28 UTC 2021 x86_64 x86_64
  CPU: Intel(R) Xeon(R) Platinum 8272CL CPU @ 2.60GHz: 
              speed         user         nice          sys         idle          irq
       #1  2593 MHz       2114 s          2 s        136 s       1085 s          0 s
       #2  2593 MHz        411 s          1 s        128 s       2816 s          0 s
       
  Memory: 6.788990020751953 GB (3357.38671875 MB free)
  Uptime: 339.0 sec
  Load Avg:  1.03  0.73  0.34
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-11.0.1 (ORCJIT, skylake-avx512)

Runtime information

Runtime Info
BLAS #threads 2
BLAS.vendor() openblas64
Sys.CPU_THREADS 2

lscpu output:

Architecture:                    x86_64
CPU op-mode(s):                  32-bit, 64-bit
Byte Order:                      Little Endian
Address sizes:                   46 bits physical, 48 bits virtual
CPU(s):                          2
On-line CPU(s) list:             0,1
Thread(s) per core:              1
Core(s) per socket:              2
Socket(s):                       1
NUMA node(s):                    1
Vendor ID:                       GenuineIntel
CPU family:                      6
Model:                           85
Model name:                      Intel(R) Xeon(R) Platinum 8272CL CPU @ 2.60GHz
Stepping:                        7
CPU MHz:                         2593.906
BogoMIPS:                        5187.81
Hypervisor vendor:               Microsoft
Virtualization type:             full
L1d cache:                       64 KiB
L1i cache:                       64 KiB
L2 cache:                        2 MiB
L3 cache:                        35.8 MiB
NUMA node0 CPU(s):               0,1
Vulnerability Itlb multihit:     KVM: Mitigation: VMX unsupported
Vulnerability L1tf:              Mitigation; PTE Inversion
Vulnerability Mds:               Mitigation; Clear CPU buffers; SMT Host state unknown
Vulnerability Meltdown:          Mitigation; PTI
Vulnerability Spec store bypass: Vulnerable
Vulnerability Spectre v1:        Mitigation; usercopy/swapgs barriers and __user pointer sanitization
Vulnerability Spectre v2:        Mitigation; Full generic retpoline, STIBP disabled, RSB filling
Vulnerability Srbds:             Not affected
Vulnerability Tsx async abort:   Mitigation; Clear CPU buffers; SMT Host state unknown
Flags:                           fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ss ht syscall nx pdpe1gb rdtscp lm constant_tsc rep_good nopl xtopology cpuid pni pclmulqdq ssse3 fma cx16 pcid sse4_1 sse4_2 movbe popcnt aes xsave avx f16c rdrand hypervisor lahf_lm abm 3dnowprefetch invpcid_single pti fsgsbase bmi1 hle avx2 smep bmi2 erms invpcid rtm mpx avx512f avx512dq rdseed adx smap clflushopt avx512cd avx512bw avx512vl xsaveopt xsavec xsaves md_clear
Cpu Property Value
Brand Intel(R) Xeon(R) Platinum 8272CL CPU @ 2.60GHz
Vendor :Intel
Architecture :Skylake
Model Family: 0x06, Model: 0x55, Stepping: 0x07, Type: 0x00
Cores 2 physical cores, 2 logical cores (on executing CPU)
No Hyperthreading hardware capability detected
Clock Frequencies Not supported by CPU
Data Cache Level 1:3 : (32, 1024, 36608) kbytes
64 byte cache line size
Address Size 48 bits virtual, 46 bits physical
SIMD 512 bit = 64 byte max. SIMD vector size
Time Stamp Counter TSC is accessible via rdtsc
TSC increased at every clock cycle (non-invariant TSC)
Perf. Monitoring Performance Monitoring Counters (PMC) are not supported
Hypervisor Yes, Microsoft

@codecov
Copy link

codecov bot commented Nov 4, 2021 •
edited
Loading

Codecov Report

Merging #400 (6ee340e) into master (1a3fa33) will not change coverage.
The diff coverage is n/a.

Impacted file tree graph

@@           Coverage Diff           @@
##           master     #400   +/-   ##
=======================================
  Coverage   92.99%   92.99%           
=======================================
  Files          52       52           
  Lines        1214     1214           
=======================================
  Hits         1129     1129           
  Misses         85       85

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 1a3fa33...6ee340e. Read the comment docs.

@theogf
Copy link
Member Author

theogf commented Nov 4, 2021

[benchmark]

@theogf
Copy link
Member Author

theogf commented Nov 4, 2021

[benchmark]

@theogf theogf added the performance critical Triggers benchmarking CI label Nov 4, 2021
@github-actions
Copy link
Contributor

github-actions bot commented Nov 4, 2021

Benchmark result

Judge result

Benchmark Report for /home/runner/work/KernelFunctions.jl/KernelFunctions.jl

Job Properties

  • Time of benchmarks:
    • Target: 4 Nov 2021 - 14:18
    • Baseline: 4 Nov 2021 - 14:20
  • Package commits:
    • Target: df6104
    • Baseline: 33d64d
  • Julia commits:
    • Target: ae8452
    • Baseline: ae8452
  • Julia command flags:
    • Target: None
    • Baseline: None
  • Environment variables:
    • Target: None
    • Baseline: None

Results

A ratio greater than 1.0 denotes a possible regression (marked with ❌), while a ratio less
than 1.0 denotes a possible improvement (marked with ✅). Only significant results - results
that indicate possible regressions or improvements - are shown below (thus, an empty table means that all
benchmark results remained invariant between builds).

ID time ratio memory ratio
["Exponential", "ColVecs", "kernelmatrixXY"] 0.89 (5%) ✅ 1.00 (1%)
["Exponential", "ColVecs", "kernelmatrix_diagX"] 0.86 (5%) ✅ 1.00 (1%)
["Exponential", "ColVecs", "kernelmatrix_diagXY"] 0.93 (5%) ✅ 1.00 (1%)
["Exponential", "RowVecs", "kernelmatrixX"] 0.94 (5%) ✅ 1.00 (1%)
["Exponential", "RowVecs", "kernelmatrixXY"] 0.94 (5%) ✅ 1.00 (1%)
["Exponential", "Vecs", "kernelmatrix_diagX"] 0.85 (5%) ✅ 1.00 (1%)
["Exponential", "Vecs", "kernelmatrix_diagXY"] 0.89 (5%) ✅ 1.00 (1%)
["SqExponential", "ColVecs", "kernelmatrixX"] 0.95 (5%) ✅ 1.00 (1%)
["SqExponential", "ColVecs", "kernelmatrixXY"] 1.06 (5%) ❌ 1.00 (1%)
["SqExponential", "RowVecs", "kernelmatrixX"] 0.90 (5%) ✅ 1.00 (1%)
["SqExponential", "RowVecs", "kernelmatrixXY"] 0.95 (5%) ✅ 1.00 (1%)
["SqExponential", "Vecs", "kernelmatrixX"] 0.88 (5%) ✅ 1.00 (1%)
["SqExponential", "Vecs", "kernelmatrix_diagX"] 0.84 (5%) ✅ 1.00 (1%)
["SqExponential", "Vecs", "kernelmatrix_diagXY"] 0.91 (5%) ✅ 1.00 (1%)

Benchmark Group List

Here's a list of all the benchmark groups executed by this job:

  • ["Exponential", "ColVecs"]
  • ["Exponential", "RowVecs"]
  • ["Exponential", "Vecs"]
  • ["SqExponential", "ColVecs"]
  • ["SqExponential", "RowVecs"]
  • ["SqExponential", "Vecs"]

Julia versioninfo

Target

Julia Version 1.6.3
Commit ae8452a9e0 (2021-09-23 17:34 UTC)
Platform Info:
  OS: Linux (x86_64-pc-linux-gnu)
      Ubuntu 20.04.3 LTS
  uname: Linux 5.11.0-1020-azure #21~20.04.1-Ubuntu SMP Mon Oct 11 18:54:28 UTC 2021 x86_64 x86_64
  CPU: Intel(R) Xeon(R) CPU E5-2673 v4 @ 2.30GHz: 
              speed         user         nice          sys         idle          irq
       #1  2294 MHz       1866 s          2 s        167 s        631 s          0 s
       #2  2294 MHz        581 s          1 s        162 s       1950 s          0 s
       
  Memory: 6.788990020751953 GB (3397.98828125 MB free)
  Uptime: 275.0 sec
  Load Avg:  1.18  0.86  0.38
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-11.0.1 (ORCJIT, broadwell)

Baseline

Julia Version 1.6.3
Commit ae8452a9e0 (2021-09-23 17:34 UTC)
Platform Info:
  OS: Linux (x86_64-pc-linux-gnu)
      Ubuntu 20.04.3 LTS
  uname: Linux 5.11.0-1020-azure #21~20.04.1-Ubuntu SMP Mon Oct 11 18:54:28 UTC 2021 x86_64 x86_64
  CPU: Intel(R) Xeon(R) CPU E5-2673 v4 @ 2.30GHz: 
              speed         user         nice          sys         idle          irq
       #1  2294 MHz       2362 s          2 s        176 s        941 s          0 s
       #2  2294 MHz        893 s          1 s        168 s       2447 s          0 s
       
  Memory: 6.788990020751953 GB (3329.55078125 MB free)
  Uptime: 357.0 sec
  Load Avg:  1.07  0.91  0.45
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-11.0.1 (ORCJIT, broadwell)

Target result

Benchmark Report for /home/runner/work/KernelFunctions.jl/KernelFunctions.jl

Job Properties

  • Time of benchmark: 4 Nov 2021 - 14:18
  • Package commit: df6104
  • Julia commit: ae8452
  • Julia command flags: None
  • Environment variables: None

Results

Below is a table of this job's results, obtained by running the benchmarks.
The values listed in the ID column have the structure [parent_group, child_group, ..., key], and can be used to
index into the BaseBenchmarks suite to retrieve the corresponding benchmarks.
The percentages accompanying time and memory values in the below table are noise tolerances. The "true"
time/memory value for a given benchmark is expected to fall within this percentage of the reported value.
An empty cell means that the value was zero.

ID time GC time memory allocations
["Exponential", "ColVecs", "kernelmatrixX"] 6.625 μs (5%) 6.73 KiB (1%) 3
["Exponential", "ColVecs", "kernelmatrixXY"] 6.425 μs (5%) 6.97 KiB (1%) 4
["Exponential", "ColVecs", "kernelmatrix_diagX"] 178.500 ns (5%) 480 bytes (1%) 2
["Exponential", "ColVecs", "kernelmatrix_diagXY"] 351.174 ns (5%) 480 bytes (1%) 2
["Exponential", "RowVecs", "kernelmatrixX"] 2.289 μs (5%) 3.67 KiB (1%) 4
["Exponential", "RowVecs", "kernelmatrixXY"] 2.678 μs (5%) 5.59 KiB (1%) 6
["Exponential", "RowVecs", "kernelmatrix_diagX"] 119.317 ns (5%) 320 bytes (1%) 2
["Exponential", "RowVecs", "kernelmatrix_diagXY"] 315.657 ns (5%) 320 bytes (1%) 2
["Exponential", "Vecs", "kernelmatrixX"] 7.625 μs (5%) 6.59 KiB (1%) 4
["Exponential", "Vecs", "kernelmatrixXY"] 7.933 μs (5%) 6.59 KiB (1%) 4
["Exponential", "Vecs", "kernelmatrix_diagX"] 179.724 ns (5%) 480 bytes (1%) 2
["Exponential", "Vecs", "kernelmatrix_diagXY"] 504.145 ns (5%) 480 bytes (1%) 2
["SqExponential", "ColVecs", "kernelmatrixX"] 6.780 μs (5%) 6.73 KiB (1%) 3
["SqExponential", "ColVecs", "kernelmatrixXY"] 6.625 μs (5%) 6.97 KiB (1%) 4
["SqExponential", "ColVecs", "kernelmatrix_diagX"] 188.167 ns (5%) 480 bytes (1%) 2
["SqExponential", "ColVecs", "kernelmatrix_diagXY"] 324.449 ns (5%) 480 bytes (1%) 2
["SqExponential", "RowVecs", "kernelmatrixX"] 2.311 μs (5%) 3.67 KiB (1%) 4
["SqExponential", "RowVecs", "kernelmatrixXY"] 2.667 μs (5%) 5.59 KiB (1%) 6
["SqExponential", "RowVecs", "kernelmatrix_diagX"] 130.012 ns (5%) 320 bytes (1%) 2
["SqExponential", "RowVecs", "kernelmatrix_diagXY"] 282.807 ns (5%) 320 bytes (1%) 2
["SqExponential", "Vecs", "kernelmatrixX"] 7.075 μs (5%) 6.59 KiB (1%) 4
["SqExponential", "Vecs", "kernelmatrixXY"] 7.075 μs (5%) 6.59 KiB (1%) 4
["SqExponential", "Vecs", "kernelmatrix_diagX"] 189.859 ns (5%) 480 bytes (1%) 2
["SqExponential", "Vecs", "kernelmatrix_diagXY"] 457.436 ns (5%) 480 bytes (1%) 2

Benchmark Group List

Here's a list of all the benchmark groups executed by this job:

  • ["Exponential", "ColVecs"]
  • ["Exponential", "RowVecs"]
  • ["Exponential", "Vecs"]
  • ["SqExponential", "ColVecs"]
  • ["SqExponential", "RowVecs"]
  • ["SqExponential", "Vecs"]

Julia versioninfo

Julia Version 1.6.3
Commit ae8452a9e0 (2021-09-23 17:34 UTC)
Platform Info:
  OS: Linux (x86_64-pc-linux-gnu)
      Ubuntu 20.04.3 LTS
  uname: Linux 5.11.0-1020-azure #21~20.04.1-Ubuntu SMP Mon Oct 11 18:54:28 UTC 2021 x86_64 x86_64
  CPU: Intel(R) Xeon(R) CPU E5-2673 v4 @ 2.30GHz: 
              speed         user         nice          sys         idle          irq
       #1  2294 MHz       1866 s          2 s        167 s        631 s          0 s
       #2  2294 MHz        581 s          1 s        162 s       1950 s          0 s
       
  Memory: 6.788990020751953 GB (3397.98828125 MB free)
  Uptime: 275.0 sec
  Load Avg:  1.18  0.86  0.38
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-11.0.1 (ORCJIT, broadwell)

Baseline result

Benchmark Report for /home/runner/work/KernelFunctions.jl/KernelFunctions.jl

Job Properties

  • Time of benchmark: 4 Nov 2021 - 14:20
  • Package commit: 33d64d
  • Julia commit: ae8452
  • Julia command flags: None
  • Environment variables: None

Results

Below is a table of this job's results, obtained by running the benchmarks.
The values listed in the ID column have the structure [parent_group, child_group, ..., key], and can be used to
index into the BaseBenchmarks suite to retrieve the corresponding benchmarks.
The percentages accompanying time and memory values in the below table are noise tolerances. The "true"
time/memory value for a given benchmark is expected to fall within this percentage of the reported value.
An empty cell means that the value was zero.

ID time GC time memory allocations
["Exponential", "ColVecs", "kernelmatrixX"] 6.800 μs (5%) 6.73 KiB (1%) 3
["Exponential", "ColVecs", "kernelmatrixXY"] 7.250 μs (5%) 6.97 KiB (1%) 4
["Exponential", "ColVecs", "kernelmatrix_diagX"] 208.333 ns (5%) 480 bytes (1%) 2
["Exponential", "ColVecs", "kernelmatrix_diagXY"] 376.526 ns (5%) 480 bytes (1%) 2
["Exponential", "RowVecs", "kernelmatrixX"] 2.433 μs (5%) 3.67 KiB (1%) 4
["Exponential", "RowVecs", "kernelmatrixXY"] 2.845 μs (5%) 5.59 KiB (1%) 6
["Exponential", "RowVecs", "kernelmatrix_diagX"] 124.736 ns (5%) 320 bytes (1%) 2
["Exponential", "RowVecs", "kernelmatrix_diagXY"] 307.830 ns (5%) 320 bytes (1%) 2
["Exponential", "Vecs", "kernelmatrixX"] 7.650 μs (5%) 6.59 KiB (1%) 4
["Exponential", "Vecs", "kernelmatrixXY"] 7.867 μs (5%) 6.59 KiB (1%) 4
["Exponential", "Vecs", "kernelmatrix_diagX"] 211.061 ns (5%) 480 bytes (1%) 2
["Exponential", "Vecs", "kernelmatrix_diagXY"] 569.435 ns (5%) 480 bytes (1%) 2
["SqExponential", "ColVecs", "kernelmatrixX"] 7.140 μs (5%) 6.73 KiB (1%) 3
["SqExponential", "ColVecs", "kernelmatrixXY"] 6.275 μs (5%) 6.97 KiB (1%) 4
["SqExponential", "ColVecs", "kernelmatrix_diagX"] 185.833 ns (5%) 480 bytes (1%) 2
["SqExponential", "ColVecs", "kernelmatrix_diagXY"] 314.671 ns (5%) 480 bytes (1%) 2
["SqExponential", "RowVecs", "kernelmatrixX"] 2.578 μs (5%) 3.67 KiB (1%) 4
["SqExponential", "RowVecs", "kernelmatrixXY"] 2.811 μs (5%) 5.59 KiB (1%) 6
["SqExponential", "RowVecs", "kernelmatrix_diagX"] 125.708 ns (5%) 320 bytes (1%) 2
["SqExponential", "RowVecs", "kernelmatrix_diagXY"] 288.421 ns (5%) 320 bytes (1%) 2
["SqExponential", "Vecs", "kernelmatrixX"] 8.050 μs (5%) 6.59 KiB (1%) 4
["SqExponential", "Vecs", "kernelmatrixXY"] 7.050 μs (5%) 6.59 KiB (1%) 4
["SqExponential", "Vecs", "kernelmatrix_diagX"] 226.001 ns (5%) 480 bytes (1%) 2
["SqExponential", "Vecs", "kernelmatrix_diagXY"] 503.590 ns (5%) 480 bytes (1%) 2

Benchmark Group List

Here's a list of all the benchmark groups executed by this job:

  • ["Exponential", "ColVecs"]
  • ["Exponential", "RowVecs"]
  • ["Exponential", "Vecs"]
  • ["SqExponential", "ColVecs"]
  • ["SqExponential", "RowVecs"]
  • ["SqExponential", "Vecs"]

Julia versioninfo

Julia Version 1.6.3
Commit ae8452a9e0 (2021-09-23 17:34 UTC)
Platform Info:
  OS: Linux (x86_64-pc-linux-gnu)
      Ubuntu 20.04.3 LTS
  uname: Linux 5.11.0-1020-azure #21~20.04.1-Ubuntu SMP Mon Oct 11 18:54:28 UTC 2021 x86_64 x86_64
  CPU: Intel(R) Xeon(R) CPU E5-2673 v4 @ 2.30GHz: 
              speed         user         nice          sys         idle          irq
       #1  2294 MHz       2362 s          2 s        176 s        941 s          0 s
       #2  2294 MHz        893 s          1 s        168 s       2447 s          0 s
       
  Memory: 6.788990020751953 GB (3329.55078125 MB free)
  Uptime: 357.0 sec
  Load Avg:  1.07  0.91  0.45
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-11.0.1 (ORCJIT, broadwell)

Runtime information

Runtime Info
BLAS #threads 2
BLAS.vendor() openblas64
Sys.CPU_THREADS 2

lscpu output:

Architecture:                    x86_64
CPU op-mode(s):                  32-bit, 64-bit
Byte Order:                      Little Endian
Address sizes:                   46 bits physical, 48 bits virtual
CPU(s):                          2
On-line CPU(s) list:             0,1
Thread(s) per core:              1
Core(s) per socket:              2
Socket(s):                       1
NUMA node(s):                    1
Vendor ID:                       GenuineIntel
CPU family:                      6
Model:                           79
Model name:                      Intel(R) Xeon(R) CPU E5-2673 v4 @ 2.30GHz
Stepping:                        1
CPU MHz:                         2294.683
BogoMIPS:                        4589.36
Hypervisor vendor:               Microsoft
Virtualization type:             full
L1d cache:                       64 KiB
L1i cache:                       64 KiB
L2 cache:                        512 KiB
L3 cache:                        50 MiB
NUMA node0 CPU(s):               0,1
Vulnerability Itlb multihit:     KVM: Mitigation: VMX unsupported
Vulnerability L1tf:              Mitigation; PTE Inversion
Vulnerability Mds:               Mitigation; Clear CPU buffers; SMT Host state unknown
Vulnerability Meltdown:          Mitigation; PTI
Vulnerability Spec store bypass: Vulnerable
Vulnerability Spectre v1:        Mitigation; usercopy/swapgs barriers and __user pointer sanitization
Vulnerability Spectre v2:        Mitigation; Full generic retpoline, STIBP disabled, RSB filling
Vulnerability Srbds:             Not affected
Vulnerability Tsx async abort:   Mitigation; Clear CPU buffers; SMT Host state unknown
Flags:                           fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ss ht syscall nx pdpe1gb rdtscp lm constant_tsc rep_good nopl xtopology cpuid pni pclmulqdq ssse3 fma cx16 pcid sse4_1 sse4_2 movbe popcnt aes xsave avx f16c rdrand hypervisor lahf_lm abm 3dnowprefetch invpcid_single pti fsgsbase bmi1 hle avx2 smep bmi2 erms invpcid rtm rdseed adx smap xsaveopt md_clear
Cpu Property Value
Brand Intel(R) Xeon(R) CPU E5-2673 v4 @ 2.30GHz
Vendor :Intel
Architecture :Broadwell
Model Family: 0x06, Model: 0x4f, Stepping: 0x01, Type: 0x00
Cores 2 physical cores, 2 logical cores (on executing CPU)
No Hyperthreading hardware capability detected
Clock Frequencies Not supported by CPU
Data Cache Level 1:3 : (32, 256, 51200) kbytes
64 byte cache line size
Address Size 48 bits virtual, 46 bits physical
SIMD 256 bit = 32 byte max. SIMD vector size
Time Stamp Counter TSC is accessible via rdtsc
TSC increased at every clock cycle (non-invariant TSC)
Perf. Monitoring Performance Monitoring Counters (PMC) are not supported
Hypervisor Yes, Microsoft

@theogf theogf removed the performance critical Triggers benchmarking CI label Nov 4, 2021
@theogf
Copy link
Member Author

theogf commented Nov 4, 2021

[benchmark]

@devmotion
Copy link
Member

when writing [benchmark] in a comment, once labeled the PR will also run at every change

So the main motivation for the PR is to be able to rerun the benchmarks automatically whenever a new commit is pushed? Couldn't this be achieved by checking the labels as well? I also wonder if it's good to rerun benchmarks automatically. I would have thought a natural workflow would be to only trigger it manually, eg before/when a PR is reviewed or merged. Basically similar to Nanosoldier in base.

@theogf
Copy link
Member Author

theogf commented Nov 4, 2021

This is what this PR is supposed to achieve, both continuous CI via the label, and ponctual trigger via comment. The coma before "once labeled" should be a full stop

@theogf
Copy link
Member Author

theogf commented Nov 7, 2021

[benchmark]

@theogf
Copy link
Member Author

theogf commented Feb 1, 2022

[benchmark]

@theogf theogf added the performance critical Triggers benchmarking CI label Feb 1, 2022
@theogf theogf changed the title Allows to trigger on comment and on labelling Allows to trigger on labelling and clean-up the process Feb 1, 2022
@theogf
Copy link
Member Author

theogf commented Feb 1, 2022

That's again one of these things that will only work once merged in master...

@theogf theogf requested a review from devmotion February 2, 2022 10:25
devmotion
devmotion reviewed Feb 2, 2022
View reviewed changes
Copy link
Member

@devmotion devmotion left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The general setup seems reasonable but of course it's difficult to review properly if we can't run the action. Maybe we just have to iterate and fix issues as they appear.

@theogf
Copy link
Member Author

theogf commented Feb 10, 2022

@devmotion I introduced your changes and I think this is ready to merge

devmotion
devmotion approved these changes Feb 10, 2022
View reviewed changes
Copy link
Member

@devmotion devmotion left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK, let's see if it works 🙂

@theogf theogf merged commit 09e76cb into master Feb 10, 2022
@theogf theogf deleted the tgf/benchmark_trigger branch February 10, 2022 22:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@devmotion devmotion devmotion approved these changes

Assignees
No one assigned
Labels
performance critical Triggers benchmarking CI
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@theogf @devmotion