Add type parameter for CachingOptimizer

[blegat]

This PR adds a type parameter for the optimizer of CachingOptimizer. It adds this type parameters in the first place as it will be more useful than the second one (see e.g. dropoptimizer!). It also changes the order of the cache and the optimizer fields for consistency.
Note that it does not implement the idea of #321 (comment) of removing the possibility to have no optimizer or to change the optimizer.
Indeed, this would require JuMP to handle these cases and I initially thought that this was the solution because I didn't have the idea of simply using the non concrete type parameter Union{Nothing, MOI.AbstractOptimizer} in the case where we want to be able to drop the optimizer and to change it and use the concrete optimizer type when we don't want to change it and the type instability becomes the bottleneck (e.g. #321).

We can now make a similar change in JuMP. Have a parameter for the moibackend type that the user can set it to the concrete type of the optimizer to have no overhead (but loosing the features of changing optimizer, droping it, ...)

Benchmarks

See #323 for benchmarks before this change

With a CachingOptimizer with concrete field type for the optimizer field (i.e. MockOptimizer{Model{Float64}}):

v0.6
 36.591 ns (4 allocations: 112 bytes)
148.012 ns (8 allocations: 224 bytes)
 24.322 ns (3 allocations: 96 bytes)
 24.871 ns (3 allocations: 96 bytes)
v0.7
 12.130 ns (1 allocation: 48 bytes)
103.417 ns (3 allocations: 112 bytes)
 13.545 ns (1 allocation: 48 bytes)
 11.785 ns

With a CachingOptimizer with Union{Nothing, OptimizerType} with concrete OptimizerType for the `optimizer field:

v0.6
 40.083 ns (4 allocations: 112 bytes)
177.939 ns (11 allocations: 288 bytes)
 29.122 ns (3 allocations: 96 bytes)
 25.401 ns (3 allocations: 96 bytes)
v0.7
 13.049 ns (1 allocation: 48 bytes)
109.670 ns (3 allocations: 112 bytes)
 11.501 ns (1 allocation: 48 bytes)
 12.421 ns (1 allocation: 48 bytes)

The overhead of the union with Void is negligible in v0.7 as suggested by #390 (comment) so it seems cleaner to use it

Closes #321

[codecov-io]

Codecov Report

Merging #390 into master will increase coverage by 0.27%.
The diff coverage is 100%.

@@            Coverage Diff             @@
##           master     #390      +/-   ##
==========================================
+ Coverage   96.48%   96.75%   +0.27%     
==========================================
  Files          39       39              
  Lines        4917     5397     +480     
==========================================
+ Hits         4744     5222     +478     
- Misses        173      175       +2

Impacted Files	Coverage Δ
src/Utilities/cachingoptimizer.jl	87.74% <100%> (-0.58%)	⬇️
src/sets.jl	90% <0%> (-10%)	⬇️
src/functions.jl	100% <0%> (ø)	⬆️
src/Test/contconic.jl	100% <0%> (ø)	⬆️
src/Utilities/functions.jl	98.82% <0%> (+1.96%)	⬆️
src/Utilities/mockoptimizer.jl	99.48% <0%> (+2.01%)	⬆️

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update dc47126...3f4db45. Read the comment docs.

[mlubin]

Let's try more explicit names for the type parameters. Something like OptimizerType and ModelType maybe?

[mlubin]

Union{Nothing,MOI.AbstractOptimizer} is a really unusual type parameter. Why not type the optimizer as Union{Nothing, OptimizerType}? My understanding is that there's very little overhead in 0.7 for Union{Nothing, T} if T is concrete.

[blegat]

Indeed, I have updated the description which a benchmark showing this

[mlubin]

so it does not support the function dropoptimizer! nor the method `resetoptimizer!.

Is this true? Can't the field be set to nothing?

[blegat]

Indeed, fixed :)

[mlubin]

Do you plan to update this to optimizer::Union{OptimizerType, Nothing}?

[blegat]

Ups, fixed