Update docstrings of linear and polynomial kernel and fix their constraints

[devmotion]

This PR updates the docstrings of the linear and polynomial kernel, consistent with the recently updated documentation of PiecewisePolynomialKernel. Moreover, the default parameters are changed to 0 and 2 instead of 0.0 and 2.0 to avoid promotions e.g. of inputs of type Float32.

Two things:

• Maybe the parameter names should be more descriptive, e.g., offset and degree, similar to recent changes in PiecewisePolynomialKernel?
  - Maybe default parameters of type Int are problematic for parameter optimization? Since Functors.fmap reconstructs kernels for the updated parameter types and not necessarily of the same type as the original kernel, it might still be fine but I haven't checked it. Even if it does not work automatically, it might not be too bad if the parameters have to be specified explicitly in such a case. At least users would get an error message if it fails whereas currently promotion happens silently.

Edit: As discussed below, the constant offset is constrained to be non-negative and the degreee of the polynomial degree is constrained to be a natural number (one could also allow zero but it does not seem particularly useful, it would just be a ConstantKernel). I renamed d to degree and reverted the default values of c to floating point numbers.

[willtebbutt]

These changes broadly look good, but I'm also a little concerned about the default values not being differentiable. There's presently some debate around how to treat Integers with regards to AD (there are reasonable arguments both ways), so perhaps it's wise to stick with Float64 for now?

[devmotion]

Sure, it might be better to keep the default values for now. I only realized that there could potentially be a problem after I had changed it. It is a bit unfortunate since Float64 seems unnecessarily invasive here. Additionally, it is a bit sad since Julia yields optimized code for integer-valued exponents and in particular a default value of 2:

julia> f(x, d) = x^d

julia> @code_typed f(3.0, 2)
CodeInfo(
1 ── %1  = (d === -1)::Bool
└───       goto #3 if not %1
2 ── %3  = Base.div_float(1.0, x)::Float64
└───       goto #12
3 ── %5  = (d === 0)::Bool
└───       goto #5 if not %5
4 ──       goto #12
5 ── %8  = (d === 1)::Bool
└───       goto #7 if not %8
6 ──       goto #12
7 ── %11 = (d === 2)::Bool
└───       goto #9 if not %11
8 ── %13 = Base.mul_float(x, x)::Float64
└───       goto #12
9 ── %15 = (d === 3)::Bool
└───       goto #11 if not %15
10 ─ %17 = Base.mul_float(x, x)::Float64
│    %18 = Base.mul_float(%17, x)::Float64
└───       goto #12
11 ─ %20 = Base.sitofp(Float64, d)::Float64
│    %21 = $(Expr(:foreigncall, "llvm.pow.f64", Float64, svec(Float64, Float64), 0, :(:llvmcall), :(x), :(%20), :(%20), :(x)))::Float64
└───       goto #12
12 ┄ %23 = φ (#2 => %3, #4 => 1.0, #6 => _2, #8 => %13, #10 => %18, #11 => %21)::Float64
└───       return %23
) => Float64

julia> @code_typed f(3.0, 2.0)
CodeInfo(
1 ─ %1 = $(Expr(:foreigncall, "llvm.pow.f64", Float64, svec(Float64, Float64), 0, :(:llvmcall), :(x), :(d), :(d), :(x)))::Float64
│   %2 = Base.ne_float(%1, %1)::Bool
│   %3 = Base.add_float(x, d)::Float64
│   %4 = Base.ne_float(%3, %3)::Bool
│   %5 = Base.not_int(%4)::Bool
│   %6 = Base.and_int(%2, %5)::Bool
└──      goto #3 if not %6
2 ─      invoke Base.Math.throw_exp_domainerror(_2::Float64)::Union{}
└──      $(Expr(:unreachable))::Union{}
3 ┄      goto #4
4 ─      return %1
) => Float64

[devmotion]

Alternatively, maybe we could just tell users to ensure that all parameters are of the correct type before training (this would also be convenient if one wants to use e.g. Float32):

using Flux

k = .... # some kernel
k_float = fmap(float, k) # `fmap` is reexported by Flux
ps = Flux.params(k_float)
... # training loop

[devmotion]

I am convinced now that we should only allow polynomials of degree d::Int. Regardless of the choice of c, the basis x^\top x' + c will be negative for some inputs. However, exponentiation of negative numbers only yields real numbers for rational exponents and is only unambigously defined for integer-valued exponents (in the other cases in addition to the real solution some complex numbers solve the corresponding equation).Therefore degree d should always be an integer and should not be trainable.

[devmotion]

Moreover, it seems that at least for odd degrees d (and hence also for the linear kernel with d = 1) the kernel is only positive-definite if offset c is non-negative, since otherwise k(x, x) = (|x|^2 + c)^d < 0 for |x| < sqrt(-c).

[devmotion]

BTW the restriction c >= 0, and implicitly d in {0,1,2,...} ("degree-d polynomial"), are also mentioned on Uncyclopedia: https://en.wikipedia.org/wiki/Polynomial_kernel#Definition

[devmotion]

Similarly, we should ensure that constant c in the constant kernel is non-negative.

[willtebbutt]

LGTM