Tensor product kernel

[devmotion]

Lately, I've been working quite a bit with tensor product kernels. I thought maybe they could be supported by KernelFunctions. However, there are still some open questions about the most reasonable approach for kernelmatrix and so on.

Each "observation" is a tuple whose first and second argument belong to different input spaces, and hence e.g. of type Tuple{Vector{Float64},Vector{Int}}. Should kernelmatrix then take tuples of multiple observations for each input space, i.e., of type Tuple{Matrix{Float64},Matrix{Int}}, or rather a vector of multiple observations, i.e., of type Vector{Tuple{Vector{Float64},Vector{Int}}}?

[devmotion]

I guess, Tuple{Vector{Float64},Vector{Int}} might be easier from a developer perspective since then one could easily separate observations in both input spaces and forward them accordingly to the corresponding kernels.

[theogf]

Yes I think the latter would allow to be more flexible for an even more general class of data.

[devmotion]

Yes, I agree. Actually, in my use case I need both a kernel on a space of distributions (the first input space) and a regular vector space (the second input space), so in contrast to the example above it's a bit unclear how to summarize everything in one vector in order to avoid having to deal with an explicit tensor product kernel.

[theogf]

By tensor product you mean that for N samples with D types of observations you would get a N^D x N^D kernel matrix?

[devmotion]

No, the name tensor product kernel has nothing to do with the kernel matrix but is just the name for a kernel that applies D different kernels to the D different types of observations, instead of applying one kernel on the product space of the D input spaces. To be consistent with the current implementation, IMO the kernel matrix would just be a matrix with the kernel evaluations on all pairs of observations. Hence if we have N samples (consisting of D input types each) it would be of size N x N, and if we have to set of N and M samples it would be of size N x M.

[theogf]

Why did we not do this before!!!!

[theogf]

I think we can merge already or would you like to add something?

[devmotion]

I think we can merge already or would you like to add something?

I guess it's usable but without a kernelmatrix or kerneldiagmatrix implementation yet since I wasn't sure which input types we want to support in that case.