Transfer NeuralKernelNetwork over from Stheno

[willtebbutt]

To be honest, this implementation probably needs some love. I've transferred it over as-is (more or less) so that it's here, but we should open an issue about it upon merging.

[theogf]

If I understand correctly, you map your samples with a series of kernel (the primitives) and then pass this to a NN with the guarantee of a positive output? I think that the docs for the kernel could be made more explicit in this sense, maybe in a series of equations?
And I see that the original name is indeed NeuralKernelNetwork so let's just keep this one :)

[devmotion]

Isn't LinearLayer just a LinearTransform with a special parameterization? And couldn't Primitives be replaced by just a tuple or vector of kernels, similar to what we use in KernelSum, KernelProduct, and KernelTensorProduct? But maybe these are things you thought should be addressed in subsequent PRs?

[willtebbutt]

But maybe these are things you thought should be addressed in subsequent PRs?

Exactly -- in this PR I'm just copying over the implementation (which was contributed to Stheno a while ago, prior to moving over to KernelFunctions etc) so that we have an implementation of this thing. I agree that this should be addressed in a follow up at some point.

I think that the docs for the kernel could be made more explicit in this sense, maybe in a series of equations?

This seems reasonable. Again, I'm keen to address this in follow up -- if you're both comfortable with it, I would just like to get this in so that future work can look at it.

I would actually be really happy not to export this for the time being so that we don't have to make a breaking change in KernelFunctions if we make a breaking change to this kernel in follow up work.

[devmotion]

Seems reasonable to me, I agree that probably it should not be exported currently.

[willtebbutt]

@devmotion @theogf if there's nothing else, is this ready to go in?

[willtebbutt]

@theogf have you seen failures like this before on 1.3? The error look really quite large, but it's running quite an old version of Zygote now... also weird that it's only happening on ubuntu.

[theogf]

Well it's Gabor doing it again. We always have CI issues with this one! Maybe we should think of a more stable implementation or examine what's going wrong.

[willtebbutt]

😬 in that case, can I merge this?