Multifidelity Bayesian Optimization with KG

[S3anaaaaz]

Hi,

I have a question about multifidelity BO with KG. It seems that the algorithm only finds the global minimum/maximum instead of minimum/maximum of high fidelity. If for example, the low fidelity has lower/higher values than high fidelity, botorch finds the minimum/maximum of low fidelity even if we want the high fidelity minimum/maximum. Am I correct? or I missed something?

Please let me know. I appreciate your time and help on this.

Thanks,
Sanaz

[saitcakmak]

Hi @S3anaaaaz. qMultiFidelityKnowledgeGradient supports multiple use cases beyond the multi-fidelity included in its name. For multi-fidelity use cases, you should be using the project operator (https://github.com/pytorch/botorch/blob/main/botorch/acquisition/knowledge_gradient.py#L373-L376) to map to the target fidelity. You can find example usage of this in the tutorials here and here.

It seems that the algorithm only finds the global minimum/maximum instead of minimum/maximum of high fidelity.

Optimizing qMultiFidelityKnowledgeGradient will give you the point that offers the most information / improvement (as defined by the acquisition function) across the whole design space. In the multi-fidelity case, this will be the point that offers most improvement on the maximizer of the target fidelity. It will not be the point maximizing the target function for any fidelity. It will just maximize the acquisition function (across all fidelities, assuming that the bounds passed to optimize_acqf include the fidelities in the search space).

[S3anaaaaz]

Hi Saitcakmak,

Thank you for your prompt response. I have used the project in the example that I was solving, however I realized it doesn't do a proper job in finding the minimum/maximum of high fidelity. For example, even in the mentioned example, https://botorch.org/tutorials/discrete_multi_fidelity_bo, the recommended point and objective value from EI and KG are very different.

Thanks,
Sanaz

[saitcakmak]

For example, even in the mentioned example, https://botorch.org/tutorials/discrete_multi_fidelity_bo, the recommended point and objective value from EI and KG are very different.

The results of the tutorial show that qMFKG does a better job in finding the maximizer of AugmentedHartmann than EI. This is demonstrated by the larger final objective value recovered by using qMFKG under the same budget.

I have used the project in the example that I was solving, however I realized it doesn't do a proper job in finding the minimum/maximum of high fidelity.

Can you explain how you used it and why do you think it doesn't do a proper job at finding the maximum of the high fidelity? Some basic information on the function you're trying to optimize, such as the dimension of the search space, would also be helpful.

[S3anaaaaz]

Thank you for your answer. The dimension of the example I am using is 1. The minimum of high-fidelity is x=-1.5 and y=-5.72, but the function finds the value y=-7.2 which is the minimum of low-fidelity.
Now, if I subtract high-fidelity function from 3, its minimum will be -8 which is lower than the minimum of low fidelity. So, in this case the function finds -8.
1D_EXAMPLE.zip

Thanks a lot

[saitcakmak]

I ran your code (commenting out lmgp_pytorch imports and replacing cost_model = AffineFidelityCostModel(fidelity_weights={1: 5.0}, fixed_cost=5.0)). It did find the minimizer of the high fidelity in my case:

recommended point:
tensor([[-1.5760,  1.0000]], dtype=torch.float64)

objective value:
tensor([5.7275], dtype=torch.float64)

final recommnedation is tensor([[-1.5760,  1.0000]], dtype=torch.float64) with y tensor([5.7275], dtype=torch.float64)

The minimum of high-fidelity is x=-1.5 and y=-5.72, but the function finds the value y=-7.2 which is the minimum of low-fidelity.

Here are plots of the high fidelity

and the low fidelity from the notebook.

I am not sure where the y=-7.2 is coming from in your case. The minimum of the low fidelity is at x=-1.28 & y=-1.13 here, which has an objective value of y=-5.23 for the high fidelity objective.

[S3anaaaaz]

Thank you so much for your answer.
I checked what you said, and it works, but it does not work if I increase the cost difference (1000 for high and 1 for low).

[saitcakmak]

it does not work if I increase the cost difference (1000 for high and 1 for low).

I'd interpret this as a feature rather than a failure :). With a cost difference of 1000x, we're essentially telling the acquisition function to avoid evaluating the high fidelity objective at all cost. Unless it is absolutely certain of the low fidelity function, thus it thinks that there's no information to be gained by evaluating it anymore, it is unlikely to ever evaluate the high fidelity function.
The problem with this is that, with too few high fidelity evaluations, there's too little information to get a reliable estimate of the high fidelity function or of the correlation between the two functions. If you can't learn a reliable correlation structure between the two functions, you can't use the evaluations of one to infer the other. So, you need let the algorithm evaluate the high fidelity function on occasion to be able to model it properly.

[S3anaaaaz]

Thank you so much
You really helped me a lot.
Appreciate it so much.

[S3anaaaaz]

Hi, Sorry I have another problem with Botorch.
While running different examples, I face this erro:

I tried with different initializations, but it didn't help. What should I do?

[Balandat]

This is numerical issues that happens if the covariances involved in the GP computations are very ill conditioned. This often happens in cases where there are a lot of repeated points. It can happen in different places of the code path though, so to understand what is going on we'd need a full stack trace or repro.

[S3anaaaaz]

Thank you so much for your answer. I attached the code. I would appreciate it if you take time and help me in solving the problem.

…

On Tue, Aug 2, 2022 at 7:40 PM Max Balandat ***@***.***> wrote: This is numerical issues that happens if the covariances involved in the GP computations are very ill conditioned. This often happens in cases where there are a lot of repeated points. It can happen in different places of the code path though, so to understand what is going on we'd need a full stack trace or repro. — Reply to this email directly, view it on GitHub <https://urldefense.com/v3/__https://github.com/pytorch/botorch/discussions/1320*discussioncomment-3312536__;Iw!!CzAuKJ42GuquVTTmVmPViYEvSg!M0gBq8U2QGUYvTbTTJrVnCGWn0kMPQcjI5mBnMUONsABylJ87uyE4CQW-teKjE6elsWjfn3ZWyjkheama1nTn9_m$>, or unsubscribe <https://urldefense.com/v3/__https://github.com/notifications/unsubscribe-auth/AY2WGMOGNMCABD55OTMM6ZLVXHLZRANCNFSM5355FYNA__;!!CzAuKJ42GuquVTTmVmPViYEvSg!M0gBq8U2QGUYvTbTTJrVnCGWn0kMPQcjI5mBnMUONsABylJ87uyE4CQW-teKjE6elsWjfn3ZWyjkheamaydhdTDh$> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[saitcakmak]

The code you shared is not runnable as it is missing lmgp_pytorch which it seems to use to run the bo loop. Without running the code, we cannot really comment on why the error happens or what you can do to fix it. Here is a common case where NotPSDError happens: You have lots of training inputs that are clustered closely in the search space (or repeated points), leading to numerical singularity of the covariance matrix.

[S3anaaaaz]

Thank you for your answer and sorry for the problem.
I attached the correct code now.
Rosen.zip

[S3anaaaaz]

I really appreciate it if you could find some time to look at my code :) .

[saitcakmak]

Hi @S3anaaaaz. I ran your code and here are the X's evaluated up to the point of failure:

Matrix not positive definite after repeatedly adding jitter up to 1.0e-06.
train_x: tensor([[ 0.3300,  0.7580,  1.0000],
        [ 0.7884,  0.3319,  1.0000],
        [ 0.7273,  0.8577,  1.0000],
        [ 0.1369,  0.1113,  1.0000],
        [ 0.0583,  0.9322,  1.0000],
        [ 0.5488,  0.0195,  1.0000],
        [ 1.0000,  0.6165,  1.0000],
        [ 0.3776,  0.4908,  1.0000],
        [ 0.4846,  0.8966,  1.0000],
        [ 0.8910,  0.1935,  1.0000],
        [ 0.6844,  1.0000,  0.7500],
        [ 0.0000,  0.3834,  0.7500],
        [ 0.2750,  0.6194,  0.7500],
        [ 0.9275,  0.0000,  0.7500],
        [ 0.8698,  0.7026,  0.7500],
        [ 0.2383,  1.0668,  1.0000],
        [ 0.1296,  1.0668,  1.0000],
        [ 0.2017,  1.0668,  1.0000],
        [ 0.6634,  1.0439,  1.0000],
        [ 0.2632,  1.0668,  1.0000],
        [-0.0122,  1.0668,  0.7500],
        [ 0.2371,  1.0668,  0.7500],
        [ 0.9050,  1.0668,  0.7500],
        [-0.0122,  1.0668,  0.7500],
        [ 0.6644,  1.0668,  0.7500],
        [-0.0122,  1.0668,  1.0000],
        [ 0.3012,  1.0668,  1.0000],
        [ 0.0861,  1.0668,  1.0000],
        [-0.0122,  1.0668,  1.0000],
        [ 0.1499,  1.0668,  1.0000],
        [ 1.0124,  1.0668,  1.0000],
        [ 1.0124,  1.0668,  1.0000],
        [-0.0122,  1.0668,  1.0000],
        [ 0.4738,  1.0668,  0.7500],
        [ 0.0680,  1.0668,  1.0000],
        [ 1.0124,  1.0668,  1.0000],
        [ 0.5797,  1.0668,  0.7500],
        [ 0.6626,  1.0668,  1.0000],
        [ 0.1237,  1.0668,  0.7500],
        [ 0.6531,  1.0668,  1.0000],
        [ 0.5566,  1.0668,  1.0000],
        [ 0.1987,  1.0668,  1.0000],
        [ 0.6315,  1.0668,  1.0000],
        [ 0.5481,  1.0668,  0.7500],
        [ 0.5488,  1.0668,  1.0000],
        [ 0.1931, -0.0395,  0.7500],
        [ 0.5774,  1.0668,  0.7500],
        [-0.0121,  1.0668,  0.7500],
        [ 0.0181,  1.0668,  0.7500],
        [ 0.2090,  1.0668,  1.0000],
        [-0.0122,  1.0668,  1.0000],
        [ 0.6884,  1.0668,  1.0000],
        [ 0.2146,  1.0668,  1.0000]], dtype=torch.float64)

The last dimension is the feature indices, so let's ignore that. We see that the second dimension is almost always the same, and the first dimension also has points that are quite close to each other or are repeated. This is generally a great recipe for numerical singularity of the covariance matrix, which leads to the error you're seeing.

You can try to force your way through by letting GPyTorch try adding larger values to the diagonal of the covariance matrix. For this, you can use the following context manager:

            with gpytorch.settings.cholesky_max_tries(6):
                mfkg_acqf = get_mfkg(model)
                new_x, new_obj, cost = optimize_mfkg_and_get_observation(mfkg_acqf)

[saitcakmak]

Note that this is not a magical bullet. If you have too many repeated observations, GPs are likely to fail eventually. After trying this with your code, I got some more iterations but things failed even with the increased Cholesky tries.