Supports add constrained variable

[ilancoulon]

I am trying to tackle #987 with this PR.

As it's my first one, I am still trying to comprehend MOI's overall design and I am aware this PR still has a lot of flaws. I hope you guys will help me correct them.

Here is what issues I still have and where you guys could help me:

• Thanks to @blegat , I created a fallback for the get(model, attr) method in order to be able to give a default value to the newly created attribute VariableBridgingCost. Not creating an intermediary get_fallback method resulted in method ambiguities since model-specific methods were already defined, such as MOI.get(mock::MockOptimizer, attr::MOI.AbstractModelAttribute). The issue here is that I still had to define a specific method for the CachingOptimizer, otherwise an error was thrown during the tests when no optimizer was attached (thrown there). I must find a way to get rid of those definitions.
• I'm not sure I really understood how the "bridging graph" works, and I'd like your opinion on what I edited in lazy_bridge_optimizer.jl.
• In copy.jl, since the NLPBlock assumes no change in the order of the variables, I didn't use any bridging cost for it, am I right?
• In copy.jl, the copying still takes the vector variables first. I did that to change as little as possible. Should we keep that?
• I added a ConstraintBridgingCost too, as @blegat first stated in [RFC] supports_constrained_variable #987, but I'm not sure how I should use it in the default_copy_to function.
• I'm not confident about where I put everything, such as the definition of the new attributes in attributes.jl, the default values in model.jl.
• Finally, I don't know where I should write the tests for this new feature, I'd be grateful if you could point me in the right direction on that.

That is a lot of questions, but I hope this PR will eventually be useful.

PS: For now, you can't really use this feature with JuMP, and we will eventually need to add this to JuMP.

[blegat]

The implementation looks good, testing now :)
I suggest you add tests in test/Utilities/copy.jl. You can fill a src = MOIU.Model{Float64}() with some constraints/variables, then you create some custom model that you create a dest model that you define with MOIU.@model and customize with supports_constrained_variable(s) and then you call index_map MOI.copy_to(dest, src). You can then check in index_map the indices of the constraints in dest and check that their value is what is expected. The indices of a MOIU.AbstractModel are unique even across different constraint type so that allows to see the order of creation easily.
You can try different examples of custom model types for dest with different combinations of supported/unsupported variables/constrained and check that the order is modifed accordingly.

[ilancoulon]

I wrote a test for supports_add_constrained_variables since the indices of AbstractModel are not unique with constraints on AbstractScalarSet. Should I write my own model to be able to test that or is this test enough?

[blegat]

Now, if you run the test with dest = full_bridge_optimizer(ReverseOrderConstrainedVariablesModel(), Float64) or dest = full_bridge_optimizer(OrderConstrainedVariablesModel(), Float64), I guess one of the two will fail as with the bridges they both support both constrained variables.
To fix this, we should implement the attribute for LazyBridgeOptimizer and return the bridge cost.

[ilancoulon]

I'm not sure what you meant here. Could you maybe write the corresponding test and I'd try to make it work?

[blegat]

I mean something like

dest = OrderConstrainedVariablesModel()
bridged_dest = MOI.Bridges.full_bridge_optimizer(dest, Float64)
index_map = MOI.copy_to(bridged_dest, src)
@test typeof(c1) == typeof(dest.constraintIndices[2])
@test typeof(c2) == typeof(dest.constraintIndices[1])

dest = ReverseOrderConstrainedVariablesModel()
bridged_dest = MOI.Bridges.full_bridge_optimizer(dest, Float64)
index_map = MOI.copy_to(bridged_dest, src)
@test typeof(c1) == typeof(dest.constraintIndices[1])
@test typeof(c2) == typeof(dest.constraintIndices[2])

[ilancoulon]

By trying the following lines:

src = MOIU.Model{Float64}()
a, c1 = MOI.add_constrained_variables(src, MOI.Nonpositives(3))
c2 = MOI.add_constraint(src, a, MOI.Nonnegatives(3))
dest = OrderConstrainedVariablesModel()
bridged_dest = MOI.Bridges.full_bridge_optimizer(dest, Float64)
index_map = MOI.copy_to(bridged_dest, src)

I get an error stating MathOptInterface.UnsupportedConstraint{MathOptInterface.VectorOfVariables,MathOptInterface.Nonpositives}: MathOptInterface.VectorOfVariables-in-MathOptInterface.Nonpositives constraint is not supported by the model..
This is because no bridge of type ::Type{MathOptInterface.VectorOfVariables}, ::Type{MathOptInterface.Nonpositives} exists in the LazyBridgeOptimizer.

I must say I did not really grasp how the bridge types work, and I'm not sure how I should go from here. Maybe that is why the issue was not tagged as a good first issue 😅

I tried implementing the attributes with:

MOI.get(b::LazyBridgeOptimizer, c::MOI.VariableBridgingCost{S}) where S<:MOI.AbstractScalarSet = MOI.get(b.model, c)
MOI.get(b::LazyBridgeOptimizer, c::MOI.VariableBridgingCost{S}) where S<:MOI.AbstractVectorSet = MOI.get(b.model, c)
function MOI.get(b::LazyBridgeOptimizer, c::MOI.ConstraintBridgingCost{F,S}) where {S <: MOI.AbstractSet, F <: MOI.AbstractFunction}
    MOI.get(b.model, c)
end

But this did not work better, I think this shows how I don't actually understand how all of this works.