How to check and improve optimization

The challenge

Parameter optimization of ODE models is challenging because

• there is no analytical solution of the ODEs, thus the objective function (and derivatives) can only be evaluated numerically with limited accuracy
• the objective function depends non-linearly on the parameters
• models might be large, i.e. have many parameters
• the amount of available data and observables is typically restricted which yields non-identifiability
• usually only rough knowledge is available about parameter ranges
• additional numerical challenges like events have to be accounted for

It is therefore essential to ensure that optimization works reliably.

Guidelines for reliable optimization

• Apply multistart-optimization (e.g. arFitLHS) in order to guarantee that
• • the optimization algorithms converges (at least locally), i.e. an optimum is found repeatedly
• • a found optimum is indeed (likeli) the global one
• Optimize the parameters by default at the log-scale (indecated in D2D by ar.qLog10). This in particular concerns strictly positive parameters, e.g. parameters that represent initial values and rate constants. Analogously, initial guesses and priors should be drawn or defined at the log-scale.
• Specify lower and upper bounds (indicated in D2D by ar.lb and ar.ub). This improves the convergence of lsqnonlin (default in D2D) and other optimization algorithms and prevents integration errors.
• Allow several orders of magnitude, i.e. use non-restritive lower and upper bounds
• Don't use finite differences for calculating derivatives. Instead, D2D exploits the so-called sensitivity equations
• Check whether optimization indeed works reliably

Checks for reliable optimization

• Convergence can be checked by multi-start optimization e.g. by arFitLHS or arLocalLHS.
• Convergence problems can be seen usually in likelihood profiles, i.e. it should be checked whether the profiles are suffiently smooth
• Look at the output of the optimization algorithm during a fit. This can be done by the option ar.config.optim.Display = 'iter'

Literature

Lessons Learned from Quantitative Dynamical Modeling in Systems Biology