James Sethna > Sloppy Models > Fitting Exponentials: Predictions without parameters

Decaying data Gamma(t) with theory and fit
Fitting exponentials to radioactive decay. The data is constructed to model the Geiger counter emission of a mixture of three common radioactive elements. The dashed line is the actual, average decay, the red line is an acceptable fit to the data (within statistical errors).

Fitting Exponentials: Prediction without parameters

We have seen that in multiparameter models the predictions are often remarkably insensitive to large variations in many sloppy directions in parameter space, leading to huge uncertainties in the parameters when fit to the data. Extracting parameters from fits to data in sloppy models is ill-posed. Under these circumstances, can we expect to be able to extract predictions from the data using the model?

Ensemble of fits, interpolating between data
Good fits with bad decay rates! Interpolation. Predictions which interpolate between data points are reliable even though the parameters are very poorly determined.

A classic example of an ill-posed inverse problem is given by radioactive decay. A mixture of three elements will produce a Geiger counter signal which theoretically will be

y(A,γ,t) = A1 e-γ1 t + A2 e-γ2 t + A3 e-γ3 t
It would take, however, enormously careful measurements (much better data than shown at right) to measure the six constants Ai and γi with any reasonable accuracy using the Geiger counter data. (Even one decay would do a good job, allowing the other two decay rates and amplitudes to wander far from their real values.) Fitting exponentials to data is a sloppy, ill-posed problem.

However, just because we don't know the parameters doesn't mean we can't make predictions. The green curves in the middle figure at right show a whole range of possible interpolations, with parameters varying wildly over huge ranges. (We sample all parameter space, weighting parameters by how well they fit the data.) You can see that the error bars are quite reasonable for predicting answers between the existing data points. This is neither surprising nor impressive - our theory is just a fancy interpolation between data points.

Ensemble of fits, extrapolating to long times
Extrapolation. Predictions which extrapolate beyond the range of the fitting data are not reliable, for sloppy models. (Notice that the vertical scale is now not the decay Γ(t), but its logarithm log(Γ(t)), so a range of ±2 is a factor of almost ten.)

But if we look at longer times, extrapolating past the data set, the fact that we don't know our parameters makes predictions basically impossible (bottom graph). In more complicated situations (like biological networks), we find that remarkably many useful predictions are like interpolations - they become precise far before the parameters are known. Of course, some predictions will always be extrapolations. Indeed, one can view the parameter fits themselves to be a kind of prediction from the model. Sloppy models are those for which the predictions of the parameters are severe extrapolations from the data points...

To understand how sloppiness reflects not intrinsic properties of the model, but a skewness in our choice of variables for the model, see Fitting Polynomials: Where is sloppiness from?.

References

More on sloppiness:


Last Modified: June 11, 2008

James P. Sethna, sethna@lassp.cornell.edu; This work supported by the Division of Materials Research of the U.S. National Science Foundation, through grant DMR-070167.

Statistical Mechanics: Entropy, Order Parameters, and Complexity, now available at Oxford University Press (USA, Europe).