Fitting exponentials to radioactive decay. The data is constructed to model the Geiger counter emission of a mixture of three common radioactive elements. The dashed line is the actual, average decay, the red line is an acceptable fit to the data (within statistical errors).

Fitting Exponentials: Prediction without parameters

We have seen that in multiparameter models the predictions are often remarkably insensitive to large variations in many sloppy directions in parameter space, leading to huge uncertainties in the parameters when fit to the data. Extracting parameters from fits to data in sloppy models is ill-posed. Under these circumstances, can we expect to be able to extract predictions from the data using the model?

Good fits with bad decay rates! Interpolation. Predictions which interpolate between data points are reliable even though the parameters are very poorly determined.

A classic example of an ill-posed inverse problem is given by radioactive decay. A mixture of three elements will produce a Geiger counter signal which theoretically will be

However, just because we don't know the parameters doesn't mean we can't make predictions. The green curves in the middle figure at right show a whole range of possible interpolations, with parameters varying wildly over huge ranges. (We sample all parameter space, weighting parameters by how well they fit the data.) You can see that the error bars are quite reasonable for predicting answers between the existing data points. This is neither surprising nor impressive - our theory is just a fancy interpolation between data points.

Extrapolation. Predictions which extrapolate beyond the range of the fitting data are not reliable, for sloppy models. (Notice that the vertical scale is now not the decay Γ(t), but its logarithm log(Γ(t)), so a range of ±2 is a factor of almost ten.)

But if we look at longer times, extrapolating past the data set, the fact that we don't know our parameters makes predictions basically impossible (bottom graph). In more complicated situations (like biological networks), we find that remarkably many useful predictions are like interpolations - they become precise far before the parameters are known. Of course, some predictions will always be extrapolations. Indeed, one can view the parameter fits themselves to be a kind of prediction from the model. Sloppy models are those for which the predictions of the parameters are severe extrapolations from the data points...

To understand how sloppiness reflects not intrinsic properties of the model, but a skewness in our choice of variables for the model, see Fitting Polynomials: Where is sloppiness from?.

References

• "Sloppy model universality class and the Vandermonde matrix", Joshua J. Waterfall, Fergal P. Casey, Ryan N. Gutenkunst, Kevin S. Brown, Christopher R. Myers, Piet W. Brouwer, Veit Elser, and James P. Sethna, Phys. Rev. Letters 97, 150601 (2006), pdf.
• Other papers on sloppy models

More on sloppiness:

• Sloppy Models
  • A sloppy systems biology model
  • What is Sloppiness?
  • What are Sloppy Models?
  • Fitting Exponentials: Prediction without parameters
  • Fitting Polynomials: Where is sloppiness from?
  • Why sloppiness? The Sloppy Universality Class
  • Differential Geometry and Sloppy Models (Transtrum)
    • The Model Manifold and Hyperribbons (Transtrum)
    • Sloppy Curvature (Transtrum)
  • Why is science possible? Sloppy models in Physics.
    • Jessie Silverberg's Huffington Post article and Katheryn McGill's vlog Interview from The Physics Factor.
    • Unedited workshop interview by Steven Reiner, Stony Brook School of Journalism; Mobile version.
    • News article on our paper showing physics is sloppy too
• Sloppy model applications
  • Do parameters matter? Fits versus measurements.
  • Experimental design in sloppy systems
  • Robustness and sloppiness
    • Parameter robustness and sloppiness
    • Environmental robustness and sloppiness: Circadian rhythms
    • Evolvability, robustness, and sloppiness
  • Estimating systematic errors for interatomic potentials and for density functional theory.
  • Learning digits with InPCA
  • Visualizing the learning trajectories of deep neural networks using InPCA

James P. Sethna, sethna@lassp.cornell.edu; This work supported by the Division of Materials Research of the U.S. National Science Foundation, through grant DMR-070167.

Statistical Mechanics: Entropy, Order Parameters, and Complexity, now available at Oxford University Press (USA, Europe).