|
|
| Triple complex: prediction errors before new measurement. Percent of total Cbl sequestered in a triple complex with Cool-1 and Cdc42. |
In designing a new experiment, how should you use your computational model? The model will predict some things well, and you can test (and perhaps falsify) the model by doing experiments there - especially if the predictions of the model seem doubtful.
On the other hand, the model (when fit to data) may not make reliable predictions, perhaps for key properties that one is particularly interested in. Then we may wish to design new experiments that are tailored to increase our understanding - to make the model more predictive for these key properties. In the statistics literature, this is called optimal experimental design.
We have seen that many computational models, especially those with many unknown parameters, are sloppy; the parameters in the model can vary over huge ranges in certain sloppy directions without changing the predictions. Much of the optimal experimental design literature asks what measurement will best lower the volume of parameter space uncertainty. For a sloppy model, this isn't a sensible question; we want to reduce the uncertainty in predictions, not parameters.
|
| EGFR trafficking network. |
Rick Cerione's group, for example, has been working on "trafficking" in certain cells from the mouse (diagram at right). When a hormone (like EGF, epidermal growth hormone, green dots) comes in contact with a cell, it sticks to a receptor (EGFR, green Ys) on the membrane (orange curve), and starts a signaling process telling the cell to change its behavior. When the hormone level drops outside, though, the EGF bound to the receptor doesn't leave by itself. The cell needs to pull the receptor inside, through the pinching off of spherical bits of membrane (called vesicles) in a process called endocytosis. Hormone receptor trafficking is this process of swallowing, digesting, and recycling of the receptor and hormone by the cell. Rick's group (Dan Baird and Qiyu Feng) has been looking at the way three proteins interact to control this process - Cbl, Cool-1, and Cdc42.
The key question they wanted to answer was whether Cbl was being tied up (sequestered) by Cool-1 and Cdc42 in a "triple complex" at around ten minutes after the hormone was introduced. Fergal Casey designed a computational model of this system, fit the existing data, and found that we did not know enough to tell whether the triple complex was tying up Cbl or not - the errors were so large that any value between 5% (negligible) and 25% (substantial) was possible (figure at top). What was the best experiment we could design to reduce the uncertainty in this measurement?
|
| Optimum experimental design. To best reduce the uncertainty in the triple complex concentration, the best feasible experiment was to measure the total concentration of activated Cdc42, either at one point just past ten minutes, or (if a series of points is allowed) at time points scattered up to about 40 minutes. |
One might immediately recommend that one measure the amount of the triple complex. (Yes, the mathematical solution does predict the obvious answer.) But that turns out not to be experimentally feasible. Allowing only measurements of things that could be measured, Fergal discovered that measuring the amount of Cdc-42 that was "activated" (bound to a GTP molecule) was the most useful in reducing the prediction uncertainties (figure at left). If you could do only one measurement, it would be best done at a bit over ten minutes; if several measurements were allowed they should be done at a sequence of early times.
As it happens, Rick came late to our group meeting when Fergal was giving his results. Rick spontaneously mentioned that he had been thinking about the best experiment to do next to figure out the puzzle of Cbl sequestration, and thought that measurements of active Cdc-42 would be the best idea!
|
| Triple complex: prediction errors after optimized Cdc42 measurement. |
The figure at right shows the expected reduction in uncertainty that Fergal found based on one extra measurement (dotted line) or multiple measurements (dashed line) of total active Cdc-42. As it happens, Dan and Qiyu's subsequent experiments performed as the theory anticipated...
So, realizing that the computational models are sloppy can guide one into
new ways of thinking about designing experiments. At least, we were able
to duplicate with a computer the results Rick Cerione intuited through wisdom
and experience. One can hope also that Fergal's methods might be better
able to cope with the increasing complexity of the biological systems being
studied.
James P. Sethna, sethna@lassp.cornell.edu; This work supported by the Division of Materials Research of the U.S. National Science Foundation, through grant DMR-070167.
Statistical Mechanics: Entropy, Order Parameters, and Complexity,
now available at
Oxford University Press
(USA,
Europe).