Dr. Curry, this is OT but you might find it of interest as material for future posts:
Comment on UQ by NW
↧
↧
Comment on UQ by robin
There are a lot of steps between where they are and anything as anal as V&V. Basic common sense coding practices, tests, code reviews, documentation, naming, commenting, specs…
I spent last night looking through a GISS codebase, it isn’t hilariously bad or anything, but it lacks the rigor and clarity of normal software. That will be holding it back in many ways, not just with bugs.
↧
Comment on UQ by Dan Hughes
The word “training” is important, because generally speaking, a skilled simulationist running buggy software will consistently obtain predictions that are more accurate and more reliable that an unskilled simulationist running perfect software.
Clearly and without question, this statement needs to be qualified relative to the nature of the bugs in the ‘buggy software’. It cannot be correct if the bug is such that failure to complete the calculation is the result. It also cannot be correct if the nature of the bug is to invalidate parts of the calculation that are critically important to the response functions of interest.
The common-sense point of NAFEMS (and its journals) is that experience has established that devoting resources to software V&V is futile, unless the V&V is accompanied by matching investments in human training with regard to the software’s foundations in physics and mathematics.
This is an interesting summary in that the objective of V&V is to determine the soundness of the software’s foundation in physics ( Validation ) and mathematics ( Verification ).
Cites to reports and papers which demonstrate that devoting resources to V&V is futile are of interest. Especially now that the concepts are being taken up by the climate science community following decades of development and applications in engineering and other areas of science.
↧
Comment on UQ by Jim2
Of course you have to understand the problem, the way climate works, the science and math in order to create a climate model. But buggy software isn’t going to embody the correct math. You really have to have a solid understanding of what you are modeling AND solid code. The idea that buggy code is an accurate model of anything is kind of silly.
↧
Comment on UQ by Steven Mosher
why do you think an accurate model is needed to for policy.
We don’t have very good flood prediction models, yet we set policy about flood plains all the time. We cannot predict earthquakes, yet we have earthquake building codes. Your notion that one needs an accurate model to set policy is not bourne out by a simple examination of what we in fact do. We do in fact make decisions under uncertainty all the time.
You think the accuracy is not good enough. That is subjective.
All of the models have decades of random cooling. THAT is the basis of santers conclusion about needing 17 years of data.
↧
↧
Comment on UQ by Steven Mosher
your lack of understanding cant be accounted for
↧
Comment on UQ by Gareth Williams
You can average over a series of real-world measurements. Assuming they are independent and drawn from some probability distribution, the average is better than any one measurement.
You can also average over many runs of a model with randomly drawn parameters (that is a Monte-Carlo simulation).
But it makes no sense at all to average over a number of different models that make fundamentally different assumptions. There is no statistical reason (Bayesian or otherwise) to think that will give a better answer than any one of them. If one of them happened to be right, you are just averaging it with wrong answers.
↧
Comment on UQ by Steven Mosher
all data is theory laden.
↧
Comment on UQ by Punksta
> We don’t have very good flood prediction models, yet we set policy about flood plains all the time.
There is a difference though. Floods could possibly go to zero, but can’t go negative, like the CO2 impact on temperature perhaps can. (Or are there combined flood-drought models?)
↧
↧
Comment on UQ by Steven Mosher
A policy maker who uses the model results might not agree.
“behaving” like the climate is over general and non testable
↧
Comment on UQ by Steven Mosher
unfortunately rutherford was wrong. No experimental answer ever agrees exactly with the prediction.
↧
Comment on UQ by Punksta
Rutherford. By an Occam argument, wouldn’t an experiment without stats be more preferred (and convincing) than one that did ?
↧
Comment on Week in review 4/13/12 by WebHubTelescope
Chief Hydrologist is a sock puppet for Captain Kangaroo. Since Kangaroo is clearly wrong on asserting that heated water does not diffuse to regions of colder concentration (or lower thermal density), he brings in a hydrologist to strengthen his claim. But the Chief is the same person, a civ who only has a cursory understanding of physics.
I always rely on real physics and supported by empirical data, both historical and current. This angers the Kangaroo, who apparently has an agenda of spreading FUD.
↧
↧
Comment on UQ by David Springer
Mosher, I’m glad to hear there are no runaway feedbacks. Tell me then what limits the positive feedbacks in climate models. I asked here for what it might be and physicistdave and Fred Moolten responded that it is 5500C (the temperature of the sun’s photosphere).
↧
Comment on UQ by David Springer
Do any of the models reproduce ice ages?
If they do then by definition some of them “show cooling”. If they do not then they are flawed.
It’s really that simple. So which is it, Mosher? Do any models show cooling or are they all flawed?
↧
Comment on UQ by David Springer
Mosher doesn’t understand the difference between a flood and global warming, evidently. There are a great many floods so with a sample size that large we use actuarial techniques to assess risk. How many catastrophic anthropogenic global warmings have there been so that we may use actuarial techniques to make risk/reward decisions?
Duh.
↧
Comment on Assessing climate model software quality by Doug Cotton
As you can see in the Appendix <a href="http://principia-scientific.org/publications/psi_radiated_energy.pdf" rel="nofollow">here</a> the long-term rate of increase in SST was about 0.06 C / decade between 1900 and 1930 and has now reduced to about 0.05 C / decade.
↧
↧
Comment on UQ by Joe's World
Judith,
Missing other parameters are not an issue?
Of course not. Just the mathematical equation is all important.
↧
Comment on The Internet: World War 3.0 (?) by Pointman
“They will not control us …”
Pointman
↧
Comment on Assessing climate model software quality by Bart R
capt. dallas 0.8 +/-0.2 | April 18, 2012 at 11:14 am |
ceteris non paribus | April 18, 2012 at 12:02 pm |
Apologies, I ought have prefaced my post with something like, “Hanc marginis exiguitas non caparet” to reflect the back-of-the-envelope nature of my speculations.
My intention is to reflect on the greater relative strength of consilience of multiple facets of a single experimental outcome than some readers might infer by he-said-she-said reasoning.
While more careful examination than I’ve given to verify all seventeen (or more) interconnected claims, the idea of prior knowledge is not appropriate to estimation of odds entirely. For one thing, it would be begging the question to assume the hypothesis right because the experiment was so successful, ergo our ‘prior knowledge’ amounts to certainty, which would certainly be implied by the statement that the results are not independent.
So we must treat them as independent until we have commonly accepted independent evidence of the nature and kind of their relationships. Frankly, one doubts if people reject the models they are likely to come to common acceptance of much about physics with those who accept the models.
For another, if the models are wrong, then certainly the outcomes are independent. Which gives rise to a one in thirteen million (give or take perhaps ten million) longshot. I’m not going to stand next to the model in a lightning storm, if so.
And it’s only one of many ways of looking at the odds, depending on one’s needs.
↧
More Pages to Explore .....
