@DS: 8 variables for Rossander (4 SAW filters) + 0 vars for AGW.
Haven’t you forgotten ToothW (D26), David? How else can Solver pick frequencies for the harmonics?
In his original post at WUWT Rossander wrote, ” I played with a few initial parameters to see what they might do, then fired off Solver with the instruction to modify the parameters below with a goal of maximizing cell U35 (MUL R2).”
Rossander then lists the 12 parameters he authorized Solver to modify. As far as he’s concerned he used 12 parameters.
In hindsight we can use the fact that Solver achieved R2 = 99.992% by setting SAW4 to zero to justify taking control of SAW4 away from Solver and zeroing it manually ourselves. This would bring Rossander’s parameters down to 10. By the rules I’ve been playing by this is fair play.
But since Solver is allowed to set all five amplitudes of the individual harmonics independently, Amp (M26) becomes redundant, that is, these six parameters are not linearly independent. This further brings Rossander’s parameters down to 9, the same number as I’m using.
I don’t see any room for further reducing the count for Rossander’s fit.
But since it’s the same number of parameters as mine, it would now be a tie were it not for the fact that we can break the tie with R2.
Never expecting this sort of competition, I had rounded my parameters to plausibly round numbers without worrying about the fifth digit of R2. Using one extra digit of precision in a couple of the parameters, very small changes to my poster’s parameters brought the 99.990% in my spreadsheet (conservatively reported as 99.98% in the poster) up to 99.997%.
So with the same number of parameters, we have 99.992% against 99.997%.
This sounds better in terms of variance of MRES as a fraction of that of MUL expressed in parts per million instead of percentages. Using AGW we get 3, replacing AGW by additional SAW parameters gives 8. Smaller is better in this case.
Rossander also said No other constraints were applied. That’s evidently incorrect as he appears to have overlooked Solver’s checked-by-default box constraining all otherwise unconstrained variables to be nonnegative. Solver found that the solution could be improved by decreasing SAW4 but when it reached zero Solver had to stop because of the checked box.
Further on Rossander wrote My analysis was trivial, limited and deeply flawed. It had to be so because it was based on no underlying hypothesis about the factors being modeled (other than the ClimSens control value). It was an exercise that took about 15 min and was intended only to illustrate the dangers of over-fitting one’s data.
With more time Rossander would presumably have noticed the oddity about SAW4, diagnosed it, and unchecked the box.
I discussed all this at WUWT three weeks ago. However with over 2000 comments here plus those on WUWT including two each by Rossander and me on this topic I can’t say I blame people for losing track of what everyone’s said in the past five weeks.
One thing that’s been nagging at the back of mind since then is that 2 more parameters (those for SAW4) should have totally nailed it. What would have happened had Rossander unchecked that box?
I tried this just now (I included Amp for old time’s sake) and Solver gave me the following fit.
D26 ToothW 1783.15…
G23 Shift 1 2219.30…
G26 Scale 1 761.44…
H23 Shift 2 4268.41…
H26 Scale 2 598.68…
I23 Shift 3 4550.48…
I26 Scale 3 1271.08…
J23 Shift 4 2741.42…
J26 Scale 4 521.23…
K23 Shift 5 3283.39…
K26 Scale 5 577.08…
M26 Amp 3748.51…
(Paradoxically Solver seems to have sent SAW4 into negative territory only to bring it back out later on. This could well have happened via continuous changes to Shift 4, J23, while J26 was negative. As John S would point out this is more easily understood in terms of the 2D geometry of the complex plane.)
The envelope, please…
R2 = 100.000%
Fourier would have smiled and said “That’s what it should have been.”
In those days mathematicians were mathematicians and didn’t need no stinking computers to see the obvious.
But this also shows how dangerously close 9 parameters are to overfitting. The “3 vs. 8″ outcome would seem to show that although 9 parameters is not completely overfitted it’s in a very tight corner.
For my poster to have even a prayer of being publishable this corner needs to be considerably enlarged. I certainly wouldn’t attempt to publish it as is. Pekka made much the same point in his first reaction to the poster, and I’ve since been mulling over the various ways of addressing this serious concern.
Climate Etc. to the rescue. Thanks to Judith for founding and hosting it.