The questions aren't necessarily directed to anyone in particular. I am tossing out these ideas so I don't lose them..

I think discussions about residuals have to start with the inherent noise in the system. For example, the SOI is a measure of the dipole difference between Tahiti and Darwin, but Tahiti /= -Darwin, and so that tells us something about the noise in the system. I recall the highest correlation coefficient that I could achieve between NINO34 and SOI was about 0.86 -- and these are both considered indices of ENSO. So trying to explain a residual is conflated with uncertainties in the data and with variations in the weather.

I am not even sure if a correlation coefficient is the best error criteria. I know it is not squared error or absolute error, which does really poorly on time series profile shapes. I have been experimenting with a metric that detects same sign excursion.

~~~

%% excursion(+X, +Y, -R)

%

% Excursion match of two arrays

%

excursion(X, Y, R) :-

mean(X, XM), XOff is -XM,

mean(Y, YM), YOff is -YM,

DXM mapdot XOff .+ X,

DYM mapdot YOff .+ Y,

DXE mapdot sign ~> DXM,

DYE mapdot sign ~> DYM,

Num dot DXE*DYE,

length(X, N),

R is ( 1 + Num/N)/2.

~~~

I have thought that the only way to evaluate the model is to compare to another model and use something like AIC or BIC to evaluate.

1. Establishing a plausible physical model

2. Determining the parameter values of the model based on known characteristics

3. Figure out an information metric for the model parameters and result

I think discussions about residuals have to start with the inherent noise in the system. For example, the SOI is a measure of the dipole difference between Tahiti and Darwin, but Tahiti /= -Darwin, and so that tells us something about the noise in the system. I recall the highest correlation coefficient that I could achieve between NINO34 and SOI was about 0.86 -- and these are both considered indices of ENSO. So trying to explain a residual is conflated with uncertainties in the data and with variations in the weather.

I am not even sure if a correlation coefficient is the best error criteria. I know it is not squared error or absolute error, which does really poorly on time series profile shapes. I have been experimenting with a metric that detects same sign excursion.

~~~

%% excursion(+X, +Y, -R)

%

% Excursion match of two arrays

%

excursion(X, Y, R) :-

mean(X, XM), XOff is -XM,

mean(Y, YM), YOff is -YM,

DXM mapdot XOff .+ X,

DYM mapdot YOff .+ Y,

DXE mapdot sign ~> DXM,

DYE mapdot sign ~> DYM,

Num dot DXE*DYE,

length(X, N),

R is ( 1 + Num/N)/2.

~~~

I have thought that the only way to evaluate the model is to compare to another model and use something like AIC or BIC to evaluate.

1. Establishing a plausible physical model

2. Determining the parameter values of the model based on known characteristics

3. Figure out an information metric for the model parameters and result