| From | Michael E. Mann | Date | Tue, 14 Oct 2003 17:27:24 -0400 |
|---|---|---|---|
| To | Tom Wigley, Kevin Trenberth, Keith Briffa, Phil Jones, Folland, Chris, tkarl@ncdc.noaa.gov, Jonathan T. Overpeck, Michael E. Mann | ||
| Subject | Fwd: Re: smoothing | ||
| Sorry--one more error. The MSE values for "minimum norm" and "minimum roughness" are switched in the figure legend. Obviously the former is a better fit... mike Date: Tue, 14 Oct 2003 17:08:49 -0400 To: Tom Wigley tkarl@ncdc.noaa.gov, jto@u.arizona.edu, mann@virginia.edu From: "Michael E. Mann" Subject: Re: smoothing Bcc: Scott Rutherford correction '1)' should read: '1) minimum norm: sets padded values equal to mean of available data beyond the available data (often the default constraint in smoothing routines)' sorry for the confusion, mike At 05:05 PM 10/14/2003 -0400, Michael E. Mann wrote: Dear All, To those I thought might be interested, I've provided an example for discussion of smoothing conventions. Its based on a simple matlab script which I've written (and attached) that uses any one of 3 possible boundary constraints [minimum norm, minimum slope, and minimum roughness] on the 'late' end of a time series (it uses the default 'minimum norm' constraint on the 'early' end of the series). Warming: you needs some matlab toolboxes for this to run... The routines uses a simple butterworth lowpass filter, and applies the 3 lowest order constraints in the following way: 1) minimum norm: sets mean equal to zero beyond the available data (often the default constraint in smoothing routines) 2) minimum slope: reflects the data in x (but not y) after the last available data point. This tends to impose a local minimum or maximum at the edge of the data. 3) minimum roughness: reflects the data in both x and y (the latter w.r.t. to the y value of the last available data point) after the last available data point. This tends to impose a point of inflection at the edge of the data---this is most likely to preserve a trend late in the series and is mathematically similar, though not identical, to the more ad hoc approach of padding the series with a continuation of the trend over the past 1/2 filter width. The routine returns the mean square error of the smooth with respect to the raw data. It is reasonable to argue that the minimum mse solution is the preferable one. In the particular example I have chosen (attached), a 40 year lowpass filtering of the CRU NH annual mean series 1856-2003, the preference is indicated for the "minimum roughness" solution as indicated in the plot (though the minimum slope solution is a close 2nd)... By the way, you may notice that the smooth is effected beyond a single filter width of the boundary. That's because of spectral leakage, which is unavoidable (though minimized by e.g. multiple-taper methods). I'm hoping this provides some food for thought/discussion, esp. for purposes of IPCC... mike ______________________________________________________________ Professor Michael E. Mann Department of Environmental Sciences, Clark Hall University of Virginia Charlottesville, VA 22903 _______________________________________________________________________ e-mail: mann@virginia.edu Phone: (434) 924-7770 FAX: (434) 982-2137 [1]http://www.evsc.virginia.edu/faculty/people/mann.shtml ______________________________________________________________ Professor Michael E. Mann Department of Environmental Sciences, Clark Hall University of Virginia Charlottesville, VA 22903 _______________________________________________________________________ e-mail: mann@virginia.edu Phone: (434) 924-7770 FAX: (434) 982-2137 [2]http://www.evsc.virginia.edu/faculty/people/mann.shtml ______________________________________________________________ Professor Michael E. Mann Department of Environmental Sciences, Clark Hall University of Virginia Charlottesville, VA 22903 _______________________________________________________________________ e-mail: mann@virginia.edu Phone: (434) 924-7770 FAX: (434) 982-2137 [3]http://www.evsc.virginia.edu/faculty/people/mann.shtml References 1. http://www.evsc.virginia.edu/faculty/people/mann.shtml 2. http://www.evsc.virginia.edu/faculty/people/mann.shtml 3. http://www.evsc.virginia.edu/faculty/people/mann.shtml | |||