FromBen SanterDateThu, 21 Feb 2008 17:12:22 -0800
ToPhil Jones
SubjectRe: Coverage

Dear Phil,

A quick question: Do you happen to have a "percentage land coverage
mask" for the HadCRUT3v data? And if so, does this exist as a netCDF file?

With best regards,

Ben
Phil Jones wrote:
>
> Ben,
> Email to Dick reminded me ! Had another phone call and I'd forgotten.
> First file is the coverage.
>
> Second is a program that reads this file - Channel 1.
>
> File is 36 by 72. 5 by 5 degs.
>
> It will start at 85-90N for the 36 subscript.
>
> for 72 it is either dateline or Greenwich.
>
> Cheers
> Phil
>
>
> At 16:53 15/02/2008, you wrote:
>> Dear Dick,
>>
>> I'm forwarding an email that I sent out several days ago. For the last
>> month, I've been working hard to respond to a recent paper by David
>> Douglass, John Christy, Benjamin Pearson, and Fred Singer. The paper
>> claims that the conclusions of our CCSP Report were incorrect, and
>> that there is a fundamental discrepancy between simulated and observed
>> temperature changes in the tropical troposphere. Douglass et al. also
>> assert that models cannot represent the "observed" differential
>> warming of the surface and troposphere. To address these claims, I've
>> been updating some of the comparisons of models and observations that
>> we did for the CCSP Report, now using newer observational datasets
>> (among them NOAA ERSST-v2 and v3). As you can see from the forwarded
>> email, the warming rates of tropical SSTs are somewhat different for
>> ERSST-v2 and v3 - ERSST-v3 warms by less than v2. Do you understand
>> why this is?
>>
>> With best regards, and hope you are well!
>>
>> Ben
>> ----------------------------------------------------------------------------
>>
>> Benjamin D. Santer
>> Program for Climate Model Diagnosis and Intercomparison
>> Lawrence Livermore National Laboratory
>> P.O. Box 808, Mail Stop L-103
>> Livermore, CA 94550, U.S.A.
>> Tel: (925) 422-2486
>> FAX: (925) 422-7675
>> email: santer1@llnl.gov
>> ----------------------------------------------------------------------------
>>
>>
>>
>> X-Account-Key: account1
>> Return-Path:
>> Received: from mail-2.llnl.gov ([unix socket])
>> by mail-2.llnl.gov (Cyrus v2.2.12) with LMTPA;
>> Wed, 13 Feb 2008 18:34:52 -0800
>> Received: from smtp.llnl.gov (nspiron-3.llnl.gov [128.115.41.83])
>> by mail-2.llnl.gov (8.13.1/8.12.3/LLNL evision: 1.6 $) with
>> ESMTP id m1E2YMTv008791;
>> Wed, 13 Feb 2008 18:34:52 -0800
>> X-Attachments: LAST_IJC_figure04.pdf
>> X-IronPort-AV: E=McAfee;i="5200,2160,5229"; a="26979778"
>> X-IronPort-AV: E=Sophos;i="4.25,349,1199692800";
>> d="pdf'?scan'208";a="26979778"
>> Received: from dione.llnl.gov (HELO [128.115.57.29]) ([128.115.57.29])
>> by smtp.llnl.gov with ESMTP; 13 Feb 2008 18:34:51 -0800
>> Message-ID: <47B3A8CB.90605@llnl.gov>
>> Date: Wed, 13 Feb 2008 18:34:51 -0800
>> From: Ben Santer
>> Reply-To: santer1@llnl.gov
>> Organization: LLNL
>> User-Agent: Thunderbird 1.5.0.12 (X11/20070529)
>> MIME-Version: 1.0
>> To: santer1@llnl.gov, Peter Thorne ,
>> Stephen Klein ,
>> Susan Solomon ,
>> John Lanzante ,
>> Melissa Free ,
>> Dian Seidel , Tom Wigley
>> ,
>> Karl Taylor ,
>> Thomas R Karl , Carl Mears
>> ,
>> "David C. Bader" ,
>> "'Francis W. Zwiers'" ,
>> Frank Wentz ,
>> Leopold Haimberger ,
>> "Michael C. MacCracken" ,
>> Phil Jones ,
>> Steve Sherwood ,
>> Tim Osborn ,
>> Gavin Schmidt ,
>> "Hack, James J." , peter gleckler
>>
>> Subject: Additional calculations
>> References: <200801121320.26705.John.Lanzante@noaa.gov>
>> <478C528C.8010606@llnl.gov>
>> <478EC287.8030008@llnl.gov>
>> <1200567390.8038.35.camel@eld443.desktop.frd.metoffice.com>
>> <7.0.1.0.2.20080117140720.022259c0@llnl.gov>
>> <1200995209.23799.95.camel@eld443.desktop.frd.metoffice.com>
>> <47962FD1.1020303@llnl.gov>
>> In-Reply-To: <47962FD1.1020303@llnl.gov>
>> Content-Type: multipart/mixed;
>> boundary="------------060600010907080200090109"
>>
>> Dear folks,
>>
>> Sorry about the delay in sending you the next version of our
>> manuscript. I decided that I needed to perform some additional
>> calculations. I was concerned that we had not addressed the issue of
>> "differential warming" of the surface and troposphere - an issue which
>> Douglass et al. HAD considered.
>>
>> Our work thus far shows that there are no fundamental inconsistencies
>> between simulated and observed temperature trends in individual
>> tropospheric layers (T2 and T2LT). But we had not performed our
>> "paired trends" test for trends in the surface-minus-T2LT difference
>> time series. This is a much tougher test to pass: differencing
>> strongly damps the correlated variability in each "pair" of surface
>> and T2LT time series. Because of this noise reduction, the standard
>> error of the linear trend in the difference series is typically
>> substantially smaller than the size of the standard error in an
>> individual surface or T2LT time series. This makes it easier to reject
>> the null hypothesis of "no significant difference between simulated
>> and observed trends".
>>
>> In the CCSP Report, the behavior of the trends in the
>> surface-minus-T2LT difference series led us to note that:
>>
>> "Comparing trend differences between the surface and the troposphere
>> exposes potential discrepancies between models and observations in the
>> tropics".
>>
>> So it seemed wise to re-examine this "differential warming" issue. I
>> felt that if we ignored it, Douglass et al. would have grounds for
>> criticizing our response.
>>
>> I've now done the "paired trends" test with the trends in the
>> surface-minus-T2LT difference series. The results are quite
>> interesting. They are at variance with the above-quoted finding of the
>> CCSP Report. The new results I will describe show that the "potential
>> discrepancies" in the tropics have largely been resolved.
>>
>> Here's what I did. I used three different observational estimates of
>> tropical SST changes. These were from NOAA-ERSST-v2, NOAA-ERSST-v3,
>> and HadISST1. It's my understanding that NOAA-ERSST-v3 and HadISST1
>> are the most recent SST products of NCDC and the Hadley Centre. I'm
>> also using T2LT data from RSS v3.0 and UAH v5.2. Here are the tropical
>> (20N-20S) trends in these five datasets over the 252-month period from
>> January 1979 to December 1999, together with their 1-sigma adjusted
>> standard errors (in brackets):
>>
>> UAH v5.2 0.060 (+/-0.137)
>> RSS v3.0 0.166 (+/-0.130)
>> HADISST1 0.108 (+/-0.133)
>> NOAA-ERSST-v2 0.100 (+/-0.131)
>> NOAA-ERSST-v3 0.077 (+/-0.121)
>>
>> (all trends in degrees C/decade).
>>
>> The trends in the three SST datasets are (by definition) calculated
>> from anomaly data that have been spatially-averaged over tropical
>> oceans. The trends in T2LT are calculated from anomaly data that have
>> been spatially averaged over land and ocean. It is physically
>> reasonable to do the differencing over different domains, since the
>> temperature field throughout the tropical troposphere is more or less
>> on the moist adiabatic lapse rate set by convection over the warmest
>> waters.
>>
>> These observational trend estimates are somewhat different from those
>> available to us at the time of the CCSP Report. This holds for both
>> T2LT and SST. For T2LT, the RSS trend used in the CCSP Report and in
>> the Santer et al. (2005) Science paper was roughly 0.13 degrees
>> C/decade. As you can see from the Table given above, it is now ca.
>> 0.17 degrees C/decade. Carl tells me that this change is largely due
>> to a change in how he and Frank adjust for inter-satellite biases.
>> This adjustment now has a latitudinal dependence, which it did not
>> have previously.
>>
>> The tropical SST trends used in the CCSP Report were estimated from
>> earlier versions of the Hadley Centre and NOAA SST data, and were of
>> order 0.12 degrees C/decade. The values estimated from more recent
>> datasets are lower - and markedly lower in the case of NOAA-ERSST-v3
>> (0.077 degrees C/decade). The reasons for this downward shift in the
>> estimated warming of tropical SSTs are unclear. As Carl pointed out in
>> an email that he sent me earlier today:
>>
>> "One important difference is that post 1985, NOAA-ERSST-v3 directly
>> ingests "bias adjusted" SST data from AVHRR, a big change from v2,
>> which didn't use any satellite data (directly). AVHRR is strongly
>> affected in the tropics by the Pinatubo eruption in 1991. If the
>> "bias adjustment" doesn't completely account for this, the trends
>> could be changed".
>>
>> Another possibility is treatment of biases in the buoy data. It would
>> be nice if Dick Reynolds could advise us as to the most likely
>> explanation for the different warming rates inferred from
>> NOAA-ERSST-v2 and v3.
>>
>> Bottom line: The most recent estimates of tropical SST changes over
>> 1979 to 1999 are smaller than we reported in the CCSP Report, while
>> the T2LT trend (at least in RSS) is larger. The trend in the observed
>> difference series, NOAA-ERSST-v3 Ts minus RSS T2LT, is now -0.089
>> degrees C/decade, which is very good agreement with the multi-model
>> ensemble trend in the Ts minus T2LT difference series (-0.085 degrees
>> C/decade). Ironically, if Douglass et al. had applied their flawed
>> "consistency test" to the multi-model ensemble mean trend and the
>> trend in the NOAA-ERSST-v3 Ts minus RSS T2LT difference series, they
>> would not have been able to conclude that models and observations are
>> inconsistent!
>>
>> Here are the observed trends in the tropical Ts minus T2LT difference
>> series in the six different pairs of Ts and T2LT datasets, together
>> with the number of "Hits" (rejections of the null hypothesis of no
>> significant difference in trends) and the percentage rejection rate
>> (based on 49 tests in each case)
>>
>> "Pair" Trend 1-sigma C.I. Hits Rej.Rate
>> HadISST1 Ts minus RSS T2LT -0.0577 (+/-0.0347) 1 (2.04%)
>> NOAA-ERSST-v2 Ts minus RSS T2LT -0.0660 (+/-0.0382) 1 (2.04%)
>> NOAA-ERSST-v3 Ts minus RSS T2LT -0.0890 (+/-0.0350) 0 (0.00%)
>> HadISST1 Ts minus UAH T2LT +0.0488 (+/-0.0371) 28 (57.14%)
>> NOAA-ERSST-v2 Ts minus UAH T2LT +0.0405 (+/-0.0403) 25 (51.02%)
>> NOAA-ERSST-v3 Ts minus UAH T2LT +0.0175 (+/-0.0370) 15 (30.60%)
>> Multi-model ensemble mean -0.0846
>>
>> Things to note:
>>
>> 1) For all "pairs" involving RSS T2LT data, the multi-model ensemble
>> mean trend is well within even the 1-sigma statistical uncertainty of
>> the observed trend.
>>
>> 2) For all "pairs" involving RSS T2LT data, there are very few
>> statistically-significant differences between the observed and
>> model-simulated "differential warming" of the tropical surface and
>> lower troposphere.
>>
>> 3) For all "pairs" involving UAH T2LT data, there are
>> statistically-significant differences between the observed and
>> model-simulated "differential warming" of the tropical surface and
>> lower troposphere. Even in these cases, however, rejection of the null
>> hypothesis is not universal: rejection rates range from 30% to 57%.
>> Clearly, not all models are inconsistent with the observational
>> estimate of "differential warming" inferred from UAH data.
>>
>> These results contradict the "model inconsistent with data" claims of
>> Douglass et al.
>>
>> The attached Figure is analogous to the Figure we currently show in
>> the paper for T2LT trends. Now, however, results are for trends in the
>> surface-minus-T2LT difference series. Rather than showing all six
>> "pairs" of observational results in the top panel, I've chosen to show
>> two pairs only in order to avoid unnecessarily complicating the
>> Figure. I propose, however, that we provide results from all six pairs
>> in a Table.
>>
>> As is visually obvious from the Figure, trends in 46 of the 49
>> simulated surface-minus-T2LT difference series pairs are within the
>> 2-sigma confidence intervals of the NOAA-ERSST-v3 Ts minus RSS T2LT
>> trend (the light grey bar). And as is obvious from Panel B, even the
>> Douglass et al. "sigma{SE}" encompasses the difference series trend
>> from the NOAA-ERSST-v3 Ts/RSS T2LT pair.
>>
>> I think we should show these results in our paper.
>>
>> The bottom line: Use of newer T2LT datasets (RSS) and Ts datasets
>> (NOAA-ERSST-v3, HADISST1) largely removes the discrepancy between
>> tropical surface and tropospheric warming rates. We need to explain
>> why the observational estimates of tropical SST changes are now
>> smaller than they were at the time of the CCSP Report. We will need
>> some help from Dick Reynolds with this.
>>
>> With best regards,
>>
>> Ben
>> ----------------------------------------------------------------------------
>>
>> Benjamin D. Santer
>> Program for Climate Model Diagnosis and Intercomparison
>> Lawrence Livermore National Laboratory
>> P.O. Box 808, Mail Stop L-103
>> Livermore, CA 94550, U.S.A.
>> Tel: (925) 422-2486
>> FAX: (925) 422-7675
>> email: santer1@llnl.gov
>> ----------------------------------------------------------------------------
>>
>>
>>
>
> Prof. Phil Jones
> Climatic Research Unit Telephone +44 (0) 1603 592090
> School of Environmental Sciences Fax +44 (0) 1603 507784
> University of East Anglia
> Norwich Email p.jones@uea.ac.uk
> NR4 7TJ
> UK
> ----------------------------------------------------------------------------
>
>
>
> ------------------------------------------------------------------------
>
> program growlandmergeetc
> dimension lnd(72,36),nlnd(72,36),ivsst(72,36),jcov(72,36)
> dimension icmb(72,36),alcov(72,36),ascov(72,36),iysst(72,36)
> dimension isdvar(72,36,12),neigsd(72,36,12)
> dimension iorigt(72,36),icount(72,36)
> dimension ash(12),anh(12),ashp(12),anhp(12)
> dimension np(12),npch(12),npinf(12),npchan(12),npsst(12)
> rad=57.2958
> ir=13
> c calculate maximum % coverage of hemisphere in cos units
> xnh=0.0
> do 20 j=1,18
> w=cos((92.5-j*5)/rad)
> do 19 i=1,72
> 19 xnh=xnh+w
> 20 continue
> c read in land fraction in %
> read(1,21)i1,i2
> 21 format(2i6)
> do 22 j=1,36
> 22 read(1,23)(jcov(i,j),i=1,72)
> 23 format(72i6)
> c set coverage of land to % of at least 25% and less than 75%
> c ocean percent is then simply the rest
> do 24 j=1,36
> do 24 i=1,72
> alcov(i,j)=0.01*jcov(i,j)
> if(alcov(i,j).le.24.9)alcov(i,j)=25.0
> if(alcov(i,j).ge.75.1)alcov(i,j)=75.0
> ascov(i,j)=100.0 - alcov(i,j)
> 24 continue
> c read in the sd of the land only datset (var corected) to assess
> c whether the neighbour check can legitimately correct values
> do 901 k=1,12
> read(4,27)ii
> do 902 j=1,36
> 902 read(4,29)(isdvar(i,j,k),i=37,72),(isdvar(ii,j,k),ii=1,36)
> 901 continue
> c read in neighbouring sd calculated from at least 4 of the
> c neigbouring 8 5 degree squares around each grid box
> do 903 k=1,12
> read(18,27)ii
> do 904 j=1,36
> 904 read(18,29)(neigsd(i,j,k),i=37,72),(neigsd(ii,j,k),ii=1,36)
> 903 continue
> c skip the first 19 years of the variance corrected land data
> c as the variance corrected SST data only starts in
> c also skip the first 19 years of the original gridded temps
> c so later can check the number of stations available per gridbox
> c per month
> do 25 k=1851,1869
> do 26 kk=1,12
> read(2,27)i1,i2
> 27 format(2i5)
> read(ir,27)i1,i2
> do 28 j=1,36
> 28 read(2,29)(lnd(i,j),i=37,72),(lnd(ii,j),ii=1,36)
> 29 format(12i5)
> do 128 j=1,36
> 128 read(ir,29)(iorigt(i,j),i=37,72),(iorigt(ii,j),ii=1,36)
> do 129 j=1,36
> 129 read(ir,29)(icount(i,j),i=37,72),(icount(ii,j),ii=1,36)
> 26 continue
> 25 continue
> c read in the land and sst data (both variance corrected)
> c reading in the land allow for the greenwich start of the land
> c and the dateline start for the SST. Output is from the dateline
> do 31 k=1870,1999
> ashy=0.0
> anhy=0.0
> if(k.ge.1901)ir=14
> if(k.ge.1951)ir=15
> if(k.ge.1991)ir=16
> if(k.ge.1994)ir=17
> do 32 kk=1,12
> npch(kk)=0
> npchan(kk)=0
> np(kk)=0
> npinf(kk)=0
> npsst(kk)=0
> c read in the original gridded land to get the station count
> c per grid box
> read(ir,27)i1,i2
> do 131 j=1,36
> 131 read(ir,29)(iorigt(i,j),i=37,72),(iorigt(ii,j),ii=1,36)
> do 132 j=1,36
> 132 read(ir,29)(icount(i,j),i=37,72),(icount(ii,j),ii=1,36)
> c read in the variance corrected land
> read(2,27)i1,i2
> write(7,27)kk,k
> do 33 j=1,36
> 33 read(2,29)(lnd(i,j),i=37,72),(lnd(ii,j),ii=1,36)
> c copy lnd array to nlnd so that the growing doesn't use already
> c infilled values
> do 34 j=1,36
> do 34 i=1,72
> 34 nlnd(i,j)=lnd(i,j)
> c read in sst data
> read(3,21)i1,i2
> do 35 j=1,36
> 35 read(3,23)(ivsst(i,j),i=1,72)
> c check land for extremes and fill in gaps (only one grid box away
> c provided there are at least 4 of the 8 surrounding boxes)
> do 41 j=1,36
> j1=j-1
> j2=j+1
> if(j1.eq.0)j1=1
> if(j2.eq.37)j2=36
> do 42 i=1,72
> sum=0.0
> nsum=0
> i1=i-1
> i2=i+1
> do 43 jj=j1,j2
> do 44 ii=i1,i2
> iii=ii
> if(iii.eq.73)iii=1
> if(iii.eq.0)iii=72
> if(jj.eq.j.and.iii.eq.i)go to 44
> if(lnd(iii,jj).eq.-9999)go to 44
> sum=sum+lnd(iii,jj)
> nsum=nsum+1
> 44 continue
> 43 continue
> if(lnd(i,j).ne.-9999)np(kk)=np(kk)+1
> if(nsum.le.3)go to 47
> sum=sum/nsum
> ndep=sum+0.5
> if(sum.lt.0.0)ndep=ndep-1
> nval=ndep
> if(lnd(i,j).eq.-9999)go to 46
> npch(kk)=npch(kk)+1
> ndep=lnd(i,j)-nval
> if(neigsd(i,j,kk).eq.-9999)go to 47
> if(iabs(ndep).le.225)go to 47
> if(iabs(ndep).lt.neigsd(i,j,kk)*2.0)go to 47
> if(icount(i,j).ge.2)go to 47
> nlnd(i,j)=nval
> npchan(kk)=npchan(kk)+1
> 48 write(6,202)k,kk,j,i,nval,lnd(i,j),ndep,isdvar(i,j,kk),
> >neigsd(i,j,kk),nlnd(i,j),nsum,icount(i,j),iorigt(i,j)
> 202 format(4i4,9i6)
> go to 47
> 46 nlnd(i,j)=nval
> npinf(kk)=npinf(kk)+1
> 47 continue
> 42 continue
> 41 continue
> c merge with marine using the weighting factors
> do 51 j=1,36
> do 52 i=1,72
> wx=0.0
> xx=0.0
> if(nlnd(i,j).eq.-9999)go to 55
> wx=wx+alcov(i,j)
> xx=xx+alcov(i,j)*nlnd(i,j)
> 55 if(ivsst(i,j).eq.-32768)go to 56
> wx=wx+ascov(i,j)
> xx=xx+ascov(i,j)*ivsst(i,j)
> 56 if(wx.ge.0.001)go to 59
> icmb(i,j)=-9999
> go to 57
> 59 aa=xx/wx
> ia=aa+0.5
> if(xx.lt.0.0)ia=ia-1
> icmb(i,j)=ia
> c writing out the land/sst merging checking when both are present
> c if(wx.ge.99.9)write(6,203)kk,j,i,ia,nlnd(i,j),ivsst(i,j),
> c >wx,alcov(i,j),ascov(i,j)
> c 203 format(6i6,3f7.1)
> 57 continue
> 52 continue
> 51 continue
> c write out the new merged file
> do 53 j=1,36
> 53 write(7,54)(icmb(i,j),i=1,72)
> 54 format(12i5)
> c calculate the hemispheric averages
> anh(kk)=0.0
> ash(kk)=0.0
> ashp(kk)=0.0
> anhp(kk)=0.0
> wx=0.0
> xx=0.0
> do 61 j=1,18
> w=cos((92.5-j*5.0)/rad)
> do 62 i=1,72
> if(icmb(i,j).eq.-9999)go to 62
> wx=wx+w
> xx=xx+w*icmb(i,j)
> 62 continue
> 61 continue
> anh(kk)=xx*0.01/wx
> anhp(kk)=wx*100.0/xnh
> wx=0.0
> xx=0.0
> do 63 j=19,36
> w=cos((j*5.0-92.5)/rad)
> do 64 i=1,72
> if(icmb(i,j).eq.-9999)go to 64
> wx=wx+w
> xx=xx+w*icmb(i,j)
> 64 continue
> 63 continue
> ash(kk)=xx*0.01/wx
> ashp(kk)=wx*100.0/xnh
> anhy=anhy+anh(kk)
> ashy=ashy+ash(kk)
> 32 continue
> anhy=anhy/12.0
> ashy=ashy/12.0
> write(8,89)k,anh,anhy
> 89 format(i4,12f6.2,f7.2)
> write(8,90)k,anhp
> 90 format(i4,12f6.0)
> write(9,89)k,ash,ashy
> write(9,90)k,ashp
> write(10,91)k,np
> write(10,91)k,npch
> write(10,91)k,npchan
> write(10,91)k,npinf
> write(10,92)
> 92 format(/)
> 91 format(i4,12i6)
> 31 continue
> stop
> end
>
>


--
----------------------------------------------------------------------------
Benjamin D. Santer
Program for Climate Model Diagnosis and Intercomparison
Lawrence Livermore National Laboratory
P.O. Box 808, Mail Stop L-103
Livermore, CA 94550, U.S.A.
Tel: (925) 422-2486
FAX: (925) 422-7675
email: santer1@llnl.gov
----------------------------------------------------------------------------