ࡱ; ~YZ  !"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWX]\[qp^_`abcdefghijklmnorstuwvxyz}{|Root Entry Fқ@.CompObjbWordDocument5!ObjectPool ɛɛ 4@  $&'()*+,-./01367:<=>?@ABCDEFGHIJKLMORSTWYZ[\]^_`abcdefghijklmnopqrstuvwxy{~ FMicrosoft Word 6.0 Document MSWordDocWord.Document.6; ࡱ; L{h hࡱ; {O ^ .1  @@& & MathTypepSymbol- 2 `7ܥe- e!#.4444,,,T66666N.|6v ^n"  )<<X !T\!^v ,hov 4(,,$44v TEMPORAL RELIABILITY OF ESTIMATES FROM CONTINGENT VALUATION Richard T. Carson, W. Michael Hanemann, Raymond J. Kopp, Jon A. Krosnick, Robert C. Mitchell, Stanley Presser, Paul A. Ruud, and V. Kerry Smith with Michael Conaway and Kerry Martin* Revised July 24, 1996 *The authors are, respectively: Associate Professor of Economics, University of California (San Diego); Professor of Agricultural and Natural Resource Economics, University of California (Berkeley); Senior Fellow, Resources for the Future; Associate Professor of Psychology and Political Science, Ohio State University; Professor of Geography, Clark University; Professor of Sociology, University of Maryland (College Park); Professor of Economics, University of California (Berkeley); and Arts and Sciences Professor, Duke University and University Fellow, Resources for the Future. Conaway and Martin are members of Natural Resource Damage Assessment, Inc. and made extensive contributions throughout the effort. The work described in this paper was funded by the Damage Assessment Office of the National Oceanic and Atmospheric Administration as part of a natural resource damage assessment under contract number 50-DGNC-1-00007. Additional support to aid in the preparation of this paper was provided Smith by the UNC Sea Grant Program under Grant No. R/MRD-. Thanks to Richard Bishop, Trudy Cameron, Nicholas Flores, Carol Jones, Norman Meade, Pierre Du Vair, Alan Randall, and two anonymous referees for comments on aspects of this work. All opinions expressed in this paper are those of the authors and should not be attributed to the National Oceanic and Atmospheric Administration, the Alfred P. Sloan Foundation, or any persons or organizations acknowledged above. Temporal Reliability of Estimates from Contingent Valuation Richard T. Carson, W. Michael Hanemann, Raymond J. Kopp, Jon A. Krosnick, Robert C. Mitchell, Stanley Presser, Paul A. Ruud, and V. Kerry Smith with Michael Conaway and Kerry Martin Abstract In 1992 the National Oceanic and Atmospheric Administration (NOAA) convened a panel of prominent social scientists to assess the reliability of natural resource damage estimates derived from contingent valuation (CV). The product of the panel's deliberations was a report that laid out a set of recommended guidelines for CV survey design, administration, and data analysis. This paper focuses on one of these guidelines -- the Panel's call for the "temporal averaging" of willingness-to-pay (WTP) responses obtained from CV surveys as one method for increasing their reliability. The panel suggested: "Time dependent measurement noise should be reduced by averaging across independently drawn samples taken at different points in time. A clear and substantial time trend in the responses would cast doubt on the 'reliability' of the finding." The purpose of this paper is to examine the temporal reliability of CV estimates. Our findings, using a CV instrument designed to measure willingness-to-pay for a program to protect Prince William Sound, Alaska from future oil spills, like the Exxon Valdez spill, exhibited no significant sensitivity to the timing of the interviews. For two samples involving independent interviews taken over two years apart, the distribution of respondents' choices "for" and "against" the protection program did not differ. Key Words: contingent valuation, natural resource damages, passive use, Exxon Valdez, reliability JEL Classification No(s).: D60, D61, K32, Q28 1. INTRODUCTION Over the past two decades the use of contingent valuation (CV) in policy analysis and academic research has grown rapidly. According to one estimate there are now almost two thousand studies in the literature dealing with CV. Special attention has focused on its use in estimating passive use value because indirect methods can only measure use related values.  While there is a substantial literature describing the theoretical foundation for nonuse or passive use values (e.g., Krutilla [1967], Plourde [1975], McConnell [ 1983]), the prospect of routinely including estimates for these losses in natural resource damages has generated considerable controversy. The 1989 Court of Appeals ruling in Ohio v. Department of the Interior held that lost passive use values should be included in damage awards resulting from injuries to natural resources due to releases of hazardous substances. Under this decision, it is unnecessary for an individual to be a direct user of a natural resource, for example as a recreationist, to hold an economic value for the resource in question. The Ohio Court also emphasized the importance of the "reliability" of the methods used to estimate natural resource damages. Because contingent valuation is currently the only technique available to measure economic values that include use and passive use, much of the current CV research has been directed at evaluating its reliability. To assess the reliability of natural resource damage estimates derived from CV, the National Oceanic and Atmospheric Administration (NOAA) convened a panel of prominent social scientists. The Panel's report concluded that: . . . under those conditions (and others specified above), CV studies convey useful information. We think it is fair to describe such information as reliable by the standards that seem to be implicit in similar contexts, like market analysis for new and innovative products and the assessment of other damages normally allowed in court proceedings. The Panel's "conditions" are a set of guidelines for CV survey design, administration, and data analysis. This paper focuses on one of these guidelines -- the Panel's call for the "temporal averaging" of willingness-to-pay (WTP) responses obtained from CV surveys as one method for increasing their reliability. The Panel suggested: Time dependent measurement noise should be reduced by averaging across independently drawn samples taken at different points in time. A clear and substantial time trend in the responses would cast doubt on the "reliability" of the findings. (P. 4609) The reasoning underlying the NOAA Panel's recommendation for temporal averaging is not clear. Measurement error can be reduced by averaging across multiple observations that are assumed to be realizations from the same underlying stochastic process. The Panel's report raises concern about the existence of time dependency in the location or scale parameters for CV constructed measures of willingness to pay. However each individual's economic value for a commodity should be expected to change with the conditions that influence any choice. In general, the prices (and availability) of substitutes and complements, level of income, and all other factors that would affect these decisions can be expected to be determinants of measures of economic values. Thus, changes in estimates of economic value, alone, are not likely to be the source of the NOAA Panel's call for attention to the temporal reliability of CV. Rather it may represent a concern over specific timing of the survey in relation to an event, such as a large oil spill, that might be subject to heightened unreliability. This hypothesis could be explained because respondents require time to evaluate the full implications of the event or because public reactions could initially entail outrage directed at the party thought to be responsible. With time, such short term responses may be often modified as more information about the cause, and the full consequences of the event becomes known. Their suggestion might be treated as a concern over the timing of a single CV survey in relation to the event giving rise to natural resource injuries. In this context temporal averaging would not improve the estimates. Their hypothesis implies WTP estimates constructed from one set of responses would be superior to those at the times that are more subject to these short term influences. Given such concerns, it is important to distinguish research on the stability of CV estimates of WTP over time from a recommendation to average the estimates for increased reliability. The alternative hypotheses providing reasons for focusing research on temporal stability all suggest temporal averaging would not improve the properties of CV estimates. Our findings use a CV instrument designed to measure WTP for a program to protect Prince William Sound, Alaska from future oil spills, like the Exxon Valdez spill. These results indicate that choices made two years after the spill are not significantly different from what are made four years after the spill. 2. TESTING THE TEMPORAL VOLATILITY HYPOTHESIS On March 24, 1989, the oil tanker Exxon Valdez left the port of Valdez, on its way to the Gulf of Alaska. It ran into the submerged rocks of Bligh Reef releasing some 11 million gallons of Prudoe Bay crude oil into the waters of Prince William Sound. As part of its damage assessment, the State of Alaska funded a CV study [Carson, et al. 1992] designed to measure the passive use losses due to the spill. With few exceptions, that study followed the survey design and administration procedures subsequently recommended by the NOAA CV Panel. The Exxon Valdez spill together with the Carson et al. study offer a unique opportunity to investigate the question posed by the NOAA Panel. By conducting a comparable analysis four years after the spill we can investigate whether the timing of this initial study was within the interval the Panel implicitly suspected could be problematic. To undertake our analysis, we compare the results of the original national face-to-face survey conducted from January to mid-April 1991 with those of a follow-up, face-to-face survey conducted in 1993 two years later, using the identical questionnaire and a comparable sample. Because of the complexity of each study and the importance of the design and survey administration to the issue of reliability, we discuss each study separately. After four field pilot tests, the original Exxon Valdez damage assessment survey was placed into the field in January of 1991, 22 months after the spill. The field administration of the survey was conducted by Westat of Rockville, Maryland using a multi-stage area probability sample of residential dwelling units (DU) drawn from the 50 United States and the District of Columbia. The Primary Sampling Units (PSUs) consisted of Westat's National Master Sample supplemented by the Honolulu SMSA. Within each of the 61 PSUs, the second-stage selections were drawn from a list of all the Census blocks in the PSU. The lists were stratified by two block characteristics: percent of the population that was black, and a weighted average of the value of owner-occupied housing and the rent of renter-occupied housing. The 334 secondary selections were then drawn with probabilities proportionate to their total population counts. In the third stage, approximately 1,600 dwelling units were drawn from the selected blocks. Within each dwelling unit, a household member 18 or older who owned, rented or paid toward the mortgage or rent was selected at random to be the respondent. The overall response rate for the original study was 75.2 percent, yielding a sample of 1,043 cases. Our second survey was conducted by the National Opinion Research Center (NORC) of the University of Chicago as part of an empirical study involving 1,408 interviewed households. Three hundred of these respondents received the original Alaska questionnaire and visuals. The remaining 1,108 households received versions of the original Alaska instrument that were modified to examine other issues not relevant to this study. The sample was composed of 12 PSUs selected from NORC's master area probability sample: Baltimore, MD; Birmingham, AL; Boston, MA; Charleston, SC; Harrisburg, PA; Ft. Wayne, IN; Manchester, NY; Nicholas County, KY; Portland, OR; Richmond, VA; Seattle, WA; and Tampa, FL. Six segments were selected from each PSU, resulting in 72 segments. 1,925 dwelling units were then randomly selected from the 72 segments. NORC's sampling staff then randomly assigned one of four interview versions of the questionnaires comprising our larger study to each selected dwelling unit in advance of the field period. The selection of the respondent for the interview was made from all individuals in the household meeting the same eligibility requirements as with the original 1991 Exxon Valdez survey. The interviews for this study were conducted over an eight-week period from May 26 to July 17, 1993 and the overall response rate was 73 percent. As in the original survey, non-English speaking households were ineligible for the survey. Due to differences in how PSUs were drawn in the first stage of sample selection, the original 1991 sample and the 1993 sample are not fully equivalent. In the 1991 sample, the first stage PSU selection followed a full probability selection scheme. The 12 PSUs in the 1993 sample were selected from NORC's master list by choosing PSUs where NORC had sufficient interviewers to conduct the study. In all subsequent stages of sample selection (i.e., choosing Census blocks, dwelling units, and respondents), the samples were drawn with identical procedures. One effect of the difference in the first stage sampling was to exclude the major metropolitan areas of New York, Philadelphia, Chicago, and Los Angeles (included in the 1991 sample) from the 1993 sample. Since the first stage sampling differs in the 1991 and 1993 samples, we provide two different procedures to adjust for sample differences. Section four presents results based on a choice function, conventionally used in tests of construct validity (Mitchell and Carson [1989]). The specification for this function was based on the construct validity test with the 1991 sample. We use this function to test for differences in the parameters associated with the factors influencing choices with the two samples. In addition, we replicated all of the analyses reported in this paper using a sub-sample of the 1991 sample that excluded the following PSUs: Bronx/Manhattan, NY; Kings/Queens/Richmond, NY; Nassau/Suffolk, NY; Philadelphia, PA; Chicago, IL; Los Angeles, CA. None of the test outcomes are changed when using this sub-sample. Therefore, we focus our discussion on analyses that compare the full 1991 sample with our 1993 replication. 3. RESULTS The questionnaire uses a referendum value elicitation format. Respondents were asked to vote on a program that, for the next ten years, would protect Prince William Sound from another oil spill causing natural resource injuries comparable to those from the Exxon Valdez spill. Questions were also asked in a double-bounded format so that if the respondent said she voted "for" the protection program then she was asked about a higher one-time cost question. Respondents answering "against" or "not sure" to the first amount were offered the program at a lower amount. Four versions of the base survey questionnaire, differing only in the amounts used in these two questions, were administered. Tests for the effects of the timing of the initial Alaska survey were undertaken in three ways: simple contingency analyses with both the first and the second response; analysis of the estimated parameters for the choice functions from each sample; and estimates of the WTP from each sample. We consider each in turn. Results for Contingency Table The first panel in Table 1 reports the percentage of respondents voting "for" or "against" adoption of the protection program based on the first question. The table displays the percentages for the two surveys, for each of the four dollar amounts used. Simple inspection of the distributions confirms that the timing of the initial Alaska survey was not impacted by its proximity to the incident. The identical survey conducted two years later provides equivalent results. The null hypothesis of equal proportions voting "for" and "against" the plan is tested four ways with these choices. Using the first question we consider: (a) votes with "don't know" and "not sure" recoded as against;  (b) "don't know" and "not sure" treated as a separate category so three responses are allowed (i.e., "for", "against" and "don't know/not sure") and (c) deleting the "don't know/not sure" responses. The next three columns in Table 1 report the chi square statistics for each possible interpretation of the choices reported with each dollar amount. None would permit rejection of the null hypothesis of equal proportions in the categories identifying the respondents' choices. The last column in Table 1 presents the results using choices from the first and second voting questions. There are four possible voting patterns based on both response questions -- for-for, for-against, against-for, and against-against. The null hypothesis of equal distribution between the two surveys can be rejected only at the $30 amount. Results for Choice Function Three estimators for the choice function were used in testing consistency as part of construct validity tests for respondents' choices in the two samples. Both probit and Weibull survival models were applied to the responses from the first question. In addition, we used the responses to both questions to develop interval censored estimates of a WTP function (i.e., the so-called double-bounded model, see Hanemann, Kanninen and Loomis [1991]) and again used a Weibull framework to evaluate the factors influencing the choices used in estimating this equation. Each of the estimators has quite different implicit assumptions. The probit was estimated in terms of the level of the tax amount (and thus is consistent with a linear random utility or WTP specification, see McConnell [1990]). It does not constrain the probability of favoring the program to unity as the tax amount declines to zero. The Weibull's location parameter assumes a model that implies independent variables in linear form will shift the log of median (or mean) WTP. It also constrains the probability to vote "for" the program to be unity when the proposed tax amount is zero. The double-bounded estimator is perhaps the most controversial approach in that it relies on the responses to both questions being governed by the same underlying probability distribution. Cameron and Quiggin [1994] have suggested violations in this assumption can bias the estimates of WTP and of the parameters in the WTP function used to describe the choices. Our primary concern here is with the consistency in the overall conclusions from fitting these models to both samples. Table 2 defines the independent variables included in all choice models. These factors correspond to the regressors selected for the original 1991 survey (see Carson et al. [1992] for a more complete discussion). Because this analysis seeks to evaluate whether replication would change conclusions about choices, we did not consider alternative specifications. Table 3 presents the probit and survival function estimates. The data from the two surveys are pooled for a total of 1,144 observations. The first column of Table 3 presents the probit results. Standard errors are shown in parentheses beside the coefficients. The variable labeled as "1993" identifies the replication sample as an intercept shift. It is not significantly different from zero, implying that under the assumption of common slope parameters, there is no shift in the choice model. The second and third columns of Table 3 imply the same conclusion, using the single and double-bounded Weibull survival models. Table 4 presents the results of relaxing the common slope parameter assumption. Each of the three models presented (probit, single-bounded survival, and double-bounded survival) contain a 1993 intercept shifting variable and interaction dummy variables (denoted N variable) for each of the independent variables (to allow testing for differences in each parameter between the two samples). The 1993 intercept shifting variable is again insignificant in all three models. With the probit and single-bounded survival estimates, only the COASTAL interaction slope effect would be judged significantly different from zero at the five percent level. Estimates for the double-bounded model imply none of the slope parameters are significantly different between the two choice functions for the two samples. Overall, then, the determinants of choices in the samples separated by two years remained stable. Willingness-to-Pay Estimates Our estimates for the mean WTP use the Turnbull [1976] non-parametric estimator based on interval censored data along with Carson et al.'s [1994] method for estimating a lower bound for the mean of the underlying WTP distribution. Assuming referendum questions with a single take-it or leave-it decision, the design of responses over proposed costs, tj allows respondents to be sorted into two groups for each cost (or tax amount). This allows the distribution function to be defined as:  EMBED Equation.2   EMBED Equation.2  EMBED Equation.2  To develop a maximum likelihood estimator for the distribution function we need only the frequences in each cell. The log-likelihood function, l, is given in equation (1).  EMBED Equation.2  (1) where Nj = number of respondents indicating "against" program at tax amount tj. Yj = number of respondents indicating "for" program at tax amount tj k = number of values for tj The lower-bound estimate of mean WTP is defined in equation (2).   EMBED Equation.2 (2) The unobserved mean is bounded from below by the estimated lower-bound mean and from above by the estimated upper-bound mean. The Turnbull lower-bound mean estimate from the 1991 sample using responses to the first voting question, is $52.80 with a standard error of $2.12. The comparable estimate for the 1993 sample is $52.81 with a standard error of $4.08. Whether or not we adjust for the effects of changes in the general price level over this time, there is no significant difference between the two samples' lower-bound means. Moreover, as one would suspect from the tests using contingency tables, our conclusions are insensitive to the treatment of "don't know/not sure" responses. Deleting them from the sample yields a lower-bound mean of $56.41 (2.21) for the 1991 sample and $57.27 (4.33) for the 1993 sample, with an asymptotic Z statistic (0.89) indicating no significant difference. Using the first and second vote choices, and the reconsideration questions to construct interval censored measures for estimating the distribution functions yields seven WTP intervals: (1) $0 to $5, (2) $5 to $10, (3) $10 to $30, (4) $30 to $60, (5) $60 to $120, (6) $120 to $250, and (7) above $250. The lower bound Turnbull mean based on these seven intervals and using the 1991 sample is $54.23 ($2.72), while the comparable estimate based on the 1993 sample is $54.02 ($5.13). As with the cross tabulations for choices alone and the choice functions, these estimates are not significantly different. 4. CONCLUSION Three features of the stated choices of our respondents that might vary over time have been examined. They are (1) the distribution of "for" and "against" votes, (2)parameters of estimated choice functions, and (3) lower-bound estimates for the mean WTP. Choices were not significantly different. Several sets of estimates for the lower-bound mean of WTP were not significantly different in real terms, and the choice functions were remarkably stable. We should expect estimates of the WTP for any object of choice to change as important aspects of the circumstances of choice change. The NOAA Panel's recommendation to consider evidence of "a clear and substantial time trend in responses" as a source of "doubt on the 'reliability' of the findings" is best interpreted as a concern about the timing of CV surveys in relationship to the date of the accident that may have prompted interest in measuring passive use losses (i.e., for a damage assessment). Our results suggest that a random sample of respondents' choices four years after the Exxon Valdez accident do not imply economic values that would be judged to be significantly different from what an independent sample selected in 1991 stated. These results are remarkably stable and have prompted some diverse responses. For example, in contrast to the NOAA Panel's concerns about too much change, one might ask is there too little change? Proponents of these questions might cite the apparent decline in the percent of people identifying environmental issues as problems that most concerned them (as reported from surveys by Roper Starch Worldwide, Inc.), as well as the decline in the percent reporting that we are spending too little on improving and protecting the environment (as reported from surveys by the National Opinion Research Center) over the approximate period covered by Exxon Valdez (1991) and NORC replication (1993) surveys. To believe that changes in these broad indicators of environmental attitudes should be reflected in CV measures of WTP one must assume that that CV responses are dominated by broad environmental attitudes rather than preferences for the specific plan to protect Prince William Sound. If this assumption is valid, one should not expect to see strong relationships between features of the plan and WTP. In this study we find such strong relationships. Others might argue that the incomes and prices faced by households changed between 1991 and 1993 and therefore there one should have expected more variability in estimates of WTP. For these concerns to be meaningful we need to be more specific about how these types of changes would be expected to influence measures of passive use values. Consider first arguments that general price inflation or changes in the availability of market goods should have changed respondents' choices more directly than what we observe. If respondents' choices are motivated by concerns that would lead to passive use values, then by definition they are not linked to changes in the prices or availability of market goods. This follows because marketed goods must be assumed to make separable contributions to individual well being from the environmental resources associated with the passive use value. This condition is implied by the definition of passive use (nonuse) values. As a result changes in the relative prices of marketed goods are unlikely to influence people's decisions for these types of environmental resources. Changes in income could influence monetary measures of passive use value. To evaluate the importance of this effect for our samples we considered respondents' reported household (before tax) income for 1990 and 1992 (for the 1991 Alaska sample and the 1993 replication). By converting the two estimates of the mean household income to 1993 dollars (using the CPI) we can compare the importance of income changes for stated choices and WTP estimates. The means are $37,231 for 1991 and $39,953 in 1993. The lower-bound mean WTP in 1993 dollars was $56.02 for the 1991 sample and $52.81 for 1993. Thus, this type of fairly simple comparison offers little to suggest lower-bound mean estimates for WTP are inconsistent with the changes we would expect based on the absence of important changes in the economic circumstances of the households in 1991 and 1993. In interpreting our findings it is important to acknowledge that this is only one test of temporal stability. Our findings do concur with the earlier test/retest studies (see Loomis [1990, 1991] as one example). Taken together with these studies they seem to suggest that the Panel's concerns about temporal stability may not be as important an issue as the Panel's overall recommendation might be interpreted to imply. Our example involved a large, exceptionally well-known incident where the media coverage alone might have been expected to influence people's choices. This is not what our results suggest. Of course, we also do not know what the pattern of responses would have been had the original survey been conducted closer to the time of the Exxon Valdez Oil Spill. As a result, they do not answer the fundamental question about when CV surveys should be conducted in relation to the timing of large, potentially controversial events like the Exxon Valdez Spill. We can say that longer term averaging or trend analysis seems unwarranted in tests of the reliability of CV surveys. Table 1: Contingency Analysis of Votes For/Against Prevention Plan Contingency Test -(2 Percent Voting For/Against Plana  First Voteb First and Second VotecFirst Dollar Amount For Against For/Against With DK Without DK1991199319911993 $10676833320.01760.03330.00070.9610 $30525648410.03840.46940.46909.35095* $60514949510.20550.24800.00401.6360$120343366670.01930.48680.00000.4837* Significantly different at the 95% level aBoth the 1991 and 1993 surveys permit respondents to reconsider their votes later in the survey. This analysis considers only the response to the first vote question and therefore does not reflect reconsideration of the vote. b"For/Against" recodes volunteered "don't know/not sure" responses as "against." With DK includes "for", "against" and "don't know/not sure" as separate categories. Without DK drops the "don't know/not sure" responses from the sample. c"First and Second Vote" base the outcome of the second vote on any reconsiderations the respondent made, that is, changing their vote from "for" to "against". Table 2: Definition of Variables Variable NameCoding of VariableconstantIntercept, equals unity for all respondents1993Coded as 1 if respondent from the 1993 replication sample, 0 otherwisewlamtdollar amount for first stated tax amountlinclogarithm of household income protestresponse coded as 1 if respondent protested that Exxon or the oil companies should pay for the plan before they were asked how they would vote, 0 otherwise.gmoreresponse coded as 1 if respondent answered B-1 as more damage and B-2 as 3 indicating great deal more damage than Exxon Valdez in absence of escort ship plan; 0 otherwisemoreresponse coded as 1 if respondent answered B-1 as more damage and B-2 as 2 indicating somewhat more damage than Exxon Valdez in absence of escort ship plan; 0 otherwiselessresponse coded as 1 if respondent answered B-1 as less damage and B-3 as a little or a lot less than Exxon Valdez in absence of escort ship plan; 0 otherwisenodamresponse coded as 1 if respondent answered B-1 as less damage and B-3 as no damage in relation to Exxon Valdez in absence of escort ship plan; 0 otherwisemworkresponse coded as 1 if respondent answered plan not completely effective (B-7) and suggest in B-8 it would reduce damage a little or a moderate amount; 0 otherwisenworkresponses coded as 1 if respondent answered plan not completely effective (B-7) and suggest in B-8 it would not reduce damage at all; 0 otherwise.nameresponses coded as 1 if respondent spontaneously named the Exxon Valdez as one of the major environment accidents caused by humans; 0 otherwisecoastalresponse coded as 1 if respondent rated as personally (A-3) protecting coastal areas from oil spills as "extremely important" or "very important"; 0 otherwise.wildresponses coded as 1 if respondent indicated (A-4) government should over next few years set aside very large amount or large amount of new land as wilderness; 0 otherwise.stenresponses coded as 1 if respondent identifies himself (or herself) as a strong environmentalist (B-17 = 1 or 2); 0 otherwise.likvisresponse coded as 1 if respondent indicates household "very likely" or "somewhat likely" to visit Alaska in future; 0 otherwise.whiteresponse coded 1 for Caucasian, 0 otherwise. Table 3: Choice Functions Variable Probit. First Vote SurvivalFirst & Second Vote Survival1993 .011 (.097) .025 ( .225) .009 ( .131)w1amt .009* (.001) - - - -linc .080 (.050) .218 ( .120) .229* ( .068)protest .944* (.113) 2.047* ( .304) 1.169* ( .145)gmore .570* (.160) 1.714* ( .515) .759* ( .228)more .693 (.960) 1.671 (2.177) .065 (1.494)less .382* (.099) .851* ( .235) .580* ( .129)nodam .366 (.300) .882 ( .595) .433 ( .363)mwork .069 (.084) .138 ( .198) .203 ( .113)nwork 1.400* (.403) 2.604* ( .694) 1.848* ( .393)name .152 (.086) .301 ( .203) .306 ( .116)coastal .288* (.107) .485* ( .244) .201 ( .139)wild .154 (.086) .403* ( .201) .335* ( .114)stenv .135 (.091) .362 ( .220) .297* ( .125)likvis .212* (.090) .519* ( .222) .247* ( .123)white .320* (.105) .701* ( .247) .287* ( .138)_cons .731 (.504) 1.478 (1.193) 1.353* ( .673) Notes: n = 1144 * indicates significance at the 95% level Table 4: Choice Functions With Full Interaction Effects VariableProbit.First Vote SurvivalFirst & Second Vote Survival1993 .905 (1.174) 1.300 (2.645) 1.371 (1.528)w1amt .009* ( .001) -- -- -- --linc .094 ( .059) .251 ( .141) .257* ( .078)protest 1.073* ( .135) 2.292* ( .348) 1.279* ( .166)gmore .591* ( .188) 1.778* ( .593) .629* ( .252)more .720 ( .956) 1.699 (2.138) .014 (1.479)less .319* ( .115) .648* ( .263) .453* ( .148)nodam .451 ( .366) 1.184 ( .716) .735 ( .415)mwork .147 ( .097) .302 ( .226) .274* ( .127)nwork 1.318* ( .407) 2.425* ( .694) 1.768* ( .393)name .141 ( .100) .229 ( .233) .253 ( .131)coastal .435* ( .126) .773* ( .287) .358* ( .159)wild .095 ( .098) .244 ( .229) .269* ( .129)stenv .236* ( .106) .575* ( .265) .386* ( .146)likvis .146 ( .105) .335 ( .252) .181 ( .143)white .342* ( .118) .791* ( .278) .335* ( .154)n_w1amt .001 ( .002) -- -- -- --n_linc .065 ( .118) .103 ( .268) .106 ( .156)n_prtest .484 ( .255) 1.003 ( .547) .463 ( .330)n_gmore .130 ( .367) .377 (1.043) .476 ( .562)n_less .330 ( .237) .821 ( .509) .552 ( .300)n_nodam .342 ( .659) 1.342 (1.354) 1.477 ( .912)n_mwork .331 ( .202) .737 ( .464) .322 ( .270)n_name .043 ( .206) .221 ( .472) .230 ( .276)n_coast .555* ( .249) 1.129* ( .561) .595 ( .325)n_wild .280 ( .207) .640 ( .482) .303 ( .277)n_stenv .384 ( .216) .781 ( .508) .455 ( .293)n_likvis .252 ( .211) .653 ( .507) .179 ( .285)n_white .094 ( .260) .344 ( .585) .254 ( .351)_cons .937 ( .591) 1.044 (1.405) 1.004 ( .781) Notes: n = 1144 * indicates significance at the 95% level References Alberini, A., 1995a, ""Testing Willingness-to-Pay Models of Discrete Choice Contingent Valuation Survey Data," Land Economics, Vol. 71, No. 1, 83-95. Alberini, A., 1995b, "Efficiency vs. Bias of Willingness-to-Pay Estimates: Bivariate and Interval Data Models," Journal of Environmental Economics and Management, Vol. 29, No 2, 169-180. Bockstael, N. B. and K. E. McConnell, 1993. "Public Goods as Characteristics of Nonmarket Commodities," Economic Journal 103(9):1244-1257. Cameron, T.A. and J. Quiggin, 1994, "Estimation Using Contingent Valuation Data from a 'Dichotomous Choice with Follow-up' Questionnaire," Journal of Environmental Economics and Management, Vol. 27, No. 3, 218-234. Carson, Richard T., Nicholas E. Flores, and W. Michael Hanemann, 1994. "On the Creation and Destruction of Public Goods: the Matter of Sequencing," unpublished paper, Department of Economics, University of California, San Diego, April. Carson, R. T., W. M. Hanemann, R. J. Kopp, J. A. Krosnick, R. C. Mitchell, S. Presser, P. A. Ruud and V. K. Smith, 1994, "Prospective Interim Lost Use Value Due to DDT and PCB Contamination in the Southern California Bight," Report to National Oceanic and Atmospheric Administration, Natural Resource Damage Assessment, Inc., La Jolla, Ca., September. Carson, R. T. and R. C. Mitchell, 1993, "The Value of Clean Water: The Public's Willingness to Pay for Boatable, Fishable and Swimmable Quality Water," Water Resources Research, Vol. 29, No. 7, 2445-2454. Carson, R. T., R. C. Mitchell, W.M. Hanemann, R. J. Kopp, S. Presser, and P. A. Ruud, 1992, "A Contingent Valuation Study of Lost Passive Use Values Resulting From The Exxon Valdez Oil Spill," Report to the Attorney General of the State of Alaska. Carson, R. T., Mitchell, R. C., Hanemann, W.M., Kopp, R. J., Presser, S. and Ruud, P. A., 1995, "Contingent Valuation and Lost Passive Use: Damages From The Exxon Valdez," Discussion Paper, Department of Economics, University of California - San Diego. Carson, R.T., J. Wright, N. Carson, A. Alberini, and N. Flores., 1995, "A Bibliography of Contingent Valuation Studies and Papers," Natural Resource Damage Assessment, Inc. La Jolla, CA. Diamond, P. and J. A. Hausman, 1994, "Contingent Valuation: Is Some Number Better than no Number," Journal of Economic Perspectives, Vol. 8, No. 4, 45-64. Hanemann, W. M., 1988. "Three Approaches to Defining 'Existence' or Nonuse Values Under Uncertainty," unpublished paper, Department of Agricultural and Resource Economics, University of California, Berkeley, July. _________ 1994, "Contingent Valuation and Economics," Journal of Economic Perspectives, Vol. 8, No. 4, 19-44. Hanemann, W. M., J. B. Loomis, and B. J. Kanninen, 1991, "Statistical Efficiency of Double-Bounded Dichotomous Choice Contingent Valuation," American Journal of Agricultural Economics, Vol. 72. No. 4, 1255-1263. Hausman, J. A., 1993, Contingent Valuation: A Critical Assessment, (Amsterdam: North Holland). Jordan, Donald L., 1993, "Newspaper Effects on Policy Preferences" Public Opinion Quarterly, Vol. 57, No. 191-204. Kanninen, B. J., 1995, "Bias in Discrete Response Contingent Valuation," Journal of Environmental Economics and Management, Vol. 28, No. 1, 114-125. Kopp, R. J. and K. A. Pease, 1996, "Contingent Valuation: Economics, Law and Politics," in Determining the Value of Non-Marketed Goods: Economic, Psychological, and Policy Relevant Aspects of Contingent Valuation Methods, R. J. Kopp, W. Pommerehne and N. Schwarz (eds.), Boston: Kluwer-Nijhoff (forthcoming). Ladd, Everett Carl and Karlyn H. Bowman, 1995. Attitudes Toward the Environment: Twenty-five Years After Earth Day (Washington, D. C.: The American Enterprise Press). Larson, Douglas, 1993. "On Measuring Existence Value," Land Economics, Vol. 69, November 377-389. Loomis, J.B., 1989, "Test-Retest Reliability of Contingent Valuation Method: A Comparison of General Population and Visitor Response," American Journal of Agricultural Economics, Vol. 71, No. 1, 76-84. ________, 1990, "Comparative Reliability of the Dichotomous Choice and Open-Ended Contingent Valuation Techniques," Journal of Environmental Economics and Management, Vol. 18, No. 1, 78-85. McClelland, G.H., W. D. Schulze, J. K Lazo, D. W. Waldman, J. K. Doyle, S. R. Elliott, and J. R. Erwin, 1992, "Methods for Measuring Nonuse Values: A Contingent Valuation Study of Groundwater Cleanup," Report to the U.S. Environmental Protection Agency. Nelson, W., 1982, Applied Life Analysis, New York: John Wiley. Portney, P.R., 1994, "The Contingent Valuation Debate: Why Economists Should Care," Journal of Economic Perspectives, Vol. 8, No. 4, 3-18. Randall, Alan, 1991. "Nonuse Benefits," in J. B. Braden and C. D. Kolstad, editors, Measuring the Demand for Environmental Commodities (Amsterdam: North Holland). Smith, V. Kerry, 1987. "Nonuse Values in Benefit Cost Analysis," Southern Economic Journal, Vol. 54 (July):19-26. Turnbull, B. W., 1976, "The Empirical Distribution Function with Arbitrarily Grouped, Censored and Truncated Data," Journal of the Royal Statistical Society, B38, 290-295. U.S. Department of Commerce, 1966. Income Distribution in the United States, by Herman P. Miller (a 1960 Census Monograph) (Washington, D.C.: U.S. Government Printing Office). PAGE  PAGE 26  See Carson, Wright, Carson, Alberini and Flores [1995].  The term passive use was first used in the ruling by the United States Court of Appeals for the District of Columbia in Ohio v. Department of the Interior, 880 F.2d 432 (D.C. Cir. 1989). The value derived from passive use has been referred to as nonuse value, existence value, and bequest value. Option value is also listed as a component of passive use value in some of the discussion explaining the ruling. The literature now generally recognizes option value as a measure of people's risk aversion for factors that might affect the ability to have access to or use environmental resources and therefore not a component of nonuse values (see Smith [1987] and Randall [1991]). In addition, subsequent research by Larson [1993] has suggested that existence values could be measured with assumptions from information about people's use of the resource to be valued. While Larson's derivation is correct, his interpretation requires specific untestable assumptions restricting individual preferences to offer the interpretation as existence values (Bockstael and McConnell [1993]).  See Diamond and Hausman [1994] and Hanemann [1994] as examples.  The opinion in Ohio v. Department of the Interior stated, On remand, DOI should consider a rule that would permit trustees to derive use values for natural resources by summing up all reliably calculated use values, however measured, so long as the trustee does not double count. [p. 87] The opinion made clear that its definition of use values included use and passive use or nonuse values.  We adopt the term "economic value" rather than simply "value" to distinguish our meaning from other uses of the word value. Economic values are defined by an individual's choices. When it is known that someone chooses to give up x in order to obtain y, we can conclude the economic value of y (termed the object of choice) is at least x.  In the debate over the appropriate uses of CV, the word "reliability" is frequently used. It is not apparent, however, that this word has the same meaning to all the participants in the debate. As noted in Kopp and Pease [1995], a recent U.S. Supreme Court decision concerning the admissibility of scientific evidence (Daubert v. _899710235Fvh˛̛Ole PIC LMETA F/ Times New Roman_- 2 j> 2 j>Times New Romanȉ- 2 `P2 ` robabilitykkkk 2 ` WTP7 2 `tkSymbol- 2 `g= 2 `Times New Romanȉ- 2 ` (~ 2 `p)~ & "System-ࡱ FMicrosoft Equation 2.0 DS Equation Equation.2ࡱ; ࡱ; Ѐ;7;d 7; F j =Probability(WTPt j )UDࡱ; ࡱ; CompObj ZObjInfo Equation Native _899710234. F̛̛Merrell Dow Pharmaceuticals, 113 S.Ct. 2786, 2795, n9 (1993)), noted that while scientists "typically distinguish between 'validity' (does the principle support what it purports to show?) and 'reliability' (does application of the principle produce consistent results?)," the Court emphasized its "reference here is to evidentiary reliability -- that is, trustworthiness." As used by the Ohio Court and in the NOAA Panel report, the reliability of a measure is the degree to which it measures the theoretical construct under investigation. However, in the empirical social sciences, this preceding definition pertains to validity, whereas reliability is defined as the extent to which the variance of the measure is not due to random sources of error. In this paper we are using the term reliability in this latter sense.  The Panel was co-chaired by two Nobel Laureate economists, Kenneth Arrow and Robert Solow. The additional members of the Panel were: Edward Leamer of the University of California, Los Angeles, Paul Portney of Resources for the Future, Roy Radner of Bell Laboratories and New York University, and Howard Schuman of the University of Michigan. The Panel's report was published in the January 15, 1993 issue of the Federal Register. Federal Register, January 15, 1993, p. 4610.  In addition to temporal averaging, the Panel also recommended: (a) the use of probability samples allowing inference to target population, (b) personal interviews, (c) careful pretesting for interviewer effects and questionnaire design, and (d) the minimization on nonresponse. The Panel also made specific recommendations for the survey itself. These recommendations included: (a) a conservative survey design (i.e., one that tends to understate values), (b) a willingness-to-pay referendum style value elicitation format, (c) accurate description of the program or policy. (d) pretesting of photographs, (e) reminder of undamaged substitute commodities, (f) adequate time lapse from the accident, (g) no-answer option, (h)yes/no followups, and (i) checks on understanding and acceptance of the object of choice presented in the CV survey.  In the case of resources that are assumed to provide a source of passive use values it is reasonable, following Hanemann [1988], to assume they make separable contributions to individual preferences. Under this premise we would expect that the amount and conditions of access to other substitute resources could influence these choices, but the relative prices of other goods making the separable contribution to preferences would not. In that case measures of economic value would respond to changes in only the aspects of the circumstances of choice related to income and non-market substitutes.  There has been some evidence that news and its sources influence public opinions. See Jordan [1993] as one example.  Carson and Mitchell [1993] also report the results of a replication study. Their study, using a CV instrument to value changes in surface water quality, showed no significant differences in estimates of willingness-to-pay (after adjusting by changes in the consumer price index) between two surveys conducted three years apart.  A complete description of the final survey and its development is provided in Carson et al. [1992].  Westat's Master Sample of 60 PSUs was selected from a list that grouped the 3,111 counties in the continental United States in 1980 into 1,179 PSUs, each consisting of one or more adjacent counties. The 1980 census was used since results from the 1990 census were not available at the time the sample was drawn. Because Alaska and Hawaii were excluded from Westat's original sampling list, a new stratum was created consisting of those two states. A random selection of PSUs from this stratum yielded the Honolulu SMSA.  Non-English speaking households were ineligible for the survey.  An approximate way to consider the power of our test of reliability is to use the Mitchell-Carson [1989, pp. 365-366] evaluation of sample requirements to isolate specified differences in means expressed in proportional terms. Given the p-value for probability of a Type I error for the test, the desired power, and an assumption about the coefficient of variation for the initial sample, Table C-4 provides (for a two sided t-test) the desired sample size when  EMBED Equation.2  = .05 and power = .90. Taking the assumed coefficient of variation (cv) as the estimate based on the lower bound mean for 1991 we have cv = .04. This is substantially below the estimates in Table C-4. Nonetheless, using cv = .1 as a conservative assumption, a sample size of 28 would be required to detect 10 percent differences in the means (32 for power = .95). Our sample of 300 clearly exceeds this standard.  Results of the larger study are contained in Carson, Hanemann, et al. [1994].  In households with more than one eligible respondent, the interviewer used a random number table to select one eligible respondent for the main interview.  The four versions of the original and follow-up surveys differed by the dollar amounts households were told they would pay in higher taxes if the prevention plan was adopted. The actual amounts used are displayed below. VersionFirst AmountSecond Amount if "for" the programSecond Amount if "against" the programA$10$30$5B$30$60$10C$60$120$30D$120$250$60  These responses were not offered by interviewers but were recorded if respondents voluntarily offered either answer.  There have been a variety of responses to the critique. Kanninen [1995] argues implicitly that the bias could be due to poor bid design. Alberini's [1995a] analysis of the properties of different bid designs also "accepts" the responses to the second question as arising from the same underlying distribution as the first. In further research, Alberini [1995b] has demonstrated the bias/variance tradeoff implied by assuming the responses to the two questions are perfectly correlated when in fact they are highly correlated would generally favor use of the double-bounded estimator.  The original 1991 and the recent 1993 data sets employed in the contingency table tests had 1,043 and 300 observations respectively for a total of 1,343 observations. In the choice function equations we employ the logarithm of income as an explanatory variable. In the 1991 and 1993 data there are 160 and 39 observations respectively that have missing income information. This reduces the size of the pooled data set that can be used to estimate the choice functions to 1,144 observations.  There are actually different hypotheses implied by each estimator. With the probit model, the parameters reflect both the location and scale parameters for the distribution. In the Weibull model, they measure the percentage change in latent WTP with a change in each independent variable.  Estimation with two questions yields interval estimates of  EMBED Equation.2  The likelihood function can be defined using these intervals. See Haab and McConnell [1995] for further illustration of the method.  This statement is true irrespective of the particular amounts used to define the intervals, although the particular tax amounts used can influence how much less the lower-bound mean is than the sample mean.  Using the consumer price index to adjust for the price increases scales the 1991 estimate by 1.061. Then the asymptotic Z statistic testing equality of the two means is 1.16, implying the null hypothesis of equality cannot be rejected at any conventional p-value.  We are grateful to John Payne for identifying this interpretation of the results.  See Ladd and Bowman, 1995  To the extent it is possible to isolate substitution relationships with other non-market resources, we might also expect that changes in these substitute resources would also influence measures of WTP. Nonetheless, the magnitude of these responses cannot be predicted a priori. At best, we have limited overall expectations from economic theory about changes in measures of WTP with changes in each individual's circumstances of choice.  The comparison of average incomes leads to a bit smaller discrepancy if we focus on the full 1993 sample. The mean in this case is $38,305 (in 1993 dollars). A larger difference in income arises with a different treatment of the right censored highest category of income. Using the U.S. Department of Commerce [1966] approach to fitting a Pareto tail to the distribution our adjusted (to 1993) mean income levels become 39,410 for the 1991 sample and 43,125 for the 1993 sample. This large discrepancy arises because a greater number of respondents in the 1993 sample reported incomes in the highest two income classes, and thus the mid-point assigned to the right censored class was greater. In the other computations the same censoring point was assigned to both samples. To evaluate whether the choices were consistent with this income increase between the Alaska and NORC surveys we computed chi square tests for each initial tax amount for the income group in this greatest income class. Only in the case of the $120 tax amount did we find significantly different choice patterns between the Alaska and NORC samples. (p = .023). A higher fraction of the NORC sample supported the plan at this cost than the Alaska sample. This is consistent with what we would expect with the higher income levels. This result should be interpreted cautiously because only 12 respondents with this income level in the NORC sample were assigned to this tax amount. 0=/00ࡱ; Ole PIC  LMETA CompObjZL=4@ࡱ; =4O P .1   & & MathTypeP &   @&ࡱ;  FMicrosoft Equation 2.0 DS Equation Equation.2ࡱ; ObjInfo1W3h0W3W31414&1W3pW3W3(24(24 Equation Native !<_899710232F̛͛Ole "ࡱ; p ?GFF ࡱ; LS{ hࡱ; S{V  u .1  @@& & MathTypepTimes New Roman-PIC 1W3h0W3 h46h46#LMETA %CompObj2ZObjInfo4 2 `1Symbol- 2 ` - 2 `= 2 `> 2 `7F% Times New Roman- 2 j> 2 j>Times New Roman - 2 `P2 ` robabilitykkkk 2 ` WTP@ 2 `)tkTimes New Roman- 2 `6 ( 2 `) &  "System-ࡱ;  FMicrosoft Equation 2.0 DS Equation Equation.2ࡱ; ࡱ; pπ?GFF 1-F j =Probability(WTP>t j )ࡱ; Equation NativeStW36886868 5_899710231 F͛͛Ole 8PIC 9Lࡱ; LE lࡱ; Ee 1 .1  & & MathTypeTimes New Roman- 2 @6lk 2 @N 2 @lk 2 @Fn 2 @! Y 2 @ lk 2 @UnMETA 1W3h0W3t; ;CompObjNZObjInfoPEquation Native Q Times New Roman+- 2 bj> 2 Zkc 2 j> 2  j> 2  j> 2 1j>Symbol- 2 @= 2 Y 2 @ + 2 @- Symbol- 2 bh={Times New Roman- 2 @4[ 2 @(( 2 @- ) 2 @7( 2 @)] Times New Roman+- 2 b1pTimes New Roman- 2 @1Symbol- 2 @F% 2 @F% &  "System-- 2 p0ࡱ;  FMicrosoft Equation 2.0 DS Equation Equation.2ࡱ; ࡱ; wwx w l=[ j=1k  N j ln(F j )+Y j ln(1-F j )] 2 p%tk ࡱ; L+y ࡱ; _899710230 1W3h0W3"F͛͛Ole:?@:? UPIC !$VLMETA Xh+y V ' .1   '&`'7 & MathType0Times New Roman- 2 iWTP@ 2 iqP 2 irob 2 in WTP@ 2 itk 2 i[tk 2 iP 2 irob 2 iatk 2 i WTP@ 2 i<%tk 2 tk 2 A P 2 S rob 2 tk 2 WTP@ 2 tk 2 Ytk 2 YX P 2 Yj r 2 Y9 ob 2 Yctk 2 YWTP@ 2 Y2tk 2 tk Times New Roman- 2 LB| 2 ukc 2 kc 2 kc 2 1 ukc 2 1 kcSymbol- 2 iu= 2 i` 2 iM  2 i< 2 i+ 2 i` 2 i 2 i#< 2 + 2 h ` 2  2 g< 2 + 2 #+ 2 Y ` 2 Y 2 Y < 2 Y|+ 2 ~` 2  - Symbol- 2 -{ 2 a-{Times New Roman- 2 i0 2 il 0 2  1 Times New Roman- 2 L1p 2 1p 2 1p 2 %2p 2 Ae2p 2 A2p 2 A3p 2 1p 2 1pTimes New Roman- 2 i ( 2 i) 2 i( 2 i&) 2 ( 2 ) 2 .` 2 .` 2 T .` 2 ".` 2 Y( 2 Y) 2 : ( 2 )Symbol- 2  F% &  "System-ࡱ FMicrosoft Equation 2.0 DS Equation Equation.2ࡱ; ࡱ; NMx M WTP LB =0Prob(0WTP<t 1 )+t 1 CompObj#%zZObjInfo&|Equation Native }_899710238)FΛ*-ЛZProb(t 1 WTP<t 2 )+t 2 Prob(t 2 WTP<t 3 )+....+t k-1 Prob(t k-1 WTP<t k )+t k (1-F k )ࡱ; ࡱ; Lࡱ; O  .1  `& @ & MathType0Symbol- 2 !a & "SystemOle 1W3h0W3 h46h46PIC (+LMETAbj (CompObj*,Z-"Syࡱ FMicrosoft Equation 2.0 DS Equation Equation.2ࡱ; ࡱ;  ;7;7; aࡱ; LT{hࡱ; ObjInfo 1W3h0W3t;-Equation Native <_899710237'0F*-Л*-ЛPOleion Native PIC  /2LMETA$&'()*+,-./0 67@CompObjFGHIJKLMO13WYZ`ObjInfofghijklmnop4uvwxyT{e  .1  @ &  & MathTypepSymbol- 2 `7F% 2 `;F% 2 `bF%Times New Roman+- 2 `( 2 `.` 2 `".` 2 `( 2 ` )).`Times New Romanm- 2 `:e 2 `ag Times New Roman- 2 j> 2  j>Symbol- 2 `J- Symbol- 2 4 -{ Times New Roman+- 2  1p &  "System- 2 @- ) 2 @7( 2 @ࡱ;  FMicrosoft Equation 2.0 DS Equation Equation.2ࡱ; ࡱ; ܀wwlw F(e.g.(F j -F j-1 )). j )+ࡱ;  Oh+'0 D h   @d Equation Native 1W3h0W3 FSummaryInformation(PICVLMETAXhg the beD:\WINWORD\TEMPLATE\NORMAL.DOT%TEMPORAL RELIABILITY OF ESTIMATESMarjorie B. McElroyMarjorie B. McElroy@_@@_8@F#Microsoft Word 6.02ࡱ; ~AD 6?FGHwx\]^!%r-.0##$$$%k&q&o't'N****@,A,R/S/T/b0c00uDP]]cuDP]cc]V uDPPccuDPcU^UcPcc(M001#144;;>>??BBBBEEKK_LM MUNVNWNSSSSlTqTJUKUUUUUUUUUVVVVVVVVVVVVV uD}5eKuD}5acvK uD}5eKuD}5acvK uD}5eKuD}5acvKuDhV]^uDP]]cuDP]cV]cPc^U uDP]uDPc6VVVVVV?W@WGWHWWWWWWWWWWXX X XXXX'Z(Z]^__:b=bbcceeGgiiiikkmmRssssstt%t&t?t@tuunvpv\w^wx"xU]ehhJPuDPcc^UuDcec uD}5eKuD}5acvK uDPPhuD uD}5eKuD}5acvKB"x/x0xBxCxDxLxMxyxzx~xxxxxxyyyyyytzuz#{${{{i|j|}}}}D~E~~~$%Á܁݁$%45}~ !01BC̃̓܃݃'(56EF\]jke]^cU]]]c_kz{΄τ4Yv$%67MN^_noʆˆۆ܆KLdevwԇՇ()?@OP_`ˈ̈ϋՋ %Vٍב/UuEbzcVU`U]ec^ce\z ј,ʛY̝:ZƞXq'[n KLơͥΥޥ\]px:;ڮlmȴɴABVcuDPc uDPc]P8 uDP8cUVcVcU@AB&'()ͼμ9:;STghij#$?@ OWuVccuDce uD}5eKuD}5avKc uD}5eKuD}5avKuD]V uDPA&BCD ! 46q!!!!!!D!D!D!D!!D!!!!!!!!!!!! !!!!!!(33xx  x999 3@333qSZ!#$%N*T/1\358;;>??A|DEE(HyJ_L!(!(!!(!(!(!(!(!(!(!(!(!(!(!( !( !(!( !(!(!(!(!(!(!(!(!(!(x3 3`0x3L33`0p333 33_LBPSSUUVVVWBWrWWWWXX)Z[]]^_bGghkoQsRssssssssss !( !(!(!(!|!|!(!!(!(!(!(!(!(!!(!(!(!(!(!(!( !(!(!( !( !(0`````"7L33x33&sssssstttttttt't(t)t*t;tAtBtOt````````      w@`p @  !`'-2    @`p @  !`'-2         OtVtWt[t\t]tetftgtsttt|t}ttttttttttttt      @`p @  !`'-2     ttttttttttttttttttttuuww@`p @  !`'-2 @`p @  !`'-2 uu u uuuu!u(u/u0u5u8u;u>uAuHuOuVu]u^uw@`p @  !`'-2 @`p @  !`'-2 ^uuunvov\w]wwwwx!x"x0xCxDxMxyxzxxxxxxxxyy%y00000000(!![[[[[[[[[[lYYYY,"lYYYY,"lYYYY,"YIYIx333%yyyytzuzzz#{${){{{{i|j|p|}}}}}}D~E~M~~~~$%,[[[[[[[[[[[[[[[[[[[[[[[[[[!!3lYYYY,"x3( +HIN\l{!!!!!!} } } } } } u} 6P3"T 6P3"x 6P3"T6P3!l4KG$    6dP3" `6dP3" `6dP3 3p3{|́΁ց /?@EScqrw} } } } } } } } } } } } } } } } } } } } } } } } }  63"T 63"x 63"T63!l4KG$  ݂͂ނ);MNS`o~ǃ׃ } } } } } } } } } } } } } } } } } } } } } } } } } } 63!l4KG$  63"T 63"x 63"T "0@PQWeu} } } } } } } } } } } } } a} !l4KG$  6P3"T 6P3"x 6P3"T6P3 63"x 63"T63!l4KG$  63"T̈́4=EYvw|"""!} } } n} hZ} 6P3"T6P3!l4KG$     6dP3"6dP3" `6dP3" `6dP3 h(3p3lH$  6(P3 " 6(3 " Dž΅υԅ /} } } } } } } } } } } t} h} 63"x 63"T63 6P3"T 6P3"x 6P3"T 6P3"T6P3!l4KG$  6P3"T 6P3"x/ABHXiyzĆՆ%5EVW]o} } } } } } } } } } } } } } } } } } } } } } } } } } 63"x 63"T63!l4KG$  63"T ƇLJχ߇#34:JZjkrƈֈ׈߈} } } } } } } } } } } } } } } } } } } } } } } } 6P3!l4KG$  63"T 63"x 63"T63߈%56?N^mnv̉܉݉+} } } } } } } } } } } } } } } } } } } } } } } } } 6P3!l4KG$  6P3"T 6P3"x 6P3"T+:IJQ`o~Šъ()2AP_`hx} } } } } } } } } } } } } } } } } } } } } } } } } 6P3"T6P3!l4KG$  6P3"T 6P3"x΋ϋ pՎÏ%} } } } } } """~~!v!v!v!v!v!m!v!0x,30x3x3lH$  6(P3 " 6(3 " 6P3"x 6P3"T6P3!l4KG$  6P3"TGbė:јq7 !!!!!!!!!!!!!!!!!!!!!!!!!! !x`%`%03Px30x3 0x300x3 Kͥ \:lȴAAͼӾ!!!! !!!!!! !!! !!!!!!!!!!! ;;!l44$ ! (0x07(x7x! #'(*/489:# ;; ;; ;; ;;!!!!!!!!!x7x!l44$ ! (0!l44$ ! (0#?!!!!037x:K@Normala c&@& Heading 1U]^$@$ Heading 2xU] @ Heading 3hU @ Heading 4h^$@$ Heading 5Uc$@$ Heading 6^c$@$ Heading 7Vc$@$ Heading 8Vc$ @$ Heading 9 Vc"A@"Default Paragraph Font"@" Normal Indent *@ Endnote Referenceh @ Footer ! @" Header !0&@10Footnote Reference]`cehk"@B" Footnote TextcORTo]c$ Ob ToCompany]cOrToFax]cOFromh]c$"O" FromCompany]c O FromPhone]cOFromFax]cODateh]cOPages]cOaToPhoneODocument]&O& Bibliogrphy ]"O" Right Par!]O"Doc Init"]O2 Tech Init#]OB Technical$]$OR$Pleading % ]8Ob8equation indent& "&O&Comments'xU]cO tablecell(]DODtitle()& 0&'()1%-UZc@O@author(*& 0&'()1%-c"O"list1 +B*O* Affiliation1 ,Vc(O( Copyright -Uc$O$Address .cDODNumber+/x& 0&'()1%-c(O( Affiliation2 0c&O& Disclaimer 1c(O"( paper title2[c&O2& paper author3c(OB( Normalbullet 4P(OR( Normalrefs5Px0Ob0table6<( "]c +@r Endnote Text7c)@ Page Number*O* Abstract text 9xxGw\- !'@)R,b--1;?HUKPTU'W`bhj  ;{L* \1|1 * "$$%&'/'(..     *. xRpu! !!!          !! !!! !$   S!)/ 7<CJQU]\\cjRpu~<_hdDz   N O s`o "%0V"xkzhijklmnq_LsOttu^u%y{ /߈+ #opqrstuvwxyz{|}~RRRRSSSSSSSSTU U::::: %!!C$W$Y$.:: endrefsFnFnJoy HallC:\TEMP\TXT_7_23.DOCRaymond J, Kopp6M:\PAPERS\TEMPORAL\DOCUMENT\LE_SUB\RE-SUB\TXT-7-24.DOCSchool of the Environment$C:\DOC1\MANUSCRI\TEMPORAL.REL\072496Marjorie B. McElroyD:\NETSCAPE\TEMP-REL.W6@HP LaserJet 5/5M - EnhancedLPT1:HPBXLAHP LaserJet 5/5M - Enhanced C@(. X@(s pHP LaserJet 5/5M - Enhanced C@(. X@(s pTimes New Roman Symbol &ArialTimesTimes New RomanWP Greek CenturyWingdingsCG Times Tms Rmn"V hzz8B^$TEMPORAL RELIABILITY OF ESTIMATESMarjorie B. McElroyMarjorie B. McElroyࡱ;