OCR Text |
Show (that is, with large confidence intervals) and few statistically significant differences across areas. It would be possible to aggregate areas for rarer events and subdivide them for more common events. That approach might solve some of the inflexibility imposed by a "one-size-fits-all" solution, but would add complexity both to analysis and to interpretation of results presented in variable ways. Standardization of Small Area Definitions. Although flexibility is desirable, standardization also has clear and important benefits. By using standard definitions for area boundaries across the various applications of small area analysis in the state, information on the small areas may be accumulated in a repository, data from different data sets and from published accounts may be integrated, and comparisons may be made that were previously not possible or feasible. The area boundaries that were designated in this study used rather general criteria such as population size, income level, and community identity. This was done with the assumption that other investigators would want to take advantage of the groundwork that has been laid here. Statistical Techniques Although the use of formal applications of statistical tests for hypothesis testing may often be unnecessary, most agree that there must be some empirical method for identifying rates that are either higher or lower than usual (Diehr, Cain, & Abdul-Salam, 1993; Diehr, Cain, & Connell, 1990; Thompson, 1998). How different must a rate be before it becomes a cause for concern or action? Rates calculated from few events or in a small population can be unreliable in the sense that a few events can create a large deviation on a trend line. Statistical tests are one method of using an independent criterion to develop professional agreement on differences that are likely to be real, versus those that may be due to sampling error. Two statistical approaches for detecting significant differences among small areas are discussed here: Confidence intervals and Bayesian smoothing techniques. Ecological fallacy and synthetic estimation are also briefly described. Confidence Intervals. Confidence intervals are calculated to describe the precision of an estimate, that is, a range within which the true value of the measure (e.g., a rate, percentage, average, etc.) is expected to occur, taking into account the variance of the measure, the sample-size, and the sampling method that was used to generate the estimate when sampling from a i population. In general, small areas yield rates with poor precision, that is, wide confidence intervals, whereas large areas yield rates with better precision. While their primary purpose is to provide information on the precision of an estimate, they can be useful in separating out true differences from sampling variation. An area may be judged as "not different" from a base rate whose estimate falls in the range of values defined by the confidence interval. One limitation with using confidence intervals to identify whether an area is significantly different from an overall state rate is the lack of independence of the area and state rates. The state includes the small area, and it is not technically appropriate to compare one area with the one that encompasses it because the two are not independent samples (Colton, 1974). Another issue is that the state rate also has a confidence interval that should be taken into consideration. Issues around the use of multiple comparisons should also be considered. A 95% confidence interval describes the range for a true value in 95% of a large number of hypothetical samples. This means that the confidence interval will not include the true value 5% of the time. If one measures rates and calculates confidence intervals for some time period in 50 small areas, none of which are truly different from the state rate, a small number will appear different by chance alone. There is currently some debate about the need to adjust for multiple comparisons (Goodman, 1998; Rothman, 1990; Savitz & Olshan, 1998; Thompson, 1998h 19982). Despite their limitations, confidence internals are relatively easy to compute, and can serve as reasonably good guides for comparing small area estimates. They are thus helpful in determining if differences are important enough to warrant attention or are likely to be chance aberrations. For information on calculating confidence intervals for various types of data, consult Colton (1974) or Levy and Lemeshow (1991). Bayesian Smoothing Techniques. A problem with most analyses of small areas is the instability of estimates from small samples. Death and birth rates are calculated based on all identified events rather than a sample of such events. Nevertheless, rates are calculated based on an arbitrary unit of time and will exhibit variability from interval to interval. That is, the calculated rate can be thought of as a "sample" of the underlying true rate. 23 |