The process of establishing interrater reliability for the circumplex model's Clinical Rating Scale

The purpose of this study was to describe the process of establishing interrater reliability for the Clinical Rating Scale (CRS). The CRS is an observational scale used by nurses and other mental health professionals to assess and plan systematic treatment interventions for families. The CRS is an operational measure of the Circumplex Model of Marital and Family Functioning. The model assesses three dimensions of family functioning: (a) cohesion, (b) adaptability, and (c) communication. To date, there have been four studies published using the CRS, and only one of these reported on the validity and reliability of the CRS. None of these studies has described the process by which raters came to agree upon the behaviors being observed in the families. In this study, using the CRS, six raters observed and videotaped interactions of families with preschool children preparing and eating supper. Through this process it became evident that the anchor-point definitions provided for the CRS needed more clarification when evaluating families with preschool children. Following observation and rating of the videotapes of young families, raters formulated anchor-point definitions and achieved interrater reliability at a level defined as acceptable. Reliability coefficients used in this study included exact percentage agreement, agreement within plus or minus one category on an 8-point scale, and kappa.