Top

Published in:

01-03-2006 | Original Article

The Role of Behavior Observation in Measurement Systems for Randomized Prevention Trials

Authors: James Snyder, John Reid, Mike Stoolmiller, George Howe, Hendricks Brown, Getachew Dagne, Wendi Cross

Published in: Prevention Science | Issue 1/2006

Abstract

The role of behavior observation in theory-driven prevention intervention trials is examined. A model is presented to guide choice of strategies for the measurement of five core elements in theoretically informed, randomized prevention trials: (1) training intervention agents, (2) delivery of key intervention conditions by intervention agents, (3) responses of clients to intervention conditions, (4) short-term risk reduction in targeted client behaviors, and (5) long-term change in client adjustment. It is argued that the social processes typically thought to mediate interventionist training (Element 1) and the efficacy of psychosocial interventions (Elements 2 and 3) may be powerfully captured by behavior observation. It is also argued that behavior observation has advantages in the measurement of short-term change (Element 4) engendered by intervention, including sensitivity to behavior change and blinding to intervention status.

Some theories may not distinguish between Core Elements 3 and 4. For example, intervention with young adolescents may entail using instruction and role playing to shape skills to resist peer influence or to refuse offers to use drugs. This same set of skills may be the desired short-term outcome. Even if this is the case, distinction between Elements 3 and 4 may be useful. Core Element 4 in contrast to Element 3 refers to the display of the target response outside of the intervention session—its generalization to natural environmental settings and its persistence over time after its acquisition and practice. Heuristically, measurement of the same response in two different contexts may be optimized by different methods.

There is another step in the training process—training PMT therapists who provide training for parents. For sake of simplicity of exposition, this additional element is not described here although observational methods and sequential analyses are clearly relevant to measurement of the training of PMT therapists as well (Forgatch et al., 2005).

This model of social exchange and influence can also be applied to therapists' training of parents in PMT skills. Parents react to actions of the PMT therapist, and these reactions may be cooperative or resistant. As such, reciprocal influence is embedded in this element. Efforts to assess this process have been made (Stoolmiller et al., 1993) and strategies by which PMT therapists can effectively deal with resistance are explicitly addressed in intervention manuals and in training of PMT therapists (Forgatch et al., 2005). These therapist–parent actions and reactions reflect sources of variation due to social Interchange (\(S _{\rm I}^2\)). This suggests that behavior observation methods are also relevant to the measurement of Core Element 1.

Bakeman, R., & Gottman, J. M. (1997). Observing interaction (2nd ed.). New York: Cambridge University Press.

Botvin, G. J. (2000). Life skills training: Promoting health and personal development. Princeton, NJ: Princeton Health Press.

Brown, C. H. (1993). Statistical methods for prevention trials in mental health. Statistics in Medicine, 12, 289–300.PubMedCrossRef

Cairns, R. B., & Green, J. A. (1979). How to assess personality and social patterns: Observations or ratings? In R. B. Cairns (Ed.), The analysis of social interactions: Methods, issues, and illustrations (pp. 209–226). Hillsdale, NJ: Erlbaum.

Carroll, K. M., Rounsaville, B. J., & Nich, C. (1994). Blind man's bluff: Effectiveness and significance of psychotherapy and pharmacotherapy blinding procedures in a clinical trial. Journal of Consulting and Clinical Psychology, 62, 276–280.PubMedCrossRef

Collins, L. M. (1991). Measurement in longitudinal research. In L. M. Collins & J. L. Horn (Eds.), Best methods for analysis of change (pp. 137–148). Washington, DC: American Psychological Association.

Crosby, L., Stubbs, J., Forgatch, M., & Capaldi, D. (1998). Family and peer process code training manual. Eugene: Oregon Social Learning Center.

Dagne, G. A., Howe, G. W., Brown, C. H., & Muthen, B. O. (2002). Hierarchical modeling of sequential behavioral data: An empirical Bayesian approach. Psychological Methods, 7, 262–280.PubMedCrossRef

DeGarmo, D. S., Patterson, G. R., & Forgatch, M. S. (2004). How do outcomes in a specified parent training intervention maintain or wane over time? Prevention Science, 5, 73–90.PubMedCrossRef

Dishion, T. J., & Kavanagh, K. (2004). Adolescent problem behavior: An intervention and assessment sourcebook for working with families in schools. New York: Guilford Press.

Dishion, T. J., Rivera, E. K., Jones, L., Verberkmoes, S., & Patras, J. (2002). Relationship process code. Unpublished Coding Manual, Child and Family Research Center, University of Oregon, Eugene.

Eddy, J. M., Dishion, T. J., & Stoolmiller, M. (1998). The analysis of intervention change in children and families: Methodological and conceptual issues embedded in intervention studies. Journal of Abnormal Child Psychology, 26, 53–69.PubMedCrossRef

Follette, W. C. (1995). Correcting methodological weaknesses in the knowledge base used to derive practice standards. In S. C. Hayes, W. C. Follette, R. M. Dawes, & K. E. Grady (Eds.), Scientific standards of psychological practice: Issues and recommendations (pp. 229–247). Reno, NV: Context Press.

Forgatch, M. S. (1994). Parenting through change: A training manual. Eugene: Oregon Social Learning Center.

Forgatch, M. S., Patterson, G. R., & DeGarmo, D. S. (2005). Evaluating fidelity: Predictive validity for a measure of competent adherence to the Oregon Model of Parent Management Training (PMTO). Behavior Therapy, 36, 3–14.PubMedCrossRef

Gardner, W., & Griffin, W. A. (1989). Methods for the analysis of parallel streams of continuously recorded behavior. Psychological Bulletin, 105, 446–455.CrossRef

Hogue, A., Liddle, H. A., & Rowe, C. (1996). Treatment adherence process research in family therapy: A rationale and some practical guidelines. Psychotherapy, 33, 332–345.

Ialongo, N., Poduska, J., Werthamer, L., & Kellam, S. (2001). The distal impact of two first grade preventive interventions on conduct problems and disorder in early adolescence. Journal of Emotional and Behavioral Disorders, 9, 146–160.CrossRef

Jones, R. R., Reid, J. B., & Patterson, G. R. (1975). Naturalistic observation in clinical assessment. In P. McReynolds (Ed.), Advances in psychological assessment (Vol. 3, pp. 42–95). San Francisco, CA: Jossey-Bass.

Kellam, S. G. (1990). Developmental epidemiologic framework for family research on depression and aggression. In G. R. Patterson (Ed.), Depression and aggression in family interaction (pp. 1–48). Hillsdale, NJ: Erlbaum.

Kent, R. N., O'Leary, K. D., Diament, C., & Dietz, A. (1974). Expectation biases in observational evaluation of therapeutic change. Journal of Consulting and Clinical Psychology, 42, 774–780.PubMedCrossRef

Kraemer, H. C. (1991). To increase power in randomized clinical trials without increasing sample size. Psychopharmacology Bulletin, 27, 217–224.PubMed

MacKinon, D. P., & Lockwood, C. M. (2003). Advances in statistical methods for substance use prevention research. Prevention Science, 4, 155–171.CrossRef

Miller, W. R., & Rollnick, S. (2002). Motivational interviewing: Preparing people to change addictive behavior. New York: Guilford Press.

Muthén, L. K., & Muthén, B. O. (1998–2004). MPLUS user's guide (3rd ed.). Los Angeles, CA: Muthén &Muthén.

Olds, D. L. (2002). Prenatal and infancy home-visiting by nurses: From randomized trials to community replication. Prevention Science, 3, 153–172.PubMedCrossRef

Patterson, G. R. (1982). Coercive family process. Eugene, OR: Castalia.

Patterson, G. R., & Reid, J. B. (1973). Interventions for aggressive boys: A replication study. Behavior Research and Therapy, 11, 383–394.CrossRef

Radke-Yarrow, M., & Zahn-Waxler, C. (1979). Observing interaction: A confrontation with methodology. In R. B. Cairns (Ed.), The analysis of social interactions: Methods, issues, and illustrations (pp. 37–66). Hillsdale, NJ: Erlbaum.

Reid, J. (2003). Development of measurement systems for randomized trials. Paper presented to the American Institutes for Research, Center for Integrating Education and Prevention Research in Schools, Washington, DC.

Reid, J. B., Patterson, G. R., & Snyder, J. (2002). Antisocial behavior in children and adolescents: A developmental analysis and model for intervention. Washington, DC: American Psychological Association.

Rotheram-Borus, M. J., Song, J., Gwadz, M., Lee, M., Van Rossem, R., & Koopman, C. (2003). Reductions in HIV risk among runaway youth. Prevention Science, 4, 173–188.PubMedCrossRef

Schelle, J. (1974). A brief report on the invalidity of parent evaluations of behavior change. Journal of Applied Behavior Analysis, 7, 341–343.CrossRef

Schrepferman, L., & Snyder, J. (2002). Coercion: The link between treatment mechanisms in behavioral parent training and risk reduction in child antisocial behavior. Behavior Therapy, 33, 339–359.CrossRef

Schrepferman, L., Snyder, J., Prichard, J., & Suarez, M. (2004). An observational system for children's social interaction with peers: Reliability and validity. Manuscript submitted for publication, Wichita State University, Wichita, KS.

Skindrud, K. D. (1973). Field observation of observer bias under overt and covert monitoring. In L. Handy & E. Mash (Eds.), Behavior change: Methodology, concepts, and practice (pp. 97–118). Champaign, IL: Research Press.

Snyder, J., & Stoolmiller, M. (2002). Reinforcement and coercion mechanisms in the development of antisocial behavior: Family processes. In J. R. Reid, J. Snyder, & G. R. Patterson (Eds.), Antisocial behavior: Prevention, intervention and basic research (pp. 65–100). Washington, DC: American Psychological Association.

Stoolmiller, M., Duncan, T. E., & Patterson, G. R. (1995). Some problems and solutions in the study of change: Significant patterns of client resistance. Journal of Consulting and Clinical Psychology, 61, 920–928.CrossRef

Stoolmiller, M., Eddy, J. M., & Reid, J. B. (2000). Detecting and describing preventive intervention effects in a universal school-based randomized trial targeting delinquent and violent behavior. Journal of Consulting and Clinical psychology, 68, 296–306.PubMedCrossRef

Walter, H. I., & Gilmore, S. K. (1973). Placebo versus social learning effects in parent training procedures designed to alter the behaviors of aggressive boys. Behavior Therapy, 4, 361–377.CrossRef

White, L., Tursky, B., & Schwartz, G. E. (1985). Placebo: Theory, research and mechanisms. New York: Guilford Press.

Willett, J. B. (1989). Some results on the reliability for longitudinal measurement of change: Implications for the design of studies of individual growth. Educational and Psychological Measurement, 49, 587–602.CrossRef

Title: The Role of Behavior Observation in Measurement Systems for Randomized Prevention Trials
Authors: James Snyder
John Reid
Mike Stoolmiller
George Howe
Hendricks Brown
Getachew Dagne
Wendi Cross
Publication date: 01-03-2006
Publisher: Springer US
Published in: Prevention Science / Issue 1/2006
Print ISSN: 1389-4986
Electronic ISSN: 1573-6695
DOI: https://doi.org/10.1007/s11121-005-0020-3

At a glance: The STEP trials

Springer Medicine

The Role of Behavior Observation in Measurement Systems for Randomized Prevention Trials

Abstract

At a glance: The STEP trials

Springer Medicine

Abstract

Please log in to get access to this content

Other articles of this Issue 1/2006

Mediators and Moderators of Parental Involvement on Substance Use: A National Study of Adolescents

Assessing the Total Effect of Time-Varying Predictors in Prevention Research

Validity of Teacher Ratings in Selecting Influential Aggressive Adolescents for a Targeted Preventive Intervention

The Mediational Role of Neurocognition in the Behavioral Outcomes of a Social-Emotional Prevention Program in Elementary School Students: Effects of the PATHS Curriculum

The Strengthening Washington D.C. Families Project: A Randomized Effectiveness Trial of Family-Based Prevention

Prevalence and Correlates of Substance Use Among Young Asian Pacific Islander Men Who Have Sex with Men