Abstract
This article provides an overview of methodological and conceptual issues underlying the assessment of change during psychotherapy with children and families. Three central considerations are discussed: (1) What changes do we measure? (2) When do we measure change? (3) How do we measure change? The difficulties with the traditional methods of assessing change are highlighted, and suggestions for a new paradigm that calls for expansion of the nomological network relevant to the analysis of change, as well as the inclusion of more assessment points and more reliable and valid measures of the change process, are presented. Illustrations of key points are provided from the Adolescent Transitions Program, a theoretically based intervention program targeting high-risk youth.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
REFERENCES
Achenbach, T. M. (1991). Manual for the Child Behavior Checklist/4–18 and 1991 profile. Burlington: University of Vermont, Department of Psychiatry.
Alexander, J. F., & Parsons, B. V. (1973). Short-term behavioral intervention with delinquent families: Impact on family process and recidivism. Journal of Abnormal Psychology, 81, 219–225.
Andrews, D. W., & Dishion, T. J. (1994). The microsocial structure underpinnings of adolescent problem behavior. In R. D. Ketterlinus & M. E. Lamb (Eds.), Adolescent problem behaviors: Issues & research (pp. 187–207). Hillsdale, NJ: Erlbaum.
Bank, L, Duncan, T., Patterson, G. R., & Reid, J. B. (1993). Parent and teacher ratings in the assessment and prediction of antisocial and delinquent behaviors. Journal of Personality, 61, 693–709.
Bank, L., Dishion, T., Skinner, M., & Patterson, G. R. (1990). Method variance in structural equation modeling: Living with glop. In G. R. Patterson (Ed.), Aggression and depression in family interactions (pp. 247–279). Hillsdale, NJ: Erlbaum.
Baron, R. M., & Kenny, D. A. (1986). The moderator-mediator variable distinction in social psychological research: Conceptual, strategic, and statistical considerations. Journal of Personality and Social Psychology, 51, 1173–1182.
Bereiter, C. (1963). Some persisting dilemmas in the measurement of change. In C. W. Harris (Ed.), Problems in measuring change (pp. 24–36). Madison: University of Wisconsin Press.
Biglan, A., Ary, D. V., Duncan, T., Smolkowski, K., & Metzler, C. W. (1997). The influence of parents and peers on adolescent problem behavior: A test of the social context model in rural communities. Manuscript submitted for publication.
Bronfenbrenner, U. (1989). Ecological systems theory. In R. Vasta (Ed.), Annals of child development, Vol. 6. Six theories of child development: Revised formulations and current issues (pp. 187–249). London: JAI Press.
Campbell, D. T., & Stanley, J. C. (1963). Experimental and quasi-experimental designs for research on teaching. In N. L. Gage (Ed.), Handbook of research teaching, (pp. 171–246). Chicago: Rand McNally.
Capaldi, D. M., & Patterson, G. R. (1991). Relation of parental transitions to boys' adjustment problems: I. A linear hypothesis. II. Mothers at risk for transitions and unskilled parenting. Developmental Psychology, 27, 489–504.
Capaldi, D. M., Chamberlain, P., Fetrow, R. A., & Wilson, J. E. (in press). Conducting ecologically valid prevention research: Recuriting and retaining a “whole village” in multimethod, multiagent studies. American Journal of Community Psychology.
Capaldi, D. M., Crosby, L., & Stoolmiller, M. (1996). Predicting the timing of first sexual intercourse for adolescent males. Child Development, 67, 344–359.
Catterall, J. S. (1987). An intensive group counseling dropout prevention intervention: Some cautions on isolating at-risk adolescents within high schools. American Education Research Journal, 24, 521–540.
Chamberlain, P. (1994). Family connections: Treatment foster care for adolescents with delinquency. Eugene, OR: Castalia.
Cicchetti, D., & Toth, S. L. (1992). The role of developmental and theory in prevention and intervention. Development and Psychopathology, 4, 489–493.
Coatsworth, J. D., Szapocznik, J., Kurtines, W., & Santisteban, D. A. (1997). Culturally competent psychosocial interventions with antisocial problem behavior in HIspanic youth. In D. M. Stoff, J. Breiling, & J. Maser (Eds.), Handbook of antisocial behavior (pp. 395–404). New York: Wiley.
Collins, L. M. (1991). Measurement in longitudinal research. In L. M. Collins & J. L. Horn, Best methods for the analysis of change (pp. 137–148). Washington, DC: American Psychological Association.
Collins, L. M., Cliff, N., & Dent, C. W. (1988). The longitudinal Guttman simplex: A new methodology for measurement of dynamic constructs in longitudinal panel studies. Applied Psychological Measurement, 12, 217–230.
Conger, R. D., Patterson, G. R., & Ge, X. (1995). A mediational model for the impact of parents' stress on adolescent adjustment. Child Development, 66, 80–97.
Cronbach, L. J., & Furby, L. (1970). How we should measure “change”—or should we? Psychological Bulletin, 74, 68–80.
Dishion, T. J., & Andrews, D. W. (1995). Preventing escalation in problem behaviors with high risk young adolescents: Immediate and 1-year outcomes. Journal of Consulting and Clinical Psychology, 63, 538–548.
Dishion, T. J., Andrews, D. W., Kavanagh, K. & Soberman, L. H. (1996). Preventive interventions for high-risk youth: The Adolescent Transitions Program. In R. D. Peters & R. J. McMahon (Eds.), Preventing childhood disorders, substance abuse, and delinquency (pp. 184–214). Thousand Oaks, CA: Sage.
Dishion, T. J., Capaldi, D., Spracklen, K. M., & Li, F. (1995). Peer ecology of male adolescent drug use. Development and Psychopathology, 7, 803–824.
Dishion, T. J., French, D. C., & Patterson, G. R. (1995). The development and ecology of antisocial behavior. In D. Cicchetti & D. J. Cohen (Eds.), Developmental psychopathology. Vol 2: Risk, disorder, and adaptation (pp. 421–471). New York: John Wiley & Sons.
Dishion, T. J., Kavanagh, K., & Kiesner, J. (in press). Adolescent Transitions Program: Assessment and intervention sourcebook. New York: Guilford Press.
Dishion, T. J., Li, F., Spracklen, K. M., Brown, G., & Haas, E. (in press). The measurement of parenting practices in research on adolescent problem behavior: A multimethod and multitrait analysis. In R. S. Ashery (Ed.), Research meeting on drug abuse prevention through family interventions. NIDA Research Monograph.
Dishion, T. J., McCord, J., & Poulin, F. (1997). Iatrogenic intervention effects when aggregating high risk youth. Unpublished manuscript.
Dishion, T. J., Patterson, G. R., & Griesler, P. C. (1994). Peer adaptation in the development of antisocial behavior: A confluence model. In L. R. Huesmann (Ed.), Aggressive behavior. Current perspectives (pp. 61–95). New York: Plenum Press.
Dishion, T. J., Patterson, G. R., & Kavanagh, K. (1992). An experimental test of the coercion model: Linking theory, measurement, and intervention. In J. McCord & R. Tremblay (Eds.), The interaction of theory and practice: Experimental studies of intervention (pp. 253–282). New York: Guilford Press.
Dishion, T. J., Reid, J. B., & Patterson, G. R. (1988). Empirical guidelines for the development of a treatment for early adolescent substance use. In R. E. Coombs (Ed.), The family context of adolescent drug use (pp. 189–224). New York: Haworth Press.
Dishion, T. J., Spracklen, K. M., Andrews, D. W., & Patterson, G. R. (1996). Deviancy training in male adolescent friendships. Behavior Therapy, 27, 373–390.
Dodge, K. A. (1993). The future of research on the treatment of conduct disorder. Special Issue: Toward a developmental perspective on conduct disorder. Development and Psychopathology, 5, 311–319.
Eddy, J. M., & Stoolmiller, M. (1997). A method for the development of reliable measures of change. Unpublished manuscript.
Eysenck, H. J. (1978). An exercise in mega-silliness. American Psychologist, 33, 517.
Fagot, B. I. (1996). Falling into gender role research. In M. R. Merrens & G. G. Brannigan (Eds.), The developmental psychologists: Research adventures across the lifespan (pp. 106–121). New York: McGraw-Hill.
Fergusson, D. M., Lynskey, M. T., & Horwood, L. J. (1993). The effect of maternal depression on maternal ratings of child behavior. Journal of Abnormal Child Psychology, 21, 245–269.
Fiske, D. (1986). Specificity of method and knowledge in social science. In D. W. Fiske & R. A. Shweder (Eds.), Metatheory in social science: Pluralisms and subjectivities (pp. 61–82). Chicago: University of Chicago Press.
Forgatch, M. S. (1991). The clinical science vortex: Developing a theory for antisocial behavior. In D. J. Pepler & K. H. Rubin (Eds.), The development and treatment of childhood aggression (pp. 291–315). Hillsdale, NJ: Erlbaum.
Forgatch, M. S., & Stoolmiller, M. (1994). Emotions as contexts for adolescent delinquency. Journal of Research on Adolescence, 4, 601–614.
Glass, G. V., McGaw, B., & Smith, M. L. (1981). Meta-analysis in social research. Beverly Hills: Sage.
Gold, M. (1970). Delinquent behavior in an American city. San Francisco: Brooks and Coleman.
Gorsuch, R. L. (1983). Factor Analysis (2nd ed). Hillsdale NJ: Lawrence Erlbaum Associates.
Green, S. M., Navratil, J. L., Loeber, R., & Lahey, B. B. (1994). Potential dropouts in a longitudinal study: Prevalence, stability, and associated characteristics. Journal of Child and Family Studies, 3, 69–87.
Henggeler, S. W., Melton, G. B., & Smith, L. A. (1992). Family preservation using multisystemic therapy: An effective alternative to incarcerating serious juvenile offenders. Journal of Consulting and Clinical Psychology, 60, 953–961.
Hollon, S. D., & Flick, S. N. (1988). On the meaning and methods of clinical significance. Behavioral Assessment, 10, 197–206.
Jacobson, N. S., & Truax, P. (1991). Clinical significance: A statistical approach to defining meaningful change in psychotherapy research. Journal of Consulting and Clinical Psychology, 59, 12–19.
Jacobson, N. S., Follette, W. C., & Revenstorf, D. (1984). Psychotherapy outcome research: Methods for reporting variability and evaluating clinical significance. Behavior Therapy, 15, 336–352.
Johnson, S. M., & Lobitz, G. K. (1975). Parental manipulation of child behavior in home observations: A methodological concern. Journal of Applied Behavior Analysis, 7, 23–31.
Kazdin, A. E. (1977). Assessing the clinical or applied importance of behavior change through social validation. Behavior Modification, 1, 427–452.
Kelly, J. G. (1988). A guide to conducting preventive research in the community: First steps. New York: Haworth Press.
Kolko, D. J. (1996). Clinical monitoring of treatment course in child physical abuse: Psychometric characteristics and treatment comparisons. Child Abuse and Neglect, 20, 23–43.
Kraemer, H. C. (1981). Coping strategies in psychiatric clinical research. Journal of Consulting and Clinical Psychology, 49, 309–319.
Kraemer, H. C. (1991). To increase power in randomized clinical trials without increasing sample size. Psychopharmacology Bulletin, 27, 217–224.
Kraemer, H. C., & Thiemann, S. (1989). A strategy to use soft data effectively in randomized controlled clinical trials. Journal of Consulting and Clinical Psychology, 57, 148–154.
Linn, R. L., & Slinde, J. A. (1977). The determination of the significance of change between pre-and posttesting periods. Review of Educational Research, 47, 121–150.
Lord, F. M. (1956). The measurement of growth. Education and Psychological Measurement, 16, 421–437.
Lord, F. M. (1963). Elementary models for measuring change. In C. W. Harris (Ed.), Problems in measuring change (pp. 21–38). Madison: University of Wisconsin Press.
McCord, J. (1978). A thirty-year follow-up of treatment effect. American Psychologist, 33, 284–289.
McCord, J. (1992). The Cambridge-Somerville Study: A pioneering longitudinal experimental study of delinquency prevention. In J. McCord & R. E. Tremblay (Eds.), Preventing antisocial behavior: Interventions from birth through adolescence (pp. 196–206). New York: Guilford Press.
McCord, W., & McCord, J. (1959). Origins of crime. New York: Columbia University Press.
Moros, K., Telfer, L. A., & Barlow, D. H. (1993). Efficacy and specific effects data on new treatments: A case study strategy with mixed anxiety-depression. Journal of Consulting and Clinical Psychology, 61, 412–420.
Olweus, D. (1979). Stability of aggressive reaction patterns in males: A review. Psychological Bulletin, 86, 852–875.
Patterson, G. R. (1974). Interventions for boys with conduct problems: Multiple settings, treatments, and criteria. Journal of Consulting and Clinical Psychology, 42, 471–481.
Patterson, G. R. (1979). A performance theory for coercive family interaction. In R. B. Cairns (Ed.), The analysis of social interactions: Methods, issues, and illustrations (pp. 119–162). Hillsdale, NJ: Erlbaum.
Patterson, G. R. (1982). A social learning approach, Vol. 3: Coercive family process. Eugene, OR: Castalia.
Patterson, G. R. (1993). Orderly change in a stable world: The antisocial trait as a chimera. Journal of Consulting and Clinical Psychology, 61, 911–919.
Patterson, G. R., & Forgatch, M. S. (1995). Predicting future clinical adjustment from treatment outcome and process variables. Psychological Assessment, 7, 275–285.
Patterson, G. R., Reid, J. B., & Dishion, T. J. (1992). A social learning approach. 4. Antisocial boys. Eugene, OR: Castalia.
Poulin, F., & Dishion, T. J. (1997). Iagrogenic effects among high-risk adolescents aggregated within interventions: An analysis of the durability and process. Unpublished manuscript.
Powers, E., & Witmer, H. (1951). An experiment in the prevention of delinquency: The Cambridge-Somerville Youth Study. New York: Columbia University Press.
Prochaska, J. O., & DiClemente, C. C. (1986). Toward a comprehensive model of change. In W. R. Miller & N. Heather (Eds.), Treating addictive behaviors: Processes of change (pp. 3–27). New York: Plenum Press.
Prochaska, J. O., Velicer, W. F., Guadagnolia, E., & Rossi, J. S. (1991). Patterns of change: Dynamic typology applied to smoking cessation. Multivariate Behavioral Research, 26, 83–107.
Reid, J. B. (1978). A social learning approach, Vol. 2: Observation in home settings. Eugene, OR: Castalia.
Reid, J. B. (1993). Prevention of conduct disorder before and after school entry: Relating interventions to development findings. Journal of Development and Psychopathology, 5, 243–262.
Reid, J. B. & Eddy, J. M. (1997). The prevention of antisocial behavior: Some considerations in the search for effective interventions. In D. M. Stoff, J. Breiling, and J. D. Maser (Eds.), The handbook of antisocial behavior (pp. 343–356). New York: Wiley.
Robins, L. N., & Pryzabeck, T. R. (1985). Age of onset of drug use as a factor in drug and other disorders. National Institute on Drug Abuse: Research Monograph Series, 56, 178–192.
Rogosa, D., Brandt, D., & Zimowski, M. (1982). A growth curve approach to the measurement of change. Psychological Bulletin, 90, 726–748.
Rogosa, D. R., & Willet, J. B. (1983). Demonstrating the reliability of the difference score in the measurement of change. Journal of Educational Measurement, 20, 335–343.
Sampson, R. J., & Laub, J. H. (1993). Crime in the making: Pathways and turning points through life. Cambridge, MA: Harvard University Press.
Schlegel, A., & Barry, (1991). Adolescence: An anthropological inquiry. New York: The Free Press.
Schmid, J., & Leiman, J. (1957). The development of hierarchical factor solutions Psychometrika, 22(1), 53–61.
Sroufe, L., & Rutter, M. (1984). The domain of developmental psychopathology. Child Development, 55, 17–29.
Stoolmiller, M. (1992). Using latent growth curves to study developmental processes. Unpublished manuscript. (Available from the Oregon Social Learning Center, 207 E. Fifth Ave., Suite 202, Eugene, OR 97401.)
Stoolmiller, M. (1994). Antisocial behavior, delinquent peer association, and unsupervised wandering for boys: Growth and change from childhood to early adolescence. Multivariate Behavioral Research, 29, 263–288.
Stoolmiller, M. (1997). Systematic biases and correlated errors of measurement in maternal ratings of the antisocial behavior of self and son. Unpublished manuscript.
Stoolmiller, M., Duncan, T., Bank, L., & Patterson, G. R. (1993). Some problems and solutions in the study of change: Significant patterns in client resistance. Journal of Consulting and Clinical Psychology, 61, 920–928.
Stouthamer-Loeber, M., van Kammen, W., & Loeber, R. (1992). The nuts and bolts of implementing large scale longitudinal studies. Violence and Victims, 7, 63–77.
Van Male, R. J. (1997). A system for coding adolescents' dyadic social interaction in videotaped intervention groups. Unpublished master's thesis, University of Oregon, Eugene.
Wampold, B. E., & Jenson, W. R. (1986). Clinical significance revisited. Behavior Therapy, 17, 302–305.
Wierson, M., & Forehand, R. (1994). Introduction to special section: The role of longitudinal data with child psychopathology and treatment: Preliminary comments and issues. Journal of Consulting and Clinical Psychology, 62, 883–886.
Willet, J. B. (1989). some results on the reliability for the longitudinal measurement of change: Implications for the design of studies of individual growth. Educational and Psychological Measurement, 49, 587–602.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Eddy, J.M., Dishion, T.J. & Stoolmiller, M. The Analysis of Intervention Change in Children and Families: Methodological and Conceptual Issues Embedded in Intervention Studies. J Abnorm Child Psychol 26, 53–69 (1998). https://doi.org/10.1023/A:1022634807098
Issue Date:
DOI: https://doi.org/10.1023/A:1022634807098