Skip to main content
Top
Published in: Translational Behavioral Medicine 2/2016

Open Access 01-06-2016 | practice and public health policy

Agile science: creating useful products for behavior change in the real world

Authors: Eric B. Hekler, PhD, Predrag Klasnja, PhD, William T. Riley, PhD, Matthew P. Buman, PhD, Jennifer Huberty, PhD, Daniel E. Rivera, PhD, Cesar A. Martin, MS

Published in: Translational Behavioral Medicine | Issue 2/2016

Login to get access

Abstract

Evidence-based practice is important for behavioral interventions but there is debate on how best to support real-world behavior change. The purpose of this paper is to define products and a preliminary process for efficiently and adaptively creating and curating a knowledge base for behavior change for real-world implementation. We look to evidence-based practice suggestions and draw parallels to software development. We argue to target three products: (1) the smallest, meaningful, self-contained, and repurposable behavior change modules of an intervention; (2) “computational models” that define the interaction between modules, individuals, and context; and (3) “personalization” algorithms, which are decision rules for intervention adaptation. The “agile science” process includes a generation phase whereby contender operational definitions and constructs of the three products are created and assessed for feasibility and an evaluation phase, whereby effect size estimates/casual inferences are created. The process emphasizes early-and-often sharing. If correct, agile science could enable a more robust knowledge base for behavior change.
Literature
1.
go back to reference Kelly M, Morgan A, Ellis S, et al. Evidence based public health: a review of the experience of the National Institute of Health and Clinical Excellence (NICE) of developing public health guidance in England. Soc Sci Med. 2010; 71: 1056–1062.CrossRefPubMed Kelly M, Morgan A, Ellis S, et al. Evidence based public health: a review of the experience of the National Institute of Health and Clinical Excellence (NICE) of developing public health guidance in England. Soc Sci Med. 2010; 71: 1056–1062.CrossRefPubMed
2.
go back to reference Rotheram-Borus MJ, Swendeman D, Chorpita BF. Disruptive innovations for designing and diffusing evidence-based interventions. Am Psychol. 2012; 67: 463.CrossRefPubMedPubMedCentral Rotheram-Borus MJ, Swendeman D, Chorpita BF. Disruptive innovations for designing and diffusing evidence-based interventions. Am Psychol. 2012; 67: 463.CrossRefPubMedPubMedCentral
3.
go back to reference Riley WT, Glasgow RE, Etheredge L, Abernethy AP. Rapid, responsive, relevant (R3) research: a call for a rapid learning health research enterprise. Clin Translat Med. 2013; 2: 1–6.CrossRef Riley WT, Glasgow RE, Etheredge L, Abernethy AP. Rapid, responsive, relevant (R3) research: a call for a rapid learning health research enterprise. Clin Translat Med. 2013; 2: 1–6.CrossRef
4.
go back to reference MRC U. Developing and evaluating complex interventions: new guidance. London: Medical Research Council; 2008. MRC U. Developing and evaluating complex interventions: new guidance. London: Medical Research Council; 2008.
5.
go back to reference Craig P, Dieppe P, Macintyre S, et al. Developing and evaluating complex interventions: the new medical research council guidance. Int J Nurs Stud. 2013; 50: 587–592.CrossRefPubMed Craig P, Dieppe P, Macintyre S, et al. Developing and evaluating complex interventions: the new medical research council guidance. Int J Nurs Stud. 2013; 50: 587–592.CrossRefPubMed
6.
go back to reference Kessler R, Glasgow RE. A proposal to speed translation of healthcare research into practice: dramatic change is needed. Am J Prev Med. 2011; 40: 637–644.CrossRefPubMed Kessler R, Glasgow RE. A proposal to speed translation of healthcare research into practice: dramatic change is needed. Am J Prev Med. 2011; 40: 637–644.CrossRefPubMed
7.
go back to reference Rapkin BD, Trickett EJ. Comprehensive dynamic trial designs for behavioral prevention research with communities: overcoming inadequacies of the randomized controlled trial paradigm. Comm Interv AIDS. 2005; 2005: 249–277. Rapkin BD, Trickett EJ. Comprehensive dynamic trial designs for behavioral prevention research with communities: overcoming inadequacies of the randomized controlled trial paradigm. Comm Interv AIDS. 2005; 2005: 249–277.
8.
go back to reference Hovell MF, Wahlgren DR, Adams MA. The logical and empirical basis for the behavioral ecological model. In: DiClemente RJ, Crosby RA, Kegler M, eds. Emerging theories and models in health promotion research and practice: Strategies for enhancing public health. San Francisco, CA: Jossey-Bass Publishers; 2009: 415–450. Hovell MF, Wahlgren DR, Adams MA. The logical and empirical basis for the behavioral ecological model. In: DiClemente RJ, Crosby RA, Kegler M, eds. Emerging theories and models in health promotion research and practice: Strategies for enhancing public health. San Francisco, CA: Jossey-Bass Publishers; 2009: 415–450.
9.
go back to reference Rothman AJ. Exploring connections between moderators and mediators: commentary on subgroup analyses in intervention research. Prev Sci. 2013; 14: 189–192.CrossRefPubMed Rothman AJ. Exploring connections between moderators and mediators: commentary on subgroup analyses in intervention research. Prev Sci. 2013; 14: 189–192.CrossRefPubMed
11.
go back to reference Greenhalgh T, Wong G, Westhorp G, Pawson R. Protocol-realist and meta-narrative evidence synthesis: evolving standards (RAMESES). BMC Med Res Methodol. 2011; 11: 115.CrossRefPubMedPubMedCentral Greenhalgh T, Wong G, Westhorp G, Pawson R. Protocol-realist and meta-narrative evidence synthesis: evolving standards (RAMESES). BMC Med Res Methodol. 2011; 11: 115.CrossRefPubMedPubMedCentral
12.
go back to reference Shepperd S, Lewin S, Straus S, et al. Can we systematically review studies that evaluate complex interventions. PLoS Med/ Public Libr Sci. 2009; 6, e1000086. Shepperd S, Lewin S, Straus S, et al. Can we systematically review studies that evaluate complex interventions. PLoS Med/ Public Libr Sci. 2009; 6, e1000086.
13.
go back to reference Riley WT, Rivera DE. Methodologies for optimizing behavioral interventions: introduction to special section. Translat Behav Med. 2014; 4: 234–237.CrossRef Riley WT, Rivera DE. Methodologies for optimizing behavioral interventions: introduction to special section. Translat Behav Med. 2014; 4: 234–237.CrossRef
14.
go back to reference Collins LM, Baker TB, Mermelstein RJ, et al. The multiphase optimization strategy for engineering effective tobacco use interventions. Ann Behav Med. 2011; 41: 208–226.CrossRefPubMedPubMedCentral Collins LM, Baker TB, Mermelstein RJ, et al. The multiphase optimization strategy for engineering effective tobacco use interventions. Ann Behav Med. 2011; 41: 208–226.CrossRefPubMedPubMedCentral
15.
go back to reference Collins LM, Murphy SA, Strecher V. The multiphase optimization strategy (MOST) and the sequential multiple assignment randomized trial (SMART): new methods for more potent eHealth interventions. Am J Prev Med. 2007; 32: S112–S118.CrossRefPubMedPubMedCentral Collins LM, Murphy SA, Strecher V. The multiphase optimization strategy (MOST) and the sequential multiple assignment randomized trial (SMART): new methods for more potent eHealth interventions. Am J Prev Med. 2007; 32: S112–S118.CrossRefPubMedPubMedCentral
16.
go back to reference Klasnja P, Hekler EB, Shiffman S, et al. Micro-randomized trials: an experimental design for developing just-in-time adaptive interventions. Health Psychol. 2016. Klasnja P, Hekler EB, Shiffman S, et al. Micro-randomized trials: an experimental design for developing just-in-time adaptive interventions. Health Psychol. 2016.
17.
go back to reference Martin CA, Desphande S, Hekler EB, Rivera DE. A system identification approach for improving behavioral interventions based on social cognitive theory. American Control Conference (ACC). 2015; 5878–5883. doi:10.1109/ACC.2015.7172261. Martin CA, Desphande S, Hekler EB, Rivera DE. A system identification approach for improving behavioral interventions based on social cognitive theory. American Control Conference (ACC). 2015; 5878–5883. doi:10.​1109/​ACC.​2015.​7172261.
18.
go back to reference Martin CA, Rivera DE, Hekler EB. An identification test monitoring procedure for MIMO systems based on statistical uncertainty estimation. IEEE Conference on Decision and Control (CDC); 2015. Martin CA, Rivera DE, Hekler EB. An identification test monitoring procedure for MIMO systems based on statistical uncertainty estimation. IEEE Conference on Decision and Control (CDC); 2015.
19.
go back to reference Weisz JR, Ng MY, Bearman SK. Odd couple? Reenvisioning the relation between science and practice in the dissemination-implementation era. Clin Psychol Sci Pract. 2014; 2: 58–74.CrossRef Weisz JR, Ng MY, Bearman SK. Odd couple? Reenvisioning the relation between science and practice in the dissemination-implementation era. Clin Psychol Sci Pract. 2014; 2: 58–74.CrossRef
20.
go back to reference Chorpita BF, Rotheram-Borus MJ, Daleiden EL, et al. The old solutions are the new problem how do we better use what we already know about reducing the burden of mental illness? Perspect Psychol Sci. 2011; 6: 493–497.CrossRefPubMed Chorpita BF, Rotheram-Borus MJ, Daleiden EL, et al. The old solutions are the new problem how do we better use what we already know about reducing the burden of mental illness? Perspect Psychol Sci. 2011; 6: 493–497.CrossRefPubMed
21.
go back to reference Chambers D, Glasgow R, Stange K. The dynamic sustainability framework: addressing the paradox of sustainment amid ongoing change. Implement Sci. 2013; 8: 117.CrossRefPubMedPubMedCentral Chambers D, Glasgow R, Stange K. The dynamic sustainability framework: addressing the paradox of sustainment amid ongoing change. Implement Sci. 2013; 8: 117.CrossRefPubMedPubMedCentral
22.
go back to reference Weber S. Success of open source. Cambridge: Harvard University Press; 2004. Weber S. Success of open source. Cambridge: Harvard University Press; 2004.
23.
go back to reference Abrahamsson P, Warsta J, Siponen MT, Ronkainen J. New directions on agile methods: a comparative analysis. Proceedings of the International Conference on Software Engineering, 2003. Abrahamsson P, Warsta J, Siponen MT, Ronkainen J. New directions on agile methods: a comparative analysis. Proceedings of the International Conference on Software Engineering, 2003.
24.
go back to reference Garlan D, Shaw M. An introduction to software architecture. Advances in software engineering and knowledge engineering. 1993; 1. Garlan D, Shaw M. An introduction to software architecture. Advances in software engineering and knowledge engineering. 1993; 1.
25.
go back to reference Ries E. The lean startup: how today’s entrepreneurs use continuous innovation to create radically successful businesses. Random House LLC; 2011. Ries E. The lean startup: how today’s entrepreneurs use continuous innovation to create radically successful businesses. Random House LLC; 2011.
26.
go back to reference Rogers Y, Sharp H, Preece J. Interaction design: beyond human-computer interaction. 3rd ed. West Sussex: Wiley; 2012. Rogers Y, Sharp H, Preece J. Interaction design: beyond human-computer interaction. 3rd ed. West Sussex: Wiley; 2012.
27.
go back to reference Maguire M. Methods to support human-centred design. Int J Hum Comput Stud. 2001; 55: 587–634.CrossRef Maguire M. Methods to support human-centred design. Int J Hum Comput Stud. 2001; 55: 587–634.CrossRef
28.
go back to reference Shadish WR, Cook TD, Campbell DT. Experimental and quasi-experimental designs for generalized causal inference. Wadsworth Cengage learning; 2002. Shadish WR, Cook TD, Campbell DT. Experimental and quasi-experimental designs for generalized causal inference. Wadsworth Cengage learning; 2002.
29.
go back to reference Fisher WW, Piazza CC, Roane HS, eds. Handbook of applied behavior analysis. New York City: Guilford Press; 2011. Fisher WW, Piazza CC, Roane HS, eds. Handbook of applied behavior analysis. New York City: Guilford Press; 2011.
30.
go back to reference Souza CRB, Redmiles D, Cheng L-T, Millen D, Patterson J. Sometimes you need to see through walls: a field study of application programming interfaces, Proceedings of the 2004 ACM conference on Computer supported cooperative work. Chicago: ACM; 2004: 63–71. Souza CRB, Redmiles D, Cheng L-T, Millen D, Patterson J. Sometimes you need to see through walls: a field study of application programming interfaces, Proceedings of the 2004 ACM conference on Computer supported cooperative work. Chicago: ACM; 2004: 63–71.
31.
go back to reference Michie S, Richardson M, Johnston M, et al. The behavior change technique taxonomy (v1) of 93 hierarchically clustered techniques: building an international consensus for the reporting of behavior change interventions. Ann Behav Med. 2013; 46: 81–95.CrossRefPubMed Michie S, Richardson M, Johnston M, et al. The behavior change technique taxonomy (v1) of 93 hierarchically clustered techniques: building an international consensus for the reporting of behavior change interventions. Ann Behav Med. 2013; 46: 81–95.CrossRefPubMed
33.
go back to reference Spruijt-Metz D, Hekler EB, Saranummi N, et al. Building new computational models to support health behavior change and maintenance: new opportunities in behavioral research. Translat Behav Med. 2015; 2015: 1–12. Spruijt-Metz D, Hekler EB, Saranummi N, et al. Building new computational models to support health behavior change and maintenance: new opportunities in behavioral research. Translat Behav Med. 2015; 2015: 1–12.
34.
go back to reference Nahum-Shani I, Hekler EB, Spruijt-Metz D. Building health behavior models to guide the development of just-in-time adaptive interventions: a pragmatic framework. Health Psychol. 2016. Nahum-Shani I, Hekler EB, Spruijt-Metz D. Building health behavior models to guide the development of just-in-time adaptive interventions: a pragmatic framework. Health Psychol. 2016.
35.
go back to reference Hekler EB, Buman MP, Poothakandiyl N, et al. Exploring behavioral markers of long-term physical activity maintenance: a case study of system identification modeling within a behavioral intervention. Health Educ Res. 2013; 40: 51S–62S. Hekler EB, Buman MP, Poothakandiyl N, et al. Exploring behavioral markers of long-term physical activity maintenance: a case study of system identification modeling within a behavioral intervention. Health Educ Res. 2013; 40: 51S–62S.
36.
go back to reference Hekler EB, Klasnja P, Froehlich JE, Buman MP. Mind the theoretical gap: interpreting, using, and developing behavioral theory in HCI research. Proceedings of SIGCHI Conference on Human Factors in Computing Systems (CHI’13). 2013; 3307–3316. Hekler EB, Klasnja P, Froehlich JE, Buman MP. Mind the theoretical gap: interpreting, using, and developing behavioral theory in HCI research. Proceedings of SIGCHI Conference on Human Factors in Computing Systems (CHI’13). 2013; 3307–3316.
37.
go back to reference Riley WT, Rivera DE, Atienza AA, et al. Health behavior models in the age of mobile interventions: are our theories up to the task? Translat Behav Med. 2011; 1: 53–71.CrossRef Riley WT, Rivera DE, Atienza AA, et al. Health behavior models in the age of mobile interventions: are our theories up to the task? Translat Behav Med. 2011; 1: 53–71.CrossRef
38.
39.
go back to reference Riley WT, Martin CA, Rivera DE, et al. The development of a control systems model of social cognitive theory. Translat Behav Med. 2016. Riley WT, Martin CA, Rivera DE, et al. The development of a control systems model of social cognitive theory. Translat Behav Med. 2016.
40.
go back to reference Dong Y, Rivera DE, Downs DS, et al. Hybrid model predictive control for optimizing gestational weight gain behavioral interventions. American Control Conference (ACC). 2013; 1970–1975. Dong Y, Rivera DE, Downs DS, et al. Hybrid model predictive control for optimizing gestational weight gain behavioral interventions. American Control Conference (ACC). 2013; 1970–1975.
41.
go back to reference Nandola NN, Rivera DE. An improved formulation of hybrid model predictive control with application to production-inventory systems. IEEE Trans Control Syst Technol. 2013; 1: 121–135.CrossRef Nandola NN, Rivera DE. An improved formulation of hybrid model predictive control with application to production-inventory systems. IEEE Trans Control Syst Technol. 2013; 1: 121–135.CrossRef
42.
go back to reference Deshpande S, Rivera DE, Younger JW, Nandola NN. A control systems engineering approach for adaptive behavioral interventions: illustration with a fibromyalgia intervention. Translat Behav Med. 2014; 4: 275–289. Errratum in 274(273), p439.CrossRef Deshpande S, Rivera DE, Younger JW, Nandola NN. A control systems engineering approach for adaptive behavioral interventions: illustration with a fibromyalgia intervention. Translat Behav Med. 2014; 4: 275–289. Errratum in 274(273), p439.CrossRef
43.
go back to reference Martin CA, Rivera DE, Hekler EB. A decision framework for an adaptive behavioral intervention for physical activity using hybrid model predictive control. American Control Conference (ACC). 2016. Martin CA, Rivera DE, Hekler EB. A decision framework for an adaptive behavioral intervention for physical activity using hybrid model predictive control. American Control Conference (ACC). 2016.
44.
45.
46.
go back to reference Borsuk ME, Stow CA, Reckhow KH. A Bayesian network of eutrophication models for synthesis, prediction, and uncertainty analysis. Ecol Model. 2004; 173: 219–239.CrossRef Borsuk ME, Stow CA, Reckhow KH. A Bayesian network of eutrophication models for synthesis, prediction, and uncertainty analysis. Ecol Model. 2004; 173: 219–239.CrossRef
47.
go back to reference Kaye M, Nelson GL, Hekler EB. Researcher-centered design of statistics: why Bayesian statistics better fit the culture and incentives of HCI. Proceedings of SIGCHI Conference on Human Factors in Computing Systems (CHI’16). San Jose, CA: ACM; 2016. Kaye M, Nelson GL, Hekler EB. Researcher-centered design of statistics: why Bayesian statistics better fit the culture and incentives of HCI. Proceedings of SIGCHI Conference on Human Factors in Computing Systems (CHI’16). San Jose, CA: ACM; 2016.
48.
go back to reference Leon AC, Davis LL, Kraemer HC. The role and interpretation of pilot studies in clinical research. J Psychiatr Res. 2011; 45: 626–629.CrossRefPubMed Leon AC, Davis LL, Kraemer HC. The role and interpretation of pilot studies in clinical research. J Psychiatr Res. 2011; 45: 626–629.CrossRefPubMed
49.
go back to reference Estrin D, Sim I. Open mHealth architecture: an engine for health care innovation. Science. 2010; 330: 759–760.CrossRefPubMed Estrin D, Sim I. Open mHealth architecture: an engine for health care innovation. Science. 2010; 330: 759–760.CrossRefPubMed
50.
go back to reference National Research Council. Toward precision medicine: building a knowledge network for biomedical research and a new taxonomy of disease. US: National Academies Press; 2011. National Research Council. Toward precision medicine: building a knowledge network for biomedical research and a new taxonomy of disease. US: National Academies Press; 2011.
51.
go back to reference Mohr DC, Schueller SM, Riley WT, et al. Trials of intervention principles: evaluation methods for evolving behavioral intervention technologies. J Med Internet Res. 2015; 17. Mohr DC, Schueller SM, Riley WT, et al. Trials of intervention principles: evaluation methods for evolving behavioral intervention technologies. J Med Internet Res. 2015; 17.
52.
go back to reference Bond RM, Fariss CJ, Jones JJ, et al. A 61-million-person experiment in social influence and political mobilization. Nature. 2012; 489: 295–298.CrossRefPubMed Bond RM, Fariss CJ, Jones JJ, et al. A 61-million-person experiment in social influence and political mobilization. Nature. 2012; 489: 295–298.CrossRefPubMed
53.
go back to reference Harmon-Jones EE, Mills JE. Cognitive dissonance: Progress on a pivotal theory in social psychology. Scientific Conferences Program, 1997, U Texas, Arlington, TX, US; This volume is based on papers presented at a 2-day conference at the University of Texas at Arlington, winter 1997; 1999. Harmon-Jones EE, Mills JE. Cognitive dissonance: Progress on a pivotal theory in social psychology. Scientific Conferences Program, 1997, U Texas, Arlington, TX, US; This volume is based on papers presented at a 2-day conference at the University of Texas at Arlington, winter 1997; 1999.
54.
go back to reference Arain M, Campbell MJ, Cooper CL, Lancaster GA. What is a pilot or feasibility study? A review of current practice and editorial policy. BMC Med Res Methodol. 2010; 10: 67.CrossRefPubMedPubMedCentral Arain M, Campbell MJ, Cooper CL, Lancaster GA. What is a pilot or feasibility study? A review of current practice and editorial policy. BMC Med Res Methodol. 2010; 10: 67.CrossRefPubMedPubMedCentral
55.
go back to reference Dallery J, Raiff BR. Optimizing behavioral health interventions with single-case designs: from development to dissemination. Translat Behav Med. 2014; 4: 1–14.CrossRef Dallery J, Raiff BR. Optimizing behavioral health interventions with single-case designs: from development to dissemination. Translat Behav Med. 2014; 4: 1–14.CrossRef
56.
go back to reference Dienes Z. Bayesian versus orthodox statistics: which side are you on? Perspect Psychol Sci. 2011; 6: 274–290.CrossRefPubMed Dienes Z. Bayesian versus orthodox statistics: which side are you on? Perspect Psychol Sci. 2011; 6: 274–290.CrossRefPubMed
57.
go back to reference Dow SP, Glassco A, Kass J, et al. Parallel prototyping leads to better design results, more divergence, and increased self-efficacy. Transactions of Computer-Human Interactions (ToCHI). 2010; 17: 18. Dow SP, Glassco A, Kass J, et al. Parallel prototyping leads to better design results, more divergence, and increased self-efficacy. Transactions of Computer-Human Interactions (ToCHI). 2010; 17: 18.
58.
go back to reference Dow SP, Klemmer SR. The efficacy of prototyping under time constraints. Design Thinking. Springer; 2011:111–128. Dow SP, Klemmer SR. The efficacy of prototyping under time constraints. Design Thinking. Springer; 2011:111–128.
59.
go back to reference Arkes HR, Ayton P. The sunk cost and Concorde effects: are humans less rational than lower animals? Psychol Bull. 1999; 125: 591.CrossRef Arkes HR, Ayton P. The sunk cost and Concorde effects: are humans less rational than lower animals? Psychol Bull. 1999; 125: 591.CrossRef
60.
go back to reference Ioannidis JP. How to make more published research true. PLoS Med/ Public Libr Sci. 2014; 11, e1001747. Ioannidis JP. How to make more published research true. PLoS Med/ Public Libr Sci. 2014; 11, e1001747.
Metadata
Title
Agile science: creating useful products for behavior change in the real world
Authors
Eric B. Hekler, PhD
Predrag Klasnja, PhD
William T. Riley, PhD
Matthew P. Buman, PhD
Jennifer Huberty, PhD
Daniel E. Rivera, PhD
Cesar A. Martin, MS
Publication date
01-06-2016
Publisher
Springer US
Published in
Translational Behavioral Medicine / Issue 2/2016
Print ISSN: 1869-6716
Electronic ISSN: 1613-9860
DOI
https://doi.org/10.1007/s13142-016-0395-7

Other articles of this Issue 2/2016

Translational Behavioral Medicine 2/2016 Go to the issue