Skip to main content
Key Specialties
Cardiology Diabetology Endocrinology Gastroenterology Geriatrics and Gerontology Gynecology and Obstetrics Hematology Infectious Disease Internal Medicine Nephrology Neurology Oncology Pediatrics Respiratory Medicine Rheumatology Surgery
Congress Coverage
2024 ACC Congress 2023 ESMO Congress 2023 EASD Congress 2023 ERS Congress 2023 ESC Congress 2023 EHA Congress 2023 ASCO Annual Meeting
Clinical Collections
Advances in Alzheimer’s

At a glance: The ONWARDS insulin icodec trials

A round-up of the ONWARDS phase 3 clinical trials evaluating the novel weekly insulin icodec in people with diabetes.
ADVANCED SEARCH
Log in
Top
Substance Abuse Treatment, Prevention, and Policy
Published in: Substance Abuse Treatment, Prevention, and Policy 1/2015

Open Access 01-12-2015 | Research

Internal consistency and measurement equivalence of the cannabis screening questions on the paper-and-pencil face-to-face ASSIST versus the online instrument

Authors: Yasser Khazaal, Anne Chatton, Grégoire Monney, Audrey Nallet, Riaz Khan, Daniele Zullino, Jean-François Etter

Published in: Substance Abuse Treatment, Prevention, and Policy | Issue 1/2015

Login to get access

Abstract

Background

Validated Internet-based screening tools for cannabis use and abuse are needed. The present study aimed to establish equivalence between the previously validated Alcohol, Smoking and Substance Involvement Screening Test (ASSIST) as a paper-and-pencil (PaP)-administered questionnaire and its online use.

Methods

Two groups of cannabis users took part in this study and the results were analyzed using structural equation modeling. One group consisted of 150 participants and was assessed with the ASSIST PaP questionnaire in a face-to-face interview (the PaP group). They were recruited from three settings: a primary health care outpatient clinic, a general psychiatric facility, and an ambulatory specialized addiction treatment facility. The other group (the Web group) comprised 1382 persons who answered the online version of the same questionnaire. This sample was drawn from people who naturalistically visited a website dedicated to helping people with cannabis addiction.

Results

The internal consistency was good for the online questionnaire (0.74) and high for the already validated PaP questionnaire (0.91). The Web group, however, had higher scores on cannabis use than did the PaP group. The results show support for configural invariance, meaning that the one-factor structure was preserved across groups, although measurement equivalence between these two survey modes was not achieved. However, when the Web group was split into two random subsamples, measurement invariance was demonstrated between them by cross-validation.

Conclusions

Measurement equivalence was not achieved between the two survey modes. Nonetheless, subanalyses of the Web group demonstrated that the cannabis screening questions of the ASSIST can be used for online screening. Differences in ASSIST scores between samples may be due to the sensitive nature of the information surveyed, with possible underreporting in face-to-face interviews, or to the different characteristics of the Web group because of the specialized nature of the website.
Literature
1.
Khazaal Y, Chatton A, Cochand S, Hoch A, Khankarli MB, Khan R, et al. Internet use by patients with psychiatric disorders in search for general and medical informations. Psychiatr Q. 2008;79:301–9.CrossRefPubMed
2.
Khazaal Y, Chatton A, Cochand S, Coquard O, Fernandez S, Khan R, et al. Brief DISCERN, six questions for the evaluation of evidence-based content of health-related websites. Patient Educ Couns. 2009;77:33–7.CrossRefPubMed
3.
Morel V, Chatton A, Cochand S, Zullino D, Khazaal Y. Quality of web-based information on bipolar disorder. J Affect Disord. 2008;110(3):265–9.CrossRefPubMed
4.
Andersson G, Bergstrom J, Hollandare F, Carlbring P, Kaldo V, Ekselius L. Internet-based self-help for depression: randomised controlled trial. Br J Psychiatry. 2005;187:456–61.CrossRefPubMed
5.
6.
Schaub M, Sullivan R, Haug S, Stark L. Web-based cognitive behavioral self-help intervention to reduce cocaine consumption in problematic cocaine users: randomized controlled trial. J Med Internet Res. 2012;14(6):e166.CrossRefPubMedCentralPubMed
7.
van Gelder MM, Bretveld RW, Roeleveld N. Web-based questionnaires: the future in epidemiology? Am J Epidemiol. 2010;172(11):1292–8.CrossRefPubMed
8.
Becker J, Hungerbuehler I, Berg O, Szamrovicz M, Haubensack A, Kormann A, et al. Development of an integrative cessation program for co-smokers of cigarettes and cannabis: demand analysis, program description, and acceptability. Subst Abuse Treat Prev Policy. 2013;8:33.CrossRefPubMedCentralPubMed
9.
Whitehead L. Methodological issues in Internet-mediated research: a randomized comparison of internet versus mailed questionnaires. J Med Internet Res. 2011;13(4):e109.CrossRefPubMedCentralPubMed
10.
Ramo DE, Prochaska JJ. Broad reach and targeted recruitment using Facebook for an online survey of young adult substance use. J Med Internet Res. 2012;14(1):e28.CrossRefPubMedCentralPubMed
11.
Fenner Y, Garland SM, Moore EE, Jayasinghe Y, Fletcher A, Tabrizi SN, et al. Web-based recruiting for health research using a social networking site: an exploratory study. J Med Internet Res. 2012;14(1):e20.CrossRefPubMedCentralPubMed
12.
Wang YC, Lee CM, Lew-Ting CY, Hsiao CK, Chen DR, Chen WJ. Survey of substance use among high school students in Taipei: web-based questionnaire versus paper-and-pencil questionnaire. J Adolesc Health. 2005;37(4):289–95.CrossRefPubMed
13.
Brigham J, Lessov-Schlaggar CN, Javitz HS, Krasnow RE, McElroy M, Swan GE. Test-retest reliability of web-based retrospective self-report of tobacco exposure and risk. J Med Internet Res. 2009;11(3):e35.CrossRefPubMedCentralPubMed
14.
Miller ET, Neal DJ, Roberts LJ, Baer JS, Cressler SO, Metrik J, et al. Test-retest reliability of alcohol measures: is there a difference between internet-based assessment and traditional methods? Psychol Addict Behav. 2002;16(1):56–63.CrossRefPubMed
15.
Miller PG, Sonderlund AL. Using the internet to research hidden populations of illicit drug users: a review. Addiction. 2010;105(9):1557–67.CrossRefPubMed
16.
Dickerson S, Reinhart AM, Feeley TH, Bidani R, Rich E, Garg VK, et al. Patient Internet use for health information at three urban primary care clinics. J Am Med Inform Assoc. 2004;11(6):499–504.CrossRefPubMedCentralPubMed
17.
Smith AB, King M, Butow P, Olver I. A comparison of data quality and practicality of online versus postal questionnaires in a sample of testicular cancer survivors. Psychooncology. 2013;22(1):233–7.CrossRefPubMed
18.
Piergiorgio C. Social Research: Theory, Methods and Technique. London: SAGE Publications; 2011.
19.
Eaton DK, Brener ND, Kann L, Denniston MM, McManus T, Kyle TM, et al. Comparison of paper-and-pencil versus Web administration of the Youth Risk Behavior Survey (YRBS): risk behavior prevalence estimates. Eval Rev. 2010;34(2):137–53.CrossRefPubMed
20.
Vallejo MA, Jordan CM, Diaz MI, Comeche MI, Ortega J. Psychological assessment via the internet: a reliability and validity study of online (vs paper-and-pencil) versions of the General Health Questionnaire-28 (GHQ-28) and the Symptoms Check-List-90-Revised (SCL-90-R). J Med Internet Res. 2007;9(1):e2.CrossRefPubMedCentralPubMed
21.
Buchanan T. Internet-based questionnaire assessment: appropriate use in clinical contexts. Cogn Behav Ther. 2003;32(3):100–9.CrossRefPubMed
22.
23.
Kelly D, Harper DJ, Landau B. Questionnaire mode effects in interactive information retrieval experiments. Inf Process Manag. 2008;44(1):122–41.CrossRef
24.
Turner CF, Ku L, Rogers SM, Lindberg LD, Pleck JH, Sonenstein FL. Adolescent sexual behavior, drug use, and violence: increased reporting with computer survey technology. Science. 1998;280(5365):867–73.CrossRefPubMed
25.
De Beuckelaer A, Lievens F. Measurement equivalence of paper-and-pencil and Internet organisational surveys: a large scale examination in 16 countries. Appl Psychol. 2009;58(2):336–61.CrossRef
26.
Spek V, Nyklicek I, Cuijpers P, Pop V. Internet administration of the Edinburgh Depression Scale. J Affect Disord. 2008;106(3):301–5.CrossRefPubMed
27.
Donker T, van Straten A, Marks I, Cuijpers P. Quick and easy self-rating of Generalized Anxiety Disorder: validity of the Dutch web-based GAD-7, GAD-2 and GAD-SI. Psychiatry Res. 2011;188(1):58–64.CrossRefPubMed
28.
Donker T, van Straten A, Marks I, Cuijpers P. Brief self-rated screening for depression on the Internet. J Affect Disord. 2010;122(3):253–9.CrossRefPubMed
29.
Lindner P, Andersson G, Ost LG, Carlbring P. Validation of the internet-administered Quality of Life Inventory (QOLI) in different psychiatric conditions. Cogn Behav Ther. 2013;42(4):315–27.CrossRefPubMed
30.
Noack R, Hofler M, Lueken U. Cannabis use patterns and their association with DSM-IV cannabis dependence and gender. Eur Addict Res. 2011;17(6):321–8.CrossRefPubMed
31.
Norberg MM, Gates P, Dillon P, Kavanagh DJ, Manocha R, Copeland J. Screening and managing cannabis use: comparing GP's and nurses’ knowledge, beliefs, and behavior. Subst Abuse Treat Prev Policy. 2012;7:31.CrossRefPubMedCentralPubMed
32.
Rooke SE, Gates PJ, Norberg MM, Copeland J. Applying technology to the treatment of cannabis use disorder: comparing telephone versus Internet delivery using data from two completed trials. J Subst Abuse Treat. 2014;46(1):78–84.CrossRefPubMed
33.
Fernandez-Artamendi S, Fernandez-Hermida JR, Muniz-Fernandez J, Secades-Villa R, Garcia-Fernandez G. Screening of cannabis-related problems among youth: the CPQ-A-S and CAST questionnaires. Subst Abuse Treat Prev Policy. 2012;7(1):13.CrossRefPubMedCentralPubMed
34.
Legleye S, Kraus L, Piontek D, Phan O, Jouanne C. Validation of the Cannabis Abuse Screening Test in a sample of cannabis inpatients. Eur Addict Res. 2012;18(4):193–200.CrossRefPubMed
35.
Bohn MJ, Babor TF, Kranzler HR. The Alcohol Use Disorders Identification Test (AUDIT): validation of a screening instrument for use in medical settings. J Stud Alcohol. 1995;56(4):423–32.CrossRefPubMed
36.
Group WAW. The Alcohol, Smoking and Substance Involvement Screening Test (ASSIST): development, reliability and feasibility. Addiction. 2002;97(9):1183–94.CrossRef
37.
Newcombe DA, Humeniuk RE, Ali R. Validation of the World Health Organization Alcohol, Smoking and Substance Involvement Screening Test (ASSIST): report of results from the Australian site. Drug Alcohol Rev. 2005;24(3):217–26.CrossRefPubMed
38.
Kosten TR, Rounsaville BJ, Kleber HD. Concurrent validity of the addiction severity index. J Nerv Ment Dis. 1983;171(10):606–10.CrossRefPubMed
39.
Heatherton TF, Kozlowski LT, Frecker RC, Fagerstrom KO. The Fagerstrom Test for Nicotine Dependence: a revision of the Fagerstrom Tolerance Questionnaire. Br J Addict. 1991;86(9):1119–27.CrossRefPubMed
40.
Sheehan DV, Lecrubier Y, Sheehan KH, Amorim P, Janavs J, Weiller E, et al. The Mini-International Neuropsychiatric Interview (M.I.N.I.): the development and validation of a structured diagnostic psychiatric interview for DSM-IV and ICD-10. J Clin Psychiatry. 1998;59:Suppl 20:22–33–quiz 34–57.
41.
Humeniuk R, Ali R, Babor TF, Farrell M, Formigoni ML, Jittiwutikarn J, et al. Validation of the Alcohol, Smoking And Substance Involvement Screening Test (ASSIST). Addiction. 2008;103(6):1039–47.CrossRefPubMed
42.
Hides L, Cotton SM, Berger G, Gleeson J, O'Donnell C, Proffitt T, et al. The reliability and validity of the Alcohol, Smoking and Substance Involvement Screening Test (ASSIST) in first-episode psychosis. Addict Behav. 2009;34(10):821–5.CrossRefPubMed
43.
Khan R, Chatton A, Thorens G, Achab S, Nallet A, Broers B, et al. Validation of the French version of the alcohol, smoking and substance involvement screening test (ASSIST) in the elderly. Subst Abuse Treat Prev Policy. 2012;7(1):14.CrossRefPubMedCentralPubMed
44.
Khan R, Chatton A, Nallet A, Broers B, Thorens G, Achab-Arigo S, et al. Validation of the French version of the alcohol, smoking and substance involvement screening test (ASSIST). Eur Addict Res. 2011;17(4):190–7.CrossRefPubMed
45.
Buchanan T, Ali T, Heffernan TM, Ling J, Parrott AC, Rodgers J, et al. Nonequivalence of on-line and paper-and-pencil psychological tests: the case of the prospective memory questionnaire. Behav Res Methods. 2005;37(1):148–54.CrossRefPubMed
46.
Cole MS, Bedeian AG, Feild HS. The measurement equivalence of Web-based and paper-and-pencil measures of transformational leadership: a multinational test. Organ Res Methods. 2006;9(3):339–68.CrossRef
47.
Streiner DL, Norman GR. Health measurement scales. 4th ed. New York: Oxford University Press; 2008.CrossRef
48.
Joreskog KG. Simultaneous factor analysis in several populations. Psychometrika. 1971;36:409–26.CrossRef
49.
Hoyle RH. Handbook of structural equation modeling. New York: The Guilford Press; 2012.
50.
Laursen BP, Little TD, Card NA. Handbook of developmental research methods. New York: The Guilford Press; 2012.
51.
Byrne BM. Structural equation modeling with AMOS. 2nd ed. New York: Routledge; 2009.
52.
Byrne BM. A primer of LISREL: basic applications and programming for confirmatory factor analytic models. New York: Springer; 1989.CrossRef
53.
Browne M, Cudeck R. Testing structural equation models. Edited by Bollen KA, Long JS. London, Sage Publications; 1993: 111–136.
54.
Hu LT, Bentler PM. Cutoff criteria for fit indexes in covariance structure analysis: conventional criteria versus new alternatives. Struct Equ Modeling. 1999;6:1–55.CrossRef
55.
Bentler PM, Chou C-P. Practical issues in structural modeling. Sociol Methods Res. 1987;16:78–117.CrossRef
56.
Kline R. Principles and practice of structural equation modeling. 3rd ed. New York: The Guilford Press; 2011.
57.
Helms JE, Henze KT, Saas TL, Mifsud VA. Treating Cronbach’s alpha reliability coefficients as data in counseling research. Couns Psychol. 2006;34(5):630–60.CrossRef
58.
Ramo DE, Prochaska JJ. Prevalence and co-use of marijuana among young adult cigarette smokers: an anonymous online national survey. Addict Sci Clin Pract. 2012;7(1):5.CrossRefPubMedCentralPubMed
59.
Khazaal Y, van Singer M, Chatton A, Achab S, Zullino D, Rothen S, et al. Does self-selection affect samples’ representativeness in online surveys? An investigation in online video game research. J Med Internet Res. 2014;16(7):e164.CrossRefPubMedCentralPubMed
60.
Buchanan T, Smith JL. Research on the Internet: validation of a World-Wide Web mediated personality scale. Behav Res Methods Instrum Comput. 1999;31(4):565–71.CrossRefPubMed
61.
Richman WL, Kiesler S, Weisband S, Drasgow F. A meta-analytic study of social desirability distortion in computer-administered questionnaires, traditional questionnaires, and interviews. J Appl Psychol. 1999;84:754–75.CrossRef
62.
Kays K, Gathercoal K, Buhrow W. Does survey format influence self-disclosure on sensitive question items? Comput Human Behav. 2012;28(1):251–6.CrossRef
63.
Duncan DF, White JB, Nicholson T. Using Internet-based surveys to reach hidden populations: case of nonabusive illicit drug users. Am J Health Behav. 2003;27(3):208–18.CrossRefPubMed
64.
Newton NC, Teesson M, Vogl LE, Andrews G. Internet-based prevention for alcohol and cannabis use: final results of the Climate Schools course. Addiction. 2010;105(4):749–59.CrossRefPubMed
65.
Hoch E, Buhringer G, Pixa A, Dittmer K, Henker J, Seifert A, et al. CANDIS treatment program for cannabis use disorders: findings from a randomized multi-site translational trial. Drug Alcohol Depend. 2014;134:185–93.CrossRefPubMed
66.
Schaub MP, Haug S, Wenger A, Berg O, Sullivan R, Beck T, et al. Can reduce–the effects of chat-counseling and web-based self-help, web-based self-help alone and a waiting list control program on cannabis use in problematic cannabis users: a randomized controlled trial. BMC Psychiatry. 2013;13:305.CrossRefPubMedCentralPubMed
Metadata
Title
Internal consistency and measurement equivalence of the cannabis screening questions on the paper-and-pencil face-to-face ASSIST versus the online instrument
Authors
Yasser Khazaal
Anne Chatton
Grégoire Monney
Audrey Nallet
Riaz Khan
Daniele Zullino
Jean-François Etter
Publication date
01-12-2015
Publisher
BioMed Central
Published in
Substance Abuse Treatment, Prevention, and Policy / Issue 1/2015
Electronic ISSN: 1747-597X
DOI
https://doi.org/10.1186/s13011-015-0002-9

Other articles of this Issue 1/2015

Substance Abuse Treatment, Prevention, and Policy 1/2015 Go to the issue

Research

Building a bonfire that remains stoked: sustainment of a contingency management intervention developed through collaborative design

Research

Russian roulette with unlicensed fat-burner drug 2,4-dinitrophenol (DNP): evidence from a multidisciplinary study of the internet, bodybuilding supplements and DNP users

Research

Traumatic experiences and re-victimization of female inmates undergoing treatment for substance abuse

Review

Polypharmacy among anabolic-androgenic steroid users: a descriptive metasynthesis

Research

Views about responsibility for alcohol addiction and negative evaluations of naltrexone

Short Report

Smoking cessation among transit workers: beliefs and perceptions among an at-risk occupational group