Skip to main content
Top
Published in: Irish Journal of Medical Science (1971 -) 1/2022

01-02-2022 | Coronavirus | Original Article

Sentiment analysis of user feedback on the HSE’s Covid-19 contact tracing app

Authors: Kaavya Rekanar, Ian R. O’Keeffe, Sarah Buckley, Manzar Abbas, Sarah Beecham, Muslim Chochlov, Brian Fitzgerald, Liam Glynn, Kevin Johnson, John Laffey, Bairbre McNicholas, Bashar Nuseibeh, James O’Connell, Derek O’Keeffe, Mike O’Callaghan, Abdul Razzaq, Ita Richardson, Andrew Simpkin, Cristiano Storni, Damyanka Tsvyatkova, Jane Walsh, Thomas Welsh, Jim Buckley

Published in: Irish Journal of Medical Science (1971 -) | Issue 1/2022

Login to get access

Abstract

Background

Digital Contact Tracing is seen as a key tool in reducing the propagation of Covid-19. But it requires high uptake and continued participation across the population to be effective. To achieve sufficient uptake/participation, health authorities should address, and thus be aware of, user concerns.

Aim

This work manually analyzes user reviews of the Irish Heath Service Executive’s (HSE) Contact Tracker app, to identify user concerns and to lay the foundations for subsequent, large-scale, automated analyses of reviews. While this might seem tightly scoped to the Irish context, the HSE app provides the basis for apps in many jurisdictions in the USA and Europe.

Methods

Manual analysis of (1287) user reviews from the Google/Apple playstores was performed, to identify the aspects of the app that users focused on, and the positive/negative sentiment expressed.

Results

The findings suggest a largely positive sentiment towards the app, and that users thought it handled data protection and transparency aspects well. But feedback suggests that users would appreciate more targeted feedback on the incidence of the virus, and facilities for more proactive engagement, like notifications that prompt users to submit their health status daily. Finally, the analysis suggests that the “android battery” issue and the backward-compatibility issue with iPhones seriously impacted retention/uptake of the app respectively.

Conclusion

The HSE have responded to the public’s desire for targeted feedback in newer versions, but should consider increasing the app’s proactive engagement. The results suggest they should also raise the backward compatibility issue, regarding older iPhones, with Apple.
Literature
1.
go back to reference Ferretti L, Wymant C, Kendall M et al (2020) Quantifying SARS-CoV-2 transmission suggests epidemic control with digital contact tracing. Science 368:6491CrossRef Ferretti L, Wymant C, Kendall M et al (2020) Quantifying SARS-CoV-2 transmission suggests epidemic control with digital contact tracing. Science 368:6491CrossRef
6.
go back to reference O’Callaghan M.E, Buckley J, Fitzgerald B et al (2020) A national survey of attitudes to COVID-19 digital contact tracing in the Republic of Ireland. Ir J Med Sci (1971-) pp. 1–25 O’Callaghan M.E, Buckley J, Fitzgerald B et al (2020) A national survey of attitudes to COVID-19 digital contact tracing in the Republic of Ireland. Ir J Med Sci (1971-) pp. 1–25
10.
go back to reference Guzman E, Maalej W (2014) How do users like this feature? A fine grained sentiment analysis of app reviews. IEEE 22nd International Requirements Engineering Conference (RE), pp. 153–162 Guzman E, Maalej W (2014) How do users like this feature? A fine grained sentiment analysis of app reviews. IEEE 22nd International Requirements Engineering Conference (RE), pp. 153–162
11.
go back to reference McIlroy S, Ali N, Khalid H, Hassan AE (2016) Analyzing and automatically labelling the types of user issues that are raised in mobile app reviews. Empir Softw Eng 21(3):1067–1106CrossRef McIlroy S, Ali N, Khalid H, Hassan AE (2016) Analyzing and automatically labelling the types of user issues that are raised in mobile app reviews. Empir Softw Eng 21(3):1067–1106CrossRef
12.
go back to reference Maalej W, Nabil H (2015) Bug report, feature request, or simply praise? On automatically classifying app reviews. IEEE 23rd International Requirements Engineering Conference (RE), pp. 116–125 Maalej W, Nabil H (2015) Bug report, feature request, or simply praise? On automatically classifying app reviews. IEEE 23rd International Requirements Engineering Conference (RE), pp. 116–125
14.
go back to reference Romesburg C (2004) Cluster analysis for researchers. Lulu Press Romesburg C (2004) Cluster analysis for researchers. Lulu Press
15.
go back to reference van Haasteren A, Vayena E, Powell J (2020) The Mobile Health App Trustworthiness Checklist: usability assessment. JMIR mHealth and uHealth 8(7):e16844CrossRef van Haasteren A, Vayena E, Powell J (2020) The Mobile Health App Trustworthiness Checklist: usability assessment. JMIR mHealth and uHealth 8(7):e16844CrossRef
16.
go back to reference Vokinger KN, Nittas V, Witt CM (2020) Digital health and the COVID-19 epidemic: an assessment framework for apps from an epidemiological and legal perspective. Swiss Medical Weekly, 150(1920) Vokinger KN, Nittas V, Witt CM (2020) Digital health and the COVID-19 epidemic: an assessment framework for apps from an epidemiological and legal perspective. Swiss Medical Weekly, 150(1920)
17.
go back to reference Kirilenko AP, Stepchenkova SO, Kim H, Li X (2018) Automated sentiment analysis in tourism: comparison of approaches. J Travel Res 57(8):1012–1025CrossRef Kirilenko AP, Stepchenkova SO, Kim H, Li X (2018) Automated sentiment analysis in tourism: comparison of approaches. J Travel Res 57(8):1012–1025CrossRef
18.
go back to reference Andreevskaia A, Bergler S, Urseanu M (2007) All blogs are not made equal: exploring genre differences in sentiment tagging of blogs. International Conference and Web and Social Media Andreevskaia A, Bergler S, Urseanu M (2007) All blogs are not made equal: exploring genre differences in sentiment tagging of blogs. International Conference and Web and Social Media
19.
go back to reference Berelson B (1952) Content analysis in communication research. American Psychological Association, APA PsycNet Berelson B (1952) Content analysis in communication research. American Psychological Association, APA PsycNet
20.
go back to reference Sharif KY, English M, Ali N et al (2015) An empirically-based characterization and quantification of information seeking through mailing lists during open source developers’ software evolution. Inf Softw Technol 57:77–94CrossRef Sharif KY, English M, Ali N et al (2015) An empirically-based characterization and quantification of information seeking through mailing lists during open source developers’ software evolution. Inf Softw Technol 57:77–94CrossRef
21.
go back to reference Herold S, Blom M, Buckley J (2016) Evidence in architecture degradation and consistency checking research: preliminary results from a literature review. Proceedings of the 10th European Conference on Software Architecture Workshops, pp. 1–7 Herold S, Blom M, Buckley J (2016) Evidence in architecture degradation and consistency checking research: preliminary results from a literature review. Proceedings of the 10th European Conference on Software Architecture Workshops, pp. 1–7
22.
go back to reference Fleiss JL (1971) Measuring nominal scale agreement among many raters. Psychol Bull 76:378–382CrossRef Fleiss JL (1971) Measuring nominal scale agreement among many raters. Psychol Bull 76:378–382CrossRef
23.
go back to reference Hartmann DP (1977) Considerations in the choice of interobserver reliability estimates. J Appl Behav Anal 10(1):103–116CrossRef Hartmann DP (1977) Considerations in the choice of interobserver reliability estimates. J Appl Behav Anal 10(1):103–116CrossRef
24.
go back to reference Landis JR, Koch GG (1977) The measurement of observer agreement for categorical data. Biometrics 33:159–174CrossRef Landis JR, Koch GG (1977) The measurement of observer agreement for categorical data. Biometrics 33:159–174CrossRef
25.
go back to reference Ha E, Wagner D (2013) Do android users write about electric sheep? Examining consumer reviews in Google Play. IEEE 10th Consumer Communications and Networking Conference (CCNC), Las Vegas, NV, pp. 149–157. Ha E, Wagner D (2013) Do android users write about electric sheep? Examining consumer reviews in Google Play. IEEE 10th Consumer Communications and Networking Conference (CCNC), Las Vegas, NV, pp. 149–157.
26.
go back to reference Buckley J, DeWille T, Exton C et al (2018) A gamification–motivation design framework for educational software developers. Journal of Educational Technology Systems 47(1):101–127CrossRef Buckley J, DeWille T, Exton C et al (2018) A gamification–motivation design framework for educational software developers. Journal of Educational Technology Systems 47(1):101–127CrossRef
27.
go back to reference Wright HK, Kim M, Perry DE (2010) Validity concerns in software engineering research. Proceedings of the FSE/SDP Workshop on (the) Future of Software Engineering Research, pp. 411–414 Wright HK, Kim M, Perry DE (2010) Validity concerns in software engineering research. Proceedings of the FSE/SDP Workshop on (the) Future of Software Engineering Research, pp. 411–414
28.
go back to reference Welsh T, Rekanar K, Abbas M et al (2020) Towards a taxonomy for evaluating societal concerns of contact tracing apps. 7th International Conference on Behavioral, Economic, and Socio-Cultural Computing (BESC) Welsh T, Rekanar K, Abbas M et al (2020) Towards a taxonomy for evaluating societal concerns of contact tracing apps. 7th International Conference on Behavioral, Economic, and Socio-Cultural Computing (BESC)
29.
go back to reference Oates BJ (2005) Researching information systems and computing. Sage Oates BJ (2005) Researching information systems and computing. Sage
30.
go back to reference Mokhtarian PL, Cao X (2008) Examining the impacts of residential self-selection on travel behavior: a focus on methodologies. Transportation Research Part B: Methodological 42(3):204–228CrossRef Mokhtarian PL, Cao X (2008) Examining the impacts of residential self-selection on travel behavior: a focus on methodologies. Transportation Research Part B: Methodological 42(3):204–228CrossRef
31.
go back to reference Heckman JJ (1990) Selection bias and self-selection. Eatwell J, Milgate M, Newman P. (eds) Econometrics. Palgrave Macmillan Heckman JJ (1990) Selection bias and self-selection. Eatwell J, Milgate M, Newman P. (eds) Econometrics. Palgrave Macmillan
32.
go back to reference Gibney S, McCarthy Technical Research Brief (2020) Profile of Smartphone Ownership and Use in Ireland Research Services and Policy Unit, Research & Development and Health Analytics Division. Available from corresponding author, on request, Department of Health Gibney S, McCarthy Technical Research Brief (2020) Profile of Smartphone Ownership and Use in Ireland Research Services and Policy Unit, Research & Development and Health Analytics Division. Available from corresponding author, on request, Department of Health
34.
go back to reference Yang H, Willis A, De Roeck A, Nuseibeh B (2012) A hybrid model for automatic emotion recognition in suicide notes. Biomedical Informatics Insights, 5: pp. BII– S8948 Yang H, Willis A, De Roeck A, Nuseibeh B (2012) A hybrid model for automatic emotion recognition in suicide notes. Biomedical Informatics Insights, 5: pp. BII– S8948
35.
go back to reference Bakshi RK, Kaur N, Kaur R, Kaur G (2016) Opinion mining and sentiment analysis. 3rd International Conference on Computing for Sustainable Global Development (INDIACom), pp. 452–455 Bakshi RK, Kaur N, Kaur R, Kaur G (2016) Opinion mining and sentiment analysis. 3rd International Conference on Computing for Sustainable Global Development (INDIACom), pp. 452–455
36.
go back to reference Hatzivassiloglou V, Wiebe J (2000) Effects of adjective orientation and gradability on sentence subjectivity in COLING Volume 1: The 18th International Conference on Computational Linguistics Hatzivassiloglou V, Wiebe J (2000) Effects of adjective orientation and gradability on sentence subjectivity in COLING Volume 1: The 18th International Conference on Computational Linguistics
Metadata
Title
Sentiment analysis of user feedback on the HSE’s Covid-19 contact tracing app
Authors
Kaavya Rekanar
Ian R. O’Keeffe
Sarah Buckley
Manzar Abbas
Sarah Beecham
Muslim Chochlov
Brian Fitzgerald
Liam Glynn
Kevin Johnson
John Laffey
Bairbre McNicholas
Bashar Nuseibeh
James O’Connell
Derek O’Keeffe
Mike O’Callaghan
Abdul Razzaq
Ita Richardson
Andrew Simpkin
Cristiano Storni
Damyanka Tsvyatkova
Jane Walsh
Thomas Welsh
Jim Buckley
Publication date
01-02-2022
Publisher
Springer International Publishing
Published in
Irish Journal of Medical Science (1971 -) / Issue 1/2022
Print ISSN: 0021-1265
Electronic ISSN: 1863-4362
DOI
https://doi.org/10.1007/s11845-021-02529-y

Other articles of this Issue 1/2022

Irish Journal of Medical Science (1971 -) 1/2022 Go to the issue