Skip to main content
Key Specialties
Cardiology Diabetology Endocrinology Gastroenterology Geriatrics and Gerontology Gynecology and Obstetrics Hematology Infectious Disease Internal Medicine Nephrology Neurology Oncology Pediatrics Respiratory Medicine Rheumatology Surgery
Congress Coverage
2024 ACC Congress 2023 ESMO Congress 2023 EASD Congress 2023 ERS Congress 2023 ESC Congress
Clinical Collections
Advances in Alzheimer’s

Keynote webinar | Spotlight on medication adherence

LIVE: Thursday 27th June 2024, 18:00-19:30 (CEST)
Join our expert panel to discover why you need to understand the drivers of non-adherence in your patients, and how you can optimize medication adherence in your clinics to drastically improve patient outcomes.
ADVANCED SEARCH
Log in
Top
Current Psychiatry Reports
Published in: Current Psychiatry Reports 12/2014

01-12-2014 | Psychiatry in the Digital Age (JS Luo, Section Editor)

New Measures of Mental State and Behavior Based on Data Collected From Sensors, Smartphones, and the Internet

Authors: Tasha Glenn, Scott Monteith

Published in: Current Psychiatry Reports | Issue 12/2014

Login to get access

Abstract

With the rapid and ubiquitous acceptance of new technologies, algorithms will be used to estimate new measures of mental state and behavior based on digital data. The algorithms will analyze data collected from sensors in smartphones and wearable technology, and data collected from Internet and smartphone usage and activities. In the future, new medical measures that assist with the screening, diagnosis, and monitoring of psychiatric disorders will be available despite unresolved reliability, usability, and privacy issues. At the same time, similar non-medical commercial measures of mental state are being developed primarily for targeted advertising. There are societal and ethical implications related to the use of these measures of mental state and behavior for both medical and non-medical purposes.
Literature
1.
2.
Kapur S, Phillips AG, Insel TR. Why has it taken so long for biological psychiatry to develop clinical tests and what to do about it? Mol Psychiatry. 2012;17:1174–9.PubMedCrossRef
3.
Biomarkers Definitions Working Group. Biomarkers and surrogate endpoints: preferred definitions and conceptual framework. Clin Pharmacol Ther. 2001;69:89–95.CrossRef
4.
Lesko LJ, Atkinson Jr AJ. Use of biomarkers and surrogate endpoints in drug development and regulatory decision making: criteria, validation, strategies. Annu Rev Pharmacol Toxicol. 2001;41:347–66.PubMedCrossRef
5.
6.
McIntyre RS, Cha DS, Jerrell JM, et al. Advancing biomarker research: utilizing ‘Big Data’ approaches for the characterization and prevention of bipolar disorder. Bipolar Disord. 2014;16(5):531–47.PubMedCrossRef
7.
8.
9.
10.
Venta L, Isomursu M, Ahtinen A et al. “My phone is a part of my soul”—how people bond with their mobile phones. In Mobile Ubiquitous Computing, Systems, Services and Technologies, 2008. UBICOMM ’08. The second international conference on IEEE, 2008; 311–17
11.
Srivastava L. Mobile phones and the evolution of social behaviour. Behav Inf Technol. 2005;24:111–29.CrossRef
12.
Korhonen I, Bardram JE. Guest editorial: introduction to the special section on pervasive healthcare. IEEE Trans Inf Technol Biomed. 2004;8:229–34.PubMedCrossRef
13.
14.
15.
Varshney U. Pervasive healthcare and wireless health monitoring. Mob Netw Appl. 2007;12:113–27.CrossRef
16.
17.
18.
19.
Berk M, Brnabic A, Dodd S, et al. Does stage of illness impact treatment response in bipolar disorder? Empirical treatment data and their implication for the staging model and early intervention. Bipolar Disord. 2011;13:87–98.PubMedCrossRef
20.
Killackey E, Yung AR. Effectiveness of early intervention in psychosis. Curr Opin Psychiatry. 2007;20:121–5.PubMedCrossRef
21.
Bulloch AG, Patten SB. Non-adherence with psychotropic medications in the general population. Soc Psychiatry Psychiatr Epidemiol. 2010;45:47–56.PubMedCrossRef
22.
Cramer JA, Rosenheck R. Compliance with medication regimens for mental and physical disorders. Psychiatr Serv. 1998;49:196–201.PubMedCrossRef
23.•
Kane JM, Perlis RH, DiCarlo LA, et al. First experience with a wireless system incorporating physiologic assessments and direct confirmation of digital tablet ingestions in ambulatory patients with schizophrenia or bipolar disorder. J Clin Psychiatry. 2013;74:e533–40. Pilot study using sensors embedded in placebo tablets.PubMedCrossRef
24.
25.
Cook IA. Biomarkers in psychiatry: potentials, pitfalls, and pragmatics. Prim Psychiatry. 2008;15:54.
26.
Wood A, Stankovic JA, Virone G, et al. Context-aware wireless sensor networks for assisted living and residential monitoring. IEEE Netw. 2008;22:26–33.CrossRef
27.
Braddock D, Rizzolo MC, Thompson M, et al. Emerging technologies and cognitive disability. J Spec Educ Technol. 2004;19:49–56.
28.
Milenković A, Otto C, Jovanov E. Wireless sensor networks for personal health monitoring: issues and an implementation. Comput Commun. 2006;29:2521–33.CrossRef
29.
30.
Pentland A. Healthwear: medical technology becomes wearable. Computer. 2004;37:42–9.CrossRef
31.
Alemdar H, Ersoy C. Wireless sensor networks for healthcare: a survey. Comput Netw. 2010;54:2688–710.CrossRef
32.
Yoo H-J. Your heart on your sleeve: advances in textile-based electronics are weaving computers right into the clothes we wear. IEEE Solid-State Circ Mag. 2013;5:59–70.
33.
Lowe SA, Ólaighin G. Monitoring human health behaviour in one’s living environment: a technological review. Med Eng Phys. 2014;36:147–68.PubMedCrossRef
34.
Jerritta S, Murugappan M, Nagarajan R, et al. Physiological signals based human emotion recognition: a review. In Signal Processing and its Applications (CSPA), 2011 I.E. 7th International Colloquium on. IEEE. 2011:410–15
35.
Alvarez-Lozano J, Frost M, Osmani V, et al. Tell me your apps and I will tell you your mood: correlation of apps usage with bipolar disorder state. ACM proceedings of 7th international conference on pervasive technologies related to assistive environments. 2014.
36.
Kotikalapudi R, Chellappan S, Montgomery F, et al. Associating Internet usage with depressive behavior among college students. IEEE Technol Soc Mag. 2012;31:73–80.CrossRef
37.•
38.
39.
40.
Pepe MS, Thompson ML. Combining diagnostic test results to increase accuracy. Biostatistics. 2000;1:123–40.PubMedCrossRef
41.
Pletcher MJ, Pignone M. Evaluating the clinical utility of a biomarker: a review of methods for estimating health impact. Circulation. 2011;123:1116–24.PubMedCentralPubMedCrossRef
42.
43.
Brenner H, Gefeller O. Variation of sensitivity, specificity, likelihood ratios and predictive values with disease prevalence. Stat Med. 1997;16:981–91.PubMedCrossRef
44.
45.
46.
Monteith S, Glenn T, Bauer M. Searching the internet for health information about bipolar disorder: some cautionary issues. Int J Bipolar Disord. 2013;1:22.PubMedCentralPubMedCrossRef
47.•
Ko JG, Lu C, Srivastava MB, et al. Wireless sensor networks for healthcare. Proc IEEE. 2010;98:1947–60. Clear overview article.CrossRef
48.
Chan M, Estève D, Fourniols JY, et al. Smart wearable systems: current status and future challenges. Artif Intell Med. 2012;56:137–56.PubMedCrossRef
49.
Healey J, Nachman L, Subramanian S, et al. Out of the lab and into the fray: towards modeling emotion in everyday life. In Pervasive computing. Springer, Berlin, 2010:156–73.
50.
51.
52.
Luxton DD, McCann RA, Bush NE, et al. mHealth for mental health: integrating smartphone technology in behavioral healthcare. Prof Psychol Res Pr. 2011;42:505.CrossRef
53.
Baig MM, Gholamhosseini H. Smart health monitoring systems: an overview of design and modeling. J Med Syst. 2013;37:9898.PubMedCrossRef
54.
55.
McAdams E, Krupaviciute A, Gehin C, Grenier E, et al. Wearable sensor systems: the challenges. In Engineering in Medicine and Biology Society, EMBC, 2011 Annual international conference of the IEEE. IEEE, 2011:3648–51
56.
Lane ND, Mohammod M, Lin M, et al. Bewell: a smartphone application to monitor, model and promote wellbeing. In 5th international ICST conference on pervasive computing technologies for healthcare. 2011: 23–6.
57.
Banaee H, Ahmed MU, Loutfi A. Data mining for wearable sensors in health monitoring systems: a review of recent trends and challenges. Sensors (Basel). 2013;13:17472–500.CrossRef
58.
Ma Y, Xu B, Bai Y, et al. Daily mood assessment based on mobile phone sensing. In Wearable and Implantable Body Sensor Networks (BSN), 2012 Ninth international conference on. IEEE. 2012: 142–7
59.
Alexe B, Hernandez MA, Hildrum KW, et al. Surfacing time-critical insights from social media. In Proceedings of the 2012 ACM SIGMOD international conference on management of data. ACM, 2012: 657–60.
60.
Agichtein E, Castillo C, Donato D, et al. Finding high-quality content in social media. In Proceedings of the 2008 International conference on web search and data mining. ACM, 2008:183–194.
61.
Wang W, Chen L, Thirunarayan K, et al. Harnessing Twitter ‘big data’ for automatic emotion identification. In Privacy, Security, Risk and Trust (PASSAT), 2012 International conference on and 2012 International conference on social computing (SocialCom). IEEE. 2012:587–592
62.•
Konopnicki D, Shmueli-Scheuer M, Cohen D, et al. A statistical approach to mining customers’ conversational data from social media. IBM J Res Dev. 2013;57:14–1. Example of creating user profiles from social media.CrossRef
63.
64.
65.
66.
Viswanathan H, Chen B, Pompili D. Research challenges in computation, communication, and context awareness for ubiquitous healthcare. IEEE Commun Mag. 2012;50:92–9.CrossRef
67.
Baldauf M, Dustdar S, Rosenberg F. A survey on context-aware systems. Int J Ad Hoc Ubiquit Comput. 2007;2:263–77.CrossRef
68.
Dey AK, Abowd GD, Salber D. A conceptual framework and a toolkit for supporting the rapid prototyping of context-aware applications. Human-computer interaction. 2001;16:97–166.CrossRef
69.
Welbourne E, Lester J, LaMarca A et al. Mobile context inference using low-cost sensors. In Location-and Context-Awareness. Springer, Berlin, 2005:254–63.
70.
Li M, Lou W, Ren K. Data security and privacy in wireless body area networks. Wireless Communications, IEEE. 2010;17:51–8.CrossRef
71.
72.
Halperin D, Kohno T, Heydt-Benjamin TS, et al. Security and privacy for implantable medical devices. IEEE Pervasive Comput. 2008;7:30–9.CrossRef
73.
74.
75.
76.
Pantic M, Sebe N, Cohn JF et al. Affective multimodal human-computer interaction. In Proceedings of the 13th annual ACM international conference on multimedia. ACM, 2005; 669–76.
77.
Schmidt A, Pfleging B, Alt F, et al. Interacting with 21st-century computers. IEEE Pervasive Comput. 2012;11:22–31.CrossRef
78.
Pentland A. Looking at people: sensing for ubiquitous and wearable computing. IEEE Trans Pattern Anal Mach Intell. 2000;22:107–19.CrossRef
79.
Sebe N, Cohen I, Gevers T, et al. Multimodal approaches for emotion recognition: a survey. In Electronic Imaging 2005, International Society for Optics and Photonics, 2005: 56–67
80.
Gunes H, Pantic M. Automatic, dimensional and continuous emotion recognition. Int J Synth Emotions (IJSE). 2010;1:68–99.CrossRef
81.•
Zeng Z, Pantic M, Roisman GI, et al. A survey of affect recognition methods: audio, visual, and spontaneous expressions. A survey of affect recognition methods: audio, visual, and spontaneous expressions. IEEE Trans Pattern Anal Mach Intell. 2009;31:39–58. Clear survey article.PubMedCrossRef
82.
Aggarwal JK, Ryoo MS. Human activity analysis: a review. ACM Comput Surv. 2011;43:16.CrossRef
83.
84.
Yan J, Liu N, Wang G, et al. How much can behavioral targeting help online advertising? In: Proceedings of the 18th international conference on world wide web. ACM. 2009:261–270.
85.
Microsoft, owner. Chung WJ, Patwa P, Markov MM, inventors. Targeting advertisements based on emotion. United States Patent Application 20120143693 A1. 6/7/12
86.
Meyer S, Rakotonirainy A. A survey of research on context-aware homes. In Proceedings of the Australasian information security workshop conference on ACSW frontiers. Australian Computer Society, Inc., 2003; 21:159–68.
87.
Chui M, Löffler M, Roberts R. The internet of things. McKinsey Quarterly 2. 2010: 1–9.
88.
Lisetti CL, Nasoz F. Using noninvasive wearable computers to recognize human emotions from physiological signals. EURASIP J Appl Signal Processing. 2004: 1672–87.
89.
D’Mello S, Picard R, Graesser A. Towards an affect-sensitive autotutor. IEEE Intell Syst. 2007;22:53–61.CrossRef
90.
Schröder M, Baggia P, Burkhardt F, et al. EmotionML—an upcoming standard for representing emotions and related states. In Affective Computing and Intelligent Interaction. Springer, Berlin. 2011:316–25.
91.
Littlewort G, Whitehill J, Wu T et al. The computer expression recognition toolbox (CERT). In Automatic Face & Gesture Recognition and Workshops (FG 2011). IEEE International conference on. IEEE. 2011:298–305
92.
93.
94.
95.
96.
97.
Overton SL, Medina SL. The stigma of mental illness. J Couns Dev. 2008;86:143–51.CrossRef
98.
Luxton DD, June JD, Kinn JT. Technology-based suicide prevention: current applications and future directions. Telemed J E Health. 2011;17:50–4.PubMedCrossRef
99.
Glascock AP, Kutzik DM. Behavioral telemedicine: a new approach to the continuous nonintrusive monitoring of activities of daily living. Telemed J. 2000;6:33–44.CrossRef
100.
Niemeijer AR, Frederiks BJ, Riphagen II, et al. Ethical and practical concerns of surveillance technologies in residential care for people with dementia or intellectual disabilities: an overview of the literature. Int Psychogeriatr. 2010;22:1129–42.PubMedCrossRef
101.
102.
103.
Glenn T, Monteith S. Privacy in the digital world: medical and health data outside of HIPAA protections. Curr Psychiatry Rep. 2014;16(11):494.PubMedCrossRef
104.
105.
Labrecque LI, Markos E, Milne GR. Online personal branding: processes, challenges, and implications. J Interact Mark. 2011;25:37–50.CrossRef
106.
Stark M, Fins JJ. Engineering medical decisions: computer algorithms and the manipulation of choice. Camb Q Healthc Ethics. 2013;22:373–81.PubMedCrossRef
107.
Petersen A, Seear K. Technologies of hope: techniques of the online advertising of stem cell treatments. New Genet Soc. 2011;30:329–46.CrossRef
108.
Lazer D, Kennedy R, King G, et al. Big data. The parable of Google Flu: traps in big data analysis. Science. 2014;343:1203–5.PubMedCrossRef
109.
Simpson MP. The future of innovation-trade secrets, property rights, and protectionism-an age-old tale. Brook L Rev. 2005;70:1121.
110.
Chang KH, Fisher D, Canny J, et al. How’s my mood and stress?: an efficient speech analysis library for unobtrusive monitoring on mobile phones. In Proceedings of the 6th international conference on body area networks. ICST (Institute for Computer Sciences, Social-Informatics and Telecommunications Engineering), 2011; 71–7.
111.
Dickerson RF, Gorlin EI, Stankovic JA. Empath: a continuous remote emotional health monitoring system for depressive illness. In Proceedings of the 2nd conference on wireless health. ACM. 2011: 5.
112.
Roh T, Bong K, Hong S, et al. Wearable mental-health monitoring platform with independent component analysis and nonlinear chaotic analysis. Conf Proc IEEE Eng Med Biol Soc. 2012;2012:4541–4.PubMed
113.
Vidal M, Turner J, Bulling A, et al. Wearable eye tracking for mental health monitoring. Comput Commun. 2012;35:1306–11.CrossRef
114.
115.
Gruenerbl A, Osmani V, Bahle G, et al. Using smart phone mobility traces for the diagnosis of depressive and manic episodes in bipolar patients. In Proceedings of the 5th augmented human international conference. ACM, 2014:38.
116.
Kappeler-Setz C, Gravenhorst F, Schumm J, et al. Towards long term monitoring of electrodermal activity in daily life. Pers Ubiquitous Comput. 2013;17:261–71.CrossRef
117.
Jashinsky J, Burton SH, Hanson CL, et al. Tracking suicide risk factors through Twitter in the US. Crisis. 2014;35:51–9.PubMed
118.
McIntyre G, Gocke R, Hyett M, et al. An approach for automatically measuring facial activity in depressed subjects. In Affective computing and intelligent interaction and workshops, 2009. ACII 2009. 3rd International conference on, IEEE, 2009:1–8
119.
Matic A, Mehta P, Rehg JM, et al. Monitoring dressing activity failures through RFID and video. Methods Inf Med. 2012;51:45–54.PubMedCrossRef
120.
Miskelly F. Electronic tracking of patients with dementia and wandering using mobile phone technology. Age Ageing. 2005;34:497–9.PubMedCrossRef
121.
122.
123.•
Lu H, Frauendorfer D, Rabbi M, et al. StressSense: detecting stress in unconstrained acoustic environments using smartphones. In Proceedings of the 2012 ACM conference on ubiquitous computing. ACM. 2012:351–360. Example of voice analysis using smartphones.
124.
Pandey S, Aly M, Bagherjeiran A, et al. Learning to target: what works for behavioral targeting. In Proceedings of the 20th ACM international conference on Information and knowledge management. ACM. 2011:1805–14.
125.
Aly M, Hatch A, Josifovski V, et al. Web-scale user modeling for targeting. In Proceedings of the 21st international conference companion on World Wide Web. ACM. 2012:3–12.
126.
127.•
128.
Sahami Shirazi A, Clawson J, Hassanpour Y, et al. Already up? using mobile phones to track & share sleep behavior. Int J Hum Comput Stud. 2013;71:878–88.CrossRef
129.
Samsung, owner. Lee H, inventor. Apparatus and method for sharing user’s emotion. United States Patent Application 20130144937 A1. 7 Aug 2012.
130.
Apple, owner. Greenzeiger MF, Phulari R, Sanghavi M, inventors. Inferring user mood based on user and group characteristic data. United States Patent Application 20140025620 A1. 23 Jan 2014.
131.
Microsoft, owner. Roseway A, Davis FA, Carroll EA, et al. inventors. Mood-actuated device. United States Patent Application 20140085181 A1. 27 Mar 2014
132.
Yahoo, owner. White W, Slaney M, inventors. System and method for generating a playlist from a mood gradient. United States Patent Application US8583615 B2. 12 Nov 2013
133.
Yahoo, owner. Kamdar G, inventor. Method and system for voice based mood analysis. United States Patent Application 20140046660 A1. 13 Feb 2014
134.
Metadata
Title
New Measures of Mental State and Behavior Based on Data Collected From Sensors, Smartphones, and the Internet
Authors
Tasha Glenn
Scott Monteith
Publication date
01-12-2014
Publisher
Springer US
Published in
Current Psychiatry Reports / Issue 12/2014
Print ISSN: 1523-3812
Electronic ISSN: 1535-1645
DOI
https://doi.org/10.1007/s11920-014-0523-3

Other articles of this Issue 12/2014

Current Psychiatry Reports 12/2014 Go to the issue

Mood Disorders (JF Greden, Section Editor)

Population-Based Initiatives in College Mental Health: Students Helping Students to Overcome Obstacles

Psychiatry in the Digital Age (JS Luo, Section Editor)

Professional Social Networking

Psychiatry in the Digital Age (JS Luo, Section Editor)

Internet Interventions for Mental Health and Addictions: Current Findings and Future Directions

Mood Disorders (JF Greden, Section Editor)

Proceed with Caution: Off-label Ketamine Treatment for Major Depressive Disorder

Women's Mental Health (D Rubinow, Section Editor)

Nonpharmacologic Treatments for Depression Related to Reproductive Events

Women’s Mental Health (D Rubinow, Section Editor)

Hormone Replacement Therapy in the Treatment of Perimenopausal Depression