Bringing Objective Digital Measures to Psychiatric Care

A persistent challenge in psychiatry is the lack of objective measures for evaluating and tracking mental health over time.

By harnessing recent technological breakthroughs in physiological monitoring and machine learning, researchers at the Johns Hopkins Applied Physics Laboratory (APL) in Laurel, Maryland, in collaboration with psychiatrists at Johns Hopkins Medicine (JHM) in Baltimore, have made significant strides toward creating objective digital measures for assessing and tracking mental health.

Tagged TEAPOT by its APL developers, the project encompasses a telehealth suite for performing “digital phenotyping” for psychiatric health to provide physicians with clinically relevant data during virtual visits. While traditional phenotyping entails deriving a person’s physical appearance from their DNA, digital phenotyping taps physiological data to estimate symptoms associated with psychiatric disorders, such as depression, anxiety and post-traumatic stress disorder.

A Complement to Personal Care

Although TEAPOT leverages recent advances in machine learning and artificial intelligence, principal investigator Erika Rashka — an APL optical engineer with experience in biometric sensing, image processing and sensor algorithms — is quick to note that the suite is not meant to replace a human practitioner, but rather, to give them access to higher-quality, actionable information about their patients.

To that end, Rashka and her APL colleagues have been collaborating closely with Peter Zandi, co-director of the Precision Medicine Center of Excellence in Mood Disorders and vice chair of Precision Medicine in the Department of Psychiatry and Behavioral Sciences within the Johns Hopkins University School of Medicine, to create outcomes that really matter to psychiatrists.

“I’m excited to be a part of this collaboration with the team at APL,” Zandi said. “This project has the potential to transform the way we provide care to those with mood disorders.”

The first step, Rashka said, was to have multiple discussions with Zandi to figure out which modalities to pursue. Psychiatrists often rely on clinical questionnaires that use the Likert scale, where patients connect numerical values to statements on emotional states and recent challenges. While helpful, Rashka said, such measures are imprecise, and the data is not usually retained in a way that supports long-term monitoring.

Measuring Emotion Objectively

Armed with knowledge about psychiatrists’ needs and effective ways to assess various mood disorders, the APL team is developing a set of tools that could be used to extract and analyze as much useful information as possible from telehealth video recordings. To objectively measure emotion, the team focused on two quantitative metrics: valence and arousal. Emotional valence relates to emotional state and ranges from pleasant to unpleasant, while arousal relates to physiological activity and ranges from low to high. Think of valence and arousal as axes to map a person’s emotional status; for example, happiness can be characterized as a pleasant, high-arousal state, whereas sadness is typically characterized by an unpleasant, low-arousal state.

This effort called for creative, collaborative work combining various sensing modalities. Rai Munoz, an APL computer vision engineer, developed a custom neural network for detecting emotional states from video data. The team also created machine learning algorithms specific to biometric analysis with support from APL software developer Diego Luna, building off work by biomedical engineer and colleague Bobby Wilson. On top of this, the team built on earlier work utilizing open-source machine learning models to predict emotional states from audio data, with additional support from software developer Taylor Kramer.

The team has designed a multimodal framework to integrate each of these models with the hope that TEAPOT will be able to use telehealth session data to produce an objective estimate of a person’s depression, anxiety or stress level that aligns with clinical questionnaires.

A tool like TEAPOT might one day help expand access to and the efficacy of mental health care in neurodiverse populations, said Hannah Cowley, an APL neuroscience researcher working on a master’s in public health from the Johns Hopkins Bloomberg School of Public Health. She has provided her clinical research experience in developing experimental protocols for work with human participants.

“TEAPOT is a fabulous effort,” Cowley said. “Our team is trying to quantify the unquantifiable in a way that could be very powerful for people who, for one reason or another, find it difficult or impossible to articulate what their mental state is to someone else.”

Sheri Lewis, a leader in APL’s Global Health Mission Area, said projects like TEAPOT are made possible by the unique synergy between the Laboratory and JHM.

“The combination of JHM’s world-class medical expertise and APL’s world-class science and engineering knowledge affords us the opportunity to develop real-world solutions that have the potential to make a positive impact on health outcomes,” Lewis said. “Mental health struggles affect everyone — young and old, civilian and military, people from every conceivable background and demographic. It’s deeply rewarding for our team to be able to improve the delivery of care in this field.”

APL’s Han Yi, Mattson Ogg, Marianne Engel and Teresa Colella have also contributed subject-matter expertise to this effort.