New AI phone apps can detect signs of mental illness

New smartphone apps, developed independently, are showing some success at detecting signs of depression and bipolar disorder.

In Poland, researchers have developed an app that analyses voice data to assess the severity of manic and depressive symptoms in people with bipolar disorder, based on features of their speech. Developing a model based on the data to predict mood states, the app has achieved an accuracy rate of 71%. The study was published in Acta Psychiatrica Scandinavica.

Bipolar disorder is a serious mental health condition marked by extreme shifts in mood, ranging from emotional highs, known as mania or hypomania, to emotional lows, or depression. Each new episode tends to worsen the course of the illness, making it critical to detect early signs of symptoms. By intervening early enough, it should be possible to prevent a full-blown episode, reducing the long-term impact on patients’ lives.

The researchers, led by Katarzyna Kaczmarek-Majer of the Polish Academy of Sciences, had two aims: to identify speech characteristics most closely correlated with these mood states and to develop a statistical model that could predict the severity of symptoms based on those characteristics.

The study involved 51 patients diagnosed with bipolar disorder, with an average age of 36 years. Twenty-eight of the participants were female. Participants installed the BDmon app, which was automatically activated whenever the participant made or received a phone call, recording the first five minutes of each conversation. During these calls, the app collected various speech parameters, such as pitch, loudness, and speech rate, and analysed them in real time. To ensure privacy, the app deleted the recording after analysing the speech characteristics, storing only the extracted features.

Participants used the app for an average of 208 days. Every three months psychiatrists regularly assessed the severity of their manic and depressive symptoms using clinical tools.

The results revealed significant correlations between certain speech features and the severity of bipolar symptoms, with marked differences between males and females. Men with more severe depressive symptoms tended to speak more quietly and with less energy, their speech was more slurred, their voice was smoother and they often made longer phone calls.

For women, however, the study found no significant correlations between speech characteristics and overall depression severity. The only exception was in cases of psychomotor retardation – a condition in which physical and mental processes slow down markedly. In women with this symptom, the researchers observed louder speech with more irregularities in voice intensity.

Men and women also had different speech patterns in cases of mania. Men with severe manic symptoms spoke louder and more energetically, with rougher voices, more variability in voice intensity, and a sharper tone. Women, in contrast, tended to speak more quietly and with less energy, using a lower-pitched voice. Their speech was also slower and more slurred, and their voices were less rough and irregular.

Using these speech patterns, the researchers developed a predictive model to estimate the severity of manic and depressive symptoms. The model was able to predict mood states with approximately 71% accuracy, suggesting that voice analysis could be used to help health care providers identify when a patient might be transitioning into a manic or depressive episode.

Analysis of a user’s pupils can help detect signs of depression

Separately, researchers at the Stevens Institute of Technology in the US have developed an app called PupilSense, which works by taking snapshots and measurements of a smartphone user’s pupils.

Sang Won Bae, an assistant professor at the Institute, said: “Previous research over the past three decades has repeatedly demonstrated how pupillary reflexes and responses can be correlated to depressive episodes.”

The system accurately calculates pupils’ diameters from 10-second “burst” photo streams captured while users are opening their phones or accessing social media.

Tested on 25 volunteers over a four-week period, the software analysed approximately 16,000 interactions with phones after collecting pupil-image data. Bae and her PhD student Rahul Islam then developed an AI tool that could differentiate between “normal” responses and abnormal ones.

Having processed the photo data and compared it with the volunteers’ self-reported moods, they found that the best iteration of PupilSense proved 76% accurate at identifying times when people did indeed feel depressed.

The app is now available open-source on the GitHub platform