How artificial intelligence detects burnouts via linguistic data
Computers are fed words and phrases every day. Trained programs can recognise or even predict patterns in this data. Scientists from the Applied Machine Intelligence research group at the Department of Technology & Computer Science at BFH now want to find out whether this possibility can be used for health prevention. Specifically: Can methods of computational linguistics be used to detect burnout more efficiently in a clinical setting? The BurnoutWords project is investigating this question.
A study in Switzerland [1] estimated that 24.2% of employees are often or always stressed at work, and 35.2% feel exhausted most of the time (22.2%) or always (13%) at the end of the working day. Sometimes such chronic stress at work can lead to burnout. The World Health Organization (WHO) has included burnout as a syndrome in the 11th Revision of the International Classification of Diseases (ICD-11) in 2019 [2].
Clinical recognition of this syndrome is sometimes difficult because the symptoms often overlap with other syndromes or disorders, such as depression. In clinical psychology, questionnaires(inventories) based on multiple-choice tick-box questions are used to identify burnout (e.g. the Maslach Burnout Inventory [3]). In the literature, this approach, although proven and regularly used in studies and in clinical practice, is sometimes questioned. Problems with such questionnaires can be that patients do not answer honestly (e.g., [4][5]) or avoid the more extreme answers (or select them particularly often) [6][7]. There is much potential in extending such questionnaires with open-ended questions, or analysing transcripts of interviews. So far, however, such approaches have not yet gained acceptance due to the time-consuming manual evaluation.
Application of computational linguistics and machine learning
However, using methods from computational linguistics and with the help of machine learning, this should be possible in the future and enable new methods for clinical psychology/psychiatry. The BurnoutWords project, which is supported by the Swiss National Science Foundation SNSF and the Hasler Foundation, is using the example of burnout recognition to investigate how indications of a syndrome or diagnosis can be automatically recognised in texts. This basic research will later enable the development of new methods involving natural language to complement existing questionnaires. The big challenge here is that such methods are very data-hungry and textual examples from affected individuals are needed in sufficiently large quantities. Rarely can data from past studies be used, but the reuse of such data is often not possible for data protection reasons. In order to develop and initially test the methods, the researchers therefore first used anonymised texts from online forums, analogous to existing research for depression detection [8]. In a next step, the methods will be further tested and improved with a dataset that will be created in cooperation with clinical institutions.
BurnoutEnsemble
The researchers were able to show that methods from the field of computational linguistics are promising. The results were described in a scientific paper published this month in the journal Frontiers in Big Data [9]. Anonymised texts from the online platform Reddit were classified into three categories, depending on the thread in which they were published and partly by means of manual selection: Burnout, depression and a control group of various other topics. Based on this, various systems (so-called classifiers) were trained using machine learning. In order to achieve the best possible result, these systems were then combined into a so-called ensemble classifier. A text segment to be evaluated by the system is evaluated by the different classifiers, which determine whether there is an indication of burnout. By voting on these different results of the individual classifiers, the ensemble classifier then determines whether an indication for burnout should be assumed or not. The proposed system is in the field of augmented intelligence and would support the clinical professional in a later application. Augmented intelligence aims to support people in their daily tasks by means of artificial intelligence, not to replace them.
What comes next?
The results from this initial project, which explores the basics, are promising and must now be further validated. On the one hand, text data from burnout sufferers will be collected in collaboration with clinical partners. The systems, based on the data from online forums used in this first phase, need to be validated. In addition, this approach makes it possible to test the methods for other languages such as German and French. Due to data availability, English was used in the above study. Due to the completely anonymised data in this study, the training data must also be extended to prevent bias. It must be ensured that the final training data for such a system covers all groups of society as well as possible.
Acknowledgement
The Applied Machine Intelligence research group would like to thank the Swiss National Science Foundation SNSF and the Hasler Foundation for supporting the project.
References
- [1] SECO (2015). The Sixth European Working Conditions Survey (EWCS).
- [2] https://icd.who.int/browse11/l-m/en#/http://id.who.int/icd/entity/129180281
- [3] Maslach, C., Jackson, S. E., & Leiter, M. P.. (1997). Maslach burnout inventory. Scarecrow Education.
- [4] Holden, R. R. (2007). Socially desirable responding does moderate personality scale validity both in experimental and in nonexperimental contexts. Can. J. Behav. Sci./Revue canadienne des sciences du comportement 39, 184. doi: 10.1037/cjbs2007015
- [5] Lambert, C. E. (2013). Identifying Faking on Self-Report Personality Inventories: Relative Merits of Traditional Lie Scales, New Lie Scales, Response Patterns, and Response Times (Kingston, ON: Queen’s University).
- [Greenleaf, E. A. (1992). Measuring extreme response style. Publ. Opin. Q. 56, 328-351.
- [7] Brulé, G., and Veenhoven, R. (2017). The ’10 excess’ phenomenon in responses to survey questions on happiness. Soc. Indicators Res. 131, 853-870. doi: 10.1007/s11205-016-1265-x
- [8] Tadesse, M. M., Lin, H., Xu, B., and Yang, L. (2019). Detection of depression-related posts in reddit social media forum. IEEE Access 7, 44883-44893. doi: 10.1109/ACCESS.2019.2909180
- [9] Merhbene, G., Nath, S., Puttick, A.R. and Kurpicz-Briki, M. (2022). BurnoutEnsemble: Augmented Intelligence to Detect Indications for Burnout in Clinical Psychology. Front. Big Data 5:863100. doi: 10.3389/fdata.2022.863100