Abstract
The psychological health of a person plays an important role in their daily life activities. The paper addresses depression issues with the machine learning model using facial expressions of the patient. Some research has already been done on visual based on depression detection methods, but those are illumination variant. The paper uses feature extraction using LBP (Local Binary Pattern) descriptor, which is illumination invariant. The Viola-Jones algorithm is used for face detection and SVM (support vector machine) is considered for classification along with the LBP descriptor to make a complete model for depression level detection. The proposed method captures frontal face from the videos of subjects and their facial features are extracted from each frame. Subsequently, the facial features are analyzed to detect depression levels with the post-processing model. The performance of the proposed system is evaluated using machine learning algorithms in MATLAB. For the real-time system design, it is necessary to test it on the hardware platform. The LBP descriptor has been implemented on FPGA using Xilinx VIVADO 16.4. The results of the proposed method show satisfactory performance and accuracy for depression detection comparison with similar previous work.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.References
WH Organization et al (2017) Depression and other common mental disorders: global health estimates. Technical Report, World Health Organization
Singh OP (2018) Closing treatment gap of mental disorders in India: Opportunity in new competency-based medical council of India curriculum. Indian J psychiatry 60:375
Cao Z, Lin C-T, Ding W, Chen M-H, Li C-T, Su T-P (2018) Identifying ketamine responses in treatment-resistant depression using a wearable forehead eeg. IEEE Trans Biomed Eng 66:1668–1679
Zhu Y, Shang Y, Shao Z, Guo G (2017) Automated depression diagnosis based on deep networks to encode facial appearance and dynamics. IEEE Trans Affect Comput 9:578–584
Uguz F, Yakut E, Aydogan S, Bayman MG, Gezginc K (2019) The impact of maternal major depression, anxiety disorders and their comorbidities on gestational age, birth weight, preterm birth and low birth weight in newborns. J Affect disord 259:382– 385
Zhao K, So H-C (2018) Drug repositioning for schizophrenia and depression/anxiety disorders: A machine learning approach leveraging expression data. IEEE J Biomed Health Inform 23:1304–1315
Pawluski JL, Lonstein JS, Fleming AS (2017) The neurobiology of postpartum anxiety and depression. Trends Neurosci 40:106– 120
Peng H, Xia C, Wang Z, Zhu J, Zhang X, Sun S, Li J, Huo X, Li X (2019) Multivariate pattern analysis of eeg-based functional connectivity: A study on the identification of depression. IEEE Access 7:92630–92641
Pancholi S, Joshi AM (2018) Portable emg data acquisition module for upper limb prosthesis application. IEEE Sensors J 18:3436–3443
Bhadoria RS, Bajpai D (2019) Stabilizing sensor data collection for control of environment-friendly clean technologies using internet of things. Wirel Pers Commun 108:493–510
Bhurane AA, Bhadoria RS (2019) Behavioral biometrics: A prognostic measure for activity recognition. In: The biometric computing: recognition and registration, p 71
Pathak Y, Arya K, Tiwari S (2019) Feature selection for image steganalysis using levy flight-based grey wolf optimization. Multimed Tools Appl 78:1473–1494
Pancholi S, Jain P, Varghese A, et al. (2019) A novel time-domain based feature for emg-pr prosthetic and rehabilitation application. In: 2019 41st annual international conference of the IEEE engineering in medicine and biology society (EMBC). IEEE, pp 5084–5087
Likforman-Sulem L, Esposito A, Faundez-Zanuy M, Clémençon S, Cordasco G (2017) Emothaw: A novel database for emotional state recognition from handwriting and drawing. IEEE Trans Human-Mach Syst 47:273–284
Nasir M, Jati A, Shivakumar PG, Nallan Chakravarthula S, Georgiou P (2016) Multimodal and multiresolution depression detection from speech and facial landmark features. In: Proceedings of the 6th international workshop on audio/visual emotion challenge, pp 43–50
Ackermann P, Kohlschein C, Bitsch JÁ, Wehrle K, Jeschke S Eeg-based automatic emotion recognition: Feature extraction, selection and classification methods, IEEE
Sharma G, Parashar A, Joshi AM (2021) Dephnn: A novel hybrid neural network for electroencephalogram (eeg)-based screening of depression. Biomed Signal Process Control 66:102393
Subhani AR, Mumtaz W, Saad MNBM, Kamel N, Malik AS (2017) Machine learning framework for the detection of mental stress at multiple levels. IEEE Access 5:13545–13556
Girard JM, Cohn JF, Mahoor MH, Mavadati S, Rosenwald DP (2013) Social risk and depression: Evidence from manual and automatic facial expression analysis. In: 2013 10th IEEE international conference and workshops on automatic face and gesture recognition (FG). IEEE, pp 1–8
He L, Jiang D, Sahli H (2015) Multimodal depression recognition with dynamic visual and audio cues, IEEE
Deshpande M, Rao V (2017) Depression detection using emotion artificial intelligence. In: 2017 international conference on intelligent sustainable systems (ICISS). IEEE, pp 858–862
Chao L, Tao J, Yang M, Li Y (2015) Multi task sequence learning for depression scale prediction from video. In: 2015 international conference on affective computing and intelligent interaction. IEEE, ACII, pp 526–531
Zhou X, Jin K, Shang Y, Guo G (2018) Visually interpretable representation learning for depression recognition from facial images. IEEE Trans Affect Comput
De Melo WC, Granger E, Hadid A (2019) Depression detection based on deep distribution learning. In: 2019 IEEE international conference on image processing (ICIP). IEEE, pp 4544–4548
de Melo WC, Granger E, Hadid A (2020) A deep multiscale spatiotemporal network for assessing depression from facial dynamics. IEEE Trans Affect Comput
Liu J-Q, Huang Y, Huang X-Y, Xia X-T, Niu X-X, Lin L, Chen Y-W (2020) Dynamic facial features in positive-emotional speech for identification of depressive tendencies. In: Innovation in Medicine and Healthcare. Springer, pp 127–134
Low C-Y, Teoh AB-J, Ng C-J (2017) Multi-fold gabor, pca, and ica filter convolution descriptor for face recognition. IEEE Trans Circuits Syst Video Technol 29:115–129
Kotsia I, Pitas I (2006) Facial expression recognition in image sequences using geometric deformation features and support vector machines. IEEE Trans Image Process 16:172–187
Ghimire D, Jeong S, Lee J, Park SH (2017) Facial expression recognition based on local region specific features and support vector machines. Multimed Tools Appl 76:7803–7821
Zhang Y-D, Yang Z-J, Lu H-M, Zhou X-X, Phillips P, Liu Q-M, Wang S-H (2016) Facial emotion recognition based on biorthogonal wavelet entropy, fuzzy support vector machine, and stratified cross validation. IEEE Access 4:8375–8385
Tikoo S, Malik N (2017) Detection of face using Viola Jones and recognition using back propagation neural network. arXiv:1701.08257
Mujahid O, Ullah Z, Mahmood H, Hafeez A (2018) Fast pattern recognition through an lbp driven cam on fpga. IEEE Access 6:39525–39531
Jain P, Joshi AM, Agrawal N, Mohanty S (2020) Iglu 2.0: A new non-invasive, accurate serum glucometer for smart healthcare. arXiv:2001.09182
Pancholi S, Joshi AM (2019) Time derivative moments based feature extraction approach for recognition of upper limb motions using emg. IEEE Sensors Lett 3:1–4
Joshi AM, Bramha A (2020) Vlsi architecture of block matching algorithms for motion estimation in high efficiency video coding. Wirel Pers Commun :1–16
Kyrkou C, Bouganis C-S, Theocharides T, Polycarpou MM (2015) Embedded hardware-efficient real-time classification with cascade support vector machines. IEEE Trans Neural Netw Learn Syst 27:99–112
Lekdioui K, Messoussi R, Ruichek Y, Chaabi Y, Touahni R (2017) Facial decomposition for expression recognition using texture/shape descriptors and svm classifier. Signal Process Image Commun 58:300–312
Tong Z, Chen X, He Z, Tong K, Fang Z, Wang X (2018) Emotion recognition based on photoplethysmogram and electroencephalogram. In: 2018 IEEE 42nd annual computer software and applications conference (COMPSAC), vol 02, pp 402–407
Murtaza M, Sharif M, AbdullahYasmin M, Ahmad T (2019) Facial expression detection using six facial expressions hexagon (sfeh) model. In: 2019 IEEE 9th annual computing and communication workshop and conference (CCWC), pp 0190–0195
Dham S, Sharma A, Dhall A (2017) Depression scale recognition from audio, visual and text analysis. arXiv:1709.05865
Peng Z, Hu Q, Dang-Zanuy J (2019) Multi-kernel svm based depression recognition using social media data. Int J Mach Learn Cybern 10:43–57
Pampouchidou A, Marias K, Tsiknakis M, Simos P, Yang F, Meriaudeau F (2015) Designing a framework for assisting depression severity assessment from facial image analysis. In: 2015 IEEE international conference on signal and image processing applications (ICSIPA), pp 578–583
Xiang J, Zhu G (2017) Joint face detection and facial expression recognition with mtcnn. In: 2017 4th international conference on information science and control engineering (ICISCE), pp 424–427
Tong Z, Chen X, He Z, Tong K, Fang Z, Wang X (2018) Emotion recognition based on photoplethysmogram and electroencephalogram. In: 2018 IEEE 42nd annual computer software and applications conference (COMPSAC), vol 2, pp 402–407
Mantri S, Patil D, Agrawal P, Wadhai V (2015) Non invasive eeg signal processing framework for real time depression analysis. In: 2015 SAI intelligent systems conference (IntelliSys). IEEE, pp 518–521
Acknowledgements
The author be thankful to SMDP-C2SD Lab, MNIT-Jaipur and Ministry of Human Resource Development (MHRD), Govt. of India to provide the support of CAD Tools to carry out the experiments. The results of the paper are carried out in SMDP-C2SD Lab, MNIT-Jaipur.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Tadalagi, M., Joshi, A.M. AutoDep: automatic depression detection using facial expressions based on linear binary pattern descriptor. Med Biol Eng Comput 59, 1339–1354 (2021). https://doi.org/10.1007/s11517-021-02358-2
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11517-021-02358-2