A Study on Detection of Emotions with the Help of Convolutional Neural Network
Abstract
Facial Expression is a mean which can be employed to find out what is running in the mind of the person whom we talk. It is always an easy task for humans to derive some insights from the expressions. But it is a complicated task for machine to derive it by using Computer Algorithms. But recent developments in the field of Computer Vision and Machine Learning have enhanced the availability of resources so that it become possible to derive certain conclusions from the input we are giving to particular Machine. This paper is planning to propose an approach to find out what emotional stage the particular person is running which is termed as Facial Emotion Recognition using Convolutional Neural Network (FERC). The FERC constitutes two parts: The first part of this Convolutional Neural Network is used to remove background from an image and second part is used to extract features from facial expressions. It consists of a database of around 10,000 images. The final layer which is a perceptron which works in series with two-layer Convolutional Neural Network. The Perceptron is used to adjust weights and exponent values after going through each iteration. Again, a background removal is applied just before generation of Emotional Vector. This will help us to avoid from multiple problems which can occur. Using Two stage CNN model we found out accuracy near about 84.92 percentage which is a better value based on insights from 24 values. The Two Layer CNN works in series where remaining layer is used to adjust weights after each iteration. FERC follows an exactly new approach compared to Single stage CNN and thus enhancing the accuracy. Further it uses a novel background elimination procedure compared to technology of EV earlier existed which could avoid coping with coping with more than one troubles that may occur. FERC is employed with about more than 28K pictures by using FER2013 dataset. We can make use of FERC in many cases including predictive learning, Lie detector etc.
References
[2] Saurav, Sumeet, Prashant Gidde, Ravi Saini, and Sanjay Singh. ”Dual integrated convolutional neural network for real-time facial expression recognition in the wild.” The Visual Computer 38, no. 3 (2022): 1083- 1096.
[3] Srimani, P. K., and R. Hegde. ”Analysis of facial expressions with respect to navarasas in bharathanatym styles using image processing.” International Journal of Knowledge Engineering 3, no. 2 (2012): 193.
[4] Mehendale, Ninad. ”Facial emotion recognition using convolutional neural networks (FERC).” SN Applied Sciences 2, no. 3 (2020): 1-8.
[5] [5]. Debnath, Tanoy, Md Reza, Anichur Rahman, Amin Beheshti, Shahab S. Band, and Hamid Alinejad-Rokny. ”Four-layer ConvNet to facial emotion recognition with minimal epochs and the significance of data diversity.” Scientific Reports 12, no. 1 (2022): 1-18.
[6] Khattak, Asad, Muhammad Zubair Asghar, Mushtaq Ali, and Ulfat Batool. ”An efficient deep learning technique for facial emotion recogni- tion.” Multimedia Tools and Applications 81, no. 2 (2022): 1649-1683.
[7] Umer, Saiyed, Ranjeet Kumar Rout, Chiara Pero, and Michele Nappi. ”Facial expression recognition with trade-offs between data augmenta- tion and deep learning features.” Journal of Ambient Intelligence and Humanized Computing 13, no. 2 (2022): 721-735.
[8] Vu, Hoai Nam, Mai Huong Nguyen, and Cuong Pham. ”Masked face recognition with convolutional neural networks and local binary patterns.” Applied Intelligence 52, no. 5 (2022): 5497-5512.
[9] Kollias, Dimitrios. Abaw: Valence-arousal estimation, expression recog- nition, action unit detection and multi-task learning challenges.” arXiv preprint arXiv:2202.10659 (2022).
[10] Gera, Darshan, S. Balasubramanian, and Anwesh Jami. ”CERN: Com- pact facial expression recognition net.” Pattern Recognition Letters 155 (2022): 9-18.
[11] Han, Jiayi, Liang Du, Xiaoqing Ye, Li Zhang, and Jianfeng Feng. ”The devil is in the face: Exploiting harmonious representations for facial expression recognition.” Neurocomputing 486 (2022): 104-113..
[12] Han, Jiayi, Liang Du, Xiaoqing Ye, Li Zhang, and Jianfeng Feng. ”The devil is in the face: Exploiting harmonious representations for facial expression recognition.” Neurocomputing 486 (2022): 104-113.
[13] Boughida, Adil, Mohamed Nadjib Kouahla, and Yacine Lafifi. ”A novel approach for facial expression recognition based on Gabor filters and genetic algorithm.” Evolving Systems 13, no. 2 (2022): 331-345.
[14] Fard, Ali Pourramezan, and Mohammad H. Mahoor. ”Ad-corre: Adap- tive correlation-based loss for facial expression recognition in the wild.”
IEEE Access 10 (2022): 26756-26768
[15] Fan, Xinqi, Mingjie Jiang, Ali Raza Shahid, and Hong Yan. ”Hierarchi- cal scale convolutional neural network for facial expression recognition.” Cognitive Neurodynamics (2022): 1-12.
[16] Giannopoulos P, Perikos I, Hatzilygeroudis I (2018) Deep learning approaches for facial emotion recognition: a case study on FER 2013. In: Hatzilygeroudis I, Palade V (eds) Advances in hybridization of intelligent methods. Springer, Berlin, pp 1–16
[17] Gajarla V, Gupta A (2015) Emotion detection and sentiment analysis of images. Georgia Institute of Technology, Atlanta
[18] Wu YL, Tsai HY, Huang YC, Chen BH (2018) Accurate emotion recognition for driving risk prevention in driver monitoring system. In: 2018 IEEE 7TH global conference on consumer electronics (GCCE). IEEE, pp 796–797
[19] Zhang K, Huang Y, Du Y, Wang L (2017) Facial expression recognition based on deep evolutional spatial-temporal networks. IEEE Trans Image Process 26(9):4193
[20] Jung H, Lee S, Yim J, Park S, Kim J (2015) Joint fne-tuning in deep neural networks for facial expression recognition. In: Proceedings of the IEEE international conference on computer vision. pp 2983–2991
[21] Zhao X, Liang X, Liu L, Li T, Han Y, Vasconcelos N, Yan S (2016) Peak- piloted deep network for facial expression recognition. In: European conference on computer vision. Springer, pp 425–442
[22] Shan C, Gong S, McOwan PW (2009) Facial expression recognition based on local binary patterns: a comprehensive study. Image Vis Comput 27(6):803
[23] Littlewort G, Whitehill J, Wu T, Fasel I, Frank M, Movellan J, Bartlett M (2011) The computer expression recognition toolbox (CERT). In: Face and gesture 2011. IEEE, pp 298–305
[24] Hsu RL, Abdel-Mottaleb M, Jain AK (2002) Face detection in color images. IEEE Trans Pattern Anal Mach Intell 24(5):696
[25] Cowie R, Douglas-Cowie E, Tsapatsoulis N, Votsis G, Kollias S, Fellenz W, Taylor JG (2001) Emotion recognition in human– computer interaction. IEEE Signal Process Mag 18(1):32
[26] Herna´ndez-Travieso JG, Travieso CM, Pozo-Ban˜os D, Alonso JB et al (2013) Expression detector system based on facial images. In: BIOSIGNALS 2013-proceedings of the international conference on bio- inspired systems and signal processing
[27] Kumar KC (2012) Morphology based facial feature extraction and facial expression recognition for driver vigilance. Int J Comput Appl 51:2
[28] Ernst H (1934) Evolution of facial musculature and facial expression. J Nerv Ment Dis 79(1):109
[29] Kim DH, An KH, Ryu YG, Chung MJ (2007) A facial expression imitation system for the primitive of intuitive human robot interaction.
In: Sarkar N (ed) Human robot interaction. IntechOpen, London
[30] Xue Yl, Mao X, Zhang F (2006) Beihang university facial expression database and multiple facial expression recognition. In: 2006 Inter- national conference on machine learning and cybernetics. IEEE, pp 3282–3287
[31] Kong SG, Heo J, Abidi BR, Paik J, Abidi MA (2005) Recent advances in visual and infrared face recognition—a review. Comput Vis Image Underst 97(1):103
[32] Dols JMF, Russell JA (2017) The science of facial expression. Oxford University Press, Oxford
[33] Parr LA, Waller BM (2006) Understanding chimpanzee facial expres- sion: insights into the evolution of communication. Soc Cogn Affect Neurosci 1(3):221
[34] Mal HP, Swarnalatha P (2017) Facial expression detection using facial expression model. In: 2017 International conference on energy, com- munication, data analytics and soft computing (ICECDS). IEEE, pp 1259–1262
[35] Xie S, Hu H (2018) Facial expression recognition using hierarchical fea- tures with deep comprehensive multipatches aggregation convolutional neural networks. IEEE Trans Multimedia 21(1):211
[36] Ratyal N, Taj I, Bajwa U, Sajid M (2018) Pose and expression invariant alignment based multi-view 3D face recognition. KSII Trans Internet Inf Syst 12:10
[37] Shamshirband, S. et al. A review on deep learning approaches in healthcare systems: Taxonomies, challenges, and open issues. J. Biomed. Inform. 113, 103627 (2021)
[38] Javanmard, R. et al. Proposed a new method for rules extraction using artifcial neural network and artifcial immune system in cancer diagnosis. J. Bionanosci. 7(6), 665–672 (2012)
[39] Rajaei, P. et al. A user-friendly tool for viral sequence analysis. Genes 12(2), 186 (2021).
[40] Alinejad-Rokny, H., Sadroddiny, E. Scaria, V. Machine learning and data mining techniques for medical complex data analysis. Neurocom- puting 276, 1 (2018)
