Autism Spectrum Disorder (ASD) is treated as fascinating, complex and neurological developmental disorder that could diagnosed in the early age usually before 3 years. Autistics are majorly characterized with impaired communication, social interaction, and repetitive, stereotyped or restricted behavior [1]. Typically developing (TD's) children are socially interactive, gazing at faces, respond on voices, showing facial emotions, attract or enjoying socially towards other children, etc. whereas the ASD children shows deficits. The causes of ASD deficits are still under debate; however social deficits are majorly caused due to the difficulties in the processing of emotional traits [2]. With the establishment of term Autism, the emotional insufficiency highlighted as a major social deficit in Autism with specific disorders in emotional reactions [3-5]. Typically Developed (TD) children had started learning emotions of faces from the very early age of life [6]. Difficulties regarding facial emotion analysis of ASD’s could recognize better by applying rapidly progressive research. The feelings or emotions of ASD's are less explainable from faces [7]. In general, faces always reflect important information about the thinking and feelings of human beings. Humans have an astounding ability to perceive and reflect the facial emotions. Human facial emotions perception thoughts are driven by either features or holistic [8]. As compare to TD’s, ASD’s are insufficient for facial emotions learning. Most of the literature had reported ASD’s with deficit in facial emotion recognition as compared to TD’s [2,9-12].

Quintin et al. [13] had analyzed the high functioning Autistics with working auditory memory by using music for recognizing emotions. Wechsler Intelligence Scale for Children, 4th edition was used for analyzing the auditory processing and working memory of children. For judgmental purpose, music clips were played along with the emotional faces appeared on a computer screen. ASD’s could not recognize or categorize raw emotions but in this case of musical emotions, the ASD’s had responded nearly as same as TD children. Autistic responses were recorded during the video playing among them. 22 inch NEC monitor, E-Prime software, P-IV Computer with Windows XP operating System was used for playing videos of different emotional faces and recording responses. The study was conducted to analyze the aftereffects paradigm for different emotional faces. Total 46 trails (45 s each) were conducted using 42 gray scale pictures as diversion trail stimuli which pretending happy, sad, surprise, disgust, fear and anger. The data related to image and face-eye fixation was recorded by using the Eye-Trac 6000 ASL. The choices of the exact and quick responses had measured against each displayed stimuli. The noted resultant frequency patterns were lessclear and less-decisive in ASD’s than the control group [14].

Emotion adaptation paradigms were reflected as useful for examining ASD’s on the parameter of emotional processing. ASD’s had resulted smaller facial emotion aftereffects (FEA) for sad emotion faces as compare to the happier faces [15]. ASD’s found more adaptive to sad aftereffects and pretends abnormal processing whereas TD’s were been found positive towards happier faces [14]. Distinct cortical pathways were noted responsible for the facial emotions processing of dynamic faces in ASD’s [15]. However, autism was also characterized by atypicalities in sensation and perception [16]. Earlier diagnosis could help in converting the abnormal emotional behavior into nearly acceptable emotions [17].

Corbett was used video modeling during conducted experiments to enhance the socio-emotions sensitivity and observational skills of the Autistic children. More than 51% was gained by autistics in their social-emotional reorganization and 18% audio tone of Autistics had also improved after including video including facial emotion photographs [18]. Sigman et al. had examined the crying and happy emotions of autistic and analyzed the change in their cardiac and attention responses during video stimuli [19]. The measured heart rate was also confirmed the interests and attraction responses of autistics as same to typically developed, but autistics had pretended ignored social behavior. Experimental findings of Iarocci et al. suggested the autistics were able to perform well in structural visual basis (like images, figures) but performs poorly in the case of difficult and changeable objects like faces, emotions, etc. [20].

Training using software’s, robots, avatar’s, etc. had found helpful for improvement of autistic social behavior. Training on FACE prototype [21] had reflected improved facial expressions of autistics. By intervening computer based program for the biological movements, 73 children, including Autism, Down syndrome and TD children were analyzed by Laine et al. [22]. The slow biological movements had made more impactful for teaching the students for reproducing facial expressions and body movements. In results TD and Down syndrome children performed better than Autistic group, however severe Autistic group got better results of learning than the milder Autistic group.

Interactive, active, realistic avatar based training software, namely “FaceSay” was used by Hopkins, 2011 for improving social, behavioral skills in Autistics. FaceSay had contained realistic avatars (animated photos of real persons) based three different games, i.e., “Amazing Gazing” for eye gaze, “Band Aid Click” for facial recognition, “Follow the Leader”. It discriminated facial expressions while photographs, drawings, emotional, facial and non-facial expressions used as stimuli. After FaceSay training, the various tests (Emotion Recognition test, the Benton Facial Recognition test, and the Kaufman Brief Intelligence Scale tests) were validated the significant improvement in their behavioral skills of Autistics [23,24]. FaceSay was also used by Rice et al. [24] for improving the ASDs with eye-gaze, metalizing, attention, affect reorganization and face reorganization skills. Emotional skills, social environment reorganization and mental ability were noted as enhanced in ASD’s after training conducted on computer based simulated environment.

Emotionally capable virtual reality avatars [25] were used where ASD’s had reflected more distributed attention to face area with long respond time as compared to TD’s.

Kandalaft et al. [26] investigated the feasibility of Virtual Reality Social Cognition Training intervention for social skills, cognition and functioning enhancement of autistics. A three dimensional virtual reality software Second Life version 2.1, Microsoft Windows XP, Audio voice manipulation software and 24 inch monitor were used for displaying user as a changeable virtual entity or Avatars with arm, body and mouth movements only. Pre and post sessions were conducted for virtual reality-social cognitive training program was attended Autistics were found with improved social facial and interaction skills. The potential impact of using 3D virtual reality, technological integration for social interaction and communication in Autistic children was investigated by Ke et al. [27]. This virtual reality program was consisted with body gesture and facial expression recognition during basic interactions in school or parties. After training on realistic virtual space, the Autistics had performed better in leading interaction, initiating interaction, maintaining interaction, responses and gesture recognition.

The high-functioning autism spectrum disorder group more readily distinguishes overt emotions such as happiness and sadness [28,29].

Emotion expressions on face are major form of non-verbal communication. The ability of ASD’s to reflect facial emotions have been treating as major area for research. The objective of study was to measure the total response total time (in secs) for reflected facial emotional expressions during the video as stimuli in TD’s and ASD’s. “No difference in total response time in seconds of TD’s and ASD’s” is defined as null hypothesis and “Difference in total response time in seconds of TD’s and ASD’s” as not null hypothesis. This paper is majorly focused on software development and it analysis for facial emotions and expressions recognition system for ASD's using Python and CV2.

Size of experiment room was of 14 feet by 12 feet. Here four walls had mounted with LED displays and Cameras approximately at 5 feet and 7 feet vertical.

Video stimuli

After detailed communiqué with parents, caretakers and professionals, data set of different videos that reflecting positive and negative emotions had collected from various online resources. These videos were displayed on four 32” LED Display Monitor (Display A, B, C, D) for better view where one display was mounted on each wall. The videos had playing length (approx.) of 10 s. Then 10 s of break with blank screen had stuffed between the pair of videos to reduce the facial emotion after effects (Figure 1).

Facial emotion recognition system

Software for Facial Emotion Recognition had created using Python 2.7 and its libraries like cv2, numpy. In Figure 2, camera’s (Cam A, B, C, D) arrangement made nearly independence form direction of face in the experimental area. This software supports with four USB connected cameras fetching face on the basis of distributed time stamps. Children dataset “The NIMH Child Emotional Faces Picture Set (NIMH-ChEFS)” was used for training the algorithm with emotions (available at http:// devepi.duhs.duke.edu/NIMH_Pictures.html). Before feeding forward for the processing of facial emotion, captured faces were processed by the initial module for establishing the identification of subject from number of faces present in scenario. The facial emotion recognition system has been mapped with video stimuli for identification of emotion. Algorithm developed for this is as follows:

Algorithm:

1. Learn the face of subject, which we want to consider as subject.

2. Machine training of face emotion recognition system using NIMH-ChEFS children dataset with normal and emotional facial expressions.

3. Start observing emotions of learned face of subject.

4. Predict by comparing subject face emotion with learned emotions.

5. Calculate time for emotional and normal reactions.

6. Record the results using file handling.

Prior to the study written Consent was obtained from all participants and legal caregivers. None of the children received any psychoactive medication during the study.

Participants

In this experiment two groups of children were participated. 10 clinically diagnosed children for ASD by meeting DSM-IV criteria and 10 TD developing children. 16 male and 04 female children having age between 4 to 12 years were part of this experiment. TD children had mental-age-matched and had no history of mental or neurological disorders. Besides, all the autistic children met Autism Diagnostic Interview-Revised (ADI-R) algorithm criteria for autism. All the participants had normal or corrected to normal vision.

Figure 1: Experimental Procedure at the place of Experimental Setup.

Participants training

To adapt children to the experiment setting and apparatus, behavioral training to each child was given before the data recording sessions. During earlier training session, the child was sat on a comfortable chair along with the caregiver in the free position. Later children trained for stay alone in free position in a room and watching videos. The goal of training was to improve tolerance for residing in room, watching videos and building confidence to stay alone in a room for 10-15 min.

Trained participants were left in the experimental environment along with a caretaker for 5-10 min in uniform manner, one at a time. A random set of 10 digital videos of 30 s of length (approx.) were played by our software on all four wall mounted LED displays. Once video started playing, software had also automatically started observing and analyzing the facial emotion reactions of subject through wall mounted cameras. Timings of emotion recognition software and stimuli were crucially synchronized for identifying the outcomes of proposed system. However timings between video playing and emotion recognition had synchronized automatically by the software. Experimental procedure has been explained in Figure 2.

The results of this experiment were measured as the total time of reflected emotions by the subjects during video stimuli’s. Total time of facial emotion responses were compared for the emotional responses of TD’s and ASD’s. As shown in Figure 3, TD’s reflected 68 as maximum and 49 as minimum emotions time. But ASD’s had executed fewer emotions with 53 as maximum and 27 as minimum. The given results reflected disagreement with the assumed null hypothesis where “No difference in total response time in seconds of TD’s and ASD’s” was predicted Figure 4.

To validate the data, two sample t-test for unequal variances was conducted had shown in Table 1. According to hypothesis ASD’s have lower facial emotion expressions as compare to TD’s. The results of t-statistic on TD’s data (Mean=58.6 s, Variance=40.93 s, Standard Deviation=6.3979 s) and ASD’s data (Mean=58.6 s, Variance=40.93 s, Standard Deviation=6.3979 s) is equal to 5.16 and the t-value is 0.0000473. Since the p-value is very low, rejects the null hypothesis.

Figure 2: Experimental Setup at the place of Experimental Procedure.

Figure 3: (a) Total time (in seconds, s) emotions by TD’s during displaying video’s; (b) Total time (in seconds, s) emotions by ASD’s during displaying video’s.

Figure 4: Standard deviation between TD's and ASD's timings.

Table 1: t-test: Two-sample assuming unequal variances.

t-test with t<0.0000473 highlighted that the “TD’s and ASD’s have different total response reaction time. As the results reflected strong evidence of a mean difference in total emotional reaction time (18.7 s) between the TD’s and ASD’s. However, ASD’s have less explainable facial emotions with comparison to TD’s.

Investigation in relation between the facial emotions of Autistic during their face processing motor activities by using a computerized approach was completed. This experiment was conducted to establish the difference in the total response time (in seconds, s) for reflected face emotional expressions during the displaying videos as stimuli in the case of TD’s and ASD’s. Python 2.7 has been utilized for programming and implementing the computerized facial emotions detection system and for video playing.