AU2020101210A4

AU2020101210A4 - Automated screening system of covid-19 infected persons by measurement of respiratory data through deep facial recognition

Info

Publication number: AU2020101210A4
Application number: AU2020101210A
Authority: AU
Inventors: Dinesh Kumar Anguraj; Vinothini C.; Priya J.; Sai Prasanthi K.; Rudra Kalyan Nayak; Ramamani Tripathy; Renjith V. Ravi; Prasanth Y.
Original assignee: Anguraj Dinesh Kumar Dr; C Vinothini Ms; J Priya Ms; K Sai Prasanthi Ms; Nayak Rudra Kalyan Dr; Tripathy Ramamani Dr; V Ravi Renjith Dr; Y Prasanth Dr
Current assignee: Anguraj Dinesh Kumar Dr; C Vinothini Ms; J Priya Ms; K Sai Prasanthi Ms; Nayak Rudra Kalyan Dr; Tripathy Ramamani Dr; V Ravi Renjith Dr; Y Prasanth Dr
Priority date: 2020-06-30
Filing date: 2020-06-30
Publication date: 2020-08-06
Anticipated expiration: 2028-06-30

Abstract

AUTOMATED SCREENING SYSTEM OF COVID-19 INFECTED PERSONS BY MEASUREMENT OF RESPIRATORY DATA THROUGH DEEP FACIAL RECOGNITION Abstract: More than 10 00 000 people were exposed to the novel coronavirus called COVID 19, which is transpired in Wuhan, China, in December 2019 and became a worldwide medical evacuation rapidly. Vigorous medication and enhanced allotment of hospital facilities in pandemic regions are substantial due to the increased rate of infectious disease. Precise and efficient medication helps to segregate the suspected cases to stabilize the spreading of infection. A severe hurdle is the lack of hospital and medical facilities in pandemic regions. COVID-19isan irregular breathing condition triggered by coronavirus infections, so numerous citizens typically wear masks to reduce the threats of being infected during the pandemic period. This invention presents FLIR E6 that uses RGB and thermal cameras for health screening to acquire the facial expressions in video frames. YOLO V3 is suggested for detecting the facial regions such as the forehead part and nostril areas. Based on the detection of the accurate region, the respiration data is extracted from the masked nostril region. Inevitably, the recurrent neural network model called Long Short Term Memory proposes for classifying as infected or healthy people based on respiratory data. This proposal exhibits that the contactless and smart screening test, which exploits the pre-scan process for virally infected people with high accuracy of detection and obtains relatively high performance. 11 P a g e AUTOMATED SCREENING SYSTEM OF COVID-19 INFECTED PERSONS BY MEASUREMENT OF RESPIRATORY DATA THROUGH DEEP FACIAL RECOGNITION Diagram breathing data iae andmaskdetection detection usn YOLO V3 FLIR thermal camera Iht xtt viralInfectedperson SflR classification using LSTM normalperson Fig 1: Data flow diagram 1| P a g e

Description

AUTOMATED SCREENING SYSTEM OF COVID-19 INFECTED PERSONS BY MEASUREMENT OF RESPIRATORY DATA THROUGH DEEP FACIAL RECOGNITION

Diagram

breathing data

iae andmaskdetection detection usn YOLO V3 FLIR thermal camera

Iht viralInfectedperson SflR xtt

classification using LSTM normalperson

Fig 1: Data flow diagram

1| P a g e

Field of Invention:

This proposal relates methods and devices to identify COVID-19 patients using facial recognition. Specifically, this invention proposes an efficient approach to identify the COVID-19 affected peoples using deep facial recognition based on the respiratory data. A handheld contactless approach to assess the wellness of people wearing masks by examining respiratory features. The deep learning methods, such as saliency, guided faster RCNN (SGFr-RCNN), promote to detect and recognize the object from video streams.

Background and prior art of the invention:

In recent months, Coronavirus2019 (COVID-19) seems to have been a massive worldwide crisis that is affected by intense abrupt breathing infection coronavirus2 (SARS-CoV-2). Globally, COVID 19 is caused by significant losses to human civilization.

Classical methods used to observe the respiration rate with the help of sensors. Sensors can be attached in the infected person that computed the respiration rate by chest or abdomen moment. However, equipment used for analysis is cumbersome, relatively high cost, and more time-consuming. At present circumstances, noninvasive analysis is uniquely suited since the contagious diseases such as COVID-19 is spreading among people while analysis.

Cho et al. applied a convolutional neural network to examine the infected people's inhalation factors for evaluating the problem of nervousness using infrared cameras.

Wang et al. promoted an in-depth learning approach along with a depth camera in real-time for categorizing anomalous breathing behaviors and gained impressive outputs. However, the main drawback is the device is not compact.

11 P a g e

Zhang et al. proposed for enhancing the accuracy of detection of the multi-view face by multi-task CNN. Still, the preliminary detection windows are generated by a weak detector that leads to less accuracy.

G. Koukiou et al. demonstrated the method to distinguish between sober and drunk people using the pixel intensity, which is situated in specific sectors such as nose, mouth, and forehead. Based on the intensities, a field of distinct aspects is obtained. Indeed this approach utilized a less quantity of samples that encountered specific features, and generalization of the classifier is not feasible.

Z. Barret et al. defined machine learning algorithms such as evolutionary-based algorithms and reinforcement learning (RL), while training to retrieve optimized system parameters. But these algorithms are trivially cumbersome and restrained more computation power.

Dong C et al. explained the deep learning algorithm called SRCNN, which exploits a single convolutional neural network for increasing the restoration quality and enhancing the analysis of low-resolution light images. But it absolves some predefined distortions.

He, Z et al. envisioned a cascaded model of deep learning method with innumerable convolutional layers. This approach scrutinized the enhancement of large scale resolution, which is utilized by interior and exterior thermal images. However, the cascaded deep networks contain relatively very less training data sources.

Kuang, X et al. suggested a conditional GAN-based network that highlights the intensity refinement of infrared image analysis. Compared to conventional methods such as histogram equalization and image transformation methods, this infrared image analysis yielded better results

At present, the infrared imaging approach is devised for tracking the patient's respiration and evaluation of the respiratory rate by using statistical and fast Fourier transform methods. Nevertheless, this method bypasses the gesture of the human body.

The extant oxygenation sensing types of equipment are relatively substantial and intangible. In a worldwide pandemic perspective, the amenable and eloquent hardware is preferred to fulfill and demand of the large-scale screening and potential

21Page applications in real-time. Nostril and mouth areas are the specific region for infrared imaging analysis to evaluate the breathing rate because the above parts only dissipate the heat between the human body and society. Yet until now, the approaches occasionally suggested evaluating the thermal breathing analysis for people who wear masks.

The objective of the invention:

• The main goal is to implement a non - invasive measuring device for living creature respiratory signals based on archived facial oriented videos. • To incorporate the advanced technologies of face-recognition with infrared imaging analysis that undergoes the strategy of breathing data analysis, people who wear masks, which is incredibly vital for the current scenario. • To promote a useful automated noninvasive screening device to locate and preserve the suspected COVID-19 cases and to leverage the crucial dispersion of coronavirus.

Summary of the invention:

Initial supervision is crucial due to the pandemic of infectious diseases such as COVID-19. Among the most exciting aspect of all control measures is the efficient and reliable detection of probable virus-infected people.

Artificial Intelligence (Al) predominantly emphasizes the prognosis of the infected people, pathogens as COVID-19, the diagnostic analysis process, detection of diseases, and predictions to battle against coronavirus.

A thermal camera called forward-looking Infrared (FLIR E6) camera is used for capturing the face videos. FUR camera includes both the thermal and RGB cameras. Advanced face recognition is proposed to identify the forehead and nostril regions from facial areas. Both the RGB camera and thermal camera capture raw facial expressions, and further analysis process facilities significantly higher efficiency and performance.

Due to inhalation and exhalation, recurrent variations in temperature that arise around the nostril regions since breathing conditions are consistent. Breathing data can be acquired by examining the heat data around its nostril regions based on thermal video frames. RGB and infrared cameras are proposed to recognize facial

3 1P a g e and mask wear regions since other algorithms lost geometric and textual details that lead error-prone to mask areas.

An object detection based convolution network called YOLOv3 is proposed to detect the accurate nostril region from facial video frames to collect the respiration information. YOLO v3 segments the video frames into different areas and predicts the specific region in bounding boxes that specify the region of interest-based on the probability of the region. The respiration data is extracted from the video by considering the maximum variance of the ROI of the masked region in which the maximum breathing signals are placed.

A recurrent neural network is applied for classification since the extracted breathing data is in time series. Long Short Term Memory (LSTM) is envisioned to classify whether the respiration condition is normal or abnormal based on the respiration data. LSTM is suitable for more extended sequence data and yields high accuracy.

Statement of the invention:

The proposal suggests the FUR E6 camera that includes both RGB and Infrared imaging. The FUR E6 camera conducts a noninvasive screening test for corona virus-infected individuals and produces video sequences. Based on the facial expressions, the respiratory data is acquired by using an Infrared camera. The object detection neural network called YOLOV3 is proposed to detect the specific nostril region to capture the region of interest. The respiratory information is extracted from video frames by taking the maximum variance of ROI. Later the Recurrent neural network called LSTM suggested classifying as infected and healthy people based on the breathing data with high accuracy.

Brief description of the Drawings:

Fig 1: Data flow Diagram

Fig 2: YOLO V3 architecture

Fig 3: Long Short Term Memory (LSTM) Architecture

41Page

A detailed description of the drawing:

Figure 1 illustrates the working flow of the automated screening device to detect the infectious person based on breathing data using facial expressions. FUR camera is utilized to capture the images using infrared, and the output image is mapped with an RGB image to recognize the people of the facial region who use masks. The YOLO V3 is proposed to detect the specific nostril region to get the respiratory data. By taking maximum variance from ROI can be considered to gather the breathing data based on the inhalation and exhalation process. For further classification of infected and healthy people, the recurrent neural network (RNN) called LSTM is devised to produce a high accuracy of detection.

Figure 2 explains the underlying architecture of YOLO v3 to detect a specific region of nostrils, correctly. YOLO V3 segments the input image into SxS gird. Each and individual cell predicts the bounding boxes to identify the specialized areas such as the nostril region. Based on ROI, the maximum variance value is used to extract the breathing data.

The Recurrent Neural network called LSTM is represented in figure 3. This model includes three layers, such as input, forget, and output layer. The softmax layer predicts infected and healthy people based on breathing data. This contactless device is used for screening the people who wear masks to detect the COVID-19 infected persons based on respiration data based on facial recognition methodology. This method obtains relatively high accuracy with better performance.

51Page

We claim that,

• FLIR camera includes both RGB and Infrared cameras that are used to capture the video sequences in the non-invasive and smart screening test for people who are wearing masks.

• YOLO V3 suggests for automatic object detection that segments the region as the forehead and masked nostril areas using a bounded box.

• LSTM proposes for classification as infected and healthy people based on the respiratory information.

• The contactless and smart device is generated for automatic detection based on breathing information with high accuracy.

11 P a g e

AUTOMATED SCREENING SYSTEM OF COVID-19 INFECTED Jun 2020

PERSONS BY MEASUREMENT OF RESPIRATORY DATA THROUGH DEEP FACIAL RECOGNITION

Diagram 2020101210

Fig 1: Data flow diagram

1|Page

Fig 2: YOLO V3 Architecture

Fig 3: LSTM Architecture

2|Page