KR102194409B1 - Face Detection and Recogniton System and Method using Multi-focusing AI Convolution Neural Network - Google Patents

Abstract

The present invention relates to a system for performing face detection and face recognition in a cost-effective method. According to the present invention, the system comprises an input unit receiving image data and a management server receiving the image data from the input unit to process the image data. The management server comprises a back-end server receiving the image data received from the input unit in a streaming type and adjusting the load and frames of the received image data, and an AI analysis server analyzing the image data adjusted by the back-end server through an AI neural network algorithm to detect or recognize a face in the image data. Accordingly, the system monitors and checks identification of a user without explicit notification to the user by unaware continuous authentication, thereby providing an effect of maintaining validity in detection/recognition without interfering with a normal flow of service.

Description

Face Detection and Recogniton System and Method using Multi-focusing AI Convolution Neural Network}

The present invention is a cost-effective method for face detection and face recognition using an AI artificial intelligence neural network learning method, employing a multi-focusing AI neural network, face detection, facial recognition, and facial recognition. Regarding a method/system that can perform authentication simultaneously, a method of operating a multi-layered BackBone neural network, sensing environmental changes (illumination, resolution) of a user image, and operating an appropriate auxiliary neural network to increase the face detection rate and recognition rate. The present invention relates to a face detection and identification system capable of responding to security attacks such as facial spoofing by using an auxiliary neural network for liveness.

In the case of a general camera, the optimal focal length is determined according to the type of lens of the camera. Image acquisition terminals such as CCTV web camera terminals also acquire images through the process of photographing a target object at a desired distance with an optimal resolution by adjusting the depth and depth according to the distance of the object to be photographed and the amount of light.

At this time, the number of pixels constituting the facial image is affected by the supported resolution of the photographing terminal, but even in the same terminal, as shown in FIG. 1, the area constituting the facial area according to the distance to the object (face). The number of pixels in is different.

That is, in the case of FIG. 1(A), which is a long distance, the number of pixels in the area constituting the facial image is small, whereas in the case of FIG. 1(C), which is close to the distance, the number of pixels in the area constituting the facial image is large.

In addition, the face detection rate and recognition rate are affected not only by the number of pixels, but also by the amount of light and illuminance/luminance.

That is, the brighter the illuminance and luminance, the greater the amount of light that reaches the camera sensor, and the pixel value (RGB) of the image file changes, affecting the face detection rate and recognition rate.

On the other hand, in the case of using the face detection/recognition system, the presence of the camera itself may give users a sense of rejection or discomfort, so it is also necessary to perform face detection and facial recognition without being recognized by the subject.

In the case of using services such as non-face-to-face authentication of financial services, in order to prevent risk factors in financial transactions due to face spoofing attacks, etc., continuous authentication without awareness, even if it is not a procedure explicitly required by customers In the background, it is necessary to check the risk factors arising from service users.

Meanwhile, Korean Patent Laid-Open No. 10-2008-0073598 discloses a technology for identifying an identity by recognizing a face from an input image, and Korean Patent Publication No. 10-2020-0017594 identifies a large-scale object in an input image. The technique of doing is disclosed.

However, the prior art as described above has the following problems.

An artificial intelligence neural network that has been trained and optimized at a specific resolution can be viewed as one template. Therefore, there are restrictions on the use of such a backbone neural network for various purposes. If face detection and recognition are to be performed simultaneously at all distances and at all resolutions, the neural network will take many parameters, overlapping layers, and more learning time.

(001) Korean Patent Application Publication No. 10-2008-0073598 (002) Korean Patent Application Publication No. 10-2020-0017594

In the present invention, when a face to be recognized included in an input image image exists in a mixture of near/far distance, or when an area with sufficient resolution and an area that is insufficient are mixed, when there is a large deviation in illuminance, the input image If the peripheral requirements for recognition/identification are uneven, facial data could not be effectively analyzed and detected with the existing single backbone neural network that performed a single artificial intelligence neural network learning.

In other words, since a general “backbone neural network for facial recognition” is a template completed by learning facial photos according to specific environmental conditions, the reference scale is always constant. Therefore, if an image of 20 pixels or an image of 200 pixels is analyzed with the same standard and scale, it is difficult to derive accurate results.

In addition, in the native app for financial transactions on mobile devices, in the case of non-face-to-face financial transactions, other portraits are used, or when other people's videos are used to hide their identity, a real object that can cope with this. Verification (Liveness authentication) function is required. Even without special sensor equipment (infrared ray sensor, distance measurement sensor, heat sensor), a security means is needed to confirm that the mobile device is real.

In addition, the present invention deviates from a facial recognition detection/identification system that is used only for a specific purpose, and is a multi-focusing face detection composed of a multilayer backbone neural network and an auxiliary neural network so that the face can be detected and recognized by adapting to the environmental conditions of the input image in general And an identification system.

According to a feature of the present invention for achieving the above object, the present invention comprises an input unit for receiving image data; And a management server that receives and processes image data from the input unit, wherein the management server receives the image data received from the input unit in a streaming format and adjusts the load and frame of the received image data. (Back-end) server; And an AI analysis server configured to detect or identify a face on the image data by analyzing the image data adjusted from the backend server through an artificial neural network algorithm.

In this case, the input unit may include a mobile terminal provided with a self-taking function through a built-in camera; A web terminal connected to a PC and capable of recognizing a plurality of people in a browser environment using a web USB camera; Alternatively, it may be configured to include any one or more of an IP camera (local terminal) that is installed in a public place to observe a number of people and configured to enable communication.

In addition, the AI analysis server receives image data input through an input device such as an IP camera, and then uses a region proposal network (RPN) neural network to specify a facial region (facial region as a result of face detection and corresponding facial location detection). A region-specific government unit that performs a task of Bounding; A K-value unit for determining the number of facial regions (faces), resolution information of the input image data, and the number of input image data; An AI analysis unit configured to include a face detection algorithm and a face recognition algorithm to analyze face information by selectively applying the face detection algorithm or the face recognition algorithm to input image data; Facial information analysis algorithm selected by the AI analysis unit, including an illuminance correction algorithm, a resolution correction algorithm, an angle correction algorithm, and an object correction algorithm to perform illuminance, resolution, angle and liveness correction for input image data. It may be configured to include an image correction unit that selectively selects and performs a correction algorithm according to the type of.

In addition, the facial recognition algorithm may include: an identification recognition algorithm for distinguishing character features of a recognized face; It may be configured to include a recognition algorithm for authentication that can confirm the biometric information of a specific person.

Meanwhile, for the selection of the analysis algorithm of the AI analysis unit, any one of the following methods may be selected.

That is, the AI analysis unit may have a facial area having a number of pixels equal to or greater than the number of pixels for facial recognition based on the number of pixels included in the facial areas included in the image to be analyzed, and having a number of pixels equal to or greater than the number of pixels for facial recognition. When the average deviation value of the facial regions is more than a preset value, a facial recognition algorithm may be selected as an analysis algorithm.

In addition, the AI analysis unit determines the number of facial areas, and when the average of the number of pixels in each facial area matches within an error range with the reference number of facial recognition pixels, the constituent pixels of each facial area If the number deviation average value is greater than or equal to a preset value, a facial recognition algorithm may be selected as an analysis algorithm, and if it is less than a preset value, a face detection algorithm may be selected as an analysis algorithm.

Finally, the AI analysis unit is used for authentication when the average of the number of pixels included in the facial areas matches within an error range with the reference pixel number for authentication, and the average deviation of the number of pixels in each facial area is equal to or greater than a preset value. A facial recognition algorithm may be selected as an analysis algorithm, and if it is less than a preset value, an identification facial recognition algorithm may be selected as an analysis algorithm.

The number of reference pixels for face detection may be 25×25 pixels per face area, the number of reference pixels for face recognition may be 50×50 pixels per face area, and the number of reference pixels for face authentication is 90 per face area. It may be x90 pixels.

And the K-value unit calculates a K value by determining the number of pixels included in the face according to the minimum recognized face size, the face ratio on the image, and the number of faces based on the resolution pixels of the input image data; The AI analysis unit may analyze facial information by selecting any one of the face detection algorithm, identification recognition algorithm, or authentication recognition algorithm according to the K value.

In addition, the input unit may be installed with a facial recognition program.

In addition, when the facial recognition program is not installed, the input unit may transmit a terminal identification number to a backend server, request an API link to the backend server, and receive an encrypted connection address to access a URL (API).

At this time, the mobile device accesses the provided URL using the default browser of the mobile device (Safari in the case of IoS, and the chrome browser in the Android device).

In addition, in order to sequentially analyze the streaming data transmitted from multiple input units at the same time, the backend server may send a response request signal to the input unit to locally delay the data transmission rate according to the response rate of the corresponding input unit. It may be configured to include a delay unit to enable.

Specifically, the delay unit of the back-end server determines the transmission rate of streaming data in consideration of the processing capabilities of the plurality of local terminals and the management server configured, and then sets the standard in terms of the number of frames per second to be transmitted. When the load processing of the input unit of a specific sequence among the input units of is increased, the number of frames transmitted per second of the corresponding input unit may be adjusted.

In addition, the illuminance correction algorithm uses LBP (Local Binary Pattern) to reduce sensitivity to changes in illuminance with respect to the pixel value of an image made in grayscale, so that illuminance correction can be used in a black and white image. For example, the illuminance correction algorithm scans the entire input image using a screen window having a size of 3×3 pixels within the input image, and compares the feature points extracted on the screen window with the center pixel of the screen window according to the difference value. It is also possible to perform correction by binarizing.

Further, the resolution correction algorithm is performed when the constituent pixels of the forward region including the facial image are 25×25 to 50×50; It is possible to perform a padding operation of masking the edge portion of the forward area including the facial image.

In addition, the real correction algorithm uses a Machine Learning Classifier using a classifier that uses only CNN neural network learning, without relying on hardware such as distance sensors and ultraviolet sensors, to prevent spoofing attacks using portrait photos and videos. However, the object detection module is used to detect spoofing attacks.

That is, the real correction algorithm may be learned through a CNN neural network to detect an object included in the input image data, and may distinguish the detected object into a real facial object, a non-real facial object, and a non-facial object.

Through learning using the AI artificial neural network according to the present invention as seen above, the following effects can be expected in a face detection and identification system using a multi-focusing multi-layer backbone neural network and an auxiliary neural network.

Face recognition images can be searched using various templates (face recognition neural networks, or detectors). A template suitable for various image resolutions and environmental conditions (illumination/angle/realistic authentication) can be configured and used for the following purposes.

1) Utilization as a marketing information tool (detection): Analysis of the floating population of the exhibition hall, seminar certification

2) Specific person detection and comparison (identity recognition): criminal, lost child search, VIP movement

3) Personal authentication (identity authentication): Processes facial recognition data with the same level of accuracy as biometric authentication (unmanned stores/access door control/non-face-to-face identity authentication)

In addition, in the present invention, the purpose of security can be achieved even without interfering with the normal service flow by monitoring and confirming the identity through the face recognition of the suspected crime without explicit notice to the user as a continuous authentication for the face. It can have an effect.

In addition, in the present invention, even when the difference in environmental factors such as a difference in resolution and a difference in illuminance of an input image is severe, there is an effect of effectively detecting or recognizing a face according to the state of the input image.

On the other hand, the present invention provides a face detection and identification system using a multi-layered Backbone AI Convolutional Neural Network and an Auxiliary AI Convolutional Neural Network (template, detector) so that the neural network can also effectively detect or recognize it. There is an effect that can be done.

1 is an exemplary view showing various examples of resolution of an input image input through an image input device.
2 is a block diagram showing the configuration of a face detection and identification system according to a specific embodiment of the present invention.
3 is an exemplary view showing a driving process according to various fields of application of the present invention.
Figure 4 is an exemplary view showing the configuration of an AI analysis server constituting a specific embodiment of the present invention.
5 is an exemplary view showing a process of using a facial recognition API between an input unit, a backend server, and a mobile device constituting a specific embodiment of the present invention.
6 is an exemplary view showing a correction principle by an object correction algorithm in a specific embodiment of the present invention.
7 is an exemplary diagram showing an operation process of a backbone neural network according to a specific embodiment of the present invention.
8 is an exemplary diagram showing size and resolution statistics of images stored in an image set.

Hereinafter, a face detection and identification system of multi-focusing content using an artificial neural network according to a specific embodiment of the present invention will be described with reference to the accompanying drawings.

Prior to the detailed description of the present invention, the facial recognition technology applied to the present invention will be briefly described first.

The K-value unit constituting the present invention may be expressed as a function of 1) input image resolution level, 2) average number of pixels per face area, 3) face ratio as shown in [Table 1] below. It can also be calculated by the number of pixels.

[Table 1] below defines the minimum required pixel value based on the maximum number of face areas that can be processed according to the camera transmission resolution for each of the face detection/face recognition/facial authentication. Depending on the processing performance of the facial detection/recognition system, the facial ratio may increase further, and in this case, a larger number of facial data can be processed.

Example of calculating the minimum pixel for facial data analysis In the case of detection resolution
division Camera transmission
resolution maximum
Face number Minimum pixel
(Length) Minimum pixel
(area) Total face width (pixel) Facial ratio
(%) SD 720 480 30 24.0 576 17.280 5.00 HD 1280 720 50 25.6 655 32.768 3.56 FHD 1920 1080 80 24.0 575 46.080 2.22 QHD 2580 1440 100 25.6 655 65.536 1.78 UHD 3840 2160 150 25.6 655 98.304 1.19 In the case of Recognitiond resolution
division Camera transmission
resolution maximum
Face number Minimum pixel
(Length) Minimum pixel
(area) Total face width (pixel) Facial ratio
(%) SD 720 480 15 48.0 2304 34.560 10.00 HD 1280 720 25 51.2 2621 65.536 7.11 FHD 1920 1080 40 48.0 2304 92.160 4.44 QHD 2580 1440 50 51.2 2621 131.072 3.56 UHD 3840 2160 75 51.2 2621 196.608 2.37 In the case of Recognitiond resolution
division Camera transmission
resolution maximum
Face number Minimum pixel
(Length) Minimum pixel
(area) Total face width (pixel) Facial ratio
(%) SD 720 480 7 102.9 10580 74.057 21.43 HD 1280 720 12 106.7 11378 136.533 14.81 FHD 1920 1080 19 101.1 10212 194.021 9.36 QHD 2580 1440 25 102.4 10485 262.144 7.11 UHD 3840 2160 39 98.5 9695 378.092 4.56

Here, as an embodiment, a method of obtaining an average pixel value per facial area is applied. That is, the following equation is applied.

A avg = (A1 + A2 +… + An) / n

Since the input images have different resolutions included in each person's facial area, the task of optimizing all resolutions in the same backbone neural network takes a lot of cost and time for learning, and speed and accuracy are different. Since they have conflicting relationships, in order to analyze and recognize images of all resolutions with a single backbone neural network, there is an inefficiency of having to calculate a neural network with a very large structure and many neural network parameters.

On the other hand, the K-value unit calculates a K value by determining the number of pixels included in the facial area according to the minimum recognized face size, the facial ratio on the image, and the number of faces based on the resolution pixels of the input image data. According to the K value, one of the facial detection algorithm, identification recognition algorithm, or authentication recognition algorithm may be selected to analyze facial information.

In the case of calculating the K value using the number of individual pixels for the facial area as described above, compared to the case of calculating the K value using the average number of pixels for the facial area, the calculation efficiency will be lowered. However, the occurrence of identification errors can be reduced.

Therefore, the method of calculating the K value can be selectively applied according to the specification of the system, the difference in perspective in the applied space, and the amount of data to be processed.

Hereinafter, a process of the present invention for face detection or face recognition using a backbone neural network will be described. The figure below illustrates the facial recognition process according to the present invention. As shown in FIG. 7, a part marked with BackBone neural network #1 is a part responsible for face detection and localization, and a part marked with BackBone neural network #2 is a neural network for facial recognition. .

Regarding the input image, the K value unit determines factors such as 1) the input image resolution level, 2) the average number of pixels per face area, and 3) the face ratio, and if it is determined for face detection, it is used for face detection. It is terminated after passing through only the BackBone neural network #1 (Face Detection Neural Network, Localization Neural Network).

In addition, if the K-value department determines that it is for facial recognition, use BackBone Neural Network #1 (Face Detection Neural Network, Localization Neural Network) for facial recognition, then go through the LandMarking process again to create a reference model, and then the image is used for facial recognition. Through the BackBone Neural Network #2 (Recogniton Neural Network), an encoding vector that is unique to the image is created.

In addition, if the K Value Department determines that it is for facial authentication, the image is uniquely encoded through BackBone neural network #1 for facial authentication (Face Detection neural network, Localization neural network) and BackBone neural network for facial authentication #2 (Recogniton neural network). Create a vector.

Hereinafter, a process of face detection or face recognition using an auxiliary neural network will be described.

7 illustrates a facial recognition process according to the present invention. As shown, the part marked with auxiliary neural network #1 is the part to which realness authentication is applied, and the part marked with auxiliary neural network #2 is the neural network where angle correction/illuminance correction is performed.

Assuming that image data is transmitted and K-value is set only for facial authentication, at this time, auxiliary neural network #1 (Liveness auxiliary neural network) processes the image data first before passing through the backbone neural network.

That is, without using a special infrared sensor or distance sensor, it is possible to determine a facial spoofing attack with an auxiliary neural network for Liveness learned using a Convolution Neural Network (CNN).

The auxiliary neural network for liveness includes an object detection module, and is a neural network that learns to extract features of an object for non-real objects (ie, photos/videos) transmitted from a mobile terminal.

The auxiliary neural network #2 is performed after completing the LandMarking work on the geometric position of the face and before proceeding with the work to determine the reference model. As long as the face is rotated up/down/left/right, it is corrected to look at the front, or if the current illuminance level is too high or too low, it is adjusted so as not to be sensitive to such changes to create a reference model.

On the other hand, in order to help understand the present invention, the resolution correction / illuminance correction will be described.

As shown in FIG. 8, millions of images are stored in the ImageNet data set, and most of them occupy more than 80% of images having a size of 40 to 140 pixels.

In addition, the total resolution of the image file actually input from the input unit varies as shown.

In other words, when we use the data of ImageNet for artificial neural network training, the neural network that has been trained will be optimized for this learned resolution range (Pixel range), and the image file containing these objects will have multiple resolutions. Expressed and will be transmitted in the input unit.

Since input images basically have different resolutions, optimizing all resolutions in the same neural network takes a lot of cost and time, and speed and accuracy have opposite relations, so a single backbone neural network is used. In order to recognize all resolutions, a neural network with a very large structure and many parameters must be calculated, resulting in a speed reduction problem.

Even in the case of illuminance, it is highly affected by environmental changes. That is, depending on the weather, a change in the illuminance occurs depending on the location of bright weather, dark weather, or living room, office, corridor, etc., and it is necessary to correct it.

Classification of illuminance by grade Case Illuminance (Lux) L1 50 L2 100 L3 200 L4 300 L5 500 L6 1000 L7 2000 L8 5000

Stable facial detection/recognition system that can actively respond to environmental changes and is relatively less affected by environmental changes when attempting to perform facial detection/recognition using appropriate backbone neural networks and auxiliary neural networks by detecting them in preparation for all environmental changes Can be provided.

FIG. 2 is a block diagram showing the configuration of a face detection and identification system according to a specific embodiment of the present invention, FIG. 3 is an exemplary view showing a driving process for various fields of application of the present invention, and FIG. 4 is a detailed diagram of the present invention. An exemplary diagram showing the configuration of an AI analysis server constituting an embodiment, and FIG. 5 is an exemplary diagram showing a process of using a facial recognition API between an input unit constituting a specific embodiment of the present invention, a backend server, and a mobile device 6 is an exemplary diagram showing a correction principle by an object correction algorithm in a specific embodiment of the present invention, and FIG. 7 is an exemplary diagram showing an operation process of a backbone neural network according to a specific embodiment of the present invention. 8 is an exemplary diagram showing size and resolution statistics of images stored in an image set.

First, as shown in Figure 2, the face detection and identification system according to the present invention is configured to include the input unit (110, 120, 130) and the management server (200).

The input unit can be applied to various types of video input devices (mobile terminal 110, web terminal 120, local terminal 130), and all devices (mobile devices, 110) that can be photographed with a camera, PC and USB It may be a connected web camera (web terminal, 120) or a Local IP camera 130.

At this time, the captured video is transmitted to the backend server 210 in the form of streaming data in real time, and the frame is appropriately divided according to the number of cameras and the amount of load.

The load distribution may be a load distribution composed of the web terminal 120, the mobile terminal 110, and the local terminal 130, respectively, or, for example, may be a load distribution between several IP cameras in the local terminal 130. .

Meanwhile, the backend server 210 induces load distribution by lowering the number of frames set in the video streaming data for all connected devices.

For example, a video transmitted from the local terminal 130 to the backend server 210 is divided into 10 to 30 frames for load distribution, and the frame data is analyzed by the AI analysis server 220. In the case of a designated transmission frame number, for example, (130), a terminal that is later than 24 FPS is identified and the fps of the terminal is lowered from 24 to 20 so that it can be transmitted so that the overall load is less affected by the terminal. To induce.

The mobile terminal 110 is provided with a self-contained camera to provide a photographing function. In addition, it is preferable that the web terminal 200 can recognize a plurality of objects (persons) in a browser environment using a web camera connected through a PC and a USB port.

In addition, the local terminal 130 may be a security IP camera installed in a public place so as to observe a number of people and configured to enable IP communication.

On the other hand, the management server 200 receives the video data received from the input unit in a streaming format, and then a back-end server 210 for adjusting the load and frame of the received video data, and the back-end It is configured to include an AI analysis server 220 that analyzes the image data adjusted from the server 210 through an artificial neural network algorithm to detect or identify a face on the image data.

It may be configured as a local server such as the BackEnd server 210 and the Al analysis server 220, or may be configured in a form separated from each other in a cloud form.

Meanwhile, as shown in FIG. 3, when an input image (streaming data) is transmitted, it is transmitted to the AI analysis server 220 through a load distribution process in the backend server 210.

Then, the AI analysis server 220 analyzes (facial detection/recognition work) using image files divided by frame units of the transferred video.

To this end, the AI analysis server 220 is configured to include a region-specific government, a K-value unit, an AI analysis unit, and an image correction unit.

The region-specific unit is a part for specifying the facial region (a task of bounding the facial region with the result of facial detection and the corresponding facial position detection) from the image data input through the facial detection algorithm neural network (RPN neural network). Face shape, skin color and boundary strabismus distinction (?), and movement when a part such as the mouth of the face is in motion (head shape / color contrast / head movement / mouse speaking) can be detected as a facial area.

Next, the K-value unit determines the number of facial areas, resolution information of the input image data, and the number of input image data in the entire image area. The method of analysis can be determined. In this case, the K value may be calculated as a specific value, but preferably may be expressed as a set of values for various characteristic elements.

In addition, the AI analysis unit is configured to include a face detection algorithm and a face recognition algorithm, and performs an image analysis of detecting (specific) a face from the entire image through the face detection algorithm on the input image data, or through a face recognition algorithm. An analysis to determine a facial image may be performed by recognizing a face including an input image.

The analysis process of the AI analysis unit and the algorithm for this will be described in detail later.

Meanwhile, the image correction unit includes an illuminance correction algorithm, a resolution correction algorithm, an angle correction algorithm, and an object correction algorithm so that illumination, resolution, angle and liveness correction for input image data can be performed. The correction algorithm is selectively selected and performed according to the type of facial information analysis algorithm selected in the part.

Hereinafter, the analysis process of the AI analysis unit and the correction process of the image correction unit will be described in more detail.

First, the analysis of the AI analysis unit is largely divided into face detection and face recognition, and the face recognition is further divided into face recognition for identification and face recognition for authentication, and can be divided into a total of three analysis.

Here, the face detection is to distinguish a person's face from the entire input image, and through this, it can be used to calculate the number of people in the image, the amount of movement of the person, and the direction of movement. In order to distinguish the face from other objects, It is preferable that the number of pixels constituting the facial area is at least approximately 25×25 pixels.

In addition, the face detection distinguishes a person's facial image from other objects, and it is not possible to specify who the facial image is, but where the area judged by the shape of the person's face (eye/ear position, etc.) It is a task (Boundering) to draw an outline by judging whether it exists in. That is, it is a step in which only the classification and location information of the object can be grasped.

Real-time identification of the number of people in a specific dense area is possible, but it is not a step to track that they are the same person. Macroscopic statistical data can be inferred using data on the number and gender of people in a specific place.

In this case, in order to detect the face of the facial image, it is preferable that the number of pixels constituting the corresponding facial area is approximately 25×25 pixels or more.

In addition, the identification facial recognition is a step of identifying a person's facial image and further identifying the characteristics of the corresponding facial, specifying who the facial image is, and characteristic information of the facial image (race, gender, age, skin color, Face shape structure, etc.) can also be identified.

At this time, in order to grasp the characteristic information of the facial image, the number of pixels constituting the corresponding facial area should be approximately 50×50 pixels or more.

Finally, the facial recognition for authentication refers to an authentication analysis used for user authentication, which identifies who the facial image is.In order to derive a reliable authentication result, the number of pixels constituting the facial area is approximately 90× It is desirable to have more than 90 pixels.

Such face detection and recognition are performed by a face detection algorithm, a recognition algorithm for identification, and a recognition algorithm for authentication, respectively, and these algorithms are composed of CNN artificial neural networks and are implemented by artificial intelligence learning.

These BackBone CNN neural networks may be classified in terms of use as shown in [Table 3] below.

Minimum recognized face size
(Min.Face.Size) The percentage of the face on the screen
(Face Ratio of window) Face count
(No. of faces) BackBone CNN#1
(Facial detection algorithm) 25x25 20% or more 50 or more BackBone CNN#2
(Recognition algorithm for identification) 50x50 10-20% 20-50 BackBone CNN#3
(Authentication recognition algorithm) 90x90 below 10 1-3

As described above, the AI analysis unit selects any one of a face detection algorithm, an identification recognition algorithm, or an authentication recognition algorithm to perform analysis on the input image data.

In addition, the analysis performed on the input image according to the selection of these algorithms performs any one of face detection, face identification, or face authentication.

At this time, basically, when the number of pixels constituting the facial images included in the input image is mostly between 25x25 and 50x50, the AI analysis unit performs face detection by a face detection algorithm, and the composition of the face images When the number of pixels is mostly between 50x50 and 90x90, face recognition for identification is performed by an identification recognition algorithm, and when the number of pixels constituting the face images is mostly 90x90 or more, the recognition algorithm for authentication By performing facial recognition for authentication.

However, when the number of pixels constituting the facial images included in the input image is divided into a similar ratio between the facial images having the reference pixel number or more and the facial images having the lower number of pixels, a criterion for selecting an analysis algorithm for this must be prepared.

Here, the number of reference pixels refers to the number of pixels that are the standard for face detection and facial recognition (the number of pixels for facial recognition, 50x50), or the number of pixels that are the criteria for the classification of face recognition for identification and facial recognition for authentication (the number of pixels for authentication, 90x90 ).

In the present invention, when the average of the number of pixels included in each of the facial areas matches within an error range with the reference number of facial recognition pixels, the facial recognition algorithm is analyzed when the average deviation of the number of pixels of each facial area is equal to or greater than a preset value. It is selected as an algorithm, and if it is less than a preset value, a face detection algorithm is selected as an analysis algorithm.

Of course, in the case of the reference pixel for authentication, similarly, when the average of the number of pixels included in each facial region is within an error range with the reference pixel number for authentication, the average deviation of the number of constituent pixels of each facial region is preset. If the value is higher than the value, the recognition algorithm for authentication is selected as the analysis algorithm, and if it is less than the preset value, the recognition algorithm for identification is selected as the analysis algorithm.

In other words, when the average of the number of pixels included in each facial area matches the reference pixel number within the error range, all facial area images are composed of the number of pixels similar to the reference pixel number, and the configuration of the facial area images The difference between the number of pixels and the number of reference pixels is large, but there are cases where the positive (+) value and the negative (-) value are similar.

In the latter case, it means that some of the facial area images are located near the image input terminal and other facial images are located far from the background. On the other hand, in the former case, since the entire facial images are clustered at the boundary of the limit of the number of possible analysis pixels, the analysis based on the low number of pixels is performed.

The selection of the analysis algorithm as described above may be determined by the K value calculated by the K-value unit.

That is, the K value is calculated by determining the number of pixels included in the face according to the minimum recognized face size, the face ratio on the image, and the number of faces based on the resolution pixels of the input image data, and the K value is a specific number. It may be converted or may be a plurality of data for various discrimination factors.

Hereinafter, various image corrections by the image correction unit will be examined.

First, the illuminance correction algorithm scans the entire input image using a screen window having a size of 3×3 pixels within the input image, and compares the feature points extracted on the screen window with the center pixel of the screen window, according to the difference value. It is also possible to perform corrections by evolution (divided by 0 and 1).

In addition, the resolution correction algorithm is performed when the constituent pixels of the forward region including the facial image are 25×25 to 50×50, and in this case, the reference requirement for facial recognition is satisfied, but the resolution is not sufficient. Is to perform corrections for

In this case, the resolution correction may be performed by a padding operation of masking an edge portion of the forward region including the facial image.

In other words, the padding operation is to facilitate detection of a small face image by creating a boundary layer in contrast to the face image in order to enhance the face detection effect even at a low resolution in the case of a face face portion having a relatively low resolution.

On the other hand, as the characteristic elements of the face of the facial image, the location/size/shape of each part (eyes, nose, mouth, cheekbones, chin), and texture of skin or hair (wrinkles/patterns, blemishes) are mainly used.

At this time, two methods of appearance base/learning base can be used for facial detection.

On the other hand, the real correction algorithm is an algorithm that corrects the object area by determining whether the object area of the input image is a facial or non-facial, and if it is a facial, whether it is a real or non-real facial (an image from which a facial image is retaken). , The algorithm is learned through a CNN neural network, detects an object included in the input image data, as shown in FIG. 6, and distinguishes the detected object into a real facial object, a non-real facial object, and a non-facial object. If the object is not a real face, it is implemented to correct the corresponding area.

According to another embodiment of the present invention, in the present invention, a facial recognition program may be installed in the input unit.

On the other hand, when the facial recognition program is not installed in the input unit, the terminal identification number is transmitted to the backend server 210, and an API link is requested to the backend server 210 to receive an encrypted URL and a connection address, Use the default browser of the mobile device (Safari for IoS, chrome browser for Android devices) to access the provided URL (API).

In addition, in order to sequentially analyze the streaming data transmitted from multiple input units at the same time, the backend server 210 transmits a response request signal to the input unit and determines the data transmission rate locally according to the response rate of the corresponding input unit. It may be configured to include a delay unit to enable delay.

The rights of the present invention are not limited to the embodiments described above, but are defined by what is described in the claims, and that a person having ordinary knowledge in the field of the present invention can make various modifications and adaptations within the scope of the rights described in the claims. It is self-evident.

The present invention relates to a system for performing face detection and face recognition using an AI artificial intelligence learning network in a cost-effective method, and in the present invention, face detection in a mobile device for non-face-to-face financial transactions, etc. By monitoring and verifying the user's identity without explicit notice to the user, it is effective in removing security risks without interfering with the normal service flow.

110: mobile terminal 120: web terminal
130; Local terminal 200: management server
210: backend server 220: AI analysis server

Claims (12)

An input unit for receiving image data;
It comprises a management server for receiving and processing image data from the input unit:
The management server,
A back-end server that receives video data received from the input unit in a streaming format and adjusts a load and a frame of the received video data;
An AI analysis server configured to analyze the image data adjusted from the backend server through an artificial neural network algorithm to detect or identify a face on the image data, comprising:
The AI analysis server,
A region specifying unit for specifying a facial region from the image data input through the RPN neural network (a task of bounding the facial region with the result of facial detection and the corresponding facial position detection);
A K-value unit for determining the number of facial areas, resolution information of the input image data, and the number of input image data;
Consisting of a face detection algorithm to identify the shape of a face image by distinguishing it from other objects, and a face recognition algorithm to identify and authenticate characteristic information of the face image, the face detection algorithm or face recognition algorithm for input image data An AI analysis unit for selectively applying the facial information to analyze facial information;
Facial information analysis selected by the AI analysis unit, including illuminance correction algorithm, resolution correction algorithm, angle correction algorithm, and real object correction algorithm so that illumination, resolution, angle and liveness correction for input image data can be performed. It includes an image correction unit that selectively selects and performs correction algorithms according to the type of algorithm:
The facial recognition algorithm,
An identification algorithm for identifying characteristic information (race, sex, age, skin color and shape structure) of a facial image;
It includes a recognition algorithm for authentication that verifies and authenticates that the person is the same as a specific person, and:
The K-value unit,
Calculating a K value according to the average number of pixels per facial area of the input image data;
The AI analysis unit,
A multi-focusing AI neural network, characterized in that facial information is analyzed by selecting one of the facial recognition algorithm, identification recognition algorithm, or authentication recognition algorithm according to the average number of average pixels per facial area of the input image. Used face detection and face identification system.
The method of claim 1,
The AI analysis unit, according to the K value,
If the average value of the average number of pixels per facial area of the input image is between the number of reference pixels for face detection (25 × 25 pixels) and the number of reference essential cells for identification (50 × 50 pixels), select a face detection algorithm to obtain facial information. To analyze;
If the average value of the average number of pixels per facial area of the input image is between the number of reference mandatory cells for identification (50×50 pixels) and the number of reference pixels for authentication (90×90 pixels), select a recognition algorithm for identification and retrieve facial information. Analyze;
If the average value of the average number of pixels per facial area of the input image exceeds the reference number of pixels for authentication (90×90 pixels), select a recognition algorithm for authentication and analyze the facial information:
If the average value of the average number of pixels per facial area of the input image matches within the error range with the number of reference required cells for identification (50×50 pixels),
When the average deviation value of the number of constituent pixels of each facial region is greater than or equal to a preset value, the facial information is analyzed by selecting an identification recognition algorithm;
When the average deviation value of the number of constituent pixels of each facial area is less than the preset value, the facial information is analyzed by selecting as a facial detection algorithm:
If the average value of the average number of pixels per facial area of the input image matches the reference number of pixels for authentication (90×90 pixels) within the error range,
When the average deviation value of the number of constituent pixels of each facial region is greater than or equal to a preset value, the facial information is analyzed by selecting a recognition algorithm for authentication;
Face detection and face identification system using a multi-focusing AI neural network, characterized in that, when the average deviation value of the number of constituent pixels of each facial region is less than a preset value, facial information is selected as an identification recognition algorithm.
The method of claim 2,
The input unit,
A mobile terminal provided with a self-taking function through a self-contained camera;
A web terminal connected to a PC and capable of recognizing a plurality of people in a browser environment using a web camera; or
A local terminal installed in an open place so as to observe a number of people and configured to enable communication; Face detection and facial identification system using a multi-focusing AI neural network, characterized in that it is configured to include any one or more of.
delete delete delete The method according to any one of claims 1 to 3,
The input unit,
A face detection and face identification system using a multi-focusing AI neural network, characterized in that a face recognition program is installed.
The method of claim 7,
The input unit,
If the facial recognition program is not installed, it transmits the terminal identification number to the backend server, requests an API link to the backend server, receives the encrypted URL and connection address, and uses the default browser of the mobile device, Face detection and face identification system using a multi-focusing AI neural network characterized by access to the provided URL (API).
The method of claim 8,
The backend server,
In order to sequentially analyze streaming data transmitted from multiple input units at the same time, it includes a delay unit that sends a response request signal to the input unit to locally delay the data transmission rate according to the response rate of the corresponding input unit. Face detection and face identification system using a multi-focusing AI neural network, characterized in that.
The method of claim 7,
The illuminance correction algorithm,
While scanning the entire input image using a screen window of 3×3 pixel size within the input image, the feature points extracted on the screen window are compared with the center pixel of the screen window, and the binarization (0 and 1 Classification) Face detection and face identification system using a multi-focusing AI neural network characterized by performing correction by numbers.
The method of claim 10,
The resolution correction algorithm,
Performed when the constituent pixels of the forward area including the facial image are 25×25 to 50×50;
A face detection and face identification system using a multi-focusing AI neural network, characterized in that a padding operation for masking an edge of a forward area including the facial image is performed.
The method of claim 7,
The real correction algorithm,
It is learned through a CNN neural network, and includes an object detection module to detect an object included in the input image data;
It is designed to classify the detected object into 1) real facial object, 2) non-real facial object, and 3) non-facial object. A face detection and face identification system using a multi-focusing AI neural network characterized by learning through a CNN artificial neural network.

Images