KR102567183B1 - A Door Control System using Face Recognition based on Continual Learning - Google Patents

Abstract

The present invention relates to a door control system through continuous learning-based face recognition, comprising: a camera installed at a door to photograph a person entering and exiting the door; a face recognition unit that identifies visitors photographed through the camera through face recognition learning for registered persons stored therein; and a control unit controlling opening and closing of the door according to a recognition result of the face recognition unit, wherein the face recognition unit includes: a face image detector detecting a face image from image data transmitted from the camera; a feature extractor extracting features from the face image detected by the face image detector; a data storage unit storing IDs of registered persons, face image data for each ID, and feature data including n (where n is a natural number) dimensional array of features included in the face image data in the form of a lookup table; and an identification module that learns the face image and feature data for each ID stored in the data storage unit and identifies the ID stored in the data storage unit through the extracted data transmitted from the feature extraction unit.

Description

A Door Control System using Face Recognition based on Continual Learning}

The present invention relates to a door control system through continuous learning-based face recognition, and more particularly, through continuous learning-based face recognition that can prevent performance degradation due to oblivion of shared attributes of data to be preserved during data update. It is about the door control system.

In general, a locking device is installed at an office door in a building to prevent intrusion by outsiders. The locking device provided at the door has a disadvantage in that it is easily exposed to outside intrusion due to a weak security due to a simple structure.

In order to solve these disadvantages, biometric technology has been widely studied as an alternative to supplement the disadvantages of access control systems using passwords or PINs (Personal Identification Numbers), which are mainly used in personal identification systems.

Unlike passwords, the biometric system is difficult to use by personal theft or copying, has little risk of being changed or lost, and enables post-tracking of users, so it has the advantage of being able to come up with an efficient plan in terms of system management. there is.

Among them, face recognition is an intuitively understandable method, so it is being actively studied in the field of computer vision and pattern recognition. systems and the like.

In general, face recognition extracts features from a face image input through a feature extraction network and recognizes a face using the extracted features. In the case of continuous addition, new face images are not always added due to the limited capacity of the database, that is, the storage device.

Conventionally, in order to eliminate this problem, when the number of database records for one person exceeds the number assigned to one person, the oldest record is deleted or the face image with the lowest similarity among images of one person is deleted.

However, if the oldest record or face image with the lowest similarity is deleted and then changed to a new face image (i.e. face image update), catastrophic forgetting occurs and the shared attributes of the data to be preserved are forgotten. There is a problem that the performance degradation occurs according to.

Republic of Korea Patent Publication No. 2020-0094649

Therefore, the present invention is to solve the above problems, and an object of the present invention is door control through continuous learning-based face recognition that can prevent performance degradation due to oblivion of shared attributes of data to be preserved during data update. to provide the system.

In order to achieve the above object, according to the present invention, the camera is installed on the door to photograph the person entering and exiting; a face recognition unit for identifying visitors photographed through the camera through face recognition learning for registered persons stored therein; and a control unit controlling opening and closing of the door according to a recognition result of the face recognition unit, wherein the face recognition unit includes: a face image detector detecting a face image from image data transmitted from the camera; a feature extractor extracting features from the face image detected by the face image detector; a data storage unit storing IDs of registered persons, face image data for each ID, and feature data including n (where n is a natural number) dimensional array of features included in the face image data in the form of a lookup table; and an identification module that learns face images and feature data for each ID stored in the data storage unit and identifies the ID stored in the data storage unit through the extracted data transmitted from the feature extraction unit.

According to the present invention, the feature extraction unit extracts feature data from a face image through Equation 1 below.

[Equation 1]

는 보정계수, x는 얼굴 이미지 데이터를 나타냄.where v _t is the characteristic data of the registered personnel,

is a correction coefficient, and x represents face image data.

According to the present invention, the identification module is characterized in that data is identified after compensating for losses as shown in Equations 2 and 3 below when identifying data.

[Equation 2]

here, is the loss function, E _x is the expected value, x is the input face image, and _pi represents the Softmax Function of the ith ID when there are IDs from 1 to L.

[Formula 3]

Here, v is the feature data of the registered person, T is the transposition vector, τ is the temperature, Q is the maximum allowable data capacity for the unregistered person, and u is the feature data of the unregistered person.

According to the present invention, the data storage unit is characterized by updating internal data with new data using Equation 4 below when it is impossible to store additional data therein.

[Formula 4]

Here, W _j and W _j are the cluster face image data closest to the face image detected by the image detector among the face image data stored in the data storage unit, and C _i and C _j are the faces contained in the i-th and j-th position buffers. Indicates the number of images.

According to the present invention, when updating old data with new data, two existing data closest to the new data to be updated are stored in one buffer using the Euclidean distance metric, and the new data is stored in the other buffer Therefore, it is possible to prevent destructive oblivion caused by the deletion of all existing data when new data is updated, thereby preventing performance degradation due to oblivion of shared attributes of data to be preserved during data update.

1 is a diagram illustrating a face recognition device based on continuous learning according to an embodiment of the present invention.
2 is a diagram illustrating a method of detecting a face image from an input image.

Hereinafter, with reference to the accompanying drawings, a preferred embodiment in which a person skilled in the art can easily practice the present invention will be described in detail. However, in the detailed description of the operating principle of the preferred embodiment of the present invention, if it is determined that a detailed description of a related known function or configuration may unnecessarily obscure the subject matter of the present invention, the detailed description will be omitted.

1 is a diagram showing a door control system through face recognition based on continuous learning according to an embodiment of the present invention.

Referring to FIG. 1 , the door control system through face recognition based on continuous learning according to an embodiment of the present invention includes a camera 10, a face recognition unit 20, and a control unit 30.

The camera 10 is installed on the door and takes pictures of the person entering and exiting the door.

The face recognition unit 20 identifies visitors photographed through the camera 10 through face recognition learning for registered persons.

The face recognition unit 20 includes an image detection unit 22, a feature extraction unit 24, a data storage unit 28, and an identification module 26.

The image detector 22 detects a face image by locating a face in an input image transmitted through the camera 10, and is composed of RetinaNet.

The feature extraction unit 24 is composed of a VGG-16 network and extracts features required for face recognition from the face image detected by the image detection unit 22. At this time, the extracted features are made to have 256-dimensional vector values.

The data storage unit 28 is an n (where n is a natural number) dimension array of IDs set for each registered person (or learner), a plurality of face image data obtained for each ID, and features extracted from each face image data. Feature data consisting of is stored in the form of a lookup table.

The ID, face image, and feature data stored in the data storage unit 28 are used as learning data for face recognition learning of the identification module 26 to be described later, and each of them according to the result identified by the identification module 26 Face images and feature data for each ID are updated.

In addition, the data storage unit 28 temporarily stores face images and feature data of unregistered persons (ie, outsiders) in addition to IDs, face images, and feature data of registered persons.

The identification module 26 identifies the ID stored in the data storage unit 28 through the extracted data transmitted from the feature extraction unit 24 by learning face images and feature data for each ID stored in the data storage unit 28 . In other words, the identification module 26 learns face images and feature data for each ID stored in the data storage unit 28, and when extraction data is input from the feature extraction unit 24, the extracted data is a face of a certain ID. If the extracted data belongs to the ID stored in the data storage unit 28 by determining whether it is close to or coincides with the image and feature data, the corresponding ID is output, and if the extracted data does not belong to the ID stored in the data storage unit 28, the number of registered persons (or the number of learners) is judged not to be, and output as an outsider.

For this reason, in the case of managing an entrant using face recognition, it is possible to manage access availability according to an identification result output from the identification module 26 .

The control unit 30 controls the opening and closing of the door according to the identification result of the person transmitted from the face recognition unit 20 .

The operation of the door control system through continuous learning-based face recognition having such a configuration is as follows.

When image data taken in real time by the camera 10 is input to the image detection unit 22 of the face recognition unit 20, the image detection unit 22 detects a face image from the input image data as shown in FIG. 2 through RetinaNet. .

The detected face image is transmitted to the feature extractor 24, and the feature extractor 24 extracts features required for face recognition from the face image detected by the image detector 22 through the VGG-16 network.

개의 유클리드 공간(D)을 갖는 룩업 테이블(V)이 저장되는데, 룩업 테이블의 ID가 L개 이기 때문에 t번째 ID의 얼굴 이미지가 입력될 때마다 아래의 수식 1과 같은 ID 별 최종 특징을 추출하게 된다.For example, when all of the people to be recognized are L, and the output value (ie, the array structure of feature information) from the identification module 26 is D, the data storage unit 28 has

A lookup table (V) having Euclidean space (D) is stored. Since the number of IDs in the lookup table is L, the final feature for each ID is extracted as shown in Equation 1 below whenever the face image of the tth ID is input do.

[Equation 1]

는 보정계수(0.01 값을 가짐), x는 얼굴 이미지 데이터를 나타냄.where v _t is the characteristic data of the registered personnel,

is a correction coefficient (has a value of 0.01), and x represents face image data.

On the other hand, the feature data (v _t ) extracted by the feature extractor 24 is transmitted to the identification module 26, and the identification module 26 transmits the extracted data from the feature extractor 24 to the data storage unit ( 28) to identify the stored ID.

That is, if the extracted data transmitted from the feature extraction unit 24 belongs to the ID stored in the data storage unit 28, the identification module 26 outputs the ID, and the extracted data is stored in the data storage unit 28. If it does not belong to the ID stored in , it is determined that it is not a registered person and output as a non-registered person (ie, an outsider).

For this reason, in the door control system to which the continuous learning-based face recognition device of the present invention is applied, the control unit 30 opens the door when there are registered personnel, and when it is an unregistered outsider, the control unit 30 sounds an alarm or does not register By providing a notification notifying that an unauthorized person is present, it is possible to induce an outsider to register an ID when he or she wants to enter.

Meanwhile, when data is identified in the identification module 26, since loss as shown in Equation 2 below occurs, the data is identified after compensating for the loss.

[Equation 2]

here, is the loss function, E _x is the expected value, x is the input face image, and _pi represents the Softmax Function of the ith ID when there are IDs from 1 to L.

[Formula 3]

Here, v is the feature data of the registered person, T is the transposition vector, τ is the temperature, Q is the maximum allowable data capacity for the unregistered person, and u is the feature data of the unregistered person.

In the door control system through continuous learning-based face recognition of the present invention, when the data storage unit 28 is full of data and no longer can store new data (ie, when it is impossible to store additional data inside), As shown in Equation 4 below, it is updated with new data using a Euclidean distance metric.

[Formula 4]

Here, W _j and W _j are the cluster face image data closest to the face image detected by the image detector among the face image data stored in the data storage unit, and C _i and C _j are the faces contained in the i-th and j-th position buffers. Indicates the number of images.

In this way, the face image data and feature data stored in the i-th buffer are changed, and new data (ie, face image data and feature data) are stored in the j-th buffer.

In this way, when the door control system through continuous learning-based face recognition according to an embodiment of the present invention updates existing data with new data, the Euclidean distance metric is used for the two existing data closest to the new data to be updated. Since new data is stored in one buffer and new data is stored in the other buffer, it is possible to prevent destructive oblivion caused by the deletion of all existing data when new data is updated, thus sharing data to be preserved when updating data. It is possible to prevent performance degradation due to property forgetting.

As described above, the detailed description of the present invention has been described with respect to the preferred embodiment of the present invention, but this is only an illustrative description of the best embodiment of the present invention, and does not limit the present invention. In addition, it is of course possible for anyone having ordinary knowledge in the technical field to which the present invention belongs to various modifications and imitations without departing from the scope of the technical idea of the present invention. Therefore, the scope of the present invention should not be limited to the described embodiments and should not be defined, but should be defined by not only the claims to be described later, but also those equivalent thereto.

10: camera 20: face recognition unit
22: image detection unit 24: feature extraction unit
26: identification module 28: data storage unit
30: control unit

Claims (4)

A camera installed at the entrance door to photograph the person entering and exiting the door;
a face recognition unit for identifying visitors photographed through the camera through face recognition learning for registered persons stored therein; and
A control unit for controlling opening and closing of the door according to a recognition result of the face recognition unit,
The face recognition unit,
a face image detector detecting a face image from the image data transmitted from the camera;
a feature extraction unit extracting features from the face image through Equation 1 below;
a data storage unit storing IDs of registered persons, face image data for each ID, and feature data including n (where n is a natural number) dimensional array of features included in the face image data in the form of a lookup table; and
Based on continuous learning, characterized in that it comprises an identification module for learning the face image and feature data for each ID stored in the data storage unit and identifying the ID stored in the data storage unit through the extracted data transmitted from the feature extraction unit. Door control system through face recognition.
[Formula 1]

where v _t is the characteristic data of the registered personnel, is a correction coefficient, and x represents face image data. delete The method of claim 1,
The identification module identifies data after compensating for losses such as Equations 2 and 3 below when identifying data.

[Equation 2]

here, is the loss function, E _x is the expected value, x is the input face image, and _pi represents the Softmax Function of the ith ID when there are IDs from 1 to L.
[Formula 3]

Here, v is the feature data of the registered person, T is the transposition vector, τ is the temperature, Q is the maximum allowable data capacity for the unregistered person, and u is the feature data of the unregistered person. The method of claim 1,
The data storage unit updates the internal data with new data using Equation 4 below when it is impossible to store additional data inside the door control system through continuous learning-based face recognition.
[Formula 4]

Here, W _j and W _j are the cluster face image data closest to the face image detected by the image detector among the face image data stored in the data storage unit, and C _i and C _j are the faces contained in the i-th and j-th position buffers. Indicates the number of images.

Images