KR20190142553A - Tracking method and system using a database of a person's faces - Google Patents

Abstract

The present invention relates to a method and system for tracking a person using a face database. According to the present invention, a method for allowing a server to track a person comprises the steps of: transmitting a face image to a tracking device when the face image including a face of a person is received; constructing a face database of the person using the face image; receiving, from the tracking device, tracking information including a tracking image of the person acquired by the tracking device using the face image; extracting a face region from the tracking image and determining whether the face region matches any one of a plurality of images stored in the face database; and as a result of the determination, if the face region matches any one of the plurality of images, transmitting the tracking information to a control server or a control terminal. According to the present invention, the face database used for tracking the person is constructed, thereby identifying the face of the person to be found in a tracking process more accurately and with high reliability.

Description

Person tracking method and system using face database {TRACKING METHOD AND SYSTEM USING A DATABASE OF A PERSON'S FACES}

The present invention relates to a method and system for tracking a person using a face database, and more particularly, to a method and system for generating a face database using deep learning and tracking a person using the same.

The most common data source used to find a specific person, for example a wanted person, suspect or missing person, is CCTV video. The police may identify a person who is looking for the naked eye in the CCTV image or search for a person whose feature information is matched in the CCTV image using the feature information of the face image.

As a conventional technology for identifying a face using a surveillance camera, a real-time face recognition system and method for CCTV images of Korean Patent Application Laid-Open No. 10-2017-0119379 are disclosed. According to the conventional method, a specific person is recognized in real time. Even if the person exposed to the image is still in a moving state, the recognition of the person is difficult to lead directly to tracking.

In addition, conventionally, a face is recognized by comparing feature information of a face included in a picture taken from a specific direction and an image collected by a surveillance camera, and the face is included in the picture and the angle of the face exposed to the surveillance camera. Since it is difficult to match, there is a disadvantage that the recognition rate is lowered.

Accordingly, there is a need for a person tracking method and system capable of more accurately searching and tracking a person by improving the reliability and accuracy of face identification during person tracking.

An object of the present invention is to solve the above-described problem, and to provide a face database used for tracking a person, so that the face of the person to be searched in the tracking process can be identified with high accuracy and high reliability.

Another object of the present invention is to provide a method for overcoming spatial limitations of a surveillance camera by tracking a person using a drone rather than a surveillance camera.

Another object of the present invention is to provide a method for quickly and error-free tracking a person moving in a wide and unspecified space such as a suspect or a missing person.

.

According to an aspect of the present invention, a method for tracking a person in a server, when receiving a face image including a face of the person, transmitting the face image to a tracking device, by using the face image. Building a face database of a person, receiving tracking information including a tracking image of the person acquired by the tracking device using the face image from the tracking device, extracting a face region from the tracking image, Determining whether the face area matches any one of a plurality of images stored in the face database, and if the face area matches any one of the plurality of images, the tracking server or the server; Characterized in that it comprises the step of transmitting to the control terminal.

According to the present invention as described above, by building a face database used for the person tracking, it is possible to identify the face of the person to find in the person tracking process with more accurate and high reliability.

In addition, according to the present invention, by tracking a person using a tracking device such as a drone rather than a surveillance camera, it is possible to overcome the spatial constraints of the surveillance camera.

In addition, according to the present invention, it is possible to quickly track a person moving in a wide and unspecified space such as a suspect or a missing person without error.

1 is a view for explaining a person tracking system according to an embodiment of the present invention,
2 is a block diagram illustrating a configuration of a server according to an embodiment of the present invention;
3 is a block diagram illustrating a configuration of a face database generation unit according to an embodiment of the present invention;
4 is a view for explaining a method for generating a corrected image according to an embodiment of the present invention;
5 is a flowchart illustrating a preprocessing step according to an embodiment of the present invention;
6 is a flowchart illustrating a person tracking method according to an embodiment of the present invention;
7 is a flowchart illustrating a face database construction step according to an embodiment of the present invention.

The above objects, features, and advantages will be described in detail with reference to the accompanying drawings, whereby those skilled in the art may easily implement the technical idea of the present invention. In describing the present invention, when it is determined that the detailed description of the known technology related to the present invention may unnecessarily obscure the gist of the present invention, the detailed description will be omitted.

The same reference numerals in the drawings are used to indicate the same or similar components, all combinations described in the specification and claims may be combined in any way. And unless specified otherwise, reference to the singular may include one or more, and reference to the singular may also include the plural expression.

The terminology used herein is for the purpose of describing particular example embodiments only and is not intended to be limiting. Singular expressions as used herein may also be intended to include the plural meanings, unless the context clearly indicates otherwise. The term "and / or," "and / or" includes any and all combinations of the items listed therein. The terms "comprising", "comprising", "comprising", "comprising", "having", "having", and the like have implicit meanings, and as such, these terms are defined by their features, It specifies steps, actions, elements, and / or components and does not exclude the presence or addition of one or more other features, integers, steps, actions, elements, components, and / or groups thereof. The steps, processes, and operations of a method described herein should not be construed as necessarily in their particular order as discussed or illustrated, unless the order of execution is specifically determined. . It should also be understood that additional or alternative steps may be used.

In addition, each component may be implemented as a hardware processor, each of the above components may be integrated into a single hardware processor, or the above components may be combined with each other to be implemented as a plurality of hardware processors.

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings.

1 is a diagram illustrating a person tracking system according to an exemplary embodiment. Referring to FIG. 1, a person tracking system according to an embodiment of the present invention may include a server 100, a terminal 50, a tracking device 200, and a police system 300.

The server 100 is a kind of control server, and manages and controls overall steps for tracking (or searching for) a person in the person tracking system according to an embodiment of the present invention. The server 100 functions as a main function of processing a face image of a person to be tracked, and for this, a face database required for tracking a person may be constructed.

When the tracking device 200 receives the tracking request signal from the server, the tracking device 200 may acquire an arbitrary tracking image by using the tracking request signal and generate tracking information including the tracking image. The tracking request signal may include a face image of a person, and may further include additional information such as missing person's face image, missing person's impression, missing place, guardian's contact information, missing person's name and age.

The tracking device 200 includes a drone as shown in FIG. 1, a commonly used surveillance camera, a ground control station (GCS), and the like, and may be a terminal used by a police officer who patrols a site.

When the tracking device 200 is a drone with a camera attached, when the tracking device 200 finds a person or a target having characteristic information similar to the feature information included in the face image included in the tracking request signal, the tracking device 200 may have 5 ~ 5 from the ground. Approaching 10m over the air, a zoom-in camera can be used to capture a person or target to obtain a tracking image. The tracking device 200 generates tracking information including at least one of a tracking image, a time at which the tracking image is captured, and location information (GPS information) at which the tracking image is captured, and transmits the generated tracking information to the server 100. Can be.

In another embodiment, the tracking device 200 may acquire a tracking image by capturing a person who has similar impression wear or wanders around the missing place by using information such as the impression wear or disappearance of the missing person included in the tracking request signal. It may be.

The tracking image acquired by the tracking device 200 may be transmitted to the server 100 to be used to determine whether a person included in the tracking image is a tracking target.

The police system 300 may be understood to mean a server or a terminal managed by an organization performing a person tracking such as a police.

2, the server 100 may include a communication unit 130, a database generation unit 150, a determination unit 170, and a notification unit 190.

The communication unit 130 is a communication module that transmits and receives data with an external device through a wired or wireless network. When the communication unit 130 receives a tracking request signal including a face image of a person, the communication unit 130 transmits a tracking request signal to the tracking device and receives tracking information from the tracking device. can do.

The terminal 50 may be a missing person report transmitted to the server 100. For example, the tracking request signal may include a face image of the missing person, an impression of the missing person, a missing person, guardian information such as a contact number of the missing person, additional information such as the name and age of the missing person, and the like.

The database generator 150 may build a face database of the person using the face image. The face database is a database that stores the generated correction image when the database generation unit 150 processes a face image of a specific person and generates a correction image that can be used for tracking a person. Can be.

For example, suppose that the server 100 receives (a) as the face image of the person A and (b) as the face image of the person B as shown in FIG. 4. The database 150 may reconstruct a person A's face database and a person B's face database by recovering an image having a low resolution, noise, or corrupted image, such as (a) or (b).

In the example of FIG. 4, since the database generator 150 cannot identify a person because the resolution of (a) received as the face image of person A is too low, the face database of person A is corrected as shown in (a3). In the case of (b) received as the face image of the person B, since some areas are damaged, the face database of the person B may be generated by recovering the damaged area as shown in (b1).

More specifically, referring to FIG. 3, the database generator 150 may include a preprocessor 153, a multi-angle image generator 155, and a storage 157.

The preprocessor 153 may remove noise and correct the resolution of the face image using the generated allele neural network, and generate a corrected image by restoring the damaged area if the damaged area exists in the face image.

Genetic allele neural networks are called GAN (Generative Adversial Network), also called hostile generating neural network. The generative allele pair neural network can generate an image similar to the original image while simultaneously training two opposing neural networks (named as a classification model and a generation model, respectively).

The preprocessor 153 may generate a corrected image by removing noise of a received face image, correcting a resolution, and recovering a damaged area by using the generated allele neural network. In the above-described example of FIG. 4, the preprocessor 153 preprocesses the received face images (a) and (b), respectively, to generate correction images (a3) and (b1).

The generational pair-pair neural network may be previously un-learned with a plurality of real face images without noise and images added with noise in order to preprocess the face image.

For example, the preprocessor 153 trains a pre-generated allele neural network to process the received face image. Specifically, referring to FIG. 5, the preprocessor 153 includes a plurality of noises in which no noise exists. In operation S10, a first training data set including an actual face image may be generated, and a second training data set may be generated as an image in which noise is added to an area of the actual face image (S20). Next, the second training dataset is input to the generational model of the generational allele neural network to generate an image from which the noise is removed (S30), and the noise is removed from the classification model of the generational allele neural network. The classification result may be obtained by inputting the received image and the first learning data set (S40). The preprocessor 153 may determine whether the accuracy of the classification result is greater than or equal to a preset reference value or less than the reference value (S50).

If the determination result in step 50, the accuracy of the classification result is less than the predetermined reference value may be reflected to the generation model to optimize the generation model (S60). Optimization of the generation model means that the weight values of each layer constituting the neural network in machine learning are adjusted to reflect the classification results.

The preprocessor 153 may learn the generation model and the classification model by repeating steps 30 to 60 until the accuracy of the classification result becomes equal to or greater than the reference value. When the learning of the generated allele neural network is completed, the preprocessor 153 may acquire the corrected image by applying the face image to the generated allele of the neural paired network.

The multi-angle image generator 155 rotates the corrected image by a predetermined angle in left, right, up, and down directions by applying a correction image to the multi-angle image generation model trained using the plurality of multi-angle face learning images and the front learning image. A plurality of angled face images can be generated. The multi-angle image generation model may also be a generative allele neural network model. In particular, the multi-angle image generation model may use the InfoGAN algorithm.

The multi-angle face learning image and the face learning image are training data for training (learning) the multi-angle image generating model. The face learning image is a face photographed from the front, and the multi-face face learning image is based on the front of the face. The image may be taken at a predetermined angle in the up and down directions. For example, the multi-angle face learning image may be an image obtained by photographing a person included in the front learning image in 5, 10, and 15 degrees in left, right, up, and down directions, in this case, The image generation model may include one or more machine learning models trained independently for each angle by using the training image and the front learning image acquired at a specific angle.

That is, the multi-angle image generation model is a machine learning model that is independently trained for each angle, such as a machine learning model trained with a plurality of learning images acquired at left 5 degrees, and a machine learning model trained with a plurality of learning images acquired at left 10 degrees. It can contain several.

The multi-angle image generation unit 155 unsupervises the multi-angle image generation model using a plurality of front learning images and a multi-angle face learning image, and applies a correction image to the multi-angle image generation model on which the learning is completed. Right, a plurality of multi-angle face images rotated at a predetermined angle in the up, down direction can be generated. As described above, when the multi-angle image generation model is a machine learning model trained using learning images acquired in 5, 10, and 15 degrees in the left, right, up, and down directions, the multi-angle image generation unit ( When 155 applies the correction image to the multi-angle image generation model, a total of 36 multi-angle face images may be generated. An example of the multi-angle face image generated by the multi-angle image generator 155 using the face image 1 received from the terminal 50 in FIG. 1 is as shown in (2) -2 of FIG. 1.

The storage unit 157 may store the multi-angle face image generated by the angle in the face database of the person to be tracked. That is, in the above-described embodiment, the face database of the person X corresponding to the face image of (1) includes a correction image (front face) in which resolution and noise are corrected, and is rotated at different angles and angles. A total of 36 face images may be stored in the face database of the person X to be tracked.

The determination unit 170 may extract a face region from the tracking image, and determine whether the face region corresponds to any one of a plurality of images stored in the face database. Referring to FIG. 1, the tracking image is an actual image 3 obtained by the tracking device 200 through a camera, etc. The determination unit 170 extracts the face region 4 from the actual image 3, and the face It may be determined whether the area 4 corresponds to any one of the plurality of images 2 stored in the face database, including the correction image and the angled face image.

As a result of the determination, when the face region corresponds to any one of the plurality of images, the notification unit 190 may transmit the tracking information to the police system 300, that is, the police server or the terminal.

Hereinafter, a person tracking method according to an embodiment of the present invention will be described with reference to FIG. 6. Referring to FIG. 6, when the server receives a tracking request signal including a face image of a person (S100), the server may transmit a tracking request signal to a tracking device (S200). The server may build a face database of the person using the face image (S300), and the tracking device may acquire a tracking image using the tracking request signal received in step 200 (S400). The tracking device generates tracking information including the tracking image (S500) and transmits the generated tracking information to the server (S600).

When the server receives the tracking information from the tracking device (S600), the server may extract a face region from the tracking image (S700). The server compares the extracted face area with the face database (S800). It may be determined whether the face region matches any one of the plurality of images stored in the face database (S900). If it is determined in operation 900 that the face region matches any one of the plurality of images, the server may transmit tracking information to the police system (police server or terminal) (S1000).

Meanwhile, in operation 300, the server may remove noise and correct the resolution of the face image using the generated allele neural network, and if the damaged area exists in the face image, the server may restore the damaged area and generate a corrected image (S310). . In addition, a plurality of multi-angle face images in which the correction image is rotated at predetermined angles in the left, right, up, and down directions by applying the correction image to the multi-angle image generation model trained using the plurality of multi-angle face learning images and the face learning image are provided. Can be generated (S330). The server may build the face database of the person by storing the corrected image and the angled face image in the face database of the person (S350).

In operation 310, the server generates a training dataset for learning the generated allele neural network used for preprocessing of the face image, and uses the same to train the generated allele neural network, which is described in detail with reference to FIG. 5. Since there is one, duplicate description is omitted.

Embodiments of the present invention disclosed in the specification and drawings are only specific examples to easily explain the technical contents of the present invention and aid the understanding of the present invention, and are not intended to limit the scope of the present invention. It will be apparent to those skilled in the art that other modifications based on the technical idea of the present invention can be carried out in addition to the embodiments disclosed herein.

Claims (7)

In how the server tracks people,
When the tracking request signal including the face image of the person is received, transmitting the tracking request signal to a tracking device;
Constructing a face database of the person using the face image;
Receiving tracking information including a tracking image acquired by the tracking device using the tracking request signal from the tracking device;
Extracting a face region from the tracking image and determining whether the face region matches any one of a plurality of images stored in the face database;
And transmitting the tracking information to a police system if the face region matches any one of the plurality of images as a result of the determination.
The method of claim 1,
Building the face database of the person
A preprocessing step of removing noise and correcting a resolution of the face image using a generated allele neural network, and generating a corrected image by recovering the damaged area if a damaged area exists in the face image;
A plurality of multi-angle face images in which the correction image is rotated by a predetermined angle in left, right, up, and down directions by applying the correction image to a multi-angle image generation model trained using a plurality of multi-angle face learning images and a front learning image. Generating a;
And storing the corrected image and the multi-angle face image in a face database of the person.
The method of claim 2,
The pretreatment step
A step of generating a first training data set consisting of a plurality of real face images in which no noise is present;
Generating a second training data set as an image in which noise is added to one region of the real face image;
C) generating an image from which the noise is removed by inputting the second training dataset to the generation model of the generational pair-pair neural network;
D) obtaining a classification result by inputting the noise-free image and the first training dataset to a classification model of the generated allele neural network;
If the accuracy of the classification result is less than a preset reference value, reflecting the classification result to the generation model;
Learning the generation model and the classification model by repeating steps c through e until the classification result is greater than or equal to the reference value;
And obtaining the corrected image by applying the face image to the generated allele pair neural network in which the learning is completed.
The method of claim 2,
The multi-angle face learning image is an image obtained by photographing a person included in the front learning image in 5, 10, and 15 degrees in left, right, up, and down directions, and the multi-angle image generation model includes a specific angle. And at least one machine learning model trained independently for each angle by using the training image acquired at and the front learning image.
The method of claim 1,
The trace request signal is
And at least one of an impression wear of the person, an expected position of the person, personal information of the person, or a guardian contact number of the person.
The method of claim 1,
The tracking information is
And at least one of a time at which the tracking image is captured and position information at which the tracking image is captured.
In a system for tracking people,
A server for processing a face image of the person;
When receiving the face image from the server, using the face image comprises a tracking device for generating tracking information including a tracking image of the person,
The server is
A communication unit for receiving the face image including the face of the person, transmitting the face image to a tracking device and receiving the tracking information from the tracking device;
A database generator configured to build a face database of the person using the face image;
A determination unit which extracts a face region from the tracking image and determines whether the face region corresponds to one of a plurality of images stored in the face database;
And a notification unit for transmitting the tracking information to a police system when the face region corresponds to any one of the plurality of images as a result of the determination.

Images