KR101990823B1 - Method, program, and apparatus for estimating location and managing identification of target using facial information - Google Patents

Abstract

According to an embodiment of the present invention, a method for location and identity management using face information comprises: capturing an image of a subject using a PTZ camera; Obtaining face information including the region of interest and coordinates of the region of interest from the image; Measuring a depth value of the image using the area of interest and a zoom value of the PTZ camera; Estimating a position of an object using the ROI and the depth value; Comparing the face information with previously stored personal information to obtain current human information including current name of the object, current recognition score, and current position information associated with the estimated position; Calling cumulative human information including a previous name of the object, a previous recognition score, and previous position information; Updating the current human information to cumulative human information if the distance between the current position and the previous position of the object is less than or equal to a threshold and the current recognition score of the object is higher than the previous recognition score; And storing the updated accumulated human information if the distance between the current position and the previous position of the object exceeds a threshold value or the current recognition score of the object is lower than the previous recognition score, By keeping human information intact, depth of object can be measured using PTZ camera without depth measurement sensor, and reliability of the identity management system can be improved.

Description

Technical Field [0001] The present invention relates to a method, a program, and an apparatus for estimating a position using face information,

[0001] The present invention relates to a method, a program, and an apparatus for location estimation and identification using face information, and more particularly to a method and apparatus for estimating a current position of an object using a size and a coordinate of a region of interest of the captured image, Program, and apparatus for managing identity by comparing and updating information and cumulative information.

Techniques for continuously capturing and tracking moving objects using a photographing device are widely used in security and surveillance systems. In addition, a system for identifying the identity of the photographed object by verifying the image of the photographed object against the stored information in the database is widely used for security, surveillance, crime investigation, etc. Recently, Techniques for simultaneously managing the identities of a large number of objects as well as a single object have been used in various fields due to the improvement. Thus, the importance of improving the reliability of the face detection / recognition system and the technique of correctly identifying the identities of the different objects photographed at the same time are emerging.

Conventionally, a method of acquiring and analyzing an image of a predetermined area using a plurality of fixed camera devices has been used to simultaneously identify and manage the identities of a plurality of objects. In recent years, however, a single PTZ (Pan-Tilt-Zoom) A method of detecting a target by acquiring and analyzing an image of a desired region by adjusting a photographing angle of the camera at a remote place is used.

Since PTZ cameras can pan, tilt, and zoom, it is possible to capture images at various positions with a single camera. Therefore, compared to installing a plurality of fixed cameras, It is inexpensive, and maintenance and repair are easy.

When using the PTZ camera, it is possible to secure a wide monitoring angle and to manage the identity by capturing images of a plurality of objects with one camera. However, when a low-priced PTZ camera having no sensor for measuring the depth value is used, Dimensional positional information of the object can not be obtained, so that it is difficult to estimate the exact position of the object.

In addition, in a technique of recognizing a face of a target by comparing the photographed image with information in a database, there is a case in which an object moves during face recognition or actually different persons are recognized as the same face due to similarity of faces with other objects There is a problem that the reliability of the face recognition result is lowered.

U.S. Patent No. 20160180147 A1

In this specification, the depth value of the camera and the object is measured in consideration of the area-of-interest information of the image taken by the PTZ camera and the zoom value of the PTZ camera, and the depth value and the area- And a technique for estimating a position of an object, specifically, a position in three-dimensional coordinates

Furthermore, the present invention aims to improve the reliability of the identity management system by updating the latest location information and the recognition score continuously under cumulative conditions.

Also, in identifying a plurality of objects, it is possible to provide a technology for preventing a problem that different objects are recognized as the same person due to a technical problem such as a low image quality of a low-cost camera, And to improve the reliability of the apparatus.

According to an aspect of the present invention, there is provided a method of position estimation and management using face information, comprising: capturing an image of a target using a PTZ camera; Obtaining face information including the region of interest and coordinates of the region of interest from the image; Measuring a depth value of the image using the area of interest and a zoom value of the PTZ camera; And estimating a position of the object using the coordinates of the area of interest and the depth value.

In one embodiment, the step of estimating the position of the object using the ROI and the depth value may further comprise the steps of: calculating, using a zoom value of the PTZ camera and a depth value of the image, Calculating a distance; Calculating three-dimensional local coordinates using the actual distance per pixel and the region of interest coordinates; And calculating three-dimensional global coordinates using the local coordinates and a pan value and a tilt value of the PTZ camera.

Comparing the face information with previously stored personal information to obtain current human information including current name of the object, current recognition score, and current position information associated with the estimated position; Calling cumulative human information including a previous name of the object, a previous recognition score, and previous position information; Updating the current human information to cumulative human information if the distance between the current position and the previous position of the object is less than or equal to a threshold and the current recognition score of the object is higher than the previous recognition score; And storing the updated accumulated human information if the distance between the current position and the previous position of the object exceeds a threshold value or the current recognition score of the object is lower than the previous recognition score, Human information can be kept intact.

In one embodiment, the step of capturing an image of an object using the PTZ camera comprises capturing a plurality of images for a predetermined time, wherein obtaining the current human information comprises: Acquiring a current recognition score with respect to the current position; Sequentially matching names for each of the plurality of images based on the respective current recognition scores; And determining the most frequent name among the plurality of matched names as the current name of the object.

In one embodiment, the sequential name matching step may include determining the same name as the current name of the object if the same name is continuously matched a predetermined number of times.

According to another aspect of the present invention, there is provided a method of position estimation and identification using face information, comprising: capturing an image of a first object and a second object using a PTZ camera; Obtaining first face information including an area of interest region and a region of interest region for the first object from the image of the first object; Obtaining second face information including an area of interest region and an area of interest coordinates for a second object from the image of the second object; Comparing the first face information with previously stored personal information to obtain first recognition information including a name and a recognition score of the first object; Comparing the second face information with previously stored personal information to obtain second recognition information including a name and a recognition score of the second object; And comparing the recognition score of the first object and the recognition score of the second object to delete the recognition information of the object having the lower recognition score if the name of the first object and the name of the second object are the same .

The present invention can be provided in a computer program for realizing a position estimation and identity management method using face information according to an aspect of the present disclosure.

According to an aspect of the present invention, there is provided an apparatus for position estimation and identification using face information, including: a PTZ camera for capturing an image of an object; A face information acquisition module for acquiring face information including an area of interest region and a region of interest coordinates from the image; A depth measurement module for measuring a depth value of the image using the area of interest and a zoom value of the PTZ camera; And a position estimation module for estimating a position of the object using the coordinates of the area of interest and the depth value.

In one embodiment, the position estimation module comprises: a first sub-module for calculating an actual distance per pixel in the image using a zoom value of the PTZ camera and a depth value of the image; A second sub-module for calculating a three-dimensional local coordinate using the actual distance per pixel and the region of interest coordinates; And a third submodule for calculating the 3D global coordinates using the local coordinates and the pan value and tilt value of the PTZ camera.

In one embodiment, the human information acquisition module compares the face information with previously stored personal information to acquire current human information including the current name of the object, current recognition score, and current position information associated with the estimated position. Further comprising a human information update module for comparing and updating the current human information with cumulative human information, wherein the cumulative human information includes a previous name, a previous recognition score, and previous position information of the object, Module updates the current human information to cumulative human information if the distance between the current position and the previous position of the object is less than or equal to a threshold and the current recognition score of the object is higher than the previous recognition score, The accumulated human information can be maintained when the distance between the previous position and the previous position is equal to or greater than the threshold value or the current recognition score of the object is lower than the previous recognition score.

In one embodiment, the PTZ camera captures a plurality of images for a preset time, and the current human information acquisition module obtains a current recognition score for each of the plurality of images, The name of each of the plurality of images is sequentially matched, and the name having the highest frequency among the plurality of matched names can be determined as the current name of the object.

In one embodiment, the human information acquisition module may determine the same name as the current name of the object when successively matching the same name consecutively a predetermined number of times when successively matching the names. According to another aspect of the present invention, there is provided an apparatus for position estimation and identification using face information, comprising: a PTZ camera for capturing an image of a first object and a second object; A first face information obtaining module for obtaining first face information including an area of interest area and a region of interest area for the first object from the image of the first object; A second face information acquisition module for acquiring second face information including an area of interest area and an area of interest area for the second object from the image of the second object; A first recognition information acquiring module for acquiring first recognition information including a first name and a recognition score of the first object by comparing the first face information with previously stored personal information; A second recognition information acquiring module for acquiring second recognition information including a name and a recognition score of the second object by comparing the second face information with previously stored personal information; And comparing the recognition score of the first object and the recognition score of the second object when the name of the first object and the name of the second object are identical to each other to recognize recognition information of an object having a lower recognition score Information deletion module.

According to an embodiment of the present invention, three-dimensional position information of an object can be obtained based on two-dimensional image information captured by a camera. According to this technique, since a low-cost camera without a depth sensor can be utilized like an RGB-D camera, not only is it useful for locating information on a single object, but also when locating information on a plurality of objects in a specific place It is useful because you can know the location of each object.

According to the embodiment provided herein, the reliability of the identity management system can be improved by continuously updating the location information of the object based on the recognition score of the object, and according to another embodiment, The reliability of the identity management system can be improved by preventing different objects from being recognized as the same person due to the similarity of faces between a plurality of objects.

1 is a flowchart illustrating a method of estimating a position using face information according to an exemplary embodiment of the present invention.
FIG. 2 is a flowchart specifically illustrating a step of estimating a position of FIG. 1 according to an embodiment.
3 is a flowchart illustrating a method of location estimation and identification using face information according to an exemplary embodiment.
4 is a flowchart illustrating a method of managing an identity using face information according to another embodiment.
FIG. 5A is a diagram illustrating shooting an image of a plurality of objects using a PTZ camera in one embodiment.
FIG. 5B is a diagram illustrating a method of determining a name of an object when the names of the first object and the second object are identically specified in one embodiment.
5C is a diagram illustrating updating cumulative human information according to an embodiment.
FIG. 6 is a block diagram illustrating the components of a location estimation and identity management apparatus using face information according to an exemplary embodiment of the present invention.
FIG. 7 is a block diagram specifically illustrating the position estimation module of FIG. 6 according to one embodiment.
8 is a block diagram showing each component of the identity management apparatus using face information according to another embodiment.

BRIEF DESCRIPTION OF THE DRAWINGS The above and other features and advantages of the present invention will be more clearly understood from the following detailed description taken in conjunction with the accompanying drawings, in which: FIG.

As used herein, terms used in the present specification are selected from the general terms that are currently widely used, while taking into consideration the functions, but these may vary depending on the intention or custom of the artisan or the emergence of new techniques. Also, in certain cases, there may be a term selected by the applicant at will, in which case the meaning will be described in the description part of the corresponding specification. Therefore, it is intended that the terminology used herein should be interpreted based on the meaning of the term rather than on the name of the term, and on the entire contents of the specification.

In addition, the embodiments described herein may be wholly hardware, partially hardware, partially software, or entirely software. As used herein, the terms "unit," "module," "device," "server," or "system" Refers to a computer-related entity. For example, a component, a module, a device, a server, or a system may refer to software such as an application for driving the hardware and / or the hardware constituting part or all of the platform.

Location estimation method using face information

Hereinafter, a method for measuring the distance to the object using the PTZ camera and obtaining the three-dimensional position information of the object using the distance value will be described in detail.

FIG. 1 is a flowchart illustrating a position estimation method using face information according to an exemplary embodiment of the present invention. Referring to FIG. 1, a step of photographing an object using a PTZ camera is performed (S10).

In this specification, a PTZ (Pan-Tilt-Zoom) camera refers to all kinds of cameras capable of horizontal pan, tilt, zoom, and the like. But are not limited to, general PTZ cameras capable of controlling the angle.

The term " object " refers to all the single or plural objects that can be photographed by the PTZ camera, and refers to a face of a person who needs identification in an identity management method described later. The image capturing method of the object may include both photographing or moving picture photographing.

In one embodiment, the PTZ camera can photograph at least one image including a plurality of objects, and performs a process of simultaneously recognizing the faces of a plurality of objects included in the image, The identity of the objects can be identified faster than the method.

Next, the step of acquiring face information including the region of interest and coordinates of the region of interest from the image of the object is performed (S20). A region of interest refers to an independent object region that is separated from the background in the entire captured image. For example, in a landscape photograph, a region including a tree, a car, or a person may be an area of interest. In the position estimation and identity management method described later, a rectangular area including the entire face of the person object may be set as a region of interest, as shown in FIG. 5B. The region of interest may be set automatically using known detection / recognition functions, or may be set arbitrarily by the user.

In one embodiment, the region of interest coordinates are two-dimensional coordinate information indicating the center point of the set ROI. For example, referring to FIG. 5C, the ROI coordinates of the withdrawal are (-200, -100) and the ROI coordinates are (200, 100). Also, the size of the ROI of the withdrawal is 100, the size of the ROI of the ROI is 50, and the size of the face region of each object is shown.

The face information includes at least the ROI information and the ROI information, and according to an embodiment, additional information necessary for identifying the object by comparing the RGB color information or the contrast information of the image with a previously stored photograph .

Next, a step of measuring the depth value of the image using the ROI information and the zoom value of the PTZ camera is performed (S30). In general, when the size of the region of interest is smaller than the reference, it can be determined that the object is farther than the reference distance, and when the object is larger than the reference distance, It can be judged that it is near. Considering further the zoom value of the PTZ camera, it can be determined that the object is farther away as the zoom value at the time of photographing is larger, even if the area of interest is the same.

Therefore, the depth value of the object, that is, the distance value (d) between the PTZ camera and the object can be calculated by the following formula.

d = W0 × zoom / W (unit: meter)

Where W is the size of the region of interest of the subject, zoom is the zoom value of the PTZ camera, and W0 is the size of the reference region of interest when d = 1 meter, zoom = 1.

Next, a step of estimating the position of the object is performed using the depth value and the ROI coordinate information (S40). Specifically, the three-dimensional position information of the object can be obtained based on the two-dimensional coordinate information of the ROI obtained in step S20 and the depth value information of the ROI measured in step S30.

In one embodiment, the location estimation method may include the following detailed steps. Referring to FIG. 2, the actual distance per pixel in the image is calculated using the zoom value of the PTZ camera and the depth value of the image (S41).

In one embodiment, the actual distance p per pixel in the image may be calculated by the following formula.

p = p0 x d / zoom (unit: meter)

Where zoom is the zoom value of the PTZ camera and p0 is the reference distance per pixel when d = 1 meter and zoom = 1. In the embodiment, the p0 value can be determined according to the sensor size and the focal distance of the PTZ camera.

Next, a step of calculating three-dimensional local coordinates based on the actual distance p per pixel, the region of interest d, and the region of interest is performed (S42), and the computed local coordinates and the pan of the PTZ camera ) Value, and a tilt value are calculated (S43).

In the embodiment, when the region of interest coordinates is (i, j), the three-dimensional local coordinate Xl and the three-dimensional global coordinate Xg can be calculated by the following equations respectively.

Xl  = (p x i, p x j, d)

Xg  = T · Xl  + b

Here, T represents an Euler 3x3 matrix reflecting the pan and tilt values of the PTZ camera, and b represents an offset of the global coordinate system and the local coordinate system.

According to the above embodiments, it is possible to measure the depth value of an image even with a low-cost PTZ camera without a separate depth measurement sensor, and to acquire three-dimensional position information of the object from the two- And it is said that commercial use is high.

How to manage identity by updating cumulative human information

Hereinafter, a method for improving the reliability of the identity management system by examining a specific condition and updating the latest location information and the recognition score of the object continuously to cumulative information will be described.

3 is a flowchart illustrating a method of location estimation and identification using face information according to an exemplary embodiment. First, each of the steps S10 to S40 of the above-described position estimation method with reference to Figs. 1 and 2 is performed. As described above, it is noted that step S40 may further include detailed steps S41 to S43 according to the embodiment as described above, and redundant description will be omitted.

Referring to FIG. 3, the current human information is obtained by comparing the face information with the previously stored personal information (S50). As described above, the face information includes at least the area of interest area and the interest area coordinate information, and further includes additional information necessary for identifying the object by comparing the RGB color information and the contrast information of the image with the previously stored photograph can do. The prestored personal information may include photograph data of the target candidate, name information matching with the photograph of each candidate, and the like, and may be stored in a memory or a server in the identity management device.

Specifically, in the face recognition process, the photographed face information is compared with the photographed data of a plurality of target candidates stored in advance (for example, the photographed data of the first and second grade students) to measure the degree of similarity with each photographed data For example, the photographed image of the photographed image is similar to 70% of the photographed image data, and the photographed image data of the photographed image is 60% similar), and the current recognition score is determined based on the similarity (for example, And 70%), and if the recognition score is equal to or greater than the threshold value, the stored name (for example, withdrawal) matched with the photograph can be determined as the current name of the target. Here, the threshold value of the recognition score may be changed according to the initial registration situation and the recognition environment.

In the face recognition process, one photographed image is compared with pre-stored personal information to determine the current name of the subject and the current recognition score, but in one embodiment, multiple images may be used to improve the confidence of the face recognition process have.

In an embodiment utilizing a plurality of images, the step S10 of photographing an object using the PTZ camera includes photographing a plurality of images for a preset time. For example, if an image of a subject is shot for 3 seconds using a PTZ camera with a shooting interval of 0.1 second, 30 images can be taken. The photographing interval and the photographing time of the camera can be arbitrarily set by the user.

In this embodiment, the step of obtaining current human information (S50) includes the steps of acquiring a current recognition score for each of the plurality of images, sequentially acquiring the name of each of the plurality of images based on the respective current recognition scores And determining the most frequent name among the plurality of matched names as the current name of the object.

For example, it is possible to measure the degree of similarity by comparing each of the photographed 30 images with the image data (for example, photograph data of the third grade first and second grade students) of the previously stored candidate in advance, Determine the current recognition score. If the recognition score is greater than or equal to the threshold, determine the name that matches the picture with the current name of each image (for example, 20 of the 30 images are in the pullout, 8 are in the image, Lt; / RTI > Finally, the most frequent name is determined by the majority of the matched plural names as the current name of the object (in the above example, it is determined by the number of withdrawals).

According to another embodiment, when the same name is continuously matched a predetermined number of times during the sequential name matching step, it may further include determining the same name as the current name of the target . For example, if you have a series of 'pull-out' matches for images 1 through 10 of 30 images, the target's name will be determined as 'pull-out' and the next step The time and resources required for the operation can be saved.

In one embodiment, when collating the photograph data of a plurality of candidate candidates, the time required for the calculation can be reduced by preferentially collating the candidate photograph data of the specified seat closest to the three-dimensional global coordinate.

Hereinafter, a method for updating human information by comparing the obtained information with previously acquired information will be described in detail.

The current human information including the current name of the object determined through the process, the current recognition score, and the current position information of the object obtained in step S40 is obtained. Then, the step of loading accumulated human information including the previous name of the object, the previous recognition score, and the previous position information is performed (S60). The accumulated human information is stored in a memory or a server in the identity management apparatus as human information (i.e., including name, recognition score, and position information stored before new shooting) obtained through the same process before the step S10 .

Next, the current human information is compared with the accumulated human information to update the human information (S70). First, the current position information of the object is compared with the previous position information to determine whether the Euclidean distance between the current position Xg and the previous position Y is within a threshold value as follows.

| Xg (i) - Y (j) | < T

If the distance is less than or equal to the threshold value, the step of comparing the current recognition score and the previous recognition score is performed. If the current recognition score is higher than the previous recognition score, the current human information is updated to the accumulated human information. That is, the updated accumulated human information includes the current name instead of the previous name, the current recognition score instead of the previous recognition score, and the current position information instead of the previous position information. The accumulated human information is stored in a memory or a server and, when the location estimation and identity management process is repeated again, loaded as cumulative human information and utilized in comparison with new current human information.

5C, the face information of the first object 100 is compared with the previously stored personal information. As a result, the face information of the current human information 1 (current name: FIRST, current recognition score: 750, (1, 1, 1)), it is compared with the cumulative human information 1 (former name: Younghee, previous recognition score: 700, previous position: (1, 1, 1)). Since the distance between the two positions is less than the threshold value because the previous position and the current position are equal to (1, 1, 1), the recognition score is compared, and since the current recognition score is higher than the previous recognition score of 700, The name, the recognition score, and the location information are all updated with the current information.

If the distance between the current position and the previous position exceeds the threshold value or is less than the threshold value but the current recognition score is lower than the previous recognition score as a result of the recognition score comparison, the accumulated human information is maintained without updating.

In one embodiment, the threshold between positions is determined by the time it takes for the process to repeat (i.e., the time it takes to perform the step S10 of imaging the subject again after the update or maintenance of cumulative human information (S70) ). The shorter the time taken to repeat the process, the more reliable the tracking of the position change of the object, so the threshold value can be set low.

Conversely, the longer the time it takes to repeat the process, the lower the confidence in tracking the change in position of the object, so that the threshold is set higher and the position can be continuously tracked even when the object moves away from the previous position. By using the above-described identity management method, the reliability of the identity management process can be improved by continuously updating the location and cumulative information of the object.

In one embodiment, if the face of the object to be tracked is completely obscured by another object, the threshold may be set higher to allow tracking to continue when the obscured face reappears, thereby improving the reliability of the identity management process.

How to manage your identity if the target's name is the same

Hereinafter, techniques for preventing identification of a plurality of objects by identifying a different object based on a technical problem such as a low image quality of a low-cost camera, a motion of an object, or a similarity of a face are provided.

4 is a flowchart illustrating a method for managing an identity using face information according to an exemplary embodiment. First, a step of photographing an image of a first object and a second object using a PTZ camera is performed (S100). Description of the general PTZ camera and description of the object to be imaged have been described above, and a duplicate description will be omitted.

Referring to FIG. 5A, objects included in a certain range are photographed using a PTZ camera. The PTZ camera can perform the face recognition process on all the objects included in the shooting range, but only the first object and the second object will be described below for clarity of explanation.

Next, the step of acquiring first face information including the region of interest and the region of interest for the first object from the image of the first object is performed (S200) The second face information including the region-of-interest size and the region-of-interest coordinates for the second object is obtained (S300). The description of the size and coordinates of the region of interest is as described above, and redundant description is omitted.

Next, the first face information is compared with the previously stored personal information to acquire first recognition information including the name and the recognition score of the first object (S400), and the second face information And acquiring second recognition information including the name and the recognition score of the second object by comparing the previously stored personal information with the previously stored personal information (S500).

The first and second face information include the size of the region of interest and the coordinates of the region of interest of the first and second objects, respectively, and are compared with pre-stored photographs such as RGB color information and contrast information of each image, And may further include additional information necessary for identification. The prestored personal information may include photograph data of the first and second candidates (for example, photograph data of the third and first grade students), name information matching each photograph of the candidates, and the like. It can be stored in a memory or server in the apparatus as described above.

The step of comparing the first face information and the second face information with previously stored personal information and acquiring respective recognition information may be performed according to the face recognition process described above. For example, a region of interest is set for each of the first object and the second object in the photographed image, and the region of interest is compared with the photograpy data of the third and first grade students in order to determine the recognition score and the matching name.

As described above, in order to improve the reliability of the face recognition process according to an exemplary embodiment, a plurality of images photographed for a predetermined time period may be used. The recognition score and name information are matched for each image, By determining the name, the reliability of the face recognition process can be improved.

If the name of the first object and the name of the second object are the same, a method of managing the identity information is performed (S600). Specifically, it is first determined whether the name of the first object and the name of the second object are the same. If the names are different from each other, it is not a logical problem, so that the obtained recognition information is stored and the process is terminated.

If the name of the first object and the name of the second object are the same, the recognition score of each object is compared. If the recognition score of the first object is higher, the recognition information of the second object is deleted, The recognition information of the first object is deleted. If the recognition scores are equal to each other, the storage of the recognition information is suspended and the operation is performed again from the imaging step (S100) of the object.

5B, the face recognition process is performed on the first object 100 in the photographed image to obtain recognition information 1 (name: withdrawal, recognition score: 750 points), and the second The recognition result 2 (name: withdrawal, recognition score: 750 points) was obtained as a result of performing the face recognition process on the object 200. The recognition result of the object area 200 (-200, -100) Since the region of interest coordinates (200, 100) are different, it was decided to have the same name "withdrawal" despite being different objects present at different locations.

In this case, even if the recognition score for each object is more than the threshold value, since the recognition result for one person is incorrect, it is possible to compare the recognition scores of the first object and the second object, Keeps the name information "withdraw" for the first object, and deletes the recognition information (name and recognition score) for the second object having the lower recognition score (700 points).

In a specific embodiment, the step of updating the accumulated human information may be further performed. If the accumulated human information of the first object is compared with the accumulated human information of the second object, if the same name exists, And the human information with low recognition score can be deleted. For example, if the first object recognition result is the "withdrawal (750 points)" and the second object recognition result is the "withdrawal (700 points)", the recognition information related to the "withdrawal" of the cumulative human information of the second object is deleted.

According to the embodiment, the second object whose recognition information has been deleted may be returned to step S100 to repeat the same process. According to the described identity management method, in the process of simultaneously identifying the identities of a plurality of objects, it is possible to prevent a problem in which different objects are recognized as the same person due to a technical problem such as low image quality of a low-cost camera, The reliability can be improved.

Such a method of location estimation and identification using face information can be implemented in the form of program instructions that can be implemented in an application or through various computer components, and the program can be recorded on a computer-readable recording medium have. The computer-readable recording medium may include program instructions, data files, data structures, and the like, alone or in combination. Program instructions that are recorded on a computer-readable recording medium may be those that are specially designed and constructed for the present invention and are known and available to those skilled in the art of computer software.

Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks and magnetic tape, optical recording media such as CD-ROMs and DVDs, magneto-optical media such as floptical disks, media, and hardware devices specifically configured to store and execute program instructions such as ROM, RAM, flash memory, and the like. Examples of program instructions include machine language code such as those generated by a compiler, as well as high-level language code that can be executed by a computer using an interpreter or the like. A hardware device may be configured to operate as one or more software modules to perform processing in accordance with the present invention, and vice versa.

Location estimation and identity management device

Hereinafter, an apparatus for estimating a position of an object by measuring depth values of a camera and an object using a PTZ camera and acquiring three-dimensional position information of the object, a face recognition unit for comparing face information with stored data, The device, and the device for managing the identity when different objects are recognized as the same person will be described with reference to the drawings.

FIG. 6 is a block diagram illustrating the components of a location estimation and identity management apparatus using face information according to an exemplary embodiment of the present invention.

6, the position and identity management apparatus 1 using face information includes a PTZ camera 10 for capturing an image of a subject, a face information acquisition module 20, a depth measurement module 30, An estimation module 40, a human information acquisition module 50, and a human information update module 60.

The Pan-Tilt-Zoom (PTZ) camera 10 refers to all kinds of cameras capable of horizontally rotating, tilting, tilting, and zooming. But is not limited to, a conventional PTZ camera that can control the camera.

The face information acquisition module 20 can be used to acquire face information including the region of interest region and coordinates of the region of interest from the photographed image. As described above, the region of interest refers to an independent object region separated from the background in the entire captured image, and in an embodiment, a rectangular region including the entire face of the person object may be set as the region of interest . The face information includes at least the ROI information and the ROI information, and according to an embodiment, additional information necessary for identifying the object by comparing the RGB color information, contrast information, .

In one embodiment, the PTZ camera 10 can capture at least one image that includes a plurality of objects, and by performing a process of simultaneously recognizing faces of a plurality of objects included in the image, It is possible to identify the objects more quickly than the face recognition method.

The depth measurement module 30 may be used to measure the depth value of the image using the area of interest and the zoom value of the PTZ camera. The detailed description of the method for measuring the depth value of the object by comparing the size of the region of interest and the reference size of the photographed image in consideration of the zoom value is as described above, and thus, the overlapping description will be omitted.

The position estimation module 40 may be used to estimate the position of the object using the ROI and the depth value. Referring to FIG. 7, the position estimation module 40 includes a first submodule for calculating an actual distance per pixel in the image, using a zoom value of the PTZ camera and a depth value of the image 41), a second sub-module (42) for calculating three-dimensional local coordinates using the actual distance per pixel and the area coordinates of interest, and a second sub module (42) for calculating a pan value, a tilt Module 43 for calculating the three-dimensional global coordinates using the values of the three-dimensional global coordinates.

The detailed description of the method for calculating the three-dimensional local coordinates and the global coordinates from the depth values using the submodules 41 to 43 is as described above, and thus a duplicated description will be omitted.

The human information acquisition module 50 compares the face information with previously stored personal information to obtain current human information including the current name of the object, the current recognition score, and current position information related to the estimated position .

As described above, the face information includes at least the area of interest area and the interest area coordinate information, and further includes additional information necessary for identifying the object by comparing the RGB color information and the contrast information of the image with the previously stored photograph can do. The prestored personal information may include photographic data of the target candidate, name information that matches the photograph of each candidate, etc., and may be stored in a memory (not shown) or server in the location estimation and identity management device 1 have.

Specifically, the human information acquisition module 50 collates the photographed face information with photographed data of a plurality of candidate candidates stored in advance, measures the degree of similarity with each photographic data, determines the current recognized score based on the degree of similarity, If the recognition score is equal to or greater than the threshold value, the stored name (for example, the withdrawal number) matching with the photograph can be determined as the current name of the object as described above.

In one embodiment, the location estimation and identity management device 1 may use multiple images to improve the reliability of the face recognition process. In this case, the PTZ camera 10 captures a plurality of images for a predetermined time, and the human information acquisition module 50 acquires a current recognition score for each of a plurality of images, It is possible to improve the reliability of the face recognition process by sequentially matching names with respect to each of the images and determining the most frequent name among the plurality of matched names as the current name of the object.

Alternatively, if the same name is continuously matched for a predetermined number of times when the names are sequentially matched, the human information acquisition module 50 determines the same name as the current name of the object, Resources can be saved.

The human information update module 60 may be used to compare and update the current human information with accumulated human information including the previous name, previous recognition score, and previous position information of the object. Specifically, when the distance between the current position and the previous position of the object is equal to or less than a threshold value and the current recognition score of the object is higher than the previous recognition score, the current human information is updated to cumulative human information, If the distance between the previous position and the previous position is equal to or greater than a threshold value or the current recognition score of the object is lower than the previous recognition score, the accumulated human information is maintained.

As described above, the threshold value between the positions can be set to be lower as the time required for the repetition of the process is shorter, and can be set higher because the longer the time taken to repeat the process, the lower the reliability of tracking the position change of the object . By using the above-described identity management method, the reliability of the identity management process can be improved by continuously updating the location and cumulative information of the object.

In one embodiment, if the face of the object to be tracked is completely obscured by another object, the threshold may be set higher to allow tracking to continue when the obscured face reappears, thereby improving the reliability of the identity management process.

Referring to Fig. 8, there is shown a simplified block diagram of an identity management device 2 using facial information according to yet another embodiment, which device 2 comprises a device for capturing an image of a first object and a second object The first face information acquisition module 22, the second face information acquisition module 24, the first recognition information acquisition module 32, the second recognition information acquisition module 34, the recognition information deletion module 34, (45).

The first and second face information acquisition modules 22 and 24 may acquire first and second face information from the images of the first and second objects photographed, And face information, respectively.

The first and second recognition information acquisition modules 32 and 34 compare the previously stored personal information to acquire the first and second recognition information each including the name and the recognition score of the first and second objects . The detailed description of the method for acquiring the name and the recognition score of the object by comparing the face information with the data stored in advance is as described above, and a duplicate description will be omitted.

The recognition information deletion module 45 compares the name of the first object and the determined name of the second object to compare the recognition score of the first object and the recognition score of the second object, The recognition information of the object having the lower recognition score is deleted. For the deleted object, the face recognition process is repeated again and a new name is determined.

In a specific embodiment, the recognition information deletion module 45 performs a step of updating the accumulated human information. When the accumulated human information of the first object is compared with the accumulated human information of the second object, if the same name exists, Can keep the larger human information and delete human information with low recognition score. For example, if the first object recognition result is the "withdrawal (750 points)" and the second object recognition result is the "withdrawal (700 points)", the human information related to the "withdrawal" of the cumulative human information of the second object is deleted.

According to the described identity management method, in the process of simultaneously identifying the identities of a plurality of objects, it is possible to prevent a problem in which different objects are recognized as the same person due to a technical problem such as low image quality of a low-cost camera, The reliability can be improved.

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it is to be understood that the invention is not limited to the disclosed exemplary embodiments or constructions, It will be understood that the invention can be variously modified and changed without departing from the spirit and scope of the invention.

1: Position estimation and identity management device
2: Identity management device
10, 12: PTZ camera
20: Face information acquisition module
30: Depth measurement module
40: Position estimation module
41: first sub-module
42: second submodule
43: third sub-module
50: Human information acquisition module
60: Human information update module
22: first face information acquisition module
24: second face information acquisition module
32: First recognition information acquisition module
34: second recognition information acquisition module
45: recognition information deletion module
100: First target
200: Second target

Claims (13)

A method for managing an identity using face information,
Capturing an image of a target using a PTZ camera;
Obtaining face information including the region of interest and coordinates of the region of interest from the image;
Measuring a depth value of the image using the area of interest and a zoom value of the PTZ camera;
Estimating a position of an object using the coordinates of the region of interest and the depth value;
Comparing the face information with previously stored personal information to obtain current human information including current name of the object, current recognition score or current position information associated with the estimated position;
Retrieving cumulative human information including a previous name of the object, a previous recognition score, or previous position information;
Updating the current human information to cumulative human information if the distance between the current position and the previous position of the object is less than or equal to a threshold and the current recognition score of the object is higher than the previous recognition score; And
Storing the updated accumulated human information,
Wherein if the distance between the current position and the previous position of the object exceeds a threshold value or if the current recognition score of the object is lower than the previous recognition score, the accumulated human information is maintained as it is How to manage.
The method according to claim 1,
Estimating a position of an object using the ROI coordinates and the depth value,
Calculating an actual distance per pixel in the image using a zoom value of the PTZ camera and a depth value of the image;
Calculating three-dimensional local coordinates using the actual distance per pixel and the region of interest coordinates; And
And calculating three-dimensional global coordinates using the local coordinates and a pan value and a tilt value of the PTZ camera.
delete The method according to claim 1,
The step of photographing an object using the PTZ camera comprises:
Taking a plurality of images for a predetermined time,
Wherein the step of acquiring the current human information comprises:
Obtaining a current recognition score for each of the plurality of images;
Sequentially matching names for each of the plurality of images based on the respective current recognition scores; And
And determining the most frequent name among the plurality of matched names as the current name of the object.
5. The method of claim 4,
Wherein the sequentially matching the names comprises:
And determining the same name as the current name of the object when the same name is continuously matched for a predetermined number of times.

A method for managing an identity using face information,
Capturing an image of a first object and a second object using a PTZ camera;
Obtaining first face information including an area of interest region and a region of interest region for the first object from the image of the first object;
Obtaining second face information including an area of interest region and an area of interest coordinates for a second object from the image of the second object;
Comparing the first face information with previously stored personal information to obtain first recognition information including a name and a recognition score of the first object;
Comparing the second face information with previously stored personal information to obtain second recognition information including a name and a recognition score of the second object; And
Comparing the recognition score of the first object with the recognition score of the second object and deleting the recognition information of the object having the lower recognition score if the name of the first object and the name of the second object are the same A method for managing identity using face information.
A computer program stored in a computer-readable recording medium for realizing a method for managing an identity using face information according to any one of claims 1, 2, and 4 to 6.
An apparatus for location estimation and identification using face information,
A PTZ camera for capturing an image of a target;
A face information acquisition module for acquiring face information including an area of interest region and a region of interest coordinates from the image;
A depth measurement module for measuring a depth value of the image using the area of interest and a zoom value of the PTZ camera;
A position estimation module for estimating a position of an object using the coordinates of the area of interest and the depth value;
A human information acquiring module for acquiring current human information including current name, current recognition score or current position information related to the estimated position by comparing the face information with previously stored personal information;
And a human information update module for comparing and updating the current human information with accumulated human information,
Wherein the accumulated human information includes a previous name of the object, a previous recognition score, or previous position information,
Wherein the human information update module updates the current human information to cumulative human information if the distance between the current position and the previous position of the object is equal to or less than a threshold value and the current recognition score of the object is higher than the previous recognition score,
Wherein when the distance between the current position and the previous position of the object exceeds a threshold value or the current recognition score of the object is lower than the previous recognition score, .
9. The method of claim 8,
The position estimation module includes:
A first sub-module for calculating an actual distance per pixel in the image, using a zoom value of the PTZ camera and a depth value of the image;
A second sub-module for calculating a three-dimensional local coordinate using the actual distance per pixel and the region of interest coordinates; And
And a third submodule for calculating the three-dimensional global coordinates by using the local coordinates and a pan value and a tilt value of the PTZ camera.
delete 9. The method of claim 8,
The PTZ camera captures a plurality of images for a predetermined time,
Wherein the current human information acquisition module acquires a current recognition score for each of the plurality of images,
And sequentially matching names for each of the plurality of images based on the respective current recognition scores,
And determines the most frequent name among a plurality of matched names as the current name of the object.
12. The method of claim 11,
Wherein the human information acquisition module determines the same name as the current name of the object when the same name is continuously matched for a predetermined number of times in sequential name matching, Device.
An apparatus for location estimation and identification using face information,
A PTZ camera for capturing an image of the first object and the second object;
A first face information obtaining module for obtaining first face information including an area of interest area and a region of interest area for the first object from the image of the first object;
A second face information acquisition module for acquiring second face information including an area of interest area and an area of interest area for the second object from the image of the second object;
A first recognition information acquiring module for acquiring first recognition information including a first name and a recognition score of the first object by comparing the first face information with previously stored personal information;
A second recognition information acquiring module for acquiring second recognition information including a name and a recognition score of the second object by comparing the second face information with previously stored personal information; And
And comparing the recognition score of the first object and the recognition score of the second object when the name of the first object and the name of the second object are the same, And a deletion module.

Images