JP5785667B1 - Person identification system - Google Patents

Abstract

A person identification system capable of accurately and quickly identifying the same person is provided. An image capturing unit for capturing a person's face, a camera-side storage unit for storing face image data obtained by capturing a person's face, a camera-side arithmetic processing unit, and the face image data as server type information A monitoring camera device having a camera-side transmission unit for transmission to the processing device, the camera device taking a photographed face image data and a predetermined face angle stored in advance in the camera-side storage unit The filter face image data for each range is collated to determine a predetermined face angle range corresponding to the face image data, and the determined face angle range is associated with the face image data as face angle range data In addition, the camera device performs facial recognition when the newly captured face image data and the face image data already captured and stored are associated with the same face angle range data. The face image data having a high degree is determined as the best shot in the associated face angle range and stored, and the best shot determination process is performed for each face angle range to delete the face image data not determined as the best shot, and then , Create the best shot total data that aggregates the face image data determined to be the best shot in the related face angle range and send it to the server type information processing device, and the server type information processing device receives the best shot total data Then, a person management database is created, and a person is identified based on the person management database. [Selection] Figure 12

Description

  The present invention relates to a person identification system in surveillance camera video.

  Conventionally, there has been a system in which visitors such as parks and amusement parks are photographed with a camera device, the photographed video data is transmitted to a server type information processing device, and the server type information processing device identifies a person by face authentication. (For example, refer to Patent Document 1).

JP 2011-5054 A

  However, even if the face image data of the same person is taken as a face image with a face angle different from that of the face image data stored in the server-type information processing device, it is determined as a different person when face authentication is performed. There was a problem that the person could not be identified. In addition, there are problems that a plurality of person identification data (person IDs) are given to a single person and accurate appearance history data cannot be obtained, and unnecessary duplicate data increases.

  Therefore, an object of the present invention is to provide a person specifying system that can specify the same person accurately and quickly.

  In order to achieve the above object, a person identification system according to the present invention includes a photographing unit for photographing a person's face, a camera-side storage unit for storing face image data obtained by photographing a person's face, and a camera side. A monitoring camera device having an arithmetic processing unit and a camera-side transmission unit for transmitting the face image data to the server type information processing device, the camera device having the captured face image data; The predetermined face angle range corresponding to the face image data is determined together with the filter face image data for each predetermined face angle range stored in advance in the camera-side storage unit, and the determined predetermined face A face angle range determination process is performed for associating the angle range with the face image data as face angle range data, and the camera device has already captured and stored the newly captured face image data. When the face image data is associated with the same face angle range data, the face image data with a high degree of face recognition is determined as the best shot in the associated face angle range and stored, and the best shot Best shot total data obtained by performing best shot determination processing for each face angle range for deleting the face image data that has not been determined as, and then totaling the face image data determined as the best shot in the associated face angle range Is generated and transmitted to the server type information processing apparatus, and the server type information processing apparatus receives the best shot total data to create a person management database, and identifies a person based on the person management database. System.

Further, the camera device performs a data transmission process for specifying a person that transmits the face image data that is associated with the face angle range data and determined as the best shot to the server type information processing apparatus as the data for specifying the person. the server type information processing apparatus, of the person management database stored in the pre-Me server-side storage unit, the face image similar the face angle range data and said received for customer identification data is associated with This is a system for reading out data and collating it with the face image data of the received person specifying data to specify a person.
Further, the server type information processing apparatus compares the face image data associated with the face angle range data whose face angle ranges are close to each other in the received best shot total data, and determines that they are not the same person. This is a system for performing the same person confirmation processing for deleting the face image data from the best shot total data.
In addition, the camera device is a system that associates the best shot total data, captured date / time data, and captured camera identification data of the camera device with each other and transmits them to the server type information processing device.

The server-type information processing device compares the face image data transmitted from the camera device with the face image data in the person management database, and when the same person determination is made, The system displays the face image data that is the best shot for each face angle range associated with the face image data in the person management database and the date and time data on the server side display unit.
The face angle range data includes first face angle range data obtained by dividing a predetermined shooting angle range into five with an angle between the camera optical axis of the shooting unit and the left and right center line of the person's face as a face angle; The second face angle range data, the third face angle range data, the fourth face angle range data, and the fifth face angle range data.
Further, the face angle range data means that the predetermined shooting angle range is −75 degrees or more and 75 degrees or less, and the first face angle range data means a range where the face angle is −75 degrees or more and less than −45 degrees. The second face angle range data means a range where the face angle is −45 degrees or more and −15 degrees or less, and the third face angle range data is a range where the face angle exceeds −15 degrees and is less than 15 degrees. The fourth face angle range data means a range where the face angle is 15 degrees or more and 45 degrees or less, and the fifth face angle range data is a range where the face angle exceeds 45 degrees and is 75 degrees or less. Is a system that means
In addition, during the tracking process for tracking a person, the camera device uses the face image data that is associated with the face angle range data and determined to be the best shot as tracking confirmation data to the server-type information processing device. The server type information processing apparatus performs the tracking confirmation data transmission process to be transmitted, and the server-type information processing apparatus transmits the latest tracking confirmation data transmitted by the camera apparatus during tracking and the past information transmitted by the camera apparatus during tracking. In this system, the tracking confirmation data is collated, and if they are not the same person, a tracking processing end command is transmitted to the camera device.

  ADVANTAGE OF THE INVENTION According to this invention, it can reduce that the face image data of the same person is determined as another person, and a person can be specified correctly and rapidly. It is possible to prevent duplicate registration in which a plurality of person identification data is given to one person, to obtain accurate appearance history data, and to prevent an increase in unnecessary duplicate data. In addition, since the best shot determination is performed by the camera device, it is possible to reduce the processing load of the server type information processing device, and to make tens of camera devices correspond to one server type information processing device. In other words, since the best shot determination is performed with an inexpensive built-in microcomputer (camera-side arithmetic processing unit) of the camera device, it is not necessary to increase the number of expensive and high-performance server-type information processing devices. Can be reduced.

1 is an overall configuration diagram showing an embodiment of the present invention. It is a block diagram which shows one Embodiment of this invention. It is explanatory drawing for demonstrating an example of a face angle. It is a block diagram for demonstrating an example of a face angle range determination process. It is a block diagram for demonstrating an example of the best shot determination process for every face angle range. It is a block diagram for demonstrating an example of the data for person specification. It is a block diagram for demonstrating an example of best shot total data. It is a block diagram for demonstrating an example of the data for person management. It is a block diagram for demonstrating an example of the same person confirmation process. It is a block diagram for demonstrating an example of a person specific process. It is a front view which shows an example of the display of a server side display part. It is a flowchart figure for demonstrating an example of a process of a camera apparatus. It is a flowchart for demonstrating an example of the same person confirmation process of a server type information processing apparatus. It is a flowchart for demonstrating an example of the person specific process of a server type information processing apparatus. It is a flowchart for demonstrating an example of a tracking person confirmation process.

Hereinafter, the present invention will be described in detail based on illustrated embodiments.
As shown in FIGS. 1 and 2, the person identification system of the present invention includes a plurality of monitoring camera devices 1 and a management server connected to the camera device 1 in a wired or wireless manner so that data (information) can be transmitted and received. And a type information processing device 3 (hereinafter also referred to as a server device 3), and a system for identifying a person existing around the camera device 1.

The plurality of camera devices 1 and the server device 3 are connected by wire or wirelessly via a LAN line, a communication line network T such as the Internet or a higher management line network, or the router 2. Further, the camera device 1 and the server device 3 may have (built in) the router 2.
In particular, if all of the plurality of camera devices 1 are wired (LAN connection) to the server device 3 and the router (hub) 2, a power outlet for each camera device 1 becomes unnecessary by PoE power supply and can be installed in various places. .

The camera device 1 is provided in a store such as a luxury clothing store or a jewelry store, an entrance / exit gate such as an amusement park or a theme park, a station premises, a playground such as a pachinko store or a slot store, or the like.
The server device 3 is provided in a supervisor room such as a staff room, a guard room, a monitoring room, or a crisis management center.

  As shown in FIGS. 1 and 2, a camera device 1 includes a photographing unit 15 such as a CCD camera or a CMOS camera, a camera-side arithmetic processing unit 12 such as a microcomputer or a CPU, and a RAM or flash memory for storing data. Camera side storage unit 11 such as camera side transmission unit 18 capable of transmitting data stored in camera side storage unit 11 to the outside (external devices such as server device 3 and router 2), and external (server device 3 etc.) And a camera-side receiving unit 19 capable of receiving data such as a command signal from. In FIG. 1 and FIG. 2, the camera side transmission unit 18 and the camera side reception unit 19 are integrally illustrated as a camera side transmission / reception unit.

  The server device 3 includes a server-side receiving unit 39 that can receive data from the camera device 1, a server-side arithmetic processing unit 32 such as a CPU for performing predetermined processing, and a RAM, ROM, and HD (hard disk) that store data. ) Server-side storage unit 31, server-side display unit 33 such as a liquid crystal monitor that can display data in the server-side storage unit 31, and server-side input such as a keyboard and mouse for instructing data search and display commands Server-type personal computer comprising: a unit 34; a server-side output unit 35 such as a printer for outputting data search results to paper; and a server-side transmission unit 38 capable of transmitting data (data files) to the outside An information processing apparatus such as a (server PC) or a server type workstation. In FIG. 1 and FIG. 2, the server-side transmission unit 38 and the server-side reception unit 39 are integrally illustrated as an on-site management transmission / reception unit.

First, functions of the camera device 1 (processing executed by the camera side arithmetic processing unit 12 and data stored in the camera side storage unit 11) will be described.
The camera device 1 performs a person detection process for detecting whether or not a person (a person to be monitored) exists based on a human silhouette.
In the human detection process, when a human silhouette is detected, it is determined that a person is present. In this way, the processing speed is improved by detecting the shape of whether or not a person exists based on the human silhouette. In the human detection process, a person may be detected by monitoring the temperature distribution.

Further, the face of the detected person is extracted and a face image is taken, and face extraction processing for creating face image data A as shown in FIG. 4 is performed.
In the face extraction process, for example, a person's face is extracted by recognizing both eyes, nose, mouth, contour, and the like, and face image (face still image) data A is created.

In addition, it is possible to perform tracking processing for tracking a person detected.
In the tracking process, after the person is detected, the part where the person exists is compared with the part where the person does not exist, the subsequent change is detected, and the person's movement vector (predicted traveling direction) is calculated to track the person. .
In addition, it is preferable that the person is tracked by following the color of the clothes that the person is wearing together with the movement vector because it is possible to prevent misidentification of the tracking and to perform tracking more accurately.
In addition, the camera device 1 estimates that the face image data A captured while being properly tracked is the same person, and associates the same person identification data (person ID) with the face image data A.

  As shown in FIG. 4, when the camera device 1 creates the face image data A, the camera device 1 creates a predetermined face angle range of the captured face image data A and filter data J stored in the camera-side storage unit 11 in advance. Each filter face image data I is collated, and a predetermined face angle range corresponding to the face image data A is determined. Further, face angle range determination processing is performed for creating face data C associated with the face image data A using the determined face angle range as face angle range data B.

  The filter data J includes a plurality (five in the illustrated example) of filter face image data I (I1 to I5) and a plurality of face angle range data B (B1 to B5) meaning the face angle range of the filter face image data I. ), And the face angle range data B is associated with the face angle range of each filter face image data I.

  In the face angle range determination process, the face image data A created in the face extraction process is compared (compared) to which of the plurality of filter face image data I associated with the face angle range data B is closest. The filter face image data I having the most matching feature points is determined, the face angle range data B associated with the determined filter face image data I is confirmed (extracted), and the confirmed face angle range data B and face extraction are confirmed. The face data C is created by associating the face image data A created by the processing.

Here, as shown in FIG. 3, as an example of the face angle range data B, the angle between the camera optical axis L1 of the photographing unit 15 of the camera device 1 and the human face left-right center line La is the face angle. A range of the face angle θ is set as θ.
The filter face image data I is reference (standard) face image data in a predetermined face angle range created by totaling and analyzing sample face image data obtained by photographing thousands or more persons at various face angles θ. .
Note that the face angle θ is not limited to the angle when the face is turned left and right (left and right face angle) as described above, for example, the angle when the face is turned up and down (up and down face angle), or tilting the neck It is also possible to use an angle such as a tilted face angle (an inclined face angle) or a combination of these.
The face angle range data B may be set as appropriate using the various face orientation angles as the face angle θ.

The face angle range data B is five face angle range data B1, B2, B3, B4, and B5 obtained by dividing a predetermined shooting angle range (θa ≦ θ ≦ θf) suitable for face recognition (face authentication) into five. is there.
The minimum face angle is a first face angle θa, an angle larger than the first face angle θa is a second face angle θb, an angle larger than the second face angle θb is a third face angle θc, and a third face angle θc Is larger than the fourth face angle θd, the fifth face angle θe is larger than the fourth face angle θd, and the sixth face angle θf is larger than the fifth face angle θe (the maximum face angle is 6th face angle θf), first face angle range data B1 not less than first face angle θa and less than second face angle θb, and second face angle range data B2 not less than second face angle θb and not more than third face angle θc The third face angle range data B3 exceeding the third face angle θc and less than the fourth face angle θd, the fourth face angle range data B4 not less than the fourth face angle θd and not more than the fifth face angle θe, and the fifth face The fifth face angle range data B5 exceeding the angle θe and not more than the sixth face angle θf.

  More specifically, the face angle range data B is obtained when the person faces the photographing unit 15 (the camera optical axis L1 and the face left-right center line La are parallel or coincide with each other) and the face angle θ Is set to 0 degree, and the left and right ones at that time are on the negative side, and the other on the left and right sides are on the positive side. Face angle range data B1, B2, B3, B4, and B5, and the first face angle range data B1 means a range in which the face angle θ is not less than −75 degrees and less than −45 degrees, and the second face angle range data B2 means a range where the face angle θ is −45 degrees or more and −15 degrees or less, and the third face angle range data B3 means a range where the face angle θ exceeds −15 degrees and less than 15 degrees, and the fourth face angle The range data B4 means a range in which the face angle θ is 15 degrees or greater and 45 degrees or less, and the fifth face angle range. Data B5 is to mean 75 degrees or less in a range beyond 45 degrees face angle theta, is set.

  Further, as shown in FIG. 5, the camera device 1 has a newly captured face image data A and face image data A of the face data C that has already been captured and stored in the camera-side storage unit 11. When the identification data is the same and associated with the same face angle range data B (the newly created face data C and the face data C already created and stored are the same person identification data and face angle range data B), the face image data A having a high degree of face recognition is determined as the best shot in the associated face angle range and stored, and the face data C not determined as the best shot Best shot determination processing is performed for each face angle range from which (face image data A) is deleted.

  Here, the face recognition degree is a degree obtained by quantifying the face-likeness. For example, the respective face-to-eye, nose and mouth placement balance values are calculated, and the face-likeness reference value stored in the camera-side storage unit 11 in advance is calculated. Are compared and the face recognition degree (face recognition score) is increased as the face closer to the reference value. Also, the face recognition program (software) increases the degree of face recognition for data that is easy to perform face authentication processing for calculating the estimated age and sex (data that is less likely to cause errors).

  Further, as shown in FIG. 6, the camera device 1 captures the face image data A determined as the best shot in the associated face angle range, the face angle range data B associated with the face image data A, and shooting. The person identification data H associated with the camera identification data E of the camera apparatus 1 and the person identification data (not shown) is created, and the person identification data transmission process for transmitting to the server apparatus 3 is performed.

  The camera identification data E may be any information that can be used to identify the camera device 1 such as the name, installation location, and identification symbol of the camera device 1. In the present invention, the “symbol” of the identification symbol includes letters and numbers.

  Further, the camera apparatus 1 uses the face data C having the same person identification data, as shown in FIG. 7, the face image data A determined as the best shot in the associated face angle range, and the face image data Best shot total data creation processing for creating best shot total data D obtained by totaling the face angle range data B associated with A is performed. “Creation” for creating the best shot total data D includes data storage processing, update processing, addition processing, and deletion processing.

  The best shot total data D is data obtained by totaling a plurality of types of face image data A (best shot data) to which the same person identification data is assigned and different face angle range data B is associated. Is the best shot in the predetermined face angle range corresponding to the first face image data A1 and the second face angle range data B2, which is the best shot in the predetermined face angle range corresponding to the first face angle range data B1. Certain second face image data A2, third face image data A3 which is the best shot in a predetermined face angle range corresponding to third face angle range data B3, and predetermined corresponding to fourth face angle range data B4 The fourth face image data A4 that is the best shot in the face angle range and the fifth face that is the best shot in the predetermined face angle range corresponding to the fifth face angle range data B5 It is data and image data A5, the collectively.

Further, as shown in FIG. 8, the camera apparatus 1 creates person management data G in which the best shot total data D, the camera identification data E, and the captured date / time data F are associated with each other, and sends them to the server apparatus 3. Data transmission processing for person management to be transmitted is performed.
The date / time data may be data having one or both of time data and date (year / month / day) data. The date / time data is information from the date and time when the camera apparatus 1 detects a person and creates the face image data A to the date and time when tracking by the tracking process becomes impossible.

Next, the processing flow of the camera device 1 will be described.
In FIG. 12, when a person appears in front of the camera device 1 in the first step S1, the person is detected by the person detection process, and the process proceeds to the second step S2 to perform the face extraction process. Face image data A associated with the person identification data is created by the face extraction process, and the process proceeds to the third step S3, where the created face image data A is subjected to a face angle range determination process, and the person identification data and the face angle range data are processed. Face data C associated with B is created.

After creating the face data C, the process proceeds to the fourth step S4, and the best shot determination process for each face angle range is performed. The face image data A (first face image data A in the associated face angle range) of the first (first) face data C photographed is determined as the best shot.
If it is determined as the best shot in the fourth step S4, the process proceeds to the fifth step S5, and face data C (face image data A, face angle range data B, and person identification data) determined as the best shot (data), The camera identification data E is associated and sent to the server device 3 as person specifying data H.
If the best shot is determined in the fourth step S4, the process proceeds to a sixth step S6, and the best shot total data D is created for the face image data A associated with the same person identification data.
After the best shot total data D is created in the sixth step S6, the process proceeds to an eighth step S8, where tracking processing based on moving object detection is performed, and tracking determination processing is performed to determine whether tracking is being performed appropriately. The face image data A photographed during tracking by this tracking process is temporarily determined (estimated) to be the same person, and the same person identification data is associated.

If the tracking is properly performed, the process returns to the human detection process (first step S1) or the face extraction process (second step S2), and the second face image data A is generated during the tracking. In step S3, face angle range determination processing is performed to create second face data C.
After the third step S3, the process proceeds to the fourth step S4, and the second (newly created) face data C is included in the first face data C (already shot and stored and included in the best shot total data D). Is determined as the best shot data in the associated face angle range, the process proceeds to the fifth step S5, and the person specifying data transmission process is performed. Do. Further, the process proceeds to the sixth step S6, and is added to the best shot total data D (best shot total data creation process).
In the fourth step S4, if the second face data C has the same face angle range data B as the face data C already captured and stored and included in the best shot total data D The face image data A of the second face data C and the face image data A of the first face data C are compared to determine which is the best shot in the associated face angle range.
If it is determined that the face image data A of the second face data C is the best shot, the process proceeds to a fifth step S5 to perform a person specifying data transmission process. Further, the process proceeds to a sixth step S6, in which the face image data A of the first face data C is deleted from the best shot data D and updated to the second face image data A (best shot total data creation process).
If it is determined that the face image data A of the first face data C is the best shot data, the process proceeds to the seventh step S7, and the second face image data A (face data C) is deleted.

And after 6th step S6 or 7th step S7, it progresses to 8th step S8, and while it is determined that tracking of a person is performed appropriately by tracking determination processing (during tracking), human detection Returning to the processing (first step S1) or the face extraction processing (second step S2), the face angle range determination processing (third step S3), the best shot determination processing for each face angle range (fourth step S4), In the case of the best shot determination, person specifying data transmission processing (fifth step S5) is performed. Moreover, the best shot total data creation process (update process, addition process, deletion process) of 6th step S6 is performed, and it progresses to 8th step S8.
If it is determined in the best shot determination process (fourth step S4) that the shot is not the best shot, the process proceeds from the fourth step S4 to the seventh step S7, and the face image data A that has not been determined as the best shot is deleted. The process proceeds to the eighth step S8. Proceeding to the eighth step S8, the process described above is repeated (loop process) while it is determined in the tracking determination process that the person is being properly tracked (during tracking). During the tracking process, it is desirable to link with the server device 3 regularly (every predetermined time or every predetermined condition) in order to improve tracking reliability. For example, the latest face image data A of the person being tracked transmitted from the camera apparatus 1 to the server apparatus 3 and the past face image data A transmitted from the camera apparatus 1 to the server apparatus 3 during tracking and stored in the server apparatus 3 The server image 3 is collated with the face image data A to calculate a tracking reliability meaning the degree that the person being tracked is the same person, and the camera device 1 has a tracking reliability equal to or higher than a predetermined threshold value. It is desirable to determine the return destination of the loop process (whether to return to the human detection process or to return to the face extraction process) depending on whether it is (higher) or less than the threshold (lower). Note that it is also desirable to refer to the proximity of a moving object as an element of tracking reliability.

  In the eighth step S8, if the person is out of the shooting area of the camera device 1 or hidden behind the object and cannot be tracked, the process proceeds to the ninth step S9 to end the tracking, and then proceeds to the tenth step S10. The person management data G that associates the best shot total data D related to one person identification data, the camera identification data E, and the shooting date / time data F with each other is generated and transmitted to the server device 3. Perform data transmission processing. Then, the process returns to the first step S1.

Next, functions of the server device 3 (processing executed by the server-side arithmetic processing unit 32 and data stored in the server-side storage unit 31) and processing flow will be described.
As shown in FIGS. 9 and 13, when the person management data G transmitted from the camera device 1 is received in the eleventh step S11, the process proceeds to the twelfth step S12, and the best of the received person management data G is obtained. In the shot total data D, the face image data A whose face angle ranges are close to each other (adjacent) are compared, and the difference of feature points (probability of non-identical person) or the number of matching feature points (same person) (Probability probability) is calculated. The same person is possible depending on whether the obtained comparison result satisfies a predetermined determination condition (whether it is significantly different from other comparison results, whether it is greater than or equal to a predetermined threshold, or within a predetermined threshold range) The face image data A having low characteristics is extracted, and it is determined whether or not the extracted face image data A is the same person by referring to the comparison result obtained above. The face image data A determined not to be the same person (non-identical person) is deleted, and the same person confirmation process is performed to leave the face image data A determined to be the same person.

For example, as shown in FIG. 9, a comparison between the first face image data A1 and the second face image data A2, a comparison between the second face image data A2 and the third face image data A3, Comparison between the face image data A3 and the fourth face image data A4 and comparison between the fourth face image data A4 and the fifth face image data A5 are performed. If the comparison result between the first face image data A1 and the second face image data A2 determines that the predetermined determination condition is not satisfied, the first face image data A1 and the second face image data A2 are the same. Extracted because the possibility of a person is low. Therefore, since it is determined that the possibility of the same person is high when the comparison result between the second face image data A2 and the third face image data A3 is referred to (confirmed), the first face image data A1 is not the same person. And the second, third, fourth, and fifth face image data A2, A3, A4, and A5 are left.
By processing in this way, it is possible to prevent erroneous determination such that it is determined that non-identical persons are the same person.
Note that the face image data A1 determined not to be the same person may remain as deletion target data without being deleted.

In the same person confirmation process, the face image data A is compared with feature points such as face eyes, nose, mouth, contour shape and arrangement balance by face authentication.
Thus, by comparing the face image data A whose face angle ranges are close (adjacent) to each other, the face angle θ is close, so that accurate and highly accurate same person confirmation processing can be performed.

  When tracking a person with the camera device 1 by the same person confirmation process, a different person is tracked by mistake, such as when a person passes by a different person or when a different person passes between the camera apparatus 1 and the person being tracked. When face image data A of a different person is actually created even though the same person identification data is given, and when it is determined as best shot data and transmitted to the server device 3 as person management data G The best shot data (face image data A) of non-identical persons can be excluded from a plurality of types of best shot data (face image data A), and the accuracy (reliability) of person identification can be improved.

  The server device 3 performs the same person confirmation process on the person management data G for each person identification data received one after another. After the same person confirmation process, the process proceeds to the thirteenth step S13, where the person management database Z (see FIG. 10) is created and stored in the server-side storage unit 31 (database creation process). The “creation process” of the database creation process includes an update process, an addition process, a deletion process, and a storage process.

  Further, in the database creation process, the face image data A of the person management data G after the same person confirmation process and the face image data A of the person management database Z (see FIG. 10) stored in the server-side storage unit 31 in advance. If there is face image data A that is determined to be the same person by face authentication, face image data A with a high degree of face recognition is stored, and low data is deleted. That is, the best shot determination processing for each face angle range for the database is performed, and the person management database Z is updated and stored in the server-side storage unit 31 (update processing, deletion processing, and storage processing).

Further, in the database creation process, the face image data A of the person management data G after the same person confirmation process is compared with the face image data A of the person management database Z, and it is determined as the same person by face authentication. If there is no face image data A, the person management data G is added to the person management database Z as new person data and stored in the server-side storage unit 31 (addition processing and storage processing).
In association with the person identification data, new person identification data may be added again before and after the same person confirmation processing in the server device 3.
In the case of the person management data G and the best shot data update process, the person identification data that has been previously stored and associated with the face image data A is prioritized and left in the person management database Z.

Further, as shown in FIGS. 10 and 14, when the server device 3 receives the person specifying data H in the fourteenth step S14, the server apparatus 3 proceeds to the fifteenth step S15 and performs the face angle range of the person specifying data H. The data B is confirmed, and the face image data A associated with the same face angle range data B as the confirmed face angle range data B is read from the person management database Z stored in the server-side storage unit 31 in advance. The face image data A of the person specifying data H is collated with the face image data A (in the person management database Z) read from the person management database Z (extracting the face image data A in the thick line frame in FIG. 10). Then, it is determined whether or not they are the same person, and a person specifying process for specifying a person is performed.
In addition, in the person management database Z, the face image data A associated with the face angle range data B adjacent to the face angle range data B of the person specifying data H is collated to specify the person. May be.

  If the person can be identified, that is, if the person management data G has already been created by the camera device 1 and the face image data A is stored in the person management database Z of the server device 3, Proceeding to 16th step S16, as shown in FIG. 11, it is the best shot for each face angle range associated with the face image data A read (extracted) from the person management database Z determined to be the same person ( A specific result display process of displaying the face image data A, the date / time data F, and the camera identification data E on the server side display unit 33 is performed. Thereafter, the process returns to the 14th step S14.

  In the specific result display process, the server-side display unit 33 calculates the appearance history obtained by counting the date and time data F, the shooting area based on the camera identification data E of the camera device 1, person identification data (person ID), and face authentication. Display estimated age and estimated gender. In addition, when personal information is known from customer information such as a membership card or a point card, the personal identification data (person ID) may be displayed in association with the name, address, contact information, and work place.

  Further, the date / time data F (appearance history) and the camera identification data E (imaging area) may be totaled and analyzed to calculate the number of visits and purchases and display them. For example, the number of times taken by the camera device 1 beside the cash register may be displayed as the number of purchases. In this way, by displaying on the server-side display unit 33, the store clerk knows whether or not the customer (customer) is likely to purchase even if he / she does not remember the customer's face and purchase history. It is possible to provide detailed services such as escorts and introduction of new products without waste (no need to spend time on customers with little intention to purchase).

The display content can be printed out on paper from the server-side output unit 35 and output by performing a predetermined operation on the server-side input unit 34.
Further, by performing a predetermined operation on the server side input unit 34, it is possible to store the total purchase price information, the complaint information, and the like in the server side storage unit 31 in association with the person identification data.

  Furthermore, it is desirable to perform tracking person confirmation processing to determine whether or not the person being tracked by the camera apparatus 1 is the same person by connecting to the server apparatus 3 during the tracking process in which the camera apparatus 1 tracks a person.

  In the tracking person confirmation process, as shown in FIG. 15, after the camera apparatus 1 starts the tracking process for tracking a person (17th step S17), the camera apparatus 1 is created during the tracking process in the 18th step S18. The server apparatus 3 uses the best shot face image data A (in the associated face angle range), the face angle range data B, the camera identification data E, and the person identification data as tracking confirmation data in association with each other. (Tracking confirmation data transmission processing for creating tracking confirmation data and transmitting it to the server device 3 is performed).

The server device 3 stores the received tracking confirmation data in the tracking person management database.
Then, the server device 3 receives the latest tracking confirmation data (the face image data A) transmitted by the camera device 1 during tracking, and the same face angle data B and person identification data as the latest tracking confirmation data. Owned (estimated by the camera device 1 as the same person), and the camera device 1 has already transmitted during tracking and has been stored in the tracking person management database of the server device 3 (the face image data A) And the same tracking person determination process for determining whether or not they are the same person (19th step S19).

If the determination result of the tracking same person determination process is not the same person in the 20th step S20, the server apparatus 3 proceeds to the 21st step S21 and transmits a tracking process end command to the camera apparatus 1. The camera device 1 receives the tracking process end command and ends the tracking process (22nd step S22). Thereafter, person management data transmission processing (see the tenth step S10 in FIG. 12) is performed. Accordingly, it is possible to reduce the face image data A that is not the same person from being included in the person management data G.
If they are the same person, the camera apparatus 1 continues tracking without transmitting a tracking process stop command.

Although the present invention can be modified in design and is not shown in the drawing, the server apparatus 3 may be provided in a plural number instead of one in the management area. The server device 3 may be provided so as to be able to transmit / receive each data (information) to / from the server device 3 in another management area via the information communication network T. The face angle range data B may be data obtained by dividing the predetermined shooting angle range into four or six. The predetermined photographing face angle range may be set to minus 90 degrees or more and 90 degrees or less.
The numbers “first” and “11th” of the names of the steps, the symbols “S1”, “S11”, and the like are given for ease of explanation and mean the order of processing. It is not a thing.
In addition, the camera device 1 stores, in other words, it can be said that the camera-side arithmetic processing unit 12 stores it in the camera-side storage unit 11. In other words, the server apparatus 3 stores the server-side arithmetic processing unit 32 in the server-side storage unit 31.

  As described above, the person identifying system of the present invention includes the photographing unit 15 for photographing a person's face, the camera-side storage unit 11 for storing the face image data A obtained by photographing the person's face, and the camera side. A monitoring camera device 1 having an arithmetic processing unit 12 and a camera side transmission unit 18 for transmitting the face image data A to the server type information processing device 3 is provided. The data A and the filter face image data I for each predetermined face angle range stored in advance in the camera-side storage unit 11 are collated to determine a predetermined face angle range corresponding to the face image data A A face angle range determination process is performed for associating the determined predetermined face angle range with the face image data A as the face angle range data B, and the camera device 1 has already captured and stored the newly captured face image data A. Face image data A Are associated with the same face angle range data B, face image data A having a high degree of face recognition is determined as the best shot in the associated face angle range and stored, and is not determined as the best shot The best shot total processing D is performed for each face angle range from which the face image data A is deleted, and then the best shot total data D is calculated by totaling the face image data A determined as the best shot in the related face angle range. Since the server type information processing apparatus 3 receives the best shot total data D and creates the person management database Z and identifies the person based on the person management database Z, The face image data A of the same person can be reduced from being determined as another person, and the person can be specified accurately and quickly. It is possible to prevent duplicate registration in which a plurality of person identification data is given to one person, to obtain accurate appearance history data, and to prevent an increase in unnecessary duplicate data. Further, since the best shot determination is performed by the camera device 1, it is possible to reduce the processing load on the server type information processing device 3, and to make tens of camera devices 1 correspond to one server type information processing device 3. That is, the microcomputer (camera-side arithmetic processing unit 12) incorporated in the camera device 1 is simple and inexpensive for a specific function, and the server-type information processing device 3 is versatile, high-performance and expensive. However, in the present invention, the latter can be reduced and the system introduction cost can be reduced as a whole.

  In addition, the camera apparatus 1 performs a person specifying data transmission process in which the face image data A that is associated with the face angle range data B and determined as the best shot is transmitted to the server type information processing apparatus 3 as the person specifying data H. The server-type information processing device 3 performs the face image data associated with the same face angle range data B as the received person specifying data H in the person management database Z stored in the server-side storage unit 31 in advance. Since A is read out and collated with the received face image data A of the person specifying data H, the person is specified, so the face image data A of the person specifying data H and all the face images in the person management database Z Compared with the case of collating the data A, the efficiency is high and the time required for the determination can be greatly reduced. Even if the face image data A from the front cannot be photographed, accurate person identification processing can be performed.

  Further, the server type information processing apparatus 3 compares the face image data A associated with the face angle range data B whose face angle ranges are close to each other in the received best shot total data D, and determines that they are not the same person. Since the same person confirmation process is performed to delete the face image data A from the best shot total data D, while tracking the person with the camera device 1, different persons are tracked incorrectly, and the same person identification data is given. When face image data A of different persons is actually created, and when it is determined as best shot data and transmitted to the server device 3 as person management data G, the best shot data of non-identical persons can be excluded, The accuracy (reliability) of person identification can be improved. In particular, it is most suitable for a station premises or a large store where a person progresses in various directions, a person makes a U-turn while walking, or a person progresses so as to overtake the person.

  Further, since the camera device 1 associates the best shot total data D, the photographed date / time data F, and the camera identification data E of the photographed camera device 1 with each other and transmits them to the server type information processing device 3, there are various In addition to the face image data A at an angle, the appearance history of the person can be obtained, and data for analyzing whether a suspicious action is being performed, whether a regular customer, or a customer can be obtained.

  The server-type information processing device 3 compares the face image data A transmitted from the camera device 1 with the face image data A in the person management database Z, and when the same person determination is made, The face image data A that is the best shot for each face angle range associated with the face image data A in the person management database Z and the date / time data F are displayed on the server-side display unit 33, so The face image data A can be confirmed, and the store clerk and the monitor can confirm and discover the person in the store or the garden. The appearance history of the person can be known, and the action corresponding to the person can be performed, thereby reducing the wasteful customer service and behavior of the store clerk.

  Further, the face angle range data B is a first face angle obtained by dividing a predetermined shooting angle range into five with an angle between the camera optical axis L1 of the shooting unit 15 and the human face left-right center line La as the face angle θ. Since the range data B1, the second face angle range data B2, the third face angle range data B3, the fourth face angle range data B4, and the fifth face angle range data B5, unnecessary face image data A can be deleted, and data Can be prevented. Face image data A in various face angle ranges can be obtained, and person confirmation and specific accuracy can be improved. It is possible to prevent an increase in unnecessary data that is difficult to perform face authentication.

  The face angle range data B is a range in which the predetermined shooting angle range is −75 degrees to 75 degrees, and the first face angle range data B1 is a range in which the face angle θ is −75 degrees to less than −45 degrees. The second face angle range data B2 means a range where the face angle θ is −45 degrees or more and −15 degrees or less, and the third face angle range data B3 is a range where the face angle θ exceeds −15 degrees and less than 15 degrees. The fourth face angle range data B4 means a range where the face angle θ is 15 degrees or more and 45 degrees or less, and the fifth face angle range data B5 is a range where the face angle θ exceeds 45 degrees and is 75 degrees or less. Therefore, face authentication can be performed easily and quickly, and the same person confirmation and person identification accuracy (accuracy) can be improved. It is possible to prevent an increase in unnecessary data that is difficult to perform face authentication.

  Further, since the camera device 1 performs a person tracking process of capturing and tracking the color of a person's clothes, the tracking device can be quickly performed, and the camera device 1 can be manufactured using an inexpensive microcomputer. When using, installation costs can be greatly reduced.

  Further, during the tracking process for tracking a person, the camera apparatus 1 uses the face image data A that is associated with the face angle range data B and determined to be the best shot as tracking confirmation data to the server type information processing apparatus 3. The server type information processing device 3 performs processing for transmitting the tracking confirmation data to be transmitted, and the server-type information processing device 3 and the latest tracking confirmation data transmitted during tracking by the camera device 1 and the past tracking confirmation transmitted by the camera device 1 during tracking. If the same person is collated and the same person is not detected, a tracking process end command is transmitted to the camera apparatus 1, so that the face image data A of the non-identical person can be reduced from being mixed into the best shot total data D. .

1 Camera device 3 Server-type information processing device
11 Camera storage
12 Camera processing unit
15 Shooting section
18 Camera side transmitter
33 Server-side display A Face image data B Face angle range data B1 First face angle range data B2 Second face angle range data B3 Third face angle range data B4 Fourth face angle range data B5 Fifth face angle range data D Best shot total data E Camera identification data F Date and time data H Person identification data I Filter face image data L1 Camera optical axis La Face left and right center line Z Person management database θ Face angle

Claims (8)

A photographing unit (15) for photographing a person's face, a camera-side storage unit (11) for storing face image data (A) obtained by photographing the person's face, a camera-side arithmetic processing unit (12), A monitoring camera device (1) having a camera-side transmission unit (18) for transmitting the face image data (A) to the server-type information processing device (3),
The camera device (1) includes the captured face image data (A) and filter face image data (I) for each predetermined face angle range stored in advance in the camera-side storage unit (11). A face angle that is collated to determine a predetermined face angle range corresponding to the face image data (A) and associate the determined face angle range with the face image data (A) as face angle range data (B) Perform range judgment processing,
Further, the camera device (1) has the same face angle range data (B) in which the newly captured face image data (A) and the face image data (A) already captured and stored are the same. The face image data (A) having a high degree of face recognition is determined as the best shot in the associated face angle range and stored, and the face image data (not determined as the best shot) ( A) Best shot total data (D) obtained by performing the best shot determination processing for each face angle range to be deleted, and then totaling the face image data (A) determined as the best shot in the associated face angle range Create and send to the server type information processing device (3),
The server type information processing apparatus (3) receives the best shot total data (D), creates a person management database (Z), and specifies a person based on the person management database (Z). Character identification system. The camera device (1) uses the face type image data (A), which is associated with the face angle range data (B) and determined to be the best shot, as the person specifying data (H), and the server type information processing device ( 3) Perform data transmission processing for identifying the person to be transmitted to
The server information processing apparatus (3), among the above-mentioned person management database stored in the pre-Me server-side storage unit (31) (Z), the same the face and received the person identification data (H) The face image data (A) associated with the angle range data (B) is read out and collated with the face image data (A) of the received person specifying data (H) to specify a person. The person identification system according to claim 1.   The server-type information processing device (3) is configured so that the face image data (A) associated with the face angle range data (B) whose face angle ranges are close to each other in the received best shot total data (D). The person identification system according to claim 1 or 2, wherein the same person confirmation processing is performed to delete the face image data (A) determined not to be the same person from the best shot total data (D).   The camera device (1) associates the best-shot total data (D), the date / time data (F) taken, and the camera identification data (E) of the taken camera device (1) with the server type The person identification system according to claim 1, 2 or 3, which is transmitted to the information processing device (3).   The server type information processing device (3) collates the face image data (A) transmitted from the camera device (1) with the face image data (A) of the person management database (Z), The face image data (A) that is the best shot for each face angle range associated with the face image data (A) of the person management database (Z) that has been determined to be the same person when the same person is determined. The person specifying system according to claim 4, wherein the date and time data (F) is displayed on the server side display section (33).   The face angle range data (B) is obtained from a predetermined image taking an angle between the camera optical axis (L1) of the photographing unit (15) and the left and right center line (La) of the person as a face angle (θ). First face angle range data (B1), second face angle range data (B2), third face angle range data (B3), fourth face angle range data (B4) and fifth face angle obtained by dividing the angle range into five. 6. The person identifying system according to claim 1, 2, 3, 4 or 5, wherein the system is range data (B5).   In the face angle range data (B), the predetermined photographing angle range is −75 degrees or more and 75 degrees or less, and in the first face angle range data (B1), the face angle (θ) is −75 degrees or more− The second face angle range data (B2) means a range where the face angle (θ) is −45 degrees or more and −15 degrees or less, and the third face angle range data (B3) ) Means a range where the face angle (θ) is greater than −15 degrees and less than 15 degrees, and the fourth face angle range data (B4) is a range where the face angle (θ) is 15 degrees or greater and 45 degrees or less. The person identification system according to claim 6, wherein the fifth face angle range data (B5) means a range in which the face angle (θ) exceeds 45 degrees and is 75 degrees or less. During the tracking process for tracking a person, the camera device (1) uses the face image data (A) that is associated with the face angle range data (B) and determined as the best shot as tracking confirmation data. Performs tracking confirmation data transmission processing to be transmitted to the server type information processing apparatus (3),
The server type information processing device (3) includes the latest tracking confirmation data transmitted by the camera device (1) during tracking and the past tracking confirmation transmitted by the camera device (1) during tracking. The person identification system according to claim 1, wherein the tracking data end instruction is transmitted to the camera device (1) when the data is compared with each other and the two persons are not the same person. .

Images