JP2007179569A - Person recognizing device, person recognizing method, passage control device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a person recognizing device, person recognizing method, and passage control device capable of improving a person recognition rate by recognizing stably with high precision without being restrained by a posture, a standing position, and a direction of a face at recognition time. <P>SOLUTION: This person recognizing device recognizes by inputting a face image of a person to be recognized 100 by a camera1 101, and detecting a face from the input face image. Information relating to the face such as a size, direction, and position of the face in the face image input at registration time and information relating to the face such as the size, direction, and position of the face of the face image input at recognition time are simultaneously displayed on a display part 104, and thus, stable and highly precise recognition can be achieved without being retrained by the posture, standing position, and direction of the face. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to a person recognition apparatus that recognizes a person's face image in security management or the like, a person recognition method, and a traffic control apparatus that controls the passage of a passerby using the person recognition apparatus and the person recognition method.

  In this type of person recognition apparatus, for example, the person recognition rate is reduced when the state such as the standing position and posture of the person differs between registration and recognition (verification). In order to reduce the decrease in the person recognition rate, if the orientation of the person at the time of recognition is different from the orientation of the person registered in advance, the orientation of the photographing camera may be changed (see, for example, Patent Document 1) A method of providing a line-of-sight guidance device for guiding the line of sight on the top (for example, see Patent Document 2) has been adopted.

In addition, a method called non-contact collation has been adopted in which a recognition operation is not started by performing a predetermined operation, but a recognition operation is automatically started only by detecting a person's face.
Japanese Patent Application Laid-Open No. 11-316836 Japanese Patent Laid-Open No. 11-191856

  In such a person recognition apparatus using biometric matching, there is a problem that recognition accuracy is lowered due to variation in input information between registration and recognition. Therefore, it is important to reproduce the same state as at the time of registration as much as possible. Therefore, as disclosed in Patent Literature 1, when the orientation of the person at the time of recognition is different from the orientation of the person registered in advance, the orientation of the photographing camera is changed, or as disclosed in Patent Literature 2. In addition, a method of providing a line-of-sight guidance device for guiding the line of sight on the device has been proposed, but it is necessary to newly provide a control circuit and a line-of-sight guidance device for controlling the orientation of the photographing camera.

  Moreover, since there is no appropriate guidance at the time of registration and it is not known what operation should be performed during a predetermined time, the effectiveness of the dictionary can be increased by learning various states. On the other hand, there is a problem that a dictionary is created from only one pattern image.

  Also, if a function that automatically starts a recognition operation at the same time that a face is detected is used, or if there is a printed matter with a human face photograph in the field of view of the image input unit, it is not detected due to a false detection. There is a problem that a meaningful recognition operation is performed.

  Therefore, the present invention makes it possible to perform stable and highly accurate recognition without being constrained by the posture, standing position, face orientation, etc. at the time of recognition, and a person recognition device and person recognition method capable of improving the person recognition rate And it aims at providing a traffic control device.

  The person recognition apparatus of the present invention includes an image input unit that inputs at least a face image of a person to be recognized, a face detection unit that detects a face area of the person from an image input by the image input unit, and the face A feature amount extracting unit that extracts a feature amount of the person's face based on a detection result of the detecting unit; a registration information holding unit that holds the feature amount extracted by the feature amount extracting unit as registration information at the time of registration; A recognition means for recognizing the person by comparing the feature quantity extracted by the feature quantity extraction means with the registration information held in the registration information holding means, and a guide pattern having a predetermined shape at the time of registration. And display means for instructing the orientation of the face image to be input by moving and displaying.

  Further, the person recognition device of the present invention includes an image input means for inputting at least a face image of a person to be recognized, a face detection means for detecting the face area of the person from the image input by the image input means, A feature amount extracting unit that extracts a feature amount of the person's face based on a detection result of the face detecting unit; and a registration information holding unit that holds the feature amount extracted by the feature amount extracting unit as registration information at the time of registration. The recognition means for recognizing the person by comparing the feature amount extracted by the feature amount extraction means with the registration information held in the registration information holding means at the time of recognition; In addition to displaying the first circle pattern indicating the position of the face image to be input and the distance from the image input means to the face, etc. Are provided with display means for displaying the detection result of the face detection means in a second circle pattern in.

  Further, the person recognition device of the present invention includes an image input means for inputting at least a face image of a person to be recognized, a face detection means for detecting the face area of the person from the image input by the image input means, A feature amount extracting unit that extracts a feature amount of the person's face based on a detection result of the face detecting unit; and a registration information holding unit that holds the feature amount extracted by the feature amount extracting unit as registration information at the time of registration. The recognition means for recognizing the person by comparing the feature amount extracted by the feature amount extraction means with the registration information held in the registration information holding means at the time of recognition; As guidance information indicating whether it should be input in such a state, the position is displayed in a first polygon pattern indicating the position of the face image to be input, the distance from the image input means to the face, and the like, Are provided with display means for displaying the detection result of the face detection means in a second polygonal pattern during recording.

  Further, the person recognition device of the present invention includes an image input means for inputting at least a face image of a person to be recognized, a face detection means for detecting the face area of the person from the image input by the image input means, A feature amount extracting unit that extracts a feature amount of the person's face based on a detection result of the face detecting unit; and a registration information holding unit that holds the feature amount extracted by the feature amount extracting unit as registration information at the time of registration. The recognition means for recognizing the person by comparing the feature amount extracted by the feature amount extraction means with the registration information held in the registration information holding means at the time of recognition; As guidance information indicating whether it should be input in such a state, it is displayed by at least one straight line indicating the position of the face image to be input, the distance from the image input means to the face, and the like. Both, and a display means for displaying a detection result of the face detection unit at the time of registration in the second at least one or more straight lines.

  Further, the person recognition device of the present invention includes an image input means for inputting at least a face image of a person to be recognized, a face detection means for detecting the face area of the person from the image input by the image input means, A feature amount extracting unit that extracts a feature amount of the person's face based on a detection result of the face detecting unit; and a registration information holding unit that holds the feature amount extracted by the feature amount extracting unit as registration information at the time of registration. A recognition means for recognizing the person by comparing the feature amount extracted by the feature amount extraction means with the registration information held in the registration information holding means at the time of recognition, and a face to be input at the time of registration Display means for displaying the orientation of the image by drawing with an arrow pattern, or displaying at the position of a circle pattern drawn in the input image.

  The person recognition method of the present invention includes an image input step of inputting at least a face image of a person to be recognized, a face detection step of detecting the face area of the person from the image input by the image input step, A feature amount extraction step for extracting the feature amount of the person's face based on the detection result of the face detection step, and at the time of registration, the feature amount extracted by the feature amount extraction step is held in registration information holding means as registration information A registration information holding step, a recognition step of recognizing the person by comparing the feature quantity extracted by the feature quantity extraction step with the registration information held in the registration information holding means, and at the time of registration And a display step for instructing the orientation of the face image to be input by moving and displaying a guide pattern of a predetermined shape.

  Further, the traffic control device of the present invention is a traffic control device for recognizing a passerby and controlling the passerby's traffic, an image input unit for inputting at least a face image of the passer, and an input by the image input unit. A face detection means for detecting a face area of the passer-by from the captured image, a feature quantity extraction means for extracting a feature quantity of the passer's face based on a detection result of the face detection means, and the feature quantity at the time of registration Registration information holding means for holding the feature quantity extracted by the extraction means as registration information, and at the time of recognition, the feature quantity extracted by the feature quantity extraction means and the registration information held in the registration information holding means are collated A recognition means for recognizing the passerby, a passage control means for controlling the passage of the passer according to the recognition result of the recognition means, and a guide pattern of a predetermined shape at the time of registration display. By causing, and a display means for indicating the orientation of the to be input face image.

  According to the present invention, it is possible to perform stable and highly accurate recognition without being constrained by the posture, standing position, face orientation, etc. at the time of recognition, and a person recognition device and person recognition method capable of improving the person recognition rate And a traffic control device can be provided.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.
First, the first embodiment will be described.
FIG. 1 schematically shows a configuration of a person recognition apparatus according to the first embodiment. This person recognition apparatus includes a camera 101 as an image input unit that captures and inputs a face image (an image including at least a face) of a person to be recognized (hereinafter also referred to as a person) 100, and the vicinity of the camera 101 (for example, Illumination unit 102 as an illuminating unit that emits light at a certain illuminance toward at least the face of the recognition target person 100 from the upper right or upper left), and the face image and various guidance input to the recognition target person 100 are displayed. And a processing unit 105 that processes a face image input from the camera 101 and performs face image recognition processing and the like.

  In the following description, image input data, extracted feature quantities, subspaces, eigenvectors for constructing subspaces, correlation matrices, status information such as registration time, date and time, location, passwords, ID codes, etc. Each piece of personal information appears. And when it says “recognition data”, it includes a partial space or an eigenvector for constructing the partial space, and when it says “registration information”, it constitutes image input data, extracted feature quantity, subspace, and subspace. Eigenvector, correlation matrix, situation information, and personal information. Accordingly, the recognition data is included in the registration information.

  The camera 101 captures and inputs a face image of the person 100 to be recognized, and includes, for example, a television camera using an image sensor such as a CCD sensor.

  The illumination unit 102 is an illumination device such as a fluorescent lamp that irradiates light at a constant illuminance from the vicinity of the camera 101 (for example, upper right or upper left) toward the face of the recognition target person 100.

  The display unit 104 is a monitor that displays a face image, various guidance, a processing result, and the like of the recognition target person 100 captured by the camera 101.

  The processing unit 105 uses a face image of a person obtained from the image input unit 106 and the image input unit 101 as an image input unit, and collates with a pattern dictionary prepared in advance, and facial parts such as a face, eyes, and nose A face detection unit 107 as a face detection unit for detecting the position of the face, a feature quantity extraction unit 108 as a feature quantity extraction unit for extracting feature quantities such as grayscale information or partial space information based on the output of the face detection unit 107, and person information At the time of registration, the registration information holding unit 109 as a registration information holding unit that holds the feature quantity extracted by the feature quantity extraction unit 108 as recognition data (registration information). At the time of person recognition, the feature value extracted by the feature quantity extraction unit 108 A recognizing unit 110 as a recognizing means for recognizing the face of the person to be recognized 100 by comparing the amount and the recognition data registered in the registered information holding unit 109; Beauty, and a display control unit 111 as display means for controlling the display unit 104.

  The image input unit 106 receives a face image from the camera 101, digitizes the image by A / D conversion, and sends it to the face detection unit 107.

  First, the face detection unit 107 detects the face area of the recognition target person 100 from the face image input by the image input unit 106. For example, a location having the highest correlation value is determined as a face region by obtaining a correlation value while moving the template prepared in advance and the image. In addition, face detection means such as a face region extraction method using an eigenspace method or a subspace method may be used.

  Next, the face detection unit 107 detects the position of the facial parts such as eyes, nose, and mouth from the detected face area. The detection method is described in, for example, literature (Kazuhiro Fukui, Osamu Yamaguchi: “Face Feature Point Extraction by Combination of Shape Extraction and Pattern Matching”, IEICE Transactions (D), vol. J80-D-II, No. 8, pp2170-2177 (1997)).

  Here, it is also possible to check the face direction based on the face detection based on the face template and the relative position of the detection coordinates of the face parts such as eyes and nose. For example, as shown in FIG. 2, the center coordinates of the face detection result are the face center coordinates (fx, fy), the face size is (fw, fh), and the left and right eye and nose detection results are (EyeLx). , EyeLy), (EyeRx, EyeRy), (NoseLx, NoseLy), (NoseRx, NoseRy), an index indicating the face method can be obtained by the following method.

Face direction (X direction) = ((EyeRx + EyeLx) / 2−fx) / fw
Face direction (Y direction) = ((EyeRy + EyeLy) / 2-fy) / fh
When the origin of these coordinates is the upper left of the screen, the numerical value becomes larger as the face faces the lower right.

  Based on the position of the face part detected by the face detection unit 107, the feature amount extraction unit 108 cuts the face region into a certain size and shape, and uses the density information as a feature amount. Here, for example, the gray value of an area of m pixels × n pixels is used as information as it is, and information of m × n dimensions is used as a feature vector.

  FIG. 3 shows face image data obtained by processing the input image by the feature amount extraction unit 108 and is obtained in time series. A partial space is calculated by obtaining a correlation matrix of feature vectors for these data and obtaining an orthonormal vector by a well-known KL expansion. The subspace calculation method calculates a subspace by obtaining a correlation matrix (or covariance matrix) of feature vectors and obtaining an orthonormal vector (eigenvector) by KL expansion. In the subspace, k eigenvectors corresponding to eigenvalues are selected in descending order of eigenvalues, and expressed using the eigenvector set.

In the present embodiment, the correlation matrix Cd is obtained from the feature vector, and the correlation matrix is obtained.

And Φ of the eigenvector is obtained. This partial space is used as a recognition dictionary for identifying a person. This information may be registered in advance and registered as a dictionary.

  Further, as will be described later, the partial space itself may be used as input data for recognition. Therefore, the result of calculating the partial space is sent to the recognition unit 110 and the registration information holding unit 109.

  The recognition unit 110 collates (compares) the recognition data (partial space) stored in the registration information holding unit 109 with the feature amount (grayscale information or partial space information) obtained by the feature amount extraction unit 108. It recognizes who the recognition target person 100 shown in the camera 101 is, or identifies whether it is a corresponding person. In order to recognize a person, it suffices to determine which person's data is most similar, and the person corresponding to the data having the maximum similarity may be used as the recognition result.

  Also, when performing facial recognition using a card, registration number, personal identification number, key, etc., the similarity to the recognition data corresponding to personal information such as each person's card or number is calculated. When the threshold value is exceeded, the person is identified.

  Means for inputting such a card, a registration number, a personal identification number, a key, and the like are necessary. When a registration number or a password is used, key input means may be provided.

  As a recognition method, when the information of the feature quantity extraction unit 108 is used, a method such as a subspace method or a composite similarity method may be used. The recognition method in this embodiment is, for example, literature (Kenichi Maeda, Sadaichi Watanabe: “Pattern Matching Method Introducing Local Structure”, IEICE Transactions (D), vol. J68-D, No. 3, pp 345-352 (1985)), a mutual subspace method which is a known recognition method is used.

In this method, both the recognition data in the registration information stored in advance and the input data are expressed as partial spaces, and the “angle” formed by the two partial spaces is defined as the similarity. The subspace input here is referred to as an input subspace. Similarly, a correlation matrix Cin is obtained for the input data string,

And the eigenvector Φin is obtained. The subspace similarity (0.0 to 1.0) of the subspace represented by two Φin and Φd is obtained and used as the similarity for recognizing it.

  The recognition unit 110 operates as in the flowchart shown in FIG. First, the recognition unit 110 operates differently depending on whether recognition work or identification work is performed (step ST1). When performing the identification operation, first, the ID code of the person to be recognized is read (step ST2). Next, the registration information (partial space) corresponding to the target ID code is read from the registration information holding unit 109 (step ST3).

  Next, as described above, in order to perform recognition using a subspace method or the like, the similarity between the subspace of each registered information and the input vector (eigenvector from the feature quantity extraction unit 108) or the input subspace is calculated. (Step ST4). Next, the similarity is compared with a preset threshold value (steps ST5 and ST6), and the identification result is output (step ST7).

  When performing the recognition work, all the data to be recognized is read from the registered information holding unit 109 (step ST8). And the similarity with each registration information is calculated (step ST9). Next, the maximum similarity is selected from the calculated similarities (step ST10), and is output as a recognition result (step ST12).

  Note that it is also possible to verify whether the recognition result is correct by determining the maximum similarity as a threshold value as in step ST11 surrounded by a broken line in FIG. 4 (step ST13). For example, when the degree of similarity is too low, it can be determined that it is not any recognition target.

  The registration information holding unit 109 can accumulate a partial space (or correlation matrix or the like) used for identifying a recognition target person, status information such as time, date and time at the time of registration, and the like. Note that image input data, extracted feature amounts, or the like may be used.

  Although the basic person recognition operation can be performed by the above processing, as shown in FIG. 5, the recognition rate is lowered due to the secular change and the secular change of the human standing position and posture. In FIG. 5, a frame a indicated by a broken line indicates a face size (face size) at the time of registration.

  For example, as shown in FIG. 5A, when the face is present in the image in the same state as that at the time of registration (when matching), the feature information can be extracted in a state close to that at the time of registration. Although the similarity to the registered dictionary tends to be high, as shown in FIGS. 5B and 5C, when the face is larger or smaller than at the time of registration, or the face is not shown in the figure Similarity is likely to decrease when the orientations are different.

  In order to suppress such problems, in the present embodiment, the display control unit 111 and the display unit 104 only display a moving image of the face of the person being photographed, for example, as shown in FIG. In addition, the detection result c such as the position, size, and direction of the face detected in the photographed image and the detection result b such as the position, size, and size of the face at the time of registration are displayed simultaneously. Thus, it is displayed in an easy-to-understand manner which state the recognition target person 100 can easily confirm with the person. As a result, it is possible to prevent the recognition process from ending without knowing that the face is not correctly captured.

  In other words, the registration information holding unit 109 also holds information such as the size, position, and direction of the face image input at the time of registration. Each value may be an average value in all images used for registration, or may be a maximum value, a minimum value, or a combination of numerical values obtained by combining these values to give a width. I do not care.

  An example of the display method in the display unit 104 is as follows. For example, the position of the face is a circle whose center coordinates are the center of gravity of the four points of the eyes and nose and whose radius is the average distance from the center coordinates to the four points of the eyes and nose. And the size can be expressed. By this method, the average face position and size at the time of registration, and the position and size when the face is detected in the input image are drawn simultaneously by changing the thickness, color, line type, It is possible to indicate how different the current state is compared to the time of registration.

  An example of the result of displaying this is shown in FIG. In FIG. 7, a circle pattern b indicated by a broken line indicates an average position and size where the face exists at the time of registration, and a circle pattern c indicated by a solid line indicates a face area detected in the input image at the time of recognition. Here, the actual input image is present in the background of the circular patterns b and c, but is omitted for easy understanding of the figure.

  In addition to the example of FIG. 7, FIG. 8 shows an example in which contents such as the face direction indicated by a curve are added. If it is front in both the vertical and horizontal directions, it will be displayed as a straight line instead of a curve, and the curvature will be increased so that the more the face moves up and down or left and right, the more convex the direction will be. In FIG. 8, a curve d (showing the center line) d shown by a broken line indicates the face direction at the time of registration, and a curve (showing the center line) e shown by a solid line shows the direction of the face at the time of recognition. Thus, the circle patterns b and c are drawn.

  FIG. 9 shows an example in which cross points in the X direction and Y direction are simply indicated by cross marks. In FIG. 9, a cross mark f indicated by a broken line indicates the face direction at the time of registration, a cross mark g indicated by a solid line indicates the face direction at the time of recognition, and these are drawn in the circular patterns b and c as shown in the figure. Is done.

  Furthermore, FIG. 10 shows an example displayed simply by arrows. In FIG. 10, an arrow h indicated by a broken line indicates the direction of the face at the time of registration, and an arrow i indicated by a solid line indicates the direction of the face at the time of recognition, and these are drawn in the circular patterns b and c as illustrated. .

  The example already described uses the average value of the face size, position, and direction at the time of registration, and is displayed as one circle or ellipse. It is also possible to give a width using the minimum value. For example, in the example of FIG. 7, examples in which the concept of the maximum value and the minimum value are entered for the size, position, and direction are shown in FIGS. 11, 12, and 13, respectively. As for the size, the maximum size and the minimum size are shown in double circles. As for the position, the circumscribed closed curve when the circle is moved within the movement range. The area indicating is indicated by two curves in each direction.

FIG. 11 shows an example in which double circle patterns b1 and b2 indicating the maximum and minimum face sizes detected during registration are drawn. FIG. 12 shows an example in which the center coordinates of the face are moved up and down between A and B during registration, and the outline region j is drawn when the circle indicating the average size of the face detected during registration is moved. An example is shown. FIG. 13 shows an example in which curves (indicating center lines) d1 and d2 indicating the maximum value and minimum value of the face orientation at the time of registration are drawn.
Further, the display method is not limited to the above-described method, and any display method may be used as long as it is a graphic that can indicate the range of the position, size, and direction of the face at the time of registration.

  In the above example, the face information is displayed as a circle or an ellipse pattern. However, the face information is not limited to the circle pattern, and may be a rectangular pattern (polygon pattern) as shown in FIG. A cross line pattern as shown in FIG. FIG. 14 shows an example in which a rectangular pattern k indicating the average position and size of the face at the time of registration is drawn. FIG. 15 shows an example in which a cross line pattern l indicating the area (vertical direction and horizontal direction) where the face was present at the time of registration is drawn.

Furthermore, instead of drawing a line, as shown in FIG. 16, the input image is not displayed in the area n other than the face area m existing at the time of registration, and the method of drawing the input image only in the existing face area m is also effective. It is.
Here, only the above example has been described, but it is also clear that there is no need to stick to the above-described graphic as long as it can indicate the presence of a face.

  In this way, by displaying the information about the face at the time of registration and the information about the input face at the time of recognition at the same time, it is possible to show an index of how the person to be recognized can easily be identified as the person. It is possible to reduce the burden of retrying due to recognition failure.

  Also, for example, guidance such as “Please keep your face a little closer to the camera” or “Please bring your face a little closer to the camera” or display similar content on the screen. Thus, the effectiveness can be further increased.

  In the present embodiment, description will be made in a form that not only holds a partial space but also holds a correlation matrix in the previous stage for calculating the partial space. The registration information holding unit 109 holds one or a plurality of recognition data corresponding to one person or a certain ID code. The partial space is stored together with accompanying information such as the acquired time. The reason for holding a plurality is that a plurality of partial spaces corresponding to a single person are simultaneously passed to the recognition unit 110 for recognition.

  As described above, according to the first embodiment, the size of the face image of the person at the time of registration is acquired, and the size and the size of the face image at the time of recognition are guided within a certain range. The facial image display means to reduce the reduction in the human recognition rate due to the difference in the illumination angle applied to the person's face and the size of the person's face image. It becomes possible.

Next, a second embodiment will be described.
Since the configuration, basic operation, and implementation method of the second embodiment are the same as those of the first embodiment described above, description thereof is omitted here.

  In the second embodiment, a face image only in a specific state is displayed by displaying on the display unit 104 guidance on how the face position, proximity to the camera, and face orientation should be when registering the face. Not only is it possible to collect face images rich in variations, and the purpose is to create an effective dictionary that leads to improved recognition accuracy.

  At the time of registration, information such as the position, size, and orientation of the face detected by the face detection unit 107 is displayed on the display unit 104. As for the display method, it is possible to display with a circular pattern, a rectangular pattern, or a cross mark as in the first embodiment.

  In addition to the information, a guidance that you want to be in this state is displayed at the same time. The display method can also be realized by replacing the information “information at the time of registration” in the first embodiment with the information “Please bring your face to this position”. . For example, it can be realized by a method such as a circle pattern, a rectangular pattern, a cross mark, and clipping that does not display a camera image except for a desired place. A display example of the display unit 104 when displayed in a circle pattern is shown in FIG. In FIG. 17, a circle pattern p indicated by a broken line indicates the position and size of the face to be collected (input), and a circle pattern q indicated by a solid line indicates the position and size of the detected face.

  As a result, when the position of the currently detected face is different from the desired position and location, a message display such as “Please move further to the right” is displayed on the display unit 104, and the desired face size is displayed. If the detection result is different from the above, a message such as “Please move closer to the camera” or “Please move away from the camera” is displayed on the display unit 104. In addition to the display on the screen, a voice message or the like may be output simultaneously.

  In addition, in order to indicate the face direction as well as the position and size of the face, the display unit 104 has an arrow to indicate contents such as “please face up slightly” and “face down slightly” Or by moving the circle. For example, FIG. 18 shows an example in which an arrow pattern r is displayed, which is an example of guidance for directing the face from the left to “front”, “right”, “left”, “up”, “down”. is there. Similarly, FIG. 19 shows an example in which the face orientation is changed depending on the position of the circular pattern s in the camera image.

  As described above, the guidance display method that can easily change the orientation of the face is not limited to these examples, and can be similarly realized by using other graphics, messages, or voices.

  As described above, according to the second embodiment, a face image that faces as many different directions as possible at the time of registration, a face image that has changed the distance from the camera, and a face image that has a different position on the screen. By collecting (inputting) and creating a registered dictionary of face images learned from the face images, accurate recognition is possible without restricting the position, face orientation, posture, etc. that stands in front of the camera during recognition. Can be performed.

Next, a third embodiment will be described.
FIG. 20 schematically shows a configuration of a person recognition apparatus according to the third embodiment. In the third embodiment, control is performed so that the recognition operation is automatically started by detecting the presence of the face of the person 100 to be recognized in front of the camera 101 with respect to the first embodiment described above. The difference is that an automatic recognition start determination unit 112 as a recognition start determination unit to be performed is added, and the other points are the same as those in the first embodiment, and thus description thereof is omitted.

  The automatic recognition start determination unit 112 automatically starts recognition using the fact that the face detection result of the face detection unit 107 has determined that “a face has been detected”. If there is a poster or the like on which a person's face is printed in the range, there is a possibility that it will operate erroneously even if there is no person.

  For this reason, the automatic recognition start determination unit 112 does not use only the face detection result of one frame, but also considers the change from the face detection result before that, and whenever a real human face appears By attaching the condition that there is a motion, for example, even if there is a poster with a face photo, it will not be erroneously detected and malfunction.

  Since the face detection unit 107 obtains information such as the face size, face center coordinates, left and right eyes, and nose detection coordinates as shown in FIG. 2, the automatic recognition start determination unit 112 performs the flowchart shown in FIG. With this processing flow, the recognition operation is automatically started only when “a face is detected continuously for a predetermined number of frames and a variation due to the movement of the face is detected”.

  Hereinafter, the flow of processing based on the flowchart shown in FIG. 21 will be described. First, a face detection counter (not shown, provided in the automatic recognition start determination unit 112) is set to "0" (step ST21), and face detection processing is performed based on the detection result of the face detection unit 107 ( Step ST22). In this face detection process, it is determined whether or not a face is detected in the current frame (step ST23). If no face is detected in the current frame, the face detection counter is set to “0” ( Step ST24) and return to step ST22.

  If a face is detected in the current frame in step ST23, it is determined whether or not a face is detected in the previous frame (step ST25). If no face is detected in the previous frame, face detection is performed. The counter is set to “0” (step ST24), and the process returns to step ST22.

  In step ST25, when a face is detected even in the previous frame, it is determined whether or not the amount of variation from the previous frame is greater than or equal to a predetermined threshold (step ST26). The detection counter is set to “0” (step ST24), and the process returns to step ST22.

  In step ST26, when the fluctuation amount is equal to or greater than a predetermined threshold value, it is determined whether or not the value of the face detection counter is equal to or greater than the predetermined threshold value (step ST27). The counter is incremented by “+1” (step ST28), and the process returns to step ST22.

  In step ST27, when the value of the face detection counter is equal to or larger than the predetermined threshold value, it is determined that the face of the recognition target person 100 exists in front of the camera 101, and the recognition processing of the recognition unit 110 is started.

Here, a method for evaluating the variation amount indicating the movement of the face will be described by taking the displacement amount of the center value of the face detection coordinates as an example. When (fx, fy) shown in FIG. 2 is the center coordinate of the detected face, the center coordinate of the face detected one frame before is (pfx, pfy), and the following variation is used. It is possible to detect the movement of the face.
Variation amount Diff = | fx−pfx | + | fy−pfy |
The fluctuation amount is zero when no face is detected in either the previous frame or the current frame, and is a value having a value of zero or more only when a face is detected continuously. Only when the fluctuation amount Diff is larger than a predetermined threshold Th, the value of the face detection counter for determining that “a moving face exists” is increased, and this condition is not satisfied. All face detection counters are reset to “0”. If this face detection counter is equal to or greater than a predetermined threshold value ThCount, it can be determined that “the face operating in consecutive frames is present”, and the recognition process may be automatically started. Is determined.

  Not limited to this, the absolute value of the displacement of the detected coordinates of the face parts such as the eyes and the nose, the absolute value of the difference between the face sizes, or the detected face coordinates (fx, fy) are the center. Even if an evaluation amount such as a difference amount or a correlation value with the previous frame in the rectangular area represented by the horizontal width ± fw / 2 and the vertical width ± fh / 2 is used, it operates without problems by determining an optimum threshold value for the used numerical value. It is clear to do.

  As described above, according to the third embodiment, even if there is an object that is likely to be erroneously detected as a face such as a poster on which the face of a person is printed in the field of view of the camera 101, there is a risk of malfunction. Therefore, the recognition operation can be automatically started without performing a predetermined operation.

Next, a fourth embodiment will be described.
In the fourth embodiment, the person recognition apparatus described in the first embodiment (or the second embodiment) uses the traffic control for recognizing the passerby's face image and controlling the passerby's pass. This is an example when applied to an apparatus.

FIG. 22 schematically shows the configuration of the traffic control device according to the fourth embodiment. This traffic control device, for example, manages entrance / exit to important facilities (such as security-oriented rooms), recognizes the face image of the user (passer), and enters the important facility based on the recognition result. Opening / closing control of the exit door is performed, and opening / closing control of the entrance / exit door 202 of the important facility 201 is performed according to the recognition results of the camera 101, the illumination unit 102, the display unit 104, the processing unit 105, and the recognition unit 110. It is comprised from the door control part 113 as a traffic control means to perform.
In FIG. 22, the configuration other than the door control unit 113 is the same as that of the person recognition apparatus in FIG.

  For example, when the obtained similarity is greater than the threshold in step ST6 in FIG. 4 or when the obtained similarity is greater than the threshold in step ST11, the recognition unit 110 notifies the door control unit 113 that “ When the door open signal is output and the obtained similarity is smaller than the threshold value, the door control unit 113 outputs a “door close” signal.

When the door control unit 113 receives the “door open” signal from the recognition unit 110, the door control unit 113 controls the entrance / exit door 202 to be in an open state, and allows the recognition target person (passer in this case) 100 to enter the room. When the “door closed” signal is received, the entrance / exit door 202 is kept closed, and the passer-by 100 is refused to enter the room.
Thus, according to the fourth embodiment, it is possible to control the passage of passers-by using the person recognition device described in the first embodiment (or the second embodiment).

  In the fourth embodiment, the case where the person recognition apparatus described in the first embodiment (or the second embodiment) is applied has been described as a representative. It is also possible to apply the person recognition device described in the embodiment. FIG. 23 shows the configuration of a case where the person recognition apparatus described in the third embodiment is applied as a fifth embodiment.

1 is a block diagram schematically showing the configuration of a person recognition device according to a first embodiment of the present invention. The figure for demonstrating the process of a face detection part. The figure for demonstrating the process of a feature-value extraction part. The flowchart for demonstrating the flow of the recognition process of a recognition part. The figure for demonstrating a state change with the time of registration. Explanatory drawing of a display part. The figure for demonstrating simultaneous drawing of registration information and detection information. The figure for demonstrating simultaneous drawing of registration information and detection information. The figure for demonstrating simultaneous drawing of registration information and detection information. The figure for demonstrating simultaneous drawing of registration information and detection information. The figure for demonstrating simultaneous drawing of registration information and detection information. The figure for demonstrating simultaneous drawing of registration information and detection information. The figure for demonstrating simultaneous drawing of registration information and detection information. The figure for demonstrating simultaneous drawing of registration information and detection information. The figure for demonstrating simultaneous drawing of registration information and detection information. The figure for demonstrating simultaneous drawing of registration information and detection information. Explanatory drawing of the guidance which shows the position and size of the face which should be collected at the time of registration which concerns on the 2nd Embodiment of this invention. Explanatory drawing of guidance at the time of registration. Explanatory drawing of guidance at the time of registration. The block diagram which shows roughly the structure of the person recognition apparatus which concerns on the 3rd Embodiment of this invention. The flowchart for demonstrating the flow of a process of an automatic recognition start determination part. The block diagram which shows roughly the structure of the traffic control apparatus which concerns on the 4th Embodiment of this invention. The block diagram which shows roughly the structure of the traffic control apparatus which concerns on the 5th Embodiment of this invention.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 100 ... Recognition person (person, passerby), 101 ... Camera (image input means), 102 ... Illumination part (illumination means), 104 ... Display part (display means), 105 ... Processing part, 106 ... Image input part ( Image input means), 107 ... Face detection section (face detection means), 108 ... Feature quantity extraction section (feature quantity extraction means), 109 ... Registration information holding section (registration information holding means), 110 ... Recognition section (recognition means) DESCRIPTION OF SYMBOLS 111 ... Display control part (display means), 112 ... Automatic recognition start determination part (recognition start determination means), 113 ... Door control part (traffic control means), 201 ... Important facility, 202 ... Door for entrance / exit.

Claims (9)

An image input means for inputting at least a face image of a person to be recognized;
Face detection means for detecting the face area of the person from the image input by the image input means;
Feature amount extraction means for extracting the feature amount of the person's face based on the detection result of the face detection means;
Registration information holding means for holding the feature quantity extracted by the feature quantity extraction means as registration information at the time of registration;
A recognizing unit for recognizing the person by comparing the feature amount extracted by the feature amount extracting unit with the registered information held in the registered information holding unit;
Display means for instructing the orientation of a face image to be input by moving and displaying a guide pattern of a predetermined shape at the time of registration;
A person recognition device comprising: An image input means for inputting at least a face image of a person to be recognized;
Face detection means for detecting the face area of the person from the image input by the image input means;
Feature amount extraction means for extracting the feature amount of the person's face based on the detection result of the face detection means;
Registration information holding means for holding the feature quantity extracted by the feature quantity extraction means as registration information at the time of registration;
A recognizing unit for recognizing the person by comparing the feature amount extracted by the feature amount extracting unit with the registered information held in the registered information holding unit;
At the time of registration, the first circle pattern indicating the position of the face image to be input and the distance from the image input means to the face is displayed as guidance information indicating the state in which the face image should be input. And display means for displaying the detection result of the face detection means at the time of registration in a second circle pattern;
A person recognition device comprising: An image input means for inputting at least a face image of a person to be recognized;
Face detection means for detecting the face area of the person from the image input by the image input means;
Feature amount extraction means for extracting the feature amount of the person's face based on the detection result of the face detection means;
Registration information holding means for holding the feature quantity extracted by the feature quantity extraction means as registration information at the time of registration;
A recognizing unit for recognizing the person by comparing the feature amount extracted by the feature amount extracting unit with the registered information held in the registered information holding unit;
Displayed as a first polygon pattern indicating the position of the face image to be input and the distance from the image input means to the face, etc., as guidance information indicating in what state the face image should be input at the time of registration And a display means for displaying the detection result of the face detection means at the time of registration in a second polygon pattern,
A person recognition device comprising: An image input means for inputting at least a face image of a person to be recognized;
Face detection means for detecting the face area of the person from the image input by the image input means;
Feature amount extraction means for extracting the feature amount of the person's face based on the detection result of the face detection means;
Registration information holding means for holding the feature quantity extracted by the feature quantity extraction means as registration information at the time of registration;
A recognizing unit for recognizing the person by comparing the feature amount extracted by the feature amount extracting unit with the registered information held in the registered information holding unit;
At the time of registration, at least one or more first information indicating the position of the face image to be input, the distance from the image input means to the face, etc. as guidance information indicating in what state the face image should be input Display means for displaying the detection result of the face detection means at the time of registration with at least one second straight line and displaying the straight line;
A person recognition device comprising: An image input means for inputting at least a face image of a person to be recognized;
Face detection means for detecting the face area of the person from the image input by the image input means;
Feature amount extraction means for extracting the feature amount of the person's face based on the detection result of the face detection means;
Registration information holding means for holding the feature quantity extracted by the feature quantity extraction means as registration information at the time of registration;
A recognizing unit for recognizing the person by comparing the feature amount extracted by the feature amount extracting unit with the registered information held in the registered information holding unit;
When registering, display means for displaying the orientation of the face image to be input by drawing with an arrow pattern, or displaying at the position of the circle pattern drawn in the input image,
A person recognition device comprising:   6. The person recognition apparatus according to claim 1, further comprising illumination means for irradiating light toward the person. An image input step of inputting at least a face image of a person to be recognized;
A face detection step of detecting a face area of the person from the image input by the image input step;
A feature amount extraction step of extracting a feature amount of the person's face based on the detection result of the face detection step;
At the time of registration, a registration information holding step for holding the feature quantity extracted by the feature quantity extraction step as registration information in a registration information holding means;
A recognition step of recognizing the person by recognizing the feature amount extracted by the feature amount extraction step and the registration information held in the registration information holding unit;
At the time of registration, a display process for instructing the orientation of the face image to be input by moving and displaying a guide pattern of a predetermined shape;
A person recognition method comprising: In a traffic control device that recognizes a passerby and controls the pass of the passerby,
Image input means for inputting at least a face image of the passer;
Face detection means for detecting a face area of the passer-by from the image input by the image input means;
Feature quantity extraction means for extracting the feature quantity of the passer's face based on the detection result of the face detection means;
Registration information holding means for holding the feature quantity extracted by the feature quantity extraction means as registration information at the time of registration;
Recognizing means for recognizing the passerby by comparing the feature amount extracted by the feature amount extracting means with the registered information held in the registered information holding means at the time of recognition;
A passage control means for controlling the passage of the passer according to the recognition result of the recognition means;
Display means for instructing the orientation of a face image to be input by moving and displaying a guide pattern of a predetermined shape at the time of registration;
A traffic control device comprising:   9. The traffic control device according to claim 8, further comprising illumination means for irradiating light toward the passerby.

Images