JP2013142930A - Image processing device, control method thereof and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image processing device capable of stably obtaining desired face recognition results in face recognition processing.SOLUTION: The image processing device detects face areas from sequentially obtained first images. With respect to the detected face areas, the image processing device determines relevant candidate face areas in the face areas which are detected from second images of a person whose face recognition data are registered. In the similarity level calculated with respect to the object face area, when the similarity level of a candidate face area has a maximum value and when it is determined that a difference between the similarity level of the maximum value and the next similarity level exceeds a first threshold value, in the next face recognition processing, it is prohibited to switch from a person as a result of the face recognition in the candidate face area to another person. While switching of face recognition result is prohibited, when a similarity level higher than the similarity level of the person as the face recognition result in the candidate face area is calculated with respect to the object face area, a piece of information representing not the person whose similarity level is calculated as higher but the person as the face recognition result in the candidate face area is output as the face recognition result in the object face area.

Description

  The present invention relates to an image processing apparatus, a control method thereof, and a program, and more particularly to a face recognition technique for specifying a person for a face image included in an image.

  Some imaging devices such as digital cameras have a face detection function for detecting the face of a person included in a subject. In such an imaging apparatus, for example, a pattern that is determined to be a person's face is determined in advance by face detection processing, and a portion that matches the pattern included in the image can be detected as a person's face. The detected human face is referred to, for example, in focus control and exposure control.

  In recent years, there is also an imaging apparatus having a face recognition function for determining whether or not a person's face detected by the face detection process matches a face registered in advance. The face recognition process performed by the imaging apparatus is performed using face recognition data in which a face image serving as a reference for face recognition or a feature amount of the face image is registered. Specifically, in the face recognition process, the similarity of the face image to be processed is calculated for each person registered in the face recognition data (face dictionary) in advance, and the person having the highest similarity is the face image. Identified as a person.

  However, the similarity calculated in the face recognition process varies depending on the state of the subject and the shooting environment. That is, when the similarity is lowered due to environmental changes or the like, an incorrect face recognition result may be obtained. In Patent Document 1, when the calculated similarity does not exceed a threshold for specifying a person, an incorrect face recognition result is displayed by displaying the group name to which the person belongs instead of the registered name. A technique for avoiding this is disclosed.

JP 2010-27034 A

  On the other hand, in the face recognition processing, a plurality of persons with similarities exceeding a threshold for specifying a person for one face image may appear as candidates. In such a case, one of the persons is specified as the face detection result, but a desired person may not be selected as the face recognition result.

  The present invention has been made in view of the above-described problems, and provides an image processing apparatus, a control method thereof, and a program capable of stably obtaining a desired face recognition result in face recognition processing. Objective.

In order to achieve the above object, an image processing apparatus of the present invention comprises the following arrangement.
Face recognition means for identifying a person included in each face area in an image based on a similarity to a person for which face recognition data is registered, and outputting information indicating the identified person as a face recognition result An image processing apparatus having an acquisition unit that sequentially acquires images, a detection unit that detects a face area from the image acquired by the acquisition unit, and face recognition data for each of the face areas detected by the detection unit And calculating means for calculating the degree of similarity with the registered person, and the target face area detected from the first image acquired by the acquiring means is acquired and included before the first image. Regarding the face area, candidate acquisition means for identifying a candidate face area having a correspondence relationship from the face areas detected from the second image for which the face recognition result by the face recognition means has already been output, and the target face area The similarity between the candidate and the person who is the face recognition result of the candidate face region has the maximum value, and the difference between the maximum value similarity and the next point similarity is the first threshold value. In the face recognition processing performed later by the face recognition means, the face recognition result of the face area corresponding to the candidate face area is changed from the person who is the face recognition result of the candidate face area to another person. Deterring means for deterring switching to a person, and the face recognizing means is similar to the person who is the face recognition result of the candidate face area when switching of the face recognition result is deterred by the deterring means. If a higher similarity is calculated for the target face area, information indicating the person who is the face recognition result of the candidate face area is output as the face recognition result of the target face area instead of the person for which the higher similarity is calculated It is characterized by doing.

  With this configuration, according to the present invention, it is possible to stably obtain a desired face recognition result in the face recognition process.

1 is a block diagram showing a functional configuration of a digital camera 100 according to an embodiment of the present invention. Flowchart illustrating face recognition processing according to an embodiment of the present invention The figure which illustrated the data structure of face area detailed information and a threshold table concerning an embodiment of the present invention The flowchart which illustrated the switching suppression determination process and stability determination process which concern on embodiment of this invention

  Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the drawings. In the following embodiment, an example in which the present invention is applied to a digital camera capable of performing face recognition processing using face recognition data as an example of an image processing apparatus will be described. However, the present invention can be applied to any device capable of executing face recognition processing using face recognition data.

  Further, in this specification, a “face image” is an image of a person's face area extracted from an image including a person. The “face dictionary” is face recognition data used for matching processing in face recognition processing, including one or more face images for each person and feature area data included in each face image. It will be explained as a thing. In the face dictionary, the number of face images to be included is determined in advance.

<Configuration of Digital Camera 100>
FIG. 1 is a block diagram showing a functional configuration of a digital camera 100 according to an embodiment of the present invention.

  The CPU 101 controls the operation of each block included in the digital camera 100. Specifically, the CPU 101 controls the operation of each block by reading out an operation program such as face recognition processing stored in the ROM 102, developing it in the RAM 103 and executing it.

  The ROM 102 is, for example, a rewritable nonvolatile memory, and stores parameters necessary for the operation of each block of the digital camera 100 in addition to an operation program such as a photographing process.

  The RAM 103 is a volatile memory, and is used not only as a development area for operation programs such as face recognition processing but also as a storage area for storing intermediate data output in the operation of each block of the digital camera 100.

  The imaging unit 105 includes an imaging element such as a CCD or CMOS sensor, an A / D conversion unit, and the like. The imaging unit 105 photoelectrically converts the optical image formed on the image sensor by the optical system 104, and outputs a captured image to which A / D conversion processing is applied. Note that the imaging unit 105 also performs processing such as sampling and gain adjustment on the input analog image signal.

  The image processing unit 106 applies predetermined image processing such as color correction and color conversion to the captured image output from the imaging unit 105. The image processing unit 106 includes a face detection unit 107 and a similarity calculation unit 108 in the present embodiment.

  The face detection unit 107 detects the face of a person who is a specific subject in an image converted into, for example, a YUV format. Specifically, the face detection unit 107 detects an area (face area) corresponding to a human face pattern according to color information and contrast distribution in the image. The face detection process performed by the face detection unit 107 outputs the position of the face area and the number of pixels as the detection result.

  The similarity calculation unit 108 analyzes the feature amount of the input face area image. Then, the similarity calculation unit 108 specifies a person corresponding to the analyzed feature amount using a face dictionary recorded in a recording medium 109 described later. Specifically, the similarity calculation unit 108 calculates a score indicating the similarity for each registered face image or its feature amount for each person registered in the face dictionary.

  The recording medium 109 is a recording device that is detachably connected to the digital camera 100 such as a built-in memory of the digital camera 100, a memory card, or an HDD. In the present embodiment, the recording medium 109 records an image photographed by the photographing process and a face dictionary referred to by the face recognition process in the digital camera 100. When the image recorded on the recording medium 109 includes a person in the image, the person obtained by the face recognition process is recorded in association with the face recognition result. In this embodiment, the face dictionary is described as being recorded on the recording medium 109, but the embodiment of the present invention is not limited to this. For example, the face dictionary may be stored in a writable area such as the ROM 102. The face dictionary may be configured to be stored in a predetermined recording area by the CPU 101 when received from an external device.

  The display unit 110 is a display device included in the digital camera 100 such as a small LCD. The display unit 110 displays a captured image output from the imaging unit 105, an image recorded on the recording medium 109, and the like. When the person identified by the face recognition process is included in the displayed image, the face recognition result can be notified to the user by, for example, displaying the person name superimposed.

  The operation input unit 111 is a user interface that the digital camera 100 includes, for example, operation members such as a power button and a shutter button. When the operation input unit 111 detects that the operation member is operated by the user, the operation input unit 111 generates a control signal corresponding to the operation content and transmits the control signal to the CPU 101.

<Face recognition processing>
A specific process of the face recognition process of the digital camera 100 according to the present embodiment having such a configuration will be described with reference to the flowchart of FIG. The processing corresponding to the flowchart can be realized by the CPU 101 reading out a corresponding processing program stored in, for example, the ROM 102, developing it in the RAM 103, and executing it. The face recognition process will be described as being started when, for example, a captured image captured by the imaging unit 105 is output. Further, the face recognition process may be performed by sequentially acquiring captured images output from the imaging unit 105, or may be sequentially performed on captured images acquired at predetermined frame intervals.

  In step S <b> 201, the CPU 101 inputs the captured image output from the imaging unit 105 to the image processing unit 106 and causes the face detection process to be executed. Specifically, the face detection unit 107 detects a face pattern of a person included in the captured image input to the image processing unit 106 and specifies a face area.

  In S202, the CPU 101 performs, for each of the face areas detected in S201 (current face area), a captured image (second image) subjected to face recognition processing before the captured image (first image) from which the face area is detected. The corresponding candidate face area that is recognized as the same subject is identified from the face areas detected from the image of FIG. Specifically, the CPU 101 reads from the RAM 103 face area specifying information for specifying a face area (past face area) in which the person detected and included in the previous face recognition process is specified. In this embodiment, the face area information includes a face identification ID for identifying the detected face area, and position information (for example, center coordinates) of the face area. The CPU 101 refers to the position information of the face area information for each of the past face areas and the position information of each of the current face areas, and specifies a corresponding candidate face area for each of the current face areas. At this time, the CPU 101 assigns the face identification ID assigned to the corresponding candidate face area to each of the current face areas.

  For each past face area, detailed face area information indicating information indicating a person included in the face area specified by the face recognition process is associated and stored in the RAM 103. In addition to the above, a method such as pattern matching may be used to identify the corresponding past face area. However, the method of comparing the current face area position with the past face area position is a processing load. This is preferable because it can be realized without increasing the height.

(Face area detailed information)
Here, face area detailed information used in the face recognition processing of the present embodiment will be described with reference to FIG. As shown in the drawing, in addition to the same face identification ID 301 as that attached to the associated past face area, the stability flag 302, the switching suppression flag 303, the face recognition ID 304, and the similarity history are included in the face area detailed information. 305 is included.

  The stability flag 302 indicates whether or not the face recognition result for the corresponding face area, that is, the face area recognized as being the same subject, is stable as a result of performing face recognition processing on the sequentially acquired captured images. Logical type information. Although a detailed stability determination method will be described later, in this embodiment, the face recognition result by the face recognition process is the same for a predetermined number of times, and the similarity calculated for the person who is the face recognition result is When it is equal to or greater than a threshold value (second threshold value) continuously for a predetermined number of times, the CPU 101 determines that the face recognition result is stable. The stability flag 302 indicates true (ON) when the face recognition result is stable, and false (OFF) when it is not stable.

  The switch suppression flag 303 is logical information indicating whether or not to suppress the face recognition result by the face recognition processing from being switched to a different person for the face area recognized as the same subject. . The switching suppression flag 303 indicates true when switching of face recognition results is suppressed, and indicates false when not suppressing.

  The face recognition ID 304 is identification information indicating a person specified in the last face recognition process. In this embodiment, the face recognition ID 304 is identification information for specifying a face image group or a feature amount group for one person among the persons registered in advance in the face recognition data recorded in the recording medium 109. The same thing shall be used. That is, by referring to the face recognition ID 304, the CPU 101 can specify face recognition data for the person specified in the last face recognition process.

  The similarity history 305 stores the similarities calculated by the similarity calculating unit 108 in the order of processing for face regions recognized as being the same subject.

  That is, in this step, by assigning the face identification ID assigned to the corresponding candidate face area to each of the current face areas, the face area associated with the candidate face area is assigned to the current face area. Detailed information is inherited and associated (candidate acquisition). That is, each face area detected in S201 is predicted to include a person who is a face recognition result for a candidate face area corresponding to the face area by this step. In the following, a person who is predicted to be included will be described as a “candidate person”.

  It should be noted that the CPU 101 deletes the face area information and the face area detailed information from the RAM 103 for the past face area that has not been selected as the corresponding candidate face area in this step.

  In step S203, the CPU 101 selects, as a target face area, one face area in which a person included in the current face area detected in step S201 is not specified.

  In step S <b> 204, the CPU 101 causes the similarity calculation unit 108 to calculate similarities with the feature amounts of all persons registered in the face recognition data for the face image indicated by the target face area. Specifically, the similarity calculation unit 108 extracts a feature amount from a face image indicated by the target face area by a predetermined method, and calculates the similarity.

  In step S <b> 205, the CPU 101 executes a switching suppression determination process that determines whether to suppress switching of face recognition results for the target face area.

(Switch suppression judgment processing)
Here, the switching suppression determination process executed in S205 will be described in detail with reference to the flowchart of FIG.

  In step S401, the CPU 101 determines whether the face recognition result for the target face area is stable. Specifically, the CPU 101 refers to the stability flag 302 of the face area detailed information associated with the target face area, and determines whether or not the face recognition result is stable. If the CPU 101 determines that the face recognition result is stable, it moves the process to S402, and if it determines that the face recognition result is not stable, completes the switching inhibition determination process.

  In S402, the CPU 101 determines whether the similarity having the maximum value is the similarity of the candidate person among the similarities of the persons registered in the face recognition data previously calculated for the target face area in S204. Judging. If the CPU 101 determines that the similarity of the candidate person has the maximum value, the CPU 101 moves the process to S403. If the CPU 101 determines that the similarity of the different person has the maximum value, the CPU 101 completes the switching inhibition determination process.

  In step S <b> 403, the CPU 101 calculates the difference between the similarity of the candidate person that is the maximum value and the similarity of the next point in the similarity of each person previously registered in the face recognition data calculated for the target face area. Is determined to exceed a predetermined threshold TH_WideGap (first threshold). If the CPU 101 determines that the difference between the similarity of the maximum value and the similarity of the next point exceeds the first threshold value, the CPU 101 moves the process to S404, and if it determines that the difference does not exceed the first threshold, completes the switching inhibition determination process. . Note that the first threshold value is set in order to determine that the person included in the target face area is a person having a maximum similarity having a maximum value in the similarity-based face recognition result.

  In step S <b> 404, the CPU 101 sets true to the face area detailed information switching inhibition flag 303 associated with the target face area.

  That is, by this switching suppression determination processing, the CPU 101 determines that the face recognition result is obtained when the face recognition result is stable for the target face area and the similarity of the next point is more than the first threshold value. Is determined to be deterministic and switching of face recognition results is to be suppressed. In other words, once a face recognition result that can be determined as deterministic is obtained, a stable face recognition result can be output by suppressing switching of the corresponding face area as described below.

  In S206, the CPU 101 determines that the maximum value in the similarity of each person registered in advance calculated in S204 for the target face area is set for the person corresponding to the similarity of the maximum value. It is determined whether or not the threshold TH_default is exceeded. In the present embodiment, the threshold value for successful face recognition will be described as different for each person registered in the face recognition data. This is because the number of face images and feature quantities registered in the face recognition data may be different for each person.

  In the face recognition process, the recognition system may be improved by calculating the degree of similarity using a plurality of face images of the same person. For this reason, the face recognition data includes one or more face images or feature amounts for each person. However, a recognition score calculated using a plurality of face images for one person, for example, when calculating a total value of similarities for each face image as a recognition score, uses one face image. It is assumed that it is higher than the calculated recognition score. In the present embodiment, the face image or feature value registered in the face recognition data for each person corresponds to the fact that the recognition score calculation criteria differ depending on the number of registered face images or feature values. The threshold for success in face recognition is set differently depending on the number. It should be noted that the information on the threshold value for successful face recognition may be stored in the recording medium 109 as a threshold value table associated with the face recognition ID for each person included in the face recognition data. In the present embodiment, as shown in FIG. 3B, the threshold table is associated with the face recognition ID 311, the stability determination threshold (high) 312, the stability determination threshold (low) 313, the face recognition success threshold 314, and the stable face recognition. Assume that four thresholds of success threshold 315 are shown. Other threshold values will be described as appropriate in the following description of steps.

  If the CPU 101 determines that the similarity of the maximum value exceeds the face recognition success threshold 504 for the person corresponding to the similarity, the process proceeds to S207, and if it does not exceed, the process proceeds to S212.

  In step S207, the CPU 101 determines whether the face recognition result for the target face area is stable. If the CPU 101 determines that the face recognition result is stable, it moves the process to S208, and if it determines that the face recognition result is not stable, it moves the process to S211.

  In step S208, the CPU 101 determines whether to suppress switching of face recognition results for the target face area. Specifically, the CPU 101 refers to the face area detailed information 303 associated with the target face area, and determines whether or not to switch the face recognition result. If the CPU 101 determines that the switching of the face recognition result is to be suppressed, it moves the process to S209, and if it determines not to suppress it, it moves the process to S211.

In step S209, the CPU 101 determines whether or not the maximum similarity for the target face area is a predetermined value that is set based on the similarity of the candidate person and exceeds a predetermined value that is higher than the similarity of the candidate person. Determine whether. Specifically, the CPU 101 sets SC_Max as the maximum similarity and SC_Cand as the similarity of the candidate person.
SC_Max> SC_Cand × α (where α> 1)
It is determined whether or not the above is satisfied. If the CPU 101 determines that the similarity of the maximum value exceeds the predetermined value, the process proceeds to S210. If the CPU 101 determines that the similarity does not exceed the predetermined value, the process proceeds to S212.

  For example, when the subject person A turns sideways from the front side, the (person A recognition score, person B recognition score) is (85, 40), (85, 45), (85 , 40), (85, 40), (80, 50), (75, 60), (75, 70), (75, 80), (75, 80), (75, 70), (75, 80 ) And (75, 65). When face images facing front are registered as face recognition data for person A and person B, the recognition score of person A is stable and high while person A is facing front (in the above example) 85) When facing sideways, the recognition score of the person A decreases (in the example, 85 → 75), and in some cases, the recognition score of another person (person B) increases (in the example, 40 → 80). At this time, if it is simply recognized that the person has the highest recognition score, the face recognition result of person A will change from person A to person B. In the present embodiment, by setting the switch inhibition flag 303 in a state where the recognition score of the person A is a stable and prominent high recognition score, the recognition score of the same level as in this step is obtained from the person A. Even if the recognition score of person B is higher, the face detection result switching is suppressed.

  In step S <b> 210, the CPU 101 sets false in the switching suppression flag 303 of the face area detailed information associated with the target face area. That is, it is determined to be deterministic when the similarity is prominently higher than the similarity of the person of the face recognition result that has been determined to be definite even though the switching of the face recognition result is being suppressed. It is possible that the face recognition result was incorrect. In other words, in this step, the CPU 101 changes the switching suppression flag 303 set to true to false so that a more likely person is output as the face recognition result, and stops switching the detection of the face detection result. The process proceeds to S211.

  In step S211, the CPU 101 sets the person whose maximum similarity is calculated as the face recognition result of the target face area. Specifically, the CPU 101 acquires the face recognition ID set in the face recognition data for the person whose maximum similarity is calculated, and performs face recognition of the face area detailed information associated with the target face area. Enter the ID 304. The CPU 101 also adds the maximum similarity to the similarity history 305 of the face area detailed information associated with the target face area.

  On the other hand, when it is determined in S206 that the maximum similarity is not greater than the face recognition success threshold for the person corresponding to the similarity, or in S209, the maximum similarity is determined not to exceed the predetermined value. In this case, the CPU 101 moves the process to S212.

  In step S212, the CPU 101 determines whether the face recognition result for the target face area is stable. If the CPU 101 determines that the face recognition result is stable, it moves the process to S213, and if it determines that the face recognition result is not stable, it moves the process to S214.

  In step S <b> 213, the CPU 101 sets the face recognition success threshold to a stable face recognition success threshold 315 (TH_Low) lower than the face recognition success threshold 314. That is, if the face recognition result for the target face area is stable, it is highly possible that the same face recognition result is obtained for the target face area detected in S201, but that it is not an error. For this reason, unlike the normal case, the CPU 101 sets a stable face recognition success threshold value 315 lower than the face recognition success threshold value 314 as a temporary face recognition success threshold value in order to widen the range of similarity regarded as face recognition success. In this way, in the digital camera 100 of the present embodiment, the same face recognition result should be obtained, but even when the similarity is lowered due to the person facing sideways or the like, it is stable. It is possible to easily obtain a face recognition result. When the face recognition result for the target face area is not stable, the face recognition threshold is set as the normal face recognition success threshold 314.

  In S <b> 214, the CPU 101 determines whether or not the candidate person's similarity with respect to the target face area exceeds the threshold value for successful face recognition. If the CPU 101 determines that the similarity of the candidate person exceeds the threshold for successful face recognition, the process proceeds to S215. If the CPU 101 determines that the similarity does not exceed the threshold, the process proceeds to S216.

  In S215, the CPU 101 sets the candidate person as the face recognition result of the target face area. Specifically, the CPU 101 acquires the face recognition ID set in the face recognition data for the candidate person, and inputs it to the face recognition ID 304 of the face area detailed information associated with the target face area. Further, the CPU 101 adds the similarity of the candidate person to the similarity history 305 of the face area detailed information associated with the target face area. If the face recognition ID of the candidate person has already been input in the face recognition ID 304 of the face area detailed information associated with the target face area, the input of the face recognition ID may be omitted.

  In step S216, the CPU 101 outputs a face recognition result indicating that the face recognition of the target face area has failed. When face recognition fails, the CPU 101 deletes the face area detailed information associated with the target face area. At this time, for example, when the real face recognition process is performed on the captured image displayed on the display unit 110 as a live view screen, the CPU 101 does not associate the face area detailed information with respect to the target face area. What is necessary is just to output to the display part 110 and notify that the face recognition process failed.

  If it is determined in S209 of the present embodiment that the maximum similarity is not greater than the predetermined value, the maximum similarity and the similarity for the candidate person can be regarded as being at the same level. That is, when a plurality of persons having the same level of similarity for one face area are obtained, the desired face recognition result may not be selected so far, but the CPU 101 switches the face recognition result. In this embodiment, a desired face detection result can be obtained stably. That is, in such a case, the CPU 101 avoids switching the face recognition result to another person by only determining whether or not the face of the candidate person is included in the target face area in the processing of S212 to S216. To do.

  In step S217, the CPU 101 executes a stability determination process for determining whether the face recognition result for the target face area is stable.

(Stability judgment processing)
Here, the details of the switching suppression determination process executed in S217 will be described with reference to the flowchart of FIG.

  In step S411, the CPU 101 determines whether the face recognition result for the target face area has changed from the previous face recognition result in the face recognition processing step described above. Specifically, the CPU 101 determines whether or not the face recognition ID 304 of the face area detailed information associated with the target face area has been changed to a different value. If the CPU 101 determines that the face recognition result for the target face area has not changed from the previous face recognition result, the CPU 101 moves the process to S412. If the CPU 101 determines that the face recognition result has changed, the stability determination process is completed.

  In step S412, the CPU 101 determines whether the face recognition result for the target face area is stable. If the CPU 101 determines that the face recognition result is not stable, it moves the process to S413, and if it determines that the face recognition result is stable, completes the stability determination process.

  In step S <b> 413, the CPU 101 refers to the similarity history 305 of the face area detailed information associated with the target face area, and the stability determination for all the past M face recognition results is higher than the face recognition success threshold 314. It is determined whether or not the threshold (high) 312 is exceeded. If the CPU 101 determines that all the similarities of the past M face recognition results are greater than or equal to the stability determination threshold (high) 312, the CPU 101 moves the process to S 415, and even one is less than the stability determination threshold (high) 312. If it is determined that there is a similarity, the process proceeds to S414.

  In step S <b> 414, the CPU 101 determines that the degrees of similarity for the past N (> M) face recognition results are all higher than the face recognition success threshold 314 and lower than the stability determination threshold (high) 312. It is determined whether this is the case. If the CPU 101 determines that all the similarities of the past N face recognition results are equal to or greater than the stability determination threshold (low) 313, the CPU 101 moves the process to S415, and even one is less than the stability determination threshold (low) 313. If it is determined that there is a similarity, the process proceeds to S416.

  In step S415, the CPU 101 sets true to the stability flag 302 of the face area detailed information associated with the target face area, and completes the stability determination process.

  In step S416, the CPU 101 sets false to the stability flag 302 of the face area detailed information associated with the target face area, and completes the stability determination process.

  After determining whether the face recognition result for the target face area is stable in this way, the CPU 101 moves the process to S218.

  In step S218, the CPU 101 determines whether or not there is still a face area in which the included person is not specified in the current face area detected in step S201. If the CPU 101 determines that there is a face area in which no person is included, the process returns to S203, and if it does not exist, the face recognition process is completed.

  As described above, the image processing apparatus according to the present embodiment can stably obtain a desired face recognition result in the face recognition process. Specifically, the image processing apparatus detects a face area from sequentially acquired images, and calculates the similarity between each detected face area and a person registered with face recognition data. The target face area detected from the first image is acquired before the first image, and the face recognition result for the included face area is already output from the second image that has been output. From this, a candidate face area having a correspondence relationship is identified. Then, the image processing apparatus has a maximum similarity between the similarity calculated for the target face area and the person who is the face recognition result of the candidate face area, and the similarity between the maximum value and the next point When it is determined that the difference from the similarity exceeds the first threshold value, the face recognition result of the face area corresponding to the candidate face area is converted into the face recognition result of the candidate face area. Switch from one person to another person. In the face recognition process, when switching of the face recognition result is suppressed, when a similarity higher than the similarity of the candidate face region with the person who is the face recognition result is calculated for the target face region, Information indicating the person who is the face recognition result of the candidate face area, not the person whose high similarity is calculated, is output as the face recognition result of the target face area.

  As described above, in the face recognition process of the present embodiment, the face recognition process can be performed while determining the stability of the face recognition result for the corresponding face area detected in the past and the suppression of the switching of the face recognition result. A stable face recognition result can be output.

[Other Embodiments]
The present invention can also be realized by executing the following processing. That is, software (program) that realizes the functions of the above-described embodiments is supplied to a system or apparatus via a network or various storage media, and a computer (or CPU, MPU, or the like) of the system or apparatus reads the program. It is a process to be executed.

Claims (6)

Face recognition means for identifying a person included in each face area in an image based on a similarity to a person for which face recognition data is registered, and outputting information indicating the identified person as a face recognition result An image processing apparatus comprising:
Acquisition means for sequentially acquiring images;
Detecting means for detecting a face area from the image acquired by the acquiring means;
Calculating means for calculating the degree of similarity between each face area detected by the detecting means and the person registered with the face recognition data;
The target face area detected from the first image acquired by the acquisition means is acquired before the first image, and the face recognition result by the face recognition means has already been output for the included face area. Candidate acquisition means for specifying candidate face areas in a correspondence relationship from the face areas detected from the second image,
In the similarity calculated for the target face area, the similarity with the person who is the face recognition result of the candidate face area has a maximum value, and the similarity between the maximum value and the similarity of the next point The face recognition result of the face area corresponding to the candidate face area in the face recognition process performed later by the face recognition means when the difference between the two faces exceeds the first threshold. Deterring means for deterring switching from the person who is the face recognition result to another person,
The face recognizing means has a similarity higher than the similarity of the candidate face area with a person as the face recognition result when the face recognizing result is suppressed by the suppressing means for the target face area. When the image processing is performed, image information is output as the face recognition result of the target face area, indicating information indicating a person who is the face recognition result of the candidate face area instead of the person whose high similarity is calculated apparatus. For one face area detected by the detection means, the face recognition results of the corresponding face areas in the images sequentially acquired by the acquisition means are the same continuously for a predetermined number of times, and the face recognition results A determination means for determining that the face recognition result for the person is stable when the similarity calculated for the person is equal to or greater than the second threshold value for the predetermined number of times;
The said suppression means performs the said suppression, when it is judged by the said judgment means that the face recognition result about the person who is a face recognition result of the said candidate face area | region is stable. Image processing apparatus. The suppression means sets the similarity calculated with respect to the target face area based on the similarity with the person who is the face recognition result of the candidate face area when switching the face recognition result is suppressed. And when there is a similarity that exceeds a predetermined value that is higher than the similarity, the suppression is stopped,
The image processing apparatus according to claim 1, wherein the face recognition unit outputs a person whose similarity exceeding the predetermined value is calculated as a face recognition result of the target face area.   The face recognizing unit is configured to determine whether the face recognition result is a face recognition result of the candidate face region in the similarity calculated for the target face region when switching of the face recognition result is suppressed by the suppressing unit. When a similarity higher than the similarity and lower than the predetermined value is calculated, information on whether or not the person is a face recognition result of the candidate face area is displayed as the face of the target face area The image processing apparatus according to claim 3, wherein the image processing apparatus outputs the result as a recognition result. Face recognition means for identifying a person included in each face area in an image based on a similarity to a person for which face recognition data is registered, and outputting information indicating the identified person as a face recognition result A control method for an image processing apparatus comprising:
An acquisition step in which the acquisition unit of the image processing apparatus sequentially acquires images;
A detection step in which the detection means of the image processing device detects a face area from the image acquired in the acquisition step;
A calculation step in which the calculation means of the image processing device calculates a similarity with each person whose face recognition data is registered for each of the face areas detected in the detection step;
The candidate acquisition means of the image processing apparatus acquires the target face area detected from the first image acquired in the acquisition step before the first image, and already includes the face for the included face area. A candidate acquisition step of identifying candidate face regions in a corresponding relationship from the face regions detected from the second image from which the face recognition result by the recognition means is output;
In the similarity calculated for the target face area by the suppression unit of the image processing apparatus, the similarity with the person who is the face recognition result of the candidate face area has a maximum value, and the maximum value When it is determined that the difference between the similarity and the similarity of the next point exceeds the first threshold value, in the face recognition process performed later by the face recognition unit, the face area corresponding to the candidate face area A deterrent step of deterring a face recognition result from switching from a person who is a face recognition result of the candidate face region to another person,
The face recognizing means has a similarity higher than the similarity with a person as a face recognition result of the candidate face area for the target face area when switching of the face recognition result is suppressed in the suppressing step. When the image processing is performed, image information is output as the face recognition result of the target face area, indicating information indicating a person who is the face recognition result of the candidate face area instead of the person whose high similarity is calculated Control method of the device.   The program for functioning a computer as each means of the image processing apparatus of any one of Claims 1 thru | or 4.

Images