JP2012160212A - Image data identification method, imaging device, program, and storage medium - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an imaging device capable of shortening time required for personal authentication.SOLUTION: An image data identification method for performing identification processing of either personal identification or facial expression identification on face image data pieces included in image data of plural frames, includes: a face detection step (S202) in which face detection means detects face image data pieces included in the image data; and an identification step (S207) in which identification means performs the identification processing on one or plural face image data pieces detected in the face detection step. In the identification step, a higher value is given to a face image data piece that indicates the same person as that of a face image data piece included in image data of a frame to be subjected to the identification processing and that is included in image data of a previous frame when the identification processing is successful than when it is unsuccessful, and the identification processing is performed on face image data pieces in descending order of the value until the number thereof reaches a predetermined upper limit.

Description

  The present invention relates to an image data identification method, an imaging apparatus, a program, and a storage medium having a personal authentication function.

  Recently, there has been a technology in which a digital camera, which is an imaging device, incorporates a technique for recognizing the face of a subject. Furthermore, a digital camera has been proposed that not only recognizes the face of the subject but also further identifies the face of the subject and personally authenticates the person who is the subject. .

  For example, the invention disclosed in Patent Document 1 proposes a personal authentication method capable of performing authentication with higher accuracy and higher speed by devising a feature point candidate extraction method used for personal authentication of a subject. Yes. Further, the invention disclosed in Patent Document 2 proposes a device for a personal authentication method in the case where the illumination condition of the subject at the time of imaging changes.

JP-A-9-251534 JP 2000-222576 A

  However, in the method proposed in Patent Document 1, it is necessary to input image data in which only one person is a subject included in the image data. Therefore, no consideration is given to a case where a plurality of persons are included as subjects in the image data, or a case where subjects other than the target person are imaged simultaneously.

  Further, in the method proposed in Patent Document 2, a plurality of persons are included as subjects included in image data, and a method of detecting a plurality of included persons is introduced. However, since the method proposed in Patent Document 2 is a method for determining the number of persons by detecting the amount of movement of a person who is a subject, the imaging device needs to be fixed. Furthermore, the method proposed in Patent Document 2 requires past image data captured at the same fixed angle of view.

  As described above, the conventionally proposed methods include a case where a plurality of persons as subjects are included, or a case where a plurality of persons are captured at different angles of view for each image frame. An imaging device has a drawback that personal authentication cannot be performed.

  In addition, in the conventional personal authentication method, a lot of time is required to perform the personal authentication process on all of the plurality of captured faces, and the personal authentication process is performed for the display operation using the EVF. Sometimes it was not in time. Conversely, in order to perform the personal authentication process within a time that is in time for the display operation in the EVF, it is necessary to reduce the frame rate of the display in the EVF. As described above, in the imaging while viewing the screen with the EVF in a state where the display frame rate is low, the position or expression of the person is not displayed in real time. Therefore, there is a risk that the most important shutter chance as a digital camera is missed. This problem can be applied not only to personal authentication for identifying an individual, but also to facial expression recognition for identifying a smile, eye meditation, and the like.

  Accordingly, an object of the present invention is to provide an imaging apparatus capable of shortening the time required for identification processing of an individual or a facial expression.

  In order to achieve the above object, an identification method of image data according to the present invention for performing identification processing of individual identification or facial expression identification on face image data included in image data of a plurality of frames. ,

  A face detection step in which a face detection means detects face image data included in the image data;

  An identification unit that performs the identification process on one or a plurality of face image data detected in the face detection step;

  In the identification step, the face image data included in the image data of the previous frame indicating the same person as the face image data included in the image data of the frame to be subjected to the identification process is failed when the identification process is successful. The identification processing is performed on face image data that gives a higher numerical value than that in the case where the numerical value is higher and reaches a predetermined upper limit.

  According to the present invention, it is possible to reduce the time required for identifying an individual or a facial expression, and it is possible to provide an imaging apparatus having an identification function capable of appropriate photographing.

It is a block block diagram of an imaging device provided with the personal authentication function which concerns on embodiment of this invention. It is a flowchart explaining operation | movement of an imaging device provided with the personal authentication function which concerns on embodiment of this invention.

<Embodiment 1>
FIG. 1 is a block diagram showing a configuration of an imaging apparatus 100 having a personal authentication function according to an embodiment of the present invention. In FIG. 1, 1 is an imaging lens and 2 is a focus lens. By controlling the focus lens 2, the in-focus state of the subject to be imaged can be controlled.

  Reference numeral 3 denotes an image sensor such as a CCD or CMOS that photoelectrically converts the light flux of the subject that has passed through the imaging lens into an electrical signal. Reference numeral 4 denotes an A / D conversion unit that performs clamp processing, gain processing, and the like on an analog image signal output from the image sensor 3 and converts the analog image signal into image data that is a digital signal.

  A signal processing unit 5 executes predetermined pixel interpolation processing and color conversion processing on the image data from the A / D conversion unit 4. In addition, the signal processing unit 5 executes predetermined calculation processing using image data obtained by imaging, and performs signal processing such as automatic white balance processing based on the obtained calculation result.

  The image data subjected to signal processing by the signal processing unit 5 is temporarily stored in the image memory 6. Also, the image data temporarily stored in the image memory 6 is supplied to the recording / reproducing unit 7 and recorded on the removable recording medium 8 at the request of the user. The image data is sent to the display control unit 9 and the display device 10 displays the image data. As the recording medium 8, various removable recording media such as an optical disk recording medium and a semiconductor recording medium can be used.

  The system control unit 11 basically controls the entire operation of the imaging apparatus 100, and normally includes a CPU. For example, the signal processing unit 5 executes a predetermined calculation process using image data obtained by imaging, and controls the focus control unit 12 and the exposure control unit 13 based on the calculation result. That is, the focus control unit 12 controls the focus lens 2 to execute automatic focus control (AF). Further, the exposure control unit 13 controls the aperture and shutter in the imaging lens 1 and further the electronic shutter of the imaging device 3 to execute automatic exposure control (AE).

  The RAM 14 connected to the system control unit 11 may store image data to be displayed on the display device 10 and supplies the image data to the display device 10 via the display control unit 9 for display. The RAM 14 can also be used as a work area for the system control unit 11.

  The image memory 6 temporarily stores captured still image data and moving image data, and is high-speed and large enough to store a predetermined number of still image data and a predetermined amount of moving image data. Storage capacity. Thus, even in the case of continuous shooting that continuously captures a plurality of still image data or panoramic imaging that combines a plurality of captured image data, the image memory 6 can be used.

  The display device 10 is used as an electronic viewfinder (hereinafter, EVF) by sequentially displaying captured image data. Further, the display device 10 is configured to be able to arbitrarily turn on / off the display according to the execution of the program in the system control unit 11. Furthermore, face detection status display, personal authentication status display, focus frame display, focus display, out-of-focus warning display, and the like are also performed.

  A recording / reproducing unit 7 is an interface with a removable recording medium 8 such as a memory card or a hard disk. Using the recording / reproducing unit 7, image data and management information attached to the image data can be transferred to and from the recording medium 8. The system control unit 11 controls the operation of the entire imaging apparatus 100. In the RAM 14 and / or ROM 15 connected to the system control unit 11, constants, variables, programs for operation of the system control unit 11, feature information of face image data of a registered person for personal authentication or identification processing, Data necessary for face detection is stored.

  Reference numeral 16 denotes a face detection unit. The face detection unit 16 detects a feature portion of a person's face by detecting edges such as eyes and a mouth from a subject included in image data to be captured. In this case, detection is executed based on data stored in the RAM 14 and / or the ROM 15 and necessary for face detection.

  In other words, the face detection process and the identification process that detect the image area data as face image data based on the area of the image data (face position and face size) occupied by the face and the likelihood of the face (face reliability) are executed. To do. Furthermore, the face detection unit 16 also detects the number of detected face image data. Since the face detection process does not need to store reference data for each individual as in the case of personal authentication, the process can be completed sufficiently earlier than the personal authentication.

  Reference numeral 17 denotes a feature information extraction unit that extracts feature information used for personal authentication from the face image data detected by the face detection unit 16.

  Reference numeral 18 denotes a main face rate calculation unit, and 19 denotes a main face selection unit. Details of these will be described later. An authentication unit 20 is based on the feature information of the registered person face image data for personal authentication stored in the RAM 14 and / or the ROM 15 and the detected face image data extracted by the feature information extraction unit 17. For example, personal authentication is performed by template matching.

  The configuration of the imaging apparatus illustrated in FIG. 1 is an example, and the configuration of the imaging apparatus according to the present invention is limited to the configuration illustrated in FIG. 1 as long as the operations described below can be performed. It is not something.

  Next, the operation of the imaging apparatus 100 according to the embodiment of the present invention will be described in detail with reference to the flowchart of FIG. First, after the process is started, the image data output from the image sensor 3 is acquired by the A / D conversion unit 4 and the signal processing unit 5 in step S201. The acquired image data is temporarily stored in the image memory 6, for example.

  In step S202, the face detection unit 16 performs face detection processing on the image data acquired in step S201 and recorded in the image memory 6. As a result of the face detection process by the face detection unit 16, the feature part of the face is detected by detecting edges such as eyes and mouth from the subject included in the image data to be captured. The face image data is detected from the image data area (face position “S” and face size “S”) occupied by the face of the person who is the subject, and the likelihood of the face (face reliability “R”). The number of face image data is detected.

  In step S203, it is determined whether the number of face image data detected by the face detection unit 16 is 2 or greater. This determination is performed by the system control unit 11, for example. If it is determined in step S203 that the number of detected face image data is 2 or more, the process proceeds to step S205, and the main face ratio calculation unit 18 executes the main face ratio calculation process. Note that the main face ratio represents a main ratio of each face image data in the image data, that is, a main degree.

  In the main face ratio calculation process in step S205, the main face ratio “M” representing the main degree is calculated for all face image data obtained by the face detection in step S202. In this case, at least the distance “D” from the center of the face image data in the image data, the size “S” of the face image data, and the face reliability “R” are used. M "is calculated. In this case, the distance “D” from the center of the face image data in the image data means the position of the face image data in the image data. The distance “D”, the size “S”, and the face reliability “R” are respectively closer to the center of the image data, larger face image data, and higher face reliability of the face image data. The main face ratio “M” is set to be large.

  The calculation of the main face ratio “M” is executed by the main face ratio calculation unit 18, and the main face ratio “M” representing the main degree is calculated by the following calculation formula.

  M = (Dis = W × D) + (Siz = W × S) + (Rel = W × R)

  As a result of the calculation using the above calculation formula, a main face selection process in which the face of the face image data having the maximum main face ratio “M” is the main face is executed in step S206. The main face selection unit 19 executes this main face selection process. The distance weight coefficient “Dis = W”, the face size weight coefficient “Siz = W”, and the face reliability weight coefficient “Rel = W” are set in the ROM 15 in advance.

  The above calculation formula is an example, and what is important here is to obtain the main face ratio “M” of the main degree in the image data of the face image data. Therefore, even if a method other than the method described here is used, if the main face ratio “M” of the main degree in the image data of the face image data can be obtained, the parameters and calculation formulas other than those in this embodiment can be used. Is possible. Of course, parameters and calculation formulas may differ depending on how the main degree is defined.

  In step S206, the main face ratio “M” that is the main face image data included in the image data that is currently being examined from the plurality of detected face image data, that is, the calculated main face ratio “M” is checked. A main face selection process for selecting face image data having the maximum “M” is executed. The main face selection process in step S206 is executed by the main face selection unit 19.

  Then, the process proceeds to step S207, and the personal authentication in the authentication unit 20 is executed for the selected face image data of one main face, whereby the personal authentication can be performed in a short time.

  If the number of faces detected in the determinations in steps S203 and S204 is 1, the main face selection process is unnecessary, and the process directly proceeds to step S207, and personal authentication is performed on the face image data of the person who is set as one main face. By performing this, it is possible to perform personal authentication in a short time. Of course, if NO in step S204, this means that no face image data was detected in step S202, that is, there was no face image data in the image data and there was no person, and step S208 was performed without performing personal authentication processing. Will proceed to.

  At this point, if it is determined in step S208 that the shutter SW1 has been pressed, the process proceeds to step S209, where the focus control unit 12 and the exposure control unit 13 are controlled to execute automatic focus control and automatic exposure control. Thereafter, in step S210, it is confirmed that the shutter SW2 is pressed, and in step S211, an imaging operation, that is, recording of image data including face image data on the recording medium 8 is executed, and the process is terminated.

  Further, by storing the face image data position, the size “S” of the face image data, and the main face ratio “M” for each image frame, when it can be determined that the change in the angle of view between the image frames is small, It is also possible to correct and process as follows.

  That is, according to the result of personal authentication processing of face image data obtained from previously captured image data, the main face rate “M” is corrected by multiplying the main face rate weight “Main = W”, After that, face image data with the largest main face ratio “M” may be used as the main face. A calculation formula for correcting the main face ratio “M” in step S206 in this case is shown below.

  M = Main = WxM

  Furthermore, the main face ratio weight “Main = W” is corrected by increasing the numerical value when personal authentication can be performed, and by decreasing the numerical value when personal authentication cannot be performed. It is also possible to increase the accuracy of the main face selection process.

  When there are a plurality of face image data whose main face ratio “M” is larger than a predetermined threshold value, it is also conceivable to execute the personal authentication process for the plurality of face image data. However, it is necessary to determine a threshold value so that the number of face image data to be personally authenticated does not increase too much. Alternatively, an upper limit on the number of subjects to be subjected to personal authentication processing may be provided, and personal authentication processing may be performed on face image data from a high main face ratio “M” to a number reaching the upper limit.

  According to the present invention, even when face image data of a plurality of persons is included as a subject in image data captured at one angle of view, personal authentication of a person intended by the user is possible. Further, since the personal authentication is performed efficiently, display processing with EVF at a high frame rate is possible, and an imaging apparatus capable of imaging without missing a photo opportunity can be provided.

  In addition, facial expression recognition processing that identifies smiles and eye meditation requires more detailed information than face detection processing, such as eye movements and mouth corner shapes, so the reference data to be stored is more than face detection processing. It is common in that there are many. Accordingly, with regard to the facial expression recognition process, it is possible to obtain an effect that the time required for the identification can be shortened by adopting a configuration in which the identification process is performed only for a face having a high main face ratio.

  The present invention can also be implemented by supplying a storage medium storing software program codes for realizing the functions of the above-described embodiments to a system or apparatus. That is, the object of the present invention can also be achieved by reading and executing a program code stored in a storage medium by a computer (or CPU or MPU) of the system or apparatus.

  In this case, the program code itself read from the storage medium realizes the functions of the above-described embodiments, and the storage medium storing the program code constitutes the present invention. As a storage medium for supplying the program code, for example, a flexible disk, a hard disk, an optical disk, a magneto-optical disk, a CD-ROM, a CD-R, a magnetic tape, a nonvolatile memory card, a ROM, or the like can be used.

  Further, by executing the program code read by the computer, not only the functions of the above-described embodiments are realized, but also an OS or the like running on the computer performs actual processing based on an instruction of the program code. Some or all may be done. Thus, the present invention includes a case where the functions of the above-described embodiments are realized by the processing.

  Furthermore, in the present invention, the program code read from the storage medium can be written in a memory provided in a function expansion board inserted into the computer or a function expansion unit connected to the computer. Therefore, the function of the above-described embodiment is realized by performing part or all of the actual processing by the CPU provided in the function expansion board or function expansion unit based on the written program code instruction. Is also included.

Claims (4)

In an image data identification method for performing identification processing of either individual identification or facial expression identification on face image data included in image data of a plurality of frames,
A face detection step in which a face detection means detects face image data included in the image data;
An identification unit that performs the identification process on one or a plurality of face image data detected in the face detection step;
In the identification step, the face image data included in the image data of the previous frame indicating the same person as the face image data included in the image data of the frame to be subjected to the identification process is failed when the identification process is successful. A method for identifying image data, wherein the identification processing is performed on face image data from a higher value to a number that reaches a predetermined upper limit. . In an imaging device that performs identification processing of either personal identification or facial expression identification on face image data included in image data of a plurality of frames,
Face detection means for detecting face image data included in the image data;
Identification means for performing the identification processing on one or a plurality of face image data detected by the face detection means,
The identification unit fails when the identification process is successful for the face image data included in the image data of the previous frame indicating the same person as the face image data included in the image data of the frame on which the identification process is performed. An image pickup apparatus that gives a higher numerical value than the case where the identification processing is performed on face image data from a higher numerical value to a number that reaches a preset upper limit.   The program for making a computer perform each process of the identification method of the image data of Claim 1.   A computer-readable storage medium storing a program for causing a computer to execute each step of the image data identification method according to claim 1.

Images