JP2017037375A - Imaging apparatus and control method thereof - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an imaging apparatus capable of accurately performing a series of face authentication processing of a subject on a pick-up image even when a tilted face may be high-possibly picked up by second imaging means.SOLUTION: An imaging apparatus 100 comprises: a main imaging unit 101 for picking up a subject; and a sub imaging unit 102 for picking up a photographer operating a main imaging unit 101 as a subject. The imaging apparatus 100 executes a series of first authentication processing which is face authentication processing of a subject picked up by the main imaging unit 101, or a piece of second authentication processing which is face authentication processing of a subject picked up by the sub imaging unit 102 using a piece of dictionary data. The imaging apparatus 100 controls the second authentication processing depending on inclination of the sub imaging unit 102.SELECTED DRAWING: Figure 1

Description

  The present invention relates to an imaging apparatus and a control method thereof.

There has been proposed an imaging apparatus having a function of photographing a subject and photographing a photographer who is operating the camera.
Japanese Patent Application Laid-Open No. 2004-133867 images a subject image with a first imaging unit, images a photographer who operates the first imaging unit with a second imaging unit, and synthesizes each video signal to record a recording medium. A video camera recorder is disclosed.

JP-A-6-165029

  In the video camera recorder disclosed in Patent Document 1, for example, when the photographer is facing the subject or the second imaging unit is tilted, the second imaging unit captures the face of the tilt. Resulting in. Therefore, when subject face authentication processing is executed on the image captured by the second imaging means using dictionary data registered based on a face facing forward, there is a high possibility that authentication will not be possible.

  The present invention is an image pickup apparatus including first and second image pickup means for picking up an image of a subject. Even when the face of the second image pickup means is highly likely to be imaged, An object of the present invention is to provide an imaging apparatus capable of performing subject face authentication processing with high accuracy.

  An imaging apparatus according to an embodiment of the present invention includes a first imaging unit that images a subject, a second imaging unit that images a photographer who operates the first imaging unit as a subject, and the first imaging unit. Authentication means for executing a first authentication process that is a face authentication process of a subject imaged by the means or a second authentication process that is a face authentication process of a subject imaged by the second imaging means; Storage means for storing dictionary data used for the authentication process or the second authentication process, and a control means for controlling the second authentication process in accordance with the inclination of the second imaging means.

  According to the imaging apparatus of the present invention, it is possible to accurately execute subject face authentication processing on a captured image even when there is a high possibility that the face of the second imaging means is captured.

It is a figure which shows the structure of the imaging device of this embodiment. It is a figure showing an example of appearance of an imaging device. 3 is a flowchart for explaining an operation process of the imaging apparatus according to the first embodiment. It is a figure which shows the example of the timing chart of a to-be-photographed object's face authentication process. It is a figure which shows the structural example of dictionary data. 6 is a flowchart illustrating an operation process of the imaging apparatus according to the second embodiment. It is a figure which shows the example of the timing chart of a to-be-photographed object's face authentication process.

  In an imaging apparatus including a plurality of imaging units, it is conceivable to perform face authentication on a subject imaged by the first imaging unit or a subject imaged by the second imaging unit to identify an individual. In face authentication, a facial feature amount (dictionary data) registered in advance is compared with a facial feature amount of a captured person, and a person is determined based on whether the similarity is an arbitrary value or more. The facial feature quantity is often obtained from organ position information such as eyes, nose and mouth. Since the first imaging unit often captures the subject's face so that it faces the front, the dictionary data used in face authentication for the image captured by the first imaging unit is composed of the front-facing face. There are many cases. Furthermore, since the organ position moves depending on the facial expression and the orientation of the face, a plurality of faces whose facial expressions are changed or rotated in the yaw direction are registered as dictionary data. Further, in face authentication for an image picked up by the second image pickup means, it is conceivable to use dictionary data prepared for the first image pickup means.

  However, when the first imaging means is used to take an image of a child as a subject, for example, the photographer checks the angle of view with a display means such as an LCD, and the child is directed toward the camera so that the child pays attention to the camera. It is often suitable. When the second image pickup means is attached to the LCD side, when the photographer is looking at the LCD, the second image pickup means can pick up the front face of the photographer, but faces the subject. Sometimes the face of the tilt is easily captured. When the LCD has a vari-angle in the vertical direction and the second imaging means moves together with the LCD, the tilted face is particularly easily imaged. Since the tilted face has a different organ position from the front face, using dictionary data registered on the basis of the front face will deviate from the organ position of the dictionary data. As a result, the similarity obtained by comparing the feature amount of the dictionary data with the feature amount of the imaged face falls below an arbitrary value, and there is a problem that the subject cannot be authenticated. In the imaging apparatus according to the present embodiment described below, it is possible to execute the subject face authentication process with high accuracy even when the tilted face is easily captured.

Example 1
FIG. 1 is a diagram illustrating a configuration of an imaging apparatus according to the present embodiment.
The imaging apparatus 100 includes a main imaging unit 101, a sub imaging unit 102, an authentication unit 103, an authentication control unit 104, an inclination detection unit 105, a dictionary data storage unit 106, a dictionary switching control unit 107, and a display device 108. The main imaging unit 101 functions as a first imaging unit that images a subject. The main imaging unit 101 outputs subject image data obtained by imaging to the authentication control unit 104.

  The sub imaging unit 102 functions as a second imaging unit that images a photographer who operates the imaging apparatus main body as a subject. The subject image data obtained by imaging is output to the authentication control unit 104. The tilt detection unit 105 detects the tilt of the sub imaging unit 102. The tilt detection unit 105 outputs the tilt detection result to the authentication control unit 104 and the dictionary switching control unit 107.

  The dictionary data storage unit 106 stores dictionary data used for the first authentication process or dictionary data used for the second authentication process. The first authentication process is a subject face authentication process for the image data output from the main imaging unit 101. The second authentication process is a subject face authentication process for the image data output by the sub imaging unit 102. The dictionary switching control unit 107 switches dictionary data stored in the dictionary data storage unit 106 in accordance with an instruction from the authentication control unit 104. The authentication unit 103 uses the dictionary data in the dictionary data storage unit 106 to execute the first authentication process or the second authentication process.

  The authentication control unit 104 controls the face authentication process by the authentication unit 103 according to the tilt detection result input from the tilt detection unit 105. In the first embodiment, the authentication control unit 104 controls the authentication unit 103 to execute the second authentication process using the switched dictionary data when the sub imaging unit 102 is inclined. For this purpose, the authentication control unit 104 instructs the dictionary switching control unit 107 to switch the dictionary data stored in the dictionary data storage unit 106. The display device 108 displays an image or UI captured by the main image capturing unit 101 or the sub image capturing 102.

FIG. 2 is a diagram illustrating an appearance example of the imaging apparatus.
FIG. 2A shows an example of the appearance of the front surface portion of the imaging apparatus 100. FIG. 2B shows an example of the appearance of the back surface of the imaging device 100. Reference numeral 201 denotes an imaging lens for the main imaging unit 101, 202 a shutter switch, and 203 a strobe. Reference numeral 204 denotes an imaging lens for the sub imaging unit 102, 205 denotes an operation button, and 206 denotes a display unit (LCD) included in the display device 108. The imaging lens 204 for the sub imaging unit 102 is fixed to the surface of the LCD 206. When the LCD 206 is movable, the imaging lens 204 for the sub imaging unit 102 also moves.

FIG. 3 is a flowchart illustrating an operation process of the imaging apparatus according to the first embodiment.
In this example, it is assumed that the dictionary data storage unit 106 initially stores first dictionary data corresponding to a face facing forward. In this example, the first dictionary data includes a face facing front and a face facing right or left by a predetermined angle with respect to the face facing front. The second dictionary data described later includes a tilted face or an overhead view.

  In S301, the main imaging unit 101 and the sub imaging unit 102 capture an image. In step S <b> 302, the authentication control unit 104 determines which of the images captured by the main imaging unit 101 and the sub imaging unit 102 is to perform face authentication processing. If it is determined that the face authentication process is to be performed on the image captured by the main imaging unit 101, the process proceeds to S303. If it is determined that the face authentication process is to be performed on the image captured by the sub imaging unit 102, the process proceeds to S306.

  In S303, the dictionary data switching control unit 107 determines whether the dictionary data is switched to the second dictionary data. If the dictionary data has not been switched, the process proceeds to S305. If the dictionary data has been switched, the process proceeds to S304.

  In S304, the dictionary data switching control unit 107 returns the dictionary data to the first dictionary data before switching. In S305, the authentication unit 103 executes the first authentication process using the first dictionary data. In step S309, the authentication processing control unit 104 determines whether imaging has been completed. When the imaging is completed, the process ends. If imaging has not been completed, the process returns to S301.

  In S306, the authentication control unit 104 determines whether the sub imaging unit 102 is tilted. If the sub imaging unit 102 is not tilted, the process proceeds to S308. Then, the authentication unit 103 uses the first dictionary data in the dictionary data storage unit 106 to execute a second authentication process. If the sub imaging unit 102 is tilted, the process proceeds to S307. Then, the dictionary data switching control unit 107 switches the dictionary data in the dictionary data storage unit 106 to the second dictionary data. Then, the process proceeds to S308, and the authentication unit 103 executes a second authentication process using the second dictionary data after switching.

  The process of S307 will be described. The dictionary data switching control unit 107 adds predetermined data, that is, dictionary data corresponding to an overhead view or a tilted face, to the first dictionary data stored in the dictionary data storage unit 106. As a result, the first dictionary data is switched to the second dictionary data. The dictionary data switching control unit 107 may change dictionary data by changing predetermined data of the first dictionary data to a dictionary data including a bird's-eye view or a tilted face. Good.

FIG. 4 is a diagram illustrating an example of a timing chart of subject face authentication processing for images captured by the main imaging unit and the sub imaging unit in the first embodiment. FIG. 5 is a diagram illustrating a configuration example of dictionary data.
In the first embodiment, the authentication control unit 104 controls the authentication unit 103 to execute the first authentication process and the second authentication process in a time-sharing manner. Note that the authentication control unit 104 may control the frequency at which the authentication unit 103 executes the first authentication process to be higher than the frequency at which the authentication unit 103 executes the second authentication process.

At time t in FIG. 4, an image is captured by the main imaging unit 101 and the sub imaging unit 102. This process corresponds to step S301 in FIG. The authentication control unit 104 determines which of the image captured by the main imaging unit 101 and the image captured by the sub imaging unit 102 is to be authenticated. This process corresponds to S302 in FIG. Here, control is performed so that authentication processing is performed on an image captured by the main imaging unit 101 at time t.

  Next, the authentication control unit 104 requests the dictionary switching control unit 107 to perform dictionary switching control, and the dictionary switching control unit 107 performs dictionary switching when dictionary switching is necessary. This process corresponds to S303 and S304 in FIG. Here, no dictionary switching is required, and no dictionary switching is performed. The authentication control unit 104 controls the authentication unit 103 to perform authentication processing on the image captured by the main imaging unit 101 using the first dictionary data in the dictionary data storage unit 106. This process corresponds to S305 in FIG.

FIG. 5 is a diagram illustrating an example of dictionary data in the dictionary data storage unit.
FIG. 5A shows the first dictionary data. FIGS. 5B and 5C show the second dictionary data. The second dictionary data shown in FIG. 5 (B) includes predetermined dictionary data (the two dictionary data on the right in FIG. 5 (A)) of the first dictionary data shown in FIG. 5 (A). This is dictionary data generated by replacing the face dictionary data of the face. The second dictionary data shown in FIG. 5C is dictionary data generated by adding the dictionary data of the face of the face to the first dictionary data shown in FIG.

  Returning to the description of FIG. The authentication control unit 104 confirms the imaging completion request. This process corresponds to S309 in FIG. Since imaging continues at this point, the process proceeds to time t + 1. At time t + 1, processing similar to that at time t is performed on the image captured by the main imaging unit 101. At time t + 2, the main imaging unit 101 and the sub imaging unit 102 capture an image. Subsequently, the authentication control unit 104 determines which of the images captured by the main imaging unit 101 and the images captured by the sub imaging unit 102 is to be subjected to the authentication process. In this example, the authentication control unit 104 controls to perform authentication processing on the image captured by the sub imaging unit 102 at time t + 2.

  Next, the inclination of the sub imaging unit 102 is confirmed. This process corresponds to S306 in FIG. The inclination of the sub imaging unit 102 is detected by the movement of the acceleration sensor or the LCD 206 on which the sub imaging unit 102 is installed. In this example, it is determined that the sub imaging unit 102 is not tilted, and the process proceeds to step S308. In step S308, the authentication control unit 104 controls the authentication unit 103 to perform an authentication process on the image captured by the sub imaging unit 102 using the dictionary data 106 as illustrated in FIG. In step S309, the imaging completion request is confirmed, the process returns to step S301, and the process proceeds to time t + 3. At time t + 3, processing similar to that at time t is performed on the image captured by the main imaging unit 101.

  At time t + 4, authentication processing for the image captured by the main imaging unit 101 at time t + 3 is not completed. Therefore, the authentication control unit 104 performs control so that the authentication process is not performed on the image captured by the main imaging unit 101 at time t + 4. At time t + 5 and time t + 6, processing similar to that at time t is performed on the image captured by the main imaging unit 101.

  At time t + 7, the authentication control unit 104 controls to perform authentication processing on the image captured by the sub imaging unit 102. At this time, since it is determined that the sub imaging unit is inclined in the yaw direction, the dictionary switching control 107 switches the first dictionary data to the second dictionary data. Then, the authentication control unit 104 controls the image captured by the sub imaging unit 102 so that authentication processing using the second dictionary data after switching is performed. Here, the authentication unit 103 is performing authentication processing on the image captured by the main imaging unit 101 at time t + 6. Therefore, the authentication control unit 104 controls to start the authentication process for the image captured by the sub imaging unit 102 at time t + 7 as soon as the authentication process is completed.

  At time t + 8, the authentication control unit 104 controls to perform authentication processing on the image captured by the main imaging unit 101. At this time, in the authentication process at time t + 7, since the dictionary data is switched to the second dictionary data, it is determined that the dictionary needs to be switched. Accordingly, the dictionary switching control unit 107 removes the dictionary data of the tilted face and the overhead view from the second dictionary data stored in the dictionary data storage unit 106, as shown in FIG. Switch to the first dictionary data. The authentication control unit 104 controls the image captured by the main imaging unit 101 so that authentication processing using the second dictionary data is performed. Here, the authentication unit 103 is performing authentication processing on the image captured by the sub imaging unit 102 at time t + 7. Therefore, the authentication control unit 104 controls to start the authentication process for the image captured by the main imaging unit 101 at time t + 8 as soon as the authentication process is completed.

  With the above processing, when there is a high possibility that the sub-imaging unit 102 is likely to capture a tilted face that is difficult to authenticate with dictionary data corresponding to a front-facing face, processing is performed by adding the tilted face dictionary data. Thus, even if the face of the tilt is imaged, it can be easily authenticated. Therefore, the face authentication process can be executed with high accuracy. In addition, since the second dictionary data after switching includes all or part of the face-facing face dictionary data, it is possible to authenticate even if the front face is imaged. Further, when the authentication process is performed on the image captured by the first main imaging unit 101, the process is performed without adding the tilt face dictionary data, so that the authentication accuracy and the processing time are not affected.

(Example 2)
FIG. 6 is a flowchart illustrating an operation process of the imaging apparatus according to the second embodiment.
S601, S602, and S606 are the same as S301, S302, and S309 in FIG. In step S <b> 603, the authentication unit 103 executes a first authentication process using the first dictionary data stored in the dictionary data storage unit 106. In step S604, the authentication control unit 104 determines whether the inclination of the sub imaging unit 102 has been detected. If the tilt of the sub imaging unit 102 is not detected, the process proceeds to S605. Then, the authentication unit 103 uses the first dictionary data in the dictionary data storage unit 106 to execute a second authentication process. When the tilt of the sub imaging unit 102 is detected, the authentication control unit 104 performs control so that the second authentication process is not performed. With this control, the authentication unit 103 does not execute the face authentication process, and the process proceeds to S606.

FIG. 7 is a diagram illustrating an example of a timing chart of subject face authentication processing for images captured by the main imaging unit and the sub imaging unit in the second embodiment.
At time t in FIG. 7, an image is captured by the main imaging unit 101 and the sub imaging unit 102. This process corresponds to step S601 in FIG. The authentication control unit 104 determines which of the image captured by the main imaging unit 101 and the image captured by the sub imaging unit 102 is to be authenticated. This process corresponds to S602 in FIG. Here, control is performed so that authentication processing is performed on the image captured by the main imaging unit 101 at time t.

  Next, the authentication control unit 104 controls the authentication unit 103 to perform authentication processing on the image captured by the main imaging unit 101 using the first dictionary data in the dictionary data storage unit 106. This process corresponds to S603 in FIG. At time t + 1, processing similar to that at time t is performed on the image captured by the main imaging unit 101.

  At time t + 2, the authentication control unit 104 controls to perform authentication processing on the image captured by the sub imaging unit 102. At this time, it is determined that the sub imaging unit 102 is not tilted. Therefore, the authentication control unit 104 controls the authentication unit 103 to perform authentication processing on the image captured by the sub imaging unit 102 using the first dictionary data in the dictionary data storage unit 106. This processing corresponds to the processing in S604 and S605 in FIG. Subsequently, at time t + 3, processing similar to that at time t is performed on the image captured by the main imaging unit 101.

  At time t + 4, the authentication process for the image captured by the main imaging unit 101 at time t + 3 is not completed. Therefore, the authentication control unit 104 performs control so that authentication processing is not performed on the image captured by the main imaging unit 101 at time t + 4.

  At time t + 5 and time t + 6, processing similar to the processing at time t is performed on the image captured by the main imaging unit 101. At time t + 7, the authentication control unit 104 controls to perform authentication processing on the image captured by the sub imaging unit 102 at time t + 2. At this time, since it is determined that the sub imaging unit 102 is tilted, the authentication control unit 104 performs control so that the authentication process for the image captured by the sub imaging unit 102 is not performed. Note that when it is detected that the sub imaging unit 102 is tilted, the process may proceed to S603 instead of S606 in FIG. 6 to perform authentication processing on the image captured by the main imaging unit 101. At time t + 8, processing similar to the processing at time t is performed on the image captured by the main imaging unit 101.

  By performing the above processing, the authentication unit 103 is authenticated by performing control so that authentication processing is not performed when there is a high possibility that a face with a face that is difficult to authenticate with front face dictionary data is captured. It can be used for authentication processing of a face that is easier to authenticate than a difficult face. As mentioned above, although the preferable Example of this invention was described, this invention is not limited to these Examples, A various deformation | transformation and change are possible within the range of the summary.

[Other Embodiments]
The present invention supplies a program that realizes one or more functions of the above-described embodiments to a system or apparatus via a network or a storage medium, and one or more processors in a computer of the system or apparatus read and execute the program This process can be realized. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

DESCRIPTION OF SYMBOLS 100 Image pick-up device 101 Main image pick-up part 102 Sub image pick-up part 103 Authentication part 104 Authentication control part 105 Tilt detection part 106 Dictionary data storage part 107 Dictionary switching control part 108 Display apparatus

Claims (13)

A first imaging means for imaging a subject;
Second imaging means for imaging a photographer who operates the first imaging means as a subject;
Authentication means for executing a first authentication process that is a face authentication process of a subject imaged by the first imaging means or a second authentication process that is a face authentication process of a subject imaged by the second imaging means When,
Storage means for storing dictionary data used for the first authentication process or the second authentication process;
An image pickup apparatus comprising: control means for controlling the second authentication process in accordance with an inclination of the second image pickup means. The control means switches dictionary data used for the first authentication processing stored in the storage means when the inclination of the second imaging means is detected,
The imaging apparatus according to claim 1, wherein the authentication unit executes the second authentication process using the dictionary data after the switching. The imaging apparatus according to claim 2, wherein the control unit switches the first dictionary data by adding predetermined data to the first dictionary data. The first dictionary data has dictionary data corresponding to the front face,
The imaging apparatus according to claim 3, wherein the predetermined data added to the first dictionary data includes dictionary data corresponding to a tilted face or an overhead view. The imaging device according to claim 3, wherein the control unit switches the first dictionary data by changing data included in the first dictionary data to predetermined data. The first dictionary data has dictionary data corresponding to the front face,
The imaging apparatus according to claim 5, wherein the predetermined data includes dictionary data corresponding to a tilted face or an overhead face. The imaging apparatus according to claim 1, wherein the control unit performs control so that the second authentication process is not performed when an inclination of the second imaging unit is detected. 8. The control unit according to claim 1, wherein the control unit controls which of the first authentication process and the second authentication process is performed by the authentication unit. 9. Imaging device. The imaging apparatus according to claim 8, wherein the control unit controls the authentication unit to execute the first authentication process and the second authentication process in a time-sharing manner. The control unit controls the frequency at which the authentication unit executes the first authentication process to be higher than the frequency at which the authentication unit executes the second authentication process. Alternatively, the imaging apparatus according to claim 9. A display unit for displaying an image obtained by imaging by the first imaging unit or the second imaging unit;
The imaging apparatus according to any one of claims 1 to 10, wherein the second imaging unit is fixed to the display unit. The imaging device according to claim 11, wherein the display unit is movable. An imaging apparatus control method comprising: a first imaging unit that images a subject; and a second imaging unit that images a photographer who operates the first imaging unit as a subject.
Executing a first authentication process that is a face authentication process of a subject imaged by the first imaging means or a second authentication process that is a face authentication process of a subject imaged by the second imaging means; ,
And a control step of controlling the second authentication process in accordance with the inclination of the second imaging means.

Images