JP2014212404A - Imaging apparatus, control method, program, and storage medium - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide such a technique that can synthesize an image of a photographer side only in such a situation where an atmosphere at the time of imaging can be expressed effectively.SOLUTION: An imaging apparatus comprises: first imaging means for imaging a subject seen from the photographer; second imaging means for imaging a subject on the photographer side; authentication means which, when the first imaging means performs imaging, authenticates second image data imaged by the second imaging means, with authentication image data registered in advance; and control means which performs control to, when the authentication by the authentication means can be performed, associate and record the first image data imaged by the first imaging means and the second image data imaged by the second imaging means, but when the authentication cannot be performed, avoid associating and recording both of the image data.

Description

  The present invention relates to an imaging apparatus having a plurality of imaging means.

  Recent mobile phones and smartphones are equipped with a normal camera (out-camera) for photographing a subject visible to the photographer and a camera (in-camera) for photographing the photographer himself or the subject on the photographer side. In an electronic device equipped with such an out-camera and in-camera, when the shutter button is pressed, the shutter of the out-camera and the in-camera are released at the same time, and the image on the in-camera side is associated with the image on the out-camera side. Can be recorded.

  As a result, not only the image of the subject but also the image on the photographer side can be recorded. Therefore, when viewing the captured image later, the atmosphere at the time of shooting can be expressed more effectively. It is possible to leave a certain image.

  For example, Patent Document 1 describes a technique in which an out camera and a main camera can be arranged in different directions to perform shooting simultaneously, and an out camera image and an in camera image are combined in an image combining mode. .

JP 2005-073161 A

  In Patent Document 1, in the image composition mode, image composition is performed even when a stranger asks to take a picture. However, even though there is no relationship between the subject and the photographer, even if a composite image of the subject is left, the atmosphere at the time of photographing cannot be expressed effectively, and there is a high possibility that it will be a failed image. Become.

  The present invention has been made in view of the above problems, and an object of the present invention is to realize a technique capable of synthesizing an image on the photographer side only in a situation where the atmosphere at the time of photographing can be effectively expressed.

  In order to solve the above-described problems and achieve the object, an imaging apparatus according to the present invention includes a first imaging unit that captures a subject that can be seen by a photographer, a second imaging unit that captures a subject on the photographer side, Authentication means for performing authentication with pre-registered authentication image data on the second image data picked up by the second image pickup means when photographing is performed by the first image pickup means; When authentication by the authentication unit is successful, the first image data captured by the first imaging unit and the second image data captured by the second imaging unit are recorded in association with each other, If the authentication is not possible, control is performed so that the first image data captured by the first imaging unit and the second image data captured by the second imaging unit are not associated and recorded. Control means.

  According to the present invention, it is possible to synthesize an image on the photographer side only in a situation where the atmosphere during photographing can be effectively expressed.

1 is a block diagram illustrating a configuration of an imaging apparatus according to an embodiment. 6 is a flowchart illustrating a shooting operation of the imaging apparatus according to the present embodiment. 3 is a flowchart showing face authentication processing in the shooting operation of FIG. 2. 3 is a flowchart showing automatic registration processing of face authentication image data in the photographing operation of FIG. 2. The figure which illustrates the setting screen of image composition mode. FIG. 4 is a diagram illustrating an out-camera image, an in-camera image, and a playback screen that are captured by the imaging apparatus according to the embodiment. The figure which illustrates the preview screen of a synthesized image, and the setting screen for setting the presence or absence of image synthesis.

  Hereinafter, embodiments for carrying out the present invention will be described in detail. The embodiment described below is an example for realizing the present invention, and should be appropriately modified or changed according to the configuration and various conditions of the apparatus to which the present invention is applied. It is not limited to the embodiment. Moreover, you may comprise combining suitably one part of each embodiment mentioned later.

  Hereinafter, an embodiment in which the present invention is applied to an imaging apparatus such as a digital camera that captures a still image or a moving image will be described.

  <Apparatus Configuration> With reference to FIG. 1, an outline of the configuration and functions of an imaging apparatus according to an embodiment of the present invention will be described.

  In FIG. 1, an imaging apparatus 100 includes an out-camera module 101 (hereinafter, “out-camera”) as a first imaging unit that captures a subject that can be seen by a photographer, and a second that captures a subject on the photographer himself or the photographer. An in-camera module 102 (hereinafter referred to as an in-camera) is mounted as an imaging unit. Each camera module 101, 102 includes a photographic lens 103, an image sensor 104, an A / D converter 105, and an image processor 106.

  The photographing lens 103 is an imaging optical system including a zoom lens, a focus lens, and a diaphragm. The image sensor 104 is an image sensor composed of a CCD, a CMOS, or the like that converts an optical image of a subject into an electrical signal.

  The A / D conversion unit 105 includes a CDS (correlated double sampling processing) circuit that removes output noise of the image sensor 104 and a non-linear amplification circuit that performs processing before A / D conversion, and an analog output from the image sensor 104. Convert the signal to a digital signal.

  The image processing unit 106 performs resize processing such as predetermined pixel interpolation and reduction and color conversion processing on the data from the A / D conversion unit 105. The image processing unit 106 performs predetermined calculation processing using the captured image data, and the system control unit 109 performs exposure control and distance measurement control based on the obtained calculation result. Here, the image processing unit 106 includes means for calculating the luminance of the subject from the output data signal after A / D conversion, and means for extracting a signal component in a specific frequency band from the calculated luminance.

  Output data from the A / D conversion unit 105 is written into a built-in memory 107 that can be accessed at high speed via the image processing unit 106. The built-in memory 107 is an electrically erasable / recordable memory, such as an EEPROM. The built-in memory 107 stores operation constants, programs, and the like for the system control unit 109. The built-in memory 107 stores face authentication image data used for face authentication processing executed by the image authentication unit 116. Here, the program is a program for executing various flowcharts described later in the present embodiment. The built-in memory 107 has a storage capacity sufficient to store a predetermined number of still images and a predetermined time of moving images and audio.

  The image recording unit 108 includes a recording medium such as a memory card and its interface, and stores the image data transferred from the built-in memory 107.

  The built-in memory 107 also serves as an image display memory (video memory). The system control unit 109 converts image display data stored in the built-in memory 107 or the image recording unit 108 into an analog signal and supplies the analog signal to the display unit 110. Thus, the display image data in the built-in memory 107 is displayed on the display unit 110. The display unit 110 is a display device such as an LCD panel, and displays a shooting screen and a focus detection area at the time of shooting, in addition to displaying images, a GUI for assisting operations, a camera state, and the like. Further, the system control unit 109 converts the digital signal stored in the built-in memory 107 into an analog signal, and sequentially transfers and displays the digital signal on the display unit 110, thereby functioning as an electronic viewfinder (EVF) and performing through image display. .

  A system control unit 109 controls the entire imaging apparatus 100. By executing the program stored in the built-in memory 107 described above, each process of this embodiment to be described later is realized. The system control unit 109 has a RAM as a system memory for developing constants and variables for operating the CPU, programs read from the built-in memory 107, and the like. The system control unit 109 also performs display control by controlling the built-in memory 107, the image recording unit 108, the display unit 110, and the like.

  The face detection module 114 performs face detection processing on the image data processed by the image processing unit 106 at, for example, a fixed period, and includes one or more face feature information ( Face position, face size, reliability). The image authentication unit 116 performs face authentication processing for comparing and collating face information detected by the face detection module 114 with previously registered image data for face authentication. The image composition unit 115 performs a process of compositing image data captured by the in-camera 102 (hereinafter, in-camera image data) and image data captured by the out-camera 101 (hereinafter, out-camera image data). The distortion correction unit 117 performs distortion correction processing of out-camera image data in face authentication processing described later. The system control unit 109 may perform each process as the face detection module 114, the image synthesis unit 115, the image authentication unit 116, and the distortion correction unit 117.

  The operation unit 111, the mode switch 112, and the photographing switch 113 are operation units for inputting various operation instructions to the system control unit 109.

  The mode switch 112 switches the operation mode of the system control unit 109 as a mode setting unit between an image recording mode and a reproduction mode. As modes included in the image recording mode, there are an auto shooting mode, an auto scene discrimination mode, a manual mode, various scene modes for shooting settings for each shooting scene, a program AE mode, a custom mode, an image composition mode, and the like. In the image composition mode, the out-camera image data (first image data) photographed by the out-camera 101 and the in-camera image data (second image data) photographed by the in-camera 102 are combined and recorded. It is.

  The shooting switch 113 is half-pressed (shooting preparation instruction) during the operation of the shutter button, and AF (autofocus) processing, AE (automatic exposure) processing, AWB (auto white balance) processing of the out camera 101 and the in camera 102 is performed. Etc. starts. The photographing switch 113 performs a series of photographing operations from reading a signal from the image sensor 104 to writing image data in the built-in memory 107 or the image recording unit 108 when the operation of the shutter button is completed, so-called full press (shooting instruction). Start.

  Each operation member of the operation unit 111 is appropriately assigned a function for each scene by selecting and operating various function icons displayed on the display unit 110, and functions as various function buttons. Examples of the function buttons include an end button, a return button, an image advance button, a jump button, a narrowing button, and an attribute change button. For example, when a menu button is pressed, various settable menu screens are displayed on the display unit 110. The user can make various settings intuitively using the menu screen displayed on the display unit 110.

  Note that as one of the operation units 111, a touch panel capable of detecting contact with the display unit 110 is provided. The touch panel and the display unit 110 can be configured integrally. For example, the touch panel is configured such that the light transmittance does not hinder the display of the display unit 110 and is attached to the upper layer of the display surface of the display unit 110. Then, the input coordinates on the touch panel are associated with the display coordinates on the display unit 110. Thereby, it is possible to configure a GUI as if the user can directly operate the screen displayed on the display unit 110. The touch panel may be any of various types of touch panels such as a resistive film method, a capacitance method, a surface acoustic wave method, an infrared method, an electromagnetic induction method, an image recognition method, and an optical sensor method. .

  <Photographing Operation> With reference to FIG. 2, the photographing operation by the imaging apparatus of this embodiment will be described. 2 is realized by reading the program stored in the internal memory 107 into the system memory and executing it by the system control unit 109. The in-camera 102 and the out-camera 101 basically perform the same operation.

  When the photographing switch 113 of the imaging apparatus 100 is turned on, the processing in FIG. 2 is started. Here, it is assumed that the imaging apparatus 100 is set to the image composition mode in advance.

  In step S <b> 201, the system control unit 109 performs simultaneous shooting with the in-camera 102 and the out-camera 101 when the shooting switch 113 is turned on.

  In S202, the image authentication unit 116 registers the in-camera image data as face authentication processing target image data (hereinafter, authentication target image data) in advance (stored in the built-in memory 107) for face authentication. Compares and collates with data. Details of the face authentication processing will be described later with reference to FIG.

  In S203, the system control unit 109 performs processing for automatically registering the in-camera image data and the out-camera image data captured in S201 as face authentication image data by the image authentication unit 116. Here, when the in-camera image data and the out-camera image data satisfy a predetermined condition in addition to the face authentication image data manually registered by the user, registration is automatically performed as face authentication image data. Details of the automatic registration processing of the face authentication image data will be described later with reference to FIG.

  In S204, the system control unit 109 determines whether the face authentication of the in-camera image data can be performed as a result of the face authentication processing in S202. If the authentication is successful, the process proceeds to S205. If the authentication is not successful, the process proceeds to S206.

  In step S205, the system control unit 109 tags and records the in-camera image data as related to the simultaneously-captured out-camera image data, and ends the process.

  In S206, the system control unit 109 tags and records the in-camera image data as not related to the simultaneously-captured out-camera image data, and ends the process.

  <Face Authentication Process> Next, the face authentication process in S202 of FIG. 2 will be described with reference to FIG.

  In S301, the system control unit 109 refers to the face authentication image data manually registered by the user among the face authentication image data registered in advance, and an evaluation value indicating the degree of coincidence with the authentication target image data ( Hereinafter, it is determined whether there is a thing whose degree of coincidence is equal to or higher than a predetermined first threshold. Here, when the face authentication image data is out-camera image data, the distortion aberration amount is different from the in-camera image data that is the authentication target image data. For this reason, the system control unit 109 performs distortion correction processing on the face authentication image data in accordance with the distortion aberration amount of the in-camera image data by the distortion correction unit 117, and then performs comparison with the authentication target image data.

  In step S302, the system control unit 109 proceeds to step S307 if the face authentication image data with the matching degree equal to or higher than the first threshold is found to be authenticated, and proceeds to step S303 if not found.

  In step S303, the system control unit 109 refers to the face authentication image data that is automatically registered and is the in-camera image data among the face authentication image data registered in advance, and matches the authentication target image data. It is determined whether or not there is a degree equal to or greater than a predetermined second threshold. Here, the automatically registered face authentication image data may be shot with a facial expression or face angle that is not suitable for authentication as compared to the face authentication image data manually registered by the user. In order to prevent this, the second threshold value is set to a value such that the authentication accuracy is higher than the first threshold value used in the determination in S301.

  In step S304, the system control unit 109 proceeds to step S307 assuming that the face authentication image data with the matching degree equal to or higher than the second threshold is found, and proceeds to step S305.

  In step S <b> 305, the system control unit 109 refers to face authentication image data that is automatically registered and is out-of-camera image data out of pre-registered face authentication image data, and the degree of coincidence with the authentication target image data. Is determined to be greater than or equal to a predetermined third threshold. Here, similarly to S301, the distortion correction unit 117 performs distortion correction processing on the face authentication image data in accordance with the distortion amount of the in-camera image data. Further, since it is assumed that the authentication accuracy is lower than that of face authentication image data that is in-camera image data that is not subjected to distortion correction, the third threshold value is determined in S303 to prevent erroneous determination. The value is set such that the authentication accuracy is higher than the second threshold used in the above.

  In step S306, the system control unit 109 proceeds to step S307 assuming that the face authentication image data with the matching degree equal to or higher than the third threshold is found, and proceeds to step S308.

  In step S307, the system control unit 109 determines that the authentication is successful and ends the process. In step S308, the system control unit 109 determines that the authentication is failed and ends the process.

  <Automatic Registration Processing of Face Authentication Image Data> Next, the automatic registration processing of face authentication image data in S203 of FIG. 2 will be described with reference to FIG.

  4, in step S401, the system control unit 109 determines whether the target image data for automatic registration processing (hereinafter, registration target image data) is in-camera image data or out-camera image data. The process proceeds to S402, and in the case of out-camera image data, the process proceeds to S407.

  In step S402, the system control unit 109 determines whether the registration target image data has been authenticated in the face authentication process in step S202 of FIG. 2, and proceeds to step S409 if it can be authenticated, and proceeds to step S403 if it cannot be authenticated.

  In S403, the system control unit 109 refers to the candidate image data in the face authentication image data registered in advance, and the degree of coincidence with the registration target image data is greater than or equal to a predetermined fourth threshold value. To determine. If a candidate image data having a matching degree equal to or greater than a predetermined fourth threshold is found, it is assumed that the candidate image data has been authenticated, and the process proceeds to S404. The process proceeds to S405. Here, the candidate image data of the face authentication image data is in-camera image data that has been captured so far, and is image data that has not been authenticated in S202, and the same subject is captured multiple times. In this case, since it is authenticated and registered as face authentication image data, it can be said to be image data of a subject that has been photographed only once.

  In S404, the system control unit 109 determines whether or not a predetermined time or more has elapsed from when the candidate image data authenticated in S403 is captured until the registration target image data is captured. If the predetermined time has elapsed, the process proceeds to S406, and if not, the process proceeds to S409. Here, if the photographing is performed a plurality of times after a predetermined time has elapsed, it is highly likely that the photographer is not a stranger but an acquaintance.

  In step S405, the system control unit 109 stores the registration target image data as candidate image data in the built-in memory 107, and ends the process.

  In step S406, the system control unit 109 stores the candidate image data authenticated in step S403 in the built-in memory 107 as face authentication image data, and ends the process.

  In S407, the system control unit 109 refers to the face authentication image data manually registered by the user among the face authentication image data registered in advance, and the degree of coincidence with the authentication target image data is a predetermined fifth. Judge whether there is something above the threshold. If a match is found that is greater than or equal to the fifth threshold value, authentication is successful and the process proceeds to S409. If not, the process proceeds to S408.

  In step S <b> 408, the system control unit 109 refers to the face authentication image data that is automatically registered and is out-of-camera image data among the face authentication image data registered in advance, and matches the authentication target image data. It is determined whether there is a degree equal to or greater than a predetermined sixth threshold. If a match is found that is greater than or equal to the fifth threshold value, authentication is successful and the process proceeds to S409. Otherwise, the process proceeds to S410. Here, in order to prevent erroneous determination, the sixth threshold value is set to a value with higher authentication accuracy than the fifth threshold value used for the determination in S407.

  In step S409, the system control unit 109 ends the process without registering the face authentication image data.

  In S410, the system control unit 109 stores the registration target image data as face authentication image data in the built-in memory 107, and ends the process.

  <Description of Image Composition Mode> Next, the image composition mode will be described with reference to FIGS.

  As shown in FIG. 5, the user can select any one of “Auto”, “ON”, and “OFF” as the setting of the image composition mode.

  When “Auto” is selected, the image data is automatically synthesized when the in-camera image data is tagged and recorded as related to the out-camera image data in S205 of FIG.

  When “ON” is selected, the image data is always combined regardless of the presence or absence of tagging indicating the relationship between the in-camera image data and the out-camera image data.

  When “OFF” is selected, image synthesis is not always performed regardless of the presence or absence of tagging indicating the relationship between in-camera image data and out-camera image data.

  In FIG. 6, reference numeral 601 denotes an out-camera image, reference numeral 602 denotes an in-camera image, and reference numeral 603 denotes a composite image obtained by combining the in-camera image 601 and the out-camera image 602. When image composition is performed, composite image data in which a small in-camera image is arranged in the vicinity of the out-camera image is recorded in the image recording unit 108.

  However, in the image composition mode, the result of the image composition when the user selects “Auto” may be different from what the user intended. For this reason, as shown in FIG. 7A, the composite image is displayed on the preview screen after the photographing is completed, and when the composite image on the preview screen is not intended as shown in FIG. It may be possible to set whether or not to manually perform image composition.

  According to the above-described embodiment, when the in-camera image is captured at the same time as the out-camera image, the in-camera image data is compared with the face authentication image data registered in advance, and only when the in-camera can be authenticated. The image is combined with the out-camera image and recorded. Thus, a composite image of the out-camera image and the in-camera image can be generated only in a situation where the atmosphere at the time of shooting can be expressed effectively.

  [Other Embodiments] The present invention is also realized by executing the following processing. That is, software (program) for realizing the functions of the present embodiment is supplied to a system or apparatus via a network or various storage media, and a computer (or CPU, MPU, etc.) of the system or apparatus reads and executes the program. It is processing.

Claims (11)

First imaging means for photographing a subject visible to the photographer;
A second imaging means for photographing a subject on the photographer side;
Authentication means for authenticating the pre-registered authentication image data for the second image data picked up by the second image pickup means when the first image pickup means is photographed;
When authentication by the authentication unit is successful, the first image data captured by the first imaging unit and the second image data captured by the second imaging unit are recorded in association with each other. When the authentication cannot be performed, control is performed so that the first image data captured by the first imaging unit and the second image data captured by the second imaging unit are not associated and recorded. And an imaging device. Automatic registration means for automatically registering the authentication image data;
Manual registration means for manually registering the authentication image data by a user;
The authentication unit performs authentication with the authentication image data registered by the manual registration unit, and performs authentication with the authentication image data of the second image data registered by the automatic registration unit when authentication is not possible, The imaging apparatus according to claim 1, wherein authentication is performed using authentication image data of the first image data registered by the automatic registration unit when authentication is not possible. The automatic registration means registers the second image data that could not be authenticated by the authentication means as candidate image data,
The second image data newly picked up by the second image pickup means and the candidate image data can be authenticated, and a predetermined time or more has elapsed since the second image data picked up the candidate image data. The candidate image data is registered as the authentication image data when the image is taken after
3. The first image data is registered as the authentication image data when authentication with the authentication image data registered by the automatic registration unit or the manual registration unit cannot be performed. The imaging device described in 1.   If the authentication image data is the first image data, the authentication means performs authentication after performing a distortion correction process on the authentication image data in accordance with the amount of distortion of the second imaging means. The imaging apparatus according to claim 3. Detection means for detecting feature information of the subject in the first image data and feature information of the subject in the second image data;
5. The apparatus according to claim 1, wherein the authentication unit collates the feature information of the subject of the second image data with the feature information of the subject of the authentication image data registered in advance. The imaging device according to item.   6. The image processing apparatus according to claim 1, further comprising image combining means for combining the first image data and the second image data recorded in association with each other by the control means. Imaging device.   When the first image data and the second image data recorded in association with each other are recorded in association with each other, a mode in which the image synthesizing unit automatically synthesizes the first image data and the second image data Regardless of whether or not the second image data is recorded in association with each other, the mode in which image data is always synthesized by the image synthesizing unit, and the first image data and the second image data are associated with each other. The imaging apparatus according to claim 6, further comprising a selection unit that allows a user to select a mode in which image data is not combined by the image combining unit regardless of whether or not the image is recorded.   Before the first image data and the second image data are recorded in association with each other, the image processing apparatus further includes setting means for displaying a composite image and setting whether or not the user performs image composition. The imaging apparatus according to claim 7. A control method for an imaging apparatus, comprising: a first imaging unit that captures a subject that is visible to a photographer; and a second imaging unit that captures a subject on the photographer side,
An authentication step of authenticating the pre-registered authentication image data with respect to the second image data imaged by the second imaging means when the first imaging means has taken an image;
If authentication by the authentication step is successful, the first image data captured by the first imaging unit and the second image data captured by the second imaging unit are recorded in association with each other. And a control step of controlling not to record the first image data and the second image data in association with each other when the authentication cannot be performed.   A program that causes a computer to function as each unit of the imaging apparatus according to any one of claims 1 to 8.   A storage medium storing a program that causes a computer to function as each unit of the imaging apparatus according to any one of claims 1 to 8.

Images