JP2011091750A - Stereoscopic image pick-up apparatus and method of controlling the same - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a stereoscopic image pick-up apparatus which has superior user friendliness. <P>SOLUTION: In the stereoscopic image pick-up apparatus includes a left-eye imaging section, a right-eye imaging section, and an imaging-mode switching means for performing switching between a stereoscopic image photographing mode, during which picked-up image data of the left-eye imaging section and picked-up image data of the right-eye imaging section are associated with each other to be made into stereoscopic image data of a subject, a two-dimensional image photographing mode during which picked-up image data, obtained by photographing at any one of the left-eye imaging section and the right-eye imaging section are turned into two-dimensional image data of the subject, when it is decided that the a main subject is included in the subject, the stereoscopic image photographing mode (step S7) is selected; and when it is decided that the main subject is not included in the subject, two-dimensional image photographing mode (step S3) is selected. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to an imaging apparatus that can capture a stereoscopic image (three-dimensional image) and a control method thereof, and more particularly, to a stereoscopic image imaging apparatus that is user-friendly and a control method thereof.

  2. Description of the Related Art Conventionally, as described in Patent Documents 1 and 2 below, for example, there are imaging devices that can capture a stereoscopic image. Stereoscopic image pickup devices are usually equipped with left and right binocular cameras, and the same subject is shot simultaneously with left and right binocular cameras, and image data shot with a right eye camera and image data shot with a left eye camera are supported. It is designed to be stored on storage media.

JP 2009-47497 A JP 2009-47496 A

  A conventional stereoscopic image capturing apparatus has been developed to capture a stereoscopic image of a subject. Therefore, any stereoscopic subject can be captured by a left and right two-lens camera as a stereoscopic image. It has become.

  For this reason, for example, even a landscape photograph of a distant view is taken with a left and right binocular camera, but actually, even if two pieces of image data taken with a left and right binocular camera are combined, Since there is no parallax, it does not become a stereoscopic image.

  Such an image can be taken as a two-dimensional image from the beginning, so that a clear image can be taken. However, the conventional stereoscopic image pickup device does not assume such a case.

  SUMMARY OF THE INVENTION An object of the present invention is to provide a user-friendly stereoscopic image capturing apparatus that automatically switches between capturing a stereoscopic image and capturing a two-dimensional image in accordance with a subject without being particularly conscious of the camera user. And providing a control method thereof.

  A stereoscopic image capturing apparatus according to the present invention relates a left-eye imaging unit, a right-eye imaging unit, captured image data of the left-eye imaging unit, and captured image data of the right-eye imaging unit, and A stereoscopic image capturing mode for converting to stereoscopic image data, and a two-dimensional image capturing mode in which captured image data captured by one of the left-eye imaging unit and the right-eye imaging unit is used as the two-dimensional image data of the subject. The photographing mode switching means for switching, and when the subject is determined and it is determined that the main subject is included in the subject, the stereoscopic image shooting mode is selected and the main subject is not included in the subject. And control means for selecting the two-dimensional image capturing mode when it is determined.

  The method for controlling a stereoscopic image capturing apparatus according to the present invention relates a left-eye imaging unit, a right-eye imaging unit, captured image data of the left-eye imaging unit, and captured image data of the right-eye imaging unit. 3D image shooting mode in which the 3D image shooting mode is used to obtain the 3D image data of the subject, and the captured image data captured by any one of the left-eye imaging unit and the right-eye imaging unit is used as the two-dimensional image data of the subject. A method of controlling a stereoscopic image pickup apparatus comprising a shooting mode switching means for switching between modes, wherein the stereoscopic image shooting mode is selected when the subject is determined and a main subject is included in the subject. The two-dimensional image capturing mode is selected when it is determined that the main subject is not included in the subject.

  According to the present invention, it is possible to automatically switch between a 3D (three-dimensional image: stereoscopic image) photographing mode and a 2D (two-dimensional image) photographing mode in accordance with the state of the main subject without a troublesome operation by the user. , Improve usability.

1 is an external perspective view of a stereoscopic image capturing apparatus according to an embodiment of the present invention. FIG. 2 is an internal functional block diagram of the stereoscopic image capturing apparatus shown in FIG. 1. It is a flowchart which shows the automatic switching processing procedure of 3D imaging | photography mode / 2D imaging | photography mode which concerns on one Embodiment of this invention. It is explanatory drawing which determines whether it is a main subject in the position in a screen. It is a flowchart which shows the automatic switching processing procedure of 3D imaging | photography mode / 2D imaging | photography mode which concerns on another embodiment of this invention. It is explanatory drawing which determines whether it is a main subject with the magnitude | size in a screen. It is a flowchart which shows the automatic switching process procedure of 3D imaging | photography mode / 2D imaging | photography mode which concerns on another embodiment of this invention. It is explanatory drawing which determines whether it is a main subject by the number in a screen. It is a flowchart which shows the automatic switching process procedure of 3D imaging | photography mode / 2D imaging | photography mode which concerns on another embodiment of this invention. It is a flowchart which shows the automatic switching process procedure of 3D imaging | photography mode / 2D imaging | photography mode which concerns on another embodiment of this invention.

  Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

  FIG. 1 is an external perspective view of a stereoscopic image capturing apparatus. The stereoscopic image capturing apparatus 10 includes a box-shaped housing 11, a right-eye imaging unit 12 </ b> R and a left-eye imaging unit 12 </ b> L provided side by side on the front of the housing 11, and left and right imaging units 12 </ b> R on the front surface of the housing 11. 12L includes a flashlight 13 provided near the center upper part, and a power switch 14, a shutter button 15, and a mode selection dial 16 provided at appropriate positions on the upper surface of the housing 11.

  A liquid crystal display screen (monitor) 35 shown in FIG. 2 is provided on the back side of the housing 11, and a through image, a mode selection screen, a menu screen, and the like are displayed.

  The right-eye imaging unit 12R includes a photographing lens 21R at the front, and the left-eye imaging unit 12L includes a photographing lens 21L at the front. An angle at which the optical axis 22R of the photographic lens 21R and the optical axis 22L of the photographic lens 21L intersect is called a convergence angle, and an actuator that controls the direction in which each of the imaging units 12R and 12L faces so that the convergence angle becomes a target value ( Drive mechanism) is built in the stereoscopic image pickup apparatus 10.

  FIG. 2 is a functional block configuration diagram inside the stereoscopic image capturing apparatus 10. The stereoscopic image capturing apparatus 10 includes an image capturing control unit 31 that controls a right eye image capturing unit (right eye camera) 12R and a left eye image capturing unit (left eye camera) 12L, and a right eye camera 12R and a left eye camera 12L. An image processing unit 32 that captures image data and performs image processing, a compression / expansion processing unit 33 that compresses or decompresses image-processed image data into a JPEG image, and the like, and temporarily stores the image data A frame memory (image memory) 34, a display control unit 36 that displays captured data and through-image data on the monitor 35, a central processing unit (CPU) 37 that performs overall control of the entire stereoscopic image capturing apparatus 10, and an internal And a memory 38, which are connected to each other via a bus 40. The CPU 37 is connected to the input unit 39 such as the power switch 14, the shutter button 15, and the mode selection dial 16 shown in FIG.

  The stereoscopic image capturing apparatus 10 further includes a driving mechanism (actuator) 41 that controls the convergence angle by driving the left eye camera 12L and the right eye camera 12R, and stereo of the captured image data of the left eye camera 12L and the right eye camera 12R. A stereo matching unit 42 that calculates the distance to the subject by performing matching processing, a face detection unit 43 that detects whether or not a “face” appears in the subject, and a captured image that has undergone image processing by the image processing unit 32 A media control unit 45 that stores data in the recording medium 44 and a switching control unit 46 that automatically switches between a 3D (three-dimensional image) photographing mode and a 2D (two-dimensional image) photographing mode are connected to the bus 40. Is done.

  The face detection unit 43 determines whether or not a face image exists in the subject, notifies the CPU 37 of the detection result, and the CPU 37 captures the subject image from the face detection result when shooting in the 3D shooting mode. Whether or not it is better to shoot in the 2D shooting mode, and outputs an instruction of the shooting mode to the switching control unit 46. Further, the CPU 37 determines whether to use the 3D shooting mode or the 2D shooting mode based on the measurement result (distance to the subject) of the stereo matching unit 42, and instructs the switching control unit 46 about the shooting mode. Output.

  Here, the 3D shooting mode refers to shooting an image of a subject using both the left and right binocular cameras 12R and 12L, performing image processing on captured image data having parallax of the left and right binocular cameras 12R and 12L, A mode in which captured image data is associated and stored in the recording medium 44 in the MPO format, which is a standard of the Camera and Imaging Products Association (CIPA), for example.

  The 2D shooting mode is a mode in which a subject image is shot using only one of the left and right binocular cameras 12R and 12L, for example, the left eye camera 12L, and the captured image data subjected to the image processing is stored in the recording medium 44. Say.

  In the stereoscopic image capturing apparatus 10 of the present embodiment, the CPU 37 that has determined the processing result of the stereo matching unit 42 or the detection result of the face detection unit 43 sets the 3D shooting mode or 2D depending on the state of the main subject in the subject. This is done by instructing the 3D / 2D shooting mode automatic switching unit 46 to select the shooting mode.

  The distance to the subject can be obtained by the stereo matching unit 42 by stereo matching processing, and when this distance is a distance at which a stereoscopic image cannot be captured by the convergence angle control, shooting is performed in the 2D shooting mode. When the distance is such that a stereoscopic image can be captured by the convergence angle control, the stereoscopic image capturing apparatus 10 automatically switches between the 3D capturing mode and the 2D capturing mode according to the following processing procedure.

  FIG. 3 is a flowchart showing a 3D mode / 2D mode automatic switching control procedure in the stereoscopic image capturing apparatus 10 described with reference to FIGS. In this flowchart, the portion for switching the 3D shooting mode / 2D shooting mode depending on the distance to the main subject obtained by the stereo matching unit 42 is omitted.

  The stereoscopic image capturing apparatus 10 according to the present embodiment statistically records past shooting records of the camera user, and determines what subject the camera user wants to shoot. If there are many people photographed from past photographing records, it is determined that the person is the main subject. When the main subject is “person”, shooting is performed in the 3D shooting mode, and when the main subject is other than a person, shooting is performed in the 2D shooting mode instead of the 3D shooting mode.

  Therefore, first, when the stereoscopic image capturing apparatus 10 is activated and the shutter button is pressed, AF (automatic focus) control is performed (step S1), and then whether or not a human “face” exists in the shooting screen is determined. Is determined (step S2).

  If the face does not exist (determination result is negative (No)), the process proceeds to step S3, the shooting mode is automatically set to the 2D shooting mode, the shooting process by 2D shooting is performed (step S4), and the shot image is generated. Processing is performed (step S5).

  If the result of determination in step S2 is that a face is present (the determination result is affirmative (Yes)), the process proceeds from step S2 to step S6, and this time the face position is within a predetermined range in the center of the shooting screen, for example, 1/3. It is determined whether it is within the range.

  As a result of the determination in step S6, when the position of the face 50 exists within the range of the central portion 1/3 of the shooting screen 51 as shown in FIG. 4A, the process proceeds from step S6 to step S7. The shooting mode is automatically set to the 3D shooting mode, an imaging process by 3D shooting is performed in step S8, and a captured image generation process is performed in step S9.

  As a result of the determination in step S6, as shown in FIG. 4B, when the position of the face 50 does not exist within the range of the center 1/3 of the shooting screen 51, the process proceeds from step S6 to step 3 to set the shooting mode. Is automatically set to the 2D shooting mode.

  After the image generation process is performed in step S5 or step S9, the process proceeds to step S10, and various signal processes (known offset process, gamma process, RGB / YC conversion) are performed on the captured and generated image data. Image processing such as processing) is performed (step S11), the processed image data is stored in the recording medium 44, and the current shooting is terminated.

  FIG. 5 is a flowchart showing a 3D mode / 2D mode automatic switching control procedure according to another embodiment instead of FIG. In the embodiment of FIG. 3, whether or not a person is the main subject is determined by whether or not the face is in the range of the center 1/3 of the screen. In this embodiment, as shown in FIG. The determination is made based on whether the size of the face in the screen 51 is equal to or greater than a predetermined threshold.

  Therefore, in the embodiment shown in FIG. 5, instead of step S <b> 6 in FIG. 3 (determination processing as to whether or not the face is in the center), the face 50 can be located at any position on the screen 51. In step S6a, it is determined whether or not the size is greater than or equal to a threshold value (for example, 2/3 of the vertical dimension of the screen 51). If a large face is captured, shooting is performed in 3D shooting mode. When the image is not captured, shooting is performed in the 2D shooting mode. Steps S1 to S11 other than step S6a are the same as those in FIG.

  FIG. 7 is a flowchart showing a 3D mode / 2D mode automatic switching control procedure according to still another embodiment instead of FIG. In the embodiment of FIG. 3, it is determined whether or not a person is the main subject by determining whether or not the face is in the range of the center 1/3 of the screen. In the embodiment of FIG. Judging by the size of the face. In contrast, in the embodiment shown in FIG. 7, as shown in FIG. 8, depending on whether or not a plurality of faces are reflected in the screen 51, even if it is not in the center of the screen, the person is the main subject. It is decided that it has become.

  Therefore, in the processing procedure of FIG. 7, step S6b is performed instead of step S6 of FIG. 3 and step S6a of FIG. 5 to determine whether or not there are a plurality of faces. It is determined that a photograph is to be taken, and shooting is performed in the 3D shooting mode. If there are not a plurality of images, shooting is performed in the 2D shooting mode. Other steps S1 to S11 other than step S6b are the same as those in FIG.

  3, 5, and 7, 3D / 2D shooting is performed according to “whether or not the face is reflected in the center”, “whether or not the face is reflected large”, and “whether or not a plurality of faces are reflected”. Although the mode is automatically switched, the 3D / 2D shooting mode may be automatically switched by combining these modes.

  FIG. 9 is a flowchart showing a 3D mode / 2D mode automatic switching control procedure according to still another embodiment. In this embodiment, the main subject is a person when one of the two determination steps S6 and S6a of “whether or not the face is in the center” or “whether or not the face is large” is affirmative. It is determined.

  That is, after it is determined in step S2 that a face has been detected, it is determined in step S6 whether or not the face is in the center. If the result is affirmative (Yes), the process proceeds to step S7 to enter the 3D shooting mode. To do.

  As a result of the determination in step S6, if the face position is not in the center (determination result is negative (No)), the process proceeds to step S6a, and it is determined whether or not the face size is greater than or equal to the threshold value. . If the determination result is affirmative (Yes), the process proceeds to step S7 to set the 3D shooting mode. If the determination result is negative (No), the process proceeds to step S3 to set the 2D shooting mode.

  FIG. 10 is a flowchart showing a 3D mode / 2D mode automatic switching control procedure according to still another embodiment. In the present embodiment, any one of the three determination steps S6, S6a, and S6b of “whether or not the face is reflected in the center”, “whether or not the face is reflected large” and “whether or not a plurality of faces are reflected”. When it is determined that at least one main subject is a person, the process proceeds to step S7 to set the 3D shooting mode.

  That is, as a result of the determination in step S6, if the face position is in the center, the process proceeds to step S7 to enter the 3D shooting mode, and if the face position is not in the center, the process proceeds to step S6a. Is determined to be greater than or equal to a threshold value.

  If a large face larger than the threshold is shown, the process proceeds to step S7 to enter the 3D shooting mode. If only a small face is shown, it is determined in step S6b whether a plurality of faces are shown. If a plurality of faces are shown, it is determined that the photo is a group photo of a plurality of people, and the process proceeds to step S7 to set the 3D shooting mode. If multiple faces are not shown, that is, if only one small face is shown at the edge of the screen, it is determined that the person is not the main subject but an irrelevant person is reflected Proceeding to S3, the 2D shooting mode is set.

  As described above, in the above-described embodiment, a camera user who often shoots a person does not need to perform a camera operation by determining whether to shoot a three-dimensional image or a two-dimensional image. Since the image pickup apparatus automatically determines and automatically switches between the 3D shooting mode and the 2D shooting mode, the number of failed photos is reduced and the usability is improved.

  In the above-described embodiment, an example in which the main subject is a person has been described, but it is needless to say that the above-described embodiment can be applied even when the main subject is not a person.

  As described above, the stereoscopic image capturing apparatus and the control method thereof according to the present embodiment include the left eye image capturing unit, the right eye image capturing unit, the captured image data of the left eye image capturing unit, and the right eye image capturing. A stereoscopic image shooting mode that associates the captured image data of a portion with the stereoscopic image data of the subject, and the captured image data captured by one of the left-eye imaging portion and the right-eye imaging portion In a stereoscopic image pickup apparatus comprising a shooting mode switching means for switching between a two-dimensional image shooting mode as image data, the stereoscopic image is determined when the subject is determined and a main subject is included in the subject. When the photographing mode is selected and it is determined that the main subject is not included in the subject, the two-dimensional image photographing mode is selected.

  Further, the stereoscopic image capturing apparatus and the control method thereof according to the embodiment are characterized in that the two-dimensional image capturing mode is selected when the distance to the main subject is a distance at which a stereoscopic image cannot be captured.

  In the stereoscopic image capturing apparatus and the control method thereof according to the embodiment, whether to set the stereoscopic image shooting mode or the two-dimensional image shooting mode based on the position, size, or number of the main subject in the subject. It is characterized by selecting.

  In addition, the stereoscopic image capturing apparatus and the control method thereof according to the embodiment are characterized by determining whether or not the main subject is a person and whether or not the main subject is in the subject by the presence or absence of a face image.

  As a result, the stereoscopic image capturing apparatus according to the present embodiment automatically switches between the 3D shooting mode and the 2D shooting mode according to the state of the main subject on the imaging apparatus side, so that the camera user can perform an appropriate operation without troublesome operations. Shooting can be performed.

  Since the stereoscopic image capturing apparatus according to the present invention automatically switches between the 3D shooting mode and the 2D shooting mode according to the state of the main subject even if the user does not perform a troublesome operation, it is possible to eliminate the failed photo. It is useful when applied to general imaging devices such as digital still cameras, digital video cameras, camera-equipped mobile phones, electronic devices with cameras such as PDAs and notebook computers, and endoscopes.

10 stereoscopic image pickup device 12R right eye image pickup unit 12L left eye image pickup unit 31 image pickup control unit 32 image processing unit (DSP)
37 Central processing unit (CPU)
42 stereo matching unit 43 face detection unit 44 recording medium 46 3D / 2D shooting mode automatic switching unit 50 face 51 shooting screen

Claims (8)

  Stereo imaging mode that associates the captured image data of the left eye imaging unit, the captured image data of the left eye imaging unit, and the captured image data of the right eye imaging unit into stereoscopic image data of the subject. And a photographing mode switching means for switching between a two-dimensional image photographing mode in which picked-up image data photographed by any one of the left-eye imaging unit and the right-eye imaging unit is used as the two-dimensional image data of the subject, and the subject The stereoscopic image shooting mode is selected when it is determined that the main subject is included in the subject, and the two-dimensional image is determined when it is determined that the main subject is not included in the subject. A stereoscopic image capturing apparatus comprising: a control unit that selects a shooting mode.   The stereoscopic image capturing apparatus according to claim 1, wherein the control unit selects the two-dimensional image capturing mode when the distance to the main subject is a distance at which a stereoscopic image cannot be captured.   3. The stereoscopic image capturing apparatus according to claim 1, wherein the control unit sets the stereoscopic image shooting mode based on a position, a size, or a number of the main subject in the subject. A stereoscopic image capturing apparatus that selects whether to set a two-dimensional image capturing mode.   The stereoscopic image capturing apparatus according to claim 3, wherein the main subject is a person and whether or not the main subject is in the subject is determined based on the presence or absence of a face image.   Stereo imaging mode that associates the captured image data of the left eye imaging unit, the captured image data of the left eye imaging unit, and the captured image data of the right eye imaging unit into stereoscopic image data of the subject. And a stereoscopic mode switching means for switching between a two-dimensional image capturing mode in which captured image data captured by one of the left-eye imaging unit and the right-eye imaging unit is used as the two-dimensional image data of the subject. A method for controlling an image pickup apparatus, wherein when the subject is determined and it is determined that the main subject is included in the subject, the stereoscopic image shooting mode is selected and the main subject is included in the subject. A method for controlling a stereoscopic image capturing apparatus that selects the two-dimensional image capturing mode when it is determined that the two-dimensional image capturing mode is not selected.   6. The method of controlling a stereoscopic image capturing apparatus according to claim 5, wherein when the distance to the main subject is a distance at which a stereoscopic image cannot be captured, the two-dimensional image capturing mode is selected. .   7. The method for controlling a stereoscopic image capturing apparatus according to claim 5 or 6, wherein the stereoscopic image capturing mode is set by the position, size, or number of the main subject in the subject. A control method of a stereoscopic image capturing apparatus for selecting whether to set an image capturing mode.   8. The method for controlling a stereoscopic image capturing apparatus according to claim 7, wherein the main subject is a person and whether or not the main subject is in the subject is determined based on the presence or absence of a face image. Control method.

Images