JP2014049890A - Imaging apparatus, imaging method, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an imaging apparatus capable of automatically selecting a lens according to a picture to be imaged, setting added information such as AR (Augmented Reality) display and such, and setting a photographic environment and such.SOLUTION: The imaging apparatus includes: a first imaging part having optical systems for a right eye and a left eye provided by being separated by a first base line length and imaging pictures for the right eye and the left eye; a second imaging part having optical systems for the right eye and the left eye provided by being separated by a second base line length smaller than the first base line length and imaging pictures for the right eye and the left eye; and a selection part that selects at least one of the first imaging part and the second imaging part on the basis of pictures output by the first imaging part and the second imaging part.

Description

  The present invention relates to an imaging apparatus, an imaging method, and a program.

A 3D stereoscopic camera that allows a user to select two lenses to be used for shooting a stereoscopic image from the three lenses provided in the camera, and captures a close-up view, a distant view, and an intermediate view according to the selected combination of lenses. It is known (see, for example, Patent Document 1).
[Prior art documents]
[Patent Literature]
[Patent Document 1] Registered Utility Model No. 3168874

  However, in the conventional method, a lens is automatically selected according to a captured image, additional information such as AR (Augmented Reality) display is set, and a shooting environment is set. I couldn't.

  In one aspect of the present invention, a first imaging unit that includes optical systems for the right eye and the left eye that are provided apart from each other by the first baseline length, and that captures images for the right eye and the left eye, A second imaging unit having optical systems for the right eye and the left eye that are provided apart from the second baseline length smaller than the first baseline length, and that captures images for the right eye and the left eye; An imaging apparatus is provided that includes a selection unit that selects at least one of the first imaging unit and the second imaging unit based on images output from the first imaging unit and the second imaging unit.

  It should be noted that the above summary of the invention does not enumerate all the necessary features of the present invention. In addition, a sub-combination of these feature groups can also be an invention.

1 is a block diagram illustrating a configuration of an imaging apparatus 10 according to the present embodiment. The external appearance perspective view of the imaging device 10 observed from the to-be-photographed object side and the user side is shown. 5 shows a flowchart of processing at the time of shooting by the imaging apparatus 10 according to the present embodiment. An example of selection of an imaging unit, setting of an AR display position, selection of an illumination unit, and selection of a sound collection unit will be shown. An example of selection of an imaging part based on an image is shown. An example of the setting of AR display based on an image is shown. 3 shows a flowchart of processing during image reproduction of the imaging apparatus 10 according to the present embodiment.

  Hereinafter, the present invention will be described through embodiments of the invention, but the following embodiments do not limit the invention according to the claims. In addition, not all the combinations of features described in the embodiments are essential for the solving means of the invention.

  FIG. 1 is a block diagram illustrating a configuration of an imaging apparatus 10 according to the present embodiment, and FIGS. 2A and 2B are external perspective views of the imaging apparatus 10 observed from the subject side and the user side. .

(Description of Imaging Device 10)
The imaging apparatus 10 according to the present embodiment includes an imaging unit for taking a distant view and an imaging unit for taking a close-up view, and by selecting an appropriate imaging unit according to an image of a subject to be taken, a user manual It is not necessary to specify the imaging unit in As shown in FIGS. 1 and 2, the imaging device 10 includes a housing 100, a grip 102, a first imaging unit 110, a second imaging unit 120, a third imaging unit 130, a first sound collection unit 140, and a second collection. Sound unit 142, encoder 144, operation switch 146, GPS sensor 148, 3G / LTE communication circuit 150, acceleration sensor 154, direction sensor 156, storage device 158, battery 160, power supply circuit 162, first illumination unit 164, second illumination Unit 166, USB connector 168, decoder 170, display driver 172, display unit 174, speaker amplifier 176, stereo speaker 178, zoom switch 180, and processing unit 190.

  The housing 100 is, for example, a horizontally long, substantially rectangular parallelepiped member that accommodates each part of the imaging device 10 therein. The grip 102 is a member for the user to hold the imaging device 10, and is, for example, a cylindrical member provided at the lower part of the housing 100.

  The first imaging unit 110 is provided on one surface of the subject side of the housing 100 and images the subject. The first imaging unit 110 is an imaging unit that captures a distant view compared to the second imaging unit 120. For example, when the subject is 50 cm or more away from the first imaging unit 110, an optimal stereoscopic image of the subject is displayed. Take an image.

  The first imaging unit 110 is a left imaging unit 112 that is an optical system for capturing an image corresponding to the user's right eye and a right imaging that is an optical system for capturing an image corresponding to the user's left eye. Part 114. The left imaging unit 112 and the right imaging unit 114 may be an imaging unit that obtains image data by causing a light beam to enter and form an image on an element such as a CCD or a CMOS. For example, the left imaging unit 112 and the right imaging unit 114 include pixels with horizontal 3000 pixels and horizontal 2000 pixels. You can do it. The first imaging unit 110 is connected to the encoder 144 and outputs analog image signals from the left imaging unit 112 and the right imaging unit 114 to the encoder 144.

  The left imaging unit 112 and the right imaging unit 114 constitute a pair of stereo cameras and capture a still image or a moving image of a subject. Accordingly, the first imaging unit 110 captures the right-eye image and the left-eye image from the left imaging unit 112 and the right imaging unit 114, and obtains a stereoscopic image with parallax. The left imaging unit 112 and the right imaging unit 114 may each have a zoom lens. Further, the left imaging unit 112 and the right imaging unit 114 may each have an autofocus (AF) mechanism.

  The left imaging unit 112 and the right imaging unit 114 are provided apart from each other by the first baseline length. For example, the left imaging unit 112 and the right imaging unit 114 are provided at a distance in the vicinity of 64 mm, which is an eye width distance (interval between the left eye and the right eye) of a human eye, for example, 50 mm to 70 mm apart. As a result, when a stereoscopic image captured by the first imaging unit 110 is taken, a stereoscopic image that is close to the image at the time of observation can be displayed.

  The second imaging unit 120 is provided on one surface of the subject side of the housing 100 and images the subject. The second imaging unit 120 is an imaging unit that captures a closer view as compared to the first imaging unit 110. For example, when the subject is located less than 50 cm from the imaging device 10, the second imaging unit 120 captures an optimal stereoscopic image of the subject. .

  The second imaging unit 120 includes a left imaging unit 122 that is an optical system for capturing an image corresponding to the user's right eye and an optical system for the right eye that captures an image corresponding to the user's left eye. A right imaging unit 124. The left imaging unit 122 and the right imaging unit 124 are disposed inside the left imaging unit 112 and the right imaging unit 114 on the subject side surface of the imaging apparatus 10. The left imaging unit 122 and the right imaging unit 124 may be imaging means for obtaining image data by causing a light beam to enter and form an image on an element such as a CCD or a CMOS. You can do it. The second imaging unit 120 is connected to the encoder 144 and outputs analog image signals from the left imaging unit 122 and the right imaging unit 124 to the encoder 144.

  The left imaging unit 122 and the right imaging unit 124 constitute a pair of stereo cameras and capture a still image or a moving image of the subject. Accordingly, the second imaging unit 120 captures the right-eye image and the left-eye image from the left imaging unit 122 and the right imaging unit 124, and obtains a stereoscopic image with parallax. The left imaging unit 122 and the right imaging unit 124 may each have a zoom lens. Further, the left imaging unit 122 and the right imaging unit 124 may each have an autofocus (AF) mechanism.

  The left imaging unit 122 and the right imaging unit 124 are provided apart from each other by a second baseline length that is smaller than the first baseline length. For example, the interval between the left imaging unit 122 and the right imaging unit 124 may be 5 mm to 50 mm. Accordingly, when a nearby subject is photographed by the second imaging unit 120, it is possible to prevent the parallax of the stereoscopic image from becoming too large and resulting in an unnatural display. Note that the second imaging unit 120 may be detachable.

  The third imaging unit 130 is provided on one surface of the housing 100 on the user side, and images the user himself / herself. The third imaging unit 130 includes a left imaging unit 132 that is an optical system for the right eye and a right imaging unit 134 that is an optical system for the left eye. The left imaging unit 132 and the right imaging unit 134 may be imaging means for obtaining image data by causing a light beam to enter and form an image on an element such as a CCD or a CMOS. You can do it. The third imaging unit 130 is connected to the encoder 144 and outputs analog image signals from the left imaging unit 132 and the right imaging unit 134 to the encoder 144.

  The left imaging unit 132 and the right imaging unit 134 constitute a pair of stereo cameras, and image the user. Accordingly, the third imaging unit 130 captures the right-eye image and the left-eye image from the left imaging unit 132 and the right imaging unit 134, and obtains a stereoscopic image with parallax. As a result, the imaging apparatus 10 displays the user's image captured by the third imaging unit 130 to the user, thereby allowing the user to confirm the user's own image as when looking at the mirror. In addition, the position of the user with respect to the imaging device 10 can be measured.

  As an example, the interval between the left imaging unit 132 and the right imaging unit 134 is set to 32 mm, which is half of the standard eye width distance. Thereby, the convergence angle when the user observes the image captured by the left imaging unit 132 and the right imaging unit 134 with the imaging device 10 is made equal to the convergence angle when the user observes his / her face with a mirror. The user can display his / her face on the imaging device 10 in the same manner as when the user looks in the mirror.

  The first sound collection unit 140 is a pair of stereo microphones provided on one surface of the subject side of the housing 100 and collects sound and the like from the subject. The first sound collection unit 140 may include a directional microphone, and thereby, even a weak sound from a distance can be collected sufficiently. Instead of the pair of first sound collection units 140, a single first sound collection unit 140 may be provided in the housing 100. The first sound collection unit 140 is connected to the encoder 144 and outputs an analog audio signal to the encoder 144.

  The second sound collection unit 142 is a pair of stereo microphones provided on one surface of the casing 100 on the subject side, and collects sound and the like from the subject. The second sound collecting unit 142 may include an omnidirectional microphone, thereby collecting sound from all directions in the vicinity. Instead of the pair of second sound collection units 142, a single second sound collection unit 142 may be provided in the housing 100. The second sound collecting unit 142 is connected to the encoder 144 and outputs an analog audio signal to the encoder 144.

  The encoder 144 receives an analog image signal output from the first imaging unit 110, the second imaging unit 120, and the third imaging unit 130, converts this into digital image data, and inputs the digital image data to the processing unit 190. In addition, the encoder 144 converts analog audio signals from the first sound collection unit 140 and the second sound collection unit 142 into digital audio data and inputs the digital audio data to the processing unit 190.

  The operation switch 146 is provided at an arbitrary location on the imaging apparatus 10 and inputs various operations of the user. The operation switch 146 is provided on the user side, the subject side, the upper surface, the lower surface, and / or the side surface of the housing 100, for example. The operation switch 146 may be provided on the grip 102. The operation switch 146 inputs an operation including a photographing instruction, an image reproduction instruction, and power ON / OFF by the user.

  The GPS sensor 148 performs positioning detection using GPS (Global Positioning System). The GPS sensor 148 supplies information regarding the detected position of the imaging device 10 to the processing unit 190.

  The 3G / LTE communication circuit 150 is a communication circuit that performs mobile communication with other devices in accordance with 3G and LTE communication standards. The imaging device 10 may include a communication circuit that performs communication according to a communication standard other than 3G and LTE. The 3G / LTE communication circuit 150 receives transmission data to other devices from the processing unit 190 and supplies received data from other devices to the processing unit 190.

  The acceleration sensor 154 detects the inclination of the imaging device 10 by measuring gravity and acceleration acting on the imaging device 10. The acceleration sensor 154 detects the orientation of the imaging device 10 by measuring the direction of gravity. The acceleration sensor 154 supplies the detected tilt of the imaging device 10 to the processing unit 190.

  The azimuth sensor 156 detects the azimuth (east, west, north, and south) on the horizontal plane of the imaging device 10 by detecting geomagnetism.

  The storage device 158 stores captured image data and collected sound data, and stores an operation program of the processing unit 190 and other necessary information.

  The battery 160 is a member that supplies power to the imaging device 10 and may be, for example, a rechargeable secondary battery such as a lithium ion battery and a nickel metal hydride battery, or a disposable primary battery. The battery 160 may be mounted inside the grip 102 as indicated by a broken line in FIG. Instead, the battery 160 may be mounted inside the housing 100.

  The power supply circuit 162 is connected to a battery 160, supplies necessary power to each circuit of the imaging apparatus 10, and controls ON / OFF of the power supply of the imaging apparatus 10.

  The first illumination unit 164 is provided on one surface of the casing 100 on the subject side, and is an illumination for momentarily illuminating a distant subject. The first illumination unit 164 may be a flash light such as a xenon tube that outputs flash light from a discharge tube.

  The second illumination unit 166 is provided on one surface of the subject side of the housing 100 and is illumination for illuminating a nearby subject. The second illumination unit 166 may be a pair of LEDs that output flash light. A single second illumination unit 166 may be provided in the housing 100 instead of the pair of second illumination units 166.

  The USB connector 168 is a connection terminal of a USB (Universal Serial Bus) device, and is provided on the housing 100 and / or the grip 102. The USB connector 168 enables communication between the imaging device 10 and another device. Further, the USB connector 168 may be connected to a power source to charge the battery 160.

  The decoder 170 decodes the digital image data input from the processing unit 190, converts it into an analog video signal, and outputs it to the display driver 172. In addition, the decoder 170 decodes the digital audio data input from the processing unit 190, converts it into an analog audio signal, and outputs the analog audio signal to the speaker amplifier 176.

  The display driver 172 generates a pulse for driving the display unit 174 based on the analog video signal from the decoder 170 and causes the display unit 174 to display an image. The speaker amplifier 176 amplifies the analog audio signal from the decoder 170 and outputs it to the stereo speaker 178.

  The display unit 174 is, for example, a display device such as a liquid crystal display device or an organic EL display device, and is provided on one surface of the housing 100 on the user side. The display unit 174 displays the stereoscopic image captured by the first imaging unit 110, the second imaging unit 120, or the third imaging unit 130. As an example, the display unit 174 may include a lenticular lens affixed to the surface, thereby displaying a right eye image and a left eye image that have parallax with respect to the user's right eye and left eye.

  The display unit 174 may be provided with a transparent touch panel 175 on the surface thereof, whereby the user inputs various operations of the imaging device 10 by touching with a finger or the like.

  The stereo speakers 178 are a pair of speakers provided on one surface of the user side of the housing 100. Instead of the stereo speaker 178, a monaural speaker may be provided in the housing 100.

  As an example, the zoom switch 180 is a switch having a W-side button (wide-angle zoom button) and a T-side button (telephoto zoom button) provided at the bottom of the display unit 174. When the T-side button is pressed during shooting by the imaging device 10, the zoom of the first imaging unit 110 and / or the second imaging unit 120 moves to the telephoto side, or the live view image on the display unit 174 is an image. Enlarged and displayed by processing. When the W-side button is pressed during shooting, the zoom of the first imaging unit 110 and / or the second imaging unit 120 moves to the wide-angle side, or the live view image on the display unit 174 is reduced by image processing. Is displayed.

  Further, when the T-side button is pressed during reproduction of the captured image, the display unit 174 may display the image captured by the first imaging unit 110. When the W-side button is pressed during reproduction, the display unit 174 may display an image captured by the second imaging unit 120.

  The processing unit 190 includes a CPU and the like, and includes an encoder 144, an operation switch 146, a GPS sensor 148, a 3G / LTE communication circuit 150, an acceleration sensor 154, a direction sensor 156, a storage device 158, a power supply circuit 162, a first illumination unit 164, The imaging device 10 is comprehensively controlled by being connected to the second illumination unit 166, the USB connector 168, the decoder 170, the touch panel 175, and the zoom switch 180.

  The processing unit 190 receives user operation information from the operation switch 146, the touch panel 175, and the zoom switch 180. The processing unit 190 transmits / receives data to / from the 3G / LTE communication circuit 150, the storage device 158, and the USB connector 168. Further, the processing unit 190 receives various data from the GPS sensor 148, the acceleration sensor 154, and the direction sensor 156. Further, the processing unit 190 controls the supply of power from the power circuit 162 and the outputs of the first lighting unit 164 and the second lighting unit 166.

  Further, the processing unit 190 receives digital image data from the first imaging unit 110, the second imaging unit 120, and / or the third imaging unit 130 from the encoder 144. Further, the processing unit 190 receives digital audio data from the first sound collecting unit 140 and / or the second sound collecting unit 142 from the encoder 144.

  When the subject is imaged, the processing unit 190 receives the presence / absence and / or color of a person in the distant view image output from the first image capturing unit 110 and the near image output from the second image capturing unit 120 received from the encoder 144. It functions as a selection unit that selects at least one of the first imaging unit 110 and the second imaging unit 120 suitable for shooting based on the degree.

  The processing unit 190 selects an imaging unit before receiving a user's imaging instruction from the operation switch 146, and selects the first imaging unit 110 or the second imaging unit 120 according to the user's imaging instruction. An image may be captured by the imaging unit, and the captured image may be stored in the storage device 158.

  Instead, the processing unit 190 causes both the first imaging unit 110 and the second imaging unit 120 to capture an image before receiving a user's imaging instruction, and responds to the user's imaging instruction from the operation switch 146. Thus, an image captured by an imaging unit selected from the first imaging unit 110 and the second imaging unit 120 may be stored in the storage device 158.

  The processing unit 190 may store the images captured by both the first imaging unit 110 and the second imaging unit 120 in the storage device 158 in association with each other. When the processing unit 190 receives a shooting instruction from the third imaging unit 130 from the user via the operation switch 146 and / or the touch panel 175, the processing unit 190 stores the image captured by the third imaging unit 130 in the storage device 158. May be.

  Further, when the subject is imaged, the processing unit 190 causes at least one of the first illumination unit 164 and the second illumination unit 166 to emit light based on images output from the first imaging unit 110 and the second imaging unit 120. It's okay.

  Further, when capturing a moving image of the subject, the processing unit 190 uses the first sound collecting unit 140 and the second sound collecting unit received from the encoder 144 based on the images output from the first image capturing unit 110 and the second image capturing unit 120. It has a function as a recording unit that records the sound from at least one from the sound unit 142 in the storage device 158.

  At the time of imaging the subject, the processing unit 190 may output the digital image data from the encoder 144 to the decoder 170 and display the live view image on the display unit 174. In this case, the processing unit 190 selects either the image output from the first imaging unit 110 or the image output from the second imaging unit 120 based on the images output from the first imaging unit 110 and the second imaging unit 120. You may select whether to display on the display part 174 as a live view image.

  For example, the processing unit 190 may select an imaging unit that outputs a live view image based on the presence and / or saturation of a person in the images output from the first imaging unit 110 and the second imaging unit 120. When a shooting instruction is received from the user while the live view image is being displayed, the processing unit 190 may cause the imaging unit that outputs the live view image to capture the image.

  When the captured moving image or still image is reproduced, the processing unit 190 reads the digital image data from the storage device 158 and outputs the digital image data to the decoder 170. When playing back the captured moving image, the processing unit 190 outputs digital audio data to be output to the stereo speaker 178 to the decoder 170.

  As described above, according to the imaging apparatus 10 of the present embodiment, a main subject such as a person is detected from the images from the first imaging unit 110 for distant view and the second imaging unit 120 for near view, and the three-dimensional structure of the main subject is obtained. An image pickup unit that picks up an image more optimally is selected and shot. Further, the imaging apparatus 10 of the present embodiment optimizes the AR display, the illumination of the subject, and the microphone based on the detection result of the main subject. Thereby, according to this embodiment, even if a user does not select an imaging part manually, the imaging device 10 can select the imaging part suitable for imaging | photography according to a to-be-photographed object, and can set an appropriate imaging environment. it can.

(Processing flow at the time of photographing by the imaging device 10)
Next, processing at the time of shooting by the imaging apparatus 10 according to the present embodiment will be described with reference to the flowchart shown in FIG. In the following description, a case will be described in which the imaging apparatus 10 captures a still image of a subject by executing the processing from S102 to S114.

  First, in S <b> 102, the processing unit 190 detects a subject from the image of the first imaging unit 110. As an example, the processing unit 190 calculates the distance of the subject in the image according to the principle of triangulation from the displacement of the position of the subject in the left imaging unit 112 and the right imaging unit 114 of the first imaging unit 110, that is, the amount of parallax. A subject existing at a shooting target distance (for example, 50 cm to infinity) of the first imaging unit 110 for shooting a distant view from the imaging device 10 is detected. Instead, the processing unit 190 may change the focus of the left imaging unit 112 and / or the right imaging unit 114 from 50 cm to infinity and detect a focused subject from the image of the first imaging unit 110. Good.

  The processing unit 190 classifies the image detection result of the first imaging unit 110 into (1) face detection, (2) high saturation, or (3) low saturation. As an example, the processing unit 190 analyzes an image of the detected subject and, if a human face is included in the image, classifies the detection result as (1) face detection. When the detected face image does not include a human face and the saturation of the subject image is equal to or higher than a predetermined reference saturation, the processing unit 190 sets the detection result as (2) high saturation. Classify.

  Here, each pixel of the image imaged by the first imaging unit 110 is expressed by RGB mixed colors, and each color is represented by a luminance value of 8 bits (0 to 255) or the like. As an example, a block of a part of pixels (for example, a block of 400 pixels in total of 20 pixels in length and width) is extracted from an image area that defines a detected subject, and the luminance of any one of RGB colors in the block of pixels is extracted. When the average value is more than 128 than one or both of the average values of the luminances of the other two colors, the processing unit 190 may classify the detection result as (2) high saturation. According to the present embodiment, the processing unit 190 takes the average value of the luminance of a plurality of pixels (for example, 400 pixels) for each pixel region, thereby reducing the influence of noise generated when focusing on a single pixel. Can be eliminated.

  If the detected subject image does not include a human face and the saturation of the subject image is less than a predetermined reference saturation, the processing unit 190 sets the detection result to (3) low saturation. Classify as degrees. When the first imaging unit 110 does not detect the subject at a distance of 50 cm to infinity, the processing unit 190 classifies the detection result as (4) no subject detection.

  Next, in S <b> 104, the processing unit 190 detects a subject from the image of the second imaging unit 120. As an example, the processing unit 190 calculates the distance of the subject in the image according to the principle of triangulation from the displacement of the position of the subject in the left imaging unit 122 and the right imaging unit 124 of the second imaging unit 120, that is, the amount of parallax. A subject existing at a shooting target distance (for example, 0 cm to 50 cm) of the second imaging unit 120 for taking a near view from the imaging device 10 is detected. Instead of this, the processing unit 190 may change the focus of the left imaging unit 122 and / or the right imaging unit 124 from 0 cm to 50 cm and detect a focused subject from the image of the second imaging unit 120. .

  The processing unit 190 classifies the detection result of the image of the second imaging unit 120 without (1) face detection, (2) high saturation, (3) low saturation, or (4) no subject detection. The processing unit 190 may classify the image detection results of the second imaging unit 120 by the same method as described in S102. Note that steps S102 and S104 may be processed in parallel or in parallel. Thereby, the imaging device 10 can complete imaging | photography rapidly.

  Next, in S106, the processing unit 190, based on the classification of the detection result of the image of the first imaging unit 110 in S104 and the classification of the detection result of the image of the second imaging unit 120 in S106, the first imaging unit. At least one suitable for shooting is selected from 110 and the second imaging unit 120.

  As an example, the processing unit 190 has a high priority in the order of (1) face detection, (2) high saturation, (3) low saturation, and (4) no subject detection for the classification of detection results of both imaging units. As a result of comparing the priorities, an imaging unit with a higher priority is selected. For example, the processing unit 190 determines whether the first imaging unit 110 and the second imaging unit are based on whether a person is captured within the shooting target distance in each of the images output from the first imaging unit 110 and the second imaging unit 120. Among the units 120, an imaging unit that outputs an image in which a person is captured is selected as an imaging unit with high priority.

  Specifically, as shown in FIG. 4A, when a human face or the like is captured in the image output from the first imaging unit 110, the first imaging unit 110 is selected (FIG. 4 ( a) 1B, 1C, 1D) When the human face is captured in the image output by the second imaging unit 120, the second imaging unit 120 is selected (2A in FIG. 4A). 3A and 4A).

  Further, for example, the processing unit 190 gives priority to an imaging unit that outputs an image having a predetermined reference saturation or higher based on the saturations of the images output from the first imaging unit 110 and the second imaging unit 120. Is selected as an imaging unit having a high value. Specifically, as illustrated in FIG. 4A, if the image output from the first imaging unit 110 is an image including a subject with high saturation equal to or higher than the reference saturation within the shooting target distance, the first imaging unit 110. (2C and 2D in FIG. 4A), and if the image output by the second imaging unit 120 is an image that includes a subject with high saturation within the shooting target distance that is equal to or higher than the reference saturation, the second imaging unit 120 (3B and 4B in FIG. 4A).

  In addition, when neither an image pickup unit detects a person nor a high-saturation subject, but the image output by the first image pickup unit 110 includes a subject with a reference saturation less than the reference saturation, the processing unit 190 performs the first operation. When the imaging unit 110 is selected (3D in FIG. 4A), and the image output by the second imaging unit 120 includes a subject whose reference saturation is less than the reference saturation, the second imaging unit 120 is selected. (4C in FIG. 4A).

  When the priority of the detection result of the first imaging unit 110 and the detection result of the second imaging unit 120 are equal, the processing unit 190 determines that both imaging units are suitable for shooting, and both imaging units (1A, 2B, and 3C in FIG. 4A). However, the detection result of any imaging unit is (4) if the subject is not detected, there is no point in executing the imaging, and therefore the processing unit 190 does not need to select any imaging unit (FIG. 4). 4D of (a). When the imaging unit is not selected, the processing unit 190 may end the process.

  For example, as illustrated in FIG. 5A, the imaging device 10 is located in the vicinity of the imaging device 10 (for example, a position 45 cm away from the imaging device 10) within the imaging range of the first imaging unit 110 and the second imaging unit 120. A case is assumed in which a person 20 and a flower 30 located far from the imaging device 10 (for example, a position 4 m away from the imaging device 10) are included.

  In this case, the processing unit 190 detects a high-saturation flower 30 (for example, a tulip represented by average luminances R229, G0, and B30) as an object from the image of the first imaging unit 110 for distant view. In addition, the processing unit 190 detects the face of the person 20 as a subject from the image of the second imaging unit 120 for foreground.

  As a result, the processing unit 190 classifies the detection result of the image of the first imaging unit 110 as (2) high saturation, and classifies the detection result of the image of the second imaging unit 120 as (1) face detection. In this case, the processing unit 190 selects the second imaging unit 120 having a high priority (2A in FIG. 4A).

  Also, for example, as shown in FIG. 5B, within the imaging range of the first imaging unit 110 and the second imaging unit 120, in the vicinity of the imaging device 10 (for example, a position 45 cm away from the imaging device 10). A case is assumed in which a rock 40 located and a flower 30 located far from the imaging apparatus 10 (as an example, a position 4 m away from the imaging apparatus 10) are included.

  In this case, the processing unit 190 detects a high-saturation flower 30 (for example, a tulip represented by average luminances R229, G0, and B30) as an object from the image of the first imaging unit 110 for distant view. Further, the processing unit 190 detects the rock 40 (for example, a rock expressed by average luminances R140, G140, and B140) as a subject from the image of the second imaging unit 120 for foreground.

  As a result, the processing unit 190 classifies the detection result of the image of the first imaging unit 110 as (2) high saturation, and classifies the detection result of the image of the second imaging unit 120 as (3) low saturation. In this case, the processing unit 190 selects the first imaging unit 110 having a high priority (2C in FIG. 4A).

  The processing unit 190 may display the live view image from the first imaging unit 110 or the second imaging unit 120 on the display unit 174. The processing unit 190 may select from which imaging unit the live view image is to be used in the same method as the selection of the imaging unit used for imaging.

  Here, when no image capturing unit is selected, the processing unit 190 may display the live view image on the display unit 174 using one image capturing unit selected by the inclination of the image capturing apparatus 10. As an example, when the acceleration sensor 154 detects that the imaging device 10 is oriented horizontally or upward, the processing unit 190 may select the first imaging unit 110 for distant view for live view image display. . Further, when the acceleration sensor 154 detects that the imaging device 10 is facing downward, the processing unit 190 may select the second imaging unit 120 for near view for live view image display.

  Next, in S108, the processing unit 190 executes processing related to AR display of additional information. The additional information may be input in advance from the user via the touch panel 175 or the like and stored in the storage device 158. Instead, the processing unit 190 analyzes the subject image, and the name, characteristic, or other Attribute information may be added. FIG. 6A shows an example in which the name “Korean beetle” is added to the image of the insect 50, and FIG. 6B shows a place name such as “XX mountain” added to the image of the mountain 60. An example is shown.

  The processing unit 190 determines an AR display shift amount, that is, a parallax amount in the left-eye image and the right-eye image of the display unit 174 based on the images output from the first imaging unit 110 and the second imaging unit 120. And stored in the storage device 158. The processing unit 190 may determine the shift amount of the AR display similarly to the selection of the first imaging unit 110 and the second imaging unit 120 in S106.

  As an example, when the second imaging unit 120 is selected in S106, the processing unit 190 sets the shift amount of the AR display to a value smaller than the shift amount of the main subject, and displays the AR display in a distant view (FIG. 4 ( 1B, 1C, 1D, 2C, 2D and 3D in b). As a result, the imaging apparatus 10 displays the AR display in a stereoscopic manner at a distance from the user with respect to the subject displayed in the vicinity.

  In this case, as illustrated in FIG. 6A, the processing unit 190 sets the shift amount of the AR display to be smaller than the shift amount of the image of the insect 50, and displays the AR display of “Scarabae” from the second imaging unit 120. 3D images are displayed in a distant view from the image of the bug 50. Thereby, the processing unit 190 makes it easy to distinguish the AR display from the image of the insect 50 displayed in a three-dimensional manner near the user.

  As another example, when the first imaging unit 110 is selected in S106, the processing unit 190 sets the AR display shift amount to a value larger than the main subject shift amount, and displays the AR display in the foreground. (2A, 3A, 4A, 3B, 4B, and 4C in FIG. 4B). As a result, the imaging device 10 displays the AR display near the user in a stereoscopic manner with respect to a subject displayed far away.

  In this case, as illustrated in FIG. 6B, the processing unit 190 sets the shift amount of the AR display to be larger than the shift amount of the image of the mountain 60, and first displays the AR display such as “XX mountain”. Three-dimensional display in the foreground from the image of the mountain 60 from the unit 110. Thereby, the processing unit 190 makes it easy to distinguish the AR display from the image of the mountain 60 displayed stereoscopically in the distance of the user.

  In addition, when the first imaging unit 110 and the second imaging unit 120 are selected in S106, the processing unit 190 detects the shift amount of the subject detected by the first imaging unit 110 and the subject detected by the second imaging unit 120. The shift amount of the AR display is set to an intermediate value of the shift amount (1A, 2B, and 3C in FIG. 4B). As a result, the processing unit 190 prevents the AR display from being displayed in a 3D display at a position that is difficult to distinguish from any of the objects when the object is displayed in 3D both near and far from the user. Can do.

  In S108, the processing unit 190 may superimpose the AR display in which the shift amount is set on the live view image and display the AR display on the display unit 174.

  Next, in S110, the processing unit 190 determines whether or not the luminance of the detected subject is equal to or less than a predetermined value in the image from the imaging unit selected in S106. If the detected luminance of the subject is equal to or lower than a predetermined value (Y in S110), the processing unit 190 advances the process to S112, and otherwise advances the process to S114.

  When both the imaging units are selected in S106, the processing unit 190 may determine whether the luminance of the subject detected by at least one of the two imaging units is equal to or lower than a predetermined value. If no image pickup unit is selected in S106, the processing unit 190 may advance the process to S114.

  In S112, the processing unit 190 selects which of the first illumination unit 164 and the second illumination unit 166 to emit light at the time of shooting based on the images output from the first imaging unit 110 and the second imaging unit 120. To do. The processing unit 190 may determine one of the illumination units in the same manner as the selection of the first imaging unit 110 and the second imaging unit 120 in S106.

  As an example, when the first imaging unit 110 for distant view is selected in S106, the processing unit 190 emits light from the first illumination unit 164 that is a discharge tube that illuminates a distant subject (1B in FIG. 4C). 1C, 1D, 2C, 2D, 3D). As a result, when the main subject is detected at a distance, the imaging apparatus 10 can perform illumination using a discharge tube with a large amount of light, and can secure a sufficient exposure amount for the subject at a distance.

  When the second imaging unit 120 for near view is selected in S106, the processing unit 190 causes the second illumination unit 166, which is an LED for illuminating a nearby subject, to emit light (2A, 3A, 4A in FIG. 4C). 3B, 4B and 4C). Thereby, the imaging device 10 can illuminate the subject with a sufficient amount of light according to the distance of the subject with high priority, such as a human face or a flower.

  When the first imaging unit 110 and the second imaging unit 120 are selected in S106, the processing unit 190 may select the second illumination unit 166, which is an LED, and prevent the light amount of the subject at a short distance from increasing. (1A, 2B, and 3C in FIG. 4 (c)). Instead, the processing unit 190 may select both the first illumination unit 164 that is a discharge tube and the second illumination unit 166 that is an LED.

  Alternatively, the processing unit 190 (4) only when the detection result of the second imaging unit 120 does not detect the subject (when corresponding to 1D, 2D, 3D, and 4D in FIG. 4C), You may select the 1st illumination part 164 which is a discharge tube. As a result, when the subject is detected in the vicinity of the imaging device 10, the imaging device 10 does not use a discharge tube with a large amount of light, so that the subject in the vicinity is strongly exposed and the difference in brightness between the distant view and the distant view increases. Can be prevented.

  At this time, when the detection result of the first imaging unit 110 is (1) face detection and the detection result of the second imaging unit 120 is (2) high saturation (corresponding to 1B in FIG. 4C), the processing unit 190 may select the second illumination unit 166. Further, when the detection result of the second imaging unit 120 is (3) low saturation (corresponding to 1C, 2C, 3C, and 4D in FIG. 4C), the processing unit 190 includes the second illumination unit. 166 may be selected, or no illumination unit may be selected.

  Next, in S <b> 114, the processing unit 190 stands by until a user's imaging instruction is received from the operation switch 146, and in response to receiving the user's imaging instruction from the operation switch 146, the processing unit 190 and the first imaging unit 110. The image is picked up by the image pickup unit selected in S106 of the two image pickup units 120. When the process of S112 has been executed, the processing unit 190 illuminates the subject using the illumination unit selected from the first illumination unit 164 and the second illumination unit 166 during imaging. The processing unit 190 superimposes the AR display of the shift amount set in S108 on the image obtained by imaging and records it in the storage device 158.

  Note that the processing unit 190 causes both the first imaging unit 110 and the second imaging unit 120 to capture an image before receiving a user's imaging instruction, and in response to the user's imaging instruction from the operation switch 146, The image captured by the imaging unit selected in S <b> 106 among the first imaging unit 110 and the second imaging unit 120 may be stored in the storage device 158.

  When storing the captured image in the storage device 158, the processing unit 190 may superimpose additional information such as the shooting date and / or subject name information on the digital image data. As an example, the processing unit 190 superimposes additional information on the left-eye and right-eye digital image data so as to have a certain shift amount (parallax amount). Accordingly, the processing unit 190 can cause the display unit 174 to display additional information added to the image as an AR display.

  Here, the processing unit 190 changes the shift amount of the position for displaying the additional information between the right-eye image and the left-eye image based on the images output from the first imaging unit 110 and the second imaging unit 120. You can do it. As an example, the processing unit 190 selects the first image capturing unit 110 for distant view based on the images output from both image capturing units to capture an image, and sets the shift amount of the position of the additional information of the subject image. The additional information is displayed on the display unit 174 so that it is larger than the shift amount so that it can be seen near the user. In addition, when the second imaging unit 120 is selected, the processing unit 190 reduces the amount of position of the additional information to be smaller than the amount of displacement of the image of the subject, and adds the additional information so that it can be seen far from the user. It is displayed on the display unit 174.

  Note that when the imaging apparatus 10 captures a moving image, the processing unit 190 does not have to execute the processes of S110 and S112. In this case, the processing unit 190 replaces the processing of S110 and S112 with the first sound collection unit 140 and the second sound collection unit 142 based on the images output from the first imaging unit 110 and the second imaging unit 120. You may perform the process which selects the sound collection part used for recording. The processing unit 190 may determine the sound collection unit used for recording, as in the selection of the first imaging unit 110 and the second imaging unit 120 in S106.

  For example, when the first imaging unit 110 for distant view is selected in S106, the processing unit 190 records the sound from the directional first sound collecting unit 140 (1B, 1C, and 1B in FIG. 4B). 1D, 2C, 2D, and 3D). The processing unit 190 records the sound from the non-directional second sound collecting unit 142 when the second imaging unit 120 for foreground is selected (2A, 3A, 4A, 3B, 4B in FIG. 4B). And 4C).

  As a result, when the main subject is detected at a distance, the imaging apparatus 10 can efficiently collect a minute sound from a far subject to be imaged. In addition, the imaging device 10 can collect sound around the subject without omission when a main subject is detected in the vicinity of the imaging device 10.

  Note that when both imaging units are selected in S106, the processing unit 190 may select both the first sound collection unit 140 and the second sound collection unit 142 (1A, 2B, and FIG. 4B). 3C). Accordingly, the processing unit 190 can select a sound collection unit suitable for the shooting conditions.

  In S102 and S104, the processing unit 190 classifies the detection results of the images of the first imaging unit 110 or the second imaging unit 120 based on the saturation of the subject. In addition to this, or instead of this, the processing unit 190 may classify the detection results based on the detected contrast of the subject.

  As described above, the imaging apparatus 10 performs the processing of the flowchart illustrated in FIG. 3, thereby selecting the imaging unit used for shooting in S <b> 106 based on the images from the first imaging unit 110 and the second imaging unit 120. In S108, the AR display shift amount is determined, and in S112, an illumination unit or a sound collection unit used for photographing is selected. Thereby, the imaging apparatus 10 can select an imaging unit suitable for shooting according to the subject and set an appropriate shooting environment without the user manually selecting the imaging unit.

(Processing flow at the time of image reproduction of the imaging device 10)
Next, processing during image reproduction of the imaging apparatus 10 according to the present embodiment will be described with reference to a flowchart shown in FIG. The processing unit 190 receives an image reproduction instruction from the user from the operation switch 146, the touch panel 175, and the like, reads digital image data from the storage device 158, and causes the display unit 174 to display an image.

  Here, when only one of the image captured by the first imaging unit 110 and the image captured by the second imaging unit 120 is stored in the storage device 158, the imaging device 10 is stored. The displayed image is displayed on the display unit 174. When both the image captured by the first imaging unit 110 and the image captured by the second imaging unit 120 are stored in the storage device 158, the imaging device 10 executes the processes from S202 to S212. Display the captured image.

  First, in S202, the processing unit 190 causes the display unit 174 to display an image captured by the second imaging unit 120 for foreground. Instead, the processing unit 190 selects either the image captured by the first image capturing unit 110 or the image captured by the second image capturing unit 120 according to the tilt of the image capturing apparatus 10 and causes the display unit 174 to display the selected image. May be.

  As an example, when the acceleration sensor 154 detects that the imaging device 10 is oriented in the horizontal direction or the upward direction, the processing unit 190 may display the image of the first imaging unit 110 on the display unit 174. Further, when the acceleration sensor 154 detects that the imaging device 10 is facing downward, the processing unit 190 may display the image of the second imaging unit 120 on the display unit 174.

  Next, in S204, the processing unit 190 determines whether either the T-side button or the W-side button of the zoom switch 180 has been pressed. The processing unit 190 advances the process to S206 when any button of the zoom switch 180 is pressed, and waits until the button is pressed when the button is not pressed.

  Next, in S206, the processing unit 190 determines which of the T-side button and the W-side button of the zoom switch 180 has been pressed. The processing unit 190 advances the process to S208 when the T-side button of the zoom switch 180 is pressed, and advances the process to S210 when the W-side button is pressed.

  In S208, the processing unit 190 displays an image captured by the first image capturing unit 110 for distant view on the display unit 174. When the display unit 174 has already displayed the image of the first imaging unit 110, the processing unit 190 may enlarge the displayed image by image processing and display the enlarged image on the display unit 174. When the display unit 174 displays a moving image, the sound collected by the first sound collecting unit 140 is reproduced simultaneously with the reproduction of the moving image. The processing unit 190 advances the process to S212.

  In S <b> 210, the processing unit 190 displays the image captured by the second image capturing unit 120 for the near view on the display unit 174. When the display unit 174 has already displayed the image of the second imaging unit 120, the processing unit 190 may reduce the displayed image by image processing and display it on the display unit 174. When the display unit 174 displays a moving image, the sound collected by the second sound collecting unit 142 is reproduced simultaneously with the reproduction of the moving image. The processing unit 190 advances the process to S212.

  In S212, the processing unit 190 determines whether or not a reproduction end instruction from the user has been received. When the user's reproduction end instruction is received from the operation switch 146, the touch panel 175, or the like, the processing unit 190 ends the process. When the user's reproduction end instruction is received from the operation switch 146, the touch panel 175, or the like, the processing unit 190 returns the process to S204.

  The imaging apparatus 10 executes the processing of the flowchart illustrated in FIG. 7, and based on the operation of the zoom switch 180 by the user, the image of the first imaging unit 110 for the distant view and the image of the second imaging unit 120 for the foreground Can be appropriately switched. As a result, the user can arbitrarily view the desired image among the image with small parallax and the image with large parallax. Note that the processing unit 190 may switch the image of the first image capturing unit 110 and the image of the second image capturing unit 120 in an instant, or may proceed in an order with a time of about 1 second.

  As mentioned above, although this invention was demonstrated using embodiment, the technical scope of this invention is not limited to the range as described in the said embodiment. It will be apparent to those skilled in the art that various modifications or improvements can be added to the above-described embodiment. It is apparent from the scope of the claims that the embodiments added with such changes or improvements can be included in the technical scope of the present invention.

  The order of execution of each process such as operations, procedures, steps, and stages in the apparatus, system, program, and method shown in the claims, the description, and the drawings is particularly “before” or “prior to”. It should be noted that the output can be realized in any order unless the output of the previous process is used in the subsequent process. Regarding the operation flow in the claims, the description, and the drawings, even if it is described using “first”, “next”, etc. for convenience, it means that it is essential to carry out in this order. It is not a thing.

DESCRIPTION OF SYMBOLS 10 Imaging device, 20 Person, 30 Flower, 40 Rock, 50 Insect, 60 Mountains 100 Case, 102 Grip, 110 1st imaging part, 112 Left imaging part 114 Right imaging part, 120 2nd imaging part, 122 Left imaging part , 124 Right imaging unit 130 Third imaging unit, 140 First sound collection unit, 142 Second sound collection unit, 144 Encoder 146 Operation switch, 148 GPS sensor, 150 3G / LTE communication circuit 154 Acceleration sensor, 156 Direction sensor, 158 Storage device 160 Battery, 162 power supply circuit, 164 first illumination unit, 166 second illumination unit 168 USB connector, 170 decoder, 172 display driver, 174 display unit 175 touch panel, 176 speaker amplifier, 178 stereo speaker 180 zoom switch, 190 processing Part

Claims (16)

A first imaging unit that has optical systems for the right eye and the left eye provided apart from the first baseline length, and that captures images for the right eye and the left eye;
A second imaging unit having optical systems for the right eye and the left eye that are provided apart from the second baseline length smaller than the first baseline length, and that captures images for the right eye and the left eye;
A selection unit that selects at least one of the first imaging unit and the second imaging unit based on images output from the first imaging unit and the second imaging unit;
An imaging apparatus comprising:   The selection unit selects at least one of the first imaging unit and the second imaging unit based on an image output from the first imaging unit and the second imaging unit before receiving a user's imaging instruction. The imaging apparatus according to claim 1, wherein an image is captured by an imaging unit selected from the first imaging unit and the second imaging unit in accordance with a user's imaging instruction.   The said selection part makes the memory | storage device preserve | save the image imaged by the imaging part selected among the said 1st imaging part and the said 2nd imaging part according to the user's imaging instruction | indication. Imaging device.   The selection unit selects at least one of the first imaging unit and the second imaging unit based on whether a person is captured in each of the images output from the first imaging unit and the second imaging unit. The imaging device according to claim 1, which is selected.   The imaging device according to claim 4, wherein the selection unit selects an imaging unit that outputs an image of a person captured from the first imaging unit and the second imaging unit.   The selection unit selects at least one of the first imaging unit and the second imaging unit based on respective saturations of images output from the first imaging unit and the second imaging unit. The imaging device according to claim 5.   The imaging apparatus according to claim 6, wherein the selection unit selects an imaging unit that outputs an image with a predetermined reference saturation or higher among the first imaging unit and the second imaging unit.   The imaging device according to any one of claims 1 to 7, wherein the first imaging unit captures a distant view compared to the second imaging unit. A first illumination unit and a second illumination unit;
The said selection part makes at least one of the said 1st illumination part and the said 2nd illumination part light-emit based on the image which the said 1st imaging part and the said 2nd imaging part output. The imaging device according to item. The first illumination unit outputs flash light by a discharge tube,
The second illumination unit outputs flash light by an LED,
The selection unit causes the first illumination unit to emit light when the first imaging unit is selected, and causes the second illumination unit to emit light when the second imaging unit is selected. Imaging device. A first sound collection unit and a second sound collection unit;
A recording unit capable of recording sound from the first sound collecting unit and the second sound collecting unit;
With
The selection unit causes the recording unit to record sound from at least one of the first sound collection unit and the second sound collection unit based on images output from the first imaging unit and the second imaging unit. The imaging device according to any one of claims 1 to 10. The first sound collection unit has a directional microphone,
The second sound collection unit has an omnidirectional microphone,
The selection unit causes the recording unit to record a sound from the first sound collection unit when the first imaging unit is selected, and the second sound collection unit when the second imaging unit is selected. The image pickup apparatus according to claim 11, wherein the sound from is recorded by the recording unit. A display unit for displaying images;
Based on the images output from the first imaging unit and the second imaging unit, the selection unit displays either the image output from the first imaging unit or the image output from the second imaging unit on the display unit. The imaging apparatus according to any one of claims 1 to 12, wherein selection is made to display. The display unit displays additional information in addition to the image,
The selection unit changes a shift amount of a position for displaying the additional information between the right-eye image and the left-eye image based on images output from the first imaging unit and the second imaging unit. Item 14. The imaging device according to Item 13. A first imaging stage in which a first imaging unit having optical systems for right eye and left eye provided apart from the first baseline length captures images for right eye and left eye;
A second imaging unit having optical systems for the right eye and the left eye provided apart from the second baseline length smaller than the first baseline length captures a right eye image and a left eye image. Imaging stage;
A selection stage in which the selection unit selects at least one of the first imaging unit and the second imaging unit based on images output from the first imaging unit and the second imaging unit;
An imaging method comprising: Computer
A first imaging unit that has optical systems for the right eye and the left eye provided apart from the first baseline length, and that captures images for the right eye and the left eye;
A second imaging unit having optical systems for the right eye and the left eye that are provided apart from the second baseline length smaller than the first baseline length, and that captures images for the right eye and the left eye;
A selection unit that selects at least one of the first imaging unit and the second imaging unit based on images output from the first imaging unit and the second imaging unit;
A program that causes an image pickup apparatus to function.

Images