JP5221827B1 - Stereoscopic image capturing apparatus and zoom operation control method - Google Patents

Abstract

  A digital camera (100) that captures a stereoscopic video, an optical system (110A and 110B) that forms an optical image of a subject, and a first and a first video that constitute the stereoscopic video from the optical image A CCD image sensor (150A and 150B) that generates a second viewpoint image (151A and 151B), and a controller (210) that controls a zoom operation of the optical system (110A and 110B). A first operation for executing the zoom operation at a first speed based on an amount of parallax in the two images, and a second operation for executing the zoom operation at a second speed slower than the first speed; Execute by selectively switching.

Description

  The present disclosure relates to a stereoscopic image capturing apparatus that captures a stereoscopic image and a zoom operation control method.

  Patent Document 1 discloses a method of obtaining a shift in the position of each subject in a video as a parallax amount from videos taken by two cameras and shifting the two images so that the average value of the parallax amounts becomes zero. Yes.

  Patent Document 2 discloses a method for preventing the convergence position of the main subject from changing before and after zooming in the zoom operation during shooting.

Japanese Patent Laid-Open No. 10-32840 JP 2001-218228 A

  The present disclosure provides a stereoscopic video imaging apparatus that can prevent a dangerous video from being shot as a 3D video by a zoom operation.

  A stereoscopic video imaging apparatus according to the present disclosure is a stereoscopic video imaging apparatus that captures a stereoscopic video, and configures the stereoscopic video from an optical system that forms an optical image of a subject and the optical image. An image sensor that generates first and second viewpoint images; and a control unit that controls a zoom operation of the optical system, the control unit based on a parallax amount in the first and second viewpoint images. A first operation for executing the zoom operation at a speed of 1 and a second operation for executing the zoom operation at a second speed slower than the first speed are selectively switched and executed.

  The stereoscopic video imaging apparatus according to the present disclosure is effective for suppressing a dangerous video from being shot as a 3D video by a zoom operation.

FIG. 1 is a schematic diagram illustrating a digital camera according to an embodiment. FIG. 2 is a flowchart of a video signal photographing operation in the digital camera according to the embodiment. FIG. 3 is a flowchart of the zoom operation in the 2D shooting mode according to the embodiment. FIG. 4 is a flowchart of the zoom operation in the 3D shooting mode according to the embodiment. FIG. 5 is a flowchart of the first calculation method of the safety evaluation value according to the embodiment. FIG. 6 is a diagram illustrating an example of the relationship between the parallax range and the safety evaluation value according to the embodiment. FIG. 7 is a flowchart of the second calculation method of the safety evaluation value according to the embodiment. FIG. 8 is a diagram for explaining a second calculation method of the safety evaluation value according to the embodiment. FIG. 9A is a diagram illustrating a parallax detection image according to the embodiment. FIG. 9B is a diagram illustrating a parallax detection image according to the embodiment. FIG. 10 is a flowchart of a zoom speed control method using the zoom direction according to the embodiment. FIG. 11A is a diagram illustrating a relationship between the safety evaluation value and the zoom speed when the zoom magnification decreases according to the embodiment. FIG. 11B is a diagram illustrating a relationship between the safety evaluation value and the zoom speed when the zoom magnification increases according to the embodiment. FIG. 12A is a diagram illustrating a relationship between the safety evaluation value and the zoom speed according to the modification of the embodiment. FIG. 12B is a diagram illustrating a relationship between the safety evaluation value and the zoom speed according to the modification of the embodiment. FIG. 13 is a flowchart of a zoom speed control method according to a modification of the embodiment. FIG. 14A is a diagram illustrating a relationship between the safety evaluation value and the zoom speed according to the modification of the embodiment. FIG. 14B is a diagram illustrating a relationship between the safety evaluation value and the zoom speed according to the modification of the embodiment. FIG. 15 is a flowchart of a zoom speed control method according to a modification of the embodiment. FIG. 16 is a flowchart of a zoom speed control method according to a modification of the embodiment.

  Hereinafter, embodiments will be described in detail with reference to the drawings as appropriate. However, more detailed description than necessary may be omitted. For example, detailed descriptions of already well-known matters and repeated descriptions for substantially the same configuration may be omitted. This is to avoid the following description from becoming unnecessarily redundant and to facilitate understanding by those skilled in the art.

  In addition, the inventors provide the accompanying drawings and the following description in order for those skilled in the art to fully understand the present disclosure, and are not intended to limit the subject matter described in the claims. Absent.

(Embodiment)
Hereinafter, an embodiment will be described with reference to FIGS.

[1-1. Configuration of digital camera]
First, a configuration of a digital camera that is an example of a stereoscopic video imaging apparatus according to the present embodiment will be described.

  An electrical configuration of the digital camera 100 according to the present embodiment will be described with reference to FIG. FIG. 1 is a block diagram showing a configuration of the digital camera 100.

  The digital camera 100 includes an optical system 110A and 110B, a zoom motor 120, an OIS (Optical Image Stabilizer) actuator 130, a focus motor 140, CCD (Charge Coupled Device) image sensors 150A and 150B, and a video processing unit 160. , Memory 200, controller 210, gyro sensor 220, card slot 230, memory card 240, operation member 250, zoom lever 260, liquid crystal monitor 270, internal memory 280, and mode setting button 290. Prepare.

  The optical system 110A includes a zoom lens 111A, an OIS 112A, and a focus lens 113A. The optical system 110B includes a zoom lens 111B, an OIS 112B, and a focus lens 113B. The optical system 110A forms a subject image at the first viewpoint. Further, the optical system 110B forms a subject image at a second viewpoint different from the first viewpoint.

  The zoom lenses 111A and 111B can enlarge or reduce the subject image by moving along the optical axis of the optical system. The zoom lenses 111A and 111B are controlled by the zoom motor 120.

  The OISs 112A and 112B have a correction lens that can move in a plane perpendicular to the optical axis. The OISs 112A and 112B reduce the blur of the subject image by driving the correction lens in a direction that cancels the blur of the digital camera 100. The correction lens can move from the center by a maximum of L in OIS 112A and 112B. The OISs 112A and 112B are controlled by the OIS actuator 130.

  The focus lenses 113A and 113B adjust the focus of the subject image by moving along the optical axis of the optical system 110A or 110B. The focus lenses 113A and 113B are controlled by a focus motor 140.

  The zoom motor 120 drives and controls the zoom lenses 111A and 111B. The zoom motor 120 is, for example, a pulse motor, a DC motor, a linear motor, or a servo motor. The zoom motor 120 may drive the zoom lenses 111A and 111B via a mechanism such as a cam mechanism or a ball screw. The zoom motor 120 may control the zoom lens 111A and the zoom lens 111B with the same operation.

  The OIS actuator 130 drives and controls the correction lenses in the OISs 112A and 112B in a plane perpendicular to the optical axis. The OIS actuator 130 is a planar coil or an ultrasonic motor.

  The focus motor 140 drives and controls the focus lenses 113A and 113B. The focus motor 140 is, for example, a pulse motor, a DC motor, a linear motor, or a servo motor. The focus motor 140 may drive the focus lenses 113A and 113B via a mechanism such as a cam mechanism or a ball screw.

  The CCD image sensor 150A captures the subject image formed by the optical system 110A to generate the first viewpoint image 151A. The CCD image sensor 150B generates the second viewpoint image 151B by photographing the subject image formed by the optical system 110B. The CCD image sensors 150A and 150B perform various operations such as exposure, transfer, and electronic shutter. Here, the first viewpoint image 151A and the second viewpoint image 151B are collectively referred to as a stereoscopic video.

  The video processing unit 160 generates first and second processed images by performing various processes on the first viewpoint image 151A and the second viewpoint image 151B. That is, the video processing unit 160 generates image data (hereinafter referred to as a review image) to be displayed on the liquid crystal monitor 270, or generates a video signal to be re-stored in the memory card 240. The video processing unit 160 performs various video processes such as gamma correction, white balance correction, and flaw correction on the first viewpoint image 151A and the second viewpoint image 151B.

  Furthermore, the video processing unit 160 generates the first and second compressed video signals by compressing each of the first and second processed images generated by the above processing using a compression format or the like conforming to the JPEG standard. The first and second compressed video signals are associated and recorded in the memory card 240. The first and second compressed video signals are preferably recorded on the memory card 240 using MPF (Multi-Picture Format) as a file format. Further, when the video signal to be compressed is a moving image, H.264 is used. Video compression standards such as H.264 / AVC are applied. Also, MPF video and JPEG (Joint Photographic Experts Group) images or MPEG (Moving Picture Experts Group) moving images may be recorded simultaneously.

  The video processing unit 160 can be realized by a DSP (Digital Signal Processor) or a microcomputer. The resolution of the review image may be the screen resolution of the liquid crystal monitor 270, or may be the resolution of image data that is compressed and formed by a compression format or the like conforming to the JPEG standard.

  The memory 200 functions as a work memory for the video processing unit 160 and the controller 210. The memory 200 temporarily stores, for example, the first and second processed images after being processed by the video processing unit 160 or the first viewpoint image 151A and the second viewpoint image 151B before being processed by the video processing unit 160. To accumulate. In addition, the memory 200 temporarily stores shooting conditions of the optical systems 110A and 110B and the CCD image sensors 150A and 150B at the time of shooting. The shooting conditions include subject distance, field angle information, ISO sensitivity, shutter speed, EV value (Exposure Value), F value, distance between lenses, shooting time, OIS shift amount, and the like. The memory 200 is, for example, a DRAM (Dynamic Random Access Memory) or a ferroelectric memory.

  The controller 210 is a control unit that controls the entire digital camera 100. The controller 210 compares the first viewpoint image 151A and the second viewpoint image 151B (or the first and second processed images) to calculate a parallax amount for each specific region of the image. Then, the controller 210 determines whether or not the calculated distribution of the parallax amount is within a predetermined range. The controller 210 calculates this determination result as a safety evaluation value. Here, the safety evaluation value is a value indicating the degree of safety as 3D video. A detailed calculation method will be described later.

  Here, the controller 210 detects the amount of parallax, for example, in units of 4 × 4 pixels. Note that the unit for detecting the amount of parallax may be any unit such as 8 × 8 pixels or 16 × 16 pixels. Further, the parallax amount may be any value as long as it is a value indicating a shift between both images.

  The controller 210 also controls the zoom motor 120 based on the calculated safety evaluation value. Details of this will be described later. The controller 210 can be realized by a semiconductor element or the like. The controller 210 may be configured only by hardware, or may be realized by combining hardware and software. The controller 210 can also be realized by a microcomputer or the like.

  The gyro sensor 220 is composed of a vibration material such as a piezoelectric element. The gyro sensor 220 vibrates a vibration material such as a piezoelectric element at a constant frequency, and generates angular velocity information by converting a force generated by the Coriolis force into a voltage. The controller 210 acquires angular velocity information from the gyro sensor 220, and drives the correction lenses in the OISs 112A and 112B in a direction to cancel the shaking of the digital camera 100 based on the angular velocity information. Thereby, the controller 210 corrects the camera shake given to the digital camera 100 by the user. The gyro sensor 220 may be any device that can measure at least the angular velocity information of the pitch angle. Further, when the gyro sensor 220 can measure the angular velocity information of the roll angle, it is possible to consider the rotation when the digital camera 100 moves in a substantially horizontal direction.

  A memory card 240 is detachable from the card slot 230. The card slot 230 can be mechanically and electrically connected to the memory card 240.

  The memory card 240 includes a flash memory or a ferroelectric memory therein, and can store data.

  The operation member 250 includes a release button. The release button receives a user's pressing operation. When the release button is pressed halfway, the controller 210 starts AF (autofocus) control and AE (automatic exposure) control. When the release button is fully pressed, the subject is photographed.

  The zoom lever 260 is a member that receives a zoom instruction that is an instruction to change the zoom magnification from the user.

  The liquid crystal monitor 270 is a display device capable of 2D display or 3D display of the first and second processed images (or the first viewpoint image 151A and the second viewpoint image 151B) read from the memory card 240. In addition, various setting information of the digital camera 100 is displayed on the liquid crystal monitor 270. For example, the liquid crystal monitor 270 displays an EV value, an F value, a shutter speed, ISO sensitivity, and the like, which are shooting conditions at the time of shooting.

  When 2D display is performed, the liquid crystal monitor 270 may select and display one of the first and second processed images (or the first viewpoint image 151A and the second viewpoint image 151B). The second processed image may be divided into screens and displayed horizontally or vertically, or the first and second processed images may be alternately displayed for each line.

  When 3D display is performed, the liquid crystal monitor 270 may display the first and second processed images in a frame sequential manner, or may overlay and display the first and second processed images.

  The internal memory 280 is a flash memory or a ferroelectric memory. The internal memory 280 stores a control program for controlling the entire digital camera 100 and the like.

  The mode setting button 290 is a button for setting a shooting mode when the digital camera 100 performs shooting. The “shooting mode” indicates a shooting scene assumed by the user, and includes, for example, (1) portrait mode, (2) macro mode, and (3) landscape mode. Further, the 2D shooting mode and the 3D shooting mode may be selected for each shooting scene. The digital camera 100 performs shooting by setting appropriate shooting parameters based on this shooting mode. The digital camera 100 may have a camera automatic setting mode in which automatic setting is performed. The mode setting button 290 is also a button for setting a playback mode of a video signal recorded on the memory card 240.

[1-2. Video signal shooting operation]
Hereinafter, in the digital camera 100, a zoom operation that is an operation of changing the zoom magnification of the optical systems 110A and 110B will be described.

  FIG. 2 is a flowchart for explaining a video signal photographing operation in the digital camera 100.

  FIG. 3 is a flowchart for explaining a zoom operation in the 2D shooting mode.

  FIG. 4 is a flowchart for explaining a zoom operation in the 3D shooting mode.

  First, when the shooting mode is changed by the user operating the mode setting button 290, the digital camera 100 acquires the changed shooting mode (S101).

  The controller 210 determines whether the acquired shooting mode is the 2D shooting mode or the 3D shooting mode (S102). When the acquired shooting mode is the 2D shooting mode, the process proceeds to S111 illustrated in FIG. On the other hand, in the 3D shooting mode, the process proceeds to S121 shown in FIG.

  When the 2D mode shooting mode is selected (2D shooting mode in S102), first, the controller 210 detects the occurrence of a zoom instruction, which is an instruction to change the zoom magnification, by operating the zoom lever 260 (S111). ). When the zoom lever 260 is operated, the controller 210 acquires a zoom instruction and starts a zoom operation (Yes in S111). If the zoom lever 260 is not operated and there is no zoom instruction, the controller 210 waits as it is (No in S111).

  When acquiring a zoom instruction (Yes in S111), the controller 210 determines whether or not the zoom magnification can be changed by the zoom instruction (S112). Specifically, when the current zoom magnification is the maximum and the zoom instruction is an instruction to increase the zoom magnification (No in S112), it is determined that the zoom magnification cannot be changed, and the process returns to waiting for the zoom instruction (S111). . If the zoom magnification is the minimum value and the zoom instruction is an instruction to lower the zoom magnification (No in S112), it is determined that the zoom magnification cannot be changed, and the process returns to waiting for the zoom instruction (S111). In cases other than the above, it is determined that the zoom magnification can be changed (Yes in S112).

  If the zoom magnification can be changed (Yes in S112), the controller 210 executes the zoom operation at the normal speed (first speed) (S113). Specifically, the controller 210 drives the zoom motor 120 so that the zoom magnification changes at a predetermined change rate in the enlargement or reduction direction designated by the zoom lever 260. Hereinafter, this zoom magnification change rate, that is, the speed at which the zoom magnification is changed is referred to as zoom speed.

  In the 2D shooting mode, unlike in the 3D shooting mode, it is preferable that the zoom speed is always constant regardless of shooting conditions or obtained images.

  On the other hand, when the 3D shooting mode is selected (3D shooting mode in S102), first, the controller 210 detects the occurrence of a zoom instruction due to the operation of the zoom lever 260 (S121). When the zoom lever 260 is operated, the controller 210 acquires a zoom instruction and starts zoom processing (Yes in S121). If the zoom lever 260 is not operated and there is no zoom instruction, the controller 210 stands by (No in S121).

  When acquiring a zoom instruction (Yes in S121), the controller 210 determines whether or not the zoom magnification can be changed by the zoom instruction (S122). The details of this process are the same as in S112. If the change is impossible (No in S122), the process returns to S121.

  When the zoom magnification can be changed (Yes in S122), the controller 210 calculates the parallax amount between the left and right images (the first viewpoint image 151A and the second viewpoint image 151B, or the first and second processed images). The safety evaluation value is calculated using the calculated parallax amount.

  Next, the controller 210 determines the zoom speed based on the calculated safety evaluation value. Specifically, the controller 210 determines whether or not the safety evaluation value is equal to or greater than a predetermined threshold (S124). If the controller 210 determines that the safety evaluation value is equal to or greater than the threshold value and the safety is high, the controller 210 sets the speed at which the zoom magnification changes to the normal speed (first speed). Then, the controller 210 changes the zoom magnification at the normal speed (S125). Specifically, the controller 210 drives the zoom motor 120 so that the zoom magnification changes in the enlargement or reduction direction designated by the zoom lever 260.

  Here, the normal speed is the same as the zoom speed in the 2D shooting mode, for example. The normal speed may be slower than the zoom speed in the 2D shooting mode.

  On the other hand, when the controller 210 determines that the safety evaluation value is less than the threshold and the safety is low, the controller 210 sets the zoom speed to a low speed (second speed). The second speed is slower than the first speed. Then, the controller 210 changes the zoom magnification at a low speed (second speed) (S126). Specifically, the controller 210 drives the zoom motor 120 so that the zoom magnification changes in the enlargement or reduction direction designated by the zoom lever 260.

  The relationship between the safety evaluation value and the zoom speed will be described later. At this time, it is also effective to display a display corresponding to the zoom speed in the viewfinder or on the back monitor, or to notify the user by sound. For example, when the safety evaluation value is low (the amount of parallax is not safe) and a low zoom speed is used, the digital camera 100 displays an icon indicating that in the viewfinder or on the back monitor screen. It is also effective to display or emit a warning sound.

[1-2-1. Determination method of safety evaluation value]
Next, the safety evaluation value calculated by the controller 210 will be described. Below, two calculation methods are demonstrated.

  FIG. 5 is a flowchart of the first calculation method of the safety evaluation value.

  First, the controller 210 compares the two images acquired from the CCD image sensors 150A and 150B to calculate a parallax amount for each minute region of the image (S131).

  This minute region is a region including a plurality of pixels such as 8 × 8 pixels. Note that the minute region may be a region including only one pixel. In addition, the controller 210 may obtain a plurality of characteristic pixels in the image as feature points, obtain correspondence between the feature points between images, and obtain the amount of parallax from the position shift.

  Here, when the amount of parallax is obtained with the direction of popping out as the + direction, the largest amount of parallax (the maximum value of the amount of parallax) among the plurality of parallax amounts indicates the amount of parallax of the subject on the most popping out side. On the other hand, the minimum value of the amount of parallax indicates the amount of parallax of the subject at the most depth side.

  Next, the controller 210 acquires the maximum parallax amount and the minimum parallax amount among the parallax amounts calculated in S131. That is, the controller 210 obtains the parallax amount of the subject that is closest to the protrusion side and the parallax amount of the subject that is closest to the depth side. Even when the direction in which the amount of parallax pops out is determined as −, the relationship between the magnitude of the amount of parallax and the direction of the pop-out and the depth is only reversed. The amount of parallax is acquired. Then, the controller 210 calculates a parallax range that is a difference between the acquired maximum parallax amount and the minimum parallax amount (S132). This parallax range indicates the difference in parallax between the closest subject and the farthest subject. Further, it is known that when this parallax range is large, it is difficult for the viewer to see and causes fatigue.

  Next, the controller 210 calculates a safety evaluation value using the parallax range calculated in S132 (S133). Specifically, the controller 210 sets a value obtained by inputting the parallax range into a predetermined monotonously decreasing function as the safety evaluation value. An example of this monotonically decreasing function is shown in FIG. In FIG. 6, when the parallax range is within a predetermined range, the safety evaluation value is 1.0, which means that the safety evaluation value is sufficiently safe. When the parallax range exceeds this range, the safety evaluation value decreases according to the excess amount, and finally becomes 0.0 which means that it is the most dangerous state. In this example, the monotone decreasing function is a continuous function, but may be a discrete function that changes stepwise. Further, in this example, the range of values that the safety evaluation value can take is 0.0 to 1.0, but is not limited to this, and may be 0.0 to -∞. In the above description, the safety evaluation value is a value obtained by inputting the parallax range into the monotonously decreasing function, but may be the parallax range itself. Further, the controller 210 may calculate the safety evaluation value by using the difference between the parallax angle of the subject having the maximum parallax amount and the parallax angle of the subject having the minimum parallax amount. In other words, any safety evaluation value may be used as long as it is a value that becomes smaller as the degree of difficulty in viewing or feeling of fatigue when the viewer views the video.

  Next, a second calculation method of the safety evaluation value will be described. FIG. 7 is a flowchart of the second calculation method of the safety evaluation value.

  First, the controller 210 calculates the amount of parallax as in S131 shown in FIG. 5 (S141).

  And the controller 210 calculates the parallax histogram of an input image by counting the appearance frequency for every parallax amount (S142). That is, in the parallax histogram, the horizontal axis is the amount of parallax, and the vertical axis is the number of small regions (appearance frequency) for which the amount of parallax is calculated.

  Next, the controller 210 counts the number of parallaxes included in the safe range, which is a parallax range that can be safely viewed, and the number of parallaxes not included in the safe range (S143). FIG. 8 is a diagram illustrating an example of the parallax histogram 301 calculated in S142. If the parallax is within the safe range 302, it means that the parallax is regarded as safe. Therefore, counting the number of parallax included in the safe range 302 means calculating the sum (area) of the frequencies of the region 303. On the other hand, counting the number of parallaxes not included in the safe range 302 means calculating the sum (area) of the frequencies of the regions 304 on both sides. In this example, the parallax distribution is shown to spread evenly on the depth side and the retraction side with 0 as the center. In reality, this is not always the case, and the distribution may be biased to either one. In such a case, the controller 210 calculates an average value of the entire disparity amount distribution, or an average value of the maximum disparity and the minimum disparity, and uses the average value so that the center of the disparity distribution is near zero. The count described above may be performed after adding an offset to the amount of parallax.

  Next, the controller 210 calculates a parallax count number Cs which is the number of parallaxes included in the safe range 302 calculated in S143, a parallax count number Cd which is the number of parallaxes not included in the safe range 302, and 1) and the safety evaluation value is calculated.

  The calculation formula of the safety evaluation value using the count numbers Cs and Cd is not limited to this, and any calculation formula may be used as long as the Cs increases, the safety evaluation value increases, and the Cd increases the safety evaluation value decreases. Anything can be used. Here, even if there is a subject having a parallax outside the safe range, if the area in the image is small, the influence on viewing is reduced. On the contrary, if the area is large, the influence on viewing is also large. As described above, the controller 210 uses the parallax count number Cs within the safe range and the parallax count number Cd outside the safe range, thereby calculating a safety evaluation value that takes these into consideration.

  Further, in the parallax amount detection processing (S131 and S141) shown in FIGS. 5 and 7, the recording images (first and second processed images, or first viewpoint image 151A and second viewpoint image 151B) are used as they are. Although an example of use is shown, it is also effective to generate a parallax detection image by correcting the angle of view of the recording image based on the zoom instruction in S121 and detect the parallax amount of the parallax detection image. . The operation in this case will be described below with reference to FIGS. 9A and 9B.

  FIG. 9A is a diagram for explaining the angle of view of the parallax amount detection image when the zoom instruction in S121 is an instruction to increase the zoom magnification.

  As shown in FIG. 9A, the controller 210 generates an image 402 by cutting out a part of the recording image 401. Here, the image 402 is an image corresponding to an image area captured when the zoom magnification is increased at a predetermined speed for a predetermined time with respect to the recording image 401.

  Next, the controller 210 generates the parallax detection image 403 by enlarging the image 402 to the original image size (the size of the recording image 401). That is, the parallax detection image 403 is an assumed image that is captured after the zoom operation is performed.

  As described above, the controller 210 can generate an assumed image after execution of the zoom operation by cropping and enlarging the image, and can detect the parallax based on the assumed image. Therefore, the controller 210 can calculate a safety evaluation value after execution of the zoom operation. In this example, the controller 210 enlarges the image after cropping the image. However, the controller 210 performs the parallax detection based on the image 402 without enlarging the image, and enlarges the obtained parallax amount with the enlargement ratio of the image. However, similar results can be obtained.

  FIG. 9B is a diagram for describing a parallax amount detection image when the zoom instruction in S121 decreases the zoom magnification.

  As illustrated in FIG. 9B, the controller 210 acquires an image 404 in which a wider range than the recording image 405 is captured. Here, the image 404 is an image corresponding to an image area captured when the zoom magnification is reduced at a predetermined speed for a predetermined time with respect to the recording image 405. For example, the CCD image sensors 150A and 150B generate images in which an area wider than the recording image 405 is captured. Then, the video processing unit 160 generates a recording image 405 by cutting out a part of the image. In addition, the controller 210 generates an image 404 using the image.

  Next, the controller 210 generates the parallax detection image 406 by reducing the obtained image 404 to the original image size (the size of the recording image 405). That is, the parallax detection image 406 is an assumed image that is captured after the zoom operation is performed.

  In this way, by providing a margin in the recording image 405 in advance so that an image of a larger area than the recording image 405 can be acquired from the CCD image sensors 150A and 150B, the controller 210 can assume the post-zoom operation. An image can be generated and parallax can be detected based on the assumed image. Therefore, the controller 210 can calculate the safety evaluation value after execution of the zoom operation. Similar to zooming in, instead of reducing the image 404 for parallax detection, the same effect can be obtained by performing parallax detection using the image 404 of the same size and reducing the obtained parallax amount at the same rate. Is obtained.

  As described above, when the zoom magnification increases, the controller 210 switches the zoom speed based on the parallax amount in the video obtained by enlarging a part of the stereoscopic video. In addition, when the zoom magnification decreases, the controller 210 switches the zoom speed based on the amount of parallax in an image obtained by reducing an image obtained by capturing a wider range than a stereoscopic image.

[1-2-2. How to determine the zoom speed]
Hereinafter, a method for determining the zoom speed based on the safety evaluation value calculated by the above-described method will be described.

  FIG. 10 is a flowchart of processing for changing the zoom magnification.

  First, the controller 210 determines whether the zoom magnification increases (telephoto) or decreases (wide angle) according to the zoom instruction (S151).

  When the zoom magnification decreases (No in S151), the controller 210 changes the zoom magnification at the normal speed (first speed) (S153).

  FIG. 11A is a diagram showing the relationship between the safety evaluation value and the zoom speed in this case. As shown in FIG. 11A, when zooming to the wide-angle side, the zoom operation is performed at the same speed V1 regardless of the safety evaluation value.

  On the other hand, when the zoom magnification increases (Yes in S151), the controller 210 determines the zoom speed based on the safety evaluation value, and changes the zoom magnification at the determined zoom speed (S152). As a method of changing the zoom magnification based on the safety evaluation value, for example, S123 to S126 shown in FIG. 4 can be used. FIG. 11B is a diagram showing the relationship between the safety evaluation value and the zoom speed in this case. As shown in FIG. 11B, when the safety evaluation value is larger than T1, the zoom operation is performed at the first speed V1, and when the safety evaluation value is smaller than T1, the first speed V1. The zoom operation is performed at a slower second speed V2. Note that the zoom speed V1 in S153 may be different from the zoom speed V1 when the safety evaluation value in S152 is large.

  Here, in general, the higher the zoom ratio, the greater the parallax, and the higher the possibility of entering a dangerous state. Therefore, the controller 210 reduces the zoom speed because the possibility of a dangerous parallax state increases when zooming to the telephoto side. On the other hand, when zooming to the wide-angle side, even if the current state is a dangerous parallax state, zooming reduces the zoom magnification and increases the possibility of shifting to a safe parallax state. Therefore, the controller 210 does not change the zoom speed.

  Although an example in which the zoom speed is changed in two steps as shown in FIG. 11B is shown here, the zoom speed may be changed in three steps as shown in FIG. 12A. Further, the zoom speed may be changed in four steps or more.

  Further, as shown in FIG. 12B, the zoom speed may continuously change in accordance with the safety evaluation value. In FIG. 12B, the zoom speed decreases linearly, but the zoom speed may decrease curvilinearly.

  Further, as a method of changing the zoom magnification based on the safety evaluation value, the following method may be used.

  FIG. 13 is a flowchart of processing for changing the zoom magnification based on the safety evaluation value in this case.

  The controller 210 determines whether the safety evaluation value is greater than or equal to the first threshold, less than the first threshold and greater than or equal to the second threshold, or less than the second threshold (S161 and S162).

  When the safety evaluation value is equal to or greater than the first threshold (Yes in S161), the controller 210 changes the zoom magnification at the normal speed (S163).

  If the safety evaluation value is less than the first threshold and greater than or equal to the second threshold (No in S161 and Yes in S162), the controller 210 changes the zoom magnification at a low speed (S164). In this case, the controller 210 may determine the zoom speed according to the safety evaluation value, and change the zoom magnification at the determined zoom speed.

  When the safety evaluation value is less than the second threshold (No in S161 and No in S162), the controller 210 prohibits changing the zoom magnification and does not change the zoom magnification (S165).

  FIG. 14A is a diagram showing the relationship between the safety evaluation value and the zoom speed in this case. As shown in FIG. 14A, while the safety evaluation value is high, the zoom speed is the first speed V1. When the safety evaluation value becomes smaller than the first threshold value T1, the zoom speed is set to the second speed V2. Further, when the safety evaluation value becomes smaller than the second threshold value T2, the zoom speed is set to zero. That is, the controller 210 stops the zoom operation.

  As illustrated in FIG. 14B, when the safety evaluation value is T1 to T2, the zoom speed may be gradually changed according to the safety evaluation value.

  In the example shown in FIGS. 14A and 14B, the zoom speed control is switched in each of three ranges, ie, a safety evaluation value range of 0 to T2, a range of T2 to T1, and a range of T1 to 1. Not limited to this, the speed control may be switched in two ranges or four ranges.

  In the above description, the zoom speed is controlled using the safety evaluation after determining the zoom direction that is the zoom magnification changing direction (telephoto side, wide angle side). A sex evaluation value may be determined. For example, the controller 210 may set the safety evaluation value to 1 regardless of the amount of parallax when zooming to the wide angle side. Even with such a method, the same processing as described above can be realized.

  Further, the application of the zoom speed determination method may be switched based on the time change of the safety evaluation value, instead of switching between zooming toward the wide angle side and zooming toward the telephoto side.

  FIG. 15 is a flowchart of the zoom speed determination process in this case.

  First, the controller 210 determines whether or not the safety evaluation value has decreased (S171). Specifically, the controller 210 samples and stores the safety evaluation value at predetermined time intervals. Then, the controller 210 determines whether or not the latest N safety evaluation values tend to decrease.

  If the safety evaluation value does not change or tends to increase (the disparity tends to be safe) (No in S171), the controller 210 zooms at a constant speed (eg, normal speed) regardless of the current safety evaluation value. The magnification is changed (S173). On the other hand, if the safety evaluation value tends to decrease (parallax is becoming dangerous) (Yes in S171), the controller 210 changes the zoom magnification at a speed based on the safety evaluation value (S172). This process is the same as S152 shown in FIG. 10, for example. Thereby, the controller 210 can directly control the zoom speed based on the change in the parallax of the image, so that the safety of the parallax is further improved.

  In addition, as shown in FIG. 16, when the safety evaluation value is decreasing (parallax is becoming a dangerous state) (Yes in S171), the controller 210 is at a low speed (second speed). The zoom magnification may be changed (S174).

  In the above description, the digital camera 100 includes the two optical systems 110A and 110B. However, the present disclosure can also be applied to a monocular 3D camera including only one optical system.

  In the above description, an example has been described in which the digital camera 100 calculates the safety evaluation value from the parallax amount and controls the zoom speed based on the safety evaluation value. The same processing can be realized even if the zoom speed is changed based on the amount.

  In the above description, the case where the normal zoom speed is constant has been described as an example. However, the normal zoom speed (first speed) may be changed according to a user operation. In this case, the low speed (second speed) may be any speed that is slower than the speed specified by the user's operation.

[1-3. Effect]
As described above, in the present embodiment, the digital camera 100 includes the optical systems 110A and 110B that form the optical image of the subject, and the first viewpoint image 151A and the first viewpoint image that configure the stereoscopic video image from the optical image. CCD image sensors 150A and 150B that generate a two-viewpoint image 151B, and a controller 210 that controls the zoom operation of the optical systems 110A and 110B are provided. The controller 210 performs a zoom operation of the optical systems 110A and 110B at a first speed based on the amount of parallax in the two images, and a second speed slower than the first speed. The second operation for executing the zoom operation of the optical systems 110A and 110B is selectively switched and executed.

  Thereby, the digital camera 100 can suppress a dangerous image from being captured as a 3D image by the zoom operation.

  Further, the controller 210 calculates a safety evaluation value indicating safety when the stereoscopic video is viewed based on the parallax amount, and when the safety evaluation value is larger than the first value, When the first operation is executed and the safety evaluation value is smaller than the first value, the second operation is executed.

  Accordingly, the digital camera 100 can selectively change the zoom operation based on the safety when the stereoscopic video is viewed, so that a dangerous video is captured as a 3D video by the zoom operation. Can be suppressed.

  Further, the controller 210 changes the second speed so as to become slower as the safety evaluation value becomes smaller.

  Thereby, when viewing safety of the stereoscopic image is low, it is possible to suppress a zoom operation that may be less secure and to notify the photographer that such a state is present. .

  Further, when the safety evaluation value is smaller than the first value and larger than the second value smaller than the first value, the controller 210 executes the second operation, and the safety evaluation value becomes the first safety evaluation value. If the value is smaller than 2, the zoom operation is prohibited.

  Thereby, when the viewing safety of the stereoscopic video is low, it is possible to prevent the zoom operation from being performed so that the zoom operation does not cause a more dangerous state. This also allows the photographer to be notified that a dangerous video is being shot.

  When the controller 210 increases the zoom magnification of the optical systems 110A and 110B, the controller 210 selectively executes the first operation and the second operation to decrease the zoom magnification of the optical systems 110A and 110B. The first operation is executed.

  As a result, the zoom operation is selectively switched only when zooming to the enlargement side, which may reduce the safety of viewing stereoscopic images, and the reduction without the risk of lowering the safety. When zooming to the side, it is possible not to switch the zoom operation. Thereby, the controller 210 can realize switching of the zoom operation based on a change in safety due to the zoom operation.

  Further, the controller 210 selectively switches between the first operation and the second operation based on the temporal change in the parallax amount.

  Thereby, for example, when the safety in viewing a stereoscopic video is lowered, the zoom operation can be suppressed and the photographer can be informed of such a state.

  The controller 210 executes the first operation when the time change tends to increase the safety, and executes the second operation when the safety tends to decrease.

  Thereby, when the viewing safety of the stereoscopic video is lowered, it is possible to suppress the zoom operation and to notify the photographer of such a state.

  Further, when the zoom magnification of the optical systems 110A and 110B increases, the controller 210 selects the first operation and the second operation based on the amount of parallax in an image obtained by enlarging a part of the stereoscopic image. Switch and execute.

  Thereby, the controller 210 can control the speed of the zoom operation based on the parallax amount in the video after changing the zoom magnification.

  In addition, when the zoom magnifications of the optical systems 110A and 110B are decreased, the controller 210 performs the first operation and the second operation based on the parallax amount in an image obtained by reducing an image obtained by capturing a wider range than a stereoscopic image. The operation is selectively switched and executed.

  Thereby, the controller 210 can control the speed of the zoom operation based on the parallax amount in the video after changing the zoom magnification.

  Further, the controller 210 calculates a parallax range that is a difference between the maximum parallax amount and the minimum parallax amount among a plurality of parallax amounts included in the stereoscopic video, and when the parallax range is smaller than the third value, When the first operation is executed and the parallax range is larger than the third value, the second operation is executed.

  Thereby, the digital camera 100 can suppress a dangerous image from being captured as a 3D image by the zoom operation.

  Further, the controller 210 counts the number of parallax amounts included in a predetermined range among a plurality of parallax amounts included in the stereoscopic video, and the counted number of parallax amounts is larger than the fourth value. In this case, the first operation is executed, and when the counted number of parallaxes is smaller than the fourth value, the second operation is executed.

  Thereby, the digital camera 100 can control the speed of the zoom operation in consideration of the safety of the entire image.

[2. Other Embodiments]
In the embodiment, the CCD image sensors 150A and 150B have been described as examples of the imaging unit (image sensor). The imaging unit only needs to capture a subject image and generate image data. Therefore, the imaging unit is not limited to the CCD image sensors 150A and 150B. However, if CCD image sensors 150A and 150B are used as the imaging unit, the imaging unit can be obtained at a low cost. A CMOS image sensor may be used as the imaging unit. If a CMOS image sensor is used as the imaging unit, it is effective for suppressing power consumption.

  In the embodiment, the controller 210 has been described as an example of the control unit. The control unit may be physically configured as long as it controls the digital camera. If the programmable microcomputer 170 is used as the control unit, the processing contents can be changed by changing the program, so that the degree of freedom in designing the control unit can be increased. The control unit may be realized by hard logic. If the control unit is realized by hardware logic, it is effective in improving the processing speed. The control unit may be composed of one element or may be physically composed of a plurality of elements. When configured by a plurality of elements, each control described in the claims may be realized by another element. In this case, it can be considered that one control unit is constituted by the plurality of elements. Moreover, you may comprise a control part and the member which has another function with one element.

  In the digital camera described in the above embodiment, each block may be individually made into one chip by a semiconductor device such as an LSI, or may be made into one chip so as to include a part or the whole.

  Here, although LSI is used, it may be called IC, system LSI, super LSI, or ultra LSI depending on the degree of integration.

  Further, the method of circuit integration is not limited to LSI, and implementation with a dedicated circuit or a general-purpose processor is also possible. An FPGA (Field Programmable Gate Array) that can be programmed after manufacturing the LSI, or a reconfigurable processor that can reconfigure the connection and setting of circuit cells inside the LSI may be used.

  Further, if integrated circuit technology comes out to replace LSI's as a result of the advancement of semiconductor technology or a derivative other technology, it is naturally also possible to carry out function block integration using this technology. Biotechnology can be applied as a possibility.

  Moreover, each process of the said embodiment may be implement | achieved by hardware, and may be implement | achieved by software. Further, it may be realized by mixed processing of software and hardware. Needless to say, when the digital camera according to the above-described embodiment is realized by hardware, it is necessary to adjust timing for performing each process. In the above embodiment, for convenience of explanation, details of timing adjustment of various signals generated in actual hardware design are omitted.

  Moreover, the execution order of the processing method in the said embodiment is not necessarily restricted to description of the said embodiment, The execution order can be changed in the range which does not deviate from the summary of invention.

  In addition, division of functional blocks in the block diagram is an example, and a plurality of functional blocks can be realized as one functional block, a single functional block can be divided into a plurality of functions, or some functions can be transferred to other functional blocks May be. In addition, functions of a plurality of functional blocks having similar functions may be processed in parallel or time-division by a single hardware or software.

  Moreover, you may combine at least one part among the functions of the stereoscopic video processing apparatus which concerns on the said embodiment and its modification.

  As described above, the embodiments have been described as examples of the technology in the present disclosure. For this purpose, the accompanying drawings and detailed description are provided.

  Accordingly, among the components described in the accompanying drawings and the detailed description, not only the components essential for solving the problem, but also the components not essential for solving the problem in order to illustrate the above technique. May also be included. Therefore, it should not be immediately recognized that these non-essential components are essential as those non-essential components are described in the accompanying drawings and detailed description.

  Moreover, since the above-mentioned embodiment is for demonstrating the technique in this indication, a various change, substitution, addition, abbreviation, etc. can be performed in a claim or its equivalent range.

  The present disclosure can be applied to a stereoscopic video imaging apparatus that can prevent dangerous video from being captured as 3D video by a zoom operation. Specifically, the present disclosure is applicable to a digital still camera, a digital video camera, a broadcasting camera, a mobile phone with a camera function, a smartphone, and the like.

100 Digital camera 110A, 110B Optical system 111A, 111B Zoom lens 112A, 112B OIS
113A, 113B Focus lens 120 Zoom motor 130 OIS actuator 140 Focus motor 150A, 150B CCD image sensor 151A First viewpoint image 151B Second viewpoint image 160 Video processing unit 200 Memory 210 Controller 220 Gyro sensor 230 Card slot 240 Memory card 250 Operation member 260 Zoom lever 270 LCD monitor 280 Internal memory 290 Mode setting button 401, 405 Recording image 402, 404 Image 403, 406 Parallax detection image

Claims (12)

A stereoscopic image capturing device for capturing a stereoscopic image,
An optical system for forming an optical image of a subject;
An image sensor for generating first and second viewpoint images constituting the stereoscopic video image from the optical image;
A control unit for controlling the zoom operation of the optical system,
The control unit is configured to execute a zoom operation at a first speed based on a parallax amount in the first and second viewpoint images, and a second speed slower than the first speed. A stereoscopic video imaging apparatus that selectively switches and executes a second operation for executing a zoom operation. The control unit calculates a safety evaluation value indicating safety when the stereoscopic video is viewed based on the parallax amount, and when the safety evaluation value is larger than a first value The stereoscopic video imaging device according to claim 1, wherein the first operation is executed, and the second operation is executed when the safety evaluation value is smaller than the first value. The stereoscopic video imaging apparatus according to claim 2, wherein the control unit changes the second speed so as to become slower as the safety evaluation value becomes smaller. The controller is
When the safety evaluation value is smaller than the first value and larger than a second value smaller than the first value, the second operation is performed;
The stereoscopic image capturing apparatus according to claim 2, wherein the zoom operation is prohibited when the safety evaluation value is smaller than the second value. The controller is
When increasing the zoom magnification of the optical system, selectively executing the first operation and the second operation based on the parallax amount,
The stereoscopic image capturing apparatus according to claim 1, wherein the first operation is performed when the zoom magnification of the optical system is decreased. The stereoscopic video imaging apparatus according to claim 1, wherein the control unit selectively switches between the first operation and the second operation based on a temporal change in the parallax amount. The control unit performs the first operation when the temporal change tends to increase safety when the stereoscopic video is viewed, and the safety tends to decrease. The stereoscopic image capturing apparatus according to claim 6, wherein the second operation is executed. When the zoom magnification of the optical system is increased, the control unit selectively selects the first operation and the second operation based on a parallax amount in an image obtained by enlarging a part of the image for stereoscopic viewing. The stereoscopic image capturing apparatus according to claim 1, wherein the stereoscopic image capturing apparatus is executed by switching. When the zoom magnification of the optical system decreases, the control unit performs the first operation and the second operation based on a parallax amount in an image obtained by reducing an image obtained by capturing a wider range than the stereoscopic image. The stereoscopic image capturing apparatus according to claim 1, wherein the three-dimensional image capturing device is selectively switched and executed. The controller is
Calculating a parallax range which is a difference between a maximum parallax amount and a minimum parallax amount among a plurality of parallax amounts included in the stereoscopic video;
The stereoscopic image capturing apparatus according to claim 1, wherein when the parallax range is smaller than a third value, the first operation is performed, and when the parallax range is larger than the third value, the second operation is performed. . The controller is
Counting the number of parallax amounts included in a predetermined range among a plurality of parallax amounts of the stereoscopic video,
The stereoscopic video shooting according to claim 1, wherein when the counted number is larger than a fourth value, the first operation is performed, and when the number is smaller than the fourth value, the second operation is performed. apparatus. An optical system for forming an optical image of a subject;
An image sensor for generating first and second viewpoint images constituting the stereoscopic video image from the optical image;
A control method of a zoom operation in a stereoscopic video imaging apparatus comprising a control unit for controlling the zoom operation of the optical system,
Based on the amount of parallax in the first and second viewpoint images, the first operation for executing the zoom operation at a first speed and the zoom operation at a second speed slower than the first speed. Selectively switching to the second operation and executing.
Control method of zoom operation.

Images