JP2012220603A - Three-dimensional video signal photography device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To solve the problem in which only a person who is experienced in three-dimensional photography can know what photography conditions a three-dimensional photograph is taken so as to obtain a desired stereoscopic feeling effect.SOLUTION: A three-dimensional video signal photography device includes: an optical system 110(a); a CCD 150(a); an optical system 110(b); a CCD 150(b) for generating video for stereoscopy; and a video processing part 160 having a function of detecting parallax information representing a parallax amount that the video for stereoscopy has, a function of adjusting photography conditions of the optical system 110(a), CCD 150(a), optical system 110(b), and CCD 150(b), and a function of setting parallax distribution information representing a distribution of determined parallax amounts. The video processing part 160 sets photography conditions of the optical system 110(a), CCD 150(a), optical system 110(b), and CCD 150(b) so that a distribution of parallax amounts represented by the detected parallax information approximates the distribution of parallax amounts represented by the parallax distribution information set by the video processing part 160.

Description

  The present invention relates to a 3D video signal photographing apparatus that photographs a stereoscopic video, for example.

  A method of reproducing a 3D video signal (hereinafter referred to as 3D viewing) by capturing left and right images with binocular parallax and displaying the captured left and right images on a display device that can independently project left and right eyes. Are known. As a method for acquiring the left and right images, a method in which two cameras are arranged side-by-side and a synchronous shooting is performed, or a method of shooting a subject image from a different viewpoint formed based on two optical systems is generally used.

  In 3D shooting using the above method, the shooting parameters of the two optical systems are adjusted in consideration of the display size and viewing distance when viewing the shot image, and the projected image is viewed directly. If the shooting is performed such that the angle of convergence, that is, the angle formed by both eyes and the subject is exactly the same, the viewer will see the same stereoscopic effect. However, such shooting will completely define the viewing environment at the time of shooting, and is impractical in practical use. Moreover, in the conventional shooting style in which the lens is changed according to the subject and the shooting scene, the optimum viewing condition changes every time the lens is changed, which is also not practical. For this reason, when viewing the subject directly and when viewing the projected image, the viewing and shooting conditions are set so that a certain effect can be obtained within a certain range, rather than strictly matching the convergence angles. A method of setting and shooting or correcting a shot 3D image is employed.

  As such a correction method, Japanese Patent Application Laid-Open No. H10-228688 calculates a position shift of each subject in a video as parallax from videos taken by two cameras, and shifts the two images so that the average value of the parallax becomes zero. Is disclosed. In 3D viewing, the subject appears to be present at the convergence position where the left-eye video and the right-eye video exactly overlap, and thus the subject projected to have a parallax of zero appears to be present on the display surface. Since the focus position of the viewer's eyes is on the display surface, such an adjustment method in which the focus position and the convergence positions of the most subjects coincide on the display surface is considered to be the most natural and easy-to-see 3D video.

Japanese Patent Laid-Open No. 10-32840

  As long as the conditions for 3D viewing cannot be strictly defined at the 3D shooting stage, the stereoscopic effect due to the video projected at the time of 3D viewing must be different from the actual one. For this reason, conventionally, as described in Patent Document 1, shooting parameter adjustment has been performed in consideration of viewers' visibility. However, 3D shooting or 3D viewing is an experience different from conventional 2D shooting / viewing, and it is a natural requirement to perform shooting / viewing utilizing this characteristic. That is, there is a demand for a function that produces a stereoscopic effect corresponding to the shooting scene while maintaining the visibility and safety. However, in order to obtain a desirable stereoscopic effect, there is a problem that only those who have sufficient experience in 3D shooting know what shooting conditions should be used for 3D shooting.

  Therefore, it is an object of the present application to make it possible to automatically adjust shooting conditions according to the set distribution of the amount of parallax so that a stereoscopic effect according to the shooting intention of the photographer can be obtained.

  In order to achieve the above object, a 3D video signal photographing device of the present invention is a 3D video signal photographing device for photographing a stereoscopic video, and shoots a subject to generate the stereoscopic video. A shooting unit, a detection unit that detects parallax information indicating the amount of parallax included in the stereoscopic video, an adjustment unit that adjusts shooting conditions in the shooting unit, and parallax distribution information that indicates a distribution of a predetermined amount of parallax. A setting unit that sets the parallax amount distribution indicated by the detected parallax information in the photographing unit so that the distribution of the parallax amount indicated by the parallax distribution information set by the setting unit approaches Set the shooting conditions.

  In this way, the 3D video signal imaging device can capture the distribution of the parallax amount in the stereoscopic video to be captured so as to approach the set distribution of the parallax amount. Thereby, for example, it is possible to automatically control the shooting parameters only by setting the distribution of the intended amount of parallax.

  Preferably, the photographing unit includes a first optical system that photographs the subject from a first viewpoint and a second optical system that photographs the subject from a second viewpoint, and the adjustment unit detects the detected When the spread of the disparity amount indicated by the disparity information is brought close to the disparity amount distribution indicated by the disparity distribution information set by the setting unit, the distance between the optical axes of the first optical system and the second optical system is adjusted. .

  In this way, by adjusting the distance between the optical axes, it becomes possible to approximate the spread of the distribution of the parallax amount indicated by the set parallax information.

  Preferably, the photographing unit includes a first optical system that photographs the subject from the first viewpoint, and a second optical system that photographs the subject from the second viewpoint, and the adjustment unit. In the case where the disparity amount distribution indicated by the setting unit is kept close to the disparity amount distribution indicated by the setting unit while maintaining the spread of the disparity amount distribution indicated by the detected disparity information, and the optical axis of the first optical system and the first The convergence angle formed by the optical axes of the two optical systems is adjusted.

  In this way, by adjusting the convergence angle, it is possible to approximate the distribution of the parallax amount indicated by the parallax distribution information set by the setting unit while maintaining the spread of the distribution of the parallax amount indicated by the detected parallax information. it can.

  Preferably, the photographing unit includes a first optical system that photographs the subject from the first viewpoint, and a second optical system that photographs the subject from the second viewpoint, and the adjustment unit. Is a lens group included in the first optical system and a lens included in the second optical system when the distribution shape of the parallax amount indicated by the detected parallax information is maintained close to the parallax amount distribution indicated by the parallax distribution information. Shift the group.

  By doing so, it is possible to approximate the distribution of the parallax amount indicated by the parallax distribution information set by the setting unit while maintaining the spread of the distribution of the parallax amount indicated by the parallax information detected by shifting the lens group. .

  Further preferably, the adjustment unit performs shooting in the shooting unit such that a part of the distribution of the parallax amount indicated by the detected parallax information approaches the distribution of the parallax amount indicated by the parallax distribution information set by the setting unit. Set conditions.

  In this way, for example, it is possible to make the distribution of the parallax amount at the protruding portion closer to the distribution of the parallax amount indicated by the parallax distribution information set by the setting unit.

  Preferably, the adjustment unit estimates a distribution of a parallax amount obtained when shooting is performed under the changed shooting condition after changing the shooting condition, and the estimated parallax amount distribution is the set parallax distribution. The imaging conditions in the imaging unit are reset so as to approach the distribution of the parallax amount indicated by the information.

  In this way, for example, the distribution of the parallax amount according to the changed shooting condition can be estimated without re-detecting the parallax amount from the stereoscopic video shot under the changed shooting condition. This makes it easy to find suitable shooting conditions because the shooting conditions can be changed repeatedly.

  Preferably, the 3D video signal photographing apparatus further includes a mode setting unit for setting a photographing mode, and the setting unit sets the parallax distribution information based on the photographing mode set by the mode setting unit.

  In this way, parallax distribution information suitable for the shooting mode can be set.

  Further preferably, when the mode setting unit sets a mode for shooting a landscape, the setting unit has a parallax amount related to retraction among the parallax amounts of the stereoscopic video that the stereoscopic video has. A disparity amount distribution that is larger than the disparity amount relating to popping out is set as disparity distribution information.

  Further preferably, when the mode setting unit sets a macro shooting mode, the setting unit has a parallax amount related to popping out of a parallax amount included in the stereoscopic video image. A disparity amount distribution that is larger than the disparity amount related to retraction among the disparity amounts is set as disparity distribution information.

  Further preferably, when the mode setting unit has set a mode for photographing a person, the setting unit has a parallax amount of substantially zero of the parallax amounts of the stereoscopic video image. A disparity amount distribution that is larger than the disparity amount related to popping out or retracting among the disparity amounts of the video is set as the disparity distribution information.

  The adjustment unit sets shooting conditions in the shooting unit such that a feature amount based on a distribution of parallax amounts indicated by the detected parallax information approaches a feature amount based on a parallax distribution indicated by information set by the setting unit. To do.

  According to the above-described invention, for example, it is possible to automatically control the shooting conditions so as to approach the distribution of the parallax amount only by setting information regarding the distribution of the target parallax amount. As a result, it is possible to shoot a stereoscopic image that expresses the target stereoscopic effect.

The schematic diagram which shows the digital camera 1 in this embodiment Flowchart for explaining video signal shooting operation in digital camera 1 Flowchart for explaining photographing operation in 2D photographing mode Flowchart for explaining photographing operation in 3D photographing mode Flowchart of an operation for determining shooting parameters from images acquired from the CCD image sensors 150 (a) and 150 (b). The figure which shows the example of the setting histogram corresponding to the imaging | photography mode set by S503. A diagram for explaining the operation to obtain the shooting parameters directly from the difference between the setting histogram and the input histogram The figure for demonstrating the determination method of the imaging | photography parameter in other embodiment.

(Embodiment)
Hereinafter, a first embodiment when the present invention is applied to a digital camera will be described with reference to the drawings.

<1-1. Configuration of Digital Camera>
First, the configuration of the digital camera will be described.

  FIG. 1 is a schematic diagram showing a digital camera 1 in the present embodiment.

  The electrical configuration of the digital camera 1 according to the present embodiment will be described with reference to FIG. FIG. 1 is a block diagram showing the configuration of the digital camera 1. The digital camera 1 includes an optical system 110 (a) and 110 (b), a zoom motor 120 (a) and 120 (b), an OIS actuator 130 (a) and 130 (b), and a focus motor 140 (a). 140 (b), CCD image sensors 150 (a) and 150 (b), video processing unit 160, memory 200, controller 210, gyro sensor 220, card slot 230, memory card 240, operation member 250, zoom lever 260, a liquid crystal monitor 270, an internal memory 280, and a mode setting button 290.

  The optical system 110 (a) includes a zoom lens 111 (a), an OIS 112 (a), and a focus lens 113 (a). The optical system 110 (b) includes a zoom lens 111 (b), an OIS 112 (b), and a focus lens 113 (b). The optical system 110 (a) forms a subject image at the first viewpoint. Further, the optical system 110 (b) forms a subject image at a second viewpoint different from the first viewpoint.

  The zoom lenses 111 (a) and 111 (b) can enlarge or reduce the subject image by moving along the optical axis of the optical system. The zoom lenses 111 (a) and 111 (b) are controlled by the zoom motor 120.

  The OIS 112 (a) and 112 (b) have a correction lens that can move in a plane perpendicular to the optical axis. The OISs 112 (a) and 112 (b) reduce the blur of the subject image by driving the correction lens in a direction that cancels the blur of the digital camera 1. The correction lens can move from the center by a maximum L in OIS 112 (a) and 112 (b). OIS 112 (a) and 112 (b) are controlled by an OIS actuator 130.

  The focus lenses 113 (a) and 113 (b) adjust the focus of the subject image by moving along the optical axis of the optical system. The focus lenses 113 (a) and 113 (b) are controlled by the focus motor 140.

  The zoom motor 120 drives and controls the zoom lenses 111 (a) and 111 (b). The zoom motor 130 may be realized by a pulse motor, a DC motor, a linear motor, a servo motor, or the like. The zoom motor 130 may drive the zoom lenses 111 (a) and 111 (b) via a mechanism such as a cam mechanism or a ball screw. Further, the zoom lens 111 (a) and the zoom lens 111 (b) may be controlled by the same operation.

  The OIS actuator 130 drives and controls the correction lenses in the OIS 112 (a) and 112 (b) in a plane perpendicular to the optical axis. The OIS actuator 130 can be realized by a planar coil or an ultrasonic motor.

  The focus motor 140 drives and controls the focus lenses 113 (a) and 113 (b). The focus motor 140 may be realized by a pulse motor, a DC motor, a linear motor, a servo motor, or the like. The focus motor 140 may drive the focus lenses 113 (a) and 113 (b) via a mechanism such as a cam mechanism or a ball screw.

  The CCD image sensors 150 (a) and 150 (b) capture the subject image formed by the optical systems 110 (a) and 110 (b) and generate a first viewpoint signal and a second viewpoint signal. The image sensor sensors 150 (a) and 150 (b) that perform the various operations such as exposure, transfer, and electronic shutter. Here, the video image sensor processing unit 160 for stereoscopic viewing by combining the first viewpoint signal and the second viewpoint signal, the first viewpoint signal generated by the CCD image sensors 150 (a) and 150 (b), and Various processes are performed on the second viewpoint signal. The video processing unit 160 performs processing on the first viewpoint signal and the second viewpoint signal, generates image data (hereinafter referred to as a review image) to be displayed on the liquid crystal monitor 270, and stores it in the memory card 240. Or generating a video signal for re-storage. For example, the video processing unit 160 performs various video processes such as gamma correction, white balance correction, and flaw correction on the first viewpoint signal and the second viewpoint signal.

  In addition, the video processing unit 160 obtains a parallax amount for each specific region of the image from the comparison of both images with respect to the first viewpoint signal and the second viewpoint signal, and calculates a parallax histogram based on the appearance frequency of each parallax amount. Further, a shooting parameter for obtaining a predetermined stereoscopic effect is determined by comparing with a predetermined histogram pattern. Detailed operation of the imaging parameter determination process will be described later.

  Here, the detection of the amount of parallax in the video processing unit 160 is performed in units of 4 × 4 pixels, for example. The unit for detecting the amount of parallax may be any unit such as 8 × 8 pixels or 16 × 16 pixels. Further, the parallax amount may be any value as long as it is a value indicating a shift between both images.

  Further, the video processing unit 160 compresses the video signal with respect to the processed first viewpoint signal and second viewpoint signal by a compression format or the like based on the JPEG standard. Two compressed video signals obtained by compressing the first viewpoint signal and the second viewpoint signal are associated with each other and recorded in the memory card 240. In addition, when two are recorded, it is desirable to record using the MPF file format. When the video signal to be compressed is a moving image, H. A moving picture compression standard such as H.264 / AVC is applied. Also, the MPF file format and the JPEG image or MPEG moving image may be recorded simultaneously.

  The video processing unit 160 can be realized by a DSP, a microcomputer, or the like. The resolution of the review image may be set to the screen resolution of the liquid crystal monitor 270, or may be set to the resolution of image data that is compressed and formed by a compression format or the like conforming to the JPEG standard.

  The memory 200 functions as a work memory for the video processing unit 160 and the controller 210. For example, the memory 200 temporarily stores the video signal processed by the video processing unit 160 or the image data input from the CCD image sensor 150 before being processed by the video processing unit 160. In addition, the memory 200 temporarily stores shooting conditions of the optical systems 110 (a) and 110 (b) and the CCD image sensors 150 (a) and 150 (b) at the time of shooting. The shooting conditions indicate subject distance, field angle information, ISO sensitivity, shutter speed, EV value, F value, distance between lenses, shooting time, OIS shift amount, and the like. The memory 200 can be realized by, for example, a DRAM or a ferroelectric memory.

  The controller 210 is a control means for controlling the whole. Control based on the imaging parameters determined by the parallax histogram obtained from the images of the CCD image sensors 150 (a) and 150 (b) is also performed. The controller 210 can be realized by a semiconductor element or the like. The controller 210 may be configured only by hardware, or may be realized by combining hardware and software. The controller 210 can be realized by a microcomputer or the like.

  The gyro sensor 220 is composed of a vibration material such as a piezoelectric element. The gyro sensor 220 vibrates a vibration material such as a piezoelectric element at a constant frequency, converts a force generated by the Coriolis force into a voltage, and obtains angular velocity information. By obtaining angular velocity information from the gyro sensor 220 and driving the correction lens in the OIS in a direction that cancels the shaking, the camera shake given to the digital camera 100 by the user is corrected. The gyro sensor 220 may be any device that can measure at least the angular velocity information of the pitch angle. Further, when the gyro sensor 220 can measure the angular velocity information of the roll angle, it is possible to consider the rotation when the digital camera 1 moves in a substantially horizontal direction.

  The card slot 230 is detachable from the memory card 240. The card slot 230 can be mechanically and electrically connected to the memory card 240.

  The memory card 240 includes a flash memory, a ferroelectric memory, and the like, and can store data.

  The operation member 250 includes a release button. The release button receives a user's pressing operation. When the release button is pressed halfway, AF control and AE control are started via the controller 210. When the release button is fully pressed, the subject is photographed.

  The zoom lever 260 is a member that receives a zoom magnification change instruction from the user.

  The liquid crystal monitor 270 displays the first viewpoint signal or the second viewpoint signal generated by the CCD image sensor 150, the first viewpoint signal read from the memory card 240, and the second viewpoint signal in 2D display or 3D display. It is a device. The liquid crystal monitor 270 can display various setting information of the digital camera 100. For example, the liquid crystal monitor 270 can display EV values, F values, shutter speeds, ISO sensitivity, and the like, which are shooting conditions at the time of shooting.

  The LCD monitor 270 may select and display the first viewpoint signal and the second viewpoint signal when 2D display is performed, and the first viewpoint signal and the second viewpoint signal may be divided into screens. The first viewpoint signal and the second viewpoint signal may be alternately displayed for each line.

  When the 3D display is performed, the liquid crystal monitor 270 may display the first viewpoint signal and the second viewpoint signal in a frame sequential manner, or overlay the first viewpoint signal and the second viewpoint signal. It doesn't matter.

  The internal memory 280 is configured by a flash memory, a ferroelectric low memory, or the like. The internal memory 280 stores a control program for controlling the entire digital camera 1 and the like.

  The mode setting button 290 is a button for setting a shooting mode when shooting with the digital camera 1. The “shooting mode” indicates a shooting scene assumed by the user, and includes, for example, a shooting mode including (1) portrait mode, (2) macro mode, and (3) landscape mode. Further, the 2D shooting mode and the 3D shooting mode may be selected for each shooting scene. The digital camera 1 performs shooting by setting appropriate shooting parameters based on this shooting mode. In addition, you may make it include the camera automatic setting mode in which the digital camera 1 performs automatic setting. The mode setting button 290 is a button for setting a playback mode of a video signal recorded on the memory card 240.

<1-2. Video signal shooting operation>
Hereinafter, a video signal photographing operation in the digital camera 1 will be described.

  FIG. 2 is a flowchart for explaining a video signal photographing operation in the digital camera 1.

  FIG. 3 is a flowchart for explaining a shooting operation in the 2D shooting mode.

  FIG. 4 is a flowchart for explaining a shooting operation in the 3D shooting mode.

  (S201) First, when the mode setting button 290 is operated to the shooting mode by the user, the digital camera 1 acquires the shooting mode after the operation.

  (S202) The controller 210 determines whether the acquired shooting mode is the 2D shooting mode or the 3D shooting mode. When the acquired shooting mode is the 2D shooting mode, the process proceeds to S301. On the other hand, in the 3D shooting mode, the process proceeds to S401.

  (S301) The controller 210 waits until the release button is fully pressed.

  (S302) When the release button is fully pressed, at least one of the image sensors 150 (a) and 150 (b) performs a shooting operation based on the shooting conditions set in the 2D shooting mode. Then, at least one of the first viewpoint signal and the second viewpoint signal is generated.

  (S303) When the video signal is generated, the video processing unit 160 performs various video processes in accordance with the 2D shooting mode on the generated video signal to generate a compressed video signal.

  (S304) When the compressed video signal is generated, the controller 210 records the compressed video signal in the memory card 240 connected to the card slot 230. When the compressed video signal of the first viewpoint signal and the compressed video signal of the second viewpoint signal are obtained, the controller 210 uses the two compressed video signals in the memory card 240 using, for example, the MPO file format. Record in association.

  (S401) On the other hand, when the 3D shooting mode is selected, the controller 210 waits until the release button is half-pressed.

  (S402) When the release button is pressed halfway, the image processor 160 obtains the parallax distribution shape from the images acquired from the CCD image sensors 150 (a) and 150 (b). Based on this and the parallax distribution shape set according to the selected shooting mode, shooting parameters are determined so that they are substantially the same.

  (S403) When the shooting parameters are determined, the controller 210 waits until the release button is fully pressed.

  (S404) When the release button is fully pressed, the image sensors of the CCD image sensors 150 (a) and 150 (b) perform a shooting operation according to the already set shooting parameters, and the first viewpoint signal and the second Generate a viewpoint signal.

  (S405) When the first viewpoint signal and the second viewpoint signal are generated, the video processing unit 160 performs various video processes in accordance with the 3D shooting mode on the generated two video signals, Two compressed video signals of the 1 viewpoint signal and the 2nd viewpoint signal are generated.

  (S 406) When two compressed video signals are generated, the controller 210 records the two compressed video signals in the memory card 240 connected to the card slot 230. When two compressed video signals are recorded on the memory card 240, the two compressed video signals are recorded in association with each other using, for example, an MPF file format.

<1-2-1. Determination of shooting parameters>
FIG. 5 shows a flowchart of the operation for determining the shooting parameters from the video acquired from the CCD image sensors 150 (a) and 150 (b) in S402.

  (S501) First, the parallax is calculated for each minute region of the image from the comparison of the images acquired from the CCD image sensors 150 (a) and 150 (b). This minute region indicates a region including a plurality of pixels such as 8 × 8 pixels. Note that the minute region may be a region including only one pixel.

  (S502) A parallax histogram of the input image is obtained by counting the appearance frequency for each parallax amount. That is, in the parallax histogram, the amount of parallax is taken on the horizontal axis, and the number of minute regions where the amount of parallax is calculated on the vertical axis. Note that the vertical axis of the parallax histogram may be the appearance probability of a minute region in which a specific amount of parallax is calculated.

  (S503) A setting histogram corresponding to the set shooting mode is selected from a database of setting histograms stored corresponding to the shooting mode. The setting histogram will be described later.

  (S504) The input histogram and the set histogram are compared, and an evaluation value related to the similarity is calculated. When the histogram is expressed with the frequency with respect to the parallax amount d being H (d), the evaluation value E is calculated by the following equation.

  However, Hr (d) represents a set histogram and Hi (d) represents an input histogram.

  That is, in this embodiment, the deviation between the input histogram and the set histogram is calculated as the evaluation value. For example, when the value of E is large, the deviation between the input histogram and the set histogram is large. Conversely, when the value of E is small, the deviation between the input histogram and the set histogram is small.

  Here, w (d) represents a weighting function for parallax, which is set corresponding to the shooting scene. For example, it is desirable for w (d) to concentrate parallax closer to the depth side than the display screen when shooting a landscape as a shooting scene. For this reason, the difference based on | Hr (d) −Hi (d) | of the parallax on the fly-out side is handled heavily, and conversely, the difference based on | Hr (d) −Hi (d) | Set the weight function that can be handled. As such a weight function, for example, a form represented by the following equation can be considered.

  However, in the above equation, d <0 indicates the amount of parallax on the back side of the screen. On the other hand, d> 0 indicates the amount of parallax on the screen projection side.

  However, w (d) is not an essential item for calculating the evaluation value. That is, it may be an evaluation value without w (d), which is equivalent to w (d) = constant. In addition, w (d) is not limited to the above mathematical formula, and may be a format in which the parallax amount on the retracting side is emphasized. Further, a format in which the parallax amount places importance on the vicinity of 0 may be used.

  Also, in the above evaluation value definition, the visibility and safety of 3D viewing are not clearly shown. However, the setting histogram and the weighting function may take a form in consideration of these. For example, according to Non-Patent Document 1, the range of the parallax angle for comfortable 3D viewing is set to be within 1 °. Therefore, a weighting function that takes a large penalty for parallax exceeding the range of ± 0.5 °. It is also effective to set w (d).

  (S505) If the calculated evaluation value E is smaller than a preset threshold value, the process is terminated. Then, the current shooting parameter is determined as a new shooting parameter. On the other hand, when the calculated evaluation value E is larger than a preset threshold value, the processing is continued to correct the imaging parameter and re-verify it.

  (S506) If the calculated evaluation value E is larger than a preset threshold value, the imaging parameter to be corrected and the correction amount are determined based on the comparison of the two histograms. For example, when the parallax amount distribution of the input histogram is adjusted to be wide or narrow, the stereo base that is the distance between the optical axes of the first optical system and the second optical system is changed.

  On the other hand, when the input histogram is translated while maintaining the shape of the input histogram, the convergence angle formed by the optical axes of the first optical system and the second optical system is adjusted. In a camera that does not have a convergence angle adjustment mechanism, the convergence angle may be artificially adjusted by OIS. Further, the convergence angle may be adjusted in a pseudo manner by trimming the video signal obtained by photographing.

  (S507) When the shooting parameters are corrected, a parallax histogram that will be obtained by shooting with the corrected shooting parameters based on the input histogram is estimated.

  This parallax histogram can be estimated by, for example, the following equation by mutually converting the parallax amount d on the horizontal axis of the histogram and the distance F between the camera and the subject.

  Here, d is the amount of parallax, sb is the stereo base, θ is the convergence angle, f is the focal length of the lens, and ss is the size of the imaging surface. T () is a function for calculating the distance F between the camera and the subject from the parallax amount d, and invT () is a function for calculating the parallax amount d from the distance F between the camera and the subject. According to this method, first, a distance F of a subject having a parallax amount of d in an image shot under shooting parameters of sb, θ, f, and ss is calculated. Next, the parallax amount d ′ of each subject when changing to a different shooting parameter is obtained with respect to the subject distance F using invT (). As a result, it is possible to estimate a histogram that would be obtained when shooting with different shooting parameters from the input histogram.

  Note that the method of estimating the input histogram is not limited to the above mathematical formula, and other methods may be used.

  The estimated parallax histogram is replaced with the input histogram, and the process returns to S504.

  By performing the above iterative process, the photographing conditions for obtaining a histogram similar to the set histogram are determined.

  Here, the method of changing the stereo base sb and the convergence angle θ of the two optical systems as imaging parameters is shown, but in the system where the stereo base cannot be changed, the focal length f of the two optical systems is changed. This is effective because the same effect can be obtained.

  Next, the setting histogram will be described with reference to the drawings.

  FIG. 6 shows an example of a setting histogram corresponding to the shooting mode set in S503.

  The histogram 601 is set so that parallax is preferentially distributed on the depth side with respect to the display surface. That is, the histogram 601 can be said to be a distribution in which the depth is emphasized. This corresponds to the landscape mode with a stereoscopic effect suitable for landscape images and the like.

  The histogram 602 is set so that parallax is concentrated on the screen. In other words, the histogram 602 has a distribution in which main subjects are arranged on the display surface and are displayed with good stereoscopic effect. This corresponds to a portrait mode such as a portrait image.

  Further, the histogram 603 is set so that parallax popping out from the screen is emphasized. This corresponds to the macro mode for enjoying the stereoscopic effect of the subject in macro photography.

<1-2-2. Modification of shooting parameter determination>
The method for determining the shooting parameter shown in 1-2-1 and FIG. 5 is to repeatedly search for a shooting parameter in which the set evaluation value and the input histogram have the smallest predetermined evaluation value. However, the present invention is not limited to this, and it is also possible to obtain the shooting parameters directly from the difference between the setting histogram and the input histogram. Such an example is shown in FIG.

  In the following, steps that perform exactly the same operations as in FIG.

  First, S501 to S503 are performed.

  (S704) Next, only the parallax maximum value, minimum value, and average value in the histogram are evaluated as evaluation values indicating the difference between the input histogram and the set histogram. That is, the evaluation value is calculated using the following formula.

  Here, dr and di indicate parallax intervals having a non-zero frequency in the setting histogram and the input histogram. Further, max () and min () indicate the maximum value and the minimum value of the argument set. When such an evaluation value is used, it is possible to obtain a shooting parameter for setting the evaluation value to 0 by direct calculation using the following method.

  (S705) In order to determine the shooting parameter for obtaining the minimum evaluation value, how far away the subject having the maximum parallax max (di) and the minimum parallax min (di) of the input histogram corresponds to the subject, It can be obtained by the above (Formula 3). Assuming that the distance corresponding to max (di) is Fmax, the distance corresponding to min (di) is Fmin. If the above-described equation invT () for obtaining the parallax from the distance is used, the following relationship is established from the maximum parallax max (dr) and the minimum parallax min (dr) of the setting histogram.

  Considering these sb 'and θ' as unknowns, these two equations become simultaneous equations, and by solving this, the imaging parameters sb 'and θ' can be directly obtained. Therefore, as described in 1-2-1, it is also possible to determine the shooting parameter without searching for the shooting parameter by repeated calculation.

<1-3. Summary>
The digital camera 1 according to the present embodiment is a digital camera 1 that captures a stereoscopic video, an optical system 110 (a) that captures a subject and generates the stereoscopic video, and a CCD image sensor 150 ( a), an optical system 110 (b), a CCD image sensor 150 (b), a video processing unit 160 for detecting parallax information indicating a parallax amount of the stereoscopic video, the optical system 110 (a), Video processing having a function of adjusting shooting conditions in the CCD image sensor 150 (a), the optical system 110 (b), and the CCD image sensor 150 (b) and a function of setting parallax distribution information indicating a distribution of a predetermined amount of parallax. Unit 160, and the video processing unit 160 indicates that the distribution of the parallax amount indicated by the detected parallax information is the parallax distribution information set by the video processing unit 160. The optical system 110 so as to approach the distribution of the to parallax amount (a), the CCD image sensor 150 (a), sets the imaging conditions in the optical system 110 (b) and the CCD image sensor 150 (b).

  In this way, the digital camera 1 can shoot so that the distribution of the parallax amount in the stereoscopic video to be shot is close to the set distribution of the parallax amount. Thereby, for example, it is possible to automatically control the shooting parameters only by setting the distribution of the intended amount of parallax.

  Preferably, the optical system 110 (a), the CCD image sensor 150 (a), the optical system 110 (b), and the CCD image sensor 150 (b) are a first optical system that captures the subject from a first viewpoint. And a second optical system that captures the subject from a second viewpoint, and the video processing unit 160 sets a parallax amount distribution spread indicated by the detected parallax information by the video processing unit 160. When approaching the distribution of the parallax amount indicated by the distribution information, the distance between the optical axes of the first optical system and the second optical system is adjusted.

  In this way, by adjusting the distance between the optical axes, it becomes possible to approximate the spread of the distribution of the parallax amount indicated by the set parallax information.

  Preferably, the optical system 110 (a), the CCD image sensor 150 (a), the optical system 110 (b), and the CCD image sensor 150 (b) are configured to take a first image of the subject from the first viewpoint. An optical system and a second optical system that captures the subject from the second viewpoint, and the video processing unit 160 maintains the spread of the distribution of the parallax amount indicated by the detected parallax information, When approaching the distribution of the parallax amount indicated by the parallax distribution information set by the video processing unit 160, the convergence angle formed by the optical axis of the first optical system and the optical axis of the second optical system is adjusted.

  In this way, by adjusting the convergence angle, the distribution of the amount of parallax indicated by the detected parallax information is maintained and the distribution of the amount of parallax indicated by the parallax distribution information set by the video processing unit 160 is maintained. be able to.

  Preferably, the optical system 110 (a), the CCD image sensor 150 (a), the optical system 110 (b), and the CCD image sensor 150 (b) are configured to take a first image of the subject from the first viewpoint. An optical system and a second optical system that captures the subject from the second viewpoint, and the video processing unit 160 maintains the disparity amount distribution shape indicated by the detected disparity information while maintaining the disparity distribution When approaching the distribution of the parallax amount indicated by the information, the lens group included in the first optical system and the lens group included in the second optical system are shifted.

  In this way, the distribution of the amount of parallax indicated by the parallax information detected by shifting the lens group is maintained, and the distribution of the amount of parallax indicated by the parallax distribution information set by the video processing unit 160 is approached. Can do.

  Also preferably, the video processing unit 160 is configured so that a part of the distribution of the parallax amount indicated by the detected parallax information approaches the distribution of the parallax amount indicated by the parallax distribution information set by the video processing unit 160. Shooting conditions in the optical system 110 (a), the CCD image sensor 150 (a), the optical system 110 (b), and the CCD image sensor 150 (b) are set.

  In this way, for example, the distribution of the amount of parallax at the protruding portion can be brought close to the distribution of the amount of parallax indicated by the parallax distribution information set by the video processing unit 160.

  Preferably, the video processing unit 160 estimates the distribution of the amount of parallax obtained when shooting is performed under the changed shooting condition after changing the shooting condition, and the estimated distribution of the amount of parallax is set as the setting. The photographing conditions in the optical system 110 (a), the CCD image sensor 150 (a), the optical system 110 (b), and the CCD image sensor 150 (b) are reset so as to approach the distribution of the parallax amount indicated by the parallax distribution information. .

  In this way, for example, the distribution of the parallax amount according to the changed shooting condition can be estimated without re-detecting the parallax amount from the stereoscopic video shot under the changed shooting condition. This makes it easy to find suitable shooting conditions because the shooting conditions can be changed repeatedly.

  Preferably, the digital camera 1 further includes a mode setting button 290 for setting a shooting mode, and the video processing unit 160 sets parallax distribution information based on a shooting mode set by the mode setting button 290. .

  In this way, parallax distribution information suitable for the shooting mode can be set.

  In addition, preferably, when the mode setting button 290 sets a mode for shooting a landscape, the video processing unit 160 has a parallax amount related to retraction among the parallax amounts of the stereoscopic video. A disparity amount distribution that is larger than the disparity amount related to popping out of the disparity amounts of the video is set as the disparity distribution information.

  Also preferably, when the mode setting button 290 is set to a mode for taking a macro, the video processing unit 160 has a parallax amount related to popping out of the parallax amount of the stereoscopic video image. A disparity amount distribution that is larger than the disparity amount related to retraction among the disparity amounts included in the video is set as the disparity distribution information.

  In addition, preferably, when the mode setting button 290 sets a mode for photographing a person, the video processing unit 160 has a parallax amount of substantially zero among the parallax amounts of the stereoscopic video image. A disparity amount distribution that is larger than the disparity amount related to popping out or retracting is set as disparity distribution information.

  Also preferably, the video processing unit 160 is configured so that the feature quantity based on the distribution of the parallax amount indicated by the detected parallax information approaches the feature quantity based on the parallax distribution indicated by the set information. ), Imaging conditions in the CCD image sensor 150 (a), the optical system 110 (b), and the CCD image sensor 150 (b) are set.

<2. Other Embodiments>
Note that the method shown in FIG. 8 may be used to determine the shooting parameters in 1-2-1.

  In the case of the imaging parameter determination method shown in FIG. 8, there is no mechanism for estimating the input histogram. Instead, after the process of S506 is completed, the operation of detecting the parallax amount is performed again.

  In the digital camera described in the above embodiment, each block may be individually made into one chip by a semiconductor device such as an LSI, or may be made into one chip so as to include a part or the whole.

  Here, although LSI is used, it may be called IC, system LSI, super LSI, or ultra LSI depending on the degree of integration.

  Further, the method of circuit integration is not limited to LSI, and implementation with a dedicated circuit or a general-purpose processor is also possible. An FPGA (Field Programmable Gate Array) that can be programmed after manufacturing the LSI or a reconfigurable processor that can reconfigure the connection and setting of circuit cells inside the LSI may be used.

  Further, if integrated circuit technology comes out to replace LSI's as a result of the advancement of semiconductor technology or a derivative other technology, it is naturally also possible to carry out function block integration using this technology. Biotechnology can be applied as a possibility.

  Moreover, each process of the said embodiment may be implement | achieved by hardware, and may be implement | achieved by software. Further, it may be realized by mixed processing of software and hardware. Needless to say, when the digital camera according to the above-described embodiment is realized by hardware, it is necessary to adjust timing for performing each process. In the above embodiment, for convenience of explanation, details of timing adjustment of various signals generated in actual hardware design are omitted.

  Moreover, the execution order of the processing method in the said embodiment is not necessarily restricted to description of the said embodiment, The execution order can be changed in the range which does not deviate from the summary of invention.

  The specific configuration of the present invention is not limited to the above-described embodiment, and various changes and modifications can be made without departing from the scope of the invention.

  According to the 3D video signal processing device, the digital imaging device, and the 3D video processing method according to the present invention, it is possible to generate a video signal that feels a natural three-dimensional feeling when viewed, so that 3D video is captured. The present invention can be applied to a digital camera, a broadcast camera, and a recorder or player that records and reproduces 3D video.

110 (a), 110 (b) Optical system 120 (a), 120 (b) Zoom motor 130 (a), 130 (b) OIS actuator 140 (a), 140 (b) Focus motor 150 (a), 150 (B) CCD image sensor 160 Video processing unit 200 Memory 210 Controller 220 Gyro sensor 230 Card slot 240 Memory card 250 Operation member 260 Zoom lever 270 Liquid crystal monitor 280 Internal memory 290 Mode setting buttons 601 602 603 Histogram

Claims (11)

A three-dimensional camera that shoots a stereoscopic image,
A photographing unit for photographing a subject and generating the stereoscopic image;
A detection unit for detecting parallax information indicating a parallax amount of the stereoscopic video;
An adjustment unit for adjusting shooting conditions in the shooting unit;
A setting unit that sets parallax distribution information indicating a distribution of a predetermined amount of parallax,
The adjustment unit is a three-dimensional camera that sets shooting conditions in the shooting unit such that the distribution of the parallax amount indicated by the detected parallax information approaches the distribution of the parallax amount indicated by the parallax distribution information set by the setting unit. The photographing unit includes a first optical system for photographing the subject from a first viewpoint and a second optical system for photographing the subject from a second viewpoint,
When the adjustment unit brings the spread of the distribution of the parallax amount indicated by the detected parallax information closer to the distribution of the parallax amount indicated by the parallax distribution information set by the setting unit, the first optical system and the second optical system The three-dimensional camera according to claim 1, wherein the distance between the optical axes is adjusted. The photographing unit includes a first optical system that photographs the subject from the first viewpoint, and a second optical system that photographs the subject from the second viewpoint,
When the adjustment unit approaches the distribution of the parallax amount indicated by the parallax distribution information set by the setting unit while maintaining the spread of the distribution of the parallax amount indicated by the detected parallax information, the optical axis of the first optical system The three-dimensional camera according to claim 1, wherein a convergence angle formed by the optical axis of the second optical system is adjusted. The photographing unit includes a first optical system that photographs the subject from the first viewpoint, and a second optical system that photographs the subject from the second viewpoint,
When the adjustment unit approaches the distribution of the parallax amount indicated by the parallax distribution information while maintaining the distribution shape of the parallax amount indicated by the detected parallax information, the lens group included in the first optical system and the second optical system The three-dimensional camera according to claim 1, wherein a lens group included in the lens is shifted.   The adjustment unit sets shooting conditions in the shooting unit so that a part of the distribution of the parallax amount indicated by the detected parallax information approaches the distribution of the parallax amount indicated by the parallax distribution information set by the setting unit. The three-dimensional camera according to claim 1.   The adjustment unit estimates the distribution of the amount of parallax obtained when shooting is performed under the changed shooting condition after changing the shooting condition, and the estimated parallax amount distribution is the parallax indicated by the set parallax distribution information The three-dimensional camera according to claim 1, wherein photographing conditions in the photographing unit are reset so as to approach a distribution of amounts. The three-dimensional camera further includes a mode setting unit for setting a shooting mode,
The three-dimensional camera according to claim 1, wherein the setting unit sets parallax distribution information based on a shooting mode set by the mode setting unit.   In the setting unit, when the mode setting unit sets a mode for shooting a landscape, the amount of parallax related to retraction among the amount of parallax included in the stereoscopic video is equal to the amount of parallax included in the stereoscopic video. The three-dimensional camera according to claim 7, wherein a distribution of a parallax amount larger than a parallax amount related to popping out is set as parallax distribution information.   The setting unit, when the mode setting unit has set a mode for shooting a macro, out of the amount of parallax of the stereoscopic video, the amount of parallax related to popping out is the amount of parallax of the stereoscopic video The three-dimensional camera according to claim 7, wherein a distribution of a parallax amount larger than a parallax amount related to retraction is set as parallax distribution information.   When the mode setting unit sets a mode for photographing a person, the setting unit has a parallax amount substantially zero among the parallax amounts of the stereoscopic video image included in the stereoscopic video image. The three-dimensional camera according to claim 7, wherein a distribution of a parallax amount larger than a parallax amount related to popping out or retracting is set as parallax distribution information.   The adjustment unit sets shooting conditions in the shooting unit such that a feature amount based on a distribution of parallax amounts indicated by the detected parallax information approaches a feature amount based on a parallax distribution indicated by information set by the setting unit. The three-dimensional camera according to claim 1.

Images