WO2015163350A1 - Dispositif de traitement d'images, dispositif d'imagerie et programme de traitement d'images - Google Patents

Dispositif de traitement d'images, dispositif d'imagerie et programme de traitement d'images Download PDF

Info

Publication number
WO2015163350A1
WO2015163350A1 PCT/JP2015/062202 JP2015062202W WO2015163350A1 WO 2015163350 A1 WO2015163350 A1 WO 2015163350A1 JP 2015062202 W JP2015062202 W JP 2015062202W WO 2015163350 A1 WO2015163350 A1 WO 2015163350A1
Authority
WO
WIPO (PCT)
Prior art keywords
parallax
amount
image data
subject
value
Prior art date
Application number
PCT/JP2015/062202
Other languages
English (en)
Japanese (ja)
Inventor
潤弥 萩原
清茂 芝崎
石賀　健一
文樹 中村
祐介 ▲高▼梨
Original Assignee
株式会社ニコン
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 株式会社ニコン filed Critical 株式会社ニコン
Publication of WO2015163350A1 publication Critical patent/WO2015163350A1/fr

Links

Images

Classifications

    • G—PHYSICS
    • G03—PHOTOGRAPHY; CINEMATOGRAPHY; ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ELECTROGRAPHY; HOLOGRAPHY
    • G03B—APPARATUS OR ARRANGEMENTS FOR TAKING PHOTOGRAPHS OR FOR PROJECTING OR VIEWING THEM; APPARATUS OR ARRANGEMENTS EMPLOYING ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ACCESSORIES THEREFOR
    • G03B35/00—Stereoscopic photography
    • G03B35/08—Stereoscopic photography by simultaneous recording
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/20—Image signal generators
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof

Definitions

  • the present invention relates to an image processing device, an imaging device, and an image processing program.
  • Patent Literature JP-A-8-47001
  • Stereo image data captured by a stereo imaging device may cause extreme parallax between the left and right images due to the placement of the subject in the scene, and viewers will feel uncomfortable and tired during viewing. was there. On the other hand, even when the parallax amount is adjusted to suppress such extremely generated parallax, there is a demand for the main subject to have a stereoscopic effect.
  • An image processing apparatus includes an acquisition unit that acquires image data, a calculation unit that calculates a parallax amount of each of a plurality of subjects included in an image of the image data, and parallax image data from the image data.
  • a determination unit that determines an adjustment parameter value for adjusting the amount of parallax applied to each of a plurality of subjects in accordance with the calculated amount of parallax, and applies the adjustment parameter value to the image data.
  • a generation unit that generates parallax image data that is an image having parallax amounts adjusted to each other.
  • An image processing program includes an acquisition step of acquiring image data, a calculation step of calculating the amount of parallax for each of a plurality of subjects included in the image of the image data, and parallax image data from the image data.
  • a determination step for determining an adjustment parameter value for adjusting the amount of parallax applied to each of a plurality of subjects in accordance with the calculated amount of parallax, and applying the adjustment parameter value to the image data And causing the computer to execute a generation step of generating parallax image data to be images having parallax amounts adjusted to each other.
  • the digital camera according to the present embodiment which is a form of the imaging device, is configured to generate a plurality of viewpoint images for one scene by one shooting. Each image having a different viewpoint is called a parallax image.
  • a parallax image In the present embodiment, a case where a right parallax image and a left parallax image from two viewpoints corresponding to the right eye and the left eye are generated will be described.
  • the digital camera in the present embodiment can generate a parallax-free image without parallax from the central viewpoint together with the parallax image.
  • FIG. 1 is a diagram illustrating a configuration of a digital camera 10 according to an embodiment of the present invention.
  • the digital camera 10 includes a photographic lens 20 as a photographic optical system, and guides a subject light beam incident along the optical axis 21 to the image sensor 100.
  • the photographing lens 20 may be an interchangeable lens that can be attached to and detached from the digital camera 10.
  • the digital camera 10 includes an image sensor 100, a control unit 201, an A / D conversion circuit 202, a memory 203, a drive unit 204, an image processing unit 205, a memory card IF 207, an operation unit 208, a display unit 209, and an LCD drive circuit 210. .
  • the direction parallel to the optical axis 21 toward the image sensor 100 is defined as the Z-axis plus direction
  • the direction toward the front of the drawing on the plane orthogonal to the Z-axis is the X-axis plus direction
  • the upward direction on the drawing is Y.
  • the axis is defined as the plus direction.
  • the coordinate axes are displayed so that the orientation of each figure can be understood with reference to the coordinate axes of FIG.
  • the photographing lens 20 is composed of a plurality of optical lens groups, and forms an image of a subject light flux from the scene in the vicinity of its focal plane.
  • the photographic lens 20 is represented by a single virtual lens arranged in the vicinity of the pupil. Further, in the vicinity of the pupil, a diaphragm 22 that restricts the incident light beam concentrically with the optical axis 21 as the center is disposed.
  • the image sensor 100 is disposed in the vicinity of the focal plane of the photographing lens 20.
  • the image sensor 100 is an image sensor such as a CCD or CMOS sensor in which a plurality of photoelectric conversion elements are two-dimensionally arranged.
  • the image sensor 100 is controlled in timing by the drive unit 204, converts the subject image formed on the light receiving surface into an image signal, and outputs the image signal to the A / D conversion circuit 202.
  • the A / D conversion circuit 202 converts the image signal output from the image sensor 100 into a digital image signal and outputs the digital image signal to the memory 203.
  • the image processing unit 205 performs various image processing on the digital image signal using the memory 203 as a work space, and generates captured image data.
  • the captured image data includes reference image data generated from the output of the non-parallax pixels of the image sensor 100 and parallax image data generated from the output of the parallax pixels of the image sensor 100, as will be described later.
  • the control unit 201 controls the digital camera 10 in an integrated manner. For example, the aperture of the diaphragm 22 is adjusted according to the set diaphragm value, and the photographing lens 20 is advanced and retracted in the optical axis direction according to the AF evaluation value. Further, the position of the photographing lens 20 is detected, and the focal length and the focus lens position of the photographing lens 20 are grasped. Further, a timing control signal is transmitted to the drive unit 204, and a series of imaging control until the image signal output from the imaging element 100 is processed into captured image data by the image processing unit 205 is managed, and the captured image Get the data.
  • control unit 201 includes a depth information detection unit 230.
  • the depth information detection unit 230 detects the subject distribution in the depth direction with respect to the scene. Specifically, the control unit 201 detects the subject distribution from the defocus amount for each subdivided area using defocus information used for autofocus.
  • the defocus information may use the output of a phase difference sensor provided exclusively, or may use the output of parallax pixels of the image sensor 100.
  • the parallax image data processed by the image processing unit 205 can also be used.
  • the subject distribution can be detected even when the focus lens is advanced and retracted without using the defocus information and the AF evaluation value by the contrast AF method is calculated for each subdivided region.
  • the image processing unit 205 processes the image signal output from the image sensor 100 to generate captured image data.
  • the image processing unit 205 includes a calculation unit 231, a determination unit 232, a parallax image data generation unit 233, and a moving image generation unit 234.
  • the calculation unit 231 calculates the parallax amount of each subject included in the image of the captured image data, that is, the unadjusted parallax amount from the left and right parallax image data described later.
  • the determination unit 232 determines a change condition for changing the parallax amount according to the parallax amount of each subject calculated by the calculation unit 231. More specifically, the determination unit 232 determines the value of the stereoscopic adjustment parameter so that the amount of parallax between output parallax images falls within the target amount of parallax.
  • This stereoscopic adjustment parameter is a parameter that is applied to adjust the amount of parallax of parallax image data when generating parallax image data from captured image data.
  • the parallax image data generation unit 233 applies parallax adjustment parameters to the captured image data to generate parallax image data that becomes images having parallax amounts adjusted to each other. Details of the calculation unit 231, the determination unit 232, and the parallax image data generation unit 233 will be described later.
  • the moving image generation unit 234 connects the parallax image data and generates a 3D moving image file.
  • the image processing unit 205 also has general image processing functions such as adjusting image data according to the selected image format.
  • the generated captured image data is converted into a display signal by the LCD drive circuit 210 and displayed on the display unit 209.
  • the data is recorded on the memory card 220 attached to the memory card IF 207.
  • the operation unit 208 functions as a part of a reception unit that receives a user operation and transmits an instruction to the control unit 201.
  • the operation unit 208 includes a plurality of operation members such as a shutter button that receives a shooting start instruction.
  • FIG. 2 is a conceptual diagram conceptually showing a state in which a part of the image sensor 100 is enlarged.
  • the basic grid 110 includes four Bayer arrays having 4 ⁇ 2 ⁇ 2 basic units in the Y-axis direction and four in the X-axis direction.
  • a green filter (G filter) is arranged for the upper left pixel and the lower right pixel
  • a blue filter (B filter) is arranged for the lower left pixel
  • R filter red filter
  • the basic grid 110 includes parallax pixels and non-parallax pixels.
  • the parallax pixel is a pixel that receives a partial light beam that is deviated from the optical axis of the photographing lens 20 out of the incident light beam that is transmitted through the photographing lens 20.
  • the parallax pixel is provided with an aperture mask having a deviated opening that is deviated from the center of the pixel so as to transmit only the partial light flux.
  • the opening mask is provided so as to overlap the color filter.
  • the parallax Lt pixel defined so that the partial light beam reaches the left side with respect to the pixel center and the parallax specified so that the partial light beam reaches the right side with respect to the pixel center by the aperture mask.
  • the non-parallax pixel is a pixel that is not provided with an aperture mask, and is a pixel that receives the entire incident light beam that passes through the photographing lens 20.
  • the parallax pixel is not limited to the aperture mask when receiving the partial light beam that is deviated from the optical axis, but has various configurations such as a selective reflection film in which the light receiving region and the reflective region are separated, and a deviated photodiode region. Can be adopted. In other words, the parallax pixel only needs to be configured to receive a partial light beam that is deviated from the optical axis, among incident light beams that pass through the photographing lens 20.
  • Pixels in the basic grid 110 are denoted by PIJ .
  • the upper left pixel is P 11
  • the upper right pixel is P 81.
  • the parallax pixels are arranged as follows.
  • the other pixels are non-parallax pixels, and are any of the non-parallax pixel + R filter, the non-parallax pixel + G filter, and the non-parallax pixel + B filter.
  • the parallax pixels When viewed as a whole of the image sensor 100, the parallax pixels are classified into one of a first group having a G filter, a second group having an R filter, and a third group having a B filter. Includes at least one parallax Lt pixel and parallax Rt pixel belonging to each group. As in the example in the figure, these parallax pixels and non-parallax pixels may be arranged with randomness in the basic lattice 110. By arranging with randomness, RGB color information can be acquired as the output of the parallax pixels without causing bias in the spatial resolution for each color component, so that high-quality parallax image data can be obtained. can get.
  • FIG. 3 is a diagram illustrating an example of processing for generating 2D image data and parallax image data.
  • the image processing unit 205 receives raw raw image data in which output values (pixel values) are arranged in the order of pixel arrangement of the image sensor 100, and executes plane separation processing for separating the raw image data into a plurality of plane data.
  • the left column of the figure shows an example of processing for generating 2D-RGB plane data as 2D image data.
  • the image processing unit 205 In generating 2D-RGB plane data, the image processing unit 205 first removes the pixel values of the parallax pixels to form a vacant lattice. Then, the pixel value that becomes the empty grid is calculated by interpolation processing using the pixel values of the surrounding pixels. For example, the pixel value of the empty lattice P 11 is obtained by averaging the pixel values of P ⁇ 1 ⁇ 1 , P 2 ⁇ 1 , P ⁇ 12 , and P 22 which are the pixel values of the G filter pixels adjacent in the diagonal direction. To calculate.
  • the pixel value of the empty lattice P 63 is calculated by averaging the pixel values of P 43 , P 61 , P 83 , and P 65 that are adjacent R filter pixel values by skipping one pixel vertically and horizontally.
  • the pixel value of the air grating P 76 is the pixel value of the adjacent B filter skipping one pixel vertically and horizontally, and averaging operation of the pixel values of P 56, P 74, P 96 , P 78 calculate.
  • the image processing unit 205 performs image processing as a general 2D image according to a predetermined format such as JPEG when generating still image data and MPEG when generating moving image data.
  • the image processing unit 205 further separates the 2D-RGB plane data for each color and performs the interpolation processing as described above to generate each plane data as reference image data. That is, three types of data are generated: Gn plane data as green reference image plane data, Rn plane data as red reference image plane data, and Bn plane data as blue reference image plane data.
  • the right column of the figure shows an example of processing for generating two G plane data, two R plane data, and two B plane data as parallax pixel data.
  • the two G plane data are GLt plane data as left parallax image data and GRt plane data as right parallax image data.
  • the two R plane data are RLt plane data and right parallax image data as left parallax image data.
  • the two B plane data are the BLt plane data as the left parallax image data and the BRt plane data as the right parallax image data.
  • the image processing unit 205 removes pixel values other than the pixel values of the G (Lt) pixels from all output values of the image sensor 100 to form a vacant lattice.
  • two pixel values P 11 and P 55 remain in the basic grid 110. Therefore, we divided into four equal basic grid 110 vertically and horizontally, the 16 pixels of the top left is represented by an output value of the P 11, is representative of the 16 pixels in the lower right in the output value of the P 55. Then, for the upper right 16 pixels and the lower left 16 pixels, average values of neighboring representative values adjacent in the vertical and horizontal directions are averaged and interpolated. That is, the GLt plane data has one value in units of 16 pixels.
  • the image processing unit 205 when generating the GRt plane data, the image processing unit 205 removes pixel values other than the pixel value of the G (Rt) pixel from all the output values of the image sensor 100 to obtain an empty grid. Then, two pixel values P 51 and P 15 remain in the basic grid 110. Therefore, the basic grid 110 is divided into four equal parts vertically and horizontally, the upper right 16 pixels are represented by the output value of P 51 , and the lower left 16 pixels are represented by the output value of P 15 . The upper left 16 pixels and the lower right 16 pixels are interpolated by averaging the peripheral representative values adjacent vertically and horizontally. That is, the GRt plane data has one value in units of 16 pixels. In this way, GLt plane data and GRt plane data having a resolution lower than that of 2D-RGB plane data can be generated.
  • the image processing unit 205 In generating the RLt plane data, the image processing unit 205 removes pixel values other than the pixel value of the R (Lt) pixel from all output values of the image sensor 100 to form a vacant lattice. Then, the primitive lattice 110, the pixel values of P 27 remains. This pixel value is set as a representative value for 64 pixels of the basic grid 110. Similarly, when generating the RRt plane data, the image processing unit 205 removes pixel values other than the pixel value of the R (Rt) pixel from all output values of the image sensor 100 to form a vacant lattice. Then, the pixel value P 63 remains in the basic grid 110. This pixel value is set as a representative value for 64 pixels of the basic grid 110.
  • RLt plane data and RRt plane data having a lower resolution than 2D-RGB plane data are generated.
  • the resolution of the RLt plane data and the RRt plane data is lower than the resolution of the GLt plane data and the GRt plane data.
  • the image processing unit 205 In generating the BLt plane data, the image processing unit 205 removes pixel values other than the pixel values of the B (Lt) pixels from all output values of the image sensor 100 to form a vacant lattice. Then, the primitive lattice 110, the pixel values of P 32 remains. This pixel value is set as a representative value for 64 pixels of the basic grid 110. Similarly, when generating the BRt plane data, the image processing unit 205 removes pixel values other than the pixel value of the B (Rt) pixel from all the output values of the image sensor 100 to obtain an empty grid. Then, the primitive lattice 110, the pixel values of P 76 remains. This pixel value is set as a representative value for 64 pixels of the basic grid 110.
  • BLt plane data and BRt plane data having a resolution lower than that of 2D-RGB plane data are generated.
  • the resolution of the BLt plane data and the BRt plane data is lower than the resolution of the GLt plane data and the GRt plane data, and is equal to the resolution of the RLt plane data and the RRt plane data.
  • image processing may be performed on the output image data so that the amount of parallax between generated images is within the target amount of parallax.
  • the image processing unit 205 generates left-view color image data and right-view color image data using these plane data.
  • color image data in which the parallax amount as a 3D image is adjusted while maintaining the blur amount of the 2D color image is generated.
  • the generation principle will be described first.
  • FIG. 4 is a diagram for explaining the concept of defocusing.
  • the parallax Lt pixel and the parallax Rt pixel receive a subject light flux that arrives from one of two parallax virtual pupils that are set symmetrically with respect to the optical axis as a partial region of the lens pupil.
  • the parallax pixel outputs an image signal obtained by photoelectrically converting only the partial light flux that has passed through the parallax virtual pupil by the action of the aperture mask that each has. Therefore, the pixel value distribution indicated by the output of the parallax pixel may be considered to be proportional to the light intensity distribution of the partial light flux that has passed through the corresponding parallax virtual pupil.
  • the output of each parallax pixel is the corresponding image point regardless of the subject luminous flux that has passed through any parallax virtual pupil. This shows a steep pixel value distribution centering on this pixel. If the parallax Lt pixels are arranged in the vicinity of the image point, the output value of the pixel corresponding to the image point is the largest, and the output value of the pixels arranged in the vicinity rapidly decreases. Further, even when the parallax Rt pixels are arranged in the vicinity of the image point, the output value of the pixel corresponding to the image point is the largest, and the output value of the pixels arranged in the vicinity rapidly decreases. That is, even if the subject luminous flux passes through any parallax virtual pupil, the output value of the pixel corresponding to the image point is the largest, and the output value of the pixels arranged in the vicinity rapidly decreases. Match each other.
  • the peak of the pixel value distribution indicated by the parallax Lt pixel corresponds to the image point, compared to the case where the object point exists at the focal position. Appearing at a position away from the pixel in one direction, and its output value decreases. In addition, the width of the pixel having the output value is increased.
  • the peak of the pixel value distribution indicated by the parallax Rt pixel appears at a position away from the pixel corresponding to the image point in the opposite direction to the one direction in the parallax Lt pixel and at an equal distance, and the output value similarly decreases. Similarly, the width of the pixel having the output value is increased.
  • the same pixel value distribution that is gentler than that in the case where the object point exists at the focal position appears at an equal distance from each other.
  • the same pixel value distribution that becomes more gentle as compared with the state of FIG. 4B appears further apart.
  • the amount of blur and the amount of parallax increase as the object point deviates from the focal position.
  • the amount of blur and the amount of parallax change in conjunction with defocus. That is, the amount of blur and the amount of parallax have a one-to-one relationship.
  • FIGS. 4B and 4C show the case where the object point shifts away from the focal position, but when the object point moves away from the focal position, as shown in FIG.
  • the relative positional relationship between the pixel value distribution indicated by the parallax Lt pixel and the pixel value distribution indicated by the parallax Rt pixel is reversed. Due to such a defocus relationship, when viewing a parallax image, the viewer visually recognizes a subject existing far behind the focal position and visually recognizes a subject present in front.
  • FIG. 5 is a graph showing changes in the pixel value distribution described in FIGS. 4B and 4C.
  • the horizontal axis represents the pixel position, and the center position is the pixel position corresponding to the image point.
  • the vertical axis represents the output value (pixel value) of each pixel. As described above, this output value is substantially proportional to the light intensity.
  • the distribution curve 1804 and the distribution curve 1805 represent the pixel value distribution of the parallax Lt pixel and the pixel value distribution of the parallax Rt pixel in FIG. 4B, respectively. As can be seen from the figure, these distributions have a line-symmetric shape with respect to the center position. Further, a combined distribution curve 1806 obtained by adding them shows a pixel value distribution of pixels without parallax with respect to the situation of FIG. 4B, that is, a pixel value distribution when the entire subject luminous flux is received, and a substantially similar shape.
  • the distribution curve 1807 and the distribution curve 1808 represent the pixel value distribution of the parallax Lt pixel and the pixel value distribution of the parallax Rt pixel in FIG. 4C, respectively. As can be seen from the figure, these distributions are also symmetrical with respect to the center position. Also, a combined distribution curve 1809 obtained by adding them shows a shape substantially similar to the pixel value distribution of the non-parallax pixels for the situation of FIG.
  • the amount of parallax expressed as an interval between peaks is adjusted while approximately maintaining the amount of blur expressed by the spread of the pixel value distribution. That is, in this embodiment, the image processing unit 205 is adjusted between the 2D image generated from the non-parallax pixel and the 3D image generated from the parallax pixel while maintaining the blur amount of the 2D image almost as it is. An image having a parallax amount is generated.
  • FIG. 6 is a diagram illustrating a pixel value distribution for explaining the concept of the adjusted parallax amount.
  • Lt distribution curve 1901 and Rt distribution curve 1902 indicated by solid lines in the figure are distribution curves in which actual pixel values of Lt plane data and Rt plane data are plotted. For example, it corresponds to the distribution curves 1804 and 1805 in FIG.
  • the distance between the peaks of the Lt distribution curve 1901 and the Rt distribution curve 1902 represents the 3D parallax amount, and the greater the distance, the stronger the stereoscopic effect during image reproduction.
  • the 2D distribution curve 1903 obtained by adding 50% each of the Lt distribution curve 1901 and the Rt distribution curve 1902 has a convex shape with no left-right bias.
  • the 2D distribution curve 1903 corresponds to a shape in which the height of the combined distribution curve 1806 in FIG. That is, an image based on this distribution is a 2D image with a parallax amount of zero.
  • the adjusted Lt distribution curve 1905 is a curve obtained by adding 80% of the Lt distribution curve 1901 and 20% of the Rt distribution curve 1902.
  • the peak of the adjusted Lt distribution curve 1905 is displaced closer to the center than the peak of the Lt distribution curve 1901 as much as the component of the Rt distribution curve 1902 is added.
  • the adjusted Rt distribution curve 1906 is a curve obtained by adding 20% of the Lt distribution curve 1901 and 80% of the Rt distribution curve 1902.
  • the peak of the adjusted Rt distribution curve 1906 is displaced closer to the center than the peak of the Rt distribution curve 1902 by the amount to which the component of the Lt distribution curve 1901 is added.
  • the adjusted parallax amount represented by the distance between the peaks of the adjusted Lt distribution curve 1905 and the adjusted Rt distribution curve 1906 is smaller than the 3D parallax amount. Therefore, the stereoscopic effect during image reproduction is alleviated.
  • the spread of each of the adjusted Lt distribution curve 1905 and the adjusted Rt distribution curve 1906 is equivalent to the spread of the 2D distribution curve 1903, it can be said that the amount of blur is equal to that of the 2D image.
  • the amount of adjustment parallax can be controlled by how much the Lt distribution curve 1901 and the Rt distribution curve 1902 are added. Then, by applying this adjusted pixel value distribution to each plane of color image data generated from pixels without parallax, the color of the left viewpoint that gives a stereoscopic effect different from that of parallax image data generated from parallax pixels Image data and right-view color image data can be generated.
  • left-view color image data and right-view color image data are generated from the nine plane data described with reference to FIG.
  • Color image data of the left viewpoint RLt c plane data is red plane data corresponding to the left viewpoint, a green plane data GLt c plane data, and three color parallax plane data BLt c plane data is blue plane data Consists of.
  • the color image data of the right-side perspective is, RRT c plane data is red plane data corresponding to the right viewpoint, a green plane data GRT c plane data, and three of BRt c plane data is blue plane Datacolor Consists of parallax plane data.
  • FIG. 7 is a diagram for explaining color parallax plane data generation processing.
  • a generation process of RLt c plane data and RRt c plane data which are red parallax planes among color parallax planes, will be described.
  • the red parallax plane is generated using the pixel value of the Rn plane data described with reference to FIG. 3, and the pixel value of the RLt plane data and the RRt plane data. Specifically, for example, when calculating the pixel value RLt mn of the target pixel position (i m , j n ) of the RLt c plane data, first, the parallax image data generation unit 233 of the image processing unit 205 stores the Rn plane data A pixel value Rn mn is extracted from the same pixel position (i m , j n ).
  • the parallax image data generating unit 233 the same pixel position of RLt plane data (i m, j n) pixel values RLt mn from the same pixel position of RRt plane data (i m, j n) pixel values from RRt Extract mn . Then, the parallax image data generation unit 233 multiplies the pixel value Rn mn by the value obtained by distributing the pixel values RLt mn and RRt mn by the value of the stereoscopic adjustment parameter C, thereby calculating the pixel value RLt cmn . Specifically, it is calculated by the following equation (1).
  • the parallax image data generation unit 233 uses the extracted pixel value Rn mn and the pixel value RLt mn as well.
  • the pixel value RRt mn is calculated by multiplying the value obtained by distributing the three-dimensional adjustment parameter C by the value. Specifically, it is calculated by the following equation (2).
  • the parallax image data generation unit 233 sequentially executes such processing from (1, 1) which is the pixel at the left end and the upper end to (i 0 , j 0 ) which is the coordinates at the right end and the lower end.
  • the same pixel position of RLt plane data (i m, j n) from instead of extracting the pixel value RLt mn extracts the pixel value GLt mn from the same pixel position of GLt plane data (i m, j n).
  • the same pixel position of the RRT plane data (i m, j n) from instead of extracting the pixel value RRT mn extracts the pixel value GRT mn from the same pixel position of GRT plane data (i m, j n) .
  • the value of each parameter of Formula (1) and Formula (2) is changed suitably, and it processes similarly.
  • the generation processing of the GLt c plane data and the GRt c plane data which are green parallax planes is completed, the generation processing of the BLt c plane data and BRt c plane data which are blue parallax planes is executed next.
  • the pixel same pixel position (i m, j n) of Rn plane data in the above description instead of extracting the pixel values Rn mn from the same pixel position of Bn plane data (i m, j n) from Extract the value Bn mn .
  • the same pixel position of RLt plane data (i m, j n) from instead of extracting the pixel value RLt mn extracts the pixel value BLt mn from the same pixel position of BLt plane data (i m, j n).
  • the same pixel position of the RRT plane data (i m, j n) from instead of extracting the pixel value RRT mn extracts the pixel value BRt mn from the same pixel position of BRt plane data (i m, j n) .
  • the value of each parameter of Formula (1) and Formula (2) is changed suitably, and it processes similarly.
  • left-view color image data (RLt c- plane data, GLt c- plane data, BLt c- plane data) and right-view color image data (RRt c- plane data, GRt c- plane data, BRt c- plane data) Is generated. That is, the color image data of the left viewpoint and the right viewpoint can be acquired by a relatively simple process as a virtual output that does not actually exist as a pixel of the image sensor 100.
  • the value of the stereoscopic adjustment parameter C can be changed within the range of 0.5 ⁇ C ⁇ 1, the amount of parallax as a 3D image is adjusted while maintaining the amount of blur of the 2D color image due to pixels without parallax. be able to. Therefore, if these image data are reproduced by a 3D image compatible reproduction device, the viewer of the stereoscopic video display panel can appreciate the 3D video in which the stereoscopic effect is appropriately adjusted as a color image.
  • the processing is simple, it is possible to generate image data at high speed and to deal with moving images.
  • the value of the stereoscopic adjustment parameter C is determined and used in a range of 0.5 ⁇ C ⁇ 1 for each subject.
  • the advantage of generating color parallax plane data using the stereoscopic adjustment parameter C determined for each subject will be described.
  • FIG. 8 is a diagram schematically showing the relationship between the parallax amount and the value of the three-dimensional adjustment parameter.
  • the horizontal axis represents the distance from the digital camera 10, that is, the depth with respect to the scene, and the vertical axis represents the amount of parallax.
  • the digital camera 10, the object located at a distance L 10 (see focus object in the drawing) are focused.
  • the threshold value of the allowable amount of parallax is predetermined as ⁇ m.
  • the plurality of subjects included in the captured image data have unadjusted parallax amounts in the left and right parallax image data.
  • the value of the three-dimensional adjustment parameter C is determined for each subject according to such an unadjusted parallax amount, and is used for adjusting the parallax amount.
  • the parallax amount m 10 unadjusted this value C 10 is adjusted to the target value m 100 of the parallax amount.
  • the target value m 100 of the parallax amount m 10 and the parallax amount of unadjusted are both 0.
  • the value of the stereoscopic adjustment parameter C is determined as C 20 for the near-point subject, and the unadjusted parallax amount m 20 is adjusted to the parallax amount target value m 200 (m 200 ⁇ m) by this value C 20 .
  • the value of the stereoscopic adjustment parameter C is determined as C 30 for the far-point subject, and the unadjusted parallax amount m 30 is set to the parallax amount target value m 300 (m 300 ⁇ ⁇ m) by this value C 30 . It has been adjusted.
  • the unadjusted parallax amount for this subject is a value indicated by the parallax amount curve 1622 in the figure.
  • the target value of the parallax amount is a value indicated by the adjusted parallax amount curve 1623 in the drawing.
  • the parallax amount curve 1622 in the figure represents the relationship between the distance from the digital camera 10 and the unadjusted parallax amount generated in each subject when it is assumed that the subject is located at each distance.
  • the parallax amount curve 1622 includes a region outside the range of ⁇ m to + m. From the viewpoint of facilitating understanding, in the figure, the region outside the range of ⁇ m to + m in the parallax amount curve 1622 is surrounded by the frame W1, and the region within the range of ⁇ m to + m is surrounded by the frame W2. Yes.
  • the adjusted parallax amount curve 1623 represents the relationship between the distance from the digital camera 10 and the target value of the parallax amount generated in each subject when it is assumed that the subject is located at each distance.
  • the target value of the parallax amount indicated by the adjusted parallax amount curve 1623 is included in the range of ⁇ m to + m within the distance range in which the parallax amount curve 1622 is set.
  • the adjustment parallax amount curve 1623 as described above has a shape in which different values of the three-dimensional adjustment parameter C are set for a subject at least in the range of ⁇ m to + m and a subject outside the range of ⁇ m to + m. It has become. The specific shape will be described later.
  • the depth relationship between subjects that is, the relationship between the distance from the digital camera 10 and the parallax amount is not broken. Specifically, the greater the distance from the digital camera 10, the smaller the parallax amount. When the parallax amount exceeds 0, the negative value increases. In other words, the parallax amount m a distance greater object from the digital camera 10, the distance and the parallax amount m b a small object meets m b ⁇ m a.
  • the parallax amount larger than + m is adjusted to be + m which is the upper limit of the allowable parallax amount, and the parallax amount smaller than ⁇ m is the lower limit of the allowable parallax amount ⁇ m It is adjusted to become. In this way, the parallax of the subject that is not the main subject is not suppressed more than necessary, and the depth relationship of the subject is not disrupted.
  • the unadjusted parallax amount along the parallax amount curve 1622 is generally maintained for the subject in the range where the unadjusted parallax amount is in the range of ⁇ m to + m. Then, the value of the three-dimensional adjustment parameter C is determined and the amount of parallax is adjusted. As a result, the stereoscopic effect is maintained in the subject within the range of ⁇ m to + m.
  • adjusting the parallax amount so that the unadjusted parallax amount is generally maintained means that the parallax amount is adjusted so that the unadjusted parallax amount is maintained or becomes a value close to the unadjusted parallax amount. Means to be adjusted.
  • the stereoscopic adjustment parameter C different from that of the subject in the range of ⁇ m to + m is included so as to be included in the range of ⁇ m to + m. Is determined and the amount of parallax is adjusted. As a result, the amount of parallax is suppressed in subjects outside the range of ⁇ m to + m, and fatigue and discomfort during viewing are reduced.
  • the amount of parallax between a subject with a large unadjusted amount of parallax and a subject with a small amount of parallax can be intentionally varied. For this reason, it is possible to reduce the amount of parallax for a non-main subject while maintaining a stereoscopic effect for the main subject, and to reduce the sense of discomfort and fatigue during viewing.
  • the unadjusted parallax amounts are the same for the plurality of subjects, and therefore the values of the three-dimensional adjustment parameter C for adjusting the parallax amount are the same. Are also equal. From this, for each subject included in the image, the value of the stereoscopic adjustment parameter C is determined according to the unadjusted parallax amount. The unadjusted parallax amounts for a plurality of subjects included in the image Is the same as determining the value of the three-dimensional adjustment parameter C.
  • the principle of determining the three-dimensional adjustment parameter C for each subject will be described.
  • the principle described below is for facilitating understanding of a lookup table 2310 described later. Therefore, the value of the stereoscopic adjustment parameter C does not necessarily have to be determined in the digital camera 10 according to this principle.
  • the three-dimensional adjustment parameter C is such that the unadjusted parallax amount is generally maintained for subjects in the range of ⁇ m to + m among a plurality of subjects included in the image. A value is calculated. For a subject outside the range of ⁇ m to + m, the value of the stereoscopic adjustment parameter C is determined so that the parallax amount is adjusted within the range of ⁇ m to + m.
  • the parallax amount of the subject is calculated from the left and right parallax image data.
  • the unadjusted parallax amounts m 10 , m 20 , and m 30 for the focused subject, the near point subject, and the far point subject are detected, respectively.
  • the distance from the digital camera 10 to the subject is calculated based on the defocus amount from the focal position. Thereby, for example, distances L 10 , L 20 , and L 30 from the digital camera 10 to the focused subject, the near point subject, and the far point subject are calculated.
  • the target value of the parallax amount corresponding to the calculated distance is determined from the adjusted parallax amount curve 1623. Specifically, a point corresponding to the calculated distance is determined from the adjusted parallax amount curve 1623, and the vertical coordinate of this point is determined as the target value of the parallax amount.
  • the target value m 100 , the target value m 200 , and the target value m 300 of the parallax amount for the focused subject, the near point subject, and the far point subject are determined.
  • the value of the stereoscopic adjustment parameter C is calculated so that the unadjusted parallax amount is adjusted to the target value of the parallax amount for each subject.
  • the value of the stereoscopic adjustment parameter C is calculated so that the parallax amount m 10 is substantially maintained.
  • the value C 20 as parallax amount m 20 unadjusted for near-point object is adjusted to the target value m 200 of the parallax amount is calculated.
  • a value C 30 is calculated such that the unadjusted parallax amount m 30 for the far point subject is adjusted to the target value m 300 for the parallax amount.
  • a 3D image is generated using the certain stereoscopic adjustment parameter C. After that, processing for calculating the parallax amount of the subject is performed. Then, by repeating this process while feeding back the calculation result, the value of the stereo adjustment parameter C when the unadjusted parallax amount is adjusted to the target value of the parallax amount is calculated.
  • a lookup table is generated in advance from the correspondence between the unadjusted parallax amount obtained in this way, the target value of the parallax amount, and the value of the stereoscopic adjustment parameter C, and this lookup table is used. Then, the value of the three-dimensional adjustment parameter C is calculated.
  • the adjusted parallax amount curve 1623 is composed of a part in the area surrounded by the frame W1 and a part in the area surrounded by the frame W2. Accordingly, the adjustment parallax amount curve 1623 can set different values of the stereoscopic adjustment parameter C for at least a subject within the range of ⁇ m to + m and a subject outside the range of ⁇ m to + m. .
  • a portion of the adjusted parallax amount curve 1623 in the region surrounded by the frame W1 is formed so that the target value of the parallax amount is included in the range of ⁇ m to + m.
  • the portion of the adjusted parallax amount curve 1623 in the region surrounded by the frame W ⁇ b> 2 is formed to approximate the parallax amount curve 1622.
  • the parallax amount smoothly transition between these subjects in the depth direction.
  • the adjusted parallax amount curve 1623 is preferably continuous in the depth direction, that is, the distance direction from the digital camera 10, and more preferably the differential value is continuous.
  • the value of the stereoscopic adjustment parameter C is determined so that the adjusted parallax amount is continuous in the depth direction. Further, the value of the stereoscopic adjustment parameter C is determined so that the differential value of the adjusted parallax amount is continuous in the depth direction.
  • the shape of the portion of the adjusted parallax amount curve 1623 in the region surrounded by the frame W2 may be adjusted. Specifically, among the parts in the region surrounded by the frame W2, the part near the boundary with the frame W1 may be deformed so as to be continuous with the part surrounded by the frame W1.
  • the shape of the portion of the adjusted parallax amount curve 1623 in the region surrounded by the frame W2 may be adjusted. Specifically, among the portions in the region surrounded by the frame W2, a portion near the boundary with the frame W1 may be deformed so that the differential value is continuous with respect to the portion surrounded by the frame W1.
  • the adjusted parallax amount curve 1623 for example, a curve that overlaps with the parallax amount curve 1622 in at least a part of the parallax amount section in the range of ⁇ m to + m and has ⁇ m as an asymptotic line is used.
  • a curve is derived from a hyperbolic tangent curve.
  • a portion surrounded by the frame W2 in the adjusted parallax amount curve 1623 is generated from the parallax amount curve 1622. Then, the portion in the region surrounded by the frame W2 is deformed so that the portion near the boundary with the frame W1 is continuous with the portion surrounded by the frame W1. At this time, the differential value of the adjusted parallax amount curve 1623 is made continuous at the boundary between the frame W1 and the frame W2. Thereby, the adjusted parallax amount curve 1623 is selected.
  • the functions of the parallax amount curve 1622 and the adjusted parallax amount curve 1623 described above depend on the imaging conditions that affect the parallax amount (for example, the aperture value, the focus position, the focal length when the photographing lens 20 is a zoom lens, etc.), respectively. Can change. Therefore, when the value of the stereoscopic adjustment parameter C is determined using the adjustment parallax amount curve 1623 according to the principle described above, the function of the adjustment parallax amount curve 1623 can be stored in the digital camera 10 for each imaging condition. That's fine.
  • FIG. 9 is a diagram showing a lookup table 2310 stored by the determining unit 232 in the present embodiment.
  • the lookup table 2310 is a table that is referred to when the determination unit 232 determines the value of the stereoscopic adjustment parameter C.
  • This lookup table 2310 is stored in advance in the storage unit of the digital camera 10. As shown in FIG. 9, in the look-up table 2310, each value (m 10 , m 20 , m 30 ,%) That the parallax amount m can take, and the value of the stereo adjustment parameter C corresponding to each value (C 10 , C 20 , C 30 ,...) Are described in pairs.
  • the value of the stereoscopic adjustment parameter C corresponding to each value within the range of ⁇ m to + m is determined so that the unadjusted parallax amount is generally maintained. Yes. Further, the value of the stereo adjustment parameter C corresponding to each value outside the range of ⁇ m to + m is determined so that the parallax amount is adjusted within the range of ⁇ m to + m.
  • Such a lookup table 2310 is generated through an experiment using a prototype, for example. Specifically, in the prototype of the digital camera 10, an imaging condition that affects the amount of parallax is set to any arbitrary condition.
  • Each captured image data is associated with a distance to the subject.
  • the amount of parallax of the subject is calculated from the left and right parallax image data in each captured image data, and the correspondence data between the amount of parallax and the distance from the digital camera 10 to the subject at the time of shooting is plotted on the coordinate plane.
  • the horizontal axis is the distance from the digital camera 10, that is, the depth with respect to the scene
  • the vertical axis is the amount of parallax.
  • a parallax amount curve 1622 is generated by generating approximate curves of these plots. Once the parallax amount curve 1622 is generated, the adjusted parallax amount curve 1623 is then selected as described above.
  • parallax amount curve 1622 and the adjusted parallax amount curve 1623 are obtained, one point on the horizontal axis is selected, and an unadjusted parallax amount corresponding to this point is detected from the parallax amount curve 1622. That is, the distance from the digital camera 10 is selected, and the unadjusted parallax amount when the subject is located at this distance is detected.
  • the value of the stereoscopic adjustment parameter C is determined such that the detected unadjusted parallax amount is adjusted to the target value of the parallax amount.
  • the unadjusted parallax amount for example, the parallax amount m 10
  • the value of the stereoscopic adjustment parameter C for example, the value C
  • the stereoscopic image is adjusted so that the unadjusted parallax amount is within the range of ⁇ m to + m.
  • the value of the adjustment parameter C (for example, the value C 20 ) is determined. Therefore, since the parallax amount can be suppressed in the subject that is not the main subject while the stereoscopic subject is left as the main subject, it is possible to reduce a sense of discomfort and fatigue during viewing.
  • a certain stereoscopic adjustment parameter C is used. After the 3D image is generated, a process for calculating the parallax amount of the subject is performed. Then, by repeating this process while feeding back the calculation result, the value of the stereo adjustment parameter C when the unadjusted parallax amount is adjusted to the target value of the parallax amount is calculated.
  • a lookup table is generated in advance from the correspondence between the unadjusted parallax amount obtained in this way, the target value of the parallax amount, and the value of the stereoscopic adjustment parameter C, and this lookup table is used. Then, the value of the three-dimensional adjustment parameter C is calculated.
  • the unadjusted parallax amount for this point and the value of the stereoscopic adjustment parameter C are stored in the lookup table 2310 in association with each other. Thereafter, similarly, another point on the horizontal axis is selected, and the unadjusted parallax amount corresponding to that point and the value of the three-dimensional adjustment parameter C are stored in the lookup table 2310 in association with each other. As a result, a lookup table 2310 is generated.
  • the value of the stereoscopic adjustment parameter C can be determined from the unadjusted parallax amount without detecting the subject distance.
  • the value of the stereoscopic adjustment parameter C can be determined by the lookup table 2310 as long as an unadjusted parallax amount can be detected regardless of the arrangement of the subject in the scene.
  • the target value of the parallax amount is determined using the adjusted parallax amount curve 1623, and the depth relationship between the subjects (the relationship between the distance from the digital camera 10 and the parallax amount) is not broken in this curve. Therefore, as long as the depth relationship of the subject is not broken in the captured image, the depth relationship of the subject is not broken even if the parallax amount is adjusted.
  • the same lookup table 2310 can be used regardless of the imaging conditions that affect the amount of parallax (aperture value, focus position, focal length when the taking lens 20 is a zoom lens). .
  • this point will be described with a specific example.
  • FIG. 10 is a diagram illustrating the relationship between the amount of parallax and the target value of the amount of parallax when the shape of the parallax amount curve varies depending on the shooting conditions.
  • the horizontal axis represents the distance from the digital camera 10
  • the vertical axis represents the amount of parallax.
  • the digital camera 10 focuses on a subject (see the focused subject in the figure) located at the distances L 11 and L 12 . Also, between the two diagrams shown in FIGS. 10A and 10B, the focal position and the aperture value as the shooting conditions that affect the parallax amount are changed. As a result, the parallax amount curve 1626 is changed. 1628 are different from each other.
  • the distance from the digital camera 10 to each subject is different from each other. Specifically, the distance from the digital camera 10 to the in-focus subject, in a scene represented by FIG. 10 (a) has a distance L 11, the distance L 12 in the scene represented by FIG. 10 (b) It has become.
  • the distance from the digital camera 10 to the near-point subject is the distance L 21 in the scene shown in FIG. 10A and the distance L 22 in the scene shown in FIG. Yes.
  • the distance from the digital camera 10 to the far point subject is the distance L 31 in the scene shown in FIG. 10A and the distance L 32 in the scene shown in FIG. Yes.
  • any parallax amount unadjusted for near-point object is a parallax amount m 20
  • the parallax amount unadjusted for far point object has a both parallax amount m 30.
  • the unadjusted parallax amount m 10 for the focused subject, the near point subject, and the far point subject in FIG. , M 20 , m 30 are respectively adjusted so that the parallax amounts m 100 , m 200 , m 300 are adjusted.
  • the unadjusted parallax amounts m 10 , m 20 , and m 30 become parallax amounts m 100 , m 200 , and m 300 for the focused subject, the near point subject, and the far point subject in FIG.
  • the value of the three-dimensional adjustment parameter C is determined so as to be adjusted.
  • the same effect as when the function of the adjusted parallax amount curve 1623 is stored in the digital camera 10 for each imaging condition can be obtained. That is, since the main subject can have a stereoscopic effect and the amount of parallax can be suppressed in a subject that is not the main subject, it is possible to reduce a sense of discomfort and fatigue during viewing.
  • FIG. 11 is a diagram for explaining changes in RGB pixel value distribution.
  • FIG. 11A shows a G (Lt) pixel, a G (Rt) pixel, and an R (Lt) pixel when a white subject light beam from an object point located at a position deviated by a certain amount from the focal position is received. , R (Rt) pixels, B (Lt) pixels, and B (Rt) pixels.
  • FIG. 11B shows R (N) pixels, G (N) pixels, and B (N) pixels that are non-parallax pixels when a white subject light beam from the object point in FIG. 11A is received. It is the graph which arranged the output value. It can be said that this graph also represents the pixel value distribution of each color.
  • FIG. 12 is a diagram illustrating the relationship between the vergence angle of the viewer and the amount of parallax.
  • the eyeball 50 represents the eyeball of the viewer, and the figure shows the right eye 51 and the left eye 52 being separated.
  • the display unit 40 reproduces non-adjusted image data whose parallax amount is not adjusted, and displays a subject 61 for the right-eye image and a subject 62 for the left-eye image.
  • Object 61 and the object 62 are the same object, so were present at a position shifted from the focal position at the time of shooting, the display unit 40 is displayed at a distance with a disparity amount D 1.
  • the viewer views the position of the lifting distance L1 (in the drawing) where the straight line connecting the right eye 51 and the subject 61 and the straight line connecting the left eye 52 and the subject 62 intersect. (Represented by a square).
  • the convergence angle at this time is ⁇ 1 as shown in the figure.
  • ⁇ 1 the convergence angle at this time.
  • the video is uncomfortable and causes eye strain. Therefore, when image processing is performed using the stereoscopic adjustment parameter in the present embodiment, adjusted image data in which the parallax amount is adjusted by the stereoscopic adjustment parameter as described above is generated.
  • the figure shows a state where the adjusted image data is reproduced over the non-adjusted image data.
  • the display unit 40 displays a subject 71 of the right-eye image and a subject 72 of the left-eye image of the adjustment image data.
  • the subject 71 and the subject 72 are the same subject, and the subjects 61 and 62 are also the same subject.
  • Object 71 and the object 72, the display unit 40 is displayed at a distance with a disparity amount D 2.
  • the viewer recognizes that the subject exists at the position of the lifting distance L2 (represented by a triangle in the figure) where the straight line connecting the right eye 51 and the subject 71 intersects with the straight line connecting the left eye 52 and the subject 72.
  • the convergence angle at this time is ⁇ 2 smaller than ⁇ 1 . Therefore, the viewer can feel the extreme feeling of lifting and can reduce the accumulation of eye strain. Note that the amount of parallax is appropriately adjusted as will be described later, so that the viewer can appreciate the video with a comfortable floating feeling (a three-dimensional effect with a feeling of depression when the defocus relationship is reversed).
  • parallax amount used as description of FIG. 12 was represented by the separation distance in the display part 40, a parallax amount can be defined in various formats. You may define by the pixel unit in picked-up image data, and you may define by the shift
  • the adjustment of the parallax amount can be executed by various methods without using the method of changing the value of the three-dimensional adjustment parameter C.
  • a method of adjusting the parallax amount without changing the value of the three-dimensional adjustment parameter C will be described.
  • FIG. 13 is a diagram schematically illustrating the relationship between the contrast indicating the sharpness of an image and the amount of parallax.
  • the horizontal axis represents the distance from the digital camera 10, and the vertical axis represents the amount of parallax and the height of contrast.
  • the digital camera 10 is focused on the main subject is located at a distance L p.
  • the contrast curve 1610 forms the highest convex curve at the distance L p that is the distance to the focal position. That illustrates how gradually blurred with increasing distance from the distance L p back and forth.
  • Parallax amount curve 1620 at a distance L p indicates parallax amount 0, than the distance L p approaches the digital camera 10 side, shows the curve slope increases. That is, the parallax amount curve 1620 shows a positive value on the near side of the distance L p , and indicates that the closer the subject is, the higher the image is visually recognized.
  • the parallax amount curve 1620 than the distance L p As the distance from the digital camera 10 side, shows the curve slope becomes smaller. That is, the parallax amount curve 1620 distance L p from indicate a negative value in the back side, it represents that it is visible sinks slowly as more distant object.
  • the subject composing the scene moves from the distance L f (the amount of parallax at this time is + m) to the distance L r ( The amount of parallax at this time may be distributed between -m). That is, if the closest near subject from the digital camera 10 exists at the distance L f and the farthest far subject exists at the distance L r , the viewer can adjust the amount of parallax without adjusting the amount of parallax in the subsequent image processing. You can enjoy 3D video comfortably.
  • the near-point object is in front of the distance L f than the distance L f '(parallax amount at this time is + m') are present, since exceeds the parallax amount allowed, viewer discomfort, fatigue Learn.
  • FIG. 14 is a diagram schematically illustrating the relationship between the subject distribution and the amount of parallax.
  • FIG. 14 corresponds to the diagram in FIG. 11 excluding the contrast curve 1610.
  • the in-focus object distance L 10 near point subject to L 20, far point object is present in L 30.
  • the parallax amount range set as the allowable range is from ⁇ m to + m
  • the value of the parallax amount curve 1620 with respect to the distance L 30 of the far-point subject is within this range.
  • the value of the parallax amount curve 1620 with respect to the distance L 20 of near-point object is over + m.
  • 14 (b) is a diagram of the subject situation, showing the concept of parallax amount when focus object is moved from the distance L 10 to the back side of the distance L 11 in FIG. 14 (a).
  • the distance L 11 is the focal position
  • the parallax amount relative to the image of the near point object has not moved (the distance L 20), as indicated by the parallax amount curve 1620, in comparison with FIG. 14 (a) It becomes considerably large. That is, the excess amount from the allowable range increases.
  • FIG. 14 (c) shows the object status of FIG. 14 (b), the near point object from the distance L 20 to the back side of the distance L 21, the concept of parallax amount when further moved to the distance L 22 It is.
  • focus position parallax amount curve 1620 remain such because of the distance L 11 draw the same curve as FIG. 14 (b), the by near point object is shifted to the rear side, the parallax amount at the time of the distance L 21 is acceptable Although exceeding the range, the excess amount is smaller than the excess amount of FIG. If further moved until the distance L 22, the parallax amount is within an allowable range.
  • the subject distribution in the depth direction with respect to the scene and the position of the subject to be focused are parameters that determine whether or not the parallax amount falls within the set allowable range.
  • FIG. 15 is a diagram schematically illustrating the relationship between the aperture value and the amount of parallax.
  • the horizontal axis represents the distance from the digital camera 10
  • the vertical axis represents the amount of parallax and the height of contrast.
  • 15A shows a state where the aperture value is F1.4
  • FIG. 15B shows a state where the aperture value is F4
  • FIG. 15C shows a state where the aperture value is F8.
  • the focal length of the taking lens 20 are the same in both states, also, the digital camera 10 is focused on the main subject is located at a distance L 10.
  • Contrast curve 1610 the highest in the distance L 10 is a distance to be focal positions in any state.
  • the aperture 22 that is, as the aperture value is increased, a relatively high value is obtained even before and after the focal length. That is, it shows that the depth of field becomes deeper as the image is taken with the aperture 22 being reduced.
  • Parallax amount curve 1620 at a distance L 10 shows the parallax amount 0, approaches the digital camera 10 side than the distance L 10, shows the curve slope increases.
  • the parallax amount curve 1620 as the distance from the digital camera 10 side than the distance L 10, shows the curve slope becomes smaller.
  • the parallax amount curve 1620 becomes gentler as the aperture value increases. That is, as compared with the case where the aperture value is F1.4, the amount of parallax in front of the focal position and the amount of parallax in the back become smaller as F4 and F8 change. If the viewer does not feel discomfort and fatigue when the amount of parallax falls within the range of ⁇ m to + m, the entire parallax amount curve 1620 falls within this range when the aperture value is F8. Even if the subject is present at any distance, the viewer can comfortably appreciate the 3D video.
  • the parallax amount exceeds + m on the short distance side of the parallax amount curve 1620.
  • the parallax amount curve 1620 at F4 exceeds + m in front of the area than the distance L 25.
  • the slope of the parallax amount curve 1620 at F4 is gentler than the slope of the parallax amount curve 1620 at F1.8, the relationship of L 25 ⁇ L 24 is established.
  • the change condition for changing the parallax amount is changed so that the parallax amount between the generated images falls within the target parallax amount (allowable parallax amount: for example, a range of ⁇ m). .
  • an imaging condition that affects the amount of parallax is changed, or a value of a stereoscopic adjustment parameter used for image processing is changed.
  • the aperture value affects the amount of parallax. Therefore, the aperture value may be changed according to the detected subject distribution so that the amount of parallax between output parallax images is within the allowable amount of parallax. For example, in the situation of FIG. 15A (the initial aperture value is F1.4, the focused subject is the distance L 10 ), and the near-point subject is present at the distance L 25 , the amount of parallax exceeds + m. Therefore, the determination unit 232 changes the aperture value from F1.4, at a distance L 25 in the F4 is a aperture value parallax amount is + m with respect to the subject.
  • the aperture value is changed to a large value not only when the near-point subject exceeds the allowable parallax amount range but also when the far-point subject exceeds the allowable parallax amount range.
  • the aperture value may be changed to a small value, that is, the direction in which the aperture 22 is opened.
  • the shutter speed can be changed to the high speed side
  • the ISO sensitivity can be changed to the low sensitivity side.
  • the relationship between the in-focus subject distance and the parallax amount curve 1620 for each aperture value is prepared in advance as a lookup table.
  • the determination unit 232 can extract and determine the aperture value to be changed by referring to the lookup table with the subject distribution and the allowable parallax amount as input values.
  • FIG. 16 is a diagram schematically showing the concept of focus shift.
  • the vertical axis and the horizontal axis are the same as those in FIG.
  • Contrast curves 1610 and the parallax amount curve 1620 focusing subject exists at a distance L 10, represents the contrast curve and a parallax amount curve when focused on the subject by moving the focus lens.
  • the peak value of the contrast curve 1610 is higher than the focus threshold E s is evaluated with focusing.
  • the parallax amount is + m 0 when referring to the parallax amount curve 1620, which exceeds the allowable parallax amount + m. Therefore, in the focus shift to correct the focus lens position in a range above the focus threshold E s, it falls within the acceptable range parallax amount at the distance L 27.
  • a parallax amount curve 1621 where the parallax amount with respect to the near-point subject is + m is selected, and a distance L p where the parallax amount is 0 in the parallax amount curve 1621 is extracted. Then, by changing the focus lens position, the distance L p as the focusing position.
  • the contrast curve 1611 is a contrast curve at this time. Since the object is present in the distance L 10 in fact, the contrast value for the subject is reduced by ⁇ e as shown. Contrast value at this time has only to above the focus threshold E s. In this way, an image shot with the focus lens position changed can be evaluated as in-focus as the image, although the contrast value for the main subject is slightly reduced, and the parallax amount for the near-point subject is within an allowable range. Yes.
  • the correction of the focusing lens position is not allowed. That is, when the parallax amount relative to the near-point object in parallax amount curve 1620 largely exceeds the allowable amount of parallax, changing the focus lens position in a range above the focus threshold E s, fit the parallax amount within the allowable range I can't. In this case, it may be used in combination with other methods such as changing the aperture value to a large value.
  • a lookup table prepared in advance as a relationship between the focused subject distance and the parallax amount curve for each aperture value may be used.
  • the determination unit 232 can extract and determine the distance L p by referring to the lookup table using the subject distribution and the allowable parallax amount as input values.
  • the control unit 201 corresponds to the distance L p to change the position of the focus lens. Control unit 201, the contrast value obtained as a result of determining whether above the focus threshold E s. If it is determined that it exceeds, the photographing sequence is continued as it is. If it is determined that the value does not exceed, the focus lens position is returned and the control shifts to another method.
  • actual control unit 201 without moving the focus lens whether the focal position is the attenuation of contrast when shifted to the L p determination unit 232 calculates the L 10, above the focus threshold E s It may be judged.
  • the contrast AF method when the focus adjustment with respect to the distance L 10, may also refer to the actual evaluation value has already been obtained.
  • the lookup table 2310 for determining the value of the three-dimensional adjustment parameter C is an imaging condition that affects the amount of parallax (a diaphragm value, a focus position, and a case where the photographing lens 20 is a zoom lens). The same can be used regardless of the focal length. Therefore, the parallax amount adjustment method as described above can be used in combination with a method of adjusting the parallax amount by changing the value of the three-dimensional adjustment parameter C.
  • FIG. 17 is a diagram schematically illustrating the relationship between the parallax amount, the value of the three-dimensional adjustment parameter, the aperture value, and the like.
  • the horizontal axis represents the distance from the digital camera 10
  • the vertical axis represents the amount of parallax.
  • the digital camera 10, the object located at a distance L 10 (see focus object in the drawing) are focused. However, the image captured by the digital camera 10, in addition to the subject located at a distance L 10, the distance L 20 and the distance L 2 one object located 30 (see a near point object and far-point object in the drawing) It is included.
  • the unadjusted parallax amounts for these three subjects are the parallax amount m 10 , the parallax amount m 20, and the parallax amount m 30 .
  • the determination unit 232 adjusts the parallax amount by changing the aperture value.
  • the parallax amount curve 1622 is transformed into the parallax amount curve 1624, and the parallax amounts given to the focused subject, the near point subject, and the far point subject become the parallax amount m 101 , the parallax amount m 201, and the parallax amount m 301 .
  • the focus position may be changed instead of the aperture value.
  • the calculation unit 231 does not adjust the parallax amount m 101 , the parallax amount m 201, and the parallax amount m 301 adjusted by changing the aperture value, the focus position, and the like, depending on the value of the stereoscopic adjustment parameter C.
  • the parallax amount for adjustment is calculated from the left and right parallax image data.
  • the determination unit 232 uses the lookup table 2310 and uses the look-up table 2310 to create a solid corresponding to the unadjusted parallax amount m 101 , parallax amount m 201, and parallax amount m 301.
  • the value of the adjustment parameter C is determined for each subject.
  • the value of the stereo adjustment parameter C determined in this way is equal to the value determined by the following method. That is, first, the adjusted parallax amount curve 1625 is generated with respect to the parallax amount curve 1624 instead of the parallax amount curve 1622. Then, the value of the stereoscopic adjustment parameter C is calculated such that the unadjusted parallax amount is adjusted to the target value of the parallax amount. In this case, for a subject whose parallax amount exceeds the allowable parallax amount, the parallax is suppressed so that the viewer does not feel discomfort or fatigue, and for a subject whose parallax amount falls within the allowable parallax amount, the parallax is more emphasized. be able to.
  • an adjusted parallax amount curve having a different shape can be obtained.
  • the same effect as that used when determining the value can be obtained. Therefore, variations in the adjustment amount of the parallax amount can be increased.
  • a function used to determine the target value of the parallax amount can be set in advance so that the adjusted parallax amount curve 1625 is obtained for each imaging condition. In this case, the amount of data increases, but the amount of parallax can be adjusted without changing the shooting conditions.
  • FIG. 18 is a diagram for explaining subject designation.
  • FIG. 18A shows a subject distribution in the depth direction from the digital camera 10 in a certain scene
  • FIG. 18B is a rear view of the digital camera 10 displaying the scene in a live view.
  • the scene is composed of an adult 301 (distance L f ), a boy 302 (distance L p ), and a girl 303 (distance L r ) in order from the digital camera 10. Then, as shown in FIG. 18B, a live view image that captures this scene is displayed on the display unit 209.
  • the boy 302 is the focused subject.
  • the amount of parallax can be suppressed by increasing the amount of adjustment of the amount of parallax for subjects that are not the main subject for the photographer. Is preferred. Accordingly, it is not always necessary to maintain the amount of parallax for a subject whose amount of parallax is in the range of ⁇ m to + m.
  • the control unit 201 receives a photographer's instruction as to which subject should be the main subject.
  • the display unit 209 displays a title 320 (for example, “Please set the main subject area”) indicating that the user instruction is accepted. In this state, the user adjusts the position and size of the frame 310 to specify a range of the area including the subject image that is desired to be the main subject.
  • the display unit 209 is provided with a touch panel 2083 as a part of the operation unit 208, and the control unit 201 acquires the output of the touch panel 2083 and determines which subject is the main subject.
  • the subject of the adult 301 not included in the frame 310 is designated as the subject whose parallax amount is desired to be reduced.
  • the determination unit 232 determines that the unadjusted parallax amount is 0.5 from the value calculated from the lookup table 2310 regardless of whether or not the unadjusted parallax amount is within the range of ⁇ m to + m.
  • a value close to is determined as the value of the three-dimensional adjustment parameter C.
  • the adjustment amount of the parallax amount becomes larger and the parallax amount is suppressed as compared with the case where the value of the stereoscopic adjustment parameter C calculated from the lookup table 2310 is used as it is.
  • the amount of adjustment of the parallax amount for each subject is, for example, as shown superimposed on each subject in FIG.
  • these adjustment amounts mean that the degree of adjustment of the parallax amount is larger as the absolute value is larger.
  • a positive value for example, +2 means that the amount of parallax is adjusted in a direction in which the distance from the digital camera 10 increases, and a negative value (for example, ⁇ 1) decreases the distance from the digital camera 10. This means that the amount of parallax is adjusted in the direction.
  • the adjustment amount of the parallax amount is reduced and the stereoscopic effect is maintained in the boy 302 and the girl 303 as the main subjects.
  • the amount of parallax adjustment is increased and the amount of parallax is reduced.
  • FIG. 19 is a processing flow in moving image shooting according to the first embodiment.
  • the flow in the figure starts when the mode button is operated by the photographer and the auto 3D video mode is started.
  • the parallax amount range is set in advance by the photographer.
  • the determination unit 232 acquires the parallax amount range set by the photographer from the system memory in step S11. In step S12, the control unit 201 executes AF and AE.
  • the control unit 201 receives from the user, via the touch panel 2083, designation of a range of an area including a subject image that is desired to be a main subject, as described with reference to FIG.
  • the control unit 201 waits for a recording start instruction for the photographer to press the recording start button.
  • step S15 When the recording start instruction is detected (YES in step S15), the control unit 201 proceeds to step S17. If no instruction is detected, the process returns to step S12. Note that after returning to step S12, the specified subject may be tracked and the process of step S13 may be skipped.
  • step S17 the control unit 201 executes AF and AE again according to the changed imaging condition.
  • step S18 the control unit 201 performs charge accumulation and readout of the image sensor 100 via the drive unit 204, and acquires captured image data as one frame. The amount of parallax between the parallax images in the captured image data acquired here does not fall within the set amount of parallax depending on the subject distribution and the shooting conditions.
  • the parallax image data generation unit 233 receives the value of the stereoscopic adjustment parameter C determined by the determination unit 232 and the captured image data, and receives color image data (RLt c plane data, GLt c plane data, left viewpoint) BLt c plane data) and right viewpoint color image data (RRt c plane data, GRt c plane data, BRt c plane data) are generated. Specific processing will be described later.
  • step S19 for the case where the photographer wants to change the main subject during moving image shooting, the control unit 201 accepts from the user the range specification of the area including the subject image that he wants to be the main subject. If the control unit 201 determines in step S21 that it has not received a recording stop instruction from the photographer, it returns to step S17 and executes the next frame process. If it is determined that a recording stop instruction has been received, the process proceeds to step S22.
  • step S22 the moving image generating unit 234 connects the continuously generated left-viewpoint color image data and right-viewpoint color image data, and executes format processing according to a 3D-compatible moving image format such as Blu-ray 3D. Generate a file. Then, the control unit 201 records the generated moving image file on the memory card 220 via the memory card IF 207, and ends a series of flows.
  • a 3D-compatible moving image format such as Blu-ray 3D.
  • the recording to the memory card 220 may be sequentially executed in synchronization with the generation of the color image data of the left viewpoint and the color image data of the right viewpoint, and the file end process may be executed in synchronization with the recording stop instruction.
  • the control unit 201 is not limited to recording on the memory card 220, and may be configured to output to an external device via a LAN, for example.
  • FIG. 20 is a processing flow of step S33 until the color image data of the left viewpoint and the parallax color image data which is the color image data of the right viewpoint are generated.
  • the parallax image data generation unit 233 acquires captured image data in step S101.
  • the captured image data is plane-separated into image data without parallax and parallax image data.
  • the parallax image data generation unit 233 executes an interpolation process for interpolating vacancies existing in the separated plane data as described with reference to FIG.
  • the parallax image data generation unit 233 initializes each variable in step S104. Specifically, first, 1 is substituted into the color variable Cset.
  • the calculation unit 231 calculates an unadjusted parallax amount for the subject displayed at the pixel at the target pixel position (i, j) in step S107 from the left and right parallax image data.
  • the determination unit 232 acquires the value of the stereoscopic adjustment parameter C corresponding to the parallax amount calculated by the calculation unit 231 from the lookup table 2310.
  • the determination unit 232 applies to a subject that is determined not to be a main subject by the photographer even if the amount of parallax is within a range of ⁇ m to + m. A value closer to 1 than the value calculated from the lookup table 2310 may be determined as the value of the stereoscopic adjustment parameter C. Further, the determination unit 232 may calculate the luminance value of the subject included in the image, and further adjust the value of the stereoscopic adjustment parameter C so that the parallax amount decreases as the luminance value decreases.
  • the value of the stereoscopic adjustment parameter C is the value determined in step S108 for the subject displayed by the target pixel.
  • the parallax image data generation unit 233 increments the parallax variable S in step S110.
  • step S111 it is determined whether or not the parallax variable S exceeds 2. If not, the process returns to step S109. If it exceeds, the process proceeds to step S112.
  • step S112 the parallax image data generation unit 233 assigns 1 to the parallax variable S and increments the coordinate variable i. Then, in step S113, it is determined whether coordinate variable i exceeds i 0. If not, the process returns to step S105. If it exceeds, the process proceeds to step S114.
  • step S114 the parallax image data generation unit 233 assigns 1 to the coordinate variable i and increments the coordinate variable j. Then, in step S115, it is determined whether coordinate variable j exceeds j 0. If not, the process returns to step S105. If it exceeds, the process proceeds to step S116.
  • step S117 the parallax image data generation unit 233 assigns 1 to the coordinate variable j and increments the color variable Cset.
  • step S118 it is determined whether or not the color variable Cset exceeds 3. If not, the process returns to step S105. If it exceeds, color image data of the left viewpoint (RLt c plane data, GLt c plane data, BLt c plane data) and color image data of the right viewpoint (RRt c plane data, GRt c plane data, BRt c plane data) If all of the above are complete, the flow returns to the flow of FIG.
  • FIG. 21 is a processing flow in moving image shooting according to the second embodiment.
  • the processing related to each processing in the processing flow of FIG. 19 is denoted by the same step number, and the description thereof is omitted except for the description of different processing and additional processing.
  • step S15 when the control unit 201 detects a recording start instruction in step S15 (YES in step S15), the process proceeds to step S16.
  • step S16 the determination unit 232 changes the shooting condition, and proceeds to step S17.
  • step S16 the determination unit 232 changes the aperture value as described with reference to FIG. 15 or changes the focus lens position as described with reference to FIG.
  • an imaging condition that affects the amount of parallax may be changed so as to be within the set range of the amount of parallax. For example, if the photographing lens 20 is a zoom lens, the focal length can be changed.
  • step S107 in the process of step S33 the calculation unit 231 calculates an unadjusted parallax amount for the subject displayed by the pixel at the target pixel position (i, j) from the left and right parallax image data.
  • This parallax amount is the parallax amount adjusted by changing the aperture value, the focus position, and the like. Also, in this flow, if the control unit 201 determines in step S21 that a recording stop instruction has not been received from the photographer, the control unit 201 returns to step S16 and executes the next frame processing.
  • FIG. 22 is a diagram illustrating a preferred opening shape.
  • each of the openings 105 and 106 has a shape in contact with a virtual center line 322 passing through the center of the pixel or a shape straddling the center line 322.
  • the shape of the opening 105 and the shape of the opening 106 are preferably the same as the respective shapes obtained by dividing the shape of the opening 104 of the non-parallax pixel by the center line 322.
  • the shape of the opening 104 is preferably equal to the shape in which the shape of the opening 105 and the shape of the opening 106 are adjacent to each other.
  • the calculation formula used by the parallax image data generation unit 233 employs the above formulas (1) and (2) using the weighted arithmetic mean, but not limited to this, various calculation formulas are adopted. Can do. For example, if the weighted geometric mean is used, it can be expressed in the same manner as the above formulas (1) and (2) Can be adopted as a calculation formula. In this case, the amount of blur maintained is not the amount of blur due to the output of the non-parallax pixel but the amount of blur due to the output of the parallax pixel.
  • FIG. 23 is a diagram for explaining cooperation between the digital camera 10 and the TV monitor 80.
  • the TV monitor 80 includes a display unit 40 made of, for example, liquid crystal, a memory card IF 81 that receives the memory card 220 taken out from the digital camera 10, a remote controller 82 that is operated by a viewer at hand, and the like.
  • the TV monitor 80 is compatible with 3D image display.
  • the display format of the 3D image is not particularly limited.
  • the right-eye image and the left-eye image may be displayed in a time-division manner, or may be an interlace in which strips are arranged in a horizontal or vertical direction. Further, it may be a side-by-side format arranged on one side and the other side of the screen.
  • the TV monitor 80 decodes a moving image file that includes the color image data of the left viewpoint and the color image data of the right viewpoint, and displays a 3D image on the display unit 40.
  • the TV monitor 80 serves as a general display device that displays a standardized moving image file.
  • the TV monitor 80 can also function as an image processing apparatus that bears at least part of the function of the control unit 201 and at least part of the function of the image processing unit 205 described with reference to FIG.
  • an image processing unit including the calculation unit 231, the determination unit 232, the parallax image data generation unit 233, and the moving image generation unit 234 described in FIG. 1 is incorporated in the TV monitor 80.
  • the digital camera 10 does not perform image processing using the stereoscopic adjustment parameter, and associates the depth information detected by the depth information detection unit 230 with the generated captured image data.
  • the TV monitor 80 determines the value of the stereoscopic adjustment parameter C for each subject with reference to the associated depth information, and performs image processing using the stereoscopic adjustment parameter C for the acquired image data. Execute.
  • the TV monitor 80 displays the 3D image with the parallax amount adjusted in this way on the display unit 40.
  • the viewer may be configured to be able to input some adjustment information during playback on the TV monitor 80.
  • the viewer can input the parallax amount range by operating the remote controller 82.
  • the TV monitor 80 acquires the input parallax amount range as adjustment information, and the determination unit 232 determines the value of the stereoscopic adjustment parameter C according to the parallax amount range. If comprised in this way, the TV monitor 80 can display the 3D image according to the preference for every viewer.
  • the parallax amount of each of the plurality of subjects included in the image of the captured image data is calculated, and the value of the stereoscopic adjustment parameter C is set for each of the plurality of subjects according to the calculated amount of parallax. Then, the value of the stereoscopic adjustment parameter C is applied to the captured image data to generate parallax image data. Therefore, unlike the case where the value of the single stereoscopic adjustment parameter C is applied to the entire captured image data, the amount of parallax adjustment is intentionally different between the main subject and the subject that is not the main subject. be able to. Therefore, since the parallax amount can be suppressed in the subject that is not the main subject while the stereoscopic subject is left as the main subject, it is possible to reduce a sense of discomfort and fatigue during viewing.
  • the value of the stereoscopic adjustment parameter C is determined so that the parallax amount is adjusted to be less than the threshold value. A sense of discomfort and fatigue can be reliably reduced.
  • the value of the three-dimensional adjustment parameter C is determined so that the adjusted parallax amount is continuous in the depth direction with respect to these subjects. Is done. Therefore, it is possible to prevent the parallax amount from changing discretely between subjects that are continuous in the depth direction. Therefore, it is possible to reliably reduce a sense of incongruity and fatigue during viewing.
  • the stereoscopic adjustment parameter C is set so that the differential value of the adjusted parallax amount is continuous in the depth direction with respect to these subjects. The value of is determined. Therefore, the change in the amount of parallax between subjects that are continuous in the depth direction can be smoothed. Therefore, it is possible to reliably reduce a sense of incongruity and fatigue during viewing.
  • the subject for which the amount of parallax is to be suppressed is specified by the user from among a plurality of subjects included in the image by applying the value of the stereoscopic adjustment parameter C, the amount of parallax of the specified subject is suppressed. Accordingly, it is possible to reliably reduce a sense of incongruity and fatigue during viewing.
  • the brightness value of each of the plurality of subjects included in the image is calculated, and the value of the stereoscopic adjustment parameter C is determined so that the parallax amount decreases as the brightness value decreases. Therefore, it is possible to prevent a large parallax from being given to a region having a small luminance value in the image, that is, a region having low brightness. Therefore, it is possible to reliably reduce a sense of incongruity and fatigue during viewing.
  • the TV monitor 80 has been described as an example of the image processing apparatus.
  • the image processing apparatus can take various forms.
  • a device such as a PC, a mobile phone, or a game device that includes or is connected to the display unit can be an image processing apparatus.
  • the configuration of outputting image data in which the amount of parallax is adjusted based on the detected depth information can of course be applied to still image shooting.
  • the still image shot in this way does not cause extreme parallax between the left and right images, and does not give the viewer a sense of incongruity.
  • the target subject is accepted by the user's instruction, but the control unit 201 may automatically select the subject.
  • the control unit 201 can set only a human image included in the scene as a target subject through the human recognition process.
  • parallax image data may be generated from image data (such as a CG image) that is not a captured image.
  • the determination unit 232 is described as determining the value of the three-dimensional adjustment parameter C using one lookup table 2310.
  • a plurality of lookup tables 2310 may be stored in the determination unit 232, and the value of the stereoscopic adjustment parameter C may be determined using the lookup table 2310 selected by the photographer.
  • the amount of parallax adjustment for each subject can be selected.
  • selecting the lookup table 2310 is equivalent to selecting a function used to determine the target value of the parallax amount, that is, a function such as the adjusted parallax amount curve 1623.
  • the plurality of lookup tables 2310 can be generated from adjusted parallax amount curves having different shapes.
  • the value of the stereo adjustment parameter C is determined using the lookup table 2310.
  • the function of the adjustment parallax amount curve 1623 is stored in the digital camera 10 for each imaging condition in accordance with the above-described principle, and the stereoscopic adjustment parameter is obtained using these functions and information on the subject distribution and the focus position in the depth direction.
  • the value of C may be determined.
  • a plurality of functions of the adjustment parallax amount curve 1623 are stored in the digital camera 10 for the same shooting condition, and the user operates the operation unit 208 to change any one of the functions. You may select as an object of use.
  • the three-dimensional adjustment parameter C when some subjects are continuous in the depth direction of the scene, the three-dimensional adjustment parameter C is set so that the differential value of the adjusted parallax amount is continuous in the depth direction for these continuous subjects. Described as determining the value. However, in such a case, it is not always necessary to determine the value of the three-dimensional adjustment parameter C so that the differential values are continuous.
  • the value of the stereoscopic adjustment parameter C may be determined so that the parallax amount is adjusted to ⁇ m.
  • the differential value is discontinuous at the boundary between the frame W1 and the frame W2.
  • the parallax is not suppressed, so Will not be lost.
  • the parallax amount is smoothly shifted between these subjects in the depth direction.
  • the subject that is continuous in the depth direction of the scene may be a part of the subject included in the image.
  • the allowable parallax threshold has been described as ⁇ m.
  • values having different absolute values may be used for the upper limit value and the lower limit value of the allowable amount of parallax.
  • Each processing flow described in this embodiment is executed by a control program that controls the control unit.
  • the control program is recorded in a built-in nonvolatile memory, and is appropriately expanded in the work memory to execute each process.
  • the control program recorded in the server is transmitted to each device via the network, and is expanded in the work memory to execute each process.
  • a control program recorded on the server is executed on the server, and each device executes processing in accordance with a control signal transmitted via the network.

Abstract

L'invention est motivée par le fait que des données d'image stéréoscopique telles que capturées par un dispositif d'imagerie stéréoscopique pourraient présenter une parallaxe se manifestant à l'extrême entre des images gauche et droite en raison de la disposition d'un sujet dans une scène, et qu'un observateur pourrait ressentir une impression d'inconfort ou de fatigue en pareil cas. En conséquence, l'invention concerne un dispositif de traitement d'images comportant: une unité d'acquisition qui acquiert des données d'image; une unité de calcul qui calcule la quantité de parallaxe pour chaque sujet d'une pluralité de sujets figurant dans une image des données d'image; une unité de détermination qui détermine, pour chaque sujet de la pluralité de sujets, en fonction des quantités calculées de parallaxe, une valeur de paramètre d'adaptation qui est appliquée lorsque des données d'image avec parallaxe sont générées à partir des données d'image et qui adapte la quantité de parallaxe; et une unité de génération qui applique la valeur de paramètre d'adaptation aux données d'image et génère des données d'image avec parallaxe qui deviennent une image caractérisée par les quantités mutuellement adaptées de parallaxe.
PCT/JP2015/062202 2014-04-22 2015-04-22 Dispositif de traitement d'images, dispositif d'imagerie et programme de traitement d'images WO2015163350A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2014088496A JP2017108194A (ja) 2014-04-22 2014-04-22 画像処理装置、撮像装置及び画像処理プログラム
JP2014-088496 2014-04-22

Publications (1)

Publication Number Publication Date
WO2015163350A1 true WO2015163350A1 (fr) 2015-10-29

Family

ID=54332513

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2015/062202 WO2015163350A1 (fr) 2014-04-22 2015-04-22 Dispositif de traitement d'images, dispositif d'imagerie et programme de traitement d'images

Country Status (2)

Country Link
JP (1) JP2017108194A (fr)
WO (1) WO2015163350A1 (fr)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113099120A (zh) * 2021-04-13 2021-07-09 南昌虚拟现实研究院股份有限公司 深度信息获取方法、装置、可读存储介质及深度相机

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2011259012A (ja) * 2010-06-04 2011-12-22 Fujifilm Corp 立体画像再生装置及び立体画像再生方法
JP2011259045A (ja) * 2010-06-07 2011-12-22 Sony Corp 立体画像表示システム、視差変換装置、視差変換方法およびプログラム
JP2012213070A (ja) * 2011-03-31 2012-11-01 Toshiba Corp 映像信号生成装置、映像信号生成方法及び制御プログラム
WO2012153378A1 (fr) * 2011-05-06 2012-11-15 富士通株式会社 Dispositif de génération d'image stéréoscopique, procédé de génération d'image stéréoscopique, programme de génération d'image stéréoscopique
JP2013118515A (ja) * 2011-12-02 2013-06-13 Canon Inc 画像処理装置およびその方法
JP2013229764A (ja) * 2012-04-25 2013-11-07 Nikon Corp 画像処理装置、撮像装置および画像処理プログラム

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2011259012A (ja) * 2010-06-04 2011-12-22 Fujifilm Corp 立体画像再生装置及び立体画像再生方法
JP2011259045A (ja) * 2010-06-07 2011-12-22 Sony Corp 立体画像表示システム、視差変換装置、視差変換方法およびプログラム
JP2012213070A (ja) * 2011-03-31 2012-11-01 Toshiba Corp 映像信号生成装置、映像信号生成方法及び制御プログラム
WO2012153378A1 (fr) * 2011-05-06 2012-11-15 富士通株式会社 Dispositif de génération d'image stéréoscopique, procédé de génération d'image stéréoscopique, programme de génération d'image stéréoscopique
JP2013118515A (ja) * 2011-12-02 2013-06-13 Canon Inc 画像処理装置およびその方法
JP2013229764A (ja) * 2012-04-25 2013-11-07 Nikon Corp 画像処理装置、撮像装置および画像処理プログラム

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113099120A (zh) * 2021-04-13 2021-07-09 南昌虚拟现实研究院股份有限公司 深度信息获取方法、装置、可读存储介质及深度相机
CN113099120B (zh) * 2021-04-13 2023-04-18 南昌虚拟现实研究院股份有限公司 深度信息获取方法、装置、可读存储介质及深度相机

Also Published As

Publication number Publication date
JP2017108194A (ja) 2017-06-15

Similar Documents

Publication Publication Date Title
US8885026B2 (en) Imaging device and imaging method
US9077976B2 (en) Single-eye stereoscopic image capturing device
JP5368350B2 (ja) 立体撮像装置
JP5814692B2 (ja) 撮像装置及びその制御方法、プログラム
JP5788518B2 (ja) 単眼立体撮影装置、撮影方法及びプログラム
JP6036840B2 (ja) 撮像装置、画像処理装置、撮像装置の制御プログラムおよび画像処理装置の制御プログラム
WO2012001970A1 (fr) Dispositif, procédé et programme de traitement d'image
JP5449551B2 (ja) 画像出力装置、方法およびプログラム
JP2014036362A (ja) 撮像装置、その制御方法、および制御プログラム
US9124866B2 (en) Image output device, method, and recording medium therefor
JP6004741B2 (ja) 画像処理装置およびその制御方法、撮像装置
WO2015163350A1 (fr) Dispositif de traitement d'images, dispositif d'imagerie et programme de traitement d'images
JP6439285B2 (ja) 画像処理装置、撮像装置および画像処理プログラム
JP2012124650A (ja) 撮像装置および撮像方法
JPWO2012001958A1 (ja) 画像処理装置および方法並びにプログラム
JP2013046395A (ja) 撮像装置及びその制御方法、プログラム、及び記録媒体
JP6070061B2 (ja) 画像処理装置、撮影装置、画像処理方法およびプログラム
JP6070060B2 (ja) 撮像装置およびプログラム
JP2024008154A (ja) 画像処理装置、表示装置および画像処理方法
JP6255753B2 (ja) 画像処理装置および撮像装置
JP2014085608A (ja) 撮像装置

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 15782683

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

NENP Non-entry into the national phase

Ref country code: JP

122 Ep: pct application non-entry in european phase

Ref document number: 15782683

Country of ref document: EP

Kind code of ref document: A1