JP5565001B2 - Stereoscopic imaging device, stereoscopic video processing device, and stereoscopic video imaging method - Google Patents

Description

  The present invention relates to a stereoscopic video imaging apparatus, a stereoscopic video processing apparatus, and a stereoscopic video imaging method for generating stereoscopic video data for perceiving stereoscopic video using video data generated by a plurality of imaging units.

  In recent years, the technology of video (stereoscopic video) that presents two images with horizontal parallax (binocular parallax) on the display and makes the viewer perceive as if the subject exists stereoscopically has attracted attention. ing. Two images used in such a technique are images captured by two imaging units having different viewpoints.

  There has been proposed a stereoscopic video imaging apparatus that adjusts the convergence angle formed by these two imaging units in accordance with the subject distance from the apparatus to the subject to be imaged and images the object at the position of the convergence point (for example, a patent) Reference 1). In addition, a technique has been proposed in which a part of video data is cut out according to the subject distance and the convergence angle is electronically controlled (for example, Patent Document 2).

Japanese Patent Publication No. 6-66967 JP-A-7-95623

  By the way, the imaging device has been reduced in size and weight in accordance with the demand of an imager who wants to improve portability, and the lens and the imaging element have been reduced accordingly. However, downsizing of lenses and image sensors greatly affects the depth of field. For example, when the depth of field becomes deeper due to the downsizing of the lens, the in-focus range becomes wider in the imaging device, and objects and backgrounds before and after the subject (main subject) that is the imaging purpose of the photographer Video data in focus up to the subject (quasi-subject) is generated.

  The same can be said for the case of using a stereoscopic video imaging device as shown in Patent Documents 1 and 2 described above, when a human directly views the subject and when the subject is imaged through the imaging unit. The focus will be different. In such video data, the viewer visually recognizes objects other than the subject that is an imaging purpose that should not be perceived and objects such as the background. Since there is more visual information to be processed than (planar image), eyes are fatigued by long-term visual recognition.

  Therefore, in view of such problems, the present invention provides a stereoscopic video imaging device, a stereoscopic video processing device, and a stereoscopic video imaging method capable of generating stereoscopic video data that allows a viewer to perceive a natural stereoscopic video. The purpose is that.

To solve the above SL problem, steric image pickup apparatus of the present invention is arranged at a position where the optical axes intersect with substantially parallel or imaging direction, a plurality of imaging units for generating a video data in each video The distance to obtain the relative distance, which is the difference between the distance between the main subject that is the subject for imaging and the imaging unit, and the distance between the quasi-subject that is a subject other than the main subject and the imaging unit, among the subjects included in the data An acquisition unit; a video correction unit that changes the sharpness of one or both of the main subject and the sub-subject based on the relative distance; and the video data is divided into predetermined blocks, and the luminance of the pixels in the divided blocks is determined. A luminance difference deriving unit that derives a difference between the maximum value and the minimum value, and when the difference is equal to or less than a predetermined threshold, the video correction unit Characterized in that no correction.

To solve the above SL problem, steric image processing apparatus of the present invention includes: a video acquisition unit that acquires stereoscopic image data for perceive a stereoscopic image using binocular parallax, of the subject included in the stereoscopic image data The distance acquisition for acquiring the relative distance that is the difference between the distance between the main subject that is the subject for imaging and the imaging unit that generated the stereoscopic video data and the distance between the quasi-subject that is a subject other than the main subject and the imaging unit A video correction unit that changes the sharpness of one or both of the main subject and the quasi-subject based on the relative distance, the stereoscopic video data is divided into predetermined blocks, and the luminance of the pixels in the divided blocks A luminance difference deriving unit that derives a difference between the maximum value and the minimum value, and when the difference is equal to or less than a predetermined threshold, the video correction unit Characterized in that no correction.

To solve the above SL problem, steric image pickup method of the present invention, in order to generate a stereoscopic image, the plurality of imaging units each of the optical axis is arranged at a position intersecting substantially parallel or imaging direction, Video data is generated, and among the subjects included in the video data, the difference between the distance between the main subject that is the subject for imaging and the imaging unit and the distance between the quasi-subject that is a subject other than the main subject and the imaging unit Obtain a certain relative distance, divide the video data into predetermined blocks, derive the difference between the maximum and minimum luminance values of the pixels in the divided blocks, and determine which of the main subject and quasi-subject based on the relative distance One or both of the sharpnesses are changed, and when the difference is equal to or smaller than a predetermined threshold, the sharpness correction is not performed for the pixels of the block whose difference is equal to or smaller than the predetermined threshold.

  As described above, the present invention can generate stereoscopic video data that allows a viewer to perceive a natural stereoscopic video.

It is a functional block diagram showing a schematic function of a stereoscopic video imaging device in the first embodiment. It is the external view which showed an example of the three-dimensional video imaging device. It is explanatory drawing for demonstrating the measurement of main subject distance. It is explanatory drawing for demonstrating a relative distance and parallax. It is explanatory drawing for demonstrating the motion vector detection of a distance acquisition part. It is explanatory drawing for demonstrating correction | amendment of parallax. It is explanatory drawing for demonstrating the example of a response | compatibility with the parallax used as a relative distance, and a definition. It is explanatory drawing for demonstrating correction | amendment of a sharpness. It is explanatory drawing for demonstrating the relationship between the main to-be-photographed object and parallax in video data when fixing a convergence angle. It is explanatory drawing for demonstrating the relationship between the main to-be-photographed object and parallax in video data when fixing a convergence angle. It is explanatory drawing for demonstrating adjustment of the convergence angle by video processing. It is explanatory drawing for demonstrating adjustment of the convergence angle by video processing. It is a flowchart which shows the flow of a process of the three-dimensional video imaging method in 1st Embodiment. It is the functional block diagram which showed the schematic structure of the three-dimensional image pick-up device in 2nd Embodiment. It is explanatory drawing for demonstrating the stereoscopic video imaging device in 3rd Embodiment.

  Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. The dimensions, materials, and other specific numerical values shown in the embodiments are merely examples for facilitating the understanding of the invention, and do not limit the present invention unless otherwise specified. In the present specification and drawings, elements having substantially the same function and configuration are denoted by the same reference numerals, and redundant description is omitted, and elements not directly related to the present invention are not illustrated. To do.

(First embodiment: stereoscopic image capturing apparatus 100)
FIG. 1 is a functional block diagram illustrating schematic functions of the stereoscopic video imaging apparatus 100 according to the first embodiment, and FIG. 2 is an external view illustrating an example of the stereoscopic video imaging apparatus 100. As shown in FIG. 1, the stereoscopic video imaging apparatus 100 includes an imaging unit 110 (indicated by 110a and 110b in the figure), an operation unit 112, a viewfinder 114, a distance measurement unit 116, and a video processing unit 118. The video merging unit 120, the video compression unit 122, the external output unit 124, and the central control unit 126 are configured. Here, a video camera is cited as the stereoscopic image capturing apparatus 100, but various electronic devices capable of imaging such as a digital still camera, a mobile phone, a PHS (Personal Handyphone System), and a PDA (Personal Digital Assistant) are employed. be able to.

  As shown in FIG. 2, the imaging unit 110 has optical axes 130 a and 130 b that are substantially parallel or intersect in the imaging direction, and when the imager holds the main body 132 of the stereoscopic video imaging device 100 horizontally, the optical axes The two imaging units 110a and 110b are arranged so that 130a and 130b exist on the same horizontal plane.

  The imaging unit 110 photoelectrically converts light incident through a focus lens 150 used for focus adjustment, a diaphragm (iris) 152 used for exposure adjustment, and an imaging lens 154 (indicated by 154a and 154b in the figure) to obtain video data. And an image pickup device 156 (indicated by 156a and 156b in the figure) for A / D conversion, and a focus lens 150, a diaphragm 152, an image pickup lens 154, and a drive circuit 158 for driving the image pickup device 156. Video data is generated in the imaging units 110a and 110b. The video data is generated by the imaging unit 110a for left eye video data to be perceived by the observer's left eye, and the imaging unit 110b generates right eye video data for the observer's right eye to perceive. It is possible to form either a moving image or a still image.

  The operation unit 112 includes operation keys including a release switch, a cross key, a joystick, and a switch such as a touch panel arranged on a display surface of a viewfinder 114 described later, and accepts an operation input from the photographer.

  The viewfinder 114 includes a liquid crystal display, an organic EL (Electro Luminescence) display, and the like, and displays image data output from the image processing unit 118 and information indicating an imaging state in conjunction with the operation unit 112 as an OSD (On-Screen Display). Display as. The photographer can grasp the subject at a desired position and occupied area by operating the operation unit 112.

  The distance measuring unit 116 measures the distance (hereinafter referred to as a main subject distance) between the stereoscopic video imaging apparatus 100 itself, in particular, the imaging unit 110, and a main subject that is a subject of imaging among subjects. The main subject distance is used when the imaging control unit 170 described later adjusts the convergence angle.

  Here, the main subject is a subject that the user mainly desires to image, and is specified by focusing, for example, in accordance with an operation input to the operation unit 112 by the photographer.

  FIG. 3 is an explanatory diagram for explaining the measurement of the main subject distance. As shown in FIG. 3A, the distance measuring unit 116 irradiates the main subject 160 with infrared rays and measures the time of flight (TOF: Time Of Flight) to reflect the infrared rays. The distance from the subject 160 is measured. In addition, as shown in FIG. 3B, even when the main subject 160 is not located in front of the stereoscopic image capturing apparatus 100, the reflection angle from the main subject 160 is reflected because the infrared irradiation angle is set wide in advance. Light is received and the main subject distance can be measured as in FIG.

  When a plurality of subjects are recognized, for example, the distance measuring unit 116 measures the main subject distance using the closest subject as the main subject 160 for imaging, or all subjects within a predetermined distance range are the main subjects. An average value of distances from the main subjects 160 is set as a main subject distance.

  Further, since the position of the focus lens 150 and the main subject distance correspond to each other, the distance measuring unit 116 acquires the focus information that is the position information of the focus lens 150 through the drive circuit 158, and based on the focus information. Thus, the main subject distance may be obtained.

  The video processing unit 118 performs R (Red) G (Green) B (Blue) processing (conversion from video data to RGB signals, γ correction, color correction, etc.) on the video data generated by the imaging unit 110, Video signal processing such as enhancement processing, noise reduction processing, and white balance adjustment processing is performed. The Y color difference signal demodulated here is used in motion vector detection described later. In addition, the white balance adjustment performed by the video processing unit 118 and the iris adjustment performed by the imaging control unit 170 use the same adjustment value so that the hue, brightness, and the like do not differ between the right-eye video data and the left-eye video data. .

  The video merging unit 120 converts the video data (right-eye video data and left-eye video data) corrected by the video correction unit 170, which will be described later, into a side-by-side format, a top-and-bottom format, and a frame sequential format, 3D video data is generated by merging with a predetermined recording method for 3D video.

  The video compression unit 122 converts the stereoscopic video data into M-JPEG (Motion-JPEG), MPEG (Moving Picture Experts Group) -2, H.264, or the like. Code data encoded by a predetermined encoding method such as H.264 is stored in an arbitrary storage medium 162. As an arbitrary storage medium 162, an optical disk medium such as a DVD (Digital Versatile Disk) or a BD (Blu-ray Disc), a medium such as a RAM, an EEPROM, a nonvolatile RAM, a flash memory, or an HDD (Hard Disk Drive) is applied. be able to. Note that the HDD is precisely a device, but for the sake of convenience, this HDD is treated as synonymous with other storage media.

  The external output unit 124 converts the stereoscopic video data into stereoscopic display data that is data of a predetermined display scheme in the stereoscopic video, such as a line sequential scheme or a frame sequential scheme, and displays the stereoscopic video connected to the stereoscopic video imaging apparatus 100. Output to the device 164. The stereoscopic image display device 164 is configured by a liquid crystal display, an organic EL display, or the like, similar to the viewfinder 114, and is formed to have different polarization characteristics for each line, for example, and displays stereoscopic display data. The viewer can view the stereoscopic video by viewing the stereoscopic display data displayed on the stereoscopic video display device 164 through glasses having different polarization characteristics on the left and right.

  In addition, the external output unit 124 uses code data that has been subjected to signal compression suitable for communication, addition of an error correction code, and the like, for example, dedicated to portable devices such as the Internet, a LAN (Local Area Network), a mobile phone, and a PHS. It may be output to the stereoscopic video display device 164 via a communication network such as a line.

  The central control unit 126 manages and controls the entire three-dimensional image pickup device 100 by a semiconductor integrated circuit including a central processing unit (CPU), a ROM storing programs, a RAM as a work area, and the like. In the present embodiment, the central control unit 126 also functions as an imaging control unit 170, a distance acquisition unit 172, a video correction unit 174, a luminance difference derivation unit 176, and a video cutout unit 178.

  The imaging control unit 170 controls imaging such as focus adjustment and exposure adjustment for the main subject 160. Specifically, the imaging control unit 170 transmits a control command for controlling imaging to the drive circuit 158 of the imaging unit 120, and the drive circuit 158 controls the focus lens 150 and the aperture according to the control command from the imaging control unit 170. 152 is adjusted.

  At this time, for example, the imaging control unit 170 sets a predetermined area at the center of each of the right-eye video data and the left-eye video data as a focus detection area used for focus adjustment, and the high-frequency component and contrast of the video data in this area. The focus lens 150 may be adjusted so that is maximized.

  Furthermore, the imaging control unit 170 converges so that the convergence point overlaps the position of the main subject distance on the vertical bisector of the connection of the imaging units 110a and 110b (the line connecting the object side principal points of the imaging units 110a and 110b). Adjust the corner (toe-in setting). The imaging control unit 170 derives the convergence angle from the main subject distance and the distance between the imaging elements 156a and 156b, and through the driving circuit 158, the imaging lenses 154a and 154b and the imaging elements 156a and 156b are derived so as to be the convergence angle. Drive. At this time, the imaging lens 154a and the imaging element 156a, and the imaging lens 154b and the imaging element 156b are interlocked according to the control of the imaging control unit 170. The imaging lenses 154a and 154b rotate at the same angle in the left-right symmetry (the optical axes 130a and 130b intersect with each other in the left-right symmetry) to adjust the convergence angle.

  Further, the adjustment of the convergence angle may be performed based on values (convergence angle, main subject distance, etc.) individually input by the photographer, for example.

  The distance acquisition unit 172 captures the distance between the main subject 160 that is the subject of imaging and the imaging unit 110 among the subjects included in the video data (main subject distance), and the quasi-subject that is a subject other than the main subject 160. A relative distance that is a difference from the distance to the unit 110 is acquired. The video correction unit 174 described later corrects the sharpness based on this relative distance.

  Since the relative distance is the distance in the imaging direction between the main subject 160 and the quasi-subject that is a subject other than the main subject 160, the parallax of the main subject 160 detected in the video data respectively captured by the imaging units 110a and 110b and the quasi-subject. It can also be expressed as a difference from the parallax of the subject. The distance acquisition unit 172 detects the parallax of the main subject 160 and the parallax of the quasi-subject between the video data captured by the imaging units 110a and 110b, and compares the difference between the detected parallax of the main subject 160 and the parallax of the quasi-subject. Distance. In the present embodiment, the parallax of the main subject 160 is the parallax of the main subject 160 detected between the video data captured by the imaging unit 110, and the parallax of the quasi-subject is between the video data captured by the imaging unit 110. The parallax of the quasi-subject detected by the above, and hereinafter simply referred to as the parallax of the main subject 160 and the parallax of the quasi-subject respectively.

  At this time, in order to facilitate the calculation, the main subject 160 itself is also represented by a relative distance 0 (zero). However, in the present embodiment, since the convergence angle is adjusted so that the main subject 160 becomes a convergence point, that is, the parallax of the main subject 160 becomes substantially zero, the distance acquisition unit 172 has a relative distance. As such, the parallax of the quasi-subject detected between the video data generated by the imaging units 110a and 110b may be obtained (derived).

  FIG. 4 is an explanatory diagram for explaining the relative distance and the parallax. The imaging control unit 170 adjusts the convergence angle according to the position of the main subject 160. Then, quasi-subjects that are subjects other than the main subject 160 also enter the angle of view ∠ABC and ∠DEF of the imaging lenses 154a and 154b shown in FIG. At this time, the position of the main subject 160 in each of the right-eye video data and the left-eye video data is the center of the screen as shown in FIG.

  In such a state that the convergence angle is adjusted according to the position of the main subject 160, the quasi-subject 184a whose distance from the imaging unit 110 is the same as the main subject 160 is the relative distance of 0, It is represented by 0 parallax in the video data for the eye and the video data for the left eye. On the other hand, the quasi-subject 184b that is relatively closer to the main subject 160 is at the right end in the left-eye video data, whereas in the right-eye video data, it is at the center as indicated by the arrow 186a. Further, the quasi-subject 184c that is relatively far from the main subject 160 is at the left end in the left-eye video data, whereas in the right-eye video data, it is at the center as indicated by the arrow 186b. The quasi-subject 184d is relatively far from the main subject 160, and is located on the right side in the left-eye video data, but is not displayed within the range in the right-eye video data as indicated by the arrow 186c.

  Thus, the relative distance can also be expressed by the parallax of a subject such as a main subject or a quasi-subject. For the derivation of the parallax, motion vector detection that identifies the same subject between the frame data is used. Here, the frame data is still image data arranged in a time series constituting one moving image. In the present embodiment, two frame data having a time difference, which are motion vector detection targets, are replaced with two pieces of left and right video data captured simultaneously. The distance acquisition unit 172 divides the video data into predetermined blocks, and derives parallax between blocks indicating the same subject for each of the divided blocks.

  FIG. 5 is an explanatory diagram for explaining the motion vector detection of the distance acquisition unit 172. In MPEG, which is a moving image compression technique, motion vector detection that detects a motion vector based on block matching is used. Here, the motion vector indicates the displacement of the same subject between two frame data as a vector. In motion vector detection in MPEG, as shown in FIG. 5A, past video data output from a frame memory is divided into predetermined blocks (areas) 190, and a block 190a selected from past frame data is selected. The most similar block 190b having the same size is extracted from the current frame data, and the motion vector 192a is detected from the positional relationship between them.

  In the present embodiment, since the distance acquisition unit 172 uses an algorithm that is almost equivalent to motion vector detection in MPEG encoding, existing motion vector detection technology can be used, and right-eye video data and left-eye video data The motion vector between is detected.

  Specifically, as shown in FIG. 5B, the distance acquisition unit 172 corresponds to the luminance (Y color difference signal) of each pixel of the comparison source block 190c selected from the left-eye video data and the pixel. The difference between the luminance of each pixel of the block 190d at an arbitrary position of the comparison target of the right-eye video data is calculated, and the sum of the luminance differences of all the pixels in the block is derived. Subsequently, the process of moving the comparison target block 190d by a predetermined distance and deriving the sum of the brightness differences is repeated, and the block 190d at the position where the sum of the brightness differences is smallest is extracted as the block 190d having the nearest brightness. The displacement is defined as a motion vector 192c of the block 190c.

  Further, as shown in FIG. 5C, the distance acquisition unit 172 displays the luminance of each pixel of the comparison source block 190e selected from the right-eye video data and the left-eye video data corresponding to the pixel. The difference with the luminance of each pixel of the block 190f at an arbitrary position of the comparison destination is taken, and the sum of the luminance differences of all the pixels is derived. Subsequently, the process of deriving the sum of the luminance differences by moving the comparison target block 190f by a predetermined distance is repeated, and the block 190f at the position where the sum of the luminance differences is the smallest is extracted as the block 190f having the closest luminance. The displacement is set as a motion vector 192e of the block 190e. Thus, motion vectors are detected in both the left-eye video data and right-eye data blocks.

  In such a motion vector detection, for example, the comparison source block is selected by moving down from the upper left of the video data to the right and moving down from the left end to the right when the right end is reached. Covered.

  Further, in the motion vector detection of the present embodiment, only the horizontal direction can be the target of the comparison target block without extracting the comparison target block from the entire range. That is, as the comparison target block 190, only the region having the same vertical coordinate as the comparison source blocks 190c and 190e (the region surrounded by the broken lines 194a and 194b in FIGS. 5B and 5C). Is the target.

  As described above, a motion vector detection technique has been established in video compression, and in this embodiment, parallax is obtained using the motion vector detection. Further, unlike the detection of motion vectors used for moving image compression, the parallax appears only as a horizontal displacement in the video, so the distance acquisition unit 172 only needs to detect a motion vector in the horizontal direction. With the configuration in which the detection of the motion vector is limited to the horizontal direction, the processing time and processing load can be significantly reduced, and the circuit can be downsized.

  As illustrated in FIG. 5, when the distance acquisition unit 172 extracts similar blocks in the right-eye video data for the left-eye video data blocks, the distance acquisition unit 172 obtains a motion vector between the blocks as the left-eye video data. The parallax of that block. Similarly, when the distance acquisition unit 172 extracts similar blocks in the left-eye video data for the right-eye video data block, the distance acquisition unit 172 obtains a motion vector between the blocks of the right-eye video data. Let it be parallax. In addition, the motion vector of the right-eye video data calculated separately as described above and the motion vector of the corresponding left-eye video data have substantially the same size, and the direction of the motion vector is opposite. It is thought that there is. Therefore, when a motion vector that does not satisfy this condition is detected, the motion vector detection accuracy may be improved by changing the parameter in motion vector detection and detecting again.

  Then, after assigning the derived parallax to each pixel in the block, the distance acquisition unit 172 corrects the assigned parallax so as to suppress a change in the parallax assigned between adjacent pixels.

  FIG. 6 is an explanatory diagram for explaining correction of parallax. The distance acquisition unit 172 derives the parallax for each block 190 as illustrated in FIG. 6, and then passes a low-pass filter for each pixel in order to suppress a sudden change in the parallax assigned to the pixel at the boundary of the block 190. As shown by a curve 198 in FIG. 6, with respect to the arrangement of pixels in the horizontal direction of the video data, a drop assigned to adjacent pixels is reduced, and the amount of change in parallax is smoothed. The smoothing so as to suppress the change in the parallax allocated between the pixels can avoid a situation in which the parallax changes suddenly at the boundary of the block, and the continuity of the video is lost to cause a sense of incongruity. it can.

  Then, the video correction unit 174 sets the relative distance that is the difference between the distance between the main subject 160 that is the subject of imaging and the imaging unit 110 and the distance between the quasi-subject that is a subject other than the main subject 160 and the imaging unit 110. Based on this, the sharpness of one or both of the main subject and the quasi-subject included in the video data is changed in units of pixels. At this time, when the relative distance is approximately 0, the video correction unit 174 may maximize the definition of the quasi-subject and decrease the definition of the quasi-subject as the relative distance increases.

  FIG. 7 is an explanatory diagram for explaining a correspondence example between the parallax used as the relative distance and the sharpness. In FIG. 7, the horizontal axis indicates parallax (relative distance), and the vertical axis indicates sharpness. In the present embodiment, the parallax takes a positive value as the quasi-subject shifts to the right and a negative value as it shifts to the left in each of the right-eye video data and the left-eye video data. For example, as in the normal curve 200, the video correction unit 174 maximizes the definition of the quasi-subject when the parallax is 0, and continuously decreases the definition of the quasi-subject as the absolute value of the parallax increases. . In addition, when the absolute value of the parallax is equal to or greater than a predetermined value, a process of blurring with the definition of the quasi-subject fixed to a predetermined minimum value may be performed. In the present embodiment, a definition of 0 indicates that no process for changing the definition of video data is performed. An example in which such sharpness correction is specifically applied to video data will be described with reference to FIG.

  FIG. 8 is an explanatory diagram for explaining the correction of the sharpness. FIG. 8A shows an example of video data before the sharpness correction, and FIG. 8B shows an example of video data after the sharpness correction. The image correcting unit 174 has a large parallax (relative distance) so that the main subject 160 without parallax, which is a subject for imaging, and the quasi-subject 184a in the vicinity of the subject 184a which has no parallax are also sharp. The large quasi-subjects 184c and 184b are corrected so that the sharpness is weak, and the quasi-subject 184d outside the angle of view in the right-eye video data is corrected so that the sharpness is further weakened.

  In the present embodiment, the sharpness is an index indicating the degree of sharpness of video data such as sharpness, for example. In the existing imaging apparatus, originally, the generated video data is subjected to a process for increasing the sharpness through an edge enhancement process that emphasizes a high frequency component having a predetermined frequency or higher. Therefore, the video correcting unit 174 uses the video data subjected to the edge enhancement processing as it is for the main subject 160 without parallax and the quasi-subject 184a in the vicinity thereof without parallax. For the quasi-subject 184d that is out of the angle of view in one of the video data, such as 184b and 184c, the sharpness is reduced by, for example, about 0 to −20 dB.

  As a result, in the video displayed on the stereoscopic video display device 164, the main subject 160 and the quasi-subject 184a are emphasized, and the other quasi-subjects 184b, 184c, and 184d are not clear and not noticeable. In particular, the quasi-subject 184d out of the angle of view in the video data for the right eye blurs, for example, about 200 TV lines out of 1080 TV lines to some extent. Here, the TV book is a unit indicating horizontal display resolution, and is an example of a unit of sharpness expressed by the number of points that can be identified when monochrome dots are displayed for each pixel.

  As described above, the stereoscopic image capturing apparatus 100 according to the present embodiment relatively strengthens the sharpness of the main subject 160 that is the subject of imaging and the quasi-subject 184a having a relative distance of 0, while the distance between the main subject 160 and the main subject 160 is increased. The sharpness of certain quasi-subjects 184b, 184c, 184d is relatively weakened. For this reason, even when the depth of field of the stereoscopic video imaging apparatus 100 is deep, it is possible to generate video data having a depth of field similar to that when a human directly visually recognizes the main subject 160.

  Further, the parallax of the main subject 160 and the quasi-subject 184a is substantially zero, and other objects are displayed in a blurred manner so that the main subject 160 and the relative subject can be displayed even when there are no glasses necessary for viewing a stereoscopic image. The quasi-subject 184a at a distance of 0 can be viewed as a natural flat image. Therefore, stereoscopic video data can be used as planar video data as it is, and the spread of stereoscopic video can be promoted.

  Furthermore, since the amount of information of the background and the quasi-subject with reduced sharpness is reduced, a long-time video can be stored on the same storage medium 162, and more channels can be transmitted in the same band.

  The video correction unit 174 further multiplies the corrected sharpness by a coefficient corresponding to the depth of field. With such a configuration, when the depth of field is already close to the human eye, the sharpness is corrected too much and the quasi-subject distant from the main subject 160 in the imaging direction is more blurred than when viewed directly. It is possible to avoid a situation in which a stranger feels uncomfortable.

  The luminance difference deriving unit 176 divides the video data into predetermined blocks, and derives the difference between the maximum value and the minimum value of the luminance in all the pixels in the divided block. Then, when the luminance difference is equal to or smaller than the predetermined threshold, the video correcting unit 174 does not perform sharpness correction on the pixels of the block whose difference is equal to or smaller than the predetermined threshold.

  A block whose difference between the maximum and minimum luminance values of the pixels contained in the block itself is less than a predetermined threshold can be regarded as a part of the background such as a wall or sky, and the image needs to be strong. Low. In addition, such a block with a small difference in luminance cannot accurately derive parallax even if motion vector detection is used. Therefore, the processing load of such a block is reduced by using a circuit that does not correct the sharpness, for example, a coring circuit. With this configuration, it is possible to avoid unnecessary correction or erroneous correction for blocks that do not require sharpness correction, and to reduce the processing load.

  In the above-described embodiment, the configuration for controlling the convergence angle is described. However, even when imaging is performed with the convergence angle fixed, the definition can be corrected. Hereinafter, a case where the convergence angle is fixed and the optical axes 130a and 130b intersect at a predetermined point will be described.

  9 and 10 are explanatory diagrams for explaining the relationship between the main subject 160 and the parallax in the video data when the angle of convergence is fixed. Here, for easy understanding, it is assumed that the main subject 160 is at an arbitrary position on the perpendicular bisector of the connection of the imaging units 110a and 110b. When the convergence angle is fixed, as shown in FIG. 9A, when the main subject 160 is closer to the imaging units 110a and 110b than the convergence point M, as shown in FIG. In the eye image data, the main subject 160 is displaced to the right side from the vertical line 10 at the center in the horizontal direction, and in the right eye image data, it is displaced to the left side from the center line 210. Conversely, as shown in FIG. 10A, when the main subject 160 is farther from the convergence point M to the imaging units 110a and 110b, as shown in FIG. In this case, the center line 210 is displaced to the left side, and the right-eye video data is displaced to the right side from the center line 210. Therefore, when the convergence angle is fixed, the main subject 160 has a parallax as shown in FIGS. 9B and 10B according to the main subject distance from the imaging units 110a and 110b. . Such parallax can be derived from the convergence angle and the main subject distance as follows.

As shown in FIG. 9, when the main subject 160 is closer to the imaging units 110a and 110b than the convergence point M, the bisector of the angle of view ∠ ABC of the imaging unit 110a is a straight line BM passing through the convergence point M, MBC is half the angle of view angle ABC. Here, since the convergence angle is fixed, the convergence point M is also fixed. ∠MBD is derived by the following formula (1).
∠MBD = arctan (line segment MD / line segment BD) (Formula 1)
Here, the line segment BD is half the distance between the imaging units 110a and 110b, and the line segment MD can be specified from the convergence angle and the line segment BD by BD / tan (convergence angle / 2). Further, ∠EBD is derived by the following formula (2).
∠ EBD = arctan (line segment ED / line segment BD) (Formula 2)
Here, the line segment ED is the acquired main subject distance. By subtracting ∠EBD from this ∠MBD, ∠MBE is derived.
∠MBE = ∠MBD−∠EBD (Formula 3)

Here, the vertical line 210 at the center in the horizontal direction of the video data for the left eye is assumed to have a horizontal coordinate of 0, the right side pixel and the horizontal coordinate increase, the left side pixel and the horizontal coordinate decrease, and the right end horizontal coordinate is g, and the horizontal coordinate at the left end is -g. The horizontal coordinate a of the main subject 160 of the left-eye video data is derived by the following formula (4) using ∠MBE.
a = g × (∠MBE / ∠MBC) (Formula 4)

  Here, since the main subject 160 is on the vertical bisector of the connection of the imaging units 110a and 110b, the horizontal coordinate of the main subject 160 of the right-eye video data is that of the main subject 160 of the left-eye video data. The value is the same as the horizontal coordinate a, and the value is -a. Therefore, the parallax of the main subject 160 can be expressed as a − (− a) = 2a.

Similarly, as shown in FIG. 10A, when the main subject 160 is farther from the imaging units 110a and 110b than the convergence point M, the bisector of the angle of view ∠ ABC of the imaging unit 110a is a straight line BM. ABM is half the angle of view. Further, ∠EBD is derived by the same equation as the above-described equation (2). Then, ∠MBE is derived by subtracting ∠MBD from ∠EBD.
∠MBE = ∠EBD−∠MBD (Formula 5)

When the horizontal coordinate is defined as in the case where the main subject 160 is closer to the convergence point M described with reference to FIG. 9, the horizontal coordinate a of the main subject 160 of the left-eye video data is expressed by the following formula ( 6).
a = −g × (∠MBE / ∠ABM) (Formula 6)

  Also here, the horizontal coordinate of the main subject 160 of the right-eye video data is the same as the horizontal coordinate a of the main subject 160 of the left-eye video data, and has the opposite value -a. Can be expressed as a − (− a) = 2a.

  Then, the distance acquisition unit 172 subtracts the parallax 2a of the main subject 160 derived from the parallax of each of the quasi-subjects and cancels the parallax 2a of the main subject 160, and The parallax difference is derived. The video correction unit 174 corrects the sharpness of the main subject 160 and the quasi-subject based on the parallax obtained by subtracting the parallax 2a of the main subject 160.

  As described above, the stereoscopic image capturing apparatus 100 has high sharpness of the main subject 160 and the quasi-subject having a relative distance of approximately 0, and is far away from the main subject 160 in the front and rear directions in the imaging direction, even when the convergence angle is fixed. As the absolute value of the relative distance increases, the sharpness can be reduced.

  Further, the distance acquisition unit 172 controls the convergence angle even when the main subject 160 is imaged at an arbitrary position before and after the display surface, which is different from the display surface. And the main subject distance are obtained, and the parallax of the main subject 160 is derived as needed, so that the sharpness correction processing of the present embodiment can be realized.

  Further, the distance acquisition unit 172 does not use the above-described convergence angle and main subject distance, but simply identifies the subject designated by the photographer through the operation unit 112 as the main subject 160 from the video data displayed on the viewfinder 114. The definition correction processing of the present embodiment can also be realized by deriving the difference between the parallax of the main subject 160 and the parallax of the quasi-subject that is a subject other than the main subject 160.

  The main subject 160 and the quasi-subject generate parallax according to the distance from the imaging unit 110. The distance from each of the imaging units 110a and 110b cannot be determined from the video data unless the angle of convergence is known. However, what is needed here is the relative distance between the main subject 160 and the quasi-subject, and the relative distance can be easily obtained from the difference between the parallax of the main subject 160 and the quasi-subject, so The sharpness can be corrected by deriving a relative distance, which is a difference from the parallax of another quasi-subject, based on the parallax of the designated main subject 160. Therefore, even when a measurement mechanism using an autofocus, an infrared sensor, or the like is not provided, the object of the present embodiment can be achieved.

  Furthermore, the adjustment of the convergence angle is not limited to the configuration in which the imaging control unit 170 actually drives the imaging lenses 154a and 154b and the imaging elements 156a and 156b, but the optical axes 130a and 130b of the imaging units 110a and 110b are fixed, It can also be done by video processing.

  The video cutout unit 178 may change the cutout range of the video data acquired from the image sensor 156 to adjust the convergence angle in a pseudo manner. However, the image sensor 156 has a larger number of pixels in the horizontal direction than video data to be finally output.

  11 and 12 are explanatory diagrams for explaining the adjustment of the convergence angle by the video processing. Here, the imaging units 110a and 110b have the imaging capabilities of the adjustment angle of view ∠ABC and ∠DEF, respectively, and arbitrary cut-out angle of view ∠A'BC 'and ∠D'EF from the adjustment angle of view ∠ABC and ∠DEF. Process to cut out '.

  For example, when trying to set the convergence point to infinity, the video cutout unit 178 performs video data from the video data 220a obtained by imaging the adjustment angle of view ∠ABC with respect to the video data for the left eye shown in FIG. Cut out 222a. Then, the cut-out video data 222a corresponds to the cut-out angle of view A′BC ′ of FIG. 11A, and the vertical line 224a at the center in the horizontal direction of the video data 222a is the imaging direction of FIG. 226a. Thus, the cut-out video data 220a is equal to the video data captured by the imaging unit having the angle of view A′BC ′ whose optical axis is perpendicular to the stereoscopic video imaging apparatus 100.

  Also in the video data for the right eye, since the video data 222b is cut out from the video data 220b obtained by imaging the adjustment angle of view ∠DEF, the cut-out video data 222b is cut out of the angle of view ∠D'EF 'shown in FIG. The vertical line 224b at the center in the horizontal direction of the video data 222b is the imaging direction 226b in FIG. 11 (a). The cut out video data 222b is equal to the video data captured by the imaging unit having the angle of view D′ EF ′ whose optical axis is perpendicular to the stereoscopic video imaging apparatus 100. In this way, it is possible to obtain video data 220a and 220b in which both optical axes are parallel toward infinity.

  Further, when the convergence point is brought close to the imaging units 110a and 110b, the video cutout unit 178 relates to the video data 222a from the video data 220a obtained by imaging the adjustment angle of view ∠ABC with respect to the video data for the left eye shown in FIG. Cut out. Then, the cut-out video data 222a corresponds to the cut-out angle of view A′BC ′ of FIG. 12A, and the vertical line 224a at the center in the horizontal direction of the video data 220a is the imaging direction of FIG. 226a. Also for the right-eye video data, by performing the same processing as shown in FIG. 12C, the vertical line 224b at the horizontal center of the video data 220b becomes the imaging direction 226b in FIG. 12A. Thus, the pseudo optical axes (imaging directions 226a and 226b) indicated by the center lines 224a and 224b between the cut-out video data 220a and the video data 220b intersect at the convergence point M as shown in FIG. Video data 220a and 220b of Kakuno BME can be obtained.

  In this way, the video cutout unit 178 changes the cutout range of the video data and artificially adjusts the convergence angle, so that the stereoscopic video imaging apparatus 100 actually drives the imaging units 110a and 110b to converge. There is no need to mount a mechanism for adjusting the corners, and the number of parts can be reduced and manufacturing can be performed at low cost.

  In the above-described stereoscopic video imaging apparatus 100, the process from the parallax derivation to the definition correction is performed uniformly. However, the present invention is not limited to this, and the stereoscopic video imaging apparatus 100 performs the parallax derivation. The sharpness correction may be performed in the stereoscopic video display device 164.

  In this case, the distance acquisition unit 172 outputs the derived parallax to the external output unit 124, and the external output unit 124 outputs the parallax and the stereoscopic video data to the stereoscopic video display device 164, respectively. At this time, since the external output unit 124 synchronizes the parallax and the stereoscopic video data, for example, an identifier for synchronization is added to each, or one stream is set so that the parallax and the stereoscopic video data correspond to each other. Can be output together.

  When the stereoscopic video display device 164 acquires the stereoscopic video data and the parallax output from the stereoscopic video imaging device 100, the stereoscopic video display device 164 corrects the sharpness of the stereoscopic video data based on the parallax, and converts it to a predetermined display method in the stereoscopic video. To display. In this case, the viewer can adjust the degree of sharpness correction by changing the designation of the main subject 160 as a reference when viewing a stereoscopic video.

  As described above, the stereoscopic video imaging apparatus 100 according to the present embodiment can generate stereoscopic video data that allows a viewer to perceive a natural stereoscopic video.

(Stereoscopic imaging method)
Furthermore, a stereoscopic video imaging method using the above-described stereoscopic video imaging device 100 is also provided. FIG. 13 is a flowchart illustrating a processing flow of the stereoscopic image capturing method according to the first embodiment.

  When the photographer instructs imaging through the operation unit 112 (YES in S260), the imaging units 110a and 110b generate video data (S262), and the distance acquisition unit 172 acquires the main subject distance through the distance measurement unit 116, for example. (S264). Then, the imaging control unit 170 adjusts the convergence angle so that the convergence point overlaps the position of the main subject distance on the vertical bisector of the connection of the imaging units 110a and 110b (S266).

  The distance acquisition unit 172 selects one block from the video data for the left eye among the video data (S268), and whether or not the difference between the maximum value and the minimum value of the pixel luminance in the block is equal to or less than a predetermined threshold value. Is determined (S270). When it is below the predetermined value (YES in S270), the process returns to the block selection step (S268). When larger than the predetermined value (NO in S270), the distance acquisition unit 172 compares with the block with the closest brightness compared to the block with the right eye video data whose vertical coordinate is the same as the block with the left eye video data. The parallax (motion vector) with the original block is derived and associated with the block of video data for the left eye (S272).

  Subsequently, the distance acquisition unit 172 determines whether there is a block that has not yet been extracted in the left-eye video data (S274), and if there is a block that has not yet been extracted (NO in S274). ), Returning to the block selection step (S268), the same processing is repeated.

  When the extraction process is completed for all the blocks of the left-eye video data (YES in S274), the distance acquisition unit 172 selects one block from the right-eye video data (S276), and the luminance of the pixel in the block is determined. It is determined whether the difference between the maximum value and the minimum value is equal to or less than a predetermined threshold (S278). When it is below the predetermined value (YES in S278), the process returns to the block selection step (S276). When larger than the predetermined value (NO in S278), the distance acquisition unit 172 compares with the block with the closest brightness compared to the block with the left eye video data whose vertical coordinate is the same as the block with the right eye video data. The parallax (motion vector) with the original block is derived and associated with the block of the right-eye video data (S280).

  Then, the distance acquisition unit 172 determines whether there is a block that has not yet been extracted from the right-eye video data (S282), and if there is a block that has not yet been extracted (NO in S282). Returning to the block selection step (S276), the same processing is repeated.

  Then, the distance acquisition unit 172 corrects (smooths) the assigned parallax so as to suppress a change in the assigned parallax between adjacent pixels (S284).

  The image correction unit 174 performs correction processing for changing the sharpness of one or both of the main subject 160 and the sub-subject based on the assigned parallax (relative distance) for each pixel (S286). Then, the video merging unit 120 synthesizes the video data with a predetermined recording method for stereoscopic video to generate stereoscopic video data (S288), and the video compression unit 122 converts the stereoscopic video data with a predetermined encoding method. The encoded data is encoded data (S290) and stored in an arbitrary storage medium 162 (S292).

  As described above, according to the stereoscopic video imaging method using the stereoscopic video imaging apparatus 100, it is possible to generate stereoscopic video data that allows a viewer to perceive a natural stereoscopic video.

(Second embodiment: stereoscopic image processing apparatus 300)
In the above-described first embodiment, the stereoscopic image capturing apparatus 100 performs the sharpness correction process according to the parallax during imaging. In the second embodiment, a stereoscopic video processing apparatus 300 that performs a sharpness correction process during reproduction will be described. Note that components that are substantially the same as those of the above-described stereoscopic video imaging apparatus 100 are denoted by the same reference numerals and description thereof is omitted.

  FIG. 14 is a functional block diagram illustrating a schematic configuration of the stereoscopic video processing device 300 according to the second embodiment. The stereoscopic video processing apparatus 300 includes a video acquisition unit 310, an operation unit 112, a video decoding unit 322, a display unit 324, and a central control unit 326. Note that components substantially the same as those in the first embodiment described above are denoted by the same reference numerals and description thereof is omitted. Here, a video acquisition unit 310, a video decoding unit 322, and a central control having different configurations are described. The section 326 will be mainly described.

  The video acquisition unit 310 acquires code data obtained by encoding stereoscopic video data for perceiving stereoscopic video by binocular parallax from the stereoscopic video imaging apparatus. The video decoding unit 322 decodes the code data and converts it into stereoscopic video data.

  The central control unit 326 manages and controls the entire stereoscopic video processing apparatus 300 by a semiconductor integrated circuit including a central processing unit (CPU), a ROM storing programs, a RAM as a work area, and the like. The central control unit 326 also functions as a distance acquisition unit 172, a video correction unit 174, a luminance difference derivation unit 176, a video cutout unit 178, and a display control unit 380.

  When the stereoscopic video processing device 300 of the present embodiment acquires stereoscopic video data from an imaging device capable of generating stereoscopic video data, the distance acquisition unit 172, like the stereoscopic video imaging device 100, causes the main subject included in the stereoscopic video data. A relative distance that is a difference between a distance between the image capturing unit 160 that generates 160-dimensional image data and a quasi-subject that is a subject other than the main subject 160 and the image capturing unit 110 is acquired. Based on the distance, the sharpness of one or both of the main subject 160 and the sub-subject is changed. The display control unit 380 converts the stereoscopic video data into a predetermined display format for stereoscopic video, and displays it on a display unit 324 that is configured by a liquid crystal display, an organic EL display, or the like and has different polarization characteristics for each line. Let

  For example, even when the depth of field of the stereoscopic video imaging device that generated the stereoscopic video data is deep, using the stereoscopic video processing device 300, the field of view is the same as when directly viewing the main subject 160. It can be viewed as a stereoscopic image with a depth equivalent to that of the human eye. Further, when the parallax of the main subject 160 and the quasi-subject is set to approximately 0, other objects are displayed in a blurred manner, so that even when there is no glasses necessary for viewing a stereoscopic image, the quasi-object with a relative distance of 0 is obtained. You can see the subject as a natural flat image.

  Further, according to the stereoscopic video processing apparatus 300 of the present embodiment, it becomes possible to correct the definition of the stereoscopic video data generated in advance afterwards, and the viewer can, for example, view the stereoscopic video. The degree of sharpness correction can be adjusted.

(Third embodiment: stereoscopic image capturing apparatus 400)
In the first embodiment described above, the stereoscopic video imaging apparatus 100 that generates video data that allows viewing of a stereoscopic video perceived from one viewpoint has been described. In the third embodiment, a stereoscopic video imaging apparatus 400 that is perceived from a plurality of viewpoints will be described. Note that components that are substantially the same as those of the above-described stereoscopic video imaging apparatus 100 are denoted by the same reference numerals and description thereof is omitted.

  FIG. 15 is an explanatory diagram for explaining a stereoscopic video imaging apparatus 400 according to the third embodiment. The stereoscopic video imaging apparatus 400 has substantially the same configuration as the stereoscopic video imaging apparatus 100, but further includes an imaging unit 110c in addition to the imaging units 110a and 110b. Here, in order to facilitate understanding, the imaging unit will be described with three examples.

  The three imaging units 110a, 110b, and 110c have their respective optical axes 130a, 130b, and 130c intersecting substantially in parallel or in the imaging direction, and when the photographer grips the main body 132 of the stereoscopic imaging device 100 horizontally, The optical axes 130a, 130b, and 130c are arranged so as to exist on the same horizontal plane.

  For example, when the distance measuring unit 116 disposed between the imaging unit 110a and the imaging unit 110b measures the main subject distance, the imaging control unit 170 adjusts the convergence angle according to the main subject distance. The imaging control unit 170 adjusts the convergence angle of the imaging units 110a and 110b so that the convergence point overlaps the position of the main subject distance on the vertical bisector of the connection of the imaging units 110a and 110b, as in the first embodiment. . For the imaging unit 110c, the imaging control unit 170 adjusts the imaging lens and the imaging element of the imaging unit 110c so that the optical axis 130c passes through the convergence points of the imaging units 110a and 110b.

  The stereoscopic image capturing apparatus 400 generates an image generated by the one located on the left side of the pair of image capturing units 110 (here, the image capturing units 110a and 110b or the image capturing units 110b and 110c) selected by the photographer through the operation unit 112. The sharpness correction processing is performed in the same manner as the above-described stereoscopic video imaging apparatus 100, with the data as left-eye video data and the video data generated by the imaging unit 110 positioned on the right as the right-eye video data.

  When viewing the stereoscopic video of the stereoscopic video data generated using such a stereoscopic video imaging apparatus 400, the photographer can combine the imaging unit 110a and the imaging unit 110b, or the combination of the imaging unit 110b and the imaging unit 110c. The desired viewpoint can be freely selected from these two viewpoints. In particular, when capturing a close-up view, two stereoscopic images that differ greatly depending on the combination of viewpoints can be enjoyed.

  As mentioned above, although preferred embodiment of this invention was described referring an accompanying drawing, it cannot be overemphasized that this invention is not limited to this embodiment. It will be apparent to those skilled in the art that various changes and modifications can be made within the scope of the claims, and these are naturally within the technical scope of the present invention. Is done.

  Note that each step in the stereoscopic image capturing method of the present specification does not necessarily have to be processed in time series in the order described in the flowchart, and may include processing in parallel or a subroutine.

  INDUSTRIAL APPLICABILITY The present invention can be used for a stereoscopic video imaging device, a stereoscopic video processing device, and a stereoscopic video imaging method that generate stereoscopic video data for perceiving stereoscopic video using video data generated by a plurality of imaging units.

100, 400 ... 3D imaging device 110 (110a, 110b, 110c) ... Imaging units 130a, 130b ... Optical axes 172, 372 ... Distance acquisition units 174, 374 ... Video correction unit 176 ... Luminance difference deriving unit 178 ... Video clipping Unit 300 ... Stereoscopic image processing device 310 ... Video acquisition unit

Claims (3)

A plurality of imaging units that are arranged at positions where the respective optical axes are substantially parallel or intersect in the imaging direction, and generate video data in each,
Among the subjects included in the video data, a relative value is a difference between a distance between a main subject that is a subject for imaging and the imaging unit, and a distance between a quasi-subject that is a subject other than the main subject and the imaging unit. A distance acquisition unit for acquiring a distance;
A video correction unit that changes the sharpness of one or both of the main subject and the quasi-subject based on the relative distance;
A luminance difference derivation unit that divides the video data into predetermined blocks and derives a difference between the maximum value and the minimum value of the luminance of the pixels in the divided block;
With
When the difference is equal to or less than a predetermined threshold, the video correction unit does not perform the sharpness correction on pixels of a block in which the difference is equal to or less than the predetermined threshold. A video acquisition unit for acquiring stereoscopic video data for perceiving stereoscopic video due to binocular parallax;
Of the subjects included in the stereoscopic video data, the distance between the main subject that is the subject for imaging and the imaging unit that generated the stereoscopic video data, and the quasi-subject that is a subject other than the main subject and the imaging unit A distance acquisition unit that acquires a relative distance that is a difference from the distance;
A video correction unit that changes the sharpness of one or both of the main subject and the quasi-subject based on the relative distance;
A luminance difference deriving unit that divides the stereoscopic video data into predetermined blocks and derives a difference between the maximum value and the minimum value of the luminance of the pixels in the divided block;
With
When the difference is equal to or smaller than a predetermined threshold, the video correction unit does not perform the sharpness correction on pixels of a block in which the difference is equal to or smaller than the predetermined threshold. In order to generate a stereoscopic video, video data is generated by a plurality of imaging units arranged at positions where respective optical axes are substantially parallel or intersect in the imaging direction,
Among the subjects included in the video data, a relative value is a difference between a distance between a main subject that is a subject for imaging and the imaging unit, and a distance between a quasi-subject that is a subject other than the main subject and the imaging unit. Get the distance,
Dividing the video data into predetermined blocks, deriving the difference between the maximum and minimum luminance values of the pixels in the divided blocks,
Based on the relative distance, the sharpness of one or both of the main subject and the quasi-subject is changed, and when the difference is equal to or smaller than a predetermined threshold, the pixel of the block whose difference is equal to or smaller than the predetermined threshold A method of imaging a stereoscopic image, wherein the sharpness is not corrected.

Images