JPH04360395A - Parallax correction device - Google Patents

Abstract

PURPOSE:To realize the parallax correction device which automatically corrects horizontal and vertical parallax of a stereo picture picked up by a camera and generates a proper stereo picture. CONSTITUTION:Left and right picture signals obtained from a camera are given to a parallax extract section 1, in which the parallax is extracted. A noise component included in the extracted parallax picture is eliminated by a low pass filter 2 and a clip processing section 3 applies clip processing to the picture. The processed parallax picture and either of the left right pictures are used to output a right picture signal and a left picture signal whose parallax are corrected are outputted to a stereo picture decoding section 4.

Description

[Detailed description of the invention]

0001

[Field of Industrial Application] This invention relates to a parallax correction device,
In particular, in image engineering, the present invention relates to a parallax correction device that corrects parallax between a left image and a right image when displaying a stereoscopic image based on the left image and the right image.

0002

[Prior art and problems to be solved by the invention] Figure 8
1 is a diagram showing an example of a conventional time-division stereoscopic image presentation device. Referring to FIG. 8, the left image is the first field, the right image is the second field, and the left and right images are temporally switched for each field (frame) and displayed on the television monitor, in front of the left and right eyes. An observer 11 views the image by wearing glasses each having a liquid crystal shutter 12 arranged thereon. At this time, the left and right liquid crystal shutters 12 are driven in synchronization with the image signal. That is, when an image for the right eye is displayed on the monitor television, the liquid crystal shutter for the left eye is driven to a state in which no light passes through, and the liquid crystal shutter for the right eye is driven to a state in which light passes. Furthermore, when an image for the right eye is displayed on a monitor television, the left and right sides are reversed. The left and right images are taken by two cameras placed horizontally on the left and right, so by merging these images with both eyes, a three-dimensional stereo image with apparent depth can be obtained.

However, in such stereo images, there is a limit to the horizontal and vertical parallax at the time when people are fused, and if this is exceeded, stereo vision and fused vision become impossible. For example, when capturing stereo images, in a method using two cameras, the alignment of the optical axes of the left and right images is delicate, and parallax exceeding a limit often occurs. Images with such inappropriate parallax settings will greatly interfere with viewing stereo images that exceed the parallax fusion range.

[0004] Therefore, the main object of the present invention is to provide a parallax correction device that can automatically correct the horizontal and vertical parallax of a stereo image recorded by a camera and generate an appropriate stereoscopic image. be.

[0005]

[Means for Solving the Problems] The invention according to claim 1 is a parallax correction device, which comprises a parallax extraction means for extracting the parallax between the left and right images by inputting stereo images of a left image and a right image, and It is configured to include a signal processing means that performs filter processing on a parallax image, and a stereo image restoring means that restores a stereo image using the output of the signal processing means and one of the left image and the right image.

The invention according to claim 2 includes, as the signal processing means of the invention according to claim 1, a low-pass filter for removing high-frequency components of the extracted parallax image, and a clip processing means for clipping the output of the low-pass filter. It consists of

[0007] In the invention according to claim 3, a filter having peaking characteristics is used as the low-pass filter according to the invention according to claim 2.

The invention according to claim 4 is characterized in that the signal processing means of the invention according to claim 1 includes a band-pass filter for removing high-frequency components and low-frequency components of the extracted parallax image;
and clipping processing means for clipping the output of the bandpass filter.

According to a fifth aspect of the present invention, the signal processing means of the first aspect of the present invention is configured to apply nonlinear processing to reduce a value near zero parallax in a parallax image to zero.

0010

[Operation] The parallax correction device according to the present invention performs filter processing on the parallax image obtained from the left image and the right image to prevent unfused parallax from occurring, and to make the parallax image correspond to one eye. Obtain stereo images of the left and right images from the images.

[0011]

DESCRIPTION OF THE PREFERRED EMBODIMENTS FIG. 1 is a schematic block diagram of an embodiment of the present invention. Referring to FIG. 1, a right image and a left image are input to a parallax extraction unit 1, and a parallax image between the right image and the left image is extracted and provided to a low pass filter (LPF) 2. The low-pass filter 2 removes high-frequency components included in the parallax image and provides the removed image to the clip processing unit 3. The clip processing section 3 performs clip processing and provides the result to the stereo image restoration section 4. The stereo image restoration unit 4 restores a stereo image based on the clipped parallax image and the left image, and outputs a right image signal and a left image signal.

FIG. 2 is a diagram for explaining the parallax detection method by the parallax extraction section shown in FIG. 1, and FIG. 3 is a diagram showing the input/output characteristics of the clip processing section shown in FIG. 4
is a diagram for explaining a stereo image restoration method by the stereo image restoration unit shown in FIG. 1, and FIG. 5 is a diagram showing input/output characteristics for nonlinear processing for noise removal.
FIG. 6 is a diagram showing the frequency characteristics of a low-pass filter with peaking characteristics.

Next, the specific operation of one embodiment of the present invention will be explained with reference to FIGS. 1 to 6. The right image signal and the left image signal captured by a camera etc. are disparity extracting unit 1
given to. The parallax extraction unit 1 extracts parallax components between left and right images. Here, the parallax component refers to a local shift (shift) between the left and right images in the horizontal direction, and this shift corresponds to the depth in stereo viewing. As a parallax component extraction method, there is, for example, a method of obtaining cross-correlation between left and right images. FIG. 2 briefly explains this method, with (a) showing the left image and (b) showing the right image. A local area of one image (in the case of a 3x3 pixel area in Fig. 2(a)) is translated in parallel, and as shown in Fig. 2(b), the shift (Dx , Dy) are obtained. Here, a specific example of calculating parallax using cross-correlation will be described. Pixels gL (xi, yi) (left eye image), g that constitute this corresponding area of the left and right screens
R(xi, yi) (right eye image), i=1, 2...9
Calculate the correlation for. Then, calculate the correlation when the area K of the image of one eye is translated by (Dx k , Dy k ), and calculate the correlation (X k , Y k ) is the parallax. Expressed as a calculation formula, S(Dx k , Dy k ) Parallax (Dx , Dy )=(Dx k , Dy k )
for minS(Dx k , Dy k ). Here, S(Dx k , Dy k ) is a function indicating the correlation between the left and right images when translated by (Dx k , Dy k ) from the reference position. If this calculation is performed while moving the area over the entire screen, and each parallax (Dx, Dy) is taken as the parallax at the center coordinates of each area, a parallax image covering the entire screen can be obtained.

This image contains a considerable amount of disturbing components due to the noise components of the image signal and the shielding components of the left and right images. Therefore, the high-frequency noise is removed by the low-pass filter 2, and the clip processing section 3 performs clip processing as shown in FIG. This makes it possible to suppress large parallax components that would not be fused when viewing stereo images. At this time, the low-pass filter 2 may be of any one-dimensional (horizontal direction), two-dimensional (horizontal, vertical direction), or three-dimensional (horizontal, vertical, temporal direction) type.

Next, the stereo image restoration unit 4 uses the parallax image signal whose high-frequency components have been removed by the low-pass filter 2 and which has been clipped by the clip processing unit 3 and the left or right input image to create left and right stereo images. restore. To explain in more detail with reference to FIG. 4, each pixel of one image shown in FIG. 4(a) is adjusted to match the horizontal parallax of the position in the parallax image shown in FIG. 4(c). ,
As shown in FIG. 4(b), the pixel is shifted from the original pixel position by Dx in the horizontal direction. Furthermore, even if the vertical parallax Dy is shifted in the vertical direction by ΔDy=D-Dy so that it becomes a constant value D, the direction of the shift is reversed depending on the sign of the parallax. Furthermore, if pixels overlap, they are averaged, and empty pixels are interpolated using pixels on the top, bottom, left and right. At this time, the vertical component of the parallax of each pixel is averaged over the entire image,
By adjusting the vertical positions of the left and right images by parallel movement according to this value, it is possible to minimize the vertical parallax component that interferes with stereo image viewing. In addition, by applying low-pass filter processing to the reconstructed left and right images,
Images that have become discontinuous by performing parallax processing may be smoothly connected.

Furthermore, as shown in FIG. 5, by reducing the noise component near zero parallax to zero, a stereo image with a high S/N ratio on the left side of the visual field can be obtained. Furthermore, by using a low-pass filter 2 having a peaking characteristic as shown in FIG. 6 and changing the magnitude of the peaking, it is possible to emphasize the sense of stereo depth at the depth edge.

Furthermore, a band pass filter is used in place of the low pass filter 2 to remove high frequency components and low frequency components.
It is also possible to apply horizontal parallax only to the portion where the depth changes. In this case, it is not possible to obtain a complete three-dimensional effect, but as shown in FIG. 7, only the parallax change component is extracted and used to construct the left and right images to obtain a pseudo three-dimensional image. Using this method, the amount of information in the parallax image is reduced by converting the depth component shown on the vertical axis in Figure 7(a) into only the parallax change component shown in Figure 7(b), reducing the transmission capacity. It has the advantage of being possible.

By the above-described processing, it is possible to obtain left and right stereo images in which the parallax components are subjected to low-pass filtering and clipping processing.

[0019]

As described above, according to the present invention, after extracting the parallax between left and right images and applying filter processing to the extracted parallax image, a stereo image is restored using either the left or right image. This makes it possible to correct large parallaxes that are harmful when viewing stereo images at each position within the screen, and generate a good stereo image. Furthermore, when transmitting one image and a parallax image, by filtering the parallax images, the bandwidth can be reduced, and therefore the transmission capacity can be reduced.

[Brief explanation of drawings]

FIG. 1 is a schematic block diagram of an embodiment of the invention.

FIG. 2 is a diagram for explaining a parallax detection method by the parallax extraction unit shown in FIG. 1;

3 is a diagram showing input/output characteristics of clip processing by the clip processing section 3 shown in FIG. 1. FIG.

FIG. 4 is a diagram for explaining a method of restoring a stereo image by the stereo image restoring section shown in FIG. 1;

FIG. 5 is a diagram showing input/output characteristics of nonlinear processing for noise removal.

FIG. 6 is a diagram showing frequency characteristics of a low-pass filter with peaking characteristics.

FIG. 7 is a diagram illustrating an example of extracting only a parallax change component.

FIG. 8 is a diagram showing an example of a conventional time-division stereoscopic image presentation device.

[Explanation of symbols]

1 Parallax extraction part 2 Low pass filter 3 Clip processing section 4 Stereo image restoration section

Claims (5)

[Claims] 1. Parallax extraction means for extracting the parallax between the left and right images by inputting stereo images of the left image and the right image, a signal processing means for performing filter processing on the parallax image extracted by the parallax extraction means, and the signal processing means. A parallax correction device comprising: a stereo image restoring means for restoring a stereo image using an output of the means and one of the left image and the right image. 2. The signal processing means includes a low-pass filter that removes high-frequency components of the parallax image extracted by the parallax extraction means, and a clip processing means that clips the output of the low-pass filter. parallax correction device. 3. The parallax correction device according to claim 2, wherein the low-pass filter has a peaking characteristic. 4. The signal processing means includes a bandpass filter for removing high-frequency components and low-frequency components of the parallax image extracted by the parallax extraction means, and a clipping process for clipping the output of the bandpass filter. 2. The parallax correction device of claim 1, comprising means. 5. The parallax correction apparatus according to claim 1, wherein the signal processing means applies nonlinear processing to reduce a value near zero parallax in the parallax image to zero.