JP2002374543A - Stereoscopic image appreciation device and display device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a stereoscopic image appreciation device and a display device, that can allow a user to simply enjoy a stereoscopic image by using a personal computer or a digital camera in a home. SOLUTION: An LCD monitor 24 of the stereoscopic image appreciation device displays two images photographed with a parallax side by side and a prism 25 restricts the visual field in a way that the two images are separately viewed with left and right eyes. An ASIC(Application Specific IC) 13 compares left right position relations of the two images and the ASIC 13 controls the display positions of the two images on the LCD monitor 24 according to the result of comparison.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a three-dimensional image viewing apparatus and a display apparatus for easily enjoying three-dimensional pictures, and more particularly, to analyzing a digital image taken by a digital camera or the like, and making it easy for anyone. The present invention relates to a stereoscopic image viewing device and a display device capable of taking and viewing a stereoscopic photograph.

[0002]

2. Description of the Related Art Conventionally, various methods for easily enjoying a stereoscopic photograph have been known. For example, the applicant for
Japanese Patent Application Laid-Open No. 7-325354 and the like have already disclosed a technique on a method for easily taking an effective three-dimensional photograph.

[0003]

However, the above-mentioned Japanese Patent Application Laid-Open No. Hei 7-325354 does not describe a simple and specific observation method.

[0004] On the other hand, studies on stereoscopic televisions and the like have been known for a long time, and a number of patent applications have been filed for stereoscopic viewing methods using lenticulars, parallax barriers, and the like.

[0005] However, since these images are reproduced using a dedicated device and photographed using a dedicated device, the photographing and observation tend to be large and highly specialized. Therefore, in many cases, a device completely different from a device possessed by a general camera user is required.

The present invention has been made in view of the above points, and provides a stereoscopic image viewing apparatus and a display apparatus which allow a user to easily enjoy a stereoscopic image using a personal computer or a digital camera in an ordinary household. The purpose is to:

[0007]

That is, the present invention provides a display means for displaying two images photographed with parallax side by side, and a visual field for separately observing the two images with right and left eyes. A control member for controlling a display position of the two images to be displayed on the display device according to a comparison result of the comparison device; It is characterized by having.

The present invention also provides a comparing means for comparing the vertical image signals of two captured images with parallax, and detecting a plurality of feature points of the vertical image signals of the two images. Detecting means, control means for determining the enlargement ratio and shift amount of each of the two images in a form in which the coordinates of the plurality of feature points in the vertical direction on the screen are aligned, and the enlargement determined by the control means Display means for displaying the two images based on the rate and the shift amount.

In the stereoscopic image viewing apparatus of the present invention,
Two images photographed with parallax are displayed side by side on the display means, and the field of view is regulated by the regulating member so that the two images can be observed separately by the right and left eyes. 2 above
The left and right positional relations of the two images are compared by the comparing unit, and the display position of the two images displayed on the display unit is controlled by the control unit according to the comparison result of the comparing unit.

Further, in the display device according to the present invention, the vertical image signals of the two images photographed with parallax are compared by comparing means, and the vertical image signals of the two images are compared. A plurality of feature points are detected by the detection means. Then, in a mode in which the coordinates of the plurality of feature points in the vertical direction on the screen are aligned, the enlargement ratio and shift amount of each of the two images are determined by the control unit. The two images are displayed on the display means based on the enlargement ratio and the shift amount determined by the control means.

[0011] In the stereoscopic image viewing apparatus according to the present invention, an image digitized and photographed with parallax is analyzed, and the image is processed so that the image can be immediately viewed stereoscopically. By using this concept, for example, stereoscopic viewing can be easily performed on a personal computer, and a stereoscopic image can be easily reproduced on a liquid crystal monitor of a digital camera.

[0012]

Embodiments of the present invention will be described below with reference to the drawings.

FIG. 2 is a diagram showing how a stereoscopic image is obtained using a general camera.

In FIG. 2, stereoscopic vision is obtained by observing images taken by the camera 2 at the left and right (L, R) positions with the left and right eyes, respectively. It becomes possible.

At this time, care must be taken to keep the subject 1 from moving. However, it is difficult for the photographer to keep the distance to the subject constant during L and R shooting. It may change. Also, which of the photographing positions L and R comes first depends on the situation at that time and the habit of the photographer. Therefore, if these problems are solved, a simple stereoscopic picture cannot be viewed.

FIG. 1 shows a configuration of a stereoscopic image viewing apparatus according to the present invention. FIG. 1A is a block diagram showing an electric system configuration of a digital camera applied to the stereoscopic image viewing apparatus.
FIG. 3B is a diagram showing a display example on the LCD monitor of FIG.

An image pickup section composed of the photographing lens 11 and the image pickup element 12 is a part for replacing a subject image with a magnitude of an electric signal. The microcomputer (ASIC) 13 also serving as an image processing IC has, for example, a portion for processing the image signal obtained in this way, a memory control function, and a function for compressing and expanding the image signal into a JPEG format.
The ASIC 13 also includes a lens driver 14,
An LCD driver 16, a memory 17, a separate flash memory 18 and a connector 19, a release switch 21 for photographing, and a switch 22 for stereoscopic photography are connected.

In the ASIC 13, the switch 2
The sequence control of the camera is performed according to the programs 1 and 22 and the program in the flash memory 18.

The photographing lens 11 constituting the image pickup unit is
Driving by the ASIC 13 through the lens driver 14 performs zooming and focusing. Then, the image signal obtained by the imaging unit is compressed and recorded in the memory 17.

The ASIC 13 is an LCD driver 1
And a function of displaying an image on the LCD monitor 24 via the LCD monitor 6. Note that a prism 25 is disposed near the LCD monitor 24.

When transferring an image signal to a personal computer (hereinafter abbreviated as a personal computer) 26, a memory 1
7 may be removed and connected to the adapter of the personal computer 26, or a method of communicating with the personal computer 26 via the connector 19 and the cable 19a. This includes:
It may be a USB format or a notebook computer card format.

The release switch 21 is a normal photographing switch. However, when taking a three-dimensional photograph, a dedicated mode may be set by operating the three-dimensional photograph switch 22. At this time, a stereo mode is displayed on the LCD monitor 24 as shown in FIG. 1B, conditions for inhibiting zooming and fixing the exposure are aligned, and a mismatch between the two images is minimized. What is necessary is just to make a device that does not occur.

Also, the half numbers in the monitor are L, R
When two images of the pair have been captured, the ratio becomes 2/2, which clearly indicates that a stereoscopic image has been obtained. The photographer is 1 /
When the number 2 is displayed, the image is shifted left and right to obtain another image.

The two images thus obtained are displayed side by side on a monitor by display control software as shown in the flowchart of FIG.

If this software is provided in the flash memory 18 of the camera, a three-dimensional image can be enjoyed through the LCD monitor of the camera. Also, if the data is taken into a hard disk or the like in the personal computer 26, a stereoscopic image can be enjoyed on the monitor of the personal computer. Here, the display operation will be described below, assuming that an image is captured in a personal computer.

First, in step S1, two images, L and R, forming a three-dimensional image are taken into, for example, a hard disk or the like in the personal computer 26. Next, in step S2, image data distributions that change along the y-direction are compared in order to make the sizes and positions of the two images thus obtained uniform.

As shown in FIGS. 4 (a) and 4 (b), the two images taken by the photographer by the photographing method as shown in FIG. May change), and vertical displacement may occur. It is difficult to arrange this by manual operation,
In the present invention, this is performed automatically.

Therefore, as shown in FIGS. 4A and 4B, a change in the image signal in the y direction of the left and right (L, R) images is obtained at the center of the screen in the x direction where the subject exists. This is based on the assumption that even if the photographer moves left and right, for example, the size of the person's face or head in the vertical direction does not change. The condition and the like may change depending on the difference between the positions of L and R), and an image change is recognized in the contour of the face, the collar of the clothes, and the like.

Step S3 in the flowchart of FIG.
Extracts feature points such as y ₁ to y ₄ shown in FIGS. 4A and 4B from the image change data thus obtained.

FIG. 5 is a flowchart showing step S in the flowchart.
3 is a subroutine for explaining the operation of "feature point extraction".

Upon entering this subroutine, step S1 is executed.
At 1 and S12, L, maximum contrast portion in a portion of the upper half of each screen in the R is a y _2, y _4. Thereby, the contour of the crown can be obtained. Alternatively, an edge between the hair and the forehead can be obtained.

Similarly, in steps S13 and S14,
The maximum contrast of the lower half of the screen is obtained from each of the L and R images, and these coordinates are defined as y ₁ and y ₃ . In this way, the difference between the magnitudes of the image magnifications that should normally match is detected as an error in the coordinate difference.

Returning to the flowchart of FIG. 3, in step S4, one of the coordinates is reduced using the coordinates obtained by the flowchart of FIG. This is specifically a subroutine as shown in FIG.

[0034] Upon entering the subroutine of the "screen size adjustment", first, at step S21, the width of the top of the head to a portion of the collar (the L-side y ₂ -y ₁ and R side of the y ₄ -y ₃ ) Is compared. As a result, when the L side is large, the process proceeds to step S22, and when the R side is large, the process proceeds to step S23. That is, these steps S22 and S23
In, the image is reduced such that the larger width is adjusted to the smaller one.

By the operation of "screen size adjustment",
Two images having two equal parallaxes can be obtained.

Returning again to the flowchart of FIG. 3, next, in step S5, the 2 obtained as described above is obtained.
L and R of the two images are determined. Then, in accordance with this determination and in step S6, display control is performed so that there is no vertical displacement. Specifically, FIG.
, One of the two images is shifted and displayed so that y ₂ = y ₄ .

Since two images suitable for stereoscopic viewing are automatically obtained by the above-described image magnification correction and position shift correction, the user only has to select two images.

Next, the automatic left / right determination in step S5 in the flowchart of FIG. 3 will be described with reference to FIGS.

FIG. 7A shows L and R images taken with parallax. The feature of these images is that in the image on the R side, the background tree 31 appears to be off the person 30 on the right side (r side) of the screen, whereas in the image on the L side, the background tree 31 is The feature is that it is shown in close contact with.

This is because, as shown in FIG. 8, when the camera is photographed at the L and R positions, the person 30 as the main subject is set at the center of the screen. The background tree 31 is photographed on the left side (l side), and the background tree 31 is photographed on the right side (r
Side).

Normally, the parallax is not so large so that the background is not largely separated. However, as shown in FIG. 7A, it is often the case that one approaches the person and the other moves away from the person. Is what happens.

In the case of a digital image, as shown in FIG. 7B, it is easy to detect a change in the image signal at the center of the screen in the y direction, and thus the obtained image signals are compared. The operation for determining the left and right images will be described with reference to FIGS. 9A and 9B.

FIG. 9 (a) shows x of the two images thus obtained.
Image distributions I ₁ and I ₂ in the directions. For the sake of simplicity, FIG. 8B is simplified. I ₁ , I ₂
When one is fixed and the other is shifted, since the same scene is photographed, there is almost no difference between the two image signals.

On the other hand, FIG._TwoStagger
And I₁Data (I ₁-I_Two) Distribution
Is shown. The face of the person is in the center in the x direction
The image position should be adjusted so that this part
It is a thing. Due to the parallax, also in the center, FIG.
As shown in (b), Δx₁The width of the image data
Large part of comes out. However, in this way
When you adjust the image, the background image is greatly shifted
In the peripheral portion, Δx as shown in FIG._TwoOf width
A part that causes a clear difference between L and R images appears. This part
L and R images are determined according to the direction in which the minute appears
Is a flowchart shown in FIG.

A subroutine for L, R image determination is started.
Then, first, as described above, in step S31, 2
Change (I) obtained when one image is traversed₁,
I_Two), Shift the center,
The position where the image of the best match is found. The next step
In step S32, the difference between the images generated at the center in this state is large.
Width of threshold part Δx₁Is detected. And step S
At 33, this width Δx ₁Next to the center that makes a big difference
A tangent is detected. A large part of this difference is on the r side.
Is an R-side image and an image on the l-side is an L-side image.
Which image is L and R
Is determined.

That is, in step S34, Δx
₂ parts of whether I ₂ parts r side is determined. here,
If it is detected that the Δx ₂ part is on the I ₂ part r side, the flow shifts to step S35 to determine that I ₂ is on the R side. Then, the process exits this routine.

If the Δx ₂ part is not on the I ₂ part r side in step S34, it is further determined in step S36 whether the Δx ₂ part is on the I ₁ part r side.
Here, if the Δx ₂ part is detected to be on the I ₁ part r side, the flow shifts to step S37 to determine that I ₁ is on the R side. Then, the process exits this routine.

[0048] In the step S36, if [Delta] x ₂ parts is determined not to be I ₁ part r side, step S38
Then, it is determined whether or not the Δx ₂ part is on the I ₂ part l side. Here, if it is detected that the Δx ₂ part is on the I ₂ part l side, the flow shifts to step S39 to determine that I ₂ is on the L side, and then exits this routine.

If it is determined in step S38 that the Δx ₂ part is not on the I ₂ part l side, it is further determined in step S40 whether the Δx ₂ part is on the I ₁ part l side.
Here, if it is detected that the Δx ₂ part is on the I ₁ part l side, the flow shifts to step S41 to determine that I ₁ is on the L side. Then, the process exits this routine.

[0050] Then, if the [Delta] x ₂ parts is detected not to be I ₁ part l side at the above step S40, step S4
Then, the process proceeds to step S2, where a warning is displayed, and the determination of L or R is left to the user.

Also, the left and right images can be distinguished by the following concept.

FIG. 7C is a diagram showing an image distribution in the x direction near the upper end of the screen. Since it is difficult to imagine that there is a subject closer to the main subject in this portion, the same image matching as in the example shown in FIG. 9 is performed using the upper part of the screen.

As shown in FIG. 11, the image distribution of I ₁ is fixed and I ₂ is shifted in the x direction.
Since R image distance is different from the matched background and the person in position x _S1, the principle of triangulation by parallax, the background image does not match (see FIG. 11 (a)). The background images match,
This is a direction in which the shift in the x direction is further reduced (FIG. 11).
(See (b)), it is possible to determine which of the images I ₁ and I ₂ is right and left.

[0054] That is, when an image did not match with a shift of x _S1 is the shift amount from the x _S1 at the position of the small x _S2 (shift amount in the x direction) was more left and right matching degree, 7
It is considered that the scene was as shown in FIG.
₁ can be considered as a left (L side) image, and I ₂ can be considered as a right (R side) image.

FIG. 12 shows an example of a subroutine for judging L and R images based on the above concept.

First, in step S51, as shown in FIG. 7 (b), a shift amount x _S1 (depending on the distance of the main subject) at which the left and right images coincide at the center of the screen is obtained.
Thereafter, at step S52, the background at the top of the screen, the shift amount x _S2 handedness matching image is obtained.

Next, in step S53, it is detected which of the images I ₁ and I ₂ corresponds to the left and right images based on the other relationships. In accordance with the detection result, the process proceeds to step S54 or S55, and it is determined which of I ₁ and I ₂ corresponds to the left or right image. According to this determination, the screen is displayed in the main flowchart of FIG.

Thus, the right and left images are displayed correctly and in the same size and position. As shown in FIG. 13, a screen 38 is provided at the center of the screen 37 of the personal computer 35, and The right and left eyes are prevented from intersecting. The partition 38 is set up and the left and right images are viewed by the observer 3.
The stereoscopic image can be easily enjoyed by viewing each of the images 9 with the left and right eyes.

Since the width between the left and right eyes differs depending on the person, the interval between the two images may be changed by operating the mouse (pointing device) 35a. In this example, the camera 10 and the personal computer 35
They are connected by a cable 36 to transmit and receive data.

Also, by incorporating software for executing the stereoscopic image viewing apparatus according to the present invention in a flash memory or the like inside the camera, a stereoscopic image can be displayed correctly and easily on a monitor LCD on the back of the camera. Can be.

At this time, if the prism 25 can be attached to the camera 10 as an adapter as shown in FIG. 14 so that the two images on the display LCD can be seen by the left and right eyes separately, In addition, stereoscopic vision can be easily realized.

As described above, according to the above-described embodiment, a stereoscopic image can be easily enjoyed by simply devising a device possessed by a general camera user.

In addition to this concept, a stereoscopic view can be easily made using a personal computer, for example, and a stereoscopic image can be displayed on a liquid crystal monitor provided in a digital camera.

According to the above embodiment of the present invention,
The following configuration can be obtained. That is, (1) display means for displaying two images photographed with parallax side by side, a regulating member for regulating a visual field so that the two images can be separated and observed by right and left eyes, and A determination unit that determines the positional relationship between the left and right images by comparing the two images with each other.

(2) The determination means compares the image signal output changes in the horizontal direction of the image, detects a portion having a large difference occurring in a state where the overall coincidence is high, and determines the difference portion within the screen. The stereoscopic image viewing apparatus according to the above (1), wherein the determination is made based on the positional relationship of:

(3) Comparison means for comparing the image signals of two captured images in the vertical direction with parallax;
Detecting means for detecting a plurality of feature points of an image signal in the vertical direction of one image; and adjusting the magnification and shift amount of each of the two images in a form in which the coordinates of the plurality of feature points in the vertical direction on the screen are aligned. And a display means for determining and displaying.

[0067]

As described above, according to the present invention, it is possible to provide a stereoscopic image viewing apparatus and a display apparatus which can easily enjoy a stereoscopic image using a personal computer or a digital camera in a general household. Can be.

[Brief description of the drawings]

1A and 1B show a configuration of a stereoscopic image viewing device of the present invention, in which FIG. 1A is a block diagram showing an electric system configuration of a digital camera applied to the stereoscopic image viewing device, and FIG. FIG. 5 is a diagram showing a display example on the LCD monitor of FIG.

FIG. 2 is a diagram showing how a stereoscopic image is obtained using a general camera.

FIG. 3 is a flowchart illustrating the operation of the embodiment of the stereoscopic image viewing apparatus according to the present invention.

FIG. 4 is a diagram for explaining feature point extraction.

FIG. 5 is a subroutine for explaining an operation of “feature point extraction” in step S3 of the flowchart in FIG. 3;

FIG. 6 is a subroutine for explaining the operation of “screen size adjustment” in step S4 of the flowchart in FIG. 3;

FIG. 7 is a diagram for describing automatic left / right determination in step S5 in the flowchart of FIG. 3;

FIG. 8 is a diagram for describing automatic left / right determination in step S5 in the flowchart of FIG. 3;

FIG. 9 is a diagram for explaining an operation of comparing left and right images by comparing image signals at the center of the screen in the y direction.

FIG. 10 is a subroutine for explaining the operation of “L, R image determination” in step S5 of the flowchart in FIG. 3;

[11] I ₁ image is an explanatory diagram for performing L, and an image determination R from I ₂ images.

FIG. 12 is a subroutine for explaining another operation example of “L, R image determination” in step S5 of the flowchart in FIG. 3;

FIG. 13 is a diagram showing an example of using a stereoscopic image viewing device according to an embodiment of the present invention.

FIG. 14 is a diagram illustrating an example in which an adapter for viewing a stereoscopic image is attached to a digital camera.

[Explanation of symbols]

 Reference Signs List 10 camera, 11 taking lens, 12 image sensor, 13 microcomputer (ASIC), 14 lens driver, 16 LCD driver, 17 memory, 18 flash memory, 19 connector, 21 release switch, 22 stereoscopic switch, 24 LCD monitor, 25 prism, 26 personal computer (personal computer), 30 main subject (person), 31 background (tree).

Claims (3)

[Claims] 1. A display means for displaying two images photographed with parallax side by side; a regulating member for regulating a visual field so as to separate and observe the two images with right and left eyes; A comparison unit that compares the left and right positional relations of the two images; and a control unit that controls a display position of the two images displayed on the display unit according to a comparison result of the comparison unit. Image viewing device. 2. The comparison means compares a horizontal image signal output change between the two images, and the control means determines whether or not a portion having a large difference generated in a state where the overall coincidence is high in the screen. The stereoscopic image viewing apparatus according to claim 1, wherein a left-right positional relationship between the two images is determined based on the positional relationship. 3. A comparing means for comparing the vertical image signals of the two captured images with parallax; and a detecting means for detecting a plurality of feature points of the vertical image signals of the two images. Control means for determining the enlargement ratio and shift amount of each of the two images in a form in which the coordinates of the plurality of feature points in the vertical direction on the screen are aligned; and the enlargement ratio and shift determined by the control means. Display means for displaying the two images based on the amount.