JPH11103473A - Stereoscopic picture display device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To attain reduction and low cost of a stereoscopic vision system circuit scale by applying a multiprocessing for a parallax video signal on the basis of each view point in a time direction and enabling an encode as a stereoscopic vision video signal for one channel. SOLUTION: A stereoscopic picture display device enables a stereoscopic vision of a pickup object picked up by adapting a lenticular lens method. In this case, a parallax video signal from each camera 1 to 5 provided with each view point is stored in corresponding frame store parts 6 to 10 with a specified frame interval and is supplied to a selector circuit 11 at the time of reading out. The selector circuit 11 applies a time axis multiplex for the parallax video from each view point and a multi video output is obtained. That is, this multi video output becomes the one in which a video is selected and is multiplexed so that it comes and goes between each view point. Thus, a picture compression processing by using a picture compression technique of an existing MPEG 2 system is enabled, it is possible to transmit the multi video signal with only a single encoder and contributes to reduction of circuit scale and low cost.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a lenticular lens type multi-view stereoscopic image display device which enables stereoscopic viewing without using glasses, and more particularly to a high efficiency compression coding of image signals for multi-view stereoscopic view. The present invention relates to a three-dimensional video display device suitable for reducing the circuit scale by reducing the number of image compression circuits that perform processing.

[0002]

2. Description of the Related Art In recent years, stereoscopic high-definition television, virtual reality (virtual reality), stereoscopic displays without glasses,
2. Description of the Related Art With the advance of three-dimensional image display technology such as moving image holography, a three-dimensional image display device capable of reproducing an image having excellent realism and realism has attracted attention.

[0003] Unlike a display device such as an ordinary television receiver, a stereoscopic image display device can display three-dimensional images, that is, can perform stereoscopic viewing, and therefore can be used for communication, broadcasting, education, medicine, exhibition, and entertainment. Applications in various fields are expected.

[0004] In general, as a method of perceiving an image three-dimensionally, there is a method of presenting an image corresponding to each independent viewpoint to the left and right eyes of a human. In such a method, for example, as in a stereo photograph, an image formed on the retinas of the left and right eyes is temporarily recorded as a photographic image, and sent to the left and right eyes to generate a three-dimensional effect. That is,
By preparing a stereo photograph, which is a so-called stereo pair, which captures the left and right images with binocular parallax, a so-called stereo pair, and observing this stereo pair using an observation mirror formed by optical elements such as lenses and mirrors , The left and right images of the stereo pair are seen as a single image in both eyes of the observer,
As a result, a fusion image is obtained in a pseudo space, and a stereoscopic video can be viewed.

[0005] However, in a method capable of stereoscopic viewing including such a method, naturally only one observer can stereoscopically view, and theoretically speaking, it is necessary to correctly stereoscopically view a stereoscopic image. The observer must observe from the center of the two left and right images. This is inconvenient when there are a plurality of observers.

Therefore, recently, in order to solve such inconveniences, images having different parallaxes corresponding to a plurality of viewpoints are prepared in advance, and even if a plurality of observers exist, they are located at the positions of the respective observers. A method has been proposed in which a corresponding image is distributed to each viewer's position by an optical system element such as a lenticular lens, thereby enabling a plurality of viewers to perform stereoscopic viewing. However, with this proposed method,
Since images of all the viewing points for the images must be recorded in advance, a large-capacity memory device for storing images is indispensable.

On the other hand, recently, digitization of image processing has progressed, and MPEG (Moving Picture Experts Group) 2
With the practical application of high-efficiency compression coding technology, such as represented by a system, it is possible to reduce the amount of information required for recording by compressing it to several tenths, for example, as in digital broadcasting and digital recording. In the transmission or recording of digital video signals, the above-mentioned image compression technique is essential.

In the MPEG2 system, the amount of information in the spatial direction is reduced by DCT (discrete cosine transform), and the amount of information in the time direction is reduced by motion-compensated interframe predictive coding.
The information amount is reduced by using a plurality of information reduction methods such as the information amount reduction by code allocation based on the appearance establishment in Huffman coding. Among them, motion compensation inter-frame predictive coding is particularly effective because it can reduce a large amount of information for a moving image. This motion-compensated inter-frame prediction coding is a type of frame difference coding, and is a method of transmitting only a difference from a previous image, and the amount of transmitted information is zero for a still image. That is, in this motion compensation inter-frame prediction coding, the motion of the moving object in the screen is detected, the position of the next moving object is predicted from the motion vector, and the difference component from the predicted position is transmitted. As a result, even for a moving image signal having an enormous amount of information, the data amount can be significantly reduced, and transmission and recording can be realized.

However, in the above-described stereoscopic video display apparatus, in order to reduce the capacity of a memory device for storing video and reduce the amount of information, a multi-viewpoint stereoscopic video signal is used for multi-view stereoscopic video signals.
If the high-efficiency compression coding method of the PEG2 system is adopted, since the video from each viewpoint is independently captured, the number of MPEG encoders and decoders as many as the number of viewpoints is required. For this reason, there has been a problem that the circuit scale of the imaging system including the stereoscopic video display device increases, and the cost becomes high.

Further, when transmitting each video data necessary for stereoscopic viewing, a transmission system having a large-capacity transmission band is required. Similarly, there is a disadvantage that the circuit becomes complicated.

[0011]

As described above, in the conventional stereoscopic video display apparatus, if a high-efficiency compression encoding method of the MPEG2 system is adopted to reduce the information amount of stereoscopic video data, Since the video data required to be viewed are captured independently of each other, MPEG2
However, there is a problem that the encoder and the decoder are required, and the cost increases due to an increase in the circuit scale of the stereoscopic imaging system.

Accordingly, the present invention has been made in view of the above-mentioned problems, and enables a parallax video signal based on each viewpoint to be multiplexed in the time axis direction and encoded as a one-channel stereoscopic video signal. Accordingly, it is an object of the present invention to provide a three-dimensional image display device capable of reducing the system circuit scale and cost.

[0013]

According to the first aspect of the present invention, there is provided a three-dimensional image display device which is arranged in a line on a horizontal line at a predetermined interval, picks up an object to be imaged at each viewpoint, and responds to each viewpoint. Imaging means for respectively outputting parallax video signals having different parallaxes, and a frame storage means provided for each of the imaging means and storing a video based on the parallax video signal from the imaging means at a predetermined frame interval; and Frame storage means corresponding to the image pickup means can be selectively controlled so as to select and input any one of the outputs of the frame storage means, and the selection order reciprocates in the juxtaposition order of the image pickup means. , A time axis multiplexing processing means for time multiplexing the input parallax video signal from the selected frame storage means and outputting a multiplexed video signal, and a multiplexed video signal from the time axis multiplexing processing means Image compression means for compressing an image by a compression method and outputting it as compressed and encoded data, and decoding the compressed and encoded data from the image compression means to obtain an original multiplexed video signal. Playback means for returning to the corresponding parallax video signal and projecting the video based on each parallax video signal, respectively, to stereoscopically display the captured video of the object to be imaged. .

According to the first aspect of the present invention, the parallax video signals for each viewpoint imaged by the imaging means are stored at predetermined frame intervals by the corresponding frame storage means. The time axis multiplex processing means can be selectively controlled so as to select and input any one of the outputs of the plurality of frame storage means, and the selection order reciprocates in the juxtaposition order of the imaging means. Then, a frame storage unit corresponding to the imaging unit is selected, and an input parallax video signal from the selected frame storage unit is time-multiplexed to output a multiplexed video signal. By performing time-division multiplexing of the captured images from each viewpoint in the moving order, the distance information of the object to be imaged measured by the movement of the viewpoint can be treated equivalently to the motion of the image. It is possible to compress information of the stereoscopic video by the compression means. Therefore, it is possible to reduce the number of image compression units provided for the number of viewpoints to one, and to reduce the circuit scale and cost. Further, the image of the object to be imaged can be displayed stereoscopically by the reproducing means.

According to a second aspect of the present invention, there is provided a three-dimensional image display device according to the first aspect,
The reproducing unit performs a reverse process of the image compression unit to obtain an original multiplexed video signal, and separates the multiplexed video signal obtained by the decoding unit into a parallax video signal for each viewpoint. A non-time-axis multiplexing processing means for outputting each; a reproduction frame storage means provided for each viewpoint and capable of storing two frames of video based on the separated parallax video signal; An image based on a parallax image signal for each viewpoint is formed on a projection panel in a predetermined arrangement, and a projector for enlarging and projecting the formed image, and a projected image from this projector are formed on a screen and formed. And a screen for displaying an image three-dimensionally using a lenticular lens.

According to the second aspect of the present invention, the operation is performed in the same manner as the above-mentioned aspect, and the projection is performed by the decoding means, the non-time-axis multiplex processing means, the frame storing means for reproduction, the reproducing means having the projector and the screen section. It can be configured as a stereoscopic image display device of the type. In this case, even if there are a plurality of observers, this stereoscopic image display device adopts a multi-lens lenticular lens system,
Stereovision can be performed from any direction.

According to a third aspect of the present invention, there is provided a three-dimensional image display apparatus according to the second aspect.
A slit-shaped shielding plate is provided on the front surface of the lenticular lens in order to project each viewpoint image as a stripe image necessary for stereoscopic viewing.

According to the third aspect of the present invention, by providing a slit-shaped shielding plate on the front surface of the lenticular lens, a projection type stereoscopic image display device of an image split type can be constituted. This makes it possible to reliably project each viewpoint image as a stripe image required for stereoscopic viewing.

[0019]

Embodiments of the present invention will be described below with reference to the drawings.

FIG. 1 is a block diagram showing an embodiment of a stereoscopic video display apparatus according to the present invention. The three-dimensional image display device of the present invention is, for example, a multi-lens type three-dimensional image display device of a lenticular lens system, and a high-efficiency compression encoding system of the MPEG2 system as an image compression encoding method of imaged video data from each viewpoint. The description is made assuming that is adopted.

As shown in FIG. 1, in the stereoscopic video display apparatus of the present invention, parallax video signals required for stereoscopic viewing are captured by cameras 1 to 5. These cameras 1 to 5 are arranged in parallel at predetermined intervals, for example, as shown in FIG. 3, so as to capture images of different parallax obtained by both eyes of a human. In this case, the viewpoint of camera 1 is viewpoint A, the viewpoint of camera 2 is viewpoint B, the viewpoint of camera 3 is viewpoint C, the viewpoint of camera 4 is viewpoint D, and the viewpoint of camera 5 is viewpoint E.

Each of the cameras 1 to 5 captures an image of a symmetric object at each viewpoint, and obtains a captured video signal, that is, a so-called parallax video signal. Frame stores 6 to 10 are connected to the cameras 1 to 5, respectively, and parallax video signals captured by the cameras are supplied to the connected frame stores 6 to 10.

The frame stores 1 to 10 store images based on the supplied parallax image signals at predetermined frame intervals. That is, the frame store unit 6 stores the image from the camera 1 installed at the viewpoint A at a predetermined frame interval, and the frame store unit 7 stores the image from the camera 2 installed at the viewpoint B at a predetermined frame interval. The frame store unit 8 stores the image from the camera 3 installed at the viewpoint C at a predetermined frame interval, and the frame store unit 9 stores the image from the camera 4 installed at the viewpoint D at a predetermined frame interval. The frames are stored at the frame intervals, and the frame store 10 stores the images from the camera 5 installed at the viewpoint E at predetermined frame intervals.

These frame store units 6 to 10 are:
It is connected to the selector circuit 11, and at the time of reading, the read parallax video signal based on each read viewpoint is supplied to the selector circuit 11.

The selector circuit 11 operates to select and switch the output sequence from the supplied parallax video signals, thereby performing time-division multiplexing of the parallax video signals from the viewpoints A to E to perform multi-view. Output video signal. FIG. 4 shows a specific example of the time axis multiplexing process in this case.

FIGS. 4A and 4B are explanatory diagrams for explaining the time axis multiplexing processing by the selector circuit 11, wherein FIG. 4A shows an image from the camera 1, FIG. 4B shows an image from the camera 2, and FIG. (C) is an image from the camera 3, and (d) is a camera 4.
4 (e) is an image from the camera 5, FIG.
(F) shows the multiplexed video output of the selector circuit 11 in which the video from each camera is multiplexed.

An image based on the multi-view parallax signal includes a viewpoint A,
FIGS. 4A to 4E show images arranged in the order of B, C, D, and E from the cameras of the corresponding viewpoint in the time axis direction. In this case, the selector circuit 11 selects the viewpoints A → B → C → D → E → E as the selection order.
The parallax video signals supplied from the respective frame stores are switched so as to reciprocate between the viewpoints such that D → C → B → A → B →... f)
To obtain a multiplexed video output as shown in FIG.

For example, the image from the camera 1 is represented by C1-1,
C1-2, C1-3,..., Images from camera 2 are converted to C2
-1, C2-2, C2-3,..., Images from the camera 3 are C3-1, C3-2, C3-3,..., Images from the camera 4 are C4-1, C4-2, C4. -3,..., The video from the camera 5 is shown as video for each frame in the time axis direction as C5-1, C5-2, C5-3,. As shown in FIG. 4 (f), C1-1, C2-1, C3-1, C4-
2, C5-1, C5-2, C4-2, C3-2, C2-
2, C1-2, C1-3, C2-3,... Are multiplexed in the selected order.

That is, by multiplexing the camera images from each viewpoint in the time axis order of movement of the viewpoint, the distance information of the object to be imaged measured by the movement of the viewpoint can be treated equivalently to the movement information of the image. For this reason, it is possible to perform information compression of multi-view stereoscopic video using an existing high-efficiency compression encoding technique such as the MPEG2 system.

Therefore, the multiplexed video output obtained by the time axis multiplexing process of the selector circuit 11 is, for example, MPEG
2 encoder 12. MPEG2 encoder 1
Reference numeral 2 outputs encoded data with a reduced data amount by subjecting the supplied multiplexed video signal to a compression encoding process of the MPEG2 system and performing image compression. The encoded data is transmitted, for example, via a transmission medium and supplied to an MPEG2 decoder 13 installed at a remote place.

In the present embodiment, the coded data from the MPEG2 encoder has been described as being transmitted via a transmission medium. However, this transmission medium is configured in place of recording / reproducing means such as a DVD, for example. Encoder 12
Is recorded in the recording / reproducing means, and the coded data reproduced so as to be reproduced is stored in the MPEG2 decoder 13.
You may make it supply to.

On the other hand, the MPEG2 decoder 13 converts the supplied encoded data into the MPEG2 encoder 1
2, the original multiplexed video signal is obtained by performing the decoding process. The restored multiplexed video signal is supplied to the selector circuit 14.

The selector circuit 14 performs a switching operation so as to distribute the video of each frame to the supplied multiplexed video signal in the order of each viewpoint. In this case, the distributed video of each viewpoint is supplied to the frame stores 15 to 19 connected to the selector circuit 14 corresponding to the respective viewpoints. Are supplied to the frame store 16, the video of the viewpoint C is supplied to the frame store 17, the video of the viewpoint D is supplied to the frame store 18, and the video of the viewpoint E is supplied to the frame store 19.

The frame store units 15 to 19
Includes a memory having a capacity capable of storing two frames of video data. In other words, by utilizing the capacity of each of the memories, it is possible to absorb a shift in the output timing of each viewpoint image from the selector circuit 14, and
The output update timings of the frame tosuars 15 to 19 can be matched. The parallax video signals of each viewpoint read from the frame store units 15 to 19 are supplied to the projector 20 connected to the frame store units.

The projector 20 projects an image group based on each viewpoint formed on a projection panel (also referred to as an LCD) by using irradiation light from a light source.
For example, as shown in FIG. 1, the light is focused on the screen 21 and transmitted, and then irradiated onto the lenticular lens 22 for stereoscopic viewing arranged on the front surface of the screen.

As shown in FIG. 5, in the pixel array of the LCD of the projector 20, images based on parallax video signals from respective viewpoints are regularly arranged in the horizontal scanning direction. Are arranged as one pixel on the LCD. That is, as shown in FIG.
One pixel on the CD is formed from viewpoints A to E, and the pixels thus formed are sequentially arranged in the horizontal scanning line direction and the vertical scanning line direction. When the projection light of the LCD is projected on the lenticular lens 22, this one pixel is projected within one pitch of the lens of the lenticular lens 22 which is formed in a semi-cylindrical shape.

Since the image of each viewpoint shown in FIG. 5 is projected on the lenticular lens 22 as a stripe image in the vertical scanning line direction, a slit-shaped light shielding plate is provided on the front surface of the lenticular lens 22, though not shown. Sometimes. A configuration called a so-called image split system is adopted. As a result, as shown in FIG. 3, it is possible to reliably project an image based on the parallax video signal from each viewpoint as a stripe image. It can project the images necessary to do so.

Next, the principle of stereoscopic vision in a multi-view stereoscopic display device employing the lenticular lens system of the present invention will be described in detail with reference to the conceptual diagrams of FIGS.

FIGS. 6A and 6B are top views of the vicinity of the screen 21 and the lenticular lens 22 shown in FIG. As shown in these figures, images are formed on the screen 21 shown in the figures based on the parallax video signals corresponding to the respective viewpoints A to E as described above. Here, in a state where each image is formed on a screen and projected on a lenticular lens 22 having a semi-cylindrical lens form, for example, as shown in FIG. When viewed, the observer sees the image C based on the viewpoint C. If the observer observes from the right side of the lenticular lens 22 as shown in FIG. 6B, the observer can see the image D based on the viewpoint D at this angle.

Therefore, by utilizing the optical system characteristics of the lenticular lens as described above, for example, FIG.
As shown in (a), when viewed from the front with respect to the lenticular lens 22, an observer can see an image D based on the viewpoint D with the right eye and an image B based on the viewpoint B with the left eye. As shown in FIG. 7B, when viewed from the right side with respect to the lenticular lens 22, an observer can see an image E based on the viewpoint E with the right eye and an image C based on the viewpoint C with the left eye. it can. Further, FIG.
As shown in (c), when viewed from the left side with respect to the lenticular lens 22, the observer can see the image C based on the viewpoint C with the right eye and the image A based on the viewpoint A with the left eye. In other words, no matter what direction the observer observes, the images with different parallaxes are incident on both eyes of the observer. It is possible to perform stereoscopic viewing of the imaged object image.

As described above, according to the stereoscopic video display apparatus of the present invention, the parallax video signals from each viewpoint necessary for stereoscopic viewing are processed by the time axis multiplexing processing circuit such as the selector circuit 11 or the like. By doing so, it is possible to perform image compression processing using an existing high-efficiency compression coding technique such as the MPEG2 method. Therefore, it is necessary to encode a time-division multiplexed multiplexed video signal as a one-channel video. Can be. As a result, the number of MPEG2 encoders conventionally required for each viewpoint can be reduced to one,
The circuit scale can be reduced to contribute to cost reduction.

Also, on the reproducing side, in this example, the coded data reproduced by the transmission or recording means can be demodulated by one MPEG2 decoder 13 into the original multiplexed video signal. Since the time axis multiplexing process can be canceled so that the multiplexed video signal demodulated by the above is distributed to each viewpoint, it is possible to restore the parallax video signal of each viewpoint necessary for stereoscopic viewing. As a result, the MPEG2 decoder conventionally required on the reproduction side is also replaced by one.
It is possible to reduce the number. That is, it is possible to reduce the cost by reducing the overall circuit scale while securing the functions of the multi-view stereoscopic image display device.

In this embodiment, the imaging side that obtains a multiplexed video signal by imaging an object to be imaged for stereoscopic viewing and the reproduction side that reproduces a stereoscopic video based on the captured multiplexed video signal are provided. Although the three-dimensional image display device configured to perform the stereoscopic vision by being integrally configured has been described, the present invention is not limited to this.
The method of transmitting the multiplexed video coded data between the PEG2 decoders 14 may be implemented as a normal broadcast mode using CATV or wireless. Accordingly, even if there are a plurality of viewers at a certain subscriber's home on the playback side, all of the plurality of viewers are images of a stereoscopic-only program broadcast from the broadcast station on the imaging side. Can be viewed three-dimensionally.

[0044]

As described above, according to the present invention,
By using the frame store units 6 to 10 and the selector circuit 11, the parallax video signal based on each viewpoint can be multiplexed in the time axis direction and encoded as a one-channel stereoscopic video signal. It becomes possible to reduce the number of MPEG2 encodings required by the number of conventional viewpoints to one. Also, it is only necessary to use one MPEG2 decoder 14 as a decoder.
As a result, it is possible to reduce the circuit scale and cost of the stereoscopic vision system.

[Brief description of the drawings]

FIG. 1 is a block diagram showing an embodiment of a stereoscopic video display device according to the present invention.

FIG. 2 is a configuration diagram showing a configuration of a projector and a screen of the apparatus of FIG.

FIG. 3 is a diagram showing a captured image of a camera installed at each viewpoint.

FIG. 4 is an explanatory diagram for explaining time axis multiplexing processing by a selector circuit.

FIG. 5 is a pixel arrangement diagram on an LCD panel in the projector.

FIG. 6 is a conceptual diagram for explaining stereoscopic vision.

FIG. 7 is a conceptual diagram for explaining stereoscopic vision.

[Explanation of symbols]

 1-5: Camera, 6-10, 15-19: Frame store section, 11, 14: Selector circuit, 12: MPEG2 encoder, 13: MPEG2 decoder, 20: Projector, 21: Screen, 22: Lenticular lens.

Claims (3)

[Claims] A plurality of image pickup means arranged in parallel at predetermined intervals on a horizontal line, each picking up an image of an object to be imaged at each viewpoint, and outputting parallax video signals having different parallaxes according to each viewpoint; A frame storage means provided for each of the means and storing a video based on a parallax video signal from the imaging means at predetermined frame intervals; and one of outputs from the plurality of frame storage means
The frame storage means corresponding to the imaging means is selected so that the selection order reciprocates in the juxtaposition order of the imaging means, and the selected frame storage means is selected from the selected frame storage means. A time axis multiplexing unit for time multiplexing the input parallax image signal and outputting a multiplexed image signal; and image compression of the multiplexed image signal from the time axis multiplexing unit by a predetermined compression method to output as compressed encoded data. Image compression means to decode the compressed coded data from the image compression means to obtain the original multiplexed video signal, further return this multiplexed video signal to a parallax video signal corresponding to each viewpoint, to each parallax video signal And a reproducing means for displaying the image of the object to be imaged three-dimensionally by projecting respective images based on the three-dimensional image. 2. The decoding device according to claim 1, wherein the reproduction unit performs a process reverse to the image compression unit to obtain an original multiplexed video signal, and a parallax image for each viewpoint from the multiplexed video signal obtained by the decoding unit. Non-time-axis multiplexing processing means for separating the signals into signals and outputting the divided signals; reproduction frame storage means provided for each viewpoint and capable of storing two frames of video based on the separated parallax video signal; A projector configured to form an image based on a parallax image signal for each viewpoint from a storage unit on a projection panel in a predetermined arrangement, and to enlarge and project the formed image; and forming an image projected from the projector on a screen. The stereoscopic video display device according to claim 1, further comprising: a screen unit for displaying the formed image three-dimensionally using a lenticular lens. 3. A lenticular lens is provided with a slit-shaped shielding plate in front of the lenticular lens to project a video image for each viewpoint as a stripe image required for stereoscopic viewing. 3. The stereoscopic video display device according to 1.