JP2840005B2 - Stereo image display system - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a stereo image display system for creating an artificial reality.

[0002]

2. Description of the Related Art Attention has been paid to virtual reality, in which a scenery, a real object, or a non-existent object at a remote place can be realized as if the real object exists there. In order to obtain this virtual reality visually,
Images recorded simultaneously by two TV cameras for the right eye and the left eye are recorded on a VTR tape or optical disk,
The image for the right eye and the image for the left eye are alternately displayed on the display device every 1/60 second, and only the image for the right eye is displayed for the right eye.
With the left eye, a stereoscopic image display system that performs control using a shutter or the like to show only the left-eye image realizes a moving image display having a three-dimensional effect.

[0003] In recent years, with the background of the improvement of the computing capability of computers and the improvement of graphic display technology, an image of an area assumed as a visual field by detecting a human intentional motion is displayed by a three-dimensional graphic display method and a stereoscopic display. A computer graphic system (CG system) that realizes visual artificial reality that is generated using visual techniques
Is being put to practical use. Such a system is expected to be applied to a flight simulator for flight training, a drive simulator for training of car driving, and the like.

[0004] In such a CG system, a vector
Point coordinate data as data, start point coordinates and end point coordinates are 1
Straight line data as a set, surface data of a set of straight line data, and attribute data such as points, straight lines, and surface colors,
Is stored in a large-capacity storage device such as a disk device as graphic data in a segment format for a sufficiently large area. When the display image is a curve, the polygonal line approximation is performed, and when the display image is a curved surface, the polyhedron approximation is performed to generate graphic data.

When the visual field changes due to the operation of a pointing device such as a mouse or the movement of a human, the computer detects the change in the visual field, calculates a new visual field, and stores the image that should be seen in the new visual field in the stored image. Generated by performing field-of-view coordinate conversion from existing graphic data. At this time, in order to realize stereoscopic vision, two images of a right-eye image and a left-eye image are generated and sequentially displayed on the display device in consideration of the parallax between the right and left eyes, and only the right-eye image is displayed with the right eye. For the left eye, control is performed by a shutter or the like to show only the left-eye image.

[0006]

In the conventional stereo image display system, a video photographed as described above is recorded as it is on a VTR or an optical disk, and this image data is read out and reproduced for display. Are better. However, there has been a problem that data recorded in a time series can be displayed as a moving image only in the order of the time series, and it is not possible to cope with a change in a visual field due to a human motion or the like.

Further, since the CG system generates a display image based on graphic data as described above,
The displayed image is an approximation of the shape, color, pattern, and the like of an actually existing object. Therefore, in order to sufficiently express the sense of reality, it is necessary to perform a very close approximation, but in this case, the amount of graphic data becomes enormous. However, in order to have a realistic feeling as a moving image, it is necessary to maintain a display speed of about 30 frames per second or more, but with the current three-dimensional graphic display technology, a graphic figure that can be displayed in a short time of about 1/30 second is required. There is a limit to the amount of the image, and it is not possible to handle the amount necessary to express an original image such as a landscape without discomfort. For this reason, there is a problem in that the figure approximation of the image must be coarse and the amount of data to be handled must be limited, resulting in a display with poor realism.

SUMMARY OF THE INVENTION The present invention has been made to solve the above problems, and is capable of responding to changes in the visual field of a sighted human and capable of displaying a highly realistic three-dimensional image. It is an object to provide an image display system.

[0009]

In the stereoscopic image display apparatus according to the present invention, the right-viewing and left-viewing moving images taken in advance are recorded on a recording medium at a position (spatial addressing) corresponding to a viewpoint position and a line-of-sight direction at the time of photographing. Data compression) and store it. At this time, a randomly accessible recording device is selected as the recording medium. Then, the display field of view is detected, read out from the compressed data storing the image corresponding to the display field of view based on spatial addressing, decompressed, and the right-eye image and the left-eye image are sequentially displayed on the display device to display a moving image. And

That is, the stereo image display device according to the present invention comprises: (a) a stereo image input unit which is arranged at a remote place and collects right-eye image data and left-eye image data in a designated visual field; b) an image transmission unit for transmitting image data collected by the stereo image input unit and data designating a field of view; and (c) a recording device capable of random access, and recording the image data transmitted by the image transmission unit. A stereo image recording unit that stores the image data in the device and reads the image data from the recording device; and (d) a stereo image that displays the right-eye image data and the left-eye image data read by the stereo image recording unit in order for a predetermined time. (E) Designating a field of view for a stereo image input unit and designating image data to be read for a stereo image recording unit. When storing the image data in the recording device, the stereo image recording unit follows the image data collection speed, compresses the image data for each screen, and then stores the image data in the recording device. When the image data is stored at a recording position determined based on the data designating the field of view, and the image data is read out from the recording device, the image data that has been subjected to the data compression is stored within a predetermined time every one screen. The data is expanded and read.

The stereo image input unit includes a first photographing device for collecting right-eye image data, a second photographing device for collecting left-eye image data, a first photographing device, and a second photographing device. A movable support device that supports the device and changes the viewpoint position and the line-of-sight direction according to instructions,
The image processing apparatus may further include a three-dimensional encoder that mixes the collected image data of the first imaging device and the collected image data of the second imaging device and outputs a three-dimensional image data signal to the image transmission unit. The three-dimensional encoder may be characterized in that the right-eye image and the left-eye image correspond to odd-numbered and even-numbered fields, respectively, and are converted into signals of the United States Television System Committee.

The image transmission unit is a first encoding device that performs pulse code modulation on the three-dimensional image data signal output from the optical image input unit, and converts a signal output from the first encoding device into optical information. A first optical transmitter for receiving and outputting light, a first optical receiver for receiving light from the first optical transmitter, and a demodulated pulse code modulation signal output from the first optical receiver. A first demodulator for outputting to the stereo image recording unit, a second encoding device for pulse code modulating a field-of-view designation data signal for image data collection of the stereo image input unit output from the image control unit,
A second optical transmitter that converts a signal output from the second encoding device into optical information and outputs the optical information, a second optical receiver that receives light from the second optical transmitter, and a second optical transmitter And a second demodulator that demodulates the pulse code modulation signal output from the receiver and outputs the demodulated signal to the stereo image recording unit. Note that the first optical transmitter and the second
The optical receiver is a component of a first spatial optical link device having an optical transmission / reception function, and the second optical transmitter and the first optical receiver are a component having an optical transmission / reception function. It may be characterized in that it is a component of the second spatial optical link device.

The stereo image recording unit includes an analog-to-digital converter for digitizing an image data signal;
A data compressor for compressing digital image data output from the analog-to-digital converter in accordance with a predetermined algorithm, a recording device capable of storing and reading compressed data output from the data compressor, and a recording device capable of writing and reading, and an output from the recording device A data decompressor for performing an inverse operation of the algorithm of the predetermined algorithm on the compressed data to reproduce image data, and a digital decompressor output from the data decompressor.
A digital-to-analog converter that uses the data signal as an image signal. The compressed data for one screen may be stored at a recording position in the recording device corresponding to the viewpoint position and the line-of-sight direction with respect to the one screen. Further, the analog-to-digital converter, the data compressor, the data decompressor, and the digital-to-analog converter may be provided for each of the three primary colors of light.

The stereo image display unit receives the image signal output from the image recording unit, converts the image signal into a predetermined scanning speed, and converts the image according to the signal output from the scanning speed converter. And a three-dimensional image display device for displaying.

The image control unit includes a computer for controlling the entire system, a first pointing device for inputting a designation of a collection field of view for stereo image input to the computer, and a collection device input from the first pointing device. A first interface for outputting designated data of the field of view to the image transmission unit, a second pointing device for inputting designation of a display field of view for determining image data to be read from the image recording unit to the computer, and a second pointing device A second interface for outputting the display field designation data input from the device or the storage position of the image data corresponding to the display field to the image transmission unit, and the right eye in synchronization with the display operation of the stereo image display unit. Or a stereoscopic eyewear device for selectively blocking the visual field of the left eye. . The first pointing device is a joystick, and the second pointing device is a joystick.
The pointing device may be a joystick or a visual field detector that measures the movement and line of sight of a human being.

[0016]

In the stereo image display apparatus of the present invention, first, image data for three-dimensional display is prepared according to the following procedure. In accordance with an operation of the joystick or the like, data for designating a collection visual field is output from the image control unit to the image transmission unit. The image transmission unit transmits the collected visual field data and transmits it to a stereo image input unit installed at a remote location. The stereo image input unit collects the right-eye image data and the left-eye image data based on the line-of-sight data and the viewpoint specified by the collected visual field data, and outputs the collected image data to the image transmission unit. The collected image data is input to the stereo image recording unit via the image transmission unit. The stereo image recording unit compresses the image data according to a predetermined algorithm (for example, the JPEG system) for each single-view screen while following the collection speed of the input image data, and then performs spatial addressing to perform data addressing. To be stored.

Next, 3 corresponding to the set visual field is performed in the following procedure.
Display a two-dimensional image. Display field data is input from the joystick or the field detector to the image controller. The image control unit analyzes the display visual field data and instructs the stereo image recording unit to read out image data corresponding to the display visual field. Upon receiving the read command, the stereo image recording unit recognizes the storage position of the display image data inside the recording device by spatial addressing, and reads out the stored compressed data from the recording device. The read compressed data is decompressed by an inverse operation of the algorithm of the compression algorithm, converted into image data for display, and then output to the stereo image display unit. The stereo image display unit sequentially displays display image data for the right eye and the left eye for display. In this display, the human visual field is controlled so that the right-eye image is displayed when the right-eye image is displayed and the left-eye image is displayed when the left-eye image is displayed. Note that the system operation time required from the reading of the compressed data from the recording device to the display of the image satisfies the display speed (24 frames / sec or more) necessary to realize the virtual reality as a moving image. .

[0018]

Embodiments of the present invention will be described below with reference to the accompanying drawings. FIG. 1 is a configuration diagram of a stereo image display system according to an embodiment. This system includes: (a) a stereo image input unit 100 that is located at a remote place and collects right-eye image data, left-eye image data, right-ear audio data, and left-ear audio data in a designated field of view. (B) an image transmission unit 200 that transmits the image / audio data collected by the stereo image input unit 100 and data specifying a visual field;
(C) The image / sound data collected by the stereo image input unit 100 is stored via the image transmission unit 200 after data compression following the collection speed of the image / sound data,
At the time of reading, a stereo image recording unit 300 that decompresses and outputs image data and audio data of one screen related to one eye within a predetermined time, and (d) right eye of one screen read from the stereo image recording unit 300 Image display unit 400 for sequentially displaying image data for one screen and image data for the left eye for one screen, and reproducing sound.
And (e) an image control unit 500 for specifying a collection visual field to the stereo image input unit 100 and specifying read image data to the stereo image recording unit 300.

Here, the stereo image input unit 100
A video camera 111 for collecting image data for the right eye,
A video camera 112 for collecting left-eye image data;
Stereo microphone 150 for recording audio
And the video camera 111, the video camera 112, and the stereo microphone 150,
A robot that is a movable support table 120 that changes the direction of the visual field and the starting point according to an instruction;
Three-dimensional encoder 1 that mixes the collected image data of the video camera 112 and the collected image data of the video camera 112 and outputs a three-dimensional image data signal in the form of an NTSC signal to the image transmission unit 200
30.

The image transmission unit 200 includes the optical image input unit 10
A function of converting the three-dimensional image data signal output from 0 into a signal of the NTSC system, converting the signal into sound and control signal information (in the case of the present embodiment, position information of the movable support, etc.), and converting the signal into a PCM signal; A PCM modem 211 having a function of demodulating an input PCM signal and outputting the demodulated PCM signal to a stereo image input unit as collected visual field data;
A spatial optical link device 221 that is electrically connected to the modem 211 and includes an optical transmitter that converts an input electrical signal into an output optical signal and an optical receiver that converts an input optical signal into an output electrical signal.
A spatial optical link device 222 optically connected to the spatial optical link device 221 and including an optical transmitter for converting an input electric signal to an output optical signal and an optical receiver for converting an input optical signal to an output electric signal; Electrically connected to the optical link device 222,
A function of demodulating a PCM signal input from the spatial optical link device 222 and outputting a three-dimensional image data signal and an audio data signal of the NTSC system to the stereo image recording unit 300 and a function of outputting control signal information to the image control unit 500 And the collected visual field data output from the image control unit 500 to a PC
A PCM modulator / demodulator 212 having a function of converting the signal into an M signal and outputting the converted signal to the spatial optical link device 222. In this embodiment, multiplex PCM transmission having the following specifications is adopted.

Transmission method: PCM-IM Transmission speed: 118.125 MHz (NRZ) Transmission capacity: One video channel ... 10.7386 MH
z (subcarrier synchronization method adopted) Audio 2 channels: 48 kHz (16-bit PCM sampling) 28-bit parallel data: ~ 20 kbps / b
it RS232C 2 channels ... ~ 20kbps / b
it

The stereo image recording section 300 is an analog / digital converter for each of the three primary colors for digitizing the image data signal.
A digital converter (not shown) and digital image data output from the analog-to-digital converter are data-compressed and output in accordance with the JPEG system, and the compressed data is input and data is expanded in accordance with the JPEG system to obtain image data. Image data compression for each of the three primary colors to be output
It comprises a decompressor, a fixed head disk device 330 for storing compressed data and capable of writing and reading, and a control device 320 for controlling access to the fixed head disk device 330. When compressing data to the fixed head disk device 330, spatial addressing is performed according to the viewpoint and the line of sight, and the compressed data is stored at the corresponding position (determined by cylinder, track, and sector) of the fixed head disk device 330.

The stereo image display unit 400 receives an image signal output from the image recording unit 300 and displays a picture according to a scanning speed converter 420 for performing double speed scanning and a signal output from the scanning speed converter 420. And a speaker system that reproduces audio according to audio data. In this embodiment, a method of attaching a polarizing filter to a monitor screen of a three-dimensional image display device and viewing the image through a polarizing glass (not shown) is adopted as a method of stereoscopic viewing of a stereo image. In addition, in the case of the mill method in which the left and right sides are alternately switched with a 60 Hz interlace using a normal video signal, display is performed every 1/30 second for one eye, and flickering due to flicker appears. Therefore, in this embodiment, double-speed scanning is performed by the scanning speed converter 420, and an image is displayed for one eye every 1/60 second.

An image control unit 500 includes a computer 510 for controlling the entire system, and a joystick or a mouse for inputting designation of a collection field of view for inputting stereo images to the computer and designation of a display field of view for determining image data to be read from the image recording unit. Input device group 520, an interface (not shown) for outputting the input designated field of view data to the image transmission unit, and storage of input designated field of view data or image data corresponding to the displayed field of view. An interface (not shown) for outputting the position to the image transmission unit, and the above-mentioned polarizing glasses for selectively blocking the visual field of the right eye or the left eye in synchronization with the display operation of the stereo image display unit. Is done.

In the stereo image display system according to the present embodiment, a three-dimensional stereo image having an artificial reality is displayed as follows.

First, image data for three-dimensional display is prepared according to the following procedure. In accordance with the operation of the joystick or the like, the viewpoint and line-of-sight direction designating the collection visual field are output from the image control unit 500 to the image transmission unit 200 as the collection visual field data. The image transmitting unit 200 transmits the collected visual field data by optical PCM to the stereo image input unit 100 installed at a remote place. The stereo image input unit 100
The robot 120 is moved in the line-of-sight direction and viewpoint specified by the collected visual field data, and the right-eye video camera 11 and the left-eye video camera 112 capture images to collect the right-eye image data and the left-eye image data. With a stereo microphone. The right-eye and left-eye image data correspond to the odd and even fields, and
The signals are combined as an SC signal and output to the image transmission unit 200. The audio data is output to the image transmission unit 200 as it is. The collected image data is input to the stereo image recording unit 300 via the image transmission unit 200. The stereo image recording unit 300 converts the input image data into a JPE image for each single-view screen while following the collection speed.
After data is compressed by the image data compression / decompression device according to the algorithm of the G system, the data is spatially addressed and stored in the data recording device 330. For example, as shown in FIG. 2, a moving image is captured while scanning the arm of the robot 120 in the horizontal direction, and sequentially scanned in the vertical direction to perform spatial addressing in the form of a matrix and recorded.

Next, 3 corresponding to the set visual field is performed in the following procedure.
Display a two-dimensional image. The data of the viewpoint and the line of sight for designating the display visual field from the joystick are stored in the image control unit 50.
Input to 0. The image control unit 500 analyzes the display visual field data and instructs the stereo image recording unit 300 to read out image data corresponding to the display visual field. Upon receiving the read command, the stereo image recording unit 300 recognizes the storage position of the display image data inside the recording device 330 by spatial addressing, and stores the stored compressed data in the recording device 3.
Read from 30. For example, if the field of view changes in an oblique direction as shown in FIG. 2, compressed data corresponding to a space address in an oblique direction is read out from a space address in a matrix. The read compressed data is expanded by an image data compressor / decompressor in accordance with the algorithm of the JPEG system, converted into display image data, and output to the stereo image display unit 400. The stereo image display unit 400 sequentially displays display image data for the right eye and the left eye for display every 1/120 second. In this display, since a polarizing filter is attached to the monitor screen of the three-dimensional display device 410, by viewing the screen through polarizing glasses, the right-eye image is displayed when the right-eye image is displayed, and the left-eye image is displayed. At the time of display, the image is viewed from the left. The JPEG method of compressing and expanding image data has a high degree of data compression and high flexibility of expansion, so that the capacity of the recording device can be saved and the access time of the compressed image data can be shortened. It is possible.

As described above, the stereoscopic image display system according to the present embodiment displays a very realistic image as a three-dimensional image in accordance with the set visual field and following the change in the set visual field.

The present invention is not limited to the above embodiment, and various modifications are possible. For example, the image transmission uses a spatial optical link, but may use an optical fiber connection.
In addition, a visual field detector that detects human foot and head movements may be used to input a display visual field, and a head mounted display or a video projector may be used as a display device.

If two head-mounted displays or two video projectors are used to display the right-eye image and the left-eye image independently, no polarizing glasses are required, and double-speed scanning for preventing flicker is unnecessary. Becomes

Further, if the collected image data is directly displayed on the three-dimensional display device, a stereo image display system capable of stereoscopically viewing an image of a remote place in real time can be provided.

[0032]

As described in detail above, the stereoscopic image display apparatus of the present invention spatially addresses images of various visual fields previously photographed as still image data and stores them as still image data. Since the addressed image data is displayed as a moving image at a natural speed, it is possible to realize a stereoscopic image display device with almost no unnaturalness in image quality and improved artificial reality.

[Brief description of the drawings]

FIG. 1 is a configuration diagram of a stereo image display system according to an embodiment of the present invention.

FIG. 2 is an explanatory diagram of an operation of the stereoscopic image display system according to the embodiment.

[Explanation of symbols]

100: stereo image input unit, 200: image transmission unit, 3
00: stereo image recording unit; 400: stereo image display unit; 500: image control unit.

────────────────────────────────────────────────── (5) References JP-A-51-90514 (JP, A) JP-A-63-201878 (JP, A) (58) Fields investigated (Int. Cl. ⁶ , DB name) H04N 13/00-17/06

Claims (11)

(57) [Claims] 1. A stereo image input unit which is arranged at a remote place and collects right-eye image data and left-eye image data in a specified visual field, and the image data and the visual field collected by the stereo image input unit. An image transmission unit that transmits data specifying the image data, and a recording device that is randomly accessible. The image data transmitted by the image transmission unit is stored in the recording device, and the image data is stored in the recording device. A stereo image recording unit that reads from the stereo image recording unit, a stereo image display unit that sequentially displays the right-eye image data and the left-eye image data read by the stereo image recording unit for a predetermined time, and a stereo image input unit. An image control unit that specifies the visual field and specifies the image data to be read from the stereo image recording unit. Wherein the stereo image recording section, when storing the image data to the recording device, following the acquisition rate of the image data, the image data 1
After compressing the data for each screen, storing the image data in a recording position of the recording device determined based on the data designating the field of view, when reading out the image data from the recording device, A stereo image display system, wherein the compressed image data is expanded and read out within the predetermined time for each screen. 2. A stereoscopic image input unit comprising: a first photographing device that collects image data for a right eye; a second photographing device that collects image data for a left eye; A movable supporting device that supports an imaging device and changes a viewpoint position and a line-of-sight direction in accordance with an instruction; and mixes the collected image data of the first imaging device with the collected image data of the second imaging device. The stereo image display system according to claim 1, further comprising: a three-dimensional encoder that outputs a three-dimensional image data signal to the image transmission unit. 3. The three-dimensional encoder according to claim 2, wherein the right-eye image and the left-eye image correspond to odd-numbered and even-numbered fields, respectively, and are converted into a signal of the United States Television System Committee. Stereo image display system. 4. An image transmission unit, comprising: a first modulator that performs pulse code modulation of the three-dimensional image data signal output from the optical image input unit; and a signal that is output from the first modulator. A first optical transmitter that converts and outputs optical information, a first optical receiver that receives light from the first optical transmitter, and a pulse code that is output from the first optical receiver After demodulating the modulated signal, a first signal is output to the stereo image recording unit.
A second modulator that pulse code modulates a field-of-view designation data signal for image data collection of the stereo image input unit output from the image control unit, and a signal output from the second modulator A second optical transmitter that converts the optical information into optical information and outputs the optical information; a second optical receiver that receives light from the second optical transmitter; and a pulse that is output from the second optical receiver. After demodulating the code modulation signal, a second output to the stereo image recording unit is provided.
The stereo image display system according to claim 1, further comprising: 5. The first optical transmitter and the second optical receiver are components of a first spatial optical link device having an optical transmitting / receiving function, and the second optical transmitter and the second optical transmitter The stereo image display system according to claim 4, wherein the first optical receiver is a component of a second spatial optical link device having an optical transmission / reception function. 6. An analog-to-digital converter for digitizing an image data signal, and a data compressor for compressing digital image data output from the analog-to-digital converter according to a predetermined algorithm. A recording device capable of writing and reading compressed data output by the data compressor; and performing an inverse operation of the algorithm of the predetermined algorithm on the compressed data output from the recording device to reproduce image data. The stereo image display system according to claim 1, further comprising: a data decompressor that performs a digital data signal output from the data decompressor, and a digital-to-analog converter that uses the digital data signal as an image signal. 7. The stereo image according to claim 6, wherein the compressed data for one screen is stored in a recording position in the recording device corresponding to a viewpoint position and a line-of-sight direction regarding the one screen. Display system. 8. The digital to analog converter, the data compressor, the data decompressor, and the digital to digital converter.
The stereo image display system according to claim 6, wherein the analog converter is provided for each of the three primary colors of light. 9. A scanning speed converter that receives an image signal output from the image recording unit and converts the image signal into a predetermined scanning speed, and according to a signal output from the scanning speed converter. The stereo image display system according to claim 1, comprising: a three-dimensional image display device that displays an image. 10. A computer for controlling the entire system, a first pointing device for inputting designation of the collection field of view of the stereo image input to the computer, and a first pointing device. A first interface for outputting designation data of the collection field of view input from the image transmission unit to the image transmission unit; and a second interface for inputting designation of a display field of view for determining image data to be read from the image recording unit to the computer. A pointing device, and a second interface for outputting the designated data of the display field of view input from the second pointing device or the storage position of image data corresponding to the display field of view to the image transmission unit, In synchronization with the display operation of the stereo image display unit, the visual field of the right eye or the left eye is selectively blocked. The stereoscopic image display system according to claim 1, further comprising: a pair of stereoscopic glasses devices. 11. The first pointing device is a joystick, and the second pointing device is a joystick.
The stereoscopic image display system according to claim 10, wherein the device is a joystick or a visual field detector that measures a movement and a line of sight of a human being.