JP2001313957A - Image distribution system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To solve the problem of a narrow range of viewpoint choice of a viewer in a conventional image distribution means because the viewpoint is fixed. SOLUTION: A recording side performs feature extraction by comparing preceding and following images in terms of time with a current image obtained from other imaging units and one imaging unit. In this case, an improvement in extraction accuracy is realized by using a real stationary object existing physically as an index of the feature extraction. Obtained characteristics, such as positional information, shape information, pattern information, voice information and sound field information, are used as three-dimensional parts for each unit time of 30 ms, and the image is transmitted. A viewing and listening side reproduces the information transmitted by using a three-dimensional plotting device and a three-dimensional sound field reproducing device. In this case, the viewing and listening of the image is possible by choosing an arbitrary viewpoint by using a device or a means capable of choosing the moving direction of the viewpoint.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

[0001] 1. Field of the Invention [0002] The present invention relates to a means for distributing video information, a device therefor, and a storage medium.

[0002]

2. Description of the Related Art Conventionally, a video distribution means for capturing a three-dimensional space as a two-dimensional plane using an image pickup device and transmitting the data in units of unit time has been generally used. There is also a means called zapping for recording a two-dimensional image from multiple viewpoints and distributing the video.

[0003]

For this reason, it has been impossible for the viewer to move or change the viewer's arbitrary viewpoint on the screen. In addition, zapping is performed only by selecting from a plurality of fixed viewpoints, and a viewer cannot select an arbitrary viewpoint.

[0004]

In recent years, a technique has been developed for capturing a certain object as a three-dimensional object using a plurality of imaging devices. There is also a technique for extracting a feature as a three-dimensional object based on a temporal change in an image by one imaging device. By applying these techniques, the recording side can capture the three-dimensional space as a two-dimensional image and then extract the three-dimensional image features. At this time, the extracted information is collectively distributed for each unit time. Viewers are becoming more common in recent years 3
The images are reconstructed and drawn in a two-dimensional drawing device, and the resulting images are output to a two-dimensional or three-dimensional display device. At this time, the viewer can obtain a wider viewpoint than before by moving arbitrarily to a viewpoint within a range that can be mathematically calculated.

[0005] A three-dimensional drawing device refers to a drawing device generally called a three-dimensional graphic accelerator or a device equivalent thereto. A two-dimensional display device refers to a conventional display device such as a TV, CRT, or LCD. A three-dimensional display device is a display device that uses a plurality of two-dimensional display devices, or outputs information of two screens in a single two-dimensional display device in a time-division manner and outputs images having different viewpoints to the left and right eyes. Alternatively, it refers to a three-dimensional display device such as a hologram.

According to claim 11, when implementing these devices, the fact that a physical stationary object is always at a fixed position in space regardless of the position of the viewpoint is utilized, and these stationary objects are used for feature extraction. In order to use as a reference, the shape and color tone information of the stationary object are registered in advance.

[0007]

DESCRIPTION OF THE PREFERRED EMBODIMENTS A recording side compares a current image with a temporally preceding and succeeding image obtained from another image pickup device or the same image pickup device in a video obtained from an image pickup device and extracts features. The obtained features are transmitted as position information and shape information summarized in a unit time of, for example, every 30 milliseconds. This unit time may be shorter. At the same time, the pattern on the surface of each of the three-dimensional images is simultaneously transmitted as pattern information. Further, the sound is corrected in consideration of the Haas effect, and the sound field is transmitted as attributed sound information and sound field information. Further, transmission information may be reduced by transmitting a difference generated by the change.

At the time of this feature extraction, an actual and physically stationary object may be used as an index for feature extraction to improve the feature extraction accuracy. Further, the imaging apparatus includes a position detecting unit in the imaging apparatus, and has a function of detecting a moving image and a moving speed of the camera itself and an imaging direction with reference to the pre-registered information of claim 11. You may.

[0009] Further, a position detecting means is provided in the relay vehicle for the input image from the image pickup device to detect the image to be picked up, the moving speed of the camera itself and the image pickup direction with reference to the pre-registered information of claim 11. It may have a function of performing while performing.

[0010] Further, a position detecting means is provided in a base station which receives an input image from the imaging device from a relay vehicle,
A function may be provided for detecting the image to be captured, the moving speed of the camera itself, and the image capturing direction with reference to the pre-registered information of claim 11.

[0011] Further, a position detecting means is provided in a display device which receives an input image from the image pickup device from a relay vehicle, and detects a picked-up image, a moving speed of the camera itself and an image pickup direction. It may have a function to perform it while referring to the pre-registration information.

Since a device such as an angular velocity sensor is generally commercially available for detecting the moving speed and direction of the camera, it is easy to detect the moving speed and the imaging direction by using them. In a simple case, detection of movement and orientation in a narrow range such as camera shake correction is generalized. Although some attempts have been made on the structure of the sound field, means for identifying the position and direction of sound generation using a difference in the phase of sounds recorded by a plurality of microphones have been developed.

Next, the viewer receives the transmitted position information,
Shape information, pattern information, audio information, and sound field information can be reconstructed every 30 milliseconds using a three-dimensional image drawing device and a three-dimensional sound field reproducing device, and can be viewed on a display device. The configuration of the three-dimensional image drawing device uses a conventional three-dimensional drawing device used for a personal computer, a game machine, or the like, and there is a method of using a device called a 5-1 surround as the three-dimensional sound field reproducing device. Alternatively, two display systems may be prepared to provide different images for each eye to provide a more realistic three-dimensional display device or output to a hologram device. In order to carry out these on a personal computer or the like, they may be recorded on a storage medium and distributed.

In implementing these devices, a viewer selects an arbitrary viewpoint by using a device for selecting a moving direction and a viewpoint of the viewpoint by input means such as a joystick, a keyboard, a keypad, and a mouse. Enables viewing of video.

[0015]

According to the first aspect, the feature extracted from the two-dimensional image obtained by photographing the three-dimensional space is transmitted as a component on the screen, so that it is not necessary to directly distribute the images from a plurality of cameras, and the viewer can enjoy the image. Can transmit information in a form that allows the user to specify the viewpoint.

According to the second aspect, since the viewer can view the work from a viewpoint intended by the viewer, a more realistic video work can be distributed.

According to the third aspect, since the sound field changes in accordance with the movement of the viewpoint, more realistic video work can be distributed.

According to the eleventh aspect, since a static object such as an actual building is used as an index, errors relating to image characteristics are reduced.

According to these inventions, it is possible to provide a viewpoint with a higher degree of freedom with respect to a video work in which the viewpoint is conventionally fixed.

Claims (11)

[Claims] 1. An apparatus for dividing and transmitting feature information obtained by using a device for extracting three-dimensional features using one or more imaging devices for each unit time, and a three-dimensional feature for each unit time Video distribution means for distributing video information using a device that draws three-dimensional features obtained from a device that receives information. 2. The video distribution means according to claim 1, wherein the viewer can control the viewpoint by a control device. 3. The video / audio distribution means according to claim 1, wherein the sound field is moved in accordance with the movement of the viewpoint of the viewer. 4. A device for dividing and transmitting feature information obtained by using a device for extracting a three-dimensional feature using one or more imaging devices for each unit time, and a three-dimensional feature for each unit time A video distribution device that distributes video information using a device that draws a three-dimensional feature obtained from a device that receives information. 5. The video distribution device according to claim 4, wherein the viewer can control the viewpoint by using the control device. 6. The video and audio distribution apparatus according to claim 4, wherein the sound field is moved in accordance with the movement of the viewpoint of the viewer. 7. An apparatus for dividing and recording feature information obtained by using a device for extracting three-dimensional features using one or more imaging devices for each unit time, and a three-dimensional feature for each unit time. A video storage medium for reproducing video information using a device for drawing three-dimensional features obtained from a device for reproducing information. 8. The video storage medium according to claim 4, wherein the viewer can control the viewpoint by a control device. 9. The video and audio storage medium according to claim 4, wherein the sound field is moved in accordance with the movement of the viewer's viewpoint. 10. A storage medium on which information for realizing the function based on claim 1 in an information processing apparatus is recorded. 11. A means for improving information detection accuracy according to claim 1, wherein when performing feature extraction, a physically existing still object is registered in advance.