JP2001028765A - Transmission reception system and transmission reception method of three-dimensional video image - Google Patents

Abstract

PROBLEM TO BE SOLVED: To generate a three-dimensional video image specific to a viewer on the basis of video source information in response to the preference of the viewer selected among a plurality of video source information sets sent from a broadcast station and to provide a three-dimensional video image full of presence that is recognized by the eyes of a player played in a soccer playground or an actor/actress playing on a stage while a viewer is resident in its house. SOLUTION: The transmission reception system is provided at least with a transmission means 1 that transmits a plurality of video source information sets D1i prepared in advance to generate a three-dimensional video image from a transmitter side to a viewer side, a reception means 2A that receives the video source information D1i sent from the transmission means 1, a selection means 4 that selects optional video source information D1i received by the reception means 2A, and a video image generating means 6 that generates a three-dimensional video image on the basis of the video source information D1i selected by the selection means 4 and a viewer selects the video source information D1i.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an ISDB (Integ) for integrating data broadcasting, audio broadcasting, and television broadcasting.
The present invention relates to a 3D video transmission / reception system and a transmission / reception method suitable for a next-generation 3D digital broadcasting system using a rated services (Digital Broadcasting) service or an interactive service network.

More specifically, an image creating means for creating a three-dimensional image based on a plurality of pieces of video material information sent from a broadcasting station or the like is provided on the receiving side, and the plurality of pieces of video material information are used in accordance with a viewer's preference. To create a viewer-specific 3D video based on the selected video material information, and to be recognized by the gaze of players playing on a soccer field or actors performing on the stage while staying at home. It is intended to allow the viewer to view a realistic 3D video.

[0003]

2. Description of the Related Art Conventionally, for a program created at a broadcasting station, a method of transmitting an analog video signal of 30 frames per second to a viewer side as two-dimensional video information has been adopted. Since a large amount of video information cannot be transmitted by this method, a digital broadcasting system for modulating a digital signal of two-dimensional video information and broadcasting the digital signal is becoming widespread.

In recent years, ISDBs (Integrated Services) for integrating data broadcasting, audio broadcasting, and television broadcasting have been developed.
s Digital Broadcasting) services have also become widespread. According to this type of ISDB service, a plurality of programs are displayed as a “headline screen” in one screen, and “weather forecast”, “news”,
"Program guide", "movie", "cooking program", "sports program", etc. can be viewed at any time. These pieces of video information are multiplexed from the broadcast station side to the viewer side and digitally transmitted. Therefore, the receiving side can select his favorite program and view "weather forecast" at any time.

[0005]

By the way, according to the conventional digital broadcasting system, "weather forecast", "news", "program guide", "movie", "cooking program" The viewer passively views the two-dimensional video information such as.

[0006] Therefore, viewer-specific “sports programs” such as “sports programs” based on video material information that the viewer actively worked on.
I cannot watch 3D images. For example, when there is a request to watch a three-dimensional image with a sense of reality that is recognized by the eyes of players playing on a soccer field or actors performing on the stage while staying at home, the following problem occurs. There is.

In the current multiplex broadcasting system, the broadcasting station does not prepare video material information necessary for stereoscopic video processing on the receiving side. Therefore, since the video material information is not sent from the broadcast station or the like, it is not possible to select the video material information according to the preference of the viewer.

As a result, it is not possible to create a viewer-specific three-dimensional video based on video material information according to the viewer's preference. Therefore, it is not possible to view a three-dimensional video with a sense of reality recognized by the eyes of players playing on the soccer field or actors performing on the stage while staying at home.

Accordingly, the present invention has been made in view of the above-described problems, and has been developed in consideration of a viewer's preference selected from a plurality of pieces of video material information transmitted from a transmitting side such as a broadcasting station. Ability to create viewer-specific 3D images based on information, as well as realistic 3D images recognized by the eyes of players playing on a soccer field or actors performing on the stage while at home. An object of the present invention is to provide a three-dimensional video transmission / reception system and a transmission / reception method capable of providing a video.

[0010]

A transmitting means for transmitting at least a plurality of pieces of video material information prepared in advance to create a three-dimensional video from a transmitting side to a viewer side, and the transmitting means are provided. Receiving means for receiving the video material information sent from the device, selecting means for selecting arbitrary video material information from the video material information received by the receiving means, and selecting the video material information based on the video material information selected by the selecting means. And a video creating means for creating a three-dimensional video by using a video transmitting / receiving system, wherein selection of video material information is performed by a viewer.

According to the three-dimensional video transmission / reception system of the present invention, at least a plurality of pieces of video material information prepared in advance for creating a three-dimensional video are transmitted from the transmission side to the viewer side by the transmission means. On the receiving side, the video material information transmitted from the transmitting means is received by the receiving means. Any video material information is selected by the viewer from the video material information received by the receiving means. The selection of the video material information at this time is performed by the viewer via the selection means. The image creating means creates a three-dimensional image based on the image material information selected by the selecting means.

Therefore, it is possible to create a viewer-specific three-dimensional video based on video material information selected according to the viewer's preference from a plurality of video material information prepared in advance on the transmission side. As a result, the viewer can not only passively watch the video provided by the broadcasting station as it is, but also view the three-dimensional video that has been actively and uniquely stereoscopically processed.

[0013] A method for transmitting and receiving a three-dimensional image according to the present invention comprises:
At least, a plurality of video material information for preparing a three-dimensional video is prepared in advance on the transmission side, and the video material information prepared on the transmission side is transmitted from the transmission side to each viewer side, and from the transmission side, The viewer receives the transmitted video material information, and the viewer selects arbitrary video material information from the video material information to create a three-dimensional video.

According to the method of transmitting and receiving a three-dimensional video according to the present invention, the viewer does not simply passively watch the video given from the broadcasting station, but the viewer actively and independently stereoscopically views the video. You can watch the processed 3D video. Therefore, it is possible to view a three-dimensional video with a sense of reality recognized by the eyes of players playing on the soccer field and actors performing on the stage while staying at home. This makes it possible to construct a next-generation three-dimensional digital broadcasting system or the like.

[0015]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS A three-dimensional video transmission / reception system and a transmission / reception method according to an embodiment of the present invention will be described below with reference to the drawings. (1) First Embodiment FIG. 1 is a perspective view showing a configuration example of a three-dimensional video transmission / reception system 100 as an embodiment according to the present invention. In this embodiment, a video creating means for creating a three-dimensional video based on a plurality of video material information sent from a broadcast station or the like is provided, and a video image is selected from the plurality of video material information according to a viewer's preference. Ability to create viewer-specific three-dimensional images based on video material information, as well as a sense of realism that is recognized by the eyes of players playing on a soccer field or actors performing on the stage while staying at home. A three-dimensional image can be provided.

A three-dimensional video transmission / reception system 1 shown in FIG.
00 is provided with at least the transmission means 1 for application to a three-dimensional digital broadcasting system or the like. This transmitting means 1 is provided in a broadcasting station or the like. In this example, a plurality of video material information D1i (i = 1 to n) prepared in advance are transmitted from the transmission side to the viewer side using the transmission means 1. Here, the video material information D1i refers to information for creating a three-dimensional video. In this example, the video material information D1i includes at least background information D11 expressing the background of the video, object information D12 expressing an object moving in the background of the video, initial arrangement information D13 of the object with respect to the background, and the object The position information D14 associated with the movement of the object and the operation information associated with the operation of the object are added. All of the video material information D1i may be prepared on the transmission side, or, of course, a part of the video material information D1i may be prepared on a storage medium such as a CD-ROM and prepared on the reception side. For example, a soccer goal or a field of a soccer game or the like is read from a CD-ROM.

The transmitting means 1 is provided with a multiplexing means 1A. Two or more pieces of video material information D1i for creating a three-dimensional video are multiplexed and transmitted to the viewer. That is, the video material information D1i is multiplexed and multiplexed from the transmitting side to the receiving side using a plurality of channels. For example, using the digital television broadcasting network, cable television communication line or Internet communication line, the video material information D
1i is transmitted to the viewer side. Digital television broadcasting networks include terrestrial channels and satellite channels. These multiplex transmission media are collectively referred to as ISDB (Integrated Server).
This is called vice Digital Broadcasting :) service.

[0018] In this example, with regard to the setting of the fee for the viewer accompanying the use of the channel related to the ISDB service, it is preferable to charge based on the number of channels used and the channel usage time.
When the pay-as-you-go system is adopted, the viewer may be charged according to the amount of the video material information within the reception time. Regarding the fee setting associated with the use of this channel, not only the viewer is targeted. For example, a television commercial can also be created from a 3D polygon image. In this case, a fee is charged to a program provider (sponsor) based on the number of channels used and the channel usage time. For pricing at this time,
The program provider may be charged according to the amount of video material information within the transmission time of the 3D commercial video.

In this example, the video material information D1i is based on the moving picture compression standard MPEG-4 (Moving Picture Expert).
s Group-4) A data stream having a data series is transmitted. This is for efficiently transmitting a large amount of video material information D1i to a limited channel. The channels are prepared in advance by the quantity of the background of the image. This is because all channels can be used efficiently by using the background of the video as a reference. In this example, the video material information D1i is transmitted from the transmission side to the reception side using a plurality of channels, and the video material information D1i is allocated to the plurality of channels. For example, operation information D15,
Other background information D11, object information D12, arrangement information D
13 and the position information D14 are allocated to different channels. In this case, the operation information D15 is transmitted in real time, and the object information D12, the arrangement information D13, and the position information D1 are transmitted.
4 is transmitted together with the background information D11 of the next video.

Further, two-dimensional video information relating to ordinary television broadcasting is transmitted using one of the above-described channels, and viewpoint video information viewed from a subject such as an athlete or an actor is used as video material information D1i. It is adapted to transmit to the receiver using one of the channels. At this time, when there are a plurality of subjects in the background of the video, a channel is assigned to each viewpoint video information of the subject. With this configuration, the receiving side can combine the viewpoint video viewed from the subject with the prepared composite image.

In this example, if the amount of video material information D1i constituting one scene of video becomes enormous, the number of channels is increased and the video material information D1i is divided and transmitted to the receiving side. Good. Or, divided video material information D1
The other video material information D1i may be transmitted to the receiving side using the remaining channels during the period when i is being transmitted to the receiving side. This is because the background information D11 may be enormous in the last scene of a certain video or the like.

On the other hand, on the viewer (receiving) side, receiving means 2A
Is provided, and receives the video material information D1i sent from the transmitting means 1 such as a broadcasting station. The receiving unit 2A is provided with an information separating unit 2A, and separates two or more pieces of video material information D1i from the multiplexed information transmitted from the transmitting unit 1. The storage means 3 is connected to the receiving means 2A so as to temporarily record the video material information D1i. The storage unit 3 has a memory capacity corresponding to the background information D11 described above. This is because the background information D11 expresses the background of one scene of the video.

In this example, the frequently used three-dimensional video information in the video material information D1i is advertised on the transmitting side during the period of displaying the initial screen of the video or in the two-dimensional video information. All the channels are allocated and transmitted during the period during which the screen is displayed. On the receiving side receiving this, the three-dimensional video information acquired during these periods is stored in the storage means 3.

For example, background information D11, object information D1
2, arrangement information D13, position information D14, and operation information D1
When the data acquisition of one group is completed, the channel is surrendered on the receiving side for data acquisition of the other group. With this configuration, the receiving side can acquire the three-dimensional video material information D1i in advance, and the transmitting side can efficiently transmit the video material information D1i.

With respect to the video material information D1i, both the background information D11 of the video and the video information with no change are transmitted from the transmitting side to the receiving side, so that the background information D11 which is not used immediately on the receiving side. Is temporarily stored in the storage means 3. Further, when receiving the video material information D1i including the two-dimensional and three-dimensional video information sent from the transmission side to the reception side, the first three-dimensional video information is a period during which the initial screen of the video is displayed or It is acquired during the period when the advertisement screen is displayed in the two-dimensional video information. This is to efficiently receive the video material information D1i multiplexed on a limited channel.

Further, the video material information D1i constituting one scene of the video includes background information D11 in the last scene or the like.
Is enormous, the video material information D1i is divided and sent using the surplus channel.
On the receiving side, the video material information D obtained during the above period
1i is temporarily stored in the storage means 3. With this configuration, a large amount of video material information D1i can be efficiently received on the receiving side.

The storage means 3 is connected to a selection means 4 for selecting arbitrary video material information D1i from the received video material information D1i. Video material information D
The selection of 1i is made by the viewer. Therefore, operation means 5 such as a push button switch and a jog dial are connected to the selection means 4, so that the video material information D1i according to the viewer's preference can be selected. In this example, the background of the video or the motion of the object related to the video material information D1i is changed based on the operation information D2 obtained by operating the operation unit 5.

In this example, the video material information D1i is prepared on the transmitting side, and with respect to the video material information D1i, a coarse reproduction mode for reproducing three-dimensional video constituting one screen in a “coarse” or “dense” mode. , The receiving side may select any of the dense reproduction modes for reproducing the data. When the coarse playback mode is selected in the case where the above-mentioned pay-as-you-go system is adopted, the amount of the video material information D1i within the reception time is smaller than that in the dense playback mode, so that the fee is low for the viewer.

An image creating means 6 is connected to the selecting means 4, and creates a three-dimensional image based on the image material information D1i selected by the viewer. For example, the image creating means 6 creates a stereo image as a basis of a three-dimensional image based on the image material information D1i.
The virtual space is represented by software for creating two-dimensional video information. At this time, the format is VRML (Virtual Reality Modeling).
(Language) method.

A display means (not shown) such as a liquid crystal display, a CRT monitor, or a synthesizing means (hereinafter also referred to as glasstron) is connected to the image creating means 6, and a three-dimensional image is displayed. The glasstron may be provided with a mechanism for detecting a viewer's eye movement. In this case, the motion of the object in the video is changed based on information (hereinafter referred to as viewpoint position detection information) D3 that has detected the viewpoint position of the viewer. For example, the motion of the object in the video is changed based on a plurality of pieces of motion information D15 of the object transmitted to the receiving side during a period in which the background of one screen of the video is displayed and the viewpoint position detection information D3 by the glasstron. Such display control is performed, or display control is performed to change the motion of an object in the video based on the operation information D15 and the operation information D2 from the operation unit 5.

Further, the change of the background or the motion of the object related to the image material information D1i is also performed when an interrupt occurs within a certain cycle time. Except for the interruption within this cycle time, an image based on the default three-dimensional image information is displayed. Thus, a three-dimensional version of the interactive television can be constructed.

Next, a description will be given of a processing example of the transmission / reception system 100 in the method of transmitting / receiving a three-dimensional video according to the present embodiment. In this example, the video material information D1i
It is assumed that all are prepared on the transmission side. of course,
A part of the video material information D1i may be stored on a storage medium such as the CD-ROM 79 and prepared on the receiving side.

First, in step A1 of the flowchart shown in FIG. 2, a plurality of pieces of video material information D1i for creating a three-dimensional video are prepared in advance on the transmitting side such as a broadcasting station. The video material information D1i includes background information D11 representing the background of a video such as a stage or a sports stadium, and object information D representing an object such as an actor or athlete moving in the background of the video.
12, the initial arrangement information D13 of the object with respect to the background, the position information D14 associated with the movement of the object, and the operation information D15 associated with the operation of the object.

Thereafter, the video material information D1i prepared on the transmitting side is multiplexed by the transmitting means 1 in step A2, and then transmitted from the transmitting side to each viewer. At this time, the video material information D1 is transmitted using a digital TV broadcast network, a cable TV communication line, or an Internet communication line.
i is transmitted to the viewer side. In this example, video material information D
1i is transmitted to the receiving side as a data stream having a data sequence of MPEG-4 which is a moving image compression standard.

Then, the video material information D1i sent from the transmitting side is received by the viewer side in step A3 using, for example, a 3D polygon receiving device.
This 3D polygon receiving apparatus will be described in an embodiment. On the viewer side, arbitrary video material information D1i is selected from the video material information D1i via the selection means 4 in step A4. At this time, the selection of the video material information D1i is performed by the viewer using the operation unit 5. Based on the video material information D1i selected by the operation unit 5, the video generation unit 6 generates a three-dimensional video.

Therefore, a viewer-specific three-dimensional video can be created based on the video material information D1i selected according to the viewer's preference from the plurality of video material information D1i prepared in advance on the transmission side. As a result, the viewer can not only passively watch the video provided by the broadcasting station as it is, but also view the three-dimensional video that has been actively and uniquely stereoscopically processed. This allows
Players who play at the soccer field while staying at home,
It is possible to view a three-dimensional image full of a sense of reality recognized by the eyes of actors performing on the stage. Also, ISD
A next-generation three-dimensional digital broadcasting system using the B service can be constructed.

(2) First Embodiment FIG. 3 shows a 3D polygon broadcasting system 200 according to each embodiment.
FIG. 3 is a block diagram illustrating a configuration example of FIG. In this example, a plurality of video material information D transmitted from a broadcast station via a satellite line
3i, a 3D polygon receiving device 202 for creating a three-dimensional video based on the video material information D1i. It can be created.

In the first embodiment, the special glasstron 20 is used.
In a viewer wearing 3 the background recognized by the line of sight of actors performing on the stage is replaced with a "forest" or "lawn" composed of 3D polygons, and a 3D image of the actor is displayed on the "lawn" It is made possible.

The 3D polygon broadcasting system 20 shown in FIG.
Reference numeral 0 includes a 3D polygon transmitting device 201, a 3D polygon receiving device 202, a special glasstron 203, and a monitor 204. The 3D polygon transmitting device 201 is a transmitting unit 1
Is installed in a broadcasting station or the like.

In this broadcasting system 200, for example, the background of a castle related to the stage broadcast of "Romeo and Juliet" prepared in advance to create a three-dimensional image, male and female actors as main characters, movement of the actors, etc. Video material information D
1i is transmitted by the 3D polygon transmitting apparatus 201 from the broadcasting station side to the 3D polygon receiving apparatus 202 possessed by the viewer via a satellite line. On the receiving side, the video material information D1i sent from the broadcast station is received by the 3D polygon receiving device 202. 3D polygon receiver 20
When the viewer selects any of the video material information D1i from the video material information D1i received by the user 2, a three-dimensional video that matches the viewer's preference is created based on the selected video material information.

In the 3D polygon receiving apparatus 202, display means such as a special glasstron 203 and a television monitor 204 are used to display a three-dimensional image. In this example, the television monitor 204 displays a two-dimensional image of the stage viewed from the audience seats, and a viewer wearing the special glasstron 203 can display a “forest” or a “lawn” composed of the 3D polygon 10. Can be displayed such that an actor exists on the screen.

The 3D polygon transmission device 201 shown in FIG. 4 is provided with an audio / video acquisition device 211, which outputs audio and video material information D1i from a program relay target such as a stage or a soccer field.
Is obtained. As the audio / video acquisition device 211, a video camera, a stereo camera, a microphone, a range finder, and the like are used. An example in which the video material information D1i is obtained by the video camera will be described with reference to FIGS.

FIGS. 40 to 45 show an example in which this type of video material information D1i is acquired by a range finder.
Will be described. Further, with respect to the video material information D1i, a plurality of light sources having different blinking patterns are attached to a specific surface of an arbitrary imageable object, and the light sources having different blinking patterns are imaged so as to flow in a predetermined imaging direction. The position information D14 of the object may be obtained by obtaining the position information of each of the light sources by performing image processing on the luminance information of the imaged light source.

For example, a plurality of light sources having different blinking patterns are attached to ornaments such as costumes, shoes, gloves, earrings, necklaces, and crowns of actors who play Romeo and Juliet, and light sources having different blinking patterns are set in predetermined imaging directions. The position information D14 of Romeo and Juliet is obtained by obtaining the position information of each of the light sources by performing image processing on the luminance information of the light source taken there. This application example will be described in a second embodiment.

Regarding this video material information D1i, in order to reduce the data transmitted from the broadcast station side to the viewer side,
What is recorded on a storage medium such as the CD-ROM 79 may be prepared in advance on the receiving side. For example, a 3D image information relating to a soccer goal or a field is stored on a CD-ROM.
79 is read. This example will be described in a second embodiment.

A computer 212 is connected to the audio / video acquisition device 211, and computer graphic data (hereinafter referred to as CG data) such as "forest", "sea", "lawn" and "sandy beach" formed by the computer 212. ) D4 may be supplied. Further, the audio / video acquisition device 211 may handle data such as a 3D TV game and a 3D movie.

The audio / video acquisition device 211 is connected with an audio encoding unit 213, a video encoding unit 214, and a data encoding unit 215 for encoding various information. The audio encoding unit 213 encodes audio such as actors and sound effects, and generates an encoded audio signal. Video encoding unit 214
Then, one program is composed, for example, the background of the stage device 40,
Image material information D1i such as background information D11, object information D12, arrangement information D13, position information D14, and motion information D15 relating to the video of the object is encoded, and a video encoded signal is generated. The data encoding unit 215 generates the CG data D4
Are encoded, and a CG data encoded signal is generated.

Multiplexing means 1A is connected to the outputs of the audio encoding unit 213, the video encoding unit 214, and the data encoding unit 215, and outputs audio encoded signals, video encoded signals, CG data encoded signals, and the like. Are multiplexed, and a plurality of transport stream packets TS
j (j = 1 to m) is generated. In this example, an encoded signal (video material information D1i) constituting one program is MPEG
-4 is encoded so as to become a data stream bit stream. The encoding method according to MPEG-4 will be described with reference to FIG. Of course, in addition to MPEG-4, MPEG
-2 may be multiplexed in the payload portion of a 188-byte transport stream (TS) packet specified.

A transmission path encoding unit 216 is connected to the output of the multiplexing means 1A, and the transport stream TS
j, TMCC (Transmission & Multiplexing)
Configuration Coding) generation, error correction and modulation are performed. For example, when predetermined TMCC information serving as control information for a receiver is input to the transmission path coding unit 216, a TMCC signal is generated based on the TMCC information. The TMCC signal is coded on the one hand to become a TMCC transmission code.

On the other hand, the transport stream TSj is synthesized in a predetermined order based on the TMCC signal, and an outer code error correction code such as a Reed-Solomon code is added. Then, after a transmission frame is formed based on the TMCC signal, energy is spread. This energy diffusion is a process for preventing the “0” or “1” signal from continuing and the spectrum after modulation from being concentrated. The energy-spread coded signal is interleaved. This process is for dispersing the burst-like errors generated in the transmission path to enhance the error correction capability. Thereafter, an inner code error correction code is added to the interleaved coded signal based on the TMCC transmission code.

Further, a transmitter 217 is connected to the output of the transmission path coding unit 216, and the coded signal to which the inner code error correction code is added is digitally modulated by the burst signal. As this modulation method, a high-efficiency TC8PSK (Trellis-coded 8-phase modulation: coding rate 2/3) method is used, and a 34.5 MHz bandwidth is used. QPSK (quadrature phase modulation: punctured convolutional coding rate 1/2, 2/3, 3/4, 5/6, 7/8) or BPSK (binary phase modulation: 1 / 2) may also be used for time division with TC8PSK. The TC8PSK-modulated digital signal is transmitted to a viewer's receiving device via a satellite line.

Here, the video material information D1i is stored in the MPEG-
4 will be described. The video encoder 300 shown in FIG. 5 has an input terminal 313 and an output terminal 314. The input terminal 313 is connected to the motion compensator 301, the shape encoder 302, and the subtractor 303.

In the motion compensating unit 301, the video material information D
In order to compensate for a motion image between frames of a video based on 1i, the redundancy in the time direction is reduced. That is, the motion compensation unit 301 generates a prediction signal for predicting the motion of the video between frames. The prediction control signal S10 is output to the shape encoding unit 302 and a bidirectional prediction memory P3 described later.

In the shape encoding section 302, the video material information D
After encoding the motion shape of the video based on 1i based on the prediction control signal S10, a motion shape signal S2 is generated. The subtracter 303 calculates the difference between the current video based on the video material information D1i and the backward prediction (past reference) image, forward prediction (future reference) image, or bidirectional prediction image. Data D21, D22, and D23 related to the backward prediction image, the forward prediction image, or the bidirectional prediction image are subtracted from the image.

A discrete cosine transform unit (hereinafter referred to as DCT) 304 is connected to the output of the subtracter 303. In order to reduce the spatial redundancy of the video, for example, an input image based on the video material information D1i is 8 It is divided into blocks of × 8 pixels. The DCT operation is performed for each block. In the DCT 304, a D composed of a DC component and an AC component
The CT coefficient is obtained.

A quantizer (Q) 305 is connected to the output of the DCT 304, and a DC component and an AC component
The T coefficient is quantized. A motion text encoding unit 306 is connected to the quantization unit 305, and a motion text signal S11 is generated after the quantized DCT coefficients are encoded as motion text. An inverse quantization unit 306 is connected to the quantization unit 305 so that the quantized DCT coefficient
It is inversely quantized into a C component and an AC component. Inverse quantization unit 30
6 is connected to an inverse discrete cosine transform unit (hereinafter referred to as IDCT) 307 to perform an inverse DCT operation.

An adder 308 is connected to the IDCT 307, and each data D21, D22, D23 and I relating to a backward prediction image, a forward prediction image or a bidirectional prediction image.
The combined image data D20 after adding the data D24 related to the output image of the DCT 307 is output. Adder 30
The frame memory 309 is connected to the output of No. 8 and the combined image data D20 by the adder 308 is temporarily stored.
The output of the frame memory is connected to the backward prediction memory P1, and holds data D21 related to the backward prediction image. A forward prediction memory P2 is connected to the frame memory, and holds data D22 related to a forward prediction image.
Further, a bidirectional prediction memory P3 is connected to the frame memory, and data D23 related to the bidirectional prediction image is held based on the prediction control signal S10.

A switch (selector) unit 310 is connected to the outputs of the backward prediction memory P1, the forward prediction memory P2, and the bidirectional prediction memory P3. Based on the switching, any one of the data D21, D22, and D23 relating to the backward prediction image, the forward prediction image, or the bidirectional prediction image is selected. The output of the switch unit 310 is the subtractor 3
03 and an adder 308.

On the other hand, a video multiplexing unit 312 is connected to the motion text coding unit 311 and the shape coding unit 302, and multiplexes the motion text signal S1 of the coded video and the coded motion shape signal S2. Be transformed into Thus, the video material information D1i can be converted into an MPEG-4 bit stream. The bit stream of the video material information D1i is obtained from the output terminal 314.

Next, an example of obtaining the video material information D1i will be described. In the example of the stage device 40 shown in FIG. 6, a video camera 401 for background photographing is arranged at the audience seat, for example, the background of the castle of the first act set on the stage is photographed, and the actor A shown in FIG. And the background information D11 expressing the castle including B.

Also, video cameras 402 and 403 for photographing objects are arranged on the left and right sides of the stage, and the movements of the actors viewed from the left and right of the stage device 40 are photographed. Actors A and B moving on the stage shown in FIG. Is obtained. Further, a video camera 404 for photographing an object position is disposed diagonally to the front right of the stage device 40. The initial arrangement information D13 of the actors with respect to the stage background and the position information D14 associated with the movement of the actors A and B moving on the stage. Is obtained.

In this example, a small CCD image pickup device 40 is mounted on the head of each of two actors (subjects) A and another actor B.
5 and 406 are attached, viewpoint images of one actor looking at the other actor are respectively captured, and operation information D15 accompanying the operation of both actors is acquired. Needless to say, the video camera 407 may be provided on the stage side, and the background or the like viewed from above the audience seat from the stage side may be photographed. As another example, the CCD is moved to a position where only the movements of the heads of both actors A and B are acquired.
A device (or a magnetic sensor) may be attached to acquire an image only in the viewpoint direction.

When the video material information D1i is prepared in advance at a broadcasting station or the like, the above-described 3D polygon transmitting device 2
After being multiplexed by 01, they are transmitted from the broadcasting station to the individual viewers. At this time, digital TV broadcasting network,
The video material information D1i is transmitted to the viewer using a cable television communication line or an Internet communication line.

Next, an example of the configuration of the 3D polygon receiving device 202 will be described. In this example, the video material information D1i related to the above-mentioned stage video is allocated to six channels and transmitted. The background information D11 is allocated to channel ch1, and the object information D12 is allocated to channel c.
h2, and the arrangement information D13 is assigned to the channel ch.
3 and its position information D14 is assigned to channel ch4.
It is assumed that the operation information D15 is allocated to the channel ch5, and the two-dimensional video information is allocated to the channel ch6.

The 3D polygon receiving device 202 shown in FIG. 8 receives a digital signal of the TC8PSK modulation system, and includes a receiving means 2, an image memory (storage means) 3, a selector (selection means) 4, an image creation means 6, A control device 88 and a jog dial 89 are provided.

The receiving means 2 has a tuner 74, a video detector 75, a noise remover 76, an error corrector 77, and an information multiplex analyzer 78. The tuner section 74 receives a digital signal of the TC8PSK modulation system transmitted from a satellite line. The tuner 74 is connected to a video detector 75, for example, a TC8PS multiplexed into six channels.
A K-modulation digital signal is detected. Video detector 7
5 is connected to a noise removing unit 76, and the digital signal transmitted in the MPEG-4 data sequence is transmitted to the OFDM (ODM).
rthogonal Frequency Division Multiplex) demodulated.

The noise removing unit 76 includes an error correcting unit 77
Are connected, and the error code of the OFDM-demodulated digital signal is corrected. An information multiplexing analysis unit 78 is connected to the error correction unit 77 as a demultiplexing unit.
The video material information D1i of the channel is demultiplexed. The background information D11 is obtained from ch1, and the object information D12 is
2, the location information D13 is obtained from ch3, the position information D14 is obtained from ch4, and the operation information D15 is c
h5, and two-dimensional video information is obtained from ch5.

The image multiplexing analysis unit 78 is connected to the image memory 3, and the background information D obtained from ch1 to ch6.
11, object information D12, arrangement information D13, position information D1
4. The motion information D15 and video material information D1i such as two-dimensional video information are temporarily stored. A selector 4 is connected to the image memory 3, and arbitrary video material information D1i is selected from the video material information D1i by a viewer using a jog dial 89 or the like as operation means.

The output of the selector 4 is supplied to the video creating means 6.
Are connected to create a three-dimensional image. Image creation means 6
Has, for example, a geometry unit 85, a raster unit 86, and a stereo image generation unit 87. That is, the geometry unit 85 is connected to the output of the selector 4, and the background information D11, the object information D12, the arrangement information D13, the position information D14, and the operation information D15 are displayed (described) in a global coordinate system for expressing a three-dimensional image. ) Is done. For example, the background recognized by the line of sight of actor B or the like performing on the stage shown in FIG. 9 is replaced with “forest” or “lawn” composed of 3D polygon 10, and actor A's three-dimensional image is displayed on the “lawn”. The image is displayed. These video composite signals are generated.

The geometry section 85 has a CD-ROM
79 is connected, and video software read from a software information source such as a 3D soccer game prepared in advance on the receiving side may be synthesized. A raster unit 86 is connected to the geometry unit 85, and a scanning method of the display unit 24, for example,
Raster processing is performed to match the horizontal scanning period and the vertical scanning period of an image so as to conform to the NTSC system or the PAL system. The raster section 86 includes a stereo image generating section 8.
7 is connected, and a stereo image is formed based on the video composite signal scanned and processed to match the scanning method of the display means 24. For example, if the display means 24 is a special glasstron 2
In the case of 03, since the viewpoint position at which the viewer looks can be detected, a stereo image is formed based on the viewpoint data.

A display means 24 such as a special glasstron 203, a TV monitor 204, or a liquid crystal display (LCD), which will be described later, is connected to the stereo image generating section 87, and video material information selected according to the viewer's preference. A viewer-specific three-dimensional image based on D1i is displayed. The above-mentioned information multiplexing analysis unit 78, image memory 3, selector 4, CD-R
OM 79, raster unit 86, and stereo image generation unit 87
Is connected to a control device 88, and the control device 8
A jog dial 89 is connected to 8. The viewer selects the video material information D1i according to his / her preference by using the jog dial 89. By this selection, the control unit 88 controls the input / output of the multiplex analysis unit 78, the image memory 3, the selector 4, the CD-ROM 79, the raster unit 86, and the stereo image generation unit 87.

Next, a configuration example of the special glasstron 203 used in the first embodiment will be described. The special glasstron 203 shown in FIG. 10 constitutes a synthesizing unit, and forms, for example, a non-transmissive head mounted display. The special glasstron 203 has a main body 21. A belt 22 is provided on the main body 21. The main body 21 is mounted on the face of the viewer like wearing glasses, and the belt 22 is fixed along the outer periphery of the head of the viewer. The main body 21 is provided with a liquid crystal display device (hereinafter, referred to as LCD) 26 for a right eye display as a first image display element and an LCD 27 for a left eye display as a second image display element.

That is, the LCD 26 attached to a position opposite to the viewer's right eye in the special glasstron 203 displays, for example, an image of the actor A recognized by the line of sight of the actor B performing on the stage shown in FIG. And 3D polygon 10
One of the stereo images synthesized with the image of the actor A existing on the background “lawn” replaced with “forest” or “lawn” composed of

The LCD 27 attached to the viewer's left eye is located on the LCD 27. The image of the actor A recognized by the line of sight of the actor B or the like performing on the stage described above and the 3D
The other of the stereo images combined with the background image replaced with “forest” or “grass” composed of polygons 10 is displayed. The special glasstron 203 is mounted on the face or head of the viewer, and the stereo image of the LCD 26 and the stereo image of the LCD 27 are guided to the viewer's eyeball. Therefore, the image of the actor A recognized by the gaze of the actor B or the like performing on the stage and the 3D polygon 10
Is synthesized in the viewer's head with the image of the actor A present on the background “lawn” replaced with “forest” or “lawn”.

The special glasstron 203 has the structure shown in FIG.
LCD 26 for right eye display and LCD for left eye display shown in FIG.
Film-shaped charge-coupled devices (hereinafter referred to as film CCDs) 4R (right) and 4L (left) are arranged in the display surface 27, and image the eye movements of an observer watching the reference plane to obtain the observer's attention. The viewpoint p is detected. In this example, the film CCDs 4R and 4L are connected to the LCD 2 facing the viewer's eyeball.
6 and the LCD 27.

In the film CCDs 4R and 4L, FIG.
For example, the imaging element 4A is arranged in a toothless manner (shaded portion) with respect to all pixels of a 4 × 6 pixel matrix shown in 2A. Therefore, the image pickup device 4A is not arranged in the white portion, and the images from the LCD 26 and the LCD 27 are passed. For the film CCDs 4R and 4L, a silicon film is partially formed on the film to form a CCD such as a photoelectric conversion element in a toothless shape.
4R and 4L are formed by attaching them in front of the LCD 26 and the LCD 27, for example.

Since the film CCDs 4R and 4L can be partially formed of a silicon film using sapphire as a base material, the LCDs 26 and 27 and the toothless film CCDs 4R and 4L are integrated into one chip. Can also. However, the transparency (light transmittance) of the chip is on the order of thin sunglasses.

The film CCD 4R captures the pupil movement of the observer's left eye as shown in FIG. 12B and the aperture state of the pupil, and the film CCD 4L captures the movement of the pupil of the observer's right eye as shown in FIG. , The aperture state of the pupil is imaged. Although the imaging performance is somewhat lowered, there is no problem in capturing the eye movement of the observer. Therefore, when the viewer wearing the special glasstron (synthesizing means) 203 adjusts the viewpoint to a specific position, the viewer relates to the video material information D1i based on the camera output signals (viewpoint position detection information) obtained from the film CCDs 4R and 4L. The background of the image or the movement of the object can be changed.

This special glasstron 203 is mounted on the face or head of the viewer shown in FIG.
And the stereo image of the LCD 27 are guided to the viewer's eyeball. In FIG. 13, the point at which the optical axes of both eyes of the viewer wearing the special glasstron 203 overlap is the point of gaze p. On the basis of the gazing point p, a stereo image of the image of the actor A and a stereo image of a virtual image of “forest” or “grass” composed of the 3D polygon 10 are synthesized in the head.

Here, the distance from the surface of the viewer's eyeball to the gazing point p is Sh, the distance from the display surface of the LCD 26 or LCD27 to the gazing point p is Sc, and the distance between the viewer's eyeball surface and the LCD 26 or LCD27 is Assuming that the offset distance from the display surface is Soff, Sh = Sc + Soff:
The viewpoint position can be detected. In this example, the positional relationship between the iris and the pupil and the shape of the pupil are recognized from the eye movements of the viewer, and the direction of the gazing point p and the separation distance S are determined based on the recognition information.
h, and background information and the like are switched based on the separation distance Sh.

As described above, the viewer's eyeball image is provided on the front surfaces of the left and right LCDs 26 and 27 on the film CCD 4R,
The present invention is not limited to the case where 4L is provided, and outputs a background image from the left-eye display LCD 27 and outputs a right-eye display LCD.
26, a film CCD 4R eye is provided on
An eyeball image may be obtained by using The reverse method may be used.

Next, an example of processing in the 3D polygon broadcast system according to the first embodiment will be described. In this example, a 3D image is created based on a plurality of pieces of image material information D1i sent from a broadcasting station via a satellite line.
It is assumed that the D-polygon receiving device 202 is installed in each home, and that a viewer-specific three-dimensional video is created based on video material information selected according to the viewer's preference from the plurality of video material information D1i. I do. Of course, the viewer
Is mounted on the head.

Further, in the viewer wearing the special glasstron 203, the background recognized by the line of sight of an actor or the like performing on the stage is replaced with a "forest" or "lawn" composed of 3D polygons 10, and the "lawn" It is assumed that a three-dimensional image of an actor is displayed above.

On the basis of these, a plurality of pieces of video material information D1i for creating a three-dimensional video are prepared in advance in the broadcasting station in step B1 of the flowchart shown in FIG.
For example, the audio-video acquisition device 211 shown in FIG. 4 expresses background information D11 expressing the background of a castle related to the stage relay of “Romeo and Juliet”, and male and female actors A and B moving in the background of the video. Material information D1i based on object information D12, initial arrangement information D13 of actors A and B with respect to the background, position information D14 associated with the movement of actors A and B, and operation information D15 associated with the movements of actors A and B. Is obtained. In addition, computer graphic data (hereinafter, referred to as CG data) such as "forest,""sea,""lawn," and "sand beach" that are processed and formed by the computer 212.
D4 is included in the video material information D1i.

Thereafter, the video material information D1i prepared on the broadcast station side is transmitted to the 3D polygon transmitting device 201 in step B2.
Are multiplexed. In the 3D polygon transmission device 201 shown in FIG. 4, for example, the video material information D1i obtained from the audio / video acquisition device 211 in step B21 is used for the audio encoding unit 213, the video encoding unit 214, and the data encoding unit 2
15 is output. The audio encoding unit 213 encodes audio such as actors and sound effects, and generates an encoded audio signal. The video encoding unit 214 constitutes one program, for example, background information D1 relating to the background of the stage apparatus and the video of the object.
1, object information D12, arrangement information D13, position information D14
And image material information D1i such as motion information D15 are encoded to generate an encoded video signal. Data encoding unit 2
At 15, the CG data D4 is encoded, and a CG data encoded signal is generated.

The multiplexing means 1A connected to the outputs of the audio encoding unit 213, the video encoding unit 214, and the data encoding unit 215 allows the audio encoded signal, the video encoded signal, the CG data encoded signal, and the like. Are multiplexed, and a plurality of transport stream packets TSj (j = 1 to m) are generated. Then, in step B22, the background information D11 relating to the above-mentioned stage video is allocated to the channel ch1, the object information D12 is allocated to the channel ch2, the arrangement information D13 is allocated to the channel ch3, and the position information D14 is allocated to the channel ch4. And the operation information D15 is assigned to the channel c.
h5, and the two-dimensional video information is transferred to channel ch6.
Assigned to. This video information may be one channel.

In this example, the coded signal (video material information D1i) constituting one program is an MPEG signal as shown in FIG.
-4 is encoded so as to become a data stream bit stream. The output of the multiplexing means 1A is output from a transmission path encoding section 216 to synthesize the transport stream TSj,
OFDM processing such as MCC generation, error correction and modulation is performed in step B23.

The coded signal after the OFDM processing is subjected to digital modulation of the TC8PSK system in the transmitter 217, and a step B is performed using a 34.5 MHz bandwidth.
At 24, the signal is transmitted to the viewer's receiving device via a satellite line. At this time, the video material information D1i is transmitted to the viewer using the digital television broadcast network. of course,
A cable television communication line or an Internet communication line may be used.

Then, the video material information D1i sent from the broadcast station side is transmitted to the viewer 3 in the step B3.
It is received by the D polygon receiving device 202. For example, the tuner section of the 3D polygon receiving apparatus 202 shown in FIG. 8 receives a digital signal of the TC8PSK modulation system transmitted from a satellite line.

The video detector 75 connected to the tuner 74 detects a TC8PSK modulation digital signal multiplexed on six channels. This video detector 7
In the noise removing unit 76 connected to 5, the digital signal transmitted in the MPEG-4 data series is subjected to noise removal and OFDM demodulation. This noise removing unit 7
In an error correction unit 77 connected to 6, the error code of the OFDM-demodulated digital signal is corrected.

In the information multiplexing analysis section 78 connected to the error correction section 77, the error-corrected 6-channel video material information D1i is demultiplexed. Background information D11 is ch
1, the object information D12 is obtained from ch2, the arrangement information D13 is obtained from ch3, and the position information D14 is c.
h4, operation information D15 is obtained from ch5,
And two-dimensional video information is obtained from ch5. These background information D11, object information D12, and arrangement information D1
3. Position information D14, motion information D15, and video material information D1i such as two-dimensional video information are temporarily stored in the image memory 3.

Then, in step B4, the video material information D
Any video material information D1i from 1i is jog dial 8
9 is selected by the viewer. The video material information D1i selected by the jog dial 89 through the control device 88 and the selector 4 is stored in the video creation unit 6
Is output to In the geometry section 85 of the image creating means 6, background information D11, object information D12, arrangement information D13, position information D14, and motion information D15 are displayed (described) in a global coordinate system for expressing a three-dimensional image.
For example, the background recognized by the line of sight of actor B or the like performing on the stage shown in FIG. 6 is replaced with “forest” or “grass” composed of 3D polygon 10, and actor A 3 is displayed on the “grass”. A two-dimensional image is synthesized, and these image synthesis signals are created.

The raster section 86 connected to the geometry section 85 is subjected to raster processing so as to match the scanning method of the display means 24. After that, the stereo image generation unit 87 connected to the raster unit 86 forms a stereo image based on the viewpoint data.

This stereo image is supplied to the special glasstron 203 in step B5. Special Glasstron 20
The LCD 26 in 3 has an image of the actor A recognized by the line of sight of the actor B and the like performing on the stage shown in FIG. 9 and the background of the 3D polygon 10 replaced with “forest” or “lawn”. One of the stereo images obtained by synthesizing the image of the actor A existing on the "grass" is displayed.

The LCD 27 displays an image of the actor A recognized by the line of sight of the actor B or the like performing on the above-mentioned stage and a background of the forest replaced by “forest” or “grass” composed of the 3D polygon 10. The other of the stereo images synthesized with the video is displayed. The stereo image of the LCD 26 and the stereo image of the LCD 27 are guided to the viewer's eyeball. Thereby, the image of actor A recognized by the line of sight of actor B or the like performing on the stage, and actor A existing on “grass” in the background replaced with “forest” or “grass” composed of 3D polygon 10 Is synthesized in the viewer's head.

As described above, according to the polygon broadcasting system in 3 according to the first embodiment, the video material selected according to the viewer's preference from a plurality of video material information D1i prepared in advance by the broadcasting station side. Viewer-specific 3 based on information D1i
You can create 3D images. Special Glasstron 2
The viewer wearing 03 can enjoy a three-dimensional image in which the actor A exists on the “forest” or “grass” composed of the 3D polygon 10 shown in FIG.

Therefore, the viewer does not simply passively watch the video provided from the broadcast station side, but can view the three-dimensional video that is actively and independently stereoscopically processed. it can. Thus, it is possible to view a three-dimensional video with a sense of reality recognized by the eyes of actors A and B performing on the stage while staying at home.
Further, a next-generation three-dimensional digital broadcasting system using an ISDB service can be provided.

In the case where the special glasstron 203 is not attached to the 3D polygon receiving apparatus 202, a two-dimensional image when the stage is viewed from the audience seat is displayed on the television monitor 204.
Will be displayed. In addition, by switching the jog dial 89, a three-dimensional composite image in which the actor A exists on the "forest" or "grass" constituted by the 3D polygon 10 can be displayed on the television monitor 204.

(3) Second Embodiment FIG. 15 is a block diagram showing a configuration example of a 3D polygon broadcast system 400 according to a second embodiment. In this example, the game device 205 is connected to the 3D polygon broadcasting system 300 according to the first embodiment, and is based on, for example, a plurality of pieces of video material information D1i transmitted from a broadcasting station via a satellite line. 3D polygon receiver 20
2, a three-dimensional video is created, and a viewer-specific three-dimensional video can be created from the plurality of pieces of video material information D1i based on video material information selected according to the viewer's preference. Along with this, Special Glasstron 2
A viewer wearing 06 is capable of displaying a three-dimensional image of a soccer player composed of 3D polygons 10 on a soccer court formed by video software provided from the game apparatus 205.

The 3D polygon broadcasting system 4 shown in FIG.
00 includes a 3D polygon transmitting device 201, a 3D polygon receiving device 202, a TV monitor 204, a game device 205, and a special glasstron 206. The components having the same names and the same reference numerals as those in the first embodiment have the same functions, and the description thereof will be omitted.

In the broadcasting system 400, for example, the background of a soccer court related to satellite relay of “World Cup Soccer”, a soccer player as a player, the movement of the soccer player, etc., prepared in advance to create a three-dimensional image. Is obtained.

The video material information D1i is provided with a video camera for photographing the background in the audience seats. For example, the background of the entire soccer court is photographed, and the background information D11 representing the soccer court including judges and soccer players. Is obtained. Also, video cameras for object shooting are arranged on the left and right sides of the soccer court, the movement of the soccer player looking at the soccer court from the left and right is shot, and object information D12 representing the soccer player moving following the ball on the soccer court is displayed. Is obtained. Further, a video camera for photographing an object position is arranged around the soccer court, and initial arrangement information D13 of the soccer player with respect to the entire background of the soccer court and position information D14 accompanying the movement of the soccer player moving in the soccer court are displayed. Is obtained.

In this example, a specific soccer player (subject)
A small CCD imaging device may be attached to the head of the soccer player to capture a viewpoint video of the goal of the soccer player, or to acquire motion information D15 associated with the motion of another soccer player. Of course, the CCD imaging device may be attached to all players. Further, a video camera may be provided on the goal side, and a background or the like as viewed on the field from the goal side may be photographed.

When the video material information D1i is prepared in advance at a broadcasting station or the like, the above-described 3D polygon transmitting device 2
After being multiplexed by 01, it is transmitted to the 3D polygon receiver 202 possessed by the viewer via a satellite line. At this time, the video material information D1i is transmitted to the viewer using a digital television broadcast network, a cable television communication line, or an Internet communication line.

On the receiving side, the video material information D1i sent from the broadcasting station is received by the 3D polygon receiving device 202. From the video material information D1i received by the 3D polygon receiving device 202, arbitrary video material information D1
When i is selected by the viewer, a three-dimensional video that matches the viewer's preference is created based on the selected video material information.

In the 3D polygon receiving apparatus 202, display means such as a special glasstron 206 and a television monitor 204 are used, and a game apparatus 205 is used.
In this example, the television monitor 204 displays a two-dimensional image of the soccer court viewed from the audience seats.
Is synthesized and displayed on the soccer court of the game apparatus 205, for example, on a soccer court using video software provided from the CD-ROM 79. The data amount of the video material information transmitted from the broadcasting station to the viewer can be reduced.

The game device 205 is an example of an image processing means, and is connected to a 3D polygon receiving device 202, a special glasstron 206, a TV monitor 204, and the like. In the game device 205, XXXXXXXXXXXX (XXXXXX
×: trade name). For example, a CD-ROM 79 in which video software such as a soccer game is recorded is mounted on the game device 205. In the game device 205, when the game software is read from the CD-ROM 79,
A soccer court is displayed on the TV monitor 204 based on the game software, and a character such as a soccer player is displayed on the court to play so as to compete in a soccer game.

In this example, at the four corners of the game apparatus 205 shown in FIG. 15, light emitting diodes (LED1 to LED4) are attached as a plurality of light sources, respectively. Is set. This reference plane has coordinates (x1, y) of four points P1 to P4.
1), (x2, y2), (x3, y3), (x4, y
4) (corresponding to a reference plane on which an image is to be synthesized in the virtual space).

The four light emitting diodes LED1 to LE
D4 has a function as a mark part.
That is, at least the blinking is performed so that the blinking pattern is different so that the mounting position is clear.

The light emitted from the light emitting diodes LED1 to LED4 is picked up by a follow-up CCD device attached to the special glasstron 206 so as to flow in a predetermined imaging direction. This panning is performed with four light emitting diodes LED1 to LED1.
This is for specifying the reference plane from the mounting position of the LED 4. The specification of the reference plane will be described with reference to FIG.

The game apparatus 205 is provided with a flicker control circuit 13 shown in FIG. 16 as control means, and controls the input and output of the light emitting diodes. For example, the blink control circuit 13
In order to make the image processing system recognize the position of the game device 205 with good reproducibility, the blink control is performed so that the blink patterns of the light emitting diodes are different.

In this example, the flicker control circuit 13
A predetermined voltage is applied to the four light emitting diodes LED1 to LED4 to control the blinking. This blinking control circuit 13 has, for example, a clock generator 61. The clock generator 61 includes a 1/2 frequency divider 62, a 1/3 frequency divider 63,
A 1/4 frequency dividing circuit 64 is connected, and a clock signal CLK1 of a predetermined frequency and this clock signal CLK1 are
The clock signal CLK that has been frequency-divided by the frequency-dividing circuit 62
2, a clock signal CLK3 frequency-divided by 1/3 in the 1/3 frequency dividing circuit 63, and a clock signal CLK4 frequency-divided by the 1/4 frequency dividing circuit 64.

Each of the clock signals CLK1 to CLK4 is supplied to each of the light emitting diodes LED through a stabilizing resistor R.
1, LED2, LED3 and LED4. The DC power supply VCC is connected to the clock generator 61.

In this example, when power is supplied to the clock generator 61 shown in FIG.
Is supplied with the clock signal CLK1, the light emitting diode LED2 is supplied with the clock signal CLK2, the light emitting diode LED3 is supplied with the clock signal CLK3, and the light emitting diode LED4 is supplied with the clock signal CLK4.
Is supplied. Therefore, the blinking patterns of the four light emitting diodes LED1, LED2, LED3, and LED4 shown in FIG. 17 can be controlled to be different.

As described above, the four light emitting diodes LED1 to LED4 of the game device 205 with the reference plane setting are
Since the blinking control is performed by the blinking control circuit 13 so as to make the blinking pattern different, when the game device 205 is panned and shot by a special photographing device such as a panning CCD device, the four light emitting diodes LED1 to LED4 are turned on. The bright spot positions of the four light emitting diodes LED1 to LED4 can be easily specified as compared with the case where the pattern is not flashed. Accordingly, the position of the game device 205 can be easily recognized by the image processing system from the positions of the four light emitting diodes LED1 to LED4.

Next, the special glasstron 206 will be described. The special glasstron 206 shown in FIG. 18 is provided with at least a panning CCD device 23 and a display unit 24. Depending on the model of the special glasstron 206, a normal CCD imaging device 25 is provided. The game device 205, the panning CCD device 23, and the image processing unit in the special glasstron 206 constitute a surface recognizing means so that an arbitrary reference plane can be recognized in the real space to which the viewer belongs. Here, the panning shot refers to a shooting mode in which signal charges are read out from a photoelectric conversion element (such as a photodiode) a plurality of times during the same field period in the panning CCD device 23.

In this example, when an interline transfer type two-dimensional imaging device having a vertical transfer unit is used as the panning CCD device 23, signals are sent from the photoelectric conversion element to the vertical transfer unit a plurality of times during the same field period. The charge is read. When a two-dimensional frame transfer type imaging device having a charge storage unit is used as the panning CCD device 23, signal charges are read from the photoelectric conversion element to the charge storage unit a plurality of times during the same field period.

Further, an image processing unit is provided in the special glasstron 206, and image processing for recognizing a reference plane or the like is performed based on image data output from the panning CCD device 23. Display means 24 is connected to this image processing unit, and the reference plane recognized by the plane recognition means is displayed. An optical unit such as a polarizing beam splitter may be provided in the special glasstron 206, and an image of a virtual body is synthesized on a reference plane in a virtual space displayed by the display unit 24. In this example, a soccer court by the video software provided from the game device 205 is displayed at a position to which the reference plane in the real space belongs, and a three-dimensional image of the soccer player constituted by the 3D polygon 10 is displayed on the soccer court. Is displayed.

That is, a normal CCD image pickup device 25 and a panning CCD device 23 are arranged side by side at a position corresponding to the viewer's eyebrows, and an external image to which the viewer belongs is imaged by the former, and the game is imaged by the latter. Device 205-4
The light-emitting diodes LED1 to LED4 are panned and shot. Therefore, the viewer can set the game device 20 with the reference plane setting.
When we turn our eyes to 5, we take a panning CCD in the direction of the reference plane.
The device 23 is turned.

Next, another special glasstron 207 will be described. The special glasstron 207 shown in FIG. 19 constitutes a transmission type head mounted display,
No ordinary CCD imaging device 25 is mounted. Therefore, the transmissive head-mounted display comprises a panning CCD device 23 and a liquid crystal shutter 2 for capturing an external image.
8 and an LCD 29 as an image display element.

For example, a panning CCD device 23 is arranged at a position corresponding to the viewer's eyebrows, and when the viewer turns his / her eyes to the game device 205, the game device 205
The four light emitting diodes LED1 to LED4 are panned. A liquid crystal shutter 28 is provided at a position corresponding to the left and right eyes of the viewer. For example, when the liquid crystal shutter 28 is opened, a real image of the game device 205 of the viewer passing through the liquid crystal shutter 28 is directly put on the eyeball. Be guided.

An LCD 29 is attached to a portion of the special glasstron 207 located beside the viewer's left or right eye, and displays a character image in the same manner as the special glasstron 206 described above. Although not shown, optical means such as a polarizing beam splitter is provided between the liquid crystal shutter 28 and the LCD 29, and an image of a soccer court by video software provided from the game device 205,
On this soccer court, a three-dimensional image of a soccer player constituted by the 3D polygon 10 is guided to the viewer's eyeball. Thereby, the virtual image on the game device 205 to which the viewer belongs and the three-dimensional video of the soccer player using the 3D polygon 10 are synthesized in the head.

Next, an example of the internal configuration of the panning CCD device 23 will be described. The panning CCD device 23 shown in FIG. On the substrate 31,
Photodiodes PHij (i = 1 to n, j = 1 to m) are arranged in a matrix of n columns × m rows as photoelectric conversion elements constituting one pixel.

In the column direction of this substrate, m
Vertical transfer units 32 are provided, and the photodiodes PH
The signal charge read from ij is transferred in the vertical direction (follow-up direction) based on the vertical read signal S1. A horizontal transfer unit 33 is connected to the vertical transfer unit 32, and the signal charges are transferred in the horizontal direction based on the horizontal readout signal S2, so that a follow shot signal SOUT is output to the output terminal. In this example, the signal charge is transferred from the photodiode PHij to the vertical transfer unit 32 at least a plurality of times during the same field period in order to perform a panning shot.

The panning CCD device 23 has a fisheye lens 35 shown in FIG. The fisheye lens 35 is provided on the optical axis of the CCD image sensor 36, for example. With the fisheye lens 35, the above-described game device 205 and the like can be imaged in a wide range. Of course, a normal lens may be used, but since the field of view is narrowed, the viewer must tilt his head more toward the game device 205.

Subsequently, the non-transmission type special glasstron 20 is used.
6 will be described. The special glasstron 206 shown in FIG. 22 includes the above-described panning CCD device 23, a normal CCD imaging device 25, an LCD 26 for right-eye display, an LCD 27 for left-eye display, and the image processing device 9. This image processing device 9 is used for the image creating means 6 shown in FIG.
, The image processing device 9 may be omitted from the special glasstron 206.

In this example, the image processing device 9 has an internal bus 41. The internal bus 41 has an internal interface (I
/ O) 42, image capture unit 43, image processing unit 44,
CPU 45, ROM 46, RAM 47, E ² PROM
(Electrically writable and erasable read-only memory) 48 and an external interface 49 are connected. Panning CCD device 23, normal CCD imaging device 2
5. LCD 26 for right eye display and LCD 2 for left eye display
7 is connected to the internal bus 41 via the internal interface 42.

This internal bus 41 has E ² as a recording medium.
The PROM 48 is connected, and an algorithm for setting a reference plane on the game device 205 is stored. Therefore, by executing this algorithm, the game device 20 in the real space can be simply and with a small amount of calculation.
5 can be recognized by the CPU 45 or the like. Thus, a 3D polygon broadcasting system in which a realistic three-dimensional image recognized by the eyes of a soccer player or the like playing on a soccer court appears on the reference plane on the game device 205 can be configured with high reproducibility. .

In this example, an external interface 49 is connected to the internal bus 41. The external interface 49 is provided with an external input terminal 410,
By directly connecting the polygon receiving device 202, the RS-232
Video signals and digital broadcast signals according to the C communication protocol can be input. In addition to the external input terminal 410, an external input terminal 411 that can be connected to the game device 205 is provided, and a stereo image of the 3D polygon 10 relating to the soccer game played on the game device 205 can be created by the 3D polygon receiving device 202. It is supplied via means 6.

Further, a ROM 46 is connected to the internal bus 41, and stores a system program for controlling the special glasstron 206, control information such as a memory reading procedure, and the like. A working RAM 47 is connected to the internal bus 41, and a system program and display information for displaying an image of the 3D polygon 10 are temporarily recorded. A CPU 45 is connected to the internal bus 41, and an internal interface 42, an image capture unit 43, an image processing unit 44, a ROM 46, a RAM 47, and an E ² PROM 4
8 input / output control, panning CCD device 23, CCD
Input / output control of the imaging device 25, the LCD 26, and the LCD 27 is performed.

An image processing section 44 is connected to the internal interface 42 as arithmetic means. For example, the game apparatus 20 is operated by a normal CCD image pickup apparatus 25 shown in FIG.
5 are acquired and the four light emitting diodes LED1 to LED4 for reference plane recognition are imaged by the follow-up CCD device 23, and the image data on the blinking pattern obtained from the CCD device 23 is controlled by the CPU 45. The image capture unit 4 via the internal interface 42 together with the instruction
3 In the image capture unit 43, the four light emitting diodes L input from the panning CCD device 23
A predetermined capture process for acquiring image data related to ED1 to LED4 is performed. The image data of this blinking pattern is expressed as a change in luminance corresponding to the passage of time.

The image capture unit 43 has an internal bus 4
The image processing unit 44 is connected to the game device 205 through which the blinking pattern synchronization is corrected or the viewer gazes with respect to the image data on which the predetermined image processing has been performed.
Is required. Thereby, the light emitting diodes LED1 to LED4 imaged so as to flow in a predetermined imaging direction
With respect to the blink pattern, the luminance signal SOUT relating to the blink pattern is subjected to image processing by the image processing unit 44, and the positions of the light emitting diodes LED1 to LED4 are obtained. Therefore, the position of the game device 205 can be easily specified. .

For example, in the image processing section 44, the panning CC
Panning signal (luminance signal) S output from D device 23
The blinking pattern of OUT is converted into a spatial arrangement pattern that forms an XY plane including four pan shot bright points P1 to P4 within the image area defined by the window W shown in FIG. Then, by scanning the arrangement pattern, at least the position coordinates (X
1, Y1), (X2, Y2), (X3, Y3), (X
4, Y4) are required. The four bright points P1 to P4 are four light emitting diodes LED1 to LED4 for setting a reference plane on the game device 205 that the viewer gazes at. Four light emitting diodes LED of the game device 205 in the real space
The position coordinates of 1 to LED4 are known, and the position coordinates are (x1, y1), (x2, y2), (x3, y3),
(X4, y4).

Therefore, the game device 20 in the real space described above
The acquired image of 5 is four light emitting diodes LED1 to LED
4 is obtained by calculating a transformation matrix projected onto the mounting position of No. 4. Here, a point (xi, yi,
0) is moved by a certain translation / rotational motion, and a point obtained by projecting the point on the image coordinate system by the perspective transformation is represented by (Xi, Yi).

[0135]

(Equation 3)

Where a1... A8 are unknown coefficients and C1
External parameters (position and direction) such as CD imaging device 25
And internal parameters such as focal length. These parameters are the position coordinates (x1, y) of a known point in the real space.
1), (x2, y2), (x3, y3), (x4, y
4) and four sets of position coordinates (X1, Y1), (Y2, Y2), (X3, Y3),
If (X4, Y4) exists, it can be obtained by solving the equation (2).

[0137]

(Equation 4)

The position coordinates of the four points obtained here (X1, Y
1), (X2, Y2), (X3, Y3), (X4, Y
By connecting 4), the game device 205 in the real space shown in FIG. 15 is recognized.

More specifically, when the moving image pickup direction is the Y axis on the arrangement pattern shown in FIG. 23 and the direction orthogonal to the Y axis is the X axis, the same direction as the moving image pickup direction by the image processing unit 44 is used. Alternatively, the luminance signal values are added in the opposite direction. When this added value is plotted on the X axis, four positions where the luminance signal value plotted on the X axis is the maximum are detected, and X coordinate values X1, X corresponding to the four positions are detected.
2, X3 and X4 are determined. Further, when the acquired image is scanned in the Y direction on the arrangement pattern, among the plurality of luminescent points arranged in the Y direction, the luminescent spot position that first emits light has a Y coordinate value Y1 corresponding to the X coordinate value. , Y2, Y3, Y4
Is required.

Here, the position coordinates of the four light emitting diodes LED1 to LED4 in the real space are defined as wi (i = 1 to 4), and the position coordinates wi of the four light emitting diodes LED1 to LED4 on the camera coordinate system. The expression vector is Ci, and L of the four light emitting diodes LED1 to LED4 is L.
Assuming that the position coordinates on the CD screen are Pi, the rotation matrix of the panning CCD device 23 is R, and the movement vector is T, Expression (3), that is, Ci = R · wi + T (3) Ci = Pi.ki (ki is a scalar). Therefore, the normal CCD imaging device 25
Of the rotation matrix R and its movement vector T,
Using this as a parameter, coordinate conversion between the real space and the virtual space can be easily performed, so that the image of the soccer player using the 3D polygon 10 can be combined with the image of the game device 205 in the virtual space. This image processing unit 4
The synthesized video subjected to the predetermined image processing in 4 is transferred again to the LCD 26 and the LCD 27 in the special glasstron 206 via the internal interface 42.

Next, an example of processing in the 3D polygon broadcast system 400 according to the second embodiment will be described. In this example, the 3D polygon receiver 202 creates a three-dimensional image based on a plurality of pieces of video material information D1i relating to a soccer relay sent from a broadcasting station via a satellite line, and views from the plurality of pieces of video material information D1i. It is assumed that a viewer-specific 3D video is created based on video material information selected according to the viewer's preference. of course,
The viewer wears the special glasstron 206 shown in FIG. At the same time, a viewer wearing the special glasstron 206 can add a 3D polygon 1 to a soccer court using video software provided from the game device 205.
It is assumed that a three-dimensional image of a soccer player composed of 0s is synthesized and displayed.

Based on the above, a plurality of pieces of video material information D1i for creating a three-dimensional video are prepared in advance in the broadcasting station in step C1 of the flowchart shown in FIG.
For example, background information D representing the entire soccer court relating to satellite relay of “World Cup Soccer” (not shown)
11, object information D12 representing a soccer player moving in the background of this video, initial arrangement information D13 of a plurality of soccer players on a soccer court, positional information D14 associated with movement of these soccer players, and The video material information D1i is obtained from the motion information D15 accompanying the motion. In addition, “forest”, “sea”, and “process” formed by the computer 212 as described in the first embodiment.
CG data D4 such as "grass" and "sandy beach" may be included in the video material information D1i.

Thereafter, the video material information D1i prepared by the broadcasting station is stored in the 3D polygon transmitting device 201 in step C2.
Are multiplexed. In the 3D polygon transmission device 201 shown in FIG. 15, for example, the video material information D1i relating to the soccer relay obtained from the audio / video acquisition device 211 in step C21 is converted into the audio encoding unit 213 and the video encoding unit 214.
And output to the data encoding unit 215. The audio encoding unit 213 encodes a soccer player, a referee's whistle, a cheering team's audio, and the like, and generates an audio encoded signal. In the video encoding unit 214, the background information D11 relating to the background of the soccer court relating to the soccer relay which constitutes one program, the object information D12 relating to the image of the soccer player, the arrangement information D13 thereof,
Image material information D1i such as the position information D14 and the operation information D15 is encoded, and a video encoded signal is generated. The data encoding unit 215 encodes CG data D4 such as the progress of the score, and generates a CG data encoded signal.

The multiplexing means 1A connected to the outputs of the audio encoding unit 213, the video encoding unit 214, and the data encoding unit 215 allows the audio encoded signal, the video encoded signal, the CG data encoded signal, etc. Are multiplexed, and a plurality of transport stream packets TSj (j = 1 to m) are generated. Then, in step C22, the background information D11 related to the soccer relay described above is performed.
Is allocated to the channel ch1, the object information D12 relating to the soccer player is allocated to the channel ch2, the arrangement information D13 is allocated to the channel ch3, the position information D14 is allocated to the channel ch4, and the operation information D15 is allocated to the channel ch4. The two-dimensional video information is allocated to channel ch6, and is allocated to channel ch6.

In this example, the coded signal (video material information D1i) constituting one program is, as shown in FIG.
-4 is encoded so as to become a data stream bit stream. The output of the multiplexing means 1A is output from a transmission path encoding section 216 to synthesize the transport stream TSj,
OFDM processing such as MCC generation, error correction and modulation is performed in step C23.

The coded signal after the OFDM processing is subjected to digital modulation of the TC8PSK system in the transmitter 217, and a step C is performed using a 34.5 MHz bandwidth.
At 24, the signal is transmitted to the viewer's receiving device via a satellite line. At this time, the video material information D1i is transmitted to the viewer using the digital television broadcast network. of course,
A cable television communication line or an Internet communication line may be used.

On the receiving side, at step C3, the game device 2
After the activation of the light emitting diode 05, the blinking control of the four light emitting diodes LED1 to LED4 is started in step C4. At this time, the 3D video data in the CD-ROM 79 is transferred to the geometry unit 85. This is because the received signal may not be completely processed by the game device 205.

At the same time, the video material information D1i sent from the broadcast station is received by the 3D polygon receiver 202 installed on the viewer side in step C5. For example, the 3D polygon receiving device 202 shown in FIG.
In the tuner section, TC8PS sent from the satellite line
A digital signal of the K modulation scheme is received.

The video detector 75 connected to the tuner 74 detects a TC8PSK modulation digital signal multiplexed on six channels. This video detector 7
In the noise removing unit 76 connected to 5, the digital signal transmitted in the MPEG-4 data series is subjected to noise removal and OFDM demodulation. This noise removing unit 7
In an error correction unit 77 connected to 6, the error code of the OFDM-demodulated digital signal is corrected.

In the information multiplexing analysis unit 78 connected to the error correction unit 77, the error-corrected 6-channel video material information D1i is demultiplexed. Background information D11 is ch
1, the object information D12 is obtained from ch2, the arrangement information D13 is obtained from ch3, and the position information D14 is c.
h4, operation information D15 is obtained from ch5,
And two-dimensional video information is obtained from ch5. These background information D11, object information D12, and arrangement information D1
3. Position information D14, motion information D15, and video material information D1i such as two-dimensional video information are temporarily stored in the image memory 3.

Thereafter, in step C6, the video material information D
Any video material information D1i from 1i is jog dial 8
9 is selected by the viewer. The video material information D1i selected by the jog dial 89 through the control device 88 and the selector 4 is stored in the video creation unit 6
Is output to In the geometry section 85 of the image creating means 6, a global coordinate system for expressing a three-dimensional image
3D video data such as a soccer court transferred from the CD-ROM 79 in step C3, background information D11 from a broadcasting station, object information D12 about soccer players, its arrangement information D13, its position information D14, and its operation information D1
5 is displayed (described).

In this example, a soccer court based on video software of a soccer game played on the CD-ROM 79 from the game apparatus 205 is replaced with a soccer court related to the relay of “World Cup Soccer”, and the 3D polygon 10 Are synthesized on the soccer player based on the soccer game software so as to exist on the soccer court based on the soccer game software.

In the raster unit 86 connected to the geometry unit 85, raster processing is performed so as to match the scanning method of the display unit 24. After that, the stereo image generation unit 87 connected to the raster unit 86 forms a stereo image based on the viewpoint data.

This stereo image is supplied to the special glasstron 206 in step C7. Special Glasstron 20
In step 6, for example, the viewer calls the first subroutine shown in FIG. 25, and sets the reference plane on the game device 205 in the real space to which the viewer belongs in step D1 of the flowchart. The shooting CCD device 23 is directed to the game device 205. After that, the four light emitting diodes LE through the blinking control circuit 13 shown in FIG.
A predetermined voltage is supplied to D1 to LED4 to blink in a predetermined blinking pattern.

In this example, the clock signal C having a predetermined frequency
LK1 is supplied to the light emitting diode LED1 through the resistor R, and a clock signal CLK2 obtained by dividing the clock signal CLK1 by １ ／ is supplied to the light emitting diode LE1 through the resistor R.
D2, the clock signal CLK3 obtained by dividing the frequency of CLK1 by 1/3 is supplied to the light emitting diode LED through the resistor R.
The clock signal CLK4 obtained by dividing the frequency of CLK1 by 1/4 is supplied to the light emitting diode LED4 through the resistor R.
Supplied to

Next, in step D2, a reference plane in real space is photographed using the ordinary CCD image pickup device 25, and a stereo image is displayed on the LCD 26 and the LCD 27. On the other hand, the panning CCD device 23 is used to pan and photograph a reference plane in real space. For example, 3D polygon 1
Game device 20 trying to synthesize 0 soccer players
5 light emitting diodes LE mounted at predetermined positions
Since D1 to LED4 blink in a different blinking pattern, the blinking pattern is imaged so as to flow in a predetermined imaging direction.

Thereafter, in step D3, image processing is performed to recognize an arbitrarily set reference plane in the real space to which the viewer belongs. The image processing unit 44 executes, for example, a video capture process in step E1 by calling a second subroutine shown in FIG. Then, in step E2, the light emitting diodes LED1 to LED4 at the four corners are recognized. Specifically, the blinking pattern of the luminance signal by the four light emitting diodes LED1 to LED4 captured by the panning CCD device 23 is converted into a spatial arrangement pattern forming an XY plane including the four luminescent points P1 to P4. You.

Thereafter, the arrangement pattern is scanned, and
At least the position coordinates (X1,
Y1), (X2, Y2), (X3, Y3), (X4, Y
4) is obtained, and the above-described equations (1) and (2) are calculated, and the four light emitting diodes LED1 to LE4 in the real space are calculated.
D4 mounting position and the position coordinates (X
1, Y1), (X2, Y2), (X3, Y3), (X
4, Y4) is obtained, and a reference plane is obtained by connecting these four points. Then, in step E3, the image processing unit 44 performs an arithmetic process based on the above equation (3), and detects the positional relationship between the panning CCD device 23 and the reference plane.

Thereafter, the process returns to step D4 of the first subroutine of FIG.
The image of the soccer court for the game is displayed on the ROM 79. Then, based on the above equation (2), the image of the soccer player using the 3D polygon 10 is synthesized on the reference plane of the virtual space.

At this time, in the special glasstron 206 worn by the viewer, a soccer player of the 3D polygon 10 kicking a soccer ball on the soccer court related to the relay of “World Cup Soccer” plays on the soccer court 10B based on soccer game software. One of the stereo images synthesized to play is displayed. Also, the LCD 27
Displays the other stereo image synthesized so that the soccer player of the 3D polygon 10 plays on the soccer court 10B based on soccer game software.

The stereo image of the LCD 26 and the LC
The D27 stereo image is guided to the viewer's eyeball. As a result, a three-dimensional image is synthesized in the viewer's head as if a soccer player kicking the soccer ball 10A on the court relating to “World Cup Soccer” is playing on the court 10B in the soccer game. .

Therefore, in the real space shown in FIG. 27, a soccer player or soccer ball 10A using the 3D polygon 10 and a soccer court 10 using the game software are placed on the reference plane.
28B, the soccer court 10B appears on the reference plane in the virtual space shown in FIG.
A soccer player or a soccer ball 10A using the 3D polygon 10 can appear.

As described above, according to the 3D polygon broadcasting system 400 according to the second embodiment, the video material selected according to the viewer's preference from the plurality of video material information D1i prepared in advance by the broadcasting station. A viewer-specific three-dimensional video can be created based on the information D1i. A viewer wearing the special glasstron 206 can play a game in which a soccer player using the 3D polygon 10 that kicks the soccer ball 10A is played on a soccer court using game software.

Therefore, the viewer does not merely passively watch the video provided by the broadcasting station, but can view the three-dimensional video which is actively and independently stereoscopically processed. it can. This allows the viewer to view a realistic three-dimensional image recognized by the eyes of a soccer player who is involved in “World Cup Soccer” while staying at home. Also, as in the first embodiment, I
A next-generation three-dimensional digital broadcasting system using the SDB service can be constructed.

When the special glasstron 206 is not mounted, a two-dimensional image when the soccer court is viewed from the seat is displayed on the television monitor 204. In addition, by switching the jog dial 89, a three-dimensional composite image of a soccer player composed of the 3D polygon 10 playing on the court 10B in the soccer game can be displayed on the television monitor 204. Since the court 10B in the soccer game is read from the CD-ROM 79, the data amount of the video material information transmitted from the broadcast station to the viewer can be reduced.

(4) Third Embodiment FIG. 29 is a perspective view showing a configuration example of a 3D polygon reception processing system 500 as a third embodiment. In this example, light source images having different blinking patterns are displayed on the TV monitor 204.
At a specific position, and the image processing system recognizes the position of the TV monitor 204 from the position of the light source image so that the image of the virtual body can be three-dimensionally combined with the two-dimensional image viewed by the viewer with good reproducibility. It was made. Note that components having the same names and the same reference numerals as those in the second embodiment have the same functions, and thus description thereof will be omitted.

In the 3D polygon reception processing system 500 shown in FIG. 29, a television image to be heard by a viewer, for example,
The video of the soccer player who plays in the match is displayed on the TV monitor 2 in the broadcast video of “World Cup Soccer” in the previous example.
04 is projected on the screen and stereoscopically displayed.

This 3D polygon reception processing system 500
Has a TV monitor 204 and a special glasstron 206 for setting a reference plane. The TV monitor 204 is, for example, a position where the viewer can view the TV monitor 204 and the special glasstron 2
06.

The TV monitor 204 displays light source images LPi at four corners in the screen shown in FIG. 29 in advance in order to provide a reference plane on which a virtual body image is to be synthesized. In this example, coordinates (x1, y1), (x2, y2), (x3, y3), and (x1, y1) of four points P1 to P4 on the TV screen at which a soccer player tries to jump out of the 3D polygon 10 are displayed.
(X4, y4) (corresponding to the position on the TV screen where the image is to be synthesized in the virtual space). The four light source images PL1 to PL4 are displayed with different blinking patterns so as to exhibit their functions as mark portions and to make the display positions clear on the TV monitor 204.

The light source images PL1 to PL4 are imaged by the panning CCD device 23 in the special glasstron 206 so as to flow in a predetermined panning imaging direction. This panning is for specifying the TV screen from the display positions of the four light source images PL1 to PL4. The specification of the TV screen is as described in FIG.

The main body 21 of this special glasstron 206
The image processing device 9 described in the second embodiment is provided therein, and performs image processing for recognizing a TV screen or the like including the light source image LPi based on the image data output from the panning CCD device 23. . Also in this example, a panning CCD device 23 and display means 24 are provided in the special glasstron 206 in addition to the image processing device 9.

Depending on the type of the special glasstron 206, a normal CCD image pickup device 25 is provided. The four light source images P displayed on the TV monitor 204 for setting the reference plane
A position recognition mechanism (surface recognition means) 7 is constituted by Li, the panning CCD device 23 and the image processing device 9 so that the display screen of the TV monitor 204 which the viewer views can be recognized. In the panning CCD device 23 which is a main part of the position recognition mechanism 7, the light source images LPi displayed with different blinking patterns are picked up at four corners in the screen of the TV monitor 204 so as to flow in a predetermined imaging direction. You. This panning is as described in the second embodiment.

The display means 24 described in the second embodiment is connected to the image processing apparatus 9 described above, and the TV screen recognized by the position recognition mechanism 7 is displayed. An optical unit such as a polarizing beam splitter may be provided in the special glasstron 206, and an image of a virtual body is synthesized on a TV screen of a virtual space displayed by the display unit 24. In this example, the position of the TV screen in the real space
It is made as if a soccer player exists with the 3D polygon 10 as a virtual body.

Next, an example of the configuration of the TV monitor 204 will be described. In this example, light source images having different blinking patterns are displayed at specific positions of a cathode ray tube (hereinafter referred to as CRT), and the position of the TV monitor 204 can be easily recognized by the image processing system from the position of the light source image. .

The TV monitor 204 shown in FIG. 30 has a display means 8 having an arbitrary image-capable screen. As the display means 8, a cathode ray tube (hereinafter referred to as CRT 8 '), a liquid crystal display panel, a plasma display panel, a flat display panel, or the like is used. Of course, the TV monitor 20
Reference numeral 4 denotes a color display format or a black-and-white display format.
And displays a three-dimensional image in response to the video signal of Within the screen of the TV monitor 204, at least three or more light source images are displayed at specific positions in the screen. In this example, rectangular light source images LP1 to LP4 are displayed at the four corners of the display video screen of the display unit 8. The light source images LP1 to LP4 may be prepared in advance on the TV broadcast station side, allocated to a dedicated channel in the same manner as the video material information D1i, multiplexed and transmitted.
A method of superimposing the light source images LP1 to LP4 on the color video signal related to the video material information D1i selected on the receiver side is also conceivable. Here, the latter case, in which the CRT 8 'is used for the display means 8, will be described.

The TV monitor 204 shown in FIG.
8 ', an image processing means 90 and a control means 66. For example, the image processing means 90 is provided with a video detection circuit 91, which converts the color video intermediate signal Sin output from the preceding video intermediate amplification circuit (not shown) into the color video signal SC.
Is detected. An addition circuit 92 is connected to an output stage of the video detection circuit 91, and a light source image signal SPi (i = 1) having a different blinking pattern during a predetermined display period of the color video signal SC.
To 4) are added.

That is, the control means 66 is connected to the input stage of the addition circuit 92, and the light source images LP1 to LP4 displayed on the screen of the CRT 8 'are controlled to blink so that the blink patterns are different from each other. The control means 66 in this example has a light source image generation section 61, and generates a pulsed light source image signal SP1 having a predetermined frequency. The light source image signal SP1 is 30
A low-frequency range excluding 50 Hz is preferable. this is,
This is because light blinking at 30 to 50 Hz may cause a viewer to have a seizure.

The clock generation circuit 61 described in the second embodiment is applied to this light source image generation section 61 '.
Frequency dividing circuit 62, 1/3 frequency dividing circuit 63, 1/4 frequency dividing circuit 6
4 is connected. A light source image signal SP1 of a predetermined frequency by a light source image generation unit 61 'and a light source image signal SP obtained by dividing the light source image signal SP1 by a half frequency dividing circuit 62.
2, a light source image signal SP3 frequency-divided by 1/3 in the 1/3 frequency dividing circuit 63, and a light source image signal SP4 frequency-divided by the 1/4 frequency dividing circuit 64.

An output circuit 65 is connected to the output stage of the light source image generation unit 61 ', 1/2 frequency dividing circuit 62, 1/3 frequency dividing circuit 63 and 1/4 frequency dividing circuit 64, and the vertical synchronizing signal SV
The light source image signals SP1 to SP4 are selected based on the horizontal synchronization signal SH and output to the addition circuit 92.

A video amplification circuit 93 is connected to the output stage of the addition circuit 92, and the light source image signal SP having a different blinking pattern added during a predetermined display period of the color video signal SC.
i is amplified. This addition circuit 92 may be connected to the output stage of the video amplification circuit 93 as shown by the broken line in FIG.
A color processing circuit 94 and a scanning processing circuit 95 are connected to an output stage of the video amplification circuit 93. Color processing circuit 9
In 4, the color video signal SC and the light source image signal SPi are supplied to the cathode and grid of the CRT 8 'as drive voltages VK and VG after predetermined gradation processing and the like.

In the scanning processing circuit 95 connected to the video amplifying circuit 93, the synchronizing signal SS is detected, and one cycle is one.
6.6 msec vertical synchronization signal SV and one cycle 63 μsec
Is output to the output circuit 65 described above. Further, the vertical synchronizing signal SV and the horizontal synchronizing signal SH
Horizontal deflection voltage VV and vertical deflection voltage V generated based on
H is supplied to the biased horizontal / vertical coils, and the similarly generated high voltage VHH is supplied to the anode of the CRT 8 '.

Thus, the light source image PLi can be synthesized and displayed on an arbitrary reception screen displayed on the CRT 8 ', and the light source image PLi can be flickered. In this example, a state where visible light of a specific wavelength X is emitted from the light source image PLi or a state where it is not emitted can be created.

It is preferable that the visible light has a color difference that cannot be recognized by human eyes. For example, the light source image PL
An x, y matrix film is stuck at the position where i flashes. For this film, a polymeric material having a variable transmission region, such as galvazole or a derivative thereof, may be used. For example, light of a specific wavelength that has passed through a green film of this type appears to the human eye as a solid green color, but the panning C
This special light, such as the CD device 23, can be detected by passing the light through a thin film filter having a specific transmission band in the image pickup device. Therefore, even if the light source image is made to blink at 30 to 50 Hz, the above-mentioned seizure of photoepilepsy can be prevented.

Next, an example of the operation of the TV monitor 204 when a light source image signal is inserted will be described. In this example, the light source image LPi is displayed using left and right several pixels of the upper and lower five lines displayed on the CRT 8 ′ of the TV monitor 204, and is displayed on both edges of one horizontal period of the color video signal SC. The light source image signals SPi (i = 1, 3) and SPj (j =
It is assumed that (2) and (4) are superimposed. For example, NTCS
In the case where the number of scanning lines is 525 and the interlaced scanning method is used, the light source image LP1 is displayed at the left end of the first, third and fifth lines of the odd-numbered frame, and the light source The image LP2 is displayed. No. 521, No. 523
The light source image LP3 is displayed at the left end of the 525th line, and the light source image LP4 is displayed at the right end of the same line.

Further, the light source image LP1 is displayed at the left end of the second and fourth lines of the even-numbered frame lines, and the light source image LP2 is displayed at the right end of the same line. Its 522
The light source image LP3 is displayed at the left end of the 524th line, and the light source image LP4 is displayed at the right end of the same line.

Under these preconditions, the control means 6
6, in the odd-numbered frames in one vertical period, the time intervals corresponding to the display of several pixels on the first, third, and fifth lines based on the vertical synchronization signal SV and the horizontal synchronization signal SH from the scanning processing circuit 95. After selecting the light source image signal SP1, the light source image signal SP2 is selected immediately before the end of the one horizontal period. As a result, the light source image signal SP1 is located at the left edge of one horizontal period of the color image signal SC shown in FIG.
Can be superimposed, and the light source image signal SP
2 can be superimposed. Therefore, the light source image LP1 can be displayed at the upper left end of the display screen shown in FIG. 30, and the light source image LP1 can be displayed at the upper right end.

The odd-numbered frames 521, 5
As for the 23rd and 525th lines, the light source image signal SP4 is selected after the light source image signal SP3 is selected as described above based on the vertical synchronization signal SV and the horizontal synchronization signal SH from the scanning processing circuit 95. As a result, the light source image signal SP3 can be superimposed on the left edge of one color period of the color image signal SC shown in FIG. 33, and the light source image signal SP4 can be superimposed on the right edge. Therefore, FIG.
The light source image LP3 can be displayed at the lower left end of the display screen shown at 0, and the light source image LP4 can be displayed at the lower right end.

In the even-numbered frames of one vertical period, the light source image signal SP1 is selected for the second and fourth lines based on the vertical synchronization signal SV and the horizontal synchronization signal SH from the scanning processing circuit 95, and then the light source image is selected. Signal S
Since P2 is selected, the color image signal S shown in FIG.
The light source image signal SP1 can be superimposed on the left edge of one horizontal period of C, and the light source image signal SP2 can be superimposed on the right edge thereof. Therefore, the light source image LP1 can be displayed at the upper left end of the display screen shown in FIG. 30, and the light source image LP1 can be displayed at the upper right end.

For the 522nd and 524th lines of the even-numbered frame, the light source image signal SP4 is selected after the light source image signal SP3 is selected based on the vertical synchronization signal SV and the horizontal synchronization signal SH from the scanning processing circuit 95. Therefore, the light source image signal SP3 can be superimposed on the left edge of one horizontal period of the color image signal SC shown in FIG. 33, and the light source image signal SP4 can be superimposed on the right edge thereof. Therefore, the light source image LP3 can be displayed at the lower left end of the display screen shown in FIG. 30, and the light source image LP4 can be displayed at the lower right end.

As a result, the TV monitor 204 preliminarily displays the light source image PL1 on an arbitrary reception screen viewed by the viewer.
~ PL4 when the composite display is performed, the T
When the light source images PL1 to PL4 displayed on the V monitor 204 are shot by a special imaging device 23 such as a CCD, the four light source images PL1 to PL4 are turned on uniformly. The positions of the light source images PL1 to PL4 can be easily specified as compared with the case. Therefore, the position of the display screen of the TV monitor 204 can be easily recognized by the image processing system such as the special glasstron 206 from the positions of the light source images PL1 to PL4.

Next, another example of the optical system of the panning CCD device 23 will be described. Follow shot CC of this example
The D device 23 is provided with a fisheye lens 35 shown in FIG. 34 in the same manner as in the second embodiment. A thin-film filter 37 is mounted on the front surface of the fisheye lens 35, and light of a specific wavelength X emitted from the four light source images PL1 to 4 on the display screen of the CRT 8 'is taken in and guided to the CCD image sensor 36. . This thin film filter 37 is like an IR filter. This IR filter has a structure in which a TiO ₂ film or a SiO ₂ film is stacked in multiple layers, and by controlling the thickness of one layer, a light transmission band (transmittance vs. wavelength) transmitting red, green, blue, and the like. It can be set freely.

In an actual television, infrared rays and ultraviolet rays are emitted from the surface of the CRT 8 ', though minutely. The wavelength band of light emitted from the light source image PLi fluctuates due to the infrared light or the ultraviolet light. In this example, an x, y matrix film is stuck at a position where the light source image PLi blinks. For the x, y matrix film, the above-mentioned transmission range varying polymer material is used. This material has a wavelength band that fluctuates due to a change in voltage and current.

When a thin film filter is used for the above-mentioned panning CCD device 23 or an x, y matrix film is used for the CRT 8 ′, the filter is affected by the color tone error of the CRT 8 ′ (or LCD) and the manufacturing variation of the thin film filter. It is necessary to calibrate the color with the characteristic. For this alignment, for example, the light source image PLi is displayed in a default color, and the panning CCD device 23 uses the light source image PLi.
It is detected whether or not light due to Li can be received.

If the light from the light source image PLi can be received, light with the default color is picked up. If the light from the light source image PLi cannot be received, the default color is changed. Then, the light of the changed color is imaged, and when the light of the light source image PLi can be received, the light of the changed color is imaged.

As a result, visible light having a color difference that cannot be recognized by the human eye and having a wavelength close to the above-described specific wavelength X, is transmitted from the transmission band of the thin film filter 37 attached to the panning CCD device 23. Deviated light source image PL
i. It can prevent viewers from epileptic seizures.

Next, an example of processing by the 3D polygon broadcast system 400 according to the third embodiment will be described with reference to FIGS. 24, 25 and 26 described above again. In this example, four light source images PL1 to PL4 are displayed in a predetermined blinking pattern on the display screen of the TV monitor 204, and the TV monitor 20 that the viewer listens to is displayed.
3D jumped out of TV video (two-dimensional image display) of 4
It is assumed that a video of a soccer player by the polygon 10 is three-dimensionally synthesized in a virtual space. Of course, the viewer wears the special glasstron 2 shown in FIG. 29 on the head.

On the premise of this, a plurality of pieces of video material information D1i for creating a three-dimensional video are prepared in advance at the broadcasting station in step C1 of the flowchart shown in FIG. For example, background information D1 representing the entire soccer court related to satellite relay of “World Cup Soccer” (not shown)
1 and object information D1 representing the soccer player who follows and kicks the soccer ball 10A in the background of this video.
2, video material information D1i is acquired from initial arrangement information D13 of a plurality of soccer players on the soccer court, position information D14 associated with the movement of these soccer players, and operation information D15 associated with the movements of the soccer players. In addition, CG data D4 such as "forest", "sea", "lawn" or "sandy beach" processed and formed by the computer 212 as described in the first and second embodiments is converted to video material information D.
1i.

Thereafter, the video material information D1i prepared on the broadcast station side is transmitted to the 3D polygon transmitting device 201 in step C2.
Are multiplexed. Regarding the multiplexing at this time, FIG.
Please refer to the operation example of the 3D polygon transmitting apparatus 201 shown in FIG.

On the receiving side, at step C3, the game device 2
After the TV monitor 204 is activated in place of the TV monitor 05, the blinking control of the four light source images PL1 to PL4 is started in step C4. In parallel with this, the video material information D1i sent from the broadcasting station is received by the 3D polygon receiving device 202 installed on the viewer side in step C5. The receiving operation of the 3D polygon receiving device 202 at this time is as described with reference to FIG.

Thereafter, in step C6, the video material information D
Any video material information D1i from 1i is jog dial 8
9 is selected by the viewer. The video material information D1i selected by the jog dial 89 through the control device 88 and the selector 4 is stored in the video creation unit 6
Is output to In the geometry unit 85 of the image creating means 6, background information D11 such as a soccer court, object information D12 relating to a soccer player, arrangement information D13 thereof, position information D14 thereof, and its operation are provided in a global coordinate system for expressing a three-dimensional image. Information D15 is displayed (described).

In this example, a soccer player who kicks soccer ball 10A is demultiplexed. A two-dimensional image related to the relay of “World Cup Soccer” is displayed on the TV monitor 204, and the soccer player composed of the 3D polygon 10 jumps out of the TV monitor 204, and the TV monitor 2
A three-dimensional image such as kicking the soccer ball 10A toward 04 is synthesized, and these image synthesis signals are created.

The raster section 86 connected to the geometry section 85 is subjected to raster processing so as to match the scanning method of the display means 24. After that, the stereo image generation unit 87 connected to the raster unit 86 forms a stereo image based on the viewpoint data.

This stereo image is supplied to the special glasstron 206 in place of the special glasstron 203 in step C7. The special glasstron 206 calls, for example, the first subroutine shown in FIG. 25 to set a reference plane on the TV monitor 204 in the real space to which the viewer belongs in step D1 of the flowchart. The user makes the panning CCD device 23 face the TV monitor 204. Thereafter, the four light source images PL1 to PL4 blink in a predetermined blink pattern through the control means 66 shown in FIG.

Next, in step D2, on the other hand, a TV image in the real space is photographed using the ordinary CCD image pickup device 25, and a stereo image is displayed on the LCD 26 and the LCD 27. On the other hand, the panning CCD device 23 is used to pan and photograph the reference surface on the TV monitor 204 in the real space. Then, in step D3, image processing is performed to recognize a reference plane arbitrarily set in the real space to which the viewer belongs. The image processing unit 44 executes, for example, a video capture process in step E1 by calling a second subroutine shown in FIG. Then, in step E2, the light source images P at the four corners
Recognize L1 to PL4. Specifically, a panning CCD
The blinking pattern of the luminance signal by the four light source images PL1 to PL4 captured by the device 23 includes four bright points P1 to P4.
Are converted into a spatial arrangement pattern that forms an XY plane including

Thereafter, the arrangement pattern is scanned, and
At least the position coordinates (X1,
Y1), (X2, Y2), (X3, Y3), (X4, Y
4) is calculated, and the above-described equations (1) and (2) are calculated. The light emission positions of the four light source images PL1 to PL4 in the real space and the position coordinates (X1, Y1) of the four points in the image processing system are calculated. ),
The relationship between (X2, Y2), (X3, Y3), and (X4, Y4) is obtained, and a reference plane is obtained by connecting these four points. Then, in step E3, the image processing unit 44 performs an arithmetic process based on the above equation (3), and detects the positional relationship between the panning CCD device 23 and the reference plane.

Thereafter, the flow returns to step D4 of the first subroutine in FIG. 25, where the two-dimensional video relating to the relay of “World Cup Soccer” displayed on the TV monitor 204 and the virtual image based on the above equation (2) 3D polygon 10 to chase and kick a soccer ball on the reference plane of space
Is synthesized with the image of the soccer player.

At this time, in the special glasstron 206 worn by the viewer, a two-dimensional video relating to the relay of “World Cup Soccer” and a soccer player of the 3D polygon 10 kicking the soccer ball 10A toward the TV monitor 204 are displayed. One of the synthesized stereo images is displayed. Also,
On the LCD 27, the other of the stereo images obtained by synthesizing the two-dimensional video relating to the relay and the soccer player of the 3D polygon 10 is displayed.

The stereo image of the LCD 26 described above and the LC
The D27 stereo image is guided to the viewer's eyeball. Thus, a three-dimensional image of a soccer player who actually kicks the soccer ball 10A on the court related to “World Cup Soccer” kicks the soccer ball 10A toward the TV monitor 204 in the virtual space is displayed in the viewer's head. Synthesized by

That is, on the reception screen of the TV monitor 204 viewed by the viewer 30 shown in FIG. 35, for example, a two-dimensional image of the “World Cup Soccer” shown in FIG. It is just being done. On the other hand, when the special glasstron 206 shown in FIG. 37 is attached, the soccer ball 1 is moved from the reception screen to the outside in the virtual space on the TV monitor 204.
A soccer player using the 3D polygon 10 that kicks OA can appear.

As described above, according to the 3D polygon broadcasting system 500 according to the third embodiment, the video material selected according to the viewer's preference from the plurality of video material information D1i prepared in advance by the broadcasting station. A viewer-specific three-dimensional video can be created based on the information D1i.

[0211] In addition, in the viewer wearing the special glasstron 206, the received image of the TV monitor 204 viewed by the viewer 30 and the soccer player of the 3D polygon 10 kicking the soccer ball 10A appearing in the virtual space. Since it is synthesized in the head, the soccer player kicking the soccer ball 10A displayed on the TV screen in the real space becomes a soccer player of the 3D polygon 10 as if jumping out of the screen. Images can be synthesized three-dimensionally.

[0212] Therefore, the viewer does not simply passively watch the video provided from the broadcast station side, but the viewer can view the three-dimensional video that is actively and independently stereoscopically processed. it can. Thus, it is possible to view a realistic three-dimensional video that realistically reproduces the movement of a soccer player relating to “World Cup Soccer” while staying at home.

In addition, in the same manner as in the first and second embodiments,
A next-generation three-dimensional digital broadcasting system using the SDB service can be constructed. In the third embodiment, the flickering patterns of the light source images PL1 to PL4 can be controlled separately from the information related to the normal TV video by attaching the x and y matrix films to the light emitting positions of the light source images PL1 to PL4.

(5) Fourth Embodiment FIG. 38 is a front view showing a configuration example of a TV monitor 208 used in the fourth embodiment. In this example, instead of the four light source images PL1 to PL4, a two-dimensional barcode image 50 shown in FIG.
Is recognized on the TV screen. of course,
The viewer has at least the normal imaging device 25 shown in FIG.
It is premised that a special glasstron 206 having the following is mounted.

The TV monitor 208 of this example displays the two-dimensional matrix code image 50 shown in FIG. 38 every frame (1/30 sec) by the interlaced scanning method. This two-dimensional matrix code image 50 is used to recognize the position of the TV screen of the TV monitor 204 in the real space. The two-dimensional barcode image 50 is composed of at least an n rows × m columns monochrome matrix displayed in black on a white background.

The two-dimensional barcode image 50 is replaced with a normal CCD shown in FIG.
The image is captured by the imaging device 25. The image processing device 9 described with reference to FIG. 22 is connected to the output stage of the imaging device 25. The image processing device 9 is provided with arithmetic means,
An imaging signal (luminance signal) output from the CD imaging device 25
From the two-dimensional matrix code image 50 to T
The position of the TV screen of the V monitor 208 is obtained.

For example, the image processing section 44 performs preprocessing. In this process, first, the acquired image is set at 2 with an appropriate threshold.
Valued. Since the barcode image portion is displayed in a black and white checkerboard pattern, the background image and the code image region can be separated quite stably by the fixed threshold value. Next, labeling is performed for each connected region of black pixels. The two-dimensional barcode image 50 will be included in any of the labeled connected regions. Therefore, in consideration of the size and aspect ratio of the circumscribed rectangle of the connected area, the background image (area) that is unlikely to include the code image area is removed.

Thereafter, a barcode image frame is applied to each element of the connected area obtained as a result of the preprocessing. For example, a checkerboard area is searched from each side of the circumscribed rectangle toward the inside, and a point sequence of the checkerboard contour is obtained.
The two-dimensional barcode image 50 is recognized by applying a line segment to the point sequence by the least square method.

The above-described TV screen is obtained by calculating a transformation matrix for projecting four vertices of the four corners of the rectangular two-dimensional barcode image 50 to vertices of a square. Here, a point (xi, yi,
0) is moved by a certain translation / rotational motion, and a point obtained by projecting the point on the image coordinate system by the perspective transformation is indicated by (Xi, Yi), and the point between them is described in the first embodiment (1). There is a similar relationship with the formula.

Therefore, these parameters are the position coordinates (x1, y1), (x2, y2),
(X3, y3), (x4, y4) and their corresponding four sets of position coordinates (X1, Y1), (Y2,
If (Y2), (X3, Y3) and (X4, Y4) exist, they can be obtained by solving the equation of the above-described equation (2).

The position coordinates (x1, y1) obtained here,
With respect to (x2, y2), (x3, y3), and (x4, y4), assuming that four vertices of a square whose side length is “1”, the surface connecting these four vertices is the TV monitor 208 in the real space.
Of the TV screen. As a result, the TV monitor 20 is obtained from the position information of the four corners of the two-dimensional matrix code image 50.
The position of 8 can be easily recognized by the image processing device 9 or the like.

Although the two-dimensional barcode image 50 on the TV monitor 208 is distorted by the attitude of the CCD imaging device 25 and perspective projection, rectangular vertices on the screen are changed to square vertices by external parameters and internal parameters. Can be projected. Therefore, the cubic 51 can be created from the position coordinates of the four corners of the two-dimensional barcode image 50 in the virtual space shown in FIG.
Can be combined with a soccer player using the 3D polygon 10. Further, when the two-dimensional barcode image 50 shown in FIG. 38 is output on an x, y matrix film, image processing can be performed separately from information related to normal TV video.

(6) Example of Acquiring Image Material Information (Part 1) FIG. 40 is a perspective view showing an example of the configuration of an object shape measuring apparatus 600 for acquiring image material according to each embodiment. In this example,
A silicon range finder is used to provide three-dimensional shape information relating to a relatively small object (subject) to a receiving side, and video material information D1i acquired by using the silicon range finder is multiplexed from a broadcasting station to a viewer side. It is something to do.

An object shape measuring device 600 shown in FIG.
It measures a three-dimensional height shape from a reference plane to an arbitrary target object. In this example, the object shape measuring device 600 has a light emitting unit 11 and a light receiving unit 12 shown in FIG.
The light emitting unit 11 and the light receiving unit 12 are engaged with each other by a hinge (hinge) 54 so as to be freely opened and closed. For example, when the object shape measuring device 600 is used, the light emitting unit 11 is opened until the angle formed between the light emitting unit 11 and the light receiving unit 12 is fixed at a predetermined position. When the device 600 is not used, the light emitting unit 11 is opened. And the light receiving unit 12 are closed so as to overlap each other. The storage shape is as if the light emitting unit 11 was covered by the light receiving unit 12 like a lid.

This light receiving section 12 also serves as a reference plane. The size of the light receiving section 12 is, for example, about 30 cm to 50 cm on one side. Considering the size that can be easily carried, the length of one side of the light receiving section 12 should be 3
Preferably, it is formed to about 5 cm. Of course, the size of the light receiving unit 12 is only an example, and is not limited to this size in consideration of use in a broadcasting station.

In this example, the light emitting section 11 has a surface light source 6 as a light source. The surface light source 6 generates light to be measured for irradiating the target object 30. The target object 30 is a ball, a doll, a robot, a small animal, or the like. A liquid crystal shutter 17 is provided above the surface light source 6, the light to be measured by the surface light source 6 is shaped into slit light, and the slit light is scanned on the target object 30. In this example, the opening and closing timing of the liquid crystal shutter 17 is controlled. For example, the image processing device 98 provided in the light receiving unit 12 performs vertical scanning such that the liquid crystal shutter 17 is opened and closed line by line. The image processing device 98 will be described with reference to FIG.

FIG. 41 is a cross-sectional view showing an example of the configuration of an object shape measuring device 600 using a laser light source, a slit lens, and a prism with respect to the surface light source 6 described above. In this example, a laser light source 14 is mounted in the light emitting unit 11. As the laser light source 14, a near infrared semiconductor laser having a wavelength of 780 μm and an output of 30 mW is used.

The laser light (measured light) generated from the laser light source 14 is shaped into slit light L0 by the slit lens 15, and the direction of the slit light L0 is polarized at 90 ° by the plurality of prisms 16. . This prism 16 is attached below the liquid crystal shutter 17.

That is, the object shape measuring device 6 shown in FIG.
The light emitting section 11 of 00 has an upper housing 96 attached to the hinge 54. The laser light source 14 is provided in the housing 96, and generates laser light. A slit lens 15 is provided downstream of the laser light source 14,
The laser light is shaped into sheet-like slit light L0. A plurality of prisms 16 are arranged in a matrix on the downstream side of the slit lens 15, and the traveling direction of the slit light L <b> 0 is polarized in a right angle direction, that is, the exit side of the housing 96.

A liquid crystal shutter (mask LC) is located on the upper surface of the housing 96 at a position located on the upper surface of the prism 16.
D) 17 is mounted, and the slit light L0 from the prism 16 is selectively passed by opening and closing the liquid crystal shutter 17. The plurality of prisms 16 and the liquid crystal shutter 17 constitute a scanning unit, and the slit light L0
Is scanned on the target object 30.

When the slit light L0 passing through the liquid crystal shutter 17 is reflected by an arbitrary target object 30 on the light emitting section,
The return light L0 'reaches the light receiving unit 12. Light receiving section 12
Has a lower housing 97 attached to the hinge 54. The light receiving lens 18 is provided on the housing 97,
The return light L0 'is taken into the housing 97. A silicon range finder 19 is provided below the light receiving lens 18, and return light (slit light) L0 'reflected from the target object 30 is detected for each pixel Pij. Here, the pixel Pij indicates a pixel existing at an arbitrary position (i, j) on the silicon range finder 19.

In the silicon range finder 19,
The timing of the return light L0 ′ passing through each pixel Pij is measured, and the angle αij between the reference plane, which is the emission angle of the slit light L0 by the prism 16 with the liquid crystal shutter 17 opened at that time, and the slit light L0. Is required. The angle between the reference plane and the line of sight of the pixel Pij is β
ij, the offset distance between the reference plane and the liquid crystal shutter 17 at the center position of the light emitting unit 11 is A, and the offset distance between the pixel Pij and the liquid crystal shutter 17 at the center position thereof is Bi.
When j is set, the three-dimensional height information Zij between the reference plane and the target object 30 is obtained by Expression (4).

[0233]

(Equation 5)

Thus, the three-dimensional height from the reference plane to the target object 30 can be measured in parallel and in real time for all the pixels in the silicon range finder 19 by only one scan of the slit light L0. it can.

Next, an example of a circuit configuration of the object shape measuring device 600 will be described. The object shape measuring device 600 shown in FIG. 42 is roughly composed of three circuit blocks. The first circuit block is a liquid crystal shutter 17. The liquid crystal shutter 17 is controlled to open and close based on the scanning signal SC, whereby the planar light to be measured by the prism 16 is shaped into slit light L0, and the slit light L0 is converted into the target object 30. Is scanned.

The second circuit block is a silicon range finder 19. In the silicon range finder, signal charges read from a photodiode (not shown)
By reading based on the vertical readout signal and the horizontal readout signal, parallel distance image measurement information including light passage timing information is output.

The third circuit block is an image processing device 98, which has an internal bus 41 for transferring the scanning signal SC and the parallel distance image measurement information. An interface (I / O) 42, an image capture unit 43,
Image processing unit 44, CPU 45, ROM 46, and RAM 4
7 is connected. The above-described liquid crystal shutter 17 and silicon range finder 19 are connected to an internal bus 41 via an interface 42.

A ROM 46 is connected to the internal bus 41, and stores a system program for controlling the object shape measuring device 600, control information such as a memory reading procedure, and the like. A working RAM 47 is connected to the internal bus 41, and a system program and parallel distance image measurement information are temporarily recorded. Further, a CPU 45 is connected to the internal bus 41, and an interface 42, an image capture unit 43, an image processing unit 44, a ROM 46, and an RA
The input and output of the M47 and the input and output of the liquid crystal shutter 17 and the silicon range finder 19 are controlled.

An image capture unit 43 is connected to the interface 42 and receives a control command from the CPU 45 to
A predetermined capture process for acquiring parallel distance image measurement information input from the silicon range finder 19 as image data is performed. An image processing unit 44 is connected to the image capturing unit 43 via the internal bus 41, and three-dimensional height shape information Zij is calculated based on parallel distance image measurement information on which predetermined image processing has been performed. Specifically, the three-dimensional height information Zij is obtained based on the above-described calculation formula.

The interface 42 has the RS-23
An external output terminal 68 based on the 2C communication protocol is connected, and is connected to a host computer or the like to enable image communication. The CCD device 69 shown by a broken line in FIG. 42 captures an image of an object.

Next, an operation example of the object shape measuring device 600 will be described. In this example, the light emitting unit 11 shown in FIG. 43 is opened until the angle αij formed between the light receiving unit 12 and the light emitting unit 11 is fixed at a predetermined position.
It is assumed that a three-dimensional shape of a sphere 30 ′ such as a tennis ball existing on the light receiving unit 12 is measured.
The reference plane is set on the light receiving unit 12.

On the premise of this, the light emitting unit 1 shown in FIG.
In 1, the liquid crystal shutter 17 is controlled to open and close based on the scanning signal SC from the image processing device 98, whereby the planar light to be measured by the prism 16 is shaped into slit light L0 and the slit light L0 Is scanned by the sphere 30 'shown in FIG.

The slit light L applied to the sphere 30 '
The irradiation position at a certain scanning timing of 0 is, for example, from the irradiation position shown in FIG. The return light L0 ′ from this irradiation position is detected by the silicon range finder 19 of the light receiving unit 12. At this time, the sphere 30 ′ inverted by the light receiving lens 18 forms an image on the silicon range finder 19.

Therefore, when the parallel range image measurement information obtained from the silicon range finder 19 is output to the image processing device 98, the three-dimensional height from the reference plane to the sphere 30 'is determined based on the parallel range image measurement information. Shape information Zij is calculated. Specifically, based on the above-described calculation formula, 3
Dimension height information Zij is obtained.

For example, when attention is paid to the line X1-X2 of the sphere 30 ', the three-dimensional shape is reflected so as to reflect the three-dimensional shape.
The three-dimensional height information Zx1 is calculated for the return light L0 ′ of the slit light L0, and the return light L of the slit light L0 is calculated.
The three-dimensional height information Zx2 is calculated with respect to 0 ', and the three-dimensional height information Z is calculated with respect to the return light L0' of the slit light L0.
x3 is calculated, three-dimensional height information Zx4 is calculated for the return light L0 'of the slit light L0, and three-dimensional height information Zx5 is calculated in real time for the return light L0' of the slit light L0. Is done. These height shape information Zx1 to Zx
By joining x5, the three-dimensional shape of X1-X2 of the sphere 30 'can be obtained.

Thus, the three-dimensional shape information (continuous height information H) of the object in the real space can be obtained, and the three-dimensional shape information Zij is transmitted from the transmission side to the viewer side. be able to. Of course, the three-dimensional shape information Zij may be obtained by rotating the object shape measuring device 600 around the target object 30. Therefore, the first
In the third to third embodiments, the image of the 3D polygon 10 can be synthesized on the virtual space on the receiving side based on the three-dimensional shape information Zij relating to various target objects prepared in advance by the broadcasting station.

(7) Example of Acquiring Video Material Information (Part 2) FIG. 45 is a perspective view showing a configuration example of the three-dimensional shape obtaining mechanism 60 for obtaining a video material according to each embodiment. In this example,
When scanning the object 30 with the slit light, the mirror 73 is used. The three-dimensional shape acquisition mechanism 60 is provided in a broadcasting station (not shown).

The three-dimensional shape obtaining mechanism 60 is provided with a laser light source 71 shown in FIG. 45, and generates a laser beam L0. A cylindrical lens 72 is provided downstream of the laser light source 71, and the laser light L0 is formed into a sheet-like slit light. A mirror 73 is disposed as a scanning unit on the downstream side of the cylindrical lens 72, and the slit light L0 is scanned on the object 30 by changing the polarization angle of the slit light L0.

When the slit light L0 is reflected by a part of the body part of the viewer or the object 30 held by the viewer,
The return light L0 'is captured by the light receiving lens 84. A silicon range finder 59 is provided below the light receiving lens 84.
Is provided, and return light (slit light) L0 ′ reflected from the object 30 is detected for each pixel Pij existing at an arbitrary position on the silicon range finder 59.

In this silicon range finder 59,
The timing of the return light L0 'passing through each pixel Pij is measured, and the return light L passing through each pixel Pij at that time is measured.
The timing of 0 'is measured, and the angle αij between the reference plane, which is the polarization angle of the mirror 73 at that time, and the slit light L0 is obtained. Here, the angle formed between the reference plane and the viewing direction of the pixel Pij is βij, and the reference plane and the mirror 73
Is the offset distance between the pixel Pij and the rotation center of the mirror 73, and Bij is the three-dimensional shape information Zij between the reference plane and the object 30.
Is obtained by the above equation (4).

Thus, the three-dimensional shape information (continuous height information H) of the object in the real space can be obtained, and the three-dimensional shape information Zij is transmitted from the transmission side to the viewer side. be able to. Therefore, in the first to third embodiments, the image of the 3D polygon 10 can be synthesized on the virtual space on the receiving side based on the three-dimensional shape information Zij relating to various target objects prepared in advance by the broadcasting station. .

In this example, the case where the silicon range finders 19 and 59 are used for the object shape obtaining means has been described. However, the present invention is not limited to this. A reference camera for capturing a reference image of the object and a reference camera A multi-view stereo camera having a detection camera that captures a reference image may be used. The three-dimensional shape information Zij of the object can be obtained based on the parallax information obtained from the reference camera and the detection camera.

A special glasstron 203 according to each embodiment,
206 is disclosed in JP-A-10-123453, JP-A-9-3
04727, JP-A-9-304730, JP-A-9-304
No. 211374, JP-A-8-160348, JP-A-8-160348
-94960, JP-A-7-325265, JP-A-7-325
The present invention can be applied to a transmission type head mounted display described in Japanese Patent Application Laid-Open No. 270714/1995 and Japanese Patent Application Laid-Open No. 7-67055.

Although the following description has been given of the case where the two-dimensional imaging device of the interline transfer system is used as the panning CCD 23 according to each embodiment, the present invention is not limited to this. The same effect can be obtained even if the above operation is performed.

In the above-described embodiment, the one-way multimedia service from the broadcast station side to the viewer side has been described. However, the present invention is not limited to this, and communication means between the transmission side and the viewer side, for example, The telephone line or the Internet network is connected, and using these communication means, the viewer sends request information (individual information) such as opinions, impressions, rebroadcasts, etc. from the viewer to the broadcasting station or program provider (sponsor). It is also possible to construct an interactive service network for transmission.

In the case where this interactive service network is constructed, shopping information may be transmitted as video material information. In this case, when the viewer performs TV shopping on the shopping information, the order information related to the shopping information is transferred to the program provider (a local store or the like), and the product price related to the shopping information is reduced. May be added to the fee associated with the viewer's use of the channel. Bank withdrawal by ID card is possible.

[0257]

As described above, according to the three-dimensional video transmission / reception system according to the present invention, a plurality of pieces of video material information sent from the transmission side to the reception side in order to create a three-dimensional video are included. And a video creating means for creating a three-dimensional video based on the video material information arbitrarily selected from the above. The viewer selects the video material information.

With this configuration, a viewer-specific three-dimensional video can be created based on video material information selected according to the viewer's preference from a plurality of video material information prepared in advance on the transmission side. Therefore, the viewer can not only passively watch the video provided from the broadcasting station, but also view the three-dimensional video that is actively and independently stereoscopically processed.

According to the method for transmitting and receiving a three-dimensional video according to the present invention, a plurality of pieces of video material information for preparing a three-dimensional video are prepared on the transmission side, and the video material information is transmitted from the transmission side to individual viewers. The viewer transmits the video material information sent from the transmitting side, and the viewer selects arbitrary video material information from the video material information to create a three-dimensional video.

[0260] With this configuration, the viewer does not simply passively watch the video provided from the broadcast station or the like.
A viewer can actively and independently view a three-dimensional video image that has been stereoscopically processed. Thus, it is possible to view a three-dimensional video with a sense of reality recognized by the eyes of players playing on the soccer field and actors performing on the stage while staying at home. INDUSTRIAL APPLICABILITY The present invention is extremely suitable for application to a next-generation three-dimensional digital broadcasting system using an ISDB service or an interactive service.

[Brief description of the drawings]

FIG. 1 is a perspective view showing a configuration example of a transmission / reception system 100 for three-dimensional video as an embodiment according to the present invention.

FIG. 2 is a flowchart illustrating a processing example of a transmission / reception system 100 of a three-dimensional video.

FIG. 3 is a block diagram illustrating a configuration example of a 3D polygon broadcast system 200 as a first embodiment.

FIG. 4 is a block diagram illustrating a configuration example of a 3D polygon transmission device 201 according to each embodiment.

FIG. 5 is a block diagram illustrating a configuration example of an MPEG-4 video encoder 300.

FIG. 6 is a perspective view showing an example of acquiring video material information D1i in the stage device 40.

FIGS. 7A and 7B are images of the front and side of a stage showing an example of acquiring video material information D1i.

FIG. 8 is a block diagram illustrating a configuration example of a 3D polygon receiving device 202 according to each embodiment.

FIG. 9 is an image diagram showing a display example in which the background recognized by the line of sight of actor B is replaced with “forest” or “grass” of 3D polygon 10;

FIG. 10 is a front view showing a configuration example of a special glasstron 203.

FIG. 11 is a conceptual diagram showing an internal configuration example of a special glasstron 203 as viewed from the top of a partially crushed glass.

FIG. 12A is a conceptual diagram showing a configuration example of film CCDs 4R and 4L viewed from above, and FIG. 12B is a conceptual diagram showing an example of left and right pupil positions.

FIG. 13 is a gazing point p when the special glasstron 203 is mounted.
It is a conceptual diagram which shows the example of positional relationship of.

FIG. 14 is a flowchart illustrating a processing example in the 3D polygon broadcast system 200 according to the first embodiment.

FIG. 15 is a block diagram illustrating a configuration example of a 3D polygon broadcast system 400 as a second embodiment.

FIG. 16 is a block diagram showing an example of an internal configuration of a blink control circuit 13 attached to the game device 205.

FIG. 17 shows the four light emitting diodes LED1 to LED
FIG. 6 is a waveform diagram illustrating a voltage supply example of FIG.

FIG. 18 is a front view showing a configuration example of a special glasstron 206 used in the 3D polygon broadcasting system 400.

FIG. 19 is a front view showing a configuration example of another special glasstron 207.

FIG. 20 is a plan view showing an example of the internal configuration of a panning CCD device 23 mounted on a special glasstron 206 or the like.

FIG. 21 is a cross-sectional view showing a configuration example of an optical system of the panning CCD device 23.

FIG. 22 is a block diagram showing an example of the internal configuration of a special glasstron 206.

FIG. 23 is a schematic diagram illustrating an example of calculating position coordinates of a reference plane according to the game apparatus 205.

FIG. 24 is a flowchart showing a processing example (main routine) in the 3D polygon broadcast system 400.

FIG. 25 is a flowchart showing an operation example (first subroutine) of the special glasstron 206.

FIG. 26 is a flowchart showing an operation example (second subroutine) of the special glasstron 206.

FIG. 27 is a perspective view showing an example of an image of the game device 205 in a real space.

FIG. 28 is a perspective view illustrating a synthesis example of a virtual image on a reference plane in a virtual space.

FIG. 29 is a perspective view showing a configuration example of a 3D polygon reception processing system 500 as a third embodiment.

FIG. 30 is a front view showing an example of the internal configuration of the TV monitor 204.

FIG. 31 is a block diagram showing an example of the internal configuration of a TV monitor 204.

FIG. 32 is an image diagram showing a waveform example of light source image signals SP1 to SP4.

FIG. 33 shows a light source image signal SPi for a video signal SC;
It is a waveform diagram which shows the example of superposition of SPj.

34 is a sectional view showing another example of the configuration of the optical system of the panning CCD device 23. FIG.

FIG. 35 is a perspective view showing a real image example of a viewer (when the special glasstron 206 is not mounted).

FIG. 36 is a front view showing a display example of a two-dimensional video in which a goal is viewed from a field displayed on the TV monitor 204.

FIG. 37: Viewer (when special glasstron 206 is attached)
FIG. 3 is a perspective view showing an example of a virtual image.

FIG. 38 is a front view showing a configuration example of a TV monitor 208 used in the fourth embodiment.

FIG. 39 is a perspective view showing an example of combining 3D polygons 10 on a two-dimensional barcode image 50.

FIG. 40 is a perspective view showing an example of the appearance of an object shape measuring device 600 for acquiring a video material according to each embodiment.

FIG. 41 is a cross-sectional view showing an example of the internal configuration of the object shape measuring device 600.

42 is a block diagram illustrating a circuit example of the object shape measurement device 600. FIG.

FIG. 43 is a perspective view showing a measurement example (part 1) of a sphere 30 ′ by the object shape measuring device 600.

FIG. 44 is a conceptual diagram showing an example (part 2) of measuring a sphere 30 ′ by the object shape measuring device 600.

FIG. 45 is a conceptual diagram showing a configuration example of a three-dimensional shape obtaining mechanism 60 for obtaining a video material according to each embodiment.

[Explanation of symbols]

DESCRIPTION OF SYMBOLS 1 ... Transmission means, 1A ... Multiplexing means, 2 ... Receiving means, 2A ... Demultiplexing means, 3 ... Storage means,
4 selection means, 5 operation means, 6 image creation means, 7 position recognition mechanism, 8, 24 display means, 9, 98 image processing apparatus, 10 ... 3D polygon, 23 ... CCD camera, 25 ... CC
D imaging device, 26 LCD for right eye display (first image display element), 27 LCD for left eye display (second image display element), 32 vertical transfer unit ( Charge transfer unit), 33: horizontal transfer unit, 50: two-dimensional bar code, 66: control means, 88: control device, 89
... Jog dial (operation means), 100 ... Transmission / reception system, 200, 400 ... 3D polygon broadcasting system, 201 ... 3D polygon transmission device (transmission means), 202 ... 3D polygon reception device (reception) Means), 203, 206, 207: special glasstron, 204, 208: TV monitor, 205: game device, 300: video encoder

Continued on the front page F term (reference) 5B050 AA08 CA05 CA08 DA07 EA19 EA28 FA02 FA06 5C061 AA01 AA29 AB01 AB08 AB11 AB12 AB16 AB20 AB24 5C082 AA02 AA27 BA41 BA46 BB01 BB03 BB44 CB01 DA22 DA26 DA87 MM05 MM10

Claims (55)

[Claims] A transmitting means for transmitting at least a plurality of pieces of video material information prepared in advance to create a three-dimensional video from a transmitting side to a viewer side; and the video material information transmitted from the transmitting means. Receiving means for receiving the video material information, selecting means for selecting arbitrary video material information from the video material information received by the receiving means, and a video for creating a three-dimensional video based on the video material information selected by the selecting means A three-dimensional video transmission / reception system, comprising: a creation unit, wherein the selection of the video material information is performed by a viewer. 2. The transmitting device according to claim 1, wherein the transmitting device and the receiving device are provided, wherein the transmitting device includes a multiplexing unit that multiplexes two or more pieces of video material information for creating a three-dimensional video. 2. The transmission / reception system according to claim 1, wherein the receiving unit includes an information separating unit that separates two or more pieces of video material information from the multiplexed information transmitted from the transmitting unit. 3. The three-dimensional video transmission / reception system according to claim 1, wherein the video material information is transmitted to a viewer using a digital television broadcasting network. 4. The three-dimensional video transmission / reception system according to claim 1, wherein the video material information is transmitted to a viewer using a cable television communication line. 5. The system according to claim 1, wherein the video material information is transmitted to a viewer using an Internet communication line. 6. The three-dimensional video transmission / reception system according to claim 1, wherein the video material information is transmitted as a data stream having an MPEG data sequence which is a moving image compression standard. 7. The three-dimensional video transmission / reception system according to claim 1, wherein the video material information is transmitted from the transmission side to the reception side using a plurality of channels. 8. The method according to claim 7, wherein the video material information is transmitted from the transmitting side to the receiving side using a plurality of channels, and the video material information is allocated to the plurality of channels. 3D video transmission and reception system. 9. The image material information includes at least background information representing a background of the image, object information representing an object moving in the background of the image, initial arrangement information of the object with respect to the background, and The 3D image transmission / reception system according to claim 1, wherein position information associated with the movement of the object and operation information associated with the operation of the object are added. 10. The transmission / reception system according to claim 9, wherein channels for transmitting the video material information are prepared in advance by the number of backgrounds of the video. 11. A case in which the video material information is transmitted from a transmission side to a reception side using a plurality of channels, wherein the operation information and other background information, object information, arrangement information, and position information are The three-dimensional image according to claim 9, wherein the motion information is transmitted in real time, and the object information, the arrangement information, and the position information are transmitted together with background information of a next image. Video transmission and reception system. 12. The motion information, background information, object information,
10. The three-dimensional video transmission / reception system according to claim 9, wherein when a group of data of the arrangement information and the position information has been fetched, the receiving side surrenders the channel to fetch data of another group. . 13. The two-dimensional video information relating to a normal television broadcast using one of the channels, wherein the video material information is transmitted from a transmission side to a reception side using a plurality of channels. The transmission / reception system of a three-dimensional image according to claim 7, wherein the transmission is performed. 14. A case where the video material information is transmitted from a transmitting side to a receiving side using a plurality of channels, wherein one of the channels is used as the video material information using video information from a viewpoint viewed from a subject. The transmission / reception system of a three-dimensional image according to claim 7, wherein the system is used to transmit to a receiving side. 15. The transmission / reception of a three-dimensional image according to claim 14, wherein the plurality of the objects exist in the background of the image, and the channel is assigned to each viewpoint image information of the object. system. 16. When receiving video material information including two-dimensional and three-dimensional video information sent from the transmitting side to the receiving side, the first three-dimensional video information displays an initial screen of the video. The three-dimensional video transmission / reception system according to claim 1, wherein the acquisition is performed during a period during which the advertisement screen is displayed or during a period when the advertisement screen is displayed in the two-dimensional video information. 17. A method for transmitting the video material information from a transmission side to a reception side using a plurality of channels, wherein a storage means for temporarily recording the video material information is provided on the reception side. The three-dimensional video transmission / reception system according to claim 1. 18. The system according to claim 17, wherein said storage means has a memory capacity corresponding to background information. 19. The three-dimensional video information frequently used in the video material information is advertised on the transmitting side during a period during which an initial screen of the video is displayed or in the two-dimensional video information. 18. The three-dimensional image according to claim 17, wherein the image data is transmitted while being allocated to all channels during a period in which a screen is displayed, and the receiving side stores the three-dimensional image information acquired during the period in a storage unit. Video transmission and reception system. 20. When the amount of video material information constituting one scene of a video becomes enormous, channels are increased to divide the video material information and transmit to a receiving side. Item 18. A three-dimensional video transmission / reception system according to item 17. 21. If the amount of video material information constituting one scene of a video is enormous, the remaining channels are used during transmission of the divided video material information. 18. The three-dimensional video transmission / reception system according to claim 17, wherein material information is transmitted to a receiving side, and the receiving side stores the video material information acquired during the period in a storage unit. 22. Regarding the video material information, all of the background information of the video that has or has not changed is transmitted from the transmitting side to the receiving side, and the background information that is not used immediately on the receiving side is the same as the background information. 2. The method according to claim 1, wherein the information is temporarily stored in a storage unit.
A three-dimensional video transmission / reception system according to claim 7. 23. Regarding the video material information, a plurality of light sources having different blinking patterns are attached to a specific surface of an arbitrary imageable object, and the light sources having different blinking patterns are imaged so as to flow in a predetermined imaging direction. 3. The three-dimensional image according to claim 1, wherein the position information of the object is obtained by performing image processing on the brightness information of the light source that has been imaged to obtain position information of each of the light sources. Transmission and reception system. 24. The three-dimensional video transmission / reception system according to claim 1, further comprising an object shape acquisition unit for acquiring three-dimensional shape information of the object with respect to the video material information. 25. A scanning device comprising: a light source for generating slit light for irradiating the object; and a plurality of prisms and a liquid crystal shutter for scanning the object with the slit light from the light source. Means, and light detecting means for detecting, for each pixel, slit light reflected from the object, wherein the light detecting means measures the timing of the slit light passing through each of the pixels P. An angle α between the reference plane and the slit light, which is an emission angle of the slit light by the prism with the liquid crystal shutter opened, is determined.An angle between the reference plane and the line of sight of the pixel P is β,
When the offset distance between the reference plane and the liquid crystal shutter at the center position of the scanning unit is A, and the offset distance between the pixel P and the liquid crystal shutter at the center position of the scanning unit is B, the reference plane and the object The three-dimensional shape information Z is expressed by the following equation:
That is, The three-dimensional video transmission / reception system according to claim 24, wherein: 26. The object shape obtaining means, comprising: a light source for generating slit light for irradiating the object; a scanning means having a mirror for scanning the object with slit light from the light source; Light detecting means for detecting slit light reflected from each pixel for each pixel, wherein the light detecting means measures the timing of the slit light passing through each pixel P, and the polarization angle of the mirror at that time An angle α between the reference plane and the slit light is determined, and an angle between the reference plane and the line-of-sight direction of the pixel P is β,
Let the offset distance between the reference plane and the center of rotation of the mirror be A
When the offset distance between the pixel P and the center of rotation of the mirror is B, the distance information Z between the reference plane and the object
Is given by the following equation: The three-dimensional video transmission / reception system according to claim 24, wherein: 27. A multi-view stereo camera having a reference camera for photographing a reference image of the object and a detection camera for photographing a reference image of the object is used as the object shape acquiring means. 25. The three-dimensional video transmission / reception system according to claim 24, wherein the three-dimensional shape information of the object is obtained based on parallax information obtained from the reference camera and the detection camera. 28. The transmission / reception of a three-dimensional video according to claim 1, wherein the video generation unit is configured to generate a stereo image serving as a basis of the three-dimensional video based on the video material information. system. 29. The virtual video camera according to claim 2, wherein the video material information is represented by software for creating three-dimensional video information.
9. The three-dimensional video transmission / reception system according to 8. 30. A display unit for displaying the three-dimensional video is provided, and in any screen that can be imaged by the display unit, at least three or more light source images are provided at a specific position in the screen. The three-dimensional video transmission / reception system according to claim 1, wherein the display is controlled and blinking is performed so that each blinking pattern of the light source image is different. 31. An apparatus according to claim 30, wherein said display means is provided, and image processing means for synthesizing and displaying said light source image on an arbitrary reception screen displayed on said display means is provided. 3. The transmission / reception system for a three-dimensional image according to item 1. 32. A case in which a light source image is displayed on the display means, wherein a film having an arbitrary transmission characteristic is mounted on a screen of a display portion of the light source image, and a specific light emitted from the light source image is specified. 31. The three-dimensional image transmitting / receiving system according to claim 30, wherein a wavelength is transmitted. 33. A position recognition unit for recognizing a position of the display unit, wherein a light source image in a screen displayed by the display unit is imaged so as to flow in a predetermined imaging direction. 31. The three-dimensional video transmission / reception system according to claim 30, wherein the luminance information is subjected to image processing to obtain position information of each of the light source images. 34. At least n displayed in black on a white background within a specific display period of an arbitrary display image of the display means.
A case where a two-dimensional matrix code image including a monochrome matrix of rows × n columns and a black frame having the same thickness as the monochrome matrix is displayed, wherein the two-dimensional matrix code image is captured, and the captured two-dimensional matrix code image is displayed. 4. The apparatus according to claim 3, wherein the position of the display means is recognized by performing image processing on a luminance signal based on the matrix code image to obtain position information of four corners of the two-dimensional matrix code image.
0. A three-dimensional video transmission / reception system according to item 0. 35. A synthesizing means for synthesizing an image of a virtual body on a reference plane of a virtual space including the display means, wherein the virtual body image is stereoscopically added to a two-dimensional television image viewed by the viewer. 31. The three-dimensional video transmission / reception system according to claim 30, wherein the three-dimensional video is transmitted and synthesized. 36. An image pickup means for picking up a two-dimensional television image of the display means, and one of a stereo image synthesized from a television image and a previously created virtual body image by the display means. A head mounted display having a first image display element and a second image display element for displaying the other of the stereo images, wherein the head mounted display is mounted on the face or head of a viewer, The three-dimensional image according to claim 35, wherein a stereo image by the first image display element and a stereo image by the second image display element are superimposed and guided to an eyeball of a viewer. Video transmission and reception system. 37. A synthesizing unit, comprising: a liquid crystal shutter for opening and closing incident light for capturing a television image of a display unit viewed by a viewer; and displaying an image of a virtual body created for synthesizing with the external world image. An image display element, and an image of a virtual object by the image display element, and a head mounted display having an optical unit that guides a television image of the display unit that has passed through the liquid crystal shutter to an eyeball of the viewer, The head-mounted display is mounted on the face or head of a viewer, and when the liquid crystal shutter is opened, an image of a virtual body by the image display element is displayed on a television image of the display unit that has passed through the liquid crystal shutter. 36. The three-dimensional video transmission / reception system according to claim 35, wherein the system is superimposed and guided to a viewer's eyeball. 38. A method according to claim 38, wherein the movement of the background or the object of the image related to the image material information is changed based on viewpoint position detection information obtained by the viewer wearing the synthesizing unit adjusting the viewpoint to a specific position. Claim 3
6. The three-dimensional video transmission / reception system according to 5. 39. A motion of an object in a video is determined based on a plurality of pieces of motion information transmitted to a receiving side during a period in which a background of one screen of the video is displayed, and viewpoint position detection information by the synthesizing unit. 39. The three-dimensional video transmission / reception system according to claim 38, wherein display control for changing is performed. 40. An operation means for operating the selection means, wherein a motion of a background or an object of the video related to the video material information is changed based on operation information obtained by operating the operation means. The three-dimensional video transmission / reception system according to claim 1, wherein: 41. A motion of an object in a video is changed based on a plurality of pieces of motion information transmitted to a receiving side during a period for displaying a background of one screen of the video and operation information by the operation means. 41. A display control is performed.
3. The transmission / reception system for a three-dimensional image according to item 1. 42. The method according to claim 1, wherein the change of the background of the image or the motion of the object according to the image material information is performed when an interrupt occurs within a predetermined cycle time.
3. The transmission / reception system for a three-dimensional image according to item 1. 43. The three-dimensional video transmitting / receiving system according to claim 42, wherein a video based on default three-dimensional video information is displayed except for an interrupt within the cycle time. 44. A two-way service network, wherein communication means is connected between the transmission side and the viewer side, and wherein the communication means transmits individual information by the viewer from the viewer side to the transmission side. The three-dimensional video transmission / reception system according to claim 1, wherein the system is constructed. 45. At least a plurality of pieces of video material information for creating a three-dimensional video are prepared in advance on the transmission side, and the video material information prepared on the transmission side is sent from the transmission side to individual viewers. Transmitting, the viewer receives the video material information sent from the transmitting side, and the viewer selects arbitrary video material information from the video material information to create a three-dimensional video. 3
How to send and receive 3D images. 46. The method of claim 45, wherein the video material information is multiplex-transmitted from the transmitting side to the receiving side using a plurality of channels. 47. The apparatus according to claim 4, wherein the video material information is all prepared on the transmission side.
6. The method for transmitting and receiving a three-dimensional video according to 5. 48. The method according to claim 45, wherein a part of the video material information is prepared on a receiving side. 49. A case in which the video material information is prepared on a transmission side, wherein the video material information is provided in a coarse reproduction mode for reproducing a three-dimensional video constituting one screen in a “coarse” or “dense” mode. 48. The method of claim 47, wherein one of the fine reproduction modes for reproduction is selected on the receiving side. 50. The three-dimensional video transmission / reception method according to claim 46, wherein a fee is charged to a viewer based on the number of channels used and the channel usage time for the fee setting accompanying the use of the channel. 51. Regarding the fee setting for the viewer,
The method of claim 50, wherein charging is performed according to the amount of video material information within the reception time. 52. The three-dimensional video transmission / reception method according to claim 46, wherein a fee is charged to a program provider based on the number of channels used and the channel usage time for the fee setting accompanying the use of the channel. 53. The three-dimensional video transmission / reception method according to claim 52, wherein the fee setting for the program provider is charged according to the amount of video material information within a transmission time. 54. A case where a two-way service network for transmitting individual information by the viewer from the viewer side to the transmission side using communication means is constructed, wherein shopping information is transmitted to the video material information. The method for transmitting and receiving a three-dimensional image according to claim 45, wherein: 55. The case where the interactive service network is constructed, wherein when a viewer performs television shopping on the shopping information, order information relating to the shopping information is transferred to a program provider. The three-dimensional video transmission / reception method according to claim 54, further comprising adding a product price related to the shopping information to a fee associated with using a channel of a viewer.