JP2000149039A - Transmitter, transmitting method, receiver, receiving method and communication system using multidimensional stream data - Google Patents

Abstract

PROBLEM TO BE SOLVED: To transmit motion data of a CG character in real time by providing a multidimensional stream data transmitting means for transmitting multidimensional stream data. SOLUTION: Time series multidimensional stream data are transmitted by a multidimensional stream data transmitting means 101, the data are received by a multidimensional stream data receiving means 102, and a CG scene is reproduced based on the received multidimensional stream data. The multidimensional stream data are provided with time information and plural- dimensional space information. Any of cable like telephone wire and radio may be used as a transmission line on which the multidimensional stream data are transmitted. In addition, a personal computer and a digital television set may be permitted as the multidimensional stream data receiving means 102 as long as it is provided with a display. Since CG data are transmitted as stream data, the original CG scene is reproduced easily, even when a part of the stream data is missing.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to stream data for forming a three-dimensional virtual space represented by three-dimensional computer graphics (hereinafter referred to as three-dimensional CG or 3DCG) based on a network such as the Internet. Related to a transmitting / receiving device.

[0002]

2. Description of the Related Art Recently, as a field of application of 3DCG, WW
Virtual malls (Virtual Mall) and electronic transactions (Electric Commerc) on the Internet such as W (World Wide Web)
e) and various related homepages are receiving attention. In particular, due to the rapid development of the Internet, an environment for easily handling relatively high-quality 3DCG such as games and movies in a home is being prepared. In the conventional WWW, a machine called a server such as a personal computer or a workstation is connected to a machine called a plurality of clients such as a personal computer via the Internet. Then, the client downloads data such as image, sound, text and arrangement information provided by the server, and reconstructs the data on the client side, so that the user can obtain necessary information. This communication between the server and the client
TCP / IP (Transmission Control / Internet Protocol)
ol).

Conventionally, data provided from the server side is:
Mainly text data and image data only, but recently VRML (Virtual Reality Modeling L
anguage) and VRML browsers have been standardized, and there is a movement to transfer 3DCG data itself such as shapes and / or scenes.

Here, the VRML will be briefly described.

[0005] HTML (Hyper Text Markup Language)
In a conventional data format mainly composed of images and texts as described above, enormous transfer time and transfer cost are required for transferring image data, particularly, moving image data. Therefore, in the current system, there is a restriction on network traffic. In contrast, in the conventional 3DCG, all of the viewpoint information and light source information including the shape are CG (Compu
(ter Graphics) data was processed using three-dimensional data obtained by calculation. The receiving side generates image data based on the three-dimensional data and displays it.

[0006] As the CG technology advances, the image quality of an image created by CG is rapidly improved, and it is much more efficient to transfer CG data as it is in terms of the data amount.

Normally, when CG data is used, data can be compressed at a data compression ratio of 1/100 or more as compared with a case where equivalent image data is transferred. Therefore, there is a movement to standardize a method of transferring 3DCG data via a network. As one approach, standardization of 3DCG data called VRML has been proposed (VRML Ver2.0). VRML Ver2.0 defines data formats such as shape data called primitives, various types of light source data, viewpoint data, and texture data, and a method of specifying rigid body movement.

On the other hand, an animation technique for generating an image in real time has recently attracted attention in the CG field. By using this real-time animation technique, a device has been devised to reproduce the real movement of the CG character mainly in commercials and movies. As one of them, a complex shape such as a human is represented by a skeletal structure, and a complex movement can be naturally reproduced by defining a movement amount of a joint of the skeleton which changes every moment.

[0009]

SUMMARY OF THE INVENTION However, VRM
In the conventional 3DCG modeling language on the Internet centered on L, there is a problem that an operation cannot be set in real time in a shape having a complicated structure such as a human.

Another problem is that it is not possible to transmit and receive real-time motion data of a CG character (such as a human) that moves realistically via a narrow-band network such as a telephone line.

Therefore, the present invention has been made in view of the above problems, and
It is an object of the present invention to provide a multidimensional stream data transmitting / receiving apparatus capable of transmitting and receiving character motion data in real time.

[0012]

The transmitting apparatus of the present invention includes multidimensional stream data transmitting means for transmitting multidimensional stream data, thereby achieving the above object.

[0013] The transmission device may include a multi-dimensional stream data identification unit that classifies the input data into a plurality of types of operation data, and a multi-dimensional stream data compression unit that compresses the plurality of types of operation data in block units. May be provided.

[0014] The multidimensional stream data compression means includes:
The compression ratio may be changed according to the type of the operation data.

The multidimensional stream data compression means includes:
A compression method may be selected according to the type of the operation data.

[0016] The multidimensional stream data compression means includes:
At least one of a plurality of types in a predetermined period
The plurality of types of motion data may be compressed in units of blocks by generating a node sequence that is time-series data including two types of motion data and extracting an arbitrary node from the node sequence. .

The multidimensional stream data transmitting means includes:
Interpolation information necessary for reproducing the extracted nodes and the node sequence may be transmitted.

The multidimensional stream data compression means includes:
The plurality of types of operation data may be compressed in block units by reducing high-frequency components of the plurality of types of operation data.

The multidimensional stream data compression means includes:
The plurality of types of operation data may be compressed in units of blocks by performing coordinate transformation on at least one type of the plurality of types of operation data.

The multidimensional stream data compression means includes:
The plurality of types of motion data may be compressed in blocks by converting data represented by three-dimensional vector coordinates into data represented by polar coordinates among the plurality of types of motion data.

[0021] The multidimensional stream data may include the type of the multidimensional stream data to be transmitted, the order of the plurality of types of operation data to be transmitted, and the order of the plurality of types of operation data to be transmitted. It may have header information or stream format description data describing the length.

[0022] The multidimensional stream data may include time information and multidimensional spatial information.

[0023] The multi-dimensional spatial information may include position information, rotation information, and scale information.

[0024] The multi-dimensional stream data may include data indicating a position, a rotation, and a scale of the CG scene.

[0025] The multi-dimensional stream data capable of representing a CG scene may be multi-dimensional animation stream data.

The multi-dimensional stream data has a data portion having a plurality of channels, and a header portion describing a stream data format of the data portion, and each of the plurality of channels has a three-dimensional data at a certain time. It may have a plurality of pieces of information about the CG character.

[0027] The three-dimensional CG character can be represented by a hierarchical structure having at least one joint, and each of the plurality of motion data has a dimension of a position vector of a reference position of the three-dimensional CG character. ,
Data having a dimension of a direction vector representing a posture at the reference position, data having an angle vector of the at least one joint and dimensions of a rotation angle around the angle vector,
At least one of the stream data having a dimension of a rotation angle about a local coordinate axis of the at least one joint may be included.

[0028] The multidimensional stream data may be a packet having a header section and a data section, and the header section may indicate the type of the packet, the order of the packet, and the length of the data section.

The receiving device includes a multidimensional stream data holding unit that holds the data compressed by the multidimensional stream data compressing unit in a multi-layer structure, and the transmitting device transmits the multidimensional stream data. From the data held in the multiplex hierarchical structure, the data held in a certain hierarchical structure according to the state of the transmission path to be applied and the reproduction speed at which the receiving device receiving the multidimensional stream data reproduces the multidimensional stream data May be selected and the selected data may be transmitted.

[0030] The transmitting apparatus may include at least one of a state of a transmission path through which the multidimensional stream data is to be transmitted, and a reproduction speed at which a receiving apparatus that receives the multidimensional stream data reproduces the multidimensional stream data.
Depending on the type, the data having the largest data amount that can be reproduced may be selected from the data held in the multi-layer structure.

[0031] The transmitting apparatus may further include a multi-dimensional stream data holding means for holding the data compressed by the multi-dimensional stream data compressing means in a file format of the multi-dimensional stream data.

[0032] The transmission method of the present invention includes a multidimensional stream data transmitting step of transmitting multidimensional stream data, whereby the above object is achieved.

The transmission method includes a multidimensional stream data identification step of classifying input data into a plurality of types of operation data, and a multidimensional stream data compression step of compressing the plurality of types of operation data in block units. May be included.

The multi-dimensional stream data compression means includes:
The compression ratio may be changed according to the type of the operation data.

The multi-dimensional stream data compression step includes:
A compression method may be selected according to the type of the operation data.

The multi-dimensional stream data compression step includes:
At least one of a plurality of types in a predetermined period
The plurality of types of motion data may be compressed in units of blocks by generating a node sequence that is time-series data including two types of motion data and extracting an arbitrary node from the node sequence. .

[0037] The transmission method may include a step of transmitting interpolation information necessary for reproducing the extracted nodes and the node sequence.

The multi-dimensional stream data compression step includes:
The plurality of types of operation data may be compressed in block units by reducing high-frequency components of the plurality of types of operation data.

The multi-dimensional stream data compression step includes:
The plurality of types of operation data may be compressed in units of blocks by performing coordinate transformation on at least one type of the plurality of types of operation data.

The multi-dimensional stream data compression step includes:
The plurality of types of motion data may be compressed in blocks by converting data represented by three-dimensional vector coordinates into data represented by polar coordinates among the plurality of types of motion data.

[0041] The multi-dimensional stream data includes a type of the multi-dimensional stream data to be transmitted, an order of the plurality of types of operation data to be transmitted, and an order of the plurality of types of operation data to be transmitted. It may have header information or stream format description data describing the length.

[0042] The multidimensional stream data may include time information and multidimensional spatial information.

The multi-dimensional spatial information may include position information, rotation information, and scale information.

[0044] The multi-dimensional stream data may include data indicating the position, rotation, and scale of the CG scene.

[0045] The multidimensional stream data capable of representing a CG scene may be multidimensional animation stream data.

The multi-dimensional stream data has a data section having a plurality of channels and a header section describing a stream data format of the data section, and each of the plurality of channels is a three-dimensional CG at a certain time. It may have a plurality of pieces of information about the character.

The three-dimensional CG character can be represented by a hierarchical structure having at least one joint, and each of the plurality of motion data has a dimension of a position vector of a reference position of the three-dimensional CG character. ,
Data having a dimension of a direction vector representing a posture at the reference position, data having an angle vector of the plurality of joints and a dimension of a rotation angle around the plurality of joints, and data about local coordinate axes of the plurality of joints; It may have at least one of the stream data having the dimension of the rotation angle.

[0048] The multidimensional stream data may be a packet having a header section and a data section, and the header section may indicate the type of the packet, the order of the packet, and the length of the data section.

[0049] The transmission method includes a multidimensional stream data holding step of holding the data compressed by the multidimensional stream data compression means in a multi-layer structure.
The transmission method is configured to perform, based on a state of a transmission path through which the multidimensional stream data is to be transmitted, and a reproduction speed for reproducing the multidimensional stream data, a certain layer from the data held in the multiplex hierarchical structure. The data held in the structure may be selected, and the selected data may be transmitted.

[0050] In the transmission method, at least one of a state of a transmission path through which the multidimensional stream data is to be transmitted and a reproduction speed at which a receiving device that receives the multidimensional stream data reproduces the multidimensional stream data may be used.
Depending on the type, the data having the largest data amount that can be reproduced may be selected from the data held in the multi-layer structure.

The transmission method may further include a multidimensional stream data holding step of holding the data compressed in the multidimensional stream data compression step in a file format of the multidimensional stream data.

[0052] The receiving apparatus of the present invention includes multidimensional stream data receiving means for receiving multidimensional stream data, thereby achieving the above object.

The multidimensional stream data identification means for identifying the data content of the multidimensional stream data from one of the header information and the stream format description information possessed by the multidimensional stream data; A multi-buffering means for sequentially reading data into a plurality of multi-buffers in block units, and a multi-dimensional stream data decompression means for decompressing the blocked multi-dimensional stream data in block units, thereby achieving the above object. You.

The multi-dimensional stream data identification means includes:
It may be determined whether the received multidimensional stream data is a channel definition packet.

[0055] The multidimensional stream data identification means includes:
It may be determined whether the received multidimensional stream data is a data packet.

When the multi-dimensional stream data identifying means determines that the received multi-dimensional stream data is a channel definition packet, the multi-dimensional stream data identifying means determines from the received multi-dimensional stream data, A time stamp, a total number of channels, and a compression ratio identifier indicating a compression ratio of channel data may be identified.

When the multidimensional stream data identifying means determines that the received multidimensional stream data is a data packet, the multidimensional stream data identifying means determines the time based on the received multidimensional stream data from the received multidimensional stream data. The stamp and the total number of channels may be identified.

[0058] The received multidimensional stream data may include compressed operation data and compression information.

The multi-dimensional stream data decompression means comprises:
The compressed operation data may be expanded based on the compression information.

The multi-dimensional stream data decompression means comprises:
The expansion rate may be determined based on the compression information.

The multi-dimensional stream data decompression means comprises:
The decompression method may be determined based on the compression information.

When the compressed operation data is generated by extracting arbitrary operation data from a plurality of operation data in a predetermined period, the multi-dimensional stream data decompression means includes: A plurality of pieces of operation data in the predetermined period may be restored based on the compression information.

[0063] The compression information may be interpolation information.

The multi-dimensional stream data decompression means comprises:
The compressed motion data may be expanded by performing coordinate transformation on the compressed motion data based on the compression information.

When the compressed motion data is represented by polar coordinates, the multi-dimensional stream data decompression means converts the compressed motion data represented by polar coordinates into data represented by three-dimensional vector coordinates. Thereby, the compressed operation data may be expanded.

[0066] The multidimensional stream data may include time information and multidimensional spatial information.

[0067] The multi-dimensional space information may include position information, rotation information, and scale information.

[0068] The multidimensional stream data may include data indicating the position, rotation, and scale of the CG scene.

[0069] The multidimensional stream data capable of representing a CG scene may be multidimensional animation stream data.

The multi-dimensional stream data has a data section having a plurality of channels, and a header section describing a stream data format of the data section, and each of the plurality of channels is a three-dimensional CG at a certain time. It may have a plurality of pieces of information about the character.

The three-dimensional CG character can be expressed by a hierarchical structure having at least one joint, and each of the plurality of motion data has data having a dimension of a position vector of a reference position of the three-dimensional CG character. ,
Data having a dimension of a direction vector representing a posture at the reference position, data having an angle vector of the at least one joint and dimensions of a rotation angle around the angle vector,
At least one of the stream data having a dimension of a rotation angle about a local coordinate axis of the at least one joint may be included.

[0072] The multidimensional stream data may be a packet having a header section and a data section, and the header section may indicate the type of the packet, the order of the packet, and the length of the data section.

[0073] The receiving apparatus may be configured to transmit the multi-dimensional stream data according to at least one of a state of a transmission path through which the multi-dimensional stream data is to be transmitted and a reproduction speed at which the receiving apparatus reproduces the multi-dimensional stream data. The transmission device for transmitting the dimensional stream data may be instructed on the compression ratio of the multidimensional stream data.

The receiving method of the present invention includes a multidimensional stream data receiving step of receiving multidimensional stream data, thereby achieving the above object.

A multidimensional stream data identification step for identifying the data content of the multidimensional stream data from one of the header information and the stream format description information of the multidimensional stream data. The method may further include a multiplex buffering step of sequentially reading data into a plurality of multiplex buffers in block units, and a multidimensional stream data decompression step of decompressing the blocked multidimensional stream data in block units.

The multi-dimensional stream data identification step includes:
It may be determined whether the received multidimensional stream data is a channel definition packet.

The multi-dimensional stream data identification step includes:
It may be determined whether the received multidimensional stream data is a data packet.

If the multi-dimensional stream data identifying step determines that the received multi-dimensional stream data is a channel definition packet, the multi-dimensional stream data identifying step determines from the received multi-dimensional stream data A time stamp, a total number of channels, and a compression ratio identifier indicating a compression ratio of channel data may be identified.

If the multi-dimensional stream data identification step determines that the received multi-dimensional stream data is a data packet, the multi-dimensional stream data identification step determines the time based on the received multi-dimensional stream data. The stamp and the total number of channels may be identified.

[0080] The received multidimensional stream data may include compressed operation data and compression information.

The multi-dimensional stream data decompression step comprises:
The compressed operation data may be expanded based on the compression information.

The multi-dimensional stream data decompression step comprises:
The expansion rate may be determined based on the compression information.

The multi-dimensional stream data decompression step comprises:
The decompression method may be determined based on the compression information.

When the compressed operation data is generated by extracting arbitrary operation data from a plurality of operation data in a predetermined period, the multi-dimensional stream data decompression step includes the steps of: A plurality of pieces of operation data in the predetermined period may be restored based on the compression information.

[0085] The compression information may be interpolation information.

The multidimensional stream data decompression step comprises:
The compressed motion data may be expanded by performing coordinate transformation on the compressed motion data based on the compression information.

If the compressed motion data is represented by polar coordinates, the multi-dimensional stream data decompression step converts the compressed motion data represented by polar coordinates into data represented by three-dimensional vector coordinates. Thereby, the compressed operation data may be expanded.

[0088] The multidimensional stream data may include time information and multidimensional spatial information.

[0089] The multidimensional space information may include position information, rotation information, and scale information.

The multidimensional stream data may include data indicating the position, rotation, and scale of the CG scene.

The multi-dimensional stream data capable of representing a CG scene may be multi-dimensional animation stream data.

The multi-dimensional stream data has a data section having a plurality of channels, and a header section describing a stream data format of the data section. It may have a plurality of pieces of information about the character.

The three-dimensional CG character can be represented by a hierarchical structure having at least one joint, and each of the plurality of motion data has data having a dimension of a position vector of a reference position of the three-dimensional CG character. ,
Data having a dimension of a direction vector representing a posture at the reference position, data having an angle vector of the plurality of joints and a dimension of a rotation angle around the plurality of joints, and data about local coordinate axes of the plurality of joints; It may have at least one of the stream data having the dimension of the rotation angle.

The multidimensional stream data may be a packet having a header section and a data section, and the header section may indicate the type of the packet, the order of the packet, and the length of the data section.

[0095] The receiving method may be arranged such that the multi-dimensional stream data is transmitted in accordance with at least one of a state of a transmission path through which the multi-dimensional stream data is to be transmitted and a reproduction speed at which the receiving apparatus reproduces the multi-dimensional stream data. The transmission means for transmitting the dimensional stream data may be instructed on the compression ratio of the multidimensional stream data.

The multidimensional stream data includes operation data and audio data, the operation data has time stamp information, the audio data has time stamp information, and the receiving device has the time stamp of the operation data. The operation data and the audio data may be reproduced in synchronization with the information and the time stamp information of the audio data.

The multi-dimensional stream data includes operation data and audio data, the operation data has time stamp information, the audio data has time stamp information, and the time stamp information and the audio data of the operation data. According to the time stamp information, the operation data and the audio data may be reproduced in synchronization with each other.

The computer-readable recording medium is
A program for causing a computer to execute the transmission method may be recorded.

The computer-readable recording medium is
A program for causing a computer to execute the receiving method may be recorded.

The communication system according to the present invention comprises: a multidimensional stream data transmitting unit for transmitting multidimensional stream data;
A multidimensional stream data receiving unit for receiving the multidimensional stream data, whereby the above object is achieved.

The multidimensional stream data transmitting section is one of an artificial satellite, a terrestrial broadcasting station, and a cable broadcasting station, and the multidimensional stream data receiving section is one of a digital television and a set top box. It may be one.

In the transmitting device, the multidimensional stream data may include data capable of representing a CG scene.

In the transmission method, the multidimensional stream data may include data capable of representing a CG scene.

In the receiving device, the multidimensional stream data may include data capable of representing a CG scene.

In the receiving method, the multidimensional stream data may include data capable of representing a CG scene.

In the communication system, the multidimensional stream data may include data capable of representing a CG scene.

[0107]

Embodiments of the present invention will be described with reference to the drawings.

(Embodiment 1) Hereinafter, a first embodiment of the present invention will be described with reference to FIGS.

The first embodiment aims at reproducing multidimensional stream data at high speed and with stable quality by transmitting and / or receiving multidimensional stream data. The multidimensional stream data is
It may include data representing a CG scene. Where CG
Data representing a scene includes geometric information such as object information, light source information, and viewpoint information.

FIG. 1 is a diagram showing an example of a CG scene displayed on the display device. FIG. 1 shows that the table is located in a room surrounded by walls.

FIG. 2 is a diagram showing a hierarchical structure when the CG scene shown in FIG. 1 is represented by an example of multidimensional stream data. As shown in FIG. 2, a room as a route has a wall and a table hanging therefrom. If there is a cup on the table, the cup hangs from the table. The room, the wall, and the table each have data for configuring them. In the example shown in FIG. 2, each of the room, the wall, and the table is a position,
It has data such as rotation and scale.

FIG. 3 shows a system 100 for transmitting and receiving multidimensional stream data according to the first embodiment.
FIG.

The system 100 shown in FIG. 3 includes a multidimensional stream data transmitting means 101 and a multidimensional stream data receiving means 102. The multidimensional stream data transmitting means 101 transmits time-series multidimensional stream data, and the multidimensional stream data receiving means 102 receives time-series multidimensional stream data. The multidimensional stream data receiving means 102 may reproduce a CG scene based on the received multidimensional stream data.

Here, the multidimensional stream data has time information and multidimensional spatial information, and time and space are called multidimensional. The multi-dimensional space information may include position information, rotation information, and scale information of the CG character. Alternatively, the multidimensional stream data is
It may have data indicating the position, rotation and scale of the CG character. Further, the multidimensional stream data may be multidimensional animation stream data indicating animation of a CG character.

The transmission path through which the multidimensional stream data is transmitted is not limited to a wire such as a telephone line, but may be wireless. Also, a multidimensional stream data receiving means 1
02 may be a personal computer or a digital television as long as it has a display device.

In this embodiment, CG data is transmitted as stream data. Therefore, even if a part of the stream data is missing, the original CG scene can be easily reproduced.

Note that the data structure of the multidimensional stream data may be the same as that of the second embodiment described later.

In the first embodiment, the multidimensional stream data transmitting means 101 may be an artificial satellite, and the multidimensional stream data receiving means 102 may be a digital television.

In the first embodiment, the multidimensional stream data transmitting means 101 may be an artificial satellite, and the multidimensional stream data receiving means 102 may be a set-top box.

Note that the artificial satellite may receive multidimensional stream data from a ground station.

In the first embodiment, the multidimensional stream data transmitting means 101 is a terrestrial broadcasting station,
The multidimensional stream data receiving means 102 may be a digital television.

In the first embodiment, the multidimensional stream data transmitting means 101 is a terrestrial broadcasting station,
The multidimensional stream data receiving means 102 may be a set-top box.

Further, in the first embodiment, the multidimensional stream data transmitting means 101 may be a cable broadcasting station, and the multidimensional stream data receiving means 102 may be a digital television.

In the first embodiment, the multidimensional stream data transmitting means 101 may be a cable broadcasting station, and the multidimensional stream data receiving means 102 may be a set top box.

[0125] A program for causing a computer to execute the above-described method for transmitting and / or receiving multidimensional stream data may be recorded on a computer-readable recording medium.

(Embodiment 2) Hereinafter, a second embodiment of the present invention will be described with reference to FIGS.

The second embodiment aims at transmitting multi-dimensional stream data at higher speed by compressing motion data based on a CG tree structure to represent a CG scene. .

FIG. 4 is a diagram showing the multidimensional stream data transmitting / receiving apparatus 10.

The transmitting means 1 transmits the multi-dimensional stream data, and the transmitted multi-dimensional stream data is received by the receiving means 2
Receive.

Here, the multidimensional stream data has time information and multidimensional spatial information, and time and space are called multidimensional. The multi-dimensional space information may include position information, rotation information, and scale information of the CG character. Alternatively, the multidimensional stream data is
It may have data indicating the position, rotation and scale of the CG character. Further, the multidimensional stream data may be multidimensional animation stream data indicating animation of a CG character.

The multi-dimensional stream data transmitting / receiving apparatus 10 shown in FIG. 4 includes a transmitting means 1, a receiving means 2, a multi-dimensional stream data storing means 31, and a multi-dimensional stream data storing means 32.

Each of the transmitting means 1 and the receiving means 2 is realized by a computer device represented by a workstation, a personal computer, and a game machine, respectively.

The transmitting means 1 may have a stream authoring means (not shown), or a stream authoring means (not shown) may be provided outside the transmitting means 1. According to the user's instructions, the stream authoring means (not shown)
The three-dimensional CG stream data representing the three-dimensional virtual space is generated. The operation of the stream authoring means will be described later.

The transmitting means 1 comprises a multidimensional stream data identifying means 11 and a multidimensional stream data compressing means 1.
2 and multi-dimensional stream data transmission means 13
The receiving means 2 includes a multi-dimensional stream data identifying means 21,
Multiple buffering means 22, multidimensional stream data decompression means 23, and multidimensional stream data receiving means 2
4

First, the flow of a series of processes in the transmitting means 1 will be described.

An example of identification of multidimensional stream data in the transmitting means 1 will be described. As an example of a multidimensional stream,
Hereinafter, a case will be described in which motion data of a three-dimensional CG character is transmitted.

FIG. 5 is a diagram showing an example of a skeleton structure of a three-dimensional CG character.

The skeleton of the three-dimensional CG character shown in FIG. 5 is a skeleton of a humanoid, and the skeleton has 22 joints. The number 1 of the joint indicates a route.

[0139] The multidimensional stream data identification means 11 in the transmission means 1 classifies a plurality of input data input to the transmission means 1.

FIG. 6 is a diagram showing an example of classification of input data to be input. Multidimensional stream data identification means 1
1 classifies input data into a plurality of types of motion data of three-dimensional CG characters and audio data SS. The plurality of types of motion data of the three-dimensional CG character obtained by classifying the input data include a position vector MS1 of the reference position of the three-dimensional CG character in the three-dimensional space, a direction vector MS2 representing the posture at the reference position, and a route. Includes information MS3 on the angle vector at each joint and the rotation angle around it.

When the three-dimensional CG character has the skeleton as described above, MS1 is a route position vector, MS2 is a route direction vector and a rotation angle around the direction vector, and MS3 is a joint vector. And a rotation angle around the rotation axis vector. The general expression of the hierarchical structure of a three-dimensional CG character is described in Japanese Patent Application No. 9-10 / 1994 by the present applicant.
No. 0458. Also, data other than the operation data and the audio data SS from the input data,
Classification may be performed by the multidimensional stream data identification unit 11.

The multi-dimensional stream data compressing means 12 in the transmitting means 1 transmits the classified operation data (MS
1 to MS3), an individual compression method is applied according to the CG tree structure and / or the type of operation data, and the operation data (MS1 to MS3) is compressed in block units. For example, multidimensional stream data compression means 1
2 changes the compression ratio according to the type of operation data.

The multidimensional stream data compression method by the multidimensional stream data compression means 12 will be described with reference to FIGS.

The movement data (MS1 to MS1) of the three-dimensional CG character classified by the multidimensional stream data identification means 11
MS3), multi-dimensional stream data compression means 1
2, blocks motion data in the time axis direction so as to form one block in a given period (time t to time t ′), as shown in FIG. Next, the multidimensional stream data compression means 12 outputs the operation data (MS1 to MS1).
Different compression methods are applied depending on the type of 3).

The multi-dimensional stream data compression means 12
A blocking method for compressing the operation data (MS1 to MS3) in block units will be described below.

The position vector MS1 of the reference position of the three-dimensional CG character has three floating-point values indicating the positions (x, y, z) in the three-axis direction. Compression can be performed.

For example, when each component of the position (x, y, z) is data represented by 4 bytes (32 bits),
The multidimensional stream data compression means 12 performs data compression by expressing each component at those positions by 2 bytes (16 bits).

In 2 bytes, in decimal number, -32768
65,536 steps to ３32767 can be expressed. By projecting a 4-byte floating point value in 65536 steps, 4-byte data can be compressed to 2 bytes. For example, the multidimensional stream data compression means 12 searches for the maximum value and the minimum value of the input floating-point value data during a predetermined period, divides the difference between the maximum value and the minimum value into 65536 equal parts, and obtains 4-byte data. In two bytes.

Although the 4-byte data is represented by 2 bytes in the above-described example, the multidimensional stream data compression means 12 sets the component at the N-byte position to be larger than 0 (N
-1) It may be represented by the following bytes.

The direction vector MS2 representing the posture of the three-dimensional CG character is a three-dimensional vector (three components) representing the axis of rotation of the three-dimensional CG character in the local coordinate system of each layer.
And information MS3 having an angle vector at each joint and a rotation angle around the joint is a three-dimensional vector (3) representing a rotation axis in the local coordinate system of each layer of the three-dimensional CG character.
Component) and one component representing the rotation angle. The multidimensional stream data compression unit 12 outputs a three-dimensional vector representing the rotation axis among the four components,
(X, Y, Z) is normalized (the magnitude of the vector is set to 1), and the three-dimensional vector coordinates are converted to polar coordinates, whereby the three components (X, Y, Z) are converted into two components (γ ,
θ). That is, the direction vector MS2 is compressed in the spatial axis direction from three components to two components,
The information MS3 may be compressed in the spatial axis direction from four components to three components.

The multi-dimensional stream data compression means 1
2, the information about the position vector MS1 of the reference position of the three-dimensional CG character and the information MS3 having the angle vector at the joint and the rotation angle around the joint may be compressed in the time axis direction. For example, the multidimensional stream data compression unit 12 generates a node sequence from at least one of a plurality of types of motion data, for example, the position vector MS1 and the information MS3, and extracts an arbitrary node from the node sequence. , Compress a plurality of types of operation data in block units.

FIG. 9A is a diagram showing 11 nodes extracted from 30 frames / block, and FIG. 9B
FIG. 14 is a diagram showing six nodes extracted from 15 frames / block.

The nodes shown in FIGS. 9A and 9B indicate at least one of the position vector MS1 and the information MS3, and the node sequence represents at least one of them in a time-series manner.

As shown in FIG. 9A, 30 frame / b
In the lock, the nodes in the block are assigned 0 to
When numbered 29, 0, 2, 5, 8, 11, 1
Nodes corresponding to 11 points such as 4, 18, 21, 24, 27, and 29 are extracted. Only the extracted nodes are output from the multidimensional stream data compression means 12, and the other nodes are not output from the multidimensional stream data compression means 12.

Further, as shown in FIG.
In the e / block, when nodes in a block are numbered from 0 to 14 in chronological order, 0, 2, 5, 9, 1
Nodes corresponding to six points such as 2 and 14 are extracted. Only the extracted nodes are output from the multidimensional stream data compression means 12, and the other nodes are not output from the multidimensional stream data compression means 12.

When the extracted nodes are output from the multidimensional stream data compression means 12, the multidimensional stream data compression means 12 outputs interpolation information necessary for restoring the original node sequence from the extracted nodes. You may.

The number of nodes extracted per block is not limited to 11 or 6. The number of nodes to be extracted is appropriately determined by a user or the like. C to be transmitted
When it is desired to improve the image quality of the G scene, the number of extracted nodes may be increased, and when transmitting the CG scene to be transmitted at a high speed, the number of extracted nodes may be reduced.

[0158] The extracted node corresponds to the receiving means 2
Then, data interpolation is performed by cubic spline interpolation or the like. In addition to the cubic spline interpolation, another interpolation method such as linear interpolation may be used.

The multidimensional stream data compression means 1
2 may reduce at least one high-frequency component of a plurality of types of operation data and compress the operation data in block units.

Next, the multidimensional stream data storage means 3
1 will be described in detail.

The stream data (MSC1 to MSC3, SSC) compressed on a block-by-block basis is stored as multidimensional stream data in a storage device in a computer usually in a multidimensional stream data storage means 31.

Here, the format of the multidimensional stream data stored in the multidimensional stream data storage means 31 will be described in detail.

The multidimensional stream data compression means 12
The compressed operation data is re-blocked, header information for transmission as multi-dimensional stream data is added to the block, and the block and header information are stored in the multi-dimensional stream data storage unit 31.

FIG. 10 is a diagram showing an example of the format of the multi-dimensional stream data stored in the multi-dimensional stream data storing means 31.

The multidimensional stream data is composed of packets, and each packet has a header section and a data section. Hereinafter, one stream data is referred to as a channel for each type of stream data (operation data), and a channel number that is uniquely determined is added to each channel (however, the order of the numbers added here is Not at all).

The multidimensional stream data stored in the multidimensional stream data storing means 31 includes a channel definition packet and a data packet as shown in FIG.
The channel definition packet is a stream that defines the channel of the data packet following the transmission of the channel definition packet. The channel definition packet and the data packet have a header part and a data part, respectively.

The header of the channel definition packet includes, for example, a packet identifier indicating that the channel definition packet itself is a channel definition packet, a time stamp indicating the time when this channel definition packet was generated from a certain reference time, and a transmission section. The total number TC of channels to be obtained has a compression ratio identifier indicating the number of nodes obtained by extracting nodes when the stream data is compressed in block units. The data part of the channel definition packet includes a total number TC of channels. Each channel has a channel identifier indicating a channel number, a data size of the channel, a channel name, and whether the channel is a position vector or a rotation angle vector. The channel type to be represented has the number of channels TC.

As described above, a data packet has a header section and a data section.

The header part of the data packet is, for example,
A packet identifier indicating that the packet is itself a data packet, a time stamp indicating the time when the data packet was generated from a certain reference time, and the total number of channels that can be transmitted by the data packet DC (the channel in the channel definition packet) Less than the total number).

In the data portion of the data packet, the total number DC
Each channel has a channel identifier representing a channel number, channel data, and a compression ratio identifier (for example, the number of extracted nodes). Note that the number of compression ratio identifiers is the same as the number of channel data.
This data packet is equivalent to one block of multidimensional stream data.

FIG. 11 is a diagram showing a configuration example of a channel definition packet and a data packet. Once the channel definition packet is transmitted at first, only the data packet may be transmitted thereafter. When it is desired to change the channel definition in the middle of the stream, the channel definition can be easily changed by transmitting the channel definition packet in the middle.

The multidimensional stream data is shown in FIG.
It is not limited to the packet shown in. For example, it may have header information or stream format description data describing the multi-level stream data, its type, the order of the plurality of operation data, and the length of the plurality of operation data.

Next, the multidimensional stream data storage means 3
1 stores a plurality of stream data having different compression ratios, and if the amount of data flowing through the transmission path is excessive or insufficient due to the state of the transmission path between the transmission means 1 and the reception means 2,
In order to use the transmission path effectively, the transmission means 1 selects stream data having an optimum compression ratio according to at least one of the state of the transmission path, the request from the reception means 2 and the request from the user. Will be described below. It is to be noted that storing a plurality of stream data having different compression rates is referred to as a multi-layered storage method.

The transmission means 1 generates stream data compressed at a certain compression rate in the block unit, and also generates several types of stream data having a compression rate different from the compression rate of the compressed stream data. Are stored in the multidimensional stream data storage means 31.

FIG. 12 is a diagram showing an example of stream data output from the transmitting means 1 when the input data received by the transmitting means 1 is 30 frames / block.

The uppermost layer shown in FIG.
The stream data that is output as one block without compressing the input data of me / block without compression. The second hierarchical level is 30 frame / block.
Is the stream data obtained by compressing the input data of 11 frames / block, and the third hierarchical level is 30 frames.
Me / block input data is simply decimated every other data, and 15 frames / block which is half the data amount
Stream data, and the lowest layer is
This is stream data obtained by compressing the above 15 frame / block stream data to 6 frame / block. The stream data of the second hierarchy and the stream data of the lowest hierarchy are interpolated by cubic spline interpolation.

[0177] By storing the multi-layered stream data in the multidimensional stream data storage means 31, it is possible to transmit stream data of an information amount according to the state of the transmission path.

In addition, according to the multi-layered storage method,
Control is performed so that the compression ratio of the stream data sent from the transmitting means 1 is optimized so as to match the capability of a control circuit constituting the receiving means 2, for example, a computer.

Next, data transmission between the transmitting means 1 and the receiving means 2 will be described.

When there is an instruction from the control unit of the transmitting means 1 or a request from the receiving means 2, the transmitting means 1
The multi-dimensional stream data stored in the storage device in the computer device, that is, the multi-dimensional stream data storage means 31, is transmitted. The data transmission medium in this case is assumed to be a two-way communication system, particularly the Internet, but may be a broadcasting system such as satellite broadcasting or terrestrial broadcasting.

Hereinafter, the flow of a series of processes of the receiving means 2 will be described.

The multidimensional stream data receiving means 24
The multidimensional stream data transmitted from the transmitting means 1 is received.

Next, an example of identification of multidimensional stream data in the receiving means 2 will be described. Hereinafter, a case where multidimensional stream data of a three-dimensional CG character is received will be described as an example of a multidimensional stream. It is assumed that the multidimensional stream data includes the above-described compressed operation data in a predetermined stream format. Note that the operation data included in the multidimensional stream data does not necessarily have to be compressed.

First, the multidimensional stream data receiving means 24 in the receiving means 2 receives the multidimensional stream data (MSC1 to MSC3) transmitted from the transmitting means 1 and transmits the transmitted multidimensional stream data (MSC1
.. MSC3) has a format as shown in FIG. 10, the multidimensional stream data identification means 21 first receives and analyzes the packet identifier, and determines whether it is a channel definition packet or a data packet.

The multidimensional stream data identification means 21
The transmitted multidimensional stream data (MSC1 to MSC)
When determining that 3) is a channel definition packet, the multidimensional stream data identification unit 21 receives and analyzes the time stamp, the total number of channels TC, and the compression ratio identifier, and further, the data portion. A channel identifier, a channel data size, a channel name, and a channel type are received and analyzed for the total number of channels TC, respectively. Here, when the multidimensional stream data identification unit 21 fails in the data analysis, the reception unit 2 requests the transmission unit 1 to retransmit the data in order to re-acquire the data. Causes the system of the transmitting means 1 and the receiving means 2 to be forcibly terminated. Here, the compression ratio identifier indicates compression information such as a compression method and a compression ratio used by the compression method.

The multidimensional stream data identification means 21
The transmitted multidimensional stream data (MSC1 to MSC)
When determining that 3) is a data packet, the multidimensional stream data identification means 21 subsequently receives and analyzes the time stamp and the total number of channels DC, and receives the channel identifier, the channel data, and the compression ratio identifier. , And this is repeated DC times the total number of channels. It should be noted that the multidimensional stream data identification means 21 has a three-dimensional C
Of the motion data of the G character, only the position vector of the reference position and the direction vector representing the posture have a special channel name, and the two are distinguished by the channel name.

When the multi-dimensional stream data identification means 21 fails to analyze the data of the data packet, the receiving means 2 sends the data to the transmitting means 1 in order to acquire the data again.
A request for retransmission of data, or the receiving means 2 forcibly terminates the system of the transmitting means 1 and the receiving means 2, or discards a data packet whose analysis has failed,
The system can be continued by receiving the next packet.

Next, the multiplex buffering means 22 in the receiving means 2 will be described.

The multiplex buffering means 22 in the receiving means 2 converts the multidimensional stream data decomposed for each block into multiplicity data corresponding to the processing performance of the computer having the receiving means 2 or the execution speed for each block. To buffer. That is, when the processing performance or execution speed of the computer is high, the multiplicity of buffering is also increased, and when the execution speed is low, the multiplicity of buffering is also reduced. At this time, data storage for buffering is performed in the multidimensional stream data storage unit 32. later,
When describing synchronous playback, multiplex buffering will be described in more detail.

Next, the multidimensional stream data decompression means 23 in the reception means 2 will be described.

In the multi-dimensional stream data decompression means 23 of the reception means 2, one block of compressed stream data (MSC1
To MSC3) are performed by individual methods according to the compression information indicated by the compression ratio identifier. For example, the multidimensional stream data decompression means 23 may determine a decompression method based on compression information. Further, the multidimensional stream data decompression means 23 may determine a decompression rate for decompressing the compressed operation data based on the compression information.

In the position vector data MSC1 of the reference position of the three-dimensional CG character, compression in the time axis direction and compression for reducing the accuracy of the value of each component are performed within one block (for example, 4 bytes of data are converted to 2 bytes). In the case where the data is compressed into data, the multidimensional stream data decompression means 23 first decompresses the compression relating to the precision of the data value (decompresses the data to 4-byte data by inverse conversion to the compression conversion). For the compression in the time axis direction, the stream data for one block is expanded by cubic spline interpolation or linear interpolation using the compression ratio identifier in the header part of the channel definition packet.

The multidimensional stream data decompression means 23
The direction vector data MSC2 is converted from polar coordinates to three-dimensional vector coordinates. For example, when the direction vector data MSC2 representing the orientation of the three-dimensional CG character has two polar coordinate components representing the rotation axis and receives them as stream data, the multidimensional stream data decompression means 23 sets the direction vector data MSC2 To 3 from polar coordinates
The direction vector representing the rotation axis of magnitude 1 may be obtained by conversion into the dimensional vector coordinates.

With respect to the angle vector at each joint of the three-dimensional CG character and the rotation angle MSC3 around the joint, the multidimensional stream data decompression means 23 is used in the same manner as the conversion from the polar coordinates to the three-dimensional vector coordinates in the direction vector representing the posture. First, three three-dimensional vector coordinate components are obtained from two polar coordinate components. Next, in the expansion of the stream data in the position vector data MSC1 at the reference position, similarly to the expansion of the stream data in the time axis direction, the multi-dimensional stream data expansion unit 23 includes a compression ratio in the header of the channel definition packet. Using the identifier, stream data of one block is expanded by cubic spline interpolation, linear interpolation, or the like.

The multidimensional stream data decompression means 23
In the above, after the expansion of the multidimensional stream data is completed, the multidimensional stream data storage means 32 stores the expanded multidimensional stream data. When there is a request for stream data from an external three-dimensional CG reproducing unit, the stream data is supplied from the multidimensional stream data storing unit 32, so that the receiving unit 2 can receive the stream data.
Real-time reproduction of a dimensional CG scene can be performed.

Hereinafter, a method of multiplex buffering, synchronous reproduction, and real-time data reproduction of multidimensional stream data in the receiving means 2 will be described in detail.

Hereinafter, a three-dimensional CG in the case where the receiving means 2 of this embodiment has a reproducing means (not shown).
A simple example of buffering one character's motion stream data and one character's audio stream data will be described.

FIG. 13 is a diagram showing the reproduction of an operation stream and an audio stream when the receiving means 2 of the present embodiment has a reproducing means (not shown). A reproduction process and a buffering process are performed on the operation stream and the audio stream.

Period for one block (time t to time t ')
In the playback process for playing back the operation stream, the buffered operation stream data
Reproducing means (not shown) reproduces the motion of the three-dimensional CG character according to the time stamp of the data included in the motion stream data. In the operation stream buffering process, for example, the block next to the block to which the operation stream data currently being played belongs,
That is, the receiving unit 2 receives the operation stream data of the block corresponding to the period (time t ′ to time t ″), and transfers the received operation stream data to the multidimensional buffering unit 2.
2 buffers the data, the multidimensional stream data identification means 21 analyzes the operation stream data, and the multidimensional stream data expansion means 23 expands the operation stream data.

To prepare for the reproduction of the operation stream data (data reproduction of the next block), the multidimensional buffering means 22 buffers the expanded operation stream data again.

The same reproduction process and buffering process as the above-described operation stream data are performed on the audio stream data. In addition, by performing the same operation when the buffering multiplicity increases,
Multiple buffering can be realized.

Here, when the stream data transmission from the receiving means 2 to the reproducing means for reproducing the three-dimensional CG data is faster than the buffering speed, that is, the reception of the stream data and the expansion of the stream data are not in time. In this case, the real-time data reproduction is maintained by requesting the transmitting means 1 from the receiving means 2 for the multidimensional stream data having a further compressed low data amount and having the optimum compression ratio for data reproduction, and receiving the data. be able to.

In this example, the minimum configuration of the multidimensional stream data transmitting / receiving apparatus as described above, that is, an example in which one transmitting means 1 is connected to one receiving means 2 has been described. As shown in FIG. 10, a plurality of receiving units 2 may be connected via a transmission unit.

As described above, the transmission means 1 may include a stream authoring means (not shown).
The basic operation of the stream authoring means (not shown) described with reference to FIG. 4 will be briefly described below.
The stream authoring unit cuts the motion stream data of the three-dimensional CG character having an arbitrary data length and the audio stream data into arbitrary lengths necessary for synchronization. Here, the data length of the operation stream data is determined by the duration of the operation, and the data length of the audio stream data is arbitrarily determined by the audio reproduction time. Then, as shown in FIG. 15, the stream authoring means adds time stamp information necessary for synchronization to the operation stream and the audio stream. Further, the stream authoring unit connects the operation stream and the audio stream to generate a new stream as needed. The stream authoring means converts the stream data thus edited into
It is registered in the transmission means 1 as input data.

When the receiving means 2 shown in FIG. 4 has a reproducing means, the reproduction of the operation stream is synchronized with the reproduction of the audio stream based on the time stamp information of the operation stream and the time stamp information of the audio stream. Is preferred. Conversely, the reproduction of the audio stream may be synchronized with the reproduction of the operation stream based on the respective time stamps.

Hereinafter, the basic operation of the multidimensional stream data transmitting / receiving apparatus according to the present embodiment will be described with reference to FIGS.

FIG. 16 is a diagram showing a flowchart of the operation of the transmitting means 1, and FIG. 17 is a diagram showing a flowchart of the operation of the receiving means 2.

First, an example of the basic operation of the transmitting means 1 will be described. In step S11 in FIG. 16, the stream data (MS1 to MS3 and SS) created by the external stream data authoring means is input to the transmitting means 1.

[0209] In step S12, the transmitting means 1 transmits the stream data (MS1 to MS) input in step S11.
S3, SS) and stream data compression (MSC)
1 to MSC3, SSC) to generate and store a multidimensional stream data packet.

[0210] In step S13, the transmitting means 1 first transmits a channel definition packet to the receiving means 2.

In step S14, the transmitting means 1 determines whether there is a request from the receiving means 2 that the channel definition needs to be changed. If a channel definition is required, the process returns to step S13.

At step S15, the transmitting means 1 transmits the data packet to the receiving means 2.

In step S16, the transmitting means 1 determines whether the amount of data flowing through the transmission path is sufficient or not. If the amount of data to be transmitted exceeds the allowable amount of the transmission path, or if there is still room in the transmission path, or if a request to change the data to be transmitted from the receiving unit 2 has been made, the process proceeds to step S17. If the transmitting unit 1 determines that there is no excess or deficiency in the data amount, the process returns to step S14.

In step S17, the transmitting means 1 selects the stream data having the optimum compression rate corresponding to the state of the transmission path or the stream data having the optimum compression rate according to the request from the receiving means 2. It returns to step S14.

Next, an example of the operation of the receiving means 2 will be described.

At step S21 in FIG. 17, the receiving means 2 receives a multidimensional stream data packet from the transmitting means 1.

At step S22, the receiving means 2 determines whether or not the stream data packet received at step S21 is a channel definition packet. If YES is determined, the process returns to step S21.

At step S23, the receiving means 2 analyzes the data packet and decomposes the data.

[0219] In step S24, the receiving means 2 determines whether or not data exists in the buffer. If NO is determined, the process returns to step S21.

In step S25, the receiving means 2 expands one block of stream data in the buffer by the number of required stream data types.

In step S26, the receiving means 2 determines in step S25 whether data for one block has been decompressed. If YES is determined, the receiving means 2 outputs the expanded stream data to the reproduction system. That is, the process proceeds to step S27. If NO is determined, the process proceeds to step S28.

At step S26, the receiving means 2 sends out the expanded stream data to the reproducing system. In step S27, the receiving unit 2 determines whether an end flag has been generated by the user. If NO is determined, the process returns to step S21. If YES is determined, the receiving means 2 ends the operation.

[0223] In the second embodiment, the transmitting means 1 may be an artificial satellite and the receiving means 2 may be a digital television.

Further, in the second embodiment, the transmitting means 1 may be an artificial satellite, and the receiving means 2 may be a set-top box.

[0225] The artificial satellite may receive multidimensional stream data from a ground station.

In the second embodiment, transmitting means 1 may be a terrestrial broadcasting station, and receiving means 2 may be a digital television.

In the second embodiment, the transmitting means 1 may be a terrestrial broadcasting station, and the receiving means 2 may be a set-top box.

In the second embodiment, the transmitting means 1 may be a cable broadcasting station, and the receiving means 2 may be a digital television.

Further, in the second embodiment, the transmitting means 1 may be a cable broadcasting station, and the receiving means 2 may be a set-top box.

[0230] A program for causing a computer to execute the above-described method for transmitting and / or receiving multidimensional stream data may be recorded on a computer-readable recording medium.

[0231]

According to the present invention, motion data and voice data of a three-dimensional CG character are separately compressed according to their types, transferred as a stream in real time, and information necessary for synchronous reproduction of each data is also obtained. Simultaneous transfer enables 3D C in a network-based transmission / reception system
The smooth movement of the G character and the sound information synchronized therewith can be transmitted and received interactively according to the user's request, and the data amount at the time of transfer can be greatly reduced.

[Brief description of the drawings]

FIG. 1 is a diagram illustrating an example of a CG scene displayed on a display device.

FIG. 2 is a diagram showing a hierarchical structure when the CG scene shown in FIG. 1 is represented by an example of multidimensional stream data.

FIG. 3 is a diagram illustrating a system 100 for transmitting and receiving multidimensional stream data according to the first embodiment.

FIG. 4 is a diagram showing a multidimensional stream data transmission / reception device 10.

FIG. 5 is a diagram illustrating an example of a skeleton structure of a three-dimensional CG character.

FIG. 6 is a diagram illustrating an example of classification of input data to be input;

FIG. 7 is a diagram showing an example in which three-dimensional CG character motion data is divided into blocks in the time axis direction.

FIG. 8 is a diagram showing a relationship between three-dimensional vector coordinates and polar coordinates.

FIG. 9A is a diagram showing 11 nodes extracted from 30 frames / block.

FIG. 9B is a diagram showing six nodes extracted from 15 frames / block.

FIG. 10 is a diagram illustrating an example of a multidimensional stream data format.

FIG. 11 is a diagram illustrating a configuration example of multidimensional stream data in a time axis direction.

FIG. 12 shows an example in which the input data received by the transmitting unit 1 is 30 fr.
FIG. 4 is a diagram illustrating an example of stream data output from the transmission unit 1 when the data is ame / block.

FIG. 13 is a diagram showing an example of multiplex buffering and synchronous reproduction in the receiving means 2.

FIG. 14 is a diagram showing an example in which a plurality of receiving means 2 are connected to a transmitting means 1;

FIG. 15 is a diagram showing an example of inserting a time stamp in the stream authoring means.

FIG. 16 is a diagram showing the operation of the transmitting means 1;

FIG. 17 is a diagram showing the operation of the receiving means 2;

[Explanation of symbols]

DESCRIPTION OF SYMBOLS 1 Transmission means 2 Receiving means 11, 21 Multidimensional stream data discriminating means 12 Multidimensional stream data compressing means 22 Multiple buffering means 23 Multidimensional stream data decompressing means 31, 32 Multidimensional stream data storing means DC Total number of channels of data packets MS1 Stream data of the position vector at the reference position of the three-dimensional CG character MS2 Stream data of the direction vector representing the attitude of the three-dimensional CG character MS3 Stream data representing the angle vector at each joint of the three-dimensional CG character and the rotation angle around it SS voice stream data TC channel definition packet total number of channels

Continued on the front page (72) Inventor Yoshiyuki Mochizuki 1006 Kazuma Kadoma, Osaka Pref.Matsushita Electric Industrial Co., Ltd. Shigeo Asahara 1006 Kadoma Kadoma, Kadoma City, Osaka Prefecture Matsushita Electric Industrial Co., Ltd.

Claims (95)

[Claims] 1. A transmitting device comprising multidimensional stream data transmitting means for transmitting multidimensional stream data. 2. A multi-dimensional stream data identification unit for classifying input data into a plurality of types of operation data, and a multi-dimensional stream data compression unit for compressing the plurality of types of operation data in block units. The transmission device according to claim 1. 3. The transmission device according to claim 2, wherein the multidimensional stream data compression unit changes a compression ratio according to a type of the operation data. 4. The transmission device according to claim 2, wherein the multidimensional stream data compression means selects a compression method according to a type of the operation data. 5. The multi-dimensional stream data compression unit generates a node sequence that is time-series data and includes at least one of a plurality of types of operation data during a predetermined period, and generates a node sequence from the node sequence. The transmission device according to claim 2, wherein the plurality of types of operation data are compressed in units of blocks by extracting an arbitrary node. 6. The transmitting apparatus according to claim 5, wherein said multidimensional stream data transmitting means transmits interpolation information necessary for reproducing said extracted nodes and said node sequence. 7. The multi-dimensional stream data compression unit according to claim 2, wherein the plurality of types of operation data are compressed in units of blocks by reducing high-frequency components of the plurality of types of operation data. Transmission device. 8. The multidimensional stream data compressing means compresses the plurality of types of operation data in units of blocks by performing coordinate transformation on at least one type of the plurality of types of operation data. The transmission device according to claim 2. 9. The multi-dimensional stream data compressing means, of the plurality of types of operation data, converts data represented by three-dimensional vector coordinates into data represented by polar coordinates, thereby obtaining the plurality of types of motion data. The transmission device according to claim 2, wherein the operation data is compressed in block units. 10. The type of the multidimensional stream data to be transmitted, the order of the plurality of types of operation data to be transmitted, and the plurality of types of operations to be transmitted. The transmission device according to claim 1, further comprising header information or stream format description data describing a data length. 11. The transmitting apparatus according to claim 1, wherein the multidimensional stream data has time information and multidimensional spatial information. 12. The transmitting apparatus according to claim 2, wherein the multi-dimensional spatial information includes position information, rotation information, and scale information. 13. The multi-dimensional stream data includes a CG
The transmitting device according to claim 1, further comprising data indicating a position, a rotation, and a scale of the scene. 14. The transmission device according to claim 1, wherein the multidimensional stream data capable of representing a CG scene is multidimensional animation stream data. 15. The multidimensional stream data has a data section having a plurality of channels, and a header section describing a stream data format of the data section, wherein each of the plurality of channels is at a certain time. 3
The transmitting device according to claim 1, wherein the transmitting device has a plurality of pieces of information about the dimension CG character. 16. The three-dimensional CG character may be represented by a hierarchical structure having at least one joint, and each of the plurality of motion data indicates a dimension of a position vector of a reference position of the three-dimensional CG character. Data having a dimension of a direction vector representing a posture at the reference position, data having an angle vector of the at least one joint and dimensions of a rotation angle around the angle vector, a local coordinate axis of the at least one joint The transmitting device according to claim 15, comprising at least one of stream data having a dimension of a rotation angle around. 17. The multi-dimensional stream data according to claim 1, wherein the multi-dimensional stream data is a packet having a header portion and a data portion, wherein the header portion indicates a type of the packet, an order of the packet, and a length of the data portion. The transmitting device according to the above. 18. The multi-dimensional stream data holding means for holding the data compressed by the multi-dimensional stream data compression means in a multi-layered structure, wherein the transmitting apparatus transmits the multi-dimensional stream data. Depending on the state of the transmission path to be performed, and the reproduction speed at which the receiving device that receives the multidimensional stream data reproduces the multidimensional stream data, the data is stored in a hierarchical structure from the data stored in the multiplex hierarchical structure. 3. The transmission device according to claim 2, wherein the transmission device selects data and transmits the selected data. 19. The transmission device, wherein the transmission device is configured to transmit at least one of a state of a transmission path through which the multidimensional stream data is to be transmitted and a reproduction speed at which a reception device that receives the multidimensional stream data reproduces the multidimensional stream data. 19. The transmitting apparatus according to claim 18, wherein, according to one, the data of the layer having the largest data amount that can be reproduced is selected from the data held in the multi-layer structure. 20. The transmission apparatus according to claim 2, further comprising a multidimensional stream data holding unit that holds the data compressed by the multidimensional stream data compression unit in a file format of the multidimensional stream data. 21. A transmission method including a multidimensional stream data transmitting step of transmitting multidimensional stream data. 22. A multidimensional stream data identification step of classifying input data into a plurality of types of operation data, and a multidimensional stream data compression step of compressing the plurality of types of operation data in block units. The transmission method according to claim 21. 23. The transmission device according to claim 22, wherein the multidimensional stream data compression unit changes a compression ratio according to a type of the operation data. 24. The transmission method according to claim 22, wherein the multidimensional stream data compression step selects a compression method according to a type of the operation data. 25. The multi-dimensional stream data compression step generates a node sequence that is time-series data and includes at least one type of operation data among a plurality of types during a predetermined period, 23. The transmission method according to claim 22, wherein the plurality of types of operation data are compressed in block units by extracting an arbitrary node. 26. The transmission method according to claim 25, further comprising a step of transmitting interpolation information necessary for reproducing the extracted nodes and the node sequence. 27. The multi-dimensional stream data compression step according to claim 22, wherein the plurality of types of operation data are compressed in blocks by reducing high-frequency components of the plurality of types of operation data. Transmission method. 28. The multi-dimensional stream data compression step: performing a coordinate transformation on at least one of the plurality of types of operation data to compress the plurality of types of operation data in block units; A transmission method according to claim 28. 29. The multi-dimensional stream data compression step, wherein the data represented by three-dimensional vector coordinates among the plurality of types of motion data is converted into data represented by polar coordinates. 23. The transmission method according to claim 22, wherein the operation data is compressed in block units. 30. The type of the multidimensional stream data to be transmitted, the order of the plurality of types of operation data to be transmitted, and the plurality of types of operations to be transmitted. 22. The transmission method according to claim 21, further comprising header information or stream format description data describing a data length. 31. The transmission method according to claim 21, wherein the multidimensional stream data has time information and multidimensional spatial information. 32. The transmission method according to claim 31, wherein the multi-dimensional spatial information includes position information, rotation information, and scale information. 33. The multi-dimensional stream data is a CG stream.
22. The transmission method according to claim 21, further comprising data indicating a position, a rotation, and a scale of the scene. 34. The transmission method according to claim 21, wherein the multidimensional stream data capable of representing a CG scene is multidimensional animation stream data. 35. The multi-dimensional stream data has a data section having a plurality of channels, and a header section describing a stream data format of the data section.
22. The transmission method according to claim 21, wherein the transmission method has a plurality of pieces of information on the dimension CG character. 36. The three-dimensional CG character may be represented by a hierarchical structure having at least one joint, and each of the plurality of motion data indicates a dimension of a position vector of a reference position of the three-dimensional CG character. Data having a dimension of a direction vector representing a posture at the reference position, data having an angle vector of the plurality of joints and a dimension of a rotation angle around the plurality of joints, local coordinate axes of the plurality of joints Having at least one of the stream data having dimensions of a rotation angle around;
The transmission method according to claim 35. 37. The multidimensional stream data according to claim 21, wherein the multidimensional stream data is a packet having a header portion and a data portion, wherein the header portion indicates a type of the packet, an order of the packet, and a length of the data portion. The transmission method described. 38. The transmission method includes a multidimensional stream data holding step of holding data compressed by the multidimensional stream data compression means in a multi-layered structure. According to the state of the transmission path to be transmitted, and the reproduction speed for reproducing the multi-dimensional stream data, the data stored in the multiplex hierarchical structure is selected from the data stored in a certain hierarchical structure, and the selected data is selected. 23. The transmission method according to claim 22, wherein the transmitted data is transmitted. 39. The transmission method, wherein: at least one of a state of a transmission path through which the multidimensional stream data is to be transmitted and a reproduction speed at which a receiving device that receives the multidimensional stream data reproduces the multidimensional stream data. 39. The transmission method according to claim 38, wherein, according to one, from among the data held in the multi-layer structure, data of a layer having the largest amount of data that can be reproduced is selected. 40. The transmission method according to claim 22, further comprising a multidimensional stream data holding step of holding the data compressed in the multidimensional stream data compression step in a file format of the multidimensional stream data. 41. A receiving device comprising multidimensional stream data receiving means for receiving multidimensional stream data. 42. A multidimensional stream data identification means for identifying data content of the multidimensional stream data from one of header information and stream format description information of the multidimensional stream data; 42. The method according to claim 41, further comprising: multiplex buffering means for sequentially reading data into a plurality of multiplex buffers in block units; and multidimensional stream data decompression means for decompressing the blocked multidimensional stream data in block units. Receiver. 43. The receiving apparatus according to claim 42, wherein said multidimensional stream data identification means determines whether said received multidimensional stream data is a channel definition packet. 44. The receiving apparatus according to claim 42, wherein said multidimensional stream data identification means determines whether said received multidimensional stream data is a data packet. 45. If the multidimensional stream data identifying means determines that the received multidimensional stream data is a channel definition packet, the multidimensional stream data identifying means determines whether the received multidimensional stream data is a channel definition packet. 44. The receiving device according to claim 43, wherein a compression ratio identifier indicating a time stamp, a total number of channels, and a compression ratio of channel data is identified. 46. When the multidimensional stream data identifying means determines that the received multidimensional stream data is a data packet, the multidimensional stream data identifying means determines the received multidimensional stream data from the received multidimensional stream data. 46. The receiving device of claim 44, wherein the receiving device identifies a timestamp, a timestamp, and a total number of channels. 47. The receiving device according to claim 41, wherein the received multidimensional stream data includes compressed operation data and compression information. 48. The receiving apparatus according to claim 47, wherein said multidimensional stream data decompression means decompresses said compressed operation data based on compression information. 49. The multi-dimensional stream data decompression means determines a decompression rate based on compression information.
9. The receiving device according to 8. 50. The receiving apparatus according to claim 48, wherein said multidimensional stream data decompression means determines a decompression method based on compression information. 51. When the compressed motion data is generated by extracting arbitrary motion data from a plurality of motion data in a predetermined period, the multi-dimensional stream data decompression means includes: 49. The receiving device according to claim 48, wherein a plurality of pieces of operation data in the predetermined period are restored based on the data and the compression information. 52. The receiving device according to claim 51, wherein said compression information is interpolation information. 53. The multi-dimensional stream data decompression unit decompresses the compressed motion data by performing coordinate transformation on the compressed motion data based on the compression information. 3. The receiving device according to claim 1. 54. When the compressed motion data is represented by polar coordinates, the multidimensional stream data decompression means converts the compressed motion data represented by polar coordinates into data represented by three-dimensional vector coordinates. 43. Decompressing the compressed motion data by converting.
3. The receiving device according to claim 1. 55. The receiving apparatus according to claim 41, wherein the multidimensional stream data has time information and multidimensional spatial information. 56. The receiving device according to claim 55, wherein the multi-dimensional spatial information includes position information, rotation information, and scale information. 57. The multi-dimensional stream data includes a CG
The receiving device according to claim 41, further comprising data indicating a position, a rotation, and a scale of the scene. 58. The receiving device according to claim 41, wherein the multidimensional stream data capable of representing a CG scene is multidimensional animation stream data. 59. The multi-dimensional stream data has a data section having a plurality of channels, and a header section describing a stream data format of the data section, wherein each of the plurality of channels has a value of 3 at a certain time.
42. The receiving device according to claim 41, wherein the receiving device has a plurality of pieces of information on a dimension CG character. 60. The three-dimensional CG character can be represented by a hierarchical structure having at least one joint, and each of the plurality of motion data indicates a dimension of a position vector of a reference position of the three-dimensional CG character. Data having a dimension of a direction vector representing a posture at the reference position, data having an angle vector of the at least one joint and dimensions of a rotation angle around the angle vector, a local coordinate axis of the at least one joint The receiving device according to claim 59, further comprising at least one of stream data having a dimension of a rotation angle around. 61. The multi-dimensional stream data according to claim 41, wherein the multi-dimensional stream data is a packet having a header part and a data part, and the header part indicates a type of the packet, an order of the packet, and a length of the data part. The receiving device according to the above. 62. The receiving device according to at least one of a state of a transmission path through which the multidimensional stream data is to be transmitted and a reproduction speed at which the receiving device reproduces the multidimensional stream data, 42. The receiving apparatus according to claim 41, wherein the receiving apparatus instructs a transmitting apparatus that transmits the multidimensional stream data with a compression ratio of the multidimensional stream data. 63. A receiving method including a multidimensional stream data receiving step of receiving multidimensional stream data. 64. A multidimensional stream data identification step of identifying data content of the multidimensional stream data from one of header information and stream format description information of the multidimensional stream data; 64. The method according to claim 63, further comprising: a multiplex buffering step of sequentially reading data into a plurality of multiplex buffers in block units; and a multidimensional stream data expansion step of expanding the blocked multidimensional stream data in block units. Receiving method. 65. The receiving method according to claim 64, wherein said multi-dimensional stream data identifying step determines whether said received multi-dimensional stream data is a channel definition packet. 66. The receiving method according to claim 64, wherein said multi-dimensional stream data identification step determines whether said received multi-dimensional stream data is a data packet. 67. If the multi-dimensional stream data identification step determines that the received multi-dimensional stream data is a channel definition packet, the multi-dimensional stream data identification step includes: 66. The receiving method according to claim 65, wherein a compression ratio identifier indicating a time stamp, a total number of channels, and a compression ratio of channel data is identified. 68. If the multidimensional stream data identification step determines that the received multidimensional stream data is a data packet, the multidimensional stream data identification step determines the received multidimensional stream data from the received multidimensional stream data. 67. The receiving method of claim 66, wherein the method identifies a time stamp, and a total number of channels. 69. The receiving method according to claim 63, wherein the received multidimensional stream data includes compressed operation data and compression information. 70. The receiving method according to claim 69, wherein said multi-dimensional stream data decompression step decompresses said compressed operation data based on compression information. 71. The multidimensional stream data decompression step determines a decompression rate based on compression information.
0. The receiving method according to 0. 72. The receiving method according to claim 70, wherein said multidimensional stream data decompression step determines a decompression method based on compression information. 73. When the compressed motion data is generated by extracting arbitrary motion data from a plurality of motion data in a predetermined period, the multi-dimensional stream data decompression step includes: 71. The receiving method according to claim 70, wherein a plurality of pieces of operation data in the predetermined period are restored based on the data and the compression information. 74. The receiving method according to claim 73, wherein said compression information is interpolation information. 75. The multi-dimensional stream data decompressing step decompresses the compressed motion data by performing coordinate transformation on the compressed motion data based on the compression information. Receiving method described in. 76. When the compressed motion data is represented by polar coordinates, the multi-dimensional stream data decompression step converts the compressed motion data represented by polar coordinates into data represented by three-dimensional vector coordinates. 65. Decompressing the compressed motion data by converting.
Receiving method described in. 77. The receiving method according to claim 63, wherein said multidimensional stream data has time information and multidimensional spatial information. 78. The receiving method according to claim 77, wherein the multi-dimensional spatial information includes position information, rotation information, and scale information. 79. The multi-dimensional stream data includes a CG
The receiving method according to claim 63, further comprising data indicating a position, a rotation, and a scale of the scene. 80. The receiving method according to claim 63, wherein the multidimensional stream data capable of representing a CG scene is multidimensional animation stream data. 81. The multi-dimensional stream data has a data section having a plurality of channels, and a header section describing a stream data format of the data section.
64. The receiving method according to claim 63, wherein the receiving method has a plurality of pieces of information about the dimension CG character. 82. The three-dimensional CG character may be represented by a hierarchical structure having at least one joint, and each of the plurality of motion data indicates a dimension of a position vector of a reference position of the three-dimensional CG character. Data having a dimension of a direction vector representing a posture at the reference position, data having an angle vector of the plurality of joints and a dimension of a rotation angle around the plurality of joints, local coordinate axes of the plurality of joints Having at least one of the stream data having dimensions of a rotation angle around;
The receiving method according to claim 81. 83. The multi-dimensional stream data according to claim 63, wherein the multi-dimensional stream data is a packet having a header section and a data section, and the header section indicates a type of the packet, an order of the packet, and a length of the data section. The receiving method described. 84. The multi-dimensional stream data according to at least one of a state of a transmission path through which the multi-dimensional stream data is to be transmitted and a reproduction speed at which the receiving device reproduces the multi-dimensional stream data. 64. The receiving method according to claim 63, wherein a transmitting unit for transmitting the compression instruction is instructed on a compression ratio of the multidimensional stream data. 85. The multi-dimensional stream data includes motion data and audio data, the motion data has time stamp information, the audio data has time stamp information, and the receiving device has The receiving device according to claim 41, wherein the operation data and the audio data are reproduced in synchronization with each other in accordance with time stamp information and the time stamp information of the audio data. 86. The multi-dimensional stream data includes motion data and audio data, the motion data has time stamp information, the audio data has time stamp information, and the time stamp information of the motion data and 64. The receiving method according to claim 63, wherein the reproduction is performed in synchronization with the operation data and the audio data in accordance with the time stamp information of the audio data. 87. A computer-readable recording medium on which a program for causing a computer to execute the transmission method according to claim 21 is recorded. 88. A computer-readable recording medium on which a program for causing a computer to execute the receiving method according to claim 63 is recorded. 89. A communication system comprising: a multidimensional stream data transmitting unit that transmits multidimensional stream data; and a multidimensional stream data receiving unit that receives the multidimensional stream data. 90. The multidimensional stream data transmitting unit is one of an artificial satellite, a terrestrial broadcasting station, and a cable broadcasting station, and the multidimensional stream data receiving unit is one of a digital television and a set top box. 90. The communication system of claim 89, wherein the communication system is one of the following: 91. The transmitting apparatus according to claim 1, wherein said multidimensional stream data includes data capable of representing a CG scene. 92. The transmission method according to claim 21, wherein the multidimensional stream data includes data capable of representing a CG scene. 93. The receiving apparatus according to claim 41, wherein said multidimensional stream data includes data capable of representing a CG scene. 94. The receiving method according to claim 63, wherein said multidimensional stream data includes data capable of representing a CG scene. 95. The communication system according to claim 89, wherein the multidimensional stream data includes data capable of representing a CG scene.