JP2002112149A - Data processing method and device, data transmission system, transmission medium - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a data transmission system where a decoding terminal can decode and display data other than video data or the like, in the case of special reproduction so as to realize distribution and decoding of scene description data, synchronization relations between the data are maintained, and the data satisfying evaluation criteria, such as a transmission bit rate, are distributed. SOLUTION: A data conversion section 7 of a server 10 outputs data used for normal reproduction, when the decoding terminal 12 makes the usual reproduction and the data conversion section 7 rewrites time information relating to the reproduction, such as a display start time and a display time or a display end time for data AU used for the normal reproduction depending on special reproduction and encodes the data and outputs the encoded data, when the decoding terminal 12 makes the special reproduction.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to multimedia data including video data, audio data, text data, graphic data and the like such as still images and moving images, and a scene for constructing a scene using the multimedia data. When special reproduction is performed in a data distribution system in which scene description data is distributed using a network, and the distributed multimedia data and scene description data are received by a decoding terminal and decoded and displayed by the decoding terminal. The present invention relates to a data processing method and apparatus, a data transmission system, and a transmission medium that are optimal for use in the present invention.

[0002]

2. Description of the Related Art Video data and the like obtained by compressing image signals of still images and moving images are distributed via a transmission medium.
FIG. 39 shows a configuration example of a conventional data distribution system in which a decoding terminal receives, decodes, and displays the data. In addition,
In FIG. 39, only the path of the video data is described to simplify the description. Also, in the following description,
Video data can be transferred to, for example, an ISO (Internatioal Organizat
ion for Standardization / IEC (International E)
In this example, a packet is distributed to a transport stream (Transport stream, hereinafter simply referred to as TS) defined by 13818-1 (so-called MPEG2 Systems).

[0003] In FIG. 39, a server 200 includes a storage unit 209 for storing video data. The video data read from the storage unit 209 is
At 04, the packet is packetized into a TS, and is further formed as distribution data 211 by the transmission unit 205, output to the transmission medium 210, and distributed to, for example, the decoding terminal 212. At this time, the TS distribution data 211 is transmitted using the protocol used in the transmission medium 210. For example, a TS that satisfies the provisions of ISO / IEC13818-1
IEC61883 "Digital Interface for cons
umer audio / video equipment, for example, IEEE (Institute of Electrical and Electron
cs Engineers) It is possible to transmit using a 1394 standard transmission medium. Note that the multiplexing unit 204 and the transmitting unit 205 may be integrated.

[0004] In the decoding terminal 212, the distribution data 211 is received by the reception unit 213 and sent to the separation unit 214.
The separating unit 214 separates the video data from the TS packet and sends the video data to the decoding unit 215. In the decoding unit 215,
Decode the encoded video data. The decoded video data is sent to, for example, a display device (not shown) and displayed as a video image.

In such a data distribution system,
For example, when performing special playback display such as fast-forward playback, frame-by-frame playback, and pause, for example, a special playback designation signal (instruction such as fast-forward playback or frame-by-frame playback) according to the operation of the terminal front panel or the remote controller by the user The signal 206 is input to the special reproduction control unit 216 of the decoding terminal 212. At this time, the special reproduction control unit 216 of the decoding terminal 212 generates a special reproduction request signal 220 for requesting the server 200 for special reproduction video data of the type specified by the special reproduction specification signal 206. The special reproduction request signal 220 is transmitted to the special reproduction control unit 201 of the server 200 via the transmission medium 210.

[0006] Upon receiving the special reproduction request signal 220, the special reproduction control unit 201 of the server 200 generates control signals 202a and 202b corresponding to the request and sends them to the corresponding multiplexing unit 204 and transmission unit 205, respectively.
The multiplexing unit 204 reads data from the storage unit 209 under the control of the trick play control unit 201 based on the control signal 202b.
The special reproduction of the type specified by the user is performed by the decoding terminal 212.
Reads out video data for special reproduction to enable. The multiplexing unit 204 packetizes the special reproduction video data into a TS and sends the packet to the transmission unit 205. The transmission unit 205 distributes the packet of the special reproduction video data to the decoding terminal 212 as distribution data 211 under the control of the special reproduction control unit 201 by the control signal 202a.

At the decoding terminal 212 when the distribution data 211 composed of the special reproduction video data is supplied, the control signals 217a and 217b for performing the special reproduction control according to the special reproduction designation signal 206 are transmitted to the decoding terminal 212. The data is output from the reproduction control unit 216 and sent to the corresponding receiving unit 213 and decoding unit 215, respectively. Receiver 21
3 receives the distribution data 211 composed of the video data for the special reproduction under the control of the special reproduction control unit 216 based on the control signal 217b, and sends it to the separation unit 214. The separation unit 214 separates the special reproduction video data from the TS packet and sends the special reproduction video data to the decoding unit 215. The decoding unit 215 decodes video data for trick play under the control of the trick play control unit 216 based on the control signal 217a. As a result, a special playback display such as fast-forward playback or frame-forward playback is performed on a display device (not shown).

[0008] The video frame encoding method specified in ISO / IEC 13818-2 includes an I-picture (Intra-code) encoding from only intra-frame data.
d picture: an intra-coded image and a B picture (Bidirectionally) to be coded using inter-frame prediction
There are a predictive-coded picture: a bidirectional predictive coded image) and a P picture (Predictive-coded picture: a forward predictive coded image). In the data distribution system shown in FIG. As the video data for reproduction, an I picture that does not use the above-described prediction processing between video frames is used. That is, I-pictures are regularly included in video data for normal reproduction to enable random access, and the I-pictures are extracted to form video data for special reproduction. As described above, in the conventional data distribution system shown in FIG. 39, when special decoding such as fast-forwarding is performed in the decoding terminal 212, the ISO /
Video data for special reproduction, such as video data consisting of only IEC13818-2 I pictures, is
From 00, it is distributed to the decoding terminal 212.

On the other hand, when the compressed video data conforming to, for example, ISO / IEC13818-2 (so-called MPEG2 video) is distributed as in the data distribution system described above, the compressed video data specified in the ISO / IEC13818-2 is used. The data must be encoded so as not to overflow and underflow the decoder buffer. The decoder buffer corresponds to an input buffer (not shown) provided in the decoding unit 215. This ISO / IEC13818-2
When data exceeds the size of the buffer specified in the above, the decoder buffer overflows, and on the other hand, if data necessary for decoding has not arrived at the time to decode, the decoder buffer underflows.

However, like the above-mentioned video data for special reproduction, video data consisting of only I-pictures has a large data amount, and may cause the decoder buffer to overflow or underflow.
For this reason, in the conventional data distribution system, special data for special reproduction different from that for normal reproduction is prepared in advance so as to enable special reproduction without overflowing or underflowing the decoder buffer. When performing special reproduction in the decoding terminal, it is necessary to deliver special data for the special reproduction. Also,
The decoding terminal also needs a special terminal capable of performing special trick-play processing different from normal trick-play processing corresponding to the special data for trick-play.

That is, according to the conventional data distribution system, in order to realize the special reproduction without causing the decoder buffer to overflow or underflow, the special data different from the special reproduction video data consisting of only the I picture described above is used. Special data for reproduction must be prepared in advance, and the special data must be distributed during special reproduction. Similarly, a decoding terminal requires a terminal having special decoding units 215 capable of handling special data for special reproduction, and a special reproduction control unit 216 includes a reception unit 213, a separation unit 214, and a decoding unit. The unit 215 needs control for data processing for special reproduction.

[0012] Under such circumstances, the present applicant has filed a patent application No. 2000-178999 and a patent application No. 2000-179.
No. 000, the server uses the video data for normal reproduction read from the storage unit and converts the data resulting from the special reproduction of the type specified by the user into an ISO / I
By converting the video data that satisfies the regulation of EC13818-2 and delivering the converted video data to the decoding terminal, it is not necessary to use and prepare in advance the special distribution data for the special reproduction as described above. Also, a technique has been proposed which has a simple configuration that does not require a special decoding terminal capable of handling special distribution data for the special reproduction.

FIG. 40 shows data obtained as a result of performing special reproduction using video data for normal reproduction, for example, according to the ISO standard.
1 shows a schematic configuration of a data distribution system that realizes conversion and output to video data satisfying the regulations of / IEC13818-2. In the example of FIG. 40, for example, video data and the like are stored in an ISO (International Organizatio).
n for Standardization) / IEC (International Ele)
In this case, a packet is distributed to a transport stream (Transport stream: TS) defined in 13818-1 (so-called MPEG2 Systems).

In FIG. 40, the server 220 has a storage unit 229 for storing multimedia data such as video data such as still images and moving images, audio data, text data, and graphic data. For example, video data is read from the storage unit 229, and the video data is sent to the multiplexing unit 224 via, for example, a data conversion unit 223 described later. The multiplexing unit 224 packetizes the data output from the data conversion unit 223 into a TS. The TS packet is further formed into distribution data 231 by the transmission unit 225, output to the transmission medium 230, and distributed to, for example, the decoding terminal 232. At this time, the distribution data 231 of the TS is transmitted using the protocol used in the transmission medium 230. For example, a TS that satisfies the provisions of ISO / IEC13818-1
IEC61883 "DigitalInterface for consu
mer audio / video equipment ”, for example, IEEE (Institute of Electrical and Electron
ics Engineers) can be transmitted using a 1394 standard transmission medium.

[0015] In the decoding terminal 232, the distribution data 231 is received by the reception unit 233 and sent to the separation unit 234.
The separating unit 234 separates video data from the TS packet and sends the video data to the decoding unit 235. In the decoding unit 235,
The supplied data is decoded, that is, the encoded video data is decoded. The decoded video data is sent to, for example, a display device (not shown) and displayed as a video image.

The decoding terminal 232 of the data distribution system
In the case where the special reproduction display is performed, for example, a special reproduction designation signal 226 corresponding to an operation by the user of the decoding terminal 232 is transmitted from the transmission medium interface unit (not shown) in the decoding terminal 232 via the transmission medium 230. Sent to server 220. This special reproduction designation signal 2
Reference numeral 26 denotes a signal including the type of special playback such as fast forward playback, rewind playback, and frame forward playback, and designation of video data and the like stored in the storage unit 229. Note that the server 220 and the decoding terminal 232 are connected to each other at a short distance, such as a home network, and the user
When the user can operate the front panel, the remote controller, and the like of the server 220, the special reproduction designation signal 226 is directly input to the server 220 by operating the front panel, the remote controller, and the like of the server 220. It is also possible.

The special reproduction designation signal 226 input to the server 220 is input to a special reproduction control unit 221 provided in the server 220. This special reproduction control unit 22
1 generates a control signal 222 for trick play control including a trick play type and video data designation in response to the trick play designation signal 226, and sends it to the data converter 223.

The data conversion section 223 reads video data from the storage section 229 under the control of the special reproduction control section 221 based on the control signal 222. Further, the data conversion unit 223 uses the video data read from the storage unit 229 to convert the data resulting from the special reproduction of the type specified by the control signal 222 into, for example, an ISO /
The video data is converted into video data satisfying the requirements of IEC13818-2 and output. That is, at this time, the data conversion unit 2
When the decoding is performed by the decoding unit 235 of the decoding terminal 232 in the same manner as in the normal reproduction, special reproduction (special reproduction specified by the user) such as fast-forward reproduction, rewind reproduction, or frame-forward reproduction is realized. Storage unit 2 for video data
29 is converted.

Here, the data conversion processing in the data conversion section 223 will be briefly described with reference to FIGS. 41 and 42.

In FIG. 41, video data for normal reproduction (video data read from the storage unit 229) encoded by MPEG2 video is converted to the data conversion unit 22.
3 shows an outline of a data conversion process for realizing fast-forward playback as an example of the special playback process and converting the video data to video data satisfying the provisions of ISO / IEC13818-2. In the figure, I represents an I picture, P represents a P picture, and B represents a B picture. Also, MPEG2 v
According to the definition of “ideo”, the encoding order (the order in which data is encoded in a bit stream) and the actual display order may differ from each other due to the fact that encoding is performed using prediction between pictures. Shows the encoding order and the display order together. FIG. 41 (a) shows the encoding order of the video data for normal reproduction, and FIG. 41 (b) shows the display order when decoding and displaying the video data for normal reproduction. FIG. 41C shows the encoding order in the case where a conversion process for special reproduction is performed such that a transition is made to the fast-forward reproduction section FS after the normal reproduction section US, and then to the normal reproduction section US. FIG. 41D shows the display order when the conversion process for the special reproduction as shown in FIG. 41C is performed.

[0021] The data conversion unit 223, the fast-forward reproduction interval FS for the special playback is performed, in the drawing E _k, E _m, E _n
As shown in, I picture during normal playback video data (a) of FIG. _{41 (I k, I m,} I n) using an extracted, further, in order not to collapse the decoder buffer,
Performs data conversion processing to insert a repeat pictures B _R therebetween I picture. Incidentally, the above repeat pictures B _R, a picture of repeating the predicted original image, when the decoding is a picture is treated as B-picture. The insertion of the repeat pictures B _R is also effective for adjusting the speed of the fast forward playback.

In FIG. 42, as in FIG.
In the data conversion unit 223, rewind reproduction as an example of special reproduction processing is realized by using the video data for normal reproduction (video data read from the storage unit 229) encoded by video, and IEC138
The outline of data conversion processing when converting into video data satisfying the rule of 18-2 will be described. FIG. 42A shows the encoding order of the video data for normal reproduction.
(B) shows the display order when the video data for normal reproduction is decoded and displayed. In (c) of FIG. 42, the process proceeds to the rewind playback section BS after the normal playback section US,
FIG. 42 shows the encoding order in the case where a conversion process for special reproduction is performed so as to return to the normal reproduction section US.
(D) shows the display order when the conversion process for the special reproduction as shown in (c) of FIG. 42 is performed.

In the data conversion section 223, E _k , E _m , E E
_As shown in FIG. 42, in order to extract the I pictures (I _k , I _m , and I _n ) from the video data for normal reproduction in FIG. 42A and change the order of the I pictures, performs data conversion processing to insert a repeat pictures B _R therebetween I picture.

As described above, the special reproduction video data converted by the data conversion unit 223 is distributed to the decoding terminal 232 through the configuration of the multiplexing unit 224 and thereafter as described above.

[0025]

By the way, in a conventional television broadcast, one image signal is displayed on a screen of an image display device, and only one audio signal is output from a speaker. In recent years, it has been considered that one scene is configured using multimedia data including video data such as still images and moving images, audio data, text data, graphic data, and the like. Note that as a method of describing the configuration of a scene using the multimedia data, there is an HTML (Hybrid) used on a so-called Internet homepage or the like.
perText Markup Language), ISO / IEC1449
MPEG4 which is a scene description method specified in 6-1
BIFS (Binary Format for the Scene), ISO /
VRML (Virtual Reality) defined in IEC14772
ity Modeling Language) and Java (trademark). Hereinafter, data describing the configuration of a scene will be referred to as a scene description.

An example of a scene description using VRML and MPEG4 BIFS will be described with reference to FIG. FIG. 43 shows the contents of the scene description. VR
In the ML, a scene description is performed using text data as shown in FIG. 43. In the MPEG4 BIFS, a scene description is performed using a binary coded version of this text data.

A scene description of VRML and MPEG4 BIFS is expressed by a basic description unit called a node. In the example of FIG. 43, the node is represented by a bold italic character. A node is a unit that describes an object to be displayed, a connection relationship between the objects, and the like, and includes data called a field to indicate characteristics and attributes of the node. For example, the Transform node in FIG. 43 is a node that can specify a three-dimensional coordinate transformation, and
The translation amount of the coordinate origin is specified in the ion field. There is also a field in which another node can be specified. For example, the Transform node in FIG. 43 has a Children field indicating a group of child nodes to be coordinate-transformed by the Transform node. For example, Shape nodes and the like are grouped by the Children field. To arrange the objects to be displayed in the scene, the nodes representing the objects are grouped together with the nodes representing the attributes, and the nodes are further grouped by the nodes representing the arrangement positions. For example, the object represented by the Shape node in FIG. 43 is arranged in the scene by applying the translation specified by the parent Transform node.

The video data and audio data are spatially and temporally arranged and displayed according to the scene description. For example, the MovieTexture node in FIG. 43 specifies that a moving image specified by ID 3 is to be pasted and displayed on the surface of a cube.

[0029]

As described above, in recent years, it has been considered that one scene is formed using multimedia data including video data, audio data, text data, graphic data, and the like. However, in the conventional data distribution system, only the video data is decoded and displayed during the trick play.

For this reason, even if multimedia data composed of, for example, video data, audio data, text data, graphic data, and the like is distributed, only video data is decoded and displayed during special reproduction. Even if data including data other than video such as audio data and subtitle text is distributed, the conventional data distribution system does not decode or display such data other than the video during special reproduction.

In view of the above, it is desired to be able to decode and display data other than video data such as audio data and subtitle text data even during special playback such as fast forward playback and rewind playback. ing.

At present, no technique or means has been realized for distributing, decoding, etc., the scene description data for constructing the scene as described above even during trick play. For this reason, in the conventional data distribution system, for example, even if one scene is configured using the above-described multimedia data and the multimedia data is distributed, the scene cannot be configured during the special reproduction. As a result, for example, a problem occurs that scenes displayed at the time of starting and ending the special reproduction are discontinuous.

In view of the above, it is desired to realize a technique and means for distributing and decoding the scene description data even during special reproduction, and decoding the scene description data.

Further, in order to distribute, decode, and display the multimedia data and the scene description data even during the trick play, the display and the like are performed while maintaining the synchronous relationship between the data. In addition, it is necessary to distribute the data as data that satisfies evaluation criteria such as a transmission bit rate (criterion for preventing the decoder buffer from being broken).

Accordingly, the present invention has been made in view of such circumstances, and enables decoding and display of data other than video data when performing special reproduction.
A data processing that realizes a method and means for distributing and decoding scene description data, and further retains a synchronization relationship between the data and distributes the data as data satisfying evaluation criteria such as a transmission bit rate. It is an object to provide a method and apparatus, a data transmission system, and a transmission medium.

[0036]

A data processing method according to the present invention is a data processing method for transmitting data coded for each predetermined coding unit to a receiving side. When performing, the data used for the normal reproduction is output, and when performing the special reproduction on the receiving side, the time information related to the reproduction of the coding unit of the data used for the normal reproduction is performed according to the special reproduction. The above-mentioned problem is solved by converting and outputting.

Further, the data processing device of the present invention is a data processing device for transmitting data coded for each predetermined coding unit to a receiving side. Output the data used for normal playback,
When performing the special reproduction on the receiving side, by including data conversion means for converting and outputting time information related to the reproduction of the encoding unit of the data used for the normal reproduction according to the special reproduction, The above-mentioned problem is solved.

Next, a data transmission system according to the present invention is a data transmission system comprising a transmitting device for transmitting data encoded for each predetermined coding unit and a receiving device for receiving the data. The apparatus outputs data used for the normal reproduction when performing the normal reproduction in the receiving apparatus, and reproduces the coding unit of the data used for the normal reproduction when performing the special reproduction in the receiving apparatus. The above-mentioned problem is solved by providing data conversion means for converting and outputting time information related to the special reproduction according to the special reproduction.

Further, the transmission medium of the present invention is a transmission medium for transmitting data coded for each predetermined coding unit by the transmitting device to the receiving device. When the receiving device performs normal reproduction, The data used for the normal reproduction is transmitted, and when the normal reproduction is performed by the receiving device, the data used for the normal reproduction is transmitted. When the special reproduction is performed by the receiving device, the data is used for the normal reproduction. The above-mentioned problem is solved by transmitting data obtained by converting time information related to reproduction of the above-mentioned coding unit of data according to the above-mentioned special reproduction.

That is, according to the present invention, for example, the display time and the display time or the display end time of the display unit of the normal reproduction data are converted into the special reproduction data by calculating and rewriting according to the special reproduction. Thus, it is possible to store and display the synchronization relationship between data even during trick play in the decoding terminal. According to the present invention,
For example, by selecting and distributing the display unit in the data for normal reproduction so as to satisfy the evaluation criteria such as the bit rate, it is possible to distribute data that satisfies the evaluation criteria such as the bit rate even during special reproduction. I do. Further, according to the present invention, to satisfy evaluation criteria such as a bit rate,
By converting and outputting the display unit in the data for normal reproduction, it is possible to distribute data that satisfies evaluation criteria such as a bit rate even during special reproduction.

[0041]

DESCRIPTION OF THE PREFERRED EMBODIMENTS Preferred embodiments of the present invention will be described below with reference to the drawings.

Video data such as still images and moving images, audio data, multimedia data such as text data and graphic data, and scene description data are distributed via a transmission medium and received by a decoding terminal for decoding. FIG. 1 shows an example of a configuration of a data distribution system according to an embodiment of the present invention, which is displayed as an image. In the following description, for example, video data and the like are converted to an ISO (International
Organization for Standardization / IEC (Inter
national Electrotechnical Commission) 13818-
1 (so-called MPEG2 Systems) is described as an example in the case of packetizing a transport stream (Transport stream: TS) and distributing it.

In FIG. 1, the server 10 includes a storage unit 9 for storing video data such as still images and moving images, audio data, multimedia data such as text data and graphic data, and scene description data. . The data read from the storage unit 9 is sent to the multiplexing unit 4 via, for example, a data conversion unit 7 described later. The multiplexing unit 4 packetizes the data output from the data conversion unit 7 into a TS. The TS packet is further formed into distribution data 22 by the transmission unit 5 and transmitted to the transmission medium 2.
1 and distributed to the decoding terminal 12, for example. At this time, the distribution data 22 of the TS is transmitted using the protocol used in the transmission medium 21.
For example, T that satisfies the provisions of ISO / IEC13818-1
S is “Digital Interface for co.
nsumer audio / video equipment "
For example, IEEE (Institute of Electrical and Electr)
onics Engineers) It is possible to transmit using a 1394 standard transmission medium. Note that the multiplexing unit 4 and the transmitting unit 5 may have an integrated configuration.

In the decoding terminal 12, the distribution data 22 is received by the reception unit 13 and sent to the separation unit 14. Separation unit 1
In 4, it separates data from the packet of the TS, and sends the decoding unit corresponding each of the plurality of decoding units 15 ₁ to 15 _n. The decoding unit 15 ₁ to 15 _n, decoding the respective supplied data, i.e., decodes the data encoded.

[0045] If the scene description data describe the configuration of the scene are being delivered, the scene synthesis unit 16 synthesizes the decoded data from the decoding unit 15 ₁ to 15 _n according to the scene description data. The combined data subjected to the scene combining by the scene combining unit 16 is sent to, for example, a display device or a speaker (not shown), and is displayed and emitted as a scene image and a sound. Note that a plurality of decoding terminals 12 may be connected.

In the case where the special reproduction display is performed at the decoding terminal 12 of the data distribution system, for example, a special reproduction designation signal 6 corresponding to an operation by the user of the decoding terminal 12 is transmitted to the decoding terminal 12 shown in FIG. It is transmitted to the server 10 via the transmission medium 21 from a transmission medium interface unit or the like that does not. The special reproduction designation signal 6 is a signal including a type of special reproduction such as fast-forward reproduction, rewind reproduction, frame-forward reproduction, and slow reproduction, and designation of data stored in the storage unit 9. If the server 10 and the decoding terminal 12 are connected in a short distance like a home network, for example, and the user can operate a front panel or a remote controller of the server 10, the front of the server 10 When a user operates a panel or a remote controller, the server 10
It is also possible to directly input the special reproduction designating signal 6 to.

The special reproduction designating signal 6 input to the server 10 is input to the special reproduction control unit 1 provided in the server 10. The trick play control unit 1 generates a trick play control signal 2 including a trick play type and data designation according to the trick play designation signal 6 and sends it to the data conversion unit 7. It should be noted that any number of data conversion units 7 may exist according to the number of data to be distributed.

The data conversion section 7 reads out data from the storage section 9 under the control of the special reproduction control section 1 by the control signal 2 and realizes the special reproduction of the type specified by the control signal 2. To the special reproduction data to be reproduced.

Hereinafter, the detailed configuration and operation of the data converter 7 in the data distribution system according to the embodiment of the present invention will be described.

FIG. 2 shows a detailed configuration of the server 10 of the data distribution system including the data conversion unit 7 according to the first embodiment of the present invention. The operation of each component other than the data conversion unit 7 is the same as described above, and a detailed description thereof will be omitted.

In FIG. 2, a data conversion section 7 of the first embodiment includes a reading section 17 for reading data from the storage section 9 under the control of the control signal 2 from the special reproduction control section 1; A time information rewriting unit for rewriting time information encoded in the output data in accordance with special reproduction; When there are a plurality of data conversion units 7, the reading unit 17 in the data conversion unit 7 may have a common configuration for all the data conversion units 7.

The reading section 17 reads the data for normal reproduction specified by the control signal 2 from the special reproduction control section 1 from the storage section 9 and sends it to the time information rewriting section 19.

The time information rewriting section 16 converts the time information of the data for normal reproduction read from the storage section 9 by the reading section 17 into the time information of the data converted according to the special reproduction. And encodes it in the output data. The time information of the data is a data arrival time, a display start time, a display end time, a display time or a decoding time. In the case of audio data, each of these time information is actually a time related to sound emission, but since image display and sound emission are related,
Expressions such as the display start time, the display end time, and the display time are used as described above. The same applies to the following description. In the first embodiment, the data whose time information has been rewritten by the time information rewriting unit 16 is sent to the multiplexing unit 4.

The conversion process of the time information in the time information rewriting unit 19 of the data conversion unit 7 will be described with reference to FIG. Note that the example of FIG. 3 shows an example of a conversion process of time information when fast-forward playback is realized.

FIG. 3A shows a case where the time information conversion processing for the special reproduction by the time information rewriting section 16 is not performed on the normal reproduction data read from the storage section 9 (ie, the decoding terminal). 12 when normal reproduction is performed). In some encoding methods such as MPEG2 video, the actual display order and the encoding order (the order in which data is encoded in the bit stream) may be different. In order to make it easy to understand, they are shown according to the display order. AU30, AU31, AU32, and the like in FIG. 3 each represent one display unit of data, and correspond to a picture in the case of video data. Data encoding is usually performed for each display unit. This display unit, that is, the coding unit, is hereinafter referred to as an AU (access unit). 1 AU
Starts the display from the display start time Ts, and the display time ΔT
The display ends at a later display end time Te. In addition,
The display time Δ of one AU generally differs depending on the encoding method.

On the other hand, FIG. 3B shows the time information conversion for the special reproduction (fast forward reproduction in this case) by the time information rewriting section 16 for the normal reproduction data read from the storage section 9. This indicates the display timing of the converted data when the process is performed, that is, when the trick play is performed in the decoding terminal 12. That is, FIG.
(B) shows that a fast forward playback section (special playback section) starts from the middle of the AU 30 'in the normal playback section, the AU 31' becomes a fast forward playback section, and the AU 32 'after the AU 31' becomes a normal playback section. It shows the display timing in the case.

Here, when fast forward reproduction is performed as special reproduction as shown in FIG. 3, the time t when the conversion processing for the special reproduction is not performed (hereinafter referred to as the time t before conversion). The relationship between the above time T and the time T ′ on the time t ′ when the conversion processing for the special reproduction is performed (hereinafter referred to as the output time t ′ after the conversion) is Will change each time you do.

For this reason, the data conversion unit 7 (time information rewriting unit 19) of the embodiment of the present invention sets the time T ′ on the time t ′ of the converted output to the time t ′ of the converted output. Using the above special reproduction start time To 'and the special reproduction start time To on the time t before the conversion (the start time on the time t before the conversion corresponding to the special reproduction start time To'), the expression ( It is calculated as in 1). T ′ = To ′ + (T−To) / n (1) where n in the expression (1) represents a reproduction speed during special reproduction, and the value of n is 2 for double-speed reproduction, and rewind reproduction. Let it be a negative value.

On the other hand, during normal reproduction, the time T ′ on the time t ′ of the converted output is changed to the time t ′ of the converted output.
The special reproduction end time Ti ′ above and the special reproduction end time Ti (special reproduction start time T
(end time on the time t before conversion corresponding to i ′), and is calculated as in equation (2). T ′ = Ti ′ + (T−Ti) (2) Also, during normal reproduction, the last special reproduction end time does not change, so that the special reproduction start time at the start of the next special reproduction is given by equation (2). Is obtained by using Equation (3). To ′ = Ti ′ + (To−Ti) (3) Based on the above equations (2) to (3), the data conversion unit 7 outputs the converted output time during normal reproduction and special reproduction. The AU display start time Ts' and display end time Te 'on t' can be calculated based on the AU display start time Ts and display end time Te on time t before conversion. Further, the display time ΔT ′ is obtained by multiplying the display time ΔT on the time t before conversion by 1 / n (n is the reproduction speed), or the display time T ′ from the display end time Te ′.
It is calculated by subtracting s'.

In this embodiment, the special reproduction start time, the special reproduction end time, and the special reproduction speed n are:
From the special reproduction control unit 1 together with the control signal 2,
It is specified to the data converter 7. The special reproduction start time, the special reproduction end time, and the special reproduction speed n may be specified by another data conversion unit (not shown). That is, for example, the data distribution system of the present embodiment
The aforementioned Patent Application No. 2000-178999 and Patent Application 200
As shown in FIG. 40 of No. 0-179000, a data conversion unit 223 for converting video data for special reproduction is provided.
When the data conversion unit 223 determines the special reproduction end time, the special reproduction start time, and the special reproduction speed in accordance with the display timing of the video data, the data conversion unit 223 transmits the special reproduction end time, The special reproduction start time and the special reproduction speed may be directly specified to the data conversion unit 7 of the present embodiment.

According to the data distribution system of the present embodiment, as described above, during normal reproduction and during special reproduction,
By calculating the display start time Ts 'and the display end time Te' of the AU on the converted output time t 'and calculating the display time ΔT', the time information rewriting unit 19 allows The display time, display end time, and display time to be encoded can be rewritten according to the special reproduction. When time information such as a decoding time and a data arrival time is also encoded in the data, the time information rewriting unit 19 also converts the time information based on the equations (1) and (2). It can be converted into time information on time t 'and output.

As described above, according to the present embodiment, when the trick play is executed in the decoding terminal 12, the time information of the normal playback data is converted after the conversion in accordance with the trick play. The data is converted into time information, and the time information is encoded into data and distributed from the server 10. That is, according to the data distribution system of the present embodiment, the distribution data received by decoding terminal 12 has already been converted to time information for special reproduction in server 10, and thus decoding terminal 12 performs decoding for special reproduction. No special processing is required, and if decoding and display are performed at a timing based on time information such as display time as in normal reproduction, a display result of the result of special reproduction is automatically obtained. That is, the decoding terminal 12 in the present embodiment does not need to be a special terminal capable of handling special distribution data for special reproduction without performing special processing for special reproduction. Furthermore, according to the present embodiment, since a plurality of data to be distributed are converted in accordance with the same playback speed, there is no deviation in synchronization between the plurality of data, and deviations may accumulate. There is no.

Next, referring to FIG. 4, which is represented in the same manner as in FIG.
A description will be given of a process of converting time information in the time information rewriting unit 19 when performing slow reproduction as special reproduction.

FIG. 4A is similar to FIG. 3A.
This shows the display timing of the data for normal reproduction on the time t before the conversion. AU40, AU41, AU42 in FIG.
Etc. each represent one display unit of data. Also,
FIG. 4B shows the converted data when the time information conversion processing for the special reproduction (in this case, slow reproduction) by the time information rewriting unit 16 is performed, similarly to FIG. 3B. This shows the display timing. That is, in FIG. 4 (b), the AU 40 'in the normal playback section becomes a slow playback section, the AU 41' becomes a slow playback section, and the AU 42 'after the AU 41' becomes a normal playback section. It shows the display timing in the case.

Here, when, for example, 0.5 × speed reproduction is performed as the special reproduction, the data conversion unit 7 according to the embodiment of the present invention is used.
In the (time information rewriting unit 19), the value of the reproduction speed n is set to 0.5, and the calculation of the expression (1) is performed.

As in the example of FIG. 4, even in the case of performing special reproduction at a reproduction speed lower than the unit speed, the time information conversion processing in the data conversion unit 7 of the present embodiment is effective as described above. Therefore, in the decoding terminal 12, if the same decoding and display as in normal reproduction is performed without special processing for slow reproduction, a display result or the like as a result of performing slow reproduction can be obtained.

Next, referring to FIG. 5, which is represented similarly to FIG.
A description will be given of a process of converting time information in the time information rewriting unit 19 when performing special reproduction such as a jump for moving a reproduction position to a temporally discontinuous display unit.

FIG. 5A is similar to FIG. 3A.
This shows the display timing of the data for normal reproduction on the time t before the conversion. AU50, AU51, AU52 in FIG.
Etc. each represent one display unit of data. Also,
FIG. 5B shows a trick play (jump in this case) by the time information rewriting unit 16 as in FIG. 3B.
Shows the display timing of the converted data when the time information conversion process is performed. That is, FIG.
In (b), the jump is performed from the middle of the AU 50 'in the normal playback section, and the special playback start time To' which is the start time of the jump and the special playback end time Ti 'which is the end time of the jump are performed. AU51 is not output, and AU5
This shows the display timing when the AU 51 ′ after the special reproduction end time Ti ′ is output following the special reproduction start time To ′ on 0 ′.

Here, in the case of a jump, since there is no reproduction speed during the special reproduction, the special reproduction control unit 1 specifies the special reproduction start time and the special reproduction end time to the data conversion unit 7. The trick play start time can be converted between the trick play start time To on the time t before the conversion and the trick play start time To 'on the time t' after the conversion by the equation (3). It can be specified either before or after conversion. The trick play end time designates the trick play end times Ti and Ti 'both before and after the conversion. However, when the trick play end time To 'on the converted time t' is equal to the trick play start time Ti ', Ti' need not be specified.

In the case of the example of FIG. 5, the data conversion unit 7
AU between jump start time To 'and end time Ti'
The AU 50 which does not output 51 and is displayed across the jump start time To 'changes or outputs the time information so that the display end time becomes To', or does not output it. Further, the AU 52 displayed across the jump end time Ti 'changes or outputs the time information so that the display time becomes Ti', or does not output it.

As in the example of FIG. 5, even in the case of performing special reproduction such as a jump for moving a reproduction position to a temporally non-continuous display unit, the time information in the data conversion unit 7 of the present embodiment is also used. Is effective in the same manner as described above. Therefore, if the decoding and display are performed in the decoding terminal 12 in the same manner as in the normal reproduction without any special processing for jumping, the display result of the result of the jump is obtained. Can be obtained.

Further, according to the present invention, scene description data describing the configuration of a scene is converted according to special reproduction, so that the scene description data can be distributed and decoded even during special reproduction. Therefore, it is possible to avoid such a disadvantage that the scene displayed at the start and end of the special reproduction is discontinuous, for example.

In the above example, when time information such as display time and decoding time is encoded and added to the data itself, the time information rewriting unit 19 of the data conversion unit 7
However, in the case where the time information is rewritten and output, for example, when the time information is added to the data by the multiplexing unit 4, the time is transmitted from the data conversion unit 7 to the multiplexing unit 4. The change of information is notified, and the multiplexing unit 4 adds the changed time information to the data. Alternatively, when the time information is added to the data by the transmission unit 5, the data conversion unit 7 similarly notifies the transmission unit 5 of the change in the time information, and the transmission unit 5 adds the time information after the change. I do.
This can be similarly applied to other embodiments described later.

By the way, in a data distribution system that distributes multimedia data such as video data, audio data, text data, and graphic data and scene description data, and decodes and displays the data, even if special reproduction is performed, the There is a demand that data be distributed as data that satisfies evaluation criteria such as a rate.

That is, the delivery data during fast-forward playback as in the example of FIG. 3 is compressed on the time axis as compared with the delivery data during normal playback, and the average bit rate is the same as that during normal playback. On the other hand, in the case of a system that distributes data via a transmission medium as in the present embodiment, the bit rate allowed during distribution depends on the transmission capacity of the transmission medium and the capability of the decoding terminal. The upper limit is determined. For example, if the bit rate of the distribution data exceeds the upper limit of the bit rate permitted for the distribution, data delay or loss occurs. In such a case, for example,
It is considered that if the bit rate of the distribution data is restricted, it is possible to prevent the bit rate of the distribution data from exceeding the upper limit bit rate permitted at the time of the distribution.

Further, for example, if the data included in the distribution data within a certain period of time relatively increases, the difficulty of decoding, scene synthesis, and display increases, and there is a risk that the display may not be correctly displayed on the decoding terminal. In such a case, it is considered that, for example, by limiting the difficulty of decoding, synthesizing, and displaying the distribution data, it is possible to reduce the risk of incorrect display at the decoding terminal.

Therefore, in the second embodiment of the present invention,
By making it possible to distribute data that satisfies evaluation criteria such as a bit rate even during trick play, data delay and loss are prevented, and a decoding terminal can correctly display a scene.

FIG. 6 shows a detailed configuration of the server 10 of the data distribution system provided with the data converter 7 according to the second embodiment of the present invention.

In FIG. 6, a data conversion unit 7 reads out data from the storage unit 9 under the control of the control signal 2 from the special reproduction control unit 1 and reads out the data from the read unit 17.
And a time information rewriting unit 19 for rewriting time information encoded in output data according to special reproduction, and a scheduler 18 for selecting an AU to be output based on an evaluation criterion such as a bit rate. Note that the data conversion unit 7 converts the time information from the time of the normal reproduction data before the conversion to the time after the conversion, encodes the time information into the data, and outputs the data. This is the same as the case of the embodiment.

The conversion processing in the scheduler 18 of the data conversion unit 7 in the second embodiment will be described with reference to FIGS.

FIG. 7 is represented in the same manner as FIG. 3, and FIG.
Represents the display timing of the data for normal reproduction on the time t before the conversion, as in FIG. A in FIG.
U70, AU71, AU72, AU73, etc. each represent one display unit of data. In addition, FIG.
3C and 3D, as in FIG. 3B, the time information conversion processing for the special reproduction (jump in this case) by the time information rewriting unit 16 is performed, and the present embodiment is used. 4 shows the display timing of the converted data when the AU is selected by the scheduler 18 according to the bit rate allowed at the time of distribution. That is, in FIG. 7 (b), the AU 71 and the AU 71 are set by the scheduler 18 in the fast-forward playback section (special playback section).
72, and the AUs 71 and 72 are subjected to time information conversion processing by the
71 ′ and AU72 ′, and the subsequent AU73 ′ indicates the display timing when a normal playback section is set. In FIG. 7C, only the AU 71 is selected by the scheduler 18 in the fast-forward playback section, and the AU 71 is converted by the time information rewriting unit 16 into an AU 71 ′, while the AU 72 is not output. , AU73 'thereafter indicates the display timing when a normal playback section is set. In addition, FIG.
In the fast-forward playback section, neither AU71 nor AU72 is selected by the scheduler 18;
Reference numeral 73 'indicates a display timing when a normal reproduction section is set.

The special playback section (fast-forward playback section) on the time t before the conversion shown in FIG.
There are two AUs, AU1 and AU72. In the case of the first embodiment, the time information of the AU71 and AU72 is converted according to the special reproduction speed, and AU71 'and U72'.
Output as However, for example, as shown in FIG. 8, when performing special reproduction (fast-forward reproduction in the examples of FIGS. 7 and 8),
The bit rate of the distribution data changes according to the reproduction speed. The bit rate changed in this way is
Exceeding the permissible bit rates of the transmission medium and the decoding terminal will cause data delays and losses.

Therefore, the scheduler 18 provided in the data converter 7 of the present embodiment selects AUs to be output and AUs not to be output so as to satisfy the bit rate allowed for the distribution data. For example, the bit rate allowed for distribution data is equal to or higher than the bit rate BR81 when only AU71 is output and AU72 is not output, and AU71 and A
If the bit rate BR80 when both U2 are output is less than BR80, the scheduler 18 determines not to output the AU72. The converted output in this case is as shown in FIG. When the bit rate allowed for the distribution data is lower than the bit rate BR81 when only the AU71 is output and the AU72 is not output, the scheduler 18 determines that neither the AU71 nor the AU72 is output. The converted output in this case is as shown in FIG. On the other hand, if the bit rate allowed for the distribution data is equal to or higher than the bit rate BR80 when both AU71 and AU72 are output,
The scheduler 18 determines to output both the AU 71 and the AU 72. The converted output in this case is as shown in FIG. The AU selected and output by the scheduler 18 in this manner is thereafter
The time information rewriting unit 19 converts the time information based on the reproduction speed of the special reproduction as described above.

As described above, according to the second embodiment, the display unit (AU) in the normal reproduction data is selected and output so as to satisfy the evaluation criterion such as the bit rate. Even during reproduction, data that satisfies evaluation criteria such as a bit rate can be distributed. The evaluation criterion is not limited to the bit rate. For example, an evaluation criterion representing the degree of difficulty in decoding data, synthesizing a scene, displaying data, or the like, such as the number of polygons allowed in a given time or the number of nodes in scene description data may be used. Alternatively, an evaluation criterion that can limit data that can be output in a certain time, such as the number of characters in text data, may be used.

Further, the data conversion unit 7 according to the second embodiment of the present invention outputs the display unit (AU) output as described above.
When selecting a display unit that is not output, the display unit in which data is encoded without using prediction between display units is output with priority, and the display unit that is encoded using prediction is not output. You can also choose to As a result, the decoding terminal can perform predictive decoding using the display unit encoded as the prediction source without using the prediction.

In the second embodiment, an example has been described in which distribution data satisfying an evaluation criterion such as a bit rate can be output depending on whether an AU is selected and output. As in the third embodiment, by converting the content of the AU itself, it is possible to output distribution data that satisfies evaluation criteria such as a bit rate.

FIG. 9 shows a detailed configuration of the server 10 of the data distribution system according to the third embodiment of the present invention.

In FIG. 9, the server 10 has the same configuration as the first and second embodiments except that a filter 23 is provided at the output stage of the data converter 7 corresponding to any of the above-described embodiments. It is common with the form.

The filter 23 is provided with the first or second filter.
The data converted by the data conversion unit 7 of the embodiment for special reproduction, that is, the AU itself is converted so as to satisfy an evaluation criterion such as a bit rate. Note that a plurality of data conversion units 7 and filters 23 may exist. That is, the filter 23 of the third embodiment
Is not limited to selecting AUs to be output and AUs not to be output, as in the data conversion unit 7 of the second embodiment.
By converting U itself, data that satisfies evaluation criteria such as a bit rate is output. For example, in the case of text data, the amount of data to be distributed is reduced by reducing the number of characters included in one AU, and the data is converted into data satisfying a desired bit rate and output.

According to the present embodiment, by converting the AU itself, it is possible to distribute data that satisfies evaluation criteria such as a bit rate even during special reproduction.
The AU input to the filter 23 has already been converted into time information according to the special reproduction by the data conversion unit 7 of the first or second embodiment. If no special processing is required and the decoding terminal 12 performs the same decoding and display processing as during normal reproduction without special processing for special reproduction, display for special reproduction is automatically performed. Can be realized.

Hereinafter, a specific example of the filter 23 will be described.

As a first specific example of the filter 23,
For example, data in a scene description is handled for each division unit, and a scene description is converted and output for each division unit so as to satisfy an evaluation criterion such as a transmission capacity.
By using the filter 23 of the first specific example in combination with the data conversion unit 7 of the first or second embodiment of the present invention, evaluation criteria such as a bit rate can be set even during trick play. Delivery of data that satisfies is enabled.

The operation of the filter 23 of the first specific example applied to the third embodiment of the present invention will be described below.

The filter 23 of the first specific example converts the input scene description based on the hierarchical information. When outputting the scene description, the filter 23 obtains decoding terminal information indicating the decoding and display capabilities of the decoding terminal 12. The decoding terminal information includes the image frame when the decoding terminal 12 displays the scene description, the upper limit of the number of nodes, the upper limit of the number of polygons, and the upper limit of multimedia data such as audio and video included therein. This is information indicating decoding and display capabilities. The filter 23 receives, in addition to the decoding terminal information, hierarchical information to which information indicating the transmission capacity of the transmission medium 22 used for distributing the scene description is added. The filter 23, based on the hierarchical information,
The scene description input is converted into scene description data having a hierarchical structure.

The third embodiment provided with the filter 23 of the first specific example
According to the data distribution system of the embodiment, as described above, by converting the scene description based on the hierarchical information, the scene description data suitable for the transmission medium 22 used for distribution can be distributed. This makes it possible to distribute a scene description that matches the performance of the decoding terminal 12.

FIG. 10 shows the procedure of the scene description conversion process in the filter 23.

In FIG. 10, the filter 23 first
In step S200, the scene description is divided into division candidate units as described later. In FIG. 10, the number of the division candidate is represented by n. Also, in order to convert the input scene description into scene description data composed of a plurality of layers, the layer of the scene description data to be output is represented by m. The layer number m starts from 0, and the smaller the number is, the more basic the layer is.

Next, the filter 23 determines in step S201
It is determined whether the division candidate n can be output as the current layer based on the layering information. For example, when the number of bytes of data allowed in the current layer is limited by the layering information, whether the output scene description of the current layer is equal to or less than the limited number of bytes even if the division candidate n is added. Find out. In step S201, when it is determined that the division candidate n cannot be output to the current layer, the process proceeds to step S202.
Go to 03.

At step S202, the filter 23
Advances the layer number m by one. That is, the output to the current layer m is ended, and thereafter, the output to the scene description data of the new layer is performed. Then, the process proceeds to step S203.

At step S203, the filter 23
Outputs the division candidate n to the current hierarchy m. And
Proceed to step S204.

At step S204, the filter 23
Determines whether all division candidates have been processed, and if so, terminates the conversion process. On the other hand, if there are still division candidates, the process proceeds to step S205.

At step S205, the filter 23
Advances the division candidate number n by one. That is, the next division candidate is set as a processing target. Then, the processing is repeated from step S201.

Here, taking the MPEG4 BIFS as an example, division in the scene description conversion processing by the filter 23 shown in FIG. 10 will be described with reference to FIG.

First, the content of the scene description data shown in FIG. 11 will be described, and then the division in the scene description processing by the filter 23 will be described.

In FIG. 11, the ransform node 302
Is a node whose 3D coordinate transformation can be specified.
In the slation field 303, the translation amount of the coordinate origin can be specified. Some fields can specify other nodes, and the structure of the scene description has a tree structure as shown in FIG. In FIG. 12, ellipses represent nodes, broken lines between nodes represent event propagation paths, and solid lines between nodes represent parent-child relationships of the nodes. For a parent node, a node representing a field of the parent node is called a child node. For example, the Transform node 302 in FIG.
Has a Children field 304 that indicates a group of child nodes whose coordinates are transformed by the Transform node.
The node 305 and the Shape node 306 are grouped as child nodes. A node that groups child nodes in the Children field in this way is called a grouping node. Grouping nodes are ISO /
Refers to a node defined in Chapter 4.6.5 of IES 14772-1, which has a field consisting of a list of nodes. 4.6 of ISO / IES14772-1.
The field name is Children, as defined in Chapter 5.
Although there are special exceptions, the Children field will be described below as including such exceptions.

To place an object to be displayed in a scene,
The node representing the object is grouped together with the node representing the attribute, and further grouped by the node representing the arrangement position. The object represented by the Shape node 306 in FIG. 11 is arranged in the scene by applying the translation specified by the parent Transform node 302. The scene description in FIG. 11 includes a Sphere node 307 representing a sphere, a Box node 312 representing a cube, and a Con representing a cone.
e node 317 and Cylinder node 322 representing a cylinder
FIG. 13 shows a result of decoding and displaying the scene description of this example.

A scene description can include user interaction. ROUTE in FIG. 11 represents the propagation of an event. When the touchTime field of the TouchSensor node 305 assigned the identifier 2 changes, the ROUTE 323 changes the value of the TouchSensor node 318 assigned the identifier 5 as an event.
It indicates that it propagates to the startTime field. In VRML, an identifier is represented by an arbitrary character string following the keyword DEF, and in MPEG4 BIFS, a numerical value called a node ID (nodeID) is used as the identifier. TouchSensor node 305 is a Shape node 306 grouped in Children field 304 of Transform node 302 as its parent node.
If the user selects, touchTime
Output as an event. Sensors that work in this way together with the Shape node attached by the grouping node are hereinafter referred to as Sensor nodes. VR
Sensor node in ML is ISO / IEC147
Pointing defined in Section 4.6.7.3 of 72-1
-device sensors and the accompanying Shape node is Se
Shape grouped to parent node of nsor node
Refers to a node. On the other hand, the TimeSensor node 318
Output elapsed time as fraction_changed event for 1 second from tTime.

The ROUTE 324 causes the TimeSensor node 3
The fraction_changed event representing the elapsed time output from 18 is a ColorInter with the identifier 6
Propagated to the set_fraction field of the polator node 319. The ColorInterpolator node 319 has a function of linearly interpolating values in the RGB color space. ColorInterpolato
The key and keyValue fields of the r node 319 are value_ch when the value of the input set_fraction field is 0.
An RGB value [000] is output as an anged event,
If the value of the input set_fraction field is 1,
This indicates that an RGB value [111] is output as an event as value_changed. If the value of the input set_fraction field is between 0 and 1, value_changed
And outputs a value obtained by linearly complementing the RGB values [000] and [111] as an event. In other words, set_
value_chan if the value of the fraction field is 0.2
An event value of RGB value [0.2 0.2 0.2] is output as ged.

According to ROUTE 325, the value val of the linear interpolation result
ue_changed is the Materi assigned the identifier 4
Propagated to the diffuseColor field of al node 314. diffuseColor represents the diffuse color of the object surface represented by the Shape node 311 to which the Material node 314 belongs. ROUTE323, ROUTE324 and ROUTE3 above
By the event propagation by 25, a user interaction is realized in which the RGB value of the displayed cube changes from [000] to [111] for one second immediately after the user selects the displayed sphere. . This user interaction is ROUTE323, ROUTE32
4, ROUTE 325, and nodes related to the propagation of the event indicated by the thick line frame in FIG. 12, and data in the scene description necessary for user interaction is hereinafter referred to as data necessary for the event propagation. , Will be called. Note that the nodes other than those indicated by the thick line frame are not related to the event.

With respect to the scene description data of FIG. 11 given as an example as described above, the filter 23 of the first specific example of the present embodiment divides the scene description into division candidate units in step S200 of FIG. .

Here, the so-called Node Insertion command
Is used, the Children field of the grouping node is used as a division unit. However, assuming that data necessary for event propagation for user interaction is not divided, three division candidates D0, D1, D2 shown in FIG.
Becomes

Gr which is the highest node in the input scene description
The division unit including the oup node 300 is a division candidate D0 where n = 0. Transform node 315 or lower node is n
= 1 as a division candidate D1. n = 1 in the division candidate D1
Shape node 316 is a grouping node Trans
Since this is a Children field of the form node 315, it can be a separate division candidate.

However, in this example, the Transform node 31
Since No. 5 has no Children field other than the Shape node 316, the Shape node 316 is not set as another division candidate. Nodes below the Transform node 320 are set as n = 2 division candidates D2. Similarly, Shape node 3
21 or less may be used as another division candidate.

The division candidate D0 with n = 0 is always output to the hierarchy m = 0. In step S201 of FIG. 10, it is determined whether the division candidate D1 of n = 1 can be output to the layer of m = 0 based on the layering information.

Next, FIG. 14 shows an example of determination in the case where the data amount allowed for the hierarchy of the scene description data to be output is designated by the hierarchy information. In the example of A in FIG. 14,
If the division candidate D1 of n = 1 is also output to the layer m = 0, the data amount exceeds the data amount permitted for the layer m = 0, so the division candidate D1 of n = 1 is output to the layer m = 0. Judge as impossible.

Therefore, according to the procedure of step S202 in FIG. 10, the output of the hierarchy m = 0 shown in B in FIG.
Is determined to include only the division candidate D0 of
1 is output. According to the procedure of step S203, the division candidate D1 of n = 1 is output to the hierarchy m = 1.

When the same procedure is performed for the next division candidate D2 of n = 2, as shown at A in FIG.
Even if the division candidate D2 of n = 2 is output to 1, the data amount allowed for the sum of the hierarchy m = 0 and the hierarchy m = 1 is not exceeded. Therefore, as shown in C in FIG. Division candidate D2
Is output to the same level m = 1 as the division candidate D1 with n = 1.

According to the above procedure, the filter 23 converts the input scene description into the converted scene description data output of the layer m = 0 shown in B of FIG. 14 and the layer m shown in C of FIG.
= 1 is converted to a scene description data output consisting of two layers.

Further, the conversion example of the scene description indicated by A in FIG. 15 is obtained by inputting the same scene description as in FIG.
An example is shown in which conversion is performed on the basis of different hierarchical information and converted into a scene description data output composed of three layers.

In other words, the scene description indicated by A in FIG. 15 is the same as the case of FIG. 14, and the converted scene description data output of layer m = 0 indicated by B in FIG. The converted scene description data output of the layer m = 1 shown in FIG. 15 and the converted data output of the layer m = 2 shown in D in FIG.

In this example of the conversion result, the transmission medium of the transmission medium used for distributing the scene description has a low transmission capacity and can transmit only the data amount allowed for the layer m = 0. Only the scene description data of the indicated hierarchy m = 0 is distributed.

Even if there is only a scene description of the hierarchy m = 0,
Since data necessary for event propagation for user interaction is not divided, the same user interaction as before conversion can be realized in the decoding terminal 12.

For a transmission medium whose transmission capacity is sufficient for the total data amount of the layers of m = 0 and m = 1, m = 0 and B shown in FIG. The scene description data of both hierarchies of m = 1 is distributed each time indicated by C in 15.

The scene description data of the layer m = 1 is the Node I
Since it is inserted into the scene description of the layer m = 0 by the nsertion command, the decoding terminal 12 can decode and display the same scene description as before conversion.

The filter 23 of the first specific example can adapt to the case where the transmission capacity of the transmission medium 22 changes by converting the scene description based on the time-varying hierarchical information. . Note that the same effect is obtained when the converted scene description data is recorded on the transmission medium 22.

In the example of the conversion result shown in FIG. 15, the decoding terminal 12 that receives, decodes, and displays the scene description has low decoding and display capabilities, and can decode and display only the data amount allowed for the layer m = 0. For the terminal 12,
Only the scene description data of hierarchy m = 0 shown in B in FIG. 15 can be distributed. Even with only the scene description of the hierarchy m = 0, data necessary for event propagation for user interaction is not divided, so that the decoding terminal 12 can realize the same user interaction as before the conversion.

Further, for the decoding terminal 12 whose decoding and display capability is sufficient for the total data amount of the layers of m = 0 and m = 1, m = 0 shown in B in FIG. And the scene description data of both the m = 1 levels shown in C in FIG.

The scene description 100 data of the hierarchy m = 1 is
Since it is inserted into the scene description of the hierarchy m = 0 by the Node Insertion command, the decoding terminal 12 can decode and display the same scene description as before conversion.

As described above, according to the first filter 23, the decoding and display capabilities of the decoding terminal 12 are dynamically changed or changed by converting the scene description based on the time-varying decoding terminal information. Terminal 1 with high performance
It becomes possible to adapt to the case where 2 is added to the distribution target.

[0130] In MPEG4 BIFS,
In order to layer the scene description, a command for inserting a node may be used, or an Inline node may be used. In addition, EXTERNPROTO described in Chapter 4.9 of ISO / IEC14772-1 may be used. EXTERNPROTO is a method of referring to a node defined by a node definition method called PROTO in external scene description data. In MPEG4 BIFS, similarly to VRML,
EXTERNPROTO can be used.

The DEF / USE described in Chapter 4.6.2 of ISO / IEC14772-1 is a system in which the node
Name it with F and use US from other places in the scene description
E makes it possible to refer to the node defined by DEF.

Also in the MPEG4 BIFS, a numerical identifier called a node ID is provided in a node in the same manner as in the DEF, and the node ID is specified from another place in the scene description to use the node in the same manner as in the USE. Is possible.

Accordingly, when the scene description is hierarchized, I
D described in Chapter 4.6.2 of SO / IEC14772-1
If the part using EF / USE is not divided into different division candidates, it is possible to perform the scene description conversion without breaking the reference relation from the USE to the DEFed node.

FIGS. 14 and 15 show examples in which the data amount allowed for each layer is used as the layering information.
Hierarchical information may be information that can determine whether or not division candidates in a scene description can be included in scene description data of a certain layer. For example, the upper limit of the number of nodes included in the layer, or computer graphics included in the layer , The number of media data such as audio and video included in the hierarchy may be limited, and a plurality of pieces of hierarchical information may be combined.

As described above, the filter 2 of the first specific example
According to No. 3, since the input scene description is converted into scene description data having a plurality of hierarchical structures, when transmitting the scene description, the hierarchical structure of the scene description is used for the purpose of saving transmission capacity. Is possible.

Further, according to the filter 23 of the first specific example, the scene description is converted into scene description data composed of a plurality of hierarchies. By deleting only the hierarchical scene description data, it is possible to save part of the information of the content described by the scene description.

In addition, what has been described above is effective in any scene description method that can be divided without depending on the type of the scene description method.

Next, the operation of the filter 23 of the second specific example applied to the third embodiment of the present invention will be described.

[0139] The filter 23 of the second specific example is shown in FIG.
As shown in FIG. 6, the scene description processing unit 24 and the ES (Elemen
tary Stream) processing unit 25 and a control unit 26 for controlling the operation thereof. The scene description data is changed by the scene description processing unit 24, and multimedia data other than the scene description data is changed by the ES processing unit 25. Possible ones can be mentioned. ES processing unit 25
Performs conversion by, for example, re-encoding data into data having a different bit rate in accordance with the transmission capacity and the capability of the decoding terminal. Further, the scene description processing unit 24 adjusts the data amount by converting the contents of the scene description according to, for example, the transmission capacity of the transmission medium 22 and the processing capability of the decoding terminal 12. By using the filter 23 including the scene description processing unit 24 and the ES processing unit 25 in combination with the data conversion unit 7 according to the first or second embodiment of the present invention, even during special reproduction. It is possible to distribute data that satisfies evaluation criteria such as a bit rate. In this case, although not shown, the decoding unit 15 of the decoding terminal 12 includes an ES decoding unit that decodes the ES to restore video data and audio data, and a scene description and decodes the scene description. And an ES scene description decoding unit that forms a scene using video, audio data, and the like based on the scene description.

Here, the filter 23 of the second specific example is used.
The data distribution system according to the third embodiment is provided with a method for addressing the problem that, when the transmittable band of the transmission medium 22 or the traffic congestion state changes, delay or loss occurs in data to be transmitted. Then, the following is performed.

The transmitting section 5 of the server 10 has a function of adding a serial number (encoded serial number) to each packet of data to be transmitted to the transmission path (transmission medium 22). Monitors data loss (data loss ratio) by monitoring the loss of a serial number (encoded serial number) added to each received packet.
It has a function to detect Alternatively, the transmission unit 5 of the server 10 has a function of adding time information (encoded time information) to the data to be transmitted to the transmission path, while the reception unit 13 of the decoding terminal 12 transmits the data received from the transmission path. And a function for monitoring the time information (encoded time information) added to the information and detecting a transmission delay based on the time information. When the receiving unit 13 of the decoding terminal 12 detects the data loss ratio of the transmission path or the transmission delay in this way,
The detection information is transmitted (reported) to the transmission unit 5 of the server 10.

The transmitting section 5 of the server 10 has a transmission state detecting function.
The transmission band and traffic congestion state of the transmission line are detected based on information such as the data loss ratio or the transmission delay of the transmission line transmitted from the receiving unit 13 of FIG. That is,
The transmission state detection function determines that the transmission path is congested if the data loss is high, or determines that the transmission path is congested if the transmission delay increases. When a bandwidth reservation type transmission path is used, the transmission state detection function
It is possible to directly know the available bandwidth (transmittable bandwidth) available to the server 10. When a transmission medium such as a radio wave that depends on weather conditions is used, the user may set the transmission band in advance according to weather conditions. Transmission state detection information by the transmission state detection function is sent to the control unit 26 of the filter 23.

The control unit 26 controls the ES processing unit 2 on the basis of the information on the transmittable bandwidth of the transmission path and the traffic congestion state.
5, for example, control is performed such that ESs having different bit rates can be selectively switched, or the ES processing unit 25 performs ISO / IEC 13818 (so-called MP
When encoding such as EG2) is performed, control such as adjusting the encoding bit rate is performed. That is, for example, when it is detected that the transmission path is congested, if the ES processing unit 25 outputs an ES having a low bit rate, it is possible to avoid data delay.

Further, for example, an unspecified number of decoding terminals 12 are connected to the server 10, and the specifications of the decoding terminals 12 are not standardized in advance, and the server 10 is directed to the decoding terminals 12 having various processing capabilities. In the case of a system configuration in which the ES 10 transmits an ES, the receiving unit 13 of each of the decoding terminals 12 has a transmission request processing function, and the transmission request processing function requests the ES according to the processing capability of the decoding terminal 12 itself. To the server 10 for transmission. This transmission request signal also includes a signal indicating the capability of the decoding terminal 12 of itself. Examples of the signal that is transmitted from the transmission request processing function to the server 10 and that indicates the capability of the decoding terminal 12 itself include, for example, memory size, resolution of the display unit, computation capability
The buffer size, the encoding format of the decodable ES, the number of decodable ESs, the bit rate of the decodable ES, and the like can be cited. The transmitting unit 5 having received the transmission request signal converts the transmission request signal into the filter 23.
To the control unit 26, and the control unit 26
The ES processing unit 25 is controlled so that an ES that matches the performance of the ES is transmitted. The ES processing unit 25
As for the image signal conversion processing when the ES is converted so as to conform to the performance of the decoding terminal 12, there is, for example, an image signal conversion processing method already proposed by the present applicant.

Further, the control unit 26 controls not only the ES processing unit 25 but also the scene description processing unit 24 in accordance with the state of the transmission path detected by the transmission state detecting function of the transmitting unit 5. The control unit 26 also controls the decryption terminal 12
Is a decoding terminal that requests a scene description according to its decoding and display performance, the receiving unit 1 of the decoding terminal 12
The ES control unit 25 and the scene description processing unit 24 are controlled in accordance with the signal indicating the capability of the decoding terminal itself transmitted from the transmission request processing function 3. Note that the control unit 26, the scene description processing unit 24, and the ES processing unit 25 may have an integrated configuration.

[0146] Hereinafter, a method of selecting a specific ES to be transmitted from the plurality of ESs by the ES processing unit 25 under the control of the control unit 26 will be described.

The control unit 26 controls each E of the plurality of ESs.
For each S, transmission priority information indicating the priority at the time of transmission is held, and in accordance with the state of the transmission path at the time of transmitting the ES or a request from the decoding terminal 12, transmission is performed in the descending order of the transmission priority. Determine possible ESs. That is, the control unit 26
Controls the ES processing unit 25 so that the transmittable ESs are transmitted in descending order of the transmission priority in accordance with the state of the transmission path when transmitting the ES or a request from the decoding terminal 12. Here, for example, the description will be made assuming that the control unit 26 holds the transmission priority information, but it may be stored in the storage unit 9.

In FIG. 17, for example, ESa, ESb, ES
An example of the transmission priority of each ES when three ESs of c exist is shown. That is, in the example of FIG.
Transmission priority of “30” and ESb transmission priority of “2”
0 ", and the transmission priority of ESc is" 10 ".
It is assumed that the smaller the transmission priority, the higher the transmission priority. Also, Ra in FIG. 17 is a transmission bit rate when transmitting ESa, Rb is a transmission bit rate when transmitting ESb, and Rc is a transmission bit rate when transmitting ESc.

Here, if the transmittable bit rate R is determined according to the state of the transmission path or a request from the decoding terminal 12, the control unit 26 sets the transmittable bit rate R in descending order of transmission priority. The ES processing unit 24 is controlled so that the ES is selected and transmitted within a range not exceeding.

That is, for example, when the relationship between the transmittable bit rate R and the transmission bit rate of each ES is expressed by Equation (4), the control unit 26 sets the E to have the highest transmission priority.
The ES processing unit 25 is controlled so that only Sc is selected and transmitted.

Rc ≦ R <(Rc + Rb) (4) Also, for example, when the relationship between the transmittable bit rate R and the transmission bit rate of each ES is expressed by Expression (5), the control unit 26 The ES processing unit 25 is controlled so that the ESc having the highest transmission priority and the ESb having the second highest (second) transmission priority are selected and transmitted.

(Rc + Rb) ≦ R <(Rc + Rb + Ra) (5) Also, for example, when the relationship between the transmittable bit rate R and the transmission bit rate of each ES is expressed by Expression (6), the control unit 26 , So that all the ESs are selected and transmitted
It controls the S processing unit 25.

(Rc + Rb + Ra) ≦ R (6) As described above, the third example including the filter 23 of the third specific example
According to the data distribution system of the embodiment, the control unit 2
6 holds transmission priority information for each ES, and in accordance with the state of the transmission path when transmitting the ES and a request from the decoding terminal 12,
By determining the ESs that can be transmitted in descending order of transmission priority, it is possible to preferentially transmit an important ES from a plurality of ESs.

In the above description, an example is described in which the selection of the ES and the conversion of the scene description are performed based on the priority set in advance. However, the priority can be changed with the conversion of the ES. It is. When the priority is changed along with the conversion of the ES, the change of the priority is performed by, for example, the ES processing unit 25.

FIG. 18 shows that the bit rate of ESa is R
An example of the transmission priority converted by the ES processing unit 25 due to the conversion to a ′ is shown. Note that FIG.
Describes an example in which the bit rate of the ESa is set to a bit rate Ra lower than the bit rate Ra in the example of FIG. 17, and as the bit rate is lowered, the transmission priority is converted to, for example, higher ( What was “30” in FIG. 17 is converted to “15” in FIG.

Further, the transmission priority is not limited to the case where the control unit 26 holds a preset value.
It can be set according to encoding parameters such as the bit rate of S and the image frame. For example, as shown in FIG.
By maintaining the relationship Ps (R) between the bit rate R of S and the transmission priority, the transmission priority can be set according to the bit rate of ES. That is, for example, it is considered that the higher the bit rate, the higher the transmission cost. Therefore, as shown in the example of FIG. 19, the higher the ES bit rate, the lower the transmission priority, and the lower the transmission cost (the lower the bit rate). ) It is possible to transmit with priority on ES.

When the ES itself has an explicit frame such as image data, the transmission priority can be set according to the frame. For example, FIG. 20 shows an example of the relationship Ps (S) between the image frame area S of the ES and the transmission priority.
By holding s (S), the transmission priority is set to ES
Can be set in accordance with the image frame. That is, in general, it is considered that the transmission cost is higher as the image frame is larger. Therefore, as shown in the example of FIG. 20, by assigning a lower transmission priority as the image frame is larger, the ES which is considered to have a lower transmission cost is prioritized. Can be transmitted.

As described above, the method of setting the transmission priority according to the encoding parameters such as the bit rate and the picture frame of the ES is performed when the ES processing unit 25 changes the transmission priority with the conversion of the ES. Can also be used. For example, if the ES processing unit 25 converts the ES having the bit rate Ra into the bit rate Ra ′, the transmission priority is set to Ps as shown in FIG.
(Ra ').

The transmission priority is determined for moving images, still images,
It may be assigned for each type of ES such as text, or for each ES encoding format. For example, assuming that the highest transmission priority is always assigned to text, text data can always be transmitted with priority even if the transmittable bit rate is limited by the state of the transmission path or the request from the decoding terminal. It becomes possible.

Further, the transmission priority can be determined based on the user's preference. That is, the server 10
The type of the user's favorite ES and the encoding format are stored by retaining the type of the user's favorite ES such as moving images, still images, and text, and the preference information such as the ES encoding format and the ES encoding parameter. , A high transmission priority can be assigned to an ES having an encoding parameter. Thereby, even if the bit rate that can be transmitted is limited according to the state of the transmission path and the request from the decoding terminal,
It is possible to preferentially transmit an ES that matches the user's preference and display the ES with high quality.

As described above, the control unit 26 holds the transmission priority information for each ES, and can transmit the transmission priority information in the descending order of the transmission priority according to the state of the transmission path at the time of transmission or the request from the decoding terminal 12. Important ES by deciding important ES
S can be transmitted with priority.

The filter 23 of the third specific example applied to the third embodiment of the present invention satisfies the evaluation criterion such as the bit rate even during special reproduction as follows. Enables data distribution. That is, the scene description processing unit 24 provided in the filter 23 of the third specific example can perform the following first to fifth scene description processes under the control of the control unit 26.

As the first scene description processing, the filter 23 of the third specific example can output a scene description suitable for the ES output from the ES processing unit 25, for example.
That is, the scene description processing unit 24 can output a scene description suitable for ES output from the ES processing unit 25 under the control of the control unit 26. Hereinafter, the first scene description processing will be specifically described with reference to FIGS.

FIG. 21 shows a display example of a scene composed of a moving image ES and a still image ES. Esi in FIG. 21 indicates a scene display area, Emv in the figure indicates a moving image ES display area in the scene display area Esi, and Es in the figure
v indicates the still image ES display area in the scene display area Esi.

FIG. 22 shows a scene description corresponding to the scene display area Esi shown in FIG.
The content when described in S is represented by text.

The scene description shown in FIG. 22 includes two cubes, and on each surface, it is specified that a moving image and a still image are pasted as a texture.
Each object has been designated for coordinate transformation by a Transform node, and is indicated by # 500 and # 502 in the figure.
Depending on the value of the translation field (the origin position of the local coordinates), the object is translated and arranged in the scene. In addition, enlargement or reduction of the object included in the Transform node is specified by the values (scaling of the local coordinates) indicated by # 501 and # 503 in the figure.

Here, for example, when it is necessary to reduce the bit rate of the distribution data due to the state of the transmission path (transmission medium 22) or a request from the decoding terminal 12,
For example, a moving image ES that requires a large amount of data during transmission
It is assumed that ES conversion processing has been performed to lower the bit rate of the ES. At this point, it is assumed that, for example, a high-resolution still image ES has already been transmitted and stored on the decoding terminal side.

In this case, in the conventional data distribution system, decoding and display are performed in the same scene configuration regardless of whether the ES bit rate is adjusted or not. Become noticeable. That is, specifically explaining with reference to the example of FIG. 21, in the conventional data distribution system, the moving image ES to be displayed in the moving image ES display area Emv in FIG.
Decoding and display (with a wide moving image ES display area that does not match the actual bit rate) with the same scene configuration as before the adjustment even if the adjustment is made to lower the bit rate of the (Display on Emv), the moving image becomes coarse (for example, the spatial resolution is coarse), and the deterioration of the image quality becomes conspicuous.

On the other hand, when the bit rate of the moving image ES is reduced, for example, as shown in FIG.
If the display area Emv is narrowed, it is considered that deterioration of the image quality of the moving image displayed in the moving image ES display area Emv (in this case, deterioration of the spatial resolution) can be made inconspicuous. Can be In the case of the present embodiment, for a still image, the still image ES has already been transmitted and stored in the decoding terminal, but the still image is, for example, a high-resolution image, and the still image ES shown in FIG. In the case where the area Esv is a narrow area that does not match the resolution, for example, as shown in FIG.
It is considered that if the S display area Esv is widened, the resolution can be fully utilized. Thus, the moving image E
A measure to narrow the S display area Emv and widen the still image ES display area Esv cannot be realized unless the scene description is changed to a scene description representing such contents.

Therefore, the scene description processing unit 24 provided in the filter 23 of the third specific example includes an ES processing unit 25
The scene description is dynamically changed and output in accordance with the bit rate adjustment of the ES in. In other words, the control unit 26 in the third specific example uses the ES
When the processing unit 25 is controlled to adjust the ES bit rate, the scene description processing unit 24 outputs a scene description suitable for the ES output from the ES processing unit 25.
It also controls the As a result, deterioration in image quality when the bit rate of a moving image is reduced as in the above-described example is made inconspicuous. In this example, in order to utilize the resolution of a still image that has already been transmitted, the moving image ES display area Emv is narrowed as shown in FIG.
A measure such as widening the S region Esv is realized.

Referring to FIG. 24, a specific operation of control unit 26 for achieving the above will be described.

In FIG. 24, when it is necessary to lower the bit rate of the distribution data due to the state of the transmission path or a request from the decoding terminal 12, at time T, the control unit 26 lowers the bit rate than the moving image ES292. The ES processing unit 25 is controlled so that the output moving image ES293 is output.

At time T, the control unit 26 sets the scene description 29 corresponding to the scene display area Esi in FIG.
The scene description processing unit 24 is controlled so that 0 is changed to the scene description 291 corresponding to the scene display area Esi in FIG. That is, the scene description processing unit 24 at this time
The scene description shown in FIG. 22 representing the scene display area Esi in FIG. 21 under the control of the control unit 26 is shown in FIG.
The scene description is converted into a scene description as shown in FIG. Note that the scene description in FIG. 25 is also indicated by the content text of the scene description described in MPEG4 BIFS, as in the case of FIG.

Compared with the scene description shown in FIG. 22, the scene description shown in FIG.
Since the value of the translation field (origin position of the local coordinates) indicated by is changed, the two cubes are moved, and the transcripts indicated by # 601 and # 603 in the figure are displayed.
According to the value of the slation field (scaling of the local coordinates), the cube with the moving image (Emv in FIG. 23) attached to the surface is converted into a smaller one, and instead, the cube with the still image (Esv in FIG. 23) attached to the surface. The conversion is large.

As in the first scene description process, for example, the process of converting the scene description shown in FIG. 22 to the scene description shown in FIG.
A process of selectively reading and transmitting a scene description (scene description in FIG. 25) corresponding to the ES output from the ES processing unit 25 from among a plurality of scene descriptions stored in the storage unit 9 in advance, or storing The scene description (scene description in FIG. 22) read from the
The process of converting to a scene description corresponding to the ES output (scene description of FIG. 25) and transmitting the same, or generating the scene description data (scene description of FIG. 25) corresponding to the ES output by the ES processing unit 25 Alternatively, it is realized by performing a process of encoding and transmitting. When a scene description method capable of describing only a change in the scene description is used, only the change may be transmitted. Further, in the above-described example, the case has been described where the moving image ES display area Emv is narrowed when the bit rate of the moving image ES is reduced. Conversely, when the bit rate is increased, the moving image ES display area Emv is reduced. Even when Emv is expanded, the scene description conversion according to the present invention can be applied. Further, in the above example, the description has been given assuming that the high-resolution still images ES are transmitted and stored in advance, but, for example, when the still images transmitted and stored in advance are of low resolution. Needless to say, a new high-resolution still image ES may be transmitted, and a corresponding scene description may be transmitted. In this embodiment, a moving image and a still image have been described as examples. However, the present invention includes a case where a scene description is changed according to the bit rate adjustment of other multimedia data.

According to the first scene description processing described with reference to FIGS. 21 to 25, the scene description representing the configuration information of the scene is converted, whereby the state of the transmission path and In addition to being able to transmit a scene description according to the request, for example, when the ES processing unit 25 converts the ES, it is possible to transmit a scene description that is optimal for the converted ES.

Next, the second scene description processing will be described.

For example, when information necessary for decoding the ES changes by converting the ES bit rate and the like from the ES processing unit 25 in accordance with the transmission path and the state of the decoding terminal 12, the filter 23 outputs the second scene. As the description processing, the scene description itself including the information necessary for decoding the ES is also converted and transmitted, so that the decoding terminal does not need to extract information necessary for decoding from the ES data itself. That is, under the control of the control unit 26, the scene description processing unit 24 performs the ES conversion process in the ES
When the information necessary for decoding the ES changes, a scene description including the information necessary for decoding the ES can be output. The information necessary for decoding the ES includes, for example, the encoding format of the ES, the buffer size required for decoding, and the bit rate. Hereinafter, the second scene description process will be specifically described with reference to the above-described drawings and FIGS. 26 and 27.

FIG. 26 shows an example of information necessary for decoding an ES used in a scene as described above with reference to FIGS. 21 and 22 by using a descriptor ObjectDescriptor defined by MPEG4.
It is described by scriptor. In the scene description in FIG. 22, the moving image to be mapped as a texture on the object surface is designated by a numerical value of 3 (= url3), which is associated with ODid = 3 which is the identifier of the ObjectDescriptor in FIG. The ES_Descriptor included in the ObjectDescriptor with the identifier ODid = 3 describes information about the ES. Further, ES_ID in the figure is an identifier for uniquely specifying an ES. This identifier ES_ID is further associated with, for example, an identifier of a header or a port number in a transmission protocol used for transmitting the ES, thereby realizing the actual ES.
Is associated with.

In the description of ES_Descriptor, Decod
An erConfigDescriptor, which is a descriptor of information necessary for decoding the ES, is included. Descriptor DecoderConfigDescriptor
Is, for example, a buffer size, a maximum bit rate, and an average bit rate necessary for decoding the ES.

On the other hand, FIG. 27 shows an example of information necessary for decoding the ES accompanying the scene description after the conversion processing in the scene description processing unit 24 corresponding to the scene shown in FIG. Defined descriptor ObjectDescript
It is described by or. Moving image changed by ES conversion (ODid is 3 and referenced from scene description)
Decoding buffer size (bufferSiseDB), maximum bit rate (maxBitRate) and average bit rate (avgBitRa)
te) is the ObjectDescriptor shown in FIG. 26 before the conversion.
The description in FIG. 27 is converted as shown in FIG. That is, in the example of FIG. 26, bufferSiseDB = 4000, maxBitRate =
1000000, avgBitRate = 1000000, but in FIG. 27, bufferSiseDB = 2000, maxBitRate = 5000000, a
It has been converted to vgBitRate = 5000000.

As in the second scene description processing, the conversion processing of the information necessary for decoding the ES accompanying the scene description is performed by the scene description processing unit 24 using a plurality of ESs stored in the storage unit 9 in advance. A process of selectively reading and transmitting information (information in FIG. 27) corresponding to the ES output from the ES processing unit 25 from among information necessary for decoding the
Alternatively, information necessary for decoding an ES (information in FIG. 26) read from the storage unit 9 is converted into information (information in FIG. 27) output from the ES processing unit 25 and necessary for decoding the ES. This is realized by performing a process of transmitting the data, or a process of encoding and transmitting information (information in FIG. 27) necessary for decoding the ES output from the ES processing unit 25.

According to the second scene description process described above, when the information necessary for decoding the ES changes by converting the bit rate of the ES according to the transmission path and the state of the decoding terminal 12, As shown in FIG. 27, by changing the information necessary for decoding the ES included in the scene description and transmitting the changed information to the decoding terminal 12, the decoding terminal 12 obtains information necessary for ES decoding from the ES data itself. It is possible to eliminate the need for extraction.

Next, the third scene description processing will be described.

As a third scene description processing, a filter 2
3 explicitly converts and outputs a scene description so as to increase or decrease the number of ESs constituting a scene, thereby enabling transmission of only ESs corresponding to the transmission band.
In No. 2, it is possible to determine the ES required for display or the like without depending on the arrival delay of the ES data or the loss of the data. That is, the scene description processing unit 24 of this example
Under the control of the control unit 26, explicitly converts and outputs the scene description so as to increase or decrease the number of ESs,
The scene description decoding function provided in the decoding unit 15 determines the ES required for display or the like without depending on the arrival delay of ES data or loss of data. Hereinafter, the third scene description processing will be specifically described with reference to the above-described drawings and FIGS. 28 and 29.

FIG. 28 shows a scene description in which, for example, the ES of a moving image is deleted from the scenes described with reference to FIGS. 21 and 22, the scene description is described in MPEG4 BIFS (written as text for easy understanding). It is. FIG. 29 shows an example of a scene displayed based on the scene description in FIG. 28. The image display area Esi includes an image ES display area (for example, a still image ES display area) Ei.
m only. Since it is possible to determine from the scene description that the ES used in the scene description in FIG. 28 is only the ES having the ODid of 4, the decoding terminal 12
Even if the video ES data with the id of 3 has not arrived,
It can be determined that it is not due to the arrival delay of S data or data loss. Furthermore, by deleting the description of the ObjectDescriptor with the ODid of 3 as in the examples of FIGS. 26 and 27, it can be determined that the moving image ES with the ODid of 3 is unnecessary.

Also, in the example of the third scene description processing, when a transmission request for temporarily reducing the processing load for decoding and composing a scene is transmitted from the decoding terminal 12, the filter 23 For example, by changing the scene description shown in FIG. 22 to the scene description shown in FIG. 28, it is possible to notify the decoding terminal 12 that the process of mapping a moving image as a texture in a scene is not explicitly required. I can do it. This makes it possible for the decoding terminal 12 to reduce the processing load for decoding a scene.

As in the third scene description processing, the above-described conversion processing from the scene description shown in FIG. 22 to the scene description shown in FIG.
A scene description (scene description in FIG. 28) corresponding to the number of ESs output from the ES processing unit 25 is selectively read out from a plurality of scene descriptions prepared in the storage unit 9 in advance and transmitted. The processing or the scene description read from the storage unit 9 is input, and the partial data (data in the scene description) corresponding to the ES that is not output is converted into a deleted scene description (the scene description in FIG. 28) and output. Processing, or when encoding and outputting a scene description,
This can be realized by performing processing that does not encode a portion corresponding to an ES that is not output.

As described above, according to the third scene description processing, by converting the scene description as described above, the scene intended on the server 10 side can be converted at the intended timing by the decoding terminal 12 side. Can be restored. Further, according to the third scene description processing, the scene description processing unit 24 can sequentially delete partial data having lower importance in the scene description until the data conforms to the transmission band or the processing performance of the decoding terminal 12. Becomes According to the third scene description processing, the decoding terminal 1
In the case where there is a margin in the processing performance of No. 2, it is possible to transmit a more detailed scene description, and thereby, it is possible to decode, display, etc. a scene optimal for the processing performance of the decoding terminal 12. Become.

Next, the fourth scene description processing will be described.

As the fourth scene description processing, the server 10 of the present embodiment converts the complexity of the scene description in accordance with the state of the transmission path and the request from the decoding terminal 12 so as to convert the scene description. The data amount can be adjusted and the processing load on the decoding terminal 12 can be adjusted. That is, the scene description processing unit 24 of this example, under the control of the control unit 26, responds to the state of the transmission path and the request from the decoding terminal 12
The data amount of the scene description is adjusted and output. Hereinafter, FIG.
The fourth scene description processing will be specifically described with reference to FIGS.

FIG. 30 shows a scene description for displaying an object described by a polygon described in MPEG4 BIFS (described as text for easy understanding). In the example of FIG. 30, the coordinates of the polygon are omitted for simplification. In the scene description of FIG. 30,
IndexedFaceSet represents a geometric object formed by connecting the vertex coordinates specified by point in Coordinate in the order specified by CoordIndex. FIG. 31 shows a display example of a scene (display example of a polygonal object) displayed by decoding the scene description of FIG.

In the fourth example of the scene description processing,
Depending on the state of the transmission path, for example, when it is desired to reduce the amount of data transmitted by the server 10 or when a transmission request to reduce the processing load is transmitted from the decoding terminal 12, the scene description processing unit 24 of the filter 23 Convert a scene description into a simpler scene description. For example, in the example of the scene description shown in FIG. 32, In represents a polygon as shown in FIG.
The dexedFaceSet is changed to a Sphere representing a sphere as shown in FIG.
Thus, the data amount itself of the scene description can be reduced, and the load of decoding processing and processing for performing scene configuration in the decoding terminal 12 can be reduced. That is, in the case of a polygon as shown in FIG. 31, each value representing a polyhedron is required, whereas in the case of a sphere as shown in FIG. 33, these values are not required. Can be reduced. On the decoding terminal 12 side, a complicated process for displaying a polyhedron becomes a simple process for displaying a sphere, and the processing load is reduced.

As in the fourth scene description process, the process of converting the scene description shown in FIG. 30 into the scene description shown in FIG. 32 is prepared in the scene description processing unit 24, for example, in the storage unit 9 in advance. Selecting and outputting a scene description that satisfies an evaluation criterion suitable for the state of the transmission path or a request from the decoding terminal 12 from among a plurality of scene descriptions described above, or a scene read from the storage unit 9. It can be realized by inputting a description and converting it into a scene description that satisfies the evaluation criteria, or by encoding and outputting a scene description that satisfies the evaluation criterion. The evaluation criterion may be any criterion representing the complexity of the scene description, such as the data amount of the scene description and the number of nodes and polygons.

As another processing method for converting the complexity of the scene description in the scene description processing section 24, FIG.
(2) The process of replacing complicated partial data with simple partial data or the reverse process, or the process of removing the partial data or the reverse process, or changing the quantization step when encoding a scene description In such a case, the amount of scene description data may be adjusted. The data amount control of the scene description by the quantization step adjustment at the time of encoding can be realized, for example, as follows. For example, MPEG4 BIFS
In, it is possible to set quantization parameters indicating use / non-use of quantization and the number of bits used for each quantization category such as coordinates, rotation axis, angle, and size. Since the quantization parameter can be changed, for example, if the number of bits used for quantization is reduced, the data amount of the scene description can be reduced.

As described above, according to the fourth scene description processing, by converting the scene description, the scene simplified as intended on the server 10 side can be restored on the decoding terminal 12 side. It becomes possible. According to the fourth scene description processing, the scene description processing unit 24
Until the transmission bandwidth or the processing performance of the decoding terminal 12 is satisfied, it is possible to delete partial data in the scene description in ascending order of importance.

Next, the fifth scene description processing will be described.

As the fifth scene description processing, the server 10
The side adjusts the bit rate of the scene description data by dividing the scene description into a plurality of decoding units in accordance with the state of the transmission path and the request from the decoding terminal 12, and performs local processing in the decoding terminal 12. It is possible to avoid concentration of load. That is, the scene description processing unit 24 of this example
Under the control of the control unit 26, the state of the transmission path and the decoding terminal 1
2, the scene description is divided into a plurality of decoding units, the transmission timing of the scene description of the divided decoding units is adjusted and output. The decoding unit of the scene description to be decoded at a certain time is the same as the AU of the coding unit. Hereinafter, the fifth scene description processing will be specifically described with reference to FIGS.

In FIG. 34, for example, a sphere, a cube, a cone,
A scene description representing four objects in a cylinder is described in MPEG4 B
It is described in one AU of IFS. FIG.
5 shows a display example of a scene displayed by decoding the scene description of FIG. 34, in which four objects of a sphere 41, a cube 42, a cone 44, and a cylinder 43 are displayed. All the scenes described in one AU shown in FIG. 34 must be decoded at the designated decoding time and reflected on the display at the designated display time. Note that this decoding time (time at which the AU should be decoded and made valid) is determined by the MPEG
4 is called DTS (Decoding Time Stamp).

In the fifth example of the scene description processing,
Depending on the state of the transmission path or a request from the decoding terminal 12,
For example, if you want to reduce the bit rate of the transmitted data,
Alternatively, when it is desired to reduce the local processing load on the decoding terminal 12, the scene description processing unit 24 of the filter 23 divides the scene description into a plurality of AUs, and shifts the DTS for each AU to thereby reduce the local description of the scene description. The bit rate is adjusted to a bit rate that matches the state of the transmission path or the request from the decoding terminal 12, and the processing amount required for decoding processing for each DTS is adjusted to a processing amount that matches the request from the decoding terminal 12.

That is, the scene description processing section 24 first divides, for example, the scene description shown in FIG. 34 into four AU1 to AU4 as shown in FIG. Here, the first A
U1 describes that an ID of 1 is assigned to a group node that performs grouping, and that it can be referenced from a subsequent AU. MPEG4 BIFS
In this example, partial scenes can be added to grouping nodes that can be referred to later.
From the second AU2 to the fourth AU4, a partial scene is defined as a child of a Group node having an ID of 1 defined in the first AU1.
A command to be added to the ren field is described.

Next, the scene description processing unit 24 specifies the first AU1 to the fourth AU4 by shifting the DTS as shown in FIG. That is, the first
The first DTS1 is designated for the AU1 of the
The second DTS2 is designated for U2, the third DTS3 is designated for the third AU3, and the fourth DTS4 is designated for the fourth AU4. Thereby, the bit rate of the local scene description data from the server 10 to the decoding terminal 12 is reduced, and the load of the local decoding process generated for each DTS is reduced in the decoding terminal 12.

As shown in FIG. 38, scenes divided into four parts as shown in FIG. 36 and decoded and displayed by DTS1 to DTS4 have objects added for each DTS as shown in FIG. In the last DTS4, a scene similar to that in FIG. 35 is obtained. That is, the sphere 41 is displayed in DTS1, and the cube 42 is further displayed in DTS2.
Is added, and a cone 44 is further added in DTS3,
In the DTS4, by further adding the cylinder 43, four objects are finally displayed.

As in the fifth scene description process, the process of converting the scene description shown in FIG. 34 into the scene description shown in FIG. 36 is prepared in the scene description processing unit 24, for example, in the storage unit 9 in advance. Selecting and outputting a scene description that satisfies an evaluation criterion suitable for the state of the transmission path or a request from the decoding terminal 12 from among a plurality of scene descriptions described above, or a scene read from the storage unit 9. The description is input and converted into scene descriptions (AU1 to AU4) divided until the above evaluation criteria are satisfied, or scene descriptions (AU1 to AU4) divided until the above evaluation criteria are satisfied
Is encoded and output for each AU.
The evaluation criterion in the fifth scene description processing includes the data amount of one AU, the number of nodes included in one AU, the number of objects, the number of polygons, and the like, and the limit of the scene included in one AU. Any standard may be used.

As described above, according to the fifth scene description processing, the scene description is divided into a plurality of AUs,
By adjusting the DTS interval for each U, it is possible to control the average bit rate of the scene description, and it is possible to reduce the load of the decoding terminal 12 on the local decoding process. The average bit rate can be calculated by dividing the total amount of data of the AU having DTS included in a certain time interval by the above time interval. And an average bit rate suitable for the request from the decoding terminal 12
The interval of TS can be adjusted. Note that, in the above example, an example in which an AU is divided is described, but a plurality of AUs can be combined.

In the above description, an example is described in which the first to fifth scene description processes are individually performed. However, a plurality of scene description processes may be simultaneously performed by arbitrarily combining the scene description processes. It is possible. In this case, it is possible to simultaneously realize the above-described effects of the combined scene description processes.

Further, in the present embodiment, MPEG4 BIFS is described as an example of a scene description, but the present invention is not limited to this, and can be applied to any scene description method. Further, for example, when a scene description method capable of describing only a change in a scene description is used, the present invention is also applicable to a case where only the change is transmitted.

Furthermore, the above-described embodiment of the present invention can be realized by a hardware configuration or by software.

In the above description, HTML and MPEG4 BIFS are given as examples of scene descriptions.
In addition, the present invention can be applied to any scene description method such as VRML and Java (trademark).

The present invention is effective for any data encoding method regardless of data types such as video data, audio data, still image data, text data, graphic data, and scene description data. . Further, the present invention can be realized by hardware or software.

[0211]

According to the present invention, when the normal reproduction is performed on the receiving side, the data used for the normal reproduction is output, and when the special reproduction is performed on the receiving side, the encoding unit of the data used for the normal reproduction is output. By converting and outputting the time information related to the reproduction according to the special reproduction, when performing the special reproduction on the receiving side, for example, decoding and display of data other than the video can be performed, and the scene description data can be converted. The data can be distributed, decoded, etc., and the data can be distributed as data that satisfies evaluation criteria such as a transmission bit rate while maintaining a synchronous relationship between the data.

[Brief description of the drawings]

FIG. 1 is a block diagram illustrating a configuration example of a data distribution system according to an embodiment of the present invention.

FIG. 2 is a block diagram illustrating a detailed configuration of a server of the data distribution system according to the first embodiment.

FIG. 3 is a diagram used for explaining a time information conversion process when fast-forward playback is performed in the first embodiment.

FIG. 4 is a diagram used to describe a process of converting time information when performing slow playback in the first embodiment.

FIG. 5 is a diagram used to explain a process of converting time information when a jump is performed in the first embodiment.

FIG. 6 is a block diagram illustrating a detailed configuration of a server of the data distribution system according to the second embodiment.

FIG. 7 is a diagram used to describe a process of converting time information when fast-forward playback is performed in the second embodiment.

FIG. 8 is a diagram used to explain a change in bit rate when performing fast-forward playback in the second embodiment.

FIG. 9 is a block diagram illustrating a detailed configuration of a server of the data distribution system according to the third embodiment.

FIG. 10 is a flowchart illustrating a flow of a division process in a filter according to a first specific example of the third embodiment;

FIG. 11 shows an MPEG4 filter in the filter of the first specific example.
FIG. 9 is a diagram used to explain a scene description division candidate by BIFS.

FIG. 12 is a diagram used for describing the structure of the scene description in FIG. 11;

13 is a diagram illustrating decoding and display results of the scene description in FIG. 11;

FIG. 14 is a diagram illustrating a conversion result of the scene description of FIG. 11;

FIG. 15 is a diagram illustrating conversion candidates having different scene descriptions in FIG. 11;

FIG. 16 is a block diagram illustrating a detailed configuration of a filter according to a second specific example of the third embodiment.

FIG. 17 is a diagram used to explain the relationship between the transmission priority, the bit rate, and three ESs in the filter of the second specific example.

FIG. 18 is a diagram used to explain a change in bit rate and a change in transmission priority.

FIG. 19 is a relationship P between the bit rate R of the ES and the transmission priority.
It is a figure showing s (R).

FIG. 20 is a relationship Ps between the image frame area S of the ES and the transmission priority.
It is a figure showing (S).

FIG. 21 is a diagram illustrating a scene display result by a scene description before conversion in the first scene description processing.

FIG. 22 is a scene description (MPEG) corresponding to the scene of FIG. 21;
4 BIFS).

FIG. 23 is a diagram illustrating a scene display result by a converted scene description in the first scene description process.

FIG. 24 is a diagram used to explain the timing of ES conversion and scene description conversion in the first scene description processing.

25 is a scene description (MPEG) corresponding to the scene of FIG.
4 BIFS).

26 shows information (MPEG4 Object) attached to the scene description in FIG. 22, which is necessary for decoding the ES corresponding to the scene in FIG. 21;
FIG. 7 is a diagram illustrating an example of a Descriptor.

FIG. 27 shows information (MPEG4 Object) attached to the scene description of FIG. 25 necessary for decoding the ES corresponding to the scene of FIG.
FIG. 7 is a diagram illustrating an example of a Descriptor.

FIG. 28 is a diagram illustrating an example of a scene description (MPEG4 BIFS) when the ES of a moving image is deleted from the scenes described in FIGS. 21 and 22.

FIG. 29 is a diagram showing a display result based on the scene description of FIG. 28.

FIG. 30 is a diagram illustrating an example of a scene description (MPEG4 BIFS) for displaying an object described by a polygon.

FIG. 31 is a diagram showing a display result by the scene description shown in FIG. 30;

FIG. 32 is a diagram illustrating an example of a scene description (MPEG4 BIFS) in which an object described by a polygon is replaced by a sphere.

FIG. 33 is a diagram showing a display result based on the scene description shown in FIG. 32;

FIG. 34 shows a scene description composed of four objects (MPEG4 BIF
It is a figure showing the example of S).

FIG. 35 is a diagram showing a display result by the scene description shown in FIG. 34;

36 is a diagram illustrating an example of each scene description (MPEG4 BIFS) obtained by dividing the scene description illustrated in FIG. 34 into four AUs.

FIG. 37 is a diagram used to explain the decoding timing of each AU shown in FIG. 36.

FIG. 38 is a diagram showing a display result based on the scene description of each AU shown in FIG. 36;

FIG. 39 is a block diagram showing a schematic configuration of a conventional data distribution system.

40 is a block diagram illustrating a schematic configuration of a data distribution system that solves a disadvantage of the data distribution system illustrated in FIG. 39.

FIG. 41 is a diagram used for a brief description of an example (fast-forward playback) of an operation of the data converter for video data in the data distribution system of FIG. 40;

FIG. 42 shows an example of the operation of the data converter for video data in the data distribution system of FIG. 40 (rewind reproduction);
It is a figure used for brief explanation of.

FIG. 43 is a diagram used to describe a scene description using VRML and MPEG4 BIFS.

[Explanation of symbols]

1 trick play control section, 7 data conversion section, 4 multiplexing section, 5 transmission section, 9 storage section, 10 server,
12 decoding terminal, 13 receiving unit, 14 separating unit, 1
5 decoding section, 16 scene synthesizing section, 17 reading section, 18 scheduler, 19 time information rewriting section, 23 filter, 24 scene description processing section,
25 ES processing unit, 26 control unit

──────────────────────────────────────────────────続 き Continued on the front page (51) Int.Cl. ⁷ Identification code FI Theme coat ゛ (Reference) H04N 5/92 H04N 5/91 L 5C064 7/025 5/92 H 7/03 7/08 A 7/035 7/13 Z 7/24 7/173 610 (72) Inventor Yoichi Yagasaki 6-35 Kita-Shinagawa, Shinagawa-ku, Tokyo F-term in Sony Corporation (reference) 5C025 CA02 CA09 CA18 CB10 DA01 5C052 AC03 AC05 CC06 CC11 DD10 5C053 FA23 GB37 KA04 KA05 LA11 LA15 5C059 KK37 MB02 MB22 PP01 PP04 PP19 PP20 RB02 RC19 RC32 RC33 RC34 SS02 UA02 UA05 5C063 AA01 AB03 AB07 AC02 AC05 AC10 CA20 CA23 CA29 CA31 CA36 DA02 DA03 DA01 DA07 BA07 EB01 BC16 BC23 BC25 BD08 BD09

Claims (40)

[Claims] In a data processing method for transmitting data encoded for each predetermined coding unit to a receiving side, when performing normal reproduction on the receiving side, data used for the normal reproduction is output. A data processing method for converting the time information related to the reproduction of the coding unit of the data used for the normal reproduction in accordance with the special reproduction and outputting the data when the special reproduction is performed on the receiving side. Method. 2. When performing the special reproduction on the receiving side, a coding unit at the time of outputting the data used for the normal reproduction is selected so as to satisfy an evaluation criterion regarding the restriction of data that can be output within a certain time. 2. The data processing method according to claim 1, wherein time information related to reproduction of the selected coding unit is converted and output according to the special reproduction. 3. The encoding unit in which the time information is converted in accordance with the trick play is converted and output so as to satisfy an evaluation criterion relating to a limit on data that can be output within a predetermined time. The data processing method according to claim 1. 4. The time information related to the reproduction of the coding unit includes a reproduction start time, a reproduction time, a reproduction end time, a decoding time, and a data arrival time when the receiving unit reproduces the coding unit. 2. The data processing method according to claim 1, comprising any one or a combination. 5. The data processing method according to claim 1, wherein time information relating to reproduction of the coding unit of data used for the normal reproduction is converted according to a reproduction speed of the special reproduction. . 6. When performing jump reproduction for moving a reproduction position to a temporally discontinuous encoding unit as the special reproduction, an output of the encoding unit corresponding to the start time and the end time of the jump reproduction is performed. 2. The data processing method according to claim 1, wherein the data processing is stopped. 7. The time information is changed so that a reproduction end time of a coding unit reproduced over a start time of the trick play becomes a start time of the trick play. Data processing method. 8. The time information is changed so that the reproduction start time of an encoding unit reproduced over the special reproduction end time becomes the special reproduction end time. Data processing method. 9. When performing a jump playback for moving a playback position to a temporally non-continuous coding unit as the special playback, a coding unit reproduced over a start time or an end time of the jump playback. 7. The data processing method according to claim 6, wherein the output is stopped. 10. The coding unit according to claim 2, wherein, when selecting the coding unit, a coding unit in which the data is coded without using prediction between coding units is preferentially selected. Data processing method described. 11. The data processing method according to claim 2, wherein the evaluation criterion is a bit rate of output data. 12. The data processing method according to claim 2, wherein the evaluation criterion is a degree of difficulty in decoding scene description data when a scene is formed from the data. 13. The data processing method according to claim 2, wherein the evaluation criterion is a degree of difficulty in scene synthesis and scene reproduction of scene description data when a scene is formed from the data. 14. A data processing device for transmitting data encoded for each predetermined encoding unit to a receiving side, wherein when performing normal reproduction on the receiving side, data used for the normal reproduction is output. When performing special reproduction on the receiving side, the reception side may include data conversion means for converting and outputting time information related to reproduction of the coding unit of the data used for the normal reproduction in accordance with the special reproduction. Characteristic data processing device. 15. The data conversion means according to claim 1, wherein when performing the special reproduction on the receiving side, the data conversion means is adapted to output the data used for the normal reproduction so as to satisfy an evaluation criterion relating to a restriction on data that can be output within a predetermined time. 15. The data processing apparatus according to claim 14, wherein the encoding unit is selected, and time information related to reproduction of the selected encoding unit is converted and output according to the special reproduction. 16. The data conversion means converts the coding unit obtained by converting the time information in response to the trick play so as to satisfy an evaluation criterion relating to a limit on data that can be output within a predetermined time, and outputs the converted coding unit. 2. The method according to claim 1, wherein
5. The data processing device according to 4. 17. The time information related to the reproduction of the coding unit includes a reproduction start time, a reproduction time, a reproduction end time, a decoding time, and a data arrival time when the reception side reproduces the coding unit. 15. The data processing device according to claim 14, wherein the data processing device includes any one or a combination. 18. The data conversion means according to claim 1, wherein said data conversion means converts time information relating to reproduction of said coding unit of data used for said normal reproduction according to a reproduction speed of said special reproduction. 15. The data processing device according to 14. 19. The data conversion means, when performing the jump reproduction for moving the reproduction position to a temporally discontinuous coding unit as the special reproduction, corresponds to the start time and the end time of the jump reproduction. The data processing apparatus according to claim 14, wherein output of the coding unit is stopped. 20. The data conversion means, wherein the time information is changed so that the reproduction end time of the coding unit reproduced over the special reproduction start time becomes the special reproduction start time. The data processing device according to claim 14, wherein 21. The data conversion means, wherein the time information is changed so that the reproduction start time of a coding unit reproduced over the special reproduction end time becomes the special reproduction end time. The data processing device according to claim 14, wherein 22. The data conversion means, when performing a jump reproduction for moving a reproduction position to a temporally discontinuous coding unit as the special reproduction, reproducing the data over a start time or an end time of the jump reproduction. 20. The data processing apparatus according to claim 19, wherein output of the coding unit to be performed is stopped. 23. The data conversion means, when selecting the coding unit, preferentially selects a coding unit in which the data is coded without using prediction between coding units. The data processing device according to claim 15, wherein 24. The data processing apparatus according to claim 15, wherein said evaluation criterion is a bit rate of output data. 25. The data processing apparatus according to claim 15, wherein the evaluation criterion is a degree of difficulty in decoding scene description data when constructing a scene from the data. 26. The data processing apparatus according to claim 15, wherein the evaluation criterion is a difficulty level of scene synthesis and scene reproduction of scene description data when a scene is composed from the data. 27. In a data transmission system including a transmitting device that transmits data encoded for each predetermined coding unit and a receiving device that receives the data, the transmitting device performs normal reproduction by the receiving device. When performing the special reproduction by the receiving device, the time information related to the reproduction of the coding unit of the data used for the normal reproduction is output to the special reproduction. A data transmission system comprising data conversion means for converting and outputting according to the data. 28. The data conversion means, when performing special reproduction in the receiving device, outputs the data used for the normal reproduction so as to satisfy an evaluation criterion relating to a restriction on data that can be output within a certain period of time. 28. The data transmission system according to claim 27, wherein the encoding unit is selected, and time information related to reproduction of the selected encoding unit is converted and output according to the special reproduction. 29. The data conversion means converts the coding unit obtained by converting the time information according to the special reproduction so as to satisfy an evaluation criterion relating to a restriction on data that can be output within a certain time, and outputs the converted coding unit. 3. The method according to claim 2, wherein
7. The data transmission system according to 7. 30. The time information related to the reproduction of the coding unit includes a reproduction start time, a reproduction time, a reproduction end time, a decoding time, and a reproduction time when reproducing the coding unit in the receiving device.
28. The data transmission system according to claim 27, comprising any one or a combination of data arrival times. 31. The data conversion means, according to a reproduction speed of the special reproduction, converts time information relating to reproduction of the coding unit of data used for the normal reproduction. 28. The data transmission system according to 27. 32. When the data conversion means performs the jump reproduction for moving the reproduction position to a temporally discontinuous coding unit as the special reproduction, the data conversion means corresponds to the start time and the end time of the jump reproduction. The data transmission system according to claim 27, wherein output of the coding unit is stopped. 33. The data conversion means, wherein the time information is changed so that the reproduction end time of a coding unit reproduced over the special reproduction start time becomes the special reproduction start time. 28. The data transmission system according to claim 27, wherein 34. The data conversion means changes the time information so that the reproduction start time of the coding unit reproduced over the special reproduction end time becomes the special reproduction end time. 28. The data transmission system according to claim 27, wherein 35. The data conversion means, when performing a jump reproduction for moving a reproduction position to a temporally discontinuous coding unit as the special reproduction, reproducing the data over a start time or an end time of the jump reproduction. 33. The data transmission system according to claim 32, wherein output of the coding unit to be performed is stopped. 36. The data conversion means, when selecting the coding unit, preferentially selects a coding unit in which the data is coded without using prediction between coding units. 29. The data transmission system according to claim 28, wherein: 37. The data transmission system according to claim 28, wherein the evaluation criterion is a bit rate of output data. 38. The data transmission system according to claim 28, wherein the evaluation criterion is a degree of difficulty in decoding scene description data when a scene is formed from the data. 39. The data transmission system according to claim 28, wherein the evaluation criterion is a degree of difficulty in scene synthesis and scene reproduction of scene description data when constructing a scene from the data. 40. In a transmission medium for transmitting data coded for each predetermined coding unit by a transmitting device to a receiving device, when the receiving device performs normal reproduction, the data used for the normal reproduction is When the normal reproduction is performed by the receiving device, the data used for the normal reproduction is transmitted. When the special reproduction is performed by the receiving device, the reproduction of the coding unit of the data used for the normal reproduction is performed. A transmission medium, wherein data obtained by converting time information related to the special reproduction according to the special reproduction is transmitted.