JP2001285832A - Data receiver, data receiving method, data transmitting method and data storage medium - Google Patents

Abstract

PROBLEM TO BE SOLVED: To compose first and second foreground pictures (adv), (mov) with a background picture (bg) at a time designated by scene description to display it. SOLUTION: This device is provided with a data request receiving means 105 which requests media data Dm1 and Dm2 of respective foreground picture to servers 13b and 13c and receives a message from the servers and a control data generation part 110a for controlling the data request receiving means 105 so that the request message of the media data may be issued to a corresponding server at a time earlier than the display starting time of each foreground picture by standby time based on information showing the standby time before starting the display of each foreground picture included in SMIL data Ds1.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

[0001] The present invention relates to a data receiving apparatus,
The present invention relates to a data reception method, a data transmission method, and a data storage medium, and more particularly to a transmission process for transmitting control data including a storage location of a media data and a reproduction start time from a server that provides the media data, And a data storage medium storing a program for performing the transmission and reception processes by a computer.

[0002]

2. Description of the Related Art In recent years, with the progress of compression coding technology of image data and audio data, and the expansion of data transmission capacity of networks such as the Internet and wireless networks, images, voices, texts and the like called media data have been developed. Services that handle this data have come to be seen.

Conventionally, such services have been mainly provided by a download method. In this download method, all media data required for reproduction is downloaded in advance from a server to a client terminal via a network, and after completion of the download, the client terminal reproduces and displays the media data.

[0004] Recently, however, in services handling media data as described above, a streaming system has been adopted instead of a download system. In this streaming method, a process of receiving media data from a server via a network at a client terminal and a process of reproducing and displaying the received media data at the client terminal are performed in parallel.

[0005] As described above, in the streaming system, the reproduction and display are performed before the reception of the media data is completed.
In a service that adopts this method, even when watching a long-time program, after requesting the program data,
The waiting time until the reproduction and display of the program data is reduced.

[0006] In the future, the service for providing media data as described above will not be limited to playback and display of media data alone such as image data and audio data, but will be limited to a plurality of media data. It is expected that the service will be extended to a service that can simultaneously play back and display media data such as moving image data, still image data, or text data.

Hereinafter, a description will be given of a process of simultaneously reproducing a plurality of media data by the streaming method and simultaneously displaying, for example, one background and two foregrounds.

FIG. 11A is a diagram for explaining the spatial arrangement of media data. Predetermined image space 1100
Is a rectangular background display area (bg area) 1110 on which a background image (bg) is displayed. In the background display area 1110, a first foreground image (adv) which is an image of an advertisement or the like is provided. Are arranged, and a rectangular second foreground display area (mov area) 1130 in which a second foreground image (mov) as a moving image is arranged. Is set.

Here, with respect to the predetermined image space 1100, the number of horizontal points corresponding to the number of pixels in the horizontal direction,
A coordinate system indicating a position in the image space 1100 is defined by the number of vertical points corresponding to the number of pixels in the vertical direction. For example, a background display area (entire scene) 111
The position of the upper left corner of 0 is a position where the number of horizontal points and the number of vertical points are each 0. The horizontal size (width) of the background display area (entire scene) 1110 is 300 points, and the background display area 11
The vertical size (height) of 10 is 200 points. The first foreground display area (adv area) 1
The position of the upper left corner of 120 is a position where the number of horizontal points is 0 and the number of vertical points is 150. The horizontal size (width) of the first foreground display area 1120 is 300
The first foreground display area 1120 has a vertical size (height) of 50 points. The position of the upper left corner of the second foreground display area (mov area) 1130 is a position where the number of horizontal points is 50 and the number of vertical points is 0. Second foreground display area 1
The horizontal size (width) of 130 is 200 points, and the vertical size (height) of second foreground display area 1130 is 150 points.

FIG. 11B is a diagram for explaining the temporal arrangement of the media data, and shows the timing at which the background image and the first and second foreground images are displayed on the predetermined image space. ing. In the example of temporal arrangement of media data shown in FIG. 11B, when the reference time T of the client terminal becomes the display start time Tbg of the background image (Tbg = 0 second), the background image (bg) is stored in the image space 1100. Appear. When the reference time T of the client terminal becomes the display start time Tadv (Tadv = 5 seconds) of the first foreground image (adv), the first foreground image (adv) is stored in the image space 1100.
Appears. Further, the reference time T of the client terminal is set to the display start time Tmov (Tmov) of the second foreground image (mov).
= 10 seconds), a second foreground image (mov) appears in the image space 1100.

In order for the client terminal to actually perform such processing of simultaneously reproducing and displaying a plurality of media data, each media data, that is, a background image (bg),
First foreground image (adv), second foreground image (mov)
(Scene description data) for synthesizing is necessary. This scene description data includes a temporal arrangement of individual media data (see FIG. 11B) and a spatial arrangement (FIG. 11A).
Reference). In addition, the scene description data includes “SMIL (Synchronized
Multimedia Integration Language) ”,“ HTML (H
yper Text Markup Language) + TIME (Timed Interac
tive Multimedia Extensions) "
Some are expressed in languages standardized by the Wide Web Consortium.

Hereinafter, SMIL, which is one of the languages for expressing the scene description data, will be described. FIG.
FIG. 4 is a diagram for describing an example of description content of scene description data by SMIL. <Smil>, </ smil>, <head>, </ head>, <layou> described at the beginning of each line of the scene description SD
t>, </ layout>, <root-layout>, <region>, <body>, <p
Character strings such as ar>, </ par>, and <video> are called elements (elements) and declare the content of the description following the element.

For example, the smil element and the / smil element
The line located between the line 710a including the element and the line 710b including the / smil element declares that the line is described in accordance with the SMIL standard. head element and / h
In the ead element, the rows located between the row 720a including the head element and the row 720b including the / head element correspond to the images (bg), (adv), and ( m
ov) declares that information for defining an area where the ov) is placed is described. Also, layout
Element and the / layout element are in particular the line 7 containing the layout element
Declares that rows 701 to 703 containing information on the arrangement area of the background image and the foreground image to be reproduced in parallel (simultaneously) are arranged between 30a and the row 730b including the / layout element. .

A root-layout element 701a declares that the description of the line 701 including this element specifies an image to be displayed as a background image (entire scene), a size of the background image, and the like. There are region elements 702a and 703a in rows 702 and 70
The description 3 declares that the size of one rectangular area where the foreground image is arranged and the position in the entire scene (image space) are specified.

The body element and the / body element include information (URL) indicating the location of the media data to be reproduced and the media in a row located between a row 740a including the body element and a row 740b including the / body element. It declares that information about the time at which data should be displayed is described. In addition, a par element and a / par element are particularly provided between a row 750a including a par element and a row 750b including a / par element, and media elements related to media data to be reproduced in parallel (simultaneously) and attributes associated therewith. Line 70 containing information
4,705 are declared as being grouped.

The video elements 704a and 705a
It declares that moving image data is specified by the description of the line including the eo element.

Further, "id", "width", and "id" described after the root-layout element, the region element, and the video element.
"Height", "left", "top", "src", "begin"
A character string such as is called an attribute, and specifies detailed information in a line including each element.

First, a root-layout element, a region element, and an id attribute in lines 701, 702, and 703 including a region element specify media data such as a background image, a first foreground image, and a second foreground image. Things.

Line 7 including a root-layout element 701a
The width attribute and the height attribute of 01 specify the width and height of the background image (entire scene). In this case, the size of the background (entire scene) is 300 points (width = "300"). , Vertical height 200 points (heigh
t = "200").

The width and height attributes of the lines 702 and 703 including the region elements 702a and 703a specify the height and width of the rectangular area, and the left and top attributes correspond to the upper left corner of the rectangular area. Is specified with respect to the upper left corner of the entire scene.

For example, in a line 702 including a region element,
region attribute value (region = adv) by id attribute (id = adv)
A first rectangular area 1120 (see FIG. 10A) in which the media data corresponding to is displayed is designated. The position of the upper left corner of the first rectangular area is defined as a position separated from the upper left corner of the image space by 0 point in the horizontal direction and 150 points in the vertical direction by the left attribute (left = 0) and the top attribute (top = 150). Has been specified. Further, as the size of the first rectangular area, a size of 300 points in width and 50 points in height is specified by a width attribute (width = 300) and a height attribute (height = 50).

In a row 703 including a region element, id
The second rectangular area 1 in which media data corresponding to the region attribute value (region = mov) is displayed by the attribute (id = mov)
130 (see FIG. 11A). This second
The position of the upper left corner of the rectangular area is 50 points horizontally and 0 point away from the upper left corner of the image space by the left attribute (left = 50) and the top attribute (top = 0). Specified. As the size of the second rectangular area, a size of 200 points in width and 150 points in height is specified by a width attribute (width = 200) and a height attribute (height = 150).

Here, the arrangement information described in the line 702 including the region element is applied to the media data specified by the region attribute value (region = adv) in the line 704 including the video element, The arrangement information described in the row 703 including the region element is applied to the media data specified by the region attribute value (region = mov) in the row 705 including the video element.

The src attribute in the rows 704 and 705 including the video elements 704a and 705a specifies the transmission scheme and the storage location of the media data on the server. The information specified by the src attribute is information necessary for separately requesting media data from the server since the media data such as the image is not attached to the SMIL data.

Here, lines 704 and 7 containing a video element
05, rtsp (Real Time Streaming Protocol), which is a protocol (procedure) for exchanging data request messages between a transmitting side and a receiving side, is specified as a transmission scheme. In a row 704 including a video element, data (adv.m) stored in the server (s2.com) as media data corresponding to the first foreground image (adv)
pg) is specified, and in the line 705 including the video element, data (mov.mpg) stored in the server (s3.com) is specified as media data corresponding to the second foreground image (mov) Have been.

Therefore, at the client terminal, RTSP (R), which is a media data transmission protocol (procedure), is used.
eal Time Streaming Protocol), the media data (adv.mpg) and the media data (adv.mpg) are transmitted to the servers (s2.com) and (s3.com) specified by the description of the lines 704 and 705 including the video element. mov.mpg) is issued. The media data is transmitted and received using RTP (Realtime Transport Protocol).

Lines 704 and 70 containing a video element
The begin attribute in No. 5 specifies the time at which the display of the media data starts when the display of the scene is started as a starting point (zero seconds). The temporal arrangement of each media data is “begin”. Determined by attributes and the like. In the description of the line 704 including the video element, the begin attribute is specified as 5 seconds (begin = "5"). That is, the temporal arrangement is specified so that the display of the first foreground image is started 5 seconds after the display of the scene is started. In the description of the line 705 including the video element, the begin attribute is specified as 10 seconds (begin = "10"). That is, the second
The temporal arrangement of the foreground image is designated so that the display is started 10 seconds after the display of the scene is started.

Next, a conventional data receiving apparatus mounted on a personal computer (personal computer) as the client terminal will be described. FIG. 13 is a block diagram for explaining a conventional data receiving device. This data receiving device 901 receives the scene description data shown in FIG.
The server requests and acquires the SMIL data shown in No. 1 from the server, and further requests and acquires the media data specified by the SMIL data from the server, and reproduces and displays the acquired media data.

That is, the data receiving device 901 includes a plurality of media data receiving means 902a and 902b for receiving and outputting image data (media data) Dm1 and Dm2 corresponding to individual images constituting a scene. The image data Dm1 and Dm2 output from the data receiving units 902902a and 902b are decoded and the decoded image data Dm1 and Dm2 are decoded.
a plurality of image decoding means 903a that output d1 and Dd2,
903b and the decoded image data D from the respective image decoding means.
A plurality of frame memories 904a and 904b for storing d1 and Dd2 in frame units, and decoded image data Dd1 and Dd2 read from each frame memory, and one scene is formed by each individual image based on the control data Dc1. And display means 905 for synthesizing and displaying image data corresponding to each individual image so that is formed.

Further, the data receiving device 901 has a third
SMIL request receiving means 906 for outputting a SMIL request signal Srd for requesting SMIL data Ds to a predetermined remote server based on the control data Dc3, and for receiving and analyzing the SMIL data Ds from the predetermined remote server. And the SMIL analysis data Da obtained by the analysis of the SMIL data, and stores information on the spatial arrangement and the temporal arrangement of the individual images corresponding to each video element as the first control data Dc1,
Based on a control data generation unit 907 that stores information on a transmission scheme of image data (media data) corresponding to an individual image and its storage location as second control data, and control data Dc2 from the data generation unit 907,
Request signal Srp for requesting image data from a remote server
And a response signal Sack to the request
And a clock request circuit 909 for providing data Sm obtained from the response signal to the data generation unit 907 and for providing time information to each unit constituting the data receiving device 901. .

The data receiving apparatus 901 includes a data receiving unit, an image decoding unit, and a frame memory.
It has the same number as the number of image data (media data) to be received. Further, the data requesting means 908 comprises:
A user operation requests scene description data for reproducing a predetermined scene.

Next, the operation will be described. FIG. 14 is a diagram for explaining a flow of a process in which the data receiving device acquires media data from a server.
al time Transport Streaming Protocol) sequence. Here, the data receiving device 901
Is mounted on a personal computer (personal computer) as a client terminal, and the data receiving device 901 is supplied with SMIL data shown in FIG. 12 as scene description data SD.

First, for example, when a user browses a home page described in HTML (Hyper Text Markup Language) using a web browser provided in a personal computer (PC), a predetermined SMIL on the home page is displayed. When the operation of clicking the area linked to the data is performed, the data receiving device 901 of the client terminal issues a SMIL request command (GET http://s1.com/scen) for requesting the SMIL data Ds.
e.smil) C1 is issued. This command C1 is sent to the server (s1.com) 13a by SMIL using HTTP.
Requires that data be provided.

The SMIL request command C1 is transmitted to the server 1
3a, the server 13a sends a reply (HTTP / 1.0 O) indicating that the command has been acknowledged.
K) R1 is issued to the client terminal, and SMIL data (scene.smil) Ds is supplied to the client terminal. Then, in the data receiving device 901 of the client terminal, the SMIL data Ds is received by the SMIL receiving means 906, and the SMIL receiving means 906
Then, a process of analyzing the SMIL data Ds is performed.
The SMIL analysis information Da obtained by the analysis of the SMIL data by the SMIL receiving unit 906 is transmitted to the control data generation unit 9.
07 and recorded.

That is, the control data generation unit 907 includes ro
Information about the size of the background image (entire scene) described as an ot-layout element, or src, top, left, and width attributes described as a video element
Stores information about attributes, height attribute, and begin attribute. Specifically, the src attribute information includes information indicating a storage location of each image data, and the top attribute information and the left attribute information respectively include a foreground image based on the upper left position of the scene. The information on the position of the rectangular area in which is placed is included. Also, width attribute information and hei
The ght attribute information includes information on the horizontal and vertical sizes of the rectangular area. More begin
The attribute information includes a display start time at which display of media data corresponding to each video element is started.

In the display means 905, the control data generator 9
The process of creating and displaying a scene based on the content stored in 07 is started. Specifically, at the same time as the start of the display processing, the background image (b
g) is displayed over the entire image space 1100. At this time, the time information output by the clock circuit 909 is set to zero.

In the SMIL data Ds, the first
The display start time of the foreground image (adv) is set to 5 seconds and the display start time of the second foreground image (mov) is set to 10 seconds.
The operation of synthesizing the image data with reference to the frame memories 904a and 904b is not performed.

Thereafter, when the time information output by the clock circuit 909 reaches 5 seconds, the data requesting means 908 and the second server 908 are controlled based on the src attribute of the video element 704a stored in the control data generator 907. (S2.com) 1
3b, a message exchange requesting image data (adv.mpg) corresponding to the first foreground image is performed using RTSP (Real Time Streaming Protocol) as a communication protocol.
This is performed using After that, the server sends the image data (adv.mpg) to the RTP (Realtime Transport Protocol).
sent using l).

More specifically, as shown in FIG. 14, a first foreground image (adv) is transmitted from a data receiving device 901 of a client terminal to a second server (s2.com) 13b.
(DESCRIBE rtsp: //s2.com/adv.mpg) C2 requesting detailed information (for example, the presence or absence of a plurality of candidate data, etc.) of the media data corresponding to.
Is issued.

When the command C2 is received by the second server 13b, a response (RTSP / 1.0 OK) R indicating that the command is accepted is received from the server 13b.
2 is issued to the client terminal, and S
DP (Session Description Protocol) information is supplied to the client terminal.

Next, from the data receiving device 901 of the client terminal to the second server (s2.com) 13b,
A preparation request command (SETUP rtsp: //s2.com/adv.mpg) C3 for requesting to prepare to provide media data corresponding to the first foreground image (adv) is issued.
Then, when the second server 13b is ready to provide the media data, a response (RTSP / 1.0 OK) R3 indicating that the command C3 has been acknowledged is issued to the client terminal.

Then, a data request command (PLAY rtsp) for requesting media data corresponding to the first foreground image (adv) from the data receiving device 901 of the client terminal to the second server (s2.com) 13b. : //s2.com
/adv.mpg) When C4 is issued, the second server 13b
Then, the response (R
TSP / 1.0 OK) R4 is issued to the client terminal, and thereafter, the second server 13b issues media data D corresponding to the first foreground image (adv.mpg).
m1 is stored in an RTP packet and sequentially supplied to the client terminal.

Then, the media data Dm1 is received by the corresponding data receiving means 902a and output to the corresponding image decoding means 903a. The image decoding means 903a performs a decoding process of the media data,
The decrypted media data Dd1 is recorded in the corresponding frame memory 904a in frame units. At this time, the media data Dm1 can be reproduced. For example, from the time when the client terminal starts requesting the media data Dm1 to the server (when the output of the counter circuit is 5 seconds), the client terminal and the Three seconds have passed between exchanging messages with the server.

In order to obtain media data from a predetermined server in this manner, the client exchanges messages between the server and the client terminal.
The time at which the foreground image becomes reproducible is later than the display start time of the first foreground image described in the SMIL data. Therefore, the display unit 905 displays the S
After a lapse of three seconds from the display start time of the first foreground image described in the MIL data, the display of the first foreground image is performed.

That is, when the time information from the clock circuit 909 reaches 8 seconds, it is determined whether or not one frame of decoded foreground image data is stored in the frame memory 904a. Is stored for one frame, the first foreground image is synthesized with the background image and displayed.

At this time, if the image data is moving image data, the image data is sequentially input to the data receiving means 902a, the image decoding means 903a sequentially decodes the image data, and the frame memory 904a stores the image data. The image data decoded by the image decoding means 903a is sequentially stored in frame units. Then, on the display unit 905, the image data corresponding to the frame stored in the frame memory 904a is sequentially combined with the data of the background image and displayed.

Thereafter, when the time information output by the clock circuit 909 reaches 10 seconds, the control data generation unit 907
Requesting means 908 and the third server (s3.com) 1 based on the src attribute of the video element 705a stored in the
3c, a message exchange requesting image data (mov.mpg) corresponding to the second foreground image is performed using RTSP (Real Time Streaming Protocol) as a communication protocol.
This is performed using After that, the image data (mov.mpg) is transmitted from the server to the RTP (Realtime Transport Protocol).
sent using l).

More specifically, as shown in FIG. 14, a second foreground image (mov) is sent from the data receiving device 901 of the client terminal to the third server (s3.com) 13c.
(DESCRIBE rtsp: //s3.com/mov.mpg) C5 requesting detailed information (for example, the presence or absence of a plurality of candidate data, etc.) on the media data corresponding to.
Is issued.

When the command C5 is received by the third server 13c, a response (RTSP / 1.0 OK) R indicating that the command is accepted is received from the server 13c.
5 is issued to the client terminal, and S
DP (Session Description Protocol) information is supplied to the client terminal.

Next, from the data receiving device 901 of the client terminal to the third server (s3.com) 13c,
A preparation request command (SETUP rtsp: //s3.com/mov.mpg) C6 for requesting preparation for providing media data (image data) corresponding to the second foreground image (mov).
Is issued. Then, when the third server 13c is ready to provide the media data, a response (RTSP / 1.0) indicating that the command C6 has been acknowledged is received.
OK) R6 is issued to the client terminal.

Then, a data request command (PLAY rtsp) for requesting media data corresponding to the second foreground image (mov) from the data receiving device 901 of the client terminal to the third server (s3.com) 13c. : //s3.com
/mov.mpg) When C7 is issued, the third server 13c
Then, a response (R) indicating that the command C7 has been acknowledged
TSP / 1.0 OK) R7 is issued to the client terminal, and thereafter, the third server 13c issues media data D corresponding to the second foreground image (mov.mpg).
m2 is stored in the RTP packet and sequentially supplied to the client terminal.

Then, the media data Dm2 is received by the corresponding data receiving unit 902b and output to the corresponding image decoding unit 903b. In the image decoding means 903b, decoding processing of the media data Dm2 is performed by
The decoding is performed in the same manner as the decoding processing of the media data Dm1, and the decoded media data Dd2 is recorded in the corresponding frame memory 904b in units of frames. At this point, the media data Dm2 can be reproduced.
For example, from the time when the request of the media data Dm2 is started from the client terminal to the server (the time when the output of the counter circuit is 10 seconds), a predetermined time is set between the message exchange between the client terminal and the server. Has passed.

In order to obtain media data from a predetermined server as described above, a message is exchanged between the server and the client terminal.
The time at which the foreground image becomes reproducible is later than the display start time of the second foreground image described in the SMIL data. Therefore, the display unit 905 displays the S
After a lapse of three seconds from the display start time of the second foreground image described in the MIL data, the second foreground image is displayed.

At this time, the display means 905 displays the background image (bg) in the background display area 1110 of the image space 1100 (see FIG. 11A), and displays the first image on the first foreground display area 1120. The foreground image (adv) is displayed, and the second foreground image (mov) is displayed on the second foreground display area 1130. That is, in the image space 1100, the background image (bg) and the first and second foreground images are combined and displayed.

[0055]

However, the conventional data receiving device 901 issues a data request message for requesting image data to the server based on the contents of the scene description by SMIL. There was such a problem.

That is, in the above scene description, the start time of the process for displaying the first foreground image (adv) is 5 seconds after the start of the display of the entire scene by the begin attribute attached to the first video element 704a. Is passed. The start time of the processing for displaying the second foreground image (mov) is determined by the second video element 705.
The begin attribute attached to “a” indicates that 10 seconds have elapsed since the start of display of the entire scene. For this reason, in the conventional data receiving apparatus 901 mounted on the client terminal (reception-side terminal), when 5 seconds have elapsed after the start of displaying the entire scene, the second server 13b transmits the first foreground image to the second server 13b. When a data request message requesting the corresponding image data is issued, and when 10 seconds have elapsed after the start of displaying the entire scene, data requesting image data corresponding to the second foreground image to the third server 13c. A request message is issued.

In this case, there is a certain delay time from when the image data is requested to the server to when the image data from the server can be displayed on the client terminal. For example, the delay time includes a time required for message exchange by the RTSP between the server and the client terminal, a time required for the server to process a command from the client terminal, and the like.

Therefore, the conventional data receiving apparatus 901 is
When a preset standby time (3 seconds in this example) has elapsed from the start of the data request to the server, an image is displayed based on the image data stored in the frame memory. ing.

As a result, in the conventional data receiving apparatus 901, at the time designated by the scene description, that is, each vi
At the time indicated in the begin attribute included in the deo element, the vide
o There is a problem that it is difficult to display media data corresponding to an element.

The time required from when image data is requested to when the image data is actually stored in the frame memory depends on the state of the network, the number of message exchanges, and the like. The temporal positional relationship between the data also fluctuates, and there is a problem that it is difficult to maintain a synchronization state among a plurality of image data.

For example, according to the scene description SD shown in FIG. 12, the display of the image corresponding to the second video element 705a starts five seconds after the display of the image corresponding to the first video element 704a starts. The time from when the image data is requested to the server to the time when the image data is actually stored in the frame memory of the data receiving apparatus 901 due to various factors such as the degree of network congestion, which should be started. Changes, a situation arises in which the image corresponding to the video element 705a is not displayed even after 5 seconds have elapsed since the display of the image corresponding to the video element 704a was started. Such a situation becomes a serious problem when the scene is a composite image including a plurality of image data related to each other.

When the media data is transmitted through a network such as the Internet where the bandwidth (that is, a constant data transmission rate) is not guaranteed, the image decoding means of the data receiving device 901 uses the data buffer of the data receiving means. It is necessary to wait for several seconds to several tens of seconds in time until a predetermined amount of received image data is accumulated, and then start decoding processing for the received image data. The process of accumulating a fixed amount of image data received in the data buffer of the data receiving unit before the decoding process by the image decoding unit starts is called pre-buffering.

When the pre-buffering is not performed, the decoding process in the data receiving apparatus is susceptible to network jitter (fluctuation in transmission rate). For example, when the decoding process is performed for each fixed amount of image data, at the time when the decoding process is performed, the image data to be decoded is not prepared, and the reproduction process of the image data is likely to be interrupted.

Therefore, considering the time required for exchanging messages with the server and pre-buffering, a message requesting each image data is sent to the server at the display time of each image data described in the SMIL data after the scene reproduction starts. The conventional data receiving apparatus 901 that issues the scene cannot reproduce an accurate scene as defined by the scene description.

Further, an appropriate pre-buffering time may be different for each bit stream (encoded data of each image data) corresponding to each image data. It is not possible to set an appropriate pre-buffering time, and the receiving terminal may experience an excess or deficiency of image data in the buffer in the data receiving means, that is, an underflow or overflow of the buffer during the decoding process on the image data. There is a possibility.

The present invention has been made in order to solve the above-mentioned problem, and it is possible to start reproduction and display of individual images constituting a scene at a time specified by scene description data. In addition, a data receiving apparatus, a data receiving method, a data transmitting method, and a data receiving method capable of performing reproduction and display processing on image data without interruption regardless of occurrence of network jitter. It is an object of the present invention to obtain a data recording medium storing a program for performing a receiving process by a computer.

[0067]

According to a first aspect of the present invention, there is provided a data receiving apparatus for transmitting media data, which is one of image data, audio data, and text data, corresponding to each element constituting a scene. What is claimed is: 1. A data receiving apparatus which acquires from a data source on a network, reproduces the acquired media data, and displays the scene, wherein the location indicating the location on the network of the data source holding each of the media data is provided. Information, first time information indicating the reproduction start time of each of the media data, and second time information corresponding to each of the media data for requesting the media data to the data source. 1 and a request for each of the media data to its data source based on the first and second time information. A time setting unit that sets the data request time to a time earlier than the reproduction start time of each of the media data by a unique time set for each of the media data; and a data set that indicates the respective media data by the location information. A data requesting unit for performing a data requesting process requesting the source at the data requesting time set by the time setting unit; and receiving media data supplied from the data source in response to a request from the data requesting unit. A second receiving unit.

According to a second aspect of the present invention, in the data receiving apparatus according to the first aspect, the first receiving unit is connected to the second receiving unit.
As the time information, it is assumed that after receiving each of the media data, time information indicating a standby time until the reproduction of each of the media data is received, the time setting unit,
The data request time of the media data is set to a time earlier than the reproduction start time of each of the media data by the standby time.

According to a third aspect of the present invention, in the data receiving apparatus according to the first aspect, the first receiving unit is connected to the second receiving unit.
As time information, time information indicating the time at which each of the media data should be requested to the corresponding data source is received, and the time setting unit sets the data request time of each of the media data to the second At the time indicated by the time information.

According to a fourth aspect of the present invention, in the data receiving apparatus according to the first aspect, the first receiving section is connected to the second receiving section.
As the time information, it is assumed that after receiving each of the media data, time information indicating a standby time until the reproduction of each of the media data is received, the time setting unit,
The data request time of each piece of media data is set to be earlier than the reproduction start time of each piece of media data by a total time of the standby time and a predetermined time set in advance.

According to a fifth aspect of the present invention, in the data receiving apparatus according to the first aspect, the first receiving section is connected to the second receiving section.
As time information, time information indicating the time at which each of the media data should be requested to the corresponding data source is received, and the time setting unit sets the data request time of each of the media data to the second Are set earlier than the time indicated by the time information.

According to the data receiving method of the present invention (claim 6), image data,
A data receiving method for acquiring media data, which is either audio data or text data, from a data source on a network, reproducing the acquired media data, and displaying the scene, wherein the media data is retained. Location information indicating the location of the data source on the network, first time information indicating the reproduction start time of the media data, and the media data corresponding to the media data to the data source. First receiving the second time information for requesting
And a data request process for requesting each of the media data to a data source indicated by the location information based on the first and second time information, based on the reproduction start time of each of the media data. A data requesting step performed earlier by a specific time set for each of the media data; and a second receiving the media data supplied from the data source in response to the data requesting process in the data requesting step. And a receiving step.

According to a seventh aspect of the present invention, in the data receiving method according to the sixth aspect, the first receiving step is performed by receiving each of the media data as the second time information. It is assumed that time information indicating a standby time until the reproduction of the media data is received, and the data requesting step includes a data request process for requesting each media data to a predetermined data source, and a reproduction of each media data. This is performed at a time earlier than the start time by the standby time.

The present invention (claim 8) is the data receiving method according to claim 6, wherein the first receiving step is performed by using the respective media data as the second time information with respect to a corresponding data source. It is assumed that time information indicating a data request time to be requested is received, and the data request step is a data request process for requesting each of the media data to a predetermined data source at the data request time. Things.

According to a ninth aspect of the present invention, in the data receiving method according to the sixth aspect, the first receiving step is performed by receiving each of the media data as the second time information. It is assumed that time information indicating a standby time until the reproduction of the media data is received, and the data requesting step includes a data request process for requesting the respective media data to a predetermined data source, The reproduction is started at a time earlier than the reproduction start time by a total time of the standby time and a predetermined time set in advance.

According to a tenth aspect of the present invention, in the data receiving method according to the sixth aspect, the first receiving step includes:
As the second time information, time information indicating a data request time at which each of the media data should be requested to the corresponding data source is received, and the data request step is performed for a predetermined data source. The data request processing for requesting each media data is performed at a time earlier than the data request time by a predetermined time.

According to the data transmission method of the present invention (claim 11), media data, which is one of image data, audio data, and text data, corresponding to each element constituting a scene, is reproduced from the media data. A data transmission method for transmitting to the receiving terminal that displays the scene, wherein location information indicating the location of the data source holding the media data on the network, and the reproduction start time of the media data. First time information, and second media information corresponding to each of the media data, for requesting the media data from the data source.
A first transmitting step of transmitting the time information of the first and second times, and the media data is transmitted in response to a request for the media data issued from a receiving terminal based on the first and second time information and the location information. And a second transmitting step of transmitting to the receiving terminal.

According to a twelfth aspect of the present invention, in the data transmission method according to the eleventh aspect, the second time information is transmitted from when the respective media data is received to when the respective media data is reproduced. This is time information indicating the standby time.

According to a thirteenth aspect of the present invention, in the data transmission method according to the eleventh aspect, the second time information indicates a data request time at which each of the media data should be requested from a corresponding data source. It is time information.

According to the data storage medium of the present invention (claim 14), media data, which is one of image data, audio data, and text data, corresponding to each element constituting a scene is transmitted from a data source on a network. A data storage medium storing a data reproduction program for performing, by a computer, a data reproduction process of acquiring and reproducing the acquired media data to display the scene, the data storage medium storing the data reproduction program and each of the media data. Location information indicating the location of the data source on the network, first time information indicating the reproduction start time of the media data, and the media data corresponding to the media data to the data source. The first process of receiving the second time information for requesting the A first program for performing the Yuta, based on the first and second time information,
A data request for requesting the respective media data to the data source indicated by the location information is made at a time earlier than the reproduction start time of the respective media data by a specific time set for each of the media data. A second program for performing the second process by a computer, and a third program for performing the third process of receiving media data supplied from the data source in response to the data request by the computer. Is included.

A data recording medium according to the present invention (claim 15) reproduces media data, which is one of image data, audio data, and text data, corresponding to each element constituting a scene. A data storage medium storing a data transmission program for performing, by a computer, a data transmission process for transmitting data to a receiving terminal that displays the scene, wherein the data transmission program is stored in a data source that holds the respective media data. Location information indicating the location on the network,
First time information for transmitting first time information indicating a reproduction start time of each of the media data and second time information corresponding to each of the media data for requesting the media source to the data source; In response to a request for each of the media data issued from a receiving terminal based on a first program for performing processing by a computer and the first and second time information and the location information, each of the media A second process of transmitting data to the receiving terminal,
And a second program to be executed by a computer.

[0082]

DESCRIPTION OF THE PREFERRED EMBODIMENTS (Embodiment 1)

FIG. 1 is a block diagram for explaining a data receiving apparatus according to the first embodiment of the present invention. The data receiving apparatus 110 according to the first embodiment receives SMIL data Ds1 as scene description data,
This is a data receiving apparatus that reproduces and displays a composite image including one background image and two foreground images in accordance with the description content of data.

That is, based on the third control data Dc3, the data receiving apparatus 110 transmits a SMIL request signal S for requesting SMIL data Ds1 to a predetermined server.
rd, and receives the SMIL data Ds1 transmitted from the predetermined server, and outputs the SMIL data Ds1.
Request receiving means 102 for analyzing the SMIL
The receiving means 102 includes a control data generation unit 110a that generates first and second control data Dc1 and Dc2 based on the analysis data Da1 obtained by analyzing the SMIL data.

The data receiving device 110 receives the image data (media data) Dm1 corresponding to the first foreground image from the server.
And decoding means 107a for decoding the received image data Dm1 and outputting the decoded image data Dd1, and a frame memory 108a for recording the decoded image data Dd1 in frame units. Further, the data receiving device 1
01 is a media data receiving unit 106b that receives image data (media data) Dm2 corresponding to the second foreground image from the server, and a decoding unit that decodes the received image data Dm2 and outputs decoded image data Dd2 10
7b and a frame memory 108b for recording the decoded image data Dd2 in frame units.

The data receiving device 110 reads out the decoded image data Dd1 and Dd2 stored in the frame memories 108a and 108b based on the first control data Dc1 from the control data generating unit 110a, and reads them. Display means 10 for displaying by combining with a background image
9, and outputs a data request signal Srp for requesting data to a required server based on the second control data Dc2 from the control data generation unit 110a.
Response signal S to the data request from the required server
data request receiving means 105 for receiving the ack.

Here, the control data generation unit 110
Analysis data Da from the SMIL request receiving means 102
1, the control command to be output as control data to the data request receiving means 105 and the display means 109 and the related information are arranged as one set of items, and the items are arranged in order from the earliest command execution time. A control data recording processing unit 103 that creates a time table and outputs time information It relating to the execution time of each control instruction in order from the earliest one. When the time information It is input, execution of the control instruction A trigger signal generating means for outputting a trigger signal to the control data recording processing unit each time the time count reaches the execution time of the set control instruction. Have been. Here, each time the control data recording processing unit 103 receives the trigger signal St from the trigger signal generation unit 104, the control data is transmitted to the data requesting unit 105 or the display unit in the order from the earliest instruction execution time. The data is output to the means 109 as control data Dc1 or Dc2.

In the figure, 101a is the data receiving device 1
01 is a clock circuit that supplies a reference clock to each section of the data receiving apparatus 01, and is the same as the clock circuit in the conventional data receiving apparatus 901. Also,

The trigger signal generating means 104
For example, a timer that can set a plurality of times, performs a timekeeping operation based on the reference clock from the clock circuit 101a, and outputs a trigger signal each time the timekeeping time reaches the set time can be used. it can.

Further, in the first embodiment, the data receiving apparatus 101 has two data receiving means, two decoding means, and two frame memories, and transmits media data corresponding to two foreground images from a server on the network. Although the example of acquiring and displaying one background image and two foreground images in combination is shown, the number of media data acquired from a server on the network is not limited to two. For example,
When acquiring three or more media data from a server on the network, the data receiving device has data receiving means, decoding means, and frame memories by the number corresponding to the number of media data to be acquired. .

FIG. 2 is a diagram showing an example of the contents of the SMIL data. It is assumed that the data receiving apparatus 110 according to the first embodiment receives the SMIL data shown in FIG. 2, for example. FIG. 3 shows a spatial arrangement (FIG. (A)) and a temporal arrangement (FIG. (B)) of media data which is the contents of the SMIL data shown in FIG. 2.

[0092] In Fig. 2, <smil>, </ smil>, <head>, </ head>,
<layout>, </ layout>, <root-layout>, <region>, <bod
Character strings such as y>, <par>, </ par>, and <video> are called elements, and declare the content of the description following the element. That is, each element 210a,
210b, 220a, 220b, 230a, 230b,
240a, 240b, 250a, and 250b are elements 710a, 710b, 720a, and 720 in FIG.
b, 730a, 730b, 740a, 740b, 750
a, 750b. Each row 20 shown in FIG.
Reference numerals 1 to 203 are the same as the respective rows 701 to 703 in FIG. However, each row 204 and 20 shown in FIG.
5 is different from each row 704 and 705 in FIG.

First, the spatial arrangement of the media data indicated by the SMIL data shown in FIG. 2 will be described. root
The -layout element 201a specifies the size of the entire scene. That is, the root-layout element 201a
Width attribute (width = "300") and heig attached to this element
The ht attribute (height = “200”) indicates that the size of the rectangular area in which the entire scene is displayed, that is, its width and height are 300 points and 200 points, respectively. The id attribute of the element 201a indicates a background image (bg) (id = "bg").

The region element 202a has a wi associated therewith.
dth attribute (width = "300") and height attribute (height = "50")
The size of the rectangular area in which the image corresponding to the element 202 is arranged, that is, the width and height thereof are respectively
It indicates that the points are 300 points and 50 points. Also, this region element 202a is attached to this
By the left attribute (left = "0") and the top attribute (top = "150"), the position of the upper left corner of this rectangular area is determined in the image space 1100.
0 points from the left edge of the image space, 1 point from the upper edge of the image space 1100
This indicates that the position is 50 points away.
The id attribute attached to the element 202a indicates the first foreground image (adv) (id = "adv").

A region attribute attached to the video element 204a indicates the first foreground image (adv) (region = "ad").
v "). Therefore, the rectangular area whose size and position are specified by the region element 202a is an area where the first foreground image (adv) is arranged (hereinafter also referred to as an adv area).

The region element 203a has a wi associated therewith.
dth attribute (width = "200") and height attribute (height = "15
0 ") indicates that the size of the corresponding rectangular area is 200 points in width and 150 points in height.
Also, the region element 203a is set such that the position of the upper left corner of this rectangular area is 50 points from the left edge of the image space 1100 by the left attribute (left = "50") and the top attribute (top = "0") attached thereto. , Is located at a position separated from the upper end of the image space 1100 by 0 point. The id attribute attached to the element 203a is the second foreground image (mo
v) (id = "mov").

A region attribute attached to the video element 205a indicates the second foreground image (mov) (region = "mo").
v "). Therefore, the rectangular area whose size and position are specified by the region element 203a is an area where the second foreground image (mov) is arranged (hereinafter also referred to as a" mov area ").

Note that the bg area is an area as a background,
The adv area is an area for displaying an advertisement or the like, and the mov area is an area for displaying a moving image or the like. After all, the adv area 1120, m obtained based on the scene description SD1
The arrangement of the ov region 1130 and the bg region 1110 is shown in FIG.
As shown in FIG. 11A, the arrangement of these regions is exactly the same as that shown in FIG.

That is, the predetermined image space 1100 includes a background display area (bg area) 1 where the background image (bg) is displayed.
In the background display area 1110, a first foreground display area (adv area) 1120 in which a first foreground image (adv) such as an advertisement is arranged, and a second foreground image (moving image) mov) is arranged in a second foreground display area (mov area) 1130. The size of the arrangement area of each image and the position in the image space are shown in FIG.
It is the same as that shown in (a).

Next, the temporal arrangement of the media data indicated by the SMIL data shown in FIG. 2 will be described. vide
The begin attribute (begin = "5s") related to the o element 204a indicates that the display of the image data corresponding to the video element 204a starts when five seconds have elapsed after the start of the display of the scene. .

Scr attribute (scr attribute) for video element 204a
= "rtsp: //s2.com/adv.mpg") is the video element 204
The image data corresponding to a is to be acquired by issuing a command requesting the image data (adv.mpg) stored in the server to the server (s2.com) by RTSP. I have.

On the other hand, the begin attribute (begin = "10s") relating to the video element 205a indicates that the display of the image data corresponding to the video element 205a starts when 10 seconds have elapsed after the start of the display of the scene. It is shown that.

Scr attribute (scr attribute) for video element 205a
= "rtsp: //s3.com/mov.mpg") is the video element 205
The image data corresponding to “a” should be acquired by issuing a command requesting the image data (mov.mpg) stored in the server (s3.com) to the server (s3.com) by RTSP. I have.

When the attribute information on each video element is summarized, as shown in FIG. 3B, the scene (background image)
5 seconds after the start of the display of the first foreground image (ad
The display of v) is started, and the display of the second foreground image (mov) is started when 10 seconds have elapsed after the start of the display of the scene.

In the video elements 204a and 205a, a prebuffering attribute is indicated. This prebufferi
The ng attribute indicates a waiting time from reception of the media data to decoding processing thereof. For example, the prebuffering attribute (prebuffering = "7") for the video element 204a
s ") represents image data (ad) corresponding to the video element 204a.
v.mpg) indicates that the decoding process waits for 7 seconds after being received by the data receiving device. The prebuffering attribute (prebuffering = "
15s ") is the image data (m
ov.mpg) indicates that the decoding process is on standby for 15 seconds after being received by the data receiving device.

Data receiving apparatus 11 according to the first embodiment
0, when the scene description data SD1 is received,
In the control data generation unit 110a, a time table is created in consideration of the standby time corresponding to each of the video elements,
It is stored in the control data generator 110a.

In this time table, the video element 20
4a, 205a corresponding to each image data (adv.mpg),
(Mov.mpg) reception processing is started 2 seconds and 5 seconds before the start of scene display, respectively, and each video element 204a, 2
When the waiting times of 7 seconds and 15 seconds have elapsed since the reception process corresponding to the reception process 05a was started, as shown in FIG.
The display of each image data (adv.mpg) and (mov.mpg) is time T
The issuance time of the control instruction is set so as to be performed at adv (Tadv = 5 seconds) and Tmov (Tmov = 10 seconds).

FIG. 4 shows that the control data generator 110a
The content of the time table stored as the content of the IL data is shown. The time table Tab includes an item corresponding to a time at which a data request or data display is performed, and a data request receiving unit 10 that is a control target that issues a control command.
5 or an item indicating the display means 109, and an item indicating a control command for the control target. Events in which a set of items of time, control target, and control command are arranged are arranged in order of time. The control target is the data request receiving means 1
In the event of the event 05, the control instruction item describes information specified by the src attribute of the video element of the SMIL data. In the event of the control target being the display unit 109, the control instruction item includes Root of SMIL data
-id, width, each related to -layout element or region element
Information specified by the height, left, and top attributes is described.

Next, the operation will be described. FIG. 5 is a diagram for explaining a flow of a process in which the data receiving device of the first embodiment acquires media data from the server, in which a message is exchanged between the data receiving device and the server; 9 illustrates transmission of media data to a receiving device.

Here, the data receiving apparatus 110 is mounted on a personal computer (personal computer) as a client terminal, similarly to the conventional data receiving apparatus 901, and the data receiving apparatus 110 has the scene shown in FIG. It is assumed that SMIL data Ds1 is supplied as data representing the description SD1.

First, for example, while a user is browsing a home page described in HTML (Hyper Text Markup Language) with a web browser provided in a personal computer (personal computer), a predetermined When a user operation of clicking an area linked to the scene description SD1 is performed, a SMIL requesting SMIL data Ds1 indicating the scene description SD1 is issued from the data receiving device 110 of the client terminal.
A request command (GET http://s1.com/scene.smil) C1 is issued. This command C1 is transmitted to the server (s1.com) 1
3a is required to provide SMIL data by HTTP.

When the SMIL request command C1 is sent to the server 1
3a, the server 13a sends a reply (HTTP / 1.0 O) indicating that the command has been acknowledged.
K) R1 is issued to the client terminal, and SMIL data (scene.smil) Ds1 is supplied to the client terminal.

Then, in the data receiving device 110 of the client terminal, the SMIL data Ds1 is received by the SMIL request receiving means 102, and the SMIL request receiving means 1
In 02, processing for analyzing the SMIL data Ds1 is performed.

SMIL in SMIL request receiving means 102
SMIL analysis data Da1 obtained by data analysis
Is supplied to the control data generation unit 110a, and is recorded in the control data recording processing unit 103. Then, the control data recording processing unit 103 performs a process of generating the time table Tab shown in FIG. 4 based on the SMIL analysis information Da1, and stores the contents of the SMIL data in a time table format.

The control data recording processing unit 103 will be briefly described below.
Will be described.
First, in the control data recording processing unit 103, the display start time indicated by the begin attribute of each video element,
The issuance time of the control command requesting the media data corresponding to each video element is obtained using the standby time (prebuffering time) designated by the prebuffering attribute of the element. Here, the issue time of the control command requesting the media data is a time obtained by subtracting the standby time from the display start time. Specifically, vi
Media data (adv.mpg) corresponding to the deo element 204a
The issue time Tpadv of the control command for requesting the scene display is -2 with respect to the display start time Tbg of the scene (Tbg = 0 second).
Second, and the issue time Tpmo of the control command requesting the media data (mov.mpg) corresponding to the video element 205a.
v is -5 seconds.

Then, the control data recording processing unit 103 determines the contents of the SMIL data based on the SMIL analysis data Da1 by using the information (the src attribute attached to the video element) required to request the media data. To sort media information and information required to display media data (information specified by the id, width, height, left, and top attributes attached to the root-layout element or region element) Is performed.

Next, the control data recording processing unit 103
Information indicating a control command requesting the media data (mov, mpg) based on the information for displaying the media data;
Information indicating the data request means that is the object of the control instruction,
In addition, event data E1 including information indicating the issue time of the control instruction as a set is created. Media data (av
d.mpg), event data E2, which is a set of information indicating a control instruction requesting the control instruction, information indicating a data requesting unit to which the control instruction is applied, and information indicating an issue time of the control instruction.
Is created.

[0118] Further, the control data recording processing section 103 performs the processing based on the information for requesting the media data.
Event data E, which is a set of information indicating a control command for displaying a background image (bg), information indicating a display means which is the target of the control command, and information indicating an issue time of the control command.
3 is created. The event data E4 is created as a set of information indicating a control command for displaying the first foreground image (adv), information indicating a display unit to which the control command is applied, and information indicating an issue time of the control command. Is done. Event data E5, which is a set of information indicating a control command for displaying the second foreground image (mov), information indicating a display means to which the control command is applied, and information indicating the issue time of the control command.
Is created.

Thereafter, the control data recording processing unit 103
Then, the above-described event data are arranged in order from the earliest issue time of the corresponding control instruction, a process of creating a time table shown in FIG. 4 is performed, and the created time table is stored.

Specifically, the scene description SD1 shown in FIG.
Then, the times (control command issuance times) at which the media data (adv.mpg) and (mov.mpg) corresponding to the video elements 204a and 205a are requested are -5 seconds and -2 seconds, respectively.
The display start times of the foreground images (adv) and (mov) are 5 seconds and 10 seconds, respectively, and the display start time of the scene (background image) is 0 seconds. Therefore, the control data recording processing unit 1
03, the first event data is event data E1, the second event data is event data E2, the third event data is event data E3, as shown in FIG. The fourth event data is event data E4, and the fifth event data is event data E5.

The control data recording processing section 103 outputs the time information of the control command from the generated time table to the signal generating means 104 in the order in which the information is issued. Is performed.

When the time information is input to the signal generating means 104, the time indicated by the time information is recorded as the set time in the input order of the time information, and the clocking operation by the clock is started. I do.

At this time, in the data receiving apparatus 110, in parallel with the time table creation processing, a predetermined server prepares for image data transmission between the data receiving apparatus and the server. Processing for exchanging messages is performed.

More specifically, as shown in FIG. 5, the data receiving device 110 of the client terminal sends a second foreground image (mov) to the third server (s3.com) 13c.
Command (DESCRIBE rtsp: //s3.com/mov.mpg) requesting detailed information (for example, the presence or absence of a plurality of candidate data, etc.) of the media data corresponding to.
a is issued.

The command C2a is transmitted to the third server 13c.
Received from the server 13c, a response indicating that the command has been acknowledged (RTSP / 1.0 OK)
R2a is issued to the client terminal, and SDP (Session Description Protocol) information is supplied to the client terminal.

Next, from the data receiving device 110 of the client terminal to the third server (s3.com) 13c,
A preparation request command (SETUP rtsp: //s3.com/mov.mpg) C3a is issued to request preparation for providing media data corresponding to the second foreground image (mov). Then, when the third server 13c is ready to provide the media data, a response (RTSP / 1.0 OK) R indicating that the command C3a has been acknowledged is received.
3a is issued to the client terminal.

Then, subsequently, from the data receiving device 110 of the client terminal, the second server (s2.com) 13b
Command (DESC) requesting detailed information (for example, the presence or absence of encoding conditions and a plurality of candidate data) on media data corresponding to the first foreground image (adv).
RIBE rtsp: //s2.com/adv.mpg) C2b is issued.

The command C2b is transmitted to the second server 13b
Received from the server 13b, a response indicating that the command C2b has been acknowledged (RTSP / 1.0
OK) R2b is issued to the client terminal, and SDP (Session Description Protocol) information is supplied to the client terminal.

Next, from the data receiving device 101 of the client terminal to the second server (s2.com) 13b,
A preparation request command (SETUP rtsp: //s2.com/adv.mpg) C3b is issued, which requests to prepare to provide media data corresponding to the first foreground image (adv). Then, when the second server 13b is ready to provide the media data, a response (RTSP / 1.0 OK) R indicating that the command C3b has been acknowledged is received.
3b is issued to the client terminal.

When the time measured by the clock reaches the set time in the signal generating means 104, the signal generating means 104 generates a trigger signal St and outputs the trigger signal St to the control data memory 103.
Here, the set time in the signal generation unit 104 is −
5, -2, 0, 5, 10 seconds, the signal generation means 1
From 04, the time measured by the clock is -5, -2,
At 0, 5, and 10 seconds, a trigger signal is sequentially output, and the control data recording processing unit 103 changes the control command included in the first event in the time table to the corresponding control target every time the trigger signal is received. Issued to

First, when the trigger signal output from the signal generating means 104 is input to the control data recording processing section 103 at time t (= -5 seconds), the control data recording processing section 10
3, the control command (PLAY rtsp: //s3.com/mov.mpg) C4a in the first event of the time table is output to the data request unit 105 which is the target of the control command.

Then, in the data request receiving means 105,
A control command (PLAY rts) from the control data recording processing unit 103
p: //s3.com/mov.mpg) Processing to output a message by RTSP for requesting image data (mov.mpg) to the third server (s3.com) 13c based on C4a Is performed.

When the third server 13c receives the message from the data request receiving means 105, the server 13c transmits the image data (mov.mpg) to the data receiving device 110 by RTP. Is done.

The image data (mov.mpg) Dm2 transmitted from the server 13c is transmitted to the media data receiving means 10
6b. Here, the image data D
m2 is a bit stream that has been compression-encoded by a standard-compliant encoding method such as MPEG. The bit stream (image data) input to the media data receiving unit 106b is sequentially output to the decoding unit 107b for each frame, and is decoded by the decoding unit 107b for each frame. The decoded image data Dd2 decoded by the decoding means 107b is stored in a frame memory 108b for each frame.
Is stored.

Next, at time t (= -2 seconds), when the trigger signal output from the signal generation means 104 is input to the control data storage processing section 103, the control data recording processing section 10
3, the control command (PLAY rtsp: //s2.com/adv.mpg) C4b in the second event of the time table is output to the data request receiving means 105 which is the target of the control command.

Then, in the data request receiving means 105,
A control command (PLAY rts) from the control data recording processing unit 103
Based on p: //s2.com/mov.mpg) C4b, a message by RTSP for requesting image data (adv.mpg) Dm1 is output to the second server (s2.com) 13b. Processing is performed.

When the second server 13b receives the message from the data request receiving means 105, the image data (adv.mpg) D is sent from the server 13b.
m1 is transmitted to the data receiving apparatus 110 by RTP.

The image data (adv.mpg) Dm1 transmitted from the server 13b is transmitted to the media data receiving means 10
6a. Here, the image data D
m1 is a bit stream that has been compression-encoded by a standard-compliant encoding method such as MPEG. The bit stream (image data) input to the media data receiving unit 106a is sequentially output to the decoding unit 107a for each frame, and is decoded by the decoding unit 107a for each frame. The decoded image data Dd1 decoded by the decoding means 107a is stored in a frame memory 108a for each frame.
Is stored.

At time t (= 0 second), when the trigger signal output from the signal generation means 104 is input to the control data recording processing section 103, the control data recording processing section 10
From 3, the control command (bg // width300 / height200) in the third event of the time table Tab is output as control data Dc1 to the display unit 109 that is the target of this control command. Then, the display unit 109 displays a control command (bg // width300 / h) from the control data recording processing unit 103.
eight200), a process of displaying the background image (bg) in the entire image space is performed. The data of the background image is previously secured in the data receiving device 110. At this time (t = 0 second), the display start time of the first and second foreground images indicated by the begin attribute attached to the video elements 204a and 205a is longer than 0 second. Adv area shown (first foreground display area) 11
20, a mov area (second foreground display area) 1130 includes:
First foreground image (adv), second foreground image (mov)
Is not displayed.

When the trigger signal St output from the signal generation means 104 at time t (= 5 seconds) is input to the control data recording processing section 103, the control data recording processing section 103 outputs the time table Tab. The control command (bg // width300 / height200) in the fourth event is transmitted to the display means 109 targeted by this control command by the control data Dc.
Output as 1. Then, the display unit 109 displays a control command (adv // left0) from the control data recording processing unit 103.
/ top150 / width300 / height50), a process of reading out the decoded image data Dd2 from the frame memory 108a for each frame is performed, and the first foreground image (adv) is read in the adv area (first foreground) in the image space 1100. Display area) 1
The image is synthesized with the background image so as to be located at 120.

Further, when the trigger signal St output from the signal generation means 104 at time t (= 10 seconds) is input to the control data recording processing section 103, the control data recording processing section 103 transmits the trigger signal St to the time table Tab. Control instruction for the fifth event (mov // left50 / top0 / width200 / heig
ht150), the display means 109 which is the target of this control instruction.
Is output as control data Dc1. Then, based on the control command (mov // left50 / top0 / width200 / height150) from the control data storage processing unit 103, the display unit 109 reads the decoded image data Dd from the frame memory 108b.
2 is performed for each frame, and the second foreground image (mov) is combined with the background image and the first foreground image so that the second foreground image (mov) is located in the mov area (second foreground display area) 1130 in the image space 1100. Is displayed.

FIG. 6 shows the control data storage processing unit 10
3 is a flowchart showing a specific process (time calculation process) of calculating a control command issuance time requesting media data in Step 3. Hereinafter, a process of calculating a control command issuance time in the control data storage processing unit 103 will be described. A brief description will be given. In the flowchart shown in FIG. 5, the first set time T1 [n] is the issue time of the control command requesting the media data corresponding to the n-th video element in the scene description SD1 (hereinafter, simply referred to as the media data request time). ), And the second set time T2 [n] is a time at which the media data corresponding to the n-th video element is displayed.

In FIG. 6, the request time T1 [n] of the media data is set to a value between the time when the control command for requesting the media data is issued to the server and the time when the media data is received, in addition to the prebuffering time. A flowchart for calculating by introducing the required time C is shown.

First, in the control data storage processing unit 103,
The first internal variable n used for the time calculation process is set to zero (Step S501). The variable n is the scene description S
It is a variable that increases by one each time the time calculation process corresponding to the video element in D1 is completed.

Next, the video element to be subjected to the time calculation processing is set to the analysis data D from the SMIL reception request means 102.
A process of determining based on d1 is performed (Step S5)
02). Normally, the video element to be subjected to the time calculation processing is sequentially selected from the head of the arrangement order among a plurality of video elements arranged in a fixed order in the scene description SD1. For this reason, in the first video element selection processing, the video element 2 out of the video elements 204a and 205a is selected.
04a is selected.

Subsequently, the control data recording processing unit 103 sets the video data in the second internal variable P used for the time calculation process.
The value (7) of the rebuffering attribute associated with the element 204a is set, and the value (5) of the begin attribute associated with the video element 204a is set in the third internal variable B used for the time calculation process. (Step S503).

After that, the control data recording processing unit 103 calculates the first set time T1 [n] based on the following (Equation 1) (step S504). T1 [n] = BPC (Equation 1)

Here, C is a constant, which is the time required from when the data request receiving means 105 issues a media data request control command to when the media data receiving means receives media data. The value of C is set by estimating the time from when the request control command is issued to when the data is received. Here, the constant C is
It is set to 0 seconds.

Therefore, the variables B, P, and C in the above (Equation 1) are set to 5, respectively, based on the scene display start time (0 second).
The first set time T1 corresponding to the first video element 204a is calculated by setting 7,0 (second).
[0] becomes -2, and the control command for requesting the media data of the first foreground image (adv) is issued two seconds before the scene display start time (0 second).

Further, the control data recording processor 103 calculates the second set time T2 [n] based on the following (Equation 2) (step S505). T2 [n] = B (Equation 2)

As a result, the first video element 204a
Is set to 5, and the display time of the first foreground image (adv) is 5 seconds after the scene display start time (0 second).

Thereafter, the control data recording processing unit 103 determines whether or not the first and second set times have been calculated for all video elements indicated in the scene description SD1. (Step S506). All vide
When the calculation of the first and second set times is completed for the o element, the first and second set times T1 [n] and T2 [n] (n = 0, 1) and a process of writing the scene display start time Tab in the time column of the time table is performed (step S508).

On the other hand, if the result of determination in step S506 is that the calculation of the first and second set times has not been completed for all video elements, the value of the variable n is incremented by one. (Step S507), the processing of steps S502 to S506 is performed.

In the data receiving apparatus 110, the vide
o When the calculation of the first and second set times for the element 204a has been completed, the calculation of the set time for the other video element 205a has not been completed, so the value of the variable n is incremented by one ( Step S50
7) Step S50 for the video element 205a
2 to step S506 are performed.

First and second video elements 205a
In the calculation of the set time, the variables B, P, C
Is set to 10, 15, 0 (second) based on the scene display start time (0 second), respectively. As a result, the first set time T1 [1] corresponding to the second video element 205a becomes -5, and the second foreground image (m
The issue time of the control command requesting the media data of ov) is 5 seconds before the scene display start time (0 second). The second set time T2 [1] corresponding to the second video element 205a is 10, and the second foreground image (m
The display time of ov) is 1 of the scene display start time (0 second).
0 seconds later.

At this time, since the calculation of the first and second set times has been completed for all video elements,
First and second set times T1 for video element 204a
[0], T2 [0], the first and second set times T1 [1], T2 [1] for the video element 205a, and the display start time Tab of the scene are described in the time column of the time table. (Step S508). That is, in the time column of the time table Tab, -5, -2, 0, 5, and 10 seconds are arranged in this order as the time information of the control instruction.

As described above, in the data receiving apparatus 110 of the first embodiment, the scene description SD1 for synthesizing and displaying the first and second foreground images (adv) and (mov) on the background image (bg) is displayed. As SMIL data D
s1 to the server 13a,
a SMIL reception requesting means 102 for receiving SMIL data Ds1 from a.
1 and Dm2 to the servers 13b and 13c, as well as data request receiving means 105 for receiving a message from the server, and information indicating the standby time before the start of display of each foreground image included in the SMIL data Ds1. A control data generating unit that controls the data request receiving unit 105 to issue the media data request message to the corresponding server at a time earlier than the display start time of each foreground image by the standby time. 110a, the foreground images can be combined with the background image at the time designated by the scene description and displayed.

By setting the standby time to a sufficiently large value in consideration of the state of the network through which the media data is transmitted, for example, the bandwidth and congestion degree, the media data reproduction processing in the data receiving apparatus is performed. Is less susceptible to network jitter, and can suppress interruption of image display during playback of media data.

Furthermore, in the data receiving apparatus 110 of the first embodiment, the control data recording
Based on the data, the time at which media data of each foreground image is requested from the server and the time at which each foreground image is displayed are managed by a time table in which information indicating each time is described. Each time the time described in the time table is reached, a control instruction for instructing the data requesting means to request media data or a control instruction for instructing the display means to start displaying media data is issued. Even when the number of foreground images constituting an image increases, a request for each media data can be made at an appropriate time before the start of display of each foreground image by comparing the clock time and the time information described in the time table. By doing so, each foreground image can be displayed favorably.

In the first embodiment, the control data storage processing unit 103 performs the process of calculating the issuance time of the control command from when the control command of the media data request is issued to the server to when the media data is received. In this example, the delay time C required during the communication is set to 0, but the delay time C is set to 0 according to the type of network to be used (for example, a network including a wireless line or a network including only wired lines). The above values may be set to appropriate values.

Further, in the first embodiment, the case has been described where the data receiving apparatus receives image data as media data. However, the media data is not limited to image data, but may be text data, audio data, or the like. In this case, the same effect as in the first embodiment can be obtained.

Also, in the first embodiment, the image data provided by the data receiving apparatus is compressed according to the MPEG method. However, the image data provided to the data receiving apparatus is a JPEG (joint photographic codi
ng experts group), GIF (graphics interchange)
format), H.264. H.261 system, H.264. The data may be compressed by another encoding method such as the H.263 method.

Furthermore, in the first embodiment, the one in which RTSP is specified as a transmission protocol for making a data request is shown as scene description data. Other protocols, such as HTTP (HyperTe
xt Transfer Protocol).

Further, in the first embodiment, the control data recording processing section 103 calculates the issue time of the control command to the data request receiving means or the display means, and the signal generating means 104 generates the control data recording processing section 103. Is set as the trigger generation time, and the trigger signal is output to the control data recording processing unit 103 each time the time counted by the signal generation unit 104 reaches the set trigger generation time. However, the issue time of the control instruction to the data request receiving means or the display means is:
The calculation may be performed by the signal generation unit 104, and the calculated time may be set as the trigger generation time. In this case, it is necessary for the control data recording processing unit 103 to manage each control instruction in an order based on the issue time.

Also, in the first embodiment, as a data receiving device, a prebuilt indicating a standby time attached to a video element.
Although the method of calculating the time at which media data is requested to the server using the ffering attribute value has been described, the data receiving device should output a data request message attached to the video element to the server, for example. Requ indicating time
The time at which media data is requested from the server may be calculated using the est attribute value.

(Embodiment 2) FIG. 7 is a block diagram for explaining a data receiving apparatus according to Embodiment 2 of the present invention.

Data receiving apparatus 120 according to the second embodiment
Is S in Embodiment 1 as scene description data.
The SMIL data Ds2 different from the MIL data Ds1 is used. Instead of the control data recording processing unit 110a that generates the control data Dc1 and Dc2 based on the SMIL data Ds1 in the data receiving device 110 of the first embodiment, It has a control data recording processing unit 120a that generates control data Dc1 and Dc2 based on the SMIL data Ds2.

FIG. 7 shows SMIL supplied as scene description data to data receiving apparatus 120 according to the second embodiment.
The content (scene description DS2) of the data Ds2 is shown.

The SMIL data Ds2 includes, as an attribute associated with the video element, a request attribute value indicating a time at which a data request message is to be output to the server instead of the prebuffering attribute in the SMIL data Ds1. That is, the SMIL data Ds2
Then, the video element 601a includes the SMIL data D
Instead of the prebuffering attribute (prebuffering = "7s") attached to the video element 201a of s1, image data (adv.mp
g) is accompanied by a region attribute (region = "-2s") indicating that a request message is output two seconds before the start of scene display. In this SMIL data Ds2,
The video element 602a includes v of the SMIL data Ds1.
The prebuffering attribute (prebuffe
Instead of ring = "15s"), the image data (mov.mpg) has a region attribute (region = "-5s") indicating that a request message is output 5 seconds before the start of scene display. ing.

In FIG. 8, a row 601 including a video element 601a is included in the scene description SD1 of the first embodiment.
The row 602 including the video element 602a corresponds to the row 201 including the video element 202a in the scene description SD1 of the first embodiment.

The control data generating section 120a in the data receiving apparatus 120 according to the second embodiment
Based on the SMIL analysis data Da2 obtained by analyzing the MIL data Ds2, the time table T shown in FIG.
control data recording processing unit 12 that creates ab, outputs the control data Dc1 and Dc2, and outputs time information It
3 and a trigger signal generating means 124 having the same configuration as the trigger signal generating means 104 in the data receiving apparatus 110 of the first embodiment.

In the data receiving apparatus 120 according to the second embodiment having such a configuration, similarly to the data receiving apparatus 110 according to the first embodiment, each media data is requested based on SMIL data Ds2 from a predetermined server. A message is issued to the corresponding server at a certain time earlier than the display start time of each foreground image, and the foreground images are combined with the background image at the time specified by the scene description. Can be displayed.

In the second embodiment, the media data request message is issued when the time measured by the data receiving device reaches the time specified by the request attribute of the SMIL data Ds2. , The timing of issuing the media data request message is
As described in the first embodiment, taking into account the constant C as the time required from transmitting the media data request message to the server to receiving the media data,
From the time specified in the request attribute, the above constant C
The data request message may be issued at a time earlier by the time indicated by.

In each of Embodiments 1 and 2, the attribute indicating the standby time before the start of display of each foreground image is set to the prebuffe
The attribute that indicates the time at which the media data request message is issued is the request attribute, and the attribute that indicates the standby time and the attribute that indicates the issue time of the request message have the same meaning. If so, another name may be used.

In the first embodiment, the standby time from when the server requests the image data corresponding to the image to when the display of the image is started is determined based on the SMIL data. However, this waiting time is
The determination may be made based on control data other than the SMIL data. For example, when the input image data is data (bit stream) encoded by the encoding process of the MPEG system, the standby time is multiplexed into a header portion corresponding to each frame of the video bit stream. Based on the VBV delay (Video Buffer Verifier) information, the delay time can be longer than the delay time indicated by the information. In this case, the following effects can be obtained.

That is, in a video decoder that receives a bit stream transmitted at a fixed transmission rate, since the video data differs for each frame, the video decoder receives the bit stream and then performs decoding processing on the bit stream. Is also different for each frame, and the VBV delay value multiplexed in the header portion of each frame of the video bit stream indicates this waiting time. Therefore, if the decoding process for the video bit stream is performed after waiting for the time indicated by the VBV delay value after receiving the video data, it is possible to prevent the buffer of the video decoder from underflowing or overflowing. . However, since the information indicating the VBV delay value is multiplexed in the bit stream itself, the VBV delay value cannot be known before receiving the bit stream.

In the first and second embodiments, the data receiving means 106a and 106b output the media data for one frame to the decoding means 107a and 107b each time the media data for one frame is received. However, the configuration of the data receiving means is not limited to this.

For example, the data receiving means 106a and 106
6b has a memory for holding the received media data, and reads the received media data from the memory at the time when the clock time in the data receiving device becomes the display start time indicated by the begin attribute in the SMIL data, and decodes the read media data. It may output to 107a and 107b. Further, the data receiving means 106a and 106b
Has a memory for holding the received media data,
When the time counted by the data receiving device becomes a certain time (for example, one second) before the display start time indicated by the begin attribute in the SMIL data (for example, 1 second), the received media data is read from the memory and decoded.
a and 107b.

However, in the configuration in which the decoding process of the received media data is started at the display start time indicated by the begin attribute, the decoding process of the media data is started at the display start time of the media data. Depending on the processing capability of the above, the decoded image data may not be stored in the frame memory at a predetermined time.

Further, in each of Embodiments 1 and 2,
Although a case has been described where SMIL data is received as control data, the control data is not limited to SMIL data. For example, the control data is XHTML (Extensible Hyper Text Markup Language) defined in W3C.
e), HTML (Hyper Text Markup Language) + TIME
(Timed Interactive Multimedia Extensions), or SDP (Session description protocol) specified by IETF (Internet Engineering Task Force), MPE
BIFS (Binary Format For S
For example, control data that designates the display start time of media data, such as cene), can provide the same effects as those of the above embodiments.

In each of Embodiments 1 and 2, the data receiving device is realized by hardware. However, the data transmitting device can be realized by software. For example, SMIL request receiving means 102, signal generating means 104, data request receiving means 105, media data receiving means 10 in data receiving apparatus 110
6a and 106b, the decoding means 107a and 107b, and the display means 109 use the CPU (Central Process
g Unit) can be implemented in a computer system using a software program programmed to perform.

Even when the data receiving apparatus 101 of the first embodiment is realized by such software, the same result as that of the first embodiment can be obtained. The software program can be stored in a storage medium such as a floppy (registered trademark) disk, an optical disk, an IC card, and a ROM cassette.

Furthermore, in each of the above embodiments, a receiving terminal (client terminal) including a data receiving device for receiving control data such as SMIL data and media data.
As a server (data transmission device) for the SMIL data, information indicating a waiting time before displaying media data (prebuffering attribute) or information indicating a time at which media data is requested to the server (request attribute) is received in SMIL data. Although the transmission to the terminal side is shown, the data transmission apparatus transmits information including information indicating the standby time and information indicating the data request time as described above in control data other than the SMIL data. Is also good.

For example, a server (data transmission device) for a receiving terminal that combines and displays a plurality of media data
Transmits control data such as response data to a request from the receiving terminal, including the above prebuffering attribute value, request attribute value or an attribute value corresponding thereto, to the receiving terminal before transmitting the media data, Media data may be transmitted according to a data request message from a terminal. Also in this case, the receiving terminal can request the media data at an appropriate data request time.

Hereinafter, a specific example of the data exchange between the data receiving device in the receiving terminal and the data transmitting device that transmits the information indicating the standby time in the control data other than the SMIL data will be briefly described. explain. FIG.
Describes a case where a data transmission device (server) transmitting media data transmits information including a pre-buffering time (standby time) in an SDP and transmits the media data between the data transmission device and the data reception device. 1 shows an example of data exchange in the first embodiment.

Here, the second server (data transmission device) 2 for transmitting the media data of the first foreground image
The third server (data transmission device) 23c that transmits the media data of the 3b and the second foreground image is to transmit the information indicating the pre-buffering time (standby time) in the SDP. First server (data transmission device) 2
3a has the same configuration as the first server 13a shown in FIG. The data receiving apparatus 130a is mounted on a personal computer (personal computer) as a client terminal.
It is assumed that SMIL data Ds representing the scene description SD shown in FIG. 12 is supplied to a as the scene description data SD. The configuration of the data receiving apparatus 130a is different from that of the conventional data receiving apparatus 901 (see FIG. 13) only in the data request receiving means 908. In the data receiving apparatus 130a, the data request receiving means includes a pre-buffering time. Information indicating (standby time) is acquired and supplied to the control data generation unit 907.

First, for example, while a user is browsing a home page described in HTML (Hyper Text Markup Language) using a web browser provided in a personal computer (PC), a predetermined When the operation of clicking the area linked to the SMIL data is performed, the SMIL data D is transmitted from the data receiving device 130a of the client terminal.
SMIL request command (GET http://s1.co
m / scene.smil) C1 is issued. This command C1
Sends HTTP to the first server (s1.com) 23a.
In order to provide SMIL data.

The SMIL request command C1 is transmitted to the server 2
3a, the server 23a sends a response (HTTP / 1.0 O) indicating that the command has been acknowledged.
K) R1 is issued to the client terminal, and SMIL data (scene.smil) Ds is supplied to the client terminal.

Then, in the data receiving apparatus 130a of the client terminal, the SMIL data Ds is received by the SMIL receiving means 906, and the SMIL receiving means 906 performs a process of analyzing the SMIL data Ds. S
The SMIL analysis information Da obtained by analyzing the SMIL data in the MIL receiving means 906 is
7 and recorded.

After that, the data receiving apparatus 130a sends the third server (s3.com) 23c detailed information on the media data corresponding to the second foreground image (mov) (for example, the Command for requesting the presence of candidate data (DESCRIBE rtsp: //s3.com/mo
v.mpg) C2a is issued.

The command C2a is transmitted to the third server 23c.
Is received from the server 23c, a response R20a indicating that the command has been acknowledged is issued to the client terminal. This response R20a includes a DESCRIBE
OK message (RTSP / 1.0 OK) 21a indicating that command C2a has been accepted, and SDP (Session
Description Protocol) information 22a.
Here, the SDP information 22a includes information necessary for the decoding process of the media data on the receiving terminal side and information necessary for the transmission of the media data. Here, in addition to these information, "(a = prebuffering: 15s) contains information. The description in the SDP information R22a (V =
0) indicates version information on the configuration of the SDP,
The description (m = video) indicates that information about the image data is described after this description. The information (a = prebuffering: 15s) is a foreground image (mov)
This indicates that the time to wait from when the media data is requested to when the media data is displayed is 15 seconds.

Next, a preparation is made for providing the media data corresponding to the second foreground image (mov) from the data receiving device 130a of the client terminal to the third server (s3.com) 23c. A preparation request command (SETUP rtsp: //s3.com/adv.mpg) C3a is issued. Then, when the third server 23c is ready to provide the media data, a response (RTSP / 1.0 OK) R indicating that the command C3a has been acknowledged is received.
3a is issued to the client terminal.

Subsequently, from the data receiving apparatus 130a, the second server (s2.com) 23b sends detailed information on the media data corresponding to the first foreground image (adv) (for example, the coding conditions and the like). Command (DESCRIBE rtsp: //s2.com/ad) that requests multiple candidate data
v.mpg) C2b is issued.

The command C2b is transmitted to the second server 23b
Is received from the server 23b, a response R20b indicating that the command has been acknowledged is issued to the client terminal. This response R20b includes a DESCRIBE
An OK message (RTSP / 1.0 OK) 21b indicating that the command C2b has been accepted, and an SDP (Session
Description Protocol) information 22b.
The SDP information 22b includes (V = 0) information, (m = vid
eo) information, etc., includes (a = prebuffering: 7s) information. This (a = prebuffering: 7s) information is the first
Indicates that the time to wait from the request of the media data (adv.mpg) of the foreground image (adv) to the display thereof is 7 seconds.

Next, a preparation is made to provide the media data corresponding to the first foreground image (adv) from the data receiving device 130a of the client terminal to the second server (s2.com) 23b. A preparation request command (SETUP rtsp: //s2.com/adv.mpg) C3b is issued. Then, when the second server 23b is ready to provide the media data, a response (RTSP / 1.0 OK) R indicating that the command C3b has been acknowledged is received.
3b is issued to the client terminal.

Then, from the data receiving device 130a of the client terminal, 15 seconds before the display start time of the second foreground image (5 seconds before the display start time of the entire scene), the third server (s3.com) ) 23c, a data request command (PLAY rtsp: //s3.com/adv.m) for requesting media data (mov.mpg) corresponding to the second foreground image (mov).
pg) C4a is issued. Then, the third server 23
In step c, a response (RTSP / 1.0 OK) R4a indicating that the command C4a has been acknowledged is issued to the client terminal. Thereafter, the third server 23c issues a second foreground image (mov. Media data Dm2 corresponding to mpg) is sequentially supplied to the client terminal by an RTP packet.

Further, from the data receiving device 130a of the client terminal, 7 seconds before the display start time of the first foreground image (2 seconds before the display start time of the entire scene), the second foreground image is displayed.
Data request command (PLAY rtsp: //s2.com/adv.m) for requesting the media (adv.mpg) corresponding to the first foreground image (adv) to the server (s2.com) 23b of
pg) C4b is issued. Then, the second server 23
In b, a response (RTSP / 1.0 OK) R4b indicating that the command C4b has been acknowledged is issued to the client terminal, and then the first foreground image (adv) is sent from the second server 23b. Are sequentially supplied to the client terminal in the form of RTP packets. After that, each media data is
It is output to the display means 905 at the display start time, and the S
It is displayed based on the analysis result of the MIL data.

As described above, the data transmission method for transmitting information from the data transmission apparatus to the data reception apparatus by including the information indicating the standby time (pre-buffering time) in control data (SDP information) other than the SMIL data is performed in real time. This is very effective for content (a video showing a live broadcast pattern of a concert) in which the initial delay time, that is, the waiting time from when the media data is requested to the server until the display thereof is started, changes.

FIG. 10 shows a case where a data transmission apparatus (server) for transmitting media data transmits information indicating a pre-buffering time (standby time) by including it in a response to a SETUP request of RTSP. 4 shows an example of data exchange between the data transmitting device and the data receiving device.

Here, the second server (data transmission device) 3 for transmitting the media data of the first foreground image
The third server (data transmission device) 33c that transmits the media data of the 3b and the second foreground image transmits information indicating the pre-buffering time (standby time) to the SE of the RTSP.
This is included in the response to the TUP request and transmitted.
The first server (data transmission device) 33a has the same configuration as the first server 13a shown in FIG. It is assumed that the data receiving device 130b is mounted on a personal computer (personal computer) as a client terminal, and that the SMIL data shown in FIG. 12 is supplied to the data receiving device 130b as scene description data Ds. I do. The data receiving device 130
The configuration of b differs from the conventional data receiving device 901 (see FIG. 13) only in the data request receiving means 908. In the data receiving device 130b, the data request receiving means
Information indicating the pre-buffering time (standby time) is obtained and supplied to the control data generation unit 907.

First, for example, while a user is browsing a home page described in HTML (Hyper Text Markup Language) using a web browser provided in a personal computer (PC), a predetermined When the operation of clicking the area linked to the SMIL data is performed, the SMIL data D is transmitted from the data receiving device 130b of the client terminal.
SMIL request command (GET http://s1.co
m / scene.smil) C1 is issued. This command C1
Sends HTTP to the first server (s1.com) 33a.
In order to provide SMIL data.

When the SMIL request command C1 is received by the first server 33a, a reply indicating that the command has been acknowledged (HTTP / 1.
(0OK) R1 is issued to the client terminal, and SMIL data (scene.smil) Ds is supplied to the client terminal.

Then, in the data receiving apparatus 130b of the client terminal, the SMIL data Ds is received by the SMIL receiving means 906, and the SMIL receiving means 906 performs a process of analyzing the SMIL data Ds. S
The SMIL analysis information Da obtained by analyzing the SMIL data in the MIL receiving means 906 is
7 and recorded.

Thereafter, the data receiving device 130b sends detailed information (for example, encoding conditions and multiple data) related to the media data corresponding to the second foreground image (mov) to the third server (s3.com) 33c. Command for requesting the presence of candidate data (DESCRIBE rtsp: //s3.com/mo
v.mpg) C2a is issued.

The command C2a is transmitted to the third server 33c.
Is received from the server 33c, a response R2a indicating that the command has been acknowledged is issued to the client terminal, and an SDP (Session Descript
ion protocol) information is supplied to the client terminal.

Next, a preparation is made to provide media data corresponding to the second foreground image (mov) from the data receiving device 130b of the client terminal to the third server (s3.com) 33c. A preparation request command (SETUP rtsp: //s3.com/mov.mpg) C3a is issued. Then, when the third server 33c is ready to provide the media data, a response R30a indicating that the command C3a has been acknowledged is issued to the client terminal.

This response R30a includes a SETUP command C
3a is accepted (RTSP /
1.0 OK) 31a and other additional information 32a. The additional information 32a includes sequence number (CSeq: 2) information and a session number (Session: 123456).
78) The information includes (a = prebuffering: 15s) information. This (a = prebuffering: 15s) information indicates that the time to wait from the request of the media data of the foreground image (mov) to the display thereof is 15 seconds. The sequence number (CSeq:
2) is one between the data transmitting device and the data receiving device.
This number is assigned to each message exchange, and the same sequence number is assigned to the issuance of a command from the receiving side and the corresponding response from the transmitting side. Therefore, although not shown here, the command (DESCRIBE r
tsp: //s3.com/mov.mpg) The sequence number (CSeq: 1) is assigned to the C2a and the response R2a. Further, the session number is given to a data transmission state established between the data transmitting device and the data receiving device.

Subsequently, the data receiving device 130b sends detailed information (for example, coding conditions and the like) on media data corresponding to the first foreground image (adv) to the second server (s2.com) 33b. Command (DESCRIBE rtsp: //s2.com/ad) that requests multiple candidate data
v.mpg) C2b is issued.

The command C2b is transmitted to the second server 33b.
Is received from the server 33b, a response R2b indicating that the command has been acknowledged is issued to the client terminal, and an SDP (Session Descript
ion protocol) information is supplied to the client terminal.

[0210] Next, preparation is made to provide media data corresponding to the first foreground image (adv) from the data receiving device 130b of the client terminal to the second server (s2.com) 33b. A preparation request command (SETUP rtsp: //s2.com/adv.mpg) C3b is issued. Then, as soon as the second server 33b is ready to provide the media data, a response R30b indicating that the command C3b has been acknowledged is issued to the client terminal.

This response R30b includes a SETUP command C
3b OK message (RTSP /
1.0 OK) 31b and other additional information 32b. The additional information 32b includes sequence number (CSeq: 2) information and a session (Session: 12345688).
Both the information and the like include (a = prebuffering: 7s) information. This (a = prebuffering: 7s) information indicates that the time to wait from the request of the media data of the foreground image (adv) to the display thereof is 7 seconds.

From the data receiving device 130b of the client terminal, 15 seconds before the display start time of the second foreground image (5 seconds before the display start time of the entire scene),
A data request command (PLAY rtsp: //s3.com/ad) for requesting the third server (s3.com) 33c for media data (mov.mpg) corresponding to the second foreground image (mov).
v.mpg) When C4a is issued, the third server 33c
Then, a response (RTSP / 1.0 OK) R4a indicating that the command C4a is acknowledged is issued to the client terminal. Thereafter, the media data Dm2 corresponding to the second foreground image (mov.mpg) is sequentially supplied from the third server 33c to the client terminal in the form of an RTP packet.

[0213] Further, the data receiving device 130b of the client terminal sends the second server (s2.s.) seven seconds before the display start time of the first foreground image (two seconds before the display start time of the entire scene). com) 33b, a data request command (PLAY rtsp: //s2.com/adv.m) for requesting media data (adv.mpg) corresponding to the first foreground image (adv).
pg) When the C4b is issued, the second server 33b responds (R) indicating that the command C4b has been acknowledged.
TSP / 1.0 OK) R4b is issued to the client terminal. Thereafter, the media data Dm1 corresponding to the first foreground image (adv) is sequentially supplied from the second server 33b to the client terminal in the form of an RTP packet. After that, each media data is
It is output to the display means 905 at the display start time, and the S
It is displayed based on the analysis result of the MIL data.

As described above, information indicating the standby time (pre-buffering time) is included in control data (response of the server to the SETUP request of the receiving terminal) other than the SMIL data and transmitted from the data transmitting apparatus to the data receiving apparatus. The data transmission method is a method for real-time, initial delay time, that is, for a content (a video showing a concert pattern that is broadcast live) in which the waiting time from when the media data is requested to the server until the display is started is changed. It will be very effective.

[0215]

According to the data receiving apparatus of the present invention (claim 1), location information indicating the location of the data source holding each media data on the network, the reproduction start time of each media data And second time information corresponding to each of the media data, for requesting the media data to the data source.
And a receiving unit for receiving the time information of the first and second times.
A data request process for requesting each of the media data to its data source based on the time information of the media data by a time earlier than the reproduction start time of the respective media data by a specific time set for each of the media data. Therefore, there is an effect that the reproduction of each media data can be started at the time specified on the transmission side.

[0216] According to the present invention (claim 2), in the data receiving apparatus according to claim 1, the respective media data is reproduced after receiving the respective media data as the second time information. Since the time information indicating the standby time until is received, a data request process for requesting each media data to a predetermined data source is performed at a time earlier than the playback start time of each media data by the standby time, On the receiving side, there is an effect that each media data can be obtained from a predetermined data source on the network during the standby time before the reproduction of each media data.
In addition, by setting the standby time to a sufficiently large value in consideration of the state of the network in which the media data is transmitted, for example, the bandwidth and congestion degree, the process of reproducing the media data in the data receiving device can be performed in the network. This also makes it less susceptible to jitter, and has the effect of suppressing interruption of image display during playback of media data.

According to the present invention (claim 3), in the data receiving apparatus according to claim 1, the second time information indicates a time at which each of the media data should be requested from a corresponding data source. The data request processing for receiving the time information and requesting each of the media data from a predetermined data source is performed at the data request time, so that the receiving side performs the data request processing from the data request time to the data reproduction start time. There is an effect that each media data can be obtained from a predetermined data source on the network. In addition, by setting the data request time to a time sufficiently earlier than the data reproduction start time in consideration of the state of the network through which the media data is transmitted, for example, the bandwidth and congestion degree, the data reception device can The media data reproduction process is less susceptible to network jitter, and also has the effect of preventing interruption of image display during media data reproduction.

According to the present invention (claim 4), in the data receiving apparatus according to claim 1, the respective media data is reproduced after receiving the respective media data as the second time information. Receiving the time information indicating the standby time up to, and performing a data request process for requesting each of the media data to a predetermined data source, from the reproduction start time of each of the media data, the standby time and a predetermined preset Since the playback is performed at a time earlier by the total time, the playback of each media data can be started at the time specified on the transmission side, and the playback processing of the media data on the reception side can reduce the jitter of the network. It is less susceptible to the influence, and there is an effect that it is possible to suppress interruption of image display during reproduction of media data.

According to the present invention (claim 5), in the data receiving apparatus according to claim 1, as the second time information, a data request time at which each of the media data should be requested to a corresponding data source. Receiving the time information indicating, and performing a data request process of requesting each of the media data to a predetermined data source at a time earlier than the data request time by a predetermined time set in advance,
Not only can the playback of each media data start at the time specified on the transmission side, but also the playback processing of the media data on the reception side is less affected by network jitter, and the media data is being played back. Thus, there is an effect that interruption of image display can be suppressed.

According to the data receiving method of the present invention (claim 6), location information indicating the location of the data source holding each media data on the network, and the reproduction start time of each media data are indicated. First time information, and second media information corresponding to each of the media data, for requesting the media data from the data source.
A data request process for requesting each of the media data to its data source based on the first and second time information,
Since the reproduction is performed at a time earlier than the reproduction start time of each of the media data by a specific time set for each of the media data, the reproduction of the media data corresponding to each element constituting the scene is designated by the transmission side. There is an effect that it is possible to start on time.

According to the present invention (claim 7), in the data receiving method according to claim 6, the respective media data is reproduced after receiving the respective media data as the second time information. Since the time information indicating the standby time until is received, a data request process for requesting each media data to a predetermined data source is performed at a time earlier than the playback start time of each media data by the standby time, On the receiving side, there is an effect that each media data can be obtained from a predetermined data source on the network during the standby time before the reproduction of each media data.
In addition, by setting the standby time to a sufficiently large value in consideration of the state of the network in which the media data is transmitted, for example, the bandwidth and congestion degree, the process of reproducing the media data in the data receiving device can be performed in the network. This also makes it less susceptible to jitter, and has the effect of suppressing interruption of image display during playback of media data.

According to the present invention (claim 8), in the data receiving method according to claim 6, the second time information indicates a time at which each of the media data should be requested from a corresponding data source. The data request processing for receiving the time information and requesting each of the media data from a predetermined data source is performed at the data request time, so that the receiving side performs the data request processing from the data request time to the data reproduction start time. There is an effect that each media data can be obtained from a predetermined data source on the network. In addition, by setting the data request time to a time sufficiently earlier than the data reproduction start time in consideration of the state of the network through which the media data is transmitted, for example, the bandwidth and congestion degree, the data reception device can The media data reproduction process is less susceptible to network jitter, and also has the effect of preventing interruption of image display during media data reproduction.

According to the present invention (claim 9), in the data receiving method according to claim 6, the respective media data is reproduced after receiving the respective media data as the second time information. Receiving the time information indicating the standby time up to, and performing a data request process for requesting each of the media data to a predetermined data source, from the reproduction start time of each of the media data, the standby time and a predetermined preset Since the playback is performed at a time earlier by the total time, the playback of each media data can be started at the time specified on the transmission side, and the playback processing of the media data on the reception side can reduce the jitter of the network. It is less susceptible to the influence, and there is an effect that it is possible to suppress interruption of image display during reproduction of media data.

According to the present invention (claim 10), claim 6
In the data receiving method described above, as the second time information, time information indicating a data request time at which each of the media data should be requested to a corresponding data source is received; Since the data request processing for requesting the media data is performed at a time earlier than the data request time by a preset predetermined time,
Not only can the playback of each media data start at the time specified on the transmission side, but also the playback processing of the media data on the reception side is less affected by network jitter, and the media data is being played back. Thus, there is an effect that interruption of image display can be suppressed.

According to the data transmission method of the present invention (claim 11), location information indicating the location of the data source holding each media data on the network and the reproduction start time of each media data are indicated. First time information, and second media information corresponding to each of the media data, for requesting the media data from the data source.
And transmitting the media data to the receiving terminal in response to the media data request issued from the receiving terminal based on the first and second time information and the location information. Therefore, on the receiving side, before the reproduction of the media data, the media data is acquired from a predetermined data source on the network based on the second time information, and the reproduction of the media data is performed.
There is an effect that it is possible to start at the time specified on the transmission side.

According to the present invention (claim 12), claim 1
In the data transmission method described in 1, the second time information is time information indicating a standby time from when each of the media data is received to when each of the media data is reproduced. There is an effect that each media data can be obtained from a predetermined data source on the network during the standby time before the reproduction of each media data. Also, by setting the standby time to a sufficiently large value in consideration of the state of the network in which the media data is transmitted, for example, the bandwidth and congestion degree, the process of reproducing the media data on the data receiving side is performed by the network. This also makes it less susceptible to jitter, and has the effect of suppressing interruption of image display during playback of media data.

According to the present invention (claim 13), claim 1
In the data transmission method according to the first aspect, the second time information is time information indicating a data request time at which each of the media data should be requested to the corresponding data source. There is an effect that each media data can be obtained from a predetermined data source on the network between the time and the data reproduction start time. Also, by setting the data request time to a time sufficiently earlier than the data reproduction start time in consideration of the state of the network where the media data is transmitted, for example, the bandwidth and congestion degree, the data reception side The media data reproduction process is less susceptible to network jitter, and also has the effect of preventing interruption of image display during media data reproduction.

According to the data recording medium of the present invention (claim 14), the data reproduction for reproducing the media data obtained from the data source on the network to display one scene by the computer is performed. As the program, location information indicating the location of the data source holding the respective media data on the network, first time information indicating the reproduction start time of the respective media data, and the respective media data are provided. A first program for performing, by a computer, a process of receiving second time information for requesting the media data from the data source;
And a data request for requesting each of the media data to the data source indicated by the location information is set for each of the media data based on the reproduction start time of the respective media data based on the second time information. And a second program for performing a process to be performed at a time earlier by a specific time by the computer, so that the media data corresponding to the individual elements constituting the scene can be stored at the time specified by the transmission side. There is an effect that data reproduction processing to be reproduced on the receiving side can be performed by software.

[0229] According to the data recording medium of the present invention (claim 15), it is possible for a computer to perform a data transmission process of transmitting media data to a receiving terminal that reproduces the media data and displays one scene. As a data transmission program, location information indicating a location on the network of a data source holding the media data, first time information indicating a reproduction start time of the media data, and media data. A first program for performing, by a computer, a corresponding process of transmitting second time information for requesting the media data to the data source;
In response to the request for each of the media data issued from the receiving terminal based on the second time information and the location information,
A process of transmitting each of the media data to the receiving terminal,
Since the second program to be executed by the computer is stored, the data transmission processing for transmitting the media data to the receiving side so that the reproduction of each media data on the receiving side is performed at the time designated by the transmitting side. Can be performed by software and wear.

[Brief description of the drawings]

FIG. 1 is a data receiving apparatus 1 according to a first embodiment of the present invention.
FIG. 10 is a block diagram for explaining FIG.

FIG. 2 shows the contents (scene description) S of SMIL data Ds1 supplied to the data receiving apparatus 110 according to the first embodiment.
It is a figure showing D1.

FIG. 3 is a diagram showing a spatial arrangement (FIG. (A)) and a temporal arrangement (FIG. (B)) of media data based on SMIL data Ds1 supplied to the data receiving apparatus 110 according to the first embodiment. .

FIG. 4 is a diagram showing a time table created by a control data storage processing unit 103 of the data receiving apparatus 110 according to the first embodiment.

FIG. 5 is a diagram illustrating a flow of a process of acquiring media data from a server by the data receiving apparatus 110 according to the first embodiment.

FIG. 6 is a diagram illustrating a process of calculating an issue time of a media data request command in the data receiving apparatus 110 according to the first embodiment.

FIG. 7 is a data receiving apparatus 1 according to a second embodiment of the present invention.
FIG. 20 is a block diagram for explaining No. 20;

FIG. 8 shows the contents (scene description) S of SMIL data Ds2 supplied to the data receiving apparatus 120 according to the second embodiment.
It is a figure which shows D2.

FIG. 9 is a diagram showing a method of transmitting information including a pre-buffering time (standby time) in an SDP as a data transmission method according to an embodiment of the present invention.

FIG. 10 is a diagram showing a method of transmitting information indicating a pre-buffering time (standby time) in a response to a SETUP request of RTSP as a data transmission method according to an embodiment of the present invention.

FIG. 11 shows SMIL supplied to a conventional data receiving apparatus.
Spatial arrangement of media data based on data Ds (Fig.
(a)) and a diagram showing a temporal arrangement (FIG. (b)).

FIG. 12 shows SMIL supplied to a conventional data receiving apparatus.
FIG. 3 is a diagram showing the contents (scene description) SD of data Ds.

FIG. 13 is a block diagram for explaining a conventional data receiving device.

FIG. 14 is a diagram illustrating a flow of a process of acquiring media data from a server by a conventional data receiving apparatus.

[Explanation of symbols]

13a, 23a, 33a First server (data transmitting device) 13b, 23b, 33b Second server (data transmitting device) 13c, 23c, 33c Third server (data transmitting device) 101a Clock circuit 102 SMIL request receiving means 103 control data recording processing unit 104 trigger signal generating means 105 data request receiving means 106a, 106b media data receiving means 107a, 107b decoding means 108a, 108b frame memory 109 display means 110, 120, 130a, 130b data receiving devices 110a, 120a control Data generation unit 201a root-layout element 202a, 203a region element 204a, 205a, 601a, 602a video element 1100 image space 1110 background arrangement area 1120 first foreground arrangement area 1130 second front Arrangement area Da1, Da2 First and second SMIL analysis data Dc1, Dc2, Dc3 First, second, third control data Dd1, Dd2 First, second decoded image data Dm1, Dm2 First, second Media data (image data) Ds1, Ds2 First and second SMIL data E1 to E5 Event It Time information Sack Response signals SD1, SD2 Scene description Srd SMIL request signal Srp Command signal St Trigger signal Tab Time table

Claims (15)

[Claims] 1. Media data corresponding to each element constituting a scene, which is one of image data, audio data, and text data, is obtained from a data source on a network, and the obtained media data is reproduced. A data receiving apparatus for displaying a scene, comprising: location information indicating a location on the network of a data source holding the media data; first time information indicating a reproduction start time of the media data;
And a first receiving unit that receives second time information for requesting the media data from the data source, corresponding to each of the media data, based on the first and second time information A data request time for requesting each of the media data from its data source, from a reproduction start time of each of the media data,
A time setting unit for setting a time earlier by a unique time set for each media data; and a data requesting process for requesting each of the media data to a data source indicated by the location information. A data requesting unit for performing at a data requesting time set by the data requesting unit, and a second receiving unit for receiving media data supplied from the data source in response to a request from the data requesting unit. Receiver. 2. The data receiving apparatus according to claim 1, wherein the first receiving unit receives the respective media data as the second time information and thereafter performs the reproduction of the respective media data. The time setting unit sets the data request time of the media data to a time earlier by the standby time than the reproduction start time of each of the media data. A data receiving device, comprising: 3. The data receiving device according to claim 1, wherein the first receiving unit indicates, as the second time information, a time indicating a time at which each of the media data should be requested from a corresponding data source. A data receiving apparatus for receiving information, wherein the time setting unit sets a data request time of each of the media data to a time indicated by the second time information. 4. The data receiving device according to claim 1, wherein the first receiving unit receives the respective media data as the second time information, and then performs the reproduction of the respective media data. The time setting unit receives the data request time of each of the media data, from the reproduction start time of each media data, the standby time and a predetermined time set in advance. A data receiving apparatus, wherein the time is set earlier by the total time of the data. 5. The data receiving apparatus according to claim 1, wherein the first receiving unit sets the second time information as a time indicating a time at which each of the media data should be requested from a corresponding data source. The time setting unit sets the data request time of each of the media data to a time earlier by a predetermined time than the time indicated by the second time information. A data receiving device, comprising: 6. Media data corresponding to each element constituting a scene, which is one of image data, audio data, and text data, is obtained from a data source on a network, and the obtained media data is played back. A data receiving method for displaying a scene, comprising: location information indicating a location on the network of a data source holding the respective media data; first time information indicating a reproduction start time of the respective media data;
And a first receiving step of receiving second time information for requesting the media data to the data source corresponding to each of the media data, and based on the first and second time information. A data request process for requesting the respective media data to the data source indicated by the location information at a time earlier than the reproduction start time of the respective media data by a specific time set for the respective media data. A data receiving method, comprising: a data requesting step to be performed; and a second receiving step of receiving media data supplied from the data source in response to the data requesting process in the data requesting step. 7. The data receiving method according to claim 6, wherein the first receiving step is performed from when the respective media data is received as the second time information to when the respective media data is reproduced. Receiving the time information indicating the standby time of the media data. In the data requesting step, the data request processing for requesting each media data to a predetermined data source is performed based on the reproduction start time of each media data. A data receiving method, which is performed at a time earlier by time. 8. The data receiving method according to claim 6, wherein the first receiving step includes, as the second time information, a data request time to request each of the media data from a corresponding data source. Receiving the time information indicating, the data requesting step, the data request processing for requesting each of the media data to a predetermined data source,
A data receiving method which is performed at the data request time. 9. The data receiving method according to claim 6, wherein the first receiving step is performed from receiving the respective media data as the second time information until reproducing the respective media data. Receiving the time information indicating the standby time of, the data requesting step, a data request processing for requesting each of the media data to a predetermined data source,
A data receiving method which is performed at a time earlier than the reproduction start time of each media data by a total time of the standby time and a predetermined time set in advance. 10. The data receiving method according to claim 6, wherein the first receiving step includes, as the second time information, a data request time to request each of the media data from a corresponding data source. Receiving the time information indicating, the data requesting step, the data request processing for requesting each of the media data to a predetermined data source,
A data receiving method, which is performed at a time earlier by a predetermined time than the data request time. 11. Corresponding to each element constituting a scene,
A data transmission method for transmitting media data, which is one of image data, audio data, and text data, to a receiving terminal that reproduces the media data and displays the scene, wherein a data source that holds the media data Location information indicating a location on the network, first time information indicating a reproduction start time of each of the media data,
And a first transmission step of transmitting second time information for requesting the media data to the data source corresponding to each of the media data, and the first and second time information and the location. Transmitting the media data to the receiving terminal in response to a request for the media data issued from the receiving terminal based on the information. 12. The data transmission method according to claim 11, wherein the second time information is time information indicating a standby time from when the respective media data is received to when the respective media data is reproduced. A data transmission method, comprising: 13. The data transmission method according to claim 11, wherein the second time information is time information indicating a data request time to request each of the media data from a corresponding data source. Data transmission method. 14. corresponding to each element constituting a scene,
Data playback for acquiring media data, which is one of image data, audio data, and text data, from a data source on a network and playing back the acquired media data to display the scene by a computer. A data storage medium storing a program, wherein the data reproduction program indicates location information indicating a location on the network of a data source holding each of the media data, and indicates a reproduction start time of each of the media data. First time information,
And a first program for performing, by a computer, a first process of receiving second time information for requesting the media data to the data source, the first program corresponding to each of the media data, Based on the first and second time information, a data request for requesting each of the media data to the data source indicated by the location information is set for each of the media data from the reproduction start time of the media data. A second program for performing, by a computer, a second process that is performed earlier by the specified inherent time, and a third process for receiving media data supplied from the data source in response to the data request. And a third program to be executed by a computer. 15. Corresponding to each element constituting a scene,
Data storing a data transmission program for performing, by a computer, data transmission processing for transmitting media data, which is one of image data, audio data, and text data, to a receiving terminal that reproduces the media data and displays the scene A storage medium, the data transmission program comprising: location information indicating a location of the data source holding the respective media data on the network; first time information indicating a reproduction start time of the respective media data. ,
And a first program for performing, by a computer, a first process for transmitting second time information for requesting the media data to the data source corresponding to each of the media data, 1. A second process for transmitting each of the media data to the receiving terminal in response to a request for each of the media data, which is issued from the receiving terminal based on the second time information and the location information, by a computer And a second program for executing the data storage medium.