JP3584787B2 - Multimedia data playback / transmission device - Google Patents

Description

[0001]
TECHNICAL FIELD OF THE INVENTION
The present invention relates to a multimedia data reproducing apparatus using a packet network, which can be realized even in a narrow band (such as ISDN) and a wide area.
[0002]
[Prior art]
Assuming that the data stored on the server is played back on the client,
The transmission / transmission method of how data on the server is transmitted to the client is a problem, and there are two methods as the method.
One is a server-based transmission method (so-called PUSH type) in which the server continues to autonomously transmit the data until a stop request comes in response to a “reproduction start” request from the client,
The other is a client-driven acquisition method (a so-called PULL type) in which a server does not have an autonomous transmission mechanism and passes data each time a client reads a request.
[0003]
In the PULL type, since the server only needs to have a normal file reading interface, it can be realized with a simple configuration. For example, when operating on a LAN, it is sufficient that the data is stored as a readable file. As long as the data on the server can be accessed without running the application on the server, the client can acquire the data at any time. Therefore, there is no problem that the service is stopped due to the failure of the application process.
[0004]
However, when trying to construct this PULL type on a wide area network, the delay from when the client requests reading to when the data is actually received cannot be ignored, and the “reception → immediate reproduction” is continuously performed. In a required real-time stream data transmission system, once a reception request is issued, a PUSH type, which can avoid a round trip delay because data is automatically transmitted in a certain band, must be mainstream.
[0005]
Therefore, a structure of a conventional multimedia data reproducing apparatus shown in FIG. 14 is known. In FIG. 14, a data sending server 1401 and a receiving terminal 1402 are connected to a packet network 1403 and send / receive multimedia data.
[0006]
The data transmission server 1401 includes a network interface 1404a, a data transmission unit 1405a, a data reception unit 1406a, a request reception unit 1410, a multi-client scheduler 1411, and a stream generation unit 1407.
[0007]
The receiving terminal 1402 includes a request transmitting unit 1412, a data receiving unit 1406b, a multiplexing / demultiplexing unit 1408 for separating data into data streams for each medium, and a multiplexing / demultiplexing unit in addition to the normal data communication means similar to the data transmission server. It comprises a receiving buffer 1409 for temporarily storing the output from the unit 1408, a media decoder 1421 for decoding a stream for each medium, and a reproducing unit 1420 for outputting multimedia information by combining the decoded data.
[0008]
The operation of reproducing the multimedia data reproducing apparatus configured as described above will be described below.
When the receiving terminal 1402 intends to receive multimedia data from the server 1401 connected to the packet network 1403, the request transmitting unit 1412 requests and transmits the data. The terminal request is received by the request receiving unit 1410 of the server 1401 via the network interface 1404b, 1404a, registered in the multi-client scheduler 1411, and in accordance with the instruction of the scheduler, the stream generation unit 1407 determines the fixed bit rate of the data to be requested. Create a stream. The generated stream is transmitted from the data transmission unit 1405a to the packet network 1403 via the network interface unit 1404a.
[0009]
The receiving terminal 1402 reads the data from the network interface unit 1404b to the data receiving unit 1406b, and separates the packet header there. The payload data is separated for each medium, temporarily stored in the corresponding reception buffer 1409, decoded by the media decoder 1421, passed to the reproduction unit 1420, and reproduced as multimedia information.
[0010]
[Problems to be solved by the invention]
Thus, in the conventional multimedia data reproducing apparatus,
Although a file in which multimedia data has been multiplexed into one has been transmitted from the server, in this case, data that depends on the time axis and data that does not depend on the time axis will be multiplexed, making it unsuitable for random access. When the playback point is jumped, it is necessary to receive in advance data necessary for starting playback from the jump destination, and the waiting time until the start of playback has been long.
[0011]
In addition, a detailed data management mechanism was needed to know necessary data at the jump destination.
In addition, the band and the area are mainly assumed to be a practical form in a LAN environment, and are not sufficient for a narrow band (such as ISDN) and a wide environment.
[0012]
The present invention solves these problems,
To provide a configuration of a multimedia data reproducing apparatus that can easily perform random access and can reduce a waiting time until the start of reproduction when jumping,
Further, providing a configuration for determining a transmission rate of a plurality of still image data according to the necessity of still image data display from multimedia content information,
In addition, to provide a configuration that attaches a transmission priority from parameters such as the display area of the still image data and can transmit at a rate according to the priority,
Also, in a configuration in which data is transmitted for each medium, overhead is large and transmission efficiency is low, but a configuration capable of increasing transmission efficiency is provided.
It is another object of the present invention to provide a system which can be realized by either a PULL type having a simple configuration or a PUSH type which is excellent in countermeasures for data transmission / reception delay.
[0013]
[Means for Solving the Problems]
In order to solve these problems, the present invention firstly provides:
ReRead file data from mote file system via networkHaving reading means corresponding to the type of media constituting the multimedia data,A remote file data reading unit,Remote file dataReading sectionDeDataDeA playback unit to code and displayA multimedia data description information indicating an attribute of data to be read and a scenario information indicating information relating to reproduction control.Multimedia content information reference sectionAnd assigning and instructing the reading unit to a transmission bit rate to each medium constituting multimedia by combining the network band usable by the apparatus and the multimedia content information from the multimedia content information reference unit. Multimedia data reading scheduler,Equipped,
PreviousThe remote file data reading section, SaidThe bit rate specified by the multimedia data reading schedulerMaintain,Read from each specified fileRowI
PreviousReadingOne of the imprints,Reads data obtained by encoding a single still image into a video format consisting of multiple frames without depending on the time information of the framesSee beforeThe display start time and display end time information of the still image data is obtained from the multimedia content information reference section, reading is started from the display start time, and reading is ended at the display end time.And
Further, when there are a plurality of still images in the data constituting the multimedia data, the display start time and the display end time of each still image are stored in the plurality of still image file reading management tables.
Information, and switch the still image file to be read only during the display period.If multiple files need to be read at the same time, adjust the bit rate given for the still image. Including a remote file data reading unit that supports a still image moving image format to be read
It is configured.
[0014]
As a result, multimedia data can be received and reproduced according to the multimedia content information, and a random access function with a short waiting time during jump can be provided.
[0015]
FurtherEven when there is a still image that overlaps in the display period, the bit rate can be read evenly divided between the still image data that are simultaneously displayed.
[0016]
Second,
A remote file data reading unit for reading file data from a remote file system via a network, having reading means corresponding to the type of media constituting the multimedia data, and a reproduction for receiving, decoding and displaying data from the reading unit A multimedia content information reference unit for referring to multimedia data description information indicating attributes of data to be read and scenario information indicating information relating to reproduction control; and a network which can be used by the present apparatus for the reading unit. A multimedia data reading scheduler for allocating and instructing a transmission bit rate to each medium constituting multimedia by combining a band and multimedia content information from the multimedia content information reference unit.
The remote file data reading unit maintains a bit rate specified by the multimedia data reading scheduler, performs reading from each specified file,
One of the reading means reads data obtained by encoding a single still image into a moving image format including a plurality of frames without depending on time information of a frame, and reads still image data from the multimedia content information reference unit. Obtain information of display start time and display end time, start reading from the display start time, end reading at the display end time,
Further,If there are multiple still images in the data that make up the multimedia data,Still image data configuration information tablePriorities are assigned to still image data according to the display area size and data size described in, and when multiple files need to be read at the same time when switching the read file based on the display start time and display end time information, The bit rate given for the image according to the priority orderReads remote file data compatible with video format for still images that performs read adjustmentWith means,
As a result, even when still images having different characteristics, importance levels, and reading states are simultaneously displayed, optimum reading that reflects the magnitude of the effect on multimedia information reproduction can be performed.
[0017]
No.3First,, SecondIn addition to the configuration
The playback unit includes a playback control interface capable of controlling playback of pause / stop release / enlarged display of an object, and sends an event notification message to the multimedia data reading unit when a playback control event occurs. With
The multimedia content information reference unit further includes scenario information indicating information on reproduction control of data to be read, and determines whether each read data depends on a time axis,
MaMultimediaThe content information reference sectionReadingFurther includes scenario information indicating information relating to reproduction control of the target data., Each read data depends on the time axisOr notOrJudge,
The multimedia data read scheduler redistributes the bit rate to the reading means of each remote file data according to the change of the reproduction status, and preferentially reads the file data of the media depending on the time axis during normal reproduction, When paused, the file data of the media independent of the time axis is read preferentially, and when the object is enlarged and displayed, the file data of the target object is read preferentially,
This allows the multimedia data reading scheduler to redistribute the bit rate to each remote file data reading unit according to the change in the playback status, and to read the file data of the media depending on the time axis preferentially during normal playback, temporarily When stopped, the file data of the media that does not depend on the time axis is read preferentially, and when the object is enlarged and displayed, the file data of the target object can be read preferentially, and the read rate can be assigned according to the playback state. is there.
[0018]
No.4To
A multimedia data reproducing system comprising a data transmitting server connected to a packet network and a receiving terminal, which is related to a transmitting side (multimedia data transmitting apparatus),
The data transmission server includes a network interface, a transmission request reception unit, and a data transmission unit that reads out file data from a local file system corresponding to each medium constituting multimedia, generates a stream, and transmits the stream to the network. ,
The receiving terminal is provided with a network interface, a request transmitting unit to the server, a data receiving unit from the server, a receiving buffer for temporarily storing received data, and data is taken out from the receiving buffer and immediately decoded and displayed. Comprising a playback unit, the stream generation unit of the data transmission server comprises a multimedia content information reference unit, referring to information such as encoding parameters of each data constituting the multimedia content,
Multimedia data for assigning and indicating a transmission bit rate to each medium constituting multimedia by combining a network band usable by the multimedia data reproducing apparatus and multimedia content information from the multimedia content information reference unit. A transmission scheduler, and a remote file data transmission unit for each of a plurality of files stored for each medium constituting the multimedia data,
The media data sending unit controls the bit rate specified by the multimedia data reading scheduler.Maintain,The specified data is sent in parallel,
One of the remote file data reading units is,A still image moving image format compatible data transmitting unit that transmits data obtained by encoding one still image into a moving image format composed of a plurality of frames,
The still image moving image format compatible data transmission unit includes a management table for transmitting a plurality of still image files,
When there are a plurality of still images in the data constituting the multimedia data, the display start time and the display end time of each still image are managed in the plurality of still image file transmission management table, and only during the display period. The transmission of the still image moving image data file is switched so as to be transmitted, and when it is necessary to simultaneously transmit a plurality of files, the bit rate given for the still image is divided equally and transmitted. Things,
The information of the display start time and the display end time of the still image data is acquired from the multimedia content information reference unit, and transmission is started from the display start time, and the transmission is ended at the display end time.
[0019]
As a result, multimedia data can be transmitted in accordance with the multimedia content information, and a random access function with a short waiting time during a jump can be provided.
Further, since it is not necessary to perform a process of requesting data from the receiving terminal to the server, it is possible to provide a configuration capable of reducing a delay in data reception.
[0020]
FurtherEven when there is a still image that overlaps in the display period, the data can be equally divided and transmitted between the still image data for simultaneously displaying the bit rate.
[0021]
No.5To,
A sending request receiving unit for receiving a sending request for data to be played from a playing device on a network, and a stream generating unit for reading file data corresponding to each medium constituting multimedia from the local file system to generate a stream. Refers to a data sending unit having a sending means corresponding to the type of media constituting the multimedia data, sending the stream to the network, and scenario information on the data including the storage location and attributes of the data to be sent. A multimedia content information reference unit, and a transmission bit rate is allocated to each medium configuring multimedia by combining the multimedia content information from the multimedia content information reference unit with a network band that can be used by the playback device; Includes a Shimesuru multimedia data transmission scheduler, the,
The media data transmission unit maintains a bit rate specified by the multimedia data transmission scheduler, performs transmission of each specified data,
For still image moving image format data transmission sectionMultipleStill imageSend filemanagementtableWhen there are a plurality of still images in the data constituting the multimedia data, priorities are assigned to the still image data according to the size and data size of the display area described in the multimedia data configuration file. When it is necessary to read multiple files simultaneously at the start and end of transmission of a transmission file based on the information of display start time and display end time, the bit rate given for a still image is not equal according to the priority order. It is equipped with a means to read in minutes.
As a result, even when still images having different characteristics, importance levels, and read states are simultaneously displayed, optimal transmission can be performed in which the degree of influence on multimedia information reproduction is reflected.
[0022]
No.6Second4,5In addition to the configuration
The playback unit of the receiving terminal includes a playback control interface capable of controlling playback of pause / stop release / enlarged display of an object. When a playback control event occurs, the multimedia data sending unit of the data sending server sends an event occurrence notification. Equipped with an event notification unit that sends out messages,The content information reference sectionSendingBe targeteddataScenario showing playback controlAffectionFurther information, Each transmission data depends on time axisOr notToJudgmentThen, the multimedia data transmission scheduler redistributes the bit rate to each media file data transmission unit according to the change of the reproduction situation, and preferentially transmits the time-dependent media file data during normal reproduction, A means is provided for sending the file data of the media which does not depend on the time axis at the time of temporary stop, and sending the file data of the target object at the time of the enlarged display of the object.
[0023]
Thereby, the multimedia data transmission scheduler redistributes the bit rate to each media data transmission unit according to the change of the reproduction status, and during the normal reproduction, preferentially transmits the media data of the media depending on the time axis and temporarily When stopped, the file data of the media that does not depend on the time axis is sent preferentially, and when the object is enlarged and displayed, the file data of the target object can be sent preferentially. It is possible.
[0030]
BEST MODE FOR CARRYING OUT THE INVENTION
Hereinafter, embodiments of the present invention will be described with reference to FIGS. The present invention is not limited to these embodiments at all, and can be implemented in various modes without departing from the gist thereof.
[0031]
(Embodiment 1)
FIG. 1 shows a system configuration diagram according to a first embodiment of the present invention. In FIG. 1, a multimedia data server 101 and a receiving terminal 102 are connected to a packet network 103, and provide and receive multimedia data, respectively. .
The multimedia data server 101 includes a network interface unit 104a and a file system 107 that stores multimedia data and can be shared via a network.
[0032]
In addition, the receiving terminal 102 is a general network interface unit 104b similar to the multimedia data server 101,
Multimedia data description information describing the storage location of each media data included in the multimedia content to be played back, encoded parameters, data size, etc., and the playback start / end time / layout position of each media related to playback control Multimedia content information reference unit 108 that refers to the described playback scenario information,
A multimedia data reading scheduler 109 for determining the distribution of the transmission bit rate for each media data;
Moving image data reading unit 110a, audio data reading unit 110b, still image moving image format data reading unit 110c as remote file data reading units for reading each media data from the remote file system,
Receiving buffers 111a to 111c for temporarily storing outputs from the remote file data reading units 110a to 110c,
A moving image decoder 112a, an audio decoder 112b, and a still image moving image decoder 112c as a media decoder for decoding each media data, and a multimedia data output unit 113 for synthesizing the decoded data to reconstruct and output as multimedia. A multimedia data reproducing unit 114 is provided.
[0033]
The operation of performing streaming playback at a fixed bit rate for the multimedia data playback device configured as described above will be described below.
When the receiving terminal 102 attempts to receive multimedia data from the multimedia data server 101 connected to the packet network 103, the receiving terminal 102 refers to the multimedia content information received by the multimedia content information Pass information.
[0034]
Here, an example of the multimedia content information is shown in FIG.
The multimedia content information reference unit holds these information as a table, and instructs the read scheduler 109 to store information on the storage locations of the respective constituent media, the reproduction start time and the reproduction end time of each medium, and the encoded data rates of audio and video data. give.
[0035]
The reading scheduler 109 has information on the network bandwidth that can be used by the playback device, and schedules at what bit rate each of the media data constituting the multimedia content should be read. Data that depends on the time axis, such as audio and video data, must be read at a rate that adds the overhead to the data rate information at the time of encoding as a minimum, and the data itself does not depend on the time axis, such as still images. Gives the remaining bit rate. This allocation rate may change dynamically as the playback scenario progresses.
[0036]
From the scheduling result of the reading scheduler 109, information on the storage location of the multimedia content, information on the allocated current bit rate, playback start time, and playback end time are passed to each remote file reading unit 110. Each of the remote file reading units 110a to 110c is realized by a thread or the like, and reads a file specified by an independent operation at a specified rate. The still image moving image format data reading unit 110c starts reading from the display start time and ends the reading at the display end time.
[0037]
The data read at each fixed rate is temporarily stored in the intermediate buffers 111a to 111c, decoded by the media decoders 112a to 112c, reconstructed by the multimedia data output unit 113, and output.
[0038]
By such an operation, a fixed network bandwidth is shared by each media data, and reading is performed at a fixed rate, and a multimedia content information is streamed by initiating a read start and end operation according to a display time of the multimedia content information. Playback is possible.
[0039]
For random access, the read pointer is moved by the specified time for the time axis dependent data, and only the non-time axis dependent data that is to be displayed in the scenario is read from the file read pointer at that time. This can be easily realized by a process of reading continuously. This eliminates the need for a waiting time for reading necessary data at the time of random access, and allows a jump operation to be started at high speed.
[0040]
A display method of reading data in a still image format in order of resolution and gradually increasing the resolution is also frequently used in network applications, but a still image is encoded in a moving image format as in the present embodiment. Since the high-frequency component is already included in the first frame from the first frame, the detailed portion can be sufficiently identified immediately after the start of the display. In addition, with a configuration in which a still image is encoded and displayed in a moving image format, a multimedia screen of a moving image and a still image can be configured using only one moving image decoder without mounting a decoder for a still image. The ability to reduce the number of decoders to be mounted has a great advantage in applications such as portable terminals.
[0041]
As described above, in the present embodiment,
As in the related art, a server that receives a data request collectively multiplexes and sends time-dependent / independent data, and a terminal that receives the data separates the data for each data type and then decodes and combines the data. Not that
The transmission rate for each medium is adjusted (108, 109), and data without time axis information is processed in the same manner as data with time axis information. That is, still image data is encoded into a moving image format (110c). ), And read at a fixed rate allocated to each piece of media data constituting the multimedia data (110a to 110c).
[0042]
As a result, multimedia data can be received and played back according to the multimedia content information. Also, since time-axis dependent / independent data is not multiplexed at a time, random access with a short waiting time at the time of jump can be achieved. The function can be provided, and the practical effect is great.
[0043]
Here, an example is shown in which the playback terminal directly reads the data stored in the server (a so-called PULL type).
The server also has a data transmission unit, and a still image moving image format data transmission unit is also provided in a configuration in which transmission is performed according to a transmission schedule according to the multimedia content information (a so-called PUSH type, that is, a reference unit 108, a scheduler 109, and The same effect can be obtained by providing the reading units 110a to 110c on the server 101 side and transmitting the read data to the terminal 102 side), and the processing of transmitting a data read request from the reproduction terminal side can be achieved. Since it becomes unnecessary, the effect of eliminating the delay of data reading can be obtained.
[0044]
(Embodiment 2)
FIG. 2 shows a system configuration diagram according to the second embodiment of the present invention. In FIG. 2, 101 to 114 have the same configuration as in the first embodiment,
The still image moving image format data reading unit 110c includes a plurality of still image file reading timing management tables 201.
[0045]
The operation of performing parallel reading of a plurality of still images according to a scenario at a fixed rate for the multimedia data reproducing apparatus configured as described above will be described below using an example of the multiple still image file reading timing management table in FIG. Will be explained.
[0046]
The movement from the time when the multimedia data reading scheduler 109 of the receiving terminal 102 specifies the distribution rate to each medium and the time when the transmission rate is specified to each remote file data reading unit is the same as in the first embodiment. From the scheduling result of the reading scheduler, information on the storage location of the multimedia content and information on the allocated current bit rate are transmitted to each remote file data reading unit 110. Each remote file reading unit is realized by a thread or the like, and reads a file specified by an independent operation at a specified rate.
[0047]
The still image moving image format data reading unit 110c can read a plurality of still images in parallel as a file to be read. As shown in FIG. 3, the multiple still image file reading timing management table 201 is a method for managing the display start time and end time of the still image in the multimedia data. Is shown. In addition, it manages how much of the file to be read has been read, and files that have been read to the end are excluded from the read target. By referring to this table, it is determined how many files should be read in parallel at each time. In accordance with this scheduling, the still image remote file reading unit divides the designated rate into a plurality of still image files to be read at that time, and reads them in parallel. The subsequent operation is the same as that of the first embodiment.
[0048]
As described above, in the present embodiment,
By providing the still image data reading unit (110c) with the multiple still image reading timing management table (201),
Even for multimedia contents including simultaneous playback of multiple still images, the still image bit rate is equally divided among the still images that are simultaneously displayed, and efficient scheduling with respect to the display time is realized. (That is, each media data shares a fixed network bandwidth and performs streaming reproduction at a fixed rate, and still image data with overlapping display times can be read and reproduced in parallel).
[0049]
For random access, the time-axis-dependent data moves the read pointer by the designated time, and the non-time-axis-dependent data moves the time pointer on the multiple still image file read timing management table 201. It can be easily realized by the process of continuously reading only those that are to be read from the file read pointer at that time and removing those that have been read from the read target,
As a result, multimedia contents including simultaneous reproduction of a plurality of still images can be reproduced, and a configuration suitable for random access can be realized.
[0050]
Here, an example is shown in which the playback terminal directly reads data stored in the server (a so-called PULL type). However, the server has a data transmission unit as in the case of the last note described in the first embodiment. The same effect can be obtained by providing a plurality of still image data transmission timing management tables (so-called PUSH type) even in a configuration in which transmission is performed according to a transmission schedule according to multimedia content information.
Further, since the process of sending a data read request from the reproduction terminal side is not required, there is obtained an effect that data reading delay is eliminated.
[0051]
(Embodiment 3)
FIG. 4 shows a system configuration diagram according to the third embodiment of the present invention. In FIG. 4, 101 to 114 have the same configuration as the second embodiment,
A still image data configuration information table 401 is provided in the still image moving image format data reading unit instead of 201.
[0052]
In the multimedia data reproducing apparatus configured as described above, an operation of prioritizing a plurality of still images and performing parallel reading will be described with reference to an example of a still image data configuration information table in FIG.
[0053]
The movement from the time when the multimedia data reading scheduler 109 of the receiving terminal 102 specifies the distribution rate to each medium and the time when the transmission rate is specified to each remote file data reading unit is the same as in the first embodiment.
[0054]
From the scheduling result of the reading scheduler, information on the storage location of the multimedia content and information on the allocated current bit rate are transmitted to each remote file reading unit 110. Each remote file reading unit is realized by a thread or the like, and reads a file specified by an independent operation at a specified rate.
[0055]
The still image moving image format data reading unit 110c can read a plurality of still images in parallel as a file to be read. The still image data configuration information table 401 manages the display start time and end time of the still image in the multimedia data as shown in FIG. 5, and which still image data is to be read in the time between events. Is shown. In addition, it manages how much of the file to be read has been read, and files that have been read to the end are excluded from the read target. By referring to this table, it is determined which file should be read in parallel at each time.
[0056]
Further, in the still image data configuration information table 401, information on the size and data size of the display area of each still image data is developed. When a plurality of still images are displayed at the same time, the size of the display area that almost matches the multimedia data playback area is assumed to be the "background", and is considered as a still image compared to a still image containing information such as a map. May have a low display priority. In addition, a display area having a relatively large display area, not the background, is considered to have a high occupancy in the display area and a high display priority.
[0057]
In accordance with the priority order from the display area, the still image remote file reading unit unequally divides the designated rate into a plurality of still image files to be read at that time and reads them in parallel. The way of division is, for example, to give a rate of 50% between adjacent priorities. If two files are read simultaneously, 3/5 and 2/5, if three files have three levels of priority, They will be distributed at the ratios of 4/19, 6/19 and 9/19. The subsequent operation is the same as that of the second embodiment.
[0058]
As described above, in the present embodiment,
If there are multiple still images in the data that make up the multimedia data, the still image dataInformation tableIt is necessary to prioritize still image data according to the size and data size of the display area described in (401), and to read a plurality of files at the same time when switching the read files based on the timing of the display start time and the display end time. In this case, there is provided a means for reading the bit rate given for the still image by unequally dividing it according to the priority order.
[0059]
As a result, even when still images having different characteristics, importance levels, and read states are displayed at the same time, optimal reading that reflects the magnitude of the effect on multimedia information reproduction is possible (that is, each media data has a fixed network bandwidth. Sharing streaming playback at a fixed rate, prioritizing still image data to reflect the size of the effect on display, distributing the optimal rate, and reading and playing back in parallel) Regarding random access, processing can be performed in the same manner as in the second embodiment even when priorities are assigned between still images.
[0060]
Here, an example in which the playback terminal directly reads data stored in the server (a so-called PULL type) has been described, but a data transmission unit is transmitted to the server in the same manner as in the last-mentioned first and second embodiments. A similar effect can be obtained by providing a still image data configuration information table (so-called PUSH type) even in a configuration in which transmission is performed according to a transmission schedule according to multimedia content information.
Further, since the process of sending a data read request from the reproduction terminal side is not required, there is obtained an effect that data reading delay is eliminated.
[0061]
(Embodiment 4)
FIG. 6 shows a system configuration diagram according to the fourth embodiment of the present invention. In FIG. 6, 101 to 114 have the same configuration as in the first embodiment. In addition, the multimedia data reproducing unit 114 in the receiving terminal 102 performs reproduction control such as pause / stop release / enlarged display of an object. It is provided with a playback control interface 601 that can be used and an event notification unit 602 that sends an event occurrence notification message to the multimedia data reading scheduler 109 when a playback control event occurs.
[0062]
The operation of the multimedia data reproducing apparatus configured as described above, in which the bit rate of each remote file data reading unit is redistributed and the parallel reading schedule is switched according to a change in the reproduction state, will be described below. The details of the exchange of an example of the multimedia data configuration information table in FIG. 7 are omitted).
The multimedia data reading scheduler 109 of the receiving terminal 102 determines an initial rate distribution to each medium, specifies a transmission rate to each remote file data reading unit, and keeps each remote file data reading unit at the specified rate. The operations of reading data from the designated file, storing the data in the corresponding intermediate buffers, decoding the data, and reproducing and outputting the data are the same as those in the first embodiment.
[0063]
Here, when a stop instruction is input to the playback control interface 601, the playback unit 114 stops the decoding of the audio and video that is the time axis-dependent data, and the event notification unit 602 outputs a “stop” message. To the multimedia data reading scheduler 109.
[0064]
The multimedia data reading scheduler 109 can determine whether or not the reproduction of the data constituting the multimedia content depends on the time axis by the multimedia content information reference unit 108. In the stop state, the reading of the audio and video of the time axis-dependent data is paused, and the still image data of the data that does not depend on the time axis or the difference data for improving the image quality of the scene stopped by the video is continuously read. The transmission path can be used effectively.
[0065]
According to such a policy, the multimedia data reading scheduler 109 determines the bit rate distribution at the time of stop, and instructs each media data reading unit. Thus, while the reading of the time-axis dependent data is stopped at the time of stopping, the transmission rate of other data can be increased, and the image quality of the reproduction state at the time of stopping can be improved.
[0066]
Further, when the instruction of “stop release” is input, the multimedia data reading scheduler 109 attempts to return to the original bit rate distribution in the same manner as in the case of “stop”. Since there is a high possibility that reading of a still image or the like has been completed, the bit rate distribution before the stop is not always the same.
[0067]
When an instruction for enlarging the visual object is input, it is necessary to send an ID indicating the visual object to be enlarged along with the event notification of “enlarged display”. The multimedia data reading scheduler keeps reading by validating only the reading rate corresponding to the object to be enlarged or increasing the priority compared to other rates.
[0068]
As described above, in the present embodiment,
It is provided with means (602 → 109 and 110a-c) for receiving a playback control signal from the multimedia data playback unit, redistributing the bit rate, and implementing reading according to the playback status.
[0069]
This allows the multimedia data reading scheduler to redistribute the bit rate to each remote file data reading unit according to the change in the playback status, and to read the file data of the media depending on the time axis preferentially during normal playback, temporarily At the time of stop, the file data of the media that does not depend on the time axis is read preferentially (that is, while the audio and video media that depend on the time axis are paused, the still images that are media that does not depend on the time axis are When the object is enlarged and displayed, the file data of the target object can be preferentially read, and an optimum reading rate can be assigned according to the reproduction state.
[0070]
Here, an example is shown in which the playback terminal directly reads data stored in the server (a so-called PULL type), but the data transmission unit is sent to the server in the same manner as in the last-mentioned first to third embodiments. In a configuration in which transmission is performed according to a transmission schedule according to multimedia content information, an event notification unit that transmits an event generation notification message to a multimedia data reading scheduler when a reproduction control event occurs is provided. The same effect can be obtained by
Further, since the process of sending a data read request from the reproduction terminal side is not required, there is obtained an effect that data reading delay is eliminated.
[0071]
(Embodiment 5)
FIG. 8 shows a system configuration diagram according to the fifth embodiment of the present invention. In FIG. 8, reference numerals 101 to 109 and 112 to 114 have the same configuration as in the first embodiment, but only one remote file data reading unit 110 is provided for a plurality of files constituting multimedia data. Further, a required slot number management table 801 for each media data is provided in the remote file data reading unit 110.
[0072]
With respect to the multimedia data reproducing apparatus configured as described above, a plurality of files are read in parallel by one processing module, the read unit is set to one slot, and at a fixed bit rate based on the required slot number management table. The reading operation will be described.
[0073]
The multimedia data reading scheduler 109 of the receiving terminal 102 determines the initial rate distribution to each medium, specifies the transmission rate for each medium to the remote file data reading unit 110, and specifies the remote file data reading unit 110. The operation of reading data from the specified file in accordance with the rate, storing the data in the intermediate buffer 111 corresponding to each material, decoding, and reproducing and outputting the data is the same as in the first embodiment.
[0074]
Here, the remote file data reading unit reads all the media data included in the multimedia data, and the media data also includes moving image format data for a still image. Here, a necessary slot number management table is used as a means for rate distribution control. The operation of the rate distribution control will be described with reference to the flowchart of FIG. 9 and numerical examples managed in the required slot number management table of FIG.
[0075]
First, the minimum transfer block size determined depending on the remote file system is set as a slot unit used in the rate distribution control in this embodiment. Then, the total bit rate of the network is divided into slot units and assigned to reading of each file data for each slot. Therefore, first, as shown by 901 in FIG. 9, the required number of slots per unit time is calculated from the bit rates allocated to the respective media data, and the initial value of the cumulative required number of slots in the required slot number management table is calculated. I do.
[0076]
For example, since the minimum transfer block size is 4096 bytes in the NetBIOS shared file mechanism, the read unit is set to this fixed number of bytes so that reading is always performed in this unit. If this is sent over a network bandwidth of 110 kbps,
110000 (bit) / 4096 (Byte) = 3.357
Therefore, if the data is read at a rate of about 3.36 slots per second, the rate can be maintained. Assuming that the video data is 60 kbps, the audio is 40 kbps, and the others are 10 kbps (that is, the distribution is 6: 4: 1), the required number of slots per unit time is 1.831, 1.221, and 0.305, respectively. . As shown in FIG. 10A, in the required slot number management table, the required number of slots per unit time is first substituted for the cumulative required number of slots of each data as an initial value.
[0077]
Next, the rate distribution reading operation is started. In this process, as shown at 902, the number of accumulated necessary slots for each piece of media data is compared, and the largest one is read only for each slot as a reading target. In FIG. 10A, since the cumulative number of required slots of video data is the largest, video data is to be read and 4096 bytes are read in slot units.
[0078]
Next, as indicated by 903, the number of allocated slots is subtracted from the cumulative required number of slots of the media data that has been actually read. As shown in FIG. 10B, 0.831 slots obtained by subtracting one slot from the initial value of 1.831 slots of video data are newly assigned. The read operation returns to 902 in FIG. 9 and reads the next largest value of the required number of slots of media data. Here, since the data is audio data, the audio data is to be read, and is read only in slot units. When reading is completed, 1 is subtracted from the cumulative number of slots required for audio data, and 0.221 is substituted. Returning to step 902 again, the number of accumulated necessary slots is compared, and the largest one at that point is to be read and read. The cumulative number of required slots may be a negative value. This operation is continued until the unit time elapses.
[0079]
When the next unit time comes, as shown at 904, an operation of adding a new required number of slots per unit time to the accumulated required number of slots is performed. Here, the required number of slots per unit time is added to the cumulative required number of slots of each media data every one second. Here, as shown in FIG. 10 (C), the initial value of 1.831, 1.221, and 0.305 is added to each of the cumulative number of required slots to obtain 2.662, 1.442, and 0.610. . The read operation is continued from 902 in FIG. 9 with the new required number of slots. In this manner, a fixed rate is distributed by the process of allocating the number of slots to media having a high necessity, and a parallel read operation of a plurality of files with a small delay with respect to a necessary read timing can be performed.
[0080]
As described above, in the present embodiment,
In order to reduce costs, etc., even under the condition that the means for reading remote files is one,
By performing a process of allocating slots by comparing the cumulative number of required slots using the required slot number management table 801, even when the bit rate is finely divided, multi-file parallel reading optimized for a file system is realized. be able to.
[0081]
Here, an example is shown in which the playback terminal directly reads data stored in the server (a so-called PULL type), but a data transmission unit is sent to the server in the same manner as in the last-mentioned first to fourth embodiments. In a configuration in which transmission is performed according to a transmission schedule in accordance with multimedia content information, the same effect can be obtained by providing the data transmission unit with a necessary slot number management table for each media data (so-called PUSH type). In addition, there is no need to send a data read request from the reproducing terminal, so that there is no delay in reading data.
[0082]
(Embodiment 6)
FIG. 11 shows a system configuration diagram according to the sixth embodiment of the present invention. In FIG. 11, reference numerals 101 to 114 denote the same components as those in the fifth embodiment. In addition, in the media data reproduction, items indicating whether or not there is a time axis dependency and slots required for media data having a time axis dependency are provided. The configuration includes an accumulated required slot number management table 1101 provided with items indicating the upper limit of the number.
[0083]
In the multimedia data reproducing apparatus configured as described above, the parallel reading of a plurality of files will be performed in one processing unit based on the required slot number management table. The operation of reading the time-axis dependent medium with priority regardless of the remaining amount will be described.
[0084]
The multimedia data reading scheduler 109 of the receiving terminal 102 determines an initial rate distribution to each medium, specifies the transmission rate for each medium to the remote file data reading unit 110, and the remote file data reading unit determines the cumulative number of required slots. The operation of reading data from a designated file at a normal rate based on the management table 1101, accumulating the data in a corresponding intermediate buffer, decoding the data, and reproducing and outputting the data is the same as in the fifth embodiment.
[0085]
When the reading speed does not fall below a predetermined bit rate per unit time and falls within a predetermined fluctuation range, the configuration of the fifth embodiment performs the rate distribution parallel reading. Here, when the fluctuation in the reading speed occurs, the time-axis-dependent media is stored by using the accumulated number-of-slots management table including the item indicating the time-axis dependence and the item indicating the upper limit of the number of accumulated required slots in that case. The priority reading operation will be described with reference to the flowchart of FIG. 12 and a numerical example managed in the required slot number management table of FIG.
[0086]
First, the required number of slots per unit time is calculated from the bit rate allocated to each piece of media data, and the process of setting the initial value of the cumulative required number of slots in the required number of slots management table is the same as in the fifth embodiment. It is. In this embodiment, next, as shown by reference numeral 1202 in FIG. 12, items indicating the reproduction time axis dependency of each media data are checked, and video and audio data have a dependency, and other data have a dependency. Substitute a value without dependencies.
[0087]
For media data that is considered to be dependent on the playback time axis, the amount of buffering and the allowable amount of data reading delay determined by the data reading control mode of the decoder are converted into slot units as shown at 1203 in FIG. Then, a value is written to the required slot number management table as the upper limit value of the cumulative required slot number. FIG. 13A shows the required slot number management table in this state.
[0088]
Next, the rate distribution read operation is started. First, as shown in 1204 of FIG. 12, the upper limit value of the cumulative number of required slots for each piece of media data is compared with the current cumulative required number of slots. Check if not. Here, if the media data exceeding the upper limit is one media data, the media data is to be read, and if there is more than one, the cumulative number of necessary slots is further compared among them, and the largest one is read, and only the slot unit is read. . If the reading speed decreases, the frequency of subtraction from the cumulative number of required slots decreases, but the addition of the required number of slots per unit time does not stop after the unit time has elapsed, so the total number of required slots Will increase.
[0089]
Here, in FIG. 13B, the still image data has the largest cumulative required slot number, but the still image data is data whose reproduction does not depend on the time axis, and the upper limit of the cumulative required slot number is not set. . Both video data and audio data are data whose reproduction depends on the time axis, and the number of accumulated necessary slots exceeds the upper limit value. Here, video data having a larger cumulative number of required slots is to be read, and 4096 bytes are read in slot units. The operation of determining the data to be read from the current value and the upper limit of the cumulative number of required slots is continued until the unit time elapses.
[0090]
When the next unit time comes, an operation of newly adding the required number of slots per unit time to the accumulated required number of slots is performed as in the fifth embodiment. In this way, by taking into account the factor of whether or not the playback depends on the time axis, the transmission rate at that time is distributed by the process of allocating the number of slots to the media with high necessity, and On the other hand, a parallel reading operation of a plurality of files with a small delay is possible.
[0091]
As described above, in the present embodiment,
By using the cumulative required slot number management table 1101 provided with items indicating the reproduction time axis dependence and the cumulative required slot number upper limit value, a process of allocating slots by comparing the cumulative required slot numbers is performed, so that effective network transmission is performed. Even when the rate decreases, the bit rate is allocated with priority on the time-axis-dependent data, and multi-file parallel reading can be realized.
[0092]
Although the example in which the playback terminal directly reads the data stored in the server (a so-called PULL type) is shown here, the data transmission unit is sent to the server in the same manner as in the last-mentioned first to fifth embodiments. In a configuration in which transmission is performed according to a transmission schedule in accordance with multimedia content information, the same effect can be obtained by providing the data transmission unit with a necessary slot number management table for each media data (so-called PUSH type). In addition, there is no need to send a data read request from the reproducing terminal, so that there is no delay in reading data.
[0093]
As described above, the embodiment of the present invention has been described in which the unidirectional media data is transmitted from the multimedia data server to the receiving terminal. However, the same configuration can be applied to the bidirectional communication terminal.
[0094]
【The invention's effect】
As described above, in the present invention, first,
According to the first or fifth method,
<First Method> Multimedia data can be received and reproduced according to multimedia content information, and a random access function with a short waiting time during jump can be provided.
<Fifth Method> Multimedia data can be transmitted in accordance with multimedia content information, and a random access function with a short waiting time during a jump can be provided. Further, since it is not necessary to perform a process of requesting data from the receiving terminal to the server, it is possible to provide a configuration capable of reducing a delay in data reception.
[0095]
Second, by the second or sixth method,
<Second Method> Even when there is a still image that overlaps during the display period, it is possible to read evenly divided still image data whose bit rate is simultaneously displayed.
<Sixth Method> Even when there are still images that overlap in the display period, the still image data for which the bit rate is simultaneously displayed can be equally divided and transmitted.
[0096]
Third, by the third or seventh method,
<Third Method> Even when still images having different characteristics, importance, and reading states are simultaneously displayed, optimum reading can be performed in which the degree of influence on multimedia information reproduction is reflected.
<Seventh Method> Even when still images having different characteristics, importance levels, and read states are simultaneously displayed, optimal transmission can be performed in which the degree of influence on multimedia information reproduction is reflected.
[0097]
Fourth, by the fourth or eighth method,
<Fourth Method> The bit rate is redistributed to each remote file data reading unit according to a change in the reproduction status, and the file data of the media depending on the time axis is preferentially read during normal reproduction, and the time is read during pause. The file data of the media independent of the axis can be read preferentially, and the file data of the target object can be read preferentially when the object is enlarged and displayed, and the reading rate can be assigned according to the reproduction state.
<Eighth Method> The bit rate is redistributed to each media data transmission unit according to the change of the reproduction status, and the media data of the media depending on the time axis is transmitted preferentially during normal reproduction, and the time is transmitted during pause. Axis-independent media file data can be sent preferentially, and when the object is enlarged and displayed, file data of the target object can be sent preferentially, and a sending rate can be assigned according to the playback state. .
[0098]
Fifth, according to the ninth or eleventh method,
<Ninth Method> Even when the division of the bit rate into each media data becomes fine, it is possible to realize multi-file parallel reading optimized for a file system.
<Eleventh Method> Even when the division of the bit rate into each media data becomes fine, it is possible to realize multi-stream transmission with high transmission efficiency optimized for a network protocol.
[0099]
Sixth, by the tenth or twelfth technique,
<Tenth Method> Even when the effective rate fluctuates in an environment where a network error is likely to occur, data depending on the time axis can be preferentially acquired.
<Twelfth Method> Flexible transmission scheduling can be performed even in an environment where the network rate tends to fluctuate.
[Brief description of the drawings]
FIG. 1 is a system configuration diagram showing a multimedia data reproducing apparatus according to a first embodiment of the present invention.
FIG. 2 is a system configuration diagram showing a multimedia data reproducing apparatus according to a second embodiment of the present invention.
FIG. 3 is a diagram showing an example of a multiple still image file read timing management table according to the second embodiment of the present invention;
FIG. 4 is a system configuration diagram showing a multimedia data reproducing device according to a third embodiment of the present invention.
FIG. 5 is a configuration diagram illustrating an example of a still image data configuration information table according to a third embodiment of the present invention.
FIG. 6 is a system configuration diagram showing a multimedia data reproducing device according to a fourth embodiment of the present invention.
FIG. 7 is a configuration diagram showing an example of a multimedia data configuration information table according to a fourth embodiment of the present invention.
FIG. 8 is a system configuration diagram showing a multimedia playback device according to a fifth embodiment of the present invention.
FIG. 9 is a flowchart according to a fifth embodiment of the present invention.
FIG. 10 is a diagram showing a numerical example of a required slot number management table according to the fifth embodiment of the present invention.
FIG. 11 is a system configuration diagram showing a multimedia playback device according to a sixth embodiment of the present invention.
FIG. 12 is a flowchart according to a sixth embodiment of the present invention.
FIG. 13 is a diagram showing a required slot number management table according to the sixth embodiment of the present invention.
FIG. 14 is a system configuration diagram showing a conventional multimedia data reproducing apparatus.
[Explanation of symbols]
101 Multimedia data server
102 receiving terminal
103 Packet Network
104, 104a, 104b Network interface unit
105, 105a, 105b Data transmission unit
106, 106a, 106b Data receiving unit
107 File System
108 Multimedia content information reference section
109 Multimedia data reading scheduler
110 Remote file data reading unit
110a Video data reading unit
110b Audio data reading unit
110c Still image moving image format data reading unit
111 Receive buffer
112 Media Decoder
112a Video decoder
112b audio decoder
112c Video decoder for still images
113 Multimedia data output unit
114 Multimedia data playback unit
201 Multiple still image file reading management table
401 Still Image Data Configuration InformationInformationTable
601 Interface for playback control
602 event notificationDepartment
801 Required slot number management table
901 Required slot number calculation processing per unit time
902 Comparison processing of the number of cumulative required slots
903 Data read processing for reading
904 Addition of required number of slots per unit time
1101 Cumulative required slot number management table
1201 Required slot number calculation processing per unit time
1202 Reproduction time axis dependency inspection processing
1203 Calculation process of upper limit value of cumulative required number of slots
1204 Comparison processing of upper limit value and current value of cumulative required slot number
1401 Data transmission server
1402 Receiving terminal
1403 Packet Network
1404 Network Interface
1405 Data transmission unit
1406 Data receiving unit
1407 Stream Generation Unit
1408 Demultiplexer
1409 Receive buffer
1410 Request receiving unit
1411 Multi-client scheduler
1412 Request sending unit
1420 playback unit
1421 media decoder

Claims (6)

Reading the file data via a network from a remote file system comprises a reading means corresponding to the type of media that constitutes the multimedia data, de-code received and the remote file data reading unit, data from the remote file data reading unit a reproducing unit for displaying a multimedia content information reference section for referring to the scenario information indicating information on the multimedia data description information and the reproduction control that indicates the attribute of the data to be read target, the relative to the reading unit device allocates the transmission bit rate to each of the media network bandwidth and Te Oh Align the multimedia content information from the multimedia content information reference unit constituting the multimedia that can be used, multimedia data read instructing Includes a scheduler, a,
The remote file data reading unit maintains the bit rate specified by the multimedia data read scheduler performs read from each specified file,
One of said reading means, it reads the encoded data to video format comprising one still picture from a plurality of frames regardless of the time information of the frame, from the previous SL multimedia content information reference unit get the information of the display start time and display end time of the still image data, the display start time to start reading from, finished reading at the display end time,
Further, when there are a plurality of still images in the data constituting the multimedia data, information of display start time and display end time of each still image is managed in a plurality of still image file reading management tables, and during the display period, If you need to switch the still image file to be read only and read multiple files at the same time, adjust the bit rate given for the still image and read the still image video format compatible remote A multimedia data reproducing apparatus , comprising: file data reading means . A remote file data reading unit for reading file data from a remote file system via a network, having reading means corresponding to the type of media constituting the multimedia data, and a reproduction for receiving, decoding and displaying data from the reading unit A multimedia content information reference unit for referring to multimedia data description information indicating attributes of data to be read and scenario information indicating information relating to reproduction control; and a network which can be used by the present apparatus for the reading unit. A multimedia that allocates and indicates a transmission bit rate to each of the media constituting the multimedia by combining the bandwidth and the multimedia content information from the multimedia content information reference unit.
Media data reading scheduler,
The remote file data reading unit maintains a bit rate specified by the multimedia data reading scheduler, performs reading from each specified file,
One of the reading means reads data obtained by encoding a single still image into a moving image format including a plurality of frames without depending on time information of a frame, and reads still image data from the multimedia content information reference unit. Obtain information of display start time and display end time, start reading from the display start time, end reading at the display end time,
Further, when there are a plurality of still images in the data constituting the multimedia data, prioritize still image data according to the size and data size of the display area described in the still image data configuration information table , If multiple files need to be read at the same time when switching the read file based on the information of display start time and display end time, the video for still images that adjusts the read bit rate given for the still images according to the priority order format corresponding remote file data characteristics and to the luma Ruchi media data reproducing apparatus that includes a reading unit. Equipped with an event notification unit that sends an event occurrence notification message to the multimedia data reading scheduler when a playback control event occurs. The playback unit includes stop / pause, stop / pause release, and enlarged / reduced object display. Having a playback control interface capable of controlling playback of
The multimedia content information reference unit further includes scenario information indicating information on reproduction control of data to be read , and determines whether each read data depends on a time axis,
The multimedia data read scheduler redistributes the bit rate to the reading means of each remote file data according to the change of the reproduction status, and preferentially reads the file data of the media depending on the time axis during normal reproduction, 3. The multimedia according to claim 1, wherein the file data of the media independent of the time axis is preferentially read during the pause, and the file data of the target object is preferentially read during the enlarged display of the object. Data playback device. A sending request receiving unit for receiving a sending request for data to be played from a playing device on a network, and a stream generating unit for reading file data corresponding to each medium constituting multimedia from the local file system to generate a stream. , and sends the stream to a network, and Lud over data delivery unit having a delivery means corresponding to the type of media that constitutes the multimedia data, the multimedia data description information indicating the attribute of the data to be delivered subject multimedia content information reference section referring, allocating transmission bit rate to each media constituting the multimedia Te Oh Align the multimedia content information from the reproduction device and network bandwidth that can be used the multimedia content information reference unit Maru to instruct And a media data transmission scheduler,
The media data sending unit maintains the bit rate designated from the multimedia data transmission scheduler, performs transmission of data specified respectively,
One of the transmitting units is configured to manage timings of a display start time and a display end time, which are display periods of each still image, when there are a plurality of still images in the data constituting the multimedia data. Using the still image file transmission management table, the transmission of the still image moving image data file is switched so as to be transmitted only during the display period, and when it is necessary to transmit a plurality of files at the same time, the still image A multimedia data transmission device, characterized in that it is a still image moving image format compatible data transmitting means for transmitting data obtained by encoding a still image into a moving image format composed of a plurality of frames by adjusting the bit rate given to the image data. A transmission request receiving unit that receives a transmission request for data to be reproduced from a reproduction device on a network; and a stream generation unit that reads file data corresponding to each medium constituting multimedia from a local file system to generate a stream.
A data transmission unit having a transmission unit corresponding to the type of media constituting the multimedia data for transmitting the stream to the network, and scenario information on the data including a storage location and attributes of the data to be transmitted. A transmission bit rate is allocated to each media constituting multimedia by combining a multimedia content information reference unit for referring to the multimedia content information from the multimedia content information reference unit with a network band usable by the playback device. And a multimedia data transmission scheduler for instructing.
The media data transmission unit maintains a bit rate specified by the multimedia data transmission scheduler, performs transmission of each specified data,
One of the transmitting means is provided, when there are a plurality of still images in the data constituting the multimedia data, a timing of a display start time and a display end time as a display period of each still image, and a timing of a display area. using a plurality still image files sent management table for managing the size and the data size, with a priority to still image data, when it is necessary to read multiple files in transmission start or end Ryoji point of feeding out files to, have rows delivery adjustment corresponding to a given bit rate for the still image priority ranking, a still image and wherein the sending the encoded data to video format consisting of a plurality of frames to the luma Ruchi media data Sending device. The multimedia content information reference unit further includes scenario information indicating reproduction control of data to be transmitted, and determines whether each transmitted data depends on a time axis ,
The multimedia data transmission scheduler redistributes the bit rate to each media file data transmission unit in accordance with a change in the reproduction status of the reproduction device, and preferentially transmits time-dependent media file data during normal reproduction. 6. The method according to claim 4 , wherein the file data of the media which does not depend on the time axis is preferentially transmitted during the temporary stop, and the file data of the target object is preferentially transmitted during the enlarged display of the object. Multimedia data transmission device.

Images