JP2005176094A - Data processor, data processing method, program and storage medium - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To appropriately divide scene description data and object description data corresponding to a type and capacity of a reception side terminal, and to a communication channel, when distributing encoded multi-media data constituted of a plurality of objects such as moving images, still images, text and CG. <P>SOLUTION: An equipment information / network analysis circuit 104 analyzes the type and throughput of the reception terminal and the condition of the communication channel, and a scene / object conversion circuit 101 judges whether to divide and transmit the scene description data 102 and the object description data 103 corresponding to analyzed results. In the case that division is required, the scene description data 102 and the object description data 103 are appropriately divided. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

  The present invention relates to a data processing apparatus, a data processing method, a program, and a storage medium that distribute multimedia data to receiving terminals connected via a network.

  MPEG-1 and MPEG-2 are known as international standards for encoding standards for compressing and encoding moving images and audio, multiplexing, transmitting or accumulating them, and demultiplexing and decoding them.

  On the other hand, ISO / IEC 14496 part 1 (MPEG-4 Systems) standardizes a method to multiplex and synchronize an encoded bit stream of multimedia data including multiple objects such as still images, moving images, audio, text, and CG. Has been.

  The MPEG-4 data stream as described above is different from conventional multimedia streams, and in addition to still images, moving images and audio data, each object such as text and CG is spatially and temporally. Information for placement includes BIFS (Binary Format for Scenes), which is an extension of VRML (Virtual Reality Modeling Language) to handle natural video and audio. Here, BIFS is information (scene description information) that describes an MPEG-4 scene in binary.

  Individual objects such as still images, moving images, and audio that make up such a multimedia stream are individually encoded and transmitted individually, so that the decoding side also individually decodes the above-described objects. The scenes are spatially arranged in accordance with the scene description information, and the time axis of each data is synchronized with the time axis inside the playback device, and the scene is synthesized and played back.

  In addition to the VRML and BIFS described above, the scene structure can be described using HTML (HyperText Markup Language), XML (eXtensible Markup Language), SMIL (Synchronized Multimedia Integration Language), XMT. (Extensible MPEG-4 Textual Format).

  When transmitting such a bit stream of multimedia data, it is required to create and transmit data having an optimal amount of information according to the capability and type of the receiving terminal and the status of the communication line. In other words, if the receiving terminal is a portable information terminal such as a mobile phone or PDA (Personal Data Assistant), or if the communication line is congested, a high compression rate is encoded on the transmitting side in advance. It is necessary to perform encoding by compressing in a format, reducing the image size, or reducing the transmission rate or frame rate.

Depending on the capabilities of the receiving terminal and the status of the communication line, information such as video and audio rates, time / space scalable switching, image size conversion, and error resilience can be controlled. Several methods for optimizing the amount, encoding, and transmitting have been proposed. For example, Patent Document 1 proposes a method for optimizing and transmitting scene description data other than moving images.
JP 2002-112149 A

  However, in the method proposed in Patent Document 1, (1) which object is prioritized when scene description data is divided and transmitted is not mentioned. (2) Object description data is divided. There is a problem that (3) the content creator cannot specify the priority order of object display.

  The present invention has been made in view of the above problems, and is received when distributing encoded multimedia data composed of a plurality of media objects such as moving images, sounds, still images, texts, and CGs. It is intended to enable display intended by the content creator according to the ability of the side.

A data processing apparatus according to an aspect of the present invention has the following configuration. That is,
Description data having object description data including a plurality of media objects, scene description data describing a temporal and spatial relationship of the plurality of media objects, and description of information necessary for associating and decoding the scene description with the media object; A data processing device for delivering multimedia data including
Determination means for determining whether or not description data needs to be divided based on external capabilities related to distribution of the multimedia data and the scene description data and object description data;
A division unit that divides each of the object description data and the scene description data in units of a part related to the media object when the determination unit determines that the division is necessary;
Each of the description data divided by the dividing means and a plurality of media objects are encoded, multiplexed to generate multimedia data and distributed to the receiving terminal.

In addition, a data processing device according to another aspect of the present invention provides:
A data processing apparatus for delivering multimedia data including a plurality of media objects and description data describing information related to reproduction of each media object to an external device,
Determining means for determining the necessity of dividing the description data based on the external capability related to the delivery of the multimedia data and the description data;
When the determination unit determines that division is necessary, based on the priority set for the media object to which the description data relates, the description data portion related to the low-priority media object is separated. Dividing means for dividing the description data;
Each of the description data divided by the dividing means and a plurality of media objects are encoded, multiplexed to generate multimedia data and distributed to the receiving terminal.

According to the present invention, when the encoded multimedia data composed of a plurality of media objects is distributed, the external capability related to the distribution (the capability of the distribution destination terminal and / or the transmission capacity of the network, etc.) Accordingly, the description data describing the information related to the reproduction can be divided and transmitted according to the priority of the media object. For this reason, it is possible to appropriately distribute the load and time required for the processing on the receiving side device for playing back a plurality of media objects. In addition, since playback is performed according to the priority of the media object, it is possible to display in accordance with the intention of the content creator.
Further, according to the present invention, since both the scene description data and the object description data are divided according to the external capability, it is possible to more appropriately realize the load distribution when reproducing the media object.

  Hereinafter, a data processing apparatus according to the present invention will be described in detail according to a preferred embodiment applied to an apparatus for distributing multimedia data with reference to the accompanying drawings.

  FIG. 1 shows a basic configuration of a multimedia data distribution apparatus 100 (hereinafter simply referred to as a distribution apparatus) and a multimedia data reception apparatus 110 (hereinafter simply referred to as a reception apparatus) in the present embodiment, and data between each circuit. It is a figure which shows a flow. A distribution apparatus 100 shown in FIG. 1 includes a scene / object conversion circuit 101, a device information / network analysis circuit 104, a scene description data encoding circuit 105, an object description data encoding circuit 106, a media bitstream storage apparatus 107, a multiplexing circuit. 108.

  The receiving apparatus 110 includes a demultiplexing circuit 111, a scene description data decoding circuit 112, an object description data decoding circuit 113, a media decoding circuit 114, a scene synthesis circuit 115, and an output device 116.

  In the distribution apparatus 100, the scene description data 102 is a screen or time structure presented to the viewer, and the above-described BIFS is adopted as a scene description language in the MPEG-4 system part. The object description data 103 is information necessary for decoding such as association between the scene description data 102 and each media object constituting the scene, an encoding method, and a packet configuration. The scene description data 102 and the object description data 103 are generated by a scene / object editing circuit (not shown), or scene description data and object description data stored in a predetermined storage device are read.

  The scene description data 102 and the object description data 103 are encoded by the scene description data encoding circuit 105 and the object description data encoding circuit 106, respectively, and input to the multiplexing circuit 108. The multiplexing circuit 108 multiplexes the input scene description data, object description data, and encoded bit stream, and distributes them to the transmission apparatus 109 as a bit stream to the receiving apparatus 110. An encoded bit stream is prepared in advance in the media bit stream storage device 107, and the multiplexing circuit 108 selects and multiplexes the encoded bit stream of the required media object from the media bit stream storage device 107. To do.

  The encoded bit stream is a still image that has been encoded with high efficiency (compression) using, for example, the well-known JPEG format, such as a high-efficiency code with the well-known MPEG-2, MPEG-4, or H-263 format. Video data, for example, voice data subjected to high-efficiency encoding such as well-known CELP (Code Excited Linear Prediction) encoding or transform domain weighted interleaved vector quantization (TWINVQ) encoding.

  In FIG. 1, reference numeral 109 denotes a transmission path represented by various networks. In this embodiment, 109 is a network for distributing processed and encoded multimedia data.

  A process in which the distribution apparatus 100 according to the present embodiment distributes multimedia data to the reception apparatus 110 will be described with reference to the flowchart of FIG.

  The distribution apparatus 100 according to the present embodiment receives a multimedia data distribution request sent from the reception apparatus 110 via the network, and receives information (hereinafter, device information) regarding the type and processing capability of the reception apparatus. The device information / network analysis circuit 104 acquires the type and processing capability of the receiving device from the received device information and analyzes the network line status (step S201). As a method for analyzing the line status of the network, for example, the transmission capacity of the line is estimated by sending predetermined data to the distribution apparatus 100 and measuring the response speed.

  Here, it is conceivable to use a framework such as CC / PP (Composite Capability / Preference Profiles) as means for transmitting and receiving device information. CC / PP is standardized in the World Wide Web Consortium (W3C), and the hardware characteristics such as the terminal device name (manufacturer name, model name), screen size, memory size, etc., are meta information based on XML. It is possible to describe using a description language. A table in which model names and capabilities are associated with each other may be prepared on the distribution device side, and the capabilities may be acquired from the model names acquired as device information.

  Next, the scene / object conversion circuit 101 reads the scene description data 102 and the object description data 103 that match the multimedia data requested to be distributed from the receiving device 110 (step S202). Then, it is determined whether or not to split the scene description data and the object description data from the analysis result in step S201 and the scene description data 102 and the object description data 103 read in step S202 (step S203).

The determination of division in step S203 includes using the encoded sizes of scene description data and object description data. This is because the size of the scene description data and the object description data corresponds to the description complexity, and the description complexity corresponds to the load of the reproduction processing of the media object. When the scene description data 102 and the object description data 103 are transmitted, if the transmission capacity of the transmission path 109 is low with respect to the size of the description data encoded, or if the processing capability of the reception device 110 is low, The scene description data and the object description data are divided. More specifically, for example, the following conditions (1) and (2):
(1) “size of description data after encoding (number of bytes)” ≦ “number of bytes of data allowed in transmission band”,
(2) “size of description data after encoding (number of bytes)” ≦ “number of bytes of data allowed for processing capability of receiving device”,
In the case of descriptive data satisfying both of the conditions (1) and (2), it is output without being divided, and when neither is satisfied, it is divided. Here, as the processing capability of the receiving apparatus, a CPU clock, a memory capacity, or the like can be used. Further, as another determination method, it may be determined according to the type of the receiving device (PDA, mobile phone, PC, etc.).

  If it is determined in step S203 that the scene description data and the object description data need not be divided, the scene description data and the object description data are transmitted without being divided (step S206). On the other hand, if it is determined that division is necessary, the scene description data and the object description data are divided in steps S204 and S205 until it is determined that further division is not necessary. Hereinafter, division of scene description data and object description data will be described with reference to specific examples.

  For example, as shown in FIG. 3, scene description data and object description data for realizing multimedia data in which a moving image 301, a moving image 302, and a still image 303 are arranged on a screen 300 are shown in FIGS. It is. Here, in order to make the explanation easy to understand, scene description data and object description data that are originally described in binary are expressed as text before the encoding process is performed.

  The scene description data 400 shown in FIG. 4 starts with a Group node. All BIFS scenes start with a kind of node called SFTopNode, but the Group node is one of SFTopNode. As child nodes of this Group node, information regarding the moving image 301, the moving image 302, and the still image 303 is described in a node 401, a node 402, and a node 403, respectively. In the node 401, an ID number indicating the location of the corresponding moving image object data 301 is described. Similarly, in the nodes 402 and 403, the locations of the corresponding moving image object data 302 and the still image object data 303 are described. An ID number to be indicated is described. It should be noted that detailed description of other nodes and fields is omitted here.

  The object description data 500 shown in FIG. 5 describes information necessary for decoding, such as the association of the scene description data and the moving images and still image media objects constituting the scene, the encoding method, the packet configuration, and the like. The The object description data is necessary at the start of a session or when a stream is added, deleted or changed during the session. A command (UPDATE OD) for updating object description data is used at the start of a session or when a stream is newly added to a scene.

The main structure of object description data is
・ ObjectDescriptorID
・ ES (Elementary Stream) Descriptor
It is.

  ObjectDescriptorID is an ID for identifying an object. As described above, an ID number indicating the location is assigned to a node that refers to a still image, audio, or moving image stream in the scene description data, and this ID number is associated with an ObjectDescriptor ID. For example, the object description data 501 to 503 are identified by ObjectDescriptor IDs 1 to 3, and this ID corresponds to a number (ID) specified by “url” in the scene description data 400. For example, the object description data 503 is identified by ObjectDescriptorID “3”, which corresponds to the ImageTexture node defined by the name IMAGE1 specified by url “3” in the scene description data 400.

  An ES Descriptor is required for each stream of still images, moving images, and audio, and each ES Descriptor is identified by ES_ID. The ES Descriptor includes a stream type and profile for determining a stream type, a buffer size necessary for the decoder, a decoder setting descriptor (DecoderConfigDescriptor) describing the maximum / average transmission rate of the stream, and the like. DecoderConfigDescriptor is information required when determining whether or not this elementary stream can be decoded on the receiving device side.

  The object description data has information on the media stream specified in the node of the scene description data. For example, the object description data related to the moving image object 301 is described in 501, and the object description data related to the moving image object 302 is The object description data relating to the still image 303 is described in 503.

  If it is determined in step S203 that the scene description data and the object description data need not be divided, the scene description data 400 shown in FIG. 4 and the object description data 500 shown in FIG. The data is input to the data encoding circuit 105 and the object description data encoding circuit 106 and encoded. The encoded scene description data 400 and the object description data 500 are multiplexed in the multiplexing circuit 108 together with the encoded bit streams of the related moving image 301, moving image 302, and still image 303 in the media bitstream storage device 107. And sent.

  On the other hand, if it is determined in step S203 that the scene description data and the object description data need to be divided, the scene / object conversion circuit 101 divides the object with low priority from the object description data as another object description data. (Step S204). Then, the node related to the object description data divided in step S204 is divided from the scene description data (step S205).

  Here, the method of dividing the scene description data and the object description data in step S204 and step S205 will be specifically described.

  When the scene description data 400 and the object description data 500 shown in FIGS. 4 and 5 are divided, first, an ObjectDescriptor having a low priority is selected from the three ObjectDescriptors (501, 502, 503) included in the object description data 500. Split as separate object data. In the example of FIG. 5, since the priority of the object (503) indicated by “ObjectDescriptorID 3” is the lowest, this ObjectDescriptor is divided as another object data.

  Here, it is conceivable to use the field streamPriority of the ES Descriptor included in the ObjectDescriptor to determine the priority. The streamPriority is a relative standard indicating the priority of this ES, and indicates that an ES having a high streamPriority value is more important than an ES having a lower streamPriority value. Therefore, in the object description data 500 to be divided, the object 503 having the lowest priority among them is divided. By changing the value of this streamPriority, the user can arbitrarily set the priority. Such an editing operation on the object description data 500 can be realized by using a normal text editing application or the like.

  According to the object description data 500 of FIG. 5, the stream priorities of the ESs represented by ES_IDs 101, 102, and 103 are 3, 2, and 1, respectively. Therefore, the ES indicated by ES_ID 103 is determined to be the lowest priority, the ES having the next highest priority is determined to be the ES indicated by ES_ID 102, and the ES having the highest priority is determined to be the ES indicated by ES_ID 103. I can do it. Therefore, in step S204, the object description data 500 shown in FIG. 5 is divided into two object description data 600 and 503 as shown in FIG.

  In step S205, the node 403 related to the object description data 503 divided in step S204 is divided from the scene description data 400. The object description data 503 is identified by ObjectDescriptorID 3, which corresponds to an ImageTexture node defined by the name IMAGE1 specified by url 3 in the scene description data 400.

  In the case of dividing the node, it is possible to divide the grouping node including the node instead of dividing only the node. In this example, the ImageTexture node defined by the name IMAGE1 is grouped. The Transform2D node 403 is divided from the scene description data 400. Therefore, in step S205, the scene description data 400 shown in FIG. 4 is divided into two scene description data 700 and 701 as shown in FIG. For example, when “divide only the node”, the node “only” corresponding to the object description data is divided, and the node is not divided in units of groups. In the example of FIG. 4, IMAGE1 (ImageTexture node) “only” is divided, and no division is performed in units of the Transform2D node 403 that groups IMAGE1. In node division, the node corresponding to the media object may be considered as a single node unit or as a node unit for grouping the nodes, and in any case, the part related to the media object is a unit. Will be divided as However, in reality, in consideration of the division efficiency, it is preferable to divide in units of nodes to be grouped rather than dividing only by a single node.

  In step S203, it is determined whether or not the object description data 600 and 503 and the scene description data 700 and 701 divided into two as shown in FIGS. If it is determined in step S203 that there is no need to further divide the scene description data and the object description data, the scene description data 600 and 503 and the object description data 700 and 701 are set as priority units as separate access units. Are sequentially input to the scene description data encoding circuit 105 and the object description data encoding circuit 106.

  The scene description data 700 and 701 encoded by the scene description data encoding circuit 105 and the object description data 600 and 503 encoded by the object description data encoding circuit 106 are time stamped by the multiplexing circuit 108, respectively. Are multiplexed and transmitted to the transmission path 109. Here, it is assumed that the divided scene description data and object description data are transmitted with a scene description command insertion command and an object description data update command added thereto, respectively.

  The time stamp includes a decoding time stamp and a composite time stamp. The decoding time stamp is a time at which encoded data is to be input to a buffer before the decoding circuit, and a synthesis time stamp is a time at which the data decoded at the decoding circuit is to be output to a memory after the decoding circuit. . In this embodiment, for simplicity, it is assumed that the same value is used for the decoding time stamp and the composite time stamp.

  In addition, the time stamps added to each access unit are added in the order of time from the highest priority to the time stamp. For example, when the time T1 is added to the scene description data 600 and the object description data 700, and the time T2 is added to the scene description data 503 and the object description data 701, time T1 <time T2. An access unit (AU) is a processing unit for time management for decoding and synthesis and for synchronization. For example, in MPEG-4 Video, encoded data (each frame) of one VOP (Video Object Plane) corresponds to one AU. In JPEG, one piece of still image data (JPEG image itself) corresponds to one AU.

  FIG. 8 is an example of a screen displayed based on the scene description data 700 and 701 and the object description data 600 and 503 divided into two by the above-described processing. The receiving apparatus 110 that has received the scene description data 700 to which the time stamp T1 is added and the object description data 600 configures and displays a scene as shown on the screen 800 at the time T1. As described above, the moving image 201 and the moving image 202 are reproduced at the time T1. Further, the receiving apparatus 110 that has received the scene description data 701 to which the time stamp T2 is added and the object description data 503 adds and synthesizes the still image 203 at the time T2. Thus, at time T2, display is performed with the same screen configuration as in FIG.

  Next, the processing when it is determined in step S203 that the scene description data and the object description data need to be further divided will be specifically described with reference to FIGS. Note that the necessity of division is determined based on the encoded size of each piece of description data after division shown in FIGS. In the present embodiment, the necessity of further division is determined using the maximum value among the encoded sizes of the description data. When the object description data 600 and 503 and the scene description data 700 and 701 shown in FIGS. 6 and 7 are divided, the object description data 503 divided from the object description data 500 is not further divided. Then, the object 502 indicated by ObjectDescriptor ID 2 having a low priority is divided from the object description data 600 as another object data.

  According to the object description data 600 of FIG. 6, the stream priorities of the ESs represented by ES_IDs 101 and 102 are 3 and 2, respectively. Therefore, it can be determined that the ES indicated by ES_ID 102 has a lower priority than the ES indicated by ES_ID 103. Therefore, from the object description data 600, the object 502 indicated by ObjectDescriptorID 2 including ES with low priority is divided as other object data. Thus, in step S204, the object description data 600 shown in FIG. 6 is divided into two object description data 501 and 502 as shown in FIG.

  In step S205, the node related to the object description data 502 divided in step S204, that is, the MovieTexture node defined by the name MOVIE2 specified by url 2 in the scene description data 700 of FIG. Divide from 700. Similar to the above-described division method, here, the Transform2D node that groups MovieTexture nodes is divided from the scene description data 700. Therefore, in step S205, the scene description data 700 shown in FIG. 7 is divided into two scene description data 1000 and 1001 as shown in FIG.

  The scene description data 1000, 1001, 701 and the object description data 501, 502, 503 divided into three as shown in FIGS. 9 and 10 are the scene access data in descending order of priority as separate access units. The data is input to the encoding circuit 105 and the object description data encoding circuit 106.

  The scene description data 1000, 1001, and 701 and the object description data 501, 502, and 503 encoded by the scene description data encoding circuit 105 are time-stamped and multiplexed by the multiplexing circuit 108, respectively, and the transmission path 109 is multiplexed. Sent to. In the time stamp added to each access unit, the time T1 is added to the scene description data 1000 and the object description data 501 in order from the highest priority, and the time T2 is added to the scene description data 1001 and the object description data 502. The time T3 is added to the scene description data 701 and the object description data 503. Here, time T1 <T2 <T3.

  FIG. 11 is an example of a screen displayed based on the scene description data and object description data divided into three as described above. The receiving apparatus 110 that has received the scene description data 1000 to which the time stamp T1 is added and the object description data 501 configures and displays a scene as shown on the screen 1100 at the time T1. As described above, the moving image 201 is reproduced at the time T1. Further, the receiving apparatus 110 that has received the scene description data 1001 to which the time stamp T2 is added and the object description data 502 adds the moving image 202 as in the screen 800 at time T2 to synthesize the scene. Further, the receiving apparatus 110 that has received the scene description data 701 to which the time stamp T3 is added and the object description data 503 adds the still image 203 and synthesizes the scene at the time T3 as in the screen 200. Thus, at time T3, display is performed with the same screen configuration as in FIG.

  In order to make the explanation easier to understand, the scene is composed of two moving images and one still image. However, the objects constituting the scene are not limited to moving images and still images. , Text, etc. can be used. Therefore, the control target for dividing the scene / object data is not limited to the still image or the moving image object, and it goes without saying that any object, CG, text, etc. constituting the moving image data can be applied. Yes.

  As described above, according to the above-described embodiment, when the encoded multimedia data composed of a plurality of media objects such as moving images, sounds, still images, texts, and CGs is distributed, Depending on the type and ability, the scene description data and the object description data can be optimized and divided, and can be encoded and transmitted. For this reason, there is an effect that the receiving apparatus can distribute the load and time required for the process of decoding the scene description data and the object description data and synthesizing the scene.

Also, it analyzes the communication line status of the network, acquires the transmission capacity of the network, and uses this to determine whether to divide the scene description data and object description data. Therefore, in an appropriate form according to the network status Multimedia data can be distributed.
Further, since the description data (in the embodiment, object priority data) describing the information relating to the reproduction of each media object is included, a description (stream priority in the embodiment) indicating the priority of the media object is included. Can easily set the priority of media objects.
In addition, a time stamp is added to a plurality of divided pieces of description data so that they are reproduced in the order according to the priority of the media object, so that the reproduction can be performed as intended by the content creator or the like.

[Other Embodiments]
In addition, as described above, the object of the present invention is to provide a storage medium storing a program code of software that realizes the functions of the embodiment to a system or apparatus, and the computer of the system or apparatus (or CPU or MPU) stores it. Needless to say, this can also be achieved by reading and executing the program code stored in the medium. In this case, the program code itself read from the storage medium realizes the functions of the above-described embodiments, and the storage medium storing the program code constitutes the present invention.

  As a storage medium for supplying the program code, for example, a floppy (registered trademark) disk, hard disk, optical disk, magneto-optical disk, CD-ROM, CD-R, magnetic tape, nonvolatile memory card, ROM, or the like is used. be able to.

  Further, by executing the program code read by the computer, not only the functions of the above-described embodiments are realized, but also an OS (operating system) running on the computer based on the instruction of the program code. Needless to say, some or all of the actual processing is performed and the functions of the above-described embodiments are realized by the processing.

  Furthermore, after the program code read from the storage medium is written to the memory provided in the function expansion board inserted into the computer or the function expansion unit connected to the computer, the function is based on the instruction of the program code. It goes without saying that the CPU or the like provided in the expansion board or the function expansion unit performs part or all of the actual processing, and the functions of the above-described embodiments are realized by the processing.

It is a figure which shows the flow of data between each circuit while showing the basic composition of the delivery apparatus in embodiment. It is a flowchart for demonstrating the process in which a delivery apparatus delivers multimedia data. It is a figure which shows the example of a screen structure of multimedia data. It is a figure which shows the example of the scene description data which implement | achieves the screen structure of FIG. It is a figure which shows the example of the object description data which implement | achieves the screen structure of FIG. It is a figure which shows the example of the object description data which divided | segmented the object description data of FIG. It is a figure which shows the example of the scene description data which divided | segmented the scene description data of FIG. It is a figure which shows the example of the screen comprised from the scene description data of FIG. 7, and the object description data of FIG. It is a figure which shows the example of the object description data which divided | segmented the object description data of FIG. It is a figure which shows the example of the scene description data which divided | segmented the scene description data of FIG. It is a figure which shows the example of the screen comprised from the scene description data of FIG. 10, and the object description data of FIG.

Claims (17)

Description data having object description data including a plurality of media objects, scene description data describing a temporal and spatial relationship of the plurality of media objects, and description of information necessary for associating and decoding the scene description with the media object; A data processing device for delivering multimedia data including
Determination means for determining whether or not description data needs to be divided based on external capabilities related to distribution of the multimedia data and the scene description data and object description data;
A division unit that divides each of the object description data and the scene description data in units of a part related to the media object when the determination unit determines that the division is necessary;
A data processing apparatus comprising: distribution means for encoding each description data divided by the dividing means and a plurality of media objects, multiplexing them to generate multimedia data, and distributing the multimedia data to the receiving terminal. A data processing apparatus for delivering multimedia data including a plurality of media objects and description data describing information related to reproduction of each media object to an external device,
Determining means for determining the necessity of dividing the description data based on the external capability related to the delivery of the multimedia data and the description data;
When the determination unit determines that division is necessary, based on the priority set for the media object to which the description data relates, the description data portion related to the low-priority media object is separated. Dividing means for dividing the description data;
A data processing apparatus comprising: distribution means for encoding each description data divided by the dividing means and a plurality of media objects, multiplexing them to generate multimedia data, and distributing the multimedia data to the receiving terminal. The description data includes scene description data describing a spatio-temporal relationship between the plurality of media objects, and object description data including a description of information necessary for associating and decoding the scene description and the media object.
3. The data processing apparatus according to claim 2, wherein the dividing unit separates a part related to a media object having a low priority for each of the object description data and the scene description data.   The determination means receives device information from the external device, analyzes the device information to acquire the processing capability of the external device, and determines whether the description data needs to be divided based on the acquired processing capability. The data processing apparatus according to claim 1, wherein the data processing apparatus is a data processing apparatus. The external device is connected via a network;
The determination means analyzes the communication line status of the network to acquire the transmission capacity of the network, and determines whether or not the description data needs to be divided based on the acquired transmission capacity. 5. The data processing device according to any one of 4 to 4. The description data includes a description indicating a priority for the associated media object;
The data processing apparatus according to claim 2, wherein the dividing unit determines a description related to the media object to be separated with reference to a description indicating the priority.   4. The data processing apparatus according to claim 3, wherein the priority level is determined according to streamPriority indicating a priority level of a stream included in the object description data.   The dividing means divides the object description data by separating a description portion corresponding to a media object having a lower priority than the object description data, and further divides the scene description data according to the divided object description data. The data processing apparatus according to claim 1 or 3, characterized in that   9. An adding means for adding a time stamp to the plurality of description data divided by the dividing means so as to be reproduced in the order according to the priority of the media object. A data processing apparatus according to any one of the above.   The data processing apparatus according to claim 1, wherein the multimedia data is data according to MPEG-4.   11. The data processing apparatus according to claim 1, wherein the scene description data is described by BIFS.   The data processing apparatus according to claim 1, wherein the media object includes a still image, a moving image, and a CG.   The dividing means divides the scene description data by separating a single node corresponding to the media object from the scene description data, or separates a node that groups nodes corresponding to the media object. The data processing apparatus according to claim 1, wherein the data processing apparatus is divided. Description data having object description data including a plurality of media objects, scene description data describing a temporal and spatial relationship of the plurality of media objects, and description of information necessary for associating and decoding the scene description with the media object; A data processing method for delivering multimedia data including a message to an external device,
A determination step of determining the necessity of dividing the description data based on the external capabilities related to the delivery of the multimedia data and the scene description data and the object description data;
A division step of dividing each of the object description data and the scene description data in units of portions related to the media object when it is determined that the division is necessary in the determination step;
A data processing method comprising: a distribution step of encoding each description data divided by the division step and a plurality of media objects, multiplexing them to generate multimedia data, and distributing the multimedia data to the receiving terminal. A data processing method for delivering multimedia data including a plurality of media objects and description data describing information related to reproduction of each media object to an external device,
A determination step of determining whether or not the description data needs to be divided based on external capabilities related to delivery of the multimedia data and the description data;
When the determination step determines that division is necessary, based on the priority set for the media object to which the description data is related, by separating the description data portion related to the media object with the lower priority A dividing step of dividing the description data;
A data processing method comprising: a distribution step of encoding each description data divided by the division step and a plurality of media objects, multiplexing them to generate multimedia data, and distributing the multimedia data to the receiving terminal.   A control program for causing a computer to execute the data processing method according to claim 14 or 15.   A storage medium storing a control program for causing a computer to execute the data processing method according to claim 14.

Images