JP4214103B2 - Data processing apparatus and data processing method - Google Patents

Description

  The present invention describes the configuration of media content in order to perform playback and delivery in accordance with viewer preference and terminal capability in viewing, playback, and delivery of media content that is continuous audiovisual information such as video, video, and audio. The present invention relates to a data processing apparatus and a data processing method for converting a description to a description of an expression to be reproduced.

  Conventionally, storage of media content is performed in units of files, and reproduction and delivery of media content are performed in units of files that store media content.

  In addition, when media content is digitized by a plurality of different methods and stored in a plurality of files, decoding processing occurs when the media content is played back. The amount of decoding processing varies depending on the digitization method. For this reason, when selecting media content, it is necessary to select media content that has been digitized by a digitization method that matches the processing capability of the terminal that reproduces the media content. In this case, the display media content is selected based on the capability of the terminal device by the receiving user by selecting the media content that matches the capability of the terminal to be used in units of files.

  Also, as a method of reproducing only a specific scene in moving image distribution using the World Wide Web, the method described in Patent Document 1 is known. FIG. 50 shows a configuration diagram of the moving image distribution apparatus described in Patent Document 1, which will be described below.

  In this moving image distribution apparatus, a scene number, a start / end frame time code, a keyword related to a scene, and a moving image file name are input in advance from the scene information input unit 3903 to the scene information storage unit 3904. Then, the scene search unit 3905 searches the scene information stored in the scene information storage unit 3904 using the search conditions input from the scene information input unit 3903. Then, the scene search unit 3905 extracts the scene number of the searched desired scene and stores it in the scenario storage unit 3907 as a scenario.

Next, the scenario editing unit 3908 changes the order of the extracted scenes and deletes unnecessary scenes as necessary. Then, the moving image transfer unit 3909 transfers the moving image data stored in the moving image file storage unit 3902 to the moving image reproducing unit 3910 in the order of the scene numbers stored in the scenario edited by the scenario editing unit 3908 and reproduces it. Note that a moving image is input from the moving image file input unit 3901 to the moving image file storage unit 3902.
JP 10-1111872 A

  However, in the conventional method of reproducing contents in units of files, all contents stored in files must be reproduced. Therefore, it is impossible to see an outline that is a summary of the content. In addition, when searching for a highlight scene in which only a part of the content is extracted or a scene that the viewer wants to see, there is a problem that the content must be referenced from the top.

  In addition, according to the method described in Patent Document 1, it is not necessary to refer to the beginning of content because the playback order of scene cuts can be specified. However, this method only has the order of scene playback as a scenario. For this reason, it is impossible to do anything other than rearranging the order of scene playback. Therefore, there is a problem that complicated reproduction such as reproduction corresponding to a plurality of media cannot be performed.

  An object of the present invention is to generate expression description data for reproducing a media segment described in the structure description data with various restrictions from the structure description data expressing the configuration of the media content. .

  In order to solve the above problems, the present invention provides an expression description that expresses the playback order, playback timing, and synchronization information of the media segments described in the structure description data from the structure description data that describes the structure of the media content. Generate data.

  Thus, by selecting several media segments from the structure description data and converting them into expression description data that represents the playback order, playback timing, and synchronization information of the media segments, synopsis, highlight scene, viewing The display form of the scene collection according to the user's preference can be obtained. In addition, since the expression description data has a reproduction order, reproduction timing, and synchronization information, a plurality of media can be associated and reproduced.

  Furthermore, the present invention stores a set of alternative data of media segments in the structure description data, and converts the data into expression description data expressing the reproduction order, reproduction timing, and synchronization information of at least one of the media segments and the alternative data. I did it.

  As a result, switching between media segment reproduction or alternative data reproduction can be performed in accordance with the capacity and communication volume of the network for delivering the media content, the capability of the terminal for reproducing the media content, and the like. In other words, it is possible to deliver and play back content using media that matches the capabilities of the terminal to be played back.

  Furthermore, the present invention is provided with a media selection unit that selects whether to reproduce a media segment or alternative data when a media segment expressed in the structure description data is reproduced.

  Thereby, even if a viewer does not select a media segment or alternative data according to the capability of the terminal, the media selection unit can automatically select the media segment or alternative data according to the capability of the terminal.

  Furthermore, the present invention further describes the score based on the context contents of each media segment in the structure description data.

  Thereby, for example, a collection of highlight scenes having various reproduction times can be generated, and the reproduction and delivery thereof can be easily performed. Further, by setting the score based on the viewpoint indicated by the keyword, by designating the keyword, it is possible to reproduce and deliver only the scene according to the user's preference.

  ADVANTAGE OF THE INVENTION According to this invention, the structure description data describing the structure of the media content by a media segment can be converted into the expression description data expressing the form which reproduces a media content. For this reason, in the playback of media content, restrictions such as playback timing and synchronization information for each media segment can be added, and various playbacks can be realized.

  According to a first aspect of the present invention, there is provided a data processing apparatus comprising: a set of time information of a media segment that is a division into which the media content is divided into an entire media content that is continuous audiovisual information or a configuration of the media content. The structure description data to be input is input, the analysis unit that acquires the time information of the media segment described in the input structure description data, and the structure description data is converted to the structure description data using the time information of the analyzed media segment. The media segment includes a playback unit, a playback timing, and a conversion unit that converts the media segment into expression description data that expresses synchronization information and outputs the data.

  With this configuration, it is possible to generate information related to reproduction from information related to the configuration of media content.

  According to a second aspect of the present invention, in the data processing apparatus according to the first aspect, the structure description data includes a set of alternative data of the media segment, and the conversion unit converts the structure description data into the medium. At least one of the segment and the alternative data is converted into expression description data representing reproduction order, reproduction timing, and synchronization information.

  As a result, information related to reproduction including selection of display media can be generated from information related to the configuration of the media content.

  According to a third aspect of the present invention, in the data processing apparatus according to the second aspect, when reproducing the media segment expressed in the structure description data, either the media segment or the alternative data is reproduced. A media selection unit for selecting the media segment, and the conversion unit, based on the selection of the media selection unit, the playback order, playback timing, and synchronization information of either the media segment or the alternative data data The expression description data that expresses is output.

  As a result, information related to reproduction including selection of display media can be generated from information related to the configuration of the media content.

  According to a fourth aspect of the present invention, in the data processing device according to any one of the first to third aspects, the expression description data is a SMIL document.

  As a result, the expression description data can be made to the international standard, so that the expression description data can be versatile.

  The data processing apparatus according to the fifth aspect of the present invention expresses the configuration of media content, which is continuous audiovisual information in which video information and audio information are synchronized, by a set of media segments that are divisions of the media content. And the structure description data describing the time information of the media segment and the score in the context content of the media segment, and the selection condition for selecting the predetermined media segment from the structure description data, and the input A selection unit that selects only the media segment having a score that meets the selection condition from structure description data, and an expression that expresses the playback order and playback timing of the media segment for the media segment selected by the selection unit. A conversion unit that converts and outputs the description data It takes Bei configuration.

  With this configuration, it is possible to select only the media segment that meets the condition from the information related to the configuration of the media content, and to generate information related to reproduction of only the selected media segment.

  According to a sixth aspect of the present invention, in the data processing device according to the fifth aspect, the structure description data includes a set of alternative data of the media segment, and the conversion unit converts the structure description data into the medium. At least one of the segment and the alternative data is converted into expression description data representing reproduction order, reproduction timing, and synchronization information.

  As a result, it is possible to select only the media segment that meets the conditions from the information related to the configuration of the media content, and to generate information related to reproduction including selection of the display media of the selected media segment.

  According to a seventh aspect of the present invention, in the data processing device according to the sixth aspect, the selection unit selects whether to reproduce the media segment or the alternative data when reproducing the selected media segment. I do.

  As a result, it is possible to select only the media segment constituting the synopsis or highlight scene from the information related to the configuration of the media content, and to generate information related to the playback of the selected media segment.

  According to an eighth aspect of the present invention, in the data processing device according to any one of the fifth to seventh aspects, the score is the importance of the corresponding media segment based on the context content of the media content.

  As a result, it is possible to select only a media segment that constitutes a synopsis or highlight scene reflecting the viewer's preference from information related to the configuration of the media content, and generate information related to playback of only the selected media segment.

  According to a ninth aspect of the present invention, in the data processing device according to any one of the fifth to eighth aspects, the media segment is assigned a viewpoint expressed by a keyword, and the score is based on the viewpoint The importance of the corresponding media segment.

  As a result, it is possible to select only a media segment that constitutes a synopsis or highlight scene reflecting the viewer's preference from information related to the configuration of the media content, and generate information related to playback of only the selected media segment.

  According to a tenth aspect of the present invention, there is provided a data processing device comprising: a set of media segments, which are divisions of the media content, comprising a configuration of media content that is continuous audiovisual information of at least one of video information and audio information. The structure description data that expresses and describes the time information of the media segment and the score in the context content of the media segment, and the selection condition for selecting the predetermined media segment from the structure description data are input and input A selection unit that selects only the media segment having a score that meets the selection condition from the structure description data, and the media segment selected by the selection unit is a playback order, a playback timing of the media segment, and Representation description expressing synchronization information Employs a configuration equipped with conversion section outputting converted into over data, the.

  With this configuration, it is possible to select only the media segment that meets the condition from the information related to the configuration of the media content, and to generate information related to reproduction of only the selected media segment.

  According to an eleventh aspect of the present invention, there is provided a data processing device that expresses a configuration of media content that is continuous audiovisual information by a set of media segments that are segments into which the media content is segmented, and time information of the media segments. The structure description data describing the score in the context contents of the media segment and the selection condition for selecting the predetermined media segment from the structure description data are input, and the selection condition is determined from the input structure description data. A selection unit that selects only the media segment having a score, and the media segment selected by the selection unit is converted into expression description data that represents the playback order, playback timing, and synchronization information of the media segment. The conversion unit to output and the expression description data The type of media content, a configuration provided with the, a reproducing unit for reproducing the media content in accordance with the contents of the representation description data.

  With this configuration, it is possible to select only the media segment that meets the condition from the information related to the configuration of the media content, and to reproduce only the selected media segment.

  A server-client system according to a twelfth aspect of the present invention is the server having the selection unit and the conversion unit, the client having the reproduction unit according to the eleventh aspect, the server, A network for connecting clients, and the representation description data is communicated between the server and the clients.

  This allows the server to select the summary and playback media according to user requirements, network conditions, playback terminal capabilities, etc., and to play only the necessary media segments on the client side. Has the effect that only necessary data need be received.

  A server-client system according to a thirteenth aspect of the present invention is a server including the selection unit according to an eleventh aspect, a client including the conversion unit and the reproduction unit according to the eleventh aspect, the server, and the server. A network to which the client is connected, and the summary structure description data in which only the media segment selected by the selection unit remains is communicated between the server and the client.

  As a result, the server side summarizes it according to user requirements, network conditions, playback terminal capabilities, etc., the client side selects the appropriate playback media that meets various conditions, and only the necessary media segments. Reproduction is possible, and only necessary data is received in communication of reproduction media.

  According to a fourteenth aspect of the present invention, there is provided structure description data that expresses the entire media content that is continuous audiovisual information or the configuration of the media content portion by a set of time information of media segments that are the divisions of the media content. To obtain the time information of the media segment described in the inputted structure description data, and using the analyzed time information of the media segment, the structure description data is reproduced in the reproduction order and reproduction of the media segment. The data processing method is characterized in that it is converted into expression description data expressing the timing and synchronization information and output.

  According to a fifteenth aspect of the present invention, a configuration of media content that is continuous audiovisual information in which video information and audio information are synchronized is expressed by a set of media segments that are divisions of the media content, and the media segments Time description, structure description data describing a score in the context content of the media segment, and a selection condition for selecting a predetermined media segment from the structure description data, and from the input structure description data Only the media segment having a score that meets a selection condition is selected, and the selected media segment is converted into expression description data that represents the playback order and playback timing of the media segment and is output. Data processing method.

  According to a sixteenth aspect of the present invention, a configuration of media content that is continuous audiovisual information of at least one of video information and audio information is expressed by a set of media segments that are divisions of the media content, and The structure description data in which the time information of the media segment, the structure description data describing the score in the context content of the media segment, and the selection condition for selecting the predetermined media segment from the structure description data are input. Only the media segment having a score that meets the selection condition is selected, and the selected media segment is converted into expression description data representing the playback order, playback timing, and synchronization information of the media segment and output. Data characterized by It is a management method.

  According to a seventeenth aspect of the present invention, the entire media content that is continuous audiovisual information or the configuration of the media content portion is expressed on a computer by a set of time information of media segments that are segments into which the media content is segmented. The structure description data is input, the time information of the media segment described in the input structure description data is acquired, and the structure description data is reproduced using the analyzed time information of the media segment. A program characterized in that it is converted into expression description data expressing the order, the timing of playback of the media segment, and the synchronization information of the media segment, and output.

  According to an eighteenth aspect of the present invention, a configuration of media content that is continuous audiovisual information in which video information and audio information are synchronized is expressed in a computer by a set of media segments that are divisions of the media content, and The structure description inputted by inputting the structure description data describing the time information of the media segment, the score in the context content of the media segment, and the selection condition for selecting the predetermined media segment from the structure description data Selecting only the media segment having a score that meets the selection condition from the data, converting the selected media segment into expression description data expressing the playback order and playback timing of the media segment, and outputting the data. It is a program that features .

  According to a nineteenth aspect of the present invention, a configuration of media content that is continuous audiovisual information of at least one of video information and audio information is expressed on a computer by a set of media segments that are divided into the media content. And the structure description data describing the time information of the media segment and the score in the context content of the media segment, and the selection condition for selecting the predetermined media segment from the structure description data, and the input Only the media segment having the score that meets the selection condition is selected from the structure description data, and the selected media segment is used as the expression description data that represents the playback order, playback timing, and synchronization information of the media segment. Convert and output It is a program characterized by.

  A twentieth aspect of the present invention is a computer-readable recording medium storing the program according to any one of the seventeenth to nineteenth aspects.

(Embodiment 1)
Embodiment 1 of the present invention will be described below with reference to the accompanying drawings. First, the configuration of the data processing system according to the first exemplary embodiment of the present invention will be described with reference to FIG. FIG. 1 is a conceptual diagram of the data processing system according to the first embodiment.

  The data processing system according to the first exemplary embodiment includes a metadata database 1001, a summary engine 1002, a description converter 1003, a playback device 1004, and a media content database 1005. Further, in the figure, what is described as 1006 is a content description that is metadata, what is described as 1007 is a selection condition, and what is described as 1008 is a summary content description that is a summary result, What is written in 1009 is a playback method description for giving an instruction to the player 1004, and what is written in 1010 is media content data.

  The metadata is data representing additional information of the media content such as bibliographic items such as the title of the media content, creation date and time, contents, and scene configuration. The database 1001 represents such a metadata database.

  The summary engine 1002 inputs content description 1006 that is structure description data representing the content and configuration of the media content from the metadata stored in the database 1001. Then, the summary engine 1002 selects only scenes that meet the selection condition 1007 input by the user from the input content description 1006. Further, the summary engine 1002 generates and outputs a summary content description 1008 that leaves only the scenes selected from the content description 1006 and deletes the rest.

  The content description 1006 and the summary content description 1008 are structural description data representing the content and configuration of the media content, and the description format is the same although the number of scenes described is different.

  The description converter 1003 receives the summary content description 1008 and expresses the description of the media playback mode, such as the playback order, playback start timing, and synchronization information when the scene described in the summary content description 1008 is played back. A reproduction method description 1009 that is data is generated and output.

  The playback device 1004 inputs a playback method description 1009 and media content data 1010 that is data to be played back according to the playback method description 1009 from the media content database 1005. The playback device 1004 plays back the media content data 1010 according to the playback order, playback start timing, synchronization information, and the like described in the playback method description 1009.

  Further, since the summary content description 1008 and the content description 1006 have the same format, the description converter 1003 can similarly generate a reproduction method description (expression description data) for the content description 1006.

  Next, the structure description data used in the content description 1006 and the summary content description 1008 will be described with reference to FIG. 2 (a), FIG. 2 (b), and FIG.

  FIG. 2A shows Document Type Definition (DTD) which is a definition for describing structure description data in XML. FIG. 2B is an example of structure description data for media content in which moving images and audio are synchronized, using MPEG1 as an example. FIG. 3 is an example of structure description data of media content in which moving images and audio are different media.

  In this embodiment, Extensible Markup Language (XML) is used as an example of a form in which structure description data is expressed on a computer.

  XML is a data description language standardized by the World Wide Web Consortium (W3C), and Ver. 1.0 was recommended on February 10, 1998. The XML ver.1.0 specification can be obtained at http://www.w3.org/TR/REC-xml.

  First, the Document Type Definition (DTD), which is a definition for describing structure description data in XML, will be described with reference to FIG.

  As shown in 201 in the figure, the contents element is composed of a par element and a mediaObject element. Further, as indicated by 202 in the figure, the contents element has a title attribute indicated by character data.

  The mediaObject element represents the media. Further, as indicated by 203 in the figure, the par element is composed of a plurality of mediaObject elements that are child elements. The par element represents that, when the contents element is composed of a plurality of mediaObject elements such as audio and video, the plurality of mediaObject elements are played back in synchronization with each other as a child element.

  Further, as indicated by 204 in the figure, the mediaObject element is composed of segment elements representing media segments. Further, as indicated by 205 in the figure, the mediaObject element is designated with the type of media by its type attribute. In this example, the media type is audio, which is audio information, video, which is video information, image, which is still image information, audiovideo, which is information in which audio and video are synchronized, and audioimage, which is audio and still image information. A type is specified. If the type attribute is not specified, the type attribute is set to audiovideo by default.

  Also, as indicated by 206 in the figure, the mediaObject element designates a media format such as MPEG1 or MPEG2 in the format attribute. Further, as indicated by reference numeral 207 in the figure, the mediaObject element designates a location where data is stored with the src attribute. By specifying Uniform Resource Locator (URL) in src attribute, you can specify the location where data is stored.

  Further, as indicated by 208 in the figure, the segment element has a start attribute and an end attribute. The start attribute and end attribute indicate the start time of the segment element and the end time of the segment element. The start attribute and the end attribute indicate the time inside the media specified by the mediaObject element. That is, the segment element specifies which part in the media specified by the mediaObject element corresponds to the start attribute and the end attribute.

  In the present embodiment, the time information of the media segment is specified by a set of start time and end time, but may be expressed by a set of start time and duration.

  Next, an example of structure description data for media content in which moving images and audio are synchronized will be described using MPEG1 as an example with reference to FIG.

  In the structure description data shown in FIG. 2B, the title “Movie etc” is specified in the contents element. In the mediaObject element type, audiovideo is specified, MPEG1 is specified as the format, and http://mserv.com/MPEG/movie0.mpg is specified as the storage location. The mediaObject element includes a segment element having time information from time 00:00:00 to 00:01:00, a segment element having time information from time 00: 01: 0 to 00:02:00, and a time 00 Segment elements having time information from 03:00 to 00:04:00 and segment elements having time information from time 00:04:00 to 00:05:00. That is, the mediaObject element has a description excluding the time 00:02:00 to 00:03:00.

  Next, an example of structure description data of media content in which moving images and audio are different media will be described with reference to FIG.

  In the structure description data shown in FIG. 3, the title “Movie etc” is specified in the contents element. In the example of FIG. 3, the contents element is composed of a mediaObject element of the video type and a mediaObject element of the audio type. Therefore, the par element synchronizes the mediaObject element whose type is video and the mediaObject element whose type is audio.

  The mediaObject element of type video specifies MPEG1 as the format and http://mserv.com/MPEG/movie0v.mpv as the storage location.

  The mediaObject element of type video has a segment element with time information from time 00:00:00 to 00:01:00 and a segment element with time information from time 00: 01: 0 to 00:02:00 And a segment element having time information from time 00:03:00 to 00:04:00 and a segment element having time information from time 00:04:00 to 00:05:00. In other words, the mediaObject element of type video is a description excluding the time 00:02:00 to 00:03:00.

  The mediaObject element of the type “audio” specifies MPEG1 as the format and “http://mserv.com/MPEG/movie0a.mp2” as the storage location. The mediaObject element of type audio has a segment element with time information from time 00:00:00 to 00:01:00 and a segment element with time information from time 00: 01: 0 to 00:02:00. And a segment element having time information from time 00:03:00 to 00:04:00 and a segment element having time information from time 00:04:00 to 00:05:00. That is, the mediaObject element whose type is audio is a description excluding the time 00:02:00 to 00:03:00.

  By the way, when the content is composed of a plurality of media, it is necessary to control playback timing and synchronization between media segments. Therefore, in this embodiment, description converter 1003 reproduces summary content description 1008 described by structure description data by using description description data that can represent the playback order of media segments, the timing of playback, and synchronization information. The description is converted into the description 1009.

  In this embodiment, Synchronized Multimedia Integration Language (SMIL) is used as expression description data. SMIL is a description language standardized by the W3C for the purpose of describing the temporal behavior of these expressions and the layout on the display screen for multiple media. On June 15, 1998, SMIL Ver 1.0 was recommended. The specification for SMIL ver. 1.0 is available at http://www.w3.org/TR/REC-smil.

  In this way, by using standardized SMIL for the expression description data, existing or future SMIL processing programs can be used, so it is versatile.

  Next, a process of converting the structure description data described in XML into expression description data expressing the playback form of media segment playback order, playback timing, synchronization information, and the like will be described with reference to FIG. FIG. 4 is a flowchart showing a procedure in which the description converter according to the first embodiment converts the structure description data into SMIL.

  First, when processing is started (step S401), in step S402, the description converter 1003 checks whether or not there is an element par element in the summary content description 1008 described by the structure description data. If the description converter 1003 determines that there is a par element in step S402, the description converter 1003 proceeds to step S406. If it is determined that there is no par element, the description converter 1003 proceeds to step S403.

  In step S403, the description converter 1003 converts the media type from the type attribute, the media format from the format attribute, and the media data URL from the src attribute of the mediaObject element of the summary content description 1008 described by the structure description data. get. Next, description converter 1003 acquires the time information of the media segment from the start attribute and end attribute of each segment element in step S404, and stores it. In step S405, the description converter 1003 generates a playback method description 1009 described in the SMIL document from the media format, media data URL, and media segment time information acquired in steps S403 and S405, and outputs them. To do.

  On the other hand, the description converter 1003 acquires the first mediaObject element in the par element in step S406. Next, in step S407, the description converter 1003 acquires the media type from the type attribute, the media format from the format attribute, and the URL of the media data from the src attribute of the acquired mediaObject element. In step S408, the description converter 1003 acquires the time information of the media segment from the start attribute and the end attribute of each segment element, and stores it.

  Next, in step S409, the description converter 1003 checks whether there is a mediaObject element that has not been checked yet in the par element. If there is a mediaObject element that has not been checked, the description converter 1003 acquires the first one in step S410, and proceeds to the processing in step S407. On the other hand, if there is no mediaObject element that has not been examined, the description converter 1003 proceeds to the process of step S411.

  Next, in step S411, description converter 1003 groups segment elements that belong to different mediaObject elements and overlap in time using the stored time information of segment elements. In step S412, the description converter 1003 generates and outputs a playback method description 1009 described in the SMIL document from the media format, media data URL, and media segment time information acquired in steps S407 and S408. .

  Next, when there is no par element in the summary content description 1008 that is the structure description data, the description converter 1003 outputs the reproduction method description 1009 that is a SMIL document from the summary content description 1008. FIG. It explains using. FIG. 5 is a flowchart in which the description converter according to the first embodiment outputs a reproduction method description that is a SMIL document from a summary content description that is structure description data.

  First, the description converter 1003 outputs a SMIL header (step S501).

  The SMIL document is composed of a header 601 and a main body 602 as shown in FIG. The header 601 is described by a head element, and the main body 602 is described by a body element. That is, the header 601 is a portion surrounded by <head> and </ head>, and the main body 602 is a portion surrounded by <body> and </ body>.

  The header can describe information such as the creator and creation date, and the layout such as where the video is displayed on the screen and where the text is displayed. The header can be omitted.

  Next, description converter 1003 surrounds the entire media segment with <seq> and </ seq> (step S502). These are seq elements, which indicate that the media segments enclosed by <seq> and </ seq> are processed, that is, played or displayed, in the order described.

  Next, description converter 1003 performs the following processing for each media segment enclosed by <seq> and </ seq>.

  First, a corresponding element is selected from the SMIL audio element, video element, ref element, and img element in accordance with the media type (step S503). The ref element is defined as a writing method that does not specify the source medium. What is specified by the ref element may be audio, video, still image, or video and audio synchronized.

  Next, description converter 1003 sets the values of the clip-begin attribute and clip-end attribute of the element selected in step S503 as follows. That is, the start attribute value of the segment element of the corresponding summary content description 1008 is set to the SMIL clip-begin attribute value, and the end attribute value is set to the clip-end attribute value (step S504). Note that “clip” refers to a time interval.

  Next, the description converter 1003 sets the value of the src attribute of the element selected in step S503 to the value of the src attribute of the mediaObject element that is the parent element of the segment element of the corresponding summary content description 1008 (step S505). Thereafter, the description of the element selected in step S503 is output.

  In this way, the description converter 1003 generates a reproduction method description 1009 that is expression description data described in SMIL from the summary content description 1008 that is structure description data.

  FIG. 7 shows a SMIL document output by the description converter 1003 from the structure description data shown in FIG. FIG. 7 is a diagram illustrating an example of a SMIL document output by the description converter according to the first embodiment.

  In the example of the document shown in FIG. 7, the information from time 00:00:00 to 00:01:00 of http://mserv.com/MPEG/movie0.mpg, http://mserv.com/MPEG/movie0.mpg. mpg time 00: 01: 0 to 00:02:00 information, http://mserv.com/MPEG/movie0.mpg time 00:03:00 to 00:04:00 information, http It is processed in the order of information from time 00:04:00 to 00:05:00 of: //mserv.com/MPEG/movie0.mpg. In the example shown in FIG. 7, the header is omitted.

  Further, a process for collecting temporally continuous clips into one may be added to output the SMIL document shown in FIG.

  In the example of the document shown in FIG. 8, information having information from time 00:00:00 to 00:02:00 of http://mserv.com/MPEG/movie0.mpg, http://mserv.com/MPEG It is processed in the order of information from 00:03:00 to 00:05:00 of /movie0.mpg. That is, the document shown in FIG. 8 performs the same processing as the example of the document shown in FIG.

  Next, when there is a par element in the summary content description 1008 that is the structure description data, the description converter 1003 outputs the reproduction method description 1009 that is a SMIL document from the summary content description 1008. FIG. It explains using. FIG. 9 is a flowchart in which the description converter according to the first embodiment outputs a reproduction method description that is a SMIL document from a summary content description that is structure description data.

  First, the description converter 1003 outputs a SMIL header (step S901). Next, description converter 1003 surrounds the entire media segment with <seq> and </ seq> (step S902). Then, the description converter 1003 encloses the media segment groups in SMIL <par> and </ par> in the order of the group with the earliest time (step S903).

  Next, description converter 1003 determines whether there are other media segments belonging to the same mediaObject element (step S904). If there are other media segments, they are enclosed in <seq> and </ seq> (step S905). ). The description converter 1003 performs the following processing for each media segment enclosed by <seq> and </ seq>.

  First, the corresponding element is selected from the SMIL audio element, video element, ref element, img element, etc. according to the type of media. Next, the values of the clip-bigen attribute and clip-end attribute of the selected element are set (step S906). In the summary content description 1008, the start attribute value of the corresponding segment element is set to the SMIL clip-begin attribute value, and the end attribute value is set to the clip-end attribute value (step S907). Next, the value of the src attribute of the selected element is set to the value of the src attribute of the mediaObject element that is the parent element of the segment element of the corresponding summary content description 1008 (step S908). Next, a description of the selected element is output.

  On the other hand, if there is no media segment belonging to the same mediaObject attribute, description converter 1003 does not enclose it with <seq> and </ seq>, and performs the same processing as that performed for each media segment.

  In this way, the description converter 1003 allows the reproduction method that is expression description data to be processed in synchronization with a plurality of media, even if the summary content description 1008 that is the structure description data is composed of a plurality of media. A description 1009 is generated.

  FIG. 10 shows a SMIL document output from the structure description data shown in FIG. FIG. 10 is a diagram illustrating an example of a SMIL document output by the description converter according to the first embodiment.

  In the example of the document shown in FIG. 10, information from time 00:00:00 to 00:01:00 of http://mserv.com/MPEG/movie0v.mpv which is video and http: // mserv. com / MPEG / movie0a.mp2 time 00:00:00 to 00:01:00 information is synchronized and video http://mserv.com/MPEG/movie0v.mpv time 00:01:00 00:02:00 information and audio http://mserv.com/MPEG/movie0a.mp2 information from time 00:01:00 to 00:02:00, and video http: / Information from 00:03:00 to 00:04:00 at /mserv.com/MPEG/movie0v.mpv and audio from http://mserv.com/MPEG/movie0a.mp2 from 00:03:00 Synchronize the information at 00:04:00, http://mserv.com/MPEG/movie0v.mpv, which is video, information from time 00:04:00 to 00:05:00 and http: //, which is audio The information from time 00:04:00 to 00:05:00 of mserv.com/MPEG/movie0a.mp2 is synchronized, and the synchronized information is processed in the order described.

  Further, as shown in FIG. 11, a SMIL document to which processing for grouping temporally continuous clips into one may be output.

  Also, in order to synchronize a plurality of clips in the par element of the SMIL document, it may be necessary to make the playback start time of a certain clip different from the playback start time of other clips. For example, audio and video may be different media objects, and the video clip may be a range in which the person is reflected, and the audio clip may be only the voice spoken by the person. In this case, the audio needs to be played back from where the person starts speaking in accordance with the video of the movement of the human mouth included in the video.

  That is, it is necessary to calculate the playback start time of each clip and start playback when that time comes. For such purposes, SMIL provides an attribute called “begin” indicating delay information in the audio element, video element, img element, and ref element. By using the begin attribute, the playback start time can be varied for each clip.

  FIG. 12 is a diagram illustrating an example of a SMIL document in which the reproduction start time is different for each clip. In the document shown in FIG. 12, by using the begin attribute, the reproduction time of information from time 00:00:00 to 00:01:00 of http://mserv.com/MPEG/movie0v.mpv which is video is used. Then, the information from time 00:00:10 to 00: 000: 40 of audio http://mserv.com/MPEG/movie0a.mp2 is sent for 10 seconds and reproduced. Also, for the information from time 00:04:00 to 00:05:00 of http://mserv.com/MPEG/movie0v.mpv that is video, http://mserv.com/MPEG/ that is audio Information from movie0a.mp2 from 00:04:15 to 00:05:00 is played for 15 seconds.

  As described above, by using the begin attribute, it is possible to synchronize a plurality of media by shifting reproduction times between the plurality of media included in the structure description data.

  As described above, according to the first embodiment, the structure description data expressing the configuration of the media content can be converted into the expression description data expressing the reproduction form of the media content. Accordingly, distribution data suitable for the user or terminal can be created by appropriately processing or selecting the structure description data in the distribution of the media content.

  Further, according to the first embodiment, even if the structure description data is composed of a plurality of media, synchronization can be established between the media. Also, synchronization between media can be achieved by shifting the reproduction timing among a plurality of media.

  In the first embodiment, the description converter 1003 converts the structure description data representing the structure of the media content into the expression description data representing the reproduction form of the media content. The processing performed by 1003 may be programmed and read by a computer for execution.

  Alternatively, a program for causing a computer to execute the processing performed by the description converter 1003 may be stored in a storage medium.

(Embodiment 2)
The second embodiment describes a media segment and its alternative data in the structure description data in order to reproduce and distribute media content tailored to the terminal, and expresses the structure description data as a reproduction form of the media segment or alternative data. Conversion to expression description data is performed. Thereby, the structure description data described by a set of alternative data such as a representative image of a media segment that is a moving image can be converted into the expression description data of the alternative data. The second embodiment will be described below.

  FIG. 13, FIG. 14 and FIG. 15 are diagrams showing examples of structure description data in the present embodiment. In the second embodiment, Extensible Markup Language (XML) is used as an example of expressing structure description data on a computer. FIG. 13 is a DTD for describing structure description data in XML. FIG. 14 is an example of structure description data for media content in which video and audio are synchronized, taking MPEG1 as an example. FIG. 15 is an example of structure description data of media content in which moving images and audio are different media.

  First, the Document Type Definition (DTD), which is a definition for describing structure description data in XML, will be described with reference to FIG.

  As indicated by reference numeral 1301 in the figure, the contents element is composed of a par element and a mediaObject element. Further, as indicated by reference numeral 1302 in the figure, the contents element has a title attribute indicated by character data. Further, as indicated by reference numeral 1303 in the figure, the par element is composed of mediaObject elements that are a plurality of child elements.

  Further, as indicated by reference numeral 1304 in the figure, the mediaObject element is composed of segment elements. Further, as indicated by reference numeral 1305 in the figure, the mediaObject element is designated by the type attribute of the media type. In this example, the media type is audio, which is audio information, video, which is video information, image, which is still image information, audiovideo, which is information in which audio and video are synchronized, and audioimage, which is audio and still image information. A type is specified. If the type attribute is not specified, the type attribute is set to audiovideo by default.

  Also, as indicated by reference numeral 1306 in the figure, the mediaObject element is a format element that specifies a media format such as MPEG1 or MPEG2 for a moving image and a format such as gif or jpeg for a still image. Also, as indicated by reference numeral 1307 in the figure, the mediaObject element specifies the location where the data is stored by the src attribute. The location where data is stored can be specified by specifying Uniform Resource Locator (URL) in src attribute.

  Further, as indicated by reference numeral 1308 in the figure, the start attribute specifies the time inside the medium specified by the mediaObject element corresponding to the start time of the segment element. The end attribute specifies the time inside the media specified by the mediaObject element corresponding to the end time of the segment element.

  In the present embodiment, the time information of the media segment is specified by a set of start time and end time, but may be expressed by a set of start time and duration.

  Further, as indicated by reference numeral 1309 in the drawing, the segment element has an alt element. The alt element represents alternative data of the corresponding media segment. Then, as indicated by reference numeral 1310 in the figure, the alt element designates a media type such as image or audio by the type attribute. In the alt element, a media format such as gif or jpeg is specified for a still image by the format attribute. In the alt element, the saved location is specified by the src attribute.

  Also, a plurality of alt elements can be specified for each media segment, and in the case of the same media, playback is performed in the order of appearance.

  The alt element has a child element pos. The alt element can specify which section in the data specified by the src attribute by the child element pos. The start attribute and end attribute of the pos element represent the start time and end time inside the media indicated by the src attribute.

  In the present embodiment, the time information is specified by a set of start time and end time, but may be expressed by a set of start time and duration.

  Next, an example of structure description data for media content in which moving images and audio are synchronized will be described using MPEG1 as an example with reference to FIG.

  In the structure description data shown in FIG. 14, the title “Movie etc” is specified in the contents element. In the mediaObject element type, audiovideo is specified, MPEG1 is specified as the format, and http://mserv.com/MPEG/movie0.mpg is specified as the storage location. The mediaObject element includes a segment element having time information from time 00:00:00 to 00:01:00, a segment element having time information from time 00:01:00 to 00:02:00, and a time 00 Segment elements having time information from 03:00 to 00:04:00 and segment elements having time information from time 00:04:00 to 00:05:00. That is, the mediaObject element has a description excluding the time 00:02:00 to 00:03:00.

  Further, the segment element having time information from time 00:00:00 to 00:01:00 is indicated by an alt element that is alternative data of audiovideo. The segment element with time information from time 00:00:00 to 00:01:00 is an alt element with type image, format jpeg, and storage location http://mserv.com/IMAGE/s0.jpg , Type is audio, format is mpeg1, save location is http://mserv.com/MPEG/movie0.mp2, alt element with time information from time 00:00:00 to 00:01:00 ing.

  A segment element with time information from time 00:01:00 to 00:02:00 is an alt whose type is image, format is jpeg, and storage location is http://mserv.com/IMAGE/s1.jpg An alt element with time information from time 00:01:00 to 00:01:30, element, type audio, format mpeg1, save location http://mserv.com/MPEG/movie0.mp2, It is configured.

  A segment element with time information from time 00:03:00 to 00:04:00 is an alt whose type is image, format is jpeg, and storage location is http://mserv.com/IMAGE/s3.jpg From an element, an alt element with time information of type audio, format mpeg1, save location http://mserv.com/MPEG/movie0.mp2, time 00:03:00 to 00:03:30 It is configured.

  A segment element with time information from time 00:04:00 to 00:05:00 is an alt whose type is image, format is jpeg, and storage location is http://mserv.com/IMAGE/s4.jpg From an element, an alt element with type audio, format mpeg1, save location http://mserv.com/MPEG/movie0.mp2, time information from 00:04:00 to 00:05:00 It is configured.

  Next, an example of structure description data of media content in which moving images and audio are different media will be described with reference to FIG.

  In the structure description data shown in FIG. 15, the title “Movie etc” is specified in the contents element. In the example of FIG. 15, the contents element includes a mediaObject element of the video type and a mediaObject element of the audio type. Therefore, the par element synchronizes the mediaObject element whose type is audio and the mediaObject element whose type is video.

  The mediaObject element of type video specifies MPEG1 as the format and http://mserv.com/MPEG/movie0v.mpv as the storage location. The mediaObject element of type video has a segment element with time information from time 00:00:00 to 00:01:00 and a segment element with time information from time 00: 01: 0 to 00:02:00 And a segment element having time information from time 00:03:00 to 00:04:00 and a segment element having time information from time 00:04:00 to 00:05:00. In other words, the mediaObject element of type video is a description excluding the time 00:02:00 to 00:03:00.

  Further, the segment element having time information from time 00:00:00 to 00:01:00 is indicated by an alt element that is alternative data of video. The segment element with time information from time 00:00:00 to 00:01:00 has an alt element with type image, format jpeg, and storage location http://mserv.com/IMAGE/s0.jpg Have been instructed. A segment element with time information from time 00:01:00 to 00:02:00 is an alt whose type is image, format is jpeg, and storage location is http://mserv.com/IMAGE/s1.jpg The element is indicated. A segment element with time information from time 00:03:00 to 00:04:00 is an alt whose type is image, format is jpeg, and storage location is http://mserv.com/IMAGE/s3.jpg The element is indicated. A segment element with time information from time 00:04:00 to 00:05:00 is an alt whose type is image, format is jpeg, and storage location is http://mserv.com/IMAGE/s4.jpg The element is indicated.

  The mediaObject element of the type “audio” specifies MPEG1 as the format and “http://mserv.com/MPEG/movie0a.mp2” as the storage location. The mediaObject element of type audio has a segment element with time information from time 00:00:00 to 00:01:00 and a segment element with time information from time 00: 01: 0 to 00:02:00. And a segment element having time information from time 00:03:00 to 00:04:00 and a segment element having time information from time 00:04:00 to 00:05:00. That is, the mediaObject element whose type is audio is a description excluding the time 00:02:00 to 00:03:00.

  Further, the segment element having time information from time 00:00:00 to 00:01:00 is indicated by the alt element which is audio alternative data. The segment element with time information from time 00:00:00 to 00:01:00 has type audio, format mpeg1, save location http://mserv.com/MPEG/movie0.mp2, time 00:00 An alt element having time information from: 00 to 00:01:00 is indicated. Also, the segment element with time information from time 00:01:00 to 00:02:00 has type audio, format mpeg1, save location http://mserv.com/MPEG/movie0.mp2, time 00 An alt element having time information from 01:00 to 00:01:30 is indicated. Also, the segment element with time information from time 00:03:00 to 00:04:00 has type audio, format mpeg1, storage location http://mserv.com/MPEG/movie0.mp2, time 00 An alt element having time information from 03:00 to 00:03:30 is indicated. Also, the segment element with time information from time 00:04:00 to 00:05:00 has type audio, format mpeg1, save location http://mserv.com/MPEG/movie0.mp2, time 00 An alt element having time information from 04:00 to 00:05:00 is indicated.

  Also in the present embodiment, SMIL is used as the expression description data as in the first embodiment. The output of the SMIL document that reproduces each media segment itself is the same as in the first embodiment.

  Hereinafter, a process in which description converter 1003 outputs a SMIL document for reproducing alternative data will be described. This is different from the flowchart of FIG. 4 in the first embodiment only in the processing of step S405 and step S412 for outputting the SMIL document. Therefore, processing different from that of the first embodiment will be described. First, the process of step S405 will be described with reference to FIG.

  First, the description converter 1003 outputs a SMIL header (step 1601). Next, description converter 1003 surrounds the entire media segment with <seq> and </ seq> (step S1602). The description converter 1003 determines whether there is alternative data of different media types for each enclosed media segment (step S1603).

  If the description converter 1003 determines in S1603 that there is no alternative data of different media types, it determines whether there are a plurality of alternative data (step S1604). When there are a plurality of alternative data, description converter 1003 surrounds the plurality of alternative data with <seq> and </ seq> (step S1605). On the other hand, the description converter 1003 performs the following processing for each alternative data without enclosing it with <seq> and </ seq> when there is only one alternative data.

  Next, description converter 1003 selects a corresponding element from the SMIL audio element, video element, img element, and the like in accordance with the type of alternative data (step S1606). Next, when the start attribute and the end attribute of the child element pos of the alt element are specified, the description converter 1003 sets the start attribute value to the SMIL clip-begin and the end attribute value to the clip-end attribute. (Step S1607). Then, the description converter 1003 sets a src attribute indicating a storage location for each alternative data (step S1608).

  On the other hand, if the description converter 1003 determines in S1603 that there is alternative data with a different media type, the description converter 1003 groups the alternative data with the same type of media type (step S1609).

  Next, description converter 1003 needs to examine alternative data having the longest duration in order to synchronize the end of reproduction of the group. Therefore, the description converter 1003 calculates the duration from the values of the start attribute and end attribute of the alternative data for each group (step S1610). However, if the media type is a still image (image) or the start attribute and end attribute are not specified, the duration of the alternative data is set to 0.

  Next, description converter 1003 sets the endsync attribute of the SMIL par element to the group with the longest duration so as to synchronize the end of playback (step S1611), and sets the entire group to <par> and </ par>. (Step S1612), and the process of S1604 is performed for each group of each media type.

  The endsync attribute is used when a plurality of media surrounded by <par> and </ par> are played back and displayed in parallel and the duration is different depending on the media. In other words, the endsync attribute is used to specify which media should be matched with the end of playback / display of all media in such a case. There are several methods for specifying media in the endsync attribute. In this embodiment, a method for specifying by the media id is used. Specifically, an id that is an identifier is assigned to an attribute of a certain type of media. Then, by setting the endsync attribute = id, the media in the same group as the media to which the id is assigned end in synchronization with the end time of the media to which the id is assigned.

  As a result, even for media that does not have a duration such as a still image and for which the display time is not specified by an attribute such as dur, the playback end time of this media is the same as that of the media with id. Can be the same as the playback time. For example, a still image can be continuously displayed while audio media is being played.

  FIG. 17 shows a SMIL document output by the above processing from the structure description data shown in FIG.

  A plurality of groups 1701 to 1704 are described in the SMIL document of FIG. The group indicated by 1701 includes alternative data whose type is image, format is jpeg and storage location is http://mserv.com/IMAGE/s0.jpg, type is audio, format is mpeg1 and storage location is http://mserv.com/MPEG/movie0.mp2 and alternative data having time information from 00:00:00 to 00:01:00. Further, a0 is assigned as the id attribute to the alternative data of the type audio. In the group 1701, id (a0) is set in the endsync attribute. Thereby, the reproduction end time of the alternative data included in the group 1701 is matched with the alternative data whose type is audio. That is, the alternative data of the type image is continuously reproduced during the reproduction time of the alternative data of the type audio.

  Note that description of the groups 1702 to 1704 is omitted.

  Next, the process of step S412 is demonstrated using FIG. First, the description converter 1003 outputs a SMIL header (step S1801). Next, description converter 1003 surrounds the entire media segment with <seq> and </ seq> (step S1802).

  Then, the description converter 1003 groups the alternative data belonging to the same mediaObject element in the chronological order of the group of media segments (step S1803), and calculates the duration from the values of the start attribute and end attribute for each group (step S1803). Step 1804). However, if the media type is a still image (image) or the start attribute and end attribute are not specified, the duration of the alternative data is set to 0.

  Next, the description converter 1003 sets the endsync attribute of the par element of SMIL so as to synchronize the end of playback with the group with the longest duration, and encloses the whole in <par> and </ par> ( Step S1805).

  Next, description converter 1003 determines whether there are a plurality of alternative data (step S1806). If there are a plurality of alternative data, description converter 1003 surrounds the plurality of alternative data with <seq> and </ seq> (step S1807). On the other hand, the description converter 1003 performs the following processing for each alternative data without enclosing it with <seq> and </ seq> when there is only one alternative data.

  Next, description converter 1003 selects a corresponding element from the SMIL audio element, video element, img element, and the like in accordance with the type of alternative data (step S1808). Next, when the start attribute and the end attribute of the child element pos of the alt element are specified, the description converter 1003 sets the start attribute value to the SMIL clip-begin attribute and the end attribute value to the clip-end. The attribute is set (step S1809). Then, the description converter 1003 sets the src attribute indicating the storage location for each alternative data (step S1810).

  Note that the SMIL document output by the process shown in FIG. 18 from the structure description data shown in FIG. 14 is the same as that shown in FIG.

  In addition, in order to synchronize each clip in the par element in the SMIL document, it may be necessary to make the playback start times different. In this case, it is necessary to calculate the playback start time of each clip and start playback when that time arrives.

  For such purposes, SMIL provides an attribute called “begin” in the audio element, video element, img element, and ref element, which can be realized by using these attributes.

  As described above, according to the second embodiment, the configuration of the whole or part of the media content is described by the time information of the media segment and a set of alternative data such as a representative image if the media segment is a moving image. The structure description data can be converted into expression description data representing the reproduction order, reproduction timing, and synchronization information of the media segment described in the structure description data or its alternative data.

  As a result, it is possible to generate information related to playback of display media tailored to the terminal from information related to the configuration of the media content. As a result, in the distribution of media content, distribution data suitable for the terminal can be created.

(Embodiment 3)
In the third embodiment, a media segment and its alternative data are described in the structure description data and data for switching the media segment and alternative data in accordance with the terminal in order to play and distribute the media content that matches the terminal. It is. Then, the structure description data is converted into expression description data that is expressed by switching the media segment and the alternative data according to the terminal.

  The third embodiment of the present invention will be described below. The expression description data of the third embodiment describes and outputs two types of media segment reproduction and alternative data reproduction in one SMIL document. As the structure description data, the data shown in FIGS. 14 and 15 is used.

  The expression description data output according to the present embodiment describes both a case where a media segment is reproduced and a case where alternative data is reproduced. When media content is reproduced based on this, it is necessary to select whether to reproduce a media segment or alternative data. Therefore, the conditions for selection are described in the expression description data.

  Since the selection condition can be described by a switch element in SMIL, the SMIL document is used as the expression description data also in the present embodiment. The switch element is to select one that meets the conditions from multiple media. The selection is evaluated in the order of the media written in the contents of the switch element, and the media that meets the conditions is selected first. Conditions are attached to the media attributes written in the contents of the switch element, such as system-bitrate and system-caption.

  In the present embodiment, the connection bit rate of the network for delivering the media content is a condition. Specifically, when the connection bit rate is 56 kbps or higher, the media segment is played back. When the connection bit rate is lower than 56 kbps, alternative data is played back.

  In the following, a process in which the description converter 1003 outputs a SMIL document that reproduces a media segment or alternative data will be described. This is different from the flowchart of FIG. 4 in the first embodiment only in a part of processing of step S405 and step S412 for outputting a SMIL document. Therefore, the process corresponding to step S405 or step S412 will be described with reference to FIG.

  First, the description converter 1003 outputs a SMIL header (step 1901). Next, description converter 1003 surrounds the entire medium with <switch> and </ switch> (step S1902). The description converter 1003 encloses the media segment with <seq> and </ seq> (step S1903), and sets the system-bitrate attribute of the seq element to system-bitrate = “56000” (step S1904).

  The system-bitrate attribute is used for condition evaluation in the switch element, and designates the bandwidth that can be used by the system in the number of bits per second. If the value is greater than or equal to this value, it is determined that the switch element meets the condition. In the above example, if the bit rate is 56000 bps or more, it is determined that the condition is met. If this condition match is the first condition match in the switch element, the media with the matched condition is selected.

  Next, the description converter 1003 performs the processing of S503 to S505 shown in FIG. 5 or S903 to S908 shown in FIG. 9 (step S1905). This outputs a SMIL document that reproduces the media segment.

  In this case, the processing procedure of step S405 or step S412 in the first embodiment can be used by ignoring the alt element representing the substitute data.

  Next, description converter 1003 does not set the system-bitrate attribute of the seq element, and surrounds this alternative data with <seq> and </ seq> (step S1906), and S1603 in FIG. 16 shown in the second embodiment. To S1612 or the processing procedure of S1803 to S1810 in FIG. 18 is performed (step S1907). As a result, the description converter 1003 outputs a SMIL document that reproduces the substitute data.

  In this way, it is possible to create a SMIL document in which it is possible to select whether to reproduce a media segment or alternative data.

  FIG. 20 shows an SMIL document output according to the third embodiment. A switch element 2000 is described in the SMIL document shown in FIG. 20, and the switch element has two seq elements 2001 and 2002 as contents. One seq element 2001 is the part from <seq system-bitrate = ”56000”> to the first </ seq>, and the other seq element 2002 is the part from <seq> to </ seq> below it. is there. This switch element evaluates <seq system-bitrate = ”56000”>. If the bit rate available to the system to be used is 56000 bps or more, the seq element 2001 is selected to satisfy this condition. If the bit rate available to the system is less than 56000 bps, the seq element 2001 is not selected and the seq element 2002 is evaluated.

  A seq element 2001 is a part indicating that the media segment is reproduced, and a seq element 2002 is a part indicating that the alternative data is reproduced. Therefore, if the bit rate that can be used by the system is 56000 bps or more, the media segment is reproduced, and if it is less than 56000 bps, the alternative data is reproduced.

  In the present embodiment, the connection bit rate of the network is used as a condition for selecting whether to reproduce the media segment or the alternative data, but other conditions may be used. In this case, however, there are conditions in which the SMIL switch element cannot be used, so it is necessary to define expression description data that extends the SMIL switch element.

  In addition, as shown in FIG. 21A, the alt of the structure description data is expanded to have a child element called “condition” that describes the condition for using the alternative data specified here, and the condition specified by the “condition” element Depending on the case, it may be divided.

  FIG. 21B shows structure description data using a child element called condition. The structure description data shown in FIG. 21B means that the expression description data is configured so that data on one line is used in the narrow band.

  In addition, in order to synchronize each clip in the par element in the SMIL document, it may be necessary to make the playback start times different. In this case, it is necessary to calculate the playback start time of each clip and start playback when that time arrives.

  For such purposes, SMIL provides an attribute called “begin” in the audio element, video element, img element, and ref element, which can be realized by using these attributes.

  As described above, according to the third embodiment, the whole or part of the media content is described by the time information of the media segment and a set of alternative data such as a representative image if the media segment is a moving image. Information indicating that the media segment described in the structure description data and its alternative data are reproduced from the structure description data, the playback timing, the playback timing, the synchronization information, and either the media segment or the alternative data for playback. Conversion to expression description data to be expressed is possible. As a result, it is possible to generate information relating to reproduction including selection of a media segment or alternative data in accordance with the terminal from information relating to the configuration of the media content.

(Embodiment 4)
In the fourth embodiment, with regard to continuous audiovisual information (media content) in which video information and audio information are synchronized, in order to reproduce and distribute only a representative part of media content such as a synopsis or a highlight scene, For the content, the configuration of the corresponding media content is expressed by a set of each segment (media segment) into which the media content is divided, and the importance based on the time information of each media segment and the context content of the corresponding media segment is described The structure description data and the threshold value of importance based on the context contents are input, and only the media segment equal to or higher than the threshold value is selected from the structure description data. Then, the media segment selected from the structure description data is converted into representation description data that represents the playback order and playback timing of the media segments as a playback mode, and is output.

  This allows you to select only the media segments that make up the synopsis or highlight scene by selecting only the media segments with the highest importance from the information related to the configuration of the media content. It can be converted into descriptive data.

  Embodiment 4 of the present invention will be described below. The fourth embodiment relates to a configuration in which alternative data for a media segment is not specified. FIG. 22 shows a block diagram of a data processing apparatus according to the fourth embodiment. In FIG. 22, what is indicated by reference numeral 1501 is a summary engine as selection means. What is described by 1502 is a description converter which is a conversion means. Reproduction method 1503 is input data and content description that is structure description data, 1504 is a selection condition, 1505 is output and expression description data. It is a description.

  FIG. 23 shows the DTD of the structure description data used in the fourth embodiment. The DTD shown in FIG. 23 is obtained by adding an attribute “score 2301” indicating the importance based on the context contents of the media segment to the segment element of the DTD shown in FIG. This importance is represented by a positive integer value, and 1 is the lowest importance.

  Next, FIG. 24 shows an example of the content description 1503 which is the structure description data of the fourth embodiment.

  As indicated by 2401 in the figure, each segment is given a score attribute indicating importance.

  In the fourth embodiment, the importance of the media segment is used as the selection condition 1504. The summary engine 1501 selects a media segment on the condition that the importance of the media segment is equal to or greater than a certain threshold value. Hereinafter, processing of the summary engine 1501 as selection means will be described with reference to the flowchart of FIG.

  First, in step S2501, the summary engine 1501 extracts the first media segment described in the content description 1503, that is, the first segment element. Next, in step S2502, the summary engine 1501 extracts the score attribute of the segment element, which is the score of the extracted media segment, and checks whether it is greater than or equal to the threshold value. If the score attribute of the first media segment is greater than or equal to the threshold, the summarization engine 1501 proceeds to the process of step S2503, and if the score attribute of the first media segment is less than the threshold, the process proceeds to step S2504. Transition.

  In step S2503, the summarization engine 1501 outputs the start attribute and end attribute values of the segment element, which are the start time and end time of the corresponding media segment, to the description converter 1502 as conversion means.

  In step S2504, the summary engine 1501 checks whether there is an unprocessed media segment. If there is an unprocessed media segment, the summarization engine 1501 proceeds to the process of step S2505, and if there is no unprocessed media segment, ends the process.

  In step S2505, the summary engine 1501 extracts the first segment element from the unprocessed media segments, and proceeds to step S2502.

  The processing of the description converter 1502 serving as the conversion means is the same as the procedure for converting the structure description data into SMIL shown in FIG.

  In the fourth embodiment, the summary engine 1501 outputs the content of the selected media segment element to the description converter 1502, and the description converter 1502 performs processing using the content, but the summary engine 1501 selects the content. It is also possible to create intermediate structure description data that leaves only the media segment, and the description converter 1502 inputs the intermediate structure description data and performs processing.

  FIG. 26 shows an example of intermediate structure description data generated from the content description 1503, which is the structure description data of FIG.

  As can be seen from reference numeral 2601 in the figure, in the intermediate structure description data, only media segments having a threshold value of 4 or more are selected and described.

  In addition, as the selection condition, the importance of the media segment is not less than a certain threshold value, but the sum of the reproduction times of the selected media segment may be not more than a certain threshold value. In this case, the summary engine 1501 sorts all media segments in descending order of importance, and selects media segments from the sorted head until the total playback time is equal to or less than the threshold and becomes the maximum. Process. Further, a combination of the media segment importance condition and the reproduction time condition may be employed.

  As described above, according to the fourth embodiment, by selecting a media segment from the importance based on the context content of the media segment, a synopsis, a highlight scene collection, and the like are constructed, and their expression description data Can be generated. As a result, it is possible to reproduce and distribute the media content only for the portion desired by the user.

  A summary content description in which the segment playback time is changed may be created in accordance with the importance of the segment.

(Embodiment 5)
The fifth embodiment is limited to the one in which the video information and the audio information are one media object in the fourth embodiment, but includes the case where it is configured by synchronization with a plurality of media objects. It is a thing.

  Embodiment 5 of the present invention will be described below. The fifth embodiment relates to a configuration in which alternative data for a media segment is not specified. The block diagram of the data processing apparatus in the fifth embodiment is the same as that shown in FIG.

  Also in the fifth embodiment, the DTD for the structure description data 1503 is shown in FIG. FIG. 27 shows an example of a content description 1503 that is structure description data in the fifth embodiment.

  In the content description 1503 shown in FIG. 27, a mediaObject element 2701 whose type is video and a mediaObject element 2702 whose type is audio are described. As shown by reference numeral 2703 in the figure, a score attribute that is an importance level is described in a segment of a mediaObject element 2701 of type video. In addition, as shown by 2704 in the figure, a score attribute which is an importance level is described in a segment of a mediaObject element 2702 whose type is audio.

  Also in the fifth embodiment, it is assumed that the selection condition 1504 is greater than or equal to a certain threshold value of the media segment. In this case, the processing of the summary engine 1501, which is a selection unit, performs the processing of the summary engine 1501 in the fourth embodiment for each mediaObject element.

  FIG. 28 shows a flowchart of processing of the summary engine 1501 in the fifth embodiment.

  First, the summary engine 1501 extracts the first mediaObject element in step S2801. Next, in S2802, the summary engine 1501 extracts the segment element that is the head of the media segments that are the contents of the extracted mediaObject element. In step S2803, the summary engine 1501 extracts the score attribute value of the segment element indicating the score of the extracted media segment, and checks whether it is equal to or greater than the threshold value.

  The summary engine 1501 proceeds to the process of step S2804 when the extracted media segment is equal to or greater than the threshold value, and proceeds to the process of step S2805 when the extracted media segment is less than the threshold value. In step S2804, the summary engine 1501 outputs the start attribute and end attribute values of the segment element, which are the start time and end time of the corresponding media segment, to the description converter 1502.

  Next, in step S2805, the summary engine 1501 checks whether there is an unprocessed media segment. If there is an unprocessed media segment, the summary engine 1501 proceeds to the process of step S2806, and if there is no unprocessed media segment, the digest engine 1501 proceeds to the process of step S2805.

  In step S2806, the summary engine 1501 extracts the first segment element from unprocessed media segments, and proceeds to the process in step S2803.

  On the other hand, the summarization engine 1501 checks in step S2807 whether or not an unprocessed mediaObject element still remains, and moves to the process of step S2808 if it still remains, and ends the process if it does not remain. In step S2808, the summary engine 1501 extracts the first mediaObject element from the unprocessed mediaObject elements, and proceeds to the process in step S2802.

  The description converter 1502 which is the conversion means of the fifth embodiment also performs processing for each mediaObject element, but the same processing as the processing of the procedure for converting the structure description data to SMIL of FIG. 4 shown in the first embodiment. I do.

  In the fifth embodiment, the summary engine 1501 outputs the content of the element of the selected media segment to the description converter 1502, and the summary engine 1502 performs processing using it. The intermediate structure description data leaving only the selected media segment may be created, and the description converter 1502 may input the intermediate structure description data and perform processing.

  FIG. 29 shows an example of intermediate structure description data generated from the content description 1503 of FIG.

  As can be seen from 2901 in the figure, in the mediaObject element whose type is video, only media segments with a threshold value of 4 or more are selected and described. Also, as can be seen from 2902 in the figure, only the media segment having a threshold value of 4 or more is selected and described in the mediaObject element whose type is audio.

  In addition, it may be necessary to make the playback start times different in order to synchronize each clip in the par element in the SMIL document. In this case, it is necessary to calculate the playback start time of each clip and start playback when that time arrives.

  For such purposes, SMIL provides an attribute called “begin” in the audio element, video element, img element, and ref element, which can be realized by using these attributes.

  As described above, according to the fifth embodiment, by selecting a media segment based on the importance based on the context content of the media segment, a synopsis, a highlight scene collection, and the like are constructed, and their expression description data Can be generated. As a result, it is possible to reproduce and distribute the media content only for the portion desired by the user.

(Embodiment 6)
Embodiment 6 of the invention will be described. In the fourth embodiment, alternative data for the media segment is not specified, whereas in the sixth embodiment, alternative data for the media segment is specified. Further, the sixth embodiment relates to a configuration in which the summary engine does not select whether to reproduce the media segment or the alternative data.

  The block diagram of the data processing apparatus in the sixth embodiment is the same as that shown in FIG.

  An example of the DTD of the structure description data used in the sixth embodiment is shown in FIG. As indicated by reference numeral 3001 in the figure, the DTD shown in FIG. 30 is obtained by adding the attribute “score” representing the importance based on the context contents of the media segment to the segment element of the DTD shown in FIG. This importance is represented by a positive integer value, and 1 is the lowest importance.

  FIG. 31 shows an example of a content description 1503 that is structure description data. As can be seen from FIG. 31, a score attribute indicating importance is described in each segment composed of alternative data.

  The processing of summary engine 1501, which is the selection means in the sixth embodiment, is the same as the processing of summary engine in the fourth embodiment. However, the summary engine 1501, which is the selection means in the sixth embodiment, outputs the alt element as a child element in addition to the start attribute and end attribute of the segment element when outputting the selected media segment.

  The processing of the description converter 1502, which is the conversion means in the sixth embodiment, is the procedure of the procedure for converting the structure description data to SMIL in FIG. 4 shown in the first, second, and third embodiments. It is the same.

  In this embodiment, the summary engine 1501 outputs the content of the selected media segment element to the description converter 1502, and the description converter 1502 is configured to perform processing using the content, but the summary engine 1501 selects the content. It is also possible to create intermediate structure description data that leaves only the media segments that have been processed, and the description converter 1502 inputs the intermediate structure description data and performs processing.

  FIG. 32 shows an example of intermediate structure description data generated from the content description 1503, which is the structure description data of FIG.

  In the structure description data shown in FIG. 32, only a segment having an importance score attribute value of 4 or more and alternative data of this segment are selected and described.

(Embodiment 7)
Embodiment 7 of the present invention will be described. In the fifth embodiment, the alternative data for the media segment is not specified, whereas in the seventh embodiment, the alternative data for the media segment is specified. Further, the seventh embodiment relates to a configuration in which alternative data for a media segment is designated and selection of whether to reproduce a media segment or alternative data is not performed in the summary engine.

  The block diagram of the data processing device in the seventh embodiment is the same as that shown in FIG.

  Also in the seventh embodiment, the DTD shown in FIG. 30 is used as the DTD for the content description 1503 that is the structure description data. FIG. 33 shows an example of a content description 1503 which is structure description data in the seventh embodiment.

  The processing of summary engine 1501, which is the selection means in the seventh embodiment, is the same as the processing of summary engine 1501 in the fifth embodiment. However, when outputting the selected media segment, the summary engine 1501 according to the seventh embodiment outputs an alt element as a child element in addition to the start attribute and end attribute of the segment element.

  The processing of the description converter 1502 in the seventh embodiment is the same as the processing of the procedure for converting the structure description data into SMIL in FIG. 4 shown in the first, second, or third embodiment. is there.

  In this embodiment, the summary engine 1501 outputs the content of the selected media segment element to the description converter 1502, and the description converter 1502 is configured to perform processing using the content, but the summary engine 1501 selects the content. It is also possible to create intermediate structure description data that leaves only the media segments that have been processed, and the description converter 1502 inputs the intermediate structure description data and performs processing.

  The structure description data shown in FIG. 34 is an example of intermediate structure description data generated from the content description 1503 in FIG.

  In the structure description data shown in FIG. 34, a segment having an importance score attribute value of 4 or more and alternative data of this segment are described separately for each type of media.

(Embodiment 8)
In the eighth embodiment, only a representative portion of media content such as a synopsis of continuous audiovisual information (media content) in which video information and audio information are synchronized and a highlight scene is reproduced by a display medium in the terminal, It is for delivery. In other words, for the media content, the structure of the corresponding media content is expressed by a set of each segment (media segment) into which the media content is divided, and important based on the time information of each media segment and the context content of the corresponding media segment. The structure description data describing the degree and the threshold value of the importance based on the context contents are input, and only the media segment equal to or higher than the threshold value is selected. Then, either the media segment or alternative data of the corresponding media segment is selected as the playback mode of the selected media segment, converted into expression description data expressing the playback order and playback timing, and output.

  As a result, by selecting only the media segments with high importance from the information on the configuration of the media content, only the media segments constituting the synopsis or highlight scene are selected, and the expression description data relating to the playback of only the selected media segments is selected. Can be converted. Therefore, it is possible to realize the selection of media in accordance with the capability of the terminal that reproduces the media content and the situation of the network to be distributed.

  An eighth embodiment of the present invention will be described. In the sixth embodiment, the alternative data of the media segment is designated and the selection of whether to reproduce the media segment or the alternative data is not performed. In the eighth embodiment, the alternative data of the media segment is designated. The user selects whether to play the media segment or the alternative data. In the eighth embodiment, the selection means is divided into media segment selection means and playback media selection means. The selection conditions are divided into segment selection conditions and playback media selection conditions.

  FIG. 35 shows a block diagram of a data processing apparatus according to the eighth embodiment. In FIG. 35, reference numeral 2801 denotes a summary engine which is a media segment selection means. Also, what is indicated by 2800 is a description converter. The description converter 2800 includes a reproduction media selection unit 2802 as reproduction media selection means and a conversion unit 2803 as conversion means.

  What is described by 2804 is input data and content description which is structure description data, what is described by 2805 is segment selection condition, what is described by 2806 is playback media selection condition by 2807 A thing is an output and represents a reproduction method description having expression description data.

  In the eighth embodiment, the content description 2804 which is the structure description data is the same as the content description 1503 in the sixth embodiment. That is, the content description 2804 uses the DTD shown in FIG. 30, and an example is shown in FIG. The segment selection condition 2805 is the same as the selection condition 1504 in the fourth embodiment or the sixth embodiment. In this case, the process of the summary engine 2801 as the media segment selection means is the same as the process of the summary engine 1501 in the sixth embodiment.

  Next, processing of the reproduction media selection unit 2802 will be described. The playback media selection unit 2802 uses the connection bit rate of the network that delivers the media content as the playback media selection condition 2806. That is, the playback media selection unit 2802 plays the media segment when the connection bit rate is 56 kbps or higher, and plays back alternative data when the connection bit rate is lower than 56 kbps. The reproduction media selection unit 2802 checks the connection bit rate, determines which one to reproduce, and notifies the conversion unit 2803.

  The conversion unit 2803 inputs the element of the media segment selected by the summary engine 2801 as the media segment selection means and the result selected by the playback media selection unit 2802, and based on the result of the playback media selection unit 2802, the expression by SMIL A reproduction method description 2807 which is description data is output.

  The process of converting the content description 2804 into SMIL by the conversion unit 2803 is the same as the process of converting the structure description data to SMIL in FIG. 4 shown in the first embodiment or the embodiment.

  In this embodiment, the summary engine 2801 outputs the content of the selected media segment element to the conversion unit 2803, and the conversion unit 2803 performs processing using the content. The intermediate structure description data that leaves only the selected media segment may be created, and the conversion unit 2803 may input the intermediate structure description data and perform processing.

  Further, although the network bit rate is used as the playback media selection condition 2806, it may be the capability of the playback terminal, a request from the user, or the like.

(Embodiment 9)
Embodiment 9 of the present invention will be described. In the ninth embodiment, the alternative data of the media segment is designated and the selection of whether to reproduce the media segment or the alternative data is not performed. In the ninth embodiment, the alternative data of the media segment is designated. The user selects whether to play the media segment or the alternative data. The ninth embodiment relates to a configuration in which the selection means selects whether to reproduce a media segment or alternative data.

  Also in the ninth embodiment, as in the eighth embodiment, the selection means is divided into a media segment selection means and a reproduction media selection means. The selection conditions are divided into segment selection conditions and playback media selection conditions. Therefore, the block diagram of the data processing apparatus in the present embodiment is the same as that shown in FIG.

  In the ninth embodiment, the content description 2804 which is the structure description data is the same as the structure description data 1503 in the seventh embodiment. That is, the content description 2804 uses the DTD shown in FIG. 30, and an example of the content description 2804 is shown in FIG. The segment selection condition 2805 is the same as that in the eighth embodiment. Therefore, the process of summary engine 2801 is the same as the process of summary engine 1501 in the seventh embodiment.

  The processing of the playback media selection unit 2802 according to the ninth embodiment uses the same processing as that described in the eighth embodiment.

  The conversion unit 2803 inputs the element of the media segment selected by the summary engine 2801 and the result selected by the reproduction media selection unit 2802, and based on the result of the reproduction media selection unit 2802, a reproduction method that is expression description data by SMIL A description 2807 is output. The conversion process from the structure description data to SMIL performed by the conversion unit 2803 is the same as the conversion procedure from the structure description data to SMIL in FIG. 4 shown in the first or second embodiment.

  In this embodiment, the summary engine 2801 outputs the content of the selected media segment element to the conversion unit 2803, and the conversion unit 2803 performs processing using the content. The intermediate structure description data that leaves only the selected media segment may be created, and the conversion unit 2803 may input the intermediate structure description data and perform processing.

  Further, although the network bit rate is used as the playback media selection condition 2806, it may be the capability of the playback terminal, a request from the user, or the like.

(Embodiment 10)
The tenth embodiment is for reproducing and distributing only a representative portion of media content that matches the user's preference regarding continuous audiovisual information (media content) in which video information and audio information are synchronized. That is, in the tenth embodiment, the configuration of the corresponding media content is expressed by a set of each division (media segment) obtained by dividing the media content, and the time information of each media segment and the keyword are expressed. The structure description data that describes the importance of the corresponding media segment based on the viewpoint of the media, the viewpoint that matches the user's preference and the threshold of the importance, are input, and the media segment that exceeds the threshold Only to choose. Then, it is converted into expression description data expressing the playback order and playback timing of the media segments as the playback mode of the selected media segment, and output. As a result, only the media segment whose importance based on the corresponding viewpoint is equal to or higher than the threshold value is selected from the information related to the configuration of the media content, and converted into the expression description data relating to the reproduction of only the selected media segment. As a result, according to the importance based on the viewpoint, it is possible to configure a collection of highlight scenes and the like according to the user's preference, and to play and deliver only that portion.

  Embodiment 10 of the invention will be described below. The tenth embodiment relates to a configuration in which alternative data for a media segment is not specified. The block diagram of the data processing apparatus according to the tenth embodiment is the same as that shown in FIG.

  FIG. 36 shows the DTD of the structure description data used in the tenth embodiment. As indicated by 3601 in the figure, the DTD shown in FIG. 36 has an element called pointOfView in order to represent a score representing importance based on the viewpoint expressed by the keyword in the segment element of the DTD shown in FIG. It is added to become a child element.

  Further, as indicated by 3602 in the figure, the pointOfView element represents a viewpoint by the viewPoint attribute, and represents the importance based on the viewpoint indicated by the viewPoint attribute by the score attribute. This importance is represented by a positive integer value, and 1 is the lowest importance. Also, multiple pointOfView elements can be attached to one segment element. FIG. 37 shows an example of a content description 1503 that is structure description data used in the tenth embodiment.

  As can be seen from FIG. 37, for each segment element, a pointOfView element, an attribute viewPoint, and an attribute score are described.

  In the tenth embodiment, the selection condition 1504 assumes that, for a certain viewpoint, the importance of the media segment based on that viewpoint is equal to or greater than a threshold value. Further, there may be a plurality of viewpoints that serve as the selection condition 1504. FIG. 38 shows a flowchart of processing in the summary engine 1501 as selection means in this case.

  First, in step S3801, the summary engine 1501 extracts a segment element that is the head media segment. Next, in step S3802, the summary engine 1501 examines all pointOfView elements that are the contents of the segment element that is the extracted media segment. Then, the summary engine 1501 checks whether there is a viewpoint attribute specified in the selection condition 1504 in the viewPoint attribute of the checked pointOfView element.

  If there is one in which the viewpoint specified in the selection condition 1504 is specified, the summary engine 1501 performs processing in step S3803 to compare the importance based on the viewpoint specified in the selection condition 1504 with a threshold value. Migrate to On the other hand, if none of the viewpoints specified in the selection condition 1504 is specified, the summarization engine 1501 does not assign importance based on the viewpoint specified in the selection condition 1504, and therefore the processing in step S3805 is not performed. Transition.

  Next, in step S3803, the summary engine 1501 checks whether the importance based on the viewpoint specified by the selection condition 1504 is equal to or greater than a threshold value. If the importance based on the designated viewpoint is equal to or greater than the threshold value, the selection condition 1504 proceeds to the process of step S3804. If the importance is less than the threshold value, the process proceeds to step S3805.

  In step S3804, the summary engine 1501 outputs the start attribute and end attribute values of the segment element indicating the start time and end time of the corresponding media segment to the description converter 1502. In step S3805, the summary engine 1501 checks whether there is an unprocessed media segment. If there is, the summary engine 1501 proceeds to the processing in step S3806. On the other hand, if there is no unprocessed media segment, the summary engine 1501 ends the process.

  In step S3806, the summary engine 1501 extracts the first segment element from unprocessed media segments, and proceeds to the process in step S3802.

  The processing of the description converter 1502 is the same as the conversion procedure from the structure description data to SMIL in FIG. 4 shown in the first embodiment.

  In the tenth embodiment, the summary engine 1501 outputs the content of the selected media segment element to the description converter 1502, and the description converter 1502 performs processing using the content, but the summary engine 1501 selects the content. It is also possible to create intermediate structure description data that leaves only the media segments that have been processed, and the description converter 1502 inputs the intermediate structure description data and performs processing.

  In addition, the selection condition is that the importance regarding a certain viewpoint of the media segment is equal to or higher than a certain threshold value, but the total sum of reproduction times of the selected media segments may be equal to or less than a certain threshold value. In this case, the summarization engine 1501 sorts all media segments in descending order of importance with respect to the designated viewpoint, and selects media segments from the sorted head until the total playback time is equal to or less than the threshold value and reaches the maximum. Processing to select is performed.

  In addition, when there are a plurality of designated viewpoints, the summarization engine 1501 may take the maximum importance of the designated viewpoints for each media segment, and may sort by the value, or may sum them up. Alternatively, an arithmetic average may be taken and sorted by that value.

  Further, it may be a combination of the importance condition and the playback time condition regarding a certain viewpoint of the media segment.

  As described above, according to the tenth embodiment, by selecting only the media segment of interest to the user from the importance based on the viewpoint expressed by the keyword added to the media segment, You can create synopses and highlight scene collections according to your preferences, and generate expression description data for them. As a result, it is possible to reproduce and distribute the media content only for the portion desired by the user.

(Embodiment 11)
Hereinafter, an eleventh embodiment of the present invention will be described. While the tenth embodiment does not have a plurality of types of media, the eleventh embodiment relates to a configuration having a plurality of types of media and no alternative data for the media segment is designated. The block diagram of the data processing apparatus in the eleventh embodiment is the same as that shown in FIG.

  Also in the eleventh embodiment, as the DTD for the content description data 1503, the one shown in FIG. 36 is used. FIG. 39 shows an example of a content description 1503 that is structure description data in the eleventh embodiment.

  As can be seen from FIG. 39, the structure description data shown in FIG. 39 has mediaObject elements of different types, and for each segment element, a pointOfView element, its viewPoint and its attribute score are described.

  Also in the embodiment, the selection condition 1504 is the same as in the tenth embodiment, and regarding a certain viewpoint, the importance of the media segment based on that viewpoint is equal to or greater than a threshold value. Further, there may be a plurality of viewpoints that serve as the selection condition 1504. In this case, the summary engine 1501 performs the summary engine 1501 processing in the tenth embodiment for each mediaObject element. FIG. 40 shows a flowchart of processing of summary engine 1501 in the eleventh embodiment.

  First, the summary engine 1501 takes out the first mediaObject element in step S4001. Next, in step S4002, the summary engine 1501 extracts a segment element that is the first media segment from the contents of the extracted mediaObject element. In step S4003, the summary engine 1501 examines all the pointOfView elements that are the contents of the segment elements that are the extracted media segments, and the viewpoint designated by the selection condition 1504 is designated in the viewPoint attribute of the examined pointOfView elements. Find out what you have.

  If there is an aspect specified in the selection condition 1504 in the viewPoint attribute of the examined pointOfView element, the summarization engine 1501 determines the importance and threshold based on the viewpoint specified in the selection condition 1504. Therefore, the process proceeds to step S4004. On the other hand, the summary engine 1501 assigns the importance based on the viewpoint specified in the selection condition 1504 when there is no viewpoint specified in the selection condition 1504 in the viewPoint attribute of the examined pointOfView element. Therefore, the process proceeds to step S4006.

  In step S4004, the summary engine 1501 checks whether the importance based on the viewpoint designated by the selection condition 1504 is equal to or greater than a threshold value. If the importance based on the viewpoint specified in the selection condition 1504 is equal to or greater than the threshold value, the summary engine 1501 proceeds to the process of step S4005 and determines the importance based on the viewpoint specified in the selection condition 1504. If it is less than the threshold value, the process proceeds to step S4006.

  In step S4005, the summary engine 1501 outputs the start attribute and end attribute values of the segment element indicating the start time and end time of the corresponding media segment to the description converter 1502. In step S4006, the summary engine 1501 checks whether there is an unprocessed media segment. If there is an unprocessed media segment, the summary engine 1501 proceeds to the process of step S4007. If there is no unprocessed media segment, the summary engine 1500 proceeds to the process of step S4008.

  In step S4008, the summary engine 1501 checks whether or not an unprocessed mediaObject element still remains. If it remains, the summary engine 1501 proceeds to the process of step S4009. The summary engine 1501 ends the process when there is no unprocessed mediaObject element remaining.

  In step S4009, the summary engine 1501 extracts the first mediaObject element from the unprocessed mediaObject elements, and proceeds to the process of step S4002.

  The description converter 1502 of the eleventh embodiment also performs processing for each mediaObject element, but otherwise the same processing as the processing of the procedure for converting the structure description data to SMIL of FIG. 4 shown in the first embodiment I do.

  In the eleventh embodiment, the summary engine 1501 outputs the content of the selected media segment element to the description converter 1502, and the description converter 1502 performs processing using the content, but the summary engine 1501 selects the content. It is also possible to create intermediate structure description data that leaves only the media segments that have been processed, and the description converter 1502 inputs the intermediate structure description data and performs processing.

  In addition, it may be necessary to make the playback start times different in order to synchronize each clip in the par element in the SMIL document. In this case, it is necessary to calculate the playback start time of each clip and start playback when that time arrives.

  For such purposes, SMIL provides an attribute called “begin” in the audio element, video element, img element, and ref element, which can be realized by using these attributes.

(Embodiment 12)
Embodiment 12 of the present invention will be described. In the tenth embodiment, alternative data for the media segment is not specified, whereas in the twelfth embodiment, alternative data for the media segment is specified. The twelfth embodiment relates to a configuration in which the selection unit does not select whether to reproduce a media segment or alternative data. The block diagram of the data processing apparatus according to the twelfth embodiment is the same as that shown in FIG.

  An example of the DTD of the structure description data used in the twelfth embodiment is shown in FIG. The DTD shown in FIG. 41 is obtained by adding an element called pointOfView to the segment element of the DTD shown in FIG. 13 so as to represent a score representing importance based on the viewpoint expressed by the keyword so as to be a child element. . The pointOfView element represents a viewpoint by a viewPoint attribute, and represents an importance level based on the viewpoint indicated by the viewPoint attribute by a score attribute. This importance is represented by a positive integer value, and 1 is the lowest importance. Also, multiple pointOfView elements can be attached to one segment element. FIG. 42 shows an example of the content description data 1503.

  As can be seen from the figure, in the content description data shown in FIG. 42, an element called pointOfView is added as a child element to the segment element of the DTD. Also, the pointOfView element describes a viewPoint attribute and a score attribute.

  The processing of summary engine 1501 in the twelfth embodiment is the same as the processing of summary engine 1501 in the tenth embodiment. However, in the process of the summary engine 1501 in the twelfth embodiment, when the selected media segment is output, in addition to the start attribute and end attribute of the segment element, an alt element that is a child element is also output.

  The processing of the description converter 1502 in the twelfth embodiment is the same as the processing of the procedure for converting the structure description data into SMIL shown in FIG. 4 shown in the first, second, or third embodiment. is there.

  In the embodiment, the summary engine 1501 outputs the content of the element of the selected media segment to the description converter 1502 in the embodiment 10, and the description converter 1502 performs processing using it. However, the summary engine 1501 may create intermediate structure description data that leaves only the selected media segment, and the description converter 1502 may input the intermediate structure description data and perform processing.

(Embodiment 13)
A thirteenth embodiment of the present invention will be described. While the eleventh embodiment does not designate media segment alternative data, the thirteenth embodiment relates to media segment alternative data designated. The thirteenth embodiment relates to a configuration in which the selection unit does not select whether to reproduce a media segment or alternative data. The block diagram of the data processing apparatus according to the thirteenth embodiment is the same as that shown in FIG.

  Also in the thirteenth embodiment, as the DTD for the content description 1503, the one shown in FIG. 41 is used. 43 and 44 show examples of the structure description data 1503 in the twelfth embodiment.

  As can be seen from the figure, the structure description data of the thirteenth embodiment has different types of mediaObject elements, and each mediaObject element has a segment element. For each segment element, a pointOfView element, its attribute viewPoint, and an attribute score are described.

  The processing of summary engine 1501 in the twelfth embodiment is the same as the processing of summary engine in the eleventh embodiment. However, when outputting the selected media segment, the summary engine 1501 according to the twelfth embodiment outputs an alt element as a child element in addition to the start attribute and end attribute of the segment element.

  The processing of the description converter 1502 in the twelfth embodiment is the same as the processing of the procedure for converting the structure description data into SMIL shown in FIG. 4 shown in the first, second, or third embodiment. is there.

  In the eleventh embodiment, the summary engine 1501 outputs the content of the selected media segment element to the description converter 1502, and the description converter 1502 performs processing using the content, but the summary engine 1501 selects the content. It is also possible to create intermediate structure description data that leaves only the media segments that have been processed, and the description converter 1502 inputs the intermediate structure description data and performs processing.

(Embodiment 14)
Embodiment 14 of the present invention will be described. While the twelfth embodiment relates to a configuration in which the selection means does not select whether to reproduce the media segment or the alternative data, the fourteenth embodiment relates to whether the selection means reproduces the media segment or the alternative data. The present invention relates to a configuration for selecting whether or not to reproduce. In the fourteenth embodiment, the selection means is divided into media segment selection means and playback media selection means. The selection conditions are divided into segment selection conditions and playback media selection conditions. Therefore, the block diagram of the data processing apparatus according to the fourteenth embodiment is the same as that shown in FIG.

  In the fourteenth embodiment, the content description data 2804 is the same as the content description 1503 in the twelfth embodiment. That is, the content description data 2804 of the fourteenth embodiment uses the DTD shown in FIG. 41, and an example of the content description data 2804 of the fourteenth embodiment is shown in FIG.

  The segment selection condition 2805 is the same as the selection condition 1504 in the tenth or twelfth embodiment. In this case, the process of summary engine 2801 is the same as the process of summary engine 1501 in the twelfth embodiment.

  Next, processing of the reproduction media selection unit 2802 will be described. The playback media selection unit 2802 uses the connection bit rate of the network that delivers the media content as the playback media selection condition 2806. That is, the playback media selection unit 2802 plays the media segment when the connection bit rate is 56 kbps or higher, and plays back alternative data when the connection bit rate is lower than 56 kbps. The reproduction media selection unit 2802 checks the connection bit rate, determines which one to reproduce, and notifies the conversion unit 2803.

  The conversion unit 2803 inputs the element of the media segment selected by the summary engine 2801 as the media segment selection means and the result selected by the playback media selection unit 2802, and based on the result of the playback media selection unit 2802, the expression by SMIL A reproduction method description 2807 which is description data is output.

  The process of converting the content description 2804 into SMIL by the conversion unit 2803 is the same as the process of converting the structure description data to SMIL in FIG. 4 shown in the first embodiment or the second embodiment.

  In this embodiment, the summary engine 2801 outputs the content of the selected media segment element to the conversion unit 2803, and the conversion unit 2803 performs processing using the content. The intermediate structure description data that leaves only the selected media segment may be created, and the conversion unit 2803 may input the intermediate structure description data and perform processing.

  Further, although the network bit rate is used as the playback media selection condition 2806, it may be the capability of the playback terminal, a request from the user, or the like.

(Embodiment 15)
Embodiment 15 of the present invention will be described. While the thirteenth embodiment relates to a configuration for selecting whether to reproduce a media segment or alternative data in the selection means, the fifteenth embodiment relates to a selection of reproducing a media segment or alternative data in the selection means. The present invention relates to a configuration for selecting whether to reproduce. Also in the fifteenth embodiment, as in the eighth embodiment, the selection means is divided into a media segment selection means and a reproduction media selection means. The selection conditions are divided into segment selection conditions and playback media selection conditions. Therefore, the block diagram of the data processing apparatus in the present embodiment is the same as that shown in FIG.

  The structure description data 2804 of the fifteenth embodiment is the same as the structure description data 1503 of the thirteenth embodiment. That is, the structure description data 2804 of the fifteenth embodiment uses the DTD shown in FIG. 41, and an example is shown in FIG. 43 and FIG.

  The segment selection condition 2805 of the fifteenth embodiment is the same as the segment selection condition 2805 of the fourteenth embodiment. Therefore, the process of summary engine 2801 is the same as the process of summary engine 1501 in the thirteenth embodiment.

  The processing of the playback media selection unit 2802 according to the fifteenth embodiment uses the same processing as that of the playback media selection unit 2802 described in the fourteenth embodiment.

  The conversion unit 2803 of the fifteenth embodiment inputs the element of the media segment selected by the summary engine 2801 and the result selected by the reproduction media selection unit 2802, and based on the result of the reproduction media selection unit 2802, the expression description by SMIL A reproduction method description 2807 which is data is output.

  The conversion process to SMIL performed by the conversion unit 2803 of the fifteenth embodiment is the same as the conversion procedure from the structure description data shown in FIG. 4 to the SMIL shown in the first or second embodiment.

  In this embodiment, the summary engine 2801 outputs the content of the selected media segment element to the conversion unit 2803, and the conversion unit 2803 performs processing using the content. The intermediate structure description data that leaves only the selected media segment may be created, and the conversion unit 2803 may input the intermediate structure description data and perform processing.

  Further, although the network bit rate is used as the playback media selection condition 2806, it may be the capability of the playback terminal, a request from the user, or the like.

(Embodiment 16)
A sixteenth embodiment of the present invention will be described. FIG. 45 shows a block diagram of a data processing apparatus in the sixteenth embodiment. In FIG. 45, 3801 indicates a structure description data database, 3802 indicates a selection unit, 3803 indicates a conversion unit, 3804 indicates a reproduction unit, What is described by 3805 represents a media content database. Also, what is written in 3806 is structure description data, what is written in 3808 is summary content description data, what is written in 3809 is expression description data, and what is written in 3810 is media content data. To express.

  The selection unit 3802, the conversion unit 3803, the structure description data 3806, the selection condition 3807, and the expression description data 3809 are the same as those described in any of Embodiments 4 to 15. The summary structure description data 3808 corresponds to intermediate structure description data in which only the selected media segment is left as shown in any of the fourth to fifteenth embodiments. The selection unit 3802 and the conversion unit 3803 can be realized by executing a program on a computer.

  As the playback unit 3804, since the description description data 3809 is expressed in SMIL, a SMIL player can be used. A SMIL player can be realized by executing a program on a computer, and free software such as Real Player of Real Networks is distributed as SMIL player software.

  In the sixteenth embodiment, the selection unit 3802 outputs the summary structure description data 3808. However, as shown in any of the fourth to fifteenth embodiments, the summary structure description data 3808 is output. Alternatively, the selected media segment may be output.

(Embodiment 17)
A server-client system according to Embodiment 17 of the present invention will be described with reference to FIG. In the seventeenth embodiment, a selection unit 3802 and a conversion unit 3803 are provided on the server 4601 side, and a playback unit 3804 is provided on the client 4602 side. In the seventeenth embodiment, the conversion unit 3803 and the reproduction unit 3804 are connected on the network 4603. Thus, the seventeenth embodiment becomes a server-client system configured to communicate the expression description data 3809 through a network.

  The processing contents performed by each processing unit are written as programs that can be executed by a computer, stored in storage media on the server 4601 side and the client 4602 side, and executed respectively.

  Instead of the structure description database 3801, the metadata database 1001, the summary engines 1002, 1501, 2801 instead of the selection unit 3802, the description converters 1003, 1502, 2800 instead of the conversion unit 3803, and the playback unit 3804 are replaced. Alternatively, the player 1004 may use the media content database 1005 instead of the media content database 3805.

  In the seventeenth embodiment, as shown in FIG. 47, the media content database 3805 may be provided in the server 4601a, and the media content data 3810 may be sent to the client 4602a via the network 4603.

(Embodiment 18)
A server-client system according to the eighteenth embodiment of the present invention will be described.

  An eighteenth embodiment will be described with reference to FIG. In the eighteenth embodiment, a selection unit 3802 is provided on the server 4701 side, and a conversion unit 3803 and a reproduction unit 3804 are provided on the client 4702 side. In the eighteenth embodiment, the selection unit 3802 and the conversion unit 3803 are connected on the network 4603. Thus, the eighteenth embodiment is a server-client system configured to communicate the summary structure description data 3808 via a network.

  The processing contents performed by each processing unit are written as a program that can be executed by a computer, stored in storage media on the server 4701 side and the client 4702 side, and executed.

  Instead of the structure description database 3801, the metadata database 1001, the summary engines 1002, 1501, 2801 instead of the selection unit 3802, the description converters 1003, 1502, 2800 instead of the conversion unit 3803, and the playback unit 3804 are replaced. Alternatively, the player 1004 may use the media content database 1005 instead of the media content database 3805.

  In the eighteenth embodiment, as shown in FIG. 49, the media content database 3805 may be provided in the server 4701a, and the media content data 3810 may be sent to the client 4702a via the network 4603.

  As described above, according to the present invention, it is possible to convert the structure description data describing the configuration of the media content by the media segment into the expression description data expressing the form for reproducing the media content. For this reason, in the playback of media content, restrictions such as playback timing and synchronization information for each media segment can be added, and various playbacks can be realized.

  Further, according to the present invention, it is possible to select whether to reproduce the media segment itself or the alternative data by describing the alternative data of the media segment in the structure description data. Accordingly, it is possible to deliver and play back content using media that matches the capacity and communication volume of the network that delivers the media content and the capability of the terminal that plays back the media content.

  In addition, according to the present invention, by further describing the score based on the context contents of each media segment in the structure description data, for example, it is possible to easily reproduce and distribute highlight scene collections having various reproduction times. In addition, since the score is based on the viewpoint indicated by the keyword, by designating the keyword, it is possible to reproduce and deliver only the scene according to the user's preference.

1 is a conceptual diagram of a data processing system according to a first embodiment of the present invention. (A) The figure which shows DTD of the structure description data in Embodiment 1 (b) The figure which shows an example of the structure description data in Embodiment 1 The figure which shows another example of the structure description data in Embodiment 1. Flow chart of conversion from structure description data to expression description data in the first embodiment The flowchart which the description converter concerning Embodiment 1 outputs the reproduction | regeneration method description which is a SMIL document from the summary content description which is structure description data Diagram showing the structure of a SMIL document FIG. 5 is a diagram illustrating an example of expression description data according to the first embodiment FIG. 5 is a diagram illustrating an example of expression description data according to the first embodiment The flowchart which the description converter concerning Embodiment 1 outputs the reproduction | regeneration method description which is a SMIL document from the summary content description which is structure description data FIG. 5 is a diagram illustrating an example of expression description data according to the first embodiment FIG. 5 is a diagram illustrating an example of expression description data according to the first embodiment FIG. 5 is a diagram illustrating an example of expression description data according to the first embodiment The figure which shows DTD of the structure description data in Embodiment 2 of this invention FIG. 10 shows an example of structure description data in the second embodiment The figure which shows another example of the structure description data in Embodiment 2. The flowchart which the description converter concerning Embodiment 2 outputs the reproduction | regeneration method description which is a SMIL document from the summary content description which is structure description data FIG. 10 is a diagram illustrating an example of expression description data according to the second embodiment The flowchart which the description converter concerning Embodiment 2 outputs the reproduction | regeneration method description which is a SMIL document from the summary content description which is structure description data Flow chart of conversion from structure description data to expression description data in the third embodiment FIG. 10 is a diagram illustrating an example of expression description data according to the third embodiment (A) The figure which shows DTD of the extension of the structure description data in Embodiment 3 (b) The figure which shows an example of the extension of the structure description data in Embodiment 3 The block diagram of the data processor in Embodiment 4 of this invention The figure which shows DTD of the structure description data in Embodiment 4 FIG. 10 is a diagram illustrating an example of structure description data according to the fourth embodiment. Flow chart in processing of selection unit in embodiment 4 FIG. 10 is a diagram illustrating an example of intermediate structure description data in the fourth embodiment The figure which shows an example of the structure description data in Embodiment 5 of this invention Flow chart in processing of selection unit in embodiment 5 The figure which shows an example of the intermediate structure description data in Embodiment 5 The figure which shows DTD of the structure description data in Embodiment 6 of this invention FIG. 18 shows an example of structure description data in the sixth embodiment. FIG. 18 is a diagram illustrating an example of intermediate structure description data in the sixth embodiment The figure which shows an example of the structure description data in Embodiment 7 of this invention FIG. 18 shows an example of intermediate structure description data in the seventh embodiment. The block diagram of the data processor in Embodiment 8 of this invention The figure which shows DTD of the structure description data in Embodiment 10 of this invention. A diagram showing an example of structure description data in the tenth embodiment Flow chart in processing of selection unit in embodiment 10 The figure which shows an example of the structure description data in Embodiment 11 of this invention Flow chart in processing of selection unit in embodiment 11 The figure which shows DTD of the structure description data in Embodiment 12 of this invention A diagram showing an example of structure description data in Embodiment 12 First diagram showing an example of structure description data in Embodiment 13 of the present invention FIG. 21 is a second diagram illustrating an example of structure description data according to the thirteenth embodiment. Block diagram of a data processing apparatus in Embodiment 16 of the present invention Block diagram of a server-client system according to a seventeenth embodiment of the present invention Block diagram of another example of server-client system of embodiment 17 18 is a block diagram of a server-client system according to an eighteenth embodiment of the present invention. Block diagram of another example of server-client system of embodiment 18 Block diagram of a conventional video distribution device

Explanation of symbols

1001 metadata database 1002, 1501, 2801 summary engine 1003, 1502, 2800 description converter 1004 player 1005 media content database 2801 media segment selection means 2802 playback media selection unit 2803 conversion unit 3801 structure description data database 3802 selection unit 3803 conversion unit 3804 Playback unit 3805 Media content database 4601, 4701 Server 4602, 4702 Client 4603 Network

Claims (10)

A plurality of media including content based on contextual content and time information consisting of a combination of playback start time and end time or duration time. Structure description data including synchronization information described in segments and representing that a plurality of media are reproduced in synchronization;
An input unit for receiving an input of a selection condition for selecting a predetermined media segment from the structure description data;
A selection unit that selects only a media segment having a score that meets the selection condition from the structure description data;
Synchronization is performed from the media type, the time information, and the synchronization information described for each media segment of summary structure description data that is intermediate structure description data that leaves only the media segment selected by the selection unit. Grouping different types of media segments to be played back, and setting the synchronization information indicating that playback is performed in synchronization with each group of the obtained media segments,
For the group of media segments, from the time information, as a playback order, set to arrange in the order of early group,
For each group of media segments, from the media type and the time information, as the playback timing, set the playback start time and end time of each media segment according to the media type, respectively.
An expression description in which the summary structure description data is arranged in the order of the set playback order for each group of media segments in which the synchronization information is set, and the playback timing set for each media segment is described. A data processing apparatus comprising: a conversion unit that converts to data and outputs the data.   The conversion unit integrates the media segments having continuous time information into one for each type of the media, resets the start time and end time of playback, and arranges the media segments in the playback order that is earlier in time. The data processing apparatus according to claim 1, wherein: The structure description data has a set of alternative data replacing the media segment,
The conversion unit sets the summary structure description data in the media segment describing the set synchronization information, sets at least one of the media segment and the alternative data, and arranges in the order of the set playback order, 2. The data processing apparatus according to claim 1, wherein the data processing device is converted into expression description data describing the set reproduction timing for each media segment.   4. The data processing apparatus according to claim 3, wherein the selection unit selects whether to reproduce the media segment or the alternative data when selecting the media segment.   5. The data processing apparatus according to claim 1, wherein the expression description data is a SMIL document.   6. The data processing apparatus according to claim 1, wherein the score is an importance level of the corresponding media segment based on the context content of the media content.   The data according to any one of claims 1 to 6, wherein a viewpoint expressed by a keyword is given to the media segment, and the score is an importance of the corresponding media segment based on the viewpoint. Processing equipment. An input unit that inputs structure description data describing the content of media content that is continuous audiovisual information, a selection unit that selects a predetermined media segment from the structure description data, and an intermediate structure that leaves only the selected media segment A data processing method in a data processing apparatus, comprising: a conversion unit that converts summary structure description data, which is description data, into expression description data and outputs the data,
The content of the media content, which is continuous audiovisual information, is divided by time for each type of media, and the score based on the time information and the context content consisting of a combination of the playback start time and end time or duration time Structure description data including synchronization information that is described in a plurality of media segments including and represents that the plurality of media are synchronized and played back;
A step of accepting a selection condition for selecting a predetermined media segment from the structure description data, enter the,
The selecting unit selecting only a media segment having a score that meets the selection condition from the structure description data;
The conversion unit is
Synchronization is performed from the media type, the time information, and the synchronization information described for each media segment of the summary structure description data that is intermediate structure description data that leaves only the media segment selected in the selection step. Grouping different types of media segments to be played back, and setting the synchronization information indicating that playback is performed in synchronization with each group of the obtained media segments,
For the group of media segments, from the time information, as a playback order, set to arrange in the order of early group,
For each group of media segments, from the media type and the time information, as the playback timing, set the playback start time and end time of each media segment according to the media type, respectively.
An expression description in which the summary structure description data is arranged in the order of the set playback order for each group of media segments in which the synchronization information is set, and the playback timing set for each media segment is described. Converting to data and outputting, and
A data processing method characterized by comprising:   The conversion unit collectively sets the media segments having continuous time information for each of the media types, resets the playback start time and end time, and arranges the media segments in the playback order that is earlier in time. 9. A data processing method according to claim 8, wherein: An input unit that inputs structure description data describing the content of media content that is continuous audiovisual information, a selection unit that selects a predetermined media segment from the structure description data, and an intermediate structure that leaves only the selected media segment A conversion unit that converts summary structure description data, which is description data, into expression description data and outputs the data, and a program used for a data processing apparatus,
Computer
In the input unit, the content of the media content, which is continuous audiovisual information, is divided by time for each type of media, and a score based on time information and contextual content consisting of a set of playback start time and end time or duration time Structure description data including synchronization information that is described in a plurality of media segments including and represents that the plurality of media are synchronized and played back;
A step of causing reception and selection condition for selecting a predetermined media segment from the structure description data, the input,
Causing the selection unit to select only media segments having a score that meets the selection condition from the structure description data;
In the converter,
Synchronization is performed from the media type, the time information, and the synchronization information described for each media segment of the summary structure description data that is intermediate structure description data that leaves only the media segment selected by the selection unit. Grouping different types of media segments to be played back, and setting the synchronization information indicating that playback is performed in synchronization with each group of the obtained media segments,
For the group of media segments, from the time information, as a playback order, set to arrange in the order of the group with the earliest time,
For each group of media segments, from the media type and the time information, the playback timing of each media segment according to the media type is set as the playback timing, respectively.
An expression description in which the summary structure description data is arranged in the order of the set playback order for each group of media segments in which the synchronization information is set, and the playback timing set for each media segment is described. Converting to data and outputting, and
A program for functioning as the data processing device by executing each of the above.

Images