JP2005166063A - Data processing apparatus, data processing method, recording medium, and program for making computer to execute the data processing method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To extract outline and a highlight scene or a scene which a viewer desires of media content. <P>SOLUTION: This data processing apparatus inputs context content description data describing a segment representing each of scenes of media contents composed of a plurality of scenes, time information representing sections of the scenes and a score based on the context contents of the media contents, selects the segment based upon the score, inputs the corresponding media contents and extracts the sections of the media contents from the time information of the selected segment. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

  In the present invention, in the viewing, playback, delivery, and accumulation of continuous audiovisual information (media content) such as video, video, and audio, the outline of the media content, the highlight scene, or only the scene that the viewer wants to see is displayed. The present invention provides a media content data processing apparatus, data processing method, recording medium, and program for reproduction and delivery.

  Conventionally, reproduction, delivery, and accumulation of media content have been performed in units of files that store media content.

  Also, as a method of searching for a specific scene of a moving image, as in Japanese Patent Laid-Open No. 10-111872, a change in the scene of a moving image (scene cut) is detected, and for each scene cut, the time code of the start frame, the end frame Time code and additional information on keywords for the scene.

  Alternatively, Carnegie Mellon University (CMU) has summarized videos by detecting scene cuts in videos, detecting human faces and captions, and detecting key phrases using voice recognition (Michael A. Smith, Takeo Kanade). , “Video Skimming and Characterization through the Combination of Image and Language Understanding Techniques”, CMU-CS-97-111, February 3, 1997).

  However, in the conventional method, when reproduction is performed in units of files, it is impossible to see a summary of the content. In addition, when searching for a highlight scene or a scene that the user wants to see, there is a problem that it is necessary to refer to the top of the content. Further, in the video delivery, there is a problem that it takes a lot of time to transmit all the file data.

  In addition, according to the method disclosed in Japanese Patent Laid-Open No. 10-111872, the search for a scene can be performed using a keyword, so that it is easy to search for a scene desired by a user. However, there is no relationship or connection between scenes in the additional information, and for example, processing when searching for one section of a story becomes difficult. In addition, since it is difficult to know which scene is important in context by searching only for keywords, it is difficult to create a synopsis or a collection of highlight scenes.

  Also, according to the CMU method, although the video can be summarized, the results are determined in a single way, so it is not possible to perform a summary with different playback times, such as a 5 minute summary and a 3 minute summary. Have difficulty. In addition, it is difficult to summarize a user's request such as selecting a scene in which a specific person is shown.

  An object of the present invention is to provide means for selecting, reproducing, and delivering only a synopsis or highlight scene, or only a scene desired by a viewer, in reproducing media content.

  It is another object of the present invention to provide means for selecting a synopsis, a highlight scene, a scene desired by a viewer, and the like so that the playback time is matched with a time desired by a user.

  Further, it is an object of the present invention to provide a means for delivering only contents such as a summary of a playback time desired by a user, a highlight scene collection, and a scene desired by the user upon delivery of the media content.

  It is another object of the present invention to provide means for adjusting the amount of data to be delivered according to the line status for communication between the server and the user.

  The summary creation device of the present invention is based on a segment representing each scene of media content composed of a plurality of scenes, time information representing scene breaks, which is attribute information of the segment, and context contents of the media content. Means for inputting context content description data describing a score, selection means for selecting a segment based on the score, content input means for inputting media content corresponding to the context content description data, and the selection Extracting means for extracting a section of the media content from the segment time information.

  With this configuration, it is possible to extract a synopsis of media content, a highlight scene, or a scene desired by the viewer.

  Further, the summary creation device of the present invention includes a segment representing each scene of media content composed of a plurality of scenes, time information representing scene breaks, which is attribute information of the segment, and at least one representing a scene. Means for inputting contextual content description data describing viewpoints represented by keywords and the score of the segment based on each viewpoint, and selection for selecting the segment based on at least one of the viewpoint and the score Means, content input means for inputting media content corresponding to the context content description data, and extraction means for extracting a section of the media content from the time information of the selected segment.

  With this configuration, it is possible to extract a synopsis of media content, a highlight scene, or a scene desired by the viewer.

  The summary creation device according to claim 1, further comprising storage means for storing the context content description data and the corresponding media content.

  In the summary creation device, a link destination of the corresponding media content is described in the context content description data, and the extraction unit extracts the link destination media content from the time information of the selected segment. Extract sections.

  In the summary creation device, the time information includes a start time and an end time of each scene.

  In the summary creation device, the time information includes a start time and a duration of each scene.

  In the summary creation device, a plurality of segments are hierarchically described in the context content description data.

  In the summary creation device, a plurality of sets of the viewpoint and the score are described as the attribute information in one segment.

  In the summary creation device, the selection unit selects a segment whose score is larger than a certain threshold.

  In the summary creation device, the selection unit selects the segments in descending order of the sum of the durations of the segments, which is the maximum when the sum of the durations of the segments is equal to or less than a threshold value.

  In the summary creation device, the selection unit selects the segment score in descending order of the sum of durations of the segments, which is close to a threshold value.

  Further, in the summary creation device, the selection unit selects a segment having a maximum sum of durations below a threshold value, in descending order of score of at least one selected viewpoint.

  In the summary creation device, the selection unit selects a segment in which the total sum of durations is in the vicinity of the threshold value in descending order of scores of at least one selected viewpoint.

  The summary creation method of the present invention is based on a segment representing each scene of media content composed of a plurality of scenes, time information representing scene breaks, which is attribute information of the segment, and context contents of the media content. Inputting context content description data in which a score is described; selecting a segment based on the score; inputting media content corresponding to the context content description data; and the selected segment Extracting the section of the media content from the time information.

  By this method, it is possible to extract a synopsis of media content, a highlight scene, or a scene desired by a viewer.

  In the summary creation method of the present invention, a segment representing each scene of media content composed of a plurality of scenes, time information representing scene breaks, which is attribute information of the segment, and at least one representing a scene A step of inputting context content description data in which a viewpoint represented by a keyword and a score of the segment based on each viewpoint are described; and a step of selecting the segment based on at least one of the viewpoint and the score And inputting the media content corresponding to the context content description data, and extracting a section of the media content from the time information of the selected segment.

  By this method, it is possible to extract a synopsis of media content, a highlight scene, or a scene desired by a viewer.

  The summary creation program of the present invention is a program for executing each step of the summary creation method. With this program, it is possible to extract a summary of media content, a highlight scene, or a scene desired by the viewer.

  The recording medium of the present invention is a computer-readable recording medium on which the above-described summary creation program is recorded. With this recording medium, it is possible to extract a synopsis of media content, a highlight scene, or a scene desired by the viewer.

  According to the present invention, it is possible to extract only the outline or highlight scene of the media content or only the scene desired by the viewer.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

[First Embodiment]
The first embodiment according to the present invention will be described below. In the present embodiment, an MPEG1 system stream moving image is assumed as media content. In this case, the media segment corresponds to one scene cut. In the present embodiment, the score is an objective importance based on the context contents in the corresponding scene.

  FIG. 1 is a block diagram of a data processing method according to the present embodiment. In FIG. 1, 101 represents a selection step and 102 represents an extraction step. The selection step 101 performs processing for selecting a scene of media content from the context content description data and outputting the start time and end time of the scene. In addition, the extraction step 102 performs processing for extracting data of a media content section that is divided by the start time and end time output by the selection step 101.

  FIG. 2 shows the configuration of the context content description data of the present embodiment. In the present embodiment, context contents are described in a tree structure. In addition, it is assumed that the sibling relationships of the tree structure are arranged in order of time from the left. In FIG. 2, the root of the tree structure indicated as <contents> represents one content, and the title of the content is attached as an attribute.

  The child element of <contents> is <section>. In <section>, a priority indicating an importance level in context contents of the corresponding scene is added as an attribute. The importance is an integer value from 1 to 5, with 1 being the least important and 5 being the most important.

  The child element of <section> is <section> or <segment>. In other words, <section> may itself be a child element. However, <section> and <segment> must not be mixed as child elements of one <section>.

  <segment> represents one scene cut, and a priority similar to <section>, start indicating a start time, and end indicating an end time are added as attributes as time information of the scene. The scene cut method may be commercially available or may be software distributed on a network, or may be performed manually. In the present embodiment, the time information is the start time and end time of the scene cut, but the same effect can be obtained by using the start time and the duration of the corresponding scene as time information. In this case, the end time of the corresponding scene is obtained by adding the duration to the start time.

  With this contextual content description data, in the case of a story such as a movie, chapters, sections, paragraphs, etc. can be described by multi-level <section>. As another example, when describing baseball, describe the times in the top <section>, describe the front and back in the child <section>, describe each batter's scene in the child <section>, Furthermore, the <section> of the child element can describe each pitch, the interval, and the result of the bat.

  As an example of expressing the context content description data of this configuration on a computer, a description in Extensible Markup Language (XML) can be used. XML is a data description language that is being standardized by the World Wide Web Consortium, and Ver. 1.0 was recommended on February 10, 1998. The XML ver. 1.0 specification is available at http://www.w3.org/TR/1998/REC-xml-19980210. 3 to 9 are examples of Document Type Definition (DTD) for describing the context content description data of this embodiment in XML and the context content description data by this DTD. 10 to 19 are contexts in which media segment representative data (dominant-data) such as representative images (video information) and keywords (sound information) is added to the context content description data shown in FIGS. An example of content description data and a DTD for describing the context content description data in XML.

  Hereinafter, the process in the selection step 101 will be described. The processing in the selection step 101 is closely related to the format of the context content description data and how to assign scores in the context content of each scene. In this embodiment, the selection step 101 focuses only on <section> having <segment> as a child element as shown in FIG. 22 (S1, S4, S5 in FIG. 23), and has a threshold value of the priority. A <section> greater than the value is selected (S2 in FIG. 23), and processing for outputting the start time and end time (S3 in FIG. 23) is performed. Therefore, the priority of <section> having <segment> as a child element is the importance between <section> having all <segment> in the content as child elements. That is, the importance in <section> surrounded by a dotted line in FIG. 22 is set to priority. Other methods of assigning priority to <section> and <segment> are optional. Note that the importance levels need not all be different values, and different elements may have the same importance level. FIG. 23 shows a flowchart of processing in the selection step in the present embodiment. For the selected <section>, the start time and end time of the scene represented by the corresponding <section> are checked from its child element <segment>. Then, the start time and end time are output.

  In the present embodiment, processing is performed by focusing on <section> having <segment> as a child element. However, it is also possible to select these by focusing on <segment>. In this case, priority is the importance between all <segments> in the content. Alternatively, the selection may be made by paying attention to the same layer among the <sections> of the upper layer that does not have <segment> as a child element. That is, processing focusing on <sections> having the same number of paths counted from <contents> or <segment> may be performed.

  Hereinafter, the operation of the extraction step 102 will be described with reference to FIG. FIG. 24 is a block diagram of the extraction step 102 according to this embodiment. In FIG. 24, the extraction step 102 in the present embodiment includes a separating unit 601, a video skimming unit 602, and an audio skimming unit 603. In the present embodiment, an MPEG1 system stream is assumed as media content. The MPEG1 system stream is obtained by multiplexing a video stream and an audio stream, and the separating unit 601 separates the multiplexed system stream into a video stream and an audio stream. The video skimming means 602 receives as input the separated video stream and the section that is the output of the selection step 101, and outputs only the data in the selected section from the input video stream. The audio skimming means 603 receives as input the separated audio stream and the section that is the output of the selection step 101, and outputs only the data in the selected section from the input audio stream.

  Hereinafter, the processing of the separating unit 601 will be described with reference to the drawings. FIG. 25 shows a flowchart of the processing of the separating means 601. The MPEG1 system stream multiplexing method is standardized by the international standard ISO / IEC IS 11172-1, and a video stream and an audio stream are multiplexed by packets. Multiplexing by packets is performed by dividing each of the video stream and the audio stream into streams of appropriate lengths called packets and adding additional information such as headers. At this time, there may be a plurality of video streams and audio streams. The header of the packet describes a stream id that can distinguish between video and audio, and a time stamp for synchronizing video and audio. The stream id can distinguish not only video and audio, but also which stream is present when there are a plurality of videos. Similarly, when there are a plurality of audio streams, they can be distinguished. In the MPEG1 system, it is configured in units of packs in which a plurality of packets are bundled. The pack includes additional information such as a multiplexing rate and reference information for time reference for synchronous reproduction as a header. Further, additional information such as the number of multiplexed video streams and the number of audio streams is attached to the head pack as a system header. The separating unit 601 first reads the number of multiplexed video streams and the number of audio streams from the system header of the first pack (S1, S2), and secures an area for storing the data of each stream (S3, S4). . Subsequently, the stream id is checked for each packet, and the packet data is written in the data area for storing the stream specified by the stream id (S5, S6). The above processing is repeated for all packets (S8, S9, S10). After processing all the data, for each stream, the video stream is output to the video skimming means 602 and the audio stream is output to the audio skimming means 603 (S11).

  The operation of the video skimming means 602 will be described below. FIG. 26 shows a flowchart of processing of the video skimming means 602. The MPEG1 video stream is standardized by the international standard ISO / IEC IS 11172-2, and is composed of a sequence layer, a GOP layer, a picture layer, a slice layer, a macroblock layer, and a block layer as shown in FIG. Has been. The minimum unit of the random access is a GOP (Group Of Pictures) layer. One picture layer corresponds to one frame. The video skimming means 602 performs data processing in units of GOP. As an initialization process, the output frame number counter C is set to 0 (S3). First, the video skimming means 602 confirms that the head of the video stream is a sequence layer header (S2, S4), stores the data (S5), and outputs the header data. Although the header of the sequence layer may appear later, the value is not allowed to be changed except for the quantization matrix. Therefore, the value is compared every time the sequence header is input (S8, S14), and the quantization matrix is obtained. If the values other than are different, an error is determined (S15). Subsequently, the video skimming means 602 detects a GOP layer header from the input data (S9). Time code data is described in the header of the GOP layer (S10). This describes the time from the beginning of the sequence. The video skimming means 602 compares this time code with the section (S1) output by the selection step 101 (S11). If the time code is not included in the selected section, the video skimming means 602 discards all data until the next GOP layer or sequence layer appears. If the time code is included in the selected section, the video skimming means 602 outputs all data until the next GOP layer or sequence layer appears (S13). However, it is necessary to change the time code of the GOP layer in order to have continuity with the data output so far (S12). Therefore, a time code to be changed is obtained using the value of the counter C. Since the counter C is the number of frames output so far, the time Tv for displaying the first frame of the GOP layer to be output this time is the picture rate that is the number of display screens per second described in C and the sequence header. Using pr, the following equation (1) is obtained.

  Tv = C / pr (1)

  Since Tv is a value in units of 1 / pr seconds, it is converted according to the MPEG1 time code format and set to the time code of the GOP layer to be output this time. Further, when outputting data of the GOP layer, the number of output picture layers is added to the counter C. The above processing is repeated until the end of the video stream (S7, S16). When a plurality of video streams are output from the separating unit 601, the above processing is performed for each video stream.

  Hereinafter, processing of the audio skimming means 603 will be described. FIG. 28 shows a flowchart of processing of the audio skimming means 603. MPEG audio is standardized by the international standard ISO / IEC IS 11172-3 and is composed of a frame called AAU (Audio Access Unit). FIG. 29 shows the structure of AAU. AAU is the smallest unit that can be decoded into audio data individually and is always composed of data of a certain number of samples Sn. Therefore, the playback time of 1 AAU can be calculated from the bit rate br, which is the transmission speed, the sampling frequency Fs, and the number of bits AAU. First, by detecting the AAU header from the audio stream (S2, S5), the number of bits A of 1 AAU can be obtained. Further, the bit rate br and the sampling frequency Fs are described in the AAU header. The sample number Sn of 1 AAU is obtained by the following equation (2).

  Sn = (L × Fs) / br (2)

  Further, the playback time Tu of 1AAU is obtained by the following equation (3) (S3).

  Tu = Sn / Fs = L / Br (3)

  When Tu is obtained, the time from the beginning of the stream can be obtained by counting the number of AAUs. The audio skimming means 603 counts the number of AAUs that have appeared so far, and calculates the time from the beginning (S7). The time is compared with the section output by the selection step 101 (S8). When the AAU appearance time is included in the selected section, the audio skimming means 603 outputs all the data of the AAU (S9). If the AAU appearance time is not included in the selected section, the audio skimming means 603 discards the AAU data. The above processing is repeated until the end of the audio stream (S6, S11). When a plurality of audio streams are output from the separating unit 601, the above processing is performed for each audio stream.

  As an effect of this embodiment, as shown in FIG. 30, the video stream and the audio stream, which are the outputs of the extraction step 102, are input to the video playback means and the audio playback means, respectively, and the video stream and the audio stream are synchronized. By reproducing, it is possible to reproduce the outline or highlight scene of the corresponding media content. Also, by multiplexing the obtained video stream and audio stream, it is possible to create an outline of the corresponding media content and an MPEG1 system stream of a highlight scene collection.

[Second Embodiment]
The second embodiment according to the present invention will be described below. This embodiment is different from the first embodiment only in the process of the selection step.

  Hereinafter, the processing of the selection step 101 in the present embodiment will be described with reference to the drawings. In the selection step 101 in the present embodiment, all priorities from the highest <section> to the <segment> that is a leaf are used. Each priority of <section> and <segment> is an objective importance in the context content. This process will be described with reference to FIG. In FIG. 31, reference numeral 1301 denotes one of the most significant <section> in the context content description data. Reference numeral 1302 denotes a child element <section> of <section> 1301. Reference numeral 1303 denotes a child element <section> of <section> 1302. Reference numeral 1304 denotes a child element <segment> of <section> 1303. In the selection step 101 in the present embodiment, an arithmetic average of all priorities on the path from <segment> to the highest <section> that is an ancestor is taken, and a <segment> whose value is equal to or greater than a threshold is selected. To do. In the example of FIG. 28, the arithmetic average pa of the attribute priority values p4, p3, p2, and p1 of <segment> 1304, <section> 1303, <section> 1302, and <section> 1301 is expressed as calculate. pa is obtained by the following equation (4).

  pa = (p1 + p2 + p3 + p4) / 4 (4)

  This pa is compared with the threshold value (S1, S2). If pa is equal to or greater than the threshold value, <segment> 1304 is selected (S3), and the attribute start and end values of <segment> 1304 are selected. It outputs as the start time and end time of the set scene (S4). The above processing is performed for all <segment> (S1, S6). FIG. 32 shows a flowchart of the processing of selection step 101 in the present embodiment.

  In this embodiment, the arithmetic average of the priority from <segment> to the highest ancestor <section> is calculated, and <segment> is selected accordingly. Select the <section> with <segment> as a child element by threshold processing by taking the arithmetic mean of the priorities from <section> having <> as a child element to the highest <section> that is the ancestor You can go. Similarly, an arithmetic average from <section> of another hierarchy to the highest <section> that is an ancestor may be taken, and <section> of that hierarchy may be selected by threshold processing.

[Third Embodiment]
The third embodiment according to the present invention will be described below. This embodiment also differs from the first embodiment only in the process of the selection step.

  Hereinafter, the processing of the selection step 101 in the present embodiment will be described with reference to the drawings. In the selection step 101 in the present embodiment, as in the processing in the first embodiment, only the <section> having <segment> as a child element is focused and selected. In this embodiment, a threshold is provided for the sum of durations of all scenes to be selected. That is, the selection is performed in the descending order of the priority of the <section> until the sum of the durations of the <section> selected so far reaches a maximum below this threshold. FIG. 33 shows a flowchart of the selection step 101 in the present embodiment. A set of <section> having <segment> as a child element is Ω (S1). First, using the attribute priority as a key, the elements <section> of Ω are sorted in descending order (S2). The <section> having the highest priority is selected from Ω (S4, S5). Remove selected <section> from Ω. By examining all the child elements <segment> of the selected <section>, the start time and end time of <section> are obtained, and the duration of <section> is calculated (S6). The sum of the durations of the <section> selected so far is obtained (S7), and if the threshold value is exceeded, the process is terminated (S8). If it is below the threshold, the start time and end time of the <section> selected this time are output (S9), and the process returns from Ω to the selection of the <section> having the highest priority. This process is repeated until the sum of durations of the selected <section> exceeds a threshold value or Ω becomes an empty set (S4, S8).

  In the present embodiment, processing is performed by focusing on <section> having <segment> as a child element. However, it is also possible to select these by focusing on <segment>. In this case, priority is the importance between all <segments> in the content. Alternatively, the selection may be made by paying attention to the same section of <section> that does not have <segment> as a child element. That is, processing focusing on <sections> having the same number of paths counted from <contents> or <segment> may be performed.

  Similarly to the second embodiment, each priority of <section> and <segment> is set as an objective importance in the context contents, and the priority from <segment> to the highest-level <section> that is an ancestor Calculate the arithmetic mean pa of and select <section> or <segment> with <segment> as a child element in descending order of pa until the sum of durations is below the threshold, The same effect can be obtained.

[Fourth Embodiment]
The fourth embodiment according to the present invention will be described below. This embodiment also differs from the first embodiment only in the process of the selection step.

  Hereinafter, the processing of the selection step 101 in the present embodiment will be described with reference to the drawings. In the selection step 101 in the present embodiment, processing is performed by paying attention to <section> having <segment> and <segment> as child elements, similarly to the processing in the first embodiment. In the present embodiment, a threshold is provided for the sum of durations of all scenes to be selected, as in the third embodiment. As in the first embodiment, the priority of <section> having <segment> as a child is the importance between <section> having all <segment> in the content as child elements. That is, the importance is set between <sections> surrounded by a dotted line in FIG. The priority of <segment> is the importance between <segment> having the same <section> as a parent element. That is, it is set as the importance between <segment> surrounded by the alternate long and short dash line in FIG.

  FIG. 35 shows a flowchart of the processing of the selection step 101 in the present embodiment. First, a set of <section> having <segment> as a child element is defined as Ω (S1). Sort Ω in descending order using priority as a key (S2). Subsequently, the <section> having the highest priority is selected from Ω (S3, S4, S5). At this time, if there are multiple <section> with the highest importance, select all. The selected <section> is set as an element of the set Ω 'and deleted from the set Ω. From the child element <segment> of the selected <section>, the start time, end time, and duration of the scene represented by the relevant <section> are obtained and stored (S6). If multiple <section> are selected, ask for them for all of them. The sum of durations of <section>, which is an element of Ω ′, is obtained (S7, S8), and compared with a threshold value (S9). If the sum of the durations is equal to the threshold value, the stored start time and end time are all output, and the process ends (S10). If the sum of durations is smaller than the threshold value, the process returns from Ω to the selection processing for <section> (S4, S5). If Ω is an empty set at this time, the stored start time and end time are all output, and the process ends (S4). If the total duration is greater than the threshold, the following processing is performed. Among the elements of the set Ω ', the <section> having the lowest importance is selected (S11). At this time, if there are a plurality of <section> having the least importance, all of them are selected. Of the child elements <segment> of the selected <section>, the least important element is deleted (S12), and the start time, end time, and duration of the corresponding <section> stored are changed (S13). ). The scene may be divided by deleting <segment>. In this case, the divided start time, end time, and duration are stored. If there is a <section> from which all <segment> have been deleted by deleting <segment>, the <section> is deleted from Ω ′. When there are a plurality of selected <section>, this process is performed for all of them. By deleting <segment>, the duration of the corresponding <section> is shortened, and the total duration is also shortened. This deletion process is repeated until the sum of the durations of the elements of Ω ′ becomes equal to or less than the threshold value. If the sum of the durations of the elements of Ω ′ is equal to or smaller than the threshold value (S14), all the stored start time and end time are output, and the process is terminated (S15).

  In this embodiment, processing is performed by paying attention to <section> having <segment> and <segment> as child elements. However, <section> and <section>, <section> The same effect can be obtained by performing processing while paying attention to the <section> of the child element.

  Also, regarding the deletion processing of <segment> when the total duration exceeds the threshold, deletion was performed from <section> with a low priority, but a threshold is set for the priority of <section>. The <segment> with the lowest importance may be deleted from all <section> below the value. Furthermore, a threshold value may be provided for <segment> priority, and <segment> below the threshold value may be deleted.

[Fifth Embodiment]
The fifth embodiment according to the present invention will be described below. In the present embodiment, an MPEG1 system stream moving image is assumed as media content. In this case, the media segment corresponds to one scene cut. In the present embodiment, the score is an objective importance based on the context contents in the corresponding scene.

  FIG. 36 is a block diagram of a data processing method according to the embodiment of the present invention. 36, 1801 represents a selection step, 1802 represents an extraction step, 1803 represents a configuration step, 1804 represents a delivery step, and 1805 represents a database. A selection step 1801 performs a process of selecting a scene of media content from the context content description data and outputting data representing the start time and end time of the scene and a file in which the scene is stored. The extraction step 1802 receives the data representing the file output from the selection step 1801, the start time, and the end time, refers to the physical content description data, and is divided from the media content file by the input start time and end time. Process to extract the data of. A configuration step 1803 performs a process of multiplexing the data output from the extraction step 1802 and configuring an MPEG1 system stream. A delivery step 1804 performs processing for delivering the MPEG1 system stream created by the composition step 1803 through a line. Reference numeral 1805 denotes a database that stores media content, its physical content description data, and context content description data.

  FIG. 37 shows the configuration of physical content description data in the present embodiment. In this embodiment, physical contents are described in a tree structure. The storage form of the media content database 1805 is not limited to storing one media content as one file, but may store one media content divided into a plurality of files. Therefore, the root of the tree structure of the physical content description data is expressed as <contents> and represents one content. The root <contents> is given the title of the corresponding content as an attribute. The child element of <contents> is <mediaobject>, which represents the stored file. An identifier id is added to <mediaobject> as an attribute for associating the link locator to the file stored as the attribute with the context content description data. Further, since the media content is composed of a plurality of files, seq indicating the order of the corresponding files in the content is also added as an attribute.

  FIG. 38 shows the configuration of the context content description data in the present embodiment. This is obtained by adding a relation with <mediaobject> of physical content description data to the context content description data in the first embodiment. That is, the child element of the root <contents> of the context content description data is <mediaobject>, and the child element of this <mediaobject> is <section>. <section> and <segment> are the same as those in the first embodiment. Corresponds to <mediaobject> in the context content description data. That is, a scene of media content described by a descendant of <mediaobject> of the context content description data is stored in a file indicated by <mediaobject> of the physical content description data having the same attribute id. Also, the time information start and end of <segment> is set to the time from the beginning of each file. That is, when one media content is composed of a plurality of files, the start time of each file is 0, and the start time of each scene is represented by the elapsed time from the start of the file in which it is stored. I will do it.

  As an example of expressing the physical content description data and the context content description data in the present embodiment on a computer, a description in Extensible Markup Language (XML) can be used. FIG. 39 shows an example of Document Type Definition (DTD) for describing the physical content description data shown in FIG. 37 in XML and the physical content description data by this DTD. 40 to 45 are examples of a DTD for describing the context content description data shown in FIG. 38 in XML and context content description data based on this DTD.

  Hereinafter, the process of the selection step 1801 will be described. Any of the methods described in the first to fourth embodiments is used as a method for selecting a scene in the selection step 1801. However, as a result, the id of <mediaobject> of the corresponding physical content description data is also output together with the start time and end time. FIG. 46 shows an example of the output of the selection step 1801 when the physical content description data is described by the XML document by the DTD shown in FIG. 39 and the context content description data is expressed by the XML document by the DTD shown in FIGS. Show. In FIG. 46, <mediaobject> id of physical content description data is described after id =, start time is described after start =, and end time is described after end =.

  Hereinafter, the process of the extraction step 1802 will be described. FIG. 47 shows a block diagram of the extraction step 1802 according to this embodiment. In FIG. 47, the extraction step 1802 in the present embodiment includes an interface unit 2401, a separating unit 2402, a video skimming unit 2403, and an audio skimming unit 2404. The interface unit 2401 receives the physical content description data and the output of the selection step 1802, takes out the media content file from the database 1805, outputs the data to the separation unit 2402, and starts the section output by the selection step 1802 The time and end time are output to video skimming means 2403 and audio skimming means 2404. Separating means 2402 separates the media content in this embodiment into a video stream and an audio stream because it is an MPEG1 system stream in which a video stream and an audio stream are multiplexed. The video skimming means 2403 receives the separated video stream and the section output by the interface means 2401 as input, and outputs only the data of the selected section from the input video stream. The audio skimming means 2402 receives the separated audio stream and the section output by the selection step 2402 as input, and outputs only the data in the selected section from the input audio stream.

  Hereinafter, processing in the interface unit 2401 will be described. FIG. 48 shows a flowchart of processing of the interface unit 2401. The interface means first inputs the physical content description data of the corresponding media content and the output of the selection step 1801 as shown in FIG. Since the time order of the file is obtained from the attribute id of the <mediaobject> of the physical content description data, the output of the selection step 1801 is sorted in order of time using the id as a key (S1). Further, the data is converted into data as shown in FIG. This is a collection of the same files, arranged in order of start time. Subsequently, the interface unit 2401 performs the following processing in order from the top of the data in FIG. First, using id, the <mediaobject> of the physical content description data is referred to, and the file name is acquired from the attribute locator. The data of the file with the corresponding file name is read from the database and output to the separating means 2402 (S2, S3). Further, all the start time and end time of the selected section in the corresponding file described after id in FIG. 49 are output to the video skimming means 2403 and the audio skimming means 2404 (S4). When the above processing is performed on all data, the processing is terminated (S5). If data still remains, the process is repeated after waiting for the separation means 2402, the video skimming means 2403, and the audio skimming means 2104 to end (S6, S7).

  Hereinafter, the process of the separation means 2402 will be described. FIG. 50 shows a flowchart of the processing of the separating means 2402. The separating unit 2402 receives the MPEG1 system stream, which is media content, from the interface unit 2401 and separates it into a video stream and an audio stream, and outputs the video stream to the video skimming unit 2403 and the audio stream to the audio skimming unit 2404 (S1). To S10), after the output is finished (S9, S11), the interface means 2401 is notified of the end of the process (S12). As shown in the flowchart of FIG. 50, the same processing as the separation unit described in the first embodiment is performed except for the notification of the end of processing.

  Hereinafter, the processing of the video skimming means 2403 will be described. FIG. 53 shows a flowchart of processing of the video skimming means 2403. As shown in the flowchart of FIG. 53, the processing similar to the video skimming means described in the first embodiment is performed except that the interface means 2401 is notified of the end of processing (S16, S17) at the end of processing. .

  Hereinafter, processing of the audio skimming means 2404 will be described. FIG. 52 shows a flowchart of processing of the audio skimming means 2404. As shown in the flowchart of FIG. 52, the same processing as the audio skimming means described in the first embodiment is performed except that the interface means 2401 is notified of the end of processing (S11, S12) when the processing is finished. .

  In the configuration step 1803, the video stream and the audio stream output from the extraction step 1802 are time-division multiplexed by the MPEG1 system multiplexing method standardized by the international standard ISO / IEC IS 11172-1. When the media content is divided into a plurality of files and stored, the extraction step 1802 outputs a video stream and an audio stream for each file, and therefore multiplexes each.

  A delivery step 1804 delivers the MPEG1 system stream multiplexed in the configuration step 1803 through a line. When the configuration step 1803 outputs a plurality of MPEG1 system streams, all are delivered in the order of output.

  In the present embodiment, when the media content is divided into a plurality of files and stored, the processing for each file is performed in the processing of the extraction step 1802, but the media content files are supported. The same effect can be obtained even if all the video streams and audio streams to be connected are output, and one MPEG1 system stream is formed in the composition step 1803 by multiplexing the video stream and the audio stream. In this case, it is necessary to change the time code in the video skimming means 2403 as follows. That is, the counter C of the number of output frames is prepared for the number of video streams, and initialization of C is performed only for the first file (S18 and S3 in FIG. 51). FIG. 53 shows a flowchart of the video skimming means 2403 in this case. In this embodiment, the context content description data and the physical content description data are described separately. However, by adding the attributes seq and locator of the physical content description data as the <mediaobject> attribute of the context content description data. , You can put them together.

[Sixth Embodiment]
The sixth embodiment according to the present invention will be described below. In the present embodiment, an MPEG1 system stream moving image is assumed as media content. In this case, the media segment corresponds to one scene cut. In the present embodiment, the score is an objective importance based on the context contents in the corresponding scene.

  FIG. 54 is a block diagram of a data processing method according to the embodiment of the present invention. 54, 3101 represents a selection step, 3102 represents an extraction step, 3103 represents a configuration step, 3104 represents a delivery step, and 3105 represents a database. The selection step 3101 performs processing for selecting a media content scene from the context content description data and outputting data representing the start time and end time of the scene and the file in which the scene is stored. This is the same as the selection step described in the embodiment. The extraction step 3102 receives the data representing the file output from the selection step 3101, the start time, and the end time, refers to the physical content description data, and is divided from the media content file by the input start time and end time This is a process similar to the extraction step described in the fifth embodiment. In the configuration step 3103, a part or all of the stream output from the extraction step 3102 is multiplexed according to the line status determined by the delivery step 3104, and the MPEG1 system stream is configured. The delivery step 3104 performs processing for judging the status of the line to be delivered and transmitting the result to the configuration step 3103 and processing for delivering the MPEG1 system stream created by the configuration step 3103 through the line. Reference numeral 3105 denotes a database storing media content, physical content description data, and context content description data.

  FIG. 55 shows a block diagram of the configuration step 3103 and the delivery step 3104 according to this embodiment. In FIG. 55, the configuration step 3103 includes a stream selection unit 3201 and a multiplexing unit 3202, and the distribution step 3104 includes a line status determination unit 3203 and a distribution unit 3204. The stream selection unit 3201 receives the video stream and audio stream output from the extraction step 3102 and the line status output from the line status determination unit 3203 as input, and the line is in a state sufficient to transmit all data. , All the streams are output to the multiplexing means 3202. If it takes a lot of time to send all the data, such as when the line is congested or the capacity is small, only a part of each of the video stream and audio stream is selected and multiplexed. To 3202. There are various combinations of selection methods in this case, such as only the base layer stream for the video stream, etc., and only monaural, stereo left, stereo right, etc. for the audio stream. However, if there is only one stream for both the video stream and the audio stream, the stream is output regardless of the line status. The multiplexing means 3202 performs time division multiplexing of the video stream and audio stream output from the stream selection means 3201 by the MPEG1 system multiplexing method standardized by the international standard ISO / IDE IS 11172-1. . The line status determination unit 3203 checks the capacity of the line to be delivered, the current usage status, and the like, and outputs it to the stream selection unit 3201. The delivery unit 3204 delivers the MPEG1 system stream multiplexed by the multiplexing unit 3202 through a line.

  In this embodiment, the stream selection unit 3201 outputs one video stream regardless of the line status. However, if the line sends all data, it takes a lot of time. Only representative images of the video stream may be selected and transmitted. As a representative image selection method, the time code of the representative image is described in the context content description data, or only a frame called an I picture that can be decoded independently is selected from each frame.

[Seventh Embodiment]
The seventh embodiment according to the present invention will be described below. In the present embodiment, an MPEG1 system stream moving image is assumed as media content. In this case, the media segment corresponds to one scene cut. In the present embodiment, the score is an importance level based on keywords such as characters and matters selected by the user or the like in the corresponding scene.

  FIG. 56 is a block diagram of a data processing method in the present embodiment. In FIG. 56, reference numeral 3301 denotes a selection step, and 3302 denotes an extraction step. A selection step 3301 performs a process of selecting a scene of the media content from the keyword of the context content description data and its score, and outputting the start time and end time of the scene. In addition, the extraction step 3302 performs a process of extracting the data of the media content section divided by the start time and the end time output by the selection step 3301.

  FIG. 57 shows the configuration of the context content description data of the present embodiment. In the present embodiment, context contents are described in a tree structure. In addition, it is assumed that the sibling relationships of the tree structure are arranged in order of time from the left. In FIG. 57, the root of the tree structure indicated as <contents> represents one content, and the title of the content is attached as an attribute.

  The child element of <contents> is <section>. In <section>, a pair (keyword, priority) of a keyword, which is a keyword indicating the content of the scene, a character, and the like, and a priority indicating the importance of the keyword is added as an attribute. priority is an integer value from 1 to 5, with 1 being the least important and 5 being the most important. The (keyword, priority) group is set so that it can be used as a key when searching for a scene or person that the user wants to see. Therefore, a plurality of (keyword, priority) pairs can be added to one <section>. For example, when describing characters, (keyword, priority) pairs are added as many as the number of characters appearing in the scene, and priority is high when many characters of the corresponding keyword appear in the scene. And so on.

  The child element of <section> is <section> or <segment>. In other words, <section> may itself be a child element. However, <section> and <segment> must not be mixed as child elements of one <section>.

  <segment> represents one scene cut, and the same (keyword, priority) pair as <section>, the start information indicating the start time, and the end indicating the end time are attributed as time information of the corresponding scene. Added. The scene cut method may be commercially available or may be software distributed on a network, or may be performed manually. In the present embodiment, the time information is the start time and end time of the scene cut, but the same effect can be obtained by using the start time and the duration of the corresponding scene as time information. In this case, the end time of the corresponding scene is obtained by adding the duration to the start time.

  With this contextual content description data, in the case of a story such as a movie, chapters, sections, paragraphs, etc. can be described by multi-level <section>. As another example, when describing baseball, describe the times in the top <section>, describe the front and back in the child <section>, describe each batter's scene in the child <section>, Furthermore, the <section> of the child element can describe each pitch, the interval, and the result of the bat.

  As an example of expressing the context content description data of this configuration on a computer, a description in Extensible Markup Language (XML) can be used. XML is a data description language that is being standardized by the World Wide Web Consortium, and Ver. 1.0 was recommended on February 10, 1998. The XML ver. 1.0 specification is available at http://www.w3.org/TR/1998/REC-xml-19980210. 58 to 66 are examples of Document Type Definition (DTD) for describing the context content description data of the present embodiment in XML and the context content description data based on this DTD. 67 to 80 are contexts in which representative data (dominant-data) of media segments such as representative images (video information) and keywords (sound information) is added to the context content description data shown in FIGS. An example of content description data and a DTD for describing the context content description data in XML.

  Hereinafter, the process in the selection step 3301 will be described. The processing in the selection step 3301 in the present embodiment is performed by paying attention to <section> having <segment> and <segment> as child elements. FIG. 81 shows a flowchart of the process of selection step 3301 in the present embodiment. In the selection step 3301 in the present embodiment, a keyword that is a key for scene selection and a threshold value of its priority are input, and from the <section> that has <segment> as a child element of context content description data, the same keyword as the key. And a <section> having a priority equal to or higher than a threshold is selected (S2, S3). Subsequently, among the <segments> of the selected <section>, only those <segments> having the same keyword as the key and having a priority equal to or higher than the threshold are selected (S5, S6). The start time and end time of the selected scene are obtained from the start and end attributes of the <segment> selected from the above processing, and are output (S7, S8, S9, S10, S11, S1, S4). ).

  In this embodiment, processing is performed focusing on <section> that has <segment> and <segment> as child elements. However, the parent-child relationship between <section> in a hierarchy and <section> that is its child element The same processing may be performed by paying attention to the above. The parent-child relationship is not limited to two layers, but the number of layers may be further increased and the same processing may be performed up to <segment>, which is a tree-structured leaf. Further, the search key may be a set of a plurality of keywords and conditions between them. The conditions between the keywords include “one”, “both”, and a combination of “one” and “both”. The selection threshold value may also be specified for each keyword when there are a plurality of keywords. The keyword serving as the search key may be received by user input, or may be configured to be automatically set by the system from a user profile or the like.

  The operation of the extraction step 3302 is the same as the extraction step described in the first embodiment.

  As an effect of the present embodiment, as shown in FIG. 82, the video stream and the audio stream that are the output of the extraction step 3302 are input to the video playback means and the audio playback means, respectively, and the video stream and the audio stream are synchronized. By playing back, it is possible to play back only the scene of the corresponding media content that the individual viewer wants to see. Also, by multiplexing the obtained video stream and audio stream, it is possible to create an MPEG1 system stream of a scene collection that the individual viewer of the media content wants to see.

[Eighth Embodiment]
The eighth embodiment according to the present invention will be described below. This embodiment is different from the seventh embodiment only in the process of the selection step.

  Hereinafter, the processing of the selection step 3301 in the present embodiment will be described with reference to the drawings. In the selection step 3301 in the present embodiment, processing is performed focusing on only <segment>. FIG. 83 shows a flowchart of the selection step 3301 in the present embodiment. As shown in FIG. 83, the selection step 3301 in this embodiment has a keyword to be a search key and a threshold of its priority as input, and has the same keyword as the key from the <segment> of the context content description data, and The <segment> whose priority is equal to or greater than the threshold value is selected (S1 to S6).

  In the present embodiment, processing is performed focusing on only <segment>, but processing may be performed focusing on <section> in a certain hierarchy. The search key may be a set of a plurality of keywords and conditions between them. The conditions between the keywords include “one”, “both”, and a combination of “one” and “both”. The selection threshold value may also be specified for each keyword when there are a plurality of keywords.

[Ninth Embodiment]
The ninth embodiment according to the present invention will be described below. This embodiment also differs from the seventh embodiment only in the process of the selection step.

  Hereinafter, the processing of the selection step 3301 in the present embodiment will be described with reference to the drawings. In the selection step 3301 in the present embodiment, as in the processing in the seventh embodiment, only the <section> having <segment> and <segment> as child elements is selected and selected. In this embodiment, a threshold is provided for the sum of durations of all scenes to be selected. That is, a selection is made such that the sum of the durations of the scenes selected so far is maximized below this threshold. FIG. 84 shows a flowchart of selection steps in the present embodiment. First, the selection step 3301 receives one keyword as a search key. Subsequently, from the <section> having <segment> as a child element, all items having a search key keyword are extracted. This set is defined as Ω (S1, S2). The elements of Ω are sorted in descending order of the priority of the keyword of the search key (S3). Subsequently, the <section> having the highest priority of the keyword of the search key is extracted from the sorted Ω (S5), and the <section> is deleted from Ω (S6). In this case, when there are a plurality of <section> having the highest priority, all of the <section> are extracted. Of the extracted <section> child elements <segment>, only <segment> having a search key is selected and added to the set Ω '(S7). The initial value of the set Ω ′ is an empty set (S2). The total duration of the scene of Ω ′ is calculated (S8) and compared with the threshold value (S9). If the sum of the durations is equal to the threshold value, all the sections of the element <segment> of Ω ′ are output, and the process ends (S14). If it is smaller than the threshold value of the duration time, the process returns to selecting the <section> having the highest priority of the keyword of the search key from Ω (S5), and the above processing is repeated. However, if Ω is an empty set, all sections of the element <segment> of Ω ′ are output, and the process is terminated (S4). When the total duration of the Ω ′ scene exceeds the threshold value, the following processing is performed. Among the elements <segment> of the set Ω ′, the <segment> having the smallest priority of the keyword of the search key is deleted (S11). In this case, if there are multiple <segment> s with the lowest priority, all <segments> are deleted. The sum of the durations of Ω ′ is calculated (S12) and compared with a threshold value (S13). If the total sum of durations is greater than the threshold value, the process returns to Ω 'to return to <segment> (S11), and this process is repeated. However, if Ω ′ is an empty set, the process is terminated (S10). If the sum of the durations is less than or equal to the threshold value, all sections of the element <segment> of Ω ′ are output, and the process ends (S14).

  In this embodiment, processing is performed focusing on <section> that has <segment> and <segment> as child elements. However, the parent-child relationship between <section> in a hierarchy and <section> that is its child element The processing may be performed paying attention to the above. Further, the parent-child relationship is not limited to two layers, and processing may be performed by further increasing the number of layers. For example, when processing in the hierarchy from the top-level <section> to <segment>, first select the top-level <section>, select the <section> that is a child element from the selected <section>, and select The process of selecting the child element from the <section> is repeated until the selection of <segment>, and the set Ω ′ of the selected <segment> is generated.

  In this embodiment, the priority of the search key keyword is in descending order. However, a threshold value may be set for priority, and the priority may be selected in descending order of priority. This threshold value may be set separately for each <section> and <segment>.

  Furthermore, in this embodiment, the search key is a single keyword, but this may be a set of a plurality of keywords and conditions between them. The conditions between the keywords include “one”, “both”, and a combination of “one” and “both”. In this case, a rule for determining the priority of keywords used for selecting or deleting <section> and <segment> is also required. An example of this rule is as follows. That is, when the condition is “one”, the largest value among the priorities of the corresponding keyword is set as the priority. In the case of “both”, the smallest priority among the priorities of the corresponding keyword is set as the priority. Even in the case of a combination of “either” and “both”, the priority value can be obtained by this rule. Even when there are a plurality of search key keywords, a threshold may be set for the priority, and processing may be performed for those having a priority equal to or higher than the threshold.

[Tenth embodiment]
The tenth embodiment according to the present invention will be described below. This embodiment is different from the seventh embodiment only in the process of the selection step.

  Hereinafter, the processing of the selection step 3301 in the present embodiment will be described with reference to the drawings. In the selection step 3301 in the present embodiment, processing is performed focusing on only <segment> as in the eighth embodiment. As in the ninth embodiment, a threshold is provided for the sum of durations of all scenes to be selected. That is, a selection is made such that the sum of the durations of the scenes selected so far is maximized below this threshold. FIG. 85 shows a flowchart of selection steps in the present embodiment. First, the selection step 3301 receives one keyword as a search key. As an initialization, the set Ω ′ is set as an empty set (S2). Subsequently, all of the <segment> having the search key keyword are extracted (S1). Let this set be Ω. The elements of Ω are sorted in descending order of the priority of the keyword of the search key (S3). Subsequently, from the sorted Ω, the <segment> having the highest priority of the keyword of the search key is extracted (S5), and the <segment> is deleted from Ω. In this case, when there are a plurality of <segment> having the largest priority, all <segment> are extracted. When Ω is an empty set, all sections of the element <segment> of Ω ′ are output, and the process ends (S4). The total duration T1 of <segment> taken out (S6), the total duration T2 of the Ω 'scene is calculated (S7), and T1 + T2 is compared with a threshold value (S8). If T1 + T2 exceeds the threshold value, all sections of the element <segment> of Ω ′ are output, and the process is terminated (S11). When T1 + T2 is equal to the threshold value, all the extracted <segment> are added as elements of Ω '(S9, S10), and all the sections of the element <segment> of Ω' are output, The process ends (S11). If T1 + T2 is smaller than the threshold value, all the extracted <segment> are added as elements of Ω ′, and the process returns to the selection process of <segment> from Ω (S10).

In the present embodiment, processing is performed focusing on only <segment>, but processing may be performed focusing on <section> in a certain hierarchy. In this embodiment, the priority of the search key keyword is in descending order. However, a threshold value may be set for priority, and the priority may be selected in descending order of priority.
Furthermore, in this embodiment, the search key is a single keyword, but this may be a set of a plurality of keywords and conditions between them. The conditions between the keywords include “one”, “both”, and a combination of “one” and “both”. In this case, a rule for determining the priority of keywords used for selecting or deleting <section> and <segment> is also required. An example of this rule is as follows. That is, when the condition is “one”, the largest value among the priorities of the corresponding keyword is set as the priority. In the case of “both”, the smallest value among the priorities of the corresponding keyword is set as the priority. Even in the case of a combination of “either” and “both”, the priority value can be obtained by this rule. Even when there are a plurality of search key keywords, a threshold may be set for the priority, and processing may be performed for those having a priority equal to or higher than the threshold.

[Eleventh embodiment]
The eleventh embodiment according to the present invention will be described below. In the present embodiment, the context content description data of the seventh to tenth embodiments differ in terms of viewpoints that are scene selection keywords and descriptions of their importance. In the seventh to tenth embodiments, as shown in FIG. 57, from the viewpoint and the viewpoint, a pair of keyword and importance (keyword, priority) is assigned to <section> and <segment> as attributes. In the present embodiment, as shown in FIG. 133, the attribute povlist is added to <contents>, and the attribute povvalue is added to <section> and <segment>. Describes the perspective and importance.

  As shown in FIG. 134, the attribute povlist represents viewpoints in vector format, and the attribute povvalue represents importance in vector format as shown in FIG. The attribute povlist and the attribute povvalue are formed by arranging the viewpoints and importance in order. For example, in FIGS. 134 and 135, the importance level regarding the viewpoint 1 is 5, the importance level regarding the viewpoint 2 is 0, the importance level regarding the viewpoint 3 is 2, and the importance level regarding the viewpoint n (where n is a positive integer). 0. Note that the importance level 0 regarding the viewpoint 2 corresponds to the fact that the attribute (keyword, priority) in which the viewpoint 2 is a keyword is not added in the seventh embodiment.

  136 to 163 and FIGS. 164 to 196 show Document Type Definition (XML) for describing the context content description data of this embodiment in Extensible Markup Language (XML) used for expressing on the computer. DTD) and an example of context content description data by this DTD. Also in the present embodiment, processing similar to that described in the seventh to tenth embodiments is performed using these context content description data.

  In this embodiment, attribute povlist is added to <contents>, and attribute povvalue is added to <section> and <segment>. However, as shown in FIG. 197, <section> and <segment> You can also add the attribute povlist. However, in the <section> or <segment> to which the attribute povlist is added, the attribute povvalue corresponds to the attribute povlist added to the <section> or <segment>. In <section> or <segment> to which attribute povlist is not added, even if attribute povvalue corresponds to attribute povlist added to <contents>, attribute povlist is not added <section> Alternatively, the attribute povlist of the nearest <section> to which the attribute povlist is added among the ancestors of <segment> may be used.

  Further, FIGS. 198 to 222 and FIGS. 223 to 252 correspond to FIG. 197, the DTD for describing the context content description data in XML used for representing on the computer, and the context content by this DTD. An example of description data is shown respectively. In the examples shown in these drawings, the attribute povvalue of <section> and <segment> to which the attribute povlist is not added corresponds to the attribute povlist added to <contents>.

[Twelfth embodiment]
The twelfth embodiment according to the present invention will be described below. In the present embodiment, an MPEG1 system stream moving image is assumed as media content. In this case, the media segment corresponds to one scene cut.

  FIG. 86 is a block diagram of a data processing method according to the embodiment of the present invention. 86, reference numeral 4101 denotes a selection step, 4102 denotes an extraction step, 4103 denotes a configuration step, 4104 denotes a delivery step, and 4105 denotes a database. In the selection step 4101, a scene of media content is selected from the context content description data, and a process of outputting data representing a start time and an end time of the scene and a file in which the scene is stored is performed. The extraction step 4102 receives the data representing the file output from the selection step 4101, the start time, and the end time, refers to the physical content description data, and is divided from the media content file by the input start time and end time Process to extract the data of. A configuration step 4103 performs a process of multiplexing the data output from the extraction step 4102 and configuring an MPEG1 system stream. A delivery step 4104 performs processing for delivering the MPEG1 system stream created by the composition step 4103 through a line. Reference numeral 4105 denotes a database storing media content, physical content description data, and context content description data.

  The configuration of the physical content description data in the present embodiment is the same as that described in the fifth embodiment. That is, the physical content description data having the configuration shown in FIG. 37 is used.

  FIG. 87 shows the configuration of the context content description data in this embodiment. This is obtained by adding a relation with <mediaobject> of the physical content description data to the context content description data in the seventh embodiment. That is, the child element of the root <contents> of the context content description data is <mediaobject>, and the child element of this <mediaobject> is <section>. <section> and <segment> are the same as those in the seventh embodiment. An attribute id is added to <mediaobject> of the context content description data, and a correspondence with <mediaobject> of the physical content description data is taken by this id. That is, a scene of media content described by a descendant of <mediaobject> of the context content description data is stored in a file indicated by <mediaobject> of the physical content description data having the same attribute id. Also, the time information start and end of <segment> is set to the time from the beginning of each file. That is, when one media content is composed of a plurality of files, the start time of each file is 0, and the start time of each scene is represented by the elapsed time from the start of the file in which it is stored. I will do it.

  As an example of expressing the physical content description data and the context content description data in the present embodiment on a computer, a description in Extensible Markup Language (XML) can be used. FIG. 39 shown in the fifth embodiment is an example of the physical content description data. 88 to 96 are examples of the DTD for describing the context content description data shown in FIG. 87 in XML and the context content description data based on this DTD.

  Hereinafter, the process of the selection step 4101 will be described. Any of the methods described in the seventh to tenth embodiments is used as a method for selecting a scene in the selection step 4101. However, as a result, the id of <mediaobject> of the corresponding physical content description data is also output together with the start time and end time. An example of the output of the selection step 4101 when the physical content description data is represented by the XML document by the DTD shown in FIG. 39 and the context content description data is represented by the XML document by the DTD shown in FIGS. It is the thing of the form similar to the thing of FIG. 46 shown in embodiment.

  The processing of the extraction step 4102 is the same as the extraction step described in the fifth embodiment. The configuration step 4103 is also the same as the configuration step described in the fifth embodiment. The delivery step 4104 is also the same as the delivery step described in the fifth embodiment.

[Thirteenth embodiment]
The thirteenth embodiment according to the present invention will be described below. In the present embodiment, an MPEG1 system stream moving image is assumed as media content. In this case, the media segment corresponds to one scene cut.

  FIG. 97 is a block diagram of a data processing method according to the embodiment of the present invention. In FIG. 97, 4401 represents a selection step, 4402 represents an extraction step, 4403 represents a configuration step, 4404 represents a delivery step, and 4405 represents a database. The selection step 4401 performs processing for selecting a scene of media content from the context content description data and outputting data representing the start time and end time of the scene and the file in which the scene is stored. This is the same as the selection step described in the embodiment. The extraction step 4402 receives the data representing the file output from the selection step 4401, the start time, and the end time, refers to the physical content description data, and is divided from the media content file by the input start time and end time. This is a process similar to the extraction step described in the twelfth embodiment. In the configuration step 4403, a part or all of the stream output from the extraction step 4402 is multiplexed according to the line status determined by the delivery step 4404, and the MPEG1 system stream is configured. It is the same as the configuration step described in the embodiment. In the delivery step 4404, the status of the delivery line is judged and the result is transmitted to the configuration step 4403, and the MPEG1 system stream created by the configuration step 4403 is delivered through the line. It is the same as the delivery step described in. Reference numeral 4405 denotes a database storing media contents, physical content description data, and context content description data.

  In the present embodiment, an MPEG1 system stream is assumed as the media content, but the same effect can be obtained with other formats as long as the time code of each screen can be obtained.

  The embodiments described below explain the summary of the modes corresponding to the invention shown in the claims. In the following, the term “sound information” is used as information about sound including sound, silence, speech, music, silence, external noise, etc., and the term “video information” is used for videos, still images, telops, etc. Used as visual information including characters. As the score, a score calculated from the content of sound information such as sound, silence, speech, music, silence, external noise, a score attached according to the presence or absence of a telop in video information, or a combination thereof can be used. . The score may be a score other than the above score.

[Fourteenth embodiment]
The fourteenth embodiment according to the present invention will be described below. FIG. 98 is a block diagram showing processing of the data processing method in the present embodiment. In the figure, 501 represents a selection step and 503 represents an extraction step. The selection step 501 is a step of selecting at least one section or scene of the media content from the score of the context content description data and outputting the selected section or scene. The selected section is, for example, the start time and end time of the selected section. In addition, the extraction step 503 is a step of performing processing for extracting only data of a media content section (hereinafter referred to as a media segment) divided by the selection section output by the selection step 501, that is, data of the selection section.

  Note that the score may be an importance level based on objective importance in contextual content, or an importance level based on keywords such as characters and matters selected by the user.

[Fifteenth embodiment]
The fifteenth embodiment according to the present invention will be described below. FIG. 99 is a block diagram showing processing of the data processing method in the present embodiment. In the figure, 501 represents a selection step, and 503 represents a reproduction step. The reproduction step 505 is a step for performing a process of reproducing only the data in the selected section divided by the selection section output from the selection step 501. Note that the selection step 501 is the same as the selection step shown in the first to thirteenth embodiments, and a description thereof will be omitted.

[Sixteenth embodiment]
The sixteenth embodiment according to the present invention will be described below. FIG. 100 is a block diagram showing processing of the data processing method in the present embodiment. In the figure, 507 represents a video selection step, and 509 represents a sound selection step. Note that the video selection step 507 and the sound selection step 509 are included in the selection step 501 shown in the fourteenth and fifteenth embodiments.

  The video selection step 507 is a step of selecting the section or scene of the video information with reference to the context content description data of the video information and outputting the selected section. The sound selection step 509 is a step of selecting a section or scene of sound information with reference to the context content description data of the sound information and outputting the selected section. The selected section is, for example, the start time and end time of the selected section. The selection section of the video information selected in the video selection step 507 and the selection section of the sound information selected in the sound selection step 509 are the extraction step 503 or the fifteenth embodiment shown in the fourteenth embodiment. Only the data in the selected section is extracted or reproduced by the reproduction step 505 shown in FIG.

[Seventeenth embodiment]
The seventeenth embodiment according to the present invention will be described below. FIG. 101 is a block diagram showing processing of the data processing method in the present embodiment. In the figure, 511 indicates a determination step, 513 indicates a selection step, 503 indicates an extraction step, and 505 indicates a reproduction step.

(Example 1)
First, in the first embodiment, the media content has a plurality of different pieces of media information at the same time, and the determination step 511 receives the physical content description data describing the data structure of the media content as input, and the capability of the receiving terminal This is a step of determining which media information is to be selected from determination conditions such as the status of the line to be delivered and a request from the user. The selection step 513 receives the data determined as the selection target in the determination step 511, the physical content description data, and the context content description data as input, and the determination step 511 refers to the input physical content description data. This is a step for performing the selection process only from the data determined to be. The extraction step 503 and the reproduction step 505 are the same as the extraction step shown in the fourteenth embodiment and the reproduction step shown in the fifteenth embodiment. The media information includes data such as video information, sound information, and text information. In the following embodiment, it is assumed that the media information includes at least one of data related to video information and sound information.

  Further, in this embodiment, different video information or sound information in the same time of the media content is assigned to the channel as shown in FIG. 102 and further to a layer in which one channel is hierarchized. Standard resolution video information is assigned to channel 1 and layer 1 for transmission, and high resolution video information is assigned to channel 1 and layer 2, and stereo sound information is assigned to channel 1 for transmitting sound information. 2 is assigned to monaural sound information. 103 and 104 are an example of Document Type Definition (DTD) for describing physical content description data in XML and physical content description data by this DTD.

  Next, the processing in the determination step 511 of this embodiment when the media content has such a channel and layer configuration will be described with reference to FIGS. First, as shown in FIG. 105, it is determined in step S101 whether there is a request from the user. In step S101, if there is a user request, determination processing SR-A based on the user request shown in FIG. 106 is executed.

  In step S101, if there is no user request, the process proceeds to step S103, and it is determined whether the receivable information is only video information, only sound information, or both video information and sound information. In this step S103, when the receivable information is only the video information, the determination process SR-B related to the video information shown in FIG. 107 is executed, and when only the sound information is determined, the determination process SR related to the sound information shown in FIG. -C is executed, and if both are video information and sound information, the process proceeds to step S105. In step S105, the capability of the receiving terminal that receives the video information and the sound information, for example, the video display capability, the sound reproduction capability, the decompression processing speed of the compressed information, etc. is determined. If the capability is high, the process proceeds to step S107. If it is lower, the process proceeds to step S109. In step S107, the status of the line for transmitting the video information and the sound information is determined. If the line is congested, the process proceeds to step S109, and if not, the process proceeds to step S111.

  Step S109 is executed when the capability of the receiving terminal is low or the line is congested. At this time, the receiving terminal uses the channel 1 and layer 1 standard resolution video information and the channel 2 monaural sound information. Receive. On the other hand, step S111 is executed when the capacity of the receiving terminal is high and the line is not congested. At this time, the receiving terminal uses the channel 1 and layer 2 high-resolution video information and the channel 1 stereo sound information. Receive.

  Next, determination processing SR-A based on a user request shown in FIG. 106 will be described. In the present embodiment, the request from the user is to select a video layer and a sound channel. First, in step S151, it is determined whether there is a request for a video by the user. In step S151, if there is a user request relating to video, the process proceeds to step S153, and if there is no user request, the process proceeds to step S159. In step S153, it is determined whether the video request by the user is to select layer 2. If YES, the process proceeds to step S155 to select layer 2 as the video information, and if NO, the process proceeds to step S157. Select. In step S159, it is determined whether or not there is a request for sound by the user. In step S159, if there is a user request regarding sound, the process proceeds to step S161, and if there is no user request, the process is terminated. In step S161, it is determined whether or not the user's sound request selects channel 1. If YES, the process proceeds to step S163 to select channel 1 as sound information, and if NO, the process proceeds to step S165 and channel 2 is selected. Select.

  Next, determination processing SR-B regarding the video information shown in FIG. 107 will be described. First, in step S171, the ability of the receiving terminal to receive video information is determined. If the ability is high, the process proceeds to step S173, and if low, the process proceeds to step S175. In step S173, the line status is determined. If the line is congested, the process proceeds to step S175, and if not congested, the process proceeds to step S177.

  Step S175 is executed when the capability of the receiving terminal is low or the line is congested. At this time, the receiving terminal receives only video information of standard resolution of channel 1 and layer 1. On the other hand, step S177 is executed when the capability of the receiving terminal is high and the line is not congested. At this time, the receiving terminal receives only high-resolution video information of channel 1 and layer 2.

  Next, determination processing SR-C regarding sound information shown in FIG. 108 will be described. First, in step S181, the ability of the receiving terminal to receive sound information is determined. If the ability is high, the process proceeds to step S183, and if low, the process proceeds to step S185. In step S183, the status of the line is determined. If the line is congested, the process proceeds to step S185, and if not, the process proceeds to step S187.

  Step S185 is executed when the capacity of the receiving terminal is low or the line is congested. At this time, the receiving terminal receives only monaural sound information of channel 2. On the other hand, step S187 is executed when the capability of the receiving terminal is high and the line is not congested. At this time, the receiving terminal receives only the stereo sound information of channel 1.

(Example 2)
Further, the second embodiment is different from the first embodiment only in the determination step S511. In the determination step 511 according to the present embodiment, the physical content description data describing the data structure of the media content is input, and the video information is determined from the determination conditions such as the receiving terminal capability, the status of the line to be delivered, and the request from the user. This is a step of determining whether to select only the sound information or only the video information and the sound information. Note that the selection step 513, the extraction step 503, and the reproduction step 505 are the same as the above-described steps, and thus description thereof is omitted.

  Next, the processing of the determination step 511 of the present embodiment will be described with reference to FIGS. 109 and 110. First, as shown in FIG. 109, in step S201, it is determined whether there is a request from the user. In step S201, if there is a user request, the process proceeds to step S203, and if there is no user request, the process proceeds to step S205. In step S203, it is determined whether or not the user request is only video information. If YES, the process proceeds to step S253 to determine only video information as a selection target. If NO, the process proceeds to step S207. In step S207, it is determined whether or not the user request is only sound information. If YES, the process proceeds to step S255 to determine only sound information as a selection target. If NO, the process proceeds to step S251 and the video information and sound information is determined. Both are determined as selection targets.

  Further, in step S205 that is advanced when there is no user request, it is determined whether the receivable information is only video information, only sound information, or both video information and sound information. In this step S205, when the receivable information is only the video information, the process proceeds to step S253, where only the video information is determined as the selection target. When the information is only the sound information, the process proceeds to step S255, where only the sound information is selected. If it is both video information and sound information, the process proceeds to step S209.

  In step S209, the line status is determined. If the line is not congested, the process proceeds to step S251, where both video information and sound information are determined to be selected, and if congested, the process proceeds to step S111. In step S211, it is determined whether sound information is included in the information delivered via the line. If YES, the process proceeds to step S255, and the sound information is determined as a selection target. If NO, the process proceeds to step S253. The video information is determined as a selection target.

(Example 3)
Also, in the third embodiment, the media content has a plurality of different video information and / or sound information at the same time, and the determination step 511 only includes the video information performed by the determination step 511 of the second embodiment, and the sound information. Which video information is determined based on judgment conditions such as receiving terminal capability, delivery line capability, and line status. / Sound information is determined as a selection target. Note that the selection step 513, the extraction step 503, and the reproduction step 505 are the same as the above-described steps, and thus description thereof is omitted.

  In the present embodiment, similar to the first embodiment, different video information or sound information at the same time possessed by the media content is allocated to each channel or layer. Is assigned standard resolution video information, channel 1 and layer 2 are assigned high resolution video information, channel 1 for transmitting sound information is stereo sound information, and channel 2 is monaural sound information. Assigned.

  Next, the process of the determination step 511 of the present embodiment will be described with reference to FIGS. As shown in FIG. 111, in the present embodiment, first, information to be selected is determined by the determination step 511 of the second embodiment (selection determination SR-D). Next, in step S301, the information determined by the selection target determination process SR-D is determined. In this step S301, when the information to be selected is only the video information, the determination process SR-E related to the video information shown in FIG. 112 is executed, and when only the information is the sound information, the sound information shown in FIG. 113 is related. Determination processing SR-F is executed, and when both the video information and the sound information are obtained, the process proceeds to step S303. In step S303, the ability of the receiving terminal to receive video information and sound information is determined. If the ability is high, the process proceeds to step S305, and if low, the process proceeds to step S307. In step S307, the line capacity such as the transmission speed is determined. If the capacity is high, the process proceeds to step S309, and if low, the process proceeds to step S307. In step S309, the line status is determined. If the line is congested, the process proceeds to step S307, and if not congested, the process proceeds to step S311.

  Step S307 is executed when the receiving terminal has a low capacity, the line capacity is low, or the line is congested. At this time, the receiving terminal performs channel 1, layer 1 standard resolution video information, 2 monaural sound information is received. On the other hand, step S311 is executed when the receiving terminal has a high capacity, the line capacity is high, and the line is not congested. At this time, the receiving terminal receives the high-resolution video information of channel 1 and layer 2, the channel 1 stereo sound information is received.

  Next, determination processing SR-E regarding the video information shown in FIG. 112 will be described. First, in step S351, the capability of the receiving terminal that receives video information is determined. If the capability is high, the process proceeds to step S353, and if low, the process proceeds to step S355. In step S353, the line capability is determined. If the capability is high, the process proceeds to step S357, and if low, the process proceeds to step S355. In step S357, the line status is determined. If the line is congested, the process proceeds to step S355, and if not congested, the process proceeds to step S359.

  Step S355 is executed when the receiving terminal has a low capacity, the line capacity is low, or the line is congested. At this time, the receiving terminal receives only the video information of the standard resolution of channel 1 and layer 1 To do. On the other hand, step S359 is executed when the receiving terminal has a high capacity, the line capacity is high, and the line is not congested. At this time, the receiving terminal receives only high-resolution video information of channel 1 and layer 2. To do.

  Next, determination processing SR-F regarding the sound information shown in FIG. 113 will be described. First, in step S371, the ability of the receiving terminal to receive sound information is determined. If the ability is high, the process proceeds to step S373, and if low, the process proceeds to step S375. In step S373, the line capacity is determined. If the capacity is high, the process proceeds to step S377, and if low, the process proceeds to step S375. In step S377, the line status is determined. If the line is congested, the process proceeds to step S375, and if not congested, the process proceeds to step S379.

  Step S375 is executed when the capacity of the receiving terminal is low, the capacity of the line is low, or the line is congested. At this time, the receiving terminal receives only the monaural sound information of channel 2. On the other hand, step S379 is executed when the receiving terminal has high capability, high line capability, and no line congestion. At this time, the receiving terminal receives only the stereo sound information of channel 1.

Example 4
In the fourth embodiment, the representative data of the media segment corresponding to each element of the lowest layer of the context content description data is added as an attribute, and the media content has a plurality of different pieces of media information at the same time. In the determination step 511, the physical content description data describing the data structure of the media content is input, and the determination conditions such as the receiving terminal capability, the capability of the line to be delivered, the status of the channel, the capability of the channel, and the request from the user To determine which media information and / or representative data is to be selected.

  Note that description of the selection step 513, the extraction step 503, and the reproduction step 505 is omitted. Note that the media information is information such as video information, sound information, text data, and the like. In the present embodiment, it is assumed that the media information includes at least one of video information and sound information. The representative data is, for example, representative image data or low-resolution video data for each media segment if it is video information, and is key phrase data for each media segment, for example, if it is sound information.

  Further, in this embodiment, as in the embodiment, different video information or sound information at the same time of the media content is assigned to the channel or layer, for example, channel 1, layer 1 for transmitting moving images. Is assigned standard resolution video information, channel 1 and layer 2 are assigned high resolution video information, channel 1 for transmitting sound information is stereo sound information, and channel 2 is monaural sound information. Is assigned.

  Next, the process of the determination step 511 of the present embodiment will be described with reference to FIGS. 114 to 118. As shown in FIG. 114, it is determined in step S401 whether there is a request from the user. In step S401, if there is a user request, determination processing SR-G based on the user request shown in FIG. 116 is executed.

  In step S401, if there is no user request, the process proceeds to step S403, and it is determined whether the receivable information is only video information, only sound information, or both video information and sound information. In this step S403, when the receivable information is only the video information, the determination process SR-H related to the video information shown in FIG. 117 is executed. When only the sound information is received, the determination process SR related to the sound information shown in FIG. -I is executed, and when the information is both video information and sound information, the process proceeds to step S405 shown in FIG.

  Step S405 is a step for determining the capability of the receiving terminal. After executing step S405, step S407 for determining the capability of the line in order and step S409 for determining whether the line is congested are executed. The discrimination step 511 of this embodiment executes these steps S405, S407 and S409, and discriminates the channel, layer or representative data of the received video information and sound information according to the following Table 1.

  Next, determination processing SR-G based on a user request shown in FIG. 116 will be described. First, in step S451, it is determined whether the user request is only video information. If YES, determination processing SR-H relating to video information is performed, and if NO, the process proceeds to step S453. In step S453, it is determined whether or not the user request is only sound information. If YES, the sound information determination process SR-I is performed. If NO, the process returns to the main routine and proceeds to step S405.

  Next, determination processing SR-H regarding the video information shown in FIG. 117 will be described. First, in step S461, the capability of the receiving terminal is determined. After executing step S461, step S463 for determining the capability of the line in order, and step S465 for determining whether the line is congested are executed. The determination processing SR-H relating to the video information of this embodiment executes these steps S461, S463, and S465, and when the terminal capability is high, the channel capability is high, and the channel is not congested, the channel 1 , Only the video information of layer 2 is received (step S471), and only the representative data of the video information is received when the terminal capability is low, the line capability is low, and the line is not congested (step S473). ). If the above conditions are not met, only the video information of channel 1 and layer 1 is received (step S475).

  Next, determination processing SR-I regarding sound information shown in FIG. 118 will be described. First, in step S471, the capability of the receiving terminal is determined. After executing step S471, step S473 for determining the capability of the line in order and step S475 for determining whether the line is congested are executed. The determination processing SR-I relating to the video information of this embodiment executes these steps S471, S473, and S475, and when the terminal capability is high and the line capability is high, and when the terminal capability is high and the line capability is high. When it is low and the line is not congested, only the sound information of channel 1 is received (step S491). When the terminal capability is low, the line capability is low, and the line is congested, only representative data of sound information is received (step S493). If the above condition is not met, only the sound information of channel 2 is received (step S495).

(Example 5)
Further, in the fifth embodiment, the determination step 511 includes the entire data of the media segment based on the determination conditions such as the reception terminal capability, the distribution line capability, and the line status, the line capability, and the request from the user, In this step, it is determined whether only the representative data of the media segment to be selected or the entire data and the representative data of the corresponding media segment are to be selected.

  In this embodiment, as in the fourth embodiment, the representative data of the media segment corresponding to each element in the lowest layer of the context content description data is added as an attribute. If this representative data is video information, For example, representative image data for each media segment or low-resolution video data, and sound information, for example, key phrase data for each media segment.

  Next, the process of the determination step 511 of the present embodiment will be described with reference to FIGS. As shown in FIG. 119, it is determined in step S501 whether there is a request from the user. In step S501, if there is a user request, determination processing SR-J based on the user request shown in FIG. 121 is executed.

  In step S501, if there is no user request, the process proceeds to step S503 to determine whether the receivable data is only the representative data of the media segment, only the entire data of the media segment, or both the representative data and the entire data. . In this step S503, when the receivable data is only representative data, the process proceeds to step S553 shown in FIG. 120, and only the representative data is determined as a selection target. When only the entire data is received, the process proceeds to step S555. Only the data is determined as a selection target, and if both the representative data and the entire data are included, the process proceeds to step S505.

  In step S505, the line capability is determined, and if the line capability is high, the process proceeds to step S507, and if low, the process proceeds to step S509. In both steps S507 and S509, it is determined whether or not the line is congested. If it is determined in step S507 that the line is not congested, the process proceeds to step S551 to determine the entire data and the representative data as selection targets. If it is determined in step S509 that the line is congested, the process proceeds to step S553, and representative data is selected. If it is determined in step S507 that the line is congested, or if it is determined in step S509 that the line is not congested, the process proceeds to step S555 and the entire data is selected.

  In the determination process SR-J by user request, first, in step S601, it is determined whether or not the user request is only representative data. If YES, the process proceeds to step S553, where only representative data is selected, and NO. If so, the process proceeds to step S603. In step S603, it is determined whether the user request is only the entire data. If YES, the process proceeds to step S555 and only the entire data is selected. If NO, the process proceeds to step S551 and both the entire data and the representative data are selected. Is selected.

[Eighteenth Embodiment]
The eighteenth embodiment according to the present invention will be described below. FIG. 122 is a block diagram showing processing of the data processing method in the present embodiment. In the figure, 501 represents a selection step, 503 represents an extraction step, and 515 represents a configuration step. Note that the selection step 501 and the extraction step 503 are the same as the selection step and the extraction step shown in the fourteenth embodiment, and thus description thereof is omitted.

  The configuration step 515 is a step of configuring a media content stream from the data of the selected section extracted by the extraction step 503. In particular, the configuration step 515 configures a stream by multiplexing the data output by the extraction step 503.

[Nineteenth embodiment]
The nineteenth embodiment according to the present invention will be described below. FIG. 123 is a block diagram showing processing of the data processing method in the present embodiment. In the figure, 501 represents a selection step, 503 represents an extraction step, 515 represents a configuration step, and 517 represents a delivery step. The selection step 501 and the extraction step 503 are the same as the selection step and the extraction step shown in the fourteenth embodiment, and the configuration step 515 is the same as the configuration step shown in the eighteenth embodiment. Description is omitted.

  The delivery step 517 is a step for delivering the stream formed by the configuration step 515 through a line. Note that the delivery step 517 may include a step of determining the status of the line to be delivered, and the configuration step 515 may include a step of adjusting the amount of data constituting the file from the status of the line determined by the delivery step 517.

[20th embodiment]
The twentieth embodiment according to the present invention will be described below. FIG. 124 is a block diagram showing processing of the data processing method in the present embodiment. In the figure, reference numeral 501 denotes a selection step, 503 denotes an extraction step, 515 denotes a configuration step, 519 denotes a recording step, and 521 denotes a data recording medium. The recording step 519 is a step of recording the stream formed by the configuration step 515 on the data recording medium 521. The data recording medium 521 records media content, context content description data, and physical content description data, and is a hard disk, memory, DVD-RAM, or the like. The selection step 501 and the extraction step 503 are the same as the selection step and the extraction step shown in the fourteenth embodiment, and the configuration step 515 is the same as the configuration step shown in the eighteenth embodiment. Description is omitted.

[Twenty-first embodiment]
The twenty-first embodiment according to the present invention will be described below. FIG. 125 is a block diagram showing processing of the data processing method in the present embodiment. In the figure, reference numeral 501 denotes a selection step, 503 denotes an extraction step, 515 denotes a configuration step, 519 denotes a recording step, 521 denotes a data recording medium, and 523 denotes a data recording medium management step. The data recording medium management step 523 is a step of reorganizing already accumulated media content and / or newly accumulated media content according to the remaining capacity of the data recording medium 521. More specifically, in the data recording medium management step 523, when the remaining capacity of the data recording medium 521 is small, the newly accumulated content is accumulated after editing, and the already accumulated media content is processed. The context content description data and the physical content description data are sent to the selection step 501, and the media content and the physical content description data are sent to the extraction step 503. The content is recorded on the data recording medium 521, and at least one of processing for deleting the media content before reorganization is performed.

  The selection step 501 and the extraction step 503 are the same as the selection step and the extraction step shown in the fourteenth embodiment, and the configuration step 515 is the same as the configuration step shown in the eighteenth embodiment. Since the recording step 519 and the data recording medium 521 are the same as the recording step and the data recording medium shown in the nineteenth embodiment, description thereof will be omitted.

[Twenty-second embodiment]
The twenty-second embodiment according to the present invention will be described below. FIG. 126 is a block diagram showing processing of the data processing method in the present embodiment. In the figure, 501 represents a selection step, 503 represents an extraction step, 515 represents a configuration step, 519 represents a recording step, 521 represents a data recording medium, and 525 represents a stored content management step. The stored content management step 525 is a step of reorganizing the media content stored in the data recording medium 521 according to the storage period. More specifically, the stored content management step 525 manages the media content stored in the data recording medium 521, and with respect to the media content that has reached a certain storage period, its context content description data, physical content description data, Is sent to the selection step 501, and the media content and physical description data are sent to the extraction step 503, thereby reorganizing the media content, recording the reorganized media content on the data recording medium 521, and This is a step of deleting media content before organization.

  The selection step 501 and the extraction step 503 are the same as the selection step and the extraction step shown in the fourteenth embodiment, and the configuration step 515 is the same as the configuration step shown in the eighteenth embodiment. Since the recording step 519 and the data recording medium 521 are the same as the recording step and the data recording medium shown in the nineteenth embodiment, description thereof will be omitted.

  Selection steps 501 and 513, extraction step 503, reproduction step 505, video selection step 507, sound selection step 509, determination step 511, configuration step 515, delivery step 517, recording step in the above fourteenth to twenty-second embodiments. 519, data recording medium management step 523 and stored content management step 525 are respectively a selection means, an extraction means, a reproduction means, a video selection means, a sound selection means, a determination means, a configuration means, a delivery means, a recording means, and a data recording medium management. As the means and the stored content management means, it can be realized as a data processing apparatus having some or all of them.

  In the above embodiment, the media content may include a data stream such as text data other than video information and sound information. In addition, each step of the above-described embodiment can be realized by storing a program for executing all or part of the operation of the step by a computer in a program storage medium and using a computer to implement the software. You may implement | achieve using the hardware circuit for exclusive use which exhibits a function.

  In the above-described embodiment, the context content description data and the physical content description data are described as separate entities. However, as shown in FIGS.

  As described above, according to the above data processing apparatus, data processing method, recording medium, and program, the context content description data having a hierarchical structure is added to the context content description data by the selection means (selection step). At least one section in the media content is selected based on the score, and in particular, only the data corresponding to the section selected by the selection means (extraction step) is extracted or reproduced by the extraction means (extraction step) Only the data corresponding to the section selected by the selection means (selection step) is reproduced by the means (reproduction step).

  Therefore, a more important scene can be freely selected from the media content, and this important selected section can be extracted or reproduced. In addition, since the context description data has a hierarchical structure composed of the top layer, the bottom layer, and other layers, you can select scenes in arbitrary units such as chapters and sections. Various selection formats can be taken, such as deleting unnecessary paragraphs.

  Also, by setting the score to indicate the importance based on the context content of the media content, and setting this score to select important scenes, for example, highlight scene collections such as programs It can be easily created, and the score indicates the importance based on the viewpoint of the keyword in the corresponding scene. By determining the keyword, a section with a higher degree of freedom can be selected. For example, by determining a keyword from a specific viewpoint such as a character or a matter, only a scene that the user wants to see can be selected.

  When the media content has a plurality of different pieces of media information at the same time, the determination unit (determination step) determines which media information is to be selected from the determination condition, and selects the selection unit (selection step). ) Is selected only from the data determined by the determining means (determination step). For this reason, since the determination means (determination step) can determine the media information of the optimum classification according to the determination condition, the selection means (selection step) can select the media information of an appropriate data amount. it can.

  Further, since the determination means (determination step) determines whether to select only video information, only sound information, or video information and sound information from the determination condition, the selection means (selection step) The time required for selecting a section to be performed can be shortened.

  Also, representative data is added as an attribute to the context content description data, and the determining means can determine the media information or representative data of the optimum classification according to these determination conditions.

  Furthermore, the determination means (determination step) determines only the entire data of the corresponding media segment, only the representative data, or both the entire data and the representative data as selection targets according to the determination condition. The means (determination step) can shorten the time required for selecting the section performed by the selection means (selection step).

  INDUSTRIAL APPLICABILITY The present invention has an effect capable of extracting a synopsis of media content, a highlight scene, or a scene desired by a viewer, and is useful for a summary creation device or the like.

It is a block diagram of the data processing method in the 1st Embodiment of this invention. It is a figure showing the data structure of the context content description data in the 1st Embodiment of this invention. It is an example of a part of XML DTD which expresses the context content description data in the 1st Embodiment of this invention on a computer, and the context content description data written by XML. It is a continuation part of the context content description data of FIG. It is a continuation part of FIG. It is a continuation part of FIG. It is a continuation part of FIG. It is a continuation part of FIG. It is a continuation part of FIG. It is an example of a part of an XML document in which representative data is added to the context content description data of FIGS. 3 to 9 and a DTD written in XML that represents the context content description data on a computer. It is a continuation part of the context content description data of FIG. It is a continuation part of FIG. It is a continuation part of FIG. It is a continuation part of FIG. It is a continuation part of FIG. It is a continuation part of FIG. It is a continuation part of FIG. It is a continuation part of FIG. It is a continuation part of FIG. It is a continuation part of FIG. It is a continuation part of FIG. It is explanatory drawing showing how to attach the importance in the 1st Embodiment of this invention. It is a flowchart of the process of the selection step in the 1st Embodiment of this invention. It is a block diagram of the extraction step in the 1st Embodiment of this invention. It is a flowchart of the process of the isolation | separation means of the extraction step in the 1st Embodiment of this invention. It is a flowchart of the process of the video skimming means of the extraction step in the 1st Embodiment of this invention. It is a block diagram of an MPEG1 video stream. It is a flowchart of the process of the audio skimming means of the extraction step in the 1st Embodiment of this invention. It is a block diagram of AAU of MPEG audio. It is a block diagram of the application in the 1st Embodiment of this invention. It is explanatory drawing of the process of the importance in the 2nd Embodiment of this invention. It is a flowchart of the process of the selection step in the 2nd Embodiment of this invention. It is a flowchart of the process of the selection step in the 3rd Embodiment of this invention. It is explanatory drawing showing how to attach the importance in the 4th Embodiment of this invention. It is a flowchart of the process of the selection step in the 4th Embodiment of this invention. It is a block diagram of the data processing method in the 5th Embodiment of this invention. It is a figure showing the data structure of the physical content description data in the 5th Embodiment of this invention. It is a figure showing the data structure of the context content description data in the 5th Embodiment of this invention. It is an example of XML DTD and XML document which express the physical content description data in the 5th Embodiment of this invention on a computer. It is the first half of an example of the XML DTD and the XML document that express the context contents description data on the computer in the fifth embodiment of the present invention. It is a continuation part of the context content description data of FIG. It is a continuation part of FIG. It is a continuation part of FIG. It is a continuation part of FIG. It is a continuation part of FIG. It is an example of the output of the selection step in the 5th Embodiment of this invention. It is a block diagram of the extraction step in the 5th Embodiment of this invention. It is a flowchart of a process of the interface means of the extraction step in the 5th Embodiment of this invention. It is an example of the result as which the interface means of the extraction step in the 5th Embodiment of this invention converted the output of the selection step. It is a flowchart of the process of the isolation | separation means of the extraction step in the 5th Embodiment of this invention. It is a flowchart of the process of the video skimming means of the extraction step in the 5th Embodiment of this invention. It is a flowchart of the process of the audio skimming means of the extraction step in the 5th Embodiment of this invention. It is a flowchart of another process of the video skimming means of the extraction step in the 5th Embodiment of this invention. It is a block diagram of the data processing method in the 6th Embodiment of this invention. It is a block diagram of a composition step and a delivery step in a 6th embodiment of the present invention. It is a block diagram of the data processing method in the 7th Embodiment of this invention. It is a figure showing the data structure of the context content description data in the 7th Embodiment of this invention. It is an example of a part of XML DTD which expresses the context content description data on the computer in the 7th Embodiment of this invention, and the context content description data written by XML. It is a continuation part of the context content description data of FIG. It is a continuation part of FIG. It is a continuation part of FIG. It is a continuation part of FIG. It is a continuation part of FIG. It is a continuation part of FIG. It is a continuation part of FIG. It is a continuation part of FIG. FIG. 67 is an example of a part of an XML document in which representative data is added to the context content description data of FIGS. 58 to 66 and a DTD written in XML that represents the context content description data on a computer. It is a continuation part of the context content description data of FIG. It is a continuation part of FIG. It is a continuation part of FIG. It is a continuation part of FIG. It is a continuation part of FIG. It is a continuation part of FIG. It is a continuation part of FIG. 74 is a continuation of FIG. It is a continuation part of FIG. 76 is a continuation of FIG. 76. It is a continuation part of FIG. It is a continuation part of FIG. 79 is a continuation of FIG. 79. It is a flowchart of the process of the selection step in the 7th Embodiment of this invention. It is a block diagram of the application in the 7th Embodiment of this invention. It is a flowchart of the process of the selection step in the 8th Embodiment of this invention. It is a flowchart of the process of the selection step in the 9th Embodiment of this invention. It is a flowchart of the process of the selection step in the 10th Embodiment of this invention. It is a block diagram of the data processing method in the 12th Embodiment of this invention. It is a figure showing the data structure of the context content description data in the 12th Embodiment of this invention. It is a part of example of XML DTD which expresses context content description data in the 5th Embodiment of this invention on a computer, and an XML document. It is a continuation part of an example of FIG. FIG. 90 is a continuation of the example in FIG. 89. FIG. 90 is a continuation of the example of FIG. FIG. 92 is a continuation of the example of FIG. 91. 92 is a continuation of the example of FIG. It is a continuation part of an example of FIG. It is a continuation part of FIG. This is a continuation of FIG. It is a block diagram of the data processing method in the 13th Embodiment of this invention. It is a block diagram of the data processing method in the 14th Embodiment of this invention. It is a block diagram of the data processing method in the 15th Embodiment of this invention. It is a block diagram of the data processing method in the 16th Embodiment of this invention. It is a block diagram of the data processing method in the 17th Embodiment of this invention. It is explanatory drawing which shows a channel and a layer. It is an example of DTD for describing physical content description data in XML, and a part of physical content description data by this DTD. It is a continuation part of the physical content description data of FIG. It is a flowchart which shows the process of the determination step of Example 1 of 17th Embodiment. It is a flowchart which shows the determination process by the user request | requirement which the determination step of Example 17 of 17th Embodiment performs. It is a flowchart which shows the determination process regarding the video information which the determination step of Example 1 of 17th Embodiment performs. It is a flowchart which shows the determination process regarding the sound information which the determination step of Example 1 of 17th Embodiment performs. It is a part of flowchart which shows the process of the determination step of Example 2 of 17th Embodiment. It is a part of flowchart which shows the process of the determination step of Example 2 of 17th Embodiment. It is a flowchart which shows the process of the determination step of Example 3 of 17th Embodiment. It is a flowchart which shows the determination process regarding the video information which the determination step of Example 3 of 17th Embodiment performs. It is a flowchart which shows the determination process regarding the sound information which the determination step of Example 3 of 17th Embodiment performs. It is a part of flowchart which shows the process of the determination step of Example 4 of 17th Embodiment. It is a part of flowchart which shows the process of the determination step of Example 4 of 17th Embodiment. It is a flowchart which shows the determination process by the user request | requirement which the determination step of Example 4 of 17th Embodiment performs. It is a flowchart which shows the determination process regarding the video information which the determination step of Example 4 of 17th Embodiment performs. It is a flowchart which shows the determination process regarding the sound information which the determination step of Example 4 of 17th Embodiment performs. It is a part of flowchart which shows the process of the determination step of Example 5 of 17th Embodiment. It is a part of flowchart which shows the process of the determination step of Example 5 of 17th Embodiment. It is a flowchart which shows the determination process by the user request | requirement which the determination step of Example 5 of 17th Embodiment performs. It is a block diagram of the data processing method in the 18th Embodiment of this invention. It is a block diagram of the data processing method in the 19th Embodiment of this invention. It is a block diagram of the data processing method in the 20th embodiment of this invention. It is a block diagram of the data processing method in the 21st Embodiment of this invention. It is a block diagram of the data processing method in the 22nd Embodiment of this invention. It is an example of a DTD that combines contextual content description data and physical content description data into one, and an XML document. It is a continuation part of the XML document of FIG. It is a continuation part of FIG. It is a continuation part of FIG. It is a continuation part of FIG. It is a continuation part of FIG. It is a figure showing the data structure of the context content description data in the 11th Embodiment of this invention. It is a figure showing the viewpoint in the 11th Embodiment of this invention. It is a figure showing the importance in the 11th Embodiment of this invention. It is an example of a part of XML DTD which expresses the context content description data on the computer in the 11th Embodiment of this invention, and the context content description data written by XML. It is a continuation part of the context content description data of FIG. 137 is a continuation of FIG. FIG. 138 is a continuation of FIG. 138. FIG. It is a continuation part of FIG. It is a continuation part of FIG. It is a continuation part of FIG. It is a continuation part of FIG. It is a continuation part of FIG. It is a continuation part of FIG. It is a continuation part of FIG. FIG. 146 is a continuation of FIG. It is a continuation part of FIG. It is a continuation part of FIG. It is a continuation part of FIG. It is a continuation part of FIG. It is a continuation part of FIG. It is a continuation part of FIG. It is a continuation part of FIG. It is a continuation part of FIG. It is a continuation part of FIG. FIG. 156 is a continuation of FIG. It is a continuation part of FIG. It is a continuation part of FIG. It is a continuation part of FIG. It is a continuation part of FIG. It is a continuation part of FIG. It is a continuation part of FIG. It is another example of the part of XML DTD which expresses the context content description data on the computer in the 11th Embodiment of this invention, and the context content description data written in XML. It is a continuation part of the context content description data of FIG. It is a continuation part of the context content description data of FIG. It is a continuation part of the context content description data of FIG. 167 is a continuation of the context content description data of FIG. 168 is a continuation of the context content description data of FIG. It is a continuation part of the context content description data of FIG. It is a continuation part of the context content description data of FIG. This is a continuation of the context content description data in FIG. 172 is a continuation of the context content description data in FIG. 172. It is a continuation part of the context content description data of FIG. It is a continuation part of the context content description data of FIG. 175 is a continuation of the context content description data of FIG. 176 is a continuation of the context content description data of FIG. It is a continuation part of the context content description data of FIG. 178 is a continuation of the context content description data in FIG. 178. It is a continuation part of the context content description data of FIG. It is a continuation part of the context content description data of FIG. 181 is a continuation of the context content description data in FIG. 18 is a continuation of the context content description data of FIG. 183 is a continuation of the context content description data in FIG. 183. FIG. 184 is a continuation of the context content description data in FIG. 184. 185 is a continuation of the context content description data in FIG. 185. 186 is a continuation of the context content description data in FIG. 186. 187 is a continuation of the context content description data in FIG. 187. FIG. 188 is a continuation of the context content description data in FIG. 188. It is a continuation part of the context content description data of FIG. This is a continuation of the context content description data of FIG. This is a continuation of the context content description data of FIG. This is a continuation of the context content description data of FIG. It is a continuation part of the context content description data of FIG. It is a continuation part of the context content description data of FIG. This is a continuation of the context content description data of FIG. It is a figure showing the data structure of the context content description data of the other aspect in the 11th Embodiment of this invention. 19 is an example of an XML DTD that represents contextual content description data on a computer and a part of contextual content description data written in XML, corresponding to FIG. 197, in the eleventh embodiment of the present invention. This is a continuation of the context content description data of FIG. It is a continuation part of FIG. It is a continuation part of FIG. This is a continuation of FIG. It is a continuation part of FIG. It is a continuation part of FIG. It is a continuation part of FIG. This is a continuation of FIG. It is a continuation part of FIG. It is a continuation part of FIG. It is a continuation part of FIG. It is a continuation part of FIG. It is a continuation part of FIG. It is a continuation part of FIG. It is a continuation part of FIG. It is a continuation part of FIG. It is a continuation part of FIG. It is a continuation part of FIG. It is a continuation part of FIG. It is a continuation part of FIG. FIG. 218 is a continuation of FIG. 218. FIG. 219 is a continuation of FIG. It is a continuation part of FIG. It is a continuation part of FIG. FIG. 26 is another example of an XML DTD representing context content description data on a computer and a part of the context content description data written in XML, corresponding to FIG. 197 in the eleventh embodiment of the present invention. It is a continuation part of the context content description data of FIG. It is a continuation part of FIG. It is a continuation part of FIG. 226 is a continuation of FIG. It is a continuation part of FIG. It is a continuation part of FIG. It is a continuation part of FIG. It is a continuation part of FIG. It is a continuation part of FIG. It is a continuation part of FIG. It is a continuation part of FIG. FIG. 234 is a continuation of FIG. 234. FIG. 235 is a continuation of FIG. 236 is a continuation of FIG. 237 is a continuation of FIG. It is a continuation part of FIG. It is a continuation part of FIG. It is a continuation part of FIG. It is a continuation part of FIG. It is a continuation part of FIG. It is a continuation part of FIG. It is a continuation part of FIG. It is a continuation part of FIG. It is a continuation part of FIG. It is a continuation part of FIG. It is a continuation part of FIG. It is a continuation part of FIG. It is a continuation part of FIG. It is a continuation part of FIG.

Explanation of symbols

101 Selection Step 102 Extraction Step 501 513 Selection Step 503 Extraction Step 505 Playback Step 507 Video Selection Step 509 Sound Selection Step 511 Determination Step 515 Configuration Step 517 Delivery Step 519 Recording Step 523 Data Recording Medium Management Step 525 Storage Content Management Step 601 Separation Means 602 Video skimming means 603 Audio skimming means 1301 Section
1302 <section>
1301 <section>
1301 Leaf <segment>
1801 selection step 1802 extraction step 1803 configuration step 1804 distribution step 1805 database 2401 interface unit 2402 separation unit 2403 video skimming unit 2404 audio skimming unit 3101 selection step 3102 extraction step 3103 configuration step 3104 distribution step 3105 database 3201 stream selection unit 3202 multiplexing unit 3203 Status determination means 3204 Delivery means 4101 Selection step 4102 Extraction step 4103 Configuration step 4104 Delivery step 4105 Database 4401 Selection step 4402 Extraction step 4403 Configuration step 4404 Delivery step 4405 Database

Claims (17)

A context describing a segment representing each scene of media content composed of a plurality of scenes, time information representing scene breaks, which is attribute information of the segment, and a score based on the context content of the media content Means for inputting content description data;
Selecting means for selecting a segment based on the score;
Content input means for inputting media content corresponding to the context content description data;
Extraction means for extracting a section of the media content from the time information of the selected segment;
A summary creation device. Segments representing each scene of media content composed of a plurality of scenes, time information representing scene segmentation, which is attribute information of the segments, viewpoints represented by at least one keyword representing scenes, and each viewpoint Means for inputting contextual content description data describing the score of the segment based on
Selection means for selecting the segment based on at least one of the viewpoint and the score;
Content input means for inputting media content corresponding to the context content description data;
Extraction means for extracting a section of the media content from the time information of the selected segment;
A summary creation device.   The summary creation device according to claim 1, further comprising storage means for storing the context content description data and the corresponding media content. In the context content description data, a link destination of the corresponding media content is described,
The summary creation device according to claim 1, wherein the extraction unit extracts a section of the linked media content from the time information of the selected segment.   The summary creation apparatus according to claim 1, wherein the time information includes a start time and an end time of each scene.   The summary creation device according to claim 1, wherein the time information includes a start time and a duration of each scene.   The summary creation device according to any one of claims 1 to 6, wherein the context content description data includes a plurality of segments described hierarchically.   The summary creation device according to claim 2, wherein a plurality of sets of the viewpoint and the score are described as the attribute information in one segment.   The summary creation device according to claim 1, wherein the selection unit selects a segment whose score is larger than a certain threshold.   The summary creation device according to claim 1, wherein the selection unit selects the segments in the descending order of the score of the segments in which the sum of durations of the segments is the maximum when the sum is not more than a threshold.   The summary creation device according to claim 1, wherein the selection unit selects the sum of the durations of the segments in the descending order of the scores of the segments in the vicinity of a threshold value.   The summary creation device according to claim 2, wherein the selection unit selects a segment in which the total sum of durations is equal to or less than a threshold value in descending order from the score of at least one selected viewpoint.   The summary creation device according to claim 2, wherein the selection unit selects a segment in which the total sum of durations is in the vicinity of a threshold value in descending order from the score of at least one selected viewpoint. A context describing a segment representing each scene of media content composed of a plurality of scenes, time information representing scene breaks, which is attribute information of the segment, and a score based on the context content of the media content Inputting content description data; and
Selecting a segment based on the score;
Inputting media content corresponding to the context content description data;
Extracting a section of the media content from time information of the selected segment;
A method for creating summaries. Segments representing each scene of media content composed of a plurality of scenes, time information representing scene segmentation, which is attribute information of the segments, viewpoints represented by at least one keyword representing scenes, and each viewpoint Entering contextual content description data describing the score of the segment based on
Selecting the segment based on at least one of the viewpoint and the score;
Inputting media content corresponding to the context content description data;
Extracting a section of the media content from time information of the selected segment;
A method for creating summaries.   A summarizing program for causing a computer to execute each step according to claim 14 or 15.   The computer-readable recording medium which recorded the program of Claim 16.

Images