JP2004127324A - Data processing apparatus, data processing method, recording medium, and program for making computer to perform the data processing method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a data processing apparatus, a data processing method, a program for making a computer to perform the same data processing method and a recording medium with the same program recorded thereon in which a required scene is freely selected out of media contents. <P>SOLUTION: The data processing apparatus inputs context content description data describing a segment representing each of scenes of media contents composed of the plurality of scenes and a score representing a degree of significance based on context contents of the media contents as attribute information of the segment, and selects the segment based upon the score. <P>COPYRIGHT: (C)2004,JPO

Description

According to the present invention, in viewing, reproducing, distributing, and storing continuous audiovisual information (media content) such as moving images, videos, and audio, only synopses and highlight scenes of the corresponding media content, or scenes that the viewer desires to see, are displayed. An object of the present invention is to provide a media content data processing device, a data processing method, a recording medium, and a program for reproducing and delivering.

Conventionally, playback, delivery, and storage of media content have been performed on a file-by-file basis that stores the media content.

As a method of searching for a specific scene of a moving image, as in Japanese Patent Application Laid-Open No. H10-111872, switching of a scene of a moving image (scene cut) is detected, and a time code of a start frame, an end frame, Time code and additional information of the keyword of the scene.

Alternatively, Carnegie Mellon University (CMU) summarizes videos by detecting scene cuts, detecting human faces and captions, and detecting key phrases by voice recognition (Michael A. Smith, Takeo Kanade). , "Video Skimming and Characterization through the Combination of Image and Language Understanding Techniques", CMU-CS-97-111, February 3, 1997).

However, in the conventional method, when the reproduction is performed in file units, it is impossible to see a synopsis of the content. Also, when searching for a highlight scene or a scene that the user wants to see, there is another problem that the user must refer to the content from the beginning. In addition, in moving image delivery, there is a problem that it takes a lot of time to transmit all the data of the file.

According to the method disclosed in Japanese Patent Application Laid-Open No. H10-111872, a scene search can be performed using a keyword, so that a search for a scene desired by a user can be easily performed. However, the additional information has no relation or connection between the scenes, and for example, it is difficult to perform processing when searching for one section of a story. Also, it is difficult to know which scene is important in context by searching only for keywords, so it is also difficult to create a synopsis or a highlight scene collection.

Also, according to the CMU method, summarization of moving images can be performed, but since the result is determined in one way, it is not possible to perform summarization with different playback times, for example, a five-minute summary and a three-minute summary. Have difficulty. In addition, it is difficult to summarize by a user's request, such as selecting a scene in which a specific person is shown.

The object of the present invention is to provide a means for selecting, synthesizing or highlighting only the scenes or highlight scenes or reproducing and distributing only the scenes desired by the viewer in reproducing the media contents.

Another object of the present invention is to provide means for selecting a synopsis, a highlight scene, a scene desired by a viewer, and the like in accordance with a time desired by the user.

Further, it is an object of the present invention to provide a means for delivering only a playback time desired by a user, a highlight scene collection, and a scene desired by a user in response to a user's request in the delivery of media content.

(4) It is another object of the present invention to provide a means for adjusting the amount of data to be delivered according to the line status of communication between the server and the user.

In the data processing apparatus according to the present invention, a segment representing each scene of a media content composed of a plurality of scenes and a score representing importance based on context content of the media content, which is attribute information of the segment, are described. Means for inputting the contextual description data thus obtained;
Selecting means for selecting a segment based on the score.

With this configuration, the required scene can be freely selected from the media contents.

In the data processing device, each scene of the media content is divided by time according to a break of the scene, and the context content description data includes, as the attribute information, time information indicating the break of the scene. It has been described.

In the above data processing device, the time information includes a start time and an end time of each scene.

In the above data processing device, the time information includes a start time and a duration of each scene.

In the above data processing device, the plurality of segments are hierarchically described in the context description data.

In the above data processing device, the context content description data has auxiliary information on context content.

In the above data processing device, the media content is at least one of video information and sound information.

In the above data processing apparatus, a link of representative data representing the segment is added to the segment.

In the above data processing device, the representative data is at least one of video information and sound information.

In the data processing method according to the present invention, a segment representing each scene of a media content composed of a plurality of scenes, and a score representing an importance based on a context content of the media content, which is attribute information of the segment, may be used. The described context content description data is input, and a segment is selected based on the score.

This method allows you to freely select the required scene from the media contents.

In the data processing method described above, each scene of the media content is separated by time according to a break of the scene, and the context content description data includes, as the attribute information, time information indicating the break of the scene. It has been described.

In the data processing method, the time information includes a start time and an end time of each scene.

In the above data processing method, the time information includes a start time and a duration of each scene.

{In addition, in the above data processing method, the plurality of segments are hierarchically described in the context description data.

In the data processing method described above, the context description data has auxiliary information related to the context.

In the above data processing method, the media content is at least one of video information and sound information.

In the above data processing method, a link of representative data representing the segment is added to the segment.

In the above data processing method, the representative data is at least one of video information and sound information.

In the program according to the present invention, a segment representing each scene of a media content composed of a plurality of scenes and a score representing importance based on context content of the media content, which is attribute information of the segment, are described. And a step of inputting the contextual description data and selecting a segment based on the score.

プ ロ グ ラ ム This program allows you to freely select the required scene from the media contents.

In the above-mentioned program, each scene of the media content is separated by time according to a break of the scene, and the context content description data describes time information indicating the break of the scene as the attribute information. ing.

The recording medium according to the present invention is a computer-readable recording medium that stores the above-described program.

(4) With this recording medium, a program for freely selecting a required scene from media contents can be recorded.

According to the present invention, a data processing apparatus, a data processing method, a program for causing a computer to execute the data processing method, and a program for causing a computer to execute the data processing method can be selected freely from media contents. A recording medium can be provided.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.

[First Embodiment]
Hereinafter, a first embodiment according to the present invention will be described. In the present embodiment, a moving image of an MPEG1 system stream is assumed as the media content. In this case, the media segment corresponds to one scene cut. In the present embodiment, the score is an objective importance based on the context content in the corresponding scene.

FIG. 1 is a block diagram of a data processing method according to the present embodiment. In FIG. 1, 101 indicates a selection step and 102 indicates an extraction step. The selecting step 101 performs a process of selecting a scene of the media content from the context description data and outputting a start time and an end time of the scene. In addition, the extraction step 102 performs a process of extracting data of a section of the media content separated by the start time and the end time output by the selection step 101.

FIG. 2 shows the structure of the context description data according to the present embodiment. In the present embodiment, context contents are described in a tree structure. Further, it is assumed that the siblings of the tree structure are arranged in chronological order from the left. In FIG. 2, the root of the tree structure described as <contents> represents one content, and the title of the content is given as an attribute.

子 The child element of <contents> is <section>. To <section>, priority indicating the importance of the context content of the scene is added as an attribute. The importance is an integer from 1 to 5, with 1 being the least important and 5 being the most important.

子 The child element of <section> is <section> or <segment>. That is, <section> may itself be a child element. However, <section> and <segment> must not be mixed as child elements of one <section>.

<Segment> indicates one scene cut, and the same priority as <section>, and start indicating start time and end indicating end time are added as attributes as time information of the corresponding scene. The method of scene cut may be software that is commercially available or distributed on a network, or may be manually performed. In the present embodiment, the time information is the start time and the end time of the scene cut, but the same effect can be obtained by using the start time and the duration of the scene as the time information. In this case, the end time of the scene is obtained by adding the duration to the start time.

文脈 With this context description data, in the case of a story such as a movie, chapters, sections, paragraphs, etc. can be described by multi-layer <section>. When describing baseball as another example, describe the times in the top <section>, describe the front and back with the child element <section>, describe the scene of each batter with the child element <section>, In addition, the <section> of the child element can describe each pitch, the interval, the result of the turn at bat, and the like.

記述 As an example of expressing the context content description data of this configuration on a computer, a description in Extensible Markup Language (XML) can be used. XML is a data description language being standardized by the World Wide Web Consortium, and Ver. 1.0 was recommended on February 10, 1998. The XML ver. 1.0 specification is available at http://www.w3.org/TR/1998/REC-xml-19980210. 3 to 9 show an example of a Document Type Definition (DTD) for describing the context content description data of the present embodiment in XML, and an example of the context content description data based on the DTD. 10 to 19 show a context in which representative data (dominant-data) of a media segment such as a representative image (video information) or a keyword (sound information) is added to the context description data shown in FIGS. An example of the content description data and a DTD for describing the context description data in XML.

Hereinafter, the processing in the selection step 101 will be described. The processing in the selection step 101 is closely related to the format of the context content description data and how to score the context content of each scene. In the present embodiment, the selection step 101 focuses only on <section> having <segment> as a child element as shown in FIG. 22 (S1, S4, S5 in FIG. 23), and there is a threshold having a value of the priority. A <section> larger than the value is selected (S2 in FIG. 23), and a process of outputting the start time and the end time (S3 in FIG. 23) is performed. Therefore, the priority of <section> having <segment> as a child element is the importance of all <section> having <segment> as a child element in the content. That is, the priority in <section> surrounded by a dotted line in FIG. 22 is set to priority. The priority of <section> and <segment> other than this is optional. Note that all importance levels need not be different values, and different elements may have the same importance level. FIG. 23 shows a flowchart of the process in the selection step in the present embodiment. For the selected <section>, the start time and the end time of the scene represented by the relevant <section> are checked from the child element <segment>. Then, the start time and the end time are output.

In the present embodiment, the processing is performed by focusing on <section> having <segment> as a child element. However, the selection may be performed by focusing on <segment>. In this case, priority is the importance of all <segments> in the content. In addition, among <section> of the upper layer not having <segment> as a child element, the selection may be performed by focusing on the same layer. That is, the processing may be performed focusing on <section> having the same number of paths counted from <contents> or <segment>.

Hereinafter, the operation of the extraction step 102 will be described with reference to FIG. FIG. 24 is a block diagram of the extraction step 102 according to the present embodiment. In FIG. 24, the extraction step 102 in the present embodiment includes a separating unit 601, a video skimming unit 602, and an audio skimming unit 603. In the present embodiment, an MPEG1 system stream is assumed as the media content. The MPEG1 system stream is obtained by multiplexing a video stream and an audio stream, and the separating unit 601 separates the multiplexed system stream into a video stream and an audio stream. The video skimming means 602 receives as input the separated video stream and the section output from the selection step 101, and outputs only data of the selected section from the input video stream. The audio skimming means 603 receives the separated audio stream and the section output from the selection step 101, and outputs only data of the selected section from the input audio stream.

Hereinafter, the processing of the separating unit 601 will be described with reference to the drawings. FIG. 25 shows a flowchart of the processing of the separating means 601. The multiplexing method of the MPEG1 system stream is standardized by the international standard ISO / IEC IS 11172-1, and a video stream and an audio stream are multiplexed by packets. Multiplexing by packets is performed by dividing each of a video stream and an audio stream into a stream of an appropriate length called a packet and adding additional information such as a header. At this time, it is said that there may be a plurality of video streams and a plurality of audio streams. In the header of the packet, a stream id capable of distinguishing between video and audio, and a time stamp for synchronizing video and audio are described. The stream id can distinguish not only a video and an audio but also a stream when there are a plurality of videos. Similarly, when there are a plurality of audio streams, it can be distinguished. In the MPEG1 system, the packet is constituted by a unit called a pack in which a plurality of packets are bundled. A multiplex rate, additional information for referring to a time reference for synchronous reproduction, and the like are added to the pack as a header. Furthermore, additional information such as the number of multiplexed video streams and the number of audio streams is added to the first pack as a system header. The separating unit 601 first reads the number of multiplexed video streams and audio streams from the system header of the first pack (S1, S2), and secures an area for storing data of each stream (S3, S4). . Subsequently, the stream id is checked for each packet, and the packet data is written to the data area for storing the stream specified by the stream id (S5, S6). The above processing is repeated for all packets (S8, S9, S10). After processing all data, the video stream is output to the video skimming means 602 and the audio stream is output to the audio skimming means 603 for each stream (S11).

Hereinafter, the operation of the video skimming means 602 will be described. FIG. 26 shows a flowchart of the processing of the video skimming means 602. The MPEG1 video stream is standardized by the international standard ISO / IEC IS 11172-2, and includes a sequence layer, a GOP layer, a picture layer, a slice layer, a macroblock layer, and a block layer as shown in FIG. Have been. The minimum unit of the random access is a GOP (Group Of Pictures) layer. One of the picture layers corresponds to one frame. The video skimming means 602 performs data processing on a GOP basis. As initialization processing, the output frame number counter C is set to 0 (S3). First, the video skimming means 602 confirms that the head of the video stream is the header of the sequence layer (S2, S4), saves the data (S5), and outputs the data of the header. Although the header of the sequence layer may appear in the future, its value cannot be changed except for the quantization matrix. Therefore, each time the sequence header is input, the value is compared (S8, S14), and the quantization matrix is changed. If any other value is different, an error is determined (S15). Subsequently, the video skimming means 602 detects the header of the GOP layer from the input data (S9). Time code data is described in the header of the GOP layer (S10). This describes the time from the beginning of the sequence. The video skimming means 602 compares the time code with the section (S1) output by the selection step 101 (S11). If the time code is not included in the selected section, the video skimming means 602 discards all data until the next GOP layer or sequence layer appears. If the time code is included in the selected section, the video skimming means 602 outputs all data until the next GOP layer or sequence layer appears (S13). However, it is necessary to change the time code of the GOP layer in order to have continuity with the data output up to that time (S12). Therefore, a time code to be changed is obtained using the value of the counter C. Since the counter C is the number of frames output so far, the time Tv for displaying the first frame of the GOP layer to be output this time is C and the picture rate, which is the number of display screens per second described in the sequence header. It is obtained by the following equation (1) using pr.

{Tv = C / pr} ・ ・ ・ (1)

Since Tv is a value in the unit of 1 / pr second, it is converted according to the MPEG1 time code format and set as the time code of the GOP layer to be output this time. When outputting data of the GOP layer, the number of output picture layers is added to the counter C. The above processing is repeated until the end of the video stream (S7, S16). When a plurality of video streams are output from the separating unit 601, the above processing is performed for each video stream.

Hereinafter, processing of the audio skimming means 603 will be described. FIG. 28 shows a flowchart of the processing of the audio skimming means 603. MPEG audio is standardized by the international standard ISO / IEC IS 11172-3, and is composed of a frame called an AAU (Audio Access Unit). FIG. 29 shows the structure of the AAU. The AAU is a minimum unit that can be independently decoded into audio data one by one, and is always configured with data having a fixed number of samples Sn. Therefore, the playback time of one AAU can be calculated from the bit rate br, which is the transmission speed, the sampling frequency Fs, and the number L of bits of the AAU. First, by detecting the AAU header from the audio stream (S2, S5), the number of bits L of one AAU can be obtained. The bit rate br and the sampling frequency Fs are described in the AAU header. The sample number Sn of 1 AAU is obtained by the following equation (2).

Sn = (L × Fs) / br ・ ・ ・ (2)

{Circle around (1)} The playback time Tu of one AAU is obtained by the following equation (3) (S3).

{Tu = Sn / Fs = L / Br} ・ ・ ・ (3)

When Tu is obtained, the time from the head of the stream can be obtained by counting the number of AAUs. The audio skimming means 603 counts the number of AAUs that have appeared so far, and calculates the time from the beginning (S7). The time is compared with the section output by the selection step 101 (S8). If the appearance time of the AAU is included in the selected section, the audio skimming means 603 outputs all the data of the AAU (S9). If the appearance time of the AAU is not included in the selected section, the audio skimming means 603 discards the data of the AAU. The above processing is repeated until the end of the audio stream (S6, S11). When a plurality of audio streams are output from the separation unit 601, the above processing is performed for each audio stream.

As an effect of the present embodiment, as shown in FIG. 30, the video stream and the audio stream output from the extraction step 102 are input to the video playback unit and the audio playback unit, respectively, and the video stream and the audio stream are synchronized. By reproducing, the synopsis and highlight scene of the corresponding media content can be reproduced. Further, by multiplexing the obtained video stream and audio stream, it is possible to create an MPEG1 system stream of a synopsis or highlight scene collection of the corresponding media content.

[Second embodiment]
Hereinafter, a second embodiment according to the present invention will be described. This embodiment differs from the first embodiment only in the processing of the selection step.

Hereinafter, the processing of the selection step 101 in the present embodiment will be described with reference to the drawings. In the selection step 101 in the present embodiment, all priorities are used from the top <section> to the leaf <segment>. The priority of each of <section> and <segment> is an objective importance in the context content. This processing will be described with reference to FIG. In FIG. 31, reference numeral 1301 denotes one of the highest-level <section> in the context description data. 1302 is a child element <section> of <section> 1301. 1303 is a child element <section> of <section> 1302. Reference numeral 1304 denotes a child element <segment> of <section> 1303. In the selection step 101 in the present embodiment, the arithmetic mean of all priorities on the path from <segment> to the highest ancestor <section> is selected, and <segment> whose value is equal to or greater than the threshold value is selected. I do. In the example of FIG. 28, the arithmetic mean pa of the attribute values p4, p3, p2, and p1 of <segment> 1304, <section> 1303, <section> 1302, and <section> 1301 is calculate. pa is obtained by the following equation (4).

{Pa = (p1 + p2 + p3 + p4) / 4} ・ ・ ・ (4)

This pa is compared with the threshold value (S1, S2). If pa is equal to or greater than the threshold value, <segment> 1304 is selected (S3), and the values of the attributes start and end of <segment> 1304 are selected. The start time and the end time of the scene are output (S4). The above processing is performed for all <segments> (S1, S6). FIG. 32 shows a flowchart of the processing of selection step 101 in the present embodiment.

In the present embodiment, the arithmetic average of the priorities from <segment> to the highest ancestor <section> is calculated, and <segment> is selected based on the arithmetic mean. Calculates the arithmetic mean of the priority from <section> with <> as a child element to the highest ancestor <section>, and selects the <section> with <segment> as a child element by threshold processing. You may go. Similarly, the arithmetic section from the <section> of another layer to the highest <section> that is the ancestor may be calculated, and the <section> of that layer may be selected by threshold processing.

[Third Embodiment]
Hereinafter, a third embodiment according to the present invention will be described. This embodiment also differs from the first embodiment only in the processing of the selection step.

Hereinafter, the processing of the selection step 101 in the present embodiment will be described with reference to the drawings. In the selection step 101 according to the present embodiment, similarly to the processing according to the first embodiment, the selection is performed by focusing on only <section> having <segment> as a child element. In the present embodiment, a threshold value is provided for the sum of the durations of all selected scenes. That is, selection is performed in descending order of the priority of <section> until the sum of the durations of the <section> selected up to that time becomes the maximum below this threshold. FIG. 33 shows a flowchart of the selection step 101 in the present embodiment. A set of <section> having <segment> as a child element is set to Ω (S1). First, the elements <section> of Ω are sorted in descending order using the attribute priority as a key (S2). <Section> having the highest priority is selected from Ω (S4, S5). Remove the selected <section> from Ω. By examining all the child elements <segment> of the selected <section>, the start time and end time of the <section> are obtained, and the duration of the <section> is calculated (S6). The sum of the continuation times of the <section> selected so far is obtained (S7), and if it exceeds the threshold value, the process is terminated (S8). If not, the start time and the end time of the currently selected <section> are output (S9), and the process returns to the selection of the <section> having the highest priority from Ω. This process is repeated until the sum of the durations of the selected <section> exceeds the threshold value or Ω becomes an empty set (S4, S8).

In the present embodiment, the processing is performed by focusing on <section> having <segment> as a child element. However, the selection may be performed by focusing on <segment>. In this case, priority is the importance of all <segments> in the content. Alternatively, the selection may be performed by focusing on the same layer of <section> having no <segment> as a child element. That is, the processing may be performed focusing on <section> having the same number of paths counted from <contents> or <segment>.

Similarly to the second embodiment, the priority of each of <section> and <segment> is set as an objective importance in the context content, and the priority from <segment> to the highest ancestor <section> of the ancestor is set. Calculate the arithmetic mean pa of and select <section> or <segment> having <segment> as a child element from the largest pa, until the sum of the durations becomes the maximum below the threshold, The same effect can be obtained.

[Fourth Embodiment]
Hereinafter, a fourth embodiment according to the present invention will be described. This embodiment also differs from the first embodiment only in the processing of the selection step.

Hereinafter, the processing of the selection step 101 in the present embodiment will be described with reference to the drawings. The selection step 101 in the present embodiment focuses on <segment> and <section> having <segment> as a child element, and performs the processing, similarly to the processing in the first embodiment. Also, in the present embodiment, as in the third embodiment, a threshold value is provided for the sum of the durations of all selected scenes. The priority of <section> having <segment> as a child is the importance of all the <section> having <segment> as a child element in the content, as in the first embodiment. That is, the importance is set between <section> enclosed by the dotted line in FIG. The priority of <segment> is the importance of <segment> having the same <section> as a parent element. That is, the importance between <segment> s enclosed by the dashed line in FIG.

FIG. 35 shows a flowchart of the process of the selection step 101 in the present embodiment. First, a set of <section> having <segment> as a child element is set to Ω (S1). Ω is sorted in descending order using priority as a key (S2). Subsequently, <section> having the highest priority is selected from Ω (S3, S4, S5). At this time, if there are a plurality of <section> s having the highest importance, all are selected. The selected <section> is set as an element of the set Ω ′ and is deleted from the set Ω. From the child element <segment> of the selected <section>, the start time, end time, and duration of the scene represented by the relevant <section> are obtained and stored (S6). If multiple <section> s are selected, ask for them for all of them. The total sum of the durations of <section>, which is an element of Ω ', is obtained (S7, S8), and compared with a threshold value (S9). If the sum of the durations is equal to the threshold, the stored start time and end time are all output, and the process ends (S10). If the total of the durations is smaller than the threshold, the process returns from Ω to the selection of <section> (S4, S5). At this time, if Ω is an empty set, the stored start time and end time are all output, and the process ends (S4). If the sum of the durations is greater than the threshold, the following processing is performed. The <section> having the lowest importance is selected from the elements of the set Ω ′ (S11). At this time, if there are a plurality of <section> s having the least importance, all of them are selected. Of the child elements <segment> of the selected <section>, the one with the lowest importance is deleted (S12), and the stored start time, end time and duration of the relevant <section> are changed (S13). ). The scene may be divided by deleting <segment>. In this case, the start time, end time, and duration of each divided part are stored. If there is a <section> from which all <segments> have been deleted by deleting the <segment>, the <section> is deleted from Ω '. If there are a plurality of selected <section> s, this process is performed for all of them. By deleting the <segment>, the duration of the relevant <section> is reduced, and the total duration is also reduced. This deletion process is repeated until the sum of the durations of the elements of Ω ′ becomes equal to or smaller than the threshold value. If the sum of the durations of the elements of Ω ′ is equal to or smaller than the threshold value (S14), the stored start time and end time are all output, and the process ends (S15).

In the present embodiment, processing is performed by focusing on <section> having <segment> and <segment> as child elements, but <section> and its child elements <section>, <section> and The same effect can be obtained even if the processing is performed by focusing on the <section> of the child element.

In addition, regarding the <segment> deletion processing when the total duration exceeds the threshold, deletion was performed from <section> with lower priority, but a threshold was set for the priority of <section>, and the threshold was set. The least significant <segment> may be deleted from all <section> s below the value. Furthermore, a threshold may be set for the priority of <segment>, and <segment> below the threshold may be deleted.

[Fifth Embodiment]
Hereinafter, a fifth embodiment according to the present invention will be described. In the present embodiment, a moving image of an MPEG1 system stream is assumed as the media content. In this case, the media segment corresponds to one scene cut. In the present embodiment, the score is an objective importance based on the context content in the corresponding scene.

FIG. 36 is a block diagram of a data processing method according to an embodiment of the present invention.
36, reference numeral 1801 denotes a selection step, 1802 denotes an extraction step, 1803 denotes a configuration step, 1804 denotes a delivery step, and 1805 denotes a database. The selection step 1801 performs a process of selecting a scene of the media content from the context description data and outputting the start time and end time of the scene and data representing a file in which the scene is stored. The extracting step 1802 receives the data representing the file output by the selecting step 1801, the start time and the end time, and refers to the physical content description data to separate the media content file from the file of the input start time and end time. The process of extracting the data of the data is performed. The configuration step 1803 performs processing for multiplexing the data output from the extraction step 1802 and configuring an MPEG1 system stream. The delivery step 1804 performs processing of delivering the MPEG1 system stream created by the configuration step 1803 through a line. Reference numeral 1805 denotes a database that stores media contents, their physical content description data, and context content description data.

FIG. 37 shows the structure of the physical content description data in the present embodiment. In the present embodiment, physical contents are described in a tree structure. The storage format of the media content on the database 1805 is not limited to one media content being stored as one file, and one media content may be divided into a plurality of files and stored. Therefore, the root of the tree structure of the physical content description data is expressed as <contents> and represents one content. The root <contents> is given the title of the corresponding content as an attribute. The <contents> child element is <mediaobject>, which represents the stored file. To <mediaobject>, an identifier id is added as an attribute for associating a link locator to a file stored as an attribute with context content description data. In addition, for when the media content is composed of a plurality of files, seq indicating the order of the files in the content is also added as an attribute.

FIG. 38 shows the configuration of context description data in the present embodiment. This is obtained by adding the association with the <mediaobject> of the physical content description data to the context content description data in the first embodiment. That is, the child element of the root <contents> of the context description data is <mediaobject>, and the child element of this <mediaobject> is <section>. <section> and <segment> are the same as in the first embodiment. Correspond to <mediaobject> in the context description data. That is, the scene of the media content described by the descendants of <mediaobject> of the context description data is stored in the file indicated by <mediaobject> of the physical description data having the attribute id of the same value. The time information start and end of <segment> is set to the time from the beginning of each file. That is, when one media content is composed of a plurality of files, the head time of each file is 0, and the start time of each scene is represented by the elapsed time from the head of the file in which it is stored. It shall be.

As an example of expressing the physical content description data and the context content description data in the present embodiment on a computer, a description in Extensible Markup Language (XML) can be used. FIG. 39 shows an example of a Document Type Definition (DTD) for describing the physical content description data shown in FIG. 37 in XML, and an example of the physical content description data based on the DTD. FIGS. 40 to 45 show an example of a DTD for describing the context description data shown in FIG. 38 in XML, and an example of the context description data based on the DTD.

Hereinafter, the processing of the selection step 1801 will be described. As a technique for selecting a scene in the selection step 1801, any one of the techniques described in the first to fourth embodiments is used. However, as a result, the id of <mediaobject> of the corresponding physical content description data is output at the same time as the start time and the end time. FIG. 46 shows an example of the output of the selection step 1801 in the case where the physical content description data is described in the XML document based on the DTD shown in FIG. 39 and the context content description data is expressed in the XML document based on the DTD shown in FIGS. Show. In FIG. 46, the id of <mediaobject> of the physical content description data is described after id =, the start time is described after start =, and the end time is described after end =.

Hereinafter, the process of the extraction step 1802 will be described. FIG. 47 shows a block diagram of extraction step 1802 according to the present embodiment. In FIG. 47, the extraction step 1802 in the present embodiment includes an interface means 2401, a separating means 2402, a video skimming means 2403, and an audio skimming means 2404. The interface means 2401 receives the physical content description data and the output of the selection step 1802 as input, extracts the file of the media content from the database 1805, outputs the data to the separation means 2402, and starts the section output by the selection step 1802. The time and the end time are output to the video skimming means 2403 and the audio skimming means 2404. The separating means 2402 separates the media content into a video stream and an audio stream because the media content in the present embodiment is an MPEG1 system stream in which a video stream and an audio stream are multiplexed. The video skimming unit 2403 receives the separated video stream and the section output by the interface unit 2401 as inputs, and outputs only the data of the selected section from the input video stream. The audio skimming means 2402 receives the separated audio stream and the section output by the selection step 2402 as inputs, and outputs only data of the selected section from the input audio stream.

Hereinafter, processing in the interface means 2401 will be described. FIG. 48 shows a flowchart of the processing of the interface means 2401. The interface means first inputs the physical content description data of the corresponding media content and the output of the selection step 1801 as shown in FIG. Since the time order of the files can be obtained from the attribute id of <mediaobject> of the physical content description data, the output of the selecting step 1801 is sorted in order of time using the id as a key (S1). Further, the data is converted into data as shown in FIG. In this example, files of the same file are put together and arranged in order of start time. Subsequently, the interface means 2401 performs the following processing in order from the top of the data in FIG. First, the file name is acquired from the attribute locator by referring to <mediaobject> of the physical content description data using id. The data of the file with the corresponding file name is read from the database and output to the separating means 2402 (S2, S3). Further, the start time and the end time of the selected section in the file, which are described after the id in FIG. 49, are all output to the video skimming means 2403 and the audio skimming means 2404 (S4). When the above processing has been performed on all data, the processing is terminated (S5). If data still remains, the above processing is repeated after waiting for the end of the processing of the separating means 2402, the video skimming means 2403, and the audio skimming means 2104 (S6, S7).

Hereinafter, the processing of the separating unit 2402 will be described. FIG. 50 shows a flowchart of the processing of the separating means 2402. The separating unit 2402 receives the MPEG1 system stream as the media content from the interface unit 2401, separates the stream into a video stream and an audio stream, and outputs the video stream to the video skimming unit 2403 and the audio stream to the audio skimming unit 2404 (S1). After the output is completed (S9, S11), the interface unit 2401 is notified of the end of the process (S12). As shown in the flowchart of FIG. 50, the same processing as that of the separating means described in the first embodiment is performed except for notification of the end of the processing.

Hereinafter, the processing of the video skimming means 2403 will be described. FIG. 53 shows a flowchart of the processing of the video skimming means 2403. As shown in the flowchart of FIG. 53, the same processing as that of the video skimming means described in the first embodiment is performed, except that a notification of the processing end is sent to the interface means 2401 at the end of the processing (S16, S17). .

Hereinafter, the processing of the audio skimming means 2404 will be described. FIG. 52 shows a flowchart of the processing of the audio skimming means 2404. As shown in the flowchart of FIG. 52, the same processing as that of the audio skimming means described in the first embodiment is performed, except that the processing means is notified to the interface means 2401 when the processing is completed (S11, S12). .

The configuration step 1803 performs time-division multiplexing of the video stream and audio stream output from the extraction step 1802 according to the multiplexing method of the MPEG1 system standardized by the international standard ISO / IEC IS 11172-1. If the media content is divided into a plurality of files and stored, the extraction step 1802 multiplexes each of the files in order to output a video stream and an audio stream for each file.

The delivery step 1804 is for delivering the MPEG1 system stream multiplexed in the configuration step 1803 through a line. When the configuration step 1803 outputs a plurality of MPEG1 system streams, all are delivered in the order of output.

In the present embodiment, when the media content is divided into a plurality of files and stored, a process for each file is performed in the process of the extracting step 1802. The same effect can be obtained even if all the video streams and audio streams are connected and output, and in the configuration step 1803, one MPEG1 system stream is configured by multiplexing the video stream and the audio stream. In this case, the time code changing process in the video skimming means 2403 needs to be performed as follows. That is, a counter C for the number of output frames is prepared for the number of video streams, and initialization of C is performed only for the first file (S18, S3 in FIG. 51). FIG. 53 shows a flowchart of the video skimming means 2403 in this case. Further, in the present embodiment, the context description data and the physical description data are described separately, but by adding the attributes seq and locator of the physical description data as <mediaobject> attributes of the context description data. , May be put together.

[Sixth Embodiment]
Hereinafter, a sixth embodiment according to the present invention will be described. In the present embodiment, a moving image of an MPEG1 system stream is assumed as the media content. In this case, the media segment corresponds to one scene cut. In the present embodiment, the score is an objective importance based on the context content in the corresponding scene.

FIG. 54 is a block diagram of a data processing method according to the embodiment of the present invention.
In FIG. 54, 3101 represents a selection step, 3102 represents an extraction step, 3103 represents a configuration step, 3104 represents a delivery step, and 3105 represents a database. The selecting step 3101 performs a process of selecting a scene of the media content from the context description data and outputting a start time and an end time of the scene and data representing a file in which the scene is stored. This is the same as the selection step described in the embodiment. The extraction step 3102 receives the data representing the file output by the selection step 3101, the start time, and the end time, and refers to the physical content description data to separate the media content file from the file of the input start time and end time. This is the same as the extraction step described in the fifth embodiment. The configuration step 3103 performs a process of multiplexing a part or all of the stream output by the extraction step 3102 according to the line condition determined by the distribution step 3104 to configure an MPEG1 system stream. The delivery step 3104 performs a process of determining the status of the line to be delivered and transmitting the result to the configuration step 3103, and a process of delivering the MPEG1 system stream created by the configuration step 3103 through the line. Reference numeral 3105 denotes a database that stores media content, its physical content description data, and context content description data.

FIG. 55 shows a block diagram of the configuration step 3103 and the delivery step 3104 according to the present embodiment. In FIG. 55, the configuration step 3103 includes a stream selection unit 3201 and a multiplexing unit 3202, and the distribution step 3104 includes a line status determination unit 3203 and a distribution unit 3204. The stream selection unit 3201 receives the video stream and audio stream output from the extraction step 3102 and the line status output from the line status determination unit 3203 as inputs, and if the line is in a state sufficient to transmit all data. , All streams are output to the multiplexing means 3202. If it takes a lot of time to send out all data, such as when the line is congested or the line is small, select a multiplexing unit by selecting only a part of each of the video stream and audio stream. Output to 3202. There are various combinations of selection methods in this case, such as only a base layer stream for a video stream, and only a monaural or only a stereo left or a stereo right for an audio stream. However, when there is only one stream for both the video stream and the audio stream, the stream is output regardless of the line condition. The multiplexing unit 3202 performs time-division multiplexing of the video stream and the audio stream output from the stream selection unit 3201 according to the multiplexing method of the MPEG1 system standardized by the international standard ISO / IDE IS 11172-1. . The line status determination unit 3203 checks the capacity of the line to be delivered, the current usage status, and the like, and outputs it to the stream selection unit 3201. The delivery unit 3204 delivers the MPEG1 system stream multiplexed by the multiplexing unit 3202 via a line.

In the present embodiment, the stream selecting means 3201 outputs a single video stream regardless of the line condition when the video stream is one. However, if it takes a lot of time to send all data through the line, Only the representative image of the video stream may be selected and transmitted. As a method of selecting a representative image, there is a method of describing the time code of the representative image in the context description data, or a method of selecting only an independently decodable I-picture frame among the frames.

[Seventh Embodiment]
Hereinafter, a seventh embodiment according to the present invention will be described. In the present embodiment, a moving image of an MPEG1 system stream is assumed as the media content. In this case, the media segment corresponds to one scene cut. In the present embodiment, the score is an importance based on the viewpoint of keywords such as characters and matters selected by the user or the like in the corresponding scene.

FIG. 56 is a block diagram of a data processing method according to the present embodiment. In FIG. 56, 3301 represents a selection step, and 3302 represents an extraction step. The selection step 3301 performs a process of selecting a scene of the media content from the keywords of the context description data and the scores thereof, and outputting the start time and end time of the scene. The extracting step 3302 performs a process of extracting data of a section of the media content separated by the start time and the end time output by the selecting step 3301.

FIG. 57 shows the structure of the context description data according to the present embodiment. In the present embodiment, context contents are described in a tree structure. Further, it is assumed that the siblings of the tree structure are arranged in chronological order from the left. In FIG. 57, the root of the tree structure described as <contents> represents one content, and the title of the content is given as an attribute.

子 The child element of <contents> is <section>. To <section>, a combination (keyword, priority) of a keyword, which is a keyword indicating the content of the scene, a character, and the like, and a priority, which indicates the importance of the keyword, is added as an attribute. priority is an integer from 1 to 5, where 1 is the least important and 5 is the most important. The (keyword, priority) group is set so that it can be used as a key when searching for a scene, a person, or the like that the user wants to see. Therefore, a plurality of (keyword, priority) pairs can be added to one <section>. For example, when describing a character, the number of (keyword, priority) pairs is added as many as the number of people appearing in the scene, and the priority is high if a person with the corresponding keyword appears in the scene in large numbers, And so on.

子 The child element of <section> is <section> or <segment>. That is, <section> may itself be a child element. However, <section> and <segment> must not be mixed as child elements of one <section>.

<segment> represents one scene cut, and a (keyword, priority) group similar to <section>, and start indicating start time and end indicating end time as time information of the corresponding scene are attributes. Will be added. The method of scene cut may be software that is commercially available or distributed on a network, or may be manually performed. In the present embodiment, the time information is the start time and the end time of the scene cut, but the same effect can be obtained by using the start time and the duration of the scene as the time information. In this case, the end time of the scene is obtained by adding the duration to the start time.

文脈 With this context description data, in the case of a story such as a movie, chapters, sections, paragraphs, etc. can be described by multi-layer <section>. When describing baseball as another example, describe the times in the top <section>, describe the front and back with the child element <section>, describe the scene of each batter with the child element <section>, In addition, the <section> of the child element can describe each pitch, the interval, the result of the turn at bat, and the like.

記述 As an example of expressing the context content description data of this configuration on a computer, a description in Extensible Markup Language (XML) can be used. XML is a data description language being standardized by the World Wide Web Consortium, and Ver. 1.0 was recommended on February 10, 1998. The XML ver. 1.0 specification is available at http://www.w3.org/TR/1998/REC-xml-19980210. FIGS. 58 to 66 show an example of a Document Type Definition (DTD) for describing the context description data of the present embodiment in XML, and an example of the context description data based on the DTD. FIGS. 67 to 80 show contexts in which representative data (dominant-data) of media segments such as representative images (video information) and keywords (sound information) are added to the context description data shown in FIGS. 58 to 66. An example of the content description data and a DTD for describing the context description data in XML.

Hereinafter, the processing in the selection step 3301 will be described. The processing in the selection step 3301 in this embodiment focuses on <section> having <segment> and <segment> as child elements. FIG. 81 shows a flowchart of the process of selection step 3301 in the present embodiment. The selection step 3301 according to the present embodiment is performed by inputting a keyword serving as a key for scene selection and a threshold value of the priority, and extracting the same keyword as the key from <section> having <segment> of context content description data as a child element. And selects a <section> whose priority is equal to or greater than the threshold (S2, S3). Subsequently, of the selected <segments> of <section>, only <segments> having the same keyword as the key and having a priority equal to or greater than the threshold are selected (S5, S6). The start time and end time of the selected scene are obtained from the start and end attributes of the <segment> selected from the above processing and output (S7, S8, S9, S10, S11, S1, S4). ).

In the present embodiment, processing is performed by focusing on <section> having <segment> and <segment> as child elements, but the parent-child relationship between <section> in a certain hierarchy and <section> as its child element The same processing may be performed by focusing on. Also, the parent-child relationship is not limited to two layers, and the same processing may be performed by increasing the number of layers and up to <segment> which is a leaf of the tree structure. Further, the search key may be a set of a plurality of keywords and conditions therebetween. Conditions between keywords include "either", "both", and a combination of "either" and "both". If there are a plurality of keywords, the selection threshold may be specified and processed for each keyword. The keyword serving as the search key may be received by a user input, or may be configured to be automatically set by the system from a user profile or the like.

The operation of the extraction step 3302 is the same as the extraction step described in the first embodiment.

As an effect of the present embodiment, as shown in FIG. 82, the video stream and the audio stream output from the extraction step 3302 are input to the video playback unit and the audio playback unit, respectively, and the video stream and the audio stream are synchronized. By playing back, only the scenes that the individual viewer wants to see of the corresponding media content can be played back. Also, by multiplexing the obtained video stream and audio stream, it is possible to create an MPEG1 system stream of a scene collection that a viewer of the corresponding media content wants to see.

[Eighth Embodiment]
Hereinafter, an eighth embodiment according to the present invention will be described. This embodiment is different from the seventh embodiment only in the processing of the selection step.

Hereinafter, the processing of the selection step 3301 in the present embodiment will be described with reference to the drawings. In the selection step 3301 in the present embodiment, processing is performed by focusing on only <segment>. FIG. 83 shows a flowchart of selection step 3301 in the present embodiment. As shown in FIG. 83, the selection step 3301 in the present embodiment is performed by inputting a keyword serving as a search key and a threshold value of the priority, and having the same keyword as the key from <segment> of the context description data, and And <segment> whose priority is equal to or greater than the threshold value (S1 to S6).

In the present embodiment, the processing is performed by focusing on only <segment>, but the processing may be performed by focusing on <section> of a certain hierarchy. The search key may be a set of a plurality of keywords and conditions between them. Conditions between keywords include "either", "both", and a combination of "either" and "both". If there are a plurality of keywords, the selection threshold may be specified and processed for each keyword.

[Ninth embodiment]
Hereinafter, a ninth embodiment according to the present invention will be described. This embodiment also differs from the seventh embodiment only in the processing of the selection step.

Hereinafter, the processing of the selection step 3301 in the present embodiment will be described with reference to the drawings. In the selection step 3301 according to the present embodiment, similarly to the processing according to the seventh embodiment, the selection is performed by focusing only on <segment> and <section> having <segment> as a child element. In the present embodiment, a threshold value is provided for the sum of the durations of all selected scenes. That is, a selection is made such that the sum of the durations of the scenes selected so far is the maximum below this threshold. FIG. 84 shows a flowchart of the selecting step in the present embodiment. First, a selection step 3301 receives one keyword serving as a search key. Next, of <section> having <segment> as a child element, all those having the keyword of the search key are extracted. This set is Ω (S1, S2). The elements of Ω are sorted in descending order of the priority of the keyword of the search key (S3). Subsequently, a <section> having the highest priority of the keyword of the search key is extracted from the sorted Ω (S5), and the <section> is deleted from Ω (S6). In this case, if there are a plurality of <sections> having the highest priority, all the <section> s are extracted. Of the extracted <section> child elements <segment>, only <segment> having a search key is selected and added to the set Ω '(S7). Note that the initial value of the set Ω 'is an empty set (S2). The sum of the durations of the scene of Ω 'is calculated (S8) and compared with the threshold (S9). If the sum of the durations is equal to the threshold value, all sections of the element <segment> of Ω ′ are output, and the process ends (S14). If the duration is smaller than the threshold value, the process returns from Ω to the selection of <section> having the highest priority of the keyword of the search key (S5), and the above processing is repeated. However, if Ω is an empty set, all sections of the element <segment> of Ω ′ are output, and the process ends (S4). If the sum of the durations of the scene of Ω ′ exceeds the threshold value, the following processing is performed. Of the elements <segment> of the set Ω ', the <segment> with the lowest priority of the keyword of the search key is deleted (S11). In this case, if there are a plurality of <segments> having the smallest priority, all the <segments> are deleted. The sum of the durations of Ω 'is calculated (S12) and compared with a threshold value (S13). If the total of the durations is larger than the threshold value, the process returns to the process of deleting <segment> from Ω '(S11), and this process is repeated. However, if Ω ′ is an empty set, the process ends (S10). If the total of the durations is equal to or less than the threshold value, all the sections of the element <segment> of Ω ′ are output, and the process ends (S14).

In the present embodiment, processing is performed by focusing on <section> having <segment> and <segment> as child elements, but the parent-child relationship between <section> in a certain hierarchy and <section> as its child element The processing may be performed by focusing on. Further, the parent-child relationship is not limited to two layers, and the processing may be performed by further increasing the layers. For example, when processing in the hierarchy from the top <section> to <segment>, first select the top <section>, select the child element <section> from the selected <section>, and select The process of selecting the child element from the selected <section> is repeated until the selection of <segment>, thereby generating a set Ω ′ of the selected <segment>.

Also, in the present embodiment, the priority of the keyword of the search key is set in descending order of priority. However, a threshold may be set for priority, and the priority may be selected in descending order of priority. This threshold may be set separately for <section> and <segment>.

Further, in the present embodiment, the search key is a single keyword, but this may be a set of a plurality of keywords and conditions therebetween. Conditions between keywords include "either", "both", and a combination of "either" and "both". In this case, a rule for determining the priority of the keyword used for selecting or deleting <section> and <segment> is also required. An example of this rule is as follows. That is, when the condition is “either”, the highest value among the priorities of the corresponding keywords is set as the priority. In the case of “both”, the smallest of the priorities of the corresponding keywords is set as the priority. In the case of a combination of “either” and “both”, the value of the priority can be obtained by this rule. Further, even when there are a plurality of keywords of the search key, a threshold value may be set for the priority, and the processing may be performed on a keyword having a priority equal to or higher than the threshold value.

[Tenth embodiment]
Hereinafter, a tenth embodiment according to the present invention will be described. This embodiment is different from the seventh embodiment only in the processing of the selection step.

Hereinafter, the processing of the selection step 3301 in the present embodiment will be described with reference to the drawings. In the selection step 3301 in the present embodiment, as in the eighth embodiment, processing is performed by focusing on only <segment>. As in the ninth embodiment, a threshold value is set for the sum of the durations of all the scenes to be selected. That is, the selection is made such that the sum of the durations of the scenes selected so far is the maximum below this threshold. FIG. 85 shows a flowchart of the selection step in the present embodiment.
First, a selection step 3301 receives one keyword serving as a search key. As initialization, the set Ω ′ is set to an empty set (S2). Next, all of the <segment> that have the keyword of the search key are extracted (S1). Let this set be Ω. The elements of Ω are sorted in descending order of the priority of the keyword of the search key (S3). Next, <segment> having the highest priority of the keyword of the search key is extracted from the sorted Ω (S5), and the <segment> is deleted from Ω. In this case, when there are a plurality of <segments> having the highest priority, all of the <segments> are extracted. If Ω is an empty set, all sections of the element <segment> of Ω ′ are output, and the process ends (S4). The total T1 of the duration of the extracted <segment> and (S6), the total T2 of the duration of the scene of Ω 'are calculated (S7), and T1 + T2 is compared with the threshold (S8). If T1 + T2 exceeds the threshold value, all sections of the element <segment> of Ω ′ are output, and the process ends (S11). If T1 + T2 is equal to the threshold, all the extracted <segments> are added as elements of Ω ′ (S9, S10), and all the sections of the elements <segment> of Ω ′ are output. The process ends (S11). If T1 + T2 is smaller than the threshold value, all the extracted <segments> are added as elements of Ω ′, and the process returns to Ω to select <segment> (S10).

In this embodiment, the processing is performed by focusing on only <segment>, but the processing may be performed by focusing on <section> of a certain hierarchy. Further, in the present embodiment, the priority of the keyword of the search key is set in descending order of priority. However, a threshold may be set for priority, and the priority may be selected in descending order of priority.
Furthermore, in the present embodiment, the search key is a single keyword, but this may be a set of a plurality of keywords and conditions between them. Conditions between keywords include "either", "both", and a combination of "either" and "both". In this case, a rule for determining the priority of the keyword used for selecting or deleting <section> and <segment> is also required. An example of this rule is as follows. That is, when the condition is “either”, the highest value among the priorities of the corresponding keywords is set as the priority. In the case of “both”, the smallest value among the priorities of the keywords is set as the priority. In the case of a combination of “either” and “both”, the value of the priority can be obtained by this rule. Further, even when there are a plurality of keywords of the search key, a threshold value may be set for the priority, and the processing may be performed on a keyword having a priority equal to or higher than the threshold value.

[Eleventh embodiment]
Hereinafter, an eleventh embodiment according to the present invention will be described. This embodiment differs from the context content description data of the seventh to tenth embodiments in the description of the viewpoint to be a keyword for scene selection and the description of its importance. In the seventh to tenth embodiments, as shown in FIG. 57, a set of a keyword and importance (keyword, priority) is assigned to <section>, <segment> as an attribute, and from the viewpoint and from that viewpoint, In the present embodiment, as shown in FIG. 133, the attribute povlist is added to <contents>, and the attribute povvalue is added to <section>, <segment>. Describes perspective and importance.

The attribute povlist represents the viewpoint in a vector format as shown in FIG. 134, and the attribute povvalue represents the importance in a vector format as shown in FIG. 135. The viewpoints and degrees of importance are arranged in order to form an attribute povlist and an attribute povvalue. For example, in FIG. 134 and FIG. 135, the importance regarding viewpoint 1 is 5, the importance regarding viewpoint 2 is 0, the importance regarding viewpoint 3 is 2, and the importance regarding viewpoint n (where n is a positive integer). 0. It should be noted that the importance level 0 relating to viewpoint 2 corresponds to the case where the attribute (keyword, priority) which is a keyword is not added to viewpoint 2 in the seventh embodiment.

FIGS. 136 to 163 and FIGS. 164 to 196 show Document Type Definitions (XML) used to express the context description data according to the present embodiment on a computer. DTD) and an example of context description data according to the DTD. Also in the present embodiment, the same processing as the processing described in the seventh to tenth embodiments is performed using these context description data.

In this embodiment, the attribute povlist is added to <contents>, and the attribute povvalue is added to <section> and <segment>. However, as shown in FIG. 197, <section>, <segment> The attribute povlist may be added to. However, in <section> or <segment> to which the attribute povlist is added, the attribute povvalue corresponds to the attribute povlist added to the <section> or <segment>. In <section> or <segment> where the attribute povlist is not added, even if the attribute povvalue corresponds to the attribute povlist added to <contents>, the attribute povlist is not added <section> Alternatively, among the ancestors of <segment>, the attribute povlist of the closest <section> to which the attribute povlist is added may be used.

FIGS. 198 to 222 and FIGS. 223 to 252 show DTDs corresponding to FIG. 197 for describing context description data in XML used for expressing on a computer, and context contents by the DTD. An example of the description data is shown below. In the examples shown in these drawings, the attribute povvalue of <section> and <segment> to which the attribute povlist is not added corresponds to the attribute povlist added to <contents>.

[Twelfth embodiment]
Hereinafter, a twelfth embodiment according to the present invention will be described. In the present embodiment, a moving image of an MPEG1 system stream is assumed as the media content. In this case, the media segment corresponds to one scene cut.

FIG. 86 is a block diagram of a data processing method according to the embodiment of the present invention. 86, reference numeral 4101 denotes a selection step, 4102 denotes an extraction step, 4103 denotes a configuration step, 4104 denotes a delivery step, and 4105 denotes a database. The selection step 4101 performs a process of selecting a scene of the media content from the context description data, and outputting data indicating a start time and an end time of the scene and a file storing the scene. The extraction step 4102 receives the data representing the file output by the selection step 4101, the start time, and the end time, and refers to the physical content description data to separate the section of the media content file from the input start time and end time. The process of extracting the data of the data is performed. The configuration step 4103 performs processing for multiplexing the data output from the extraction step 4102 to configure an MPEG1 system stream. The delivery step 4104 performs a process of delivering the MPEG1 system stream created by the configuration step 4103 via a line. Reference numeral 4105 denotes a database that stores media contents, their physical content description data, and context content description data.

物理 The structure of the physical content description data in the present embodiment is the same as that described in the fifth embodiment. That is, the physical content description data having the configuration shown in FIG. 37 is used.

FIG. 87 shows the configuration of context description data in the present embodiment. This is obtained by adding the association with the <mediaobject> of the physical content description data to the context content description data in the seventh embodiment. That is, the child element of the root <contents> of the context description data is <mediaobject>, and the child element of this <mediaobject> is <section>. <section> and <segment> are the same as in the seventh embodiment. An attribute id is added to <mediaobject> of the context description data, and the id is used to correspond to <mediaobject> of the physical description data. That is, the scene of the media content described by the descendants of <mediaobject> of the context description data is stored in the file indicated by <mediaobject> of the physical description data having the attribute id of the same value. The time information start and end of <segment> is set to the time from the beginning of each file. That is, when one media content is composed of a plurality of files, the head time of each file is 0, and the start time of each scene is represented by the elapsed time from the head of the file in which it is stored. It shall be.

As an example of expressing the physical content description data and the context content description data in the present embodiment on a computer, a description in Extensible Markup Language (XML) can be used. FIG. 39 shown in the fifth embodiment is an example of the physical content description data. FIGS. 88 to 96 show examples of a DTD for describing the context description data shown in FIG. 87 in XML, and an example of the context description data based on the DTD.

Hereinafter, the processing of the selection step 4101 will be described. As a technique for selecting a scene in the selection step 4101, any of the techniques described in the seventh to tenth embodiments is used. However, as a result, the id of <mediaobject> of the corresponding physical content description data is output at the same time as the start time and the end time. When the physical content description data is represented by the DTD XML document shown in FIG. 39 and the context content description data is represented by the DTD XML document shown in FIGS. 88 to 96, an example of the output of the selection step 4101 is a fifth example. This is similar to the embodiment of FIG. 46 shown in the embodiment.

The processing of the extraction step 4102 is the same as the extraction step described in the fifth embodiment. The configuration step 4103 is the same as the configuration step described in the fifth embodiment. The delivery step 4104 is the same as the delivery step described in the fifth embodiment.

[Thirteenth embodiment]
Hereinafter, a thirteenth embodiment according to the present invention will be described. In the present embodiment, a moving image of an MPEG1 system stream is assumed as the media content. In this case, the media segment corresponds to one scene cut.

FIG. 97 is a block diagram of a data processing method according to the embodiment of the present invention. In FIG. 97, 4401 represents a selection step, 4402 represents an extraction step, 4403 represents a configuration step, 4404 represents a delivery step, and 4405 represents a database. The selecting step 4401 performs a process of selecting a scene of the media content from the context description data and outputting a start time and an end time of the scene and data representing a file in which the scene is stored. This is the same as the selection step described in the embodiment. The extraction step 4402 receives the data representing the file output from the selection step 4401, the start time, and the end time, and refers to the physical content description data to separate the section of the media content file from the input start time and end time. This is the same as the extraction step described in the twelfth embodiment. The configuration step 4403 performs a process of multiplexing a part or all of the stream output by the extraction step 4402 and configuring an MPEG1 system stream according to the line condition determined by the delivery step 4404. It is the same as the configuration step described in the embodiment. The delivery step 4404 is for determining the status of the line to be delivered and transmitting the result to the configuration step 4403, and performing the process of delivering the MPEG1 system stream created by the configuration step 4403 through the line. Is the same as the delivery step described in. Reference numeral 4405 denotes a database that stores media contents, their physical description data, and context description data.

In the present embodiment, the MPEG1 system stream is assumed as the media content, but the same effect can be obtained in other formats as long as the time code of each screen can be obtained.

The following embodiments are intended to describe the summary of embodiments corresponding to the inventions set forth in the claims. Hereinafter, the term "sound information" is used as information on sound including sound, silence, speech, music, silence, external noise, and the like, and the term "video information" is used for moving images, still images, telops, and the like. Used as visual information including characters. The score can be a score calculated from the content of sound information such as sound, silence, speech, music, silence, external noise, or a score attached according to the presence or absence of a telop in video information, or a combination thereof. . The score may be a score other than the above score.

[Fourteenth Embodiment]
Hereinafter, a fourteenth embodiment according to the present invention will be described. FIG. 98 is a block diagram showing processing of the data processing method according to the present embodiment. In the figure, 501 represents a selection step, and 503 represents an extraction step. The selection step 501 is a step of selecting at least one section or scene of the media content from the score of the context description data and outputting the selected section or scene. The selected section is, for example, a start time and an end time of the selected section. Further, the extraction step 503 is a step of performing processing for extracting data of a section (hereinafter, referred to as a media segment) of the media content divided by the selection section output by the selection step 501, that is, only data of the selection section.

The score may be an importance based on the objective importance in the context content, or may be an importance based on a keyword such as a character or a matter selected by a user or the like.

[Fifteenth Embodiment]
Hereinafter, a fifteenth embodiment according to the present invention will be described. FIG. 99 is a block diagram showing processing of the data processing method according to the present embodiment. In the figure, reference numeral 501 denotes a selection step, and 503 denotes a reproduction step. The reproduction step 505 is a step of performing processing for reproducing only data of the selected section divided by the selected section output by the selection step 501. Note that the selection step 501 is the same as the selection steps described in the first to thirteenth embodiments, and thus the description is omitted.

[Sixteenth embodiment]
Hereinafter, a sixteenth embodiment according to the present invention will be described. FIG. 100 is a block diagram showing processing of the data processing method according to the present embodiment. In the figure, reference numeral 507 denotes a video selection step, and 509 denotes a sound selection step. Note that the video selection step 507 and the sound selection step 509 are included in the selection step 501 shown in the fourteenth and fifteenth embodiments.

The video selection step 507 is a step of selecting a section or a scene of the video information with reference to the context description data of the video information, and outputting the selected section. The sound selection step 509 is a step of selecting a section or a scene of the sound information with reference to the context description data of the sound information, and outputting the selected section. The selected section is, for example, a start time and an end time of the selected section. The selection section of the video information selected in the video selection step 507 and the selection section of the sound information selected in the sound selection step 509 are the same as the extraction step 503 or the fifteenth embodiment described in the fourteenth embodiment. In the reproduction step 505 shown in (1), only the data in the selected section is extracted or reproduced.

[Seventeenth embodiment]
Hereinafter, a seventeenth embodiment according to the present invention will be described. FIG. 101 is a block diagram showing processing of the data processing method according to the present embodiment. In the figure, 511 indicates a determination step, 513 indicates a selection step, 503 indicates an extraction step, and 505 indicates a reproduction step.

(Example 1)
First, in the first embodiment, the media content has a plurality of different pieces of media information at the same time, and the determining step 511 determines that the physical content description data describing the data configuration of the media content is input and the capability of the receiving terminal is determined. This is a step of determining which media information is to be selected from determination conditions such as the status of a line to be delivered and a request from a user. The selection step 513 receives the data determined as the selection target in the determination step 511, the physical content description data, and the context content description data, and refers to the input physical content description data to determine the selection target 511. This is the step of performing the selection process only from the data determined to be. Note that the extraction step 503 and the reproduction step 505 are the same as the extraction step shown in the fourteenth embodiment and the reproduction step shown in the fifteenth embodiment, respectively, and thus description thereof will be omitted. The media information includes data such as video information, sound information, and text information. Hereinafter, in the present embodiment, it is assumed that the media information particularly includes at least one of data related to video information and sound information.

Also, in the present embodiment, different video information or audio information at the same time of the media content is assigned to a channel as shown in FIG. 102, and further to a layer in which one channel is hierarchized. Standard resolution video information is assigned to channel 1 and layer 1 for transmission, and high resolution video information is assigned to channel 1 and layer 2. Stereo sound information is assigned to channel 1 for transmitting sound information. 2 is assigned monaural sound information. FIGS. 103 and 104 show an example of a Document Type Definition (DTD) for describing the physical content description data in XML and physical content description data based on the DTD.

Next, the processing of the determination step 511 of this embodiment when the media content has such a channel and layer configuration will be described with reference to FIGS. First, as shown in FIG. 105, in step S101, it is determined whether there is a request from the user. In step S101, if there is a user request, a determination process SR-A based on the user request shown in FIG. 106 is executed.

If there is no user request in step S101, the process proceeds to step S103 to determine whether receivable information is only video information, only audio information, or both video information and audio information. In this step S103, when the receivable information is only the video information, the determination process SR-B for the video information shown in FIG. 107 is executed, and when the receivable information is only the audio information, the determination process SR for the audio information shown in FIG. If -C is executed and both the video information and the sound information are present, the process proceeds to step S105. In step S105, the ability of the receiving terminal to receive the video information and the sound information, for example, the video display ability, the sound reproduction ability, the decompression processing speed of the compressed information, and the like are determined. If lower, the process proceeds to step S109. In step S107, the state of the line transmitting the video information and the sound information is determined. If the line is congested, the process proceeds to step S109, and if not, the process proceeds to step S111.

Step S109 is executed when the capability of the receiving terminal is low or the line is congested. At this time, the receiving terminal transmits the standard resolution video information of channel 1 and layer 1 and the monaural sound information of channel 2 Receive. On the other hand, step S111 is executed when the capacity of the receiving terminal is high and the line is not congested. At this time, the receiving terminal transmits the high-resolution video information of channel 1 and layer 2 and the stereo sound information of channel 1. Receive.

Next, the determination process SR-A based on a user request shown in FIG. 106 will be described. In the present embodiment, the request from the user is to select a video layer and a sound channel. First, in a step S151, it is determined whether or not there is a request for a video by the user. In this step S151, if there is a user request for video, the process proceeds to step S153, and if not, the process proceeds to step S159. In step S153, it is determined whether the user's request for video is to select layer 2. If YES, the flow advances to step S155 to select layer 2 as video information. If NO, the flow advances to step S157 to go to layer 1 Select In the step S159, it is determined whether or not there is a request for a sound by the user. In this step S159, if there is a user request for sound, the process proceeds to step S161, and if there is no user request, the process ends. In step S161, it is determined whether or not the user's request for sound selects channel 1. If YES, the process proceeds to step S163 to select channel 1 as sound information. If NO, the process proceeds to step S165 and channel 2 Select

Next, the determination process SR-B for the video information shown in FIG. 107 will be described. First, in step S171, the capability of the receiving terminal that receives video information is determined. If the capability is high, the process proceeds to step S173, and if low, the process proceeds to step S175. In step S173, the state of the line is determined. If the line is congested, the process proceeds to step S175, and if not, the process proceeds to step S177.

Step S175 is executed when the capacity of the receiving terminal is low or the line is congested. At this time, the receiving terminal receives only the channel 1 and layer 1 standard resolution video information. On the other hand, step S177 is executed when the capacity of the receiving terminal is high and the line is not congested. At this time, the receiving terminal receives only high-resolution video information of channel 1 and layer 2.

Next, the determination process SR-C for the sound information shown in FIG. 108 will be described. First, in step S181, the capability of the receiving terminal that receives the sound information is determined. If the capability is high, the process proceeds to step S183, and if the capability is low, the process proceeds to step S185. In step S183, the state of the line is determined. If the line is congested, the process proceeds to step S185, and if not, the process proceeds to step S187.

Step S185 is executed when the capacity of the receiving terminal is low or the line is congested. At this time, the receiving terminal receives only monaural sound information of channel 2. On the other hand, step S187 is executed when the capacity of the receiving terminal is high and the line is not congested. At this time, the receiving terminal receives only the stereo sound information of channel 1.

(Example 2)
Further, the second embodiment is different from the first embodiment only in the determination step S511. The determination step 511 according to the present embodiment uses the physical content description data describing the data configuration of the media content as input, and determines the video information based on the determination conditions such as the capability of the receiving terminal, the status of the line to be delivered, and the request from the user. It is a step of determining whether to select only the sound information, only the sound information, or any of the video information and the sound information. Note that the selection step 513, the extraction step 503, and the reproduction step 505 are the same as the above-described steps, and thus description thereof will be omitted.

Next, the processing of the determination step 511 of this embodiment will be described with reference to FIGS. First, as shown in FIG. 109, in step S201, it is determined whether there is a request from the user. In step S201, if there is a user request, the process proceeds to step S203. If there is no user request, the process proceeds to step S205. In step S203, it is determined whether the user request is only video information. If YES, the flow advances to step S253 to determine only video information as a selection target, and if NO, the flow advances to step S207. In step S207, it is determined whether the user request is only sound information. If YES, the process proceeds to step S255 to determine only the sound information as a selection target. If NO, the process proceeds to step S251 to perform the video information and the sound information. Are determined to be selection targets.

{Circle around (5)} In step S205, which is advanced when there is no user request, it is determined whether the receivable information is only video information, only audio information, or both video information and audio information. In this step S205, when the receivable information is only the video information, the process proceeds to step S253, and only the video information is determined as the selection target. Is determined, and if both are the video information and the sound information, the process proceeds to step S209.

In step S209, the state of the line is determined. If the line is not congested, the process proceeds to step S251, where both the video information and the sound information are determined as selection targets. In step S211, it is determined whether or not the information delivered via the line includes sound information. If YES, the process proceeds to step S255 to determine the sound information as a selection target. If NO, the process proceeds to step S253. Determines that the video information is to be selected.

(Example 3)
Further, in the third embodiment, the media content has a plurality of different video information and / or sound information at the same time, and the determining step 511 determines that only the video information performed by the determining step 511 of the second embodiment is the sound information. In addition to determining whether only video information or audio information is to be selected, the video information is further determined based on the determination conditions such as the capability of the receiving terminal, the capability of the line to be delivered, and the status of the line. It is determined whether the sound information is to be selected. Note that the selection step 513, the extraction step 503, and the reproduction step 505 are the same as the above-described steps, and thus description thereof will be omitted.

In the present embodiment, as in the first embodiment, different video information or audio information at the same time of the media content is assigned to each channel or layer. For example, channels 1 and layer 1 for transmitting a moving image are assigned to channels and layers. Is standard resolution video information, channel 1 and layer 2 are allocated high resolution video information, and channel 1 for transmitting sound information is stereo sound information and channel 2 is monaural sound information. Have been assigned.

Next, the processing of the determination step 511 of this embodiment will be described with reference to FIGS. As shown in FIG. 111, in the present embodiment, first, information to be selected is determined by the determination step 511 of the second embodiment (selection target determination SR-D). Next, in step S301, the information determined by the selection target determination process SR-D is determined. In this step S301, when the information to be selected is only the video information, the determination process SR-E for the video information shown in FIG. 112 is executed. The determination process SR-F is executed, and if it is both video information and sound information, the process proceeds to step S303. In step S303, the capability of the receiving terminal that receives the video information and the sound information is determined. If the capability is high, the process proceeds to step S305, and if the capability is low, the process proceeds to step S307. In step S307, the capability of the line such as the transmission speed is determined. If the capability is high, the process proceeds to step S309, and if the capability is low, the process proceeds to step S307. In step S309, the state of the line is determined. If the line is congested, the process proceeds to step S307, and if not, the process proceeds to step S311.

Step S307 is executed when the capacity of the receiving terminal is low, the capacity of the line is low, or the line is congested. At this time, the receiving terminal transmits video information of standard resolution of channel 1 and layer 1 and channel 2 monaural sound information. On the other hand, step S311 is executed when the capacity of the receiving terminal is high, the capacity of the line is high, and the line is not congested. At this time, the receiving terminal transmits the high-resolution video information of channel 1 and layer 2 and the channel 1 stereo sound information.

Next, the determination process SR-E for the video information shown in FIG. 112 will be described. First, in step S351, the capability of the receiving terminal that receives the video information is determined. If the capability is high, the process proceeds to step S353, and if the capability is low, the process proceeds to step S355. In step S353, the capability of the line is determined. If the capability is high, the process proceeds to step S357, and if the capability is low, the process proceeds to step S355. In step S357, the state of the line is determined. If the line is congested, the process proceeds to step S355, and if not, the process proceeds to step S359.

Step S355 is executed when the capacity of the receiving terminal is low, the capacity of the line is low, or the line is congested. At this time, the receiving terminal receives only the standard resolution video information of channel 1 and layer 1 I do. On the other hand, step S359 is executed when the capacity of the receiving terminal is high, the capacity of the line is high, and the line is not congested. At this time, the receiving terminal receives only high-resolution video information of channel 1 and layer 2 I do.

Next, the determination process SR-F relating to the sound information shown in FIG. 113 will be described. First, in step S371, the capability of the receiving terminal that receives the sound information is determined. If the capability is high, the process proceeds to step S373, and if low, the process proceeds to step S375. In step S373, the capability of the line is determined. If the capability is high, the process proceeds to step S377, and if the capability is low, the process proceeds to step S375. In step S377, the state of the line is determined. If the line is congested, the process proceeds to step S375, and if not, the process proceeds to step S379.

Step S375 is executed when the capacity of the receiving terminal is low, the capacity of the line is low, or the line is congested. At this time, the receiving terminal receives only monaural sound information of channel 2. On the other hand, step S379 is executed when the capacity of the receiving terminal is high, the capacity of the line is high, and the line is not congested. At this time, the receiving terminal receives only the stereo sound information of channel 1.

(Example 4)
In the fourth embodiment, the representative data of the media segment corresponding to each element of the lowest layer of the context description data is added as an attribute, and the media content has a plurality of different pieces of media information at the same time. The determination step 511 receives the physical content description data describing the data configuration of the media content as input, and determines determination conditions such as the capability of the receiving terminal, the capability of the line to be delivered, and the status of the line, the capability of the line, and the request from the user. , The step of determining which media information and / or representative data is to be selected.

The description of the selection step 513, the extraction step 503, and the reproduction step 505 is omitted. The media information is information such as video information, sound information, and text data. Hereinafter, in the present embodiment, it is assumed that the media information includes at least one of the video information and the sound information. The representative data is, for example, representative image data or low-resolution video data for each media segment in the case of video information, and is key phrase data for each media segment in the case of sound information.

Further, in the present embodiment, similarly to the embodiment, different video information or audio information at the same time of the media content is assigned to each channel or layer. For example, channel 1 for transmitting a moving image, layer 1 Is assigned high-definition video information to channels 1 and 2, stereo sound information is transmitted to channel 1 for transmitting sound information, and monaural sound information is transmitted to channel 2. Is assigned.

Next, the processing of the determination step 511 of this embodiment will be described with reference to FIGS. As shown in FIG. 114, in step S401, it is determined whether there is a request from the user. In step S401, if there is a user request, a determination process SR-G based on the user request shown in FIG. 116 is executed.

In step S401, if there is no user request, the process proceeds to step S403, and it is determined whether the receivable information is only video information, only audio information, or both video information and audio information. In this step S403, when the receivable information is only the video information, the determination process SR-H for the video information shown in FIG. 117 is executed, and when the receivable information is only the audio information, the determination process SR for the audio information shown in FIG. When -I is executed and both the video information and the sound information are present, the process proceeds to step S405 shown in FIG.

Step S405 is a step of determining the capability of the receiving terminal. After executing Step S405, Step S407 of sequentially determining the capability of the line and Step S409 of determining whether the line is congested are performed. The determination step 511 of this embodiment executes these steps S405, S407 and S409 to determine the channel, layer or representative data of the video information and audio information to be received as shown in Table 1 below.

Next, the determination process SR-G based on the user request shown in FIG. 116 will be described. First, in step S451, it is determined whether the request by the user is only video information. If YES, the determination process SR-H for video information is performed, and if NO, the process proceeds to step S453. In step S453, it is determined whether or not the request by the user is only sound information. If YES, a determination process SR-I relating to sound information is performed. If NO, the process returns to the main routine and proceeds to step S405.

Next, the determination process SR-H for the video information shown in FIG. 117 will be described. First, in step S461, the capability of the receiving terminal is determined. After performing step S461, step S463 for sequentially determining the capability of the line and step S465 for determining whether the line is congested are performed. The determination process SR-H relating to the video information of the present embodiment executes these steps S461, S463, and S465, and when the capability of the terminal is high, the capability of the line is high, and the line is not congested, the channel 1 , Only the video information of layer 2 is received (step S471). When the capability of the terminal is low, the capability of the line is low, and the line is not congested, only the representative data of the video information is received (step S473). ). If the above conditions are not satisfied, only the video information of channel 1 and layer 1 is received (step S475).

Next, the determination process SR-I for the sound information shown in FIG. 118 will be described. First, in step S471, the capability of the receiving terminal is determined. After executing step S471, step S473 for sequentially determining the capability of the line and step S475 for determining whether the line is congested are performed. The determination process SR-I relating to the video information in the present embodiment executes these steps S471, S473, and S475, and when the capability of the terminal is high and the capability of the line is high, and when the capability of the terminal is high and the capability of the line is low. When it is low and the line is not congested, only the sound information of channel 1 is received (step S491). When the terminal has a low capacity, the line has a low capacity, and the line is congested, only the representative data of the sound information is received (step S493). If the above condition is not satisfied, only the sound information of channel 2 is received (step S495).

(Example 5)
Further, in the fifth embodiment, the determination step 511 determines the entire data of the media segment from the determination conditions such as the capability of the receiving terminal, the capability of the line to be delivered, the status of the line, the capability of the line, and the request from the user. This is a step of determining whether only the representative data of the media segment to be selected, or the entire data or the representative data of the corresponding media segment is to be selected.

In this embodiment, as in the fourth embodiment, representative data of a media segment corresponding to each element of the lowest layer of the context description data is added as an attribute. If the representative data is video information, For example, it is representative image data or low-resolution video data for each media segment, and if it is sound information, for example, it is key phrase data for each media segment.

Next, the processing of the determination step 511 of this embodiment will be described with reference to FIGS. As shown in FIG. 119, in a step S501, it is determined whether or not there is a request from the user. In step S501, if there is a user request, a determination process SR-J based on the user request shown in FIG. 121 is executed.

If there is no user request in step S501, the flow advances to step S503 to determine whether receivable data is only the representative data of the media segment, only the entire data of the media segment, or both the representative data and the entire data. . In this step S503, when the receivable data is only the representative data, the process proceeds to step S553 shown in FIG. 120, and only the representative data is determined as a selection target. When the receivable data is only the entire data, the process proceeds to step S555 to perform the entire process. It is determined that only the data is to be selected, and if it is both the representative data and the entire data, the process proceeds to step S505.

In step S505, the line capacity is determined. If the line capacity is high, the process proceeds to step S507, and if the line capability is low, the process proceeds to step S509. In both steps S507 and S509, it is determined whether the line is congested. If it is determined in step S507 that the line is not congested, the process proceeds to step S551, where the entire data and the representative data are determined to be selected. If it is determined in step S509 that the line is congested, the flow advances to step S553 to select the representative data. If it is determined in step S507 that the line is congested, or if it is determined in step S509 that the line is not congested, the process proceeds to step S555 to select the entire data.

In the determination process SR-J based on a user request, first, in step S601, it is determined whether the user request is only representative data. If YES, the process proceeds to step S553 to select only the representative data. If it is, the process proceeds to step S603. In step S603, it is determined whether the user request is only the entire data. If YES, the process proceeds to step S555 to select only the entire data. If NO, the process proceeds to step S551 to perform both the entire data and the representative data. Is selected.

[Eighteenth Embodiment]
Hereinafter, an eighteenth embodiment according to the present invention will be described. FIG. 122 is a block diagram illustrating processing of the data processing method according to the present embodiment. In the figure, 501 represents a selection step, 503 represents an extraction step, and 515 represents a configuration step. Note that the selection step 501 and the extraction step 503 are the same as the selection step and the extraction step described in the fourteenth embodiment, and thus description thereof will be omitted.

The configuration step 515 is a step of configuring a media content stream from the data of the selected section extracted by the extraction step 503. In particular, the configuration step 515 multiplexes the data output from the extraction step 503 to configure a stream.

[Nineteenth Embodiment]
Hereinafter, a nineteenth embodiment according to the present invention will be described. FIG. 123 is a block diagram showing processing of the data processing method according to the present embodiment. In the figure, 501 indicates a selection step, 503 indicates an extraction step, 515 indicates a configuration step, and 517 indicates a delivery step. Note that the selection step 501 and the extraction step 503 are the same as the selection step and the extraction step shown in the fourteenth embodiment, and the configuration step 515 is the same as the configuration step shown in the eighteenth embodiment. Description is omitted.

The delivery step 517 is a step of delivering the stream formed by the configuration step 515 via a line. The delivery step 517 may include a step of determining the status of the line to be delivered, and the configuration step 515 may include a step of adjusting the amount of data constituting the file based on the status of the line determined by the delivery step 517.

[Twentieth embodiment]
Hereinafter, a twentieth embodiment according to the present invention will be described. FIG. 124 is a block diagram showing processing of the data processing method according to the present embodiment. In the figure, reference numeral 501 denotes a selection step, 503 denotes an extraction step, 515 denotes a configuration step, 519 denotes a recording step, and 521 denotes a data recording medium. The recording step 519 is a step of recording the stream composed by the composition step 515 on the data recording medium 521. The data recording medium 521 records media content, its context content description data, and physical content description data, and is a hard disk, a memory, a DVD-RAM, or the like. Note that the selection step 501 and the extraction step 503 are the same as the selection step and the extraction step shown in the fourteenth embodiment, and the configuration step 515 is the same as the configuration step shown in the eighteenth embodiment. Description is omitted.

[Twenty-first embodiment]
Hereinafter, a twenty-first embodiment according to the present invention will be described. FIG. 125 is a block diagram illustrating processing of the data processing method according to the present embodiment. In the figure, reference numeral 501 denotes a selection step, 503 denotes an extraction step, 515 denotes a configuration step, 519 denotes a recording step, 521 denotes a data recording medium, and 523 denotes a data recording medium management step. The data recording medium management step 523 is a step of reorganizing already stored media contents and / or newly stored media contents based on the remaining capacity of the data recording medium 521. More specifically, the data recording medium management step 523 includes a step of, when the remaining capacity of the data recording medium 521 is small, a process of accumulating the newly accumulated content after the editing is performed, and a process of accumulating the media content already accumulated. Then, the context content description data and the physical content description data are sent to the selection step 501, and the media content and the physical content description data are sent to the extraction step 503, thereby reorganizing the media content, The content is recorded on the data recording medium 521, and at least one of the processing of deleting the media content before the reorganization is performed.

Note that the selection step 501 and the extraction step 503 are the same as the selection step and the extraction step shown in the fourteenth embodiment, and the configuration step 515 is the same as the configuration step shown in the eighteenth embodiment. The recording step 519 and the data recording medium 521 are the same as the recording step and the data recording medium described in the nineteenth embodiment, and thus description thereof will be omitted.

[Twenty-second embodiment]
Hereinafter, a twenty-second embodiment according to the present invention will be described. FIG. 126 is a block diagram illustrating processing of the data processing method according to the present embodiment. In the figure, reference numeral 501 denotes a selection step, 503 denotes an extraction step, 515 denotes a configuration step, 519 denotes a recording step, 521 denotes a data recording medium, and 525 denotes a stored content management step. The stored content management step 525 is a step of reorganizing the media content stored in the data recording medium 521 according to the storage period. More specifically, the stored content management step 525 manages the media content stored in the data recording medium 521, and, for a media content that has reached a certain storage period, the context content description data and the physical content description data. To the selection step 501 and send the media content and the physical content description data to the extraction step 503 to reorganize the media content, record the reorganized media content on the data recording medium 521, and This is a step of deleting media content before composition.

Note that the selection step 501 and the extraction step 503 are the same as the selection step and the extraction step shown in the fourteenth embodiment, and the configuration step 515 is the same as the configuration step shown in the eighteenth embodiment. The recording step 519 and the data recording medium 521 are the same as the recording step and the data recording medium described in the nineteenth embodiment, and thus description thereof will be omitted.

The selection steps 501 and 513, the extraction step 503, the reproduction step 505, the video selection step 507, the sound selection step 509, the determination step 511, the configuration step 515, the delivery step 517, and the recording step in the above fourteenth to twenty-second embodiments. 519, a data recording medium management step 523, and a stored content management step 525 include a selection unit, an extraction unit, a reproduction unit, a video selection unit, a sound selection unit, a determination unit, a configuration unit, a delivery unit, a recording unit, and a data recording medium management, respectively. As a means and a stored content management means, it can be realized as a data processing device having a part or all of them.

In the above embodiment, the media content may include a data stream such as text data other than the video information and the sound information. In addition, each step of the above-described embodiment may be realized by storing a program for executing all or a part of the operation of the step by a computer in a program storage medium and realizing the software using a computer. It may be realized by using a dedicated hardware circuit that performs a function.

In the above embodiment, the context description data and the physical description data are described in different entities, but may be combined as shown in FIGS. 127 to 132.

As described above, according to the above-described data processing apparatus, data processing method, recording medium, and program, the context content description data having the hierarchical structure is added to the context content description data by the selection unit (selection step). At least one section in the media content is selected based on the selected score. In particular, the extraction means (extraction step) extracts or reproduces only data corresponding to the section selected by the selection means (selection step). By means (reproduction step), only data corresponding to the section selected by the selection means (selection step) is reproduced.

For this reason, a more important scene can be freely selected from the media contents, and the important selected section can be extracted or reproduced. Also, since the context description data has a hierarchical structure consisting of the top layer, the bottom layer, and other layers, scenes can be selected in arbitrary units such as chapters and sections. A variety of selection formats can be used, such as deleting unnecessary paragraphs.

Also, by setting the score to indicate importance based on the contextual content of the media content, and setting this score to select an important scene, for example, a highlight scene collection of a program or the like can be used. Creation can be performed easily, and the score indicates the importance based on the viewpoint of the keyword in the corresponding scene. By determining the keyword, a section having a higher degree of freedom can be selected. For example, by determining a keyword from a specific viewpoint such as a character or a matter, it is possible to select only a scene that the user wants to see.

When the media content has a plurality of different pieces of media information at the same time, the judging means (judging step) judges which media information is to be selected from the judging conditions. ) Performs the selection process only from the data determined by the determination means (determination step). For this reason, since the determining means (determining step) can determine the media information of the optimal division according to the determining condition, the selecting means (selecting step) can select the media information having an appropriate data amount. it can.

Further, since the judging means (judging step) judges whether only the video information, only the sound information, or the video information and the sound information are to be selected from the judging conditions, the selecting means (selecting step) The time required for selecting a section to be performed can be reduced.

{Circle around (4)} Representative data is added to the context description data as an attribute, and the determining means can determine the media information or the representative data in the most appropriate category according to these determination conditions.

Further, the determination means (determination step) determines that only the entire data of the corresponding media segment, only the representative data, or both the entire data and the representative data are to be selected according to the determination condition. The means (judgment step) can reduce the time required for selection of the section performed by the selection means (selection step).

The present invention is useful for a data processing device, a data processing method, a recording medium, a program, and the like that can freely select a required scene from media contents.

FIG. 2 is a block diagram of a data processing method according to the first embodiment of the present invention. FIG. 4 is a diagram illustrating a data structure of context description data according to the first embodiment of the present invention. FIG. 2 is an example of an XML DTD that expresses context content description data on a computer according to the first embodiment of the present invention, and a part of the context content description data written in XML. FIG. It is a continuation part of the context content description data of FIG. It is a continuation of FIG. It is a continuation of FIG. It is a continuation part of FIG. It is a continuation of FIG. It is a continuation part of FIG. It is an example of a part of an XML document in which representative data is added to the context content description data of FIGS. 3 to 9 and a DTD written in XML that expresses the context content description data on a computer. It is a continuation part of the context description data of FIG. It is a continuation part of FIG. It is a continuation part of FIG. It is a continuation part of FIG. It is a continuation part of FIG. It is a continuation part of FIG. It is a continuation part of FIG. It is a continuation part of FIG. It is a continuation part of FIG. It is a continuation part of FIG. It is a continuation part of FIG. FIG. 4 is an explanatory diagram illustrating how to assign importance in the first embodiment of the present invention. It is a flow chart of processing of a selection step in a 1st embodiment of the present invention. FIG. 3 is a configuration diagram of an extraction step according to the first embodiment of the present invention. It is a flowchart of the process of the separation means of the extraction step in the first embodiment of the present invention. It is a flowchart of the process of the video skimming means of the extraction step in the first embodiment of the present invention. FIG. 3 is a configuration diagram of an MPEG1 video stream. It is a flowchart of the process of the audio skimming means of the extraction step in the first embodiment of the present invention. FIG. 3 is a configuration diagram of an AAU for MPEG audio. It is a block diagram of the application in the 1st Embodiment of this invention. It is an explanatory view of processing of importance in a 2nd embodiment of the present invention. It is a flow chart of processing of a selection step in a 2nd embodiment of the present invention. It is a flow chart of processing of a selection step in a 3rd embodiment of the present invention. It is an explanatory view showing how to attach importance in a 4th embodiment of the present invention. It is a flow chart of processing of a selection step in a 4th embodiment of the present invention. It is a block diagram of a data processing method in a 5th embodiment of the present invention. It is a figure showing the data structure of the physical content description data in 5th Embodiment of this invention. It is a figure showing the data structure of the context content description data in 5th Embodiment of this invention. It is an example of the XML DTD which expresses the physical content description data on a computer in 5th Embodiment of this invention, and an example of an XML document. It is a first half of an example of an XML DTD and an XML document that expresses context content description data on a computer according to the fifth embodiment of the present invention. It is a continuation part of the context description data of FIG. 42 is a continuation of FIG. 41. 42 is a continuation of FIG. 42. It is a continuation part of FIG. It is a continuation part of FIG. It is an example of the output of the selection step in the fifth exemplary embodiment of the present invention. It is a block diagram of an extraction step in a 5th embodiment of the present invention. It is a flow chart of processing of an interface means of an extraction step in a 5th embodiment of the present invention. It is an example of the result of having converted the output of the selection step by the interface means of the extraction step in the fifth embodiment of the present invention. It is a flow chart of processing of a separation means of an extraction step in a 5th embodiment of the present invention. It is a flow chart of processing of video skimming means of an extraction step in a 5th embodiment of the present invention. It is a flow chart of processing of audio skimming means of an extraction step in a 5th embodiment of the present invention. It is a flow chart of another processing of video skimming means of an extraction step in a 5th embodiment of the present invention. It is a block diagram of a data processing method in a 6th embodiment of the present invention. It is a block diagram of a composition step and a delivery step in a sixth embodiment of the present invention. It is a block diagram of a data processing method in a 7th embodiment of the present invention. It is a figure showing the data structure of the context content description data in 7th Embodiment of this invention. It is an example of the DTD of XML which expresses the context content description data on the computer in 7th Embodiment of this invention, and a part of context content description data written in XML. It is a continuation part of the context description data of FIG. It is a continuation part of FIG. It is a continuation part of FIG. It is a continuation part of FIG. It is a continuation part of FIG. It is a continuation part of FIG. It is a continuation part of FIG. It is a continuation part of FIG. This is an example of a part of an XML document in which representative data is added to the context content description data of FIGS. 58 to 66, and an example of a DTD written in XML that expresses the context content description data on a computer. This is the continuation of the context description data of FIG. It is a continuation part of FIG. It is a continuation part of FIG. It is a continuation part of FIG. It is a continuation part of FIG. It is a continuation part of FIG. This is a continuation of FIG. 73. This is a continuation of FIG. 74. It is a continuation of FIG. It is a continuation part of FIG. It is a continuation of FIG. 77. It is a continuation of FIG. 78. It is a continuation part of FIG. It is a flow chart of processing of a selection step in a 7th embodiment of the present invention. It is a block diagram of the application in the 7th embodiment of the present invention. It is a flow chart of processing of a selection step in an 8th embodiment of the present invention. It is a flow chart of processing of a selection step in a 9th embodiment of the present invention. It is a flow chart of processing of a selection step in a 10th embodiment of the present invention. FIG. 33 is a block diagram of a data processing method according to a twelfth embodiment of the present invention. FIG. 39 is a diagram illustrating a data structure of context description data according to a twelfth embodiment of the present invention. It is a part of an example of an XML DTD that expresses context description data on a computer according to the fifth embodiment of the present invention, and an XML document. It is a continuation of the example of FIG. 88. It is a continuation of the example of FIG. 89. It is a continuation of the example of FIG. 90. This is a continuation of the example of FIG. 91. It is a continuation of the example of FIG. 92. It is a continuation of the example of FIG. 93. This is a continuation of FIG. 94. It is a continuation part of FIG. FIG. 33 is a block diagram of a data processing method according to a thirteenth embodiment of the present invention. FIG. 35 is a block diagram of a data processing method according to a fourteenth embodiment of the present invention. FIG. 35 is a block diagram of a data processing method according to a fifteenth embodiment of the present invention. FIG. 33 is a block diagram of a data processing method according to a sixteenth embodiment of the present invention. FIG. 51 is a block diagram of a data processing method according to a seventeenth embodiment of the present invention. FIG. 3 is an explanatory diagram showing channels and layers. It is an example of a DTD for describing physical content description data in XML and a part of the physical content description data based on the DTD. This is a continuation part of the physical content description data of FIG. It is a flow chart which shows processing of a judgment step of Example 1 of a 17th embodiment. 31 is a flowchart illustrating a determination process according to a user request performed by a determination step of Example 1 of the seventeenth embodiment. 31 is a flowchart illustrating a determination process regarding video information performed by a determination step of Example 1 of the seventeenth embodiment. It is a flow chart which shows the judgment processing about sound information performed by the judgment step of Example 1 of a 17th embodiment. 39 is a part of a flowchart showing the processing of the determination step in Example 2 of the seventeenth embodiment. 39 is a part of a flowchart showing the processing of the determination step in Example 2 of the seventeenth embodiment. 35 is a flowchart illustrating a process of a determination step in Example 3 of the seventeenth embodiment. 35 is a flowchart illustrating a determination process regarding video information performed by a determination step of Example 3 of the seventeenth embodiment. 39 is a flowchart illustrating sound information determination processing performed by a determination step of Example 3 of the seventeenth embodiment. 39 is a part of a flowchart showing the processing of the determination step in Example 4 of the seventeenth embodiment. 39 is a part of a flowchart showing the processing of the determination step in Example 4 of the seventeenth embodiment. 31 is a flowchart illustrating a determination process according to a user request performed by a determination step in Example 4 of the seventeenth embodiment. 35 is a flowchart illustrating a determination process regarding video information performed by a determination step of Example 4 of the seventeenth embodiment. 39 is a flowchart illustrating sound information determination processing performed by a determination step of Example 4 of the seventeenth embodiment. It is a part of flowchart which shows the process of the determination step of Example 5 of the seventeenth embodiment. It is a part of flowchart which shows the process of the determination step of Example 5 of the seventeenth embodiment. 35 is a flowchart illustrating a determination process according to a user request performed by a determination step of Example 5 of the seventeenth embodiment. [Fig. 129] A block diagram of a data processing method according to an eighteenth embodiment of the present invention. [Fig. 129] A block diagram of a data processing method according to a nineteenth embodiment of the present invention. [Fig. 129] A block diagram of a data processing method according to a twentieth embodiment of the present invention. FIG. 39 is a block diagram of a data processing method according to a twenty-first embodiment of the present invention. FIG. 33 is a block diagram of a data processing method according to a twenty-second embodiment of the present invention. This is an example of a DTD in which context content description data and physical content description data are combined into one, and an XML document. This is the continuation of the XML document in FIG. 127. It is a continuation of FIG. It is a continuation part of FIG. 130 is a continuation of FIG. 130. It is a continuation part of FIG. It is a figure showing the data structure of context description data in the eleventh embodiment of the present invention. FIG. 33 is a diagram illustrating a viewpoint in the eleventh embodiment of the present invention. It is a figure showing the importance in 11th Embodiment of this invention. It is an example of the DTD of XML which expresses the context description data on a computer in the eleventh embodiment of the present invention, and an example of a part of the context description data written in XML. It is a continuation part of the context description data of FIG. It is a continuation part of FIG. It is a continuation part of FIG. It is a continuation part of FIG. 140 is a continuation of FIG. 140. It is a continuation part of FIG. It is a continuation part of FIG. This is a continuation of FIG. This is a continuation of FIG. 144. It is a continuation part of FIG. It is a continuation part of FIG. This is a continuation of FIG. It is a continuation part of FIG. It is a continuation part of FIG. 150 is a continuation of FIG. 150. It is a continuation part of FIG. It is a continuation part of FIG. It is a continuation part of FIG. It is a continuation part of FIG. This is a continuation of FIG. It is a continuation part of FIG. It is a continuation part of FIG. It is a continuation part of FIG. It is a continuation part of FIG. 160 is a continuation of FIG. 160. It is a continuation part of FIG. It is a continuation part of FIG. It is another example of the DTD of XML which expresses the context content description data on the computer in 11th Embodiment of this invention, and a part of context content description data written in XML. This is the continuation of the context description data of FIG. This is a continuation part of the context description data of FIG. This is the continuation of the context description data of FIG. This is the continuation of the context description data of FIG. This is the continuation of the context description data of FIG. This is the continuation of the context description data of FIG. This is the continuation of the context description data of FIG. This is a continuation part of the context description data of FIG. This is the continuation of the context description data of FIG. This is the continuation of the context description data of FIG. This is a continuation part of the context description data of FIG. This is a continuation part of the context description data of FIG. This is a continuation part of the context content description data of FIG. This is the continuation of the context description data of FIG. This is the continuation of the context description data of FIG. This is the continuation of the context description data of FIG. This is the continuation of the context description data of FIG. FIG. 181 is a continuation part of the context description data. This is a continuation part of the context description data of FIG. This is a continuation part of the context description data in FIG. 183. This is the continuation of the context description data of FIG. This is a continuation part of the context content description data of FIG. This is a continuation part of the context description data of FIG. This is the continuation of the context description data of FIG. 187. It is a continuation part of the context content description data of FIG. This is the continuation of the context description data of FIG. This is the continuation of the context description data of FIG. This is a continuation part of the context description data of FIG. This is a continuation part of the context description data of FIG. This is the continuation of the context description data of FIG. 193. This is a continuation part of the context description data of FIG. This is a continuation part of the context description data of FIG. FIG. 39 is a diagram illustrating a data structure of context content description data of another aspect in the eleventh embodiment of the present invention. FIG. 39 illustrates an example of an XML DTD for representing context content description data on a computer and a part of the context content description data written in XML, corresponding to FIG. 197 according to the eleventh embodiment of the present invention. This is the continuation of the context description data of FIG. It is a continuation part of FIG. It is a continuation part of FIG. This is a continuation of FIG. It is a continuation part of FIG. This is a continuation of FIG. This is a continuation of FIG. This is a continuation of FIG. 205. This is a continuation of FIG. 206. This is a continuation of FIG. This is a continuation of FIG. This is a continuation of FIG. It is a continuation part of FIG. This is a continuation of FIG. It is a continuation of FIG. This is a continuation of FIG. This is a continuation of FIG. 214. This is a continuation of FIG. It is a continuation part of FIG. This is a continuation of FIG. This is a continuation of FIG. It is a continuation of FIG. It is a continuation part of FIG. It is a continuation part of FIG. FIG. 39 shows another example of the DTD of XML for expressing the context description data on a computer and a part of the context description data written in XML, corresponding to FIG. 197, in the eleventh embodiment of the present invention. This is a continuation part of the context description data of FIG. This is a continuation of FIG. It is a continuation part of FIG. It is a continuation part of FIG. This is a continuation of FIG. This is a continuation of FIG. It is a continuation part of FIG. It is a continuation part of FIG. It is a continuation part of FIG. It is a continuation part of FIG. This is a continuation part of FIG. This is a continuation part of FIG. This is a continuation part of FIG. It is a continuation part of FIG. This is a continuation of FIG. This is a continuation of FIG. It is a continuation part of FIG. This is a continuation of FIG. It is a continuation part of FIG. This is a continuation part of FIG. This is a continuation part of FIG. This is a continuation of FIG. This is a continuation part of FIG. This is a continuation part of FIG. This is a continuation part of FIG. This is a continuation of FIG. This is a continuation part of FIG. It is a continuation part of FIG. It is a continuation part of FIG.

Explanation of reference numerals

101 Selection Step 102 Extraction Step 501, 513 Selection Step 503 Extraction Step 505 Playback Step 507 Video Selection Step 509 Sound Selection Step 511 Judgment Step 515 Construction Step 517 Delivery Step 519 Recording Step 523 Data Recording Medium Management Step 525 Storage Content Management Step 601 Separation Means 602 Video skimming means 603 Audio skimming means 1301 section
Section 1302 <section>
Section 1301 <section>
1301 leaf <segment>
1801 selection step 1802 extraction step 1803 construction step 1804 delivery step 1805 database 2401 interface means 2402 separation means 2403 video skimming means 2404 audio skimming means 3101 selection step 3102 extraction step 3103 construction step 3104 delivery step 3105 database 3201 stream selection means 3202 multiplexing means 3203 Situation determination means 3204 Delivery means 4101 Selection step 4102 Extraction step 4103 Configuration step 4104 Delivery step 4105 Database 4401 Selection step 4402 Extraction step 4403 Configuration step 4404 Delivery step 4405 Database

Claims (21)

Context content description data in which a segment representing each scene of a media content composed of a plurality of scenes and a score representing importance based on the context content of the media content, which is attribute information of the segment, are input. Means to
A selection unit for selecting a segment based on the score. 2. The scene according to claim 1, wherein each scene of the media content is separated by time according to a break of the scene, and the context content description data describes time information indicating the break of the scene as the attribute information. Data processing device. The data processing apparatus according to claim 2, wherein the time information includes a start time and an end time of each scene. The data processing device according to claim 2, wherein the time information includes a start time and a duration of each of the scenes. 5. The data processing device according to claim 1, wherein the plurality of segments are described hierarchically in the context description data. The data processing device according to any one of claims 1 to 5, wherein the context content description data has auxiliary information relating to context content. 7. The data processing device according to claim 1, wherein the media content is at least one of video information and sound information. 8. The data processing device according to claim 1, wherein a link destination of representative data representing the segment is added to the segment. The data processing apparatus according to claim 8, wherein the representative data is at least one of video information and sound information. Context content description data in which a segment representing each scene of a media content composed of a plurality of scenes and a score representing importance based on the context content of the media content, which is attribute information of the segment, are input. And
A data processing method for selecting a segment based on the score. 11. The scene according to claim 10, wherein each scene of the media content is separated by time according to a break of the scene, and the context content description data describes time information indicating the break of the scene as the attribute information. Data processing method. 12. The data processing method according to claim 11, wherein the time information includes a start time and an end time of each scene. 12. The data processing method according to claim 11, wherein the time information includes a start time and a duration of each of the scenes. 14. The data processing method according to claim 10, wherein a plurality of the segments are described hierarchically in the context description data. The data processing method according to any one of claims 10 to 14, wherein the context content description data has auxiliary information relating to context content. 16. The data processing method according to claim 10, wherein the media content is at least one of video information and sound information. 17. The data processing method according to claim 10, wherein a link destination of representative data representing the segment is added to the segment. 18. The data processing method according to claim 17, wherein the representative data is at least one of video information and sound information. Context content description data describing a segment representing each scene of a media content composed of a plurality of scenes and a score representing importance based on the context content of the media content, which is attribute information of the segment, is input. And a program for causing a computer to execute a step of selecting a segment based on the score. 20. The scene according to claim 19, wherein each scene of the media content is separated by time according to a break of the scene, and the context content description data describes time information indicating the break of the scene as the attribute information. program. A computer-readable storage medium on which the program according to claim 19 or 20 is recorded.

Images