JP2004343781A - Video content caption generating method, video content caption generating unit, digest video programming method, digest video programming unit, and computer-readable recording medium on which program for making computer perform method is stored - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To generate a caption with a reasonable and smooth sentence line for a viewer (user) by making clear previous-and-next connection and relation of the caption generated from each video scene. <P>SOLUTION: A video content caption generating unit 100 is provided with a caption generating section 101, into which two or more textual information consisting of fragmented strings that describe content of each searched video scene are input to generate the caption which describes the video content of the video scene using the textual information; a video content determination section 102 which determines the content of each video scene from the input textual information; and a connection expression select section 103 which selects a connection expression from "SO", "BUT", "AND", "FURTHERMORE", "OR", according to the relation between the previous and the next video scenes, based on the determination result by the video content determination section 102; and the caption generating section 101 uses the connection expression selected with the connection expression select section 103 to connect the captions for the appropriate previous and next video scenes and output them. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

  The present invention describes the video content of each clipped video scene when adding a supplementary information of video (video information) as an index with the digitization of broadcasting and creating a digest version of the video using the index. Method of generating a video content description, a video content description generation device and a recording medium, and a digest video programming method of creating a digest video program using the generated video content description; The present invention relates to a digest video programmer and a recording medium.

2. Description of the Related Art In recent years, digitalization of broadcasting is rapidly progressing on a worldwide scale, and preparations for BS (Broadcast Satellite) digital broadcasting and terrestrial digital broadcasting are steadily progressing. As a result, the television viewing mode changes rapidly, and not only the conventional real-time viewing mode but also the storage-based viewing mode and the non-linear viewing mode are possible.
Here, a description will be given of a digest creation system in a non-linear viewing mode that has been proposed by the present applicants. The present applicant first searches for a video scene to which supplementary information is added as an index, searches for a video scene assumed to be an important scene using the index, and creates a digest version of the video (digest video). We devised a system, and in this digest creation system, the video scenes that were determined to be important scenes also included audio commentary, so it was sufficient to generate a summary of the cross-sectional index as an explanatory sentence. I've been thinking about the process of generating an explanatory note for video content. In addition, when creating a digest video using an index, a digest creation device that creates a digest video that reflects the preference of a viewer (user) who uses the video is also proposed.
The details of the above technology are disclosed in Non-Patent Documents 1 to 3.

Takako Hashimoto, et al .: "Study of Digest Viewing Method Using Program Index", Proceedings of the Society of Image Information and Television Engineers, March 1999, p. 7-12. Takako Hashimoto et al .: "Prototype of Digest Creation Method Using Program Index", Data Engineering Workshop (DEWS'99) Proceedings CD-ROM, March 1999. Takako Hashimoto, et al .: "Trial Production of Digest Creation Method for TV Receiving Terminal", ADBS99 Proceedings, December 1999.

However, the above-described process for generating a description of a video content has the following problems.
First, for each video scene that is a search result, a description is generated independently using each fragmentary index, so that the description before and after and the relevance are unclear. It was not possible for a user (user) to generate a description of the flow of a sentence that was smooth without any discomfort.

  Second, since the description is generated using only the fragmentary index of each video scene as the search result, what meaning does each video scene as the search result have in the preceding and following video scenes? It was not possible to generate a foreword sentence or an afterword sentence as an explanatory sentence to clarify.

  Third, it is possible to create a digest video reflecting the preference of a viewer (user) who uses the video in the digest creation device. However, in the above-described description generation processing of video content, a digest video (video scene) is used. The description is generated only from the fragmentary index given to the user, and the description reflecting the preference of the viewer (user) cannot be generated.

  Furthermore, according to the conventional technology, it is possible to use a digest video created using a digest creation device as a simple program by directly reproducing the digest video, but it is possible to automatically create a program from the digest video, It has not been possible to create a program by giving an effect reflecting the tastes of viewers (users).

  The present invention has been made in view of the above, and clarifies the connection before and after and the relevance of a description generated from each video scene so that a viewer (user) can smoothly flow a sentence without discomfort. The first object is to generate a description sentence.

  In addition, the present invention has been made in view of the above, and as a brief explanatory sentence for clarifying the meaning of each video scene as a search result in the preceding and following video scenes A second object is to enable generation of a preamble sentence or a postscript sentence.

  Further, the present invention has been made in view of the above, and it is a third object of the present invention to be able to generate an explanatory note reflecting the taste of a viewer (user).

  In addition, the present invention has been made in view of the above, and a digest video for automatically creating a program from a digest video and creating a program with an effect reflecting the taste of a viewer (user). It is a fourth object of the present invention to provide a method for converting a program or a program for converting a digest video.

  In order to solve the above-mentioned problem and achieve the object, the invention according to claim 1 provides a fragment for explaining the content of each video scene retrieved as a digest video scene from one video stream. When a plurality of character information including a character string or information that can be converted into a character string is added, a description generating step of generating a description describing the video content of the video scene using the character information is performed. In the video content description generation method having, in the video content determination step of determining the content of each video scene from the character information, based on the determination result of the video content determination step, by the relationship between the previous and next video scene, sequential, A connection expression selection step of selecting a connection expression from among inverse connection, parallel, addition, and selection. In using the selected connection representation, characterized by connecting a description of the front and rear of the video scene corresponding.

  In addition, the invention according to claim 2 provides, for each video scene retrieved as a digest video scene from a video stream structured using a hierarchical structure, a fragmentary character string or A video content description step having a description generation step of generating a description describing the video content of a video scene using the character information when a plurality of character information composed of information that can be converted to a character string is added; In the generation method, a video content determining step of determining the content of each video scene from the character information, and, based on a determination result of the video content determining step, a sequential connection, a reverse connection, a parallel connection, and an addition based on a relationship between preceding and following video scenes. , A connection expression selecting step of selecting a connection expression from the selections, wherein the explanatory sentence generating step includes the connection selected in the connection expression selecting step. With current, characterized by connecting a description of the front and rear of the video scene corresponding.

  According to a third aspect of the present invention, in the method of generating a description of a video content according to the second aspect, the description generating step includes the step of generating the description of a video scene of a certain hierarchy by using the hierarchical structure. And generating an introductory sentence that is an introductory sentence of the explanatory note from the character information of the video scene in the upper hierarchy of the video scene in the hierarchy together with the explanatory text indicating the video content of the video scene in the hierarchy. I do.

  According to a fourth aspect of the present invention, in the method for generating a description of a video content according to the second or third aspect, the description generation step includes the step of: Utilizing a hierarchical structure, together with a descriptive sentence indicating the video content of the video scene of the relevant layer, and generating a postscript to be a postscript of the descriptive sentence from the character information of the video scene of the higher hierarchical level of the video scene of the relevant layer Features.

  According to a fifth aspect of the present invention, in the method for generating a description of a video content according to any one of the first to fourth aspects, the description generation step further includes generating a description of the video scene. The sentence expression of the explanatory note is changed according to the preference of the user using preset user preference information.

  Further, according to a sixth aspect of the present invention, for each video scene retrieved from one video stream as a scene for a digest video, information that can be converted into a fragmentary character string or a character string that describes the content is described. In the case where a plurality of pieces of text information consisting of: are added, the video content description generation device includes a description generation unit that generates a description that explains the video content of the video scene using the text information. From the video content determination means for determining the content of each video scene from the video content determination means, based on the relationship between the previous and next video scene, connected expression from sequential, reverse connection, parallel, addition, selection And a connection expression selecting unit for selecting the connection expression selected by the connection expression selection unit. Characterized in that it connects.

  In addition, according to the invention according to claim 7, for each video scene retrieved as a digest video scene from a video stream structured using a hierarchical structure, a fragmentary character string or When a plurality of character information composed of information that can be converted into a character string is added, a video content description having a description generating means for generating a description describing the video content of the video scene using the character information In the generation device, a video content determining means for determining the content of each video scene from the character information, and, based on a determination result of the video content determining means, a sequential connection, a reverse connection, a parallel connection, an addition, And a connection expression selecting means for selecting a connection expression from the selections, wherein the explanatory sentence generating means uses the connection expression selected by the connection expression selecting means to select a connection expression. Characterized by connecting a description of a video scene.

  According to an eighth aspect of the present invention, in the video content description generating apparatus according to the seventh aspect, the description generating means further includes: By using the structure, a preamble sentence, which is a preamble of the explanation, is generated from the character information of the video scene of the hierarchy higher than the video scene of the hierarchy together with the description indicating the video content of the video scene of the hierarchy. And

  According to a ninth aspect of the present invention, in the video content description generating apparatus according to the seventh or eighth aspect, the description generating means further includes: when generating a description about a video scene of a certain hierarchy, Using the hierarchical structure, a postscript that is a postscript of the explanatory note is generated from the character information of the video scene of the layer higher than the video scene of the layer, together with the description indicating the video content of the video scene of the layer. It is characterized by.

  According to a tenth aspect of the present invention, there is provided a computer readable program recording a program for causing a computer to execute the method for generating a description of a video content according to any one of the first to fifth aspects. Recording medium.

  Further, the invention according to claim 11 causes the user to input each video scene retrieved from one video stream as a scene for a digest video, and a description of the video content created for each video scene, In addition to the reproduction of each of the video scenes, a digest video program creation method for creating a digest video program by providing a description of the video content in audio or text via a preset virtual character. An input step of inputting a degree value of the emotional reaction of the virtual character to the video content of each video scene together with the description of the video scene and the video content, and based on the degree value for each video scene. An effecting step of effecting effecting processing of the emotional expression of the virtual character.

  The invention according to claim 12 is a program for converting a digest video into a digest video program by inputting the description, the preamble, the postscript, and the degree value of each video scene together with the video scene for the digest video. The method, in addition to reproducing each of the video scenes, provides the explanatory note, the preamble sentence, and the postscript sentence via voice through a preset virtual character, and based on the degree value for each video scene. And performing an effecting process of effecting the emotional expression of the virtual character.

  According to a thirteenth aspect of the present invention, each of the video scenes retrieved from one video stream as a scene for a digest video, and a description, an introductory sentence, a postscript sentence of each video scene created in advance, and the video A digest video programmer that inputs a degree value indicating a degree of a user's emotional change with respect to content and creates a digest video program. A video file generating means for generating a video file by associating the description, the preamble, the postscript and the degree value, a program definition file database storing at least various configuration information of a program including at least a virtual character as a program definition file, A plurality of emotion expression levels are set, and one production method is defined for each of the emotion expression levels. An effect definition database storing a template and the video file are input, a degree of emotional expression is determined for each image file based on a degree value, and an emotional expression according to the degree of emotional expression is determined from the effect definition database. Selecting means for selecting an effect template, and inputting the program definition file, the video file, and the effect template. Based on the selected effect template for each video file, at least the reproduction timing of the video scene, An effect processing means for performing a program effect processing for each video file by setting the setting of the explanatory note, the preamble sentence, the postscript sentence and the output timing of the sound output as the sound of the character, and the operation of the virtual character; It is characterized by having.

  According to a fourteenth aspect of the present invention, in the digest video programming apparatus according to the thirteenth aspect, the various configuration information of the program in the program definition file includes at least one virtual character, a studio set of the program, and a camera. It is characterized by comprising information such as the number and position, CG lighting, CG props, sound, program title, supermarket settings, and the like.

  The invention according to claim 15 is the digest video programming device according to claim 13 or 14, wherein the degree value has emotion type information for indicating an emotion type such as emotion, anger, and so on, The effect definition database stores a plurality of effect templates that are classified as emotion type information and the degree of emotional expression as a key index, and the selecting means selects the effect template when the effect value is selected. Based on this, the emotion type information and the degree of emotion expression used as the key index are determined, and all the effect templates are selected from the effect definition database.

  According to a sixteenth aspect of the present invention, in the digest video programming apparatus according to the fifteenth aspect, the degree value can be composed of a plurality of degree values. A plurality of effect templates in which a plurality of emotion type information and a degree of the emotional expression of the plurality of emotion type information are classified as a key index are stored, and the selecting means, when selecting the effect template, Based on the plurality of degree values, a plurality of emotion type information to be used as a key index and a degree of emotion expression of the plurality of emotion type information are determined, and all corresponding effect templates are selected from the effect definition database. And

  According to a seventeenth aspect of the present invention, there is provided the digest video programming apparatus according to any one of the thirteenth to sixteenth aspects, further comprising the step of designating a desired program definition file from a plurality of program definition files. A plurality of program definition files are stored in the program definition file database, and the effect processing means, when performing the program effect processing for each video file, through the specifying means The present invention is characterized in that a designated program definition file is inputted, and the program effect processing for each video file is performed based on various kinds of configuration information.

  The invention according to claim 18 is the digest video programming device according to claim 17, wherein the effect template is set with program environment information to which a defined effect method can be applied, The effect processing means, when there are a plurality of effect templates selected by the selecting means, refers to the program environment information of each effect template, in a program environment provided by a program definition file specified through the specifying means. The present invention is characterized in that one of the executable effect templates is selected, and the program effect processing is performed for each video file.

  According to a nineteenth aspect of the present invention, in the digest video programming apparatus according to the eighteenth aspect, an effect method defined in the effect template is used when one digest image is programmed. Use number limitation information for limiting the number of times can be set, and the effect processing means selects one of the executable effect templates, and then selects the use number limitation information if the effect template includes use number limitation information. The number of times the effect template has been used in the past is compared with the use count limitation information to determine whether or not the effect template can be used, and if not usable, select another executable effect template. Features.

  According to a twentieth aspect of the present invention, in the digest video programming apparatus according to any one of the fifteenth to nineteenth aspects, the program effect processing for each image file in the effect processing means is performed by the selecting means. When the selection of the rendering template for one video file is completed, a sequential processing function for selecting and processing the rendering template to be used, and waiting for the selection means to complete the selection of the rendering template for all the video files, and After selecting an effect template to be used in the file, it has a batch processing function for processing, and the effect processing means, when performing processing using the batch processing function, all of the selected by the selection means With reference to the effect templates, for each set of effect templates having the same emotion type information and emotion expression degree, Is obtained, and if a plurality of different effect templates exist in one set from the set selected a plurality of times, the effect templates are selected such that the number of times of selecting each effect template is uniform. It is characterized by the following.

  According to a twenty-first aspect of the present invention, in the digest video programming apparatus according to the twentieth aspect, the effect template is one of emotion type information and a degree value associated with the degree of emotional expression included in each effect template. It is possible to set designation information that specifies that the video file having the highest degree value or the video file having the lowest degree value be used for the program effect processing, and the effect processing means is provided with the designation information. If the effect template exists, the corresponding effect template is relatively compared with the degree values of all the selected video files, and the corresponding effect template is directed to the video file having the maximum degree value or the minimum degree value. It is characterized in that it is used only for processing.

  The invention according to claim 22 is a computer-readable recording medium on which is recorded a program for causing a computer to execute the digest video programming method according to claim 11 or 12.

  According to the first and second aspects of the present invention, the content of each video scene is determined from the character information, and a connection expression is selected from sequential connection, reverse connection, parallel, addition, selection, etc., based on the relationship between the preceding and following video scenes. Then, using the selected connection expression, a description of the video content that connects the description of the relevant video scene before and after is generated, so the connection and relevance of the description before and after the description generated from each video scene are clarified. Thus, it is possible to generate a description of the flow of the sentence that is smooth for the viewer (user) without discomfort.

  According to the third aspect of the present invention, when generating a description about a video scene of a certain hierarchy of each video scene obtained as a search result from a video stream structured using a hierarchical structure, Using a hierarchical structure, together with a descriptive sentence indicating the video content of the video scene of the relevant layer, to generate a foreword to be a preamble of the descriptive sentence from the character information of the video scene of the upper layer of the video scene of the relevant layer, An introductory sentence can be generated as a brief explanatory sentence for clarifying the meaning of each video scene as a search result in the preceding and following video scenes.

  According to the fourth aspect of the present invention, when generating a description about a video scene of a certain hierarchy of each video scene obtained as a search result from a video stream structured using a hierarchical structure, By using the structure, along with a descriptive sentence indicating the video content of the video scene of the hierarchy and a text postscript to be a postscript of the descriptive sentence from the character information of the video scene of the hierarchy higher than the video scene of the hierarchy, search is performed. A postscript sentence can be generated as a brief explanatory sentence for clarifying the meaning of each of the resulting video scenes in the preceding and following video scenes.

  Further, according to the invention according to claim 5, when generating the description of the video scene, the sentence expression of the description is changed according to the user's preference using the preset user preference information. In order to change the description, it is possible to generate an explanatory note reflecting the taste of the viewer (user).

  According to the inventions according to claims 6 and 7, a video content determining means for determining the content of each video scene from character information, and a relationship between preceding and following video scenes based on a determination result of the video content determining means, Connection expression selecting means for selecting a connection expression from sequential connection, reverse connection, parallel, addition, and selection, and the description generating means uses the connection expression selected by the connection expression selection means to select a connection expression before and after the connection expression. To connect the descriptions of the video scenes, clarify the connection and relevance of the description generated from each video scene and the relatedness, and create a description of the flow of the text that is smooth for the viewer (user) without discomfort Can be generated.

  Further, according to the invention according to claim 8, when generating a description about a video scene of a certain hierarchy of each video scene obtained as a search result from a video stream structured using a hierarchical structure, By using the structure, together with a descriptive sentence indicating the video content of the video scene of the layer and a text information of a video scene of a layer higher than the video scene of the layer, a preamble sentence that becomes a preamble of the description is generated by searching. An introductory sentence can be generated as a brief explanatory sentence for clarifying the meaning of each of the resulting video scenes in the preceding and following video scenes.

  According to the ninth aspect of the present invention, when generating a description about a video scene of a certain hierarchy of each video scene obtained as a search result from the video stream structured using the hierarchical structure, Utilizing this, together with the descriptive text indicating the video content of the video scene of the hierarchy, and the character information of the video scene of the hierarchy higher than the video scene of the hierarchy, to generate a postscript that is the postscript of the description, A postscript sentence can be generated as a brief explanatory sentence for clarifying what each video scene has in the preceding and following video scenes.

  According to a tenth aspect of the present invention, a program for causing a computer to execute the method for generating a description of a video content according to any one of the first to fifth aspects is recorded, and the program is stored in a computer. By reading and executing the method, it is possible to achieve the same effect as the method of generating a description of a video content according to any one of the first to fifth aspects.

  According to the eleventh aspect of the present invention, a degree value of the emotional reaction of the virtual character to the video content of each video scene is input together with the description of the video scene and the video content. A program of a digest video that automatically creates a program from a digest video and creates a program with an effect reflecting the taste of a viewer (user) in order to produce an emotional expression of a virtual character based on the program. Method can be provided.

  According to the twelfth aspect of the present invention, in addition to the reproduction of each video scene, an explanatory sentence, a preamble sentence, and a postscript sentence are provided in audio via a preset virtual character. A digest video that automatically creates a program from a digest video and creates a program that reflects the viewer's (user's) preferences in order to produce an emotional expression of the virtual character based on the degree value. Can be provided.

  According to the thirteenth aspect of the present invention, there is provided a video file generating means for generating a video file by associating a description, a preamble, a postscript, and a degree value for each video scene as a processing unit of program conversion. A program definition file database storing at least various configuration information of a program including at least a virtual character as a program definition file, a plurality of degrees of emotion expression, and an effect template defining one effect method for each degree of emotion expression And input a video file, determine the degree of emotional expression based on the degree value for each video file, and from the effect definition database, create an effect expression template corresponding to the degree of emotional expression from the effect definition database Entering a selection means to select, a program definition file, a video file, and a rendering template, Based on the effect template selected for each video file, at least the playback timing of the video scene, the setting of the explanatory note, preamble sentence, postscript sentence to be output as the sound of the virtual character and the output timing of the sound, and the operation of the virtual character Is set, the effect processing means for performing the program effect processing for each video file is provided, so that the program is automatically created from the digest video and the effect reflecting the taste of the viewer (user) is provided. It is possible to provide a method for converting a digest video into a program in which a program is created.

  According to the fourteenth aspect of the present invention, the various types of configuration information of the program in the program definition file include at least one virtual character, a studio set of the program, the number and position of cameras, CG lighting, CG props, sound, and program. Since it is composed of information such as a title and a supermarket setting, various programs can be constructed by setting or changing these various types of configuration information.

  Further, according to the invention according to claim 15, the degree value has emotion type information for indicating the type of emotion such as emotion, emotion and so on, and the effect definition database stores the emotion type information and the degree of emotion expression in the effect definition database. A plurality of effect templates classified as key indexes are stored, and when selecting an effect template, the selection means determines the emotion type information and the degree of emotion expression to be used as the key index based on the degree value. Since all the effect templates are selected from the effect definition database, expressions of fine emotions such as emotions and emotions can be reflected in the effect.

  According to the sixteenth aspect of the present invention, the degree value can be composed of a plurality of degree values, and the effect definition database includes a plurality of emotion type information and a plurality of emotion type information. A plurality of effect templates in which the degree of expression is classified as a key index are stored, and when selecting an effect template, a plurality of emotion type information to be used as a key index based on a plurality of degree values when selecting an effect template is stored. And to determine the degree of emotional expression of multiple emotion type information, and to select all applicable production templates from the production definition database, multiple emotion types such as "Frustrated and unfortunate" Fine and complex emotional expressions can be reflected in the production.

  According to the seventeenth aspect of the present invention, there is further provided a designation means for designating a desired program definition file from among the plurality of program definition files, and the program definition file database stores a plurality of program definition files. When performing a program production process for each video file, the production processing unit inputs the program definition file specified through the specification unit, and based on the corresponding various configuration information, the video processing unit. , The setting or change of the digest video program can be easily performed.

  According to the eighteenth aspect of the present invention, in the effect template, the program environment information to which the defined effect method can be applied is set, and the effect processing means determines whether the effect template selected by the selecting means is selected. When there are a plurality of effect templates, one of the effect templates executable in the program environment provided by the program definition file specified through the specifying means is referred to by referring to the program environment information of each effect template, and a video file unit is selected. Since the program effect processing is performed, consistency between the effect method defined in the effect template and the environment of the program to be created can be easily achieved. In other words, it is possible to always produce a program with an effect that does not cause discomfort.

  According to the invention according to claim 19, in the effect template, use number limitation information for limiting the number of times the defined effect method is used when one digest video is converted into a program can be set. The effect processing means, after selecting one of the executable effect templates, if the effect template is set to use number limitation information, the effect processing means determines the number of times the selected effect template has been used in the past and the use number limitation information. It is determined whether or not it can be used by comparison, and if it is not available, to select another executable effect template, the same effect template, in other words, the same effect method is repeated to some extent or more. It can be set so as not to be used, and it is possible to avoid making the production rut and prevent the production from getting tired. Further, if the usage count limitation information is set to be used only once, it can be effectively used only once in one digest video program.

  According to the twentieth aspect of the present invention, in the program effect processing for each video file in the effect processing means, when the selection of the effect template of one video file is completed by the selection means, the effect template to be used is selected and processed. And a batch processing function that waits for the selection means to select the effect templates for all video files, selects an effect template to be used for each image file, and then processes the batch. When performing processing using the batch processing function, the effect processing means refers to all effect templates selected by the selecting means, and for each set of effect templates having the same emotion type information and the same degree of emotion expression. The number of times the set was selected is determined, and among the sets selected multiple times, a plurality of different effect templates are included in one set. If there is, the effect template is selected so that the number of selection of each effect template is uniform, so when creating a digest video program, if there is no time, use the sequential processing function to perform the program effect processing Is performed, and if there is a sufficient time, the batch processing function can be used, whereby a natural effect in which repetition is less noticeable can be performed.

  According to the twenty-first aspect of the present invention, the effect template is a video file having the highest degree value or the lowest degree among the degree values associated with the emotion type information and the degree of emotion expression included in each effect template. It is possible to set designation information for designating the use of the video file having the value in the program production process. If there is a production template in which the designation information is set, the production process is selected. Since the degree values of all the video files are relatively compared and the corresponding rendering template is used only for the program rendering processing of the video file having the maximum degree value or the minimum degree value, the rendering template, that is, the rendering effect, is effectively obtained. You can choose the method.

  According to the invention of claim 22, a program for causing a computer to execute the digest video programming method according to claim 11 or 12 is recorded, and the program is read and executed by the computer. The same effect as that of the digest video programming method according to the eleventh or twelfth aspect can be obtained.

  Hereinafter, a method for generating a description of a video content, a method for generating a video description, a method for converting a digest video into a program, a device for converting a digest video into a program, and a computer readable recording program for causing a computer to execute the method will be described. An embodiment of a recording medium will be described in detail with reference to the accompanying drawings.

[Embodiment 1]
FIG. 1 is a schematic configuration diagram of the video content description generating apparatus according to the first embodiment. The video content description generating apparatus 100 according to the first embodiment inputs a plurality of pieces of character information composed of fragmentary character strings that explain the content of each video scene retrieved as a digest video scene from a digest creation engine (not shown). A description generating unit 101 that generates a description describing the video content of the video scene using the text information; a video content determination unit 102 that determines the content of each video scene from the input text information; And a connection expression selection unit 103 for selecting a connection expression from sequential connection, reverse connection, parallel, addition, and selection based on the relationship between the preceding and following video scenes based on the determination result of the determination unit 102.

  Here, it is assumed that a video stream structured using a hierarchical structure is used. For example, in the structuring using a hierarchical structure, the entire image is set as the highest layer, and the highest layer is divided into logically meaningful video scenes (units of the image) to form the next layer. This can be easily realized by sequentially dividing and structuring the video scene so that the scene is further divided into the next layer. Each of the video scenes of the structured video stream has a plurality of pieces of character information composed of fragmentary character strings (or information that can be converted into character strings) describing the contents as indexes. I do.

  A technology for searching for a digest video scene from a structured video stream by a digest creation engine and outputting each searched video scene and a fragmentary character string (character information) describing the content thereof. With respect to the technology, a technology previously filed by the present applicants (for example, Japanese Patent Application No. 11-058916, “Digest Creation Apparatus, Digest Creation Method, and Computer-Readable Computer Recorded with Program for Making Computer Execute the Method”) Recording medium ").

  The description generating unit 101 uses the connection expression selected by the connection expression selecting unit 103 to connect and output the description of the preceding and succeeding video scenes. Further, when generating a description about a video scene of a certain hierarchy, the description generation unit 101 uses a hierarchical structure to provide a description indicating the video content of the video scene of the hierarchy, and a description of the video scene of the hierarchy. From the character information of the video scene of the upper layer, a preamble sentence which is a preamble of the explanation and a postscript which is a postscript of the explanation are generated.

In the above configuration, the operation will be described in the order of (1) connection expression addition processing, and (2) preamble / postscript generation processing.
(1) Connection Expression Addition Processing This connection expression addition processing is executed by the joint work of the video content determination unit 102 and the connection expression selection unit 103. In the process of adding the connection expression, an explanatory sentence is generated from a character string (character information) that describes each video scene, and instead of presenting them continuously, attention is paid to the content relationship between the preceding and following video scenes. Add the appropriate connection between the two legends. As a result, the flow of a sentence composed of a plurality of explanatory sentences in which the explanatory sentences of each video scene are arranged is smoothed, which helps the viewer understand the situation.

  First, a description will be given of a function that inputs character information of two video scenes cut out as digest videos by the digest creation engine, analyzes the contents of the video scenes, and determines the relationship between them. Hereinafter, this function is referred to as a connection relation determination function.

Generally, there are the following five types of connection relations, and the connection relation discrimination function sets any one of these as a return value.
1. Parallel: Represents the meaning of arranging.
Example: Also and / or and.
2. Addition: Represents the meaning of addition.
Example: And, moreover, additionally.
3. Choice: Represents the meaning of choosing one or the other.
Example: Or, or, or, or.
4. Junction: Indicates that what is stated before is the cause or reason for what is stated later.
Example: Therefore, therefore, and therefore, so, then,
Because.
5. Inverse: An indication that what is said before and what is after are inversely related.
Example: But, but, but, but, but, but,
However.

  Character information input from the digest creation engine is given as an argument of the connection relation determination function. In the first embodiment, in addition to a fragmentary character string that describes the contents of a video scene, a digest creation engine calculates the value of an importance determination parameter described later, and generates a video content description generation device as character information. 100, and the video content description generating apparatus 100 uses the value of the importance determination parameter together with the character string as an argument of the connection relation determination function.

Hereinafter, a connection relation determination function for a baseball program will be specifically described as an example. Representative connection expressions in the case of a baseball program include the following additional connection and reverse connection expressions.
* Additional connections between video scenes with additional points:
Example: more → "<more>, one-out, runner second base, third base, Kiyohara home run ,,,,"
* Reverse connection expression when missing a score chance:
Example: However, → "I went to Runner Takahashi 3rd base. <But> I fell to the 4th Kiyohara center fly ,,,,,"

When the video for which the description is to be generated is baseball, the importance determination parameters used in the connection relation determination function are as follows. In addition, each takes a positive value.
* Attack level (importance judgment parameter)
Indicates an aggressively important level. Raises during aggressive events such as hits and home runs.
* Excitement level (importance judgment parameter)
Show audience expectations and excitement. For example, the value increases when the batting order is a turn at the cleanup of the third, fourth or fifth or when the runner is on the third base and has a specific chance.
* Pitcher level (importance judgment parameter)
Shows the pitcher and defensive tone. The value increases during strikes and continuous strikeouts.

  FIG. 2 shows an algorithm of the connection relation determination function. In the example of this algorithm, in order to simplify the explanation, in the case where the class of the structure of the video for which the description is to be generated is a small class such as a bat or pitching class (in other words, the hierarchy of the video scene is a lower hierarchy) Case) and a case such as a large inning class (in other words, a case where the layer of the video scene is a higher layer). In the former, the calculated value is biased by the excitement level using [attack level-pitcher level] as an index (content index level). The magic numbers α, β, and γ are set to 5, 6, and 0, respectively. The relationship between innings is calculated based on the change in score.

  In the case of baseball, the return value of the connection relation discrimination function is either addition or reverse connection. However, in exceptional cases, it is not possible to deny having a value other than this, but in most cases, it is considered that these two types are used. Note that the connection relation determination function does not depend on the viewer's preference. For example, regardless of the fans of either team, reversal is a reverse connection, and adding points is addition.

(2) Foreword / Postscript Generation Processing When generating a description of a certain video scene, the description generation unit 101 converts various situations at that time, preconditions, and the like as a preamble as needed. Present. In addition, a certain video scene is described, and before the description of the next video scene, information on the result of the video scene as a whole is presented as a postscript if necessary. These preamble sentences and postscript sentences are generated from the character information of the parent scene of the corresponding video scene (the video scene of the higher hierarchy) using the hierarchical structure of the video scene.

Specifically, for example, in the case of a baseball video, a preamble sentence (expression of the preamble) is generated from information indicating a situation of a video scene being processed at that time.
For example, as character information added to the parent scene,
・ Score situation ・ Attack team name ・ Out count ・ Base runner ・ Pitcher name ・ Batter name ・ If there is a ball count, such as "5 backs, giant attack, one out, runners 2nd and 3rd base ,," Strings can be automatically generated as preambles.

In addition, the postscript sentence (expression of the postscript) is information about the result, for example,
-Results of Matches-Results of base runners-Information on results such as score results is generated from information indicating the situation at that time.

  As described above, according to the video content description generating method and the video content description generating device of the first embodiment, the content of each video scene is determined from the character information, Select a connection expression from among concatenation, parallel, addition, selection, etc., and use the selected connection expression to generate a description of the video content that connects the description of the relevant video scene before and after. It is possible to clarify the connection and relevance of the description sentence before and after and generate a description of the flow of the sentence that is smooth for the viewer (user) without discomfort.

  In addition, when generating a description about a video scene of a certain hierarchy of each video scene obtained as a search result from a video stream structured using a hierarchical structure, the hierarchical structure is used to generate a description of a higher hierarchy. In order to generate a foreword sentence that is a foreword of the explanatory note from the character information of the video scene, a brief explanation to clarify what each search result video scene has in the preceding and following video scenes A foreword sentence can be generated as a simple sentence. Similarly, in order to generate a postscript sentence that is a postscript of the explanatory note from the character information of the video scene of the upper layer of the video scene of the relevant layer, how each video scene as a search result is A trailing sentence can be generated as a brief explanatory sentence for clarifying the meaning.

[Embodiment 2]
FIG. 3 is a schematic configuration diagram of a video content description generating apparatus according to the second embodiment. The video content description generating apparatus 200 according to the second embodiment inputs a plurality of pieces of character information composed of fragmentary character strings that explain the content of each video scene retrieved as a digest video scene from a digest creation engine (not shown). A description generating unit 201 that generates a description explaining the video content of the video scene using the character information; and a degree of a user's emotional change in the video content (preference level) for each video scene in advance. ), A storage unit 202 in which a plurality of parameters are defined and stored as emotion degree parameters, a setting unit 203 for setting user preference information, an emotion degree parameter and a preference corresponding to each video scene. And a calculation unit 204 that calculates a degree value (preference level value) of a user's emotional reaction to each video scene using the information. It is.

  The plurality of emotion degree parameters are added to the video scene and the degree (preference level) is quantified by a combination of the contents of the character information and the contents of the preference information. The calculation unit 204 calculates a degree value (preference level value) using the digitized degree.

  Further, in the second embodiment, when the description generating unit 201 generates the description explaining the video contents of the video scene using the character information, the degree value ( An emotion expression sentence indicating an emotion expression is added based on the value of the preference level).

In the configuration described above, the operation of the emotion expression generation process (emotional expression addition process), which is a main part of the second embodiment, will be specifically described.
In the emotion expression generation process, when generating a description from the character information of each video scene, not only the facts are objectively stated, but also the expression method is changed using the preference information of the viewer. For example, if the search result is content that is happy for the viewer, an expression full of joy is generated, and if the search result is sad content, an expression expressing sad feeling is generated. In the emotion expression generation process according to the second embodiment, the emotion expression is represented by an explanatory sentence (sentence) using the preference information of the viewer, but the emotion expression generation process itself includes music in a video, color tone of a screen, and the like. Can be reflected in the effect of the virtual character, the expression of the virtual character speaking the explanation, and the like.

  Here, a series of flows of emotion expression generation processing by the storage unit 202, the setting unit 203, the calculation unit 204, and the description generation unit 201 are described as a calculation function of a viewer's preference level (degree) with respect to search results (hereinafter, emotion level). The algorithm will be described below.

  Hereinafter, the case of baseball will be described as an example. FIG. 4 shows an algorithm of the emotion degree discrimination function. The preference level is calculated first from the standpoint that the user is an attacking team fan. When the user's preference set in the preference information is the defense team, the sign is reversed last. In other words, when the attacking team is riding on the flow of attack, the degree of joy (the degree of the user's emotional change: positive change) increases, but the degree of sadness (the user's The degree of emotional change: negative change) increases.

  The value is amplified as the user's preference level increases. The amplification adjustment value φ is set to “5” in the figure. This makes it possible to express emotional changes of the user, for example, when a favorite player is appearing, a good scene is more happy and a bad scene is more sad.

  This algorithm is created on the assumption that the user's preference set by the preference information is the same as the team to which the favorite player belongs and the team to which he supports.

  In the video content description generating apparatus 200 according to the second embodiment, when character information of each video scene is input, the calculation unit 204 reads out the emotion degree parameter corresponding to each video scene from the storage unit 202, and sends the parameter to the setting unit 203. With reference to the set preference information, preference information corresponding to the emotion degree parameter and corresponding character information are set and calculation is performed to obtain the value of the preference level of the target video scene. Next, the description generation unit 201 inputs character information of each video scene to generate a description explaining the video content of the video scene, and based on the preference level (degree value) obtained by the calculation unit 204. An emotion expression sentence indicating an emotion expression is added. For example, when the value of the preference level of the video scene is (preference level> θ), an emotion expression sentence indicating an emotion of being happy is added. If the description is "Two-out, runner third base, Takahashi will reverse in a timely hit", add the emotional expression "I did it." And add "two-out, runner third base, Takahashi. Will be reversed with a timely hit of. I did it. "

  As described above, according to the video content description generation method and the video content description generation device of the second embodiment, the description generation unit generates the description describing the video content of the video scene using the character information. At this time, since an emotional expression sentence indicating an emotional expression is added based on the degree value, it is possible to generate an explanatory sentence that reflects the taste in conformity with the sense of the viewer (user). In other words, by generating a description that reflects the preference more flexibly (or stepwise) in accordance with the degree value of the emotional reaction of the user, a personal description according to the preference of the user is generated. A sentence can be created.

  Further, in the second embodiment, the degree of the plurality of emotion degree parameters is quantified by a combination of the contents of the character information and the contents of the preference information added to the video scene. Is used to calculate the degree value, and it is possible to generate an explanatory sentence reflecting tastes, further adapted to the senses of the viewer (user). In other words, an emotional expression can be added more flexibly (or stepwise) in accordance with the degree of the emotional reaction of the user, and a personalized description reflecting the user's preference is created. be able to.

[Embodiment 3]
In the third embodiment, a method of generating a description of a video content using a description generation algorithm based on a hierarchical structure of the video will be described. FIG. 5 illustrates an explanatory note generation function (explanatory note generation algorithm) according to the third embodiment. As shown in the figure, the description generation function sequentially generates a description by using a recursive call while using the connection relation determination function and the emotion degree determination function described in the first or second embodiment.

  For example, when generating a description for a video scene, it is first checked whether the video scene is at the top of a class (hierarchy). If it is the head, there is no previous video scene, so the connection relation determination function is not called. For each class hierarchy, a preamble sentence and a postscript sentence are added to a set of class instances at the same level. For example, in baseball, a preamble sentence such as "5 backs, giant attack, one out, runners 2 and 3rd base" is generated from character information. As a postscript sentence, a score status at the end of the inning, a brief description of the inning and the like are generated.

  The calculated emotion level value is used in various parts of the explanation generating function. In the generation of the preamble sentence, if the positive value is happy, an expression such as "happily ,,," is added. Conversely, a negative value: If sad, add an expression such as "Unfortunately ,,,". In addition, in the case of generating a postscript sentence, add expressions such as "It was really good" or "It was a completely disappointing result".

  FIG. 6 is an explanatory diagram showing the order in which explanatory texts are generated for a certain game when the explanatory text generation function of the third embodiment is used. If the viewer is a Hiroshima fan, the explanations are as follows (1) to (17). If a corresponding sentence is not required, no explanation is generated. In the figure, the order in which explanatory sentences are generated is described using arrows and numbers.

(1) On October 3, a match between Hiroshima and Giants was held at Tokyo Dome.
(2) One table, Hiroshima attack,
(4) At Eto's turn at bat, there was a solo home run.
(5) It was good.
(6) After finishing the table once, Hiroshima preempted 1-0 by Eto's home run.
You.
(7) Unfortunately, one time, it was immediately reversed by a giant
Was.
(8) First (9) Leader Batter Kawasho made a hit in front of the center.
(10) The giant's counterattack has begun.
(11) Further (12) Matsui is on the foreball.
(13) Runner, first and second base. Hiroshima, pinch.
(14) In addition, unfortunately (15) Takahashi's timely hit gives 2 points. It is a reversal with giant 1-2.
(17) I finished the back once, and I was turned over by Hiroshima 1: 2 and a giant. True
I'm sorry.

  As described above, according to the third embodiment, the description of the video content is generated by using the description generation algorithm based on the hierarchical structure of the video. Therefore, in addition to the effects of the first and second embodiments, The description can be more clearly created using the hierarchical concept expression, and the sentence can be made more readable. In particular, according to the third embodiment, it is possible to obtain an effect that the hierarchical structure of a video can be used for general purposes.

  Here, a description will be given of a case where the method for generating a description of a video content and the video content description generating apparatus according to the first to third embodiments are applied to a digest creation system. FIG. 7 is a schematic diagram of a digest creation system that incorporates the video content description generation method of the present invention as a video text generation function, and outputs a scene (video scene) cut out by the digest generation engine and a brief description thereof. After that, an overview of the entire system up to how it is finally displayed on the TV device will be described. In the figure, a user interface (UIF) for performing an operation interactively with a TV viewer is called a program viewing user interface, and is hereinafter abbreviated as PV (Program Viewer).

  The description (character string) and the video scene generated by the digest creation engine are input to a description generation function, and are generated as a description including a connection expression, an emotion expression, and a structural expression. The generated description, each video scene, the calculated connection type and preference level are passed to the PV.

  Since PV is a user interface for viewing TV programs, it is required to have an action description capability that can be performed in a TV program scenario. TVML is known to satisfy this requirement. The technology of TVML is described in Hayashi, Orihara, Shimoda, et al .: "Language specifications and CG description method of TV program description language TVML", 3rd Intelligent Information Median Symposium, pp. 75-80, 1997. It is described in.

TVML is a language widely described and widely used as a language for describing a scenario of a TV program. Therefore, the PV interpreter can use the TVML interpreter as a TV program description language. The following functions of the TVML can be realized by calling the TVML interpreter from the PV interpreter.
* Selection and arrangement of CG characters and movements in scenarios (tilting the head, etc.)
* Camera position setting, switching between multiple cameras, pan / tilt * Video and audio file playback * Video effect * Subtitle display

  The moving image output by the digest creation system is finally reproduced by the moving image reproduction function of TVML. Further, in the PV description language, it is desirable to be able to describe a TV-like effect such as a change in illumination at a scene transition or a zoom-in action of a camera.

  In addition, as shown in the figure, a database of the character design, the character vocabulary thereof, and the like is stored as a TVML library. For example, the PV can search the vocabulary database of the currently selected character, find the dialogue when that character speaks that type of connected language, and embed it in the code. Specifically, in the case of multilingual support, processing is performed such as using the inverse concatenation expression “but”, “but”, “however”, etc. depending on the character.

  In the digest creation system as described above, in addition to realizing a simple video search query, how to present a digest video obtained as a search result in an easy-to-understand manner is a major problem. It is clear that incorporating the sentence generation method and the video content explanation sentence generation device as one explanation sentence generation function is very useful for solving this problem.

  The method for generating a description of video content according to the first to third embodiments described above can be realized by executing a program prepared in advance by a computer according to the above-described description and the procedure shown in each flowchart (algorithm). . This program is provided by being recorded on a computer-readable recording medium such as a hard disk, a floppy (R) disk, a CD-ROM, an MO, and a DVD. Or it can be distributed over a network.

[Embodiment 4]
Embodiment 4 shows a digest video programming method and a digest video programming apparatus of the present invention. The digest video programming apparatus according to the fourth embodiment inputs each video scene retrieved as a digest video scene from one video stream, and a description of the video content created for each video scene. In addition to the reproduction of each video scene, a digest video program is created by providing a description of the video content in voice or text via a preset virtual character. Of the virtual character with respect to the video content of each video scene, and the effect of expressing the emotion of the virtual character is produced based on the degree value for each video scene.

  In addition to the video scene for the digest video, the description, the preamble, the postscript, and the degree value of each video scene generated by the video content description generation device 200 according to the second embodiment are input, and the digest video program is displayed. create. At this time, in addition to the reproduction of each video scene, an explanatory sentence, an introductory sentence, and a postscript sentence are provided in audio via a preset virtual character, and the emotion of the virtual character is determined based on the degree value for each video scene. This is to produce an expression.

  FIG. 8 is a block diagram showing a digest video programmer 400 according to the fourth embodiment. Note that reference numeral 200 denotes the video content description generating apparatus according to the second embodiment described above. As a precondition, each video scene searched by the video content description generating apparatus 200 as a scene for a digest video is used. A description value, an introductory sentence, an introductory sentence, and a degree value indicating the degree of emotional change of the user with respect to the video content and a super (caption) are generated, and these six pieces of information are converted into a digest video programmer 400. Shall be passed to

In the fourth embodiment, these six pieces of information will be described in the following manner.
1) Video scene (video scene)
2) Foreword explanation (foreword)
3) Event description (description)
4) Postscript explanation (postscript)
5) Super (super)
6) Emotion level parameter (degree value having emotion type information)

  However, of these six pieces of information, information other than the video scene may be generated as needed, or information may not be set as needed. Here, the emotion level parameter has emotion type information for indicating the type of emotion such as emotion, anger, and so on. Emotion type information includes specific types of emotion, such as "happy," "fun," "smiley," "surprised," "sad," "frustrated," "sorry," "relief," etc. Is set.

  Further, the emotion level parameter can be composed of a plurality of emotion level parameters. For example, an emotion level parameter having an emotion type information of "regret" and an emotion level parameter having an emotion type information of "sorry" The emotion level parameter of one video scene may be configured using the two emotion level parameters. By using a plurality of emotion level parameters in this way, it is possible to express an emotion such as "regretful and disappointing" by combining the contents of each emotion type information and use it as information.

  The digest video programming apparatus 400 according to the fourth embodiment includes a video file generation unit 401, a program definition file database 402, an effect definition database 403, an effect template selection unit 404, an effect processing unit 405, and a PVML interpreter 406. , A TVML player 407, and a TV (television: display device) 408. Although not shown in the drawing, a designation unit for designating a desired program definition file from a plurality of program definition files described later is provided. This designation means can be easily constituted by a display screen of a personal computer, a keyboard or the like.

  The video file generation unit 401 inputs an event description, a preamble description, a postscript description, a super (caption), and an emotion level parameter for each video scene from the video content description generation device 200, and processes the program for processing. Then, a video file is generated by associating an event description, a preamble description, a postscript description, and an emotion level parameter for each video scene.

  The program definition file database 402 stores various types of program configuration information as program definition files. Examples of the various types of program configuration information in the program definition file include at least one virtual character, a studio set of the program, and a camera. There is information such as the number and position, CG lighting, CG props, sound, program title, supermarket settings, and the like. It should be noted that a plurality of the program definition files are stored in advance, and addition and change can be easily performed by setting various types of configuration information in a predetermined format.

  The effect definition database 403 stores a plurality of effect templates, and each effect template has one at least for each degree of emotional expression set in a plurality of stages (for example, very, usually, a few three stages). The production method is defined. These effect templates are classified and stored using emotion type information and the degree of emotion expression as key indices.

  The effect definition database 403 also stores a plurality of effect templates in which a plurality of emotion type information and a degree of emotion expression of the plurality of emotion type information are classified as a key index.

  Furthermore, in the effect template, program environment information indicating a program environment to which the defined effect method is applicable is set, and the defined effect method is used several times in one digest video program. Use number limitation information for limiting whether or not it is possible is set as necessary.

  The effect template is a program of a video file having the highest emotion level parameter or a video file having the lowest emotion level parameter among emotion level parameters associated with the emotion type information and the degree of emotion expression included in each effect template. Specifying information for specifying the use for the effect processing can be set as necessary.

  The effect template selection unit 404 inputs a video file, determines the degree of emotion expression based on the emotion level parameter for each image file, and from the effect definition database 403, the effect template of the emotion expression according to the degree of emotion expression. Select Specifically, when selecting an effect template, the emotion type information and the degree of emotion expression to be used as a key index are determined based on the emotion level parameter, and all applicable effect templates are selected from the effect definition database 403. I do.

  When the emotion level parameter of one video file is composed of a plurality of emotion level parameters, the effect template selection unit 404 selects a key based on the plurality of emotion level parameters when selecting the effect template. A plurality of emotion type information to be used as an index and a degree of emotion expression of the plurality of emotion type information are determined, and all corresponding effect templates are selected from the effect definition database.

  The effect processing unit 405 inputs the program definition file, the video file, and the effect template, and based on the effect template selected for each image file, at least the reproduction timing of the video scene and the event description to output as the sound of the virtual character. By setting the sentence, the preamble description, the postscript description, the audio output timing, and the operation of the virtual character, the program effect processing is performed for each video file. At this time, the program definition file to be used uses the program definition file specified via the specifying means.

  In the above configuration, the outline of the process of the digest video programmer 400 will be described with reference to FIG. The digest video programmer 400 first inputs the input files (foreword description, event description, postscript, super, emotion level parameter) generated by the video content description generator 200, and the video (each for digest video). Video scene). Naturally, a plurality of video scenes for digest video are output, and the input file is generated and output for each video scene. Depending on the video scene, there may be no foreword explanation and afterword explanation.

  Further, as described in the first to third embodiments, the above three explanatory sentences already include an emotion expression, a connection expression, and a hierarchical structure expression. The emotion level parameter (degree value having emotion type information), which is the basis for creating the emotion expression in the video content description generation apparatus 200, is used as it is by the digest video programmer 400 for use in determining the production of the video scene. Passed.

  As shown in S901 in FIG. 9, the video file generation unit 401 generates a video file in correspondence with the input video scene, preamble description, event description, postscript description, super, and emotion level parameter.

  The effect template selection unit 404 determines an emotion ID (degree of emotion expression) from an emotion level parameter for each video file (that is, for each video scene) as shown in S902 to S904 in FIG. Specifically, the application range of the numerical value (level value) of the emotion level parameter is defined for each emotion ID in advance as the emotion expression definition file, and the emotion ID belonging to the emotion level parameter is determined for each scene (S902). , S903), and selects all applicable effect templates from the effect definition database 403 using the obtained emotion ID as a key index (search key).

  Here, when the emotion level parameter is composed of a plurality of emotion level parameters, the emotion ID in which the plurality of emotion level parameters are defined is targeted, and all the emotion levels are set using the plurality of emotion level parameters as a key index. An emotion ID that matches the parameter is determined, and all corresponding effect templates are selected from the effect definition database 403. For example, if the emotion level parameters are two of p1 and p2, and if it is in the range of (p1: -5 to -3) and (p2: 5 to 6), the emotion ID is set to "regretful and disappointing."

  In the emotion expression definition file, a plurality of definitions of the emotion ID and the value range pattern of the emotion ID are arranged, and the emotion ID that matches the pattern first is selected in order from the top.

Next, the effect template selection unit 404 selects an effect template corresponding to the selected emotion ID. Basically, the relationship between the emotion ID and a certain effect template prepared in advance is a one-to-many relationship. The reason why a plurality of effect templates are prepared for one emotion ID is to provide a variety of effects so as not to be a boring program. For example, for the emotion ID “Very happy”, by preparing a set of four production templates with the following production methods defined, when the “Very happy” scene comes, Can select one effect template.
(Direction Method 1) Make your face bright red and stand up (Direction Method 2) Weep with tears (Direction Method 3) Sing for three years (Direction Method 4) Break a ball and fly a dove

  A point to be noted when defining an effect template is to first set an effect environment framework (program environment information). For example, it is necessary to determine how many virtual characters to appear, what to use as props, and the like. Similarly, it is necessary to set the program environment in the program definition file. By setting the program environment (effect environment) in both the effect template and the program definition file in this way, the same environment can be used consistently in one program.

  For example, it is initially determined that there are two caster virtual characters, and if a corresponding program definition file is determined, then only effects (effect templates) that match the environment framework of two people are combined. An environment identifier (program environment information) is described in the effect template and the program definition file, and is used to confirm that the environment is the same.

In the fourth embodiment, the effect template and the program definition file are described in PVML. When creating an effect template, an effect is defined using the following two types of variables.
(Variable 1) Information passed from the video content description generation device 200
Example: "Event description" is a variable & etnetscript
(Variable 2) Items defined in the program definition file
Example: The virtual character is & Castnn (nn is a subscript)
Music and sound effect files & Soundnn

Since the effect template simply writes PVML code using the defined variables, synchronization between contents can be freely described. For example, the following synchronization method can be considered.
(1) First, a virtual character speaks a foreword explanation.
(2) Next, the following is performed in parallel.
(2a) Reproduction of a video scene (2b) Talking of an event description (2c) Super (caption) display (3) Then, a postscript description is spoken.

  The effect processing unit 405 executes S905 shown in FIG. First, a program definition file, a video file, and a rendering template are input, and at least the playback timing of a video scene and an event description and a preamble description output as a virtual character sound based on the rendering template selected for each video file. By setting the sentence, the postscript description, the output timing of the sound, and the operation of the virtual character, the program effect processing is performed for each video file. At this time, the program definition file to be used uses the program definition file specified via the specifying means.

  Further, when there are a plurality of effect templates selected by the effect template selection unit 404, the program environment information of each effect template is referred to and matched (matched) with the program environment of the program definition file specified via the specifying means. Then, one of the effect templates (that is, executable effect templates) that match is selected, and a program effect process is performed for each video file.

  Further, after selecting one of the executable effect templates, the effect template selecting unit 404 sets the number of times the selected effect template has been used in the past and the number of times of use in the case where the effect template has been set with use number limitation information. The information is compared with the information to determine whether or not it is usable. If not, another executable effect template is selected.

  Specifically, the rendering processing unit 405 determines the rendering template of each video scene, and then embeds actual data in the above-described variables of the rendering template (PVML code) of each video scene while referring to the program definition file. Go. The processing flow of FIG. 9 illustrates batch processing for finally creating a PVML code in a lump. On the other hand, when it is desired to proceed with the process interactively with the program user, a sequential process of generating and executing a PVML code for each video scene is performed.

  Further, as shown in S906 in FIG. 9, a separate definition group file for creating a subroutine for sharing a group of actions and effects is created, and a separate definition group file in which the effects are encapsulated is designated to perform a series of effects. You can also choose.

  As described above, when the selection of an effect template for one video file is completed, the effect processing unit 405 selects the effect template to be used as a program effect process for each image file, After the effect template selection is completed, a batch process is performed for selecting an effect template to be used in each video file and then processing the selected effect template.

  As another modified example of performing the batch processing, for example, with reference to all the effect templates selected by the effect template selecting unit 404, for each set of effect templates having the same emotion type information and the same degree of emotion expression, The number of times the set is selected is determined, and when a plurality of different effect templates exist in one set among the sets selected multiple times, the effect templates are selected such that the number of times of selecting each effect template is uniform. May be selected. In other words, the number of times selected for each emotion ID is determined, and among the emotion IDs selected a plurality of times, for the emotion IDs that select a plurality of effect templates, the number of times each effect template is selected is made uniform. Select an effect template.

  Further, as another modified example of performing the batch processing, the effect processing unit 405 determines that, when there is an effect template in which the specified information is set, the effect processing unit 405 determines whether the corresponding effect template is selected for all the video files. The emotion level parameters may be relatively compared, and the corresponding effect template may be used only for the program effect processing of the video file having the maximum emotion level parameter or the minimum emotion level parameter.

  Next, an example of a digest video program screen displayed on the TV 408 according to the fourth embodiment will be described with reference to FIGS. As shown in the figure, the screen of the TV 408 (PVUI screen) includes a material display area 1001 for displaying video and displaying subtitles and superimposed characters, and displaying a motion of a virtual character and a studio effect (including a set, lighting, and a camera position). It is composed of three logical areas including a studio area 1002 to be used and an operation menu area 1003 used for selecting an operation menu by a user (viewer).

  In the fourth embodiment, the number of the above-mentioned areas is one each, and a tiled layout without overlapping is adopted. The reason why the tiled layout is used as the multi-window display mode is that it is easier for users who are unfamiliar with computers to adjust to the display mode, depending on the computer operation skills of the user. It may be selectable.

FIG. 11 shows an example of a final PVML code created as a digest video program by the digest video programmer 400 according to the fourth embodiment.
First, for one video scene of the digest video, the virtual character (BOB) first speaks the preamble description, and then, the virtual character (BOB) speaks the event description and the reproduction of the video scene is performed in parallel. It is described as follows.

  Note that the <head> portion corresponds to the program definition file portion, and the <body> portion is the program body. The parallel processing and the sequential processing are described by <par> and <seq> tags, respectively.

  In principle, the language specification of PVML is <method, target object, parameter sequence related to method>, but since many methods may be described for the target object, the target “<Set>” is prepared as a tag to be specified. Frequently used macros such as hurray movements are separately defined in advance as a PVML library.

The position layout description described in the <head> section will be described. In the layout specification of the head section, a vertical division (<vertical>) is performed on the left and right sides of the screen, and a horizontal division (<horizontal>) is performed on the left half in advance. The size is linked while maintaining the relationship of the divided tree information. Therefore, the operation menu area becomes large and the studio area becomes small by the following size change. In the case of the screen shown in FIG. 10A, the screen is changed to the screen shown in FIG.
<viewchange area = "display" duration = "2"
dstx = "0" dsty = "0" dstheight = "500" dstwidth = "500"/>

  Since the PVML used in the fourth embodiment calls and uses various functions of SMIL and TVML, the contents of the production are restricted by the specifications of SMIL and TVML, but the description language is not particularly limited. It is obvious that other description languages can be applied to the digest video programming method and the digest video programming apparatus of the present invention.

  In the above-described fourth embodiment, the consistency between the explanation of the virtual character (the audio output of the foreword explanation, the event explanation, and the postscript explanation), the reproduction of each video scene of the digest video, and the display of the supermarket is easily achieved. And can be synchronized. This makes it possible to give a presentation that is easy to understand. In the created program, the virtual character is explained and explained by the virtual character, and the degree value (emotion level parameter) calculated by the image content explanation generating device 200 of the second embodiment is used. Since the virtual character is rendered with emotions and emotions, when the created program is evaluated, the emotional expression can help the viewer to comprehend and understand, and it is easy for the viewer to feel comfortable.

  The digest video program conversion method according to the fourth embodiment described above can be realized by executing a previously prepared program on a computer according to the procedure described in the above description. This program is provided by being recorded on a computer-readable recording medium such as a hard disk, a floppy (R) disk, a CD-ROM, an MO, and a DVD. Or it can be distributed over a network.

FIG. 1 is a schematic configuration diagram of a video content explanation sentence generation device according to a first embodiment. FIG. 4 is an explanatory diagram illustrating an algorithm of a connection relation determination function according to the first embodiment. FIG. 9 is a schematic configuration diagram of a video content description generating device according to a second embodiment. FIG. 14 is an explanatory diagram illustrating an algorithm of an emotion degree discrimination function according to the second embodiment. FIG. 14 is an explanatory diagram illustrating an explanatory note generation function (explanatory note generation algorithm) according to the third embodiment; FIG. 14 is an explanatory diagram showing in which order a description is generated for a certain game when the description generation function of the third embodiment is used. BRIEF DESCRIPTION OF THE DRAWINGS FIG. 1 is a schematic diagram of a digest creation system that incorporates a video content description method according to the present invention as a video text generation function. FIG. 13 is a block diagram of a digest video programmer according to a fourth embodiment. It is explanatory drawing which shows the outline | summary flow of a process of the program conversion apparatus of a digest video. FIG. 14 is an explanatory diagram showing an example of a screen of a digest video program displayed on a TV according to the fourth embodiment. FIG. 15 is an explanatory diagram showing an example of a final PVML code created as a digest video program by the digest video programmer of the fourth embodiment.

Explanation of reference numerals

REFERENCE SIGNS LIST 100 video description generator 101 description generator 102 video determination unit 103 connection expression selector 200 video description generator 201 description generator 202 storage unit 203 setting unit 204 calculator 400 digest video generator 401 Video file generation unit 402 Program definition file database 403 Production definition database 404 Production template selection unit 405 Production processing unit 406 PVML interpreter 407 TVML player 408 TV

Claims (22)

For each video scene retrieved as a digest video scene from one video stream, a plurality of pieces of character information composed of a fragmentary character string or information that can be converted into a character string that describes the content are added. In the video content description generation method having a description generation step of generating a description describing the video content of the video scene using the character information,
A video content determining step of determining the content of each video scene from the character information;
Based on the determination result of the video content determination step, according to the relationship between the previous and subsequent video scene, forward, reverse, parallel, addition, connection expression selection step of selecting a connection expression from among the selection,
The method of generating a video content description, wherein the description generation step connects the description of the relevant video scene before and after the connection using the connection expression selected in the connection expression selection step. For each video scene searched as a digest video scene from a video stream structured using a hierarchical structure, a fragmentary character string or a plurality of information that can be converted into a character string that describes the content of the video scene is described. When the text information is added, the video content description generation method having a description generation step of generating a description describing the video content of the video scene using the text information,
A video content determining step of determining the content of each video scene from the character information;
Based on the determination result of the video content determination step, according to the relationship between the previous and subsequent video scene, forward, reverse, parallel, addition, connection expression selection step of selecting a connection expression from among the selection,
The method of generating a video content description, wherein the description generation step connects the description of the relevant video scene before and after the connection using the connection expression selected in the connection expression selection step.   The description generating step includes the step of generating a description of a video scene of a certain hierarchy by using the hierarchical structure, together with a description indicating the video content of the video scene of the hierarchy, 3. The method according to claim 2, wherein a preamble sentence, which is a preamble to the explanation sentence, is generated from the character information of the video scene of the upper hierarchy.   The description generating step includes the step of generating a description of a video scene of a certain hierarchy by using the hierarchical structure, together with a description indicating the video content of the video scene of the hierarchy, 4. The method according to claim 2, wherein a postscript sentence to be a postscript of the explanatory note is generated from the character information of the video scene of the upper hierarchy.   The description generating step further includes, when generating the description of the video scene, changing the sentence expression of the description according to the user's preference using preset user preference information. 5. The method according to claim 1, further comprising the steps of: For each video scene retrieved as a digest video scene from one video stream, a plurality of pieces of character information composed of a fragmentary character string or information that can be converted into a character string that describes the content are added. In the video content description generation device having a description generation means for generating a description describing the video content of the video scene using the character information,
Video content determining means for determining the content of each video scene from the character information,
Based on the determination result of the video content determination means, by the relationship of the previous and next video scene, sequential, reverse connection, parallel, addition, connection expression selection means for selecting a connection expression from among selection,
A video content description generating apparatus, wherein the description generating means connects the description of the preceding and succeeding video scenes using the connection expression selected by the connection expression selecting means. For each video scene searched as a digest video scene from a video stream structured using a hierarchical structure, a fragmentary character string or a plurality of information that can be converted into a character string that describes the content of the video scene is described. When the text information is added, in the video content description generation device having a description generation means for generating a description to explain the video content of the video scene using the text information,
Video content determining means for determining the content of each video scene from the character information,
Based on the determination result of the video content determination means, by the relationship of the previous and next video scene, sequential, reverse connection, parallel, addition, connection expression selection means for selecting a connection expression from among selection,
A video content description generating apparatus, wherein the description generating means connects the description of the preceding and succeeding video scenes using the connection expression selected by the connection expression selecting means.   Further, when generating a description about a video scene of a certain hierarchy, the description generating means uses the hierarchical structure to provide a description indicating the video content of the video scene of the hierarchy and the video of the hierarchy. 8. The video content description generating apparatus according to claim 7, wherein a preamble sentence which is a preamble of the description is generated from the character information of the video scene of a hierarchy higher than the scene.   Further, when generating a description about a video scene of a certain hierarchy, the description generating means uses the hierarchical structure to provide a description indicating the video content of the video scene of the hierarchy and the video of the hierarchy. 9. The video content description generation apparatus according to claim 7, wherein a postscript text that is a postscript of the description text is generated from the character information of the video scene in the upper layer of the scene.   A computer-readable recording medium having recorded thereon a program for causing a computer to execute the method for generating a description of a video content according to claim 1. Each video scene retrieved as a digest video scene from one video stream and a description of the video content created for each video scene are input, and in addition to the reproduction of each video scene, A digest video programming method for creating a digest video program by providing a description of the video content by voice or text via a preset virtual character,
An input step of inputting a degree value of the emotional reaction of the virtual character to the video content of each video scene, together with the description of the video scene and the video content,
An effecting step of performing an effecting process of an emotional expression of the virtual character based on the degree value for each video scene;
A method for converting a digest video into a program. Along with the video scene for the digest video, the description of each video scene, a preamble, a postscript, and a degree value are input, and a digest video programming method of creating a digest video program,
In addition to the reproduction of each of the video scenes, the description, the preamble, and the postscript are provided by voice through a preset virtual character, and the virtual character of the virtual character is provided for each video scene based on the degree value. An effect step for performing an effect expression process,
A digest video program conversion method comprising: Each video scene retrieved from one video stream as a scene for a digest video, and a description of each video scene created in advance, a preamble, a postscript, and a change in the emotional state of the user to the video content. A digest video programming device for inputting a degree value indicating the degree and creating a digest video program,
Video file generating means for generating a video file by associating the explanatory note, preamble sentence, postscript sentence, and degree value for each video scene as a processing unit of program conversion;
A program definition file database storing various configuration information of programs including at least virtual characters as a program definition file,
An effect definition database in which a plurality of emotion expression levels are set, and for each of the emotion expression degrees, an effect template defining one effect method is stored;
Selecting means for inputting the video file, determining a degree of emotional expression based on a degree value for each video file, and selecting an effect expression effect template according to the degree of emotional expression from the effect definition database; and ,
The program definition file, the video file, and the effect template are input, and at least the reproduction timing of the video scene and the description sentence as the sound of the virtual character are output based on the selected effect template for each video file. Effect processing means for performing program effect processing for each video file by setting the sentence, the setting of the postscript sentence and the output timing of the sound, and the operation of the virtual character;
A digest video programmer comprising:   The various types of configuration information of the program in the program definition file include at least one virtual character, and information such as the studio set of the program, the number and position of cameras, CG lighting, CG props, sound, program title, supermarket settings, and the like. The digest video programming apparatus according to claim 13, characterized in that: The degree value has emotion type information for indicating the type of emotion such as emotions and emotions,
The effect definition database stores a plurality of effect templates that are classified as emotion type information and the degree of emotion expression as a key index,
The selecting means, when selecting the effect template, determines the emotion type information and the degree of emotion expression to be used as a key index based on the degree value, and determines all applicable effect templates from the effect definition database. 15. The digest video programmer according to claim 13, wherein the digest video is selected. Further, the degree value can be composed of a plurality of degree values,
The effect definition database stores a plurality of effect templates in which a plurality of emotion type information and a degree of emotion expression of the plurality of emotion type information are classified as a key index,
The selecting means determines a plurality of emotion type information to be used as a key index and a degree of emotional expression of the plurality of emotion type information based on the plurality of degree values when selecting the effect template. 16. The digest video programming apparatus according to claim 15, wherein all of the effect templates are selected from the definition database. Further, a designation unit for designating a desired program definition file from a plurality of program definition files is provided,
The program definition file database stores a plurality of program definition files,
The effect processing means, when performing the program effect processing of the video file unit, input the program definition file specified via the specifying means, based on the various configuration information corresponding, the video file unit 17. The digest video programming device according to claim 13, wherein the program rendering process is performed. In the effect template, program environment information to which a defined effect method can be applied is set,
When there are a plurality of effect templates selected by the selecting means, the effect processing means refers to the program environment information of each effect template, and provides a program environment provided in a program definition file specified via the specifying means. 18. The digest video programming apparatus according to claim 17, wherein one of the rendering templates executable in step (a) is selected, and the program rendering process is performed for each video file. In the effect template, it is possible to set use number limitation information for limiting the number of times the defined effect method is used when making one digest video into a program,
The effect processing means, after selecting one of the executable effect templates, if the effect template has been set with use count limitation information, the selected effect template has been used in the past and the use count limit. 19. The digest video program according to claim 18, wherein the information is compared with the information to determine whether or not the digest video can be used, and if not, another executable effect template is selected. apparatus. The program effect processing for each video file in the effect processing means includes a sequential processing function of selecting and processing an effect template to be used when the selection means has finished selecting an effect template for one video file; It has a batch processing function to wait until the selection of the rendering templates for all video files is completed, select the rendering template to be used for each video file, and then process it.
The effect processing means, when performing processing using the batch processing function, referring to all effect templates selected by the selecting means, the effect type information and the degree of emotion expression of the effect template of the same emotion expression For each set, the number of times that the set is selected is obtained, and when a plurality of different effect templates exist in one set among the sets selected a plurality of times, the number of times of selecting each effect template is made uniform. 20. The digest video programming apparatus according to claim 15, wherein an effect template is selected. The effect template is used for the program effect processing of the video file having the highest degree value or the video file having the lowest degree value among the degree values associated with the emotion type information and the degree of the emotion expression included in each effect template. It is possible to set the specification information that specifies
When there is a rendering template in which the designation information is set, the rendering processing means relatively compares the degree values of all the video files for which the rendering template is selected, and determines the rendering template to be the maximum degree. 21. The digest video programming apparatus according to claim 20, wherein the digest video programming apparatus is used only for program effect processing of a video file having a value or a minimum degree value.   13. A computer-readable recording medium having recorded thereon a program for causing a computer to execute the digest video programming method according to claim 11 or 12.

Images