JP6917788B2 - Summary video generator and program - Google Patents

Description

The present invention relates to an apparatus and a program for generating a summary video in the field of video processing using a computer and a hard disk.

Conventionally, in broadcasting stations, there is an increasing need for "contents for the Internet" such as program HP as a medium for increasing viewers' interest in programs. For online content, it is desirable to distribute a digest video (summary video) consisting of only important video sections in the program video.

However, manually producing a summary video is very costly in terms of both labor and cost. Therefore, a technique for automatically generating a summary video is desired. As a technique for automatically generating a summary video, for example, the methods of Patent Documents 1 to 5 have been proposed.

In the method of Patent Document 1, the video is divided into a plurality of video sections for each modal of the video, the similarity between the two video sections is obtained, and the video sections are clustered based on the similarity. Then, a representative video section is extracted from each of the plurality of clusters, and a summary video is generated by combining the representative video sections.

In the method of Patent Document 2, a corresponding section is generated between videos based on the similarity of each video, a common video section and an individual video section are extracted from the corresponding section, a common summary section is selected from the common video section, and a common summary section is selected. Select an individual summary section from the individual video sections. Then, the common summary section and the individual summary section are integrated to generate a summary video.

The method of Patent Document 3 selects one or more video sections from a plurality of video sections based on metadata and feature quantities, obtains a set of video sections that maximizes the evaluation value of a predetermined evaluation function, and obtains a video. A summary video is generated by combining a set of intervals.

In the method of Patent Document 4, the image feature amount and the audio feature amount are obtained from the video, the importance of the image processing unit is calculated based on the image feature amount and the audio feature amount, and the summary image is based on the importance. To generate.

In the method of Patent Document 5, a divided video having a short time is generated from the original video, the block area is regarded as a visual word for each divided video, a score is calculated based on the feature amount of the visual word, and the score is divided in descending order. Select a video to generate a summary video.

Japanese Unexamined Patent Publication No. 2014-179906 Japanese Unexamined Patent Publication No. 2013-126233 Japanese Unexamined Patent Publication No. 2012-19305 Japanese Unexamined Patent Publication No. 2014-33417 Japanese Unexamined Patent Publication No. 2012-10265

The methods of Patent Documents 1 and 2 described above generate a summary video based on the degree of similarity between video sections or the presence or absence of a common section. However, since these methods are premised on the repeated appearance of similar video sections, it is difficult to apply them to general broadcast program images in which similar video sections do not always appear repeatedly.

Further, the method of Patent Document 3 described above generates a summary video based on a basic image feature amount, an audio feature amount, and added metadata. However, it is necessary to add detailed metadata to the video in advance, and the processing load is high.

Further, the methods of Patent Documents 4 and 5 described above generate a summary video based on a basic image feature amount and an audio feature amount without using metadata. Since these methods do not use metadata, the processing load is lower than that of the method of Patent Document 3.

All of these methods of Patent Documents 1 to 5 generate a summary video, but do not consider the effect appearing in the video. Therefore, the summary video may not include important scenes from the viewpoint of production. Here, the production means expressing and effectively showing a predetermined intention based on a script or the like. For example, the production of a broadcast program image includes elements such as telop display, appearance of a main performer or guest, zooming in or panning of a camera, and CG image for explanation. These elements are more likely to be used in important scenes in the video.

In general, it is desirable that the summary video is composed of only the video of important sections. For this reason, it has been desired to obtain the degree of effect of the effect on the video, specify the section with high effect as an important section, and combine the video of the important section to generate a summary video.

Therefore, the present invention has been made to solve the above-mentioned problems, and an object of the present invention is to generate a summary video composed of only videos of important sections by considering the effect appearing in the video. The purpose is to provide a generator and a program.

In order to solve the above problem, the summary video generation device according to claim 1 is a summary video generation device that generates a summary video from a video, and includes a cut division unit that divides the video into a plurality of cut videos in cut units, and the above. A scene generation unit that integrates a series of cut images of the same scene from the plurality of cut images divided by the cut division unit to generate a scene image for each scene, and the plurality of cuts divided by the cut division unit. For each of the images, a score calculation unit that calculates a score indicating the importance of the effect for each of a predetermined number of different effects, and a score calculation unit for each of the plurality of cut images divided by the cut division unit. Based on the total score calculation unit that calculates the total score based on the score for each effect calculated by the above, and the summary video from the plurality of cut images based on the total score calculated by the total score calculation unit. A summary video generation unit that selects the cut video that constitutes the above and generates the summary video, and the summary video generation unit selects the cut video from the scene video generated by the scene generation unit. The feature is that the number of the cut images selected from the scene images does not exceed a predetermined value.

Further, the summary video generation device according to claim 2 is the effect calculated by the score calculation unit for each of the plurality of cut videos by the total score calculation unit in the summary video generation device according to claim 1. It is characterized in that the total score is calculated based on the score for each and the weighting coefficient for each effect set in advance.

Further, in the summary video generation device according to claim 3, in the summary video generation device according to claim 1 or 2 , the score calculation unit determines the area where an object related to the effect appears in the video, and is related to the effect. The score is calculated based on the amount of movement of the target to be performed or the probability that the target related to the effect appears.

Further, in the summary video generation device according to claim 4, in the summary video generation device according to any one of claims 1 to 3 , the summary video generation unit is divided by the cut division unit. From the cut video, the number of the cut video to be selected from the scene video generated by the scene generation unit is determined in descending order of importance of the effect according to the total score calculated by the total score calculation unit. The summary video selection unit that selects the cut video that constitutes the summary video and the cut selected by the summary video selection unit until the total length of the summary video exceeds a predetermined value so as not to exceed the value. It is characterized by including a summary video output unit that connects videos in chronological order and generates the summary video.

Further, the program of claim 5 is characterized in that the computer functions as the summary video generation device according to any one of claims 1 to 4.

As described above, according to the present invention, it is possible to generate a summary video composed of only the video of an important section by considering the effect appearing in the video.

It is a block diagram which shows the structural example of the summary video generation apparatus by embodiment of this invention. It is a figure explaining the cut series V _C [1], ..., V _C [N _C ] and the scene series V _S [1], ..., V _S [N _S]. It is a block diagram which shows the structural example of the element score calculation part, and the input / output data example. It is a block diagram which shows the structural example of the total score calculation part, and the input / output data example. It is a block diagram which shows the structural example of the summary video generation part, and the input / output data example. It is a flowchart which shows the processing example of the summary video generation part. It is a figure explaining the flowchart of FIG.

Hereinafter, embodiments for carrying out the present invention will be described in detail with reference to the drawings. In the present invention, the degree of effect due to the effect (for example, telop, performer, amount of movement of camera work, CG image for explanation, etc.) appearing in the image is determined as the importance, and the section with high effect is specified as the important section. It is characterized by combining images of important sections to generate a summary image. As a result, a summary video composed of only the video of the important section is generated.

〔overall structure〕
First, the overall configuration of the summary video generator according to the embodiment of the present invention will be described. FIG. 1 is a block diagram showing a configuration example of a summary video generator according to an embodiment of the present invention. The summary video generation device 1 includes a cut division unit 10, a scene generation unit 11, an element score calculation unit 12, a total score calculation unit 13, and a summary video generation unit 14.

Cut dividing unit 10 receives the program video, cut program image cut unit video _{V C [i] (i =} 1, ..., N C) performs "Cut image division processing" is divided into cut Generates a cut series V _C [1], ..., V _C [N _C ] consisting of video V _{C [i].} Then, the cut division unit 10 outputs the _{cut series V C} [1], ..., V _C [N _{C ] composed of the cut video V C} [i] to the scene division unit 11 and the element score calculation unit 12.

Parameter i = 1, ..., N _C represents the cut video V _C [i] number (cut number), N _C indicates the number of cut video V _C [i]. The cut image V _C [i] is an image taken continuously without a break until the camera is switched.

The "cut video division process" is known, and for details, refer to, for example, Japanese Patent Application Laid-Open No. 2008-33749.

The scene generation unit 11 inputs the _{cut series V C} [1], ..., V _C [N _{C ] composed of the cut video V C} [i] from the cut division unit 10. Then, the scene generation unit 11 _{integrates the cut video V C} [i] of the same scene to detect the scene boundary, and the scene video V _S [j] (j = 1, ..., N _S ) for each scene. Performs the "scene video generation process" to generate the scene series V _S [1], ..., V _S [N _S ] _{consisting of the scene video V S [j].} The scene generation unit 11 outputs the _{scene series V S} [1], ..., V _S [N _{S ] composed of the scene video V S} [j] to the summary video generation unit 14.

Parameter j = 1, ..., N _S denotes the scene image V _S [j] number (scene number), N _S denotes the number of scene image V _S [j]. The scene video V _S [j] is a video that summarizes a series of cut video V _{C [i] in a certain scene.}

The "scene image generation process" is known, and for details, refer to, for example, Japanese Patent Application Laid-Open No. 2014-225118 and Japanese Patent Application Laid-Open No. 2014-33355.

FIG. 2 is a diagram illustrating a cut series V _C [1], ..., V _C [N _C ] and a scene series V _S [1], ..., V _S [N _S ]. The cut series V _C [1], ..., V _C [N _C ] is generated by dividing the program video into cut units, and the scene series V _S [1], ..., V _S [N _S]. ] Is generated by integrating the cut series V _C [1], ..., V _C [N _C ] for each scene.

In the example of FIG. 2, the scene image V _S [1] is an image in which the cut image V _C [1], V _C [2], and V _C [3] are integrated, and the scene image V _S [2] is Cut video This is a video that integrates _{V C} [4] and V _{C [5].} The scene video V _S [N _S ] is a video that integrates _{the cut video V C} [N _C -1] and V _C [N _C]. In this way, the cut video V _C [i] belongs to any of the scene video V _S [j].

Returning to FIG. 1, the element score calculation unit 12 inputs the _{cut series V C} [1], ..., V _C [N _{C ] composed of the cut video V C} [i] from the cut division unit 10. _{Then, the element score calculation unit 12 sets the cut image V for each cut image V C} [i] for each of the four element productions consisting of "telop", "face recognition", "camera work", and "CG image-likeness". _{Calculate the element scores S 1} [i] to S ₄ [i] that represent the importance based on _{C [i].}

The element score, which indicates the degree of importance, reflects the degree of effect of the effect that appears in _{the cut video V C [i].} The element scores of "telop", "face recognition", "camera work", and "CG image-likeness" for the cut video V _{C [i] are the telop score S 1} [i], face recognition score S ₂ [i], and camera, respectively. Work score S ₃ [i] and CG image quality score S ₄ [i].

The element score calculation unit 12 has element scores S ₁ [i] to S ₄ [i] (telop score S ₁ [i], face recognition score S ₂ [i], camera work score S ₃ [i], and CG image-likeness. Element score series consisting of score S _{4 [i]) S 1} [1], ..., S ₁ [N _C ], S ₂ [1], ..., S ₂ [N _C ], S ₃ [1] ], ..., S ₃ [N _C ], S ₄ [1], ..., S ₄ [N _C ].

The element score calculation unit 12 includes a cut series V _C [1], ..., V _C [N _{C ] composed of cut images V C} [i], and element scores S ₁ [i] to S ₄ [i] ( Element score series S ₁ [1], ... consisting of telop score S ₁ [i], face recognition score S ₂ [i], camera work score S ₃ [i] and CG image-likeness score S _{4 [i])} , S ₁ [N _C ], S ₂ [1], ..., S ₂ [N _C ], S ₃ [1], ..., S ₃ [N _C ], S ₄ [1], .. ., S ₄ [N _C ] is output to the total score calculation unit 13. The details of the element score calculation unit 12 will be described later.

From the element score calculation unit 12, the total score calculation unit 13 includes a cut series V _C [1], ..., V _C [N _{C ] composed of cut images V C} [i], and an element score S ₁ [i]. Element score series consisting of ~ S _{4 [i] S 1} [1], ..., S ₁ [N _C ], S ₂ [1], ..., S ₂ [N _C ], S ₃ [1] , ..., S ₃ [N _C ], S ₄ [1], ..., S ₄ [N _C ].

The total score calculation unit 13 integrates the element scores S ₁ [i] to S ₄ [i] for _{each cut video V C} [i] using preset weighting coefficients W _{1 to} W _{4, and the total score is calculated.} Calculate S [i] and generate the total score series S [1], ..., S [N _{C] consisting of the total score S [i].} Then, the total score calculation unit 13 includes a cut series V _C [1], ..., V _C [N _{C ] composed of the cut video V C} [i], and a total score series S composed of the total score S [i]. Output [1], ..., S [N _C ] to the summary video generation unit 14. The details of the total score calculation unit 13 will be described later.

The summary video generation unit 14 is from the total score calculation unit 13, from the cut series V _C [1], ..., V _C [N _{C ] consisting of the cut video V C} [i], and from the total score S [i]. Enter the total score series S [1], ..., S [N _C]. Further, the summary video generation unit 14 inputs the _{scene series V S} [1], ..., V _S [N _{S ] composed of the scene video V S} [j] from the scene generation unit 11.

The summary video generation unit 14 refers to the total score S [i] and the scene video V _{S [j], and constitutes the summary video V C} [i] until the length of the entire summary video exceeds a predetermined value. Select. Then, the video summary generation unit 14 generates a video summary by concatenating cut video V _C selected [i] (in ascending order of frame numbers) in a time series sorted, and outputs the video summary. The details of the summary video generation unit 14 will be described later.

[Element score calculation unit 12]
Next, the element score calculation unit 12 shown in FIG. 1 will be described in detail. _{As described above, the element score calculation unit 12 cuts each cut image V C} [i] for each of the four element productions consisting of "telop", "face recognition", "camera work", and "CG image-likeness". Based on the video V _C [i], the telop score S ₁ [i], the face recognition score S ₂ [i], the camera work score S ₃ [i], and the CG video-likeness score S ₄ [i] are calculated.

FIG. 3 is a block diagram showing a configuration example and an input / output data example of the element score calculation unit 12. The element score calculation unit 12 includes a telop area detection unit 20, a face recognition processing unit 21, a camera work calculation unit 22, a CG image-likeness calculation unit 23, a telop score calculation unit 24, a face recognition score calculation unit 25, and a camera work score calculation. A unit 26 and a CG image-like score calculation unit 27 are provided.

The element score calculation unit 12 _{inputs the cut series V C} [1], ..., V _C [N _{C ] consisting of the cut video V C} [i] from the cut division unit 10, and inputs this to the total score calculation unit. Output to 13.

Ticker area detection unit 20, from the cut dividing unit 10, a cut line consisting of cut video _{V C [i] V C [} 1], ..., V C [N C] Enter the cut video V _C [i ], The frame image P [i, n] is sampled for each T frame, and the image series P [i, 1], ..., P [i, N _P ] consisting of the frame image P [i, n] is generated. do. Then, the telop region detection unit 20 calculates the image series P [i, 1], ..., P [i, N _P ] composed of the frame image P [i, n] by the face recognition processing unit 21 and the CG image-likeness. Output to unit 23.

The parameters n = 1, ..., N _P indicate the number of the frame image P [i, n], and N _P indicates the number of sampled frame images P [i, n].

The telop area detection unit 20 performs "telop area detection processing" for detecting the area where the telop is displayed for each frame image P [i, n], calculates the area of the telop area, and calculates the area of the telop area, and the frame image P [ _{Calculate the area ratio r TL} [i, n] of the telop region to i, n]. The ticker area detection unit 20, the frame image P [i, n] area of telop area for the ratio r _TL [i, n] consists of area ratio sequence _{r TL [i, 1],} ..., r TL [ i, N _P ] is generated and output to the telop score calculation unit 24. The area of this telop area is the area where the "telop", which is the object related to the production, appears in the image.

The "telop region detection process" is known, and for details, refer to, for example, Japanese Patent Application Laid-Open No. 2013-30963.

C_TLは正規化定数であり、予め設定される。 The telop score calculation unit 24 is _subjected to an area ratio series r _TL [i, 1], .. Enter ., r _TL [i, N _P ]. Then, the telop score calculation unit 24 uses the following formula for each cut video V _C [i] based on the area ratio r _TL [i, n], and the telop score S _{1 of the cut video V C [i].} Calculate [i].

C _TL is a normalization constant and is preset.

The range of the telop score S ₁ [i] is 0 ≤ S ₁ [i] ≤ 1. The telop score S ₁ [i] becomes a large value as the area of the telop region in the frame image P [i, n] is large, and becomes a small value as the area is small. That is, the telop score S ₁ [i] becomes larger as the frame image P [i, n] having a wider telop display area appears in the cut video V _{C [i].} The telop score S ₁ [i] sets _{the area ratio r TL} [i, n] of the frame image P [i, n], which has the widest area where the telop is displayed _{, from 0 to 1 in the cut video V C [i].} The value is normalized in the range up to.

The telop score calculation unit 24 generates a telop score series S ₁ [1], ..., S ₁ [N _{C ] composed of the telop score S 1} [i], and outputs this to the total score calculation unit 13.

The face recognition processing unit 21 _{inputs a cut series V C} [1], ..., V _C [N _{C ] composed of a cut image V C} [i] from the cut division unit 10, and also inputs a telop area detection unit 20. From, input the image series P [i, 1], ..., P [i, N _{P] consisting of the frame image P [i, n].}

The face recognition processing unit 21 has, for each frame image P [i, n], the main performer ID [m] in the target program (M main performer ID [1], ..., ID [M]]. ) Is subjected to "face recognition processing" for recognizing these faces, and the face area F [i, n, k] is detected. It is assumed that K face regions F [i, n, k] are detected from the frame image P [i, n]. The face recognition processing unit 21 generates K face region sequences F [i, n, 1], ..., F [i, n, K] composed of face regions F [i, n, k].

The parameters m = 1, ..., M indicate the number of the main performer ID [m], and M indicates the number of the main performer ID [m]. Further, the parameters k = 1, ..., K indicate the number of the face area F [i, n, k], and K is the face area F [i, n] detected from the frame image P [i, n]. Indicates the number of n, k].

The "face recognition process" is known, and for details, refer to, for example, Japanese Patent Application Laid-Open No. 2017-333372.

_{The face recognition processing unit 21 calculates the area ratio r FC} [i, n, k] of the face area F [i, n, k] to the frame image P [i, n], and K pieces of the area ratio r _{FC. Generate an area ratio series r FC} [i, n, 1], ..., r _FC [i, n, K] consisting of [i, n, k].

The face recognition processing unit 21 performs a "face probability calculation process" for calculating the _{probability p FC} [i, n, k, m] that the face area F [i, n, k] is the face of the performer ID [m]. _{And generate a probability series p FC} [i, n, k, 1], ..., p _FC [i, n, k, M] consisting of M probabilities p _FC [i, n, k, m]. do. This probability p _FC [i, n, k, m] is the probability that the "face" of the performer ID [m], which is the target related to the production, appears.

The "face probability calculation process" is known, and for details, refer to, for example, Japanese Patent Application Laid-Open No. 2017-333372, as in the "face recognition process".

_{The face recognition processing unit 21 has an area ratio series r FC} [i, n, _{k] composed of an area ratio r FC} [i, n, k] of the face region F [i, n, k] with respect to the frame image P [i, n]. 1], ..., r _FC [i, n, K] and the face area F [i, n, k] in the frame image P [i, n] are the faces of the performer ID [m]. _{Face recognition score calculation unit 25 with probability series p FC} [i, n, k, 1], ..., p _FC [i, n, k, M] consisting of probability p _FC [i, n, k, m] Output to.

The face recognition score calculation unit 25 receives from the face recognition processing unit 21 the area ratio of the face area F [i, n, k] to the frame image P [i, n] r _FC [i, n, k]. The performers are the series r _FC [i, n, 1], ..., r _FC [i, n, K], and the face area F [i, n, k] in the frame image P [i, n]. _{Probability series p FC} [i, n, k, 1], ..., p _FC [i, n, k, M _{] consisting of the probability p FC} [i, n, k, m] that is the face of ID [m] ] Is entered.

C’_FCは、パラメータw_FC[i,n,k]の最小値を定める値であり、予め設定される。 The face recognition score calculation unit 25 uses the following formula to determine the probability that the face area F [i, n, k] in the frame image P [i, n] is the face of the performer ID [m] p _FC [i, _{Calculate the parameter w FC} [i, n, k] based on n, k, m].

_C'FC is a value that defines the minimum value of the parameter w _FC [i, n, k] and is preset.

The parameter w _{FC [i, n, k] has the maximum probability p FC} [i, n, k, m] in the face region F [i, n, k] in the frame image P [i, n]. The lower the value, the larger the value, and the higher the maximum probability p _FC [i, n, k, m], the smaller the value.

_{The face recognition score calculation unit 25 uses the following formula to obtain the area ratio r FC} [i, n, k] of the face region F [i, n, k] to the frame image P [i, n] and the above formula (2). _{), The parameter R FC} [i, n] is calculated based on the parameter w _FC [i, n, k].

The parameter R _{FC [i, n] is the parameter w FC} [i, n, k] of the face area F [i, n, k] in the frame image P [i, n] and the face area F [i, n, k]. It is a value obtained by multiplying the area ratio r _FC [i, n, k] of k] and adding the multiplication result for all face regions F [i, n, k].

The parameter R _FC [i, n] _{becomes larger as the parameter w FC} [i, n, k] is larger (the _{lower the probability p FC} [i, n, k, m]), and the parameter w _FC [i] The smaller the, n, k] (the _{higher the probability p FC} [i, n, k, m]), the smaller the value. The parameter R _FC [i, n] is the area ratio _{r FC [i, n, k} ] higher, becomes a large value, as the area ratio _{r FC [i, n, k} ] is small, a small value ..

C_FCは正規化定数であり、予め設定される。 Face recognition score calculation unit 25, by the following equation, for each cut image V _C [i], the parameter R _FC [i, n] calculated the at Equation (3) based on, the cut video V _C [ Calculate the face recognition score S ₂ [i] of i].

C _FC is a normalization constant and is preset.

The range of the face recognition score S ₂ [i] is 0 ≤ S ₂ [i] ≤ 1. _{The larger the parameter R FC} [i, n], the larger the face recognition score S ₂ [i], _{and the smaller the parameter R FC} [i, n], the smaller the value. In other words, the face recognition score S ₂ [i], the probability _{p FC [i, n, k} , m] is as low, becomes a large value, the probability _{p FC [i, n, k} , m] higher, smaller The higher the area ratio r _FC [i, n, k], the larger the value, and the _{lower the area ratio r FC} [i, n, k], the smaller the value.

The probability p _FC [i, n, k, m] is the probability that the face area F [i, n, k] is the face of the main performer ID [m], so the probability p _FC [i, n, m] If k, m] is low, there is a high probability that it is the face of the non-main performer ID [m] (guest's face), and if the probability p _FC [i, n, k, m] is high, it is the guest's face. It means that the probability of being is low.

Therefore, the face recognition score S ₂ [i] is such that the larger the total area where the face of the main performer ID [m] appears in the frame image P [i, n] included in _{the cut video V C [i], the larger the total area.} The larger the value, the smaller the total area where the face of the main performer ID [m] appears, the smaller the value. In addition, the face recognition score S ₂ [i] becomes a larger value as the probability that the guest's face appears in the frame image P [i, n] included in the cut image V _{C [i] becomes larger, and the guest's face becomes larger.} The lower the probability of appearance, the smaller the value. In other words, the face recognition score S ₂ [i] is such that in the cut video V _C [i], the frame image P [i, n] with a large total area where the face of the main performer ID [m] appears appears. The larger the value, the larger the value as the frame image P [i, n], which has a high probability of the guest's face appearing, appears.

The face recognition score calculation unit 25 generates a face recognition score series S ₂ [1], ..., S ₂ [N _{C ] consisting of the face recognition score S 2} [i], and outputs this to the total score calculation unit 13. Output.

The camera work calculation unit 22 _{inputs the cut series V C} [1], ..., V _C [N _{C ] consisting of the cut video V C} [i] from the cut division unit 10, and "camera work detection processing". I do. The camera work calculation unit 22, for each cut image V _C [i], the zoom, obtains a video sequence given camera work occurs V _CW [i, q] such as bread, video section V _CW [i, Generate a video section sequence V _CW [i, 1], ..., V _CW [i, N _{CW] consisting of q].}

Parameter q = 1, ..., N _CW indicates the number of video segments _{V CW [i, q],} N CW , the video segment is detected from the cut video _{V C [i] V CW [} i, q ] Indicates the number.

The "camera work detection process" is known, and for details, refer to, for example, Japanese Patent Application Laid-Open No. 10-243340.

Camera work calculation unit 22, the video section V _CW [i, q] to calculate the movement amount of the camera for normalization of the amount of motion of the camera by the length of the diagonal of the image (division) motion ratio r _CW [i , q] is calculated to generate a _{motion rate series r CW} [i, 1], ..., r _CW [i, N _{CW ] consisting of motion rate r CW} [i, q]. Then, the camera work calculation unit 22 sets the motion rate _{series r CW} [i, 1], ..., r _CW [i, N _{CW ] composed of the motion rate r CW} [i, q] into the camera work score calculation unit 26. Output to. This movement rate r _CW [i, q] is the movement rate of the "camera work" that is the object related to the production.

C_CWは正規化定数であり、予め設定される。 The camera work score calculation unit 26 _{obtains a movement rate series r CW} [i, 1], ..., r _CW [i, N _CW ] composed of movement rates r CW [i, q] from the camera work calculation unit 22. Input and calculate the camera work score S ₃ [i] of the _{cut video V C} [i] based on the motion rate r _CW [i, q] for _{each cut video V C} [i] by the following formula. do.

C _CW is a normalization constant and is preset.

The range of the camera work score S ₃ [i] is 0 ≤ S ₃ [i] ≤ 1. The camera work score S ₃ [i] becomes a large value as the amount of movement of the camera increases, and becomes a small value as the amount of movement of the camera decreases in the cut image V _{C [i].} That is, the camera work score S ₃ [i] becomes larger as the video section V _CW [i, q] with a large amount of camera movement appears in the cut video V _{C [i].}

The camera work score calculation unit 26 generates a camera work score series S ₃ [1], ..., S ₃ [N _{C ] consisting of the camera work score S 3} [i], and outputs this to the total score calculation unit 13. Output.

The CG image-likeness calculation unit 23 _{inputs the cut series V C} [1], ..., V _C [N _{C ] consisting of the cut image V C} [i] from the cut division unit 10, and also inputs the telop area detection unit. From 20, the image series P [i, 1], ..., P [i, N _P ] consisting of the frame image P [i, n] is input.

The CG image-likeness calculation unit 23 performs "identification processing using the recognition model of the support vector machine (SVM)" for each frame image P [i, n], and the probability that it is a CG image p _CG [i, n]. Is calculated. Then, the CG image-likeness calculation unit 23 generates a _{probability series p CG} [i, 1], ..., p _CG [i, N _{P ] composed of the probability p CG} [i, n] which is a CG image. _{The range of the probability p CG} [i, n] that is a CG image _{is 0 ≦ p CG} [i, n] ≦ 1.

The support vector machine is made to learn in advance the difference between the image feature of the CG image and the image feature of the image that is not the CG image. CG images tend to be highly saturated and have artificial texture features. Therefore, as the image features, a histogram of the S value (saturation) of the HSV color space, an edge direction histogram, a fractal feature with high classibility of natural objects and artificial objects, and the like are used.

Since the "identification process using the recognition model of the support vector machine (SVM)" is known, detailed description thereof will be omitted here. For details of fractal features, refer to, for example, Japanese Patent Application Laid-Open No. 2001-56820.

CG image likeness calculator 23, the probability p _CG is CG image [i, n] probability sequence p _CG consisting [i, 1], ..., p CG [i, N P] CG image likeness score calculated Output to unit 27.

From the CG image-likeness calculation unit 23, the CG image-likeness score calculation unit 27 is a _{probability series p CG} [i, 1], ..., p _CG [i, _{n] consisting of the probability p CG} [i, n] of the CG image. Enter N _P ]. Then, the CG video-like score calculation unit 27 uses the following formula to _{obtain a CG video-like score S 4} [i _{] for each cut video V C} [i] _{based on the probability p CG} [i, n] of the CG video. ] Is calculated.

The range of the CG image-likeness score S ₄ [i] is 0 ≦ S ₄ [i] ≦ 1. The CG image-likeness score S ₄ [i] is such that the larger the maximum value _{of the probability p CG} [i, n] of the frame image P [i, n] included in _{the cut image V C [i] is the CG image.} It becomes a large value, and the _{smaller the maximum value of the probability p CG} [i, n] which is a CG image, the smaller the value. That is, the CG image-likeness score S ₄ [i] is so large that a frame image P [i, n] having a high probability of being a CG image p _{CG [i, n] appears in the cut image V C [i].} It becomes a value.

The CG video-like score calculation unit 27 generates a CG video-like score series S ₄ [1], ..., S ₄ [N _{C ] consisting of the CG video-like score S 4} [i], and calculates the total score. Output to unit 13.

[Comprehensive score calculation unit 13]
Next, the total score calculation unit 13 shown in FIG. 1 will be described in detail. As described above, the total score calculation unit 13 determines the _{telop score S 1} [i], the face recognition score S ₂ [i], the camera work score S ₃ [i], and the CG image quality for _{each cut image V C} [i]. The total score S [i] is calculated by integrating the scores S _{4 [i].}

FIG. 4 is a block diagram showing a configuration example and an input / output data example of the total score calculation unit 13. The total score calculation unit 13 includes a weight coefficient setting unit 30 and a score calculation unit 31.

_{The total score calculation unit 13 inputs the cut series V C} [1], ..., V _C [N _{C ] consisting of the cut video V C} [i] from the element score calculation unit 12, and generates a summary video. Output to unit 14.

The weighting coefficient setting unit 30 has a total score S for each element of the _{telop score S 1} [i], the face recognition score S ₂ [i], the camera work score S ₃ [i], and the CG image-likeness score S _{4 [i]. Set the weighting coefficient W r} (r = 1, ..., 4) that determines the degree of reflection in [i]. Then, the weighting coefficient setting unit 30 _{outputs the weighting coefficients W 1 to} W ₄ to the score calculation unit 31. The weighting factors W _{1 to} W ₄ are freely determined and preset by the operation of the user (creator of the summary video).

The score calculation unit 31 inputs the telop score series S ₁ [1], ..., S ₁ [N _{C ] consisting of the telop score S 1} [i] from the telop score calculation unit 24 of the element score calculation unit 12. , From the face recognition score calculation unit 25, input the face recognition score series S ₂ [1], ..., S ₂ [N _C ] _{consisting of the face recognition score S 2 [i].} Further, the weighting coefficient setting unit 30 inputs the camera work score series S ₃ [1], ..., S ₃ [N _{C ] composed of the camera work score S 3 [i] from the camera work score calculation unit 26. , The CG image-likeness score sequence S 4} [1], ..., S ₄ [N _C ] consisting of the CG image-likeness _{score S 4} [i] is input from the CG image-likeness score calculation unit 27. Further, the score calculation unit 31 inputs _{weight coefficients W 1 to} W _{4 from the weight coefficient setting unit 30.}

The score calculation unit 31 uses the following formula to _{add a weighting coefficient W to the telop score S 1} [i], the face recognition score S ₂ [i], the camera work score S ₃ [i], and the CG image-likeness score S ₄ [i]. _{By multiplying each of r} and adding the multiplication results, the total score S [i] _{of the cut video V C [i] is calculated.}

The score calculation unit 31 generates a total score series S [1], ..., S [N _{C ] consisting of the total score S [i] of the cut video V C} [i], and the summary video generation unit 14 generates the total score series S [1], ..., S [N C]. Output to.

Thus, in accordance with the weighting factor W _r, the total score S score of elements corresponding to the weight coefficient W _r is reflected [i] is calculated. For elements with a high weighting factor W _r , the score of that element is greatly reflected in the total score S [i], _{and for elements with a low weighting factor W r} , the score of that element is the total score S [i]. ] Is not reflected so much.

For example, if a large value is set for the weight coefficient W _{2 of the face recognition score S 2} [i] and a small value is set for the _{other weight coefficients W 1 , 3 and 4, the face recognition score S 2} [i] A total score S [i] that largely reflects is generated. Then, the summary video generation unit 14, which will be described later, generates a summary video in which many faces of the main performer ID [m] or the faces of the guests appear. Similarly, when a large value is set for the weighting coefficient W ₄ of the CG image-likeness score S ₄ [i] and a small value is set for the _{other weighting factors W 1 , 2, 3, the CG image-likeness score S 4} A total score S [i] that largely reflects [i] is generated. Then, the summary video generation unit 14, which will be described later, generates a summary video in which many CG images appear.

[Summary video generation unit 14]
Next, the summary video generation unit 14 shown in FIG. 1 will be described in detail. As described above, the summary video generation unit 14 refers to the total score S [i] and the scene video V _S [j], and selects the _{cut video V C} [i] until the length of the entire summary video exceeds a predetermined value. Then, the selected cut video V _C [i] is concatenated to generate a summary video.

FIG. 5 is a block diagram showing a configuration example and an input / output data example of the summary video generation unit 14. The summary video generation unit 14 includes a summary video selection unit 40 and a summary video output unit 41.

The summary video selection unit 40 is from the total score calculation unit 13, from the cut series V _C [1], ..., V _C [N _{C ] consisting of the cut video V C} [i], and from the total score S [i]. Enter the total score series S [1], ..., S [N _C]. Further, the summary video selection unit 40 inputs the _{scene series V S} [1], ..., V _S [N _{S ] composed of the scene video V S} [j] from the scene generation unit 11.

The summary video selection unit 40 _{sorts all the cut video V C} [i] in descending order of importance based on the total score S [i], and for the sorted cut video V _C [i [i]]. _{In order, the cut video V C} [I [i]] is selected until the total length of the selected cut video V _C [I [i]] (the length of the entire summary video) exceeds a predetermined value. The order of importance refers to the order in which the effect of the production that integrates "telop", "face recognition", "camera work", and "CG image-likeness" is high.

When the summary video selection unit 40 selects the cut video V _C [I [i]], the number _{of the cut video V C} [I [i]] selected in the scene video V _{S [j] is set to a predetermined value.} Do not exceed. The digest video selection unit 40 outputs the cut video V _C selected [I [i]] in the video summary output unit 41.

Summary video output unit 41, the digest video selection unit 40 receives the cut image V _C selected [I [i]], and connecting the selected cut image V _C [I [i]] in a time-series Summary It generates a video _{V C [i '1] ...} V C [i' L], and outputs the video summary _{V C [i '1] ...} V C [i' L]. L is the number of the number of cut video V _C which is selected [i], i.e. summarized video _{V C [i '1] ...} V C [i' L].

[Operation of summary video generation unit 14]
FIG. 6 is a flowchart showing a processing example of the summary video generation unit 14, and FIG. 7 is a diagram illustrating the flowchart of FIG. In the summary video selection unit 40 of the summary video generation unit 14, the cut series V _C [1], ..., V _C [in descending order of the score of the total score series S [1], ..., S [N _C]. N _C ] is sorted and the cut series V _C [I [1]], ..., V _C [I [N _C ]] is generated (step S601).

For example, as shown in FIG. 7, the scores of the total score series S [1], ..., S [N _C ] are S [8]> S [4]> S [1]> S [10]> S [ 3]>. .. .. > In the case of S [20], the cut series after sorting is V _C [I [1]] = V _C [8], V _C [I [2]] = V _C [4] by the processing of step S601. , V _C [I [3]] = V _C [1], V _C [I [4]] = V _C [10], V _C [I [5]] = V _C [3] ,. .. .. , V _C [I [N _C ]] = V _C [20].

Returning to FIG. 6, the video summary selection unit 40 as an initial setting, and set the parameter i = 1, ..., all the flags the Select [i] for N _C to "false" (Select [i] = false (i = 1, ..., N C)), the parameter j = 1, ..., all counts count [j] of the N _S is set to 0 (count [j] = 0 (j = 1, ..., N _S ) (step S602). Note that this parameter i is used for the explanation of FIGS. 6 and 7, and the cut image V _C [i], the total score S [i], etc. Different from parameter i.

Flag the Select [i], when the cut video V _C at step S604 and step S605 described later [I [i]] is selected in a part of the video summary, "true" is set, the cut video V _C [ If I [i]] is not selected for the summary video, the default "false" is maintained. Count Count [j] is the scene image V _S [j] of the parameter j scene number, in the scene image V _S [j] Cut belonging to the video _{V C [I [i]]} , the video summary one _{The number of cut images V C} [I [i]] selected in the section is shown.

Below, the parameters i = 1, ..., N _C are associated with each of the sorted cut series V _C [I [1]], ..., V _C [I [N _{C]], and after sorting} The summary video is selected for each cut video V _C [I [i]]. Selection process video summary, as in step S606 to be described later, the total number of frames of the selected cut image V _C as summarized video [I [i]] is performed until more than a predetermined value T _MAX.

The summary video selection unit 40 sets the parameter i to 1 (i = 1, step S603), sets the flag Select [i] to "true", or sets the count Count [J [i]] to a predetermined value. It is determined whether or not a condition larger than _{N MAX is satisfied (step S604).} That is, in the summary video selection unit 40, the cut video V _C [I [i]] of the parameter i is selected as a part of the summary video, or the cut video V _C [I [i]] of the parameter i belongs to. in scene number J [i] in the scene image _{V S [J [i]]} , the number of cut video has been selected as part of the video summary is determined whether conditions are satisfied is greater than a predetermined value N _MAX.

J [i] _{indicates the scene number to which the cut video V C} [I [i]] belongs. N _MAX indicates the maximum value of the number of _{cut video V C} [I [i]] selected from one scene video V _S [J [i]] and is preset.

In step S604, the summary video selection unit 40 determines that the flag Select [i] is set to "true" or the count Count [J [i]] does not satisfy the condition larger than the _{predetermined value N MAX.} In the case (step S604: N), _{select the cut video V C} [I [i]] as a part of the summary video, set the flag Select [i] to "true" (Select [i] = true), and count. Increment Count [J [i]] (Count [J [i]] = Count [J [i]] + 1) (step S605). That is, in the summary video selection unit 40, when "true" is not set in the flag Select [i] ("false" is set), and the count Count [J [i]] is set from the _{predetermined value N MAX.} If it is not large, the process of step S605 is performed. In other words, in the summary video selection unit 40, the cut video V _C [I [i]] is not selected as a part of the summary video, and the _{scene number J to which the cut video V C} [I [i]] belongs. If the number of cut images selected as a part of the summary image in the scene image V _{S [J [i]] of [i] is not larger than the predetermined value N MAX} , the process of step S605 is performed.

For example, as shown in FIG. 7, among the flags Select [1], ..., Select [N _c ] set to "false" in the initial setting by step S605, the flags Select [1], ... , Select [5] etc. are changed to "true".

Returning to FIG. 6, the summary video selection unit 40 performs all cuts in which the flag Select [i] is set to “true” (Select [i] = true) from the parameter i = 1 to the current parameter i. It calculates the total number of frames of the video _{V C [I [i]]} , determines whether the total number of frames is greater than the predetermined value T _MAX (step S606). That is, in the summary video selection unit 40, whether or not the total number of frames of _{all the cut video V C [i] selected as a part of the summary video is larger than the predetermined value TMAX} , that is, the length of the entire summary video. Saga determines greater or not than a predetermined value T _{MAX. TMAX} indicates the length (number of frames) of the summary video that the user wants to generate, and is preset.

Summary video selection unit 40, in step S606, the case where the total number of frames of the selected cut image V _C [i] as part of the video summary is not greater than the predetermined value T _MAX (step S606: N) , that is, when the length of the entire video summary is not greater than the predetermined value T _MAX, the process proceeds to step S604. In this case, since the flag of the parameter i Select [i] is set to "true" at step S605, the process proceeds from step S604 to step S607, the next cut video _{V C [I [i + 1} ]] Is processed.

On the other hand, the video summary selection unit 40, in step S606, the case where the total number of frames of the selected cut image V _C [i] as part of the digest video is determined to be greater than the predetermined value T _MAX (step S606: Y ), that is, when the length of the entire digest video is determined to be greater than the predetermined value T _MAX, the process proceeds to step S611.

On the other hand, the summary video selection unit 40 determines in step S604 that the flag Select [i] is set to "true" or the count Count [J [i]] satisfies a condition larger than the _{predetermined value N MAX.} (Step S604: Y), the parameter i is incremented (i = i + 1, step S607). That is, when the flag Select [i] is set to "true", the summary video selection unit 40 performs the process of step S607. Alternatively, when the count Count [J [i]] is _{larger than the predetermined value N MAX} , the summary video selection unit 40 performs the process of step S607.

In other words, the summary video selection unit 40 _{processes the next cut video V C} [I [i + 1]] _{when the cut video V C} [I [i]] is selected as a part of the summary video. To do so, the process proceeds to step S607. Alternatively, the summary video selection unit 40 is selected as a part of the summary video in _{the scene video V S} [J [i]] of the scene number J [i] to which the _{cut video V C [I [i]] belongs.} When the number of cut images is larger than the _{predetermined value N MAX} , the process of step S607 is performed for the processing of the remaining cut images V _{C [I [i]] in the scene image V S [J].} The remaining cut video V _C [I [i]] in this case is not selected as the summary video.

Summary video selection unit 40 determines whether the parameter i is greater than the predetermined value N _C (i> N _C, step S608). That is, the summary video selection unit 40 _{determines whether or not the processing of all the cut video V C} [I [i]] is completed.

Summary video selection unit 40, in step S608, the case where the parameter i is not greater than the predetermined value N _C (step S608: N), the process proceeds to step S604, the the cut video V _C [I [i]] Process.

On the other hand, when the summary video selection unit 40 determines in step S608 that the parameter i is _{larger than the predetermined value N C} (step S608: Y), that is, the processing of all the cut video V _C [I [i]] is performed. When completed, the predetermined value N _MAX is incremented (N _MAX = N _MAX +1, step S609).

As a result, the predetermined value N _MAX (the maximum value of the number of _{cut images V C} [i [i]] selected from one scene image V _{S [j [i]]) used in step S604 is incremented.} .. Then, when the process proceeds from step S610 to step S603, which will be described later, processing is performed in order from _{the cut video V C [I [i]] with the first parameter i = 1. Therefore, the cut video V C} [I [i]] selected as a part of the summary video is added until the condition of step S606 is satisfied.

The summary video selection unit 40 _{determines whether or not the predetermined value N MAX} is larger than the predetermined value Nc (step S610), and determines that the predetermined value N _MAX is not larger than the predetermined value Nc (step S610: N). ), The process proceeds to step S603. On the other hand, when the summary video selection unit 40 determines that the predetermined value N _MAX is larger than the predetermined value Nc (step S610: Y), the process proceeds to step S611.

The summary video output unit 41 shifts from step S606 (Y) or step S610 (Y), and all the cut video V _C [I [i]] selected in step S605 (flag Select [i] = true). All the cut images V _C [I [i]]) that are are sorted in chronological order and concatenated. Then, the summary video output unit 41 _{generates and outputs the connected video V C} [i ' ₁ ] ... V _C [i' _L ] as a summary video (step S611). L indicates the number of selected cut images V _C [I [i]].

For example, as shown in FIG. 7, according to step S611, all cut images in which the flag Select [i] = true V _C [I [1]] = V _C [8], V _C [I [2]] = V _C [4], V _C [I [3]] = V _C [1], V _C [I [4]] = V _C [10], V _C [I [5]] = V _C [3], etc. Are sorted and concatenated in chronological order. The linked image _{V C [i '1] =} V C [1], V C [i' 2] = V C [3], V C [i '3] = V C [4] ,. .. .. _{, V C [i 'L]} = V C [25] is generated as a summary image.

As described above, according to the abstract image generating apparatus 1 according to an embodiment of the present invention, the cut dividing unit 10, cuts a program image video _{V C [i] (i =} 1, ..., N C) divided Then, the scene generation unit 11 _{integrates the cut video V C} [i] of the same scene to generate the scene video V _S [j] (j = 1, ..., N _S ).

The element score calculation unit 12 has an element score S ₁ [i _{] for each cut image V C} [i] for each production of four elements consisting of "telop", "face recognition", "camera work", and "CG image-likeness". ] ~ S ₄ [i] (telop score S ₁ [i], face recognition score S ₂ [i], camera work score S ₃ [i], and CG image-likeness score S ₄ [i]) are calculated.

The total score calculation unit 13 calculates the total score S [i] of the cut video V _{C [i] based on the element scores S 1} [i] to S ₄ [i] and the weighting coefficients W _{1 to} W _4.

The summary video generation unit 14 refers to the total score S [i] and the scene video V _S [j], and selects and selects the _{cut video V C} [i] until the length of the entire summary video exceeds a predetermined value. A summary video is generated by sorting and concatenating the cut video V _{C [i] in chronological order.}

As a result, a summary video is generated in consideration of elements that tend to occur in important scenes in the program production, such as each element of "telop", "face recognition", "camera work", and "CG image-likeness" in the program image. be able to.

That is, by considering the effect appearing in the program image, it is possible to generate a summary image composed of only the images of the important sections having a high degree of effect by the effect. Then, by distributing such a summary video as content for the Internet, it can be used as a medium for increasing the viewer's interest in the program.

Further, according to the summary video generation device 1 according to the embodiment of the present invention, the total score calculation unit 13 calculates _{the total score S [i] of the cut video V C} [i] when the element score S ₁ [i] is calculated. ] ~ S _{4 The weighting factors W 1 to} W ₄ set for each [i] are now used.

This makes it possible in accordance with the weighting coefficients W ₁ to _W-4, reflecting the scores of the element of the weighting factors W ₁ to _W-4 to the total score S [i]. In other words, it is possible to generate various variations of summary video according to the user's intention by freely setting which element is emphasized to create the summary video for the weighting factors W _{1 to} W _4. can.

Further, according to the summary video generation device 1 according to the embodiment of the present invention, when the summary video generation unit 14 _{selects the cut video V C} [i] for generating the summary video, the scene video V _S [j] ], The number of _{cut video V C} [i] selected in] is prevented from exceeding the _{specified value N MAX.} As a result, the cut video V _C [i] is not selected biased to a specific scene, so that the summarized video biased to a specific scene is not generated. That is, a summary video is generated in consideration of the entire scene of the program video.

Although the present invention has been described above with reference to embodiments, the present invention is not limited to the above-described embodiments, and various modifications can be made without departing from the technical idea. In the above-described embodiment, the summary video is generated with the types of production as "telop", "face recognition", "camera work", and "CG image-likeness", but the types of production are limited to these four elements. Other elements may be used instead of the ones. For example, a summary video may be generated by using loudness, music, conversation, a specific object, or the like as elements of production. Further, a summary video may be generated by using an arbitrary predetermined number of elements of these effects.

As described above, the element score represents the importance that reflects the degree of the effect of the effect. Therefore, when the effect is "loudness", the element score is set to a value proportional to the loudness, for example. .. Also, if the effect is "music", element score is set in accordance with the time ratio that music plays in the cut video V _C [i]. Further, when the effect is a "specific object", the element score is set based on the area ratio of the area where the object is detected and the probability that the object is the object, as in the case of the above-mentioned "face recognition". ..

As the hardware configuration of the summary video generation device 1 according to the embodiment of the present invention, a normal computer can be used. The summary video generator 1 is composed of a computer including a volatile storage medium such as a CPU and RAM, a non-volatile storage medium such as a ROM, and an interface. Each function of the cut division unit 10, the scene generation unit 11, the element score calculation unit 12, the total score calculation unit 13, and the summary video generation unit 14 provided in the summary video generation device 1 uses a program describing these functions in the CPU. It is realized by executing each. These programs are stored in the storage medium, read by the CPU, and executed. In addition, these programs can be stored and distributed in storage media such as magnetic disks (floppy (registered trademark) disks, hard disks, etc.), optical disks (CD-ROM, DVD, etc.), semiconductor memories, etc., and can be distributed via a network. You can also send and receive.

1 Summary video generation device 10 Cut division unit 11 Scene generation unit 12 Element score calculation unit 13 Comprehensive score calculation unit 14 Summary video generation unit 20 Telop area detection unit 21 Face recognition processing unit 22 Camera work calculation unit 23 CG image quality calculation unit 24 Telop score calculation unit 25 Face recognition score calculation unit 26 Camera work score calculation unit 27 CG video-likeness score calculation unit 30 Weight coefficient setting unit 31 Score calculation unit 40 Summary video selection unit 41 Summary video output unit

Claims (5)

In a summary video generator that generates a summary video from video
A cut division unit that divides the image into a plurality of cut images in cut units,
A scene generation unit that integrates a series of cut images of the same scene from the plurality of cut images divided by the cut division unit and generates a scene image for each scene.
A score calculation unit that calculates a score indicating the importance of the effect for each of a predetermined number of different effects for each of the plurality of cut images divided by the cut division unit.
A total score calculation unit that calculates the total score based on the score for each effect calculated by the score calculation unit for each of the plurality of cut images divided by the cut division unit.
Based on the total score calculated by the total score calculation unit, a summary video generation unit that selects a cut video constituting the summary video from the plurality of cut videos and generates the summary video is provided .
The summary video generation unit
When selecting the cut video from the scene video generated by the scene generation unit, the feature is that the number of the cut video selected from the scene video does not exceed a predetermined value. Summary video generator. In the summary video generator according to claim 1,
The total score calculation unit
A summary characterized in that, for each of the plurality of cut images, the total score is calculated based on the score for each effect calculated by the score calculation unit and the weighting coefficient for each effect set in advance. Video generator. In the summary video generator according to claim 1 or 2.
The score calculation unit
The score is calculated based on the area where the object related to the effect appears in the image, the amount of movement of the object related to the effect, or the probability that the object related to the effect appears. Summary video generator. In the summary video generator according to any one of claims 1 to 3,
The summary video generation unit
From the plurality of cut images divided by the cut division unit, the scene images generated by the scene generation unit are generated in descending order of importance of the effect according to the total score calculated by the total score calculation unit. A summary video selection unit that selects the cut video constituting the summary video until the total length of the summary video exceeds the predetermined value so that the number of the cut video selected from the list does not exceed the predetermined value.
A summary video output unit that generates the summary video by connecting the cut videos selected by the summary video selection unit in chronological order.
A summary video generator characterized by being equipped with. A program for operating a computer as a summary video generator according to any one of claims 1 to 4.

Images