JP2018206292A - Video summary creation device and program - Google Patents

Abstract

To create a video summary which is constituted of only an image between important zones by taking into consideration a performance appearing in the image.SOLUTION: A cut division part 10 of a video summary creation device 1 divides a program video into cut videos V[i], a scene creation part 11 creates a scene video V[j] by integrating the cut videos V[i] having the same scenes. An element score calculation part 12 calculates element scores S[i] to S[i] with respect to the respective cut videos V[i] at each performance such as a "telop". An integration score calculation part 13 calculates an integration score S[i] of the cut videos V[i] on the basis of the element scores S[i] to S[i] and a weight coefficient Wto W.A video summary creation part 14 refers to the integration score S[i] and the scene video V[j], selects the cut videos V[i] until a length of a video summary as a whole exceeds a prescribed value, and creates the video summary by sorting and connecting the selected cut videos V[i] in time series. By this constitution, the video summary taking into consideration the performance such as the "telop" can be created.SELECTED DRAWING: Figure 1

Description

  The present invention relates to an apparatus and program for generating summary video in the field of video processing using a computer and a hard disk.

  2. Description of the Related Art Conventionally, in broadcasting stations, there is an increasing need for “net contents” such as the program HP as a medium for increasing the interest of viewers in programs. In content for the Internet, it is desirable to distribute a digest video (summary video) composed only of important video sections in a program video.

  However, manually producing a summary video is very expensive in terms of both labor and cost. For this reason, a technique for automatically generating a summary video is desired. As techniques for automatically generating a summary video, for example, methods of Patent Documents 1 to 5 have been proposed.

  In the method of Patent Document 1, a video is divided into a plurality of video sections for each modal of the video, the similarity between two video sections is obtained, and the video sections are clustered based on the similarity. Then, a representative video section is extracted from each of the plurality of clusters, and a summary video is generated by combining the representative video sections.

  The method of Patent Document 2 generates a corresponding section between videos based on the similarity for each video, extracts a common video section and an individual video section from the corresponding section, selects a common summary section from the common video section, Select an individual summary section from an individual video section. Then, the summary video is generated by integrating the common summary section and the individual summary section.

  The method of Patent Document 3 selects one or more video segments from a plurality of video segments based on metadata and feature amounts, obtains a set of video segments that maximizes an evaluation value of a predetermined evaluation function, and A summary video is generated by combining sets of sections.

  The method of Patent Document 4 obtains an image feature amount and an audio feature amount for a video, calculates the importance of a video processing unit based on the image feature amount and the audio feature amount, and summarizes the video based on the importance. Is generated.

  The method of Patent Literature 5 generates a divided video with a short time from the original video, regards the block area as a visual word for each divided video, calculates a score based on the feature amount of the visual word, and divides the score in descending order of score. Select video and generate summary video.

JP 2014-179906 A JP 2013-126233 A JP 2012-19305 A JP, 2014-33417, A JP 2012-10265 A

  The methods described in Patent Documents 1 and 2 generate summary videos based on the similarity between video sections or the presence or absence of a common section. However, since these methods are based on the premise that similar video sections appear repeatedly, it is difficult to apply to general broadcast program videos in which similar video sections do not always appear repeatedly.

  Further, the method disclosed in Patent Document 3 described above generates a summary video based on a basic image feature amount, audio feature amount, and assigned metadata. However, it is necessary to give detailed metadata to the video in advance, and the processing load is high.

  In addition, the methods disclosed in Patent Documents 4 and 5 generate a summary video based on basic image feature amounts and audio feature amounts without using metadata. Since these methods do not use metadata, the processing load is low compared to the method of Patent Document 3.

  These methods of Patent Documents 1 to 5 all generate a summary video, but do not consider the effects that appear in the video. For this reason, the summary video may not include an important scene from the viewpoint of production. Here, the term “production” refers to expressing effectively so as to achieve a predetermined intention based on a script or the like. For example, the production of broadcast program video includes elements such as telop display, appearance of a main performer or guest, camera zoom-in or panning, and explanatory CG video. These elements tend to be used for important scenes in video.

  In general, it is desirable that the summary video is composed only of video in an important section. For this reason, it has been desired to determine the degree of effect of the effect on the video, identify a highly effective section as an important section, and combine the videos in the important section to generate a summary video.

  Therefore, the present invention has been made to solve the above-described problems, and its purpose is to take into account the effects that appear in the video and to generate a summary video that can generate a summary video composed only of video in an important section. To provide a generation device and a program.

  In order to solve the above-mentioned problem, the summary video generation device according to claim 1 is a summary video generation device that generates a summary video from a video, a cut division unit that divides the video into a plurality of cut videos in a cut unit, For each of the plurality of cut videos divided by the cut dividing unit, for each predetermined number of different effects, a score calculating unit that calculates a score representing the importance of the effect, and the plurality of the divided divided by the cut dividing unit For each of the cut video, based on the score for each effect calculated by the score calculation unit, based on the total score calculated by the total score calculation unit and the total score calculation unit that calculates a total score A summary video generation unit that selects the cut video constituting the summary video from the plurality of cut videos, and generates the summary video; Characterized by comprising.

  The summary video generation device according to claim 2 is the summary video generation device according to claim 1, wherein the total score calculation unit calculates the effect calculated by the score calculation unit for each of the plurality of cut videos. The total score is calculated based on each score and a preset weighting factor for each effect.

  The summary video generation device according to claim 3 is the summary video generation device according to claim 1, further comprising: cutting video of the same scene from the plurality of cut videos divided by the cut division unit. A summary generation video generation unit, wherein the summary video generation unit does not exceed a predetermined value so that a number of the cut video selected from the scene video generated by the scene generation unit does not exceed a predetermined value. It is characterized by selecting a cut video that constitutes.

  The summary video generation device according to claim 4 is the summary video generation device according to any one of claims 1 to 3, wherein the score calculation unit causes an object related to the effect to appear in the video. The score is calculated based on the area, the amount of movement of the object related to the effect, or the probability that the object related to the effect will appear.

  The summary video generation device according to claim 5 is the summary video generation device according to claim 3, wherein the summary video generation unit calculates the total score from the plurality of cut videos divided by the cut division unit. The number of cut videos selected from the scene videos generated by the scene generation unit in order of the importance of the effect according to the total score calculated by the unit does not exceed a predetermined value. Until the total length of the summary video exceeds a predetermined value, the summary video selection unit that selects the cut video constituting the summary video, and the cut video selected by the summary video selection unit are connected in time series, A summary video output unit for generating the summary video.

  Furthermore, a program according to a sixth aspect causes a computer to function as the summary video generation apparatus according to any one of the first to fifth aspects.

  As described above, according to the present invention, it is possible to generate a summary video composed only of video in an important section by taking into consideration the effects that appear in the video.

It is a block diagram which shows the structural example of the summary video generation apparatus by embodiment of this invention. It is a figure explaining cut series V _C [1], ..., V _C [N _C ] and scene series V _S [1], ..., V _S [N _S ]. It is a block diagram which shows the structural example and example of input-output data of an element score calculation part. It is a block diagram which shows the structural example and example of input-output data of a comprehensive score calculation part. It is a block diagram which shows the structural example and example of input / output data of a summary image | video production | generation part. It is a flowchart which shows the process example of the summary image | video production | generation part. It is a figure explaining the flowchart of FIG.

  Hereinafter, embodiments for carrying out the present invention will be described in detail with reference to the drawings. The present invention obtains the degree of effect by the effects appearing in the video (for example, telop, performer, camerawork movement amount, CG video for explanation, etc.) as the importance, and identifies the high-efficiency section as the important section, A summary video is generated by combining videos of important sections. As a result, a summary video composed only of video in an important section is generated.

〔overall structure〕
First, the overall configuration of a summary video generation apparatus according to an embodiment of the present invention will be described. FIG. 1 is a block diagram illustrating a configuration example of a summary video generation apparatus according to an embodiment of the present invention. The summary video generation apparatus 1 includes a cut division unit 10, a scene generation unit 11, an element score calculation unit 12, a total score calculation unit 13, and a summary video generation unit 14.

The cut dividing unit 10 inputs a program video, performs a “cut video dividing process” for dividing the program video into cut videos V _C [i] (i = 1,..., N _C ) in units of cuts, A cut sequence V _C [1],..., V _C [N _C ] composed of the video V _C [i] is generated. Then, the cut dividing unit 10 outputs the cut series V _C [1],..., V _C [N _C ] including the cut video V _C [i] to the scene dividing unit 11 and the element score calculating unit 12.

Parameter i = 1, ..., N _C represents the cut video V _C [i] number (cut number), N _C indicates the number of cut video V _C [i]. The cut video V _C [i] is a video shot continuously without a break before the camera is switched.

  Note that “cut video division processing” is already known, and for details, see, for example, Japanese Patent Laid-Open No. 2008-33749.

The scene generation unit 11 inputs the cut sequence V _C [1],..., V _C [N _C ] composed of the cut video V _C [i] from the cut division unit 10. Then, the scene generation unit 11 detects the scene boundary by integrating the cut video V _C [i] of the same scene, and the scene video V _S [j] (j = 1,..., N _S ) for each scene. To generate a scene sequence V _S [1],..., V _S [N _S ] composed of the scene video V _S [j]. The scene generation unit 11 outputs the scene series V _S [1],..., V _S [N _S ] composed of the scene video V _S [j] to the summary video generation unit 14.

Parameter j = 1, ..., N _S denotes the scene image V _S [j] number (scene number), N _S denotes the number of scene image V _S [j]. The scene image V _S [j] is an image in which a series of cut images V _C [i] in a certain scene are collected.

  Note that “scene video generation processing” is known, and for details, see, for example, Japanese Patent Application Laid-Open Nos. 2014-225118 and 2014-33355.

Figure 2 is a cut sequence _{V C [1], ...,} V C [N C] and scene sequence V _S [1], ..., a diagram illustrating the V _S [N _S]. The cut sequence V _C [1], ..., V _C [N _C ] is generated by dividing the program video into cut units, and the scene sequence V _S [1], ..., V _S [N _S ] Is generated by integrating the cut sequences V _C [1], ..., V _C [N _C ] for the same scene.

In the example of FIG. 2, the scene image V _S [1] is an image obtained by integrating the cut images V _C [1], V _C [2], and V _C [3], and the scene image V _S [2] is This is an integrated video of cut video V _C [4] and V _C [5]. The scene image V _S [N _S ] is an image obtained by integrating the cut images V _C [N _C -1] and V _C [N _C ]. Thus, the cut video V _C [i] belongs to one of the scene videos V _S [j].

Returning to FIG. 1, the element score calculation unit 12 inputs a cut sequence V _C [1],..., V _C [N _C ] composed of the cut video V _C [i] from the cut division unit 10. Then, the element score calculation unit 12 performs, for each cut video V _C [i], for each cut video V _C [i] for each production of the four elements including “telop”, “face recognition”, “camera work”, and “CG video quality”. Element scores S ₁ [i] to S ₄ [i] representing importance are calculated based on _C [i].

The element score representing the importance reflects the degree of effect due to the effect appearing in the cut video V _C [i]. The element scores of “telop”, “face recognition”, “camera work” and “CG image quality” for the cut video V _C [i] are the telop score S ₁ [i], the face recognition score S ₂ [i], and the camera, respectively. The work score S ₃ [i] and the CG video likelihood score S ₄ [i] are used.

The element score calculation unit 12 includes element scores S ₁ [i] to S ₄ [i] (telop score S ₁ [i], face recognition score S ₂ [i], camera work score S ₃ [i], and CG image likelihood. Element score series S ₁ [1], ..., S ₁ [N _C ], S ₂ [1], ..., S ₂ [N _C ], S ₃ [1 consisting of scores S ₄ [i]) ], ..., S ₃ [N _C ], S ₄ [1], ..., S ₄ [N _C ] are generated.

The element score calculation unit 12 includes a cut sequence V _C [1],..., V _C [N _C ] composed of cut videos V _C [i] and element scores S ₁ [i] to S ₄ [i] ( Element score series S ₁ [1], composed of telop score S ₁ [i], face recognition score S ₂ [i], camera work score S ₃ [i], and CG video-likeness score S ₄ [i]) ... , S ₁ [N _C ], S ₂ [1], ..., S ₂ [N _C ], S ₃ [1], ..., S ₃ [N _C ], S ₄ [1], .. , S ₄ [N _C ] is output to the total score calculation unit 13. Details of the element score calculation unit 12 will be described later.

The total score calculation unit 13 receives from the element score calculation unit 12 a cut sequence V _C [1],..., V _C [N _C ] composed of the cut video V _C [i], and an element score S ₁ [i]. to S ₄ [i] element score line S ₁ [1] consisting _{of, ..., S 1 [N C} ], S 2 [1], ..., S 2 [N C], S 3 [1] , ..., S ₃ [N _C ], S ₄ [1], ..., S ₄ [N _C ] are input.

The total score calculation unit 13 integrates the element scores S ₁ [i] to S ₄ [i] using the preset weighting factors W _{1 to} W ₄ for each cut video V _C [i], and generates a total score. S [i] is calculated, and a total score series S [1],..., S [N _C ] composed of the total score S [i] is generated. Then, the total score calculation unit 13 includes a cut sequence V _C [1], ..., V _C [N _C ] composed of the cut video V _C [i], and a total score sequence S composed of the total score S [i]. [1],..., S [N _C ] are output to the summary video generation unit 14. Details of the total score calculation unit 13 will be described later.

The summary video generation unit 14 receives the cut score V _C [1],..., V _C [N _C ] including the cut video V _C [i] and the total score S [i] from the total score calculation unit 13. The total score series S [1], ..., S [N _C ] is input. In addition, the summary video generation unit 14 receives the scene series V _S [1],..., V _S [N _S ] including the scene video V _S [j] from the scene generation unit 11.

The summary video generation unit 14 refers to the overall score S [i] and the scene video V _S [j], and the cut video V _C [i] constituting the summary video until the total length of the summary video exceeds a predetermined value. Select. Then, the summary video generation unit 14 generates a summary video by sorting and connecting the selected cut video V _C [i] in time series (in order of frame number), and outputs the summary video. Details of the summary video generation unit 14 will be described later.

[Element Score Calculation Unit 12]
Next, the element score calculation unit 12 shown in FIG. 1 will be described in detail. As described above, the element score calculation unit 12 performs the cut for each cut video V _C [i] for each production of four elements including “telop”, “face recognition”, “camera work”, and “CG video quality”. Based on the video V _C [i], a telop score S ₁ [i], a face recognition score S ₂ [i], a camera work score S ₃ [i], and a CG video likelihood score S ₄ [i] are calculated.

  FIG. 3 is a block diagram illustrating a configuration example and input / output data example of the element score calculation unit 12. The element score calculation unit 12 includes a telop area detection unit 20, a face recognition processing unit 21, a camera work calculation unit 22, a CG video likelihood calculation unit 23, a telop score calculation unit 24, a face recognition score calculation unit 25, and a camera work score calculation. Unit 26 and a CG video likelihood score calculation unit 27.

The element score calculation unit 12 inputs a cut sequence V _C [1],..., V _C [N _C ] composed of the cut video V _C [i] from the cut division unit 10, and uses this as a total score calculation unit 13 is output.

Ticker area detection unit 20, from the cut dividing unit 10, a cut line consisting of cut video _{V C [i] V C [} 1], ..., V C [N C] Enter the cut video V _C [i ] Sample frame image P [i, n] every T frames to generate image sequence P [i, 1], ..., P [i, N _P ] consisting of frame images P [i, n] To do. Then, the telop area detection unit 20 calculates the image sequence P [i, 1],..., P [i, N _P ] composed of the frame images P [i, n] as the face recognition processing unit 21 and the CG image likelihood. To the unit 23.

Parameters n = 1,..., N _P indicate the numbers of the frame images P [i, n], and N _P indicates the number of sampled frame images P [i, n].

The telop area detection unit 20 performs “telop area detection processing” for detecting an area in which a telop is displayed for each frame image P [i, n], calculates the area of the telop area, and calculates the frame image P [ The area ratio r _TL [i, n] of the telop area to i, n] is calculated. The ticker area detection unit 20, the frame image P [i, n] area of telop area for the ratio r _TL [i, n] consists of area ratio sequence _{r TL [i, 1],} ..., r TL [ i, N _P ] is generated and output to the telop score calculation unit 24. The area of the telop area is an area where “telop”, which is a target related to the production, appears in the video.

  Note that “telop area detection processing” is already known, and for details, see, for example, Japanese Patent Application Laid-Open No. 2013-30963.

C_TLは正規化定数であり、予め設定される。 The telop score calculation unit 24 sends an area ratio series r _TL [i, 1],... _Composed of the area ratio r _TL [i, n] of the telop area to the frame image P [i, n] from the telop area detection unit 20. ., r Enter _TL [i, N _P ]. Then, the telop score calculation unit 24 calculates the telop score S _{1 of the} cut video V _C [i] based on the area ratio r _TL [i, n] for each cut video V _C [i] according to the following formula. [i] is calculated.

C _TL is a normalization constant and is set in advance.

The range of the telop score S ₁ [i] is 0 ≦ S ₁ [i] ≦ 1. The telop score S ₁ [i] is larger as the area of the telop area is larger in the frame image P [i, n], and is smaller as the area is smaller. That is, the telop score S ₁ [i] increases as the frame image P [i, n] having a wider area in which the telop is displayed appears in the cut video V _C [i]. The telop score S ₁ [i] is an area ratio r _TL [i, n] of the frame image P [i, n] having the widest area where the telop is displayed in the cut video V _C [i]. It becomes a value normalized in the range between.

The telop score calculation unit 24 generates a telop score series S ₁ [1],..., S ₁ [N _C ] composed of the telop score S ₁ [i], and outputs this to the total score calculation unit 13.

The face recognition processing unit 21 inputs a cut sequence V _C [1],..., V _C [N _C ] composed of the cut video V _C [i] from the cut dividing unit 10, and a telop area detection unit 20. , P [i, 1],..., P [i, N _P ] composed of frame images P [i, n] are input.

  For each frame image P [i, n], the face recognition processing unit 21 selects the main performer ID [m] (M main performer IDs [1],..., ID [M]) in the target program. The “face recognition process” for recognizing these faces is performed, and the face area F [i, n, k] is detected. It is assumed that K face areas F [i, n, k] are detected from the frame image P [i, n]. The face recognition processing unit 21 generates K face area sequences F [i, n, 1],..., F [i, n, K] composed of face areas F [i, n, k].

  Parameters m = 1,..., M indicate the number of the main performer ID [m], and M indicates the number of the main performer ID [m]. Parameters k = 1,..., K indicate the number of the face area F [i, n, k], and K indicates the face area F [i, n] detected from the frame image P [i, n]. n, k].

  Note that “face recognition processing” is known, and for details, see, for example, Japanese Patent Application Laid-Open No. 2017-33372.

The face recognition processing unit 21 calculates an area ratio r _FC [i, n, k] of the face area F [i, n, k] with respect to the frame image P [i, n], and K area ratios r _FC. An area ratio series r _FC [i, n, 1], ..., r _FC [i, n, K] consisting of [i, n, k] is generated.

The face recognition processing unit 21 performs a “face probability calculation process” for calculating the probability p _FC [i, n, k, m] that the face area F [i, n, k] is the face of the performer ID [m]. And generate a probability sequence p _FC [i, n, k, 1], ..., p _FC [i, n, k, M] consisting of M probabilities p _FC [i, n, k, m] To do. This probability p _FC [i, n, k, m] is the probability that the “face” of the performer ID [m], which is the target related to the production, appears.

  The “face probability calculation process” is already known, and for details, refer to, for example, Japanese Patent Application Laid-Open No. 2017-33372 as in the “face recognition process”.

The face recognition processing unit 21 has an area ratio series r _FC [i, n, k] composed of area ratios r _FC [i, n, k] of the face region F [i, n, k] with respect to the frame image P [i, n]. 1], ..., r _FC [i, n, K] and face area F [i, n, k] in frame image P [i, n] are the faces of performer ID [m] A probability sequence p _FC [i, n, k, 1], ..., p _FC [i, n, k, M] consisting of probabilities p _FC [i, n, k, m] Output to.

The face recognition score calculation unit 25 receives an area ratio composed of the area ratio r _FC [i, n, k] of the face region F [i, n, k] with respect to the frame image P [i, n] from the face recognition processing unit 21. The series r _FC [i, n, 1], ..., r _FC [i, n, K] and the face area F [i, n, k] in the frame image P [i, n] Probability sequence p _FC [i, n, k, 1], ..., p _FC [i, n, k, M] consisting of the probability p _FC [i, n, k, m] that is the face of ID [m] ] Is entered.

C’_FCは、パラメータw_FC[i,n,k]の最小値を定める値であり、予め設定される。 The face recognition score calculation unit 25 calculates the probability p _FC [i,] that the face area F [i, n, k] in the frame image P [i, n] is the face of the performer ID [m] by the following formula. The parameter w _FC [i, n, k] is calculated based on n, k, m].

C ′ _FC is a value that determines the minimum value of the parameter w _FC [i, n, k], and is set in advance.

The parameter w _FC [i, n, k] has a maximum probability p _FC [i, n, k, m] in the face area F [i, n, k] in the frame image P [i, n]. The lower the value, the larger the value, and the higher the probability of maximum p _FC [i, n, k, m], the smaller the value.

The face recognition score calculation unit 25 calculates the area ratio r _FC [i, n, k] of the face area F [i, n, k] with respect to the frame image P [i, n] by the following formula and the formula (2) The parameter R _FC [i, n] is calculated based on the parameter w _FC [i, n, k] calculated in (1).

The parameter R _FC [i, n] is included in the face area F [i, n, k] in the parameter w _FC [i, n, k] of the face area F [i, n, k] in the frame image P [i, n]. This is a value obtained by multiplying the area ratio r _FC [i, n, k] of k] and adding the multiplication results for all face regions F [i, n, k].

The parameter R _FC [i, n] increases as the parameter w _FC [i, n, k] increases (the probability p _FC [i, n, k, m] decreases). The parameter w _FC [i, n] , n, k] is smaller (the probability p _FC [i, n, k, m] is higher), the smaller the value is. The parameter R _FC [i, n] is the area ratio _{r FC [i, n, k} ] higher, becomes a large value, as the area ratio _{r FC [i, n, k} ] is small, a small value .

C_FCは正規化定数であり、予め設定される。 Face recognition score calculation unit 25, by the following equation, for each cut image V _C [i], the parameter R _FC [i, n] calculated the at Equation (3) based on, the cut video V _C [ The face recognition score S ₂ [i] of i] is calculated.

C _FC is a normalization constant and is set in advance.

The range of the face recognition score S ₂ [i] is 0 ≦ S ₂ [i] ≦ 1. Face recognition score S ₂ [i], the higher the parameter R _FC [i, n] is large, it becomes a large value, as the parameter R _FC [i, n] is small, a small value. In other words, the face recognition score S ₂ [i], the probability _{p FC [i, n, k} , m] is as low, becomes a large value, the probability _{p FC [i, n, k} , m] higher, smaller Thus, the higher the area ratio r _FC [i, n, k], the larger the value, and the lower the area ratio r _FC [i, n, k], the smaller the value.

The probability p _FC [i, n, k, m] is the probability that the face area F [i, n, k] is the face of the main performer ID [m], so the probability p _FC [i, n, If k, m] is low, there is a high probability that it is a non-main performer ID [m] face (guest face), and if the probability p _FC [i, n, k, m] is high, the guest face This means that the probability of being is low.

Therefore, the face recognition score S ₂ [i] is larger as the total area where the face of the main performer ID [m] appears in the frame image P [i, n] included in the cut video V _C [i] The larger the value, the smaller the total area in which the face of the main performer ID [m] appears. In addition, the face recognition score S ₂ [i] increases as the probability that the guest's face appears in the frame image P [i, n] included in the cut video V _C [i] increases. The lower the probability of appearing, the smaller the value. That is, the face recognition score S ₂ [i] is such that the frame image P [i, n] having a large total area in which the face of the main performer ID [m] appears in the cut video V _C [i] The value increases as the frame image P [i, n] with a higher probability of appearance of the guest's face appears.

The face recognition score calculation unit 25 generates a face recognition score series S ₂ [1],..., S ₂ [N _C ] composed of the face recognition score S ₂ [i], and sends this to the total score calculation unit 13. Output.

The camera work calculation unit 22 inputs the cut sequence V _C [1],..., V _C [N _C ] composed of the cut video V _C [i] from the cut dividing unit 10 and performs “camera work detection processing”. I do. Then, the camera work calculation unit 22 obtains a video section V _CW [i, q] in which predetermined camera work such as zooming and panning has occurred for each cut video V _C [i], and the video section V _CW [i, Generate a video interval series V _CW [i, 1], ..., V _CW [i, N _CW ] consisting of q].

Parameter q = 1, ..., N _CW indicates the number of video segments _{V CW [i, q],} N CW , the video segment is detected from the cut video _{V C [i] V CW [} i, q ] Number.

  The “camera work detection process” is already known, and for details, see, for example, Japanese Patent Laid-Open No. 10-243340.

Camera work calculation unit 22, the video section V _CW [i, q] to calculate the movement amount of the camera for normalization of the amount of motion of the camera by the length of the diagonal of the image (division) motion ratio r _CW [i , q] to generate a motion rate sequence r _CW [i, 1],..., r _CW [i, N _CW ] composed of the motion rates r _CW [i, q]. Then, the camera work calculation unit 22 _{converts the} motion rate sequence r _CW [i, 1],..., R _CW [i, N _CW ] composed of the motion rates r _CW [i, q] into the camera work score calculation unit 26. Output to. This motion rate r _CW [i, q] is a motion rate of “camera work” that is a target related to the effect.

C_CWは正規化定数であり、予め設定される。 The camera work score calculation unit 26 receives the motion rate sequence r _CW [i, 1], ..., r _CW [i, N _CW ] composed of the motion rate r _CW [i, q] from the camera work calculation unit 22. type, by the following equation, for each cut image V _C [i], based on the movement rate r _CW [i, q], calculated camerawork score S ₃ [i] of the cut video V _C [i] To do.

C _CW is a normalization constant and is set in advance.

The range of the camera work score S ₃ [i] is 0 ≦ S ₃ [i] ≦ 1. In the cut video V _C [i], the camera work score S ₃ [i] increases as the camera movement amount increases, and decreases as the camera movement amount decreases. That is, the camera work score S ₃ [i] increases as the video section V _CW [i, q] with a large amount of camera motion appears in the cut video V _C [i].

The camera work score calculation unit 26 generates a camera work score series S ₃ [1],..., S ₃ [N _C ] composed of the camera work score S ₃ [i], and sends this to the total score calculation unit 13. Output.

The CG video quality calculation unit 23 receives a cut sequence V _C [1],..., V _C [N _C ] composed of the cut video V _C [i] from the cut division unit 10 and a telop area detection unit. 20, an image sequence P [i, 1],..., P [i, N _P ] composed of frame images P [i, n] is input.

The CG video likelihood calculation unit 23 performs “discrimination processing using a recognition model of a support vector machine (SVM)” for each frame image P [i, n], and a probability p _CG [i, n] that is a CG video. Is calculated. Then, the CG video likelihood calculation unit 23 generates a probability sequence p _CG [i, 1],..., P _CG [i, N _P ] composed of the probability p _CG [i, n] that is a CG video. The range of the probability p _CG [i, n] that is a CG image is 0 ≦ p _CG [i, n] ≦ 1.

  The support vector machine is made to learn in advance a difference between an image feature of a CG video and an image feature of an image that is not a CG video. CG images tend to be highly saturated and have artificial texture features. Therefore, as the image feature, an S value (saturation) histogram in the HSV color space, an edge direction histogram, or a fractal feature with high classification of natural objects and artifacts is used.

  Since “identification processing using a support vector machine (SVM) recognition model” is known, detailed description thereof is omitted here. For details of the fractal feature, refer to, for example, Japanese Patent Laid-Open No. 2001-56820.

The CG image likelihood calculation unit 23 calculates a probability score p _CG [i, 1], ..., p _CG [i, N _P ] composed of the probability p _CG [i, n], which is a CG image, as a CG image likelihood score. To the unit 27.

The CG video likelihood score calculating unit 27 receives a probability sequence p _CG [i, 1],..., P _CG [i, i] including the probability p _CG [i, n] of the CG video from the CG video likelihood calculating unit 23. Enter N _P ]. Then, the CG video likelihood score calculation unit 27 calculates the CG video likelihood score S ₄ [i] for each cut video V _C [i] based on the probability p _CG [i, n] that is a CG video for each cut video V _C [i]. ] Is calculated.

The range of the CG video likelihood score S ₄ [i] is 0 ≦ S ₄ [i] ≦ 1. The CG video likelihood score S ₄ [i] is larger for the frame image P [i, n] included in the cut video V _C [i], as the maximum value of the probability p _CG [i, n] of the CG video is larger. The larger the value, the smaller the maximum value of the probability p _CG [i, n] that is a CG video, the smaller the value. That is, the CG video likelihood score S ₄ [i] increases as the frame image P [i, n] having a high probability p _CG [i, n] that is a CG video appears in the cut video V _C [i]. Value.

The CG video quality score calculation unit 27 generates a CG video quality score series S ₄ [1],..., S ₄ [N _C ] composed of CG video quality scores S ₄ [i], and calculates a total score. To the unit 13.

[Total Score Calculation Unit 13]
Next, the total score calculation unit 13 illustrated in FIG. 1 will be described in detail. As described above, the total score calculation unit 13 determines, for each cut video V _C [i], the telop score S ₁ [i], the face recognition score S ₂ [i], the camera work score S ₃ [i], and the CG video likelihood. The score S ₄ [i] is integrated to calculate the total score S [i].

  FIG. 4 is a block diagram illustrating a configuration example and input / output data example of the total score calculation unit 13. The total score calculation unit 13 includes a weight coefficient setting unit 30 and a score calculation unit 31.

The total score calculation unit 13 receives the cut sequence V _C [1], ..., V _C [N _C ] composed of the cut video V _C [i] from the element score calculation unit 12, and generates the summary video. To the unit 14.

The weighting coefficient setting unit 30 calculates the total score S for each element of the telop score S ₁ [i], the face recognition score S ₂ [i], the camera work score S ₃ [i], and the CG video likelihood score S ₄ [i]. A weight coefficient W _r (r = 1,..., 4) that determines the degree of reflection on [i] is set. Then, the weight coefficient setting unit 30 outputs the weight coefficients W _{1 to} W ₄ to the score calculation unit 31. The weighting factors W _{1 to} W ₄ are freely determined and set in advance by the operation of the user (summary video producer).

The score calculation unit 31 inputs the telop score series S ₁ [1],..., S ₁ [N _C ] composed of the telop score S ₁ [i] from the telop score calculation unit 24 of the element score calculation unit 12. The face recognition score series S ₂ [1],..., S ₂ [N _C ] composed of the face recognition score S ₂ [i] is input from the face recognition score calculation unit 25. Further, the weighting factor setting unit 30 inputs a camera work score series S ₃ [1],..., S ₃ [N _C ] composed of the camera work score S ₃ [i] from the camera work score calculation unit 26. The CG video likelihood score series S ₄ [1],..., S ₄ [N _C ] composed of the CG video likelihood score S ₄ [i] is input from the CG video likelihood score calculation unit 27. Moreover, the score calculation unit 31 inputs weighting factors W _{1 to} W ₄ from the weighting factor setting unit 30.

The score calculation unit 31 calculates the weight coefficient W to the telop score S ₁ [i], the face recognition score S ₂ [i], the camera work score S ₃ [i], and the CG video likelihood score S ₄ [i] according to the following formula. The total score S [i] of the cut video V _C [i] is calculated by multiplying each _r and adding the multiplication results.

The score calculation unit 31 generates a total score series S [1],..., S [N _C ] composed of the total score S [i] of the cut video V _C [i], and this is generated as the summary video generation unit 14. Output to.

Thus, in accordance with the weighting factor W _r, the total score S score of elements corresponding to the weight coefficient W _r is reflected [i] is calculated. For an element for which the weighting factor _Wr is set high, the score for that element is greatly reflected in the overall score S [i], and for an element for which the weighting factor _Wr is set low, the score for that element is the overall score S [i ] Is not reflected so much.

For example, when a large value is set for the weight coefficient W ₂ of the face recognition score S ₂ [i] and a small value is set for the other weight coefficients W ₁ , ₃ , ₄ , the face recognition score S ₂ [i] A total score S [i] that greatly reflects is generated. Then, a summary video in which a face of the main performer ID [m] or a guest's face appears is generated by the summary video generation unit 14 described later. Similarly, to set a large value to the weighting factor W ₄ of the CG image likeness score S ₄ [i], if you set the smaller value to the other weight coefficient W _{1, 2, 3} are, CG image likeness score S ₄ A total score S [i] that greatly reflects [i] is generated. Then, a summary video in which many CG videos appear is generated by the summary video generation unit 14 described later.

[Summary Video Generation Unit 14]
Next, the summary video generation unit 14 shown in FIG. 1 will be described in detail. As described above, the summary video generation unit 14 refers to the overall score S [i] and the scene video V _S [j], and selects the cut video V _C [i] until the total length of the summary video exceeds a predetermined value. Then, the selected cut video V _C [i] is connected to generate a summary video.

  FIG. 5 is a block diagram illustrating a configuration example and input / output data example of the summary video generation unit 14. The summary video generation unit 14 includes a summary video selection unit 40 and a summary video output unit 41.

The summary video selection unit 40, from the total score calculation unit 13, from the cut sequence V _C [1], ..., V _C [N _C ] consisting of the cut video V _C [i], and the total score S [i]. The total score series S [1], ..., S [N _C ] is input. In addition, the summary video selection unit 40 inputs the scene series V _S [1],..., V _S [N _S ] including the scene video V _S [j] from the scene generation unit 11.

The summary video selection unit 40 sorts all the cut videos V _C [i] in descending order of importance based on the overall score S [i], and the sorted cut videos V _C [I [i]] are sorted. In sequence, the cut video V _C [I [i]] is selected until the entire length of the selected cut video V _C [I [i]] (the total length of the summary video) exceeds a predetermined value. The order of importance is the order in which the effect of the production combining “telop”, “face recognition”, “camera work”, and “CG image quality” is high.

SUMMARY image selection section 40, when selecting the cut video _{V C [I [i]]} , the number of cut images selected in the scene image _{V S [j] V C [} I [i]] is a predetermined value Do not exceed. The summary video selection unit 40 then outputs the selected cut video V _C [I [i]] to the summary video output unit 41.

Summary video output unit 41, the digest video selection unit 40 receives the cut image V _C selected [I [i]], and connecting the selected cut image V _C [I [i]] in a time-series Summary Video V _C [i ' ₁ ] ... V _C [i' _L ] is generated, and summary video V _C [i ' ₁ ] ... V _C [i' _L ] is output. L is the number of selected cut videos V _C [i], that is, the number of summary videos V _C [i ′ ₁ ]... V _C [i ′ _L ].

[Operation of Summary Video Generation Unit 14]
FIG. 6 is a flowchart illustrating an example of processing of the summary video generation unit 14, and FIG. 7 is a diagram illustrating the flowchart of FIG. The summary video selection unit 40 of the summary video generation unit 14 generates the cut sequences V _C [1], ..., V _C [in descending order of the scores of the overall score series S [1], ..., S [N _C ]. N _C ] are sorted to generate a cut sequence V _C [I [1]],..., V _C [I [N _C ]] (step S601).

For example, as shown in FIG. 7, the score of the total score series S [1],..., S [N _C ] is S [8]> S [4]> S [1]> S [10]> S [ 3] ＞． . . When> S [20], the cut sequence after sorting is processed as V _C [I [1]] = V _C [8], V _C [I [2]] = V _C [4] by the processing in step S601. , V _C [I [3]] = V _C [1], V _C [I [4]] = V _C [10], V _C [I [5]] = V _C [3],. . . , V _C [I [N _C ]] = V _C [20].

Returning to FIG. 6, the video summary selection unit 40 as an initial setting, and set the parameter i = 1, ..., all the flags the Select [i] for N _C to "false" (Select [i] = false (i = 1, ..., N _C )), set all counts Count [j] for parameters j = 1, ..., N _S to 0 (Count [j] = 0 (j = 1,..., N _S ) (step S 602) Note that this parameter i is used for the explanation of FIG.6 and FIG.7, and includes cut video V _C [i], total score S [i], etc. Different from parameter i.

The flag Select [i] is set to “true” when the cut video V _C [I [i]] is selected as a part of the summary video in steps S604 and S605 described later, and the cut video V _C [ If I [i]] is not selected for the summary video, the default “false” is maintained. Count Count [j] is the scene image V _S [j] of the parameter j scene number, in the scene image V _S [j] Cut belonging to the video _{V C [I [i]]} , the video summary one Indicates the number of cut images V _C [I [i]] selected.

Hereafter, the parameters i = 1, ..., N _C are made to correspond to the sorted cut sequences V _C [I [1]], ..., V _C [I [N _C ]], respectively. A summary video selection process is performed for each cut video V _C [I [i]]. The summary video selection processing is performed until the total number of frames of the cut video V _C [I [i]] selected as the summary video exceeds a predetermined value T _MAX as in step S606 described later.

The summary video selection unit 40 sets the parameter i to 1 (i = 1, step S603), the flag Select [i] is set to “true”, or the count Count [J [i]] is a predetermined value. It is determined whether or not a condition greater than N _MAX is satisfied (step S604). That is, the video summary selection unit 40 genera parameter i of the cut video V _C [I [i]] is selected in a part of the video summary or parameter i of the cut video _{V C, [I [i]} ] In the scene video V _S [J [i]] with the scene number J [i], it is determined whether or not the condition that the number of cut videos selected as a part of the summary video satisfies a condition larger than a predetermined value N _MAX is determined.

J [i] indicates the scene number to which the cut video V _C [I [i]] belongs. N _MAX indicates the maximum value of the number of cut videos V _C [I [i]] selected from one scene video V _S [J [i]], and is preset.

In step S604, the summary video selection unit 40 determines that the flag Select [i] is set to “true” or that the count Count [J [i]] does not satisfy the condition greater than the predetermined value _NMAX . In the case (step S604: N), the cut video V _C [I [i]] is selected as a part of the summary video, the flag Select [i] is set to “true” (Select [i] = true), and the count is performed. Count [J [i]] is incremented (Count [J [i]] = Count [J [i]] + 1) (step S605). That is, the summary video selection unit 40, when “true” is not set in the flag Select [i] (“false” is set), and the count Count [J [i]] is greater than the predetermined value N _MAX . If not, the process of step S605 is performed. In other words, the summary video selection unit 40 does not select the cut video V _C [I [i]] as a part of the summary video and the scene number J to which the cut video V _C [I [i]] belongs. In the scene video V _S [J [i]] of [i], when the number of cut videos selected as part of the summary video is not greater than the predetermined value N _MAX , the process of step S605 is performed.

For example, as illustrated in FIG. 7, among the flags Select [1],..., Select [N _c ] that are initially set to “false” in step S605, the flags Select [1],. , Select [5] etc. are changed to "true".

Returning to FIG. 6, the summary video selection unit 40 performs all the cuts in which the flag Select [i] is set to “true” (Select [i] = true) from the parameter i = 1 to the current parameter i. The total number of frames of the video V _C [I [i]] is calculated, and it is determined whether or not the total number of frames is greater than a predetermined value T _MAX (step S606). That is, the summary video selection unit 40 determines whether or not the total number of frames of all cut videos V _C [i] selected as a part of the summary video is greater than the predetermined value T _MAX , that is, the length of the entire summary video. Whether or not is greater than a predetermined value T _MAX is determined. T _MAX indicates the length (number of frames) of the summary video that the user wants to generate and is preset.

When the summary video selection unit 40 determines in step S606 that the total number of frames of the cut video V _C [i] selected as part of the summary video is not greater than the predetermined value T _MAX (step S606: N). That is, if it is determined that the total length of the summary video is not greater than the predetermined value T _MAX , the process proceeds to step S604. In this case, since the flag Select [i] of the parameter i is set to “true” in step S605, the process proceeds from step S604 to step S607, and the next cut video V _C [I [i + 1]]. Is performed.

On the other hand, when the summary video selection unit 40 determines in step S606 that the total number of frames of the cut video V _C [i] selected as part of the summary video is larger than the predetermined value T _MAX (step S606: Y). ), That is, if it is determined that the length of the entire summary video is greater than the predetermined value T _MAX , the process proceeds to step S611.

On the other hand, the summary video selection unit 40 determines in step S604 that the flag Select [i] is set to “true” or that the count Count [J [i]] satisfies the condition greater than the predetermined value _NMAX. If so (step S604: Y), the parameter i is incremented (i = i + 1, step S607). That is, the summary video selection unit 40 performs the process of step S607 when “true” is set in the flag Select [i]. Or, digest video selection unit 40, when the count Count [J [i]] is greater than the predetermined value N _MAX, the process of step S607.

In other words, when the cut video V _C [I [i]] is selected as a part of the summary video, the summary video selection unit 40 performs processing of the next cut video V _C [I [i + 1]]. Therefore, the process proceeds to step S607. Alternatively, the summary video selection unit 40 is selected as a part of the summary video in the scene video V _S [J [i]] of the scene number J [i] to which the cut video V _C [I [i]] belongs. If the number of cut videos is larger than the predetermined value N _MAX , the process of step S607 is performed to process the remaining cut videos V _C [I [i]] in the scene video V _S [J]. The remaining cut video V _C [I [i]] in this case is not selected as the summary video.

The summary video selection unit 40 determines whether or not the parameter i is greater than the predetermined value N _C (i> N _C , step S608). That is, the summary video selection unit 40 determines whether or not processing of all cut videos V _C [I [i]] has been completed.

When the summary video selection unit 40 determines in step S608 that the parameter i is not greater than the predetermined value N _C (step S608: N), the summary video selection unit 40 proceeds to step S604, and the cut video V _C [I [i]]. Perform the process.

On the other hand, if the summary video selection unit 40 determines in step S608 that the parameter i is larger than the predetermined value N _C (step S608: Y), that is, the processing of all cut video V _C [I [i]] is performed. If completed, the predetermined value N _MAX is incremented (N _MAX = N _MAX +1, step S609).

As a result, the predetermined value N _MAX (the maximum value of the number of cut images V _C [I [i]] selected from one scene image V _S [j [i]]) used in step S604 is incremented. . Then, when the process proceeds from step S610, which will be described later, to step S603, processing is performed in order from the first cut video V _C [I [i]] with the parameter i = 1. Therefore, the cut video V _C [I [i]] selected as a part of the summary video is added until the condition of step S606 is satisfied.

The summary video selection unit 40 determines whether or not the predetermined value N _MAX is larger than the predetermined value Nc (step S610), and determines that the predetermined value N _MAX is not larger than the predetermined value Nc (step S610: N ), The process proceeds to step S603. On the other hand, if the summary video selection unit 40 determines that the predetermined value N _MAX is larger than the predetermined value Nc (step S610: Y), the summary video selection unit 40 proceeds to step S611.

The summary video output unit 41 proceeds from step S606 (Y) or step S610 (Y), and all the cut videos V _C [I [i]] (flag Select [i] = true) selected in step S605. All cut videos V _C [I [i]]) are sorted and connected in time series. Then, the summary video output unit 41 generates and outputs the concatenated video V _C [i ′ ₁ ]... V _C [i ′ _L ] as a summary video (step S611). L indicates the number of selected cut videos V _C [I [i]].

For example, as shown in FIG. 7, in step S611, all the cut videos V _C [I [1]] = V _C [8], V _C [I [2]] = V with the flag Select [i] = true. _C [4], V _C [I [3]] = V _C [1], V _C [I [4]] = V _C [10], V _C [I [5]] = V _C [3] etc. Are sorted and concatenated in time series. Then, the concatenated images V _C [i ′ ₁ ] = V _C [1], V _C [i ′ ₂ ] = V _C [3], V _C [i ′ ₃ ] = V _C [4],. . . , V _C [i ′ _L ] = V _C [25] is generated as a summary video.

As described above, according to the summary video generation device 1 according to the embodiment of the present invention, the cut dividing unit 10 divides the program video into the cut video V _C [i] (i = 1,..., N _C ). Then, the scene generation unit 11 integrates the cut video V _C [i] of the same scene to generate the scene video V _S [j] (j = 1,..., N _S ).

The element score calculation unit 12 generates an element score S ₁ [i] for each cut video V _C [i] for each production of the four elements including “telop”, “face recognition”, “camera work”, and “CG image quality”. ] To S ₄ [i] (telop score S ₁ [i], face recognition score S ₂ [i], camera work score S ₃ [i], and CG video likelihood score S ₄ [i]) are calculated.

The total score calculation unit 13 calculates the total score S [i] of the cut video V _C [i] based on the element scores S ₁ [i] to S ₄ [i] and the weighting factors W _{1 to} W ₄ .

The summary video generation unit 14 refers to the overall score S [i] and the scene video V _S [j], selects and selects the cut video V _C [i] until the total length of the summary video exceeds a predetermined value. A summary video is generated by sorting and connecting the cut video V _C [i] in time series.

  As a result, a summary video is generated that takes into consideration factors that are likely to occur in scenes important for program production, such as the elements of “telop”, “face recognition”, “camera work”, and “CG video” in the program video. be able to.

  In other words, by taking into account the effects that appear in the program image, it is possible to generate a summary image that is composed only of images of important sections where the effect of the effects is high. Then, by distributing such summary video as content for the Internet, it can be used as a medium for increasing the interest of viewers in the program.

Also, according to the summary video generation device 1 according to the embodiment of the present invention, the total score calculation unit 13 calculates the element score S ₁ [i] when calculating the total score S [i] of the cut video V _C [i]. ] To S ₄ [i] The weight coefficients W _{1 to} W ₄ set for each are used.

This makes it possible in accordance with the weighting coefficients W ₁ to _W-4, reflecting the scores of the element of the weighting factors W ₁ to _W-4 to the total score S [i]. That is, whether to make a video summary strong emphasis on which elements, by setting freely relative to the weight coefficient W ₁ to _W-4, to generate a video summary of the many variations in line with the intentions of the user it can.

Further, according to the summary video generation apparatus 1 according to the embodiment of the present invention, the summary video generation unit 14 selects the scene video V _S [j] when selecting the cut video V _C [i] for generating the summary video. The number of cut images V _C [i] to be selected in [] is prevented from exceeding a predetermined value N _MAX . As a result, the cut video V _C [i] is not selected biased to a specific scene, so that a summary video biased to the specific scene is not generated. That is, a summary video in consideration of the entire scene of the program video is generated.

  The present invention has been described with reference to the embodiment. However, the present invention is not limited to the above-described embodiment, and various modifications can be made without departing from the technical idea thereof. In the embodiment, the summary video is generated with the production type as “telop”, “face recognition”, “camera work”, and “CG video quality”, but the production type is limited to these four elements. Instead of this, other elements may be used. For example, a summary video may be generated using sound volume, music, conversation, a specific object, etc. as elements of production. Moreover, you may make it produce | generate a summary image | video using arbitrary predetermined number elements among these elements of production.

As described above, the element score represents the degree of importance reflecting the effect level of the effect. Therefore, when the effect is “sound volume”, the element score is set to a value proportional to the magnitude, for example. . Further, when the production is “music”, the element score is set according to the proportion of time that the music is flowing in the cut video V _C [i]. When the production is “specific object”, the element score is set based on the area ratio of the area in which the object is detected and the probability that the object is the same as in the case of the “face recognition” described above. .

  Note that a normal computer can be used as the hardware configuration of the summary video generation apparatus 1 according to the embodiment of the present invention. The summary video generation apparatus 1 is configured by a computer including a volatile storage medium such as a CPU and a RAM, a non-volatile storage medium such as a ROM, an interface, and the like. The functions of the cut dividing unit 10, the scene generating unit 11, the element score calculating unit 12, the total score calculating unit 13, and the summary video generating unit 14 included in the summary video generation device 1 are stored in the CPU as a program describing these functions. Each is realized by executing. These programs are stored in the storage medium and read out and executed by the CPU. These programs can also be stored and distributed in a storage medium such as a magnetic disk (floppy (registered trademark) disk, hard disk, etc.), optical disk (CD-ROM, DVD, etc.), semiconductor memory, etc. You can also send and receive.

DESCRIPTION OF SYMBOLS 1 Summary video generation apparatus 10 Cut division part 11 Scene generation part 12 Element score calculation part 13 Comprehensive score calculation part 14 Summary video generation part 20 Telop area detection part 21 Face recognition processing part 22 Camera work calculation part 23 CG video likelihood calculation part 24 Telop score calculation unit 25 Face recognition score calculation unit 26 Camera work score calculation unit 27 CG video likelihood score calculation unit 30 Weight coefficient setting unit 31 Score calculation unit 40 Summary video selection unit 41 Summary video output unit

Claims (6)

In a summary video generation device that generates a summary video from a video,
A cut division unit that divides the video into a plurality of cut videos in a cut unit;
For each of the plurality of cut videos divided by the cut dividing unit, for each predetermined number of different effects, a score calculation unit that calculates a score representing the importance of the effect;
For each of the plurality of cut videos divided by the cut division unit, a total score calculation unit that calculates a total score based on the score for each effect calculated by the score calculation unit;
Based on the total score calculated by the total score calculation unit, from the plurality of cut videos, select a cut video constituting the summary video, and generate a summary video generation unit,
A summary video generation apparatus comprising: The summary video generation device according to claim 1,
The total score calculation unit
For each of the plurality of cut videos, the total score is calculated based on a score for each effect calculated by the score calculation unit and a preset weighting factor for each effect. Video generation device. In the summary video generation device according to claim 1 or 2,
Furthermore, a scene generation unit that generates a cut video of the same scene as a scene video from the plurality of cut videos divided by the cut division unit,
The summary video generation unit includes:
Summary video generation comprising: selecting a cut video constituting the summary video so that the number of the cut videos selected from the scene video generated by the scene generation unit does not exceed a predetermined value apparatus. In the summary image generation device according to any one of claims 1 to 3,
The score calculation unit
The score is calculated based on an area where an object related to the effect appears in the video, an amount of movement of the object related to the effect, or a probability that an object related to the effect appears. Summary video generator. The summary video generation device according to claim 3,
The summary video generation unit includes:
From the plurality of cut images divided by the cut dividing unit, the scene images generated by the scene generating unit in the order of importance of the effect according to the total score calculated by the total score calculating unit. A summary video selection unit that selects the cut video constituting the summary video until the total length of the summary video exceeds a predetermined value so that the number of the cut videos to be selected does not exceed a predetermined value;
A summary video output unit for generating the summary video by connecting the cut video selected by the summary video selection unit in time series; and
A summary video generation apparatus comprising:   The program for functioning a computer as the summary image | video production | generation apparatus as described in any one of Claim 1-5.

Images