JP2014187687A - Device and method for extracting highlight scene of moving image - Google Patents

Abstract

PROBLEM TO BE SOLVED: To extract a specified highlight scene including a face of a person from a moving image.SOLUTION: A highlight scene extracting device includes: a frame extracting circuit 2 which detects a face of a person from a frame image; a scene composition determining circuit 4 for generating a scene composition feature C including a plurality of feature values determined by a position and a size of the detected face in the frame image; a scene storage device 5 which previously stores a range of the plurality of feature values determined by the position and the size of the face of the person in at least one specified highlight scene as a scene composition condition D; a highlight scene extracting device 6 for determining whether the feature value of the scene composition feature C is within the range of the feature value of the scene composition condition D or not; and an index generation circuit 7 for giving an index of the highlight scene to the frame image when the feature value of the scene composition feature C is within the range of the feature value of the scene composition condition D.

Description

  The present invention relates to a moving image highlight scene extraction apparatus and method, and more particularly to an apparatus and method for extracting a specific highlight scene including a human face from a moving image.

  As a technique for extracting a human face from a moving image and creating a face index, for example, the invention of Patent Document 1 is known.

  The invention of Patent Document 1 includes a moving image storage unit, a face image tracking unit, a representative face determination unit, and a face index construction unit. For each frame image extracted from the moving image storage unit, the face image tracking unit detects a face in each frame image, and when the detected face is in the same position in the frame in a continuous frame, The images are tracked as images of the same person, and one set of frame group information is generated. Next, the representative face determination unit selects, as a representative face image, a group close to the front face from the group of frames. The face index constructing unit constructs a face index using the selected representative face image as a face image key. For example, by selecting a face image key displayed as a thumbnail image, the user can search and reproduce a scene in which a person of the face image appears.

JP 2008-252296 A

P. Viola, M. Jones, "Rapid Object Detection Using a Boosted Cascade of Simple Features", IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), Vol. 1, pp. 511-51, 2001

  The moving image includes various highlight scenes according to the content. For example, in a scene where a drama is exciting, the faces of two people may be displayed large, and in an important scene or an interview such as sports, one person may be displayed large.

  In the invention of Patent Document 1, scenes that can be selected from a moving image by a user are limited to those specified by face image keys. Therefore, in the invention of Patent Document 1, the user can search and reproduce a scene in which a person with a face image key appears, but it is difficult to search for and reproduce a specific highlight scene. It was.

  An object of the present invention is to solve the above-described problems and provide a moving image highlight scene extraction apparatus and method for extracting a specific highlight scene including a human face from a moving image.

A highlight scene extraction device for moving images according to an aspect of the present invention includes:
Means for detecting a human face from a frame of a moving image;
Means for generating a scene composition feature including a plurality of feature values determined by the position and size of the detected face in the frame;
Storage means for storing in advance a plurality of feature value ranges determined by the position and size of a person's face in at least one specific highlight scene as scene composition conditions;
First determination means for determining whether or not a feature value of the scene composition feature is within a range of feature values of the scene composition condition;
Indexing means for assigning an index of a highlight scene to the frame when the feature value of the scene composition feature is within the range of the feature value of the scene composition condition is provided.

  According to the present invention, a specific highlight scene including a human face can be extracted from a moving image.

It is a block diagram which shows the structure of the highlight scene extraction apparatus which concerns on Embodiment 1 of this invention. It is a flowchart which shows the index production | generation process performed by the highlight scene extraction apparatus of FIG. FIG. 2 is a diagram illustrating an example in which a face of a person P0 is detected from a frame image 100 by the face detection circuit 3 of FIG. 2 is a diagram illustrating an example in which faces of a plurality of persons P0 and P1 are detected from a frame image 101 by the face detection circuit 3 of FIG. 6 is a table showing a scene composition feature C of a frame image including a plurality of persons P0 to PN determined by the scene composition determination circuit 4 of FIG. It is a figure which shows the example of the frame image 110 containing the highlight scene of a drama. It is a figure which shows the attention area 111 and the face Fi of the attention person Pi in the frame image 110 of FIG. It is a figure which shows the scene composition characteristic C in the frame image 110 of FIG. It is a block diagram which shows the structure of the highlight scene extraction apparatus which concerns on Embodiment 2 of this invention. 10 is a flowchart showing a scene selection process S5A executed by the moving image genre determination circuit 11, the scene composition selection circuit 12, and the face tracking circuit 13 of FIG. It is a figure which shows the example of the frame image 120 containing an interview scene. It is a figure which shows the example of the scene composition characteristic C of the moving image containing an interview scene. It is a block diagram which shows the structure of the highlight scene extraction apparatus which concerns on Embodiment 3 of this invention. It is a flowchart which shows the index production | generation process performed by the highlight scene extraction apparatus of FIG.

  Embodiments of the present invention will be described below with reference to the drawings. Throughout the drawings, similar components are designated by the same reference numerals.

Embodiment 1 FIG.
FIG. 1 is a block diagram showing a configuration of a highlight scene extraction apparatus according to Embodiment 1 of the present invention. 1 includes a moving image input circuit 1, a frame extraction circuit 2, a face detection circuit 3, a scene composition determination circuit 4, a scene storage device 5, a highlight scene determination circuit 6, an index generation circuit 7, and A moving image storage device 8 is provided.

  The moving image input circuit 1 acquires a moving image from a recording medium such as a tuner, DVD or Blu-ray, or a network such as a LAN or the Internet. The moving image input circuit 1 may acquire a moving image from a moving image storage device 8 to be described later. The frame extraction circuit 2 generates a series of frame images from the input moving image. The face detection circuit 3 detects a person's face from each generated frame image. In order to detect a face from a frame image, for example, the method of Non-Patent Document 1 can be used.

FIG. 3 is a diagram showing an example of detecting the face of the person P0 from the frame image 100 by the face detection circuit 3 of FIG. FIG. 4 is a diagram illustrating an example in which faces of a plurality of persons P0 and P1 are detected from the frame image 101 by the face detection circuit 3 of FIG. The frame image 100 in FIG. 3 has a size h × w. 3 includes a person P0, and the face of the person P0 is detected as a rectangular area F0 having vertices A0 and B0. The face F0 of the person P0 is, for example, a square area, the position is specified by the center of gravity G0 = (x ₀ , y ₀ ), and the size thereof is specified by the side length l ₀ . 4 includes persons P0 and P1, and the face F1 of the person P1 is, for example, a square area, the position of which is specified by the center of gravity G1 = (x ₁ , y ₁ ), and the size thereof is It is specified by the side length l ₁ .

  The scene composition determination circuit 4 generates a scene composition feature C including a plurality of feature values determined by the position and size of the detected face in the frame image. An exciting scene or an important scene in a moving image can be defined by the spatial arrangement of characters. Therefore, the feature value of the scene composition feature C defines the spatial arrangement of the characters.

FIG. 5 is a table showing a scene composition feature C of a frame image including a plurality of persons P0 to PN determined by the scene composition determination circuit 4 of FIG. Each element C _ij (1 ≦ i, j ≦ N) of the scene composition feature C is defined as follows for an arbitrary pair of persons Pi and Pj.

[Equation 1]
C _ij = (x _i , y _i , h _i , s _ij , d _ij )
[Equation 2]
h _i = l _i / h
[Equation 3]
s _ij = l _j / l _i
[Equation 4]
d _ij = sqrt ((x _i -x _j ) ² + (y _i -y _j ) ² ) / h

Here, x _i and y _i indicate the position (center of gravity) of the face of the person Pi. h _i is the size of the face of the person Pi normalized by the size h of the frame image. s _ij is a ratio of the face sizes of the persons Pi and Pj. d _ij is a distance between the faces of the persons Pi and Pj normalized by the size h of the frame image. In other words, each element C _ij of the scene composition feature C indicates the relationship between the feature value (position x _i , y _i and size h _i ) that identifies the face of the person Pi itself and the face of another person Pj. And feature values (size ratio s _ij and distance d _ij ). Note that a general frame image is horizontally long (w> h), and therefore the length h of the short side is used in Equations 2 and 4.

  The scene storage device 5 stores in advance as a scene composition condition D a range of a plurality of feature values determined by the position and size of a person's face in at least one specific highlight scene. In other words, the scene composition condition D indicates a predetermined range of feature values of a scene composition feature C that is considered to be possessed by the highlight scene when the frame image is a specific highlight scene.

  An exemplary scene composition condition D is shown below.

[Equation 5]
_{r 0/3 ≦ x i /} h ≦ r 0 × 2/3
[Equation 6]
1/3 ≦ y _i / h ≦ 2/3
[Equation 7]
1/2 ≦ h _i ≦ 1
[Equation 8]
0.8 ≦ s _ij ≦ 1.2
[Equation 9]
h _i × 1.5 ≦ d _ij ≦ h _i × 2.5

Here, the coefficient r ₀ = w / h. The scene composition conditions D in Equations 5 to 9 indicate cases where the moving image is a drama. In the case of a drama, there are a hero and an important person related to the hero, and the hero and the important person are often displayed in an important scene in the development of the scenario. The scene composition feature C and scene composition condition D of such a scene will be described below with reference to FIGS.

FIG. 6 is a diagram illustrating an example of a frame image 110 including a drama highlight scene. FIG. 7 is a diagram showing the attention area 111 and the face Fi of the attention person Pi in the frame image 110 of FIG. FIG. 8 is a diagram showing a scene composition feature C in the frame image 110 of FIG. The frame image 110 in FIG. 6 includes a person Pi that is a person of interest (the main character) and a person Pj that is a peripheral person. First, a person Pi having a face Fi that is displayed in a large area of attention 111 in the center of the frame image 110 is determined as a person of interest (FIG. 7), and a person Pi near the person Pi that is the person of interest Similarly, a person Pj having a face Fj displayed large is determined as a peripheral person (FIG. 8). The attention person and the peripheral person are persons whose face positions (x _i , y _i ; x _j , y _j ) and sizes (h _i ; h _j ) satisfy Expressions 5 to 7. Furthermore, when the frame image 110 further satisfies the expressions 8 and 9, it is determined that the frame image 110 is a drama highlight scene. In addition, the range of the feature value illustrated in Formula 5 to Formula 9 is not a fixed value, and can be arbitrarily set at the time of design.

  The scene composition condition D stored in the scene storage device 5 is not limited to the range of feature values exemplified in Equations 5 to 9, and is considered to be included in other highlight scenes other than the drama. A range of feature values of the scene composition feature C may be stored.

  The highlight scene determination circuit 6 determines whether the feature value of the scene composition feature C generated by the scene composition determination circuit 4 is within the range of the feature value of the scene composition condition D stored in the scene storage device 5. judge. Next, when the feature value of the scene composition feature C is within the range of the feature value of the scene composition condition D, the index generation circuit 7 assigns the highlight scene index to the frame image.

  The determination by the highlight scene determination circuit 6 will be described with the following code.

[Equation 10]
for i: = 1 to N do
for j: = 1 to N do
if i ≠ j then
val: = calc_corr (C _ij , D)
if val == true then
make_index (i, j, D)
end
end

According to Equation 10, the function calc_corr (C _ij , D) is executed for all elements of i ≠ j among the elements C _ij (1 ≦ i, j ≦ N) of the scene composition feature C of a certain frame image. When the function calc_corr (C _ij , D) returns a value of true (true), the function make_index (i, j, D) is executed. The function calc_corr (C _ij , D) determines whether or not the feature value of the scene composition feature C of a certain frame image is within the range of the feature values of the scene composition condition D of Formulas 5 to 9, and when YES Returns the value of true (true), and returns the value of false (false) when NO. Here, when the function calc_corr (C _ij , D) returns the value of true, the index generation circuit 7 executes the function make_index (i, j, D) to add the index of the highlight scene to the frame image. To do. Specifically, the index generation circuit 7 stores the time stamp information of the frame image in the moving image storage device 8 to indicate that the highlight image is included in the frame image.

  The moving image storage device 8 stores the index of the highlight scene and the moving image acquired by the moving image input circuit 1. The user can search for a highlight scene in the moving image by using the index of the highlight scene when the moving image is reproduced. A storage device that stores the index of the highlight scene and a storage device that stores the moving image acquired by the moving image input circuit 1 may be provided separately.

  FIG. 2 is a flowchart showing an index generation process executed by the highlight scene extraction apparatus of FIG. In step S1 of FIG. 2, the moving image input circuit 1 acquires a moving image. In step S2, the frame extraction circuit 2 generates a frame image from the acquired moving image. In step S3, the face detection circuit 3 detects a face from the generated frame image. In step S4, the scene composition determination circuit 4 generates a scene composition feature using the detected face information. In the scene selection process in step S5, the highlight scene determination circuit 6 reads the scene composition condition from the scene storage device 5. In step S6, the highlight scene determination circuit 6 determines whether the feature value of the scene composition feature is within a predetermined range of the feature value of the scene composition condition. If YES, the process proceeds to step S7. If NO, the process proceeds to step S8. In step S <b> 7, the index generation circuit 7 assigns an index of the highlight scene to the frame and stores the index in the moving image storage device 8. In step S8, the scene composition determination circuit 4 generates the next frame image from the acquired moving image, and returns to step S3.

  As described above, the highlight scene extraction apparatus according to Embodiment 1 of the present invention can extract a specific highlight scene including a human face from a moving image.

  In the present invention, attention is paid to the fact that the intention of the photographer of the moving image (whether it is a highlight scene) appears as the arrangement of persons in the frame image. In the present invention, a scene composition feature including a plurality of feature values determined by the position and size of the detected face in the frame image is generated as the arrangement of the person. According to the present invention, it is determined whether or not the feature value of the scene composition feature generated from the frame image is within the range of the feature value of the scene composition condition stored in advance, and a specific high level is selected from the moving image. Only light scenes can be searched and played.

  In the conventional technique, attention is paid only to whether or not a face appears in a moving image, a face is detected from each frame image, and a search face index is created for a portion where the face appears. On the other hand, in the first embodiment of the present invention, a specific highlight scene in which a certain person appears can be extracted by using the scene composition feature.

Embodiment 2. FIG.
In general, a moving image is associated with a specific genre and may include different highlight scenes depending on the genre. In the second embodiment of the present invention, different scene composition conditions are selected according to the genre of moving images.

  FIG. 9 is a block diagram showing a configuration of a highlight scene extraction apparatus according to Embodiment 2 of the present invention. The highlight scene extraction device of FIG. 9 includes a moving image genre determination circuit 11, a scene composition selection circuit 12, and a face tracking circuit 13 in addition to the components of the highlight scene extraction device of FIG. 9 includes an index generation circuit 7A that operates in accordance with a signal output from the face tracking circuit 13, instead of the index generation circuit 7 of FIG. The scene storage device 5 stores in advance as a scene composition condition a range of a plurality of feature values determined by the position and size of a person's face in a highlight scene of at least one specific genre.

  The moving image genre determination circuit 11 acquires genre information (drama, news, sports, etc.) indicating the genre of the moving image from, for example, EPG (electronic program guide) information in a broadcast wave. Next, the moving image genre determination circuit 11 determines whether or not the moving image is a genre including an interview scene. When the moving image is a genre including an interview scene (such as news or sports), the scene composition selection circuit 12 selects a scene composition condition corresponding to a highlight scene of the same genre as the genre from the scene storage device 5 and selects The face tracking circuit 13 tracks the detected face in the frame image over a certain period of time. On the other hand, when the moving image is a genre that does not include an interview scene (such as a drama), the scene composition selection circuit 12 selects a scene composition condition corresponding to a highlight scene of the same genre as the genre from the scene storage device 5. The face tracking circuit 13 does not perform face tracking.

  Here, with reference to FIG. 11 and FIG. 12, face tracking by the face tracking circuit 13 will be described.

FIG. 11 is a diagram illustrating an example of a frame image 120 including an interview scene. FIG. 12 is a diagram illustrating an example of a scene composition feature of a moving image including an interview scene. The moving image input circuit 1, the frame extraction circuit 2, and the face detection circuit 3 operate in the same manner as in the first embodiment. When the face F10 of the person P10 included in the frame image 120 is detected at time t ₀ in FIG. 12, the face tracking circuit 13, each of the plurality of frame images temporally continuous from the frame image at time t ₀ It is determined whether or not the amount of change in the position or size of the face detected in the adjacent frame image is smaller than a predetermined threshold value. In other words, the face tracking circuit 13 satisfies whether the following expression is satisfied with respect to the threshold value th _d of the change amount of the face position or the threshold value th _h of the change amount of the face size. Determine whether.

[Equation 11]
th _d ≦ sqrt ((x (n + 1) −x (n)) ² + (y (n + 1) −y (n)) ² )
[Equation 12]
th _h ≦ abs | h (n + 1) −h (n) |

  Here, the integer n indicates the discretized time t.

In the example of FIG. 12, over the time period from time t ₀ to time t ₁ , the frame image 120 includes a substantially constant face F10, and therefore neither of the equations 11 and 12 holds. In the time period from time t ₀ to time t ₁ , frame images in which the amount of change in the position or size of the face is smaller than the threshold th _d or th _h are continuous, and this time period is referred to as “tracking time length ta”. Call.

The face tracking circuit 13 determines whether or not the tracking time length ta exceeds a predetermined threshold th ₁ when either of the expressions 11 and 12 is satisfied. When tracking the time length ta exceeds the threshold th _1, considered frame image 120 includes interview scene (i.e., scene same person's face in the center of the frame image is displayed larger over time) . At this time, the face tracking circuit 13 instructs the scene composition selection circuit 12 to select a scene composition condition corresponding to the interview scene from the scene storage device 5 and send it to the highlight scene determination circuit 6, and further, the face tracking circuit 13 notifies the start time _{t 0} of the tracking time length ta in the index generating circuit 7A. Meanwhile, when the track length of time ta has not exceeded the threshold th _1, the frame image 120 is considered to contain no interview scene. At this time, the face tracking circuit 13 selects a scene composition condition corresponding to the genre of the moving image determined by the moving image genre determination circuit 11 from the scene storage device 5 and sends it to the highlight scene determination circuit 6. The selection circuit 12 is instructed.

  Based on the scene composition condition sent from the scene composition selection circuit 12, the highlight scene determination circuit 6 determines that the feature value of the scene composition feature is within the range of the feature value of the scene composition condition, as in the first embodiment. It is determined whether or not.

When the tracking time length ta exceeds the threshold th ₁ (that is, when the tracking time length ta is input from the face tracking circuit 13), the index generation circuit 7A has a feature value of the scene composition feature. when in the range of feature values of the scene composition condition is indexed highlight scene frame image of the start time t ₀ of the track duration ta. When the tracking time length ta is equal to or less than the threshold th ₁ (that is, when the tracking time length ta is not input from the face tracking circuit 13), the feature value of the scene composition feature is the feature of the scene composition condition. When it is within the value range, the index of the highlight scene is given to the current frame image.

FIG. 10 is a flowchart showing a scene selection process S5A executed by the moving image genre determination circuit 11, the scene composition selection circuit 12, and the face tracking circuit 13 of FIG. The scene selection process S5A in FIG. 10 is executed instead of the scene selection process S5 in FIG. In step S11 of FIG. 10, the moving image genre determination circuit 11 acquires genre information of moving images. In step S12, the moving image genre determination circuit 11 determines whether or not the moving image is a genre including an interview scene. If YES, the process proceeds to step S13. If NO, the process proceeds to step S16. In step S13, the face tracking circuit 13 tracks the detected face in the frame image for a predetermined time. In step S14, the face tracking circuit 13 determines whether track duration ta of the facial image is greater than the threshold th _1, when YES, the process proceeds to step S15, and if NO then the process proceeds to step S16 . In step S15, the scene composition selection circuit 12 sends the scene composition condition corresponding to the interview scene to the highlight scene determination circuit 6, and proceeds to step S6 in FIG. In step S16, the scene composition condition corresponding to the genre of the moving image determined by the moving image genre determination circuit 11 is sent to the highlight scene determination circuit 6, and the process proceeds to step S6 in FIG.

  As described above, according to the highlight scene extraction apparatus according to Embodiment 2 of the present invention, different scene composition conditions can be selected according to the genre of moving images.

  The scene composition of the highlight scene differs depending on the genre of the moving image. Therefore, genre information indicating the genre of the moving image is acquired, and a specific highlight scene is extracted from the moving image of the specific genre by using a scene composition condition corresponding to a highlight scene of the same genre as the moving image genre. can do.

Embodiment 3 FIG.
In general, since video information and audio information are closely related to each other in a moving image, when determining a highlight scene, a scene composition feature and an audio feature having high relevance to the scene composition feature are used. As a result, the highlight scene can be extracted more accurately.

  FIG. 13 is a block diagram showing a configuration of a highlight scene extraction apparatus according to Embodiment 3 of the present invention. The highlight scene extraction device of FIG. 13 includes a scene storage device 5B and a highlight scene determination circuit 6B instead of the scene storage device 5 and the highlight scene determination circuit 6 of FIG. 1, and further includes an audio feature generation circuit 21. . The audio feature generation circuit 21 generates audio features by separating audio information from the moving image, and sends the audio features to the highlight scene determination circuit 6B. In addition to the range of feature values of the scene composition condition, the scene storage device 5B stores in advance a plurality of feature value ranges related to audio information before and after at least one specific highlight scene as audio conditions. In other words, when the frame image is a specific highlight scene, the audio condition indicates a range of feature values of audio features that are considered to be included in audio information before and after the highlight scene.

  FIG. 14 is a flowchart showing an index generation process executed by the highlight scene extraction apparatus of FIG. Steps S1 to S4 in FIG. 14 are the same as steps S1 to S4 in FIG. In step S21, the audio feature generation circuit 21 separates audio information from the moving image (step S1) acquired by the moving image input circuit 1, and in step S22, generates an audio feature from the audio information to determine a highlight scene. Send to circuit 6B. In the scene selection process in step S5B, the highlight scene determination circuit 6B reads the scene composition condition and the audio condition from the scene storage device 5B. In step S6B, the highlight scene determination circuit 6B determines whether or not the feature values of the scene composition feature and the audio feature are within the predetermined ranges of the feature values of the scene composition condition and the audio condition, If YES, the process proceeds to step S7. If NO, the process proceeds to step S8. Steps S7 to S8 in FIG. 14 are the same as steps S7 to S8 in FIG.

  As the audio feature, a feature highly relevant to the scene composition feature to be extracted can be used. For example, in a highlight scene, the volume (sound volume) | V | is often large, so the volume | V | is used as an audio feature, and the volume threshold value | Vth | is used as an audio condition. Also good. In this case, in step S6B, the highlight scene determination circuit 6B determines whether or not the volume | V | is larger than the threshold value | Vth |. Further, since the frequency of the sound becomes high in a case where the audience is excited in the highlight scene, the frequency of the sound | Fv | is used as the audio feature, and the frequency threshold value | Fth | is used as the audio condition. Also good. The highlight scene determination circuit 6B determines whether the frequency | Fv | is higher than the threshold value | Fth |.

  As described above, according to the highlight scene extraction apparatus according to Embodiment 3 of the present invention, a person can be identified from a moving image by combining a scene composition feature from video information and an audio feature from audio information. It is possible to extract a specific highlight scene including the face of the user more accurately.

  The moving image highlight scene extracting apparatus and method of the present invention can be applied to a hard disk recorder or the like for recording moving images.

DESCRIPTION OF SYMBOLS 1 Moving image input circuit, 2 Frame extraction circuit, 3 Face detection circuit, 4 Scene composition determination circuit, 5, 5B Scene storage device, 6, 6B Highlight scene determination circuit, 7, 7A Index generation circuit, 8 Moving image storage device , 11 Moving image genre determination circuit, 12 Scene composition selection circuit, 13 Face tracking circuit, 100, 101, 110, 120 Frame image, 111 Region of interest, A0, B0 vertex, F0, F1, Fi, Fj, F10 Face, G0 , G1, Gi, Gj, G10 Center of gravity, P0, P1, Pi, Pj, P10 Person.

Claims (10)

Means for detecting a human face from a frame of a moving image;
Means for generating a scene composition feature including a plurality of feature values determined by the position and size of the detected face in the frame;
Storage means for storing in advance a plurality of feature value ranges determined by the position and size of a person's face in at least one specific highlight scene as scene composition conditions;
First determination means for determining whether or not a feature value of the scene composition feature is within a range of feature values of the scene composition condition;
A highlight scene of a moving image comprising: index means for assigning an index of a highlight scene to the frame when the feature value of the scene composition feature is within the range of the feature value of the scene composition condition Extraction device. The above video is associated with a specific genre,
The storage means stores in advance as a scene composition condition a plurality of feature value ranges determined by the position and size of a person's face in a highlight scene of at least one specific genre,
The moving image highlight scene extraction device is:
Means for acquiring genre information indicating the genre of the moving image;
Means for selecting a scene composition condition corresponding to a highlight scene of the same genre as the moving image genre from the storage means;
2. The moving image according to claim 1, wherein the first determination unit determines whether or not a feature value of the scene composition feature is within a range of feature values of the selected scene composition condition. Highlight scene extraction device. The feature value of the scene composition feature includes, when the frame includes a plurality of persons, a ratio of face sizes of the plurality of persons, and a distance between the faces of the plurality of persons,
3. The moving image high image according to claim 1, wherein the scene composition condition includes a range of a ratio of face sizes of a plurality of persons and a range of a distance between the faces of the plurality of persons. Light scene extraction device. The moving image highlight scene extraction device is:
Determining whether or not the amount of change in the position or size of the face detected in each adjacent frame among a plurality of frames temporally continuous from the first frame is smaller than the first threshold, A second determination means for determining whether or not a time length of continuous frames having a change amount smaller than the first threshold exceeds a second threshold;
The index means is configured such that when the time length for which the amount of change is smaller than the first threshold exceeds a second threshold and the feature value of the scene composition feature is the scene composition The highlight of a moving image according to any one of claims 1 to 3, wherein an index of a highlight scene is assigned to the first frame when it is within a range of feature values of a condition. Scene extraction device. The moving image includes video information and audio information,
The highlight scene extraction device further includes means for generating an audio feature including a plurality of feature values related to the audio information,
The storage means stores in advance as a sound condition a range of a plurality of feature values related to sound information before and after at least one specific highlight scene,
The first determination means determines whether each feature value of the scene composition feature and the audio feature is within a range of each feature value of the scene composition condition and the audio condition,
The index means assigns an index of a highlight scene to the frame when the feature values of the scene composition feature and the audio feature are within the range of the feature values of the scene composition condition and the audio condition. The highlight scene extraction apparatus for moving images according to any one of claims 1 to 4. Detecting a human face from a frame of a moving image;
Generating a scene composition feature including a plurality of feature values determined by the position and size of the detected face in the frame;
Pre-stored in the storage means as a scene composition condition a range of a plurality of feature values determined by the position and size of a person's face in at least one specific highlight scene;
Determining whether the feature value of the scene composition feature is within the range of the feature value of the scene composition condition;
A highlight scene extraction method for moving images, comprising: a step of assigning an index of a highlight scene to the frame when a feature value of the scene composition feature is within a range of feature values of the scene composition condition . The above video is associated with a specific genre,
The storage means stores in advance as a scene composition condition a plurality of feature value ranges determined by the position and size of a person's face in a highlight scene of at least one specific genre,
The highlight scene extraction method of the above moving image is as follows:
Acquiring genre information indicating the genre of the moving image;
Selecting from the storage means a scene composition condition corresponding to a highlight scene of the same genre as the moving image genre;
7. The highlight scene extraction of a moving image according to claim 6, further comprising a step of determining whether or not the feature value of the scene composition feature is within the range of the feature value of the selected scene composition condition. Method. The feature value of the scene composition feature includes, when the frame includes a plurality of persons, a ratio of face sizes of the plurality of persons, and a distance between the faces of the plurality of persons,
8. The moving image high level according to claim 6, wherein the scene composition condition includes a range of a ratio of face sizes of a plurality of persons and a range of a distance between the faces of the plurality of persons. Light scene extraction method. The highlight scene extraction method of the above moving image is as follows:
Determining whether or not the amount of change in the position or size of the face detected in each adjacent frame among a plurality of frames temporally continuous from the first frame is smaller than the first threshold, Determining whether or not a time length of continuous frames having a change amount smaller than the first threshold exceeds a second threshold;
When the length of time for which the amount of change is smaller than the first threshold exceeds the second threshold, and the feature value of the scene composition feature is the feature value of the scene composition condition The moving image highlight scene according to claim 6, further comprising a step of assigning an index of a highlight scene to the first frame when it is within the range. Extraction method. The moving image includes video information and audio information,
The highlight scene extraction method of the above moving image is as follows:
Generating a voice feature including a plurality of feature values related to the voice information;
Pre-stored as a sound condition a range of feature values related to sound information before and after at least one specific highlight scene;
Determining whether each feature value of the scene composition feature and the audio feature is within a range of each feature value of the scene composition condition and the audio condition;
Adding a highlight scene index to the frame when the feature values of the scene composition feature and the audio feature are within the range of the feature values of the scene composition condition and the audio condition. The highlight scene extraction method for moving images according to any one of claims 6 to 9.

Images