JP2006331271A - Representative image extraction apparatus and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To highly accurately extract a representative image from moving images. <P>SOLUTION: A representative image extraction apparatus for extracting the representative image from an inputted moving image comprises: a representative image determination means for determining a representative candidate image from a prescribed section included in the moving image; a face image recognition means for recognizing a face image by collating the representative candidate image with a plurality of face templates having at least a preliminarily set face image and personal identification information corresponding to the face image; and a representative image decision means for searching a representative image on the basis of a collated result corresponding to the personal identification information obtained by the face image recognition means and then outputting the searched representative image. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to a representative image extraction device and a representative image extraction program, and more particularly to a representative image extraction device and a representative image extraction program for extracting a representative image from a moving image with high accuracy.

  In recent years, various broadcasting networks such as BS digital broadcasting and cable TV have become widespread in addition to normal terrestrial television broadcasting, and the number of programs that viewers can view increases and the number of programs that they want to view also increases. There is a tendency. Thus, the viewer needs a lot of time to view all the programs.

  Therefore, there is a service that extracts representative images in order to make viewers understand the general contents of programs to be broadcast, generates summary programs with reduced time using the extracted images, and provides them to viewers. is there.

  In addition, as one method for extracting such a representative screen, face recognition, voice recognition, motion recognition, facial expression recognition, object recognition in a moving image, and a technique combining them are effective methods. There is an image recognition method (for example, see Non-Patent Document 1).

The technique disclosed in Non-Patent Document 1 determines a representative image from given candidate images using a weighted linear sum of outputs from a plurality of representative face discriminators. Each representative face determiner uses a different feature amount designed to determine a representative face. As specific examples of feature amounts used for the determiner, a set of five types of feature amounts including face orientation, size, brightness, movement amount, and eye detection result has been proposed.
Kosuke Hirasawa et al., “Development of Best Shot Face Image Recording System”, The Institute of Electronics, Information and Communication Engineers, PRMU 2004-103, HIP 2004-43 (2004-11)

  By the way, in the method using a plurality of determiners for determining the representative image as described above, the result of integrating the outputs from the plurality of individually optimized determiners is used. However, the evaluation value (feature value) of each determiner is set by the system designer based on subjective judgment on the nature of the target image (face image) that is supposed to represent the feature of the representative image. For this reason, the selected representative image is not necessarily an image advantageous for specifying an individual in the face image recognition process.

  Even if the characteristics (features) of the representative image assumed by the system designer reflect the characteristics of the face image that is advantageous for individual identification, internal variables (parameters) such as threshold values are set. Therefore, it is necessary to make adjustments in advance for each determination device, and the convenience in operating the system is low.

  Furthermore, there is no standard for the number of discriminators used for face image recognition processing, and in general, errors with respect to learning data prepared for parameter adjustment improve as the number of discriminators (features) increases. Is done. However, if the number of discriminators is increased in order to improve the discrimination accuracy for learning data, there is an inherent risk that the discrimination ability (robustness) for unknown data will be reduced due to over adaptation to learning data. Become.

  SUMMARY An advantage of some aspects of the invention is that it provides a representative image extraction device and a representative image extraction program for extracting a representative image from a moving image with high accuracy.

  In order to solve the above problems, the present invention employs means for solving the problems having the following characteristics.

  The invention described in claim 1 is a representative image extracting device that extracts a representative image from an input moving image, a representative image determining unit that determines a representative candidate image from a predetermined section included in the moving image; By comparing the representative candidate image with a preset face image and a plurality of face templates having at least personal identification information corresponding to the face image, face image recognition means for recognizing the face image, and the face image recognition means And a representative image determining unit that searches for a representative image based on a matching result corresponding to the obtained personal identification information and outputs the searched representative image.

  According to the first aspect of the present invention, a representative image can be extracted from a moving image with high accuracy. Thereby, for example, information on the contents of a huge amount of video (moving images), particularly information on a projected person can be summarized using a representative still image. Therefore, intuitive and simple browsing of moving images can be realized.

  According to a second aspect of the present invention, the representative image determination unit searches for a representative image based on specific personal identification information based on a predetermined condition from a result of collation with the representative candidate image by the plurality of face templates. And a representative image output means for extracting a representative image from the representative candidate image based on the image frame obtained by the representative image searching means or the appearance time information of the image frame. To do.

  According to the second aspect of the present invention, the configuration of the entire apparatus can be simplified and the representative image can be extracted with high accuracy by using the collation result in the face recognition for the processing for determining the representative image. Can do. In addition, robustness with respect to fluctuating unknown input data can be improved.

  In the invention described in claim 3, the representative image search means, for each image frame included in the representative candidate image, a matching score in a specific face template among the plurality of face templates, and the The image frame or the appearance time information is searched based on a difference in score value with a matching score in a face image other than a specific face template.

  According to invention of Claim 3, a highly accurate representative image can be extracted rapidly based on the difference of a score value.

  The invention described in claim 4 is characterized in that the representative image search means searches for the image frame or the appearance time information in which the difference in the score values is a maximum or a predetermined threshold value or more.

  According to the fourth aspect of the present invention, an optimum representative image or an optimum plurality of representative images can be extracted according to the purpose of the representative image.

  The invention described in claim 5 is characterized in that the representative image search means normalizes a sum of a plurality of matching scores obtained for each face template to a constant value.

  According to the fifth aspect of the present invention, when the normalized value is used, the representative image can be extracted only by paying attention to the maximum value of the matching score.

  According to a sixth aspect of the present invention, in a representative image extraction program for causing a computer to execute a representative image extraction process for extracting a representative image from an input moving image, the representative candidate is extracted from a predetermined section included in the moving image. A face that recognizes a face image by collating a representative image determination process for determining an image, the representative candidate image, and a plurality of face templates having at least a preset face image and personal identification information corresponding to the face image. The computer is caused to search for a representative image based on the image recognition process and a matching result corresponding to the personal identification information obtained by the face image recognition process, and to output the searched representative image.

  According to the sixth aspect of the present invention, a representative image can be extracted from a moving image with high accuracy. In addition, by generating an execution program for extracting a representative image in the present invention, a representative image extraction process can be realized at a low cost without requiring a special apparatus configuration. Also, the representative image extraction process can be easily realized by installing the program.

  According to the present invention, a representative image can be extracted from a moving image with high accuracy.

<Outline of the present invention>
The present invention uses the parameters used for the face recognition process also when determining the representative image, thereby making the evaluation standard of the representative image directly related to the superiority or inferiority of the conditions in the face recognition process. A representative image is extracted from a moving image with high accuracy. Further, by making the representative image determination process common with the internal process of face recognition, the configuration of the entire apparatus can be simplified.

<Embodiment>
In the following, a preferred embodiment of a representative image extraction apparatus and a representative image extraction program according to the present invention having the above-described features will be described in detail with reference to the drawings.

  FIG. 1 is a diagram illustrating a configuration example of a representative image extraction apparatus according to the present invention. The representative image extraction apparatus 1 shown in FIG. 1 is configured to include a representative candidate image determination unit 10, a face image recognition unit 20, and a representative image determination unit 30.

  The representative candidate image determination unit 10 includes a representative candidate section determination unit 11 and a representative candidate image storage unit 12. Further, the face image recognition means 20 is configured to include a face template storage means 21 and a face template collation means 22. Further, the representative image determination unit 30 is configured to include a representative image search unit 31 and a representative image output unit 32.

  The representative candidate section determining means 11 outputs a set of start points and end points for determining the section of the representative candidate image from the input image (moving image). Further, the representative candidate section determination unit 11 outputs a representative candidate image (a group of continuous still images from the start point to the end point) including the determined start point and end point to the representative candidate image storage unit 12.

  The representative candidate image storage unit 12 stores the continuous still images in the representative candidate section determined by the representative candidate section determination unit 11 together with time information in the moving image. The representative candidate image storage unit 12 outputs the stored representative candidate image to the face image recognition unit 20 and the representative image determination unit 30.

  The face template storage unit 21 in the face image recognition unit 20 inputs and stores a preset face template. That is, the face template storage unit 21 stores a face template including information necessary for identifying each person's face (such as a face image and personal identification information corresponding to the face image). Further, the face template storage unit 21 outputs the stored face template to the face template collation unit 22.

  The face template matching unit 22 determines a representative face template from the representative candidate images obtained by the representative candidate image determining unit 10 and a plurality of face templates obtained by the face template storage unit 21. Further, the face template matching unit 22 records history information (matching) of representative face identification numbers as personal identification information corresponding to the representative face templates and matching scores (similarities between representative candidate images and face templates) for all face templates. (Score table) is output to the representative image determination means 30.

  The representative image search means 31 in the representative image determination means 30 inputs the representative face identification number obtained from the face image recognition means 20 and the history information of the matching score, and based on these information, the matching score of the representative face template and The time information of the image frame or the image frame in the moving image in which the difference from all the matching scores of the other face templates is maximized is searched from the representative candidate section.

  The representative image search means 31 can also search for a representative image by normalizing the sum of a plurality of matching scores obtained for each of a plurality of face templates registered in the face template collation means 22, for example. . Further, when performing the process of normalizing the sum of the matching scores as described above, the representative image search unit 31 sets the time (representative image appearance time) at which the matching score of the representative face template is maximized as the representative candidate section. Search from inside. As described above, when using the normalized value, it is possible to extract the representative image only by paying attention to the maximum value of the matching score.

  The representative image search means 31 outputs the searched maximum time (representative image appearance time) to the representative image output means 32. As for the above-described search method, an example is shown in which a single still image representing the contents of a given series of moving images is searched. However, the present invention is not limited to this. With reference to the threshold value, a plurality of image frames whose difference in score values is equal to or greater than the threshold value or times corresponding to image frames may be output to the representative image output unit 32. Thereby, an optimal representative image or a plurality of optimal representative images can be extracted according to the purpose of the representative image.

  The representative image output means 32 searches the representative candidate image obtained from the representative candidate image determination means 10 for an image (representative image) corresponding to the representative image appearance time obtained from the representative image search means 31, and extracts it by the search. The processed representative image is output.

  The representative image extraction apparatus 1 described above can extract a representative image from a moving image with high accuracy. Specifically, based on the representative candidate image given by the representative candidate image determining means 10, the history of matching scores for each face template calculated by the face template matching means 22 is output to the representative image determining means 30, and the information is By using it for processing in the representative image search means 31, the entire apparatus can be simplified to the minimum necessary configuration. In addition, robustness with respect to fluctuating unknown input data can be improved. That is, by using the history of internal parameters of face image processing in variable template matching, face image recognition can be realized with high accuracy.

  Next, specific processing contents in the representative candidate image determination unit 10, the face image recognition unit 20, and the representative image determination unit 30 will be described.

<Representative candidate image determining means 10>
The representative candidate image determining means 10 estimates a section where the same person is highly likely to be reflected based on texture, color change, etc., from the input continuous still image or moving image data, and starts the section And end point. In particular, when the candidate image section may be fixed, only the start point may be output, and for the end point, a time obtained by adding a fixed length of delay time to the time of the start point may be used instead.

  The representative candidate image determining unit 10 required in the present embodiment is not particularly limited as long as it is a method that can estimate a section where the same person is shown from a given image. For example, the Japanese Patent Application filed by the present applicant for a method for estimating the start time and the end point by using the color and texture information and the information of the volume of the sound, the time when the composition of the moving image changes. The technique shown in 2004-149972 "Summary program generation device and summary program generation program" can be used.

  The above-described method generates a high-precision summary program by efficiently extracting images and sounds that make it easy to understand the contents of the program. More specifically, for example, a drama video and audio are taken into a computer, an image is extracted based on a portion (cut point) indicating the switching of the video, a line start, etc., and data including the extracted image and frame information, etc. (Image data structure) is stored. Next, the user deletes or changes the image data structure recorded on the GUI (Graphical User Interface), and extracts the image data structure corresponding to the condition set by the user.

  In this way, the representative candidate image obtained by the representative candidate section determining unit 11 is output to the representative candidate image storing unit 12. As a method other than the above, for example, the start point and the end point can be determined using the content description (metadata) of the broadcast video, and thereby the representative candidate section can be extracted efficiently and with high accuracy. it can.

<Face image recognition means 20>
The face image recognition means 20 calculates the degree of similarity (matching score) with the face image in the representative candidate image for each data (face template) related to the face image registered in advance. A specific method for calculating the matching score is not particularly limited as long as the state of the target image (face image) to be matched reflects the difficulty of personal identification. For example, a matching score is calculated from the calculated spatial frequency component and the distortion energy of a graph connecting each feature point in a feature point composed of a characteristic part of each organ of the face (end of the eye, apex of the nose, etc.). It is possible to apply a method for calculating. The above-described method is described in Japanese Patent Application No. 2003-388203 “Face Image Recognition Device and Face Image Recognition Program” filed by the present applicant.

  The above-described method takes a plurality of deformed sample images assumed for an object (face) in face image recognition, and extracts statistical information (variance / covariance matrix) of feature points plotted on the sample images. And use it. In other words, since the movement direction and movement position of each feature point are associated with each other to some extent depending on the facial expression, facial image recognition is performed by applying the movement information to the feature point of the image for which a person is desired to be identified.

  Since the image given as a sample only needs to be able to cover the main deformation patterns that the object (face) can take, the registered image object itself (for example, in the personal authentication system using a face image, the registered person itself) There is no need to shoot the deformation. In other words, images with different facial expressions given as deformation samples do not shoot images with different facial expressions in the registered image, but the relationship in which the facial deformation patterns are similar, or the facial expression of the sample image Adaptation is possible as long as the relationship includes facial expressions that a person can take.

  Specifically, only the top few local similarities (spatial frequency component similarities) calculated for each feature point are summed, and the sum is matched with each face template and the input face image. Define as score. Therefore, it is possible to reflect the lack of feature points with high local similarity due to face orientation and occlusion, that is, the state of the face related to the difficulty of personal identification in the magnitude of the score.

  If the given image is a continuous still image, the matching score is integrated for each of the plurality of face templates for all the given images, and the face template that gives the maximum value is output as the representative face template.

  The face image recognition means 20 shown in the present embodiment is not particularly limited as long as it is a method capable of calculating a matching score for each face template and outputting a template and a matching score related to the representative face image from the calculated result. Absent.

<Specific Example of Face Template Matching Unit 22>
Here, a specific example of the face template matching unit 22 in the present embodiment will be described with reference to the drawings. FIG. 2 is a diagram illustrating an example of a registered face image used for template matching. Moreover, FIG. 3 is a figure which shows an example for demonstrating the mode of face recognition.

  A large number of template images (registered face images) such as the person 40 (ID = 2) shown in FIG. Further, in the person 40 shown in FIG. 2, information indicating what kind of person this person is and the angle in the image, the feature point positional relationship, and the like are stored in association with the image.

  As shown in FIG. 2, the face template collation unit 22 specifies the person 40 shown in the image frames 0 to 20 [frame] stored by the representative candidate image storage unit 12, for example, Recognition processing is performed by matching with the template.

  When such processing is performed, as described above, for example, a maximum of nine feature points 42 such as the eyes, the corners of the eyes, and the nose vertices in the face image set in the face area 41 in the preset image. The collation is performed from the position to the face direction and the positional relationship of each point. In addition to the nine points, feature points including the end points of the mouth may be used. Further, for example, predetermined feature points (3 points, 5 points, etc.) displayed in the image out of 9 points are selected in order from the top according to the degree of matching with the face template at each feature point, The face image may be recognized based on the selected feature point.

  Here, assuming that a face image as shown in FIG. 2 is stored as the face template 21, the face template matching means 22 performs matching with each frame and creates a matching score with each frame image. Further, the face template matching unit 22 performs the same process for face images other than the image shown in FIG. 2 stored in the face template storage unit 21, and the matching score table as a result of the matching is represented by a representative face identification number ( And the personal identification information) to the representative image search means 31.

  Here, FIG. 4 is a diagram illustrating an example of the matching score table. As shown in FIG. 4, the matching score table 50 outputs a matching score for each frame for each face image (personal identification information: ID) stored in the face template storage unit 21.

  In the output data shown in FIG. 4, “frame” indicates an image frame included in the video, and specifically indicates time information corresponding to the frame. “Iteration” indicates a relative time from the start point of the representative candidate section, “track_no” indicates a number for identifying each face area, and “track-> age” indicates the length of the history of the detected face area. "Track-> cfgroup_no" indicates the serial number of the feature point constellation pattern on the representative candidate image used for collation.

  “Pose” indicates the direction of the face template (for example, the front is 0, and if it is facing right, it is +0 to +90, and if it is facing left, it is −0 to −90). “cen_x” indicates the center position (X coordinate) of the face area, “cen_y” indicates the center position (Y coordinate) of the face area, and “rad” indicates the size of the face area. Furthermore, “max_evid” indicates a matching score required in the present embodiment, and “max_evid_id” indicates a personal identification number as personal identification information.

  Note that the information included in the matching score table 50 is not limited to this. Further, the matching score table 50 does not need all of the above-described information, and may have at least personal identification information, a matching score, and appearance time information for the frame.

<Representative image determination means 30>
The representative image determination unit 30 searches for the optimum representative image by comparing the matching score for each individual identification number from the above-described matching score table shown in FIG.

  Here, FIG. 5 is a diagram illustrating an example of a result of the representative image search in the present embodiment. In the representative image determination means 30, the representative image search means 31 includes one registered face image (ID) registered as a face template and all other registered images (ID) as one method for searching for a representative image. The average of the matching scores is compared, and an image frame having the maximum difference is extracted as a representative image. Thus, by searching for a representative image based on the score value of the matching score, a highly accurate representative image can be quickly extracted.

  For example, as a result of comparing the registered face image ID = 2 shown in FIG. 2 described above with other registered face image IDs, a matching score is obtained at the 16th frame from the preset start point as shown in FIG. The difference in (matching score) is maximized. That is, in 16 frames, there is a high possibility that a person with registered face image ID = 2 is displayed.

  Therefore, the representative image search means 31 outputs information of 16 frames to the representative image output means 32 as the appearance time information of the representative image. The representative image search means 31 may directly output the image frame when there is a corresponding image frame.

  The representative image output unit 32 extracts the image of the 16th frame from the representative candidate image stored in the representative candidate image storage unit 12, and, as shown in an example of the output result of the representative image shown in FIG. The registered face image ID = 2 is output as a representative image. Thereby, automatic extraction of an image (representative image) from a moving image can be realized with high accuracy.

  The representative image search method described above is not limited to this in the present invention. In the above description, a search method using a specific registered face image and all other registered face images is shown, but among all the other registered face images, the highest rank (for example, higher rank) from the one with the highest matching score. Search may be performed using up to the fifth place.

  The above-described representative image search means 31 shows an example in which one representative image with the largest difference in scores is extracted. However, the present invention is not limited to this. For example, matching as shown in FIG. In the score comparison, a threshold value may be set in advance, and all image frames or time information whose score difference is equal to or greater than the threshold value may be extracted.

By using the representative image extraction apparatus 1 described above, a representative image can be extracted from a moving image with high accuracy. As a result, for example, information related to the contents of a vast amount of video (moving images), in particular, information related to a projected person is summarized using representative still images, thereby realizing intuitive and simple browsing of moving images. be able to.
Further, by using a parameter reflecting the face image recognition process for determination of a representative image, it is possible to realize extraction of a representative image that is robust against fluctuations in the input image.

  Here, the above-described representative image extraction apparatus 1 can extract a representative image with a dedicated apparatus configuration. For example, the representative image extraction apparatus 1 generates an execution program capable of causing a computer to execute a representative image extraction process. By installing a program in a general-purpose personal computer, workstation, or the like, the representative image extraction process in the present invention can be realized.

<Hardware configuration>
Here, a hardware configuration example of a computer capable of executing the representative image extraction processing according to the present invention will be described with reference to the drawings. FIG. 7 is a diagram showing an example of a hardware configuration capable of realizing the representative image extraction process in the present invention.

  7 includes an input device 61, an output device 62, a drive device 63, an auxiliary storage device 64, a memory device 65, a CPU (Central Processing Unit) 66 for performing various controls, and a network connection device. 67, which are connected to each other by a system bus B.

  The input device 61 has a pointing device such as a keyboard and a mouse operated by the user, and inputs various operation signals such as execution of a program from the user. The output device 62 has a display for displaying various windows and data necessary for operating the computer main body for performing processing according to the present invention, and displays the program execution progress, results, and the like by the control program of the CPU 66. can do.

  Here, in the present invention, the execution program installed in the computer main body is provided by the recording medium 68 such as a CD-ROM. The recording medium 68 on which the program is recorded can be set in the drive device 63, and the execution program included in the recording medium 68 is installed from the recording medium 68 to the auxiliary storage device 64 via the drive device 63.

  The auxiliary storage device 64 is a storage means such as a hard disk, and can store an execution program according to the present invention, a control program provided in a computer, and the like, and can perform input / output as necessary.

  The CPU 66 performs various operations and data input / output with each hardware component based on a control program such as an OS (Operating System) and an execution program read and stored by the memory device 65. Each process in the representative image extraction can be realized by controlling the process. Various data necessary during the execution of the program can be acquired from the auxiliary storage device 64 and stored.

  The network connection device 67 obtains an execution program from another terminal connected to the communication network by connecting to a communication network or the like, or an execution result obtained by executing the program or an execution in the present invention The program itself can be provided to other terminals.

  With the hardware configuration as described above, the representative image extraction process can be realized at a low cost without requiring a special device configuration. Also, the representative image extraction process can be easily realized by installing the program.

<Representative image extraction process>
Next, a processing procedure in the execution program will be described using a flowchart. FIG. 8 is a flowchart showing an example of a representative image extraction processing procedure in the present invention. First, a moving image for extracting a representative image is input (S01), and a representative candidate section for extracting a representative candidate image from the input moving image is determined (S02). Note that, as described above, the representative candidate section outputs a set of start points and end points of the section of the representative candidate image based on the start and end of audio from the moving image, scene switching, and the like. In S02, a representative candidate image in which continuous still images in the representative candidate section are associated with time information in the moving image is extracted.

  Next, the representative candidate image (continuous still image group) acquired by the process of S02 is collated with the face image of the face template registered in advance (S03). In the process of S03, a representative face template is determined from the representative face candidate image and the face template as described above, and personal identification information indicating the representative face template and a matching score table are generated.

  Next, based on the personal identification information and the matching score table, an image frame or time that maximizes the difference between the matching score of the representative face template and the matching score of other face templates or a predetermined face template is searched. (S04), an image corresponding to the image frame or time is output as a representative image (S05).

  Here, it is determined whether or not an unprocessed moving image exists (S06). If an unprocessed moving image exists (YES in S06), the process returns to S01 to input an unprocessed moving image and subsequent processing. I do. If there is no unprocessed moving image (NO in S06), the representative image extraction process ends.

  As described above, a representative image can be extracted from a moving image with high accuracy by the representative image extraction process. In addition, by generating an execution program for extracting a representative image in the present invention, a representative image extraction process can be realized at a low cost without requiring a special apparatus configuration. Also, the representative image extraction process can be easily realized by installing the program.

  As described above, according to the present invention, a representative image can be extracted from a moving image with high accuracy. For example, it is possible to summarize information related to the contents of a vast amount of video (moving images), particularly information related to a projected person, using representative still images. Therefore, intuitive and simple browsing of moving images can be realized.

  Further, by using the collation result in face recognition for the processing for representative image determination, the configuration of the entire apparatus can be simplified, and the representative image can be extracted with high accuracy. In addition, robustness with respect to fluctuating unknown input data can be improved.

  In the above-described embodiment, an example in which a face image is extracted as a representative image extraction example has been described. However, the present invention is not limited to this. For example, in the case of a sign language video, a finger endpoint is a representative feature point. Images can be extracted and representative images other than faces and human images can be extracted with high accuracy by applying the above-mentioned processing with a predetermined portion as a feature point for animals, plants, buildings, etc. it can.

  Although the preferred embodiment of the present invention has been described in detail above, the present invention is not limited to the specific embodiment, and various modifications, within the scope of the gist of the present invention described in the claims, It can be changed.

It is a figure which shows one structural example of the representative image extraction apparatus in this invention. It is a figure which shows an example of the registration face image used for template collation. It is a figure which shows an example for demonstrating the mode of face recognition. It is a figure which shows an example of a matching score table. It is a figure which shows an example of the result of the representative image search in this embodiment. It is a figure which shows an example of the output result of a representative image. It is a figure which shows an example of the hardware constitutions which can implement | achieve the representative image extraction process in this invention. It is a flowchart which shows an example of the representative image extraction process sequence in this invention.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Representative image extraction apparatus 10 Representative candidate image determination means 11 Representative candidate area determination means 12 Representative candidate image storage means 20 Face image recognition means 21 Face template storage means 22 Face template collation means 30 Representative image determination means 31 Representative image search means 32 Representative Image output means 40 Person 41 Face area 42 Feature point 50 Matching score table 61 Input device 62 Output device 63 Drive device 64 Auxiliary storage device 65 Memory device 66 CPU
67 Network connection device 68 Recording medium

Claims (6)

In a representative image extraction device that extracts a representative image from an input moving image,
Representative image determining means for determining a representative candidate image from a predetermined section included in the moving image;
Face image recognition means for recognizing a face image by comparing the representative candidate image with a plurality of face templates having at least a preset face image and personal identification information corresponding to the face image;
A representative image extracting apparatus comprising: a representative image determining unit that searches for a representative image based on a matching result corresponding to the personal identification information obtained by the face image recognition unit and outputs the searched representative image. The representative image determination means includes
Representative image search means for searching for a representative image based on specific personal identification information based on a predetermined condition from a matching result with the representative candidate image by the plurality of face templates;
The representative image output means for extracting a representative image from the representative candidate image based on the image frame obtained by the representative image search means or the appearance time information of the image frame. Representative image extraction device. The representative image search means includes
For each image frame included in the representative candidate image, a difference in score value between a matching score in a specific face template and a matching score in a face image other than the specific face template among the plurality of face templates. The representative image extraction apparatus according to claim 2, wherein the image frame or the appearance time information is searched based on the information. The representative image search means includes
4. The representative image extraction device according to claim 3, wherein the image frame or the appearance time information whose difference in score values is maximum or equal to or greater than a preset threshold is searched. The representative image search means includes
The representative image extraction apparatus according to claim 3 or 4, wherein a sum of a plurality of matching scores obtained for each face template is normalized to a constant value. In a representative image extraction program for causing a computer to execute a representative image extraction process for extracting a representative image from an input moving image,
Representative image determination processing for determining a representative candidate image from a predetermined section included in the moving image;
A face image recognition process for recognizing a face image by comparing the representative candidate image with a face image set in advance and a plurality of face templates having at least personal identification information corresponding to the face image;
A representative image extraction program for causing a computer to execute a representative image determination process of searching for a representative image based on a matching result corresponding to the personal identification information obtained by the face image recognition process and outputting the searched representative image.