JP2008181303A - Image management program and image management device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To effectively utilize the recognition result of the face of a person by storing images into folders. <P>SOLUTION: A face recognition part 103a recognizes the face of a person included in an object image. An image classification part 103b classifies object images on the basis of the result of the face recognition by the face recognition part 103a, and stores the object images in folders prepared in an HDD 104 according to the classification result. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to an image management program and an image management apparatus for managing images taken by a camera.

  The following image distribution servers are known. This image distribution server identifies a person in the image by recognizing the face of the person in the image, and automatically distributes the image to the identified person (for example, Patent Document 1).

JP 2004-326281 A

  However, in the conventional image distribution server, the recognition result of the person's face in the image is used only for the distribution of the image, and the recognition result of the person's face cannot be used effectively yet.

The present invention recognizes a person's face included in the target image, classifies the target image based on the result of face recognition, and stores data for accessing the target image in a folder according to the classification result. Features.
In the present invention, a folder is created for each person in advance, and data for accessing the target image is stored in the folder for the person included in the target image based on the recognition result of the person's face. May be.
Alternatively, in the present invention, a folder is created for each group including at least one person, and the group folder to which the person included in the target image belongs is based on the recognition result of the person's face. Data for accessing the target image may be stored.
When folders are created for each group, the number of persons belonging to the group is compared with the number of persons included in the target image, and the number of persons belonging to the group is determined from the number of persons included in the target image. If there are too many, it is preferable not to recognize the faces of persons belonging to the group. Moreover, it is preferable to set the number of print target images to one for the persons belonging to the same group among the persons included in the print target image regardless of the number of persons.

  According to the present invention, the recognition result of a person's face can be used effectively.

-First embodiment-
FIG. 1 is a block diagram showing a configuration of an embodiment of an image management apparatus according to the first embodiment. As the image management apparatus 100, for example, a personal computer (personal computer) is used. The image management apparatus 100 includes an operation member 101, an image input I / F (image input interface) 102, a control device 103, an HDD (hard disk drive) 104, and a monitor 105.

  The operation member 101 includes various input devices operated by a user such as a keyboard and a mouse. The image input I / F 102 is a connection interface for connecting an external image input device such as a digital camera or an image storage device, and is used for capturing a management target image (target image) described later from the image input device. Is. As the image input I / F 102, for example, a USB interface or a wireless LAN interface is used. As the monitor 105, for example, a liquid crystal display is used.

  The HDD 104 stores various data such as image data captured via the image input I / F 102 and programs executed by the control device 103 as described later. In the present embodiment, as will be described later, the captured image data is classified and stored in a plurality of folders for each person in the image by the control device 103. For this purpose, a folder for sorting images is created in the HDD 104 for each person who wants to sort images in advance by the user. In each folder, a face image (registered face image) of a person to be sorted in the folder is stored.

  FIG. 2 is a diagram illustrating a specific example of a folder structure for storing images according to the first embodiment. As shown in FIG. 2, for example, an image storage folder 2a is created in the HDD 104, and folders 2b to 2e for persons A to D are created in advance in the image storage folder 2a. That is, a folder is created in advance in the HDD 104 for each person. The person A folder 2b stores the registered face image of the person A, and the person B folder 2c stores the registered face image of the person B. Similarly, a registered face image of the person C and a registered face image of the person D are stored in the folder 2c for the person C and the folder 2d for the person D, respectively.

  The control device 103 includes a CPU, a memory, and other peripheral circuits, and functionally includes a face recognition unit 103a and an image classification unit 103b.

  The face recognition unit 103a performs known face recognition processing one by one on the target image (input image) captured via the image input I / F 102, and an image of the human face portion (input) from the image Face image). At this time, when a plurality of human faces are included in the image (exposed), a plurality of input face images are extracted. Then, the face recognition unit 103a represents information representing the positional relationship between the facial feature points based on the input face image, that is, the positional relationship between the eyes, nose, mouth, and the like as the feature points for identifying the individual's face. Information is calculated as face feature data. Similarly, face feature data is calculated for registered face images stored in each folder. Note that the face recognition process and the calculation method of face feature data are omitted because they are known techniques.

  The face recognition unit 103a identifies the registered face image having the highest degree of coincidence with the input face image by comparing the calculated face feature data of the input face image with the face feature data of the registered face image. Then, the image classification unit 103b receives the image including the input face image in the folder in which the registered face image having the highest degree of coincidence with the input face image, that is, the input image captured via the image input I / F 102 is input. Remember. Thereby, each input image can be sorted and stored in a folder for a person included in the image.

  If a single input image includes a plurality of human faces, the input image is copied and distributed to a plurality of folders for each person. For example, when the faces of persons A and B are included in the input image, the input images are copied and stored in the person A folder 2b and the person B folder 2c shown in FIG. Is done.

  At this time, when one image is copied and stored in each of the person A folder 2b and the person B folder 2c, one image is stored in different folders in duplicate. There is a possibility that the capacity of the HDD 104 will be compressed. Therefore, in order to avoid this, when a plurality of human faces are included in one image, the image classification unit 103b stores only one input image in the HDD 104, The A folder 2b and the person B folder 2c may store virtual copies of the input images. The virtual copy means data for accessing the input image. For example, a method of storing a link to the actual input image in a folder for each person is conceivable. Thereby, it is possible to distribute the input image to the folder for each person while avoiding the capacity of the HDD 104 being compressed.

  FIG. 3 is a flowchart illustrating processing of the image management apparatus 100 according to the first embodiment. The processing shown in FIG. 3 is executed by the control device 103 as a program (image management program) that is activated when an input image is captured via the image input I / F 102.

  In step S <b> 10, the face recognition unit 103 a performs a known face recognition process on the input image, and extracts an image of a human face portion, that is, an input face image from the image. Thereafter, the process proceeds to step S20, and the face recognition unit 103a calculates information representing the positional relationship between the facial feature points based on the input face image as facial feature data, and the process proceeds to step S30.

  In step S30, the face recognition unit 103a identifies the registered face image having the highest degree of coincidence with the input face image by comparing the calculated face feature data of the input face image with the face feature data of the registered face image. . Thereafter, the process proceeds to step S40, and the image classification unit 103b obtains an image including the input face image in the folder in which the registered face image having the highest matching degree with the input face image is stored, or data for accessing the input image. Sort and store, and end the process.

According to the first embodiment described above, the following operational effects can be obtained.
(1) The face recognition unit 103a recognizes the face of a person included in the input image (target image), and the image classification unit 103b classifies the input image based on the result of the face recognition, and obtains a classification result. The input image is stored in the corresponding folder. As a result, each input image can be sorted and stored in a folder corresponding to the person in the image.

(2) When a plurality of human faces are included in one input image, the image classification unit 103b stores only one input image in the HDD 104, and each folder has an input Instead of the image, a virtual copy of the input image, that is, data for accessing the input image is stored. As a result, it is possible to distribute images to folders according to the person while avoiding the capacity of the HDD 104 being compressed.

(3) A folder is created in advance in the HDD 104 for each person, and the image classification unit 103b is created for the person included in the input image based on the recognition result by the face recognition unit 103a. Each input image is stored in the folder. Thus, each input image can be managed by being distributed to a folder for each person included in the image.

-Second embodiment-
In the second embodiment, by storing a plurality of registered face images in a distribution destination folder, it is possible to distribute group unit images composed of at least one person in one folder. To do. Thereby, for example, if the registered face images for the whole family are stored in one folder, all the images including the family faces are stored in the folder for the family.

  FIG. 4 is a block diagram showing a configuration of an embodiment of an image management apparatus according to the second embodiment. In FIG. 4, the same components as those in the block diagram shown in FIG. 1 in the first embodiment are denoted by the same reference numerals, and differences will be mainly described. A printer 106 a is connected to the image management apparatus 100 via a predetermined connection interface (connection I / F) 106. The control device 103 further includes a print number setting unit 103c. A specific example of processing by the print number setting unit 103c will be described later.

  In the second embodiment, a folder is created in the HDD 104 for each group including at least one person. For example, an image storage folder having a structure as shown in FIG. 5 is created in the HDD 104. That is, folders 5b to 5e for groups A to D are created as lower folders of the image storage folder 5a in the HDD 104. In each of the folders A to D for the groups A to D, as described above, at least one face registration image is stored. For example, when the group A folder 5b is a family folder, registered face images for the whole family or several members of the family are stored in the folder 5b.

  Then, as in the first embodiment, the face recognition unit 103a compares the face feature data of the input face image with the face feature data of the registered face image, and thus has the highest degree of matching with the input face image. Specify the registered face image. The image classification unit 103b stores the input image in a folder in which all registered face images having the highest degree of coincidence with the input face image extracted from the input image are stored. In other words, the image classification unit 103b only uses the input image for the group composed of all the persons when all the persons corresponding to the registered face images stored in the folder are included in the input image. The input image is stored in the folder for the group.

  For example, when the group A folder 5b shown in FIG. 5 is used as a family folder, and registered face images for four families are stored in the folder 5b, all four of them are included. Input images are sorted and stored in the folder 5b. That is, an input image including only four family members or an input image including four family members and other persons is stored in the folder 5b. On the other hand, an input image that includes less than 3 persons out of 4 family members is not a target for distribution to the folder 5b, and is a predetermined folder such as an image storage folder. It is stored in 5a. As a result, only images including the entire family can be stored together in one folder.

  In the present embodiment, as described above, the image classification unit 103b applies the input image only when all of the registered face image persons stored in advance in the folder are included in the input image. Store in a folder. Therefore, the input images that may be stored in each folder are registered in which the number of input face images extracted from the input images (the number of persons included in the input images) is stored in the folder. It is limited to images that are equal to or greater than the number of face images (number of persons belonging to a group).

  Therefore, the face recognition unit 103a calculates the number of registered face images stored in each folder in advance and the number of human faces included in the input image. If the number of registered face images stored is greater than the number of human faces included in the input image, the registered face images stored in the folder are subject to face recognition processing. It may be outside. As a result, registered face images stored in a folder that cannot be an input image distribution destination can be excluded in advance from the target of face recognition processing, and the processing by the face recognition unit 103a can be speeded up. .

  When the user gives an instruction to print an image recorded in the HDD 104, the print number setting unit 103c determines the number of images to be printed according to the image classification result by the image classification unit 103. Specifically, the number-of-prints setting unit 103c displays the number of persons included in the image and the persons belonging to the group of the folder in which the image is stored, that is, the registered face image stored in the folder. The number of images to be printed is determined according to the number of images.

  When people are grouped as in the present embodiment, only one image is printed for the group. Therefore, the number-of-printing-number setting unit 103c sets the number of printed sheets to 1 for a person belonging to the group of the folder in which the image is stored among the persons included in the image, regardless of the number of persons. The number of printed sheets is set to one sheet, and for each other person, one sheet is set for each person.

  Specifically, the print number setting unit 103c performs the following process to set the number of prints. First, the number-of-printing-number setting unit 103c instructs the face recognition unit 103a to execute the above-described face recognition processing on an image (print target image) instructed to be printed by the user. The face recognition unit 103a performs a face recognition process on the print target image, extracts a face image from the print target image, and sets this as a person included in the print target image. Then, from the extracted face images, the face image having the highest degree of coincidence with the registered face image in the folder in which the print target image is stored is selected and set as a person belonging to the group.

  The number-of-printing-number setting unit 103c sets the number of printed sheets to one for persons belonging to the group selected by the face recognition unit 103a among the persons included in the print target image, and the other persons In contrast, one print number is set for each person. For example, when the group A folder 5b shown in FIG. 5 is a family folder and the registered face images for four families are stored in the folder 5b, the print target image includes four families. A case where six persons are included will be described. At this time, the number-of-printing-number setting unit 103c sets a total of three sheets, that is, one sheet for four family members and two sheets for the other two persons. In this way, it is possible to automatically set the optimum number of prints in consideration of the fact that one image needs to be printed on the group.

  Then, the print number setting unit 103 c outputs the image data of the print target image and the set print number data to the printer 106 a connected via the connection I / F 106. Thereby, the set number of print target images is output from the printer 106a.

  FIG. 6 is a flowchart showing image storage processing of the image management apparatus 100 according to the second embodiment. The process shown in FIG. 6 is executed by the control device 103 as a program that is activated when an input image is captured via the image input I / F 102. In FIG. 6, the same processes as those in FIG. 3 described in the first embodiment are given the same step numbers and the description thereof is omitted, and the differences will be mainly described.

  In step S50, the image classification unit 103b determines whether or not all of the registered face image people stored in advance in the folder are included in the input image, that is, whether or not all members of the group are included. To do. If a positive determination is made, the process proceeds to step S60. In step S60, the input image is stored in the group folder, and the process ends. On the other hand, if a negative determination is made, the process proceeds to step S70, where the input image is stored in a specified folder, and the process ends.

  FIG. 7 is a flowchart illustrating image printing processing of the image management apparatus 100 according to the second embodiment. The process shown in FIG. 7 is executed by the control device 103 as a program that is activated when an image to be printed is designated by the user and an instruction to execute printing is given.

  In step S110, as described above, the face recognition unit 103a performs face recognition processing on the print target image, extracts a face image from the print target image, and includes the person included in the print target image. And Thereafter, the process proceeds to step S120, where the face recognition unit 103a selects a face image having the highest degree of coincidence with the registered face image in the folder in which the print target image is stored from the extracted face images. A person belonging to. Thereafter, the process proceeds to step S130.

  In step S130, the print number setting unit 103c sets the number of prints to 1 for the persons belonging to the group selected by the face recognition unit 103a, and the other persons included in the print target image. Is set to the number of printed sheets. Thereafter, the process proceeds to step S140, and the number-of-printing-number setting unit 103c outputs the image data of the image to be printed and the set number of printed sheets to the connected printer 106a via the connection I / F 106, and ends the process. To do.

  In the second embodiment, the image storage process shown in FIG. 6 and the image printing process shown in FIG. 7 are executed by starting different programs. You may perform as a series of processes by starting a program.

According to the second embodiment described above, the following effects can be obtained in addition to the operational effects of the first embodiment.
(1) The image classification unit 103b is used only for a group of people who are registered face images stored in a folder in advance in the input image. It was determined that the image was an image, and the input image was stored in the folder for that group. As a result, only images including all the members belonging to the group can be collectively stored and managed in one folder.

(2) The face recognition unit 103a stores the number of registered face images (number of persons belonging to a group) stored in advance in each folder and the number of person faces included in the input image (in the input image). If the number of registered face images stored is greater than the number of person faces included in the input image, it is stored in that folder. The registered face images are excluded from the face recognition process. As a result, registered face images stored in a folder that cannot be an input image distribution destination can be excluded in advance from the target of face recognition processing, and the processing by the face recognition unit 103a can be speeded up. .

(3) The number-of-printing-number setting unit 103c sets the number of printed sheets to 1 for the persons belonging to the group among the persons included in the print target image, and sets the number of persons for the other persons. The number of prints is set to one for each. In this way, it is possible to automatically set the optimum number of prints in consideration of the fact that one image needs to be printed on the group.

-Modification-
The image management apparatus 100 according to the above-described embodiment can be modified as follows.
(1) In the above-described first and second embodiments, the example in which the face recognition process and the image distribution process are performed one by one on the input image captured via the image input I / F 102 has been described. That is, the example in which the face recognition process and the image distribution process are performed on the input image every time the input image is input via the image input I / F 102 has been described. However, the input images input via the image input I / F 102 are accumulated and stored in a predetermined folder in the HDD 104 as they are, and when the user instructs the image distribution, the face recognition process and An image distribution process may be executed to distribute images to a plurality of folders. In addition, face recognition processing and image distribution processing are executed when the number of images stored in a predetermined folder in the HDD 104 reaches a predetermined number set in advance or when a predetermined image distribution timing comes. The images may be distributed to a plurality of folders.

(2) In the first and second embodiments described above, the example in which the registered face image is stored in advance in each folder of the distribution destination has been described. However, a folder for storing the registered face image may be provided separately and stored in the folder. Then, a table in which the registered face image and the input image distribution destination folder are associated with each other is prepared, and the image classification unit 103b refers to the table and associates it with the registered face image having the highest matching degree with the input face image. The specified folder may be specified, and the input image may be sorted and stored in the specified folder. Thereby, the registered face images can be managed together in one folder.

(3) In the first and second embodiments described above, the example in which a personal computer is used as the image management apparatus 100 has been described. However, the image management apparatus 100 may be applied to other devices such as a PDA, a mobile phone, or a digital camera. For example, FIG. 8 shows a block diagram when the image management apparatus 100 according to the second embodiment is mounted on a digital camera. As shown in FIG. 8, the digital camera 200 includes a lens 107 and an image sensor 108, and is input via the lens 107 instead of the input from the image input I / F 102 in the image management apparatus 100. An input image is obtained by capturing an image of the subject to be captured by the image sensor 108.

  The digital camera 200 includes a memory card slot 109 instead of the HDD 104, and a folder having the structure shown in FIG. 5 is created in the memory card inserted into the memory card slot 109. As in the second embodiment, the face recognition unit 103a performs face recognition processing on the input image, and the image classification unit 103b stores the input image based on the recognition result by the face recognition unit 103a. The data is sorted and stored in a folder created in the memory card inserted into the card slot 109. Although the case where the image management apparatus 100 according to the second embodiment is mounted on a digital camera has been described here, the same applies to the case where the image management apparatus 100 according to the first embodiment is mounted on a digital camera.

  Alternatively, the personal computer and the server may be connected via a communication line, and the image transferred from the personal computer may be used as an input image, and the image may be classified and stored on the server side. As a specific example in this case, an online album service provided on the Internet can be considered. That is, a user accesses an online album providing service site and transfers images to be included in the album from the personal computer to the server. The server accepts the transferred image as an input image, classifies each image for each person, and creates an online album according to the classification result. The classification result is transferred to the personal computer, and the user can check the classification result on the monitor.

(4) In the first and second embodiments described above, an example in which the image classification unit 103b performs a process of sorting and storing images in folders according to the respective classification results has been described. However, the classification result may be displayed on the monitor 105 and presented to the user. For example, the classification result display screen as shown in FIG. 9 may be displayed on the monitor 105, and the folder name of each folder may be displayed in association with the thumbnail image of the image stored in each folder. In the second embodiment described above, the user may be able to designate a print target image by selecting a thumbnail image on the classification result display screen.

(5) In the second embodiment described above, the image classification unit 103b only applies the input image to the group when the input image includes all of the registered face image persons stored in advance in the folder. An example of storing in a folder for use has been described. However, if the input image includes at least one of the registered face images stored in the folder in advance, it is determined that the image is for the group, and the input image is used for the group. You may make it memorize | store in this folder. Thereby, images including at least one member belonging to the group can be collectively stored in one folder.

  Note that the present invention is not limited to the configurations in the above-described embodiments as long as the characteristic functions of the present invention are not impaired.

It is a block diagram which shows the structure of one Embodiment of the image management apparatus in 1st Embodiment. It is a figure which shows the specific example of the folder structure for image storage in 1st Embodiment. It is a flowchart which shows the process of the image management apparatus 100 in 1st Embodiment. It is a block diagram which shows the structure of one Embodiment of the image management apparatus in 2nd Embodiment. It is a figure which shows the specific example of the folder structure for image storage in 2nd Embodiment. It is a flowchart which shows the image storage process in 2nd Embodiment. 12 is a flowchart illustrating image printing processing according to the second embodiment. It is a block diagram which shows the structure of one Embodiment at the time of applying an image management apparatus to a camera. It is a figure which shows the specific example of a classification result display screen.

Explanation of symbols

DESCRIPTION OF SYMBOLS 100 Image management apparatus, 101 Operation member, 102 Image input I / F, 103 Control apparatus, 103a Face recognition part, 103b Image classification part, 103c Print number setting part, 104 HDD, 105 monitor, 106 Connection I / F, 106a Printer, 107 Lens, 108 Image sensor, 109 Memory card slot

Claims (6)

On the computer,
A face recognition procedure for recognizing a human face included in the target image;
An image management program for classifying the target image based on a recognition result obtained by the face recognition procedure and executing a storage control procedure for storing data for accessing the target image in a folder corresponding to the classification result. The image management program according to claim 1,
The folder is created in advance for each person,
The storage control procedure stores data for accessing the target image in a folder for a person included in the target image based on a recognition result obtained by the recognition procedure. The image management program according to claim 1,
The folder is created for each group including at least one person,
The storage control procedure stores data for accessing the target image in a folder for a group to which a person included in the target image belongs based on a recognition result obtained by the recognition procedure. Management program. The image management program according to claim 3,
The face recognition procedure compares the number of persons belonging to the group with the number of persons included in the target image, and the number of persons belonging to the group is greater than the number of persons included in the target image. An image management program characterized in that, when there are many, the faces of persons belonging to the group are not subject to recognition. The image management program according to claim 3 or 4,
On the computer,
Further executing a print number setting procedure for setting the number of prints of the print target image stored in the folder;
The number-of-printing-number setting procedure sets, for a person belonging to the same group among persons included in the print target image, the number of prints of the print target image is set to one regardless of the number of persons. An image management program. Execution means for executing the image management program according to any one of claims 1 to 5;
An image management apparatus comprising storage means for storing the target image.

Images