JP2009147644A - Image processing device, control method thereof, and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To enlarge face images of persons included in an image to be easily identified and easily specify the positions of the face images in a whole image. <P>SOLUTION: The face images are recognized from image data, and positions thereof are acquired. A layout of the face images is determined according to the positions of the face images. According to the determined layout, the face images are displayed in a display device. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to an image processing apparatus that handles image data obtained by photographing a person using an image input apparatus such as a digital camera, and a control method and program thereof.

  Conventionally, an image obtained by photographing a subject with a digital camera is captured in a personal computer and displayed on the monitor to check the composition of the entire image and the appearance of the subject, particularly a person's face. Here, the user has searched for and confirmed the person's face from the image. When an image includes many persons, the person's face becomes small, and thus confirmation is difficult.

Therefore, a face is extracted from an image and the face image is enlarged and displayed (see Patent Document 1). In Patent Document 1, the face of a person recognized from the entire image is displayed side by side in a specific area of the screen, whereby the state of the person's face can be individually confirmed.
JP 2007-019678 A

  However, in Patent Document 1, face images are displayed vertically in a specific area regardless of the position in the entire image. Therefore, it is difficult to specify the position of the face image in the entire image, particularly when a large number of human faces are included in the image.

  Further, in Patent Document 1, only a face image whose face is recognized is displayed, and a face which is actually a face but is turned sideways or face is not recognized because the eyes are meditated is displayed separately. There wasn't.

  The present invention has been devised in view of the above-described problems, and an object of the present invention is to allow a user to easily confirm an image including a plurality of human face images.

  In order to achieve the above object, according to the first invention of the present application, a face recognition unit that recognizes a face image from image data, an acquisition unit that acquires the position of the face image, and the position according to the position of the face image There is provided an image processing apparatus comprising: a determination unit that determines a layout of a face image; and a display control unit that displays the face images side by side according to the layout on a display device.

  According to the present invention, the user can easily confirm the face images of the person included in the image individually and can easily identify the position of each face image in the entire image. It becomes possible to do efficiently.

Example 1
Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

  The image processing apparatus in the present embodiment recognizes the face of a person reflected in the image data stored in the external storage device in accordance with the image selection application program. Then, the recognized face images are extracted, and a graphical user interface (GUI) arranged while maintaining the relative positions of the faces in the image data is displayed on the display device.

  FIG. 1 is a block diagram illustrating a configuration of a computer apparatus (information processing apparatus) that implements the image processing apparatus according to the present exemplary embodiment. A CPU (Central Processing Unit) 100 controls the operation of the entire computer apparatus. The memory 120 stores programs, data, parameters, and the like necessary for the operation of the computer device. An external storage device 130 typified by a hard disk or the like stores an OS (Operating System), image selection application software, and image data. The display memory 110 expands the image data stored in the external storage device 130 and displays it on the display device 150. The operation of the user of the image processing apparatus is input via the input device 140 such as a keyboard or a pointing device.

  Note that the image processing apparatus according to the present embodiment realizes the operation described below by the CPU 100 reading and executing the OS and image selection application software program codes stored in the external storage device 130. The image selection application software may be supplied to the CPU 100 from a removable storage medium. Therefore, the program code itself realizes the functions of the following embodiments, and a storage medium storing the program code also constitutes the present invention.

  FIG. 2 is a diagram showing a module configuration realized by an image selection application program installed in the image processing apparatus of this embodiment. Image reading means 200 for reading image data from the external storage device 130, image display control means 210 for displaying a plurality of images (including original images and face images) on the display device, and recognizing a human face included in the image data The face recognition unit 220 is configured. Further, although it is estimated that the face is actually captured, the face recognizing unit 220 includes a failed face image detecting unit 230 that detects a region that could not be recognized. Further, the image processing apparatus includes a face position acquisition unit 240 that acquires position information of a face image that has been face-recognized, and a rearrangement unit 250 that rearranges the face images in a designated layout.

  The operation of the image processing apparatus of this embodiment will be described with reference to the flowchart of FIG.

  In step S <b> 600, the image reading unit 200 reads image data from the external storage device 130. In this embodiment, it is assumed that image data of a group photograph as shown in FIG. 3 is read. The image data 300 is obtained by photographing 14 people from person A to person N in order in three rows.

  In step S601, the face recognition unit 220 performs face recognition processing on the image data read in step S600. As a result of the face recognition process, a face image area recognized as a face in the image data is acquired.

  In step S602, the face position acquisition unit 240 acquires position information (coordinates) of the face image. This detects the position in the image data for each face image, acquires the relative position information of the face images, and registers them in the memory 120.

  On the other hand, in step S601, even if the face is actually a person, if the face is turned sideways or the eyes are closed, there is not enough information for face recognition. Is not recognized as a face image. For example, let us consider a case where the person at the position “G” in the image data 300 in FIG. Here, in the case of a group photo, people are often photographed side by side at regular intervals. Therefore, if there is irregularly dispersed position information among the position information of the face image acquired in step S602, it can be predicted that there is a person who has not been recognized among them.

  Therefore, in step S603, the failed face image detection unit 230 first detects an area where there is a failed face image that has not been recognized in step S601, from the position information of the face image acquired in step S602.

  That is, the unsuccessful face image detection unit 230 obtains the distance between adjacent face images in accordance with the position information of each face image acquired in step S602. In the image data of FIG. 3, “F” and “H” are recognized as face images, and “F” and “H” are determined to be adjacent to each other based on their position information, and the distance between them is determined. Since the distance between the face images “F” and “H” is larger than the average of the distances between the centers of the other adjacent face images, the failure occurs between “F” and “H”. It is estimated that the face image “G” exists. In this way, even if the face recognition unit 220 performs an accurate recognition process and recognizes an area that is not recognized as a face image, an area that is actually estimated to have a face is recognized as a failed face image. Can do. In particular, it is highly probable that the photographing part of the face that the face recognition unit 220 could not recognize is unsuccessful, and this is useful when the user confirms the image data.

  In step S604, the face position acquisition unit 240 acquires the position information of the area searched in step S603 and registers it in the memory 120 as the position information of the failed face image.

  In step S605, the rearranging unit 250 reads the position information of the face image from the memory 120, and determines the layout of the face image and the failed face image.

  FIG. 4 shows the face image recognized in step S601 registered in the memory 120 and the failed face image estimated in step S603. An area surrounded by a dotted rectangular frame 401 in the image data 300 is a face image or a failed face image.

  In step S606, the image display control unit 210 causes the display device 150 to display the face image and the failed face image side by side according to the layout determined in step S605. Here, the face image and the failed face image may be enlarged and displayed at a predetermined magnification larger than the size cut out from the image data. Further, the face image and the failed face image may be displayed after being adjusted so that their sizes are equal.

  A display example in step S606 is shown in FIG. In FIG. 5A, the face image and the failed face image are displayed side by side according to the layout determined in step S605 on another screen independent of the screen on which the original image data is displayed. In FIG. 5B, the face image and the failed face image are displayed side by side in accordance with the layout determined in step S605 over the original image data. Thus, by enlarging and displaying the face image and the failed face image, the user can confirm in detail the appearance of each person's face.

  Note that the face image recognized in step S601 and the failed face image estimated in step S603 may be displayed separately. For example, an icon may be displayed near the face image or the failed face image, or the color of the rectangle or the line type may be changed between the face image and the failed face image. As a result, since a person who is photographed in a state where his / her face is not recognized by facing his / her eyes or facing sideways, it is likely that the user has failed to shoot, so the user can easily recognize a face that may fail.

  Further, only the failed face image estimated in step S603 may be displayed on the screen. By selectively displaying only the failed face images, the user can intensively confirm the face images that have a high possibility of failure, and the time required for the work can be shortened.

  In step S601, when face recognition is performed under a plurality of types of conditions, an area that is not face-recognized because some conditions are not satisfied may be stored in the memory 120 as a face candidate image. . For example, the skin color recognition is performed for pixels in a certain area, but the face recognition is not performed because the pupil recognition is not performed. In step S603, the failed face image detection unit 230 compares the position of the face candidate image with the estimated position and overlaps each other, and the area where the face candidate image exists is set as the area where the failed face image exists. You may make it guess. By estimating using both the position condition and the face recognition condition, the accuracy of the result of estimating that the face image exists is improved.

  As described above, according to the present embodiment, the photographed face is displayed in a state in which the positional relationship in the entire image is maintained, so that the user can locate each face in the entire image. It becomes easy to recognize whether it exists.

(Example 2)
In the case of a group photo with a large number of people, there are many face images in the image data, so if all of them are displayed at once, it will be cumbersome and difficult to see. Therefore, in this embodiment, a case will be described in which a face image included in image data is displayed on the screen for each group in a state where the positional relationship in the image data is maintained. In the following, description of parts similar to those of the first embodiment will be omitted, and parts unique to the present embodiment will be described in detail.

  The image processing apparatus according to this embodiment has a module configuration as shown in FIG.

  After step S604 described above, the number counting unit 720 calculates the sum of the number of face images recognized in step S601 and the number of failed face images detected in step S603, and determines whether the number is larger than the predetermined number. . When it is determined that the total is larger than the predetermined number of people, the group creating unit 710 automatically groups the face image and the failed image according to the respective position information.

  For example, when it is determined that the sum of face images and failure images is greater than 20, grouping is performed for each horizontal row as shown in FIG. Which face image and failure image belong to can be determined by the value of the Y coordinate of the position information. Further, the grouping is not limited to one horizontal row, but may be performed for each predetermined area.

  Then, each group generated by the grouping is displayed on the display device 150 in order. The group may be switched and displayed in response to the operation of the input device 140 by the user, or the group may be automatically switched and displayed when a predetermined time has elapsed.

  According to the present embodiment, by classifying face images and failure images into a plurality of groups and displaying them for each group, the user can easily see each face, and in which position in the entire image the user is located. Can be recognized.

It is a block diagram which shows the structure of the computer apparatus in one Example of this invention. It is a block diagram which shows the module structure of the image processing apparatus in one Example of this invention. It is a figure which shows the image data of the group photograph in one Example of this invention. It is a figure which shows the position of the face image contained in the image data in one Example of this invention. It is a figure which shows the example of a display of the face image contained in the image data in one Example of this invention. It is a flowchart figure which shows operation | movement of the image processing apparatus in one Example of this invention. It is a block diagram which shows the module structure of the image processing apparatus in one Example of this invention. It is a figure which shows the example of the group of the face image contained in the image data in one Example of this invention.

Explanation of symbols

100 CPU
110 Display Memory 120 Memory 130 External Storage Device 140 Input Device 150 Display Device

Claims (9)

Face recognition means for recognizing a face image from image data;
Obtaining means for obtaining the position of the face image;
Determining means for determining a layout of the face image according to a position of the face image;
An image processing apparatus comprising: display control means for arranging the face images according to the layout and displaying the face images on a display device. Detecting means for detecting a failed face image not recognized by the face recognition means according to the position of the face image;
The determination means further determines a layout of the failed face image according to a position of the failed face image, and the display control means arranges the failed images according to the layout and displays them on the display device. The image processing apparatus according to 1.   3. The image processing apparatus according to claim 2, wherein the display control unit distinguishes the face image and the failed face image from each other and displays them on the display device. The face recognition means further comprises a registration means for recognizing an area that satisfies all of a plurality of conditions as a face image and registering an area that satisfies a part of the conditions as a face candidate image,
The image processing apparatus according to claim 2, wherein the detection unit further performs the detection according to a face candidate image registered in the registration unit.   The image processing apparatus according to claim 1, wherein the display control unit enlarges the face image at a predetermined magnification and displays the face image on the display device. Further comprising group creation means for creating a group of face images according to the position of the face image;
6. The image processing apparatus according to claim 1, wherein the display control unit displays the face image on the display device for each group. A measuring means for measuring the number of the face images;
The image processing apparatus according to claim 6, wherein the group creating unit creates the group when the number of the face images is larger than a predetermined number. Recognizing a face image from image data;
Obtaining a position of the face image;
Determining the layout of the face image according to the position of the face image;
And a step of displaying the face images side by side according to the layout on a display device. Recognizing a face image from image data on a computer;
Obtaining a position of the face image;
Determining the layout of the face image according to the position of the face image;
And causing the display device to display the face images side by side according to the layout.

Images