JP4522323B2 - Image processing apparatus and control method thereof - Google Patents

Description

  The present invention relates to an image processing technique for detecting a face area from an image and performing person verification.

  With the spread of image processing related devices such as digital cameras and scanners, the number of digital images handled in ordinary households and businesses is increasing. The face collation process is one of means for efficiently classifying these enormous numbers of digital images according to the contents. By using the face matching process, it is possible to easily classify the image file by the person who is the subject and the number of people, not the information accompanying the image as a character string such as the shooting date and time and the image size.

  On the other hand, it is conceivable that the target of face matching processing is to specify a very large image size or a huge number of image groups. Therefore, not only the accuracy but also the calculation amount is reduced in the processing at high speed. Operation is also a very important issue.

  As a method for improving the processing speed, a method is generally used in which a face information database for each image is created using the first matching result, and the second and subsequent matching speeds are increased (for example, patent literature). 1).

  Further, in Patent Document 2, a human face region is detected from a face image, and facial parts such as eyes and mouth are extracted based on the detection result, and a facial feature amount is extracted based on the extraction result. A method for evaluating the degree of similarity with a registered face image database is disclosed.

  FIG. 5 is a functional block diagram showing a configuration of a face image search apparatus as a conventional technique.

In the figure, 501 is an image capturing unit that captures an image, 502 is a face region detecting unit that detects a hair region and a skin color region from the captured face image, and 503 is an eye, mouth from the skin color region. , A face part extraction unit for extracting the position of the nose, 504, a feature amount extraction unit for extracting feature amounts such as hairstyle feature amounts and face part shapes, and 505, a face part extraction unit 503 and feature amount extraction 504 indicates a database for registering and holding face information extracted by the unit 504, and 506 indicates an evaluation processing unit that reads information from the database 505 when a search process is requested, and performs comparison / collation processing with a search target. ing.
Japanese Patent Laid-Open No. 2003-150617 JP 2003-178304 JP

  In the above-mentioned Patent Document 1, it is necessary to extract information relating to a face area from an image to be collated in the first collation in which a face information database that can be referred to in face collation processing is not created in advance. At this time, since the size pattern of the face area to be searched increases as the image size of the target image increases, more time is required for face area detection. This problem can be avoided if the image is reduced to an appropriate image size in advance, but extra time is required for processing unrelated to the face area detection processing in order to reduce the image quality.

  In Patent Document 2, the processing time for detecting a facial part from the target image increases according to the image size of the target image, so it is difficult to maintain real-time characteristics when the image size is very large. There is. In addition, if the target image is larger than a certain size, it may be possible to use a reduced version. However, if a large number of images are input, the processing time for the reduction process becomes enormous. Problem arises.

The present invention has been made in view of the above problems, and an object thereof is to realize high-speed and reduce processing time by using the reduced image image of the subject image image detection of the face area person verification process.

In order to solve the above-described problems and achieve the object, an image processing apparatus according to the present invention is an image processing apparatus that detects a face area from an image and performs a person matching process . a primary detector for detecting a face region with respect to the reduced image, when the face area is detected by the primary detector, the size shrinking small image size than the reduced image subjected to detection by the primary detecting means secondary detecting means for detecting a face area on the image or the target image itself, a collating unit for performing person verification using the face region detected by said secondary detection means, a specific size of the target image If a face area is not detected even when the primary image is detected for the reduced image, control is performed so that the secondary image is not detected for the target image. And a control means.

According to the present invention, by using a reduced image of the subject image image detection of the face area, it is possible to perform human material collation processing at high speed and with high accuracy.

  DESCRIPTION OF EXEMPLARY EMBODIMENTS Hereinafter, exemplary embodiments for realizing the invention will be described in detail with reference to the accompanying drawings.

  The embodiment described below is an example as means for realizing the present invention, and should be appropriately modified or changed according to the configuration and various conditions of the apparatus to which the present invention is applied. It is not limited to the embodiment.

  Further, the present invention supplies a recording medium (recording medium) storing software program codes for realizing an image processing method of an embodiment described later to the system or apparatus, and the computer (or CPU or MPU) of the system or apparatus. Needless to say, this is also achieved by reading and executing the program code stored in the storage medium.

  In the present embodiment, an image captured by a digital camera is used as a target image for face (person) collation processing, and an image associated therewith is thumbnail data created in the file of the target image by the digital camera or a digital camera. A reduced image created outside the target image file, a reduced image created by an application running on the PC based on the target image, and a reduced image created by the OS on the PC based on the target image are used.

  FIG. 1 is a block diagram showing a configuration of an image processing apparatus according to an embodiment of the present invention, and FIG. 2 is a flowchart showing face matching processing and matching result display processing according to the first embodiment.

  In FIG. 1, reference numeral 101 denotes an arithmetic processing unit (CPU) that controls the operation of face collation processing. Reference numeral 102 denotes a hard disk drive (HDD) in which digital data such as an execution program and encoded target image data is recorded in a readable format. Reference numeral 103 denotes a memory device, which is configured such that a program code and target image data recorded in the hard disk drive 102 are loaded and executed by the CPU 101.

  Reference numeral 104 denotes an operation unit including a keyboard and a mouse, and is an input unit for collation operation. The CPU 101 detects this input and notifies the program loaded in the memory device for processing. Reference numeral 105 denotes a display unit (display) such as a CRT display or a liquid crystal panel, which displays a processing target and an image group of processing results.

  Reference numeral 106 denotes an external storage medium drive for reading an external storage medium such as a CD-ROM or a DVD-ROM. In this embodiment, the program and image data recorded on the hard disk drive 102 are used. However, when the program and image data are recorded on an external storage medium, the program and image data may be executed via the external storage medium drive 106. Similarly, the present invention can be applied to a form in which a program and image data are loaded via a network.

  Next, the face matching process used in this embodiment will be described.

The face area detection process (S210, S309, S312, S317) and the face collation process (S212, S314) described later in FIGS. 2 and 3 use the method proposed by Laurenz Wiskott et al. (Laurenz Wiskott, Jean-Marc Fellous, Norbert Kruger, and Christoph von der Malsburg, "Face Recognition by Elastic Bunch Graph Matching" IEEE Trans. On Pattern Analysis and Machine Intelligence, vol.19, no.7, pp.775-779 , 1997)
These processes are performed in the following order.
1. Search for a human face from the search target image.
(1-1) Rough contour detection of a face image is performed on an image to be searched using a wavelet transform method. That is, a face graph including a small number of nodes is applied to an image, and a face area is detected from the target image.
(1-2) Next, the face position detected roughly is used to correct the tilt and size of the face, and then wavelet transform is used again to make up a large number of nodes including individual positions of the face part. Create a face graph. The face graph created here is face data that characterizes the original face image.
(1-3) The above operation is repeated for a plurality of search target images, and only images that can be determined as faces are taken as images of people.
2. A similar face is detected from the search target image.
(2-1) As in the case of 1., rough contour detection of a face image is performed on a search target image using a wavelet transform method.
(2-2) Next, the face position detected roughly is used to correct the tilt, size, etc. of the face, and then the wavelet transform is used again to form a large number of nodes including individual positions of the face part. Create a face graph. The face graph created here is face data that characterizes the face of the search target image.
(2-3) The feature amount of the face graph created as described above and the face graph specified by the search condition are compared, and if the difference is less than or equal to the threshold value, it is determined that the face is similar.
(2-4) The above operation is repeated for a plurality of search target images.

  Steps S210, S309, S312, and S317 are used to detect a face area from an image using the method described in 1. above, and steps S212 and S314 are performed using the method described in 2. above to collate with a person specified by a search condition. Is to do.

  Next, a method for selecting a reduced image to be a target image for face area detection in the present embodiment will be described with reference to FIG.

  The target image 401 is an image file captured by a digital camera and taken into the hard disk 102 drive on the PC. The reduced image 402 is information created by the digital camera as file header information of the target image 403.

  The reduced image 403 is a separate file created in association with the target image 401 by the digital camera when the target image 401 is captured, and has a larger image size than the reduced image 402. The reduced image 403 is also taken into the hard disk drive 102 on the PC, like the target image 401.

  The reduced image 404 is a thumbnail created for image browsing by a face matching application in which face matching processing is implemented. This thumbnail is not created for use in the face matching process, but is created to satisfy the image browsing function of the face matching application. This image size is equal to the reduced image 402.

  The reduced image 405 is a thumbnail file automatically created by the OS for the target image 401, and the image size is equal to the reduced image 402 and the reduced image 404. The reduced image 404 and the reduced image 405 are created as separate files from the target image.

  In the reduced image selection process, a reduced image having a large image size is used preferentially. Since it is considered that the image size of the reduced image created by the digital camera, the OS, and the face matching application hardly changes, the image size is not acquired for each image. The order in which the image size is considered to be large is determined in advance, reduced images are acquired in this order, and the reduced image obtained first is used for the face matching process. In the case where the image sizes are equal, a priority order from the viewpoint of image quality is determined.

  In the present embodiment, the reduced image 304 created by the face matching application is considered to have higher image quality than the reduced image 405 created by the OS, and the reduced image 304 is used preferentially. Based on the above selection criteria, in the present embodiment, the reduced images 403, 402, 404, and 405 are preferentially used in the order. Although the face area included in these reduced images is smaller than that in the target image, in this embodiment, the face area detection method that can detect a face area of an arbitrary size is used, so the reduced image is represented as a face area. It can be set as a detection target image, and a face region can be correctly detected even if it is a reduced image.

[First Embodiment]
With reference to FIG. 2, the face matching process and the matching result display process according to the first embodiment will be described.

  First, when the face matching process is started, it is determined in step S201 whether an image has been selected.

  If no image has been selected, the process returns to immediately before step S201. If it has been selected, the image is set as a target image for face matching processing in step S202.

  Next, when the target image is set in step S202, the process proceeds to step S203.

  In step S203, it is determined whether the collation processing button has been selected. If the collation processing button has not been selected, the process returns to immediately before step S201. If it has been selected, the process proceeds to step S204 to determine the presence of a reduced image. Steps S204 to S207 are processes for determining whether or not a reduced image exists, and the determination is performed according to the above-described priority order.

  First, in step S204, it is determined whether there is a reduced image 403 associated with the target image. If it exists, the process proceeds to step S209, and the reduced image 403 is set as a face area detection target image. If not, the process proceeds to step S205.

  In step S205, it is determined whether there is a reduced image 402 associated with the target image. If it exists, the process proceeds to step S209, and the reduced image 402 is set as a face area detection target image. If not, the process proceeds to step S206.

  In step S206, it is determined whether there is a reduced image 404 associated with the target image. If it exists, the process proceeds to step S209, and the reduced image 404 is set as a face area detection target image. If not, the process proceeds to step S207.

  In step S207, it is determined whether there is a reduced image 405 associated with the target image. If it exists, the process proceeds to step S209, and the reduced image 405 is set as a face area detection target image. If not, the process proceeds to step S208.

  In step S208, if there is no reduced image associated with the target image, a reduced image of VGA size is created from the target image, the process proceeds to step S209, and this reduced image is set as a face area detection target.

  After setting each reduced image as a face area detection target in step S209, the process proceeds to step S210.

  In step S210, face area detection processing is performed on the reduced image, and the process proceeds to step S211.

  In step S211, it is determined whether a face area is detected in step S210. If the face area has been detected, the process proceeds to step S212. If the face area has not been detected, the process proceeds to step S213.

  In step S212, face collation processing is performed using the above-described algorithm using the face area detected from the reduced image. When the face matching process is completed, the process proceeds to step S213.

  Note that the face area detected from the reduced image as a result of the face matching process is stored in association with the area in the target image using the coordinate information.

  In step S213, the face matching process result is displayed. The display content is information regarding whether a face area has been detected for the target image and, if detected, the result of face matching processing.

  In step S214, it is determined whether search termination processing has been performed. If the end process is not performed, the process returns to step S201 to perform the second and subsequent face collation processes. If the termination process has been performed, the face matching process is terminated.

  With the above processing, the face matching process is quickly performed, and the matching result can be presented.

[Second Embodiment]
Next, with reference to FIG. 3, the face matching process and the matching result display process according to the second embodiment will be described.

  In FIG. 3, the specific image size used in steps S311 and S316 is half the width of the main body image in both the vertical and horizontal widths.

  In the face area detection in steps S309, S312 and S317, the detection threshold of the face area is changed according to the image size of the target image. Specifically, the detection threshold is set higher as the image size is larger. This is for the purpose of preventing undetected images having a small image size and suppressing excessive detection in images having a large image size.

  In FIG. 3, first, when the face matching process is started, it is determined in step S301 whether an image has been selected.

  If no image has been selected, the process returns to immediately before step S301. If it has been selected, the image is set as a target image for face matching processing in step S302.

Next, when the target image is set in step S302, the process proceeds to step S303.
In step S303, it is determined whether a search processing button has been selected. If the search process button has not been selected, the process returns to immediately before step S301. If it has been selected, the process proceeds to step S304 to determine whether or not a reduced image exists. Steps S304 to S307 are first reduced image presence determination processing, and determination is performed according to the above-described priority order.

  In the first reduced image existence determination process, first, in step S304, it is determined whether there is a reduced image 403 associated with the target image. If it exists, the process proceeds to step S308. If not, the process proceeds to step S305.

  In step S305, it is determined whether there is a reduced image 402 associated with the target image. If it exists, the process proceeds to step S308. If not, the process proceeds to step S306.

  In step S306, it is determined whether there is a reduced image 404 associated with the target image. If it exists, the process proceeds to step S308. If not, the process proceeds to step S307.

  In step S307, it is determined whether there is a reduced image 405 associated with the target image. If it exists, the process proceeds to step S308. If not, the process proceeds to step S316.

  In step S308, the reduced image designated in any of steps S304 to S307 is set as the target image for the primary face area detection process, and the process proceeds to step S309.

After performing the primary face area detection process in step S309, it is determined in next step S310 whether or not a face area has been detected, and if detected, the process proceeds to step S312. If not detected, the process proceeds to step S311.

  In step S311, it is determined whether the image used for the primary face area detection process is larger than a specific size. If it is larger, the face area is not detected from the target image, and the process proceeds to step S315. If it is smaller, the process proceeds to step S316.

  In steps S316 and S317, a reduced image of a specific size is generated from the target image, face area detection processing is performed using the generated reduced image as the target image, and the process proceeds to step S318.

  In step S318, it is determined whether or not a face area has been detected as a result of performing face area detection on the generated reduced image. If a face area has been detected, the process proceeds to step S312. If not detected, it is determined that no face area has been detected from the target image, and the process proceeds to step S315.

  In step S312, when a face area is detected in the reduced image, secondary face area detection processing is performed using the target image itself. The image used as a detection target in this step may be an image larger than the reduced image used in the face area detection processing in steps S309 and S317, and an image that satisfies the condition is newly created from the target image and used. But you can. After this step, the process proceeds to step S313.

  In step S313, it is determined whether a face area is detected by the secondary face area detection process. If detected, the process proceeds to step S314; otherwise, the process proceeds to step S315.

  In step S314, a face matching process is performed using the detection result of the face area, and the process proceeds to step S315.

  In step S315, the face matching process result is displayed. The display content is information regarding whether a face area has been detected for the target image and, if detected, the result of face matching processing.

  Note that the face area detected from the reduced image as a result of the face matching process is stored in association with the area in the target image using the coordinate information.

  With the above processing, the face matching process is quickly performed, and the matching result can be presented.

[Other Embodiments]
The embodiment according to the present invention has been described in detail using specific examples. However, the present invention can take an embodiment as a system, apparatus, method, program, storage medium (recording medium), or the like. Specifically, the present invention may be applied to a system composed of a plurality of devices, or may be applied to an apparatus composed of a single device.

  In the present invention, a software program (in the embodiment, a program corresponding to each flowchart shown in the drawings) for realizing the functions of the above-described embodiment is directly or remotely supplied to the system or apparatus, and the computer of the system or apparatus is provided. Is also achieved by reading and executing the supplied program code.

  Accordingly, since the functions of the present invention are implemented by computer, the program code installed in the computer also implements the present invention. In other words, the present invention includes a computer program itself for realizing the functional processing of the present invention.

  In that case, as long as it has the function of a program, it may be in the form of object code, a program executed by an interpreter, script data supplied to the OS, or the like.

  As a recording medium (storage medium) for supplying the program, for example, a floppy (registered trademark) disk, hard disk, optical disk, magneto-optical disk, MO, CD-ROM, CD-R, CD-RW, magnetic tape, non-volatile Memory cards, ROM, DVD (DVD-ROM, DVD-R), and the like.

  As another program supply method, a client computer browser is used to connect to an Internet homepage, and the computer program of the present invention itself or a compressed file including an automatic installation function is downloaded from the homepage to a recording medium such as a hard disk. Can also be supplied. It can also be realized by dividing the program code constituting the program of the present invention into a plurality of files and downloading each file from a different homepage. That is, the present invention includes a WWW server that allows a plurality of users to download a program file for realizing the functional processing of the present invention on a computer.

  In addition, the program of the present invention is encrypted, stored in a storage medium such as a CD-ROM, distributed to users, and key information for decryption is downloaded from a homepage via the Internet to users who have cleared predetermined conditions. It is also possible to execute the encrypted program by using the key information and install the program on a computer.

  In addition to the functions of the above-described embodiments being realized by the computer executing the read program, the OS running on the computer based on the instruction of the program is a part of the actual processing. Alternatively, the functions of the above-described embodiment can be realized by performing all of the processes and performing the processing.

  Furthermore, after the program read from the recording medium is written in a memory provided in a function expansion board inserted into the computer or a function expansion unit connected to the computer, the function expansion board or The CPU or the like provided in the function expansion unit performs part or all of the actual processing, and the functions of the above-described embodiments are realized by the processing.

1 is a block diagram illustrating a configuration of an image processing apparatus according to an embodiment of the present invention. It is a flowchart which shows the image processing procedure of 1st Embodiment which concerns on this invention. It is a flowchart which shows the image processing procedure of 2nd Embodiment which concerns on this invention. It is a figure explaining the face area detection process of this embodiment. It is a functional block diagram which shows the structure of the face image search apparatus as a prior art.

Explanation of symbols

101 CPU
102 HDD
103 memory
104 Operation unit
105 display
106 External storage media drive
107 Internal bus
401 Target image for face matching
402 Thumbnail created inside target image file by digital camera
403 Reduced image created outside the target image file by the digital camera
404 Thumbnail created by application
Thumbnail created by 405 OS

Claims (14)

An image processing apparatus that detects a face area from an image and performs a person matching process,
Primary detection means for detecting a face area with respect to a reduced image of the target image to be subjected to person verification;
Two to when the face area is detected by the primary detection means detects a face area relative to the size shrinking small image or the target image itself image size than the reduced image subjected to detection by the primary detecting means A next detection means;
Collation means for performing person collation using the face area detected by the secondary detection means ;
If a face area is not detected even when detection by the primary detection unit is performed on a reduced image having a specific size or more of the target image, detection by the secondary detection unit is performed on the target image. And an image processing apparatus comprising: a control unit that controls the image processing so as not to occur. If the face area is not detected even if the face area is detected by the primary detection unit for the reduced image of the target image, it is determined whether the size of the reduced image of the target image is larger than a predetermined size. Image size determination means;
If it is determined that the reduced image of the target image is smaller than the predetermined size by the image size determination unit, further comprising a reduced image generation means for generating a reduced image of the predetermined size from the target image,
The primary detection means detects a face area for the reduced image created by the reduced image creation means,
The secondary detection means reduces the target image whose face area has not been detected by the primary detection means when the primary detection means detects a face area from the reduced image created by the reduced image creation means. There line detects a face area relative to the size shrinking small image or the target image itself image size than the image,
The control means does not detect the target image by the secondary detection means when the primary detection means does not detect a face area from the reduced image created by the reduced image creation means. the image processing apparatus according to claim 1, characterized in that control. Determining means for determining whether a plurality of reduced images of the target image exist;
A plurality of reduced small image determined by the previous SL-size constant means further comprises a primary detection image selecting means for selecting on the basis of the previously stated priority a reduced image to be used for the detection of the face region by the primary detecting means the image processing apparatus according to claim 1 or 2, characterized in that. The image processing according to any one of claims 1 to 3 , wherein the reduced image having an image size larger than the reduced image of the target image is an image having an image size equal to or smaller than the size of the target image. apparatus. If it is determined that multiple condensed small images are present by pre-Symbol-size constant means, said primary detecting image selecting means selects one of them as a large reduced image of the image size than the reduced image of the target image The image processing apparatus according to claim 3 , wherein: The target image is a digital image data stored in the storage unit in encoded predetermined format, large reduced image of the image size than the reduced image of the reduced image and the target image of the target image inside the object image the image processing apparatus according to any one of claims 1 to 5, characterized in that present in the. The target image is a digital image data stored in the storage unit in encoded predetermined format, large reduced image of the image size than the reduced image of the reduced image and the target image of the first target image of the target image the image processing apparatus according to any one of claims 1 to 5, characterized in that exist outside. Said primary detecting image selection means, the image processing apparatus according to claim 3, characterized in that selecting a reduced image in the data inside the target image preferentially. The image processing apparatus according to claim 3 , wherein the primary detection image selection unit preferentially selects an image having a large image size. It said primary detecting means and said secondary detecting means, an image processing apparatus according to any one of claims 1 to 9, characterized in that the image size of the reduced image can change the detection accuracy. Display means for displaying the collation result obtained by the collation means on a display unit;
And means for associating the verification result to the target image,
It means for associating said, the detected face area by the secondary detecting means, any one of claims 1 to 10, characterized in that associated with the area in the target image using the coordinate information The image processing apparatus according to item 1. A control method for an image processing apparatus that detects a face area from an image and performs a person matching process,
A primary detection step of detecting a face area with respect to a reduced image of the target image to be subjected to person verification;
Two to when the face area is detected by the primary detection step detects a face area relative to the size shrinking small image or the target image itself image size than the reduced image subjected to detection by the primary detecting step A next detection step;
A collation step of performing person collation using the face area detected by the secondary detection step ;
When a face area is not detected even when detection is performed by the primary detection step on a reduced image having a specific size or more of the target image, detection by the secondary detection step is performed on the target image. And a control step of controlling so that there is no control . A computer of an image processing apparatus that detects a face area from an image and performs person matching processing,
Primary detection means for detecting a face area with respect to a reduced image of the target image to be subjected to person verification;
Two to when the face area is detected by the primary detection means detects a face area relative to the size shrinking small image or the target image itself image size than the reduced image subjected to detection by the primary detecting means A next detection means;
Collation means for performing person collation using the face area detected by the secondary detection means ;
If a face area is not detected even when detection by the primary detection unit is performed on a reduced image having a specific size or more of the target image, detection by the secondary detection unit is performed on the target image. A program for functioning as a control means for controlling so as not to exist . A computer of an image processing apparatus that detects a face area from an image and performs person matching processing,
Primary detection means for detecting a face area with respect to a reduced image of the target image to be subjected to person verification;
Two to when the face area is detected by the primary detection means detects a face area relative to the size shrinking small image or the target image itself image size than the reduced image subjected to detection by the primary detecting means A next detection means;
Collation means for performing person collation using the face area detected by the secondary detection means ;
If a face area is not detected even when detection by the primary detection unit is performed on a reduced image having a specific size or more of the target image, detection by the secondary detection unit is performed on the target image. A computer-readable recording medium having recorded thereon a program for causing it to function as a control means for controlling so as not to occur .

Images