JP5044664B2 - Human image search device - Google Patents

Description

  The present invention relates to a person image search apparatus, and more particularly to a person image search apparatus that searches an image in which the same person as that shown in a search target image is copied from a search target image group.

  Various types of apparatuses that search an image showing a specific person from a search target image group have been proposed so far. For example, in Patent Document 1, a human face region is detected from a captured face image, facial parts such as eyes and mouth are extracted based on the detection result, and the facial feature amount is based on the extraction result. And a facial image search apparatus that suppresses the amount of calculation has been proposed by evaluating the similarity with a registered facial image held in a database.

JP 2003-178304 A

  However, in the face image search device of Patent Document 1, for example, feature quantities of a plurality of parts of a face such as eyes, nose, mouth, and the like are extracted, and the degree of similarity with a face image to be compared is evaluated. When the number of face images to be compared increases, there is a drawback that it takes time to search.

  The present invention has been made in view of such circumstances, and an object of the present invention is to provide a person image search apparatus capable of quick search.

In order to achieve the above object, the invention according to claim 1 is a person image search device that searches an image in which a person who is the same as a person copied in a search target image is copied from a search target image group. A comparison target setting unit that sets a face part to be used, and a comparison target setting unit that includes a person's face shown in the search target image and a person's face shown in each image of the search target image group. A comparison / determination unit that compares an image having a certain correlation as a similar image, an output unit that outputs an image determined as a similar image by the comparison / determination unit to a display device, and a re-search The comparison target setting unit adds and sets the face parts to be compared in a predetermined order when a re-search is instructed by the re-search instruction unit. A person image retrieval device characterized by Subjected to.

  According to the present invention, the face of the person shown in the search target image and the face of the person shown in each image of the search target image group are compared with the face parts set in the comparison target setting unit, An image having a certain correlation is determined to be a similar image and searched from the search target image group. When a re-search is instructed by the re-search instruction unit, the comparison target setting unit additionally sets a face part to be compared, and the re-search is performed. Thereby, it is possible to quickly search for a result that the user can satisfy.

  According to the human image search apparatus according to the present invention, a quick search can be performed.

Block diagram showing the configuration of the person image search device The flowchart which shows the procedure of the search process in the person image search apparatus of FIG. The flowchart which shows the procedure of the setting process of the comparison object in the person image search device of FIG. The block diagram which shows embodiment of the person image search device concerning this invention Flowchart showing search processing procedure Flow chart showing the procedure of comparison target setting processing

  The best mode for carrying out a person image retrieval apparatus according to the present invention will be described below with reference to the accompanying drawings.

  FIG. 1 is a block diagram showing a configuration of a person image search apparatus. A person image search apparatus 10 shown in FIG. 1 includes an image input unit 12, a search target detection unit 14, a comparison target setting unit 16, a comparison target extraction unit 18, a comparison determination unit 20, and an output unit 22. For example, it is realized by a personal computer incorporating a predetermined program.

  The image input unit 12 performs search target image input processing. Here, the search target image is person image data in which a person to be searched is copied, and is supplied from, for example, a digital camera or an auxiliary storage device connected to the person image search device 10.

  The search target detection unit 14 performs a process of detecting the size (number) of the search target image group. The search target image group is a collection of arbitrary image data regardless of the presence or absence of a person, and is stored as an image database 24 in an auxiliary storage device connected to the person image search device 10 or a data server connected through a network, for example. Has been.

  The comparison target setting unit 16 performs processing for setting a face part to be compared according to the size of the search target image group detected by the search target detection unit 14. Here, for example, if the search target image group is 10000 or more, “eyes” is compared, if “9999 to 1000”, “eyes, mouth”, and if less than 999 are 100, “eyes, mouth, nose” are compared. It shall be set to the target.

  The comparison target extraction unit 18 detects a human face area from the search target image input from the target image input unit 12 and the image (search target image) stored in the image database 24, and is set by the comparison target setting unit 16. The extracted face parts are processed.

  In the detection of the face area, a pixel having a color close to the color designated as the skin color is extracted from the original image, and the extracted area is detected as the face area. In this process, for example, in the color space for distinguishing the skin color from other colors, the range of the skin color in the color space is determined from the information of the skin color sampled in advance, and whether the color of each pixel is in the determined range or not. This is done by determining whether or not.

  In addition, the eyes that are face parts are extracted by, for example, detecting and extracting an area having a lower luminance value than the face area from the detected face area. For example, the mouth is extracted by detecting an area having a luminance value lower than that of the face area in the range below the extracted eyes. In the extraction of the nose, a rough region of the nose is specified between the eyes and the mouth, and a process of emphasizing the lateral edge is performed on this region. Then, a process of projecting the luminance value in the horizontal direction is performed on the obtained image, and the position having the smallest luminance value is determined as the nose position.

  The comparison determination unit 20 compares the face part of the search target image extracted by the comparison target extraction unit 18 with the face part of the search target image, and determines an image having a certain correlation as a similar image. That is, correlation processing is performed on the face part of the search target image and the face part of the search target image to obtain a correlation value, and an image within a preset threshold is determined as a similar image. For example, when “eyes and mouth” are extracted by the comparison target extraction unit 18, “eyes” of the search target image and “eyes” of the search target image, “mouth” of the search target image and “mouth” of the search target image. Is determined to be a similar image when both the “eyes and mouth” are within a preset threshold value.

  The output unit 22 outputs the search target image determined as a similar image by the comparison determination unit 20 to the display device 26.

  Next, the procedure of the search process of the person image search apparatus 10 configured as described above will be described with reference to the flowchart shown in FIG.

  First, a search target image is input by the image input unit 12 (step S10). As the search target image, for example, an image photographed with a digital camera may be taken in, or a specific one may be selected from images stored in the auxiliary storage device and taken as a search target image.

  When the search target image is input, next, the search target detection unit 14 detects the size of the search target image group (step S11). As described above, since the search target image group is provided as the image database 24, the search target detection unit 14 accesses the image database 24 and detects the size of the search target image group.

  When the size of the search target image group is detected, the comparison target setting unit 16 sets a face part to be compared based on the detected size of the search target image group (step S12).

  FIG. 3 is a flowchart illustrating the procedure of the comparison target setting process. First, it is determined whether the number of detected search target image groups is 10,000 or more (step S20). If it is determined that the number of detected search target image groups is 10,000 or more, “eyes” is set as the comparison target (step S21). On the other hand, when it is determined that the detected search target image group is not 10,000 or more, it is next determined whether the detected search target image group is 1000 or more and less than 9999 (step S22). If it is determined that the detected search target image group is 1000 or more and less than 9999, “eyes and mouth” is set as the comparison target (step S23), and 1000 or more and 9999 detected search target image groups are set. If it is determined that the number is not less than 1000, it is determined that the number of search target image groups is less than 1000, and “eyes, mouth, and nose” are set as comparison targets (step S24). The face parts to be compared are set as described above.

  When the setting of the face part to be compared is completed, the comparison object extracting unit 18 performs the process of extracting the face part set as the comparison object from the input search target image (step S13). For example, when “eye” is set as the comparison target, an image of “eye” is extracted from the search target image, and when “eye and mouth” is set, “eye and mouth” is extracted from the search target image. Are extracted respectively.

  Next, the first search target image is extracted from the image database 24 (step S14), and the extraction process of the facial part set as the comparison target for the extracted search target image is performed by the comparison target extraction unit 18. Performed (step S15).

  When the comparison target face part is extracted from the search target image, the extracted face part of the search target image is compared with the face part extracted from the search target image, and it is determined whether or not they are similar. That is, a correlation value between the face part extracted from the search target image and the face part extracted from the search target image is calculated (step S16), and whether the correlation value is within a preset threshold value is compared and determined 20. (Step S17).

  As a result, when it is determined that the correlation value between the face part extracted from the search target image and the face part extracted from the search target image is within the threshold value, both are determined to be similar images, and are determined to be similar images. The search target image is output from the output unit 22 and displayed on the display device 26 (step S18). On the other hand, if it is determined that the correlation value between the face part extracted from the search target image and the face part extracted from the search target image is not within the threshold, both are determined to be dissimilar images, and the image display is not performed. Proceed to the next step.

  As described above, when it is determined that the search target image is similar to the search target image, the search target image is output to the display device 26. Thereafter, it is determined whether or not search processing has been performed for all images in the image database (step S19). If it is determined that search processing for all images has been performed, the processing ends.

  On the other hand, if it is determined that the search process for all images has not been completed, the process returns to step S14, the next search target image is extracted, and the above-described similarity determination process is performed.

  As described above, according to the human image search apparatus 10 of this configuration, the comparison target is set according to the size of the search target image group, and therefore, always within a certain time regardless of the size of the search target image group. The target image can be searched.

  In the above configuration, “eyes”, “mouth”, and “nose” are raised as settable comparison targets, but facial parts that can be set as comparison targets are not limited thereto. For example, in addition to these face parts, “brow”, “hairstyle”, “face outline” and the like may be added to the settable elements.

  Moreover, the setting example of a comparison object is an example, and is not limited to this.

  FIG. 4 is a block diagram showing an embodiment of a person image search device according to the present invention. As shown in the figure, the human image search device 30 of the present embodiment includes an image input unit 12, a comparison target extraction unit 18, a comparison determination unit 20, an output unit 22, a comparison target setting unit 32, The re-search instruction unit 34 is implemented by, for example, a personal computer incorporating a predetermined program.

  The configuration of the person image search apparatus 10 having the configuration shown in FIG. 1 is the same as the configuration of the person image search device 10 described above except that the comparison target detection unit 14 is not provided and the re-search instruction unit 34 is provided and the operation of the comparison target setting unit 32 is different. The same. Therefore, the same components as those of the human image search apparatus 10 having the configuration shown in FIG.

  The comparison target setting unit 16 of the person image search apparatus 10 having the configuration shown in FIG. 1 described above sets the face parts to be compared in accordance with the size of the search target image group. The comparison target setting unit 32 of the search device 30 sets a face part to be compared in accordance with an instruction from the re-search instruction unit 34. That is, for example, in the first search, “eyes” is set as a comparison target, and when a re-search instruction is issued from the re-search instruction unit 34, “eyes and mouth” are set as comparison targets. When a re-search instruction is issued from the re-search instruction unit 34, “eyes, mouth and nose” are set as comparison targets. The comparison target extraction unit 18 extracts the face parts set by the comparison target setting unit 32 from the search target image and the search target image.

  The re-search instruction unit 34 for instructing re-search is constituted by, for example, a keyboard, and an instruction is output by a key operation of the keyboard. Alternatively, an instruction is output by clicking a re-search execution button displayed on the display device 26.

  Next, the procedure of the search process of the person image search apparatus 30 of the present embodiment configured as described above will be described with reference to the flowchart shown in FIG.

  First, a search target image is input by the image input unit 12 (step S30). The search target image input method is the same as that of the person image search apparatus 10 having the configuration shown in FIG.

  When the search target image is input, the comparison target setting unit 32 sets a face part to be compared (step S31).

  FIG. 6 is a flowchart illustrating the procedure of the comparison target setting process. First, it is determined whether or not this is the first search (step S40). If it is determined to be the first search, “eye” is set as the comparison target (step S41). On the other hand, if it is determined that the search is not the first, it is determined whether or not the search is the second time (step S42). If it is determined that the search is the second time, “eye and mouth” is set as the comparison target (step S43). If it is determined that the search is not the second time, it is further determined whether or not the search is the third time. (Step S44). If it is determined that the search is the third time, “eyes, mouth, and nose” is set as the comparison target (step S45). If it is determined that the search is not the third time, it is determined that the search is the fourth time or later. Then, an error display is output to the display device 26 (step S46). That is, a message indicating that the search cannot be performed again with higher search accuracy is displayed on the display device 26.

  In this way, the face parts to be compared are set according to the number of re-searches. Here, since it is the first search, “eye” is set as the comparison target. When the comparison target setting is completed, the comparison target extraction unit 18 performs processing for extracting the face part set as the comparison target from the input search target image (step S32).

  Next, the first search target image is extracted from the image database 24 (step S33), and the extraction process of the face part set as the comparison target for the extracted search target image is performed by the comparison target extraction unit 18. Performed (step S34).

  When the comparison target face part is extracted from the search target image, the extracted face part of the search target image is compared with the face part extracted from the search target image, and it is determined whether or not they are similar. That is, the correlation value between the face part extracted from the search target image and the face part extracted from the search target image is calculated (step S35), and the comparison determination unit 20 determines whether the correlation value is within a preset threshold value. (Step S36).

  As a result, when it is determined that the correlation value between the face part extracted from the search target image and the face part extracted from the search target image is within the threshold value, both are determined to be similar images, and are determined to be similar images. The search target image is output from the output unit 22 and displayed on the display device 26 (step S37). On the other hand, if it is determined that the correlation value between the face part extracted from the search target image and the face part extracted from the search target image is not within the threshold, both are determined to be dissimilar images, and the image display is not performed. Proceed to the next step.

  As described above, when it is determined that the search target image is similar to the search target image, the search target image is output to the display device 26. Thereafter, it is determined whether or not search processing has been performed for all images in the image database (step S38). If it is determined that search processing for all images has been performed, the processing ends.

  On the other hand, if it is determined that the search processing for all images has not been completed, the process returns to step S33, the next search target image is extracted, and the above-described similarity determination processing is performed.

  When the search processing for all images is completed, the user looks at the result output to the display device 26 and determines whether or not re-search is necessary. When a search with higher accuracy is desired, the re-search instruction unit 34 inputs a re-search instruction.

  The presence or absence of a re-search instruction by the re-search unit 34 is determined (step S39), and if a re-search instruction is issued, the process returns to step S31, and after the comparison target is reset, under the reset comparison target, The analogy determination process described above is performed.

  As described above, according to the human image search apparatus 30 of the present embodiment, the image search is performed while gradually increasing the search accuracy in accordance with the re-search instruction, so that the user can quickly search for a satisfactory result. it can.

  The face parts that can be set as comparison targets are not limited to those in the above example, as in the person image search apparatus 10 having the configuration shown in FIG. 1. For example, “eyes”, “mouth”, “nose” "Brow", "hairstyle", "contour" and the like may be added to the settable elements.

  In the above embodiment, the present invention is realized by a personal computer in which a predetermined program is incorporated. However, the function is realized by incorporating the predetermined program in a computer built in a digital camera, a mobile phone with a camera, or the like. You may make it make it.

  DESCRIPTION OF SYMBOLS 10 ... Person image search device, 12 ... Image input part, 14 ... Search object detection part, 16 ... Comparison object setting part, 18 ... Comparison object extraction part, 20 ... Comparison determination part, 22 ... Output part, 24 ... Image database, 26 ... Display device, 30 ... Human image search device, 32 ... Comparison target setting unit, 34 ... Re-search instruction unit

Claims (4)

In the person image search device for searching for an image in which the same person as the person copied in the search target image is searched from the search target image group,
A comparison target setting unit for setting a face part to be compared;
The face of the person shown in the search target image and the face of the person shown in each image of the search target image group are compared with the parts set in the comparison target setting unit, and have a certain correlation. A comparison determination unit that determines the image as a similar image;
An output unit that outputs an image determined as a similar image by the comparison determination unit to a display device;
A re-search instruction section for instructing re-search,
And the comparison target setting unit adds and sets face parts to be compared in a predetermined order when the re-search instruction unit instructs re-search. .   The number of re-searchable times is set in advance, and when the re-search instructing unit is instructed to re-search beyond the number of re-searchable times, the display device includes means for outputting an error display. The person image search device according to claim 1.   The facial parts that can be set as comparison targets include at least eyes, mouth, and nose, and the comparison target setting unit sets the facial parts to be compared one by one in a predetermined order in response to a re-search instruction. The person image search apparatus according to claim 1, wherein the person image search apparatus is additionally set.   The person image search device according to claim 1, wherein the comparison target setting unit sets an eye as a comparison target in an initial search.

Images