JP6276504B2 - Image detection apparatus, control program, and image detection method - Google Patents

Description

  The present invention relates to a technique for extracting a feature amount from an image.

  As described in Patent Document 1, various techniques for extracting feature amounts from images have been proposed. Non-Patent Document 1 describes LBP (Local Binary Pattern) and LTP (Local Ternary Pattern) used in feature quantity extraction.

JP 2010-44438 A

Xiaoyang Tan and Bill Triggs, “Enhanced Local Texture Feature Sets for Face Recognition under Difficult Lighting Conditions”, IEEE Transactions on Imgage processing, Volume 19, Issue 6, pp.1635-1650, June 2010

  Now, when extracting a feature amount from an image, it is necessary to obtain an appropriate feature amount.

  Therefore, the present invention has been made in view of the above points, and an object thereof is to provide a technique capable of obtaining an appropriate feature amount.

In order to solve the above problems, one aspect of an image detection device according to the present invention is an image detection device that detects a detection target image from a processing target image, and includes a plurality of pixels included in the image included in the processing target image. Each of which is a pixel of interest, an evaluation value acquisition unit that obtains a plurality of types of evaluation values indicating the relationship between the pixel value of the pixel of interest and a plurality of surrounding pixel values, and the plurality of pixels that are included in the image seeking co-occurrence frequencies for evaluation value, determining a characteristic amount obtaining section for the co-occurrence frequency and feature amount, based on the feature quantity, whether the image is likely to be the detection target image The image detection device captures the processing target image between the plurality of types of evaluation values and uses the evaluation values used in the acquisition of the feature values used by the classifier. Use according to the brightness of the environment Divide.

In the aspect of the image detection device according to the present invention, the feature amount acquisition unit may perform the feature amount without normalizing the co-occurrence frequency for a certain type of evaluation value included in the plurality of types of evaluation values. And

In the aspect of the image detection apparatus according to the present invention, the evaluation value acquisition unit may be configured to obtain an evaluation value of a certain type included in the plurality of types of evaluation values in a diagonal direction with respect to the target pixel. Pixel values are not used.

Further, in one aspect of the image detection device according to the present invention, the evaluation value acquisition unit, when obtaining the certain type of evaluation value, surrounds the target pixel in the upper right, upper left, lower right, and lower left directions. All pixel values are not used.

Further, in one aspect of the image detection device according to the present invention, the evaluation value acquisition unit includes, for each of a plurality of surrounding pixel values used for acquiring the certain kind of evaluation value, the surrounding pixel value and the pixel of the target pixel. 1 bit indicating the relationship with the value is generated, and the binary code composed of the plurality of bits generated for the plurality of surrounding pixel values is related to the number of bit changes when the plurality of bits are viewed in order. Rather, it is used as a certain kind of evaluation value.

Further, in one aspect of the image detection apparatus according to the present invention, the plurality of types of evaluation values include a first type of evaluation value, and the evaluation value acquisition unit is used for acquiring the first type of evaluation value. For each of a plurality of surrounding pixel values, one bit indicating the relationship between the surrounding pixel value and the pixel value of the target pixel is generated, and a binary code including a plurality of bits generated for the plurality of surrounding pixel values is It was used as the first type of evaluation value, wherein one bit for the first type of evaluation value, surrounding pixel value is equal to or less than the value obtained by subtracting a predetermined amount from the pixel value of the pixel of interest "1" If it is larger than the value, “0” is indicated.

In the aspect of the image detection apparatus according to the present invention, the plurality of types of evaluation values include a second type of evaluation value, and the evaluation value acquisition unit is used for acquiring the second type of evaluation value. For each of a plurality of surrounding pixel values, one bit indicating the relationship between the surrounding pixel value and the pixel value of the target pixel is generated, and a binary code including a plurality of bits generated for the plurality of surrounding pixel values is Used as a second evaluation value, the 1 bit relating to the second type of evaluation value indicates “1” if the surrounding pixel value is equal to or greater than a value obtained by adding a predetermined amount to the pixel value of the target pixel. If it is less than the value, “0” is indicated.

  In one aspect of the image detection apparatus according to the present invention, the detection target image is a human face image.

One aspect of the control program according to the present invention is a control program for controlling an image detection device that detects a detection target image from a processing target image. The control program includes (a) the processing target image. Obtaining a plurality of types of evaluation values indicating the relationship between a pixel value of the target pixel and a plurality of surrounding pixel values, with each of a plurality of pixels included in the image included in the image included in the image, and (b) included in the image seeking co-occurrence frequencies for the evaluation values obtained for the plurality of pixels, the steps of the co-occurrence frequency and feature amount, based on (c) the feature amount, the image is the detection target Determining whether there is a high possibility that the image is an image, and the evaluation value used in the acquisition of the feature value used in the step (c) is, among the plurality of types of evaluation values, Imaging of the processing target image It is intended to make proper use according to the brightness of the border.

An aspect of the image detection method according to the present invention is an image detection method for detecting a detection target image from a processing target image, wherein: (a) each of a plurality of pixels included in the image included in the processing target image A step of obtaining a plurality of types of evaluation values indicating the relationship between the pixel value of the pixel of interest and a plurality of surrounding pixel values, and (b) the evaluation values obtained for the plurality of pixels included in the image seeking co-occurrence frequencies for the steps of the co-occurrence frequency with the feature amount, (c) on the basis of the feature quantity, the process determines whether the image is likely to be the detection target image The evaluation value used in the acquisition of the feature value used in the step (c) is determined according to the brightness of the imaging environment of the processing target image between the plurality of types of evaluation values. Use properly.

  According to the present invention, an appropriate feature amount can be obtained.

It is a figure which shows the structure of an image processing system. It is a figure which shows the structure of the several functional block with which an image detection apparatus is provided. It is a figure for demonstrating operation | movement of a detection part. It is a figure for demonstrating operation | movement of a detection part. It is a figure for demonstrating operation | movement of a detection part. It is a figure for demonstrating operation | movement of a detection part. It is a figure for demonstrating operation | movement of a detection part. It is a figure for demonstrating operation | movement of a detection part. It is a figure which overlaps and shows a detection result frame on a process target image. It is a figure which shows the structure of a feature-value extraction apparatus. It is a figure for demonstrating operation | movement of an evaluation value acquisition part. It is a figure which shows an example of an attention pixel value and a some surrounding pixel value. It is a figure for demonstrating the production | generation method of LBP. It is a figure for demonstrating the production | generation method of LBP. It is a figure which shows an example of a process target image. It is a figure which shows an example of an LPB map image. It is a figure for demonstrating the production | generation method of LTP. It is a figure for demonstrating the production | generation method of positive LTP. It is a figure for demonstrating the production | generation method of negative LTP. It is a figure which shows an example of a positive LTP map image. It is a figure which shows an example of a negative LTP map image. It is a figure which shows the positional relationship between an attention pixel and several surrounding pixels. It is a figure for demonstrating the method of calculating | requiring the pixel value in the peripheral position of the diagonal direction whose distance from an attention pixel is "1". It is a figure which shows an example of a one-dimensional evaluation value histogram. It is a figure which shows an example of multiple types of evaluation value pairs. It is a figure for demonstrating operation | movement of the feature-value acquisition part. It is a figure for demonstrating operation | movement of the feature-value acquisition part. It is a figure for demonstrating operation | movement of the feature-value acquisition part. It is a figure which shows an example of a two-dimensional evaluation value histogram. It is a figure for demonstrating the production | generation method of an output value map. It is a figure for demonstrating the production | generation method of an output value map. It is a figure which shows an example of the output value map which concerns on this Embodiment. It is a figure which shows an example of the output value map produced | generated based only on the 1st feature-value. It is a figure for demonstrating operation | movement of the modification of a feature-value acquisition part. It is a figure for demonstrating operation | movement of the modification of a feature-value acquisition part. It is a figure which shows a mode that an imaging environment is bright. It is a figure which shows a mode that an imaging environment is dark. It is a figure which shows the structure of the modification of an image processing system. It is a figure which shows the structure of the modification of an image detection apparatus. It is a figure which shows the frequency curve showing the frequency distribution about the brightness | luminance difference between adjacent pixels.

  FIG. 1 is a diagram illustrating a configuration of an image processing system 50 including an image detection apparatus 1 according to an embodiment. The image processing system 50 includes an image detection device 1 and an imaging device 5. The imaging device 5 captures an image and outputs image data indicating the captured image to the image detection device 1. The image detection apparatus 1 detects a detection target image from a captured image indicated by input image data. The image processing system 50 is used in, for example, a surveillance camera system, a digital camera system, or the like. In the present embodiment, the detection target image is, for example, a human face image. Hereinafter, simply speaking “face image” means a human face image. A captured image to be detected from the detection target image is referred to as a “processing target image”. The detection target image may be an image other than the face image. For example, the detection target image may be an image of the whole person.

  As shown in FIG. 1, the image detection apparatus 1 includes a CPU (Central Processing Unit) 2 and a storage unit 3. The storage unit 3 includes a ROM (Read Only Memory), a RAM (Random Access Memory), and the like. The storage unit 3 stores a control program 4 for controlling the operation of the image detection apparatus 1 and the like. Various functions of the image detection apparatus 1 are realized by the CPU 2 executing the control program 4 in the storage unit 3. In the image detection apparatus 1, a plurality of functional blocks as shown in FIG. 2 are formed by executing the control program 4.

  As shown in FIG. 2, the image detection apparatus 1 includes an image input unit 11, a detection unit 12, and an output value map generation unit 15 as functional blocks. Note that various functions provided in the image detection apparatus 1 may be realized by hardware circuits instead of functional blocks.

  A plurality of pieces of image data respectively indicating a plurality of images sequentially taken by the imaging device 5 are sequentially input to the image input unit 11. The image input unit 11 outputs image data indicating the processing target image. The image input unit 11 may use each image obtained by the imaging device 5 as a processing target image, or may use an image obtained every few seconds among images obtained by the imaging device 5 as a processing target image. In the imaging device 5, for example, L (L ≧ 2) images are captured per second. That is, the imaging frame rate in the imaging device 5 is Lfps (frame per second).

  In the image captured by the imaging device 5, M (M ≧ 2) pixels are arranged in the row direction, and N (N ≧ 2) pixels are arranged in the column direction. The resolution of the image picked up by the image pickup device 5 is, for example, VGA (Video Graphics Array), and M = 640 and N = 480.

  Hereinafter, the size of an area in which m (m ≧ 1) pixels are arranged in the row direction and n (n ≧ 1) pixels are arranged in the column direction is represented by mp × np (p is a pixel meaning). In addition, among a plurality of values arranged in a matrix, a value located in the m-th row and the n-th column with reference to the upper left may be referred to as an m × n-th value. In addition, in a plurality of pixels arranged in a matrix, a pixel located in the m-th row and the n-th column with respect to the upper left may be referred to as an m × n-th pixel.

  The detection unit 12 uses the image data output from the image input unit 11 to detect a face image for the processing target image. The output value map generation unit 15 generates an output value map indicating the distribution in the processing target image with respect to the detection accuracy value indicating the likelihood as the face image based on the detection result of the detection unit 12.

  Next, the operation of each block of the image detection apparatus 1 will be described in detail.

<Detection process>
As shown in FIG. 2, the detection unit 12 includes a feature amount extraction unit 13 and a discriminator 14. Using the detection frame, the detection unit 12 performs a detection process of detecting, as a detection result region, a region that is highly likely to be a face image having the same size as the detection frame in the processing target image. Hereinafter, simply “detection processing” means this detection processing in the detection unit 12. The detection process will be described in detail later.

  The detection unit 12 uses a plurality of types of detection frames having different sizes in order to detect face images of various sizes in the processing target image. In the detection unit 12, for example, 30 types of detection frames are used.

  In the present embodiment, as will be described later, the feature amount extraction unit 13 extracts a feature amount from an image. The feature amount extraction unit 13 needs to use an image having a reference size (normalized size) as a target image from which the feature amount is extracted.

  On the other hand, in the present embodiment, the plurality of types of detection frames having different sizes include a detection frame having the same size as the reference size and a detection frame having a size different from the reference size. Hereinafter, a detection frame having the same size as the reference size is referred to as a “reference detection frame”, and a detection frame having a size different from the reference size is referred to as a “non-reference detection frame”. In the present embodiment, the detection frame having the smallest size among the plurality of types of detection frames is the reference detection frame. Therefore, the size of the non-reference detection frame is larger than the reference size. The size of the reference detection frame is, for example, 16p × 16p. The plurality of types of detection frames include, for example, a non-reference detection frame having a size of 18p × 18p and a non-reference detection frame having a size of 20p × 20p.

  In the present embodiment, when performing detection processing for a processing target image using a reference detection frame, the detection unit 12 moves the reference detection frame with respect to the processing target image, A face image is detected for the image to determine whether the image is highly likely to be a face image. And the detection part 12 makes the area | region (image in a reference | standard detection frame) determined with possibility that it is a face image high in a process target image as a detection result area | region.

  On the other hand, when performing detection processing using the non-reference detection frame for the processing target image, the detection unit 12 changes the size of the non-reference detection frame so that the size matches the reference size. Then, the detection unit 12 changes the size of the processing target image in accordance with the size change of the non-reference detection frame. The detection unit 12 detects a face image for an image in the non-reference detection frame while moving the non-reference detection frame whose size has been changed with respect to the processing target image whose size has been changed, It is determined whether or not the image is likely to be a face image. Then, the detection unit 12 performs the size change based on the region (the image in the non-reference detection frame after the size change) that is determined to be highly likely to be a face image in the processing target image that has undergone the size change. A region that has a high possibility of being a face image in a processing target image of an original size that is not known is identified, and the region is set as a detection result region.

  Hereinafter, the processing target image after the size change when the non-reference detection frame is used for the processing target image and the detection processing is performed is referred to as a “size-changed image”. Further, the non-reference detection frame after the size change when the detection process is performed using the non-reference detection frame for the processing target image is referred to as a “size change detection frame”.

  Thus, in the present embodiment, the operation of the detection unit 12 when the detection unit 12 performs detection processing on the processing target image using the reference detection frame, and the detection unit 12 performs processing on the processing target image. Thus, the operation of the detection unit 12 when performing the detection process using the non-reference detection frame is different. The operation of the detection unit 12 will be described in detail below.

  In the detection unit 12, when the reference detection frame is used for the detection process, the feature amount extraction unit 13 sets a reference detection frame for the processing target image, and within the reference detection frame in the processing target image. A plurality of feature amounts are extracted from the image. On the other hand, when a non-reference detection frame is used for the detection process, the feature amount extraction unit 13 resizes the non-reference detection frame with respect to the size-changed image obtained by resizing the processing target image. The size change detection frame obtained in this way is set, and a plurality of feature amounts are extracted from the image in the size change detection frame in the size change image. Hereinafter, the image in the reference detection frame and the image in the size change detection frame from which the feature amount is extracted may be collectively referred to as “in-frame image”.

  Here, since the size of the reference detection frame matches the reference size, the size of the image in the reference detection frame in the processing target image becomes the reference size. In addition, since the size of the size change detection frame matches the reference size, the size of the image in the size change detection frame in the size change image becomes the reference size. Therefore, the feature amount extraction unit 13 can always extract the feature amount from the image of the reference size. Note that the feature quantity extraction method in the feature quantity extraction unit 13 will be described in detail later.

  The discriminator 14 includes a feature vector composed of a plurality of feature amounts extracted from the in-frame image by the feature amount extraction unit 13 and a weight vector composed of a plurality of weight coefficients generated based on the learning sample (learning sample image). Based on the above, a real value indicating the probability that the in-frame image is a face image is calculated. Specifically, the feature amount extraction unit 13 obtains an inner product of the feature vector and the weight vector for the in-frame image, and adds a predetermined bias value to the inner product, and the value in the in-frame image is obtained. A real value indicating the certainty of the face image is used. Hereinafter, a real value indicating the likelihood of being a face image is referred to as a “detection accuracy value”. The detection accuracy value calculated by the discriminator 14 indicates the face image likelihood (face likelihood) of the image in the reference detection frame or the image in the size change detection frame. In the classifier 14, for example, SVM (Support Vector Machine) or Adaboost is used.

  If the calculated detection accuracy value is greater than or equal to the threshold value, the discriminator 14 determines that there is a high possibility that the in-frame image is a face image. That is, when the reference detection frame is used, the discriminator 14 determines that the image in the reference detection frame in the processing target image is a region that is highly likely to be a face image having the same size as the reference detection frame. To do. When the non-reference detection frame is used, the discriminator 14 is an area where the image in the size change detection frame in the size change image is likely to be a face image having the same size as the size change detection frame. Judge that there is.

  On the other hand, if the calculated detection accuracy value is less than the threshold, the discriminator 14 determines that there is a high possibility that the in-frame image is not a face image. That is, when the reference detection frame is used, the discriminator 14 determines that the image in the reference detection frame in the processing target image is not an area that is highly likely to be a face image having the same size as the reference detection frame. To do. In addition, when the non-reference detection frame is used, the discriminator 14 determines that the image in the size change detection frame in the size change image is likely to be a face image having the same size as the size change detection frame. Judge that there is no.

  When the discriminator 14 determines that the image in the reference detection frame in the processing target image is a region that is highly likely to be a face image having the same size as the reference detection frame, the image is used as a detection result region, and the reference detection is performed. Let the frame be the detection result frame.

  If the discriminator 14 determines that the image in the size change detection frame in the size change image is a region that is highly likely to be a face image having the same size as the size change detection frame, the identifier 14 temporarily detects the outer shape frame of the region. The result frame. Then, the discriminator 14 identifies a region that is highly likely to be a face image having the same size as the non-reference detection frame in the processing target image that is the original image of the resized image based on the temporary detection result frame, The area is set as a detection result area, and the outer frame of the detection result area is set as a final detection result frame.

<Detection process using reference detection frame>
Next, the detection unit 12 moves the reference detection frame relative to the processing target image, and determines whether the image in the reference detection frame is likely to be a face image. The operation will be described. 3-6 is a figure for demonstrating the said operation | movement of the detection part 12. FIG. The detection unit 12 detects a face image for an image in the reference detection frame while raster scanning the reference detection frame.

  As shown in FIG. 3, the feature amount extraction unit 13 first sets a reference detection frame 100 at the upper left of the processing target image 20, and extracts a plurality of feature amounts from the image in the reference detection frame 100. The discriminator 14 obtains a detection accuracy value for an image in the reference detection frame 100 based on a feature vector composed of a plurality of feature amounts extracted by the feature amount extraction unit 13 and a weight vector composed of a plurality of weight coefficients. . When the calculated detection accuracy value is equal to or greater than the threshold value, the classifier 14 determines that the region in the upper left reference detection frame 100 in the processing target image 20 is highly likely to be a face image. The region is set as a detection result region, and the reference detection frame 100 which is the outer frame of the region is set as a detection result frame.

  Next, the feature amount extraction unit 13 moves the reference detection frame 100 slightly to the right in the processing target image 20. For example, the feature amount extraction unit 13 moves the reference detection frame 100 to the right by one pixel or several pixels. Then, the feature quantity extraction unit 13 extracts a plurality of feature quantities from the image in the reference detection frame 100 after movement in the processing target image 20.

  Thereafter, the discriminator 14 uses the feature vector composed of a plurality of feature amounts extracted by the feature amount extraction unit 13 and the weight vector composed of a plurality of weight coefficients to determine the image in the reference detection frame 100 after movement. The detection accuracy value is obtained. When the calculated detection accuracy value is equal to or greater than the threshold value, the discriminator 14 determines that the image in the reference detection frame 100 after movement is highly likely to be a face image, and determines the image. In addition to the detection result area, the reference detection frame 100 after movement, which is the outer frame of the image, is used as the detection result frame.

  Thereafter, the detection unit 12 operates in the same manner, and as illustrated in FIG. 4, when the reference detection frame 100 moves to the right end of the processing target image 20, the detection unit 12 detects the image in the right end reference detection frame 100. The detection accuracy value is obtained. If the obtained detection accuracy value is equal to or greater than the threshold value, the detection unit 12 sets the image in the rightmost reference detection frame 100 as a detection result region, and uses the rightmost reference detection frame 100 as a detection result frame. To do.

  Next, as illustrated in FIG. 5, the feature amount extraction unit 13 moves the reference detection frame 100 to the left end of the processing target image 20 while slightly lowering the reference detection frame 100, and then performs a plurality of features from the image in the reference detection frame 100. Extract the amount. The feature amount extraction unit 13 moves the reference detection frame 100 downward by, for example, one pixel or several pixels in the vertical direction (column direction). Thereafter, the discriminator 14 uses the feature vector composed of a plurality of feature amounts extracted from the feature amount extraction unit 13 and the weight vector composed of a plurality of weight coefficients to determine the image in the current reference detection frame 100. Find and output the detection accuracy value. When the calculated detection accuracy value is equal to or greater than the threshold value, the classifier 14 determines that the image in the current reference detection frame 100 is likely to be a face image, and detects the image. In addition to the result area, the reference detection frame 100 is set as a detection result frame.

  Thereafter, the detection unit 12 operates in the same manner, and as shown in FIG. 6, when the reference detection frame 100 moves to the lower right of the processing target image 20, the detection unit 12 moves to the lower right of the reference detection frame 100. The detection accuracy value for the image is obtained. If the obtained detection accuracy value is equal to or greater than the threshold value, the detection unit 12 sets the image in the lower right reference detection frame 100 as a detection result region and uses the lower right reference detection frame as the detection result frame. And

  As described above, the detection unit 12 uses the reference detection frame to detect, as a detection result region, an area that is highly likely to be a face image having the same size as the reference detection frame in the processing target image. In other words, the detection unit 12 specifies a face image having the same size as the reference detection frame in the processing target image using the reference detection frame.

<Detection process using non-reference detection frame>
When the detection unit 12 performs the detection process using the non-reference detection frame, the feature amount extraction unit 13 is configured so that the size of the non-reference detection frame matches the reference size (the size of the reference detection frame). The non-reference detection frame is resized. Then, the feature amount extraction unit 13 changes the size of the processing target image by the same amount as the size change ratio for the non-reference detection frame.

  In this embodiment, since the reference size is 16p × 16p, for example, when a non-reference detection frame having a size of Rp × Rp (R> 16) is used, the feature amount extraction unit 13 The non-reference detection frame is reduced by multiplying the vertical width (vertical width) and horizontal width (horizontal width) of the reference detection frame by (16 / R), respectively, to generate a size change detection frame. Then, the feature amount extraction unit 13 reduces the processing target image by multiplying the vertical width (number of pixels) and the horizontal width (number of pixels) of the processing target image by (16 / R), and generates a size-changed image. After that, the detection unit 12 moves the size change detection frame with respect to the size change image and moves the feature amount from the image in the size change detection frame in the same manner as the process described with reference to FIGS. Based on the extracted feature amount, it is determined whether there is a high possibility that the image in the size change detection frame is a face image having the same size as the size change detection frame. That is, using the size change detection frame, the detection unit 12 performs processing for detecting a region that is highly likely to be a face image having the same size as the size change detection frame in the size change image. Hereinafter, this process is referred to as a “size-changed version detection process”. Further, the reference detection frame and the size change detection frame used for extracting the feature amount from the in-frame image are collectively referred to as “feature amount extraction frame”. The feature amount extraction frame does not include the non-reference detection frame before the size change.

  In the size change version detection process, the detection unit 12 sets a size change detection frame for the size change image, and the image in the size change detection frame may be a face image having the same size as the size change detection frame. If it is determined that the property is high, the size change detection frame that is the outer frame of the image is set as a temporary detection result frame.

  In the detection unit 12, when at least one temporary detection result frame is obtained for the size-changed image, the discriminator 14 determines the at least one temporary detection result frame according to the processing target image that is the source of the size-changed image. Convert to detection result frame.

  Specifically, the classifier 14 first sets at least one obtained temporary detection result frame for the resized image. FIG. 7 is a diagram illustrating a state in which the temporary detection result frame 110 is set for the size-changed image 120. In the example of FIG. 7, a plurality of temporary detection result frames 110 are set for the resized image 120.

  Next, as shown in FIG. 8, the classifier 14 processes the resized image 120 by enlarging (resizing) the resized image 120 in which the temporary detection result frame 110 is set and returning it to the original size. The target image 20 is converted. As a result, the temporary detection result frame 110 set in the size-changed image 120 is also enlarged, and the temporary detection result frame 110 is converted into a detection result frame 150 corresponding to the processing target image 20, as shown in FIG. The A region in the detection result frame 150 in the processing target image 20 is a detection result region that is highly likely to be a face image having the same size as the non-reference detection frame in the processing target image 20. Thereby, the detection unit 12 has a detection result region that is highly likely to be a face image having the same size as the non-reference detection frame in the processing target image, based on the temporary detection result frame 110 obtained by the size-changed version detection process. Identified.

  As described above, when the detection unit 12 performs the detection process on the processing target image using the non-reference detection frame, the non-reference detection frame that has been resized so that the size matches the reference size, The size-changed version detection process is performed using the processing target image whose size has been changed according to the size change of the reference detection frame. Thereby, even when a detection frame having a size different from the reference size is used, the feature amount extraction unit 13 can extract the feature amount from the image of the reference size. And the detection part 12 pinpoints the detection result area | region with high possibility that it is a face image of the same size as a non-reference | standard detection frame in a process target image based on the result of a size-change version detection process. Accordingly, the detection unit 12 performs detection processing using the non-reference detection frame.

  The detection unit 12 performs the detection process as described above using each of a plurality of types of detection frames. Thereby, when a face image is included in the processing target image, a detection result area (area that is highly likely to be a face image) and a detection result frame (outer frame of an area that is likely to be a face image) Is obtained, and a detection accuracy value corresponding to the detection result frame is obtained. The detection accuracy value corresponding to the detection result frame obtained for the processing target image indicates the probability that the image in the detection result frame in the processing target image is a face image.

  FIG. 9 is a diagram illustrating a state in which the detection result frame 150 obtained for the processing target image 20 is arranged so as to overlap the processing target image 20. As shown in FIG. 9, detection results frames 150 having various sizes are obtained by performing detection processing using a plurality of types of detection frames having different sizes. This means that face images of various sizes included in the processing target image 20 are detected.

<Feature extraction process>
Next, the operation of the feature amount extraction unit 13 will be described in detail. FIG. 10 is a diagram illustrating a configuration of the feature amount extraction unit 13. As shown in FIG. 10, the feature amount extraction unit 13 includes an evaluation value acquisition unit 130 and a feature amount acquisition unit 131.

  The evaluation value acquisition unit 130 generates an evaluation value map including a plurality of evaluation values arranged in a matrix form from the processing target image. The feature amount acquisition unit 131 acquires the feature amount of the face image using the evaluation value map. Here, the evaluation value is a value indicating the relationship between the pixel value of the target pixel and a plurality of pixel values around the target pixel. Hereinafter, the pixel value of the target pixel may be referred to as a “target pixel value”. In addition, pixel values around the pixel of interest may be referred to as “ambient pixel values”. In the present embodiment, for example, peripheral pixel values in eight directions around the target pixel are used to acquire the evaluation value.

<Method for generating evaluation value map>
The evaluation value acquisition unit 130 generates an evaluation value map corresponding to the detection frame for each of a plurality of types of detection frames used in the detection process. Hereinafter, the evaluation value map corresponding to the reference detection frame is referred to as a “reference evaluation value map”. The evaluation value map corresponding to the non-reference detection frame is referred to as a “non-reference evaluation value map”.

  Here, as will be described later, in the feature amount extraction for the processing target image, (M−2) p × (N−2) p, which is smaller than the original size (Mp × Np) by one surrounding pixel. A processing target image is used. That is, in the detection processing described with reference to FIGS. 3 to 6 described above, a processing target image that is smaller than the original size by one surrounding pixel is used. This processing target image is particularly referred to as an “extraction processing target image”. The processing target image 20 shown in FIGS. 3 to 6 is actually an “extraction processing target image”. In addition, when a feature amount for a size-changed image is extracted, a size-changed image that is smaller than the original size by one surrounding pixel is used. Hereinafter, this size-changed image is particularly referred to as “extraction size-changed image”.

  The reference evaluation value map is generated from the processing target image. In the reference evaluation value map, (M−2) evaluation values are arranged in the row direction and (N−2) evaluation values are arranged in the column direction, as in the extraction processing target image. −2) × (N−2)) multiple evaluation values. The plurality of evaluation values constituting the reference evaluation value map respectively correspond to the plurality of pixels constituting the extraction processing target image. Specifically, the m × n-th evaluation value in the reference evaluation value map corresponds to the m × n-th pixel in the extraction processing target image. Each evaluation value in the reference evaluation value map indicates the relationship between the pixel value of the target pixel and a plurality of surrounding pixel values when the corresponding pixel is the target pixel.

  In addition, a plurality of evaluation values arranged in a matrix that form a non-reference evaluation value map corresponding to a non-reference detection frame is a size-changed image corresponding to the non-reference detection frame (the size of the non-reference detection frame). The processing target image whose size has been changed by the same ratio as the change ratio) corresponds to each of a plurality of pixels constituting an extraction size-changed image obtained by reducing the size by one surrounding pixel. The arrangement of the plurality of evaluation values in the non-reference evaluation value map is the same as the arrangement of the plurality of pixels constituting the extraction size-changed image. The m × n-th evaluation value in the non-reference evaluation value map corresponds to the m × n-th pixel in the extraction size-changed image. Each evaluation value in the non-reference evaluation value map indicates a relationship between the pixel value of the target pixel and a plurality of surrounding pixel values when the corresponding pixel is the target pixel.

  When generating the reference evaluation value map, the evaluation value acquisition unit 130 sets a calculation frame 200 having a size of 3p × 3p on the upper left of the processing target image 20, as shown in FIG. Then, the evaluation value acquisition unit 130 sets the central pixel of the nine pixels in the calculation frame 200 as the target pixel.

  Next, the evaluation value acquisition unit 130 obtains an evaluation value indicating the relationship between the pixel value of the target pixel and the eight surrounding pixel values around the target pixel. The evaluation value acquisition unit 130 uses the eight surrounding pixel values as the pixel value of the upper left pixel, the pixel value of the upper right pixel, the pixel value of the upper right pixel, the pixel value of the right pixel, , The pixel value of the pixel immediately below, the pixel value of the lower left pixel, and the pixel value of the left pixel are used. In the present embodiment, the evaluation value is represented by 8 bits. Then, the evaluation value acquisition unit 130 sets the obtained evaluation value to 1 in the reference evaluation value map corresponding to the central pixel in the calculation frame 200, that is, the 1 × 1 pixel of the extraction processing target image. X First value. As the evaluation value, for example, LBP or LTP can be used. The method for obtaining LBP and LTP will be described in detail later.

  FIG. 12 is a diagram illustrating an example of pixel values for nine pixels in the calculation frame 200. In the present embodiment, the pixel value used for obtaining the evaluation value is, for example, luminance. In the present embodiment, the pixel value is represented by 8 bits. Therefore, the pixel value takes a value from “0” to “255” in decimal. The pixel value may be a color difference component. In the following, values such as pixel values are assumed to be values expressed in decimal notation unless otherwise specified.

  In the example of FIG. 12, the target pixel value 210 is “57”. Also, the pixel value of the upper left pixel of the target pixel, the pixel value of the upper right pixel, the pixel value of the upper right pixel, the pixel value of the right pixel, the pixel value of the lower right pixel, the pixel value of the lower right pixel, the lower left pixel The pixel value of the pixel and the pixel value of the left pixel are “50”, “55”, “65”, “75”, “79”, “59”, “48”, and “49”, respectively. Yes.

  When the evaluation value acquisition unit 130 obtains an evaluation value corresponding to the calculation frame 200 at the upper left of the processing target image 20, the evaluation value acquisition unit 130 moves the calculation frame 200 to the right by one pixel in the processing target image 20. Then, the evaluation value acquisition unit 130 sets the central pixel of the nine pixels in the calculation frame 200 after movement as a target pixel, and calculates an evaluation value indicating the relationship between the target pixel value and the eight surrounding pixel values. Ask. The evaluation value acquisition unit 130 uses the calculated evaluation value of the reference evaluation value map corresponding to the center pixel in the calculation frame 200 after movement, that is, the 1 × 2 pixel of the extraction target image. The value is 1 × 2nd.

  Next, the evaluation value acquisition unit 130 moves the calculation frame 200 further to the right by one pixel in the processing target image 20. Then, the evaluation value acquisition unit 130 sets the central pixel of the nine pixels in the calculation frame 200 after movement as a target pixel, and calculates an evaluation value indicating the relationship between the target pixel value and the eight surrounding pixel values. Ask. The evaluation value acquisition unit 130 uses the calculated evaluation value of the reference evaluation value map corresponding to the center pixel in the calculation frame 200 after movement, that is, the 1 × 3rd pixel of the extraction target image. The value is 1 × 3.

  Thereafter, the evaluation value acquisition unit 130 raster scans the calculation frame 200 pixel by pixel to the lower right of the processing target image 20, and determines the center pixel of the calculation frame 200 at each position of the calculation frame 200. An evaluation value is obtained as the target pixel. Thereby, a plurality of evaluation values respectively corresponding to a plurality of pixels constituting the extraction processing target image, that is, a plurality of pixels excluding one pixel around the processing target image, are generated, and are composed of the plurality of evaluation values. A reference evaluation value map is completed. Note that for each pixel around the processing target image, each pixel does not become a target pixel, and thus an evaluation value corresponding to each pixel cannot be obtained.

  When generating the non-reference evaluation value map corresponding to the non-reference detection frame, the evaluation value acquisition unit 130 supports the non-reference detection frame instead of setting the calculation frame 200 for the processing target image. A calculation frame 200 is set for the size-changed image. Then, in the same manner as described above, the evaluation value acquisition unit 130 obtains an evaluation value at each position of the calculation frame 200 while raster-scanning the calculation frame 200 pixel by pixel with respect to the size-changed image. As a result, a plurality of pixels constituting the extraction size-changed image corresponding to the non-reference detection frame, that is, a plurality of pixels respectively corresponding to a plurality of pixels excluding one peripheral pixel of the size-changed image corresponding to the non-reference detection frame. An evaluation value is generated, and a non-reference evaluation value map corresponding to the non-reference detection frame, which is composed of the plurality of evaluation values, is completed. The evaluation value acquisition unit 130 generates a non-reference evaluation value map for each of a plurality of types of non-reference detection frames. Note that, for each pixel around the size-changed image, each pixel does not become a target pixel, and thus an evaluation value corresponding to each pixel cannot be obtained.

<Specific examples of evaluation values>
Next, LBP and LTP used as evaluation values will be described.

<LBP>
When the LBP is generated, for each of a plurality of surrounding pixel values, 1 bit (hereinafter referred to as “relation display bit”) indicating the relationship between the surrounding pixel value and the target pixel value is generated. If the difference value obtained by subtracting the target pixel value from the surrounding pixel value is greater than or equal to zero, the value of the related display bit is “1”, and if it is less than zero, the value of the related display bit is “0”. Then, an 8-bit binary code (hereinafter referred to as “related display code”) composed of a plurality of related display bits obtained for a plurality of surrounding pixel values becomes LBP, and the LPB is used as an evaluation value. Specifically, a value representing the relation display code as the LBP in decimal is used as the evaluation value.

  For example, assume that a target pixel value 210 and a plurality of surrounding pixel values 220 as shown in FIG. 12 are obtained for a plurality of pixels in the calculation frame 200. The evaluation value acquisition unit 130 obtains a difference value 250 obtained by subtracting the target pixel value 210 from the surrounding pixel value 220 for each surrounding pixel value 220. As shown in FIG. 13, the difference values 250 between the surrounding pixel values 220 for the upper left, right above, right upper, right, lower right, right under, lower left, and left surrounding pixel values 220 are “−7”, respectively. , “−2”, “8”, “18”, “22”, “2”, “−9”, and “−8”. Then, the evaluation value acquisition unit 130 compares each of the obtained plurality of difference values 250 (eight difference values 250 in this example) with zero. When the difference value 250 for the surrounding pixel value 220 is greater than or equal to zero, the evaluation value acquisition unit 130 sets the value of the relationship display bit 260 for the surrounding pixel value 220 to “1”, and the difference value 250 is zero. If it is less, the value of the relation display bit 260 for the surrounding pixel value 220 is set to “0”. In the example of FIGS. 12 and 13, as shown in FIG. 14, the relationship display bits 260 for the surrounding pixel values 220 on the upper left, directly above, upper right, right, lower right, directly below, lower left, and left are respectively “0”. , “0”, “1”, “1”, “1”, “1”, “0”, “0”. Then, the evaluation value acquisition unit 130 generates an 8-bit relation display code as an LBP by arranging the plurality of relation display bits 260 obtained for the plurality of surrounding pixel values 220 in a predetermined order. When the evaluation value acquisition unit 130 obtains the relationship display code, the evaluation value acquisition unit 130 sets a value representing the relationship display code in decimal as an evaluation value corresponding to the target pixel. In the present embodiment, for example, the relationship display bit 260 of the upper left surrounding pixel value 220, the relationship display bit 260 of the upper right surrounding pixel value 220, the relationship display bit 260 of the upper right surrounding pixel value 220, and the right surrounding pixel value 220, a relation display bit 260 of the lower right surrounding pixel value 220, a relation display bit 260 of the lower right surrounding pixel value 220, a relation display bit 260 of the lower left surrounding pixel value 220, and a left surrounding pixel value 220. The relationship display bits 260 are arranged in the order of the relationship display bits 260, and the relationship display code is generated. In the example of the pixel value in FIG. 12, as shown in FIG. 14, “00111100” is the relation display code, and the value “60” expressed in decimal is the evaluation value corresponding to the target pixel.

  As can be understood from the above description, it can be said that the LBP indicates the state (distribution state) of the pixel value around the pixel of interest. Therefore, it can be said that LBP indicates a local texture. Hereinafter, an evaluation value map in which each evaluation value is an LBP (more precisely, a value represented by a decimal number) is referred to as an “LBP map”.

  FIG. 15 is a diagram illustrating an example of the processing target image 20. FIG. 16 shows a grayscale LBP map image 30 obtained by imaging each LBP map by using each evaluation value in the LBP map generated based on the processing target image 20 shown in FIG. 15 as luminance. FIG. From the LBP map image 30 shown in FIG. 16, the texture for the processing target image 20 can be read.

<LTP>
When the LTP is generated, unlike the LBP, ternary data indicating the relationship between the surrounding pixel value and the target pixel value is generated for each of the plurality of surrounding pixel values. Here, in LTP, in order to suppress the influence of noise, an offset value is used when ternary data indicating the relationship between the surrounding pixel value and the target pixel value is generated.

  Specifically, when the absolute value of the difference value between the surrounding pixel value and the target pixel value is less than a predetermined offset value, the value of the ternary data is set to “0”. When the absolute value of the difference value between the surrounding pixel value and the target pixel value is equal to or larger than the offset value and the surrounding pixel value is larger than the target pixel value, the value of the ternary data is “1”. Is done. That is, when the surrounding pixel value is equal to or greater than the value obtained by adding the offset value to the target pixel value, the value of the ternary data is “1”. When the absolute value of the difference value between the surrounding pixel value and the target pixel value is equal to or larger than the offset value and the surrounding pixel value is smaller than the target pixel value, the value of the ternary data is “−1”. It is said. That is, when the surrounding pixel value is equal to or smaller than the value obtained by subtracting the offset value from the target pixel value, the value of the ternary data is “−1”. As described in Non-Patent Document 1, the offset value is set to “5”, for example.

  As described above, even when at least one of the surrounding pixel value and the target pixel value is affected by noise, the offset value is provided when the magnitude relationship between the surrounding pixel value and the target pixel value is determined. , It can be suppressed that the magnitude relationship is erroneously determined.

  LTP is a ternary code (ternary code) obtained by arranging a plurality of ternary data obtained for a plurality of surrounding pixel values in a predetermined order. For example, the upper left surrounding pixel value ternary data, the upper right surrounding pixel value ternary data, the upper right surrounding pixel value ternary data, the right surrounding pixel value ternary data, the lower right surrounding pixel value The ternary code obtained by arranging the ternary data, the ternary data of the immediately lower surrounding pixel value, the ternary data of the lower left surrounding pixel value, and the ternary data of the left surrounding pixel value is LTP.

  FIG. 17 shows a plurality of ternary data 270 respectively corresponding to the plurality of surrounding pixel values 220 when the target pixel value 210 and the plurality of surrounding pixel values 220 as shown in FIG. 12 are obtained. FIG. In the example of FIG. 15, LTP is (−1) 01110 (−1) (− 1).

  When LTP is used as an evaluation value, LTP is not used as it is, but either positive LTP or negative LTP obtained from LTP is used.

  The positive LTP is obtained by converting the LTP into a binary code focusing on only “1” included in the LTP. Specifically, all values other than “1” in LTP are converted to “0”, and the binary code obtained thereby becomes positive LTP. FIG. 18 is a diagram showing a positive LTP corresponding to the LTP shown in FIG. In the LTP shown in FIG. 17, that is, (-1) 01110 (-1) (-1), when all values other than "1" are converted to "0", as shown in FIG. A positive LTP is obtained. The 8 bits constituting the positive LTP are the upper left surrounding pixel value, the upper right surrounding pixel value, the upper right surrounding pixel value, the right surrounding pixel value, the lower right surrounding pixel value, and the immediately below surrounding pixel value in order from the top. , Respectively corresponding to the lower left surrounding pixel value and the left surrounding pixel value.

  As can be understood from the above description, each bit of the positive LTP indicates the relationship between the surrounding pixel value corresponding to the bit and the target pixel value. Each bit of the positive LTP indicates “1” if the surrounding pixel value corresponding to the positive LTP is equal to or greater than the value obtained by adding only the offset value to the target pixel value, and indicates “0” if it is less than the value. ". Therefore, it can be said that the positive LTP indicates a state (distribution state) of a pixel value larger than the pixel value of the target pixel around the target pixel. Therefore, it can be said that the positive LTP also shows a local texture like the LBP. The evaluation value acquisition unit 130 uses a value representing the positive LTP in decimal as an evaluation value. In the example of FIG. 18, the value “56” representing the binary code “00111000” in decimal is used as the evaluation value. Hereinafter, an evaluation value map having positive LTPs (more precisely, values expressed in decimal) as evaluation values is referred to as a “positive LTP map”.

  On the other hand, the negative LTP is obtained by converting the LTP into a binary code focusing on only “−1” included in the LTP. Specifically, all values other than “−1” in LTP are converted to “0” and “−1” is converted to “1”, and the resulting binary code becomes a negative LTP. FIG. 19 is a diagram showing a negative LTP corresponding to the LTP shown in FIG. In the LTP shown in FIG. 17, that is, (-1) 01110 (-1) (-1), all values other than "-1" are converted to "0" and "-1" is converted to "1". Then, as shown in FIG. 19, a negative LTP of “10000011” is obtained. The 8 bits constituting the negative LTP are the upper left surrounding pixel value, the upper right surrounding pixel value, the upper right surrounding pixel value, the right surrounding pixel value, the lower right surrounding pixel value, and the immediately below surrounding pixel value in order from the top. , Respectively corresponding to the lower left surrounding pixel value and the left surrounding pixel value.

  As can be understood from the above description, for each bit of the negative LTP, the relationship between the surrounding pixel value corresponding to the bit and the target pixel value is shown. Each bit of the negative LTP indicates “1” if the surrounding pixel value corresponding to the bit is equal to or less than the value obtained by subtracting the offset value from the target pixel value, and indicates “0” if the value is larger than the value. Show. Therefore, it can be said that the negative LTP indicates a state (distribution state) of a pixel value smaller than the pixel value of the target pixel around the target pixel. Therefore, it can be said that the negative LTP also shows a local texture like the LBP and the positive LTP. The evaluation value acquisition unit 130 uses a value representing the negative LTP in decimal as an evaluation value. In the example of FIG. 19, the value “131” representing the binary code “10000011” in decimal is used as the evaluation value. Hereinafter, an evaluation value map having negative LTPs (more precisely, values expressed in decimal) as evaluation values is referred to as a “negative LTP map”.

  FIG. 20 shows a grayscale positive LTP map image obtained by imaging each positive evaluation value in the positive LTP map generated based on the processing target image 20 shown in FIG. It is a figure which shows 40p. FIG. 21 shows a grayscale negative LTP map image obtained by imaging each negative LTP map by setting each evaluation value in the negative LTP map generated based on the processing target image 20 shown in FIG. 15 as luminance. It is a figure which shows 40n. From the positive LTP map image 40p shown in FIG. 21 and the negative LTP map image 40n shown in FIG. 21, the texture of the processing target image 20 can be read.

  Hereinafter, when there is no need to particularly distinguish LBP, negative LTP, and positive LTP, each may be referred to as a “texture expression code”.

<About uniform>
As described above, in the present embodiment, the evaluation value is represented by 8 bits, and thus can take 256 types of values from 0 to 255.

  On the other hand, as can be understood from the description below, the number of feature quantities extracted from the in-frame image, that is, the number of dimensions of the feature vector extracted from the in-frame image, is the kind of value that the evaluation value can take. Depends on the number. Therefore, in order to reduce the processing amount in the detection unit 12, it is effective to limit the number of types of values that the evaluation value can take.

  Therefore, using the concept of uniform, which is sometimes used when obtaining LBP and LTP, it is considered to limit (reduce) the number of types of values that an evaluation value can take. In the following, the limitation on the number of types of values that the evaluation value can take using uniform will be described.

  First, the number of bit changes (bit inversion) when the 8 bits constituting the texture expression code such as LBP are viewed in order is obtained. The direction in which the 8 bits constituting the texture expression code are viewed in order may be from the high order to the low order, or from the low order to the high order. When the number of bit changes obtained for the texture expression code is 2 or less, the texture expression code is assumed to be uniform. On the other hand, when the number of bit changes obtained for the texture expression code exceeds two, that is, three or more times, the texture expression code is not uniform.

  With respect to the texture expression code determined to be uniform, a value representing the texture expression code in decimal is used as the evaluation value.

  On the other hand, the texture expression code that is not uniform is not used as an evaluation value as it is generated based on the target pixel value and surrounding pixel values affected by noise. The texture expression code determined not to be uniform is converted into a specific binary code of 8 bits, and a value representing the specific binary code in decimal is used as an evaluation value. The specific binary code may be anything as long as it is an 8-bit binary code other than the uniform 8-bit binary code. For example, “10101010” is adopted as the specific binary code.

  For example, it is assumed that the texture expression code is “01000000”. For example, when “01000000” is viewed in order from the high order, it changes from “0” to “1” between the first and second bits from the high order, and the second and third bits from the high order. In the meantime, it changes from “1” to “0”. Therefore, since the bit changes twice, “01000000” becomes uniform, and the value “64” representing “01000000” in decimal is used as the evaluation value.

  Further, it is assumed that the texture expression code is “00001111”. When viewed in order from the higher order, “0” changes to “1” between the fourth and fifth bits from the upper order, and the bit changes once. Therefore, “00001111” becomes uniform, and a value “15” representing “00001111” in decimal is an evaluation value.

  Further, it is assumed that the texture expression code is “00110011”. When viewed in order from the high order, it changes from “0” to “1” between the second and third bits from the high order, and from “1” to “1” between the fourth and fifth bits from the high order. It changes to “0”, and changes from “0” to “1” between the 6th and 7th bits from the top. Therefore, since the bit change is 3 times, “00110011” is not uniform. Therefore, “00110011” is converted into a specific binary code “10101010”, and a value “170” representing “10101010” in decimal is used as an evaluation value.

  Further, it is assumed that the texture expression code is “01010101”. Looking at it in turn, the bit change is 7 times, so that “01010101” is not uniform. Therefore, “01010101” is converted into a specific binary code “10101010”, and a value “170” representing “10101010” in decimal is used as an evaluation value.

  In this way, when the number of bit changes when the 8 bits constituting the texture expression code are viewed in order exceeds two, the texture expression code is converted into a specific binary code, and the specific binary code is converted. By using a value representing a code in decimal as an evaluation value, the types of values that the evaluation value can take are limited from 256 types to 59 types.

  Evaluation value acquisition section 130 according to the present embodiment uses uniform to limit the types of values that evaluation values can take to 59, in order to reduce the number of dimensions of feature vectors.

  For convenience of explanation, numbers 0 to 58 are assigned to 59 types of values that can be evaluated. Hereinafter, this embodiment may be described using this number.

<About pixel value interpolation>
In order to improve the accuracy of the evaluation value, it is preferable to use the surrounding pixel values at a plurality of surrounding positions having the same distance from the target pixel for the plurality of surrounding pixel values used for obtaining the evaluation value. .

  On the other hand, in the generation of the LBP, “distance from the point of interest” is used as a parameter indicating how far the surrounding pixel value is compared with the target pixel value up to a range away from the target pixel. When the distance from the point of interest is “1”, as described above, eight surrounding pixel values are used to obtain the LBP. The distance 1 indicates the vertical and horizontal distances between pixels. The same applies to LTP.

  FIG. 22 is a diagram showing a positional relationship between the target pixel 300 and the eight surrounding pixels 310a to 310h in the calculation frame 200. As shown in FIG. 22, in the nine pixels in the calculation frame 200, the distance a between the target pixel 300 and the surrounding pixels in the vertical direction or the horizontal direction, and the target pixel 300 and the surrounding pixels in the diagonal direction. Is different from the distance b.

  As described above, when the pixel values of the eight peripheral pixels 310a to 310h existing around the target pixel 300 are used as the eight peripheral pixel values used when obtaining the evaluation value, the target pixel and the peripheral pixels The distance between the position corresponding to the value is “1” in the vertical direction and the horizontal direction, but is not “1” in the oblique direction. Therefore, in this case, the plurality of surrounding pixel values used for obtaining the evaluation value are not the surrounding pixel values at the plurality of surrounding positions having the same distance from the target pixel.

  Therefore, as for the surrounding pixel values in the oblique direction, the pixel values of the surrounding pixels in the oblique direction are not used, but at the surrounding position 400 in the oblique direction where the distance from the target pixel is “1” (see FIG. 22). It is desirable to use pixel values.

  The pixel value at the peripheral position 400 in the oblique direction whose distance from the target pixel is “1” can be obtained by pixel value interpolation processing such as bilinear interpolation processing.

  FIG. 23 is a diagram for explaining a method of obtaining the pixel value at the peripheral position 400 in the upper right direction where the distance from the target pixel is “1” using bilinear interpolation processing. In the example of FIG. 23, the pixel value of the target pixel 300, the pixel value of the surrounding pixel 310b in the upper direction, the pixel value of the surrounding pixel 310c in the upper right direction, and the pixel value of the surrounding pixel 310d in the right direction are “57” and “ 55, “65” and “75”. Further, the pixel value of the pixel 320a existing in the upper direction of the surrounding pixel 310c is “50”, and the pixel value of the pixel 320b existing in the right direction of the surrounding pixel 310c is “70”.

  In the example of FIG. 23, the ratio of the vertical distance between the target peripheral position 400 and the pixel 320a and the vertical distance between the peripheral position 400 and the peripheral pixel 310d is y1: y2. Then, the interpolation value Y0 in the vertical direction is obtained using the following equation (1).

  When the ratio of the distance in the left-right direction between the surrounding position 400 and the surrounding pixel 310b and the distance in the left-right direction between the surrounding position 400 and the pixel 320b is x1: x2, the following formula ( 2) is used to determine the left-right direction interpolation value X0.

  Then, the pixel value Z0 at the peripheral position 400 is obtained using the following equation (3).

  The pixel value at the upper left peripheral position where the distance from the target pixel is “1”, the pixel value at the lower left peripheral position where the distance from the target pixel is “1”, and the distance from the target pixel is “1” The pixel value at the peripheral position in the lower right direction that is “can be obtained in the same manner.

  When obtaining the evaluation value, the pixel value at the peripheral position in the diagonal direction where the distance from the target pixel is “1” is used as the peripheral pixel value, not the pixel value of the peripheral pixel existing in the diagonal direction of the target pixel. By using it, a more accurate evaluation value can be obtained. The evaluation value acquisition unit 130 may use the pixel values of the surrounding pixels in the oblique direction as the surrounding pixel values, or use the pixel values at the surrounding position in the oblique direction whose distance from the target pixel is “1”. You may do it.

<Description of operation of feature quantity acquisition unit>
Next, the operation of the feature amount acquisition unit 131 will be described in detail. The feature amount acquisition unit 131 acquires a feature amount for each of a plurality of types of detection frames using the evaluation value map generated by the evaluation value acquisition unit 130 corresponding to the detection frame. Further, when the feature amount acquisition unit 131 acquires the feature amount of the processing target image, the feature amount acquisition unit 131 uses an extraction processing target image that is smaller than the original size by one surrounding pixel. Further, when the feature amount acquisition unit 131 acquires a feature amount for the size-changed image, the feature-size acquisition unit 131 uses an extraction size-changed image that is smaller than the original size by one surrounding pixel. Hereinafter, the extraction processing target image and the extraction size-changed image used for acquiring the feature amount may be collectively referred to as “extraction target image”.

  In the present embodiment, for example, two types of feature values are used. Specifically, the frequency (frequency) of the evaluation value within the feature amount extraction frame set in the evaluation value map is used as the feature amount. That is, the frequency (frequency) of each of 59 types of values that can be taken as evaluation values within the feature value extraction frame set in the evaluation value map is used as the feature value. Hereinafter, this feature amount is referred to as a “first feature amount”.

  Furthermore, in the present embodiment, the co-occurrence frequency (co-occurrence frequency) of evaluation values within the feature amount extraction frame set in the evaluation value map is used as the feature amount. That is, the combination amount (frequency) of the possible values of the evaluation value in the feature amount extraction frame set in the evaluation value map is used as the feature amount. Hereinafter, this feature amount is referred to as a “second feature amount”. It can be said that the second feature amount indicates the co-occurrence of the evaluation value. The feature amount acquisition unit 131 inputs a feature vector composed of both the first and second feature amounts acquired from the in-frame image to the discriminator 14. Hereinafter, how to obtain the first and second feature amounts will be described.

<About the first feature>
When the reference detection frame is set at a position of the extraction processing target image as shown in FIGS. 3 to 6, the feature amount acquisition unit 131 generates the reference evaluation value generated by the evaluation value acquisition unit 130. A reference detection frame is set at the same position as the certain position with respect to the map. Then, the feature amount acquisition unit 131 generates a one-dimensional evaluation value histogram indicating the frequency distribution (frequency distribution) of evaluation values within the reference detection frame set in the reference evaluation value map.

  FIG. 24 is a diagram illustrating an example of a one-dimensional evaluation value histogram. The horizontal axis of FIG. 24 shows numbers 0 to 58 assigned to 59 types of evaluation values. The vertical axis in FIG. 24 indicates the frequency of evaluation values having the number values shown on the horizontal axis in a plurality of evaluation values within the reference detection frame set in the reference evaluation value map. In the present embodiment, since there are 59 types of values that can be taken as evaluation values, the one-dimensional evaluation value histogram has 59 bins.

  When the feature quantity acquisition unit 131 generates a one-dimensional evaluation value histogram, each of the frequencies in the 59 bins in the one-dimensional evaluation value histogram is set for the image in the reference detection frame set as the extraction processing target image. The first feature value is used. As a result, 59 first feature values are extracted from the image within the reference detection frame set as the extraction target image.

  When the reference detection frame is raster scanned with respect to the extraction processing target image, the feature amount acquisition unit 131 uses the reference evaluation value map as a reference at each position of the reference detection frame as described above. 59 first feature amounts are acquired from the image within the detection frame.

  The feature amount acquisition unit 131 acquires the first feature amount in the same manner even when the non-reference detection frame is used. When the size change detection frame (feature amount extraction frame) is set at a position of the extraction size change image corresponding to the non-reference detection frame, the feature amount acquisition unit 131 corresponds to the non-reference detection frame. For the non-reference evaluation value map generated by the value acquisition unit 130, the size change detection frame is set at the same position as the certain position. Then, the feature quantity acquisition unit 131 generates a one-dimensional evaluation value histogram indicating the frequency distribution of evaluation values within the size change detection frame set in the non-reference evaluation value map.

  When the feature quantity acquisition unit 131 generates a one-dimensional evaluation value histogram, the frequency in 59 bins in the one-dimensional evaluation value histogram is set for each image in the size change detection frame set as the extraction size change image. The first feature amount. As a result, 59 first feature amounts are extracted from the image within the size change detection frame set in the size change image for extraction.

  The feature amount acquisition unit 131 rasterizes the non-reference evaluation value map as described above at each position of the size change detection frame when raster-scanning the size change detection frame with respect to the size change image for extraction. It is used to obtain 59 first feature values from the image within the size change detection frame. For each of a plurality of types of non-reference detection frames, the feature amount acquisition unit 131 raster-scans the size change detection frame corresponding to the non-reference detection frame with respect to the extracted size-changed image corresponding to the non-reference detection frame. However, 59 first feature values are acquired from the image in the size change detection frame using the non-reference evaluation value map corresponding to the non-reference detection frame.

  Each of the 59 first feature values (frequency in 59 bins of the one-dimensional evaluation value histogram) extracted from the in-frame image may be normalized using the following equation (4). By normalizing the first feature value, the first feature value is less affected by the change in the imaging environment of the imaging device 5.

  Here, v indicates the first feature value before normalization, and V indicates the first feature value after normalization. Further, k indicates the number of the first feature amounts extracted from the in-frame image, and k = 59 in the present embodiment. And v (i) represents the first feature value before normalization of the i-th number when the first to k-th numbers are assigned to the k first feature values. Note that ε is a constant provided so that v is not divided by zero in the expression on the right side of Expression (4).

<About the second feature>
When the reference detection frame is set at a certain position of the extraction processing target image, the feature amount acquisition unit 131 has the same position as the certain position with respect to the reference evaluation value map generated by the evaluation value acquisition unit 130. Set the reference detection frame to. Then, the feature amount acquisition unit 131 generates a two-dimensional evaluation value histogram indicating the distribution of evaluation value co-occurrence frequencies within the reference detection frame set in the reference evaluation value map.

  In the present embodiment, the frequency distribution of combinations of values that can be taken by two evaluation values in a predetermined relative positional relationship within the reference detection frame (feature amount extraction frame) set in the reference evaluation value map is obtained. A two-dimensional evaluation value histogram is generated. Hereinafter, a pair of two evaluation values having a predetermined relative positional relationship is referred to as an “evaluation value pair”. In this embodiment, a two-dimensional evaluation value histogram is generated for each of a plurality of types (plural sets) of evaluation value pairs having different relative positional relationships between the two evaluation values constituting the evaluation value pair. The In the present embodiment, for example, 30 types of evaluation value pairs are used. Hereinafter, in the evaluation value pair, one evaluation value is referred to as a “first evaluation value”, and the other evaluation value is referred to as a “second evaluation value”.

  FIG. 25 is a diagram illustrating an example of 30 types of evaluation value pairs. FIG. 25 shows a relative positional relationship between the first evaluation value 500 and the second evaluation value 510 within the feature amount extraction frame. In FIG. 25, white circles indicate the first evaluation value 500. In FIG. 25, black circles indicate the second evaluation values 510, and 30 types of second evaluation values 510 having different positions within the feature amount extraction frame are illustrated.

  In the present embodiment, one type of evaluation value pair is formed by the first evaluation value 500 shown in FIG. 25 and one type of second evaluation value 510 shown in FIG. In FIG. 25, since there are 30 types of second evaluation values 510, 30 types of evaluation value pairs are formed. Hereinafter, the evaluation value pair to be described is referred to as “target evaluation value pair”.

  26 to 28 show two-dimensional evaluation value histograms indicating frequency distributions for combinations of values that can be taken by the target evaluation value pair in the reference detection frame 100 set by the feature amount acquisition unit 131 in the reference evaluation value map. It is a figure for demonstrating operation | movement of the said feature-value acquisition part 131 at the time of producing | generating. 26 to 28, the target evaluation value pair is configured by a first evaluation value 500 and a second evaluation value 510 that are adjacent to each other in the left-right direction.

  In the present embodiment, the feature amount acquisition unit 131 sets the evaluation value as the first evaluation value 500 in order from the upper left to the lower right (along the raster scan direction) in the reference detection frame 100, and the first evaluation value 500. The combination of the second evaluation value 510 and the first evaluation value 500 paired with each other is stored. Then, the feature quantity acquisition unit 131 generates a two-dimensional evaluation value histogram indicating the frequency distribution for the stored combinations of the plurality of sets. This point will be described in detail below.

  As shown in FIG. 26, the feature amount acquisition unit 131 first sets the first upper evaluation value in the reference detection frame 100 as the first evaluation value 500, and sets the evaluation value adjacent to the right as the second evaluation value 510. A combination of the evaluation value 500 and the second evaluation value 510 is stored.

  Next, as shown in FIG. 27, the feature quantity acquisition unit 131 sets the second evaluation value as the first evaluation value 500 when viewed from the upper left in the reference detection frame 100 in the right direction, and sets the evaluation value adjacent to the right as the first evaluation value. As the second evaluation value 510, a combination of the first evaluation value 500 and the second evaluation value 510 is stored.

  Thereafter, the feature quantity acquisition unit 131 sequentially sets the evaluation values in the reference detection frame 100 in the raster scan direction as the first evaluation value 500, and sets the evaluation value on the right side of the first evaluation value 500 as the second evaluation value. As each value 510, for each first evaluation value 500, a combination of the first evaluation value 500 and the second evaluation value 510 paired therewith is stored. In FIG. 28, the second evaluation value viewed from the lower right in the reference detection frame 100 in the left direction is the first evaluation value 500, and the evaluation value adjacent to the right is the second evaluation value 510. It is shown.

  Note that the feature amount acquisition unit 131 sets the evaluation value as the first evaluation value 500 in order from the upper left to the lower right in the reference detection frame 100, and the second evaluation value 510 and the first evaluation value 500 paired with the first evaluation value 500. When the second evaluation value 510 that is paired with the first evaluation value 500 cannot be determined when the combination of the first evaluation values 500 is stored, the first evaluation value 500 and the second evaluation that is paired with the first evaluation value 500 are stored. The combination with the value 510 is not stored. For example, in the example of FIGS. 26 to 28, when the lower right evaluation value in the reference detection frame 100 is the first evaluation value 500, the second evaluation value 510 that is paired with the first evaluation value 500 cannot be determined. A combination of the first evaluation value 500 and the second evaluation value 520 paired therewith is not stored.

  When the combination of the first evaluation value 500 and the second evaluation value 510 is completed, the feature amount acquisition unit 131, based on the stored combination of the plurality of combinations, the frequency distribution regarding the combination of values that the target evaluation value pair can take Is generated.

  FIG. 29 is a diagram illustrating an example of a two-dimensional evaluation value histogram for a target evaluation value pair. The first axis along the X direction in FIG. 29 shows numbers 0 to 58 assigned to 59 types of values that can be taken by the first evaluation value of the target evaluation value pair. In addition, the second axis along the Y direction in FIG. 29 shows numbers 0 to 58 assigned to 59 types of values that can be taken by the second evaluation value of the target evaluation value pair. The third axis along the Z direction in FIG. 29 is the first evaluation value having the number indicated by the first axis in the feature amount extraction frame set in the evaluation value map, and the second axis. The frequency of combination with the second evaluation value having the value of the indicated number is shown. That is, the third axis in FIG. 29 is the first evaluation value having the number indicated on the first axis and the number indicated on the second axis in a plurality of combinations stored for the target evaluation value pair. It shows how many combinations with the second evaluation value having the value of.

  For example, when there are eight combinations of the first evaluation value having a value of number 0 and the second evaluation value having a value of number 2 in a plurality of combinations stored for the target evaluation value pair, The frequency in the bin corresponding to the number 0 indicated on the first axis and the number 2 indicated on the second axis is “8”. Further, in the combination of a plurality of sets stored for the target evaluation value pair, when there are three combinations of the first evaluation value having the value of number 2 and the second evaluation value having the value of number 1, The frequency in the bin corresponding to the number 2 indicated on the first axis and the number 1 indicated on the second axis is “3”. In the present embodiment, the two-dimensional evaluation value histogram has 3481 (= 59 × 59) bins.

  As described above, the feature amount acquisition unit 131 generates a two-dimensional evaluation value histogram for each of the 30 types of evaluation value pairs. Thereby, 30 two-dimensional evaluation value histograms are generated.

  When the feature quantity acquisition unit 131 generates 30 two-dimensional evaluation value histograms, the feature amount acquisition unit 131 extracts, for each of the 30 two-dimensional evaluation value histograms, frequencies of 3481 bins in the two-dimensional evaluation histogram. The second feature amount for the image within the reference detection frame set as the processing target image. Accordingly, 104430 (= 3481 × 30) second feature values are extracted from the image within the reference detection frame set as the extraction processing target image.

  When the reference detection frame is raster scanned with respect to the extraction processing target image, the feature amount acquisition unit 131 uses the reference evaluation value map as a reference at each position of the reference detection frame as described above. 104430 second feature values are acquired from the image within the detection frame.

  The feature amount acquisition unit 131 acquires the second feature amount in the same manner even when the non-reference detection frame is used. When the size change detection frame is set at a position of the extraction size change image corresponding to the non-reference detection frame, the feature amount acquisition unit 131 generates the evaluation value acquisition unit 130 corresponding to the non-reference detection frame. The size change detection frame is set at the same position as the certain position for the non-reference evaluation value map. Then, the feature amount acquisition unit 131 generates a two-dimensional evaluation value histogram indicating the distribution of co-occurrence frequencies of evaluation values within the size change detection frame set in the non-reference evaluation value map for each of the 30 evaluation value pairs. Generate.

  When the feature quantity acquisition unit 131 generates 30 two-dimensional evaluation value histograms, for each of the 30 two-dimensional evaluation value histograms, the frequency of 3481 bins in the two-dimensional evaluation value histogram is calculated. The second feature amount of the image within the size change detection frame set in the size change image for extraction is used. As a result, 104430 second feature values are extracted from the image within the size change detection frame set in the size change image for extraction.

  The feature amount acquisition unit 131 rasterizes the non-reference evaluation value map as described above at each position of the size change detection frame when raster-scanning the size change detection frame with respect to the size change image for extraction. Using this, 104430 second feature values are acquired from the image within the size change detection frame. For each of a plurality of types of non-reference detection frames, the feature amount acquisition unit 131 raster-scans the size change detection frame corresponding to the non-reference detection frame with respect to the extracted size-changed image corresponding to the non-reference detection frame. However, 104430 second feature values are acquired from the image in the size change detection frame using the non-reference evaluation value map corresponding to the non-reference detection frame.

  As in the case of the first feature amount, each of the 104430 second feature amounts (frequency in 104430 bins of the two-dimensional evaluation value histogram) extracted from the in-frame image is expressed by the above equation (4). Normalization. As a result, the second feature amount is not easily affected by a change in the imaging environment of the imaging device 5. When the second feature value is normalized using Expression (4), v is the second feature value before normalization, and V is the second feature value after normalization. Further, k is the number of second feature values extracted from the in-frame image, and k = 104430. Then, v (i) is the second feature amount before normalization of the i-th number when the first to k-th numbers are assigned to the k second feature amounts.

<About feature vectors input to the classifier>
When the feature quantity acquisition unit 131 extracts 59 first feature quantities and 104430 second feature quantities from the in-frame image, a feature vector obtained by arranging these 104489 (= 59 + 104430) feature quantities in order. Is generated. The number of dimensions of the feature vector is “104489”. Then, the feature quantity acquisition unit 131 inputs the generated feature vector to the classifier 14. As described above, the discriminator 14 calculates a detection accuracy value indicating the likelihood that the in-frame image is a face image based on the input feature vector and the weight vector. The weight vector used in the discriminator 14 is generated based on a feature vector composed of 59 first feature values and 104430 second feature values extracted from the learning sample in the same manner as described above.

  As described above, the feature quantity extraction unit 13 according to the present embodiment acquires a feature vector composed of 104489 feature quantities from the in-frame image and inputs the feature vector to the discriminator 14.

  If the values that the evaluation value can take using uniform are not limited, the number of bins in the one-dimensional evaluation value histogram is 256, and the number of bins in the two-dimensional evaluation value histogram is 1966080 (= 256). × 256 × 30). Therefore, the dimension number of the feature vector in this case is 1966336 (= 256 + 1966080).

<Output value map generation processing>
The output value map generation unit 15 generates an output value map indicating the distribution in the processing target image with respect to the detection accuracy value indicating the likelihood (face image likelihood) as the face image based on the detection result in the detection unit 12. To do.

  Specifically, the output value map generation unit 15 arranges (M−2) values in the row direction and (N−2) values in the column direction, as in the extraction processing target image. Consider a map 600 consisting of a total of ((M−2) × (N−2)) values. Then, the output value map generation unit 15 sets one detection result frame for the processing target image as the target detection result frame, and maps the frame 610 having the same size as the target detection result frame to the map 600 at the same position as the target detection result frame. Set for. FIG. 30 is a diagram showing a state in which a frame 610 is set for the map 600.

  Next, the output value map generation unit 15 sets “0” for each value outside the frame 610 in the map 600, and for each value within the frame 610, a detection accuracy value (target detection result) corresponding to the target detection result frame. This is determined using a detection accuracy value obtained as a result of detecting a face image with respect to an image in the detection frame that is a frame. If the size of the object detection result frame is 16p × 16p, for example, there are 16 values in the frame 610, 16 in the row direction and 16 in the column direction, for a total of 256 values. If the size of the target detection result frame is, for example, 20p × 20p, there are a total of 400 values in the frame 610, 20 in the row direction and 20 in the column direction. FIG. 31 is a diagram for explaining a method of determining each value in the frame 610.

  The output value map generation unit 15 sets the value of the center 611 in the frame 610 as the detection accuracy value corresponding to the target detection result frame obtained by the detection unit 12. Then, the output value map generation unit 15 sets the other values in the frame 610 to the outside from the center 611 in the frame 610 according to the normal distribution curve with the value at the center 611 of the frame 610 as the maximum value. Is gradually reduced. Thereby, each of the plurality of values constituting the map 600 is determined, and the map 600 corresponding to the target detection result frame is completed.

  As described above, the output value map generation unit 15 generates a plurality of maps 600 respectively corresponding to a plurality of detection result frames for the processing target image. Then, the output value map generation unit 15 combines the plurality of generated maps 600 to generate an output value map.

  Specifically, the output value map generation unit 15 adds the m × n-th value of the plurality of generated maps 600, and uses the obtained addition value as the m × n-th detection accuracy value of the output value map. And In this way, the output value map generation unit 15 obtains each detection accuracy value constituting the output value map. Thereby, an output value map indicating the distribution of detection accuracy values in the processing target image is completed. The output value map is composed of ((M−2) × (N−2)) detection accuracy values, like the extraction target image. By referring to the output value map, it is possible to specify a region having a high likelihood of a face image in the processing target image. That is, the face image in the processing target image can be specified by referring to the output value map.

  FIG. 32 is a diagram showing an output value map for the processing target image 20 superimposed on the processing target image 20. FIG. 32 shows an output value map when a negative LTP map is used as the evaluation value map. In FIG. 32 and FIG. 33 to be described later, for easy understanding, the output value map is shown by dividing the magnitude of the detection accuracy value into, for example, five stages from the first stage to the fifth stage. In the output value maps shown in FIGS. 32 and 33, sand areas are hatched for areas belonging to the fifth stage where the detection accuracy value is the largest, and left for areas belonging to the second largest fourth stage. Rising hatching is shown. Also, in the output value maps in FIGS. 32 and 33, a region belonging to the third stage where the detection accuracy value is the third largest shows hatching that rises to the right, and belongs to the second largest second stage. For the region, vertical hatching is shown. In the output value maps shown in FIGS. 32 and 33, hatching is not shown for the region belonging to the first stage having the smallest detection accuracy value.

  In the output value map shown in FIG. 32, the detection accuracy value in the region corresponding to the face image in the processing target image 20 is high. This means that the face image included in the processing target image 20 is properly detected.

  In FIG. 33, unlike the present embodiment, only the first feature amount (frequency of each bin in the one-dimensional evaluation value map) is extracted from the in-frame image, and the discriminator 14 includes only the first feature amount. It is a figure which shows the output value map at the time of the feature vector being input superimposed on the process target image. FIG. 33 shows an output value map when a negative LTP map is used as the evaluation value map, as in FIG.

  As shown in FIG. 33, in the output value map obtained by performing face detection using only the first feature amount, the non-face image in the processing target image 20 has a large detection accuracy value. There are many areas corresponding to. Therefore, the possibility that a face image is erroneously detected increases.

  As described above, in the image detection apparatus 1 according to the present embodiment, the detection accuracy for the face image is higher than when the face detection is performed using only the first feature amount.

  When generating the output value map, the image detection device 1 specifies a face image in the processing target image based on the output value map. Specifically, the image detection apparatus 1 specifies a region where the detection accuracy value is equal to or greater than a threshold value in the output value map, and a region existing at the same position as the region in the processing target image is a face image. Certify. Then, when displaying the processing target image on the display device, the image detection device 1 surrounds the face image in the processing target image with a square frame or the like.

  Further, the image detection apparatus 1 may compare a face image registered in advance with a face image specified in the processing target image and determine whether or not they match. Then, when the face image registered in advance and the face image specified in the processing target image do not match, the image detection device 1 performs mosaic processing on the face image in the processing target image, The processing target image may be displayed on a display device. As a result, when the image detection apparatus 1 according to the present embodiment is used in a surveillance camera system, even when a face image of a neighbor's person is photographed by the surveillance camera, the face image cannot be recognized. can do. That is, a privacy mask can be realized.

  As described above, in the feature quantity extraction device 13 (feature quantity extraction unit 13) according to the present embodiment, the co-occurrence frequency for the evaluation value indicating the relationship between the target pixel value and a plurality of surrounding pixel values is used as the feature quantity. Therefore, it is possible to obtain a feature amount suitable for image detection such as face image detection. Therefore, as in the present embodiment, the feature amount extraction device 13 is used in the image detection device 1 that detects the detection target image from the processing target image, and the feature amount extraction device 13 is based on the feature amount extracted from the image. The determination accuracy can be improved by determining whether or not the image is highly likely to be a detection target image. Therefore, erroneous detection of the detection target image can be suppressed. That is, the detection accuracy for the detection target image is improved.

  Further, by detecting a face image using a feature quantity based on LBP, negative LTP, or positive LTP indicating the local texture of the image, only HOG (Histgrams of Oriented Gradients) feature quantity or Haar-like feature quantity is obtained. The detection accuracy for the face image can be improved as compared with the case of using.

  In the present embodiment, since the detection accuracy of the detection target image is improved by using the second feature amount, the first and second feature amounts are not normalized in order to reduce the processing amount. For this reason, even if the first and second feature quantities are easily affected by changes in the imaging environment, the detection accuracy for the detection target image can be maintained.

  In the above example, the feature vector input to the discriminator 14 includes both the first feature value and the second feature value. However, the first feature value may not be included. That is, the feature vector only needs to include at least the second feature amount (co-occurrence frequency of evaluation values).

  The feature vector includes at least two types of second feature values acquired from the LBP map, second feature values acquired from the positive LTP map, and second feature values acquired from the negative LTP map. Two feature quantities may be included. For example, the feature vector may include a second feature amount acquired from the LBP map, a second feature amount acquired from the positive LTP map, and a second feature amount acquired from the negative LTP map. The second feature value acquired from the positive LTP map and the second feature value acquired from the negative LTP map may be included.

  In addition, the feature vector may include other types of feature amounts such as an HOG feature amount and a Haar-like feature amount.

<Various modifications>
Hereinafter, various modified examples of the present embodiment will be described.

<First Modification>
When extracting feature values from the image within the frame, the image within the frame is divided into a plurality of blocks, the feature values are extracted individually from each block, and the feature values for the extracted plurality of blocks are included in the frame. A feature amount for an image may be used. Thereby, independent feature amounts can be extracted for each of a plurality of blocks constituting the in-frame image. Therefore, even when a part of the face is hidden, it is possible to appropriately detect the face image of the face from the processing target image. Hereinafter, this modification will be described by taking as an example a case where the in-frame image is divided into four blocks in a matrix.

  As shown in FIG. 34, when the feature quantity acquisition unit 131 sets a feature quantity extraction frame 710 at a certain position of the extraction target image 700, the feature quantity acquisition unit 131 sets 4 images in the feature quantity extraction frame 710 (in-frame images) in a matrix. The image is divided into two image blocks 720. In addition, when the feature amount acquisition unit 131 sets the feature amount extraction frame 710 at the same position as the feature amount extraction frame 710 set in the extraction target image 700 with respect to the evaluation value map 800, the region in the feature amount extraction frame 710 is displayed. Is divided into four evaluation value blocks 820 in a matrix. Then, for each of the four evaluation value blocks 820, the feature amount acquisition unit 131 uses the plurality of evaluation values included in the evaluation value block 820 as described above and the plurality of first feature amounts and the plurality of first values. 2 Find the feature quantity. The number of first feature values and the number of second feature values depends on the number of evaluation values included in the evaluation value block 820. Hereinafter, the plurality of first feature amounts and the plurality of second feature amounts obtained for one evaluation value block 820 are collectively referred to as a “feature amount group”.

  When the feature amount acquisition unit 131 obtains a feature amount group of the evaluation value block 820 for each of the plurality of evaluation value blocks 820, the feature amount acquisition unit 131 sets the feature amount group for the image block 720 at the same position as the evaluation value block 820. Feature amount. Thereby, the feature amount is extracted independently from each of the four image blocks 720 constituting the in-frame image. When the feature quantity acquisition unit 131 extracts the feature quantities from the four image blocks 720 constituting the in-frame image, the classifier 14 uses a vector composed of the feature quantities of the four image blocks 720 as a feature vector for the in-frame image. To enter. The discriminator 14 calculates a detection accuracy value indicating the likelihood that the in-frame image is a face image based on the input feature vector and weight vector.

<Second Modification>
As described above, by using the co-occurrence frequency of the evaluation value as a feature amount, it is possible to suppress erroneous detection of the detection target image. Therefore, in order to reduce the processing amount, the number of dimensions of the feature vector is reduced. Even so, the detection accuracy of the detection target image can be maintained.

  Therefore, in this modification, the evaluation value acquisition unit 130 does not use the surrounding pixel values in the oblique direction with respect to the target pixel when obtaining the evaluation value. For example, the evaluation value acquisition unit 130 does not use all the surrounding pixel values in the upper right, upper left, lower right, and lower left directions for the target pixel. Thereby, in the acquisition of the evaluation value, only the surrounding pixel value in the upward direction, the surrounding pixel value in the downward direction, the surrounding pixel value in the right direction, and the surrounding pixel value in the left direction are used. It is expressed by 4 bits, and the information amount of the evaluation value is reduced. Therefore, the possible values of the evaluation value are 16 types of 0 to 15, and the number of types of values that the evaluation value can take is reduced. Note that by not using at least one surrounding pixel value among the surrounding pixel values in the upper right, upper left, lower right, and lower left directions, the number of types of values that the evaluation value can take can be reduced. it can.

  Thus, the number of types of values that can be taken by the evaluation value is reduced, whereby the number of bins in the one-dimensional evaluation value histogram and the two-dimensional evaluation value histogram can be reduced. Therefore, it is possible to reduce the processing amount of the feature vector generation processing in the feature amount extraction unit 13 and to reduce the processing amount in the classifier 14.

  Further, in the case of obtaining the evaluation value, when the surrounding pixel value in the oblique direction is not used, it is necessary to obtain the pixel value at the surrounding position in the oblique direction whose distance from the target pixel is “1” by the pixel value interpolation process. Therefore, the processing amount in the evaluation value acquisition unit 130 is further reduced.

  In addition, when all the surrounding pixel values in the upper right, upper left, lower right, and lower left directions are not used in obtaining the evaluation value, the evaluation value is expressed in 4 bits, so uniform is used as described above. Even if the kinds of values that the evaluation value can take are limited, the effect is not so much. Therefore, in this case, uniform is not used to limit the types of values that the evaluation value can take. That is, when the evaluation value acquisition unit 130 looks at a plurality of bits in order, a texture expression code composed of a plurality of bits (4 bits) (more precisely, a value represented by a decimal number). It is used as an evaluation value regardless of the number of bit changes. This eliminates the need to limit the types of values that can be taken by the evaluation value, thereby reducing the amount of processing in the evaluation value acquisition unit 130.

  If the evaluation value is expressed in 4 bits and the types of values that the evaluation value can take using uniform are not limited, the number of bins in the one-dimensional evaluation value histogram is 16, and the two-dimensional evaluation value histogram The number of bins is 256 (= 16 × 16). Therefore, the feature vector extracted from the in-frame image is composed of 7696 (= 16 + 256 × 30) feature amounts and has 7696 dimensions.

<Third Modification>
As shown in FIG. 36, when the imaging environment is bright, such as in the daytime, the contrast between a relatively dark part such as the eyes and the surroundings of a person's face becomes clear. Therefore, when the imaging environment is bright, the facial image is based on the feature amount acquired from the negative LTP map including the negative LTP indicating the distribution of pixels darker than the target pixel around the target pixel. Detection accuracy can be improved by performing detection.

  On the other hand, as shown in FIG. 37, when the imaging environment is dark, such as at night, the brightness of a human face, such as a cheek, and the surroundings become clear. Therefore, when the imaging environment is dark, the facial image is based on the feature amount acquired from the positive LTP map including the positive LTP indicating the distribution of pixels brighter than the target pixel around the target pixel. Detection accuracy can be improved by performing detection.

  Therefore, in the image processing system 50 according to this modification, when the imaging environment of the imaging device 5 is bright, the image detection device 1 detects a face image using a negative LTP map, and the imaging device 5 When the imaging environment is dark, the face image is detected using the positive LTP map. Hereinafter, the image processing system 50 according to this modification will be described in detail.

  FIG. 38 is a diagram showing a configuration of an image processing system 50 according to this modification. As shown in FIG. 38, in the image processing system 50 according to the present modification, the illuminance sensor 5a is provided in the imaging device 5. The illuminance sensor 5a detects the illuminance of the imaging environment in the imaging device 5, and outputs a detection signal indicating the detected illuminance.

  FIG. 39 is a diagram illustrating functional blocks of the image detection apparatus 1 according to the present modification. As shown in FIG. 39, the image detection apparatus 1 according to the present modification example determines whether or not the imaging environment in the imaging apparatus 5 is bright based on the detection signal output from the illuminance sensor 5a. It has. The determination unit 16 refers to a detection signal output from the illuminance sensor 5a when the detection unit 12 performs a detection process on the processing target image. Then, the determination unit 16 determines that the imaging environment is bright if the illuminance indicated by the detection signal is equal to or greater than the threshold value, and determines that the imaging environment is dark if the illuminance is less than the threshold value.

  In this modification, the feature quantity extraction unit 13 generates a negative LTP map when the determination unit 16 determines that the imaging environment is bright. Then, the feature quantity extraction unit 13 generates a feature vector using the generated negative LTP map and inputs it to the classifier 14. On the other hand, when the determination unit 16 determines that the imaging environment is dark, the feature amount extraction unit 13 generates a positive LTP map. Then, the feature quantity extraction unit 13 generates a feature vector using the generated positive LTP map and inputs it to the classifier 14.

  Thus, the detection accuracy for the face image is further improved by switching the type of the evaluation value map to be used depending on whether the imaging environment is bright or dark.

<Fourth Modification>
When the pixel value is luminance and the pixel value is expressed by 8 bits, the detection accuracy for the face image is improved by setting the offset value used for generating the LTP to “8”. The reason for this will be described below.

  FIG. 40 shows a frequency curve (frequency curve) 900 indicating a frequency distribution (frequency distribution) of luminance differences between adjacent pixels for the face image sample, and a frequency distribution of luminance differences between adjacent pixels for the non-face image sample. It is a figure which shows the frequency curve 910. FIG. The horizontal axis in FIG. 40 indicates the values that the luminance difference between pixels can take. The vertical axis in FIG. 40 indicates the frequency (frequency) of the luminance difference having the value indicated on the horizontal axis. The frequency curves 900 and 910 are obtained as follows.

  First, various face image samples are prepared. Next, for each of a plurality of pixels constituting the face image sample, a luminance difference between the pixel and each of three adjacent pixels (a pixel immediately below, a pixel on the lower right, and a pixel on the right) is obtained. . The process for obtaining the luminance difference is performed for each of the prepared face image samples. Then, a frequency distribution for a plurality of luminance differences obtained for a plurality of face image samples is obtained. Next, each frequency in the obtained frequency distribution is divided by the number of face image samples to generate an average frequency distribution for one face image sample. Thereafter, a histogram indicating the frequency distribution is generated. The frequency curve 900 is obtained by connecting vertices of a plurality of bins of a generated histogram with a curve.

  Also, various non-face image samples are prepared. Then, for each of a plurality of pixels constituting the non-face image sample, a luminance difference between the pixel and each of three adjacent pixels (a pixel immediately below, a pixel on the lower right, and a pixel on the right) is obtained. . The process for obtaining the luminance difference is performed for each of the prepared non-face image samples. Then, a frequency distribution for a plurality of luminance differences obtained for a plurality of non-face image samples is obtained. Next, each frequency in the obtained frequency distribution is divided by the number of non-face image samples to generate an average frequency distribution for one non-face image sample. Thereafter, a histogram indicating the frequency distribution is generated. The frequency curve 910 is obtained by connecting the vertices of a plurality of bins of the generated histogram with a curve.

  As shown in FIG. 40, the frequency curve 910 indicating the frequency distribution of the luminance difference between adjacent pixels for the non-face image sample has a shape close to a normal distribution curve as a whole. This is because no feature is seen in the luminance difference between adjacent pixels for the non-face image, and the fact that the luminance difference peaks around 2-3 is due to the influence of noise.

  On the other hand, the frequency curve 900 indicating the frequency distribution of the luminance difference between adjacent pixels for the face image sample is similar to the frequency curve 910 in the portion where the luminance difference is less than 8, which is similar to the normal distribution curve. However, in the portion where the luminance difference is 8 or more, it is deviated from the normal distribution curve and is not similar to the frequency curve 910. Since a characteristic is seen in the luminance difference between adjacent pixels for the face image, the frequency curve 900 is not inherently a normal distribution curve, but due to the influence of noise, a normal distribution is obtained in a portion where the luminance difference is less than 8. The shape seems to be close to a curve.

  Here, if the frequency curves 900 and 910 are both normal distribution curves, the frequency curves 900 and 910 do not cross each other. However, as shown in FIG. 40, since the frequency curves 900 and 910 intersect with a luminance difference of 8, a facial image feature appears when the luminance difference is 8 or more, and the shape of the frequency curve 910 is a normal distribution. It can be seen that it is broken from the curve.

  As described above, when the frequency curve 900 for the face image sample and the frequency curve 910 for the non-face image sample are compared, if the luminance difference between adjacent pixels is 8 or more, they are non-similar. . From this, it can be considered that when the luminance difference between adjacent pixels is 8 or more, the influence of noise is reduced and the feature of the face image appears.

  Therefore, when extracting the feature amount using the negative LTP map or the positive LTP map, the offset value when generating the LTP is set to “8”. Thereby, the negative LTP or the positive LTP can appropriately represent the feature of the face image, and by identifying the face image based on the feature amount extracted using the negative LTP map or the positive LTP map, The detection accuracy for the face image is improved.

  Although the image processing system 50 has been described in detail above, the above description is illustrative in all aspects, and the present invention is not limited thereto. The various examples described above can be applied in combination as long as they do not contradict each other. And it is understood that the countless modification which is not illustrated can be assumed without deviating from the scope of the present invention.

DESCRIPTION OF SYMBOLS 1 Image detection apparatus 4 Control program 13 Feature-value extraction apparatus (feature-value extraction part)
130 evaluation value acquisition unit 131 feature amount acquisition unit

Claims (10)

An image detection device for detecting a detection target image from a processing target image,
Each of a plurality of pixels included in an image included in the processing target image is a target pixel, and an evaluation value acquisition unit that obtains a plurality of types of evaluation values indicating a relationship between a pixel value of the target pixel and a plurality of surrounding pixel values;
Seeking co-occurrence frequencies for the evaluation values obtained for the plurality of pixels included in the image, the feature amount acquisition unit to feature quantity the co-occurrence frequency,
A discriminator for determining whether the image is likely to be the detection target image based on the feature amount;
The image detection device uses the evaluation value used in the acquisition of the feature amount used in the classifier according to the brightness of the imaging environment of the processing target image between the plurality of types of evaluation values. Different image detection devices. The image detection apparatus according to claim 1.
The plurality of types of evaluation values include a first type of evaluation value,
The evaluation value acquisition unit generates, for each of a plurality of surrounding pixel values used in the acquisition of the first type of evaluation value, 1 bit indicating a relationship between the surrounding pixel value and the pixel value of the target pixel, A binary code composed of a plurality of bits generated for a plurality of surrounding pixel values is used as the first type of evaluation value,
The 1 bit relating to the first type of evaluation value indicates “1” if the surrounding pixel value is equal to or smaller than a value obtained by subtracting a predetermined amount from the pixel value of the target pixel, and “1” if the surrounding pixel value is larger than the value. An image detection device indicating 0 ″. An image detection apparatus according to any one of claims 1 and 2.
The plurality of types of evaluation values include a second type of evaluation value;
The evaluation value acquisition unit generates, for each of a plurality of surrounding pixel values used in the acquisition of the second type of evaluation value, 1 bit indicating a relationship between the surrounding pixel value and the pixel value of the target pixel, A binary code composed of a plurality of bits generated for a plurality of surrounding pixel values is used as the second type of evaluation value,
The 1 bit relating to the second type of evaluation value indicates “1” if the surrounding pixel value is equal to or greater than a value obtained by adding a predetermined amount to the pixel value of the target pixel, and “1” if the surrounding pixel value is less than the value. An image detection device indicating 0 ″. The image detection apparatus according to any one of claims 1 to 3 ,
The image detection apparatus , wherein the feature amount acquisition unit sets the feature amount without normalizing the co-occurrence frequency for a certain type of evaluation value included in the plurality of types of evaluation values . The image detection apparatus according to any one of claims 1 to 3 ,
The image detection apparatus , wherein the evaluation value acquisition unit does not use surrounding pixel values in an oblique direction with respect to a target pixel when obtaining a certain type of evaluation value included in the plurality of types of evaluation values . The image detection apparatus according to claim 5 ,
The evaluation value acquiring unit, in determining said certain type of evaluation value, not used all the upper right, upper left, around the pixel values of the lower right and lower left directions with respect to the target pixel, the image sensing apparatus. The image detection device according to claim 6 ,
The evaluation value acquisition unit generates, for each of a plurality of surrounding pixel values used for acquisition of the certain type of evaluation value, one bit indicating a relationship between the surrounding pixel value and the pixel value of the target pixel, Image detection that uses a binary code composed of multiple bits generated for the surrounding pixel values as an evaluation value of that kind regardless of the number of bit changes when the multiple bits are viewed in order Equipment . An image detection apparatus according to any one of claims 1 to 7 ,
The image detection apparatus, wherein the detection target image is a human face image. A control program for controlling an image detection device that detects a detection target image from a processing target image,
In the image detection device,
(A) obtaining a plurality of types of evaluation values indicating the relationship between a pixel value of the target pixel and a plurality of surrounding pixel values, with each of the plurality of pixels included in the image included in the processing target image as a target pixel;
(B) seeking co-occurrence frequencies for the evaluation values obtained for the plurality of pixels included in the image, the steps of the co-occurrence frequency and feature amount,
(C) executing a step of determining whether the image is likely to be the detection target image based on the feature amount;
The evaluation value used in the acquisition of the feature value used in the step (c) is selectively used among the plurality of types of evaluation values according to the brightness of the imaging environment of the processing target image. Control program. An image detection method for detecting a detection target image from a processing target image,
(A) obtaining a plurality of types of evaluation values indicating the relationship between a pixel value of the target pixel and a plurality of surrounding pixel values, with each of the plurality of pixels included in the image included in the processing target image as a target pixel;
(B) seeking co-occurrence frequencies for the evaluation values obtained for the plurality of pixels included in the image, the steps of the co-occurrence frequency and feature amount,
(C) determining whether the image is likely to be the detection target image based on the feature amount,
Image detection in which the evaluation value used in the acquisition of the feature value used in the step (c) is selectively used between the plurality of types of evaluation values according to the brightness of the imaging environment of the processing target image Method.