WO2011092848A1

WO2011092848A1 - Object detection device and face detection device

Info

Publication number: WO2011092848A1
Application number: PCT/JP2010/051257
Authority: WO
Inventors: 智明吉永; 茂喜長屋; 武洋藤田
Original assignee: 株式会社日立製作所
Priority date: 2010-01-29
Filing date: 2010-01-29
Publication date: 2011-08-04
Also published as: JPWO2011092848A1

Abstract

The disclosed object detection device resolves the issue of deterioration of detection accuracy upon occurrence of deformation to an object that had occurred with conventional object detection devices by way of the following means. In an object detection device for determining whether an object or a non-object is represented by performing determination processing for feature values obtained from a plurality of feature patterns, for an image that has been determined not to represent an object, determination is performed by using a new classifier into which is input a feature value for which calculation has already been performed regarding whether the image represents a certain specific deformed object or a non-object. As a result, it is possible to detect with high precision even an object for which a specific deformation has been applied. In addition, simply by altering the classifier, high-speed detection becomes possible.

Description

Object detection device and face detection device

The present invention relates to an object detection device that detects a specific object such as a face from an image.

There are, for example, Non-Patent Document 1 and Patent Document 1 as a method for detecting a specific object such as a face from an input image.

The technique described in Non-Patent Document 1 detects a face in an image by determining whether a face image of 20 × 20 pixels is a face or a non-face using a plurality of image features called haar. is there.

Patent Document 1 detects a face in a plurality of directions by preparing a different feature pattern for each face direction and estimating the face direction, and then performing face detection processing specialized for the estimated face direction.

JP 2008-173628 A

In Non-Patent Document 1, when a change occurs such as turning to the side or rotating with respect to the face to be detected, the reaction deteriorates in many image features. Arise. That is, the detection rate for a face in which a specific change has occurred is low.

Further, in Patent Document 1, since the direction estimation is performed using the feature pattern for each of a plurality of directions, there is a problem that the processing time increases, and further, if the face direction estimation is wrong, the face cannot be detected. For this reason, it has been necessary to improve the detection rate for a face in which a specific change such as rotation has occurred without adding an enormous feature amount evaluation calculation.

The present invention has been made in view of the above problems, and an object of the present invention is to improve the detection rate for a face in which a specific change has occurred.

This application discloses a plurality of means for solving the above problems. One of them is, for example, the configuration described in the claims.

According to the present invention, the object detection rate can be increased.

1 is a block diagram illustrating a configuration of an object detection apparatus according to a first embodiment. It is explanatory drawing which shows the example of a process of the image input part 101 of FIG. It is a figure which shows the example of the feature pattern processed by the feature evaluation part 102 of FIG. It is a table which shows an example of the data of the feature pattern used by the object discrimination | determination stored in feature pattern DB110 of FIG. 2 is a flowchart illustrating a procedure of processing performed by a change object determination unit 105 in FIG. 1. It is a figure which shows the example of the weight with respect to each feature-value used by the change object discrimination | determination part 105 of FIG. It is a figure which shows the high-speed object determination of Example 2. FIG. FIG. 10 is a diagram illustrating an example of feature amounts used in the third embodiment. FIG. 10 is a configuration diagram illustrating a configuration of an object detection apparatus according to a fourth embodiment. FIG. 10 is a configuration diagram illustrating a configuration of an object detection apparatus according to a fifth embodiment. FIG. 10 is a diagram illustrating an example of a setting screen for performing parameter setting in the fifth embodiment.

Hereinafter, examples will be described.

Example 1 will be described with reference to FIG. The object detection unit 100 includes an image input unit 101, a feature pattern DB 110, a feature evaluation unit 102, a feature amount storage unit 103, an object determination unit 104, a changed object determination unit 105, and a determination result output unit 106. Each unit described above may be configured by hardware. Further, it may be a module combining hardware and software.

The operation of the object detection unit shown in FIG. 1 will be described using a case where a face is detected as an example. As an object to be detected by the object detection unit, in addition to the face, other objects such as a person, a car, and a sign may be targeted. For simplification of description, an operation for detecting a front face of a person from an input image will be described as an example of an object detection operation.

The image input unit 101 receives an imaging module such as a camera, a reproduction image of a pre-recorded image, and the like, and outputs an image region 203 for performing a discrimination process between a face and a non-face to the feature evaluation unit 102. The feature pattern DB 110 stores a feature pattern for determining whether it is a face or a non-face. The feature evaluation unit 102 calculates feature amounts for a plurality of feature patterns defined in the feature pattern DB 110 for the input image region 203 and stores them in the feature amount storage unit 103. The feature amount storage unit 103 stores the feature amount obtained from the feature evaluation unit 102. The object discriminating unit 104 performs discrimination processing of a face or a non-face based on a plurality of feature value values obtained by the feature evaluation unit 102. As a result, if it is determined as a face, the result is output to the determination result output unit 106, and if it is determined as a non-face, the result is output to the change object determination unit 105. The changed object discriminating unit 105 discriminates whether the image area 203 discriminated as a non-face is a face or a non-face that has undergone a specific change, such as a sideways face, using the feature amount stored in the feature amount storage unit 103. In the change object determination unit 105, in order to find an object different from the object determination unit 104, a determination process different from that of the object determination unit 104 is performed. The discrimination result in the change object discrimination unit 105 is output to the discrimination result output unit 106. In the discrimination result output unit 106, the image region determined as the face by the object discrimination unit 104 or the image region discriminated as the face having undergone the specific change by the changed object discrimination unit 105 is set as a face, and the other is determined as a non-face. Is output.

With the above configuration, for a face that can no longer be detected due to a change such as being directed obliquely laterally, a discrimination process that focuses on the feature value indicating the unique output for the changed face It is possible to perform face detection that is robust against slight changes. Furthermore, since the discrimination process for the changed face can be performed as it is using the feature amount obtained in the front face detection, the above discrimination process can be performed at high speed.

Note that, as an example of the specific change with respect to the face, changes such as the face turning obliquely in the horizontal direction, the face rotating, or a part of the face being hidden can be considered. Hereinafter, as an example of object detection, an example will be described in which a face facing diagonally to the left and right is detected with high accuracy. One or a plurality of specific change patterns of the face determined by the object detection unit may be used.

Details of the operation of the image input unit 101 in FIG. 1 will be described with reference to FIG. With respect to the input image 200 obtained by the image input unit 101, the

faces

201 and 202 to be detected exist at arbitrary positions and in arbitrary sizes. In order to cope with this, the image input unit 101 cuts out image regions 203 having a plurality of positions and sizes on the input image 200, for example, in a raster scan form, and outputs them to the feature evaluation unit 102. In the feature evaluation unit 102 and later, a process for determining whether a face is a non-face is performed on a plurality of image areas in a single input image 200. As a result, a face of an arbitrary size present at an arbitrary location in the input image 200 is detected.

The feature pattern DB 110 in FIG. 1 will be described. The feature pattern DB 110 defines a plurality of image feature parameters used for face discrimination. FIG. 3 is an example of a defined image feature pattern. The feature pattern in FIG. 3 is composed of a black rectangle 301 and a white rectangle 302, and the feature amount is obtained by the difference in the sum of the pixel values in the rectangle. FIG. 4 is an example of a face discrimination parameter table 400 in which the feature parameters are defined. N image features h _i (i∈N) are defined to discriminate between a face and a non-face, and a weight α when performing face determination for each feature and a weight β when performing right oblique side face determination , A weight γ for the left oblique side face determination is defined.

Details of the operation of the feature evaluation unit 102 in FIG. 1 will be described. The feature evaluation unit 102 calculates a plurality of feature amounts for the input image region 203. If the image area 203 input to the image feature pattern h _i defined in the feature pattern DB 110 is a vector I, the obtained feature quantity h _i (I) can be obtained by Equation 1.

The feature evaluation unit 102 stores the N feature amounts obtained by the above calculation in the feature amount storage unit 103.

In addition, although the example which used the rectangular feature as the feature-value evaluated by the feature evaluation part 102 was shown, according to the object made into a detection target, EOH (Edge of Orientation Histograms), HOG (Histogram of Orientation Gradients), etc. differ. A feature amount may be used.

Details of the operation of the object determination unit 104 in FIG. 1 will be described. The object discriminating unit 104 calculates face likelihood based on each feature quantity h _i (I) obtained by the feature evaluation unit 102 and discriminates whether the image region 203 is a face or a non-face. As an example of face discrimination, the AdaBoost classifier calculates the face likelihood F (I) by a linear sum function using the face discrimination weight α _i for each feature h _i described in the face discrimination parameter table 400 of FIG. To do.

A larger face likelihood F (I) indicates that the face is more likely to be a face. If the face likelihood F is greater than or equal to the threshold Th _F , it is determined to be a face, and if it is less, it is determined to be a non-face. If the discrimination result is a face, the object discrimination unit 104 outputs the result to the discrimination result output unit 106. If the determination result is a non-face, the result is output to the change object determination unit 105.

Details of the operation of the change object determination unit 105 in FIG. 1 will be described with reference to FIG. FIG. 5 is a flowchart showing steps of determining the right oblique face and the left oblique face in the change object discriminating unit 105.

In step 501, the feature quantity h _i (I) for the image area 203 stored in the feature quantity storage unit 103 is read.

In step 502, the likelihood R for the right oblique face is calculated from the feature amount. The likelihood R is calculated in step 502 using the same feature amount as that of the object determination unit 104, but a different determination process is performed because a determination target is different. The right oblique face likelihood R is calculated by the calculation of Equation 3 which is a function of a linear sum using the right oblique face weight β _i for each feature h _i described in the face discrimination parameter table 400 of FIG.

In step 503, the right oblique face likelihood _R is compared with the right oblique face discrimination threshold Thre _R, and if it is equal to or greater than the threshold, the process proceeds to step 507, and if not, the process proceeds to step 504.

In step 504, the likelihood L for the left oblique face is calculated as in the case of the right oblique face.

The left oblique face likelihood L is a mathematical expression that is a function of a linear sum using the left oblique face weight γ _i for each feature h _i described in the face discrimination parameter table 400 of FIG. 4 is calculated.

In step 505, the left oblique face likelihood _L is compared with the left oblique face discrimination threshold Thre _L, and if it is equal to or greater than the threshold, the process proceeds to step 507, and if not, the process proceeds to step 506.

FIG. 6 is a diagram showing the concept of face likelihood calculation in

steps

502 and 504 of FIG. FIG. 6A is a diagram showing calculation of face likelihood for a front face, while FIGS. 6B and 6C show calculation of right diagonal face likelihood and left diagonal face likelihood, respectively. FIG. In FIG. 6, the weights for the feature amounts 601 to 605 are expressed by shading. In FIG. 6A, almost equal weights are assigned to the feature amounts 601 to 605 in order to discriminate a general front face. On the other hand, FIG. 6B shows that weights are small for feature amounts such as h1, h3, and h4, and features such as h2, h5, and the like are large. Yes. As a result, it is possible to discriminate between the right diagonal face and the non-face based on different criteria from the front face and non-face determination.

Step 506 is a step visited when it is determined that neither the right diagonal face nor the left diagonal face is detected. For this reason, in step 506, the final discrimination result in the change object discrimination unit is determined as a non-face. On the other hand, step 507 is visited when it is determined that the face is a right diagonal face or a left diagonal face. For this reason, step 507 determines the discrimination result as a face.

By taking the above processing flow, it is possible to re-determine whether the image area determined to be a non-face by the object determination unit 104 is a right diagonal face or a left diagonal face. This re-discrimination process is performed using a plurality of feature amounts calculated in advance and stored in the feature amount storage unit 103, so that it is not necessary to newly calculate a feature amount for the image region 203 and can be determined at high speed.

In addition, although the linear sum with respect to several feature-values was used for the discrimination | determination of a deformed face, the discrimination methods different in the object discrimination | determination part 104 and the change object discrimination | determination part 105 using discriminators, such as PCA and a nonlinear support vector machine (SVM). You may discriminate | determine using.

As described above, in the first embodiment, the face detection apparatus detects a face from an input image, and includes an image input unit that inputs an image, and a feature that calculates a feature amount for the image obtained by the image input unit. A linear sum is calculated by the evaluation unit, the feature amount storage unit that stores the calculated feature amount, the feature amount obtained by the feature evaluation unit and the front face function, and based on the calculated linear sum, An object discriminating unit that discriminates whether the image is a front face or a non-front face; and an image that has been discriminated as a non-front face by the object discriminating unit; Calculates a linear sum with an oblique face function with a different weighting factor, and determines whether the image is a non-face or an oblique face, and a discrimination result that outputs the result of the object discrimination Characterized by having an output part It discloses a face detection apparatus that. With this face detection device, it is possible to determine a face with an obstruction, so that the face detection rate is improved.

Furthermore, if the feature points to which the weighting coefficient of the function for partial occlusion face is included in the change object discriminating unit include part of the feature points of the front face, the feature amount is not recalculated and the calculation amount is reduced. Therefore, high-speed face detection can be realized.

Example 2 will be described with reference to FIG. The present embodiment is an example in which face detection is performed at high speed by using a cascade type discriminator as shown in FIG.

In FIG. 7, the face / non-face discriminator in the object discriminating unit 104 has a configuration in which a plurality of discriminators 710 to 730 are connected in cascade. The discriminator 1 (710) uses only the feature value set (701) from the feature value 1 to the feature value A (A <N) out of the N feature values stored in the feature value storage unit 103. Processing is performed to determine whether the image area is a face or a non-face. Here, when it is determined that the face is non-face (when the determination result of the discriminator 1 (710) is “False”), the processing ends. At this time, the feature amount calculation in the feature evaluation unit 102 is completed only after the feature amount A is processed. On the other hand, when the image area is determined to be a face (when the determination result of the identifier 1 (710) is “True”), the determination processing is left to the identifier 2 (720). The discriminator 2 (720) performs discrimination processing using the feature amount set (702) from the feature amounts A + 1 to B, and similarly determines whether the image area is a face or a non-face. When the above processing is performed up to the last discriminator S (730) and the discriminator S (730) discriminates the face, the discrimination result is set as a face. However, if the discriminator S (730) determines False, the change object discriminating unit 105 determines whether the face is an oblique face or a non-face. The changing object discriminating unit 105 uses the N feature amounts stored in the feature amount storage unit (103) used in the discriminating processes of the discriminators 1 to S to determine whether the left and right diagonal faces are not in accordance with the flowchart of FIG. Determine if it is a face.

With the above-described configuration, it is possible to detect the face at high speed because it is possible to end the discrimination process halfway without performing all the feature amount calculations for an area that is clearly determined to be a non-face that is neither a face nor an oblique face.

Example 3 will be described with reference to FIG. FIG. 8A shows an example of images of feature amounts (801) to (803) in which a high weight value is set when the face of a person wearing sunglasses is detected in the change object discriminating unit (105). FIG. FIG. 8B is a diagram showing an example of images of feature amounts (804) to (807) in which a high weight value is set when detecting the face of a person with a mask similarly. By using the object detection unit 100 having the configuration of FIG. 1, it is possible to perform a discrimination process in which feature quantities that react to the exemplified sunglasses and mask are weighted, and to detect the presence / absence of a wearing item on the face. Is possible.

It is possible to collect specific metadata information for a face by determining whether or not such a wearing item is present in the change object discriminating unit 105 and outputting additional information on the presence or absence of the wearing item for the face in the discrimination result output unit 106. Become.

In addition to sunglasses and masks, glasses, beards, hats, eye patches, and the like can be considered as examples of wearing items for the face. A classifier for discriminating each change object is prepared in the feature pattern DB 110 in advance. It is possible to determine whether the face is equipped with these wearing items.

As described above, the third embodiment is a face detection device that detects a face from an input image, and includes an image input unit that inputs an image, and a feature that calculates a feature amount for the image obtained by the image input unit. A linear sum is calculated by the evaluation unit, the feature amount storage unit that stores the calculated feature amount, the feature amount obtained by the feature evaluation unit and the front face function, and based on the calculated linear sum, An object discriminating unit that discriminates whether the image is a front face or a non-front face; and an image that has been discriminated as a non-front face by the object discriminating unit; Calculates a linear sum with a partially occluded face function with a different weighting coefficient, and a change object discriminating unit that detects whether the image is a non-face or a face that is partially occluded by something. Disclosed is a face detection device characterized by having That. With this face detection device, it is possible to determine a face with an obstruction, so that the face detection rate is improved.

The configuration of the apparatus according to the fourth embodiment will be described with reference to FIG. The present embodiment is an example in which the object detection apparatus according to the first to third embodiments is mounted on an imaging apparatus such as a monitoring camera or a digital camera, a display, or a video recording apparatus.

FIG. 9 is a configuration diagram illustrating the configuration of the object detection device according to the fourth embodiment. In FIG. 9, an object detection apparatus (900) includes an image input unit (909), an image memory (902), a CPU (903), a RAM (904), a ROM (905), a detection result recording unit (906), an interface ( 907) and an output device (908).

In the object detection apparatus (900) of the present embodiment, a target object is detected from an image obtained by a camera that is an imaging unit (901). The CPU (903) in the object detection apparatus (900) of the present embodiment corresponds to the object detection unit (100) shown in FIG. 1 of the first embodiment, and executes each arithmetic processing of the object detection unit (100) as a program. This is realized by performing arithmetic processing by the CPU (903).

In this embodiment, the CPU (903) performs arithmetic processing according to the detection method in the object detection unit (100) to detect the object.

The object detection result for each sequence is recorded in the detection result recording unit (906). The detection result is converted into an appropriate form through the interface (907) and output to the output device (908). Here, the output device may be a display, a printer, a PC, or the like.

In this embodiment, it is possible to perform arithmetic processing as an object detection device by an information processing device such as a computer.

According to the fourth embodiment described above, it is possible to realize an imaging device, a display, and a video recording device having an object detection function for detecting an object existing in an image with high accuracy.

The configuration of the apparatus according to the fifth embodiment will be described with reference to FIG. In this embodiment, the object detection apparatus (900) according to the fourth embodiment is provided with the input device (1010) and the setting control unit (1020) shown in FIG. It is an example which comprises the object detection apparatus (1000) which can do.

In the object detection apparatus (1000) of the present embodiment, a command for parameter adjustment in the change object determination process in the change object determination unit (105) of FIG. 1 is received from the input device (1010). The setting control unit (1020) that has received this command performs parameter control such as ON / OFF control of discrimination processing for the deformed object in the changed object discrimination unit (105) and sensitivity adjustment. As a result, it is possible to limit or expand the discrimination target for a plurality of deformed objects prepared in advance to be discriminated in the object detection device.

FIG. 11 is a diagram showing an example of a parameter setting screen (1100) for performing parameter setting in deformed object discrimination with the input device (1010). The parameter setting screen (1100) includes an object discrimination parameter (1101) and deformed object discrimination parameters (1102 to 1107). The object discrimination parameter (1101) controls the sensitivity of face detection in the object discrimination unit (104). For example, when high sensitivity is set, it is easy to determine a face by loosening the threshold _TF of the face discrimination parameter table (400) defined in the feature pattern DB (110).

In the deformed object discriminating parameters (1102 to 1107), whether or not the changing object discriminating unit (105) performs discriminating processing on each deformed object such as a left-right slanted face and a left-right rotated face, and how sensitivity adjustment is performed. Set the parameters. The parameter information of each item set in this way is sent to the setting control unit (1020), and the setting control unit controls the detection process in the CPU.

By adopting the above configuration, it is possible to perform object detection specialized for the deformed object to be detected. This makes it possible to adjust the detection sensitivity for the product and change the characteristics according to the environment in which it is used. For example, when face detection in a surveillance camera image is taken as an example, the face appearing in the image varies depending on the installation status of the camera. Depending on the relationship between the camera installation position and the person flow line in the installation environment, a situation in which many downward faces appear or a situation in which more right faces appear than left faces may occur. By setting a deformed object pattern to be determined for each camera, a desired face can be detected with high accuracy.

DESCRIPTION OF SYMBOLS 100 ... Object detection part 101 ... Image input part 102 ... Feature evaluation part 103 ... Feature-value storage part 104 ... Object discrimination | determination part 105 ... Change object discrimination | determination part 106 ... Discrimination result output part 110 ... Feature pattern DB
200:

input image

201, 202 ... face 203 ...

image area

301, 302 ... rectangular feature 400 ... face discrimination parameter table 501 to 507 ... step 601 to 605 ... rectangular feature 701 to 703 ... feature quantity set 710 ... classifier 1
720 ... Classifier 2
730 ... Classifier S
801 to 807 ... Rectangular feature 900 ... Object detection device 901 ... Imaging unit 902 ... Image memory 903 ... CPU
904 ... RAM
905 ... ROM
906 ... Detection result recording unit 907 ... Interface 908 ... Output device 909 ... Image input unit 1000 ... Object detection device 1010 ... Input device 1020 ... Setting control unit 1100 ... Parameter setting screen 1101 ... Object discrimination parameters 1102 to 1107 ... Deformed object discrimination parameters

Claims

An object detection device for detecting a specific object from an input image,
An image input unit for inputting an image;
A feature evaluation unit that calculates a feature amount for the image obtained by the image input unit;
A feature amount storage unit for storing the calculated feature amount;
An object discriminating unit that discriminates whether the image is an object or a non-object by using the feature amount obtained by the feature evaluation unit;
For an image that is determined not to be an object by the object determination unit, a specific change occurs in the image by a determination method that is different from the object determination unit using the feature amount stored in the feature amount storage unit. A change object discriminator for discriminating whether the object is a non-object,
An object detection apparatus comprising: a discrimination result output unit for outputting the object discrimination result.
In claim 1,
The object detection apparatus, wherein the change object determination unit includes an input unit for adjusting a type of a deformed object to be determined and a sensitivity of the determination.
In claim 1,
The object detection device, wherein the change object determination unit determines whether the object is a deformed object or a non-object by performing a linear determination process using the feature amount calculated by the feature evaluation unit.
In claim 1,
The object detection apparatus according to claim 1, wherein the change object determination unit performs determination using a support vector machine having the feature amount calculated by the feature evaluation unit as an input.
In claim 1,
An object detection apparatus characterized in that the change object discrimination unit discriminates an object that has undergone in-plane / out-plane rotation.
In claim 1,
The object detection apparatus, wherein the change object determination unit determines an object in which a part of the object is blocked.
A face detection device for detecting a face from an input image,
An image input unit for inputting an image;
A feature evaluation unit that calculates a feature amount for the image obtained by the image input unit;
A feature amount storage unit for storing the calculated feature amount;
An object discriminating unit that calculates a linear sum of the feature amount obtained by the feature evaluation unit and the front face function, and discriminates whether the image is a front face or a non-front face based on the calculated linear sum;
For an image determined as a non-front face by the object determination unit, a linear sum is calculated from the feature amount stored in the feature amount storage unit and the object determination unit using an oblique face function with a different weighting coefficient. Then, a change object determination unit that determines whether the image is a non-face or an oblique face,
A face detection apparatus comprising a discrimination result output unit for outputting the result of object discrimination.
In claim 7,
The face detection device, wherein the feature point that assigns the weighting coefficient of the oblique face function in the change object determination unit includes a part of the feature point of the front face.
A face detection device for detecting a face from an input image,
An image input unit for inputting an image;
A feature evaluation unit that calculates a feature amount for the image obtained by the image input unit;
A feature amount storage unit for storing the calculated feature amount;
An object discriminating unit that calculates a linear sum of the feature amount obtained by the feature evaluation unit and the front face function, and discriminates whether the image is a front face or a non-front face based on the calculated linear sum;
For an image determined as a non-front face by the object determination unit, the feature amount stored in the feature amount storage unit and the object determination unit perform a linear sum with a partial occlusion face function with a weighting factor changed. A face detection apparatus comprising: a change object determination unit that calculates and detects whether the image is a non-face or a face that is partially covered by something.
In claim 9,
The face detection device, wherein the feature point to which the weighting coefficient of the partial occlusion face function is included in the change object determination unit includes a part of the feature point of the front face.