JP2010219862A - Face detection device, and imaging apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To appropriately detect the face of an imaged person under backlight. <P>SOLUTION: An imaging apparatus 2 includes a face detection section 41 which detects a moving body from an image imaged by an imaging section 10, calculates an extent of backlight based on luminance information of a moving body area including the detected moving body, determines an offset value for binarization processing corresponding to the calculated extent of backlight, binarizes the image based on the determined offset value, and detects the face of a person of a subject from the binarized image. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to a face detection apparatus and an imaging apparatus that detect a face from a captured image.

  2. Description of the Related Art Conventionally, a technique for capturing a person in a digital camera and detecting a person's face from the captured image is known. Also known is a digital camera that detects a person's face from the captured image and performs person exposure by performing automatic exposure control so that the person's face is properly exposed according to the brightness of the detected face area. (For example, refer to Patent Document 1).

JP 2003-107555 A

  However, in a configuration in which exposure control using conventional face detection is performed, there is a possibility that the detection of a person's face may not be performed well in the backlight when it is difficult to detect the pattern of the person's face from the captured image. For this reason, at the time of backlighting, there is a possibility that automatic exposure control using human face detection cannot be performed well.

  An object of the present invention is to appropriately detect the face of an imaged person even under backlight.

In order to solve the above-described problem, a face detection device according to claim 1 is provided.
A moving object is detected from the captured image, a backlight level is calculated based on luminance information of a moving object area including the detected moving object, and an offset value for binarization processing is determined according to the calculated backlight level. Then, the image processing apparatus includes a face detection unit that binarizes the image based on the determined offset value and detects the face of the subject person from the binarized image.

According to a second aspect of the present invention, in the face detection device according to the first aspect,
The face detection unit calculates a difference value between an average luminance value of a luminance evaluation area that does not include the moving object in the moving object area and an average luminance value of the luminance evaluation area that includes the moving object in the moving object area. Calculated as the degree of backlight.

The invention according to claim 3 is the face detection apparatus according to claim 1 or 2,
The face detection unit calculates a difference value between an average luminance value of a luminance evaluation area of the captured image and an average luminance value of a target pixel, and calculates the calculated difference value and the calculated offset value. In comparison, the pixel of interest is binarized with a pixel of interest having a difference value larger than the offset value as a black pixel and other pixels of interest as white pixels.

An imaging device according to a fourth aspect of the present invention provides:
The face detection device according to any one of claims 1 to 3,
An imaging unit that captures the image;
An automatic exposure control unit that determines whether or not a face has been detected by the face detection unit, and performs exposure control on the imaging unit according to luminance information of the detected face area when a face is detected; .

  According to the present invention, it is possible to appropriately detect an image of a person's face even under backlight.

It is a figure which shows a vending machine provided with the imaging device of embodiment which concerns on this invention. It is a block diagram which shows the function structure of an imaging device. It is a flowchart which shows an automatic exposure control process. It is a figure which shows the moving body area in an image. It is a figure which shows the attention pixel and luminance evaluation area for a binarization process. It is a figure which shows another attention pixel and brightness | luminance evaluation area for a binarization process. It is a figure which shows all the areas in an image. It is a figure which shows the center area in an image. It is a figure which shows the division area in an image.

  Embodiments according to the present invention will be described in detail with reference to the accompanying drawings. The present invention is not limited to the illustrated example.

  First, with reference to FIG. 1, a mode in which the imaging apparatus of the present embodiment is used will be described. FIG. 1 shows a vending machine 1 including an imaging device 2 according to the present embodiment.

  As shown in FIG. 1, the imaging device 2 of the present embodiment is fixed to a vending machine 1. The vending machine 1 is assumed to be a vending machine that sells cigarettes as a product that requires checking the age of the adult. As a product for which the age verification of the adult is necessary, alcohol or the like may be used. The vending machine 1 images the person 3 who is the customer of the product with the imaging device 2, analyzes the captured image data to perform age authentication of the person 3, and if the product is authenticated as an adult, Sell.

  It is assumed that the vending machine 1 is installed outdoors. For this reason, it is assumed that the person 3 may be imaged under backlight by the imaging device 2 depending on the position of the sun 4 in the time zone.

  Next, the internal configuration of the imaging apparatus 2 will be described with reference to FIG. FIG. 2 shows a functional configuration of the imaging apparatus 2. The imaging device 2 includes an imaging unit 10, a signal processing unit 20, an image memory 30, and a control unit 40.

  The imaging unit 10 is a digital movie camera that images a subject. The imaging unit 10 includes an optical system, an imaging device, an A / D (Analog to Digital) conversion unit, and the like. The optical system includes various lenses, a diaphragm mechanism, and the like. The image sensor is an image sensor such as a CCD (Charge Coupled Device), CMOS, etc., and photoelectrically converts the optical signal of the subject image formed by the optical system into an analog electrical signal to obtain an analog image signal (RGB data). Output. RGB data is image data represented by red, blue, and green component signals. Further, it is assumed that the exposure of the imaging unit 10 can be adjusted by adjusting the aperture value of the aperture mechanism of the imaging unit 10, the exposure time (shutter speed) of the imaging device, and the sensitivity. The A / D converter converts an analog image signal input from the image sensor into digital image data and outputs the digital image data. The imaging unit 10 generates and outputs image data for each frame of a moving image. The imaging speed of the imaging unit 10 is, for example, 30 fps.

  The signal processing unit 20 is a signal processing circuit for various signal processing such as interpolation processing, color processing (color difference conversion processing for converting RGB data into YUV data), luminance signal processing (edge enhancement processing, gamma correction processing), and the like. The signal processing unit 20 performs various types of signal processing on the image data (RGB data) input from the imaging unit 10 and converts the image data into YUV data. The YUV data is image data represented by a luminance signal Y, a blue component color difference signal U (Cb), and a red component color difference signal V (Cr). The signal processing unit 20 stores the converted YUV data in the image memory 30 and outputs it to the authentication unit (not shown). The authentication unit performs image analysis on the image data input from the signal processing unit 20 and performs age authentication of the subject person.

  Further, when photometric area information (photometric area coordinate information) is input from the control unit 40, the signal processing unit 20 stores the coordinate information in a register (not shown), and stores image data corresponding to the coordinate information. It has a function of reading out from the image memory 30 and calculating luminance information (average luminance value) of the photometric area in the image of the read out image data according to the input photometric area information and outputting it to the control unit 40. To do.

  The image memory 30 is composed of a RAM (Random Access Memory), and temporarily stores image data (YUV data) for each frame output from the signal processing unit 20.

  The control unit 40 controls each unit of the imaging device 2. The control unit 40 includes a CPU (Central Processing Unit), a RAM, and a ROM (Read Only Memory). In the control unit 40, a designated program among the programs stored in the ROM is expanded in the RAM, and various processes are executed in cooperation with the expanded program and the CPU. It is assumed that an automatic exposure control program is stored in the ROM. The control unit 40 includes a face detection unit 41 and an automatic exposure control unit 42 as functional units according to the automatic exposure control program.

  The face detection unit 41 detects a moving object from the image of the image data captured by the imaging unit 10, calculates the degree of backlight based on the luminance information of the moving object area including the detected moving object, and calculates the calculated degree of backlight An offset value for binarization processing corresponding to the image is determined, the image is binarized based on the determined offset value, and the face of the subject person is detected from the binarized image. The automatic exposure control unit 42 determines whether or not a face has been detected by the face detection unit 41. When a face is detected, the image capturing unit 10 performs exposure control according to the luminance information of the detected face area. To do.

  Next, the operation of the imaging device 2 will be described with reference to FIGS. FIG. 3 shows the flow of the automatic exposure control process. FIG. 4 shows the moving object area 100 in the image F. FIG. 5 shows a pixel of interest 201 and a luminance evaluation area 202 for binarization processing. FIG. 6 shows a pixel of interest 201 and a luminance evaluation area 203 for binarization processing. FIG. 7 shows a central area 301 in the image F. FIG. 8 shows the entire area 302 in the image F. FIG. 9 shows a divided area 303 in the image F.

  In the imaging apparatus 2, for example, the automatic exposure control process based on the automatic exposure control program is executed by the control unit 40, triggered by the power-on input. In addition, the image data (YUV data) of each frame generated by imaging the subject by the imaging unit 10 and the signal processing unit 20 with the power-on as a trigger is stored in the image memory 30 in the imaging order, and Output to the authentication unit.

  As shown in FIG. 3, first, the face detection unit 41 determines whether or not a plurality of image data including the latest one frame of image data that has not been read is stored in the image memory 30 (step). S1). If a plurality of image data including the latest one frame of image data that has not been read is not stored (step S1; NO), the process proceeds to step S1.

  When a plurality of image data including the latest one frame of image data that has not been read is stored (step S <b> 1; YES), the face detection unit 41 obtains the latest two frames of image data from the image memory 30. Read (step S2). Step S2 is a step of reading out image data of a plurality of frames used for the moving object detection process, and image data of three frames or more may be read out.

  Then, the moving object detection is performed by the face detection unit 41 based on the image of the image data read in step S2 (step S3). In step S <b> 3, an area in which a luminance value changes more than a predetermined value between a plurality of images in a plurality of frames of image data is detected as a moving object. The moving object detection in step S3 is performed by a known moving object detection technique. As the moving object, a person who is a customer of the vending machine 1 is assumed. The height of the person differs depending on the person, and when the person poses for authentication in front of the lens of the imaging unit 10, the person (face) may not be located at the same position in the image. For this reason, detecting a moving object in an image is effective in calculating the degree of backlight and detecting a human face, which will be described later.

  And it is discriminate | determined by the face detection part 41 whether the moving body was detected by step S3 (step S4). When a moving body is not detected (step S3; NO), the process proceeds to step S1. If a moving object is detected (step S3; YES), the moving object area is determined (set) in the area surrounding the moving object part detected in step S3 in the image of the latest one frame of image data read in step S2. (Step S5).

  In step S5, for example, as shown in FIG. 4, when the moving object M is detected in the image F, the moving object area 100 including the moving object M is set. The moving object area 100 includes luminance evaluation areas 101, 102, and 103 for calculating the degree of backlight, which will be described later. The luminance evaluation areas 101, 102, and 103 are sequentially arranged at the upper left, upper right (area not including the moving object M), and lower center (area including the moving object M) of the moving object area 100.

  Then, the luminance information is calculated by the face detection unit 41 in the moving object area determined in step S5 (step S6). Specifically, the average luminance value for each pixel in the luminance evaluation areas 101, 102, 103 of the moving object area 100 is calculated. Then, from the luminance information calculated in step S6, the average luminance value of the luminance evaluation area (luminance evaluation area 101 + luminance evaluation area 102) not including the moving object M and the luminance evaluation area (luminance evaluation area 103) including the moving object M are included. ) And an average luminance value are calculated (step S7). The difference value calculated in step S7 is a value that represents the degree of backlight in the captured image.

  And the offset value according to the difference value (degree of backlight) calculated at step S7 is set (step S8). More specifically, the greater the degree of backlighting, the smaller the offset value is set. The offset value is set, for example, in a range of 9 levels from 0 to 8. For example, when the difference value exceeds 200, the offset value is set to 0 because the degree of backlighting is very large. If the difference value is smaller than 130, the offset value is set to 8 because the degree of backlighting is not so large. In the normal lighting state, the offset value is set to about 8. When the difference value is 130 to 200, the difference value is divided into nine equal parts, and the offset value is assigned to each section in order from 8 to 0, and the offset value of the section corresponding to the difference value is set.

  Then, the face detection unit 41 uses the offset value calculated in step S8 to perform binarization processing on the latest one frame of image data read out in step S2, thereby obtaining binarized data. (Step S9).

  In step S9, for example, the upper left pixel is selected as the target pixel in the image of the latest one frame of image data. Then, the luminance value of the selected target pixel is acquired, and the average luminance value of the pixels in the luminance evaluation area corresponding to the target pixel is calculated. The luminance evaluation area is set in advance. For example, as shown in FIG. 5, a luminance evaluation area 202 having an annular pattern around the target pixel 201 is used. Further, as shown in FIG. 6, a luminance evaluation area 203 having a rectangular pattern above and below the target pixel 201 may be used. In addition, as long as the shape and the number of pixels of the luminance evaluation area are suitable for face detection (extraction of eyes, nose, and mouth), other luminance evaluation areas may be used. Furthermore, a configuration in which a plurality of luminance evaluation areas are overlaid may be employed.

Then, a difference value between the calculated average luminance value of the luminance evaluation area and the acquired luminance value of the target pixel being selected is calculated. And it is discriminate | determined whether following Formula (1) is materialized about the calculated difference value.
Average luminance value in luminance evaluation area−luminance value of target pixel> offset value set in step S8 (1)

  When Expression (1) is satisfied, a black pixel is set at the selected target pixel position. When the expression (1) is not satisfied, a white pixel is set at the selected target pixel position. Then, the pixel of interest is moved and selected by one pixel to the right, and the black pixel or the white pixel in the pixel of interest after the movement is set. When the pixel of interest is moved, if the pixel of interest before the movement is on the rightmost side, the pixel of interest is moved to the leftmost pixel below one column. In this manner, the movement of the target pixel and the setting of the black pixel or the white pixel are repeated, and binarized data that is image data including the black pixel and the white pixel is generated.

  Then, face detection processing is performed using the binarized data generated in step S8 (step S10). For the face detection process in step S10, a known face detection technique for detecting a face area from binarized data is used. As a face detection method, for example, there is a method of specifying an eye, nose and mouth in binarized data by pattern matching and detecting a face (area) including the specified eye, nose and mouth.

  Then, the automatic exposure control unit 42 determines whether or not a face is detected in step S10 (step S11). When a face is detected (step S11; YES), the automatic exposure control unit 42 calculates face area luminance information that is an average luminance value of the face area detected in step S10 (step S12). In step S12, for example, the coordinate information of the face area obtained in step S10 and the latest one frame of image data stored in the image memory 30 are sent to the signal processing unit 20 (note that it is sent out). The image data may be the image data used in the process of step S10, or the latest image data imaged and generated by the imaging unit 10). The signal processing unit 20 calculates the luminance information of the face area according to the input coordinate information of the face area and outputs it to the automatic exposure control unit 42. Similarly, in step S6, the face detection unit 41 causes the signal processing unit 20 to calculate the luminance information of the luminance evaluation area in the moving object area of the image data.

  Then, the automatic exposure control unit 42 performs automatic exposure control on the imaging unit 10 according to the luminance information of the face area calculated in step S12 (step S13), and the process proceeds to step S1. More specifically, the automatic exposure control unit 42 adjusts the aperture value of the aperture mechanism, the exposure time of the image sensor, and the sensitivity to appropriate values according to the luminance information of the face area. The imaging unit 10 is controlled to an appropriate exposure.

  When the face is not detected (step S11; NO), the automatic exposure control unit 42 calculates the luminance information of the photometric area that is the average luminance value of the default photometric area set in advance, and the calculated photometric area Depending on the luminance information, automatic exposure control is performed on the imaging unit 10 (step S14), and the process proceeds to step S1. The default photometric area is selected from, for example, the center area 301 located at the center of the image F shown in FIG. 7, the whole area 302 of the image F shown in FIG. 8, or a plurality of divided areas 303 obtained by dividing the image F. It may be an area or another area. In step S <b> 14, for example, the coordinate information and image data of the photometric area are input from the automatic exposure control unit 42 to the signal processing unit 20. The signal processing unit 20 calculates luminance information of the photometric area according to the input coordinate information of the photometric area, and outputs it to the automatic exposure control unit 42.

  As described above, according to the present embodiment, the face detection unit 41 detects a moving object from the image data image captured by the image capturing unit 10, and includes the detected moving object in the captured image data image. The degree of backlight based on the luminance information in the image is calculated, an offset value for binarization processing is determined according to the calculated degree of backlight, and an image of the captured image data is determined based on the determined offset value Is binarized, and the face of the subject person is detected from the binarized image. For this reason, it is possible to appropriately calculate the degree of backlighting by specifying a person as a moving body area, and it is possible to appropriately detect the face of the person who is captured even under backlighting.

  The face detection unit 41 also includes an average luminance value of the luminance evaluation areas 101 and 102 that do not include the moving object M in the moving object area 100 of the captured image, and an average luminance value of the luminance evaluation area 103 that includes the moving object M. Is calculated as the degree of backlight. For this reason, the degree of backlight can be calculated more accurately.

  In addition, the face detection unit 41 calculates a difference value between the average luminance value of the luminance evaluation areas 202 and 203 of the captured image and the average luminance value of the target pixel 201, and the calculated difference value and the calculation Compared with the offset value, binarized data is generated with a pixel of interest whose difference value is larger than the offset value as a black pixel and other pixels of interest as white pixels. For this reason, appropriate binarized data can be generated even under backlight using an offset value corresponding to the degree of backlight.

  Further, the automatic exposure control unit 42 determines whether or not a face is detected by the face detection unit 41, and when a face is detected, the exposure control according to the luminance information of the detected face area is captured. To part 10. For this reason, automatic exposure control can be appropriately performed even under backlight.

  The description in the above embodiment is an example of a suitable face detection apparatus and imaging apparatus according to the present invention, and the present invention is not limited to this.

For example, in the above embodiment, the difference value of the luminance information of the moving object area of the image is set as the degree of backlighting, and the binarized offset value is determined according to the degree of backlighting. However, the present invention is not limited thereto. It is not a thing. For example, the face detection unit 41 calculates the average luminance values of the luminance evaluation areas 101, 102, and 103 of the moving object area 100 in FIG. 4, depending on whether or not the following expressions (2) and (3) are satisfied. Then, it is determined whether the light is backlit.
Average luminance value of luminance evaluation area 101> Average luminance value of luminance evaluation area 103 (2)
Average luminance value of luminance evaluation area 102> Average luminance value of luminance evaluation area 103 (3)

  When the face detection unit 41 determines that the light is backlit, the offset value for binarization processing is set to a relatively low value (eg, 3), and when it is determined that the light is not backlit, the offset value May be set to a relatively high value (for example, 8). That is, (average luminance value of luminance evaluation area 101−average luminance value of luminance evaluation area 103) and (average luminance value of luminance evaluation area 102−average luminance value of luminance evaluation area 103) are respectively expressed by the first backlight. The offset value is set to 3 when the first backlight level and the second backlight level are both greater than 0. Otherwise, the offset value is set to 8. Is set.

  Moreover, in the said embodiment, although the imaging device 2 was set as the structure provided in the vending machine 1, it is not limited to this. For example, the imaging device 2 may be provided in another face detection device such as an ATM (Automated Teller Machine) such as a bank or a convenience store, or an authentication device for a visitor to a predetermined area.

  In the above embodiment, the face detection unit 41 and the automatic exposure control unit 42 are configured to cause the signal processing unit 20 to calculate the luminance information of the specific area of the image of the image data. However, the present invention is not limited to this. It is not a thing. The face detection unit 41 and the automatic exposure control unit 42 may be configured to calculate luminance information of a specific area of the image of the image data.

  Further, the detailed configuration and detailed operation of each part constituting the imaging device 2 in the above embodiment can be appropriately changed without departing from the spirit of the present invention.

DESCRIPTION OF SYMBOLS 1 Vending machine 2 Imaging device 10 Imaging part 20 Signal processing part 30 Image memory 40 Control part 41 Face detection part 42 Automatic exposure control part 3 Person 4 Sun

Claims (4)

  A moving object is detected from the captured image, a backlight level is calculated based on luminance information of a moving object area including the detected moving object, and an offset value for binarization processing according to the calculated backlight level is determined. And a face detection device including a face detection unit that binarizes the image based on the determined offset value and detects a face of a person as a subject from the binarized image.   The face detection unit calculates a difference value between an average luminance value of a luminance evaluation area that does not include the moving object in the moving object area and an average luminance value of the luminance evaluation area that includes the moving object in the moving object area. The face detection apparatus according to claim 1, wherein the face detection apparatus calculates the degree of backlight.   The face detection unit calculates a difference value between an average luminance value of a luminance evaluation area of the captured image and an average luminance value of a target pixel, and calculates the calculated difference value and the calculated offset value. The face detection apparatus according to claim 1 or 2, wherein the image is binarized by comparing the pixel of interest having a difference value larger than the offset value as a black pixel and the other pixel of interest as a white pixel. The face detection device according to any one of claims 1 to 3,
An imaging unit that captures the image;
An automatic exposure control unit that determines whether or not a face has been detected by the face detection unit, and performs exposure control on the imaging unit according to luminance information of the detected face area when a face is detected; An imaging apparatus comprising:

Images