JP2010028370A - Imaging device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an imaging device mounting a face detection function which makes processing faster without lowering the detection accuracy of a face. <P>SOLUTION: The imaging device comprises: an imaging means for outputting image data corresponding to a subject; and a face detection function which carries out a step scan which shifts a rectangular area of a predetermined image size by a predetermined step width to change target position inside the image data outputted from the imaging means, while evaluating whether or not small image data contains a person's face for each small image data in the rectangular area at each step position to detect the person's face (S104 to S108). The face detection function sets a step width to a next rectangular area based on an evaluation value (the number of cleared hierarchies L) as a degree of certainty containing the person's face in the evaluation made in the rectangular area at a certain step position (S110), to carry out the step scan. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to an imaging apparatus such as a digital camera provided with an imaging element that acquires image data.

  In recent years, in an imaging apparatus such as a digital camera, a function that is controlled so that a face area detected by detecting a human face in a captured image is focused and exposed, and a detected face are registered in advance. When these functions are realized by software processing, the processing becomes heavy, which takes processing time, and may not be suitable for actual use. As a method for lightening the processing, simply using a low-resolution image or performing a thinning-out processing makes the processing light, but this reduces the accuracy of detection or authentication. In view of this, various methods have been developed in order to increase the processing speed or accuracy when realizing face detection and face authentication processing by software.

  For example, in Patent Document 1, by using a lookup table or the like, a determination value is associated with each feature amount independently, thereby realizing high processing accuracy. Speeding up is realized by reducing it. However, in this method, since all regions in the target image are scanned with the same step width, it can be said that a scan that is obviously not a face is wasted.

  Further, in Patent Document 2, the face size of the subject is specified based on the distance information obtained from the focusing operation, and the face detection operation is performed within an area defined by the face size, thereby reducing unnecessary scanning. Is possible. However, this method has a problem that the focusing operation has to be performed once.

JP 2004-362468 A JP 2006-25238 A

  The present invention has been made in view of the above problems in the prior art, and an object of the present invention is to provide an imaging apparatus equipped with a face detection function that speeds up processing without reducing the accuracy of face detection.

The present invention provided to solve the above problems is as follows.
[1] A rectangular area (rectangular area S) having a predetermined image size is predetermined in an imaging unit (imaging sensor 104) that outputs image data corresponding to a subject and image data (image 20) output from the imaging unit. While performing step scan that changes the target position by shifting the step width (step width St), it is evaluated whether or not the small image data includes a human face for each small image data of the rectangular area at each step position. And a face detection function (S104 to S108, S203 to S205) for detecting a human face, wherein the face detection function is based on the evaluation performed in a rectangular area at a certain step position. A step width to the next rectangular area is set (S110, S207) based on the evaluation value that is the degree of probability including the human face (step S110, S207). An image pickup apparatus (FIGS. 2, 3, and 7) characterized by
[2] The face detection function hierarchically determines whether or not the small image data in the rectangular area includes a person's face, and if the determination at the highest layer (LayerN) is cleared, The imaging apparatus according to [1], wherein the image is evaluated to include a face, and the evaluation value is determined based on the number of cleared layers (hierarchy L) (FIG. 6).
[3] The face detection function obtains a similarity between the small image data in the rectangular area and template data (face feature data) that is facial feature data. The image pickup apparatus according to [1], wherein the evaluation value is determined based on a similarity to the template data (FIG. 8).
[4] The face detection function according to the above [2] or [3], wherein the face detection function sets a step width to the next rectangular area in a relationship inversely proportional to the evaluation value and performs step scan Imaging device.

As an effect of the present invention, according to the invention of claim 1, for example, when the evaluation value of the rectangular area is lower than a predetermined level, that is, when the possibility of being a face is low, the area is not a face (or does not include a face). Therefore, the next rectangular area does not overlap with the original rectangular area or has a step width that reduces the overlap, thereby reducing the amount of calculation and speeding up the detection process.
According to the invention of claim 2, when the evaluation value of the rectangular area (the number of cleared hierarchies) is lower than a predetermined level (the highest hierarchy), that is, when the possibility of being a face is low, the area is not a face ( The next rectangular area does not overlap the original rectangular area, or the step width is such that the overlap is small, thereby reducing the amount of calculation and speeding up the detection process. And In addition, by determining hierarchically, a rectangular area that is unlikely to be a face is determined not to be a face (not including a face) at an early stage and moves to the next rectangular area. The number of detection processes can be reduced and the detection process can be speeded up.
According to the invention of claim 3, when the evaluation value (similarity with the template data) of the rectangular area is lower than a predetermined level (threshold), that is, when the possibility of being a face is low, the area is not a face (face The next rectangular area does not overlap with the original rectangular area, or the step width is such that the overlap becomes small, thereby reducing the amount of calculation and speeding up the detection process. To do.
According to the invention of claim 4, by setting the step width to the next rectangular area in a relationship inversely proportional to the evaluation value, the evaluation value of the rectangular area is lower than a predetermined level (the possibility of being a face). When the area is low, the area is determined not to be a face (does not include a face), and the next rectangular area can be set to a step width that does not overlap with the original rectangular area or the overlap becomes small. It is possible to reduce the amount and speed up the detection process.

The configuration of the imaging device according to the present invention will be described below.
FIG. 1 is an external view of a digital still camera which is an aspect of an imaging apparatus according to the present invention, and FIG. 2 is a system schematic diagram inside the digital still camera.

First, an outline of the operation of the digital still camera will be described.
The digital still camera (hereinafter referred to as camera) 10 is activated in the recording mode by setting the shooting / playback switching mode dial 2 in FIG. 1 to the shooting mode and pressing the power button 1. The mode dial 2 and the power button 1 are included in the operation unit 107 in FIG. 2, and when the CPU 105c detects that the power button 1 is turned on while the mode dial 2 is in the shooting mode, the CPU 105c The driver 108 is controlled to move the lens barrel unit to the photographing enabled position. Furthermore, power is turned on to the image sensor 104, the LCD display 5, and the like to start the operation.

When the power of each part is turned on, the operation in the finder mode is started.
In the finder mode, light incident on the light receiving sensor (CCD, CMOS) 104a through the lens 101 is converted into electricity and sent to the A / D converter 104b. The A / D converter 104b converts the input analog signal into 12-bit digital data. Each signal converted into a digital signal is taken into the sensor I / F unit 105a in the digital signal processing IC 105, converted into a YUV signal in a format that can be displayed by the YUV conversion unit 105d, and framed by the memory controller 105b. It is written in the memory 106. The YUV signal is read by the memory controller 105b and sent to the TV or LCD monitor via the display output control unit 105f for display. This process is performed at 1/30 second intervals, and a finder mode display updated every 1/30 seconds is obtained.

  The image sensor 104 is a sensor having 8 million pixels. During still image shooting, all pixels can be output every 1/30 seconds. In the finder mode, output is performed by reducing the number of pixels to 640 × 480 pixels every 1/30 seconds by combining peripheral pixels with addition and thinning.

  The sensor I / F unit 105a of the digital signal processing IC 105 detects an AF evaluation value that indicates the degree of focus on the screen, an AE evaluation value that detects the subject brightness, and a subject color from the digital RGB signal captured in the block. The calculated AWB evaluation value is calculated. These data are read out as feature data to the CPU 105c and used for each of AE, AF, and AWB processes.

  The AF evaluation value is created by, for example, an output integrated value of a high frequency component extraction filter or an integrated value of a luminance difference between adjacent pixels. When in the in-focus state, the edge portion of the subject is clear, so the high frequency component is the highest. By utilizing this, at the time of the focus detection operation by AF, the AF evaluation value at each focus lens position is acquired, and AF is executed with the point where the maximum is obtained as the focus position.

  The AE evaluation value and the AWB evaluation value are created from the respective RGB integration values. For example, the screen is equally divided into 256 areas (16 horizontal divisions and 16 vertical divisions), and the RGB integration of each area is calculated. The CPU 105c reads the RGB integrated values, and in AE, the brightness of each area is calculated, and an appropriate exposure time is determined from the brightness distribution. In AWB, an AWB control value that matches the color of the light source is determined from the RGB distribution. The AE and AWB processes are continuously performed during the finder mode.

  When the release button (shutter button) 3 shown in FIG. 1 is operated, an AF operation for detecting a focus position and a still image recording process are performed.

  When the release button 3 is pressed, when a still image shooting start signal is taken into the CPU 105c from the camera operation unit 107 in FIG. 2, the CPU 105c drives the lens 101 via the motor driver 108 in synchronization with the frame rate, thereby performing hill-climbing AF. Execute. When the AF target range is the entire region from infinity to close, the focus lens moves to each focus position from close to infinity, or from infinity to close, and each frame (= each) created by the digital signal processing IC 105 The CPU 105c reads the AF evaluation value at the (focus position). The focus lens is moved to the in-focus position with the point at which the AF evaluation value at each focus position is maximized as the in-focus position.

  It is converted into a digital RGB signal output from the image sensor 104 after the AF is completed, and stored in a frame memory (SDRAM) 106 via a digital signal processing IC 105. The digital RGB signal is read again into the digital signal processing circuit, converted into YUV data by the YUV converter 105d, and written back to the frame memory 106.

  At the time of capturing a still image, the YUV converted image data is sent to the image compression / decompression circuit 105e in the digital signal processing IC 105. The YUV data sent to the image compression / decompression circuit 105 e is compressed and written back to the frame memory 106. The compressed data in the frame memory 106 is read out via the digital signal processing IC 105 and stored in the data storage memory.

Next, a face detection method will be described.
As a method for detecting a person image from a subject image, the following method is known, and in this embodiment, any one of the methods is used.
(1) Television Society Journal Vol. 49, no. 6, pp. No. 787-797 (1995) “Proposal of a modified HSV color system effective for face area extraction” is a method of extracting a face area by converting a color image into a mosaic image and paying attention to a skin color area.
(2) Journal of the Institute of Electronics, Information and Communication Engineers, Vol. 74-D-II, no. 11, pp. 1625-1627 (1991) “A method for extracting a face region from a still gray scene image”, the geometrical structure of each part constituting the head of a front human figure such as hair, eyes and mouth. Of extracting the head region of a frontal person using various shape features.
(3) As shown in “Video phone face area detection and its effect” in Image Lab 1991-11 (1991), in the case of a moving image, the contour edge of a person image generated by a subtle movement of the person between frames To extract a frontal portrait using GIS.

(First embodiment)
Next, face detection processing, which is an operation that characterizes the present invention, will be described.
FIG. 3 shows a flow of face detection processing for detecting a human face from images by performing face detection processing on the image captured by the imaging optical system of the camera 10 of FIGS. ). Note that the camera 10 has a face detection function (face detection module), and a face detection processing mode for operating the face detection module can be selected from a menu displayed on the LCD display 5, for example.

  When the operation mode of the camera 10 is a mode for performing face detection processing, the face detection module copies one piece of image data (hereinafter also referred to as an image) from the monitoring image at a certain timing (S101). The CPU 105c performs face detection processing on the copied image and determines whether or not there is a human face in the image. Here, the face detection module hierarchically determines whether or not a human face is included in the small image data in the target rectangular area, and clears the determination in the highest hierarchy (final hierarchy). It is evaluated that it includes a human face. That is, for a plurality of hierarchies from a lower hierarchy determined based on the rough features of the face to an upper hierarchy determined based on the finer features, the small image data clears the determination in the corresponding hierarchy. In this case, a step-by-step determination of going up to the next higher level and receiving a determination is received, and if the determination at the corresponding layer is NG, the evaluation is completed at that step.

  In the face detection process, the similarity between the facial feature data of the person stored in the memory (SDRAM 106) in advance and the symmetrical rectangular area (small image data) in the image is calculated. The feature data in the memory (SDRAM 106) is stored in a plurality of layers, and the similarity of the feature data is calculated for each layer.

  Here, the feature data is, for example, two or three small rectangles (S1, S2 in FIG. 4A, S3, S4 in (b), (c) in the target rectangular area S as shown in FIG. In S5 to S7), the degree of similarity is the data when the weighted sum (feature evaluation value) of the average luminance value of each small rectangle is equal to or greater than the threshold value. In this embodiment, the weighted sum of the average luminance values of the small rectangles is called a feature evaluation value.

  The feature data in the shallow hierarchy (lower hierarchy) is a relatively rough judgment as to whether the target rectangular area is a face (or whether it includes a face). For example, as shown in FIG. The small rectangles S1 and S2 in the target rectangular area S are held as large ones, and the number of feature data is small. Then, the small rectangles S3 to S7 as shown in FIGS. 4B and 4C are made small so that the feature of the face is judged as the hierarchy becomes deeper, and the number of feature data in the hierarchy is large. It will become.

  The similarity in each hierarchy is calculated by obtaining a feature evaluation value in each feature data in the hierarchy. The more the feature data whose feature evaluation value exceeds the threshold value, the higher the similarity. Assuming that the feature evaluation value of the feature data j in the hierarchy i is Evj and Evj is equal to or greater than the threshold Thj, and the similarity is simj, the similarity Simi of the hierarchy i is calculated by the following equation (1).

  In FIG. 3, the first target rectangular area is determined from the image data (S102). Here, it is the upper left edge of the image. In this embodiment, as shown in FIG. 5, step scanning is performed while the target rectangular area S is gradually shifted by a predetermined step width from the upper left end of the image 20 to the lower right end.

  In the target rectangular area S, first, the first hierarchy (Layer 1) is selected with the hierarchy L = 1 (S103), and the similarity between the feature data of the first hierarchy and the target rectangular area S is calculated (S104). Here, if the similarity is greater than or equal to a predetermined threshold, the target rectangle is assumed to match the feature data of that hierarchy (clear determination). If it matches the feature data and the hierarchy is the final hierarchy, it is determined that the rectangular area is a face (or includes a face), and information on the rectangular area S is registered in the memory (SDRAM 106) (S108).

  If the hierarchy is not the final hierarchy, the hierarchy L is set to L = L + 1 (S107), and the similarity between the target rectangular area S and the feature data of the next hierarchy is calculated.

  If the feature data in the hierarchy is not matched in step S105 (NG determination), it is determined that the target rectangular area S is not a face (does not include a face) at that time, and the process proceeds to step S109.

  If it is evaluated whether or not the target rectangular area S is a face (whether or not it includes a face), it is confirmed whether or not the entire face detection target area in the image 20 has been scanned (S109). Here, if the area to be scanned still remains, the next rectangular area S ′ is based on the evaluation value which is the degree of probability including the human face in the evaluation performed in the immediately preceding target rectangular area S. Set the step width to. That is, the step width to the next rectangular area is determined according to the hierarchy L (number of cleared hierarchies) for which the similarity with the feature data is calculated in the immediately preceding rectangular area S, and the next rectangular area S ′ is determined (S110). ).

FIG. 6 shows an example of a change in the step width to the next rectangular area S ′ according to the hierarchy L.
FIG. 6A shows a case where the NG determination is made in the higher hierarchy, here, all the hierarchies N, in the hierarchy N-1, and the possibility of including a face around the rectangular area S is high. The step width to the rectangular area S ′ is small, and the overlap with the previous target rectangular area S is large. On the other hand, FIG. 6B shows a case where an NG determination is made in the lower hierarchy, here, the third hierarchy. Since there is a low possibility of including a face around the rectangular area S, the next target rectangular area S ′ is displayed. The step width is large, and there is no overlap with the previous target rectangular area S.

  At this time, the step width to the next rectangular area S ′ may be set in a relationship inversely proportional to the evaluation value. Alternatively, the step width may be reduced if the evaluation value is large, and the step width may be increased if the evaluation value is small. In this embodiment, when scanning is performed in a rectangular area S having a horizontal width W, the step width St to the next rectangular area in the case of NG in the hierarchy n is St = W * (N−n) / N It shall be calculated by

  When it is evaluated whether or not a certain target rectangular area is a face, if the entire face detection target area has been scanned (Yes in S109), the face detection processing for the image ends, and then face detection is performed. When the mode is continued (Yes in S111), the process returns to step S101, a new image is copied from the monitoring data, and face detection processing is similarly performed on the image. If the face detection mode is not continued (No in S111), the process ends here (END).

  As described above, when the evaluation value (the number of cleared layers) of the target rectangular region S is lower than a predetermined level (the highest layer), that is, when the possibility of being a face is low, the region is not a face (the face is The next rectangular area S ′ is not overlapped with the original rectangular area S, or the step width St is such that the overlap is reduced, thereby reducing the amount of calculation and speeding up the detection process. Can do. Further, by determining the target rectangular area S in a hierarchical manner, a rectangular area having a low possibility of being a face is determined not to be a face (not including a face) at an early stage, and the next rectangular area is determined. Therefore, the amount of calculation is reduced, and the detection process can be speeded up.

(Second Embodiment)
FIG. 7 shows a flow of face detection / face authentication processing in the second embodiment.
When the operation mode of the camera 10 is a mode for performing face detection processing, the face detection module copies one piece of image data (hereinafter also referred to as an image) from the monitoring image at a certain timing (S201). The CPU 105c performs face detection processing on the copied image, and determines whether there is a human face in the image.

  In the face detection process, the sum of the similarities between the feature data of a person's face that is stored in advance in the memory (SDRAM 106) and the symmetric rectangular region S in the image 20 is calculated as the similarity between the target rectangular region and the feature data. Shall be. Here, the facial feature data and the similarity calculation method are the same as those in the first embodiment.

  Next, the first target rectangular area is determined from the image (S202). Here, as in the first embodiment, the upper left edge of the image is used. In this embodiment, as in the first embodiment, step scanning is performed while gradually shifting the target rectangular area S by a predetermined step width from the upper left end to the lower right end of the image 20 as shown in FIG. Shall.

  Here, the face detection module obtains the similarity between the small image data and the template data (face feature data) in the target rectangular area S, and evaluates that the human face is included if the similarity is equal to or greater than a threshold value. That is, the similarity with the feature data is calculated for the target rectangular area S first determined in step S202 (S203). Next, the similarity is compared with a preset threshold value (S204), and if the similarity is equal to or greater than a predetermined threshold value, the target rectangular area matches the facial feature data (clear determination, S204). Yes). When the target rectangular area S matches the feature data, the target rectangular area is registered as the face (or includes a face) in the memory (SDRAM 106) (S206).

  If it is determined whether or not the target rectangular area is a face, and there is an unscanned area in the face detection target area in the image 20 (No in S206), the process was performed in the immediately preceding target rectangular area S. A step width to the next rectangular area S ′ is set based on an evaluation value that is a degree of probability including the human face in the evaluation. That is, the step width for determining the next target rectangular area S ′ is calculated based on the similarity to the template data (feature data) in the immediately preceding rectangular target area S (S207).

  FIG. 8 shows an example of changing the step width based on the similarity of the target rectangular area. Here, the width of the immediately preceding target rectangular area S is W, the similarity is Simi, and the step width St corresponding thereto is calculated as St = W / Simi as shown in FIG. The rectangular area S ′ is an area shifted to the right by the step width St from the immediately preceding target rectangular area S (FIG. 8B).

  When it is determined whether or not a certain target rectangular area is a face, if the entire face detection target area has been scanned (Yes in S206), the face detection process for the image is terminated and the face detection mode is completed. Is continued (Yes in S208), the process returns to step S201, a new image is copied from the monitoring data, and the face detection process is similarly performed on the image. If the face detection mode is not continued (No in S208), the process ends here (END).

  As described above, when the evaluation value (similarity with template data (feature data)) of the target rectangular area S is lower than a predetermined level (threshold), that is, when the possibility of being a face is low, the area is not a face. Since the next rectangular area S ′ is determined not to include a face, the next rectangular area S ′ is not overlapped with the original rectangular area S, or the step width St is set so as to reduce the overlap, thereby reducing the amount of calculation and speeding up the detection process. Can be

  Although the present invention has been described with the embodiments shown in the drawings, the present invention is not limited to the embodiments shown in the drawings, and other embodiments, additions, modifications, deletions, etc. Can be changed within the range that can be conceived, and any embodiment is included in the scope of the present invention as long as the effects and advantages of the present invention are exhibited.

1 is an external view illustrating a configuration of a digital camera that is an imaging apparatus according to the present invention. It is a block diagram which shows the outline | summary of the imaging system of the digital camera of FIG. 6 is a flowchart illustrating a processing procedure in a face detection mode according to the first exemplary embodiment. It is explanatory drawing of the characteristic data used by this invention. It is a figure which shows the mode of the step scan in image data. It is explanatory drawing of the example of a change of the step width according to the evaluation value in Example 1. FIG. 12 is a flowchart illustrating a processing procedure in a face detection mode according to the second embodiment. It is explanatory drawing of the example of a change of the step width according to the evaluation value in Example 2. FIG.

Explanation of symbols

1 Power button 2 Shooting / playback dial 3 Release button 4 Optical viewfinder 5 LCD display 6 Shooting lens 7 Flash 10 Camera (digital camera)
20 images (image data)
101 lens (zoom / AF lens)
102 Aperture and Color Filter 103 Mechanical Shutter 104 Image Sensor 104a Light Receiving Sensor 104b A / D Converter 105 Signal Processing IC
105a Sensor I / F part 105b Memory controller 105c CPU
105d YUV conversion unit 105e Image compression / decompression circuit 105f Display output control unit 105g Resize processing unit 105h Media I / F unit 106 SDRAM (memory)
107 Operation unit 108 Motor driver 109 ROM
S, S 'rectangular area S1, S2, S3, S4, S5, S6, S7 small rectangle

Claims (4)

Imaging means for outputting image data according to the subject;
In the image data output from the imaging means, while performing a step scan that changes the target position by shifting a rectangular area of a predetermined image size by a predetermined step width, the small image data of the rectangular area at each step position A face detection function for evaluating whether the small image data includes a person's face and detecting the person's face,
The face detection function sets a step width to the next rectangular area based on an evaluation value that is a degree of certainty including a human face in the evaluation performed in the rectangular area at a certain step position, An imaging apparatus that performs scanning. The face detection function hierarchically determines whether or not the small image data in the rectangular area includes a human face, and evaluates that the human face is included if the determination in the highest hierarchy is cleared. Is,
The imaging apparatus according to claim 1, wherein the evaluation value is determined based on a cleared number of hierarchies. The face detection function obtains the similarity between the small image data in the rectangular area and the template data that is facial feature data, and evaluates that the human face is included if the similarity is equal to or greater than a threshold value.
The imaging apparatus according to claim 1, wherein the evaluation value is determined based on a similarity with the template data.   The imaging apparatus according to claim 2, wherein the face detection function performs step scanning by setting a step width to the next rectangular area in a relationship inversely proportional to the evaluation value.

Images