JP5189913B2 - Image processing device - Google Patents

Description

  The present invention relates to an image processing apparatus, and more particularly to a technique for detecting a human face from a subject image obtained by photographing.

  A technique for detecting a human face from a subject image obtained by photographing with a digital camera or the like is known. For example, Patent Document 1 below discloses that a face image is detected from a subject image, and a circular, square, or oval icon is superimposed on the image and displayed in correspondence with the face image. When detecting a face, it is described that human eyes are detected from an image and feature points that are facial organs (main parts) other than the eyes are extracted.

JP 2008-98755 A

  Generally, when detecting a human face from a subject image, there is a negative correlation between the size of the face to be detected and the processing time required for detection. That is, the processing time required for detection increases as the size of the face to be detected decreases, and the processing time required for detection decreases as the size of the face to be detected increases.

  On the other hand, the closer the distance to the person, the larger the face size in the subject image of the person, and the higher the detection priority than when the distance is far and the face size is small.

  Therefore, when detecting a human face, it first detects a large face that requires processing time but is relatively high in size, then detects a medium-sized face, and finally It is conceivable to perform processing so as to detect a face having a small size and output the result as a detection result.

  However, it is assumed that a large-sized face cannot be detected first and then a process of detecting a medium-sized face is detected, and then a large-sized face appears in the subject image (digital camera). Panning or zooming in), this face cannot be detected in the process of detecting a medium-sized face, and even if the process moves to the process of detecting the next smaller face, the face still cannot be detected. For the first time in the process of moving to the next detection cycle and detecting a large face again for the first time, after passing through a process for detecting a medium face and a process for detecting a small face after that, Since it is output as a detection result, the timing of detecting and outputting a large face is delayed as a result.

  The result of face detection is used for controlling AE (automatic exposure control), AF (automatic focus control), AWB (automatic white balance), etc., in addition to displaying icons superimposed on the subject image as in the prior art. Therefore, the face detection timing delay causes a delay in the control of these AE, AF, and AWB. Although it is effective to increase the processing speed of the CPU of the face detection circuit or the face detection engine and shorten the detection cycle, the processing load on the CPU increases or the cost increases.

  An object of the present invention is to provide an imaging apparatus capable of quickly detecting a large face.

  The present invention is an image processing apparatus that detects a human face from a subject image, an imaging unit that captures the subject image, and a detection unit that detects the face from the subject image, and is relative to at least the face. A process for detecting a large face and a process for detecting a relatively small face. First, a process for detecting a relatively large face is performed, and a process for outputting the detection result is performed. And detecting means for executing a process of detecting a relatively small face after that is repeated a plurality of times and outputting the detection result.

  In one embodiment of the present invention, the detecting means detects a relatively large face as the face, detects a relatively medium face, and detects a relatively small face. First, execute a process to detect a relatively large face and output the detection result multiple times, and then execute a process to detect a relatively medium size face. The detection result is output, the process of detecting a relatively large face is executed, and the process of outputting the detection result is repeated a plurality of times, and then the relatively small face is detected. Execute the process and output the detection result. In addition, the detection unit first repeats the process of detecting a relatively large face and outputting the detection result three times, and thereafter performing the process of detecting a relatively medium size face. Execute and output the detection result, then repeat the process of detecting the relatively large size face and outputting the detection result three times, then detect the relatively small size face The detection result is output.

  The present invention is also an image processing apparatus for detecting a human face from a subject image, an imaging means for capturing the subject image, and a detection means for detecting the face from the subject image, at least as the face It has a process to detect a relatively large face and a process to detect a relatively small face. First, it executes a process to detect a relatively large face and outputs the detection result. Then, the process of detecting a relatively small face is executed and the detection result is output, and then the process of detecting a relatively large face is executed again and the result is output. And detecting means for executing processing for detecting a relatively small face again and outputting the result.

  In one embodiment of the present invention, the detection unit detects a relatively small size face and a process of detecting a relatively large size face as a face and a process of detecting a relatively medium size face. First, execute a process of detecting a relatively large face and output the detection result, and then execute a process of detecting a relatively medium size face and detect the result. Is output twice, then the process of detecting a relatively large face is executed and the result is output, and then the process of detecting a relatively small face is executed and The process of outputting the result is repeated twice.

  Further, in another embodiment of the present invention, the detection means detects a face while masking a predetermined area on both left and right sides of the subject image and excluding it from the processing target, and detects a plurality of the large-sized faces. A face is detected without performing masking in any of the detection processes.

  The present invention is also an image processing apparatus for detecting a human face from a subject image, an imaging means for capturing the subject image, and a detection means for detecting the face from the subject image, at least as the face A process for detecting a relatively large size face, a process for detecting a relatively medium size face, and a process for detecting a relatively small size face. The process of masking and detecting a relatively large face is executed three times and the detection result is output three times. Thereafter, a predetermined area on both the left and right sides of the subject image is masked to obtain a relatively medium size. Executes processing to detect a face and outputs the detection result, and then executes processing to detect a relatively large face by masking a predetermined area on both the left and right sides of the subject image and outputs the detection result Do The process is repeated twice, and after that, a process of detecting a relatively large face without masking is executed and the detection result is output, and then a predetermined area on both the left and right sides of the subject image is masked to make a relative And detecting means for executing processing for detecting a small-sized face and outputting the detection result.

  According to the present invention, a large face can be quickly detected from a subject image.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings, taking a digital camera as an example of an imaging apparatus. However, the present invention is not limited to a digital camera, and can be applied to other imaging devices such as a video camera.

  FIG. 1 shows a configuration of a digital camera in the present embodiment. The photographing optical system 10 generally shown as a lens includes a photographing lens, a diaphragm, and a mechanical shutter in detail. The photographing lens further includes a focus lens, a zoom lens, and an iris. Each lens and shutter of the photographing optical system 10 are driven by a motor or a driver in accordance with a command from the AE / AF control unit 22.

  The image sensor 12 converts the subject image formed by the photographing optical system 10 into an electrical signal and outputs it as R, G, and B image signals. The image pickup device 12 is configured by a CCD sensor or a CMOS sensor, and accumulated charges are sequentially read according to a timing signal from a timing generator (TG).

  The analog front end (AFE) 14 is an analog signal processing unit, which removes fixed pattern noise by taking the difference between the voltage value at the time of amplification, gain adjustment, reset and the voltage value at the time of signal output for the analog image signal. A correlated double sampling (CDS) process is performed.

  The AD converter 16 converts the image signal supplied from the analog front end (AFE) 16 into a digital signal.

  The image processing circuit 18 includes a CPU, and includes known gain correction, gamma correction, synchronization processing, RGB-YC separation, noise reduction processing, edge enhancement processing, and JPEG compression processing as functional blocks. On the other hand, gain correction, gamma correction, synchronization processing, and the like are executed to generate image data of Y and C signals. The gain correction is to adjust the gray balance by correcting a different gain for each channel of the input RAW data (spatial incomplete sampling data before RGB imaging). The gamma correction is to match the characteristics of an image sensor such as a CCD with the input / output characteristics of the LCD. The synchronization process generates an RGB3 image from a single-layer CCD with a Bayer array. The generated image data is stored in the memory card 20. In addition, a predetermined decompression process is performed on the compressed image data to generate uncompressed image data. The uncompressed image data is displayed on the LCD 24 as a captured image. Further, the image processing circuit 18 includes a circuit for generating a preview image to be displayed on the LCD 24, a circuit for detecting a face image from the subject image, and an AWB (auto white balance) circuit. An organic EL may be used instead of the LCD 24.

  The AE / AF control unit 22 includes a CPU, and calculates physical quantities necessary for AE (automatic exposure control) and AF (automatic focus control). Specifically, the brightness of the subject is detected from the image signal, and the distance to the subject is detected. Appropriate exposure is determined from the brightness of the subject, and the aperture value, shutter speed, and ISO sensitivity are determined.

  In addition to a single processor, the image processing circuit 18 may include an MPU that controls the entire system and a DIP (digital image processor) that processes image data at a high speed as its sub-processor.

  FIG. 2 shows a part of functional blocks of the image processing circuit 18. The image processing circuit 18 includes a preview image generation unit 18a, a face detection unit 18b, and an AWB unit 18c. The preview image generation unit 18 a creates a preview image by the above-described processes and outputs it to the LCD 24. A preview image is an image obtained by thinning out image data. The face detection unit 18b detects a human face from a preview image as a subject image. The method of detecting the face is arbitrary, and it is possible to detect the human eye from the subject image, or to detect the feature of the face other than the eyes (the nose, mouth, hair, face contour shape), the elliptical shape of the head And so on. These may be detected in combination with each other. Alternatively, the skin color portion may be extracted from the preview image, the degree of matching between the face template and the skin color portion image may be obtained, and the highly matched portion may be detected as a face. As described above, although the face detection method itself is arbitrary, the face detection unit 18b in the present embodiment detects the face from the large face to the medium face, and then from the small face to the face in the conventional manner. Instead of outputting the result, the face is detected in a specific order. The face detection order will be further described later. The face detection unit 18 b outputs the face detection result to the AWB unit 18 c and the AE / AF control unit 22. The AWB unit 18c determines the light source based on the R / G and B / G distributions of the image signal in the facial region detected by the face detection unit 18b. A gain value for each of the R, G, and B signals is determined from the determined light source. The AE / AF control unit 22 determines the appropriate exposure and the proper focus value by detecting the brightness and the distance to the face of the face portion detected by the face detection unit 18b.

  The AF control of the AE / AF control unit 22 is any one of face AF that follows (tracks) while focusing on the detected face, multi-AF, and area tracking AF that focuses on and tracks the subject in the center of the screen. Alternatively, the focus control may be performed by selectively switching. The distance measurement method may be either a contrast detection method or a phase detection method. The number of AF points for multi-AF is arbitrary, and may be any of 3, 5, 7, or more.

  In addition, the image processing circuit 18 includes various memories such as a ROM, an EEPROM, and a RAM. The ROM stores a control program including an OS executed by the CPU of the image processing circuit 18 and various data. The face detection process executed by the face detection unit 18b is also executed by this control program. User setting data is stored in the EEPROM. The RAM operates as a working memory for the CPU.

  FIG. 3 schematically shows face detection processing in the face detection unit 18b. When detecting the face from the preview image, the face detection unit 18b divides the size of the face to be detected into three levels, large, medium, and small, and detects them in a predetermined order. Of course, large, medium, and small are relative, and for example, 60 to 240 are set as large, 40 to 60 are set as medium, and 30 to 40 are set as small. In addition, when the preview image is, for example, 320 pixels wide × 240 pixels high, a predetermined area on both the left and right sides is masked (masking), and an area of 240 pixels wide × 240 pixels high, which is a substantially central area of the preview image. Is a search target area for face detection. However, it is not desirable to limit all face detection processing to only the masked area. This is because a human face may exist at the end of the subject image. Of course, all areas of the preview image may be set as search target areas for face detection without performing masking. For masking, it is only necessary to exclude predetermined areas at the left and right ends from the search target.

  In FIG. 3, first, the face detection unit 18b detects a large-sized face after masking the left and right ends of the preview image. The detection method is as described above. For example, the skin color portion is extracted from the subject image, the matching degree between the face template and the skin color portion image is obtained, and the highly matching portion is detected as the face. When detecting a large face, a large template is prepared. Then, the face detection result is output (first detection step). The detection result is, for example, the detected face size and detection position. After outputting the face detection result, the preview image is masked again to detect a large size face and output the face detection result (second detection step). If a large-sized face is detected in this detection step, tracking control is performed so as to follow the detected face. That is, even when the detected face portion moves, the moved face is continuously detected. The tracking control includes, for example, searching around the detected face position by pattern matching, or calculating a motion vector and tracking the face based on the motion vector. Techniques for performing AE and AF following the detected face are known. When the tracking cannot be performed, retry is performed a predetermined number of times (for example, 5 times), and when the retry fails, the tracking control is stopped and the process returns to the face detection algorithm again. Next, after masking the preview image again, a large face is detected, and a face detection result is output (third detection step).

  After the large-size face detection process is repeated a total of three times, the preview image is masked and a medium-size face is detected. When detecting a medium-sized face, a medium-sized template is prepared. Then, the face detection result is output (fourth detection step). If a medium-sized face is detected in this detection step, tracking control is performed so as to follow the detected face. When the follow-up cannot be performed, retry is performed a predetermined number of times, and when the retry fails, the process returns to the face detection algorithm again. Next, after masking the preview image, a large face is detected again. Then, the face detection result is output (fifth detection step). Next, after masking the preview image, a large face is detected again. Then, the face detection result is output (sixth detection step). Next, a large-sized face is detected without masking the preview image, that is, by searching all 320 pixels × 240 pixels. Then, the face detection result is output (seventh detection step). The reason why masking is not performed in the seventh detection step is that face images may exist in the areas masked in the previous detection steps, that is, the left and right end portions. If a large face is detected in this detection step, tracking control is performed so as to follow the detected face. When the follow-up cannot be performed, retry is performed a predetermined number of times, and when the retry fails, the tracking control is stopped and the process returns to the face detection algorithm again. Needless to say, when the detected face is followed, it is executed without masking. That is, the tracking range is the entire preview image. Finally, a small face is detected after masking the preview image. Then, the face detection result is output (eighth detection step). When a small face is detected in this detection step, tracking control is performed so as to follow the detected face. If the tracking cannot be performed, retry is performed a predetermined number of times. If the retry fails, the process returns to the face detection algorithm again. After executing the eighth detection step, the process returns to the first detection step again, and the same detection process is repeated.

  As described above, the face detection unit 18b performs the face detection process in the order from the first detection step to the eighth detection step. In the first, second and third detection steps, a large size face is detected, and in the fourth detection step, a medium size face is detected. In the fifth, sixth, and seventh detection steps, a large face is detected again, and in the eighth step, a small face is detected. In each of the first to sixth and eighth detection steps, face detection is performed by masking the left and right ends of the preview image, and in the seventh detection step, face detection is performed without masking the left and right ends of the preview image.

  FIG. 4 shows face detection processing in the present embodiment. FIG. 5 shows a conventional face detection process for comparison. Conventionally, a large-sized face to a small-sized face are detected in a certain detection cycle T, and the detection result is output. Then, in the next detection cycle T, detection is again made from a large size face to a small size face, and the detection result is output. Therefore, if a large-sized face appears in the subject image after shifting to detection of a medium-sized face without detecting a large-sized face in the detection period T, the detection period T The detection result of the large size face cannot be output, and the detection result of the large size face is obtained only when the next detection cycle T is completed.

  On the other hand, in the present embodiment, as shown in FIG. 4, the first detection step and the second detection step are executed in the detection cycle T, the third detection step and the fourth detection step are executed in the next detection cycle T, and In the detection cycle T, the fifth detection step and the sixth detection step are executed, and in the next detection cycle T, the seventh detection step and the eighth detection step are executed. In each of the first detection step to the third detection step, a large-sized face is detected, and the detection result is output when each detection step is completed. Even when a face appears, it is possible to detect the large-sized face in the second detection step or the third detection step and quickly output the result.

  FIG. 6 shows the output timing of face detection in this embodiment and the prior art. Assume that a large-sized face appears in the subject image at time t1. In the prior art, face detection is output after 2T. On the other hand, in this embodiment, face detection is output after 3 / 4T.

  In the present embodiment, the first detection step to the sixth detection step and the eighth detection step mask the preview image and limit the search range for face detection to the central area of the preview image. It is possible to detect an existing face efficiently and quickly, and in the seventh detection step, the entire preview image is used as a search range without masking, so that the face located at the end can be detected reliably. Can do. As described above, when a large size face is detected, the processing time is shorter than when a medium size or small size face is detected. Therefore, the processing time does not increase greatly even if masking is not applied when detecting a large face in the seventh detection step. That is, it is desirable to limit the processing timing without masking only when detecting a large face. Further, the timing for processing without masking may be executed in any of a plurality of steps for detecting a large-sized face when the first to eighth detection steps are set as one processing cycle. It is desirable to do it in steps near the end. Specifically, it is desirable to carry out at the fifth to seventh detection steps instead of the first to third detection steps, more preferably at the sixth or seventh detection step, and at the seventh detection step. Most desirable.

  As mentioned above, although embodiment of this invention was described, this invention is not limited to this, A various deformation | transformation is possible.

For example, in this embodiment, as shown in FIG.
First detection step: large size detection Second detection step: large size detection Third detection step: large size detection Fourth detection step: medium size detection Fifth detection step: large size detection Sixth detection step: large size detection Detection step: Large size detection Eighth detection step: Small size detection, but as shown in FIG.
First detection step: Large size detection Second detection step: Medium size detection Third detection step: Large size detection Fourth detection step: Medium size detection Fifth detection step: Large size detection Sixth detection step: Small size detection Seventh Detection step: Large size detection Eighth detection step: Small size detection may be used. Even in this case, it goes without saying that the detection result is output for each detection step. Further, it is desirable that masking is not performed in the seventh detection step, and masking is performed in other detection steps.

Besides this, medium size face detection is not performed,
First detection step: large size detection Second detection step: large size detection Third detection step: large size detection Fourth detection step: small size detection Fifth detection step: large size detection Sixth detection step: large size detection Detection step: Large size detection Eighth detection step: Small size detection can be performed, or small size face detection is not performed.
First detection step: large size detection Second detection step: large size detection Third detection step: large size detection Fourth detection step: medium size detection Fifth detection step: large size detection Sixth detection step: large size detection Detection step: Large size detection Eighth detection step: Medium size detection can also be performed. In any case, it is desirable not to apply masking in the seventh detection step.

Furthermore, instead of making the first to eighth detection steps as one processing cycle, the first to sixth detection steps as one processing cycle more simply,
First detection step: Large size detection Second detection step: Large size detection Third detection step: Medium size detection Fourth detection step: Large size detection Fifth detection step: Large size detection Sixth detection step: Small size detection Well or
First detection step: large size detection Second detection step: large size detection Third detection step: small size detection Fourth detection step: large size detection Fifth detection step: large size detection Sixth detection step: small size detection Good. In these cases, it is desirable not to apply masking in the fifth detection step.

  In the present embodiment, each detection step is a step of processing all search target ranges in the preview image. In the present embodiment, processing for detecting a face of a specific size (large size face) is performed for all of the search target range, and the result is output each time. Even when the face cannot be detected, it continues. It can be said that the process of detecting faces of the same size is executed, or the process of detecting faces of the same size again after one detection step. In this embodiment, a large-sized face is regarded as having a high priority, and a large-sized face is detected quickly. However, a medium-sized or small-sized face is preferentially detected according to user settings. It would be possible to do that. When a medium-sized face is preferentially detected, for example, the following processing is performed.

First detection step: Medium size detection Second detection step: Medium size detection Third detection step: Medium size detection Fourth detection step: Large size detection Fifth detection step: Medium size detection Sixth detection step: Medium size detection Detection step: Medium size detection Eighth detection step: Small size detection However, when the result of face detection is used for AE, AF, or AWB, it is generally desirable to detect a large size face preferentially.

1 is an overall configuration diagram of an embodiment. It is a detailed block diagram of the image processing circuit in FIG. It is explanatory drawing which shows the face detection process of embodiment. It is explanatory drawing of the face detection process and result output timing of embodiment. It is explanatory drawing of the face detection process of a prior art, and a result output timing. It is explanatory drawing which shows the difference in the result output timing of the conventional apparatus of embodiment. It is explanatory drawing of the face detection process and result output timing of other embodiment.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 10 Image | photographing optical system, 12 Image pick-up element, 14 Analog front end (AFE), 16 AD converter, 18 Image processing circuit, 20 Memory card, 22 AE / AF control part, 24 LCD.

Claims (7)

An image processing apparatus for detecting a human face from a subject image,
An imaging means for imaging a subject image;
Detection means for detecting the face from the subject image, comprising at least a process for detecting a relatively large face as the face and a process for detecting a relatively small face; Detecting means for repeatedly executing a process for detecting a large-sized face and outputting the detection result a plurality of times, and subsequently executing a process for detecting a relatively small-sized face and outputting the detection result When,
An image processing apparatus comprising: The apparatus of claim 1.
The detection means includes a process of detecting a relatively large face as the face, a process of detecting a relatively medium size face, and a process of detecting a relatively small size face. The process of detecting a large face and outputting the detection result is repeated a plurality of times, then the process of detecting a relatively medium size face is executed and the detection result is output. After that, the process of detecting a relatively large size face and outputting the detection result is repeated a plurality of times, and then the process of detecting a relatively small size face is executed and the detection result is obtained. An image processing apparatus for outputting. The apparatus of claim 2.
The detection means first repeats the process of detecting a relatively large face and outputting the detection result three times, and then executes the process of detecting a relatively medium size face. The process of outputting the detection result, then executing the process of detecting a relatively large size face and outputting the detection result is repeated three times, and thereafter detecting the relatively small size face And an output of the detection result. An image processing apparatus for detecting a human face from a subject image,
An imaging means for imaging a subject image;
Detection means for detecting the face from the subject image, comprising at least a process for detecting a relatively large face as the face and a process for detecting a relatively small face; The process of detecting a large face is executed and the detection result is output, and then the process of detecting a relatively small face is executed and the detection result is output. Detecting means for executing a process for detecting a large-sized face and outputting the result, then executing a process for detecting a relatively small-sized face again and outputting the result;
An image processing apparatus comprising: The apparatus of claim 4.
The detection means includes a process of detecting a relatively large face as the face, a process of detecting a relatively medium size face, and a process of detecting a relatively small size face. The process of detecting a large-sized face and outputting the detection result is repeated twice, followed by the process of detecting a relatively medium-sized face and outputting the detection result twice. Thereafter, a process of detecting a relatively large face is executed and the result is output, and then a process of detecting a relatively small face is executed and the result is output twice. An image processing apparatus. In the apparatus in any one of Claims 1-5,
The detecting means detects a face while masking a predetermined area at both right and left ends of the subject image and excluding it from the processing target, and performs masking in any of the plurality of processes for detecting the large size face. An image processing apparatus for detecting a face without detecting a face. An image processing apparatus for detecting a human face from a subject image,
An imaging means for imaging a subject image;
Detecting means for detecting the face from the subject image, at least a process for detecting a relatively large face as the face, a process for detecting a relatively medium size face, and a relatively small face First, a process of detecting a relatively large face by masking a predetermined area on both the left and right sides of the subject image and outputting the detection result is repeated three times. Then, mask the predetermined areas on both left and right sides of the subject image and execute a process of detecting a relatively medium-sized face and output the detection result, and then mask the predetermined areas on both left and right sides of the subject image The process of detecting a large face and outputting the detection result is repeated twice, and then the process of detecting a relatively large face without masking is executed and the detection is performed. And outputs the result, and detecting means for outputting a detection result then performing the processing to detect the face of a relatively small size by masking a predetermined area of the left and right ends of the subject image,
An image processing apparatus comprising:

Images