JP4810440B2 - IMAGING DEVICE, ITS CONTROL METHOD, PROGRAM, AND STORAGE MEDIUM - Google Patents

Description

  The present invention relates to a technique for identifying an identification target (photographing target), more specifically, a position of a person's face from image data.

As an imaging device, a digital camera that converts a subject image captured by a solid-state imaging device such as a CCD image sensor into digital image data and records the digital image data on a recording medium such as a built-in memory or a memory card is already well known. Some digital cameras detect a human face from a photographed image, and perform focusing and exposure control on the detected face (see Patent Document 1).
JP 2002-512255 A

  In the imaging apparatus having the function of detecting a human face from a photographed image as described above, the size of the face that can be detected is limited due to the performance of the face detection system, and a small face that is smaller than a certain size is detected. I can't. Therefore, when detecting a small face that cannot be detected by the face detection system, there is a method of performing face detection after enlarging the captured image. In this case, since there is an upper limit on the image size input to the face detection system, a method of cutting out a part of the captured image and enlarging it to the same size as the original captured image is used.

  FIG. 5A shows a photographed image, and this image is usually input to the face detection system for face detection. On the other hand, when face detection is performed after enlarging the photographed image, an area inside the photographed image is designated as shown in FIG. 5B, and the designated area is restored as shown in FIG. 5C. The image is enlarged to the same size as the image of FIG. 5A, and the image is input to the face detection system to perform face detection.

  However, with this method, it is possible to detect a face that is included in a portion that is clipped and enlarged in the captured image, but it is impossible to detect a face that exists outside the cropped portion even if it is a large face. There is.

  Accordingly, the present invention has been made in view of the above-described problems, and an object of the present invention is to enable detection of as small a face as possible while suppressing omission of detection of a face outside the enlarged range of the photographed image. It is.

In order to solve the above-described problems and achieve the object, an imaging apparatus according to the present invention includes an imaging unit that captures a subject image, a display unit that displays a captured image captured by the imaging unit, and the captured image. Detecting means for detecting a target photographing object from the first image, first detection processing for detecting the photographing object from the first image obtained from the picked-up image by the detecting means, and the first obtained from the picked-up image. A switching unit that switches between a second detection process in which a region smaller than the first image is cut out, and the detection unit detects the shooting target from a second image enlarged to the same size as the image before the cutting. the switching means, the first detection processing result of detected plurality of even one of the imaging target, enters a preset specified region in said first image If not, continue to the first detection processing, all of the plurality of the imaging subject, if has entered a preset specified region in the first image, the above specified area size Is switched to the second detection process as an area smaller than the first image , and if all the sizes of the plurality of imaging targets detected by the second detection process are equal to or larger than a threshold, It switches to the detection process of 1 again, It is characterized by the above-mentioned.

The image pickup apparatus control method according to the present invention includes an image pickup process for picking up a subject image, a display process for displaying a picked-up image picked up by the pick-up step, and a first image obtained from the picked-up image. A first detection step for detecting an object, and a region that is smaller than the first image obtained from the captured image is cut out, and the shooting target is detected from a second image that has been enlarged to the same size as the image before cutting. plurality and the second detection step, and a switching step of switching between said first detection step and the second detection step, wherein the switching step is detected result of said first detection step of If even one of the imaging objects does not fall within the preset designated area in the first image, the first detection process is continued, and all of the plurality of imaging objects are 1 image If you have entered the Oite preset designated region, the switching or designated area of the size of the area in the second detection step as an area smaller than the first image, the second detection If all the sizes of the plurality of photographing objects detected in the process are equal to or larger than a threshold, the process is switched to the first detection process again .

A control method of an imaging apparatus according to the present invention includes: a display step of displaying an imaging step for imaging an object image, an image captured by the imaging step,
Before SL cut a first detection step of detecting a photographing target from the first image obtained from the captured image, the smaller area than the first image obtained from the captured image, the same size as the image before clipping and a second detection step of detecting the imaging object from a second image obtained by enlarging, and a switching step of switching between said first detection step and the second detection step, the switching step, the If all of the plurality of photographing objects detected by the first detection step are within a designated area set in advance in the first image, the second detection step is switched to the second detection step. If all the sizes of the plurality of photographing objects detected by the detection step are equal to or larger than a threshold value, switching to the first detection step is performed .

  A program according to the present invention causes a computer to execute the above control method.

  A storage medium according to the present invention stores the above program.

  According to the present invention, it is possible to detect as small a face as possible while suppressing omission of detection of a face outside the enlarged range in the photographed image.

  Hereinafter, a preferred embodiment of the present invention will be described in detail with reference to the accompanying drawings.

(Configuration of imaging device)
FIG. 1 is a block diagram showing a schematic configuration of an imaging apparatus according to an embodiment of the present invention.

  In FIG. 1, a light beam representing a subject image is collected by a lens system 1 and is incident on an image sensor 2 composed of a CCD, a CMOS sensor, or the like. The analog signal processing circuit 3 performs analog signal processing such as correlated double sampling on the image signal output from the image sensor 2 that photoelectrically converts the subject image. The image signal output from the analog signal processing circuit 3 is converted into digital data by the A / D conversion circuit 4 and input to the control circuit 12 and the digital signal processing circuit 5. The digital signal processing circuit 5 performs digital signal processing such as gamma correction and white balance processing. The image signal output from the digital signal processing circuit 5 is sent to the display device 9 via the memory 6. Then, a subject image is displayed on the display screen of the display device 9. An electronic viewfinder (live view display) is realized by sequentially displaying on the display device 9 image signals (captured images) that have been subjected to sequential photoelectric conversion (imaging) by the image sensor 2 and image processing.

  The image signal output from the digital signal processing circuit 5 is also input to the face detection processing circuit 10. In the face detection processing circuit 10, a face image (photographing target) is detected from the input image signal. This face image detection process is performed on each image signal that has been sequentially photoelectrically converted by the image sensor 2 for live view display.

  Here, when a shutter button (not shown) is pressed, the subject image is captured by the image sensor 2 (not only for live view display but also for actual recording and storage) to represent the subject image. Image data is obtained. The image data is given from the digital signal processing circuit 5 to the memory 6 and temporarily stored. The image data is read from the memory 6 and recorded on the memory card 8 by the recording control circuit 7.

  Note that the shutter button is connected to the release switch 14, and the switch SW1 of the release switch 14 is turned on by half-pressing the shutter button, and shooting such as AE (automatic exposure) processing, AF (autofocus) processing, and the like. A preparatory operation is performed. When the shutter button is fully pressed, the switch SW2 of the release switch 14 is turned on, and the photographing operation is performed.

(Operation of imaging device)
Here, a description will be given of an operation in the face detection processing circuit 10 in the imaging apparatus of FIG. 1 that switches between a case where a face is detected by enlarging a captured image and a case where a face is detected without being enlarged depending on conditions.

  FIG. 2 is a flowchart showing the operation of face detection processing for detecting a face from a captured image.

  Note that the operation shown in FIG. 2 is performed in the above-described live view display state.

  First, in step S11, image data that has been subjected to photoelectric conversion and image processing by the imaging device 2 at a certain timing is acquired. This acquisition of image data is an operation for acquiring an image to be displayed in the live view.

  In step S12, face detection is performed on the image data.

  Various techniques are known for detecting the face area. For example, there is a method using learning represented by a neural network. In addition, there is a technique for identifying parts having physical shape features such as eyes, nose, mouth, and facial contour from image data using template matching. In addition, there is a technique that detects a feature amount of image data such as skin color or eye shape and uses statistical analysis (for example, Japanese Patent Laid-Open Nos. 10-232934 and 2000-48184). reference). In addition, it is determined whether the previous face area is near the detected position, the color near the face area is determined in consideration of the color of clothes, or the face identification is performed closer to the center of the screen. There is a method of setting the threshold value for this purpose low. Alternatively, there is a method of tracking the main subject by specifying a region in which the main subject exists in advance, storing a histogram and color information, and obtaining a correlation value. In this embodiment, a pair of eyes (both eyes), a nose, a mouth, and a face outline are detected from the shape, and a face area identification process is performed by a method of determining a person's face area from these relative positions. Yes. Then, a square area surrounding the face area is set along the size of the detected face area, and the coordinates of the square area are obtained. The coordinates of the rectangular area surrounding the face area become an identification area indicating the position of the main subject. The area surrounding the face area is not limited to a square, and may be another shape such as an ellipse or a shape that follows the detected outline of the face.

  Next, it progresses to step S13 and it is determined whether the face was detected. If a face is detected, the process proceeds to step S14, and if no face is detected, the process proceeds to step S16.

  In step S14, it is determined whether or not the face detected in step S12 is within a designated area on the screen. As shown in FIG. 3A, for example, assuming that two faces A and B are detected, if the detected faces A and B are all within the designated area, the process proceeds to step S16. As shown in FIG. 3B, when the detected face A is in the designated area but there is at least one face that is not in the designated area, such as face B, the process proceeds to step S14a. move on.

  In step S14a, a photographing process described later is performed, and the process proceeds to step S15.

  In step S15, it is determined whether or not shooting has been completed. If shooting has been completed, the processing of this flowchart is terminated. If shooting has not ended, the process returns to step S11 and the above-described flow is repeated.

  If it is determined in step S13 that no face is detected, or if it is determined that all the faces detected in step S14 are within the designated area, the process proceeds to step S16.

  In step S16, image data for live view is newly acquired.

  In step S17, the image is enlarged and face detection is performed as shown in FIG.

  Then, it progresses to step S18 and it is determined whether the face was detected. If a face is detected, the process proceeds to step S18a, and if no face is detected, the process proceeds to step S14a. In step S18a, a photographing process described later is performed, and the process proceeds to step S19.

  In step S19, it is determined whether or not the shooting is finished. If the shooting is finished, the processing of this flowchart is finished. If shooting has not ended, the process proceeds to step S20.

  In step S20, it is determined whether or not the minimum face size among the faces detected by enlarging the image in step S17 is greater than or equal to a threshold value. If the minimum face size is equal to or larger than the threshold value, the process proceeds to step S11, and the process returns to the process of newly acquiring image data for live view and executing face detection. If the minimum face size is less than the threshold value, the process proceeds to step S16 to newly acquire live view image data, and the process returns to the process of enlarging and detecting the face in step S17.

  Next, FIG. 4 is a flowchart showing photographing process steps which are subroutines executed in steps S14a and S18a of FIG.

  When the photographing process step is started, first, in step S41, it is determined whether or not the shutter button is half-pressed and the switch SW1 of the release switch 14 is turned on. If the switch SW1 is ON, the process proceeds to step S42. If the switch SW1 is not ON, this subroutine is terminated and the process returns to step S15 or step S19 in FIG.

  In step S42, the luminance of the subject is detected by a photometric sensor (not shown), the aperture value of the lens system 1 is calculated, and the AE process for calculating the shutter speed is performed. Further, an in-focus state is detected from the contrast of the image data from the image sensor 2, and an AF process is performed to drive the focus lens of the lens system 1 so that the subject image is in the in-focus state.

  Next, in step S43, it is determined whether or not the shutter button is fully pressed and the switch SW2 of the release switch 14 is ON. If the switch SW2 is ON, the process proceeds to step S44. If the switch SW2 is not ON, the subroutine is terminated and the process returns to step S15 or step S19 in FIG.

  In step S44, a photographing process for performing the main exposure of the image sensor 2 is executed using the aperture value and the shutter speed calculated in step S42.

  In step S45, the image captured in step S44 is displayed on the display device 9.

  In step S46, the image data photographed in step S44 is recorded on the memory card 8.

  When these processes are finished, this subroutine is finished, and the process returns to step S15 or step S19 in FIG.

  As described above, according to the present embodiment, face detection is first performed without enlarging an image. After that, it is determined whether or not all the faces detected in step S14 are within the specified area. If there is a face that does not contain at least one face, the face is not enlarged at the next face detection. Perform detection. As the designated area here, as shown in FIG. 3B, an area for enlarging an image or an area inside the area is designated. By doing so, when a face is detected outside the designated area, the face detection is performed without enlarging the image at the next face detection. The adverse effect of the face not being detected can be prevented. Also, if all faces are within the specified area, there is no fear that the face will not be detected even if the image is enlarged and detected, so the next time face detection is performed, the image is enlarged and face detection is performed. .

  Also, if face detection is not performed at all when face detection is performed without first enlarging the image, the face may exist but may not have been detected due to its small size. At the time of detection, the face is detected by enlarging the image.

  When the face is detected by enlarging the image in step S17, if no face is detected, there is no small face. Therefore, the face is not enlarged at the next face detection. Perform detection. Further, when a face is detected when the face is detected by enlarging the image, it is determined whether or not the detected minimum face size is greater than or equal to a threshold value. As the threshold value, the minimum face size that can be detected by the face detection system is designated. If the minimum face size in the screen is less than the threshold value, the face cannot be detected without enlarging the image. Therefore, the face is detected by enlarging the image at the next face detection. If the minimum face size on the screen is equal to or greater than the threshold, the face can be detected without enlarging the image. It can be carried out.

  As described above, the case where the face is detected by enlarging the image and the case where the face is detected without being enlarged are switched depending on conditions such as the position of the detected face and the size of the face as in the present embodiment. Accordingly, it is possible to detect even a smaller face, and it is possible to suppress an adverse effect that the face is not detected when the face is outside the enlarged region by enlarging and detecting the image. Furthermore, when all faces can be detected without enlarging the image, the area where the face is not detected does not occur when enlarging so that the image is not enlarged. It can be carried out. Further, depending on the detected face position and face size, the image enlargement ratio may be set in multiple stages, such as the first enlargement ratio and the second enlargement ratio. For example, as the size of the smallest detected face becomes smaller, the enlargement ratio of the image read for the next face detection can be increased. Alternatively, it is also possible to increase the enlargement ratio of an image to be read for the next face detection as the position of the detected face closest to the center is closer to the center.

(Other embodiments)
The object of each embodiment is also achieved by the following method. That is, a storage medium (or recording medium) in which a program code of software that realizes the functions of the above-described embodiments is recorded is supplied to the system or apparatus. Then, the computer (or CPU or MPU) of the system or apparatus reads and executes the program code stored in the storage medium. In this case, the program code itself read from the storage medium realizes the functions of the above-described embodiments, and the storage medium storing the program code constitutes the present invention. Further, by executing the program code read by the computer, not only the functions of the above-described embodiments are realized, but the present invention includes the following cases. That is, based on the instruction of the program code, an operating system (OS) running on the computer performs part or all of the actual processing, and the functions of the above-described embodiments are realized by the processing.

  Furthermore, the following cases are also included in the present invention. That is, the program code read from the storage medium is written into a memory provided in a function expansion card inserted into the computer or a function expansion unit connected to the computer. Thereafter, based on the instruction of the program code, the CPU or the like provided in the function expansion card or function expansion unit performs part or all of the actual processing, and the functions of the above-described embodiments are realized by the processing.

  When the present invention is applied to the above storage medium, the storage medium stores program codes corresponding to the procedure described above.

1 is a block diagram illustrating a schematic configuration of an imaging apparatus according to an embodiment of the present invention. It is a flowchart which shows the operation | movement of the face detection process which detects a face from a picked-up image. It is the figure which showed the example of the area | region designated of the image. It is a flowchart which shows the imaging | photography process step which is a subroutine performed by step S14a and step S18a of FIG. It is the figure which showed the example in the case of expanding an image and detecting a face.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Lens system 2 Image pick-up element 3 Analog signal processing circuit 4 A / D conversion circuit 5 Digital signal processing circuit 6 Memory 7 Recording control circuit 8 Memory card 9 Display apparatus 10 Face detection processing circuit 12 Control circuit 14 Release switch

Claims (5)

Imaging means for capturing a subject image;
Display means for displaying a captured image captured by the imaging means;
Detecting means for detecting a target photographing object from the captured image;
A first detection process for detecting the object to be photographed by the detection unit from the first image obtained from the captured image, and a region smaller than the first image obtained from the captured image, and before the cropping A switching unit that switches between a second detection process in which the detection unit detects the imaging target from a second image enlarged to the same size as the image,
If the switching means does not fall within the designated area set in advance in the first image, even if one of the plurality of shooting targets detected as a result of performing the first detection processing is not included, 1 is continued, and if all of the plurality of photographing objects are in a designated area set in advance in the first image, an area larger than the designated area is selected as the first area. Switch to the second detection process as a region smaller than the image of the image , and if all the sizes of the plurality of photographing objects detected by the second detection process are equal to or larger than a threshold value, the first detection process is performed again. An imaging device characterized by switching.   The imaging apparatus according to claim 1, wherein the photographing target is a human face. An imaging process for capturing a subject image;
A display process for displaying the captured image captured by the imaging process;
A first detection step of detecting a shooting target from a first image obtained from the captured image;
A second detection step of cutting out a region smaller than the first image obtained from the captured image and detecting the shooting target from a second image enlarged to the same size as the image before cutting;
A switching step of switching between the first detection step and the second detection step,
In the switching step, if even one of the plurality of photographing objects detected as a result of performing the first detection step does not fall within the designated area preset in the first image, the first step is performed . 1 is continued, and if all of the plurality of photographing objects are in a designated area set in advance in the first image, an area larger than the designated area is selected as the first area. Switch to the second detection step as an area smaller than the image of the image , and if all the sizes of the plurality of photographing objects detected by the second detection step are equal to or larger than a threshold, the first detection step is performed again. A method for controlling an imaging apparatus, characterized by switching.   A program for causing a computer to execute the control method according to claim 3.   A computer-readable storage medium storing the program according to claim 4.

Images