JP5004876B2 - Imaging device - Google Patents

Description

  The present invention relates to an imaging apparatus that obtains image data by converting an output of an imaging device that converts incident light into an electrical signal into a digital value. In particular, the present invention relates to an imaging apparatus having a function of detecting the face of a subject.

  Conventionally, when taking a picture of a person, there is a problem that the focus is not on the person but on the background because of the contrast between the person who is the main subject and the background. In order to solve such a problem, an imaging apparatus that focuses on a person by detecting a face in the screen and focusing on the position of the detected face has been developed (see Patent Document 1).

  However, according to Patent Document 1, there is a problem that it takes time to shoot because face detection processing is executed regardless of whether or not a person is present in the shooting screen.

  In order to solve this problem, there has been proposed an imaging device that performs face detection when a photographer's voice is detected by a microphone provided in the imaging device (see Patent Document 2).

  According to Patent Document 2, face detection processing is not executed when the imaging apparatus is in landscape shooting mode, face detection processing is executed when in the person shooting mode, and the voice of the photographer is detected when the mode is not in either mode. Only when the face detection process is executed.

As for face detection from image data, those described in Non-Patent Documents 1 and 2 are known. Further, the position and size of the face can be estimated by detecting eyes by the methods described in Patent Documents 3 to 6.
JP 2001-215403 A JP 2005-318084 A Japanese Patent Laid-Open No. 3-17696 JP-A-4-255015 JP-A-5-300601 Japanese Patent Laid-Open No. 9-251342 Television Society Journal Vol. 49, no. 6, pp. 787-797 (1995), “Proposal of Modified HSV Color System Effective for Face Area Extraction” The Institute of Electronics, Information and Communication Engineers Vol. 74-D-II, no. 11, pp. 1625-1627 (1991), “Method for extracting a face region from a still gray scene image”

  However, in the above conventional imaging device, there is no difference between the face detection processing in the case of reliably taking a person and the face detection processing in the case where the imaging device cannot determine whether or not the person has been taken. Processing was not optimized.

  The objective of this invention is providing the imaging device which can improve the precision of a face detection process.

In order to achieve the above object, an imaging apparatus according to the present invention includes an imaging unit that acquires image data by photoelectrically converting a subject image, and an evaluation for detecting a face from the captured image obtained by the imaging unit. A face detection unit that calculates a value, compares the evaluation value with a threshold value to perform face determination, and determines that the face is a face if the evaluation value is greater than the threshold value; and a voice detection unit that detects voice The face detection means changes the threshold value according to a detection result of the voice detection means.

  According to the imaging apparatus of the present invention, it is possible to improve the accuracy of face detection processing.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

  FIG. 1 is an external perspective view of a digital camera as an imaging device according to an embodiment of the present invention.

  The apparatus main body 1 includes an optical viewfinder 2, a power switch (button) 3, a release switch 4 that is pressed when shooting a still image or a moving image, a zoom lever 5 for changing a shooting angle of view, and a mode switch 6. .

  The mode switch 6 switches various modes in the apparatus main body 1. More specifically, when the mode switch 6 is set to the icon mark 1a printed on the back surface of the apparatus main body 1, it is possible to switch to the still image recording mode. Further, when the mode switch 6 is set to the icon mark 1b, it is possible to switch to the moving image recording mode. In addition, when the mode switch 6 is set to the icon mark 1c, the mode can be switched to the reproduction mode.

  The liquid crystal panel 7 is a display unit provided on the back surface of the apparatus main body 1 and displays a subject image before photographing formed on the light receiving surface of the image sensor through a photographing lens as a through image or recording after photographing. Play and display the recorded image.

  The operation unit 8 is an operation switch for an operator to perform various operations. Specifically, the operation unit 8 is a display switch for switching a display on the liquid crystal panel 7, a menu switch, a print switch, or a SET switch.

  The cross switch 9 is a four-way switch (upper switch, lower switch, right switch, left switch) arranged in a cross.

  FIG. 2 is a block diagram of the digital camera of FIG.

  Hereinafter, the configuration will be described together with the operation (function).

  In FIG. 2, the barrier 101 covers the imaging system including the photographing lens 102 of the apparatus main body 1, thereby preventing the imaging system from becoming dirty or damaged. There is a photographing lens 102, a shutter 103 having a diaphragm function, and an imaging unit (imaging device) 104 including a CCD, a CMOS device, or the like that converts an optical image into an electrical signal (photoelectric conversion).

  The A / D converter 105 converts an analog signal into a digital signal. The A / D converter 105 is used when an analog signal output from the imaging unit 104 is converted into a digital signal, or when an analog signal output from the audio control unit 106 is converted into a digital signal.

  The timing generation unit 107 supplies a clock signal and a control signal to the imaging unit 104, the A / D converter 105, the audio control unit 106, and the D / A converter 108. The timing generation unit 107 is controlled by the memory control unit 109 and the system control unit 110.

  The image processing unit 111 performs resizing processing and color conversion processing such as predetermined pixel interpolation and reduction on the data from the A / D converter 105 or the data from the memory control unit 109.

  The image processing unit 111 performs predetermined calculation processing using the captured image data, and the system control unit 110 performs exposure control and distance measurement control based on the obtained calculation result. Thereby, AF (autofocus) processing, AE (automatic exposure) processing, and EF (flash pre-emission) processing of the TTL (through-the-lens) method are performed.

  The system control unit 110 calculates an evaluation value for detecting a face from the captured image obtained by the imaging unit 104 as an imaging unit, compares the evaluation value with a threshold value, performs face determination, and evaluates the evaluation value. Functions as face detection means for determining that the face is larger than the threshold. Details thereof will be described in step S301 of FIG.

  The image processing unit 111 further performs predetermined calculation processing using the captured image data, and also performs TTL AWB (auto white balance) processing based on the obtained calculation result.

  Output data from the A / D converter 105 is written into the memory 112 via the image processing unit 111 and the memory control unit 109 or directly via the memory control unit 109. The memory 112 stores image data obtained by the imaging unit 104 and converted into digital data by the A / D converter 105 and image data to be displayed on the image display unit 23 including the liquid crystal panel 7.

  Note that the memory 112 is also used for storing audio data recorded in the microphone 21 (21a, 21b), a still image, a moving image, and a file header when configuring an image file. Therefore, the memory 112 has a storage capacity sufficient to store a predetermined number of still images, a moving image and sound for a predetermined time.

  The compression / decompression unit 113 compresses and decompresses image data by adaptive discrete cosine transform (ADCT) or the like. The compression / decompression unit 113 reads a captured image stored in the memory 112 using the shutter 103 as a trigger, performs compression processing, and writes the processed data in the memory 112.

  The compression / decompression unit 113 performs decompression processing on the compressed image read into the memory 112 from the recording unit 201 of the recording medium 200 and writes the processed data to the memory 112.

  The image data written to the memory 112 by the compression / decompression unit 113 is filed in the file unit of the system control unit 110. Then, the data is recorded in the recording unit 201 via the interface (I / F) 114, the connector 115, the connector 203 on the recording medium 200 side, and the interface (I / F) 202. The memory 112 also serves as an image display memory (video memory).

  The D / A converter 108 converts the image display data stored in the memory 112 into an analog signal and supplies the analog signal to the image display unit 23. The image display unit 23 converts the display image data written in the memory 112 into an analog signal via the D / A converter 108 on the display device such as the liquid crystal panel 7 and displays it.

  The audio signal output from the microphone 21 is supplied to the A / D converter 105 via the audio control unit 106 configured by an amplifier or the like, converted into a digital signal by the A / D converter 105, and then subjected to memory control. The data is stored in the memory 112 by the unit 109.

  On the other hand, the audio data recorded on the recording medium 200 is read into the memory 112 and then converted into an analog signal by the D / A converter 108. The voice control unit 106 drives the speaker 22 with this analog signal and outputs a voice.

  The nonvolatile memory 116 is an electrically erasable / recordable memory, and for example, an EEPROM or the like is used. The nonvolatile memory 116 stores (records) constants, programs, and the like for operation of the system control unit 110. Here, the program is a program for executing various flowcharts described later in the present embodiment.

  The system control unit 110 executes programs stored in the nonvolatile memory 116, thereby realizing each process of the present embodiment described later. The system memory 117 is a RAM. In the system memory 117, constants and variables for operation of the system control unit 110, programs read from the nonvolatile memory 116, and the like are expanded (stored).

  The zoom lever 5, the mode switch 6, the first shutter switch 51, the second shutter switch 52, the operation unit 8 and the cross switch 9 are operation means for inputting various operation instructions to the system control unit 110.

  The mode switch 6 can switch the operation mode of the system control unit 110 to any one of a still image recording mode, a moving image recording mode, a reproduction mode, and the like. The first shutter switch 51 is turned on during the halfway operation of the release switch 4 provided in the apparatus main body 1 and generates a first shutter switch signal SW1.

  The system control unit 110 starts operations such as AF processing, AE processing, AWB processing, and EF processing in response to the first shutter switch signal SW1.

  The second shutter switch 52 is turned on when the operation of the release switch 4 is completed (fully pressed), and generates a second shutter switch signal SW2. In response to the second shutter switch signal SW2, the system control unit 110 starts a series of shooting processing operations from reading a signal from the imaging unit 104 to writing image data on the recording medium 200.

  Each operation member of the operation unit 8 is appropriately assigned a function for each scene by selecting and operating various function icons displayed on the image display unit 23, and functions as various function switches. Examples of the function switch include an end switch, a return switch, an image feed switch, a jump switch, a narrowing switch, and an attribute change switch.

  For example, when the menu switch is pressed, a menu screen on which various settings can be made is displayed on the image display unit 23. The operator can make various settings intuitively using the menu screen displayed on the image display unit 23 and the cross switch 9 or the SET switch. The power switch 3 switches between power on and power off.

  The power supply control unit 118 includes a battery detection circuit, a DC-DC converter, a switch circuit that switches a block to be energized, and the like, and detects whether or not a battery is attached, the type of battery, and the remaining battery level. Further, the power supply control unit 118 controls the DC-DC converter based on the detection result and an instruction from the system control unit 110, and supplies a necessary voltage to each unit including the recording medium 200 for a necessary period.

  The power supply unit 119 includes a primary battery such as an alkaline battery or a lithium battery, a secondary battery such as a NiCd battery, a NiMH battery, or a Li battery, an AC adapter, or the like. Connectors 54 and 55 connect the power supply unit 119 and the power supply control unit 118.

  An RTC (Real Time Clock) 120 measures the date and time. The RTC 120 holds a power supply unit therein separately from the power supply control unit 118 and keeps counting time even when the power supply unit 119 is turned off. The system control unit 110 sets a system timer using the date and time acquired from the RTC 120 at the time of activation, and executes timer control.

  The interface 114 serves as an interface between the recording medium 200 such as a memory card or a hard disk or a tuner card and the apparatus main body 1. The connector 115 connects the recording medium 200 and the tuner card to the interface 114. The recording medium attachment / detachment detection unit 121 detects whether the recording medium 200 or a tuner card is attached to the connector 115.

  The recording medium 200 is a memory card or a hard disk in FIG. The recording medium 200 includes a recording unit 201 composed of a semiconductor memory, a magnetic disk, or the like, an interface 202 with the apparatus main body 1, and a connector 203 for connecting the recording medium 200 and the apparatus main body 1.

  The connectors 115 and 203 conform to the SDI / O card expansion standard, and a tuner card conforming to the SDI / O card expansion standard can be attached and detached in addition to the recording medium described above.

  The communication unit 122 performs various communication processes such as RS232C, USB, IEEE1394, P1284, SCSI, modem, LAN, and wireless communication. A connector (antenna in the case of wireless communication) 123 connects the apparatus main body 1 to another device via the communication unit 122.

  FIG. 3 is a flowchart showing a procedure of photographing (imaging) processing executed by the digital camera of FIG.

  The process shown in FIG. 3 is executed by the system control unit 110. For example, the system control unit 110 includes a CPU (not shown), and implements the processing shown in FIG. 3 by executing a control program stored in the system memory 117, for example.

  When the shooting operation is started in FIG. 3, in step S <b> 301, the system control unit 110 performs face detection processing for detecting whether or not a human face exists in the through-displayed image signal. This face detection process will be described later with reference to FIG.

  When a human face is detected in the face detection process, the system control unit 110 uses the face position coordinates, size (width, height), number of detections, reliability coefficient, and the like detected in the image signal as face information. Store in the memory 117. If no face is detected in the face detection process, 0 is set in the area such as position coordinates, size (width, height), number of detections, reliability coefficient, etc. in the system memory 117.

  Subsequently, in step S302, it is determined whether or not the first shutter switch signal SW1 is turned on. If the first shutter switch signal SW1 is OFF, the face detection process in step S301 is executed again, and if it is ON, the process proceeds to the next step S303.

  In step S303, the system control unit 110 performs a distance measurement process to focus the photographing lens 102 on the subject, and performs a photometry process to determine an aperture value and a shutter time (shutter speed).

  In the photometric process, a flash is set if necessary. At this time, if a face is detected in step S301, it is possible to perform distance measurement within the detected face range.

  Next, in step S304, the ON / OFF state of the second shutter switch signal SW2 is determined. If the second shutter switch signal SW2 is turned on while the first shutter switch signal SW1 is turned on, the process proceeds from step S304 to step S306.

  When the second shutter switch signal SW2 is not turned on and the first shutter switch signal SW1 is also canceled (step S305), the process returns from step S305 to step S301.

  Further, while the first shutter switch signal SW1 is ON and the second shutter switch signal SW2 is OFF, the processes in steps S303 to S305 are repeated.

  When the second shutter switch SW2 is pressed (when the second shutter switch signal SW2 is turned on), in step S306, the system control unit 110 executes photographing processing (exposure processing) including exposure processing and development processing.

  In the exposure process, the image data obtained through the image pickup unit 104 and the A / D converter 105 is sent via the image processing unit 111 and the memory control unit 109 or directly from the A / D converter 105. The data is written into the memory 112 via 109.

  In the development process, the system control unit 110 reads out the image data written in the memory 112 using the memory control unit 109 and, if necessary, the image processing unit 111, and performs various processes.

  After shooting, in step S307, the system control unit 110 executes a recording process in which the image data obtained by the shooting process is written to the recording medium 200 as an image file.

(First embodiment)
FIG. 4 is a flowchart showing the procedure of the first embodiment of the face detection process executed in step S301 of FIG.

  When the face detection process starts, the face evaluation value of the image is calculated in step S401. The face evaluation value is a numerical value representing the face-likeness of the area included in the image, for example, the degree of matching with the face template in the pattern matching method, and is calculated from the layout of feature points such as eyes, nose, and mouth. It is a feature quantity.

  When finding the degree of matching with a face template, edge extraction is performed within the area of the image where face detection processing is performed, a predetermined face template is compared with the extracted edge, and the degree of similarity with the face template is calculated. To do.

  The face template is scanned within a region where face detection processing is performed, and the similarity with the face template is sequentially calculated for each target pixel. Techniques for obtaining these face evaluation values are known and disclosed in, for example, Japanese Patent Laid-Open No. 8-63597.

  Next, in step S402, the face evaluation value calculated in step S401 is compared with a predetermined threshold value.

  If the image to be subjected to the face detection process includes an area where the face evaluation value is equal to or greater than the threshold value, the process proceeds to step S403, where the target area is determined to be a face, and the face detection process is terminated. If there is no region where the face evaluation value is equal to or greater than the threshold value in step S402, the process proceeds to step S404.

  In step S <b> 404, voice detection is performed using the microphone 21 provided in the apparatus main body 1.

  In step S405, the presence / absence of sound is determined. If sound is detected, the process proceeds to step S406.

  In step S406, the threshold value to be compared with the face evaluation value is lowered. This is because it is assumed that a person is present within the shooting range since the sound is detected.

  After changing the threshold value, in step S407, the face evaluation value obtained in step S401 is compared with the threshold value. If the face evaluation value is greater than or equal to the threshold value, the process proceeds to step S403, where it is determined that the target area is a face.

  On the other hand, if no sound is detected in step S405, or if there is no region where the face evaluation value is equal to or greater than the threshold value in step S407, the face detection process is terminated without detecting a face.

  As described above, in the first embodiment, the threshold value used as a criterion for face determination is lowered when voice is detected. For this reason, for example, it is possible to detect an object that cannot be detected by normal face detection processing, such as shooting in the dark, sideways, and eye meditation, which contributes to an improvement in the face detection rate.

  In the first embodiment, the face determination criterion is based on whether or not the face evaluation value is greater than or equal to the threshold value. For example, if the face evaluation value falls within a certain range between the upper limit and the lower limit. A technique for determining a face is also known. In that case, it is possible to improve the face detection rate by changing the upper and lower limits of the range that is determined to be a face when speech is detected to widen the range.

(Second Embodiment)
In the second embodiment, it is assumed that the threshold value is lowered by detecting the voice of a person. Further, the sound source position is detected, and the threshold value of only a part of the region including the sound source position is lowered.

  FIG. 5 is a flowchart showing the procedure of the second embodiment of the face detection process executed in step S301 of FIG.

  When the face detection process starts, a voice detection process is executed in step S501, and the presence or absence of voice is determined in step S502. If it is determined that there is sound, it is then determined in step S503 whether the sound is a person's voice. If it is determined that the sound is a person's voice, it is determined in step S504 whether the sound is within the shooting angle of view.

  A series of these voice / sound source determination techniques are known and disclosed in, for example, Japanese Patent Application Laid-Open No. 05-215833.

  FIG. 6 is a diagram illustrating a configuration example of a sound source direction detecting unit used in the sound source direction detecting process executed in step S501 of FIG.

  In FIG. 6, the output signal of the microphone 21 (21a, 21b) having high directivity passes only a specific frequency without attenuation by the band-pass filter 602 (602a, 602b). The sound pressure difference detection circuit 603 compares the sound pressure levels output from the microphones 21 a and 21 b and outputs a difference value between the sound pressure levels to the system control unit 110.

  The sound pressure level output by the microphone 21 decreases as the sound source 601 moves away from the directional characteristic pattern of the microphone 21. For this reason, if there is a difference in the sound pressure levels output from the microphones 21a and 21b, it can be detected that there is a sound source 601 that emits an audio signal on the side of the microphone that outputs a high level. Can be detected to be in front of.

  Here, it is possible to detect the sound source position of a person's uttered sound by setting the frequency of the specific band that the bandpass filter 602 passes without attenuation to, for example, around 2 KHz that is a frequency band uttered by a person. .

  Further, by using two directional microphones 21, one for each of the left and right sides of the apparatus main body 1, it is possible to detect the sound source position in the left-right direction relative to the apparatus main body 1. Furthermore, by providing another microphone 21 at a different position in the vertical direction, the position of the sound source can be detected in the vertical direction with respect to the apparatus main body 1. Increasing the number of microphones 21 further increases the accuracy of detecting the sound source position.

  As another technique for detecting the position of the sound source, a technique using a phase difference between audio signals output from a plurality of microphones 21a and 21b provided in the apparatus main body 1 is also known. This is because, if there is a difference in the distance from the sound source 601 to each of the microphones 21a and 21b, a phase difference occurs in the output signal of the microphone 21, so that the direction of the sound source is specified by calculation from the known inter-microphone distance and sound speed. It is. Details are disclosed in Japanese Patent Application Laid-Open No. 07-140527.

  Furthermore, the above-described sound source position detection method may be properly used depending on the focal length of the photographing lens 102.

  The closer the sound source 601 is to the side from the front of the apparatus body 1, the greater the difference in distance from the sound source 601 to each of the microphones 21a and 21b, and the greater the phase difference. Therefore, sound source position detection by the phase difference method is suitable near the wide angle.

  On the other hand, when the sound source 601 is in the vicinity of the front of the apparatus main body 1, if the microphone 21 has a narrow sound detection range and has a narrow sound detection range, a sound pressure difference occurs even if the sound source position is slightly different. Therefore, sound source position detection by the sound pressure difference method using the directional microphone 21 is suitable near the telephoto position.

Returning to FIG. 5, if it is determined in step S504 that the sound source 601 is within the angle of view, the process proceeds to step S505. In step S505, a partial region of the shooting range including the sound source direction is set, and in the next step S506, processing for lowering a threshold value used as a face determination reference is performed only in the region including the sound source position set in step S505. .

  Processing to lower the threshold value used as a reference for face detection is performed, or in step S502 to step S504, the sound source 601 cannot be detected, the sound is not a person's uttered sound, the sound source 601 is not in each captured image, etc. When the determination is made, the process proceeds to step S507.

  In step S507, as in step S401 of the first embodiment, the face evaluation value of the shooting area is calculated.

  In step S508, the calculated face evaluation value is compared with a threshold value. If there is an area where the face evaluation value is equal to or greater than the threshold value, the face detection process ends after it is determined in step S509 that the face evaluation value is equal to or greater than the threshold value. The face detection process ends without detecting a face.

  As described above, in the second embodiment, the type of voice is distinguished based on whether it is a person's voice. Therefore, it is possible to improve the face detection rate by lowering the threshold only when there is a person within the shooting angle of view. If there is no person, the threshold will not be lowered unnecessarily. , Increase in false detection can be suppressed.

  Further, since the sound source direction is detected and the threshold value is lowered only in a partial region including the sound source, it is possible to suppress an increase in false detection in a region where no person exists.

  Furthermore, since the sound source direction detection method is changed depending on the focal length of the photographing lens 102, the detection accuracy of the sound source direction can be further improved.

1 is an external perspective view of a digital camera as an imaging apparatus according to an embodiment of the present invention. It is a block diagram of the digital camera of FIG. 3 is a flowchart illustrating a procedure of photographing (imaging) processing executed by the digital camera of FIG. 2. It is a flowchart which shows the procedure of 1st Embodiment of the face detection process performed by FIG.3 S301. It is a flowchart which shows the procedure of 2nd Embodiment of the face detection process performed by step S301 of FIG. It is a figure which shows the structural example of the sound source direction detection means used for the sound source direction detection process performed by step S501 of FIG.

Explanation of symbols

1 Device body 21 Microphone
104 imaging unit 106 audio control unit 110 system control unit 111 image processing unit 603 sound pressure difference detection circuit

Claims (11)

Imaging means for acquiring image data by photoelectrically converting a subject image;
An evaluation value for detecting a face is calculated from the photographed image obtained by the imaging means, and the face is determined by comparing the evaluation value with a threshold value. Face detection means for determining that there is,
Voice detection means for detecting voice,
The imaging apparatus according to claim 1, wherein the face detection unit changes the threshold value in accordance with a detection result of the voice detection unit. Imaging means for acquiring image data by photoelectrically converting a subject image;
A face detection unit that calculates an evaluation value for detecting a face from the captured image obtained by the imaging unit, and determines that the face is a face if the evaluation value is within a predetermined range;
Voice detection means for detecting voice,
The imaging apparatus according to claim 1, wherein the face detection unit changes the predetermined range according to a detection result of the voice detection unit. It said speech detection means, the imaging apparatus according to claim 1 or 2, wherein the detecting the human vocal sound as the sound. It said speech detection means, by detecting the frequency of the frequency band of the utterance of a person, the image pickup apparatus according to claim 3, wherein the detecting the human vocal sound as the sound. The imaging apparatus according to claim 1, wherein the sound detection unit detects a direction of the sound source from the detection result of the sound.   The imaging apparatus according to claim 5, wherein the sound detection unit detects the direction of the sound source using a phase difference between output signals of a plurality of microphones.   The imaging apparatus according to claim 5, wherein the sound detection unit detects a direction of the sound source using a sound pressure difference between output signals of a plurality of microphones. The sound detection means detects a direction of the sound source using a phase difference of output signals of a plurality of microphones when a focal length of a photographing lens provided in the apparatus main body is close to a wide angle. 5. The imaging device according to 5 . The sound detection means detects a direction of the sound source by using a sound pressure difference between output signals of a plurality of microphones when a focal length of a photographing lens provided in the apparatus main body is near telephoto. 5. The imaging device according to 5 . Said face detecting means, said selected regions comprising a sound source direction detected voice detection means of claim 5 to 9, characterized in that changing the threshold value or the predetermined range in the selected region The imaging device according to any one of the above. Evaluation value for detecting the face image pickup device according to any one of claims 1 to 10, characterized in that a similarity between the template in the pattern matching method.

Images