JP5182308B2 - Imaging apparatus and program thereof - Google Patents

Description

  The present invention relates to an imaging apparatus and a program thereof.

There is a technique for detecting the orientation of a face in an imaging apparatus, for example, a camera.
(Patent Document 1)

JP Patent Publication No. 2007-280374

  However, according to the technique of the above-mentioned Patent Document 1, even if the orientation of the face is detected, imaging control considering the orientation of the face has not been performed at the time of imaging.

  SUMMARY An advantage of some aspects of the invention is that it provides an imaging apparatus capable of performing imaging control in consideration of the direction of a subject and a program thereof.

In order to achieve the above object, an image pickup apparatus according to a first aspect of the present invention includes an image pickup means for picking up a subject, a subject detection means for detecting a subject in image data picked up by the image pickup means, and the subject detection means. An identification unit that identifies a direction in which the detected subject is facing , and a focus movement by changing a search lens movement range of the focus lens according to the direction in which the subject identified by the identification unit is facing. And control means for performing focus control so as to detect the lens position.

To achieve the above object, a program according to a second aspect of the present invention provides a computer including an imaging unit that images a subject, a subject detection unit that detects a subject in image data captured by the imaging unit, and the subject An identification unit that identifies a direction in which the subject detected by the detection unit is facing , and a focus lens search movement range is changed according to the direction in which the subject identified by the identification unit is facing , and the subject is focused. It is characterized by functioning as a control means for controlling the focus so as to detect an in-focus lens position.

  According to the present invention, it is possible to perform predetermined control for imaging based on the detected face orientation.

1 is a block diagram of a digital camera according to an embodiment of the present invention. It is a flowchart which shows operation | movement of the digital camera 1 of 1st Embodiment. An example of a face in the detected frame image data, a certain area to be subjected to face detection processing based on the detected face orientation, and a prediction area specified based on the detected face orientation FIG. 6 is a diagram illustrating an example of a frame image captured in step S2 after the AE process in step S6 and an example of a prediction area 33 that is a target of the photometric area in the AE process in step S6. It is a figure which shows the mode of a movement of focus lens 2A, and a figure which shows an example of the mode of the direction of the detected face. It is a flowchart which shows operation | movement of the digital camera 1 of 2nd Embodiment. It is a figure for demonstrating the exposure control in 2nd Embodiment. It is a flowchart which shows operation | movement of the digital camera 1 of 3rd Embodiment. It is a figure which shows an example of the mode of the attention area 44 detected based on the direction of the detected face.

Hereinafter, the first embodiment will be described in detail with reference to the drawings as an example in which the imaging apparatus of the present invention is applied to a digital camera.
[First Embodiment]
A. Configuration of Digital Camera FIG. 1 is a block diagram showing a schematic electrical configuration of a digital camera 1 that implements the imaging apparatus of the present invention.
The digital camera 1 includes an imaging lens 2, a lens driving block 3, an aperture 4, a CCD 5, a vertical driver 6, a TG (timing generator) 7, a unit circuit 8, a DMA controller (hereinafter referred to as DMA) 9, a CPU 10, and a key input unit 11. , Memory 12, DRAM 13, DMA 14, image generation unit 15, DMA 16, DMA 17, display unit 18, DMA 19, compression / decompression unit 20, DMA 21, flash memory 22, face detection unit 23, and bus 24.

  The imaging lens 2 is composed of a plurality of lens groups (not shown) and has at least a focus lens 2A. A lens driving block 3 is connected to the focus lens 2A. The lens driving block 3 includes a focus motor that drives the focus lens 2A along the optical axis direction and a focus motor driver that drives the focus motor in accordance with a control signal sent from the CPU 10 (not shown).

The diaphragm 4 includes a drive circuit (not shown), and the drive circuit operates the diaphragm 4 in accordance with a control signal sent from the CPU 10.
The diaphragm is a mechanism that controls the amount of light incident on the CCD 5.
The exposure amount is determined by the aperture value (aperture level) and the shutter speed.

  The CCD 5 is scanned and driven by the vertical driver 6, photoelectrically converts the intensity of light of each color of the RGB value of the subject image at a constant period, and outputs it to the unit circuit 8 as an imaging signal. The operation timing of the vertical driver 6 and the unit circuit 8 is controlled by the CPU 10 via the TG 7. The CCD 5 has a function as an electronic shutter, and the electronic shutter is controlled via the CPU 10 via the vertical driver 6 and TG 7.

  A TG 7 is connected to the unit circuit 8, a CDS (Correlated Double Sampling) circuit that holds the imaged signal output from the CCD 5 by correlated double sampling, and an AGC that performs automatic gain adjustment of the imaged signal after the sampling. (Automatic Gain Control) circuit and an A / D converter that converts the analog signal after the automatic gain adjustment into a digital signal. The image pickup signal obtained by the CCD 5 passes through the unit circuit 8 and is then Bayered by the DMA 9 The data is stored in the buffer memory (DRAM 13).

The CPU 10 is a one-chip microcomputer that has functions to perform AF processing, recording processing, display processing, and the like, and controls each unit of the digital camera 1.
In particular, the CPU 10 controls the control unit 101 that restricts the area of the face detected by the face detection unit 23 based on the face orientation detected by the face detection unit 23, and the AE processing unit 102 that performs AE processing based on the image data. Have

The key input unit 11 includes a plurality of operation keys such as a half-press operation, a fully-pressable shutter button, a mode switching key, a recording button, a cross key, and a SET key, and sends an operation signal according to the user's key operation to the CPU 10. Output to.
The memory 12 stores a control program and necessary data necessary for the CPU 10 to control each unit of the digital camera 1, and the CPU 10 operates according to the program.

  The DRAM 13 is used as a buffer memory for temporarily storing image data picked up by the CCD 5 and also as a working memory for the CPU 10.

The DMA 14 reads out image data of Bayer data stored in the buffer memory and outputs it to the image generation unit 15.
The image generation unit 15 performs processing such as pixel interpolation processing, γ correction processing, and white balance processing on the image data transmitted from the DMA 14, and also generates a luminance color difference signal (YUV data). That is, it is a portion that performs image processing.
The DMA 16 stores the image data (YUV data) of the luminance / color difference signal subjected to image processing by the image generation unit 15 in a buffer memory.

The DMA 17 outputs image data of YUV data stored in the buffer memory to the display unit 18.
The display unit 18 includes a color LCD and its driving circuit, and displays an image of the image data output from the DMA 17.

The DMA 19 outputs the YUV data image data and the compressed image data stored in the buffer memory to the compression / decompression unit 20, and buffers the image data compressed by the compression / decompression unit 20 and the decompressed image data. It is stored in memory.
The compression / decompression unit 20 is a part that performs compression / decompression of image data (for example, compression / decompression in JPEG or MPEG format).
The DMA 21 reads the compressed image data stored in the buffer memory and records it in the flash memory 22 or stores the compressed image data recorded in the flash memory 22 in the buffer memory.

The face detection unit 23 performs face detection processing for detecting a face in the captured image data. This face detection process detects the face and also detects the face orientation. Since this face detection process is a well-known technique, it will not be described in detail. For example, characteristic data obtained when a general human face stored in advance is viewed from any direction (eye, eyebrows, nose when viewed from any direction). , Mouth, ear, and contour data of the entire face, etc.) and image data are compared with each other to detect a region that matches the facial feature data at a predetermined value or more as a face, and at the specified value or more. The direction of the matched face feature data is detected as the detected face direction.
Note that eye, nose, and mouth feature data may be included, the eyes, nose, and mouth may be detected, and the face orientation may be detected from the detected positional relationship between the eyes, nose, and mouth.

B. Operation of Digital Camera 1 Hereinafter, the operation of the digital camera 1 in the first embodiment will be described with reference to the flowchart of FIG.

  When the user presses the recording button of the key input unit 11 fully while the through image is displayed in the imaging mode (when an operation signal corresponding to the pressing operation of the recording button is sent from the key input unit 11), the moving image is captured. It is determined that the recording process has started, and the CPU 10 determines whether or not the imaging timing has come (step S1). Here, since the subject is imaged at a frame rate of 30 fps, the imaging timing arrives at 1/30 s (second) intervals.

  If it is determined in step S1 that the imaging timing has not arrived, the process stays in step S1 until the imaging timing arrives. When the imaging timing has arrived, the CPU 10 performs an imaging process. This imaging process is performed based on the shutter speed, aperture value, and gain value set by the AE process in step S6 described later. That is, the CCD 5 is exposed with the set shutter speed and aperture value, the image data stored in the CCD 5 is read, and the AGC of the unit circuit 8 automatically gains the read frame image data according to the set gain value. Then, the frame image data of the luminance / color difference signal after the automatic gain adjustment generated by the image generation unit 15 is stored in the buffer memory.

  Note that the first imaging after the moving image capturing / recording process is started is that the AE process is not performed in step S6, which will be described later. The imaging process is performed with the shutter speed, aperture value, gain value, and the like set by the AE process performed based on the AE process.

  Next, the CPU 10 outputs the frame image data captured in step S2 most recently stored in the buffer memory to the face detection unit 23, and the face detection unit 23 performs face detection on the most recently captured frame image data. A detection process is performed (step S3). Information indicating the face area detected by the face detection process and information indicating the face direction are sent to the CPU 10. At this time, the face detection process is performed according to control by the control unit 101 of the CPU 10 in step S4 described later. In addition, since the face detection control is not performed in step S4 to be described later after the moving image capturing process is started, the face detection is performed on the entire area of the frame image data or a predetermined or arbitrary area. Detection is performed.

  Next, the control unit 101 of the CPU 10 controls the face detection unit 23 so as to perform face detection processing that places importance on the detected direction of the face (step S4). The face detection process that places importance on the direction of the face is a face detection process that is performed only on a predetermined area based on the direction in which the face is facing. This is because it is considered that a person usually moves in the direction in which the face is facing.By performing face detection on a predetermined area in which the direction in which the face is facing is increased, the processing load of the face detection process is reduced. While reducing, it is possible to quickly detect the face. That is, it is possible to quickly follow the face detected by this control, and to reduce the processing load of the tracking.

Next, the AE processing unit 102 of the CPU 10 specifies a prediction region based on the detected face orientation (step S5). The prediction area based on the face direction refers to a predetermined area in which the prediction area is shifted from the detected face area in the direction of the face direction. This is because it is generally considered that a person moves in the direction in which the face is facing. The AE process is performed with emphasis on the identified prediction area.
FIG. 3 (A) is based on a face in the detected frame image data, a predetermined area to be subjected to face detection processing based on the detected face orientation, and the detected face orientation. An example of the state of the identified prediction area is shown.

As shown in FIG. 3A, the frame image data includes a face 31 and the direction of the face 31 is the right direction. Therefore, the predetermined area 32 to be subjected to the face detection process is in the direction in which the face 31 faces. It can be seen that this is a long area. By setting the target area for this face detection processing to the predetermined area 32, the processing load for the next face detection and tracking can be reduced.
3A, it can be seen that the prediction area 33 is a predetermined area that is shifted from an area where the face 31 is located in a direction in which the face 31 faces. The size of the prediction area 33 may be the same size as the detected face size, or may be a predetermined size.

Returning to the flowchart of FIG. 2, after the operation of step S5, the AE processing unit 102 of the CPU 10 performs an AE process in which the specified prediction region 33 is emphasized (step S6).
That is, among the frame image data captured in step S2 most recently stored in the buffer memory, the specified prediction area 33 is set as a photometry area, and AE processing is performed based on the luminance signal of the image data in the photometry area. Do. By this AE process, the shutter speed, aperture value, and gain value are set.

  Next, the CPU 10 performs AF processing on the face based on the detected face orientation (step S7). The AF process based on the detected face means that the search movement direction of the focus lens 2A is changed based on the detected face orientation.

Here, FIG. 4A shows the movement of the focus lens 2A.
As shown in FIG. 4A, the focus lens 2A can be moved from the lens end to the lens end, and the AF process searches and moves the focus lens to detect the focus lens position. The focus lens position is detected based on the high-frequency component of the image data at the position, and the focus lens 2A is moved to the detected focus lens position. Here, one lens end side is a lens position that focuses on a near subject, and the other lens end side is a lens position that focuses on a far subject.

At this time, the search movement direction of the focus lens 2A is changed according to the detected face orientation.
Here, FIG. 4B is a diagram illustrating an example of the detected face orientation.
As shown in FIG. 4B, when the detected face is facing the right front, the face is expected to approach the digital camera 1, and therefore the focus lens 2A is moved. Then, the focus lens position is detected by moving the search to the lens end side where the near subject is in focus.

  Further, as shown in FIG. 4C, when the detected face is facing rightward, the face is expected to move away from the digital camera 1, so that the focus lens 2A is turned off. Then, the focus lens is detected by moving the search to the lens end side where the far object is in focus.

  In other words, when the detected face orientation is closer to the digital camera 1, the focus lens 2A is searched and moved from the lens end side where the subject close to the current lens position is in focus, and the detected face orientation is detected. Is in the direction away from the digital camera 1, the focus lens 2 </ b> A is searched for from the lens end side where the subject far from the current lens position is in focus.

In addition, even when the detected face is away from the digital camera 1 but is not close to the digital camera 1 (for example, when the face is facing sideways), the focus lens 2A may not be moved (AF The focus lens 2A is narrow in the front and rear direction (the direction of the lens end that focuses on the near subject and the direction of the lens end that focuses on the far subject) from the current lens position. The focus lens position may be detected by moving within the range. This is because when the distance from the digital camera 1 to the subject does not change, the focus is not shifted or even if it is shifted.
As described above, the AF process can be quickly performed by changing the search movement direction of the focus lens 2A according to the detected face orientation.

  Returning to the description of the flowchart of FIG. 2, when the AF processing is performed on the face based on the detected face orientation, the CPU 10 immediately captures the frame imaged in step S <b> 2 recorded in the buffer memory. The image data is compressed by the compression / decompression unit 20, and the compressed frame image data is recorded in the flash memory 22 via the DMA 21 (step S8).

  Next, the CPU 10 determines whether or not to end the moving image capturing / recording process (step S9). This determination is made based on whether or not an operation signal corresponding to the full press operation of the shutter button has been sent from the key input unit 11 again.

  If it is determined in step S9 that the moving image capturing / recording process is to end, a moving image file is generated based on the recorded frame image data (step S10). If it is determined in step S9 that the moving image capturing / recording process is not ended, the process proceeds to step S1. Return and repeat the above operation. That is, if it is determined in step S1 that the imaging timing has arrived, in step S2, the CCD 5 is exposed with the shutter speed and aperture value set by the AE process in step S6, and the frame image data stored in the CCD 5 is read out. The AGC of the circuit 8 automatically adjusts the frame image data read from the CCD 5 according to the set gain value, and the frame image of the luminance / chrominance signal after the automatic gain adjustment generated by the image generator 15 is obtained. Store in buffer memory.

  FIG. 3B shows an example of the state of the frame image captured in step S2 after the AE process in step S6 and the state of the prediction area 33 that is the target of the photometric area in the AE process in step S6. is there.

  As shown in FIG. 3B, it can be seen that the face 31 on the frame image data is on the prediction region 33 that is the target of the AE process. That is, as shown in FIG. 3A, the area where the face is detected is not used as the photometry area, but the photometry area is shifted in the direction in which the face is facing. By performing the AE process in advance as a photometric area, it is possible to capture image data in which the face located at the time of imaging is appropriately exposed. That is, conventionally, as shown in FIG. 3A, the detected face area is used as a photometric area to perform the AE process and the imaging process is performed. It is possible to correct the problem that the areas are different and the actually captured face is not properly exposed.

  As described above, in the first embodiment, predetermined control for imaging can be appropriately performed based on the detected face orientation. For example, since the face detection process that places importance on the direction of the detected face direction is performed, the processing load for tracking can be reduced, and the face can be tracked quickly. Further, since the AE process is performed by specifying a photometric area that places importance on the detected face orientation, it is possible to capture image data in which the face area has an appropriate exposure amount even when the subject is moving. In addition, since the focus lens search movement is performed with an emphasis on the detected face orientation, AF processing can be quickly performed following the face.

[Second Embodiment]
Next, a second embodiment will be described.
D. Operation of Digital Camera 1 The second embodiment also implements the imaging device of the present invention by using the digital camera 1 having the same configuration as that shown in FIG.
The operation of the digital camera 1 in the second embodiment will be described below with reference to the flowchart of FIG.

  When the user presses the recording button of the key input unit 11 fully while the through image is displayed in the imaging mode (when an operation signal corresponding to the pressing operation of the recording button is sent from the key input unit 11), the moving image is captured. The CPU 10 determines that the recording process has been started, and determines whether or not the imaging timing has come (step S21). Here, since the subject is imaged at a frame rate of 30 fps, the imaging timing arrives at 1/30 s (second) intervals.

If it is determined in step S21 that the imaging timing has not arrived, the process stays in step S21 until the imaging timing has arrived. When the imaging timing has arrived, the CPU 10 performs an imaging process (step S22). This imaging process is performed based on the exposure conditions (shutter speed, aperture value, gain value) set most recently in step S28 or step S30 described later. That is, the CCD 5 is exposed with the set shutter speed and aperture value, the image data stored in the CCD 5 is read, and the AGC of the unit circuit 8 automatically gains the read frame image data according to the set gain value. Then, the frame image data of the luminance / color difference signal after the automatic gain adjustment generated by the image generation unit 15 is stored in the buffer memory.

Note that since the exposure condition is not set in the later-described step S28 or step S30 after the moving image capturing / recording process is started, the first image capturing is performed for the through image display before the moving image capturing / recording process. The imaging process is performed with the shutter speed, aperture value, gain value, and the like set by the AE process performed based on the frame image data.

  Next, the CPU 10 outputs the frame image data captured in step S22 most recently stored in the buffer memory to the face detection unit 23, and the face detection unit 23 performs face detection on the frame image data captured most recently. A detection process is performed (step S23). Information indicating the face area detected by the face detection process and information indicating the face direction are sent to the CPU 10. At this time, the face detection process is performed according to control by the control unit 101 of the CPU 10 in step S24 described later. Note that the first face detection after the moving image capturing / recording process is started is that face detection control is not performed in step S24, which will be described later, so that the entire area of the frame image data or a predetermined or arbitrary area is detected. Face detection is performed.

  Next, the control unit 101 of the CPU 10 controls the face detection unit 23 so as to perform face detection processing with an emphasis on the detected direction of the face (step S24). This control is the same as step S4 of FIG. 2 described in the first embodiment.

  Next, the CPU 10 performs an AF process on the face based on the detected face orientation (step S25). The AF process based on the detected face means that the search movement direction of the focus lens 2A is changed based on the detected face orientation, as in step S7 of FIG. 2 of the first embodiment. is there.

Next, the CPU 10 causes the compression / decompression unit 20 to compress the frame image data captured in step S22 most recently recorded in the buffer memory, and records the compressed frame image data in the flash memory 22 via the DMA 21. (Step S26).

  Next, the CPU 10 determines whether or not the detected face has entered the prediction region 33 most recently specified in step S31 described later (step S27). Here, the face entering the specified prediction area 33 indicates a case where the detected face is ½ or more in the prediction area 33, but the detected face is in the prediction area 33. If even a little enters, it may be determined that a face has entered the specified prediction area 33, or only when all of the detected faces have entered the prediction area 33. It may be determined that a face has entered.

In addition, when the prediction area | region specification of step S32 mentioned later is not performed, it branches to N at step S27. For example, after the imaging in the first step S22 after the moving image imaging / recording process is started, the prediction area 33 has not been specified yet, and therefore the process branches to N in step S27.

If it is determined in step S27 that the detected face has entered the most recently specified prediction area 33, the AE processing unit 102 of the CPU 10 uses the prediction area calculated most recently in step S33, which will be described later, as an exposure condition. (Step S28), the process proceeds to step S32 . As a result, in the next step S22, an image is captured under the exposure condition based on the set prediction region 33.

On the other hand, if it is determined in step S27 that the detected face is not included in the most recently specified prediction region 33, the CPU 10 determines whether or not the operation of the previous step S30 has ended (step S29). .
If it is determined in step S29 that the operation of the previous step S30 has not ended, the process proceeds to step S31. If it is determined that the operation of the previous step S30 has ended, the AE processing unit 102 of the CPU 10 determines the detected face area. The exposure condition is calculated and set as the photometric area (step S30), and the process proceeds to step S31. That is, the exposure conditions (aperture value, shutter speed, gain value) are calculated and set based on the luminance signal of the image data of the detected face area of the most recently captured frame image data. As a result, in the next step S22, an image is captured under the exposure condition based on the set face area.

In step S31, the CPU 10 determines whether or not the detected face orientation has changed. This determination is made based on whether or not the direction of the most recently detected face matches the direction of the previously detected face at a predetermined value or more, and if it matches at a predetermined value or more, the face direction changes. Judge that it is not.
For example, when the face orientation is not detected before the most recent face orientation detection, such as when the face orientation is detected for the first time, the detected face orientation is Judge that it has changed.

  If it is determined in step S31 that the orientation of the face has not changed, the process proceeds to step S34 as it is. If it is determined in step S31 that the orientation of the face has changed, the already specified prediction region 33 cannot be maintained. Since it is considered that the face does not move in the direction of the predicted region 33, the process proceeds to step S32. Note that even if it is determined that the face orientation has not been detected and the face orientation has changed before the most recent face orientation detection, the prediction region has not been identified, so that the process returns to step S32. Will go on.

  In step S32, the AE processing unit 102 of the CPU 10 identifies the prediction region based on the face orientation most recently detected in step S23. The prediction area based on the face direction refers to a predetermined area in which the prediction area is shifted from the detected face area in the direction of the face direction. This is because it is generally considered that a person moves in the direction in which the face is facing. Exposure conditions are calculated with emphasis on the identified prediction area.

  Next, the AE processing unit 102 of the CPU 10 calculates the exposure condition by placing importance on the identified prediction region 33 (step S33), and proceeds to step S34. That is, the specified prediction area 33 of the most recently captured frame image data is set as a photometry area, and exposure conditions (aperture value, shutter speed, gain value) are calculated based on the luminance signal of the image data in the photometry area. .

  In step S34, the CPU 10 determines whether or not to end the moving image capturing / recording process. This determination is made based on whether or not an operation signal corresponding to the full press operation of the shutter button has been sent from the key input unit 11 again.

If it is determined in step S34 that the moving image capturing / recording process is to end, a moving image file is generated based on the recorded frame image data (step S35). If it is determined in step S34 that the moving image capturing / recording process is not ended, the process proceeds to step S1. Return and repeat the above operation. In other words, if it is determined in step S21 that the imaging timing has arrived, in step S22, the CCD 5 is exposed with the shutter speed and aperture value set in step S28 or step S30 , and the frame image data stored in the CCD 5 is read out. The AGC of the unit circuit 8 automatically adjusts the gain of the frame image data read from the CCD 5 in accordance with the set gain value, and the frame of the luminance color difference signal after the automatic gain adjustment generated by the image generator 15. Store the image in the buffer memory.

FIG. 6 is a diagram for explaining exposure control in the second embodiment.
FIG. 6A is assumed to be frame image data captured for the first time after the start of the moving image recording process.
A face 31 in FIG. 6A indicates a face detected, an area 34 indicates a detected face area, and an area 33 indicates a prediction area specified based on the detected face orientation. Yes.

First, when frame image data is captured for the first time after the start of the moving image recording process, the process branches to N in step S27, and the exposure condition based on the detected face area 34 (face area 34 shown in FIG. 6A). Is calculated and set ( step S30 ). Then, it is determined in step S31 that the orientation of the face has changed, and based on the detected orientation of the face 31, a prediction area 33 as shown in FIG. 6A is specified (step S32). An exposure condition is calculated using the photometric area (step S33).
If it is determined in step S34 that the moving image recording process is not to be ended, an image is captured in step S22 under the exposure condition calculated based on the face area of FIG.

FIG. 6B is a diagram illustrating an example of the state of the frame image data captured at this time. The prediction region 33 in FIG. 6B is the prediction region 33 identified most recently, that is, the same region as the prediction region 33 shown in FIG.
As shown in FIG. 6B, since the detected face 31 is more than half in the prediction area 33, it is determined in step S27 that a face has entered the prediction area 33, and the prediction area 33 already calculated is entered. Based on the exposure condition, that is, the exposure condition calculated based on the image data of the prediction area 33 in FIG. 6A is set (step S28).
The next frame image data is captured under the exposure condition based on the set prediction area 31.

FIG. 6C is a diagram illustrating an example of the state of the frame image data captured at this time.
As can be seen from FIG. 6C, the face 31 on the frame image data is on the prediction region 33. That is, if it is determined that the face 31 has entered the prediction area 33, the image is captured under the exposure condition based on the prediction area 33 calculated in advance, and thus image data in which the face located at the time of image capture has an appropriate exposure amount is captured. be able to.

In addition, unlike the first embodiment in which the prediction area 33 based on the face direction is specified and the specified prediction area 33 is always the photometric area, when the face 31 enters the specified prediction area 33, the specification is performed. Since the image is taken under the exposure condition calculated in advance based on the predicted region 33, appropriate exposure follow-up can be performed quickly. That is, conventionally, since the AE process is performed using the detected face area as the photometry area as shown in FIG. 6A, the AE process takes time, and the next shooting is performed. Can correct the problem that the face actually captured does not have proper exposure because the face has already moved.
Further, since shooting is performed under an exposure condition calculated in advance after the subject enters the prediction area 33, appropriate imaging processing can be performed even when the subject's face is facing sideways but not moving. . In addition, even when the subject starts to move from the upper body where the subject has stopped, the direction of the subject's face can be predicted from the direction of the subject's face, so appropriate exposure tracking is quickly performed on the subject's face. be able to.

As described above, in the second embodiment, predetermined control for imaging can be appropriately performed based on the detected face orientation. For example, the prediction area 33 is specified based on the detected face orientation, exposure conditions are calculated using the specified prediction area 33 as a photometric area, and detection is performed until a face enters the specified prediction area 33. The exposure control is performed based on the specified face area, and when a face enters the specified prediction area, the image is captured under the exposure condition based on the calculated prediction area 33. Even if the subject is stopped or moving, appropriate imaging processing can always be performed. In addition, when the subject starts moving from a stopped state, an appropriate imaging process can be performed while following the subject quickly.
In addition, since the focus lens search movement is performed with an emphasis on the detected face orientation, AF processing can be quickly performed following the face.

[Modification 1]
The first and second embodiments can be modified as follows.

(1) In the first and second embodiments, the prediction area 33 based on the detected face orientation is specified, and AE processing and calculation of exposure conditions are performed using the specified prediction area 33 as a photometric area. However, regardless of the detected face orientation, the photometry area is the entire area of the image data, and the AE processing and the exposure condition calculation are performed with emphasis on the prediction area 33 based on the detected face orientation direction. You may do it. That is, the luminance component of the image data of the area shifted in the direction of the detected face direction may be weighted to calculate the AE process and the exposure condition.
If it demonstrates using FIG. 3 (A), while making all the area | regions of image data into a photometry area | region, the predetermined area | region 33 is weighted and AE processing will be performed. 6A, the entire area of the image data is set as a photometric area, and the predetermined area 33 is weighted to calculate the exposure condition.
In short, what is necessary is just to perform AE processing and calculation of exposure conditions with emphasis on a predetermined area based on the detected face orientation.

(2) In the first and second embodiments, the detected amount of movement of the face is detected, and based on the detected amount of movement, the face of the predetermined region 32 that is the target of face detection processing You may make it change the length of the direction which is facing, and the magnitude | size of the predetermined area | region 32. FIG.
If it demonstrates using FIG. 3 (A), according to the detected amount of movement of the face 31, the length of the direction of the direction of the detected face of this predetermined area | region 32 will be changed.
When the amount of motion of the face 31 is small, increasing the direction of the detected face direction of the predetermined area 32 that is the target of the face detection process or increasing the predetermined area 32 only increases unnecessary processing. In addition, when the amount of movement of the face 31 is large, the face 31 is not detected when the direction of the detected face of the region 32 to be subjected to the face detection process is shortened or the predetermined region 32 is made small. Because there is. As a result, the face can be detected quickly and the burden of the face detection process can be reduced.
The detection of the amount of movement of the face may be performed by detecting the amount of movement based on the position of the face detected by face detection, or by detecting the motion vector of the face using a block matching method or the like. The amount may be detected.
Further, in consideration of both the direction of the detected face 31 and the amount of movement, the length of the predetermined area 32 in the direction of the detected face direction may be changed.
Note that, as in the prediction region 33, the predetermined region 32 may be shifted from the face region in the direction of the detected face direction. In this case, based on the detected amount of motion, the amount of shift from the detected face area of the predetermined area 32 in the direction of the face is changed.

(3) In the first and second embodiments described above, the detected amount of motion of the detected face is detected, and the detected face of the prediction region 33 to be identified based on the detected amount of motion The amount of shift from the region to the direction of the face may be changed.
If it demonstrates using FIG. 3 (A), according to the detected motion amount of the face 31, the amount to shift of the photometry area | region 33 to specify will be changed. That is, when the amount of motion is large, the amount of shift is increased accordingly, and when the amount of motion is small, the amount of shift is decreased accordingly.
Thus, based on the detected amount of movement of the face 31, it is possible to identify the area where the face 31 is supposed to be located at the time of imaging as the predicted area 33, and the face area appropriately positioned at the time of imaging is appropriately exposed. Can be captured.
The detection of the amount of motion of the face may be performed by detecting the amount of motion based on the position of the face detected in the past by the face detection, or by detecting the motion vector of the face using a block matching method or the like. Thus, the amount of motion may be detected.
Note that the size of the prediction region 33 to be specified may be changed based on the detected amount of motion. This also makes it possible to identify the area where the face 31 is supposed to be located at the time of imaging based on the detected amount of movement of the face 31 as the prediction area 33, and the face area properly positioned at the time of imaging is appropriately exposed. The amount of image data can be captured.

(4) In the first and second embodiments described above, the control of the face detection unit 23 that performs the face detection process that places importance on the detected face orientation lengthens the detected face orientation direction. The face detection process is performed on the predetermined area 32 so as to detect an area matching the feature data of the predetermined subject at a predetermined value or more as a subject, but face detection is performed on all areas. And the predetermined value of face detection in a region other than the predetermined region 32 where the direction of the detected face is long may be set higher than the predetermined value in the predetermined region, or the detected face The predetermined value of face detection in the predetermined area 32 having a long direction may be made lower than the predetermined value in an area other than the predetermined area.
Further, the number of feature data of a predetermined subject to be searched in the predetermined area may be larger than the number in the area other than the predetermined area.
In addition, the feature data is searched in the predetermined area first, and if the area that matches the feature data at a predetermined value or more is not detected in the predetermined area, the feature data is searched in the area other than the predetermined area. Also good.
This makes it easier to detect a face within a predetermined range area in which the detected face direction is long.

(5) In the first and second embodiments, the case of the moving image capturing / recording process has been described. However, the present invention may be applied even when displaying a through image in the moving image capturing mode or the still image capturing mode. .
In other words, in the case of the first embodiment, when the through image display is applied, after the operation of step S7 in FIG. 2, the operation of step S8 is not performed and the process proceeds to step S9 as it is, and the moving image is displayed in step S9. It is determined whether or not to perform an imaging recording process or a still image imaging and recording process, and if it is determined not to perform the process, the process returns to step S1. If it is determined in step S8 that the moving image capturing / recording process and the still image capturing / recording process are to be performed, the process proceeds to the imaging / recording process. In the case of the second embodiment, after the operation in step S25 in FIG. 5, the operation proceeds to step S27 without performing the operation in step S26. In step S34, the moving image capturing / recording process or the stationary operation is performed. It is determined whether or not the image capturing / recording process is to be performed. If it is determined not to be, the process returns to step S21.
At this time, when the still image capturing / recording process is performed, the exposure condition performed based on the predicted region 33 or the face region most recently specified in step S6 in FIG. 2 or step S28 or step S30 in FIG. May be replaced with exposure conditions for still image capturing.
Alternatively, autofocus may be performed on the most recently detected face before taking a still image.

(6) In the first and second embodiments, depending on the detected face orientation, the length of the predetermined area 32 that is the target of the face detection process and the size of the predetermined area 32 You may make it change.
Further, the amount of the prediction area 33 to be identified that is shifted from the detected face area toward the face direction may be changed according to the detected face direction.
For example, even if the face is moving at the same speed, if the face is facing sideways, the moving distance of the face is the longest between the captured image data, and the face orientation is In the case of the diagonally forward direction and the diagonally backward direction, the movement distance of the face between the captured image data is shorter than that of the sideways image, and when the face is facing the front and directly behind, the distance between the image data is one. This is because the movement distance is shortened.
As a result, it is possible to capture image data in which the face area positioned at the time of image capture has an appropriate exposure amount, to quickly detect the face, and to reduce the burden of face detection processing. .
Note that, as in the prediction region 33, the predetermined region 32 may be shifted from the face region in the direction of the detected face direction. In this case, based on the detected face direction, the amount of the predetermined area 32 to be shifted from the detected face area in the direction of the face is changed. Further, the size of the specified prediction area 33 may be changed based on the detected face orientation . Also by this, the above effects can be obtained.

  (7) When the moving speed of the subject is slower than the predetermined speed, exposure control as in the second embodiment is performed, and when the moving speed of the subject exceeds the predetermined speed, the first embodiment You may make it switch automatically so that exposure control like this may be performed. The moving speed of the subject may be detected based on the face position detected in the past by the face detection, or may be detected by detecting the motion vector of the face using a block matching method or the like. The speed may be detected.

(8) In the first and second embodiments, the face is made to follow by face detection by the face detection unit 23, but the face detection unit 23 is made to follow the face detected by the block matching method or the like. It may be. That is, the detected face area is tracked by detecting where the image data of the detected face area is in the frame image data captured later by the block matching method.
Also at this time, face tracking is performed with emphasis on the orientation of the face detected by the face detection unit 23. That is, the search range of the detected face is changed based on the detected face orientation.

For example, when the frame image data is captured as shown in FIG. 3A, the detected face is the face 31, so the search range is set to the region 32 instead of the entire region of the frame image data. . That is, the detected face is tracked by detecting where the image data of the detected area of the face 31 is in the area 32 of the frame image data captured thereafter. Thereby, a face can be followed quickly.

In this case, although it is possible to follow the face, since the direction of the following face is unknown, the face detection unit 31 follows the face area by detecting the face area. The direction of the face is detected. In this case, since face detection is performed on the following face area, the processing load of face detection can be reduced, and the face can be detected quickly.
The face tracking by the block matching method is also a kind of face detection because the face area is detected in principle, but the face detection by the face detection unit 23 is different from the face detection by the face detection unit 23 in advance. The face detection by the block matching method is to compare and collate the image data of the captured face area with the frame image data captured after that, while comparing and collating with the recorded face feature data.

(9) In the second embodiment, when a prediction region where a subject is supposed to move is specified and the subject enters the specified prediction region, an appropriate value within the prediction region calculated in advance is determined. Although the imaging control is performed under the conditions, it is determined whether or not the subject has moved, and the imaging control may be performed under the conditions calculated in advance when the subject moves in the predicted direction. Good. In other words, the prediction of the direction of movement and the prediction of the predicted area from the direction of the face are performed, and when it is determined that the face has moved in the predicted direction, the appropriate conditions within the prediction area calculated in advance are used. Perform imaging control.
In this case, since the imaging control is performed under appropriate conditions in the prediction area specified in the next imaging after detecting the movement of the subject, it is detected that the subject has entered the specified prediction area because the movement speed of the subject is fast. Even in the next imaging, it is possible to prevent the problem that the subject is not in the prediction area.

  (10) In the first and second embodiments, the human face is detected. However, instead of the human face, the face of an animal such as a cat or dog or the entire human body (head, torso) , A predetermined object such as an arm, a leg, or the like) may be detected. At this time, the direction of the predetermined subject is also detected.

  (11) In the first and second embodiments, the area to be face-detected is the predetermined area 32 and the area to be subject to exposure control is the prediction area 33, but the area to be face-detected In addition, the area subjected to exposure control may be the same. That is, face detection and exposure control may be performed on the predetermined area 32 described above, or face detection and exposure control may be performed on the prediction area 33.

  (12) Moreover, the aspect which arbitrarily combined the said modification (1) thru | or (11) in the range which is not inconsistent may be sufficient.

[Third Embodiment]
Next, a third embodiment will be described.
E. Operation of Digital Camera 1 The third embodiment also realizes the imaging apparatus of the present invention by using the digital camera 1 having the same configuration as that shown in FIG. Note that the CPU 10 includes a target area detection unit that detects a target area based on the detected orientation of each face.
The operation of the digital camera 1 in the third embodiment will be described below with reference to the flowchart of FIG.

  When the still image capturing mode is set by the user operating the mode switching key of the key input unit 11, the CPU 10 starts capturing the subject by the CCD 5 at a predetermined frame rate, and is sequentially generated and buffered by the image generating unit 15. A so-called through image display is started in which the frame image data of the luminance color difference signal stored in the memory (DRAM 13) is displayed on the display unit 18 (step S51).

  Next, the CPU 10 determines whether or not the user has pressed the shutter button halfway (step S52). This determination is made based on whether or not an operation signal corresponding to a half-press operation of the shutter button is sent from the key input unit 11.

  If it is determined in step S52 that the shutter button is not half-pressed, the process stays in step S52 until it is determined that the shutter button is half-pressed. If it is determined that the shutter button is half-pressed, the CPU 10 stores the latest stored in the buffer memory. Is output to the face detection unit 23, and the face detection unit 23 is caused to perform face detection processing on the most recently captured frame image data (step S53). Information indicating the face area detected by the face detection process and information indicating the face direction are sent to the CPU 10.

Next, the CPU 10 determines whether or not a plurality of faces are detected by the face detection process (step S54).
If it is determined in step S54 that a plurality of faces have been detected, the attention area detection unit of the CPU 10 detects the attention area based on the detected orientation of each face (step S55). This attention area refers to an area where a person with a detected face is looking (attention).
In the detection of the attention area, an area where the extension lines in the detected direction of each face intersect is detected as the attention area.

FIG. 8 is a diagram illustrating an example of the state of the attention area detected based on the detected face orientation.
Referring to FIG. 8, it can be seen that the face 41 and the face 42 are looking at the subject 43, and the subject of interest (the subject of interest) is the subject 43.
Here, the face 41 and the face 42 are detected, and a predetermined area centering on a point where the extension lines in the direction of the detected faces intersect is the attention area 44.

  Returning to the flowchart of FIG. 8, when the attention area is detected, the CPU 10 determines whether or not the attention area 44 is detected by the attention area detection unit (step S <b> 56).

  If it is determined in step S56 that the attention area 44 has been detected, the AE processing unit 102 of the CPU 10 sets the detected attention area 44 as a photometry area, and based on the luminance component of the image data in the photometry area, the AE for still images. In addition to performing the process, the CPU 10 performs an AF process on the area of interest (step S57) and proceeds to step S59. In this AF process, the focus lens 2A is moved, and the focus lens 2A is moved to the focus lens position based on the high-frequency component of the image data of the area of interest. At this time, information indicating the attention area (for example, attention frame) may be displayed on the through image.

  On the other hand, if it is determined in step S54 that a plurality of faces are not detected, or if it is determined in step S56 that the attention area 44 cannot be detected, the AE processing unit 102 of the CPU 10 determines the detected faces. Using the area as a photometry area, AE processing for still images is performed based on the luminance component of the image data in the photometry area, and the AF processing unit of the CPU 10 performs AF processing on the detected face (step S58), the process proceeds to step S59. In this AF process, the focus lens 2A is moved, and the focus lens 2A is moved to the focus lens position based on the high-frequency component of the image data of the detected face area.

  At this time, if there are a plurality of detected faces, the AE process may be performed based on all the face areas, or the AE process may be performed based on any one of the face areas. May be. In addition, when there are a plurality of detected faces, the AF process may be performed so that all the detected faces are in focus, or the AF process may be performed on any one of the faces. Good.

In step S59, the CPU 10 determines whether or not the shutter button has been fully pressed by the user.
If it is determined in step S59 that the shutter button is not fully pressed, the process stays in step S59 until it is fully pressed. If it is determined that the shutter button is fully pressed, the CPU 10 is set by the AE process in step S57 or step S58. The still image capturing process is performed under the exposure conditions, and the image data of the luminance color difference signal generated by the image generation unit 15 and compressed by the compression / decompression unit 20 is recorded in the flash memory 22 via the DMA 21 (step S60).

  As described above, in the third embodiment, predetermined control for imaging can be appropriately performed based on the detected face orientation. For example, when a plurality of faces are detected, an area where the extension lines in the direction of the detected faces intersect is detected as the attention area 44 in which the imaged person is paying attention. Since the AE process and the AF process are performed on the area 44, it is possible to appropriately control the exposure and to focus on the subject that the subject is paying attention to.

[Modification 2]
The third embodiment can be modified as follows.

(01) In the third embodiment, if it is determined in step S59 in FIG. 7 that the shutter button is not fully pressed, the process stays in step S59 until it is fully pressed. If it is determined that the button is not fully pressed, the process may return to step S53. That is, the continuous AF process may be performed in which the area of interest is tracked and the face is tracked until the shutter button is fully pressed.
Further, although the case of still image capturing has been described, the present invention may be applied to the case of moving image capturing.

  (2) In the third embodiment, the region where the extension lines in the direction of each detected face intersect is detected as the attention area. However, a part of each detected face (2 In the direction of more than one face, when an extension line in the direction of the direction intersects in an area, the area may be detected as an attention area.

  (3) In the third embodiment, the AE process is performed using the detected area of interest or the detected face area as the photometric area, but the entire area of the captured image data is the photometric area. AE processing may be performed by weighting at least the detected attention area.

  (4) In the third embodiment, the human face is detected. However, instead of the human face, the face of an animal such as a cat or dog or the entire human body (head, torso, arms, A predetermined subject such as a whole body such as a foot) may be detected. At this time, the direction of the predetermined subject is also detected.

  (5) In the third embodiment, the case of still image capturing has been described. However, the present invention may be applied during moving image capturing. That is, the AE process and the AF process in the detected attention area 44 may be periodically performed during moving image capturing.

  (6) Moreover, the aspect which arbitrarily combined the said modification (1) thru | or (5) in the range which is not inconsistent may be sufficient.

  Further, in each of the above embodiments, the face detection unit 23 detects the face and also detects the orientation of the detected face. However, the face detection unit 23 detects the face and detects it. A means for detecting the orientation of the face may be provided separately.

The above-described embodiments and modifications of the present invention are merely examples as the best embodiments, and are described in order to better understand the principle and structure of the present invention. And is not intended to limit the scope of the appended claims.
Therefore, it should be understood that all the various variations and modifications that can be made to the above-described embodiments of the present invention are included in the scope of the present invention and protected by the appended claims. .

  Finally, in each of the above embodiments, the case where the imaging apparatus of the present invention is applied to the digital camera 1 has been described. However, the present invention is not limited to the above embodiment, and the main point is to image a subject. Applicable to any device that can be used.

1 Digital Camera 2 Imaging Lens 3 Lens Drive Block 4 Aperture 5 CCD
6 Vertical driver 7 TG
8 Unit circuit 9 DMA
10 CPU
11 Key input section 12 Memory 13 DRAM
14 DMA
15 Image generator 16 DMA
17 DMA
18 Display 19 DMA
20 Compression / decompression unit 21 DMA
22 Flash memory 23 Face detection unit 24 Bus

Claims (2)

Imaging means for imaging a subject;
Subject detection means for detecting a subject in the image data imaged by the imaging means;
Identification means for identifying a direction in which the subject detected by the subject detection means is facing ;
A control means for controlling focus so as to detect a focus lens position in which the subject is in focus by changing a search movement range of the focus lens according to a direction in which the subject identified by the identification means is facing ;
An imaging apparatus comprising: A computer having an imaging means for imaging a subject,
Subject detection means for detecting a subject in the image data imaged by the imaging means;
Identification means for identifying a direction in which the subject detected by the subject detection means is facing ;
According to the direction in which the subject identified by the identification unit is facing , the focus lens search movement range is changed to function as a control unit that performs focus control so as to detect a focus lens position in focus on the subject. A program characterized by

Images