JP5323243B2 - Image processing apparatus and control method thereof - Google Patents

Description

  The present invention relates to an image processing apparatus and a control method thereof, and more particularly to an image processing apparatus having a subject detection function and a control method thereof.

  In an imaging apparatus using a photoelectric conversion element such as a digital camera or a video camera, an object can be detected from a captured image using an image processing technique. For this reason, an imaging apparatus that performs automatic focus control (AF) for focusing on a detected subject and automatic exposure control (AE) for achieving a proper exposure of the detected subject is also realized.

  For example, in an EVF image (also called a live image, a live view image, a live view image, or the like) that is continuously captured in order to make the display device included in the image sensor function as an electronic viewfinder (EVF), subject detection is continuously performed. Think about what to do. In this case, for example, a method of detecting an object from an individual image using an image recognition technique such as face detection (see Patent Document 1) or a method of detecting (tracking) a movement of an area designated as an object area ( (See Patent Document 2).

  In face detection, facial features such as eyes and mouth are extracted from an image, and a subject is detected from each image by determining whether the face is a face. On the other hand, in the method of tracking the subject area, a subject is detected from each image by searching another frame image for an area highly correlated with the subject area selected in a certain frame image. Hereinafter, in this specification, for convenience, detection of a subject using image recognition such as face detection is referred to as “face detection”, and detection of a subject based on motion detection of a subject region is referred to as “moving object detection”. However, it should be noted that this name does not include the intention to limit the subject to a human face.

JP 2007-274587 A JP 2001-243478 A

Face detection requires recognition of multiple parts that make up the face, so subject detection accuracy is high, but computation is complex and processing takes time, so subject detection speed (follow-up performance) can detect moving objects. Inferior. In contrast, moving object detection detects a subject by calculating the difference between a plurality of frame images, so it can be performed faster than face detection, but detection accuracy becomes difficult when the brightness of the subject changes. Inferior to face detection.
As described above, in the conventional tracking method of the subject area, the detection speed is lowered if priority is given to the detection accuracy of the subject, and the detection precision is lowered if priority is given to the detection speed.

  The present invention has been made in view of such a problem of the prior art, and an object thereof is to realize an image processing apparatus capable of appropriately tracking a subject area and a control method therefor.

  In order to solve the above problems, an image processing apparatus according to the present invention detects a predetermined subject existing in a frame image by detecting a part having a predetermined characteristic from a frame image of a moving image. And a second detection unit that tracks a region where a predetermined subject detected by the first detection unit exists in the moving image by searching for a similar region between the images of the frame. Determining means for determining a region in the moving image where a predetermined subject is present based on at least one of the detection result by the first detection means and the detection result by the second detection means, If the predetermined condition is satisfied, the region where the predetermined subject exists is determined based on the detection result by the first detection means, not by the detection result by the second detection means. When the predetermined condition is not satisfied, a region where the predetermined subject exists is determined based on the detection result by the first detection unit and the detection result by the second detection unit. The conditions include at least one of a gain adjustment amount applied to the moving image exceeding a predetermined threshold and an exposure time for one frame image being longer than a predetermined threshold. It is what.

  Similarly, in order to solve the above-described problem, in the control method of the image processing apparatus according to the present invention, the first detection unit detects a part having a predetermined feature from the image of the frame of the moving image. In the first detection step, the first detection step of detecting a predetermined subject existing in the image of the first image and the second detection means search for a similar region between the images of the frame. A second detection step for tracking a region where the detected subject is detected, and a determination unit based on at least one of the detection result in the first detection step and the detection result in the second detection step; A determination step of determining a region where a predetermined subject exists in the moving image, and the determination means in the determination step detects the detection result in the second detection step when the predetermined condition is satisfied. Regardless of the case, a region where a predetermined subject exists is determined based on the detection result in the first detection step, and when the predetermined condition is not satisfied, the detection result in the first detection step and the second Based on the detection result in the detection step, a region where a predetermined subject exists is determined, and the predetermined condition is that the gain adjustment amount applied to the moving image exceeds a predetermined threshold value, And an exposure time for an image of one frame is longer than a predetermined threshold value.

  With such a configuration, according to the present invention, an image processing apparatus capable of appropriately tracking a subject area and a control method thereof can be realized.

It is a block diagram which shows the function structural example of the digital camera as an example of the image processing apparatus which concerns on the 1st Embodiment of this invention. 3 is a flowchart showing a subject detection operation in the digital camera according to the first embodiment of the present invention. It is a figure which shows typically the influence which the change of the image quality by a sensitivity and the change of an image quality gives to the precision of a moving body detection. It is a figure which shows typically the example of the frame display in the digital camera which concerns on the 1st Embodiment of this invention. It is a flowchart which shows the to-be-photographed object detection operation | movement in the digital camera which concerns on the 2nd Embodiment of this invention. It is a figure for demonstrating typically the problem of the moving body detection when a shutter speed is slow. It is a figure which shows typically the effect of the 2nd Embodiment of this invention.

Hereinafter, preferred and exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings.
<First Embodiment>
The image processing apparatus according to the first embodiment of the present invention first detects a subject area with high accuracy by face detection, and performs moving object detection using the detected subject area, thereby detecting the subject detection accuracy and detection speed ( (Trackability) can be made compatible.

FIG. 1 is a block diagram illustrating a functional configuration example of a digital camera as an example of an image processing apparatus according to the first embodiment of the present invention.
(Configuration of digital camera)
The operation unit 101 is a user interface for a user of the digital camera 100 to input various instructions to the digital camera 100, and includes an input device such as a switch or a button.

  The operation unit 101 includes a shutter switch. The control unit 102 is notified of the signal SW1 when the shutter switch is half-pressed and the signal SW2 when the shutter switch is fully pressed.

  The control unit 102 controls the operation of each unit of the digital camera 100 in accordance with an instruction from the operation unit 101 and realizes the function of the digital camera 100. The control unit 102 includes, for example, a CPU, a nonvolatile memory that stores a program executed by the CPU, and a RAM for reading the program and using it as a work area.

  The control unit 102 has an automatic exposure control (AE) function that calculates subject luminance from digital image data output from an image processing unit 105 (to be described later) and automatically determines at least one of the shutter speed and the aperture according to the shooting mode. Realize. Further, the control unit 102 notifies the A / D conversion unit 104 of a gain adjustment amount corresponding to the set sensitivity. The set sensitivity may be a fixed sensitivity set by the user, or may be a sensitivity dynamically set by the control unit 102 based on the result of the AE process. In addition, the control unit 102 determines whether or not the flash unit 111 needs to emit light at the time of actual shooting according to the flash setting, the automatically determined shutter speed, and the like. When the flash emission is determined, the control unit 102 instructs the EF processing unit 110 to turn on the flash. When the EF processing unit 110 receives a flash-on instruction from the control unit 102, the EF processing unit 110 controls the flash unit 111 and causes the flash unit 111 to emit light in accordance with the opening timing of the shutter of the exposure mechanism 109.

  Further, the control unit 102 drives the focus lens of the lens 108a using the lens driving unit 108, and detects a change in contrast of the digital image data output from the image processing unit 105, thereby realizing an automatic focusing control function. .

  The image sensor 103 is a photoelectric conversion device such as a CCD image sensor or a CMOS image sensor, and converts a subject optical image formed through the lens 108a and the exposure mechanism 109a into an analog electrical signal (analog image data) in units of pixels. To do.

  The lens 108a has an AF function, and drives the focus lens in accordance with control from a lens driving unit 108 described later. The exposure mechanism 109a has an aperture and a mechanical shutter, and opens the optical path between the lens 108a and the image sensor 103 with the aperture and shutter speed according to the control of a mechanical drive unit 109, which will be described later. To expose.

  The A / D conversion unit 104 performs correlated double sampling, gain adjustment, A / D conversion, and the like on the analog image data output from the image sensor 103, and outputs the result as digital image data. The gain adjustment amount (amplification factor) to be applied is given from the control unit 102. If the gain is large, as a result, the noise component included in the image also becomes large.

  The image processing unit 105 performs image processing such as white balance correction, pixel interpolation processing, gamma correction processing, and color difference signal generation on the digital image data output from the A / D conversion unit 104, for example, processing YUV image data Output as completed digital image data.

  The face detection unit 106 functions as a first subject detection unit that performs subject detection by image recognition. The face detection unit 106 detects a human face as an example of a subject from the digital image data output from the A / D conversion unit 104. Then, the face detection unit 106 sends information (face information) related to the face area, such as the position and range (size) of the detected face, reliability (probability as a face), and the like to the control unit 102 and the moving object detection unit 115. Notice. The face position may be the center coordinates of the face area.

  A known face detection technique can be used for face detection in the present embodiment. As a known face detection technique, a method based on learning using a neural network or the like, template matching is used to search a part having a characteristic shape of eyes, nose, mouth, etc. from an image, and if the degree of similarity is high, it is regarded as a face There are methods. In addition, many other methods have been proposed, such as a method that detects image feature amounts such as skin color and eye shape and uses statistical analysis. In general, a plurality of these methods are combined to improve the accuracy of face detection. Specific examples include a face detection method using wavelet transform and image feature amount described in JP-A-2002-251380.

  The moving body detection unit 115 functions as a second subject detection unit that performs subject detection by moving body detection. The moving object detection unit 115 discriminates a moving object from two images of digital image data continuous in time series output from the image processing unit 105, and calculates its position, range, and movement amount. The moving object detection unit 115 of this embodiment includes an angular velocity sensor (not shown), and also detects the movement of the digital camera 100. Note that by using the face information detected by the face detection unit 106, the moving object detection unit 115 can distinguish the movement of the background from the movement of the subject, and the moving object information (position and range) for each of the background and the subject. , Movement amount).

  The EVF display unit 107 includes a display device such as an LCD, and displays an image based on the digital image data processed by the image processing unit 105.

  The format conversion unit 112 generates a data file for recording conforming to, for example, DCF (Design Fule for Camera File System) from the digital image data output from the image processing unit 105. The format conversion unit 112 performs encoding into a JPEG format, generation of a file header, and the like in the process of generating a data file.

  The image recording unit 113 records the data file generated by the format conversion unit 112 in a built-in memory of the digital camera 100, a removable medium attached to the digital camera 100, or the like.

  The external connection unit 114 is an interface for connecting the digital camera 100 to an external device such as a PC (personal computer) or a printer. The external connection unit 114 communicates with an external device in conformity with general standards such as USB, IEEE 1394, IEEE 802.11, and the like.

(Description of operation)
Next, the operation of the digital camera 100 of this embodiment will be described.
First, when the user of the digital camera 100 turns on a power switch included in the operation unit 101, the control unit 102 detects this, and powers each unit constituting the digital camera 100 from a battery or an AC input (not shown). Supply.

  The digital camera 100 of this embodiment is configured to start an EVF display operation when power is supplied. Specifically, the mechanical shutter of the exposure mechanism 109a is opened, and the image sensor 103 is exposed. The charges accumulated in each pixel of the image sensor 103 are sequentially read out at a cycle that realizes a predetermined frame rate, and output to the A / D converter 104 as analog image data. Thus, in this embodiment, the image for EVF display is acquired by continuously capturing images using a so-called electronic shutter.

  The A / D conversion unit 104 performs correlated double sampling, gain adjustment, A / D conversion, and the like on the analog image data output from the image sensor 103, and outputs the result as digital image data.

  The image processing unit 105 performs image processing such as white balance correction, pixel interpolation processing, gamma correction processing, and color difference signal generation on the digital image data output from the A / D conversion unit 104, for example, processing YUV image data Output as completed digital image data.

  The face detection unit 106 detects a human face by image recognition from the digital image data output from the A / D conversion unit 104, and detects the position and range of the detected face, reliability (probability of the face), and the like. The control unit 102 is notified of information related to the face area (face information).

  The moving object detection unit 115 detects a moving object from images of a plurality of digital image data continuous in time series output from the image processing unit 105, and calculates its position, range, and movement amount. When moving object detection is performed on three or more consecutive images, the moving object detection unit 115 uses the detection results obtained for the first two sheets to continue detection for the next two or more sheets. To do. Furthermore, the moving object detection unit 115 also detects the movement of the digital camera 100 using an angular velocity sensor.

  The control unit 102 determines a gain adjustment amount to be notified to the A / D conversion unit 104. Also, the control unit 102 determines whether to determine the subject region using only the subject detection result by the face detection unit 106 or to determine the subject region using the subject detection result by the moving body detection unit 115 according to the gain adjustment amount. Switch. Details of the switching control will be described later.

The image processing unit 105 performs various types of image processing on the digital image data output from the A / D conversion unit 104, and outputs processed digital image data.
Further, the EVF display unit 107 sequentially displays images according to the image data output from the image processing unit 105.

  The control unit 102 controls each unit so as to repeatedly execute the EVF display process as long as the signal SW1 is not notified from the operation unit 101 (that is, notification that the shutter switch is half-pressed).

  On the other hand, when the control unit 102 receives the notification of the signal SW1, AF and AE processes are performed using the latest captured image when the notification is received, and the in-focus position and the exposure condition are determined. Further, the control unit 102 determines whether or not the flash unit 111 needs to emit light. Whether or not to emit the flash may be determined in advance by using the operation unit 101 and may be determined by reading the setting data, or may be automatically determined by detecting the darkness of the surroundings. You may do it.

  The control unit 102 stands by until the notification of the signal SW1 is continued from the operation unit 101 and until the notification of the signal SW2 (that is, the notification of full pressing of the shutter switch) is received. If the notification of the signal SW1 is interrupted before receiving the notification of the signal SW2, the control unit 102 resumes the EVF display process.

  Upon receiving the notification of the signal SW2, the control unit 102 controls the EF processing unit 110 to cause the flash unit 111 to emit light, causes the flash unit 111 to perform pre-emission, calculates the light emission amount, and weights the EF frame. Process. Then, the control unit 102 instructs the EF processing unit 110 to cause the flash unit 111 to emit light at the main light emission amount calculated by the pre-light emission. When the flash unit 111 is not caused to emit light, the control unit 102 proceeds to the main photographing process without performing the above-described light control.

  In the main photographing process, the image pickup process during the EVF display process, the exposure mechanism 109a is controlled by the aperture and the shutter speed determined in the AE process, and the image pickup element 103 is exposed. Mainly different in many respects.

  In addition, since the actual shooting is a process of capturing an image for recording, the processed digital image data output from the image processing unit 105 is converted into a recording data file by the format conversion unit 112 and the image recording unit 113. Is recorded on the recording medium.

(Subject detection operation)
Next, the subject detection operation in the digital camera 100 of the present embodiment will be described using the flowchart shown in FIG.
First, the control unit 102 sets an initial exposure value for capturing an image for EVF display (S201). The initial exposure value can be set as appropriate within a range in which a predetermined EVF image frame rate can be realized.

  Then, in order to realize the set exposure value, the control unit 102 controls the aperture of the exposure mechanism 109a through the mechanical drive unit 109 to expose the image sensor (S202), and captures an EVF image at a cycle corresponding to the shutter speed. Reading from the element 103 (S203).

  Next, the control unit 102 determines whether or not the captured EVF image corresponds to a predetermined face detection cycle (S204). This face detection cycle can be determined as the number of frames of the EVF image for which face detection is continuously performed, for example, according to the processing capability of the face detection unit 106.

  When the captured EVF image corresponds to the face detection cycle, the control unit 102 disables the moving object detection unit 115 and enables the face detection unit 106 to execute face detection processing on the EVF image and acquire subject information (face information). (S205). If the EVF image does not correspond to the face detection cycle, face detection by the face detection unit 106 is not performed and the previously detected subject information is held.

  Next, the control unit 102 determines whether or not the captured EVF image corresponds to a predetermined moving object detection cycle (S206). This moving object detection cycle can be determined as the number of frames of the EVF image to be detected according to the processing capability of the moving object detection unit 115, for example. Note that the moving object detection period is shorter than the face detection period.

  When the captured EVF image corresponds to the moving object detection cycle, the control unit 102 disables the face detection unit 106, enables the moving object detection unit 115, performs moving object detection (subject tracking), and acquires the moving distance and direction of the subject. To do.

  The moving object detection unit 115 detects the movement of the subject between the EVF image (image b) that is the object of moving object detection and the past EVF image (image a) that is adjacent in time series. Specifically, the moving object detection unit 115 determines the moving object for the face area detected by the face detection unit 106 if the image a corresponds to the face detection cycle, and if the image a corresponds to the moving object detection cycle. Moving object detection is performed on the face area tracked by the detection.

  The moving object detection unit 115 searches the image b for an area most similar to the face area in the image a, and sets the object moving distance and moving direction as the object vector (Tx, Ty) from the difference between the center coordinates of the two areas. Calculate (S207). When moving object detection is performed on three or more consecutive images, the moving object detection unit 115 uses the detection results obtained for the first two sheets to continue detection for the next two or more sheets. To do. Since the moving object detection cycle is shorter than the face detection cycle, moving object detection is performed a plurality of times before the face detection result is updated. If a face area is detected by the face detection unit 106, tracking the subject area by moving object detection based on the face area can track the movement of the face without performing face detection during that time. It becomes possible.

  Here, as described above, since face detection is subject detection based on image recognition, it takes time due to the complexity of the calculation. However, since face features such as eyes and cheeks are extracted, detection accuracy is high. On the other hand, the moving object detection is a method of detecting a region having a high similarity to a region detected as a subject in the previous EVF image as a subject region in the current EVF image. Therefore, in moving object detection, it cannot be determined whether or not the subject area is a face area, and although it is inferior to face detection in terms of accuracy, face detection is not performed because the computational complexity is lower than face detection. This is useful as a process for tracking the subject in between.

  However, when the gain adjustment amount applied to the captured image in the A / D conversion unit 104 is increased in order to increase sensitivity, such as when the subject brightness is low, the noise component in the EVF image increases, and the accuracy of moving object detection increases. It will decrease.

  The reason will be described. Live images have a lower frame rate because they need to be displayed as moving images, and the maximum shutter speed is limited. For example, when the frame rate is 30 frames / second, the shutter speed cannot be slower than 1/30 seconds. Therefore, when the subject brightness is low, the live image is captured by increasing the sensitivity, specifically, by increasing the gain of the pixel signal. As the sensitivity (gain) increases, the random noise component in the captured image also increases.

FIG. 3 is a diagram schematically illustrating a change in image quality due to sensitivity and an influence of the change in image quality on the accuracy of moving object detection.
3A shows an image taken with normal sensitivity, FIG. 3B shows an image taken with higher sensitivity than FIG. 3A, and FIG. 3C shows FIG. The images captured with higher sensitivity than b) are schematically shown. A random noise component appears in FIG. 3B and further increases in FIG.

  For example, if the state of FIG. 3 (a) continues, even if a method of detecting a subject area first by face detection and then switching to moving object detection to track the subject area is possible, erroneous detection by moving object detection is possible. The nature is low. Note that the face frame 31 is an example of a mark that is superimposed and displayed on the live image in order to notify the user of the detected subject area (face area).

  However, for example, when face detection is performed in the state of FIG. 3A and the state is changed to the state of FIG. 3B after switching to moving body detection, the peripheral region of the original subject region (the region indicated by the dotted frame 32) There is a possibility of false detection. As a result, even if the subject does not move between images, the detected subject region starts to vary within the range indicated by the arrows.

  When the gain further increases and the state shown in FIG. 3C is reached, the random noise component increases, so that the range of variation due to erroneous detection becomes wider.

  As a result, a region that is not the subject region is erroneously detected as a subject region after movement. Moreover, in the moving object detection, it is not possible to confirm whether or not the detected area is a correct subject area (in this case, the face area) even if erroneously detected, so the erroneous detection was performed in the moving object detection performed until the next face detection cycle. There is also a possibility that the subject area continues to be tracked.

  Therefore, in this embodiment, the control unit 102 determines whether or not the gain adjustment amount applied to the captured image exceeds a predetermined threshold (S208). The gain adjustment amount may be a value corresponding to the sensitivity setting when capturing an EVF image. In this case, the threshold value may be a gain adjustment amount corresponding to, for example, ISO 1600. If the gain adjustment amount exceeds the threshold, the combined use of face detection and moving object detection is stopped, and the subject region is determined based on only face detection (S209). Noise components that are superimposed on the image due to an increase in gain adjustment amount are not preferable for face detection.However, in face detection, face-specific feature values such as eyes and cheeks are extracted. The probability of detection is much smaller compared to moving object detection.

  Specifically, in S209, the control unit 102 uses the face area detected by the most recent face detection as the subject area in the EVF image until the next face detection cycle. In this case, the moving object detection unit 115 performs the moving object detection until the gain adjustment amount falls below the threshold, but the detection result may not be used, or the moving object detection operation itself by the moving object detection unit 115 is performed. It may not be performed. In the latter case, the control unit 102 functions as a stopping unit.

  On the other hand, if the gain adjustment amount is equal to or smaller than the predetermined threshold value in S208, the control unit 102 uses the area tracked by the moving object detection based on the detection result of the face detection as the subject area.

  The control unit 102 can superimpose and display a frame (face frame 31) indicating the subject area on the EVF image in order to notify the user of the detected subject area. When the gain adjustment amount exceeds the threshold value, the frame display is performed using only the result of the face detection, so that the irregularity of the face frame 31 as shown by the dotted line in FIG. 3B or FIG. A stable frame display can be realized by avoiding movement.

FIG. 4 is a diagram schematically showing an example of frame display in the present embodiment, and FIGS. 4A to 4C are respectively the same EVF images as FIGS. 3A to 3C. It corresponds.
Assume that face detection is performed in the state of FIG. 4A, and then the gain adjustment amount increases, and in the state of FIG. 4B, the gain adjustment amount exceeds the threshold value. In this case, as shown in FIGS. 4B and 4C, the face frame 33 is displayed using the information of the subject area obtained by the face detection in FIG. 4A as it is. Therefore, it is possible to avoid the phenomenon that the display position of the face frame becomes unstable due to a decrease in moving object detection accuracy as shown in FIG. 3B or 3C.

  The gain adjustment amount threshold is an example, and may be another value. Further, the threshold value of the gain adjustment amount does not need to be a fixed value, and may be variable according to the base noise level of the image sensor 103 and other noise factors. For example, the higher the temperature of the CCD image sensor, the higher the noise level in the captured image, so the gain adjustment amount threshold is lowered.

  In S211, the control unit 102 determines whether or not the signal SW1 is notified. If the signal SW1 is not notified, the process returns to S202 and the EVF display process is continued.

  On the other hand, if the signal SW1 is notified in S211, the control unit 102 performs AE control in S212 and AF control in S213. At this time, if the subject area is detected, the control unit 102 can perform AE control so that the subject area is properly exposed, or perform AF control so that the subject area is focused. When the subject area is not detected, the control unit 102 can perform AE control and AE control based on a predetermined exposure control mode and focus detection area.

  Next, the control unit 102 determines whether or not the signal SW2 is notified (S214), and if notified, performs the main photographing process under the exposure conditions set by the AE control and the AF control (S216). On the other hand, if the signal SW2 is not notified, the control unit 102 confirms the state of the signal SW1 (S215). If the notification is canceled, the process returns to S202. If the notification continues, the process returns to S214. .

  As described above, the present embodiment is a gain applied to imaging data in an image processing apparatus capable of subject detection by image recognition and subject detection by tracking a subject area detected by image recognition by moving object detection. Monitor the amount of adjustment. When the gain adjustment amount exceeds a predetermined threshold, a subject area based only on the subject detection result by image recognition is used. When the gain adjustment amount is equal to or smaller than the threshold value, a subject area based on the result of subject detection by moving object detection is further used.

  When the gain adjustment amount exceeds the threshold value and there is a high possibility that the accuracy of moving object detection will decrease, only the result of subject detection by image recognition is used, so tracking performance is sacrificed somewhat, but erroneous detection of the subject region is prevented. It is possible. As a result, it is possible to avoid problems such as focusing control on a background deviated from the subject or incorrect exposure setting during actual photographing, and improving the accuracy of AE control and AF control.

  In addition, when displaying the subject area (frame display or the like), it is possible to prevent a change in the display position caused by erroneous detection of the subject area due to moving object detection, and to perform stable display.

  On the other hand, when the gain adjustment amount is equal to or smaller than the threshold value, the subject area based on the subject detection result by moving object detection is used in addition to the subject detection by image recognition, so that subject detection having both accuracy and followability can be realized. it can.

<Second Embodiment>
Next, a second embodiment of the present invention will be described. In the first embodiment, attention is paid to a noise component, which is one of the factors that reduce the accuracy of moving object detection, and an image is used to detect a subject area depending on whether the gain adjustment amount exceeds a predetermined threshold. It was switched between using recognition and moving object detection together or using only image recognition.

  In this embodiment, attention is paid to subject blur, which is another factor that reduces the accuracy of moving object detection, and image recognition is performed for subject region detection depending on whether the exposure time (shutter speed) is longer than a predetermined threshold. And whether to use motion detection together or only image recognition.

  When moving objects are imaged at a slow shutter speed, subject blurring tends to occur. For this reason, it is difficult to detect a moving object based on the correlation between images, and the destination of the face area detected by face detection may not be detected. On the other hand, the face detection takes time to calculate, but it is stronger than the moving object detection against the subject blur and can maintain the higher object detection accuracy than the moving object detection.

  FIG. 6 is a diagram for schematically explaining the problem of moving object detection when the shutter speed is slow. In FIG. 6, it is assumed that the subject (person) moves at a speed at which the shutter speed of the EVF image is blurred. For example, the face detection is successful in FIG. In this case, even when switching from FIG. 6 (b) to moving object detection, the position of the face area detected in FIG. 6 (a) is correctly detected based on the correlation between the images in FIG. 6 (b). I can't do it. As a result, the subject area cannot be tracked correctly. In addition, since it is impossible to determine whether the detected area is a face or not by moving object detection, the area that is not a face is continuously tracked in subsequent moving object detection. (FIGS. 6C to 6E). As a result, the display position of the face frame 31 remains shifted from the subject.

  FIG. 5 is a flowchart for explaining the operation of the digital camera according to the present embodiment. The same operation steps as those in FIG. 2 are denoted by the same reference numerals, and redundant description is omitted. Further, since the digital camera of the present embodiment may have the same configuration as the digital camera 100 described with reference to FIG. 1 in the first embodiment, the following description is also given using the components shown in FIG. To do.

  As is clear from the comparison between FIG. 5 and FIG. 2, except for the determination processing in S508, the same processing as in the first embodiment may be performed, and therefore only the determination processing in S508 will be described. As described above, in the present embodiment, when the exposure time (shutter speed) is longer (slower) than a predetermined threshold (for example, 1/30 second), the detection of the subject area by the combined use of face detection and moving object detection, Switch to subject area detection by face detection only.

  FIG. 7 is a diagram schematically illustrating the effect of the present embodiment. In FIG. 7A, when it is determined that the exposure time is longer than the threshold, setting the subject area based only on the face detection sacrifices the tracking ability somewhat (FIG. 7D). Since the subject can be detected correctly when the face is detected (FIG. 7E), the face frame 31 does not deviate from the subject. On the other hand, when the moving object detection is used together, as shown in FIGS. 7B and 7C, the face frame 31 is displaced from the subject.

  As described above, the present embodiment can achieve the same effects as those of the first embodiment. Note that the threshold value of the exposure time in this embodiment does not have to be a fixed value. For example, the motion vector of the subject may be calculated from between the subject regions obtained in the face detection cycle, and the threshold may be set shorter as the subject moving speed is faster (the motion vector is larger).

<Other embodiments>
It should be noted that the first embodiment and this embodiment can be implemented in combination. That is, when the gain adjustment amount exceeds the threshold value or the exposure time is longer than the threshold value, it is possible to switch from the detection of the subject area by the combined use of face detection and moving object detection to the detection of the subject area by only face detection. .

The above-described embodiment can also be realized in software by a computer of a system or apparatus (or CPU, MPU, etc.).
Therefore, the computer program itself supplied to the computer in order to implement the above-described embodiment by the computer also realizes the present invention. That is, the computer program itself for realizing the functions of the above-described embodiments is also one aspect of the present invention.

  The computer program for realizing the above-described embodiment may be in any form as long as it can be read by a computer. For example, it can be composed of object code, a program executed by an interpreter, script data supplied to the OS, but is not limited thereto.

  A computer program for realizing the above-described embodiment is supplied to a computer via a storage medium or wired / wireless communication. Examples of the storage medium for supplying the program include a magnetic storage medium such as a flexible disk, a hard disk, and a magnetic tape, an optical / magneto-optical storage medium such as an MO, CD, and DVD, and a nonvolatile semiconductor memory.

  As a computer program supply method using wired / wireless communication, there is a method of using a server on a computer network. In this case, a data file (program file) that can be a computer program forming the present invention is stored in the server. The program file may be an executable format or a source code.

Then, the program file is supplied by downloading to a client computer that has accessed the server. In this case, the program file can be divided into a plurality of segment files, and the segment files can be distributed and arranged on different servers.
That is, a server apparatus that provides a client computer with a program file for realizing the above-described embodiment is also one aspect of the present invention.

  In addition, a storage medium in which the computer program for realizing the above-described embodiment is encrypted and distributed is distributed, and key information for decrypting is supplied to a user who satisfies a predetermined condition, and the user's computer Installation may be allowed. The key information can be supplied by being downloaded from a homepage via the Internet, for example.

Further, the computer program for realizing the above-described embodiment may use an OS function already running on the computer.
Further, a part of the computer program for realizing the above-described embodiment may be configured by firmware such as an expansion board attached to the computer, or may be executed by a CPU provided in the expansion board. Good.

Claims (5)

First detecting means for detecting a predetermined subject existing in the image of the frame by detecting a part having a predetermined feature from the image of the frame of the moving image;
A second detection unit that tracks a region where the predetermined subject detected by the first detection unit exists in the moving image by searching for a similar region between the images of the frame;
Determining means for determining an area where the predetermined subject exists in the moving image based on at least one of a detection result by the first detection means and a detection result by the second detection means;
When the predetermined means satisfies the predetermined condition, the area where the predetermined subject exists is based on the detection result by the first detection means instead of the detection result by the second detection means. If the predetermined condition is not satisfied, an area where the predetermined subject exists is determined based on the detection result by the first detection means and the detection result by the second detection means. To do,
The predetermined condition is that at least one of a gain adjustment amount applied to the moving image exceeds a predetermined threshold and an exposure time for an image of one frame is longer than a predetermined threshold. An image processing apparatus comprising:   The image processing apparatus according to claim 1, wherein the gain adjustment amount is a value that increases as an imaging sensitivity setting when the image is captured is high.   The image processing apparatus according to claim 1, wherein the first detection unit detects a human face as the predetermined subject. A first detection step in which a first detection unit detects a predetermined subject existing in the image of the frame by detecting a part having a predetermined characteristic from the image of the frame of the moving image;
The second detection means searches for a similar region between the images of the frame, thereby tracking a region where the predetermined subject detected in the first detection step is present in the moving image. Detection process of
A determining step for determining a region in the moving image where the predetermined subject is present based on at least one of the detection result in the first detection step and the detection result in the second detection step; Have
In the determination step, when the predetermined condition is satisfied, the determination means does not depend on the detection result in the second detection step, but based on the detection result in the first detection step, the predetermined subject. If the predetermined condition is not satisfied and the predetermined condition is not satisfied, the predetermined subject exists based on the detection result in the first detection step and the detection result in the second detection step. Which determines the area to be
The predetermined condition is that at least one of a gain adjustment amount applied to the moving image exceeds a predetermined threshold and an exposure time for an image of one frame is longer than a predetermined threshold. An image processing apparatus control method comprising:   The program for functioning a computer as each means of the image processing apparatus of any one of Claim 1 thru | or 3.