JP2013042514A - Target image detection device, control method therefor, control program, recording medium with the program recorded thereon, and electronic apparatus provided with target image detection device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To quickly output a result of detection of a face image.SOLUTION: A control part 20 of a digital camera 10 is a target image detection device for detecting a face image from within a photographic image. The control part 20 comprises: an image acquisition part 22 for acquiring the photographic image; a condition determination part 23 for determining, among a plurality of detection conditions for detecting a face image, a detection condition different from a previous detection condition; a face detection part 24 for detecting a face image in accordance with the detection condition determined by the condition determination part 23 for the photographic image acquired by the image acquisition part 22; and a position information output part 26 for outputting position information on the detected face image to an image composition part 17.

Description

  The present invention relates to a target image detection device that detects a target image from an original image, a control method thereof, a control program, a recording medium that records the program, and an electronic apparatus that includes the target image detection device.

  2. Description of the Related Art Conventionally, there is an increasing need for automatically recognizing a shooting target or a display target in order to provide an easy-to-use user interface or automatically acquire an optimal image in an imaging device, a display device, or the like. As an example of such a need, a human face is detected and adjusted so that the detected face has an optimum focus and exposure, or a character is recognized, and the recognized character area is automatically OCR (Optical Character). Recognition).

  In general, in order to realize such a function, advanced image processing is required, and a large amount of calculation is required. Moreover, since the above functions generally require real-time properties, high-speed processing is required. For this reason, it is often difficult to realize the above functions in an embedded device whose processing capability, power consumption, and cost are restricted, such as a digital camera or a portable communication terminal.

  Various methods for speeding up the processing have been proposed for this problem. For example, in the image processing apparatus described in Patent Document 1, the reduced images are sequentially template-matched in the order in which the resolution sequentially changes to detect a region where a specific subject is photographed, and the processing is interrupted after detection As a result, the processing speed is increased. Further, in the face detection method described in Patent Document 2, detection is performed in order from the largest face, and when a face is detected, the area where the face has already been detected is excluded from the detection target area and the detection is continued. Thereby, since the detection target area can be reduced every time a face is detected, the processing can be accelerated.

JP 2007-135115 A (published May 31, 2007) JP 2006-301847 A (published on November 2, 2006)

  The image processing apparatus of Patent Document 1 is for following the movement of a specific subject. For this reason, in the process of detecting an area where an unspecified subject whose number has not been determined is detected, it is difficult to interrupt the process after detection, and it is difficult to increase the speed of the process. Further, in the face detection method of Patent Document 2, since the face detection for the detection target region is repeated in order from the large face to the small face, the processing still takes time.

  The present invention has been made in view of the above-described problems, and an object of the present invention is to provide a target image detection device that can quickly output a detection result.

  A target image detection device according to the present invention is a target image detection device that detects a target image from an original image, and an acquisition unit that acquires the original image and a plurality of detection conditions for detecting the target image Among the first detection condition different from the previous detection condition, and the target image under the first detection condition determined by the determination means from the first original image acquired by the acquisition means. Detecting means for detecting the image, and output means for outputting information that can identify the target image detected by the detecting means to a second original image following the first original image, wherein the determining means comprises: And a second detection condition different from the first detection condition based on a history of association between the detection condition and the detection result indicating whether or not the detection means has detected the target image. Further determined That.

  The target image detection device control method according to the present invention is a target image detection device control method for detecting a target image from an original image, the acquisition step for acquiring the original image, and the target image Of the plurality of detection conditions for detection, the determination step determines the first detection condition different from the previous detection condition and the first original image acquired in the acquisition step. A detection step for detecting the target image under the first detection condition, and an output for outputting information capable of specifying the target image detected in the detection step to a second original image following the first original image A correspondence between a certain detection condition and a detection result indicating whether or not the target image is detected under the detection condition in the detection step. Based on only the history is characterized by further determining a different second detection condition from the first detection condition.

  In the target image detection device according to the present invention, the detection unit further detects the target image under a second detection condition determined by the determination unit from the second original image, and the output unit includes the detection unit. The information that can identify the target image detected by may be further output to a third original image that follows the second original image.

  The target image detection device according to the present invention may further include a tracking unit that tracks the target image detected by the detection condition with an original image following the original image including the target image.

  In the target image detection device according to the present invention, the tracking unit tracks the target image with the second original image under a tracking condition included in the first detection condition, and the output unit includes the tracking unit. Information that can identify the target image tracked by the means may be further output to a third original image following the second original image.

  The target image detection device according to the present invention further includes correction means for correcting the tracking result tracked by the tracking means based on the association history, and the output means outputs the tracking result corrected by the correction means. The target image tracked by the tracking unit may be further output as information that can be specified to a third original image following the second original image.

  In the target image detection device according to the present invention, the correction unit may correct the tracking result tracked by the tracking unit with an average of numerical values included in the detection condition associated with the detection result.

  In the target image detection device according to the present invention, when the number of detection results indicating that the target image has been detected decreases by changing the detection condition, the correction unit prioritizes the detection condition before the decrease. In this case, the tracking result tracked by the tracking unit may be corrected.

  In the target image detection device according to the present invention, the determining unit changes the use frequency of the detection condition according to the number of times of detection by the detection condition so far, and the plurality of detections based on the changed use frequency. Of the conditions, a detection condition different from the previous detection condition may be determined.

  In the target image detection device according to the present invention, the determination unit may detect a detection condition different from the previous detection condition among the plurality of detection conditions according to a combination of the plurality of detection conditions for detecting the target image. You may decide to.

  In the target image detection device according to the present invention, the determination unit changes the order of the plurality of detection conditions for detecting the target image based on a predetermined regularity, and thereby among the plurality of detection conditions, The detection condition may be determined differently from the previous detection condition.

  The target image detection device according to the present invention includes a result history storage unit that stores a history of correspondence between a detection condition and a detection result indicating whether the detection unit has detected the target image under the detection condition. Further, it may be provided.

  In the target image detection device according to the present invention, the determination unit may sequentially determine a detection condition from a plurality of detection conditions for detecting the target image.

  In the target image detection device according to the present invention, the acquisition unit may acquire a series of still images included in a moving image as the original image.

  The electronic apparatus according to the present invention may include the target image detection device.

  The electronic apparatus according to the present invention further includes: a display unit that displays information; and an image combining unit that combines the original image with an image indicating the detection result detected by the target image detection device and displays the image on the display unit. You may prepare.

  Each unit in the target image detection device can be executed on a computer by a control program. Furthermore, the control program can be executed on any computer by storing the control program in a computer-readable recording medium.

  As described above, when the target image detection device according to the present invention acquires the original image, the source image acquired under a detection condition different from the previous detection condition among the plurality of detection conditions for detecting the target image. Since the target image is detected from the image and the detection result is output, the detection result can be output in a short time compared to the conventional technique, but the detection result can be output in a short time. There is an effect that other devices can quickly use the detected result.

It is a block diagram which shows schematic structure of the digital camera which is one Embodiment of this invention. It is a figure which shows the example of the detection conditions of face detection. It is a flowchart which shows the processing operation | movement of the face detection process in the said digital camera, and a face tracking process. It is a figure which shows the example of the synthesized image displayed in the said digital camera, and the comparative example of the synthesized image displayed by a prior art in time series. It is a block diagram which shows schematic structure of the digital camera which is other embodiment of this invention. It is a flowchart which shows the processing operation | movement of the face detection process in the said digital camera, and a face tracking process. It is a figure for demonstrating an example which correct | amends the tracking result of the face tracking in the said digital camera.

[Embodiment 1]
An embodiment of the present invention will be described with reference to FIGS. FIG. 1 shows a schematic configuration of a digital camera according to the present embodiment. As shown in the figure, a digital camera (electronic device) 10 is photographed by a photographing unit 11 and a photographed image (original image) subjected to image processing by an image processing unit 12 is displayed on a display unit 13 as a live view image. . In addition, when the user presses a shutter button (not shown) of the operation unit 14, the digital camera 10, after the captured image subjected to image processing by the image processing unit 12 is compressed by the image compression unit 15, It is stored in the image recording unit 16.

  The digital camera 10 according to the present embodiment first acquires a captured image that has been subjected to image processing by the image processing unit 12, and among the plurality of detection conditions for detecting a face image (target image), Change to another detection condition. Next, a face image is detected from the acquired captured image under the changed detection condition. Then, a frame image (information that can specify the target image) surrounding the area of the detected face image is generated, and the generated frame image is combined with the captured image by the image combining unit (image combining unit) 17. And displayed on the display unit 13 as a live view image.

  As a result, the detection result is output in a short time, although it is a detection result of a part of the face image as compared with the conventional technique. As a result, the frame image can be quickly displayed using the output detection result. In addition, a face image is detected under still another detection condition for the subsequent photographed image, and another detection result is output in a short time. Therefore, when the captured image is a moving image, the number of frame images gradually increases, so that the display state of the frame images becomes smoother than in the conventional technique.

  Next, details of the digital camera 10 of this embodiment will be described. As described above, the digital camera 10 includes the photographing unit 11, the image processing unit 12, the display unit 13, the operation unit 14, the image compression unit 15, the image recording unit 16, and the image composition unit 17. Furthermore, as shown in FIG. 1, the digital camera 10 includes a control unit (target image detection device) 20 and a storage unit 21.

  The photographing unit 11 shoots a subject, and includes, for example, an optical system such as a lens group, a diaphragm, and an image sensor, and a circuit system such as an amplifier and an A / D converter. Examples of the image sensor include a CCD and a complementary metal-oxide semiconductor (CMOS) image sensor. The photographing unit 11 generates a photographed image by the photographing described above, converts it into photographed image data, and transmits it to the image processing unit 12.

  The image processing unit 12 performs image processing such as matrix calculation, γ correction, and white balance adjustment on the captured image data from the imaging unit 11. Further, the image processing unit 12 creates display image data, recording image data, and face detection image data from the image-processed image data, and the display unit 13, the image compression unit 15, and the control unit, respectively. To the unit 20. Note that the data of the captured images for display and face detection may be the same.

  The image synthesis unit 17 synthesizes an image based on an instruction from the control unit 20 with the captured image from the image processing unit 12. The image composition unit 17 transmits composite image data, which is a composite image, to the display unit 13.

  In the present embodiment, the image composition unit 17 receives position information of the face image detected by the control unit 20, and based on the received position information, a frame image such as a rectangular image or a circular image that appropriately surrounds the face image. Is generated. Further, the image composition unit 17 receives the position information of the face image tracked by the control unit 20, and generates a frame image that appropriately surrounds the face image based on the received position information. Then, the image composition unit 17 synthesizes the generated frame image with the captured image from the image processing unit 12. The synthesized image is transmitted to the display unit 13 and displayed on the display unit 13.

  The frame image related to the face image detected by the control unit 20 and the frame image related to the face image tracked by the control unit 20 may be the same or different. Further, the frame image may be a graphic that can indicate that the face image is included inside, and for example, may be an image that displays only the vicinity of the vertex of a rectangle.

  The display unit 13 includes a display device such as an LCD (Liquid Crystal Display Element), a CRT (Cathode Ray Tube), or a plasma display. The display unit 13 displays and outputs various information such as characters and images based on the composite image data from the image composition unit 17.

  The operation unit 14 receives various inputs from the user through user operations, and includes an input button, a touch panel, and other input devices. The operation unit 14 converts information operated by the user into operation data and transmits the operation data to the control unit 20. Note that other examples of the input device include a keyboard, a numeric keypad, and a pointing device such as a mouse.

  The image compression unit 15 compresses captured image data from the image processing unit 12 according to a method such as JPEG or MPEG. The image compression unit 15 records the compressed captured image data in the image recording unit 16.

  The image recording unit 16 records information, and particularly records data of a captured image compressed by the image compression unit 15. The image recording unit 16 is configured by a nonvolatile storage device such as a flash memory or a hard disk.

  The control unit 20 comprehensively controls operations of various configurations in the digital camera 10. The control unit 20 is configured by a computer including a CPU (Central Processing Unit) and a memory, for example. And operation control of various composition is performed by making a computer run a control program. This program may be in a form that reads and uses what is recorded on a removable medium such as a flash memory, or may be in a form that reads and uses what is installed in a hard disk or the like. In addition, a form in which the above program is downloaded and installed on a hard disk or the like is also conceivable. Details of the control unit 20 will be described later.

  The storage unit 21 includes a nonvolatile storage device such as a flash memory and a ROM (Read Only Memory), and a volatile storage device such as a RAM (Random Access Memory). Examples of the contents stored in the nonvolatile storage device include the above-described control program, OS (operating system) program, other various programs, operation setting values in the image processing unit 12, input character data, and the like. Examples of the operation setting values in the image processing unit 12 include white balance values set at the time of shipment of the apparatus and maintenance, various parameter values related to image processing when adjusting the brightness and darkness of a captured image, and the like. On the other hand, examples of the contents stored in the volatile storage device include a working file and a temporary file. Details of the storage unit 21 will be described later.

  In the present embodiment, the control unit 20 detects a face image from a captured image, and detects the detected face image from subsequent captured images, with respect to a moving image composed of a plurality of captured images in time series. By doing so, it has a face tracking function for tracking the face image. It should be noted that the face image is detected by using a known face detection technique such as template matching, skin color area detection, face outline detection, or facial feature point detection. be able to.

  FIG. 1 shows a configuration related to the face detection function and the face tracking function in the control unit 20 and the storage unit 21. As illustrated, the control unit 20 includes an image acquisition unit (acquisition unit) 22, a condition determination unit (determination unit) 23, a face detection unit (detection unit) 24, a face tracking unit (tracking unit) 25, and position information output. The unit (output means) 26 is provided. The storage unit 21 includes a condition storage unit 27 and a result history storage unit 28.

  The condition storage unit 27 stores in advance a plurality of detection conditions for detecting the face image (hereinafter referred to as “face detection detection conditions”). FIG. 2 shows an example of detection conditions for the face detection. Note that the illustrated arrows indicate the scan direction of the template. (A) of the figure is a template used in the template matching and shows a plurality of templates having different sizes. Moreover, (b) of the figure has shown the some template which performed the rotation (RIP (Rotation In Plane)) in the image with respect to the said template. Moreover, (c) of the figure has shown the some template which changed the observation direction (ROP (Rotation Off Plane)) with respect to the said template.

  FIG. 2D shows a plurality of search ranges in which a search is performed using the template in a captured image that is a face detection target. Further, (e) in the figure shows a plurality of templates having different feature amounts. Other detection conditions include a plurality of image filters for favorably performing face detection, a plurality of similarity calculation methods for calculating a similarity with a template, and the like.

  By the way, it is considered that a person does not move so much in a very short time such as one or several frames. Therefore, a face image detected in a certain captured image can be detected in the subsequent captured image in the vicinity of the position of the previously detected face image under the same or similar detection condition as the previous detection condition. is expected. Accordingly, the tracking condition for tracking the face image is included in the detection condition for detecting the face image.

  The result history storage unit 28 stores the detection results and detection conditions of the face detection and the tracking results and detection conditions of the face tracking as a result history. Examples of the detection result and the tracking result include detection date / time, size, and position information of the detected face image.

  The image acquisition unit 22 acquires captured image data from the image processing unit 12. When the image acquisition unit 22 acquires the captured image data, the image acquisition unit 22 notifies the condition determination unit 23 of the fact and sends the acquired captured image data to the face detection unit 24 and the face tracking unit 25.

  When the condition determination unit 23 receives the notification from the image acquisition unit 22, the condition determination unit 23 refers to the detection condition in the result history storage unit 28, and detects from the plurality of detection conditions in the condition storage unit 27 that is different from the previous detection condition. The condition is determined. In the present embodiment, the condition determining unit 23 sequentially determines a plurality of detection conditions necessary for detecting all face images from the captured image. The condition determination unit 23 sends the determined detection condition to the face detection unit 24.

  In addition, when receiving the notification from the image acquisition unit 22, the condition determination unit 23 determines the tracking condition based on the detection history and tracking history of the face image in the result history storage unit 28. The condition determining unit 23 sends the determined tracking condition to the face tracking unit 25. The condition determination unit 23 may use a plurality of detection conditions in the condition storage unit 27 when determining the tracking condition.

  When the face detection unit 24 receives the captured image data from the image acquisition unit 22, the face detection unit 24 detects the face image from the captured image based on the detection condition from the condition determination unit 23. The face detection unit 24 sends the detection result to the position information output unit 26 and accumulates the detection result and the detection condition in the result history storage unit 28. Note that the specific processing of face detection performed by the face detection unit 24 is the same as the conventional processing, and thus the description thereof is omitted.

  Upon receiving the captured image data from the image acquisition unit 22, the face tracking unit 25 tracks the face image from the captured image based on the tracking condition from the condition determining unit 23. Specifically, the face tracking unit 25 detects the previously detected face image from the captured image based on the detection condition suitable for the face tracking from the condition determining unit 23. The face tracking unit 25 sends the tracking result to the position information output unit 26 and accumulates the tracking result and the tracking condition in the result history storage unit 28. In addition, since the specific process of the face tracking which the face tracking part 25 performs is the same as that of the past, the description is abbreviate | omitted.

  The position information output unit 26 outputs position information of the detected face image to the image composition unit 17 based on the detection result from the face detection unit 24. The position information output unit 26 outputs position information of the tracked face image to the image composition unit 17 based on the tracking result from the face tracking unit 25.

  As the position information of the face image output by the position information output unit 26, any information can be used as long as it is information that can specify the area of the face image in the captured image. Examples of the position information of the face image include a combination of center coordinates and width in the face image area, and a combination of coordinates of four vertices of a rectangle indicating the boundary of the face image area.

  FIG. 3 shows processing operations of face detection processing and face tracking processing in the digital camera 10 having the above-described configuration. As shown in the figure, when the image acquisition unit 22 acquires one captured image in the moving image captured by the imaging unit 11, the condition determination unit 23 includes a detection history of a face image as a result history in the result history storage unit 28, and Referring to the tracking history, it is determined whether or not the detected face image (detected target) exists (step S10. Hereinafter, it may be simply referred to as “S10.” The same applies to other steps. .) If not, the process proceeds to step S15.

  When the detected target exists, the condition determination unit 23 determines the tracking condition from the result history (S11), and the face tracking unit 25 tracks the detected face image with the determined tracking condition ( S12). Next, the face tracking unit 25 stores the tracking result and the tracking condition in the result history storage unit 28, while the position information output unit 26 outputs the position information of the face image based on the tracking result to the image composition unit 17. (S13). Accordingly, the image composition unit 17 generates a frame image related to the face image tracked by the face tracking unit 25, combines the generated frame image and the captured image, and displays and outputs the frame image via the display unit 13.

  Next, the condition determination unit 23 determines whether or not all the detected objects have been processed (S14). When all the processes are not performed, the process returns to step S11 and the above processing operation is repeated. On the other hand, if all processing is performed, the process proceeds to step S15.

  In step S15, the condition determination unit 23 refers to the detection condition in the result history storage unit 28 and changes the detection condition from the previous detection condition. Next, the face detection unit 24 detects a face image under the changed detection condition (S16). The face detection unit 24 stores the detection result and the detection condition in the result history storage unit 28, while the position information output unit 26 outputs the position information of the face image based on the detection result to the image composition unit 17 ( S17). As a result, the image composition unit 17 generates a frame image related to the face image detected by the face detection unit 24, combines the generated frame image and the captured image, and outputs the frame image via the display unit 13. Thereafter, the processing operation is terminated.

  FIG. 4 shows an example of the composite image. (A) of the figure has shown the example of the synthesized image displayed in the digital camera 10 of this embodiment in time series. Moreover, (b) of the figure is a comparative example, and shows an example of a composite image displayed in a conventional digital camera in time series. In the illustrated example, the number of detection conditions for performing all face detection is five.

  In the present embodiment, as shown in FIG. 4A, face detection is performed on the first captured image a1 under the first detection condition, and a rectangular frame image b1 surrounding the detected face image. Is combined with the second photographed image a2 and displayed. Next, face detection is performed on the second captured image a2 under a second detection condition that is different from the first detection condition, and a rectangular frame image b2 surrounding the detected face image is captured at the third position. It is synthesized and displayed on the image a3. Further, face tracking is performed on the second captured image a2 under tracking conditions suitable for the face image surrounded by the frame image b1, and a rectangular frame image c1 surrounding the tracked face image is captured third. It is synthesized and displayed on the image a3.

  Next, face detection is performed on the third captured image a3 under a third detection condition different from the first and second detection conditions, and four rectangular frame images b3 surrounding the detected face image are obtained. The second captured image a4 is combined and displayed. Further, face tracking is performed on the third photographed image a3 under tracking conditions suitable for each of the two face images surrounded by the frame images c1 and b2, and a rectangular frame image surrounding the tracked face image. c1 and c2 are combined with the fourth captured image a4 and displayed.

  Next, face detection is performed on the fourth captured image a4 under a fourth detection condition different from the first to third detection conditions, and a rectangular frame image b4 surrounding the detected face image is 5 The second captured image a5 is combined and displayed. Also, face tracking is performed on the fourth captured image a4 under tracking conditions suitable for each of the three face images surrounded by the frame images c1, c2, and b3, and a rectangular shape surrounding the tracked face image is obtained. The frame images c1 to c3 are combined with the fifth captured image a5 and displayed.

  Hereinafter, by repeating this, a rectangular frame image b5 surrounding the face image detected in the sixth captured image a6, the fifth captured image a5, and the face tracked in the fifth captured image a5 The rectangular frame images c1 to c4 surrounding the image are combined and displayed. In other words, frame images c1 to c4 and b5 obtained as a result of face detection under all detection conditions are displayed in the sixth composite image.

  On the other hand, conventionally, as shown in FIG. 4B, face detection is sequentially performed on the first captured image a1 under the first to fifth detection conditions, and the detection is performed after the face detection is completed. The rectangular frame images d1 to d5 surrounding all the face images are combined with the captured image and displayed. For this reason, it takes time until the frame images d1 to d5 are displayed. In the illustrated example, frame images d1 to d5 obtained by face detection for the first captured image a1 are combined with the sixth captured image a6 and displayed.

  In FIG. 4, the position of each subject in the captured image is the same between the captured images a1 to a6. However, since the subject actually moves, it is often different between the captured images a1 to a6. At the same time, the deviation often increases.

  Comparing (a) and (b) of FIG. 4, the frame image showing the results of face detection and face tracking is started more quickly than the conventional case, although it is a part. It can be understood that the time for the user to wait for the detection of the face image can be shortened. In addition, since the number of frame images gradually increases, the display state of the frame images becomes smoother than in the conventional technique. Further, since frame images indicating the remaining results of face detection and face tracking are sequentially displayed, it can be understood that new entertainment can be provided to the user.

  In the present embodiment, as shown in FIG. 4A, face detection under the second detection condition is performed on the second captured image a2, and the frame image b2 indicating the detection result is the next. It is combined with the (third) captured image a3. On the other hand, in the prior art, as shown in FIG. 4B, face detection under the second detection condition is performed on the first captured image a1, and the frame image d2 indicating the detection result is the sixth. Is combined with the captured image a6.

  Therefore, in the present embodiment, the period between the shooting time of the shot image that is the target of face detection and the shooting time of the shot image in which the frame image indicating the detection result is combined is shorter than in the conventional case. It can be understood that the deviation from the face image in the photographed image synthesized with the frame image becomes small.

  Further, in the present embodiment, as shown in FIG. 4A, among the frame images combined with the sixth photographed image a6, the frame images c1 to c4 are once for the fifth photographed image. The frame image b5 relates to the face image detected with respect to the fifth photographed image. That is, in the present embodiment, the frame image combined with the sixth captured image a6 relates to the face image in the immediately preceding (fifth) captured image a5. On the other hand, in the prior art, as shown in FIG. 4B, all of the frame images d1 to d5 combined with the sixth captured image a6 relate to the face image detected with respect to the first captured image a1. Is.

  Therefore, the frame image has a shorter period between the shooting time of the shot image from which the frame image is generated and the shooting time of the shot image in which the frame image is combined than in the conventional case. It can be understood that the deviation from the face image in the captured image in which the frame image is synthesized is further reduced.

[Embodiment 2]
Next, another embodiment of the present invention will be described with reference to FIGS. FIG. 5 shows a schematic configuration of the digital camera according to the present embodiment. The digital camera 10 shown in FIG. 1 is different from the digital camera 10 shown in FIG. 1 in that the function of the condition determination unit 30 is different from that of the face detection unit 24 and the face tracking unit 25 to the position information output unit 26. The difference is that a correction unit (correction means) 31 is added, and the other configurations are the same. In addition, the same code | symbol is attached | subjected to the structure and processing operation similar to the structure and processing operation demonstrated in the said embodiment, and the description is abbreviate | omitted.

  The condition determination unit 23 shown in FIG. 1 sequentially determines a plurality of detection conditions necessary for detecting all face images. In this case, by acquiring a plurality of captured images included in the moving image, it is possible to output the entire detection result of the face image for all of the plurality of detection conditions as in the conventional technique.

  On the other hand, the condition determination unit 30 of the present embodiment dynamically changes and determines the detection condition based on the detection result history in the result history storage unit 28. Information on conditions to be dynamically changed is stored in the condition storage unit 27.

  For example, when face detection using a sideways face image template is not performed in a predetermined period as shown in the left image of FIG. 2C, the detection frequency under the detection condition using the template may be reduced. It is done. This is because in the above case, it is considered that the face of the subject is facing the direction of the digital camera 10.

  In addition, when a face image is suddenly no longer detected from the region of the captured image from which the face image has been detected, the detection frequency under the detection condition of the face image can be increased. In the above case, it is considered that the subject is out of the frame from the digital camera 10 or the subject is turned back, so that the face of the subject is facing in the opposite direction to the digital camera 10, and the face image is again displayed. This is because the possibility of being detected is high.

  Thus, the efficiency of face detection can be improved by dynamically changing the detection condition based on the history of detection results.

  Below, methods for dynamically changing the detection conditions based on the history of the detection results are collectively described. The first method is to change the use frequency of the detection condition according to the number of times of detection by the detection condition so far. For example, when the number of times of detection in a certain area of a captured image is large, the frequency of using the area as a detection condition can be increased. In addition, when the number of detections of a face orientation is small, the frequency of using the orientation template as a detection condition may be reduced.

  The second method is to change the use frequency and / or combination of detection conditions in accordance with the combination of detection conditions. For example, if the number of times that a face image is detected simultaneously in the lower right area and the upper left area in the captured image is small, detecting the face image in one area reduces the frequency of using the other area as a detection condition. Is mentioned. In addition, when the possibility of simultaneously detecting the face image in the first direction and the face image in the second direction rotated 180 ° within the image from the first direction is low, the face image in the first direction is detected. Then, the frequency of using the second direction as a detection condition can be reduced.

  In addition, for example, when there are many face images of 30 pixels in the center area of the photographed image and many face images of 20 pixels in the peripheral area, the size of the detected face image tends to vary depending on the area in the photographed image. Is present, the frequency of using the size of the face image as a detection condition may be changed according to the region.

  The third method is to change the order of detection conditions based on the regularity when regularity exists in the order of detection conditions. For example, when the size of the face image once detected tends to decrease with time, the face image size may be used as a detection condition in order from the largest to the smallest. In addition, if it is predicted that the face image once detected is lost again in a while, it is possible to increase the frequency of using the detection condition for a while.

  The result correction unit 31 corrects the tracking result from the face tracking unit 25 based on the detection result from the face detection unit 24. The result correction unit 31 sends the corrected detection result and tracking result to the position information output unit 26.

  Conventionally, a face image is detected in order for each of a plurality of detection conditions required to detect all face images from a captured image, and the detection result is corrected based on the plurality of detection results, The corrected detection result was output. For example, when both a face image having a size of 20 pixels and a face image having a size of 30 pixels are detected for a certain area of the photographed image, the face image having an average value of 25 pixels as a size. It is mentioned to correct to.

  In contrast, in the present embodiment, when a captured image is acquired, some of the plurality of detection conditions are determined, a face image is detected under the determined detection conditions, and a detection result is output. For this reason, the accuracy of the detection result is reduced as compared with the conventional case in which the detection result is corrected based on the detection result of all of the plurality of detection conditions.

  Therefore, in the present embodiment, the result correction unit 31 corrects the tracking result from the face tracking unit 25 based on the detection result from the face detection unit 24 as described above. The result correction unit 31 may further correct the tracking result from the face tracking unit 25 based on the result history in the result history storage unit 28.

  FIG. 7 illustrates an example of tracking result correction. (A) of the figure shows a composite image at a certain time point. As shown in the figure, the composite image is a frame corresponding to a 30-pixel face image detected by face detection with respect to a photographed image e1 at a certain time point and a photographed image (not shown) immediately before that time point. Image f1. When the face detection unit 24 performs face detection on the captured image e1 shown in (a) of the figure using a 20-pixel template, as shown by a broken line in (b) of the figure, a face of 20 pixels is obtained. Assume that two images g1 and g2 are detected. At this time, the face image of 30 pixels surrounded by the frame image f1 and the face image g1 of 20 pixels are detected in the same region in the photographed image, and thus are considered to be the same face.

  Therefore, a 30-pixel face image (not shown) tracked under a tracking condition suitable for the face image surrounded by the frame image f1 with respect to the photographed image e1 shown in FIG. Is corrected to a face image of 25 pixels, which is the average of 30 pixels and 20 pixels, and the correction result is sent to the position information output unit 26. Note that the face image g2 detected for the first time in a region other than the region is sent to the position information output unit 26 without being corrected.

  At this time, a composite image displayed on the display unit 13 is shown in FIG. As shown in the figure, the component image is tracked and corrected by the following captured image e2 and frame image f2 corresponding to the 20-pixel face image detected by face detection with respect to the captured image e1, and face tracking. And a frame image h1 corresponding to a 25-pixel face image. Comparing (a) and (c) in the figure, it can be understood that the frame image h1 after correction is closer in size to the actual face image size than the frame image f1 before correction. Therefore, the accuracy of the face image can be gradually improved.

  Below, methods for correcting the tracking result based on the detection result will be described together. The first method is a method of correcting a tracking result with an average of detection results when a face image presumed to be the same subject is detected under a plurality of detection conditions. For example, when the number of detected face images in almost the same area of the captured image reaches a predetermined number of frames, the face image size in the tracking result is corrected or detected with the average value of the detected face image size. For example, the face image direction in the tracking result may be corrected with the average value of the face image direction. In addition, the tracking position of the face image in the tracking result may be corrected so that the movement trajectory of the face image estimated to be the same subject becomes smooth.

  In addition, when face images presumed to be the same subject are detected under a plurality of detection conditions, the tracking result is corrected by giving priority to the detection result of the appropriate detection condition. For example, if the number of detections is reduced by changing the detection condition of the image filter, the detection result of the previous detection condition is assumed to be more appropriate, and a weighted average is calculated with a weight on the detection result. For example, the tracking result may be corrected based on the result.

  FIG. 6 shows processing operations of face detection processing and face tracking processing in the digital camera 10 configured as described above. As shown in the figure, when the image acquisition unit 22 acquires one captured image in the moving image captured by the imaging unit 11, the condition determination unit 30 refers to the result history in the result history storage unit 28, and the detected target is It is determined whether or not it exists (S10). If not, the process proceeds to step S21.

  When the detected target exists, the condition determining unit 30 determines the tracking condition from the result history (S11), and the face tracking unit 25 tracks the detected face image with the determined tracking condition ( S12). Next, the face tracking unit 25 stores the tracking result and the tracking condition in the result history storage unit 28 (S20).

  Next, the condition determination unit 30 determines whether or not all the detected objects have been processed (S14). When all the processes are not performed, the process returns to step S11 and the above processing operation is repeated. On the other hand, if all processing is performed, the process proceeds to step S21.

  In step S21, the condition determination unit 30 refers to the detection condition in the result history storage unit 28 and changes the detection condition from the previous detection condition. At this time, the condition determination unit 30 further changes the detection condition in consideration of the history of detection results in the result history storage unit 28.

  Next, the face detection unit 24 detects a face image under the changed detection condition (S16). Then, the face detection unit 24 stores the detection result and the detection condition in the result history storage unit 28 (S22).

  Next, the result correction unit 31 corrects the tracking result in step S12 based on the detection result in step S16 (S23). Then, the position information output unit 26 outputs the position information of the face image based on the detection result and the position information of the face image based on the corrected tracking result to the image composition unit 17 (S24). Thus, the image composition unit 17 generates and generates a frame image related to the face image detected by the face detection unit 24 and a frame image related to the face image tracked by the face tracking unit 25 and corrected by the result correction unit 31. The frame image and the captured image are combined and displayed and output via the display unit 13. Thereafter, the processing operation is terminated.

  The present invention is not limited to the above-described embodiments, and various modifications are possible within the scope shown in the claims, and embodiments obtained by appropriately combining technical means disclosed in different embodiments. Is also included in the technical scope of the present invention.

  For example, in the above-described embodiment, the components 22 to 28, 30, and 31 relating to face detection and face tracking are provided in the control unit 20, but may be provided in the image processing unit 12. In the above-described embodiment, the detection result and the tracking result are displayed and output as a frame image. However, based on the detection result and the tracking result, the photographing unit 11 such as focus adjustment and focus adjustment can be controlled.

  Moreover, in the said embodiment, the person's face image is detected and tracked from the picked-up image. However, it can also be applied to animation and other moving images. In addition, an image of a person's upper body or whole body may be detected, or an image of a moving object such as a ball, a vehicle, a baggage being carried, or the like may be detected. Furthermore, even a stationary object can be applied because it moves on the captured image if the digital camera 10 is moved. That is, the present invention is applicable to detection of an image of an object that moves on an acquired time-series image.

  In the above-described embodiment, the present invention is applied to the digital camera 10, but the present invention can be applied to any electronic device having a photographing unit such as a mobile phone with a camera. Furthermore, the present invention can be applied to an image processing device that acquires a moving image and performs image processing. Such an image processing device is expected to be applied not only to the digital camera 10 but also to a printer.

  Finally, each block of the digital camera 10, in particular, the control unit 20 and the image processing unit 12 may be configured by hardware logic, or may be realized by software using a CPU as follows.

  That is, the digital camera 10 is a storage device (recording device) such as a CPU that executes instructions of a control program that realizes each function, a ROM that stores the program, a RAM that expands the program, a memory that stores the program and various data, and the like. Medium). An object of the present invention is to provide a recording medium on which a program code (execution format program, intermediate code program, source program) of a control program for the digital camera 10 which is software for realizing the functions described above is recorded so as to be readable by a computer. This can also be achieved by supplying the digital camera 10 and reading and executing the program code recorded on the recording medium by the computer (or CPU or MPU).

  Examples of the recording medium include a tape system such as a magnetic tape and a cassette tape, a magnetic disk such as a floppy (registered trademark) disk / hard disk, and an optical disk such as a CD-ROM / MO / MD / DVD / CD-R. Card system such as IC card, IC card (including memory card) / optical card, or semiconductor memory system such as mask ROM / EPROM / EEPROM (registered trademark) / flash ROM.

  Further, the digital camera 10 may be configured to be connectable to a communication network, and the program code may be supplied via the communication network. The communication network is not particularly limited. For example, the Internet, intranet, extranet, LAN, ISDN, VAN, CATV communication network, virtual private network, telephone line network, mobile communication network, satellite communication. A net or the like is available. Also, the transmission medium constituting the communication network is not particularly limited. For example, even in the case of wired such as IEEE 1394, USB, power line carrier, cable TV line, telephone line, ADSL line, etc., infrared rays such as IrDA and remote control, Bluetooth ( (Registered trademark), 802.11 wireless, HDR, mobile phone network, satellite line, terrestrial digital network, and the like can also be used. The present invention can also be realized in the form of a computer data signal embedded in a carrier wave in which the program code is embodied by electronic transmission.

[Additional Notes]
A target image detection device according to the present invention is a target image detection device that detects a target image from an original image, and in order to solve the above problem, an acquisition unit that acquires the original image, and the target image Of the plurality of detection conditions for detection, a determination means for determining a detection condition different from the previous detection condition, and the detection condition determined by the determination means for the original image acquired by the acquisition means, the target The image processing apparatus includes a detection unit that detects an image and an output unit that outputs a detection result detected by the detection unit.

  In addition, a control method for a target image detection device according to the present invention detects a target image from an original image, and in order to solve the above problem, an acquisition step of acquiring the original image, and the target Of a plurality of detection conditions for detecting an image, a determination step for determining a detection condition different from the previous detection condition, and a detection condition determined by the determination means for the original image acquired in the acquisition step Thus, the method includes a detection step of detecting the target image and an output step of outputting a detection result detected in the detection step.

  Here, examples of the target to be detected include a moving object such as a human face, upper body, whole body, ball, and vehicle, or a part thereof.

  According to the above configuration and method, when the original image is acquired, the target image is acquired with respect to the acquired original image under a detection condition different from the previous detection condition among a plurality of detection conditions for detecting the target image. Detect and output the detection result. Therefore, a part of the target image is detected as compared with the conventional technique in which the target image is detected for each original image under a plurality of detection conditions for detecting the target image, and all detection results are output all at once. Although it is a result, the detection result can be output in a short time. As a result, the output detection result can be quickly used by other devices (for example, a device that controls display, a device that controls imaging, etc.).

  The target image detection device according to the present invention further includes a result history storage unit that stores a history of detection results detected by the detection unit, and the determination unit further includes a history of detection results in the result history storage unit. Based on the above, the detection condition may be determined. Alternatively, in the target image detection device according to the present invention, the determination unit may sequentially determine detection conditions from among a plurality of detection conditions for detecting the target image.

  When sequentially determining, all the detection results of the target image can be output for all of the plurality of detection conditions as in the conventional technique. Furthermore, when the original image is a plurality of still images included in the moving image, and the subsequent still image is acquired before all the detection results of a certain still image are output in the above-described conventional technique, Since the detection result for the subsequent still image can be output, the detection result for the still image that is newer than before can be included.

  On the other hand, when determining a detection condition based on the history of detection results, for example, a detection condition that is highly likely to detect a target image increases the frequency that is determined as the detection condition, and may detect a target image. It is conceivable that the detection condition with a low is to reduce the frequency determined as the detection condition. Therefore, the efficiency of detecting the target image is improved.

  By the way, since the detection target is the moving object or the like, when the original image is a plurality of still images included in the moving image, the position of the target image may deviate from the detection position as time passes.

  Therefore, in the target image detection device according to the present invention, the acquisition unit acquires a plurality of still images included in a moving image as the original image, and the original image at a certain time acquired by the acquisition unit is obtained. And a tracking unit that tracks the target image detected by the detection unit with an original image at a time point after the time point, and the output unit further outputs a tracking result tracked by the tracking unit. It is preferable.

  In this case, since the target image once detected is tracked and the tracking result is output, the above problem can be avoided. The tracking unit may track the target image based on the tracking condition. Furthermore, the tracking condition may be acquired from the detection condition.

  By the way, in the present invention, since the target image is detected under some detection conditions and the detection result is output, the detection result is compared with the conventional technique that outputs the detection result after detecting the target image under all the detection conditions. There are cases where the accuracy of is low.

  Thus, the target image detection device according to the present invention further includes a correction unit that corrects the tracking result tracked by the tracking unit based on the detection result detected by the detection unit, and the output unit includes the tracking unit. Instead of the tracking result tracked by the means, it is preferable to output the tracking result corrected by the correcting means.

  In this case, the detection result under another detection condition can be reflected in the tracking result, so that the accuracy of the detection result can be gradually improved. Note that the correction means may further correct based on the history of detection results.

  Note that the electronic device including the target image detection device having the above-described configuration can also exhibit the above-described effects.

  Further, the electronic apparatus further includes a display unit that displays information, and an image combining unit that combines an image indicating a detection result detected by the target image detection device with the original image and displays the image on the display unit. May be. In this case, although it is a part, the display of the image indicating the detection result can be quickly started, and thereafter the image indicating the remaining detection result can be sequentially displayed. As a result, it is possible to shorten the time for which the user waits for detection of the target image, as compared with the conventional technique in which images indicating all detection results are displayed all at once. In addition, since images indicating the detection results are sequentially displayed, new entertainment can be provided to the user. Furthermore, since the remaining detection results are for the original image after the original image where the first detection is performed, the target image is selected for the image indicating the detection result and the currently displayed original image. Deviation from an image indicating a detection result when detected can be suppressed.

  As described above, when a digital camera according to the present invention acquires a captured image, the digital camera detects a face image from the acquired captured image under a detection condition different from the previous detection condition. Since the detection result can be output in a short time, the present invention can be applied to any electronic device that detects a target image from an original image, such as a printer, in addition to a camera-equipped mobile phone.

10 Digital camera (electronic equipment)
DESCRIPTION OF SYMBOLS 11 Image pick-up part 12 Image processing part 13 Display part 14 Operation part 15 Image compression part 16 Image recording part 17 Image composition part (image composition means)
20 Control unit (target image detection device)
21 Storage Unit 22 Image Acquisition Unit (Acquisition Means)
23.30 Condition determining unit (determining means)
24 face detection unit (detection means)
25 Face tracking part (tracking means)
26 position information output unit (output means)
27 Condition storage unit 28 Result history storage unit 31 Result correction unit (correction means)

Claims (18)

A target image detection device for detecting a target image from an original image,
Obtaining means for obtaining the original image;
Determining means for determining a first detection condition different from the previous detection condition among the plurality of detection conditions for detecting the target image;
Detecting means for detecting the target image under a first detection condition determined by the determining means from a first original image acquired by the acquiring means;
Output means for outputting information that can identify the target image detected by the detection means to a second original image that follows the first original image;
The determination unit is different from the first detection condition based on a history of association between a certain detection condition and a detection result indicating whether the detection unit has detected the target image under the detection condition. A target image detection device, wherein the second detection condition is further determined. The detection means further detects the target image under a second detection condition determined by the determination means from the second original image,
2. The target image according to claim 1, wherein the output unit further outputs information that can identify the target image detected by the detection unit to a third original image that follows the second original image. Detection device.   The target image detection device according to claim 1, further comprising a tracking unit that tracks a target image detected by the detection condition with an original image following the original image including the target image. The tracking means tracks the target image in the second original image under a tracking condition included in the first detection condition,
The target image according to claim 3, wherein the output unit further outputs information that can identify the target image tracked by the tracking unit to a third original image that follows the second original image. Detection device. A correction means for correcting the tracking result tracked by the tracking means based on the history of the correspondence;
The output means further outputs the tracking result corrected by the correction means to a third original image following the second original image as information that can identify the target image tracked by the tracking means. The target image detection device according to claim 3 or 4.   The target image detection device according to claim 5, wherein the correction unit corrects the tracking result tracked by the tracking unit with an average of numerical values included in a detection condition associated with the detection result.   When the number of detection results indicating that the target image has been detected is decreased by changing the detection condition, the correction unit preferentially applies the detection condition before the decrease, thereby the tracking unit The target image detection device according to claim 5, wherein the tracking result tracked by is corrected.   The determination means changes the use frequency of the detection condition according to the number of detections so far according to a certain detection condition, and based on the changed use frequency, the previous detection condition among the plurality of detection conditions is The target image detection device according to claim 1, wherein different detection conditions are determined.   The determination unit, according to a combination of a plurality of detection conditions for detecting the target image, to determine a detection condition different from the previous detection condition among the plurality of detection conditions. The target image detection device according to any one of 1 to 8.   The determining means changes the order of a plurality of detection conditions for detecting the target image based on a predetermined regularity, so that a detection condition different from the previous detection condition among the plurality of detection conditions. The target image detecting device according to claim 1, wherein the target image detecting device is determined.   2. A result history storage unit that stores a history of correspondence between a detection condition and a detection result indicating whether or not the detection unit has detected the target image under the detection condition. 11. The target image detection device according to any one of 1 to 10.   The target image detection device according to claim 1, wherein the determination unit sequentially determines a detection condition from a plurality of detection conditions for detecting the target image.   The target image detection device according to claim 1, wherein the acquisition unit acquires a plurality of still images included in a moving image as the original image.   An electronic apparatus comprising the target image detection device according to any one of claims 1 to 13. A display for displaying information;
The electronic apparatus according to claim 14, further comprising an image composition unit configured to synthesize an image indicating a detection result detected by the target image detection device with the original image and display the image on the display unit.   A control program for operating the target image detection device according to any one of claims 1 to 13, wherein the control program causes a computer to function as each means.   A computer-readable recording medium on which the control program according to claim 16 is recorded. A method of controlling a target image detection device that detects a target image from an original image,
An acquisition step of acquiring the original image;
Of the plurality of detection conditions for detecting the target image, a determination step for determining a first detection condition different from the previous detection condition;
From the first original image acquired in the acquisition step, a detection step of detecting the target image under the first detection condition determined in the determination step;
Outputting the information that can identify the target image detected in the detection step to a second original image following the first original image, and
The determination step is different from the first detection condition based on a history of association between a detection condition and a detection result indicating whether or not the target image is detected under the detection condition in the detection step. A control method for a target image detection device, further comprising determining a second detection condition.

Images