JP2005202543A - Object extracting method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an object extracting method for extracting a part on which an object is imaged from image data photographed by a digital camera or a digital cam coder or the like. <P>SOLUTION: In this face detecting method for detecting the face of an object, a face detection region is reset by acquiring the photographic information of an image when face detection can not be performed in an initial face detection step, and the face detection step is executed again. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

  The present invention belongs to a subject extraction method for extracting a portion where a subject is captured from image data taken by a digital camera, a digital camcorder, or the like.

  In recent years, digital still cameras and digital video cameras have exploded in popularity. In addition, it is not uncommon for mobile phones and information portable terminals to be equipped with camera functions.

  The number of digital images taken with these digital devices tends to increase as compared to conventional silver halide cameras. For images taken with a silver salt camera, film development and printing on photographic paper are essential, but digital images basically do not require these processes. Also, digital images can be stored and stored easily and at low cost due to the spread of storage devices such as PCs.

  On the other hand, since the number of film photographing cameras has increased, it has become difficult to quickly search for an image desired by the user.

Japanese Patent Application Laid-Open No. 2002-358522 proposes a method of detecting a face area of an image and enlarging the detected face area to display a thumbnail list. With this configuration, it is possible to detect a face area as an attention area and display the enlarged portion of the face area, thereby enabling a system that improves the searchability of digital images.
JP 2002-358522 A JP 2003-163827 A

  Various methods have already been proposed for detecting the face area. As a basic face area detection method, a configuration is generally used in which a skin color area in image data is searched and a determination is made based on whether or not a face pattern exists in the skin color area.

  On the other hand, the amount of digital image data by a digital camera or the like is dramatically increasing against the background of increasing the number of pixels of an image sensor and increasing the speed of data processing. If face detection processing is executed with high accuracy on image data with a large amount of data, a long processing time is required. In order to detect a face at high speed, the detection accuracy must be lowered.

  In view of this, Japanese Patent Laid-Open No. 2003-163827 proposes a configuration in which the accuracy of the face detection process is switched depending on the shooting mode at the time of image shooting. When the photographing mode is the portrait mode, it is estimated that the face area in the image is large, and the processing is omitted to increase the speed. In other modes, the processing is executed with high precision. By this processing, both processing time and accuracy are achieved.

  However, Japanese Patent Laid-Open No. 2003-163827 is a proposal based on an image photographing apparatus such as a digital camera. Therefore, it is not appropriate to adapt to other purposes as it is because of the difference in the processing method of face detection and the calculation capability of the control unit.

  In order to solve the above problems, in the subject extraction method according to claim 1 of the present invention, an image input step, an attention area detection step of the image, and photographing information of the image are acquired. And a step of determining an attention area of the image based on the attention area detection result, and when the attention area fails to be detected, the attention area search range is determined from the imaging information, It is characterized in that the attention area detection step is re-executed.

  The subject extraction method described in claim 2 is characterized in that the attention area detection step performs face detection in an image.

  Further, in the subject extraction method described in claim 3, the shooting information distinguishes at least a person shooting mode and other shooting modes.

  The subject extraction method described in claim 4 is characterized in that the condition for re-execution of the attention area detection step is limited to that the person photographing mode is determined based on the photographing information.

  The subject extraction method described in claim 5 is characterized in that the attention area search range when the attention area detection step is re-executed includes a step of determining from the focal position information of the photographing information.

  When it is determined that there is no face area as a result of the face detection process performed on an image by the subject extraction method described in claims 1 to 5 of the present invention, the re-detection process is performed using information stored at the time of shooting. Extract and expand the range for executing. This can be expected to improve the accuracy of face detection during redetection processing.

  In addition, the face detection accuracy can be switched according to the computing capability of the device to be mounted.

  Embodiments of the present invention will be described below with reference to the drawings.

  In this embodiment, the subject extraction method of the present invention is mounted on an image display device such as a digital television.

  FIG. 1 is a block diagram showing a configuration of an image display apparatus to which an embodiment of the present invention is applied.

  Here, reference numeral 101 denotes the entire image display apparatus. A control CPU 201 controls the entire image display apparatus 101. An image display unit 203 includes a device that can realize a thin screen such as a liquid crystal display or a plasma display. The image display unit 203 displays an image instructed to the control CPU 201.

  Further, the present invention is also a device that displays a superimposed list display of a plurality of image data that is a feature of the present invention.

  An image storage unit 205 includes a nonvolatile storage device such as a hard disk. A large amount of images taken by a digital camera, a digital video camera, etc., and captured by an image display device can be accumulated and stored by a user operation.

  Reference numeral 207 denotes a memory card. The memory card 207 is detachable. The memory card 207 can also be connected to an image capturing device such as a digital camera or a digital video camera, and the captured image data can be taken into the image display device 101 via the memory card 207.

  Reference numeral 209 denotes an external input terminal. Via this terminal, image data of an image photographing device such as a digital camera or a digital video camera can be directly taken into the image display apparatus 101.

  Reference numeral 211 denotes an image decoder unit. Image data captured from the image storage unit 205, the memory card 207, and the external input terminal 209 is converted into a data format that can be handled in the image display apparatus 101 such as the control CPU 201.

  Reference numeral 213 denotes a face detection processing unit. It is determined whether or not there is a person image in the captured image data, and if there is a person image, processing for specifying the area is performed. Details of the processing will be described later.

  Reference numeral 215 denotes a cache memory. It is used for temporarily holding the image data decoded by the image decoder 211, holding the image data displayed on the image display unit 203, holding the temporary data of the face detection processing unit 213, and the like.

  In the present image display apparatus, the present invention is implemented in order to perform a display effect such as enlarging a detected place based on a face detection result.

  FIG. 2 is a flowchart showing a processing flow of face detection processing used as one method of attention area detection.

  This processing is executed by the face detection processing unit 213 from image data to be displayed prior to execution of the overlapping list display. After the processing starts, the process proceeds to step 101.

  In step 101, a process of searching for an area including skin color data from the color data in the image data is executed. After processing, go to step 103.

  In step 103, pattern matching processing is performed between the target image and an image pattern having a pattern or a mouth shape indicating facial features. After processing, go to step 105.

  In step 105, the processing branches depending on whether or not a face area exists in the target image based on the processing results of steps 101 and 103. That is, if a face area exists, the process proceeds to step 107. If the face area does not exist, this process ends.

  In step 107, the information of the area determined as the face area based on the processing results in steps 101 and 103 is written to the cache memory 215, and the process is returned to the control CPU 201, thereby ending this process.

  FIG. 3-A is a diagram illustrating an example of image data. FIG. 3B is a diagram illustrating an example of a region-of-interest determination process based on the determination result of the face area after the face detection process.

  Face detection processing is executed on the image data in which a human image is captured as shown in FIG. 3A by the face detection processing unit 213 in the processing flow described with reference to FIG. As shown in FIG. 3B, the areas determined as the face area are portions 303 and 305. As shown in the figure, the face area is recognized as circular graphic data.

  The face detection process has a good detection rate when the ratio of the face as the subject to the entire image data is large. However, when the ratio is low, there is a tendency that the face cannot be detected. Processing in this case will be described with reference to FIGS. 4-A and 4-B.

  FIG. 4-A is a diagram illustrating an example of image data. FIG. 4-B is a diagram illustrating an example of a re-detection range of face detection based on shooting information.

  Image data taken with a digital camera or the like can hold information at the time of shooting in the image data (EXIF information). This EXIF information includes position information at the time of shooting. Therefore, in the present invention, when the face detection process fails, the focal position information of the EXIF information is read. This is the range indicated by 403 and 405 in FIG.

  Based on this focal position information, a re-detection range for face detection is determined. A region indicated by reference numeral 407 in FIG.

  When the face area cannot be detected from the target image, the re-detection area is cut out from the focal position information of the EXIF information and determined, and then the face detection process is re-executed.

  FIG. 5 is a diagram showing an example of an image file format.

  Various formats have been proposed for digital images taken with a digital camera or the like. Recently, in addition to the image data 501, a format including additional information at the time of photographing has become widespread.

  In the present invention, the focus position information 503 which is the additional information is used. In the focal position information 503, information indicating which part of the image data 501 is focused is written.

  FIG. 6 is a diagram illustrating a process flow for determining whether or not the face detection process is re-executed.

  This process is also executed by the control CPU 201 when the control CPU 201 determines that the face detection process is necessary. When executing, the process proceeds to step 201 first.

  In step 201, the face detection process described with reference to FIG. After the face detection process is completed, the process proceeds to step 203.

  In step 203, the process branches depending on whether or not a face area has been detected as a result of the face detection in the previous step. That is, if the face area can be detected, the process ends. If not detected, the process proceeds to step 205.

  In step 205, a process of reading the focal position information 503 from the target image data is executed. After processing, go to step 207.

  In step 207, a setting for enlarging and cutting the periphery from the focal position information 503 is executed. After execution, the process proceeds to step 209.

  In step 209, the area determined in the previous step is scanned again, and face detection processing is performed. After the detection process ends, this process ends.

  FIG. 7 is a diagram illustrating an example in which the face detection processing result is applied to image display.

  Here, the area 601 is an area where the face is detected. In this image display apparatus, when a large amount of image data is displayed in this way, display control is performed so that the region 601 is always exposed on the screen. Thereby, a user's image search property improves.

  This embodiment is also an example in which the present invention is applied to an image display apparatus. Since the basic configuration is the same as that of the first embodiment, description thereof is omitted.

  In this embodiment, in order to further improve the accuracy of redetection of the face detection area, the shooting mode 505 at the time of shooting an image with the image information described in FIG. 5 is used.

  FIG. 8 is a diagram illustrating a flow of processing for determining whether or not to re-execute the face detection processing in the present embodiment. After the processing starts, the process proceeds to step 301.

  In step 301, the face detection process described in FIG. After the face detection process is completed, the process proceeds to step 303.

  In step 303, the process branches depending on whether or not a face area has been detected as a result of the face detection in the previous step. That is, if the face area can be detected, the process ends. If not detected, the process proceeds to step 305.

  In step 305, a process of reading the shooting mode 505 from the target image data is executed. After the processing, the process proceeds to step 307.

  In step 307, it is determined whether or not the content of the photographing mode 505 read in the previous step is a portrait (person image) mode, and the process is branched. That is, if the content of the shooting mode 505 is the portrait mode, it is determined that there is a high possibility that a person image is captured in the image, and the process proceeds to step 309. If the content of the shooting mode 505 is not a portrait, there is a high possibility that there is no person image in the image, so this process ends.

  In step 309, processing for reading the focal position information 503 from the data of the target image is executed. After processing, go to step 311.

  In step 311, a setting for enlarging and cutting the periphery from the focal position information 503 is executed. After execution, the process proceeds to step 209.

  In step 313, the area determined in the previous step is scanned again, and face detection processing is performed. After the detection process ends, this process ends.

It is the block diagram which showed the structure of the image display apparatus to which the Example of this invention is applied. It is the flowchart which showed the processing flow of the face detection process used as one method of attention area detection. FIG. 3-A is a diagram illustrating an example of image data. FIG. 3B is a diagram illustrating an example of the determination result of the face area after the face detection process. FIG. 4-A is a diagram illustrating an example of image data. FIG. 4-B is a diagram illustrating an example of a re-detection range of face detection based on shooting information. It is the figure which showed an example of the format of an image file. It is the figure which showed the flow of the process which judges whether re-execution of a face detection process is performed. It is the figure which showed the example which applied the face detection process result to the image display. FIG. 10 is a diagram illustrating a process flow for determining whether or not to re-execute a face detection process in the second embodiment.

Claims (5)

  A step of inputting an image, a region of interest detection step of the image, a step of acquiring photographing information of the image, and a step of determining a region of interest of the image based on the region of interest detection result, An object extraction method characterized by determining the attention area search range from the imaging information and re-executing the attention area detection step when the attention area detection fails.   The subject extraction method according to claim 1, wherein the attention area detection step performs face detection in an image.   The subject extraction method according to claim 1, wherein the photographing information distinguishes at least a person photographing mode and another photographing mode.   The subject extraction method according to claim 3, wherein a condition for re-execution of the attention area detection step is limited to the case where the photographing information is determined to be a person photographing mode.   5. The subject extraction method according to claim 1, further comprising: determining the attention area search range when the attention area detection step is re-executed based on focal position information of the photographing information.

Images