JP4773924B2 - IMAGING DEVICE, ITS CONTROL METHOD, PROGRAM, AND STORAGE MEDIUM - Google Patents

Description

  The present invention relates to a technique for identifying an identification target, specifically, a position of a person's face from image data.

  A technique is known in which feature data of an object (hereinafter referred to as a main subject) that satisfies an identification target is registered in advance and collated with this to identify the object from image data.

  For example, Patent Document 1 discloses a method of specifying a region where a face exists by calculating a correlation value between face image data registered in advance and image data to be identified.

  Further, in Patent Document 2, the region in which image data is divided into a plurality of frequency bands and the possibility that a face is likely to exist is limited, and the face image data registered in advance for the limited region A method for calculating a correlation value is disclosed.

  An example in which these technologies are applied to a camera will be described.

  In Patent Document 3, a main subject is automatically detected from image data by a technique such as shape analysis, and a focus detection area is displayed for the detected main subject. And the method of performing a focus adjustment operation | movement with respect to this focus detection area is disclosed.

  Further, Patent Document 4 discloses a method of identifying a region where a face as a main subject exists by extracting feature points from image data and setting a focus detection area according to the size of this region. .

  Further, Patent Document 5 discloses a method of identifying a region where a face exists by extracting feature points from image data and performing zoom driving according to the size of this region.

Further, in Patent Document 6, a method is provided in which a face position is detected before shooting, and a frame is displayed at the detected face position immediately after shooting, and the face is enlarged to confirm whether the face is in focus. It is disclosed.
JP 09-251534 A JP-A-10-162118 JP 2003-107335 A JP 2004-317699 A JP 2004-320286 A JP 2005-323015 A

  By the way, so-called compact type digital cameras are mainly used in live view display in which images are sequentially displayed on a liquid crystal display on the back of the camera at the time of shooting, and the liquid crystal display is used as an electronic viewfinder. In such a camera, during live view display, face detection is always performed from the image data captured by the image sensor, and the area where the face exists is surrounded by a frame and displayed so that the user can identify the face. The result can be confirmed. However, when it is attempted to detect the position of the face with high accuracy from the image data captured by the image sensor, the processing takes time. For this reason, when the frame is displayed after the face detection process is completed, the person has already moved to another position on the screen, and the position of the frame may not match the actual face position of the person. That is, the display position of the frame may not follow the movement of the person's face. In such a case, the camera determines that there is a person's face at the position where the frame is displayed. Therefore, the exposure or focus is actually adjusted to the position of the frame where there is no person's face. May go crazy. There is also a problem that the position of the displayed frame does not match the position of the actual person's face, and the user feels uncomfortable.

  Therefore, the present invention has been made in view of the above-described problems, and an object thereof is to improve the followability of the face position detection result to the actual face position without reducing the face detection accuracy. That is.

In order to solve the above-described problems and achieve the object, an imaging apparatus according to the present invention uses a detection unit that detects a target position from image data and an image data generated by the imaging element. Display means for displaying the display, display control means for causing the display means to display a display indicating the position of the object detected by the detection means, and storage means for storing the image data generated by the image sensor. When the detection unit detects the position of the target from the image data generated by the image sensor before the photographing operation for storing the image data generated by the image sensor in the storage unit, the storage in comparison with the case of detecting the position of the target from the image data stored in the unit, at least, it sets a narrow range of the image data to detect the target, and, biopsy And performing possible to one of the sets a narrow range of tilt of the object.
In addition, an imaging apparatus according to the present invention includes a detection unit that detects a target position from image data, a display unit that displays an image using image data generated by the imaging device, and a detection unit that detects the image. Display control means for causing the display means to display a display indicating the position of the target, and a shutter switch for instructing storage means to store image data, and the detection means includes the shutter switch When the position of the target is detected from the image data generated by the image sensor before the operation of the shutter, the position of the target is detected from the image data generated by the image sensor after the shutter switch is operated. in comparison with the case where, at least, it sets a narrow range of the image data to detect the target, and, inclination of the object to be detected And performing one of setting a narrow range.

The image pickup apparatus control method according to the present invention includes a detection step of detecting a target position from image data, and a display step of displaying an image on a display unit using image data generated by the image pickup device. A display control step for displaying on the display means a display indicating the position of the object detected in the detection step, and a photographing step for storing the image data generated by the image sensor in the storage means, In the detection step, when the position of the target is detected from the image data generated by the imaging device before the photographing step, the position of the target is detected from the image data stored in the storage unit compared to, at least, it sets a narrow range of the image data to detect the target, and, either by setting narrow range of the target slope to be detected And performing.
The image pickup apparatus control method according to the present invention includes a detection step of detecting a target position from image data, and a display step of displaying an image on a display unit using image data generated by the image pickup device. A display control step for displaying on the display means a display indicating the position of the object detected in the detection step, and a photographing step for storing image data in the storage means in response to an operation of a shutter switch, In the detection step, when detecting the position of the target from the image data generated by the image sensor before the shutter switch is operated, the image data generated by the image sensor after the shutter switch is operated in comparison with the case of detecting the position of the object from at least be set narrow range of the image data to detect the target, and the detection And performing one of setting a narrow range of inclination of the subject in ability.

  A program according to the present invention causes a computer to execute the above control method.

  A storage medium according to the present invention stores the above program.

  According to the present invention, it is possible to improve the followability of the detection result of the face position with respect to the actual face position without reducing the face detection accuracy.

  DESCRIPTION OF EXEMPLARY EMBODIMENTS Hereinafter, preferred embodiments of the invention will be described in detail with reference to the accompanying drawings.

  The embodiment described below is an example as means for realizing the present invention, and should be appropriately modified or changed according to the configuration and various conditions of the apparatus to which the present invention is applied. It is not limited to the embodiment.

(First embodiment)
FIG. 1 is a block diagram illustrating a configuration of an imaging apparatus 100 according to the first embodiment of the present invention. In the present embodiment, a digital camera will be described as an example of a device having a function of displaying an identification area (specifically, a human face area) of a main subject. Other devices such as a camera and a surveillance camera may be used.

  In FIG. 1, reference numeral 101 denotes an objective lens group that forms a subject image on an image sensor 103 to be described later, and reference numeral 102 denotes a light amount adjusting device including an aperture device and a shutter device. Reference numeral 103 denotes an image sensor such as a CCD or CMOS sensor that converts an object image formed by the objective lens group 101 into an electric signal. An analog signal processing circuit 104 performs clamp processing, gain processing, and the like on the analog output signal of the image sensor 103. An analog / digital (hereinafter referred to as A / D) converter 105 converts the output of the analog signal processing circuit 104 into a digital signal. The image data output from the A / D converter 105 is written into the memory 108 via a digital signal processing circuit 107 and a memory control circuit 106 described later, or directly via the memory control circuit 106.

  A digital signal processing circuit 107 performs pixel interpolation processing and color conversion processing on data from the A / D converter 105 or data from the memory control circuit 106. The digital signal processing circuit 107 uses the data from the A / D converter 105 to calculate a value indicating the in-focus state of the subject, a luminance value, and white balance adjustment.

  The digital signal processing circuit 107 is equipped with a subject identification circuit 115 for previously registering feature data of the main subject and collating it with the feature data to identify the main subject from the image data. In the present embodiment, the subject identification circuit 115 detects a facial feature such as eyes and mouth registered as feature data from data from the A / D converter 105 or data written in the memory 108. Thus, an area where a human face exists is identified. In this embodiment, the position of the area where the human face that is the main subject exists is used as the identification area.

  The system control circuit 112 controls the exposure control circuit 113 and the focus control circuit 114 based on the calculation result of the digital signal processing circuit 107. Specifically, focus control processing, exposure control processing, and light control processing using a subject image that has passed through the objective lens group 101 are performed. In addition, the system control circuit 112 includes a display control circuit 117 that controls display of a scene frame indicating the identification area based on the identification result of the subject identification circuit 115.

  The memory control circuit 106 controls the analog signal processing circuit 104, the A / D converter 105, the digital signal processing circuit 107, the memory 108, and a digital / analog (hereinafter referred to as D / A) converter 109. The data A / D converted by the A / D converter 105 is sent via the digital signal processing circuit 107 and the memory control circuit 106, or the data A / D converted by the A / D converter 105 is directly sent to the memory control circuit 106. The data is written to the memory 108 via the.

  Data to be displayed on the display device 110 is written in the memory 108, and the data written in the memory 108 is displayed on the display device 110 described later via the D / A converter 109. Further, the memory 108 stores captured still images or moving images. The memory 108 can also be used as a work area for the system control circuit 112.

  Reference numeral 110 denotes a display device that includes a liquid crystal monitor that displays an image generated from image data obtained by photographing, and functions as an electronic viewfinder if subject images obtained using the image sensor 103 are sequentially displayed. The display device 110 can arbitrarily switch the display on and off according to an instruction from the system control circuit 112. When the display is turned off, the power consumption of the imaging device 100 is reduced as compared with the case where the display is turned on. it can. Further, the display device 110 displays an operation state, a message, and the like of the imaging device 100 using characters, images, and the like in accordance with a command from the system control circuit 112.

  Reference numeral 111 denotes an interface for transferring image data and management information attached to the image data to / from external storage media such as a memory card and a hard disk, and peripheral devices such as other computers and printers. When the interface 111 is configured using a PCMCIA card, a CF (Compact Flash (registered trademark)) card, or the like, a variety of communication cards may be connected. Examples of the various communication cards include LAN cards, modem cards, USB cards, IEEE 1394 cards, P1284 cards, SCSI cards, communication cards such as PHS, and the like.

  Reference numeral 116 denotes an attitude detection circuit that detects the attitude of the imaging apparatus 100 and outputs a detection result to the system control circuit 112.

  The system control circuit 112 controls the overall operation of the imaging apparatus 100. Constants, variables, programs, and the like for operation of the system control circuit 112 are stored in a memory (not shown) in the system control circuit 112.

  The exposure control circuit 113 drives the aperture device and shutter device of the light amount adjusting device 102. The focus control circuit 114 drives the focusing lens and zoom lens of the objective lens group 101.

  FIG. 2 is a diagram illustrating some of the components of the digital signal processing circuit 107.

  A color conversion circuit 201 separates and generates the luminance signal Y and the color difference signals Cr and Cb from the RGB signal of the image data output from the A / D converter 105.

  Reference numeral 202 denotes a color processing circuit that performs white balance processing on the color difference signals Cr and Cb and achromatic processing for thinning a high-luminance or low-luminance color.

  A luminance processing circuit 203 performs filter processing, luminance correction processing, gamma processing, and the like on the luminance signal.

  Reference numeral 204 denotes an edge enhancement circuit that performs edge enhancement by applying filter processing to the luminance signal.

  Reference numeral 205 denotes a band pass filter (BPF) or a high pass filter (HPF) for extracting a high frequency signal from the luminance signal after the luminance processing.

  Reference numeral 206 denotes a scene detection circuit that detects a scene change during imaging.

  The digital signal processing circuit 107 outputs a YCrCb signal obtained by combining the luminance signal Y and the color difference signals Cr and Cb as image data.

  The operation of the digital camera according to this embodiment will be described below with reference to the flowcharts of FIGS. A program for executing this processing is stored in a memory in the system control circuit 112 and is executed under the control of the system control circuit 112.

  FIG. 3 is a flowchart for explaining the operation in the main process in the imaging apparatus 100 according to the present embodiment.

  This process is started when the power is turned on, for example, after battery replacement.

  In step S101, the system control circuit 112 initializes various flags and control variables in the internal memory.

  In step S102, the system control circuit 112 initially sets the image display of the display device 110 to OFF.

  In step S <b> 103, the system control circuit 112 determines whether there is a problem in the operation of the imaging apparatus 100 with respect to the remaining capacity of the power source and the operation status. If the system control circuit 112 determines that there is a problem, the system control circuit 112 proceeds to step S105, displays a predetermined warning by an image or sound using the display device 110, and then returns to step S103. If the system control circuit 112 determines that there is no problem with the power supply, the process proceeds to step S104.

  In step S104, the system control circuit 112 determines whether or not the operation state of the storage medium has a problem in the operation of the imaging apparatus 100, particularly in the data recording / reproduction operation with respect to the storage medium. If the system control circuit 112 determines that there is a problem, the process proceeds to step S105 described above, and a predetermined warning is displayed by an image or sound using the display device 110, and then the process returns to step S103. If the system control circuit 112 determines that there is no problem in the operation of the storage medium, the process proceeds to step S106.

  In step S <b> 106, the system control circuit 112 uses the display device 110 to display a user interface (hereinafter referred to as UI) of various setting states of the imaging device 100 using images and sounds. If the image display of the display device 110 is on, UI display of various setting states of the imaging device 100 may be performed by using the display device 110 by an image or sound. Thus, various settings are made by the user.

  In step S107, the system control circuit 112 sets the image display of the display device 110 to ON.

  In step S108, the system control circuit 112 sets an observation (live view) display state in which images generated based on image data obtained by the image sensor 103 are sequentially displayed. In this observation display state, the display device 110 functions as an electronic finder by sequentially displaying data sequentially written in the memory 108 on the display device 110.

  In step S109, the system control circuit 112 performs focus control processing and exposure control processing based on the obtained image data. At this time, if information regarding an identification area that is an area where a face exists (hereinafter referred to as a face area) is obtained, the focus control process and the exposure control process are performed with priority on the identification area.

  In step S110, the system control circuit 112 sets the face detection level to 1.

  In step S111, the system control circuit 112 causes the subject identification circuit 115 to perform face identification processing for identifying a face area from the image data. Details of the face detection process will be described later. Note that the face detection process performed here is performed, for example, to adjust the exposure or focus on the face of a person considered to be the main subject, as will be described later.

  In step S112, the system control circuit 112 receives the identification result by the subject identification circuit 115 and causes the digital signal processing circuit 107 to perform a process of displaying an identification display indicating the identification area.

  In the present embodiment, as shown in FIG. 7, a frame is displayed as an identification display on the portion where the face is located. In the present embodiment, a square frame is displayed so as to surround the face according to the size of the face.

  In step S113, the system control circuit 112 checks whether or not the shutter switch SW1 is pressed. If not, the system control circuit 112 returns to step S103, but if the shutter switch SW1 is pressed, the process proceeds to step S114. The shutter switch SW1 is a switch that is turned on by half-pressing a release button (not shown).

  In step S114, the system control circuit 112 performs exposure so that the luminance value of the identification area is appropriate, or the weight of the identification area is larger than other areas so that the luminance value of the entire screen is appropriate. The control circuit 113 performs exposure control.

  In accordance with the result of the exposure control at this time, if necessary, it is determined that a flash device (not shown) emits light at the time of photographing. Further, in order to prevent the white balance from becoming inappropriate due to the influence of the skin color, the white balance is adjusted by excluding the identification area or by making the weighting smaller than that of the other areas. Further, the focus control circuit 114 is caused to perform focus control so that the identification area is in focus.

  In step S115, the system control circuit 112 continues the observation display state until the shutter switch SW2 is operated. The shutter switch SW2 is a switch that is turned on by fully pressing a release button (not shown).

  In step S116, the system control circuit 112 checks whether the shutter switch SW2 is pressed. If not, the system control circuit 112 proceeds to step S117. If the shutter switch SW2 is pressed, the system control circuit 112 proceeds to step S118.

  In step S117, the system control circuit 112 checks whether or not the shutter switch SW1 is pressed. If not, the system control circuit 112 returns to step S103, and if the shutter switch SW1 is pressed, returns to step S116.

  In step S <b> 118, the system control circuit 112 performs a shooting process for writing the captured image data in the memory 108. The exposure control circuit 113 drives the diaphragm device according to the exposure control result in step S114, opens the shutter, and exposes the image sensor 103. If necessary, the flash device is caused to emit light, and the shutter is closed when the set exposure time has elapsed. The charge signal output from the image sensor 103 is written into the memory 108 as image data for storage via the A / D converter 105, the digital signal processing circuit 107, and the memory control circuit 106. Further, the system control circuit 112 uses the memory control circuit 106 and the digital signal processing circuit 107 to read the image data for storage written in the memory 108 and execute vertical addition processing. The digital signal processing circuit 107 sequentially performs color processing to generate display image data, which is written in the memory 108 again.

  In step S119, the system control circuit 112 sets the face detection level to 2.

  In step S120, the system control circuit 112 causes the subject identification circuit 115 to perform face identification processing for identifying a face area from the image data. Details of the face detection process will be described later. The result of the face detection processing performed here is used as information for accurately specifying the position of a person's eyes when, for example, a red-eye phenomenon occurs during flash photography. Therefore, the accuracy of face position detection is required. Alternatively, it may be used as information for classifying a photograph of a person or a specific person.

  In step S121, the system control circuit 112 receives the identification result from the subject identification circuit 115 and causes the digital signal processing circuit 107 to perform a process of displaying an identification display indicating the identification area.

  In the present embodiment, as shown in FIG. 7, a frame is displayed at the portion where the face is located.

  In step S122, the display device 110 displays the captured image using the display image data obtained in step S118.

  In step S123, the system control circuit 112 reads the image data for storage written in the memory 108, and executes various image processing using the memory control circuit 106 and the digital signal processing circuit 107 as necessary. Further, the system control circuit 112 performs a recording process of writing the image data for storage compressed into the storage medium after performing the image compression process. At this time, the position information of the face area identified in step S120 is also written together with the image data.

  In step S124, the system control circuit 112 waits for the set display time to elapse and proceeds to step S125.

  In step S125, the system control circuit 112 sets the display device 110 to the observation display state and returns to step S103. Thereby, after confirming the image photographed with the display apparatus 110, it can be set as the observation display state for the next imaging | photography.

  Next, the face detection process in steps S111 and S120 of FIG. 3 which is a feature of the present embodiment will be described.

  Various techniques are known for detecting the face area. For example, there is a method using learning represented by a neural network. In addition, there is a technique for identifying parts having physical shape features such as eyes, nose, mouth, and facial contour from image data using template matching. In addition, there is a technique that detects a feature amount of image data such as skin color or eye shape and uses statistical analysis (for example, Japanese Patent Laid-Open Nos. 10-232934 and 2000-48184). reference). In addition, it is determined whether the previous face area is near the detected position, the color near the face area is determined in consideration of the color of clothes, or the face identification is performed closer to the center of the screen. There is a method of setting the threshold value for this purpose low. Alternatively, there is a method of tracking the main subject by specifying a region in which the main subject exists in advance, storing a histogram and color information, and obtaining a correlation value. In this embodiment, a pair of eyes (both eyes), a nose, a mouth, and a face outline are detected from the shape, and a face area identification process is performed by a method of determining a person's face area from these relative positions. Yes. Then, a square area surrounding the face area is set along the size of the detected face area, and the coordinates of the square area are obtained. The coordinates of the rectangular area surrounding the face area become an identification area indicating the position of the main subject. The area surrounding the face area is not limited to a square, and may be another shape such as an ellipse or a shape that follows the detected outline of the face.

  FIG. 8 is a diagram for explaining image data in the present embodiment.

  An image sensor having a photoelectric conversion unit of 3000 pixels in the X direction and 2000 pixels in the Y direction is used.

  FIG. 4 is a flowchart for explaining the operation of the face detection process in the present embodiment.

  In the face detection process illustrated in FIG. 4, the face detection range is switched according to the face detection level.

  In step S201, the system control circuit 112 determines whether or not the face detection level is 1. If the face detection level is 1, the process proceeds to step S202. If the face detection level is not 1, the process proceeds to step S220. In the present embodiment, as shown in FIG. 3, when the face detection level is not 1, the face detection level is 2.

  In steps S202 to S203, a face detection range is set when the face detection level is 1.

  The face detection range when the face detection level is 1 is a range surrounded by a thick line in FIG.

  In step S202, the system control circuit 112 sets the start point coordinates (X1, Y1) of the face detection range.

  Specifically, X1 = 999 and Y1 = 499 (see FIG. 9).

  In step S203, the system control circuit 112 sets the end point coordinates (X2, Y2) of the face detection range.

  Specifically, X2 = 1999 and Y2 = 1499 (see FIG. 9).

  In step S204, the system control circuit 112 sets the start point coordinates (X, Y) for feature extraction.

  Specifically, X = X1 and Y = Y1.

  In step S205, the system control circuit 112 sets a feature extraction area.

  Specifically, based on the start point coordinates of the feature extraction region specified in step S204, a region for a predetermined pixel α in the X direction and a predetermined pixel β in the Y direction is set (see FIG. 10).

  In step S206, the system control circuit 112 performs feature extraction for the designated region.

  In step S207, the system control circuit 112 updates the start point coordinates (X, Y) for feature extraction.

  Specifically, X = X + 1 is calculated. At this time, when X + α> X2, X = X1 and Y = Y + 1 are calculated.

  In step S208, the system control circuit 112 sets a feature extraction area.

  Specifically, based on the start point coordinates of the feature extraction area specified in step S207, an area for a predetermined pixel α in the X direction and a predetermined pixel β in the Y direction is set (see FIG. 10).

  By doing so, the feature extraction region moves as indicated by the arrow within the range surrounded by the thick line in FIG. 10, and the feature extraction processing is performed for the entire range of the region surrounded by the thick line.

  In step S209, the system control circuit 112 determines whether or not feature extraction has been completed for the entire face detection range by determining whether the conditions of X + α> X2 and Y + β> Y2 are satisfied. If not completed, the process returns to step S206. If completed, the process proceeds to step S210 and the face detection process is terminated.

  In step S220, the system control circuit 112 sets the start point coordinates (X1, Y1) of the face detection range.

  Specifically, X1 = 0 and Y1 = 0 are set (see FIG. 11).

  In step S221, the system control circuit 112 sets the end point coordinates (X2, Y2) of the face detection range.

  Specifically, X2 = 2999 and Y2 = 1999 are set (see FIG. 11).

  That is, in this embodiment, the face detection range when the face detection level is 2 is the entire screen as shown in FIG.

  In step S222, the system control circuit 112 sets the start point coordinates (X, Y) for feature extraction.

  Specifically, X = X1 and Y = Y1.

  In step S223, the system control circuit 112 sets a feature extraction region.

  Specifically, based on the start point coordinates of the feature extraction area specified in step S222, an area for a predetermined pixel α in the X direction and a predetermined pixel β in the Y direction is set (see FIG. 12).

  In step S224, the system control circuit 112 performs feature extraction for the specified region.

  In step S225, the system control circuit 112 updates the starting point coordinates (X, Y) for feature extraction.

  Specifically, X = X + 1 is calculated. At this time, when X + α> X2, X = X1 and Y = Y + 1 are calculated.

  In step S226, the system control circuit 112 sets a feature extraction area.

  Specifically, based on the start point coordinates of the feature extraction area specified in step S207, an area for a predetermined pixel α in the X direction and a predetermined pixel β in the Y direction is set (see FIG. 12).

  By doing so, the feature extraction region moves as indicated by the arrow within the range surrounded by the thick line in FIG. 12 (the entire screen in this embodiment), and the entire range of the region surrounded by the thick line is moved. A feature extraction process is performed for.

  In step S227, the system control circuit 112 determines whether or not the feature extraction of the face detection range is completed by determining whether the conditions of X + α> X2 and Y + β> Y2 are satisfied. If not completed, the process returns to step S224. If completed, the process proceeds to step S210 to end face detection. When feature extraction in the face detection range is completed in a feature extraction area of a certain size, the feature extraction area is changed by changing the values of α and β, and feature extraction in the face detection range is performed again. Also good.

  As described above, in the present embodiment, in the face detection operation before the shooting operation, specifically, in the face detection operation in the standby state (when displaying live view), the face detection level is set to 1 in step S110, In next step S111, face detection is performed. In the face detection performed in step S111, the face detection range is limited to the vicinity of the center as shown in FIGS. Further, in the face detection operation after the photographing operation, specifically, in the face detection operation in the photographed image (image reproduction), the face detection level is set to 2 in step S119, and the face detection is performed in the next step S120. . In the face detection performed in step S120, as shown in FIGS. 11 and 12, the face detection range is not limited.

  If a person is moving in the state before the shooting operation and the face detection operation takes time, if the position of the face area can be identified by the face detection processing, the actual face position will be There is a high possibility that the focus control process and the exposure control process using the position of the face area will not function effectively. Therefore, in the face detection operation before the shooting operation, more specifically, in the face detection operation in the standby state (live view display), the processing time is shortened by limiting the face detection range. Improves the ability to follow the movement. In addition, the face detection operation after the shooting operation, more specifically, the face detection operation in the captured image (when the image is played back) does not cause the above-mentioned problems, and therefore the detection is not limited without limiting the face detection range. A face detection result with less image quality is obtained. As described above, in the present embodiment, the area of the face detection range is varied depending on whether priority is given to the processing time required for face detection or suppression of face detection omission.

(Second Embodiment)
Hereinafter, a second embodiment of the face detection process will be described. The configuration of the imaging device and the main operation of the imaging device are the same as those in the first embodiment shown in FIGS.

  FIG. 5 is a flowchart for explaining the operation of the face detection process in the second embodiment.

  In step S301, the system control circuit 112 determines whether or not the face detection level is 1. If the face detection level is 1, the process proceeds to step S302. If the face detection level is not 1, the process proceeds to step S310.

  In step S302, the system control circuit 112 generates image data for feature extraction. When generating image data, pixel thinning processing is performed in order to reduce the generated size.

  In step S303, the system control circuit 112 performs feature extraction using the image data generated in step S302 described above. The feature extraction method is the same as in the first embodiment.

  In step S310, the system control circuit 112 generates image data for feature extraction. When generating image data, pixel thinning processing is not performed.

  In step S311, the system control circuit 112 performs feature extraction using the image data generated in step S310 described above. The feature extraction method is the same as in the first embodiment.

  As described above, in the present embodiment, in the face detection operation before the shooting operation, specifically, in the face detection operation in the standby state (when displaying live view), the face detection level is set to 1 in step S110, In next step S111, face detection is performed. In face detection performed in step S111, image data subjected to pixel thinning processing is generated (step S302), and feature extraction is performed (step S303). Further, in the face detection operation after the photographing operation, specifically, in the face detection operation in the photographed image (image reproduction), the face detection level is set to 2 in step S119, and the face detection is performed in the next step S120. . In the face detection performed in step S120, image data not subjected to pixel thinning processing is generated (step S310), and feature extraction is performed (step S311).

  As described above, in the face detection operation before the photographing operation, specifically, the face detection operation in the standby state, the image data subjected to the pixel thinning process is used, so that the amount of data handled in the face detection operation can be reduced. it can. For this reason, since the processing time can be shortened, the followability of the face detection to the movement of the person's face can be improved.

  In addition, the face detection operation after the shooting operation, more specifically, the face detection operation in the captured image (during image playback) uses image data that has not been subjected to pixel thinning processing, and therefore has a smaller face than before the shooting operation. Can be detected, and a highly accurate face detection result can be obtained. As described above, the number of pixels of the image data used for the face detection operation is varied depending on whether processing time required for face detection is given priority or accuracy is given priority.

(Third embodiment)
Hereinafter, a third embodiment of the face detection process will be described. The configuration of the imaging device and the main operation of the imaging device are the same as those in the first embodiment shown in FIGS.

  FIG. 6 is a flowchart for explaining the operation of the face detection process in the third embodiment. In the method of detecting the face area by detecting the nose, mouth, and face outline, if the face in the image data is tilted, there are cases where these characteristic parts cannot be identified. In order to reduce detection omissions, a method of detecting a face area while rotating image data that is a detection target of the face area little by little can be considered.

  In step S401, the system control circuit 112 determines whether or not the face detection level is 1. If the face detection level is 1, the process proceeds to step S402. If the face detection level is not 1, the process proceeds to step S410.

  In step S402, the system control circuit 112 sets a detection range of the face tilt angle.

  Specifically, the angle is −45 degrees to +45 degrees.

  In step S403, the system control circuit 112 rotates the image by −45 degrees.

  In step S404, the system control circuit 112 performs feature extraction. The feature extraction method is the same as in the first embodiment.

  In step S405, the system control circuit 112 rotates the image by +5 degrees.

  In step S406, the system control circuit 112 determines whether or not the detection has been completed up to +45 degrees. If it has been completed, the process proceeds to step S407, and if not, the process returns to step S404.

  In step S410, the system control circuit 112 sets the detection range of the face tilt angle.

  Specifically, it is set to −180 degrees to +180 degrees.

  In step S411, the system control circuit 112 rotates the image by −180 degrees.

  In step S412, the system control circuit 112 performs feature extraction. The feature extraction method is the same as in the first embodiment.

  In step S413, the system control circuit 112 rotates the image by +5 degrees.

  In step S414, the system control circuit 112 determines whether or not the detection has been completed up to +180 degrees. If completed, the process proceeds to step S407, and if not completed, the process returns to step S412.

  As described above, in the present embodiment, in the face detection operation before the shooting operation, specifically, in the face detection operation in the standby state (when displaying live view), the face detection level is set to 1 in step S110, In next step S111, face detection is performed. In the face detection performed in step S111, the detection range of the face inclination is limited to −45 degrees to +45 degrees (step S402), and feature extraction is performed (step S404). Further, in the face detection operation after the photographing operation, specifically, in the face detection operation in the photographed image (image reproduction), the face detection level is set to 2 in step S119, and the face detection is performed in the next step S120. . In the face detection performed in step S120, the detection range of the face inclination is not limited, and feature extraction is performed from −180 degrees to +180 degrees (step S410) (step S412).

  As described above, the face detection operation before the photographing operation, more specifically, the face detection operation in the standby state, can reduce the processing time by limiting the detection range of the face inclination. The followability to the movement can be improved.

  Further, in the face detection operation after the photographing operation, specifically, the face detection operation on the photographed image does not limit the detection range of the face inclination, a highly accurate face detection result can be obtained.

(Fourth embodiment)
In the embodiment described above, the camera automatically switches between the face detection operation in the standby state and the face detection operation after the shooting operation before the shooting operation. Thus, the user may arbitrarily switch.

(Other embodiments)
In the above-described embodiment, the face detection operation performed before the shooting operation and the face detection operation performed when image data is displayed immediately after shooting are described as examples of different face detection levels. Yes, but not limited to this. Regardless of the shooting operation, the face detection level of the face detection operation performed in the reproduction mode in which the image data stored in the storage medium is read and arbitrary image data is displayed is set to 2, and the face detection level in the standby state is set to 1. Also good. This is because it is not necessary to obtain a face detection result in real time immediately after shooting or in the playback mode, and there is no problem in the face detection operation before the shooting operation that affects the focus control process or the exposure control process. Therefore, it is desirable to prioritize accuracy over time required for face detection operation immediately after shooting or in a state where the playback mode is set.

  Further, although the description has been given taking a human face as an example of the detection target, the present invention is not limited to this. For example, in the first embodiment, since the areas of the detection range are made different, for example, a method for detecting a region satisfying a predetermined color condition as a target, or a target for a region satisfying a predetermined luminance condition Even if it is the method of detecting as, it is possible to apply. In the second and third embodiments, for example, a detection method that targets a region that satisfies a predetermined shape condition that affects detection time and detection accuracy depending on the resolution and orientation of the target object. It is possible to apply. The target object is not limited to a person's face, but can be limited to a specific individual, a child, or an animal other than a human depending on the purpose of detection.

  Further, as the method for setting the face detection level, the area of the detection range, the number of pixels, and the allowable range of the inclination of the subject have been described as examples, but other methods are also conceivable. For example, a method of changing the number of conditions necessary for recognizing the face as a detection target, or a method of changing the upper limit value and the lower limit value of the detectable face size can be considered.

  The object of each embodiment is also achieved by the following method. That is, a storage medium (or recording medium) in which a program code of software that realizes the functions of the above-described embodiments is recorded is supplied to the system or apparatus. Then, the computer (or CPU or MPU) of the system or apparatus reads and executes the program code stored in the storage medium. In this case, the program code itself read from the storage medium realizes the functions of the above-described embodiments, and the storage medium storing the program code constitutes the present invention. Further, by executing the program code read by the computer, not only the functions of the above-described embodiments are realized, but the present invention includes the following cases. That is, based on the instruction of the program code, an operating system (OS) running on the computer performs part or all of the actual processing, and the functions of the above-described embodiments are realized by the processing.

  Furthermore, the following cases are also included in the present invention. That is, the program code read from the storage medium is written in a memory provided in a function expansion card inserted into the computer or a function expansion unit connected to the computer. Thereafter, based on the instruction of the program code, the CPU or the like provided in the function expansion card or function expansion unit performs part or all of the actual processing, and the functions of the above-described embodiments are realized by the processing.

  When the present invention is applied to the above storage medium, the storage medium stores program codes corresponding to the procedure described above.

1 is a block diagram illustrating a configuration of an imaging apparatus according to a first embodiment of the present invention. It is a figure which shows a part of component of the digital signal processing circuit of the imaging device in the 1st Embodiment of this invention. It is a flowchart for demonstrating the operation | movement in the main process routine in the 1st Embodiment of this invention. It is a flowchart for demonstrating the operation | movement of the face detection in the 1st Embodiment of this invention. It is a flowchart for demonstrating the operation | movement of the face detection in the 2nd Embodiment of this invention. It is a flowchart for demonstrating the operation | movement of the face detection in the 3rd Embodiment of this invention. It is a figure which shows the example of a display of an identification area | region. It is a figure which shows image data. It is a figure which shows the face detection range in the 1st Embodiment of this invention. It is a figure which shows the relationship between the face detection range and the position of a feature extraction area | region in the 1st Embodiment of this invention. It is a figure which shows the face detection range in the 1st Embodiment of this invention. It is a figure which shows the relationship between the face detection range and the position of a feature extraction area | region in the 1st Embodiment of this invention.

Explanation of symbols

DESCRIPTION OF SYMBOLS 100 Image pick-up device 101 Objective lens group 102 Light quantity adjustment device 103 Image pick-up element 104 Analog signal processing circuit 105 A / D converter 106 Memory control circuit 107 Digital signal processing circuit 108 Memory 109 D / A converter 110 Display device 111 Interface 112 System control Circuit 113 Exposure control circuit 114 Focus control circuit 115 Subject identification circuit 116 Posture detection circuit 117 Display control circuit

Claims (11)

Detecting means for detecting the position of the target object from the image data;
Display means for displaying an image using image data generated by the image sensor;
Display control means for causing the display means to display a display indicating the position of the object detected by the detection means;
Storage means for storing image data generated by the image sensor,
In the case where the detection unit detects the position of the target from the image data generated by the image sensor before the photographing operation for storing the image data generated by the image sensor in the storage unit, Compared with the case where the position of the target is detected from stored image data, at least the range of the image data for detecting the target is set narrow, and the range of the tilt of the target that can be detected imaging device and performing either to the narrow configuration.   The display means displays an image using image data stored in the storage means by the photographing operation, and the detection means detects the position of the object from the image data used for the display. Item 2. The imaging device according to Item 1.   2. The display unit according to claim 1, wherein the display unit reproduces the image data stored in the storage unit to display an image, and the detection unit detects the position of the target from the image data used for the display. The imaging device described. Detecting means for detecting the position of the target object from the image data;
Display means for displaying an image using image data generated by the image sensor;
Display control means for causing the display means to display a display indicating the position of the object detected by the detection means;
A shutter switch for instructing the storage means to store the image data;
When the detection unit detects the position of the target from image data generated by the image sensor before the shutter switch is operated, the image generated by the image sensor after the shutter switch is operated. Compared to the case where the position of the target is detected from data, at least a range of the image data for detecting the target is set narrowly, and a range of the inclination of the target that can be detected is set narrowly imaging device and performs either.   The display means displays an image using image data generated by the image sensor after the shutter switch is operated, and the detection means detects the position of the target from the image data used for the display. The imaging apparatus according to claim 4.   The imaging apparatus according to claim 1, further comprising a focus control unit that performs focus control using image data of the position of the target detected by the detection unit.   The imaging apparatus according to claim 1, further comprising an exposure control unit that performs exposure control using image data of the position of the target detected by the detection unit. A detection step of detecting the position of the target object from the image data;
A display step of displaying an image on display means using image data generated by the image sensor;
A display control step of displaying on the display means a display indicating the position of the target detected in the detection step;
A photographing step of storing image data generated by the image sensor in a storage means,
In the detection step, when the position of the target is detected from image data generated by the imaging device before the photographing step, the position of the target is detected from the image data stored in the storage unit. compared to the case, and wherein at least, it sets a narrow range of the image data to detect the target, and, to do any of the possible sets a narrow range of the target slope to be detected Control method for imaging apparatus. A detection step of detecting the position of the target object from the image data;
A display step of displaying an image on display means using image data generated by the image sensor;
A display control step of displaying on the display means a display indicating the position of the target detected in the detection step;
A photographing step of storing image data in the storage means according to the operation of the shutter switch,
In the detection step, when the position of the target is detected from image data generated by the image sensor before the shutter switch is operated, an image generated by the image sensor after the shutter switch is operated Compared to the case where the position of the target is detected from data, at least a range of the image data for detecting the target is set narrowly, and a range of the inclination of the target that can be detected is set narrowly control method for an imaging apparatus which is characterized in that one of the.   A program for causing a computer to execute the control method according to claim 8 or 9.   A computer-readable storage medium storing the program according to claim 10.

Images