JP2010028418A - Imaging apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To achieve the high-precision continuous setting of a face region in image data successively formed from a CCD image sensor even if the face of a person is turned from a front to a side. <P>SOLUTION: A digital camera 1 includes: a CCD image sensor 12; a flash memory 16 where face information is stored; a controller 17 for determining whether the face information exists or not in the image data formed from the CCD image sensor 12; a controller 17 for setting a face region in a position where the face information is determined to exist in the image data when it is determined that the face information exists, and storing the color information of the image data in the face region in a buffer memory 15; and a controller 17 for setting the face region in the image data based on the color information when it is determined that the face information does not exist and when the color information is stored in the buffer memory 15. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to an imaging apparatus such as a digital camera.

  There exist some which are described in patent document 1 in the conventional imaging device. The imaging apparatus disclosed in Patent Document 1 is “to enable tracking of a subject even when a situation occurs in which a face to be tracked cannot be detected in a moving image”.

The imaging apparatus disclosed in Patent Document 1 “estimates the approximate position of the current face based on the position of the face detected in the past, acquires a plurality of surrounding information corresponding to the approximate position, and is acquired in the past. The current face position is estimated on the basis of the position where the surrounding information most similar to the surrounding information is acquired. "
JP 2007-42072 A

  As described above, in a general imaging apparatus, it is possible to detect a person's face by pattern matching using information related to a face (hereinafter referred to as face information).

  However, the imaging device cannot detect a face from image data generated by the imaging device if a person facing the front of the camera faces sideways. That is, the imaging apparatus can detect a face with high accuracy for a person facing forward, but cannot detect a face with high accuracy for a person facing sideways. This is because template information such as the shape of the face, the color of the face, and the positions of the eyes and nose is used for the face information.

  In short, in pattern matching, a subject can be detected with high accuracy when the subject faces a predetermined direction. However, when the subject faces in a direction other than the predetermined direction, the subject detection accuracy becomes extremely low.

  Therefore, in order to solve the above-described problem, the present invention is directed to image data that is sequentially generated from the imaging unit, even when the subject is facing a predetermined direction, even when the subject is facing a direction other than the predetermined direction. An object of the present invention is to provide an imaging device capable of continuously setting a region related to a high accuracy.

That is, the imaging apparatus of the present invention captures a subject image formed by an optical system and sequentially generates image data, a storage medium storing first feature information indicating the characteristics of the subject,
In the image data generated from the imaging means, when it is determined that the first feature information is present by the determining means for determining whether or not the stored first feature information exists, and the determining means, A feature region is set at a position where it is determined that the first feature information exists in the image data, and second feature information indicating the feature of the image data in the set feature region is stored in the storage medium. If the first feature information is determined not to exist by the first setting means and the determination means, and the second feature information is stored in the storage medium, the stored second feature Second setting means for setting the feature region in the image data based on the information.

  In this way, even if the first feature information does not exist in the image data, the feature region can be set based on the second feature information.

  Preferably, the second setting means determines whether to set a feature region based on the second feature information according to the content of the stored second feature information, and sets the feature region to If it is determined to set, the feature region is set in the image data based on the stored second feature information.

  Preferably, the image processing apparatus further includes first counting means for counting the number of times that the first setting means has continuously set the characteristic area for the image data sequentially generated from the imaging means as the first setting number. Then, the second setting means determines whether to set a feature area based on the second feature information according to the number of times set by the first counting means, and determines to set the feature area. In this case, the feature region is set in the image data based on the stored second feature information.

  Preferably, for the image data sequentially generated from the imaging unit, the time from when the first setting unit sets the feature region until the feature region is not set is counted as the first set time. First timing means is further provided. The second setting means determines whether or not to set a feature area based on the second feature information according to the first set time counted by the first timing means, and sets the feature area. If so, the feature region is set in the image data based on the stored second feature information.

  Preferably, the apparatus further includes luminance acquisition means for acquiring information related to the brightness of the shooting environment. Then, the second setting unit determines whether to set a feature region based on the second feature information according to the information on the brightness acquired by the luminance acquisition unit, and determines to set the feature region. In this case, the feature region is set in the image data based on the stored second feature information.

  Preferably, the first setting means is designated by a position where it is determined that the first feature information exists in the image data when the determination means determines that the first feature information exists. The third feature information indicating the feature of the image data in the specific area is stored in the storage medium. Then, the second setting means determines whether to set a feature region based on the second feature information according to the content of the stored third feature information, and determines to set the feature region. In this case, the feature region is set in the image data based on the stored second feature information.

  Preferably, the image processing apparatus further includes second counting means for counting the number of times that the second setting means has continuously set the characteristic area as the second setting number for the image data sequentially generated from the imaging means. The second setting means determines whether or not to set a feature area based on the second feature information according to the second set count counted by the second counting means. If it is determined to set, the feature region is set in the image data based on the stored second feature information.

  Preferably, for the image data sequentially generated from the imaging unit, the time from when the second setting unit sets the feature region until the feature region is not set is counted as a second set time. Second timing means is further provided. The second setting means determines whether or not to set a feature area based on the second feature information according to a second set time counted by the second timing means, and sets the feature area If so, the feature region is set in the image data based on the stored second feature information.

  Preferably, the second setting means further determines whether or not to set the feature region based on the second feature information according to a position where the feature region based on the second feature information is set, and If it is determined that a feature area is to be set, the feature area is set in the image data based on the stored second feature information.

  Preferably, the information processing apparatus further includes movement information acquisition means for acquiring information related to movement of the own apparatus. Then, the second setting means determines whether or not to set a feature area based on the second feature information according to the information related to movement acquired by the movement amount acquisition means, and determines to set the feature area. In this case, the feature region is set in the image data based on the stored second feature information.

  Preferably, the first feature information is information relating to a face, and the second feature information is information relating to a color.

  Preferably, the feature area is a face area.

  That is, according to the imaging apparatus of the present invention, when the first feature information does not exist in the image data, the feature region is set based on the second feature information, so that the subject is facing the predetermined direction. Even when facing in a direction other than a predetermined direction, the area related to the subject can be continuously set with high accuracy for the image data sequentially generated from the imaging means.

Hereinafter, an embodiment of the present invention will be described with reference to the drawings. An example in which the present invention is applied to a digital camera will be described.
(Embodiment 1)
1. Overview of Digital Camera The digital camera 1 determines whether or not face information stored in advance in the flash memory 16 is present in the image data generated from the CCD image sensor 12 (detects a face from the image data). To do). When the digital camera 1 determines that one piece of face information exists in the image data (when one face can be detected), the digital camera 1 sets the face area, and the characteristics of the image data in the first face area are set. The indicated color information is stored in the buffer memory 15. On the other hand, when the digital camera 1 determines that face information does not exist in the image data and the color information is stored in the buffer memory 15, a face area is set in the image data based on the color information. To do.

  If the digital camera 1 determines that there are a plurality of pieces of face information in the image data, a plurality of face areas are set in the image data. The digital camera 1 selects a main subject in the set face area. The digital camera 1 stores color information indicating the characteristics of image data in the face area selected as the main subject in the buffer memory 15.

2. Configuration FIG. 1 is a perspective view of a digital camera 1 according to the present embodiment. FIG. 2 is a block diagram showing a configuration of the digital camera 1 according to the present embodiment. The configuration will be specifically described below.

  The optical system 11 collects optical signals from the subject and forms a subject image on the CCD image sensor 12. The optical system 11 includes an objective lens 111, a zoom lens 112, a focus lens 113, and an aperture / shutter 114.

  The zoom lens 112 is driven by the zoom driving unit 115 and adjusts the range (view angle) in which the subject image is captured. That is, the zoom lens 112 can adjust the focal length. The focus lens 113 is driven by the focus drive unit 116 to adjust the focus. The zoom drive unit 115 includes a cam mechanism for moving the zoom lens 112 and an actuator for driving the cam mechanism, and drives the zoom lens 112 in accordance with control from the controller 17. The focus driving unit 116 includes a cam mechanism for moving the focus lens 113, an actuator for driving the cam mechanism, and the like. The focus driving unit 116 drives the focus lens 113 in accordance with control from the controller 17.

  The diaphragm / shutter 114 is a member that serves as both a diaphragm and a shutter. The aperture / shutter 114 is driven by an aperture / shutter driver 117. The stop is composed of five blades and adjusts the amount of light passing through the optical system 11. The shutter is opened and closed to adjust the amount of light hitting the CCD image sensor 12 with time. The aperture / shutter drive unit 117 includes a cam mechanism and an actuator for driving the cam mechanism, and drives the aperture / shutter in accordance with control from the controller 17.

  In the present embodiment, the diaphragm / shutter 114 that serves both as a diaphragm and a shutter is provided. However, the present invention is not limited to this, and the diaphragm and the shutter may be provided separately. In this case, for example, the shutter is not provided in the optical system, and only the diaphragm is provided in the optical system. At this time, the shutter may be provided between the optical system and the CCD image sensor. In this way, since the shutter is provided by a member different from the optical system, it can be applied to a lens exchange type digital camera.

  The CCD image sensor 12 captures a subject image formed by the optical system 11 and generates image data. The CCD image sensor 12 captures a subject image based on the pulse signal supplied from the timing generator 121 and generates image data. The timing generator 121 is configured by an integrated circuit such as an LSI and supplies a pulse signal to the CCD image sensor 12. Here, the pulse signal that the timing generator 121 supplies to the CCD image sensor 12 supplies, for example, 30 frame readout pulse signals per second. As a result, the CCD image sensor 12 can acquire image data of a frame every 1/30 second.

  The AD converter 13 converts the image data generated by the CCD image sensor 12 into digital data.

  The image processing unit 14 performs predetermined processing on the image data converted into digital data. As the predetermined processing, gamma conversion, YC conversion, electronic zoom processing, compression processing, expansion processing, and the like can be considered, but are not limited thereto.

  The buffer memory 15 functions as a work memory when the image processing unit 14 performs image processing and when the controller 17 performs control processing. The buffer memory 15 can be realized by, for example, a DRAM.

  The flash memory 16 is used as a built-in memory. The flash memory 16 can store programs for controlling the controller 17 and setting values in addition to the image processed image data. In the present embodiment, face information is stored. The face information is used to detect a person's face from the image data. That is, the face information is used when determining whether face information exists in the image data. As the face information, for example, template information corresponding to the contour of the entire face and facial components (eyes, nose, ears, etc.) can be considered.

The controller 17 controls each part of the digital camera 1. The controller 17 may be realized by a microcomputer or a hard-wired circuit. Various controls of the controller 17 are as follows . This will be described in detail in various controls of the controller .

  The shake detector 30 is composed of a gyro sensor or the like, and detects vibration of the own device. The shake detector 30 detects a first gyro sensor that detects vertical rotational vibration about the horizontal axis of the digital camera 1 and a first rotational sensor that detects horizontal rotational vibration about the vertical axis of the digital camera 1. 2 gyro sensors. That is, the shake detector 30 detects the motion angular velocity of the digital camera 1. The controller 17 obtains the motion angular velocity from the shake detector 30 and integrates this angular velocity to calculate the angular velocity of the digital camera 1. This angular velocity is the shake amount of the digital camera 1. The shake amount is calculated by a value such as −300 to 300 deg / sec, for example.

  The card slot 18 is a slot for inserting and removing the memory card 19. The card slot 18 may have a function of controlling the memory card 19. The memory card 19 includes a flash memory or the like. The memory card 19 can store the image data image-processed by the image processing unit 14.

  The liquid crystal monitor 20 displays image data and various settings of the digital camera 1. Note that an organic EL display or the like can be used instead of the liquid crystal monitor 20.

  The operation unit 21 is a general term for operation members attached to the exterior of the digital camera 1. The operation unit 21 may be a cross key or a push button.

  In the present embodiment, there is a shutter switch 211 as one push button of the operation unit. The shutter switch 211 is provided on the upper surface of the digital camera 1 and can detect half-press and full-press from the user. When a half-press or full-press operation is performed by the user, the shutter switch 211 outputs an operation signal corresponding to the operation to the controller 17.

  In the present embodiment, a zoom lever 212 is provided as one member of the operation unit 21. The zoom lever 212 changes the focal length of the digital camera 1 in accordance with an operation from the user. The zoom lever 212 outputs an operation signal to the controller 17 when operated by the user. As a result, the controller 17 can send a control signal for changing the focal length to the zoom lens 112.

3. Based on the face information stored in the flash memory 16, the various control controllers 17 of the controller determine whether or not face information exists in the image data generated by the CCD image sensor 12.

  When the controller 17 determines that one or more pieces of face information exist in the image data, the controller 17 sets a face area in the image data. Thereafter, the controller 17 stores the color information corresponding to the image data in the face area in the buffer memory 15. On the other hand, when the controller 17 determines that face information does not exist in the image data and color information is stored in the buffer memory 15, the controller 17 sets a face area in the image data based on the color information.

  When the face area is set in the image data, the controller 17 causes the liquid crystal monitor 20 to display the image data in which the face area is set.

3-1. Determination of whether face information is present in image data The controller 17 determines whether face information stored in the flash memory 16 is present in the image data. Specifically, the controller 17 reads face information from the flash memory 16. The controller 17 determines whether or not face information exists in the image data by template matching. Since template matching is a conventional technique, description thereof is omitted.

3-2 When the controller 17 determines that there is one face information in 3-1, the controller 17 determines in the image data that there is one face information in the image data. A face area is set at a position where it is determined that face information exists (FIG. 3A). Thereafter, the controller 17 stores color information corresponding to the image data in the face area in the buffer memory 15 (FIG. 3A). At this time, the controller 17 stores the position where the face area is set in the image data in the buffer memory 15 as the set position. If the color information and the setting position are stored in association with each other, the controller 17 can easily find out when using the information. As the setting position, position information on the upper left end of the face area may be used.

  In this embodiment, the color information in the image data in the face area is obtained by converting each pixel of the image data into a hue color space. Here, the hue can be expressed by an angle as a phase angle of 0 to 360 degrees. That is, when the color space is the HSV color space, it can be expressed by a positional relationship of R = 0 / G = 120 / B = 240 as shown in FIG. Therefore, for example, in FIG. 4, the angle of the corresponding color is θ. Thereby, the color information in the image data can be expressed by a histogram. In the present embodiment, 0 to 359 degrees are divided into 40, and the number of pixels existing in each region is calculated. The horizontal axis of the histogram in FIG. 5 represents divided areas (“1” to “40”, for example, “1” corresponds to 0 to 8 degrees), and the vertical axis exists in the first face area. This represents the number of pixels in each area.

3-3 When the controller 17 determines that a plurality of face information is present in 3-1, the controller 17 determines that a plurality of face information is present in 3-1. And determine the main subject from the face area. The method for determining the main subject is the same as in the conventional technique. For example, an area closest to the center position of the image in the image data in the face area may be selected as the main subject area. An area having a large size may be set as the area of the main subject.

  When the controller 17 selects the face area of the main subject, the controller 17 stores color information indicating the characteristics of the image data in the face area of the main subject in the buffer memory 15. At this time, the position where the face area is set in the image data is stored in the buffer memory 15 as the set position.

  Note that color information and setting positions may be stored for all or some of the plurality of face regions.

3-4 When the controller 17 determines that face information does not exist in 3-1, the controller 17 determines that face information does not exist in 3-1, and the color information is stored in the buffer memory 15. If so, the face area is set based on the color information.

  Specifically, the controller 17 reads color information and a set position from the buffer memory 15. Next, as shown in FIG. 3B, the controller 17 searches for an area closest to the color information in the image data included in a predetermined area (hereinafter referred to as a search area) from the set position. The method for searching for the area that most closely approximates the color information is a conventional technique, and therefore details are omitted. Thereafter, the controller 17 sets the face area in the area where the color information is most approximated in the image data in the search area.

  Note that the controller 17 does not set the second face area when it is determined in 3-1 that the face area does not exist and color information is not stored in the buffer memory 15. The controller 17 may not set the second face area when there is no color information approximated at a predetermined level or higher in the search area of the image data.

4). The comparison digital camera 1 according to the embodiment and the present invention is an example of an imaging apparatus according to the present invention. The CCD image sensor 12 is an example of an imaging unit of the present invention. The buffer memory 15 and the flash memory 16 are examples of the storage medium of the present invention. The controller 17 is an example of a determination unit of the present invention. The controller 17 is an example of a first setting unit of the present invention. The controller 17 is an example of a first setting unit of the present invention. The face information is an example of first feature information of the present invention. The color information is an example of second feature information of the present invention. The face area is an example of the characteristic area of the present invention.

5. Operation The operation of the digital camera 1 configured as described above will be described. This operation example is performed when the digital camera 1 is set to a subject photographing mode such as a portrait mode after the power of the digital camera 1 is changed from OFF to ON.

5-1. Basic operation (Fig. 6)
First, the controller 17 controls the CCD image sensor 121 to drive the CCD image sensor 12. The CCD image sensor 12 generates image data (A1). Next, the controller 17 reads face information from the flash memory 16. The controller 17 determines whether or not face information exists in the image data generated by the CCD image sensor 12 (A2).

  If the controller 17 determines that face information exists in the image data, the controller 17 sets a face area in the image data (A3). The controller 17 determines whether or not a plurality of face information exists in the image data (A4). If the controller 17 determines that there are a plurality of face areas in the image data, the controller 17 selects a face area to be the main subject from the plurality of face areas (A5).

  When one face information is present, the controller 17 relates to the color information and setting of the image data in the face area regarding the face area or the face area selected as the main subject when there are a plurality of face information. The position is stored in the buffer memory 15. That is, the controller 17 stores the color information indicating the characteristics of the image data in the face area and the set position of the face area in the image data in the buffer memory 15 (A6). Next, the process proceeds to step A9.

  On the other hand, if it is determined in step A2 that face information does not exist in the image data, the controller 17 determines whether color information is stored in the buffer memory 15 (A7). If the color information is not stored in the buffer memory 15, the controller 17 shifts the process to step A1. On the other hand, when the color information is stored in the buffer memory 15, the controller 17 reads the color information and the set position from the buffer memory 15. Then, the controller 17 sets a face area in the image data based on the color information and the setting position (A8).

  Thereafter, the controller 17 displays the image data in which the face area is set on the liquid crystal monitor 20 (A9). Then, the controller 17 returns to step A1, and causes the CCD image sensor 12 to generate the next image data.

  An operation example will be described with reference to FIG. First, as shown in FIG. 7A, when face information exists in the image data, the controller 17 sets a face area in the image data based on the face information. At this time, the controller 17 stores the color information corresponding to the image data in the face area in the buffer memory 15. Next, in the image data generated from the CCD image sensor 12 as well, as shown in FIG. 7B, a face area is set in the image data based on the face information. At this time, the controller 17 stores the color information corresponding to the image data in the face area in the buffer memory 15. FIG. 7C is similar to FIGS. 7A and B. Then, in the image data generated from the CCD image sensor 12, when the controller 17 determines that no face area exists in the image data as shown in FIG. 7D, the controller 17 stores the color stored in the buffer memory 15. A face area is set based on the information. Accordingly, the digital camera 1 can set the face area in the image data based on the color information even when it is determined that the face area does not exist in the image data.

  Here, in the above operation, it is determined whether or not the face area can be set with the face information. If the face area cannot be set, the face area is set with the color information. In this way, since tracking of face information can be prioritized over tracking of color information, it is possible to further reduce tracking of a subject that is different from the subject to be tracked, such as tracking of color information. That is, when the face area is detected in the image data, the color information is reset (stored), and therefore, when setting the face area based on the color information, it is easy to continuously set the face area for a desired subject. Become. That is, the tracking accuracy can be improved.

Although the basic operation has been described in “ 5-1. Basic operation (FIG. 6) ”, in the following cases, when face information does not exist in the image data (when a face cannot be detected from the image data) ), The face area may be set based on the color information.

5-2. Operation considering color information (Fig. 8)
In the above configuration, the digital camera 1 determines whether face information is present in the image data. However, the determination of whether or not face information exists in the image data (hereinafter referred to as face detection) may erroneously detect a non-face. Therefore, when the face cannot be detected from the image data, the digital camera 1 determines whether or not to set the face area based on the color information according to the content of the color information stored in the buffer memory 15. When the digital camera 1 determines that the face area is set based on the color information, the digital camera 1 sets the face area in the image data based on the color information.

  When the controller 17 determines that the face information does not exist in the image data, and the color information stored in the buffer memory 15 includes information on the skin color at a predetermined ratio or more (for example, 40% or more), It is determined that the face area is set based on the color information. In the case of the histogram of FIG. 5, for example, a divided region “2 to 4” (hue angle 9 to 35 degrees) can be used as the skin color information. For this reason, if there are 40% or more of all the pixels of the divided region “2-4” in the histogram, it is determined that the face region is set based on the color information.

  An operation example in this case is as shown in FIG. That is, the determination condition in step A7 is step B7. In step B <b> 7, the controller 17 determines whether or not the color information is stored in the buffer memory 15 and the content of the color information includes 40% or more of information related to the skin color. Steps B1 to B9 other than step B7 are the same processes as steps A1 to A9.

5-3. An operation that takes into account the number of times the face area is continuously set based on face information (FIG. 9)
In the above configuration, the digital camera 1 performs face detection on the image data sequentially generated by the CCD image sensor 12. However, face detection may detect a face instantaneously (for example, only for three frames). In such a case, a non-face is often detected as a face. Therefore, the digital camera 1 sets the face area based on the color information when the face cannot be detected from the image data and the face is detected stably. That is, the digital camera 1 counts the number of times that the face area is continuously set based on the face information with respect to the image data sequentially generated from the CCD image sensor 12 as the face setting number (the first embodiment of the present invention An example of one counting means). Further, when the face cannot be detected from the image data, the digital camera 1 determines whether or not to set the face area based on the color information according to the number of face settings. When the digital camera 1 determines that the face area is set based on the color information, the digital camera 1 sets the face area in the image data based on the color information.

  The controller 17 determines that the face area is set based on the color information when it is determined that the face information does not exist in the image data and the face setting count is equal to or more than a predetermined number (30 times or more). The number of face settings is stored in the buffer memory 15. In addition, flag information for determining whether or not the face area is continuously set is stored in the buffer memory 15. The flag information is used to determine whether or not a face area has been set based on face information in the previous image data. The flag information has “1” and “0” as information. When the flag information is “1”, it indicates that the face area is set based on the face information for the immediately preceding image data. This indicates that the face area is set based on the color information for the immediately preceding image data.

  An example of the operation in this case will be described with reference to FIG. Description of operations similar to those in FIG. 6 is omitted. When starting this operation, the number of face settings stored in the buffer memory 15 is set to “0”. The flag information stored in the buffer memory 15 is set to “0”.

  First, the controller 17 performs the operations of Steps C1 to C6 similar to the operations of Steps A1 to A6. After step C6, the controller 17 determines whether the setting of the face area based on the face information is continuous (C7). That is, based on the flag information stored in the buffer memory 15, the controller 17 determines whether or not a face area based on face information has been set for the immediately preceding image data. Accordingly, when the flag information is “1”, the controller 17 adds “1” to the number of face settings, assuming that the setting of the face area is continuous (C8). On the other hand, when the flag information is “0”, the controller 17 sets “1” as the number of face settings stored in the buffer memory 15 on the assumption that the setting of the face area is not continuous (C9). After step C8 or C9, the controller 17 sets “1” in the flag information stored in the buffer memory 15 (C10). Then, the controller 17 performs the same operation as in Step A9 (C14).

On the other hand, if it is determined in step C2 that face information does not exist in the image data, the controller 17 sets “0” in the flag information stored in the buffer memory 15 (C11). After this, the controller 17
It is determined whether color information is stored in the buffer memory 15 and the number of face settings stored in the buffer memory 15 is “30” or more (C12). If the color information is not stored in the buffer memory 15 or the number of face settings stored in the buffer memory 15 is less than “30” in Step C11, the controller 17 proceeds to Step C1. On the other hand, when the color information is stored in the buffer memory 15 and the number of face settings stored in the buffer memory 15 is “30” or more in Step C11, the controller 17 performs the same operation as in Step A8. Perform (C13). Thereafter, the controller 17 performs the same operation as in Step A9 (C14).

  In this operation example, it is determined whether to set a face area based on the color information according to the number of face settings. However, the present invention is not limited to this, and it may be determined whether or not to set the face area based on the color information according to the face setting time (the time during which face information can be detected continuously). In this case, the digital camera 1 uses a controller 17 that counts the time for which the face area is continuously set based on the face information as the face setting time for the image data sequentially generated from the CCD image sensor 12 (first aspect of the present invention). An example of one time measuring means). In the case of this operation, the calculation may be performed by converting the number of times of face setting to 1/30 seconds (reading time of one frame), for example. Therefore, in this operation example, the face setting time is stored in the buffer memory 15, and “1/30” is added to the face setting time in step C8. In step C9, “Set the face setting time to“ 1/30 ”. In Step C12, “Is color information stored and face setting time is“ 1 ”or more?” ”

5-4. Operation considering information around the position where the face area is set based on the face information (FIG. 10)
Generally, there is a possibility that exposed arms and shoulders having a skin color exist around the face of a person. In such a case, when tracking the face area with color information, the digital camera may determine a non-face part (exposed arm, shoulder, etc.) as a face and set the face area. Therefore, the digital camera 1 acquires color information (specific information) from the periphery (specific area) of the face area set based on the face information for the immediately preceding image data. Further, when the face cannot be detected from the image data, the digital camera 1 determines whether or not to set the face area based on the color information according to the specific information acquired from the specific area. When the digital camera 1 determines that the face area is set based on the color information, the digital camera 1 sets the face area in the image data based on the color information.

  When the face area is set based on the face information, the controller 17 acquires the specific information of the image data in the specific area around the face area. The specific area of the present embodiment is a surrounding area excluding a portion adjacent to the face area. For example, in the case of FIG. 11A, the face area 50A and the specific area 60A are obtained. In the case of FIG. 11B, the face area 50B and the specific area 60B are obtained. It should be noted that the reason why the part adjacent to the face area is excluded from the specific area is that there is a neck under the face.For example, when information on this neck is acquired, it is determined whether there are exposed arms and chests around the face. This is because it cannot be done. The digital camera 1 stores the acquired specific information in the buffer memory 15.

  If the controller 17 determines that face information does not exist in the image data, and the specific information stored in the buffer memory 15 includes skin color information at a predetermined rate (5%) or more, the controller 17 is based on the color information. It is determined that no face area is set. For example, if information about color is acquired from image data in the specific area as shown in FIG. 11A as the specific information of the specific area, the controller 17 determines to set the face area based on the color information. On the other hand, if information about color is acquired from the image data in the specific area as shown in FIG. 11B, the controller 17 determines that the face area is not set based on the color information. In the case of FIG. 11B, since the subject's hand is in the specific area, it is determined that the specific information in the specific area includes the skin color and the face area is not set based on the color information.

  An operation example in this case will be described with reference to FIG. Description of operations similar to those in FIG. 6 is omitted.

  First, the controller 17 performs the same operation as the operations in steps A1 to A6 (D1 to D6). After the step D6, the controller 17 specifies information on the color of the image data in the specific area specified by the face area. Obtained as information (D7). Then, the controller 17 performs the same operation as in Step A9 (D10).

  On the other hand, if it is determined in step D2 that face information does not exist in the image data, the controller 17 stores the color information in the buffer memory 15, and in the specific information stored in the buffer memory 15, information on the skin color Is less than a predetermined ratio (D8). In step D8, if the color information is not stored in the buffer memory 15 or if the specific information stored in the buffer memory 15 is equal to or higher than the predetermined ratio, the controller 17 proceeds to step D1. On the other hand, when the color information is stored in the buffer memory 15 in step D8, and the specific information stored in the buffer memory 15 is less than the predetermined ratio, the controller 17 is the same as in step A8. The operation is performed (D9). Thereafter, the controller 17 performs the same operation as in Step A9 (D10).

  It should be noted that the specific area may be made larger or smaller depending on the size of the face area. This is because when the face area is large, the body of the subject is also moved to image data.

5-5. Operation in consideration of information on brightness of shooting environment (FIG. 12)
When the shooting environment becomes dark, the reliability of the information about the color decreases. That is, in such a case, if the face area is set based on the color information, the accuracy in setting the face area is lowered. Therefore, the digital camera 1 acquires information regarding the brightness of the shooting environment. That is, the digital camera 1 includes a controller 17 (an example of the luminance acquisition unit of the present invention) that acquires information related to the brightness of the shooting environment from the image data generated by the CCD image sensor 12. Further, when the face cannot be detected from the image data, the digital camera 1 determines whether or not to set a face area based on the color information according to the acquired information regarding brightness. When the digital camera 1 determines to set a face area based on the color information, the digital camera 1 sets the face area in the image data based on the color information.

  The controller 17 acquires information about brightness from the image data. This information on brightness is luminance information. The controller 17 receives the image data YC-converted by the image processing unit 14, and calculates an average value of the luminance of the image from this image data. Since the method for calculating the average value of luminance is a conventional technique, description thereof is omitted. As a result, the controller 17 acquires luminance information. For example, when the luminance information is expressed by 8 bits, the luminance information can be expressed by “0” to “255”.

  If the controller 17 determines that face information does not exist in the image data and the luminance information is less than a predetermined value (for example, “30”), the controller 17 does not set the face area based on the color information. To. That is, the controller 17 sets the face area based on the color information when the luminance information is greater than or equal to a predetermined value.

  The operation will be described with reference to FIG. Description of operations similar to those in FIG. 6 is omitted.

First, the controller 17 performs the same operation as that of steps A1 to A6 (E1 to E6). The controller 17 performs the same operation as step A9 after step E6 (E10).
On the other hand, if it is determined in step E2 that face information does not exist in the image data, the controller 17 acquires luminance information from the generated image data (E7). Thereafter, the controller 17 determines whether or not the color information is stored in the buffer memory 15 and the acquired luminance information is equal to or greater than a predetermined value (for example, “30”) (E8). In step E8, if the color information is not stored in the buffer memory 15 or the acquired luminance information is less than “30”, the controller 17 proceeds to step E1. On the other hand, when the color information is stored in the buffer memory 15 and the acquired luminance information is “30” or more in step E8, the controller 17 performs the same operation as in step A8 (E9). Thereafter, the controller 17 performs the same operation as in Step A9 (E10).

5-6. Operation in consideration of the number of times the face area is continuously set based on the color information (FIG. 13)
If the face cannot be detected from the image data and the face area setting based on the color information is continuously performed a plurality of times, a subject different from the face may be tracked. Therefore, when the face cannot be detected from the image data and the face area is set based on the color information for a plurality of times, the digital camera 1 does not set the face area based on the color information. That is, the digital camera 1 counts the number of times the face area is continuously set based on the color information for the image data sequentially generated from the CCD image sensor 12 as the number of times of color setting. An example of 2 counting means). If the face cannot be detected from the image data, the digital camera 1 determines whether or not to set the face area based on the color information according to the number of color settings. When the digital camera 1 determines that the face area is set based on the color information, the digital camera 1 sets the face area in the image data based on the color information.

  When the controller 17 determines that face information does not exist in the image data and the number of color settings is equal to or greater than a predetermined number (less than 90), the controller 17 determines to set the face area based on the color information. The number of color settings is stored in the buffer memory 15.

  An operation example in this case will be described with reference to FIG. Description of operations similar to those in FIG. 6 is omitted. When starting this operation, the number of color settings stored in the buffer memory 15 is set to “0”.

  First, the controller 17 performs the same operation as that in steps A1 to A6 (F1 to F6). The controller 17 performs the same operation as in step A9 (F11).

  On the other hand, if it is determined in step F2 that face information does not exist in the image data, the controller 17 stores the color information in the buffer memory 15 and the color setting count stored in the buffer memory 15 is “ Whether it is less than 90 "is determined (F7). If the color information is not stored in the buffer memory 15 or the number of color settings stored in the buffer memory 15 is “90” or more in Step F7, the controller 17 stores the color settings stored in the buffer memory 15. “0” is set to the number of times (F10). On the other hand, if the color information is stored in the buffer memory 15 and the number of color settings stored in the buffer memory 15 is less than “90” in step F7, the controller 17 performs the same operation as in step A8. Perform (F8). Then, the controller 17 adds “1” to the number of color settings stored in the buffer memory 15 (F9). Thereafter, the controller 17 performs the same operation as in Step A9 (F11).

  In this operation example, it is determined whether or not to set the face area based on the color information according to the number of color settings. However, the present invention is not limited to this, and it may be determined whether or not to set the face area based on the color information according to the color setting time (the time when the face area is continuously set based on the color information). In this case, the digital camera 1 controls the controller 17 that counts the time for which the face area is continuously set based on the color information as the color setting time for the image data sequentially generated from the CCD image sensor 12 (the first embodiment of the present invention). 2) an example of time keeping means. In the case of this operation, the calculation may be performed by converting the above number of times of color setting into, for example, 1/30 second (reading time of one frame). Therefore, in this operation example, the color setting time is stored in the buffer memory 15, and “1/30” is added to the color setting time in Step F9. In step F10, “0” is set for the color setting time. In step F7, "Is color information stored and color setting time is less than" 3 "?" ”

5-7. Operation considering the range for setting the second face area (FIG. 14)
When shooting a person as in portrait mode, the photographer often moves the digital camera so that the face of the subject is as centered as possible. Therefore, the digital camera 1 does not set the face area based on the color information when the position set for the face area based on the color information is out of the setting range. That is, when the face cannot be detected from the image data, the digital camera 1 determines whether or not to set the face area based on the color information according to the setting position of the face area based on the color information. When the digital camera 1 determines that the face area is set based on the color information, the digital camera 1 sets the face area in the image data based on the color information.

  Here, if the controller 17 according to the present embodiment determines that face information is present in the image data, the face region based on the color information is detected when the set position of the face region is moved outside the set range in the image data. Do not set. That is, when the set position of the face area is within the set range in the image data, the face area is set based on the color information.

  The setting range of the embodiment may be, for example, as shown in FIG. In other words, the size of the setting range may be changed according to the size of the face area set based on the face information. This is because when the face detected in the image data is large, the range in which the face area moves is considered to move over the entire screen. On the other hand, when the face detected in the image data is small, the range in which the face area moves is considered to move near the center of the screen. FIG. 15A shows an example when the face area is large. FIG. 15C shows an example when the face area is small.

  In the embodiment, if the setting position of the face area based on the color information is partially out of the setting range, the face area is not set based on the color information. For example, in FIG. 16, in the case of (A), the face area is set based on the color information, and in the case of (B), the face area is not set based on the color information.

  An operation example in this case will be described with reference to FIG. Steps G1 to G7 are the same as steps A1 to A7. When the controller 17 determines that the color information is stored in step G7, the controller 17 determines whether or not the setting position of the face area based on the color information is within the setting range (G8). When the set position is included in the set range, the controller 17 sets a face area based on the color information (G9). Thereafter, the controller 17 performs the same process as in Step A9 (G10). On the other hand, if the set position is not included in the set range, the process proceeds to step G1.

5-8. Operation considering the amount of blurring (Fig. 17)
If the digital camera is moved greatly, it is considered that the face cannot be detected from the next image data. In such a case, if the face area is set based on the color information, the face area is set even though the face of the subject does not actually exist. Therefore, the digital camera 1 determines whether or not to set the face area based on the color information according to the shake amount calculated by the controller 17 (an example of the movement amount acquisition unit of the present invention). When the digital camera 1 determines that the face area is set based on the color information, the digital camera 1 sets the face area in the image data based on the color information.

  The controller 17 sets the face area based on the color information when the blur amount is included in the predetermined range (−200 to 200 deg / sec).

  The operation will be described with reference to FIG. Description of operations similar to those in FIG. 6 is omitted.

First, the controller 17 performs the same operation as the operations in steps A1 to A6 (H1 to H6). The controller 17 performs the same operation as step A9 after step H6 (H10).
On the other hand, if it is determined in step E2 that face information does not exist in the image data, the controller 17 calculates the amount of blurring of the digital camera 1 (H7). Thereafter, the controller 17 determines whether or not the color information is stored in the buffer memory 15 and the calculated blur amount is included in the predetermined range (H8). In step H8, if the color information is not stored in the buffer memory 15 or the calculated blur amount is not included in −200 to 200 deg / sec, the controller 17 proceeds to step H1. On the other hand, when the color information is stored in the buffer memory 15 and the calculated blur amount is included in −200 to 200 deg / sec in step H8, the controller 17 performs the same operation as in step A8 (H9). ). Thereafter, the controller 17 performs the same operation as in Step A9 (H10).

6). Summary The digital camera 1 captures a subject image formed by the optical system 11 and sequentially generates image data, a flash memory 16 that stores face information, and a CCD image sensor 12. The controller 17 that determines whether or not the stored face information exists in the image data, and when the controller 17 determines that the face information exists, the image data is located at the position where the face information is determined to exist. A controller 17 that sets the face area and stores the color information of the image data in the set face area in the buffer memory 15, and the controller 17 determines that no face information exists, and the buffer memory 15, when color information is stored, image data based on the stored color information It includes a controller 17 for setting the face area, to.

In this way, when face information does not exist in the image data, the face area is set based on the color information, so that the face of the subject faces from the front direction to the direction other than the front direction. Even in such a case, the face area can be continuously set with high accuracy for the image data sequentially generated from the CCD image sensor.
(Other embodiments)
Embodiment 1 was illustrated as embodiment of this invention. However, the present invention is not limited to the first embodiment, and can be implemented in other embodiments. Therefore, other embodiments of the present invention will be described collectively below.

  In the first embodiment, the example in which face information is used as the first feature information has been described. However, the present invention is not limited to this, and for example, authentication image information used for solid authentication can be used. For this authentication image information, a person, that is, a subject (dog, cat, etc.) different from a human may be used. Furthermore, the information is not limited to image information, and any information may be used as long as it is characteristic information of the subject.

  In this case, for example, the setting is changed from the solid authentication to the setting of the feature region based on the color information automatically, so that the tracking is performed with the accuracy higher than the accuracy without storing a plurality of pieces of image information for the solid authentication. Will be able to. For example, image information obtained by photographing a dog is stored. A dog is detected by solid authentication based on this image information. At this time, information regarding the color of the detected dog is stored. After this, if a dog cannot be detected by solid authentication, it is tracked with color information.

  In the present embodiment, the face area is set using face information stored in advance in the buffer memory 15. However, the information is not limited to this, and may be information acquired from the outside immediately before shooting.

  Further, in the present embodiment, the feature information is classified using the hue as the information regarding the color. However, the present invention is not limited to this, and saturation or brightness may be used, or information combining these may be used.

  In the present embodiment, an example in which information about color is used as the second feature information has been described. However, the present invention is not limited to this, and for example, information on contrast or / and information on luminance can be used.

  That is, the present invention is not limited to the above embodiment, and can be implemented in various modes.

  The present invention can be applied to an imaging apparatus. For example, it can be used for a digital camera, a video camera, a mobile terminal with a camera, and the like.

1 is a perspective view of a digital camera according to a first embodiment of the present invention. 1 is a block configuration diagram of a digital camera according to a first embodiment of the present invention. The figure for demonstrating the difference in operation | movement with the case where face information exists in the image data concerning Embodiment 1 of this invention, and the case where it does not exist. The figure for demonstrating the color information concerning Embodiment 1 of this invention. The figure for demonstrating the color information concerning Embodiment 1 of this invention. Flowchart for explaining the basic operation of the digital camera according to the first embodiment of the present invention. The figure for demonstrating operation | movement of the digital camera concerning Embodiment 1 of this invention. Flowchart for explaining an operation example 5-2 of the digital camera according to the first embodiment of the present invention. Flowchart for explaining an operation example 5-3 of the digital camera according to the first exemplary embodiment of the present invention. Flowchart for explaining operation example 5-4 of the digital camera according to the first exemplary embodiment of the present invention. The figure for demonstrating the specific area | region of the operation example 5-4 of the digital camera concerning Embodiment 1 of this invention. Flowchart for explaining operation example 5-5 of the digital camera according to the first exemplary embodiment of the present invention. Flowchart for explaining operation example 5-6 of the digital camera according to the first exemplary embodiment of the present invention. Flowchart for explaining operation example 5-7 of the digital camera according to the first exemplary embodiment of the present invention. The figure for demonstrating the setting range of the operation example 5-7 of the digital camera concerning Embodiment 1 of this invention. The figure for demonstrating the operation | movement according to the setting range in the operation example 5-7 of the digital camera concerning Embodiment 1 of this invention. Flowchart for explaining operation example 5-8 of the digital camera according to the first exemplary embodiment of the present invention.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Digital camera 11 Optical system 12 CCD image sensor 121 Timing generator 13 AD converter 14 Image processing part 15 Buffer memory 16 Flash memory 17 Controller 20 Liquid crystal monitor 21 Operation part 50 Face area 60 Specific area 70 Setting range

Claims (12)

Imaging means for capturing a subject image formed by an optical system and sequentially generating image data;
A storage medium storing first feature information indicating features of the subject;
Discriminating means for discriminating whether or not the stored first feature information exists in the image data generated from the imaging means;
When it is determined by the determining means that the first feature information is present, a feature region is set at a position where the first feature information is determined to be present in the image data, and within the set feature region First setting means for storing second characteristic information indicating characteristics of the image data in the storage medium;
When it is determined by the determining means that the first feature information does not exist, and the second feature information is stored in the storage medium, the image data is based on the stored second feature information. Second setting means for setting the feature region in the inside,
An imaging apparatus comprising: The second setting means includes
Further, according to the content of the stored second feature information, it is determined whether to set a feature region based on the second feature information, and when it is determined to set the feature region, the stored second feature information 2) setting the feature region in the image data based on the feature information;
The imaging device according to claim 1. A first counting unit that counts the number of times that the first setting unit has continuously set the characteristic region as the first set number of times for the image data sequentially generated from the imaging unit;
The second setting means includes
Further, according to the number of times set by the first counting means, it is determined whether to set a feature area based on the second feature information, and if it is determined to set the feature area, the stored second 2) setting the feature region in the image data based on the feature information;
The imaging device according to claim 1. First time measuring means for measuring the time from when the first setting means sets the feature area to the time when the feature area is no longer set as the first set time for the image data sequentially generated from the imaging means. In addition,
The second setting means includes
Further, it is determined whether or not to set a feature area based on the second feature information according to the first set time measured by the first timing means, and if it is determined to set the feature area, the storage Setting the feature region in the image data based on the second feature information
The imaging device according to claim 1. It further comprises a luminance acquisition means for acquiring information regarding the brightness of the shooting environment,
The second setting means includes
Further, it is determined whether or not to set a feature area based on the second feature information according to the information about the brightness acquired by the luminance acquisition means, and if it is determined to set the feature area, the stored Setting the feature region in the image data based on second feature information;
The imaging device according to claim 1. The first setting means includes
When it is determined by the determining means that the first feature information is present, the feature further indicates a feature of the image data in the specific area designated by the position where the first feature information is determined to be present in the image data. Storing third feature information in the storage medium;
The second setting means includes
Based on the content of the stored third feature information, it is determined whether to set a feature region based on the second feature information, and when it is determined to set the feature region, the stored second feature information Setting the feature region in the image data based on feature information;
The imaging device according to claim 1. A second counting unit that counts the number of times that the second setting unit has continuously set the characteristic area as the second set number of times for the image data sequentially generated from the imaging unit;
The second setting means includes
Further, according to the second set number counted by the second counting means, it is determined whether or not to set a feature region based on the second feature information, and if it is determined to set the feature region, Setting the feature region in the image data based on the second feature information
The imaging device according to claim 1. Second time measuring means for measuring the time from when the second setting means sets the characteristic area to the time when the characteristic area is no longer set as the second setting time for the image data sequentially generated from the imaging means; In addition,
The second setting means includes
Further, it is determined whether to set a feature area based on the second feature information according to a second set time measured by the second timing means, and when it is determined to set the feature area, the storage Setting the feature region in the image data based on the second feature information
The imaging device according to claim 1. The second setting means includes
Further, according to the position where the feature area based on the second feature information is set, it is determined whether to set the feature area based on the second feature information. Setting the feature region in the image data based on the second feature information.
The imaging device according to claim 1. It further comprises movement information acquisition means for acquiring information relating to movement of the own device,
The second setting means includes
Further, it is determined whether to set a feature area based on the second feature information in accordance with the movement-related information acquired by the movement amount acquisition means. Setting the feature region in the image data based on second feature information;
The imaging device according to claim 1. The first feature information is information about a face,
The second feature information is information relating to color.
The imaging device according to claim 1. The imaging device according to claim 1, wherein the feature region is a face region.

Images