JP2010014783A - Photographing device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To identify a specified person who should be given priority, with a high probability and to automatically focus the person even when identification of the specified person only by features of the face is difficult. <P>SOLUTION: A face region detecting unit 56 extracts a face region of a person from the image data of the subject acquired by a photographing unit 53, while a face peripheral region detecting unit 57 extracts a region other than the face. A specified person information recording unit 58 preliminarily has a record of characteristics information about the face (face information) of the specified person and characteristics information (additional information) about colors and features of clothes or the like other than the face. When a first object identifying unit 59 fails in identification of the specified person by the face information, a second object identifying unit 60 uses the additional information and determines whether or not the registered characteristics information about the specified person are included in the subject; and when the second identifying unit determines that the information is included in the subject, the unit determines the face region of the person as a focusing object. An AF controlling unit 55 performs AF control to focus the identified focusing object. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to a photographing apparatus such as a digital still camera or a portable terminal equipped with a camera function, and more particularly to a technique for focusing during photographing.

  For example, when taking a photograph or the like using a photographing apparatus such as a digital still camera, the target is selected so that an object of particular interest (for example, a person) can be clearly photographed in an image of the subject to be photographed. In general, the optical system is focused on the part of the object. In an imaging apparatus equipped with an automatic focus adjustment mechanism (AF), the focal length of the optical system is automatically adjusted so as to focus on a specific location within the imaging range (for example, the center of the imaging range). It is common. Recently, a technique for recognizing a person's face in a subject image and automatically focusing on the face area has been widely used.

  By the way, for example, when photographing a plurality of persons at the same time, there is a case where it is desired to focus on a specific person preferentially among a plurality of persons existing in the same subject. For example, there are cases where a specific person such as his / her family or friends is included in a plurality of persons. However, since such a specific person does not always exist at the center of the shooting range, it is difficult to automatically focus on the specific person.

  For example, Patent Document 1 discloses a technique for providing a convenient function in such a case. In the technique of Patent Document 1, a person's face included in the shooting range is automatically recognized by image processing, and the recognized face portion is automatically focused. Further, image information of a specific face is registered in advance, and when the face included in the image to be photographed matches the registered face, the face is automatically focused. Note that luminance information included in the image data is used for face matching.

For example, Patent Document 2 discloses a technique for confirming which position in an image is in focus after automatically adjusting the focal position. Patent Document 3 discloses a technique for recognizing a face included in an image and adjusting the position so that the face is arranged at the center of the screen and displaying the enlarged image when a person is photographed. Patent Document 4 discloses a technique for automatically enlarging and displaying a pre-selected focus position in a screen immediately after shooting.
Japanese Patent Laid-Open No. 2002-333651 Japanese Patent No. 3958861 JP 2005-318515 A JP 2004-64259 A

  If the technique of the above-mentioned patent document 1 is adopted, when shooting a subject including a plurality of persons, if a specific person registered in advance, such as a family or a friend, is included in the plurality of persons, the specific person Can be automatically focused on the face of a specific person even if is not located at the center of the shooting range.

  However, it is difficult to accurately check whether a person's face in the subject image matches the face of the specific person, and there may be a situation in which a specific person cannot be recognized among multiple persons using only the facial features. Is also expensive. For example, when identifying whether or not a specific person is based only on luminance information indicating facial features, the face of the specific person is tilted by 60 degrees or more with respect to the shooting direction in a state where the specific person is facing sideways. Face recognition becomes extremely difficult.

  As described above, when the face cannot be correctly recognized because the specific person is facing sideways, there is a problem in that the specific person cannot be focused. In this case, if an attempt is made to take a picture after sufficient recognition of a specific person is made, it takes time until the picture is taken and a photo opportunity may be missed.

  The present invention has been made in view of the above circumstances, and it is possible to identify a specific person with higher accuracy and automatically focus even in a situation where it is difficult to identify only with the facial features of the specific person. An object of the present invention is to provide a possible photographing apparatus.

The present invention provides a photographing unit for photographing a subject, first image information in a human face region, and second image information in a peripheral region of the face from image data of the subject obtained by the photographing unit. A specific area detecting unit for detecting the face, face information including feature information related to the face of the specific person, and a specific person information recording unit for recording additional information including feature information related to other than the face of the specific person, and the specific area detection If the first image information of the face area detected by the section is compared with the face information recorded in the specific person information recording section, and the first image information matches the face information, Detected by the specific region detecting unit when the first object identifying unit that identifies the region as a focus target and the first image information of the detected face region and the recorded face information do not match Of the area around the face When the additional information is included in the second image information, the face area corresponding to the peripheral area is set as the focus object. A second object identification unit for identifying; a focus adjustment unit that focuses the imaging unit on the position of the focus object identified by the first object identification unit or the second object identification unit; An imaging device is provided.
With the above-described configuration, even if it is difficult to identify only the facial features of a specific person, it is possible to identify the specific person with higher accuracy by using additional information other than the face. For this reason, it becomes possible to focus on a specific target person more reliably and quickly. As the feature information related to other than the face, for example, it is assumed to use information related to clothes worn by a specific person, glasses and accessories, a bag possessed by the specific person, and the like.

The present invention provides a photographing unit for photographing a subject, first image information in a human face region, and second image information in a peripheral region of the face from image data of the subject obtained by the photographing unit. A specific area detecting unit for detecting the face, face information including characteristic information regarding the face of the specific person, and additional information including characteristic information regarding the clothes of the specific person, and the specific area detecting unit When the first image information of the face area detected in step 1 is compared with the face information recorded in the specific person information recording unit, and the first image information and the face information match, this face area Is detected by the specific area detection unit when the first image information of the detected face area does not match the recorded face information. 2nd area around the face When the image information and the additional information recorded in the specific person information recording unit are compared, and the second image information includes additional information related to the clothes, the face area corresponding to the peripheral area is selected as the focus object. And a focus adjustment unit that focuses the photographing unit on the position of the focus object identified by the first object identification unit or the second object identification unit. A photographing apparatus including the above is provided.
With the above configuration, even in a situation where it is difficult to identify only with the facial features of a specific person, it is possible to identify the specific person with higher accuracy by using additional information including feature information about clothes. For this reason, it becomes possible to focus on a specific target person more reliably and quickly.

Further, the present invention is the above-described photographing apparatus, wherein the specific person information recording unit records feature information including luminance information as the face information, and records feature information including color difference information as the additional information. The first object identification unit compares the face information with luminance information in the first image information, and the second object identification unit performs the second image information. And comparing with the additional information using the color difference information.
With the above configuration, even in situations where it is difficult to identify only with the characteristics of the face of a specific person, it is possible to identify whether or not the person is a specific person with high accuracy by matching the color of clothes, for example, using color difference information It becomes possible. Further, it is possible to easily identify a specific person by determining the color.

Further, the present invention is the above-described imaging device, wherein the first object identification unit calculates a likelihood indicating a probability of matching between the first image information and the face information, and the likelihood. And the first threshold value and a second threshold value smaller than the first threshold value, and when the likelihood is equal to or greater than the first threshold value, a match is determined, and the second object identification is performed. The unit compares the second image information with the additional information only when the likelihood is between the first threshold and the second threshold in the first object identification unit. This includes identifying the face area of a specific person.
With the above configuration, it is possible to more appropriately determine whether or not the face region and the peripheral region correspond to a specific person by calculating and determining the likelihood indicating the degree of coincidence.

Moreover, this invention is said imaging | photography apparatus, Comprising: A said 2nd target identification part is located below with respect to the face area detected by the said specific area detection part as said 2nd image information. This includes identifying the peripheral area of the face of a specific person using color information of the peripheral area as compared with the additional information.
With the above configuration, by comparing the color of clothes or the like located below the face area, it is possible to identify and detect a specific person with high accuracy.

Further, the present invention is the above-described photographing apparatus, wherein the specific person information recording unit records color information relating to the face, upper body, and lower body of the specific person as the additional information, and the second object The identification unit includes, as the second image information, a face region detected by the specific region detection unit, a region corresponding to the upper body close to the face region, and a region corresponding to the lower body close to the upper body The color information is used to compare the color arrangement of the second image information and the additional information to identify the face peripheral area of the specific person.
With the above configuration, the color of the person's face, upper body, and lower body and their arrangement are compared to determine consistency, so that the specific person can be more reliably identified and detected using additional information other than the face. For example, even when there are multiple persons in the subject wearing clothes similar to the upper body clothes of a specific person, the determination accuracy can be improved by determining the combination of the upper body clothes and the lower body clothes. Enhanced.

Further, the present invention is the above-described photographing device, wherein the specific person information recording unit records a plurality of feature information including color difference information and luminance information as the additional information, and the second object The identification unit performs comparison with the first additional information using the color difference information in the second image information and comparison with the second additional information using the luminance information in the second image information. This includes identifying the area around the human face.
With the above-described configuration, it is possible to improve the determination accuracy of a specific person by comparing features such as clothes other than the face using luminance information in addition to color difference information as additional information. For example, even if there are multiple persons in the subject wearing clothes of the same color as the clothes of a specific person, the shape of clothes, the shape of items worn such as glasses, and belongings such as bags The determination accuracy can be improved by determining the shape and the like together.

  According to the present invention, it is possible to provide an imaging apparatus capable of identifying a specific person with higher accuracy and automatically focusing even in a situation where it is difficult to identify only by the facial features of the specific person.

  In the following embodiments, a configuration example applied to a camera function unit mounted on a portable electronic device such as a mobile phone terminal or a digital still camera is shown as an example of a photographing apparatus.

  FIG. 1 is a block diagram showing a schematic configuration of a photographing apparatus according to an embodiment of the present invention. The imaging apparatus of the present embodiment includes an optical system 52, an imaging unit 53, a focus adjustment mechanism 54, an AF control unit 55, a face area detection unit 56, a face peripheral area detection unit 57, a specific person information recording unit 58, and a first object. An identification unit 59 and a second object identification unit 60 are provided.

  The optical system 52 includes a plurality of lenses and the like, and guides light incident from an arbitrary subject 51 to the imaging surface of the imaging unit 53 to form an image. The focus adjustment mechanism 54 connected to the optical system 52 moves the position of the lens of the optical system 52 by a driving force such as a motor, and adjusts the focus of the light imaged on the imaging surface of the imaging unit 53. The photographing unit 53 photographs the subject 51, and includes an imaging device such as a two-dimensional CCD image sensor. The photographing unit 53 converts the subject image incident on the imaging surface of the surface into an electric signal of a two-dimensional image as image data. Output.

  An AF (automatic focus adjustment) control unit 55 controls the focus adjustment mechanism 54 based on image data output from the imaging unit 53 or a signal output from another sensor (such as a distance measuring sensor (not shown)), and the subject AF control is performed to automatically adjust the focus of the optical system 52 so that a specific position on 51 is correctly focused. In the normal operation mode, the AF control unit 55 performs AF control so that the focus is achieved at a predetermined position such as the center of the screen. Further, in a special mode for focusing on a person, the AF control unit 55 positions the focus object to be focused based on a signal input from the first object identification unit 59 or the second object identification unit 60. And AF control is performed so that the focus is correctly adjusted at the position of the focus object to be prioritized on the subject 51. Here, the focus adjustment mechanism 54 and the AF control unit 55 realize the function of the focus adjustment unit.

  The face area detection unit 56 performs predetermined image processing on the luminance signal included in the image data output from the imaging unit 53, thereby determining the area of each person's face in the subject image (subject image image) before shooting. The extracted face area data is output.

  The face peripheral area detection unit 57 belongs to each person in the subject image (subject image image) before photographing by performing predetermined image processing on the luminance signal included in the image data output from the photographing unit 53. An area other than the face is extracted, and data of the extracted area is output. The area other than the face belonging to each person includes an upper body area, a lower body area, an article (for example, a bag) area of the person, and the like. For example, when a person is wearing glasses, the glasses area is also extracted as an area other than the face. As data of the area other than the face, not only luminance information but also color difference information is used. Here, the face area detection unit 56 and the face peripheral area detection unit 57 are configured to detect the first image information in the human face area and the second image information in the face peripheral area from the subject image data. The function of the area detection unit is realized.

  The specific person information recording unit 58 has a non-volatile memory, and information related to a specific person extracted by the face area detection unit 56 and the face peripheral area detection unit 57 from images captured by the image capturing unit 53 at a certain point in time, Feature information on registered personal identification clothes and personal belongings is recorded and stored as specific personal information. As the specific person, for example, it is assumed to select a family or a friend of the user (owner or the like) of the photographing apparatus. Therefore, when recording information related to a specific person from a photographed image, a person designated by an input operation such as a button by the user of the photographing apparatus may be selected as the specific person.

  The specific person information recorded by the specific person information recording unit 58 includes facial feature information obtained by analyzing the luminance signal of the face area of the specific person (for example, the distance between both eyes, hereinafter referred to as “face information”), Includes facial feature information (for example, clothing color and shape, hereinafter referred to as “additional information”) obtained by analysis of color difference signals and luminance signals of areas other than the face of a specific person, shooting date, and the like. . Further specific examples will be described later.

  The first object identifying unit 59 registers in the image data of the subject image obtained by the photographing unit 53 with the feature information regarding the face area of the person extracted by the face area detecting unit 56 and the specific person information recording unit 58 in advance. It is determined whether or not both are the same person by comparing with the feature information regarding the face area of the specified person. If it is determined that they are the same person, the corresponding person extracted by the face area detecting unit 56 is identified as a specific person, the face area is determined as a focus object, and the focus object area is determined. Position information is given to the AF control unit 55.

  The second object identification unit 60 performs the following process when the first object identification unit 59 cannot identify the person detected by the face area detection unit 56 from the specific person. That is, the second object identifying unit 60 includes, in the image data of the subject image obtained by the photographing unit 53, feature information related to a region other than the person's face extracted by the face peripheral region detecting unit 57, and specific person information in advance. By comparing the feature information related to the area other than the face of the specific person registered in the recording unit 58, it is determined whether or not both are the same person. If it is determined that they are the same person, the corresponding person extracted by the face peripheral area detection unit 57 is identified as a specific person, the face area is determined as a focus object, and the area of the focus object is determined. Is given to the AF control unit 55.

  That is, in the present embodiment, the first object identification unit 59 determines whether or not the person included in the subject 51 photographed by the photographing unit 53 matches the specific person registered in the specific person information recording unit 58. If the determination cannot be made only by using the feature information (additional information) of the region other than the face, the second object identifying unit 60 makes the determination. At this time, if a specific person registered in advance in the specific person information recording unit 58 is included in the subject 51, the identification result of the first object identification unit 59 or the second object identification unit 60 is used. The AF control unit 55 grasps the position of the face area of the person corresponding to the specific person as the focus object in the image data of the subject image obtained by the photographing unit 53, and this area (the face of the specific person) ) Automatically adjusts the focus adjustment mechanism 54 so that it is in focus.

  FIG. 2 is a block diagram showing the configuration of the main part of the photographing apparatus of this embodiment. The imaging apparatus of this embodiment includes a camera device 10, a display device 20, an application unit 100, an AF control unit 200, a priority determination unit 300, a luminance component determination unit 310, a color difference component determination unit 320, a priority registration unit 400, and imaging control. The unit 500 is provided.

  Of the components shown in FIG. 1, the optical system 52, the imaging unit 53, and the focus adjustment mechanism 54 are included in the camera device 10, and the AF control unit 55 corresponds to the AF control unit 200. Further, the face area detection unit 56, the face peripheral area detection unit 57, the first object identification unit 59, and the second object identification unit 60 correspond to the priority determination unit 300, and the specific person information recording unit 58 is a priority registration unit 400. Corresponding to Here, the first object identifying unit 59 includes the function of the luminance component determining unit 310 of the priority determining unit 300, and the second object identifying unit 60 is the color difference component determining unit 320 and the luminance component determining unit 310 of the priority determining unit 300. Includes features.

  The camera device 10 is a device for photographing an image of an arbitrary subject, and performs focus adjustment according to an instruction from the AF control unit 200 and performs photographing according to an instruction from the photographing control unit 500. The image data of the subject image before photographing or the subject image after photographing obtained by the camera device 10 is delivered to the application unit 100 via the photographing control unit 500. The camera device 10 captures a color image, and output image data includes luminance information (Y) and color difference information (Cb / Cr).

  The display device 20 is a device having a two-dimensional image display function such as a liquid crystal display panel, for example, and performs display based on instructions and information from the imaging control unit 500. At this time, display of the subject image in the shooting area of the camera device 10 (preview display) and display of the shot subject image (post-view display) are performed.

  The application unit 100 is an application program (camera application) for controlling a function of taking a photograph. This application program is executed by a processor for control processing provided in the photographing apparatus. Each function of the camera is realized by executing the camera application in the application unit 100, and a series of processes such as AF control and AE (automatic exposure adjustment) control before photographing, photographing processing, and data storage after photographing are performed. Control is implemented.

  The AF control unit 200 receives an instruction from the application unit 100 and performs AF control for automatically focusing the optical system of the camera device 10 on a specific position of the subject. At this time, the priority determination unit 300 confirms where to focus. The priority determination unit 300 receives a request from the AF control unit 200 and determines whether there is a target (person) to be focused preferentially during focus adjustment.

  The luminance component determination unit 310 determines the presence / absence of a priority portion when focusing on the basis of the luminance component (information on the face region and additional information on the peripheral region of the face) at the time of shooting. At this time, the luminance component determination unit 310 compares the face information and additional information of the specific person registered in advance in the priority registration unit 400 with the subject image in the shooting region, and based on the degree of coincidence of information, the focus priority Judgment of presence / absence of location

  The color difference component determination unit 320 determines the presence / absence of a priority portion when focusing is performed based on the color difference component (color information such as clothes) at the time of shooting. At this time, the color difference component determination unit 320 compares the color information of the peripheral area of the specific person registered in advance in the priority registration unit 400 with the subject image in the shooting area, and based on the degree of coincidence of the information, the focus priority Judgment of presence / absence of location That is, the luminance component determination unit 310 determines the luminance component, whereas the color difference component determination unit 320 performs the same determination using the color difference component.

  The priority registration unit 400 records and holds feature information of a priority target (a person's face, clothes, etc.) in order to register feature information about a specific person to be prioritized when focusing. At this time, the luminance component and the color difference component are extracted based on the image data obtained by photographing the priority object (person, clothes, etc.) and are used as registration data used in the luminance component determination unit 310 and the color difference component determination unit 320.

  The imaging control unit 500 is a component that controls the imaging operation itself of the camera. In other words, the photographing control unit 500 controls the photographing operation of the subject image captured by the camera device 10 by the photographing operation (shutter pressing) signal from the application unit 100 after the focus adjustment and the exposure adjustment are completed.

  Next, the operation of the photographing apparatus according to the present embodiment will be described. FIG. 3 is a flowchart showing an outline of operations in the photographing apparatus of the present embodiment.

  First, the application unit 100 inputs a user's shooting start operation, starts a camera application, and starts shooting processing (step S11). Thereafter, when the user performs an operation for instructing the AF control by half-pressing the shutter button or the like, the application unit 100 inputs the AF control operation, and instructs the AF control unit 200 to execute the AF control (step). S12).

  In the AF control process, the AF control unit 200 first executes a priority determination process for determining the presence / absence of a priority portion when focusing is performed (step S13). In this priority determination process, as a priority condition for deciding which position to focus on, it is confirmed whether the feature information of the face area or the area other than the face is registered, and if the priority condition is registered Determines whether the registration information matches the image data. At this time, first, the registration of the priority condition is confirmed with respect to the feature information (face information: luminance component) of the face area, and whether there is a portion that matches the face information in the image data of the subject image obtained by the camera device 10. It is determined whether or not (step S14). If there is no registration of face information, or if there is no matching part in the image data, then confirm the registration of priority conditions for feature information (additional information: color difference component, luminance component) of areas other than the face, It is determined whether there is a portion that matches the additional information in the image data of the subject image (step S15). This priority determination process will be described in detail later.

  Here, when the priority condition is registered and the feature information of the registered priority condition is matched with the subject in the comparison with the subject, the corresponding position is set as the AF priority position. The focus position (focus object) is determined (step S16), and the focus is preferentially adjusted here. On the other hand, if the priority condition is not registered or if there is no matching point in the comparison between the feature information of the priority condition and the shooting target even if it is registered, for example, the center portion of the subject image is AF The focus position is determined as the standard position (step S17), and the focus is adjusted here.

  Next, the AF control unit 200 sends an instruction to the camera device 10 to adjust the focus of the optical system to the AF priority position or AF standard position determined as the focus position, and performs focus adjustment at the corresponding focus position (step). S18). Upon receiving an instruction from the AF control unit 200, the camera device 10 adjusts the focus of the optical system by driving the lens in the optical axis direction so as to be in focus at the focus position.

  Next, when the user performs an operation for instructing execution of shooting by pressing a shutter button or the like, the application unit 100 inputs this shooting operation, and instructs the shooting control unit 500 to execute a shooting operation (step S19). ). When receiving a shooting execution instruction from the application unit 100, the shooting control unit 500 instructs the camera device 10 to perform shooting.

  Then, the shooting control unit 500 controls the shooting process (step S20). The camera device 10 shoots a subject to be photographed in accordance with an instruction from the photographing control unit 500, and passes image data of a subject image obtained by photographing to the photographing control unit 500. After receiving the image data of the subject image from the camera device 10, the imaging control unit 500 performs control for displaying the image data as a captured image on the display device 20, and at the same time, delivers it to the application unit 100. Thereafter, the application unit 100 saves the image data of the subject image received from the shooting control unit 500 in the image storage memory as a process after shooting (step S21).

  FIG. 4 is a flowchart showing a specific example of the operation of priority determination processing in the photographing apparatus of the present embodiment. Here, when face information (luminance information) related to a face area of a specific person and additional information (color difference information, brightness information) related to a peripheral area of the face are registered in the priority registration unit 400 in advance, The processing procedure of is illustrated.

  When a subject including a person is photographed by the camera device 10, first, in step S31, the priority determination unit 300 determines whether a face exists in the subject image. At this time, it is determined whether or not at least one person's face has been extracted from the image data of the subject before photographing. If a face cannot be detected from the subject image in step S31, the process proceeds to step S32, and the AF control unit 200 sets a predetermined AF standard position, for example, a position corresponding to the center of the subject image to be a focus position. Then, the camera device 10 is controlled to focus on this position.

  If a face can be detected from the subject image in step S31, the process proceeds to step S33. In step S33, the luminance component determination unit 310 of the priority determination unit 300 compares the detected face information of the person with the face information of the specific person registered in the priority registration unit 400 using the luminance component, and determines whether or not they match. Determine.

  When actually comparing face information, for example, it is determined whether or not the relative proportions of the lengths of the facial features such as eye width, mouth width, and eye-nose distance match. . However, due to the influence of differences in shooting conditions, even if the person in the subject and the registered specific person are the same person, they do not always match completely. Therefore, the luminance component determination unit 310 outputs a comparison result based on the likelihood value representing the likelihood of matching.

  FIG. 5 is a schematic diagram showing a correspondence relationship between the likelihood range and the determination result. In FIG. 5, (a) shows an example of determining either coincidence / non-coincidence, and (b) shows an example of outputting three determination results in the present embodiment. As shown in FIG. 5A, when matching or mismatching is determined under one condition, when the likelihood is expressed by a value in the range of 0 to 100%, for example, a determination criterion is set to 75%. If the value is high, it is matched, and if the likelihood is low, it is not matched. However, if there is a disagreement at a value close to the criterion, there is a high possibility that it actually matches.

  Therefore, in the present embodiment, as shown in FIG. 5B, when the likelihood is equal to or higher than a threshold A (for example, 80%) that is the first threshold, it is determined that the values match, and the likelihood is set to the second threshold. If it is less than a certain threshold B (for example, 60%), it is determined that there is a mismatch. Further, when the likelihood is in the range between the thresholds A and B, it is regarded as semi-coincidence (approximately coincidence), and re-determination is performed according to other additional conditions. In the case of quasi-coincidence, there is a possibility that it is actually coincident or there is a possibility that it is not coincident. Therefore, by determining whether the person is a specific person using additional information, the determination accuracy is improved. To do.

  Therefore, the priority determination unit 300 proceeds to step S34 when the likelihood is determined to be coincident with the threshold A or higher as a result of the comparison at step S33, and when it is determined that the likelihood is equal to or smaller than the threshold B with step S38. If the likelihood is between the thresholds A and B and it is determined as semi-match, the process proceeds to step S35, and if the likelihood cannot be determined, the process proceeds to step S39. As a case where determination is impossible, for example, there is a high possibility that the face is a face, but it is difficult to make a coincidence determination because the eyes are hidden by a hand.

  In step S34, the priority determination unit 300 determines a person whose matching is detected by the luminance component determination unit 310 or the chrominance component determination unit 320, that is, the face area of the registered specific person as the focus object, and the AF control unit 200. Controls the camera device 10 to focus on the focus position corresponding to the focus object.

  In step S <b> 35, the priority determination unit 300 determines whether or not feature information of a region other than the face related to the specific person is registered as additional information in the priority registration unit 400. If additional information is registered, the process proceeds to step S36, and if not registered, the process proceeds to step S38. Here, it is assumed that the additional information includes first additional information (for example, color information such as clothes based on color difference information) and second additional information (for example, shape information such as clothes based on luminance information).

  In step S36, the color difference component determination unit 320 of the priority determination unit 300 relates to the first additional information related to the specific person registered in advance in the priority registration unit 400 and the area around the face of the person detected from the subject image. The feature information is compared with the color difference component to determine whether or not they match. In this comparison of additional information, as in the case of comparing face information, matching, semi-matching, and mismatching are determined based on likelihood. Here, if it is determined to be coincident, the process proceeds to step S34, if it is determined to be semi-coincidence, the process proceeds to step S37, and if not coincident or cannot be determined, the process proceeds to step S38.

  In step S37, the luminance component determination unit 310 of the priority determination unit 300 relates to the second additional information related to the specific person registered in advance in the priority registration unit 400 and the area around the face of the person detected from the subject image. The feature information is compared with the luminance component to determine whether or not they match. Here, if it is determined that they match, the process proceeds to step S34, and if they do not match or cannot be determined, the process proceeds to step S38.

  Even if it is not possible to make a determination in step S33 and the face information of the person in the subject image and the specific person cannot be matched, if additional information is registered, step S39 is similar to step S36. , A match with the first additional information is determined. That is, the color difference component determination unit 320 of the priority determination unit 300 uses the color difference component to store the first additional information related to the specific person registered in advance and the feature information about the area around the face of the person detected from the subject image. Compare and determine whether they match. Here, when it determines with a match, it progresses to step S40, and when it does not correspond or a determination is impossible, it progresses to step S38.

  In step S40, the luminance component determination unit 310 of the priority determination unit 300 uses the second additional information related to the specific person registered in advance and the feature information related to the area around the face of the person detected from the subject image. Compare by component and determine if they match. Here, if it is determined that they match, the process proceeds to step S41, and if they do not match or cannot be determined, the process proceeds to step S38.

  In step S41, as in step S34, the priority determination unit 300 determines the person whose face is detected by the luminance component determination unit 310 or the color difference component determination unit 320, that is, the face area of the registered specific person as the focus object. The AF control unit 200 controls the camera device 10 so as to focus on the focus position corresponding to the focus object.

  In step S38, the priority determination unit 300 determines any face area detected in the subject image as the focus object, and the AF control unit 200 sets the focus position corresponding to the focus object. The camera device 10 is controlled to focus.

  By the above priority determination process, the brightness information of the face, such as when a specific person to be preferentially photographed faces in the direction transverse to the photographing direction, or when a part of the face is hidden by a hand, etc. Even in situations where it is difficult to identify a person using only the feature information, it is possible to more accurately identify whether or not there is a match with a specific person by making a match determination using additional information, and focus on the face of the specific person Can be adjusted automatically.

  Next, a specific example of processing for determining whether or not the feature information in the subject image matches the feature information of the specific person based on the additional information that is the feature information of the region other than the face will be described.

  FIG. 6 is a schematic diagram illustrating a specific example of additional information. Here, color information (first additional information) and shape information (second additional information) are used as additional information other than face information among the feature information of a specific person registered in the priority registration unit 400. And In the example shown in FIG. 6, the color difference component (Cb / Cr) information for each of the face, upper body, and lower body regions is registered as the color information of the first additional information. In addition, as the shape information of the second additional information, the shape of the upper body clothes (outerwear) and the shape of the lower body clothes (trousers) are registered.

  FIG. 7 is a schematic diagram showing an example of the relative positional relationship between the face, upper body, and lower body constituting a person. The color difference component determination unit 320 of the priority determination unit 300 compares the color information of the first additional information and determines whether it matches the registered additional information of the specific person. Since the color of a person's face and clothes around the person are often different for each person, the colors of the face, upper body, and lower body are compared. In many cases, a person can also be specified by the positional relationship of these colors, so the colors of the person's face and its surroundings are compared with their arrangement.

  When the subject person is in a standing posture, as shown in FIG. 7 (a), the upper body is located relatively below the face, and the lower body is located further below the upper body. If the posture is lying, it is positioned in the same order in the right direction from the face or in the opposite left direction as shown in FIG. Since the positional relationship of this face → upper body → lower body does not change, the order of these colors is never changed. Therefore, the color difference component determination unit 320 can identify whether or not the color information matches the color information of the specific person by determining the color arrangement match in addition to the color match determination of the face, upper body, and lower body. it can. A highly accurate determination result can be obtained by comparing the color information of the first additional information in addition to the face information of the specific person.

  On the other hand, the luminance component determination unit 310 of the priority determination unit 300 compares the shape information of the second additional information and determines whether it matches the registered additional information of the specific person. Regarding the shape information of the additional information, in addition to the shape of the clothes shown in FIG. 6, information of a more detailed characteristic shape can be registered. For example, shape information such as hairstyle, wrinkles, glasses, jacket collar, trouser / skirt distinction, long-sleeve / short-sleeve distinction, possessed bag, and the like.

  FIG. 8 is a schematic diagram showing a specific example regarding the shape information of the additional information. In the example of FIG. 8, a numerical value representing the distance from the face and coordinate data representing each feature point of the contour of the corresponding pattern are registered for each of the hairstyle, glasses, collar, pocket, front opening, and lower body. It is. In the priority registration unit 400, various pieces of shape information as described above are registered as the second additional information of the specific person, and the luminance component determination unit 310 compares these pieces of shape information in combination to perform a match determination. Do. In this way, by comparing the shape information of the second additional information in addition to the face information of the specific person, a highly accurate determination result can be obtained.

  Note that additional information may change even for the same person, especially if it is during the day, but it is relatively unlikely to change, but it may change compared to the situation on the previous day or earlier. Get higher. FIG. 9 is a schematic diagram showing an example of the invariant probability of the additional information. In the example of FIG. 9, the probability that the shape information of each of the hairstyle, glasses, collar, pocket, front opening, and lower body does not change is illustrated.

  FIG. 10 is a schematic diagram showing a specific example of shape information including invariant probability as additional information. For example, like the invariant probability information of the additional information shown in FIG. 9, the invariant probability of the day and the invariant probability before that are registered in advance in the priority registration unit 400 as constants for each item of shape information. Then, when acquiring the additional information of the specific person registered in the priority registration unit 400, the luminance component determination unit 310 determines whether the registration date of the additional information of the specific person matches the current date. By adding invariant probability information according to the additional information, it is reflected as information indicating the possibility of change for each item of the additional information. As a result, as shown in FIG. 10, additional information is generated by adding an element of invariant probability to the additional information form information shown in FIG.

  FIG. 11 is a flowchart showing a processing procedure for determining matching of a specific person using the additional information shown in FIG. The luminance component determination unit 310 uses the second additional information to determine whether the face person detected in the subject image and the specific person registered in the priority registration unit 400 match, as shown in FIG. Perform the following process.

  In step S <b> 51, the luminance component determination unit 310 confirms that the condition information and the luminance information of the image related to the person to be compared match the condition information (see FIG. 10) of the additional information of the specific person registered in the priority registration unit 400. judge. At this time, each item of the shape information is determined in order (hairstyle, glasses, collar, pocket, front opening, lower body order). Then, an item that matches the image to be compared with the condition of the shape information is extracted.

  In step S52, the luminance component determination unit 310 calculates and analyzes the likelihood related to the comparison between the person to be compared and the specific person with respect to the shape information of the individual item extracted in step S51. At this time, first, in step S53, based on the coordinate data representing the contour of the corresponding item, the contour matching degree between the person to be compared and the specific person is calculated based on the likelihood. Next, in step S54, based on a numerical value representing the distance from the face of the corresponding item, the distance coincidence between the person to be compared and the specific person is calculated based on the likelihood. In step S55, based on the likelihood of the contour coincidence obtained in step S53 and the likelihood of the distance coincidence obtained in step S54, the coincidence representing the likelihood of matching for the corresponding item (for example, hairstyle). The degree is calculated by the likelihood. Here, the contour matching degree 90% and the distance matching degree 80% are integrated to obtain the item matching degree of 0.9 × 0.8 = 72%. Note that the value of the invariant probability may be taken into account when calculating the degree of coincidence.

  Next, in step S56, the luminance component determination unit 310 determines whether or not the degree of coincidence (likelihood) obtained in step S55 is greater than or equal to a predetermined first reference value (for example, 75%). Identify. Here, if the degree of coincidence is equal to or higher than the first reference value, the process proceeds to step S57, where it is judged as “match” and the result is output.

  On the other hand, if the degree of coincidence is less than the first reference value in step S56, the process proceeds to step S58. In step S58, the luminance component determination unit 310 determines whether or not the degree of coincidence (likelihood) obtained in step S55 is equal to or less than a predetermined second reference value (for example, 30%), and identifies whether or not they match. Here, if the degree of coincidence is equal to or less than the second reference value, the process proceeds to step S59, where it is determined as “mismatch” and the result is output.

  If the degree of coincidence (likelihood) obtained in step S55 is an intermediate value (between the first reference value and the second reference value), the process returns to step S51 and relates to the next item (such as glasses). The degree of coincidence is determined as an evaluation. Thereafter, the same processing is repeated.

  In addition, when finally determining the match between the person to be compared and the specific person, the likelihood indicating the degree of match regarding the feature information of the face information detected from the luminance and the first additional information (color difference information) Based on the likelihood representing the degree of coincidence and the likelihood representing the degree of coincidence related to the second additional information (form information), these likelihoods are comprehensively evaluated to determine whether or not there is a match Also good.

  As described above, according to the present embodiment, the feature information of a specific person registered in advance is compared with the feature information about the face and the face surrounding the face included in the subject, and the face area of the person that matches the specific person Can automatically focus on. In this case, additional information on clothing other than the face is added even in situations where it is difficult to identify the face of a specific person, such as when the person is facing sideways or a part of the face is hidden by a hand. Based on this, a specific person can be identified with high accuracy, and a person who should be given priority can be quickly focused and photographed.

  For example, when shooting multiple people at the same time as one subject, the characteristics (color, design, etc.) of the clothes worn by the people are often different from each other, and the possibility of changing to different clothes on the same day For example, if information on clothes of a specific person photographed on the same day is registered as additional information, whether the person wearing the clothes is a specific person depends on whether the characteristics of the clothes match. Can be identified. Since the clothes worn by each person are likely to be replaced with different ones every day, it is desirable to update the additional information to be registered with the clothes information of the day each time.

  It should be noted that the present invention is not limited to those shown in the above-described embodiments, and those skilled in the art can also make changes and applications based on the description in the specification and well-known techniques. Yes, included in the scope of protection.

  The present invention has an effect that it is possible to identify a specific person with higher accuracy and automatically focus even in a situation where it is difficult to identify only by the facial features of the specific person, for example, digital It is useful as a photographing device such as a still camera or a portable terminal equipped with a camera function.

1 is a block diagram showing a schematic configuration of a photographing apparatus according to an embodiment of the present invention. The block diagram which shows the structure of the principal part of the imaging device of this embodiment. The flowchart which shows the outline | summary of operation | movement in the imaging device of this embodiment. The flowchart which shows the specific example of the operation | movement of the priority determination process in the imaging device of this embodiment. Schematic diagram showing correspondence between likelihood range and determination result Schematic diagram showing a specific example of additional information Schematic diagram showing an example of the relative positional relationship between the face, upper body, and lower body that make up a person Schematic diagram showing a specific example of additional information shape information Schematic diagram showing an example of invariant probability of additional information Schematic diagram showing a specific example of shape information including invariant probability as additional information The flowchart which shows the process sequence which performs a matching determination of a specific person using the additional information of FIG.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 Camera device 20 Display device 51 Subject 52 Optical system 53 Image | photographing part 54 Focus adjustment mechanism 55 AF control part 56 Face area detection part 57 Face periphery area detection part 58 Specific person information recording part 59 1st target identification part 60 2nd object Object identification unit 100 Application unit 200 AF control unit 300 Priority determination unit 310 Luminance component determination unit 320 Color difference component determination unit 400 Priority registration unit 500 Shooting control unit

Claims (7)

A shooting section for shooting the subject;
A specific area detecting unit that detects first image information in a face area of a person and second image information in a peripheral area of the face from image data of a subject obtained by the photographing unit;
A specific person information recording unit that records face information including characteristic information related to the face of the specific person and additional information including characteristic information other than the face of the specific person;
When the first image information of the face area detected by the specific area detection unit is compared with the face information recorded in the specific person information recording unit, and the first image information and the face information match A first object identification unit for identifying the face area as a focus object;
When the first image information of the detected face area and the recorded face information do not match, the second image information of the peripheral area of the face detected by the specific area detector and the specific person information Second object identification that compares the additional information recorded in the recording unit and identifies the face area corresponding to the peripheral area as the focus object when the additional information is included in the second image information And
A focus adjustment unit that focuses the photographing unit on the position of the focus object identified by the first object identification unit or the second object identification unit;
An imaging device comprising: A shooting section for shooting the subject;
A specific area detecting unit that detects first image information in a face area of a person and second image information in a peripheral area of the face from image data of a subject obtained by the photographing unit;
A specific person information recording unit that records face information including characteristic information about the face of the specific person and additional information including characteristic information about the clothes of the specific person;
When the first image information of the face area detected by the specific area detection unit is compared with the face information recorded in the specific person information recording unit, and the first image information and the face information match A first object identification unit for identifying the face area as a focus object;
When the first image information of the detected face area and the recorded face information do not match, the second image information of the peripheral area of the face detected by the specific area detector and the specific person information A second object for comparing the additional information recorded in the recording unit and identifying the face area corresponding to the peripheral area as a focus object when the second image information includes additional information about the clothes An object identification unit;
A focus adjustment unit that focuses the photographing unit on the position of the focus object identified by the first object identification unit or the second object identification unit;
An imaging device comprising: The imaging device according to claim 1 or 2,
The specific person information recording unit records feature information including luminance information as the face information, and records feature information including color difference information as the additional information,
The first object identification unit performs comparison with the face information using luminance information in the first image information,
The second object identification unit is a photographing apparatus that performs comparison with the additional information using color difference information in the second image information. The imaging device according to claim 1 or 2,
The first object identification unit calculates a likelihood indicating the likelihood of the match between the first image information and the face information, and based on the likelihood, the first threshold value, and the first threshold value. Comparing with a small second threshold, and determining a match if the likelihood is greater than or equal to the first threshold;
The second object identification unit includes the second image information and the second image information only when the likelihood is between the first threshold and the second threshold in the first object identification unit. An imaging device that identifies a face area of a specific person by comparing with additional information. The imaging device according to claim 1 or 2,
The second object identification unit uses the color information of the peripheral area positioned below the face area detected by the specific area detection unit as the second image information and compares it with the additional information. And an imaging device for identifying the area around the face of a specific person. The imaging device according to claim 1 or 2,
The specific person information recording unit records color information regarding the face, upper body, and lower body of the specific person as the additional information.
The second object identification unit includes, as the second image information, a face region detected by the specific region detection unit, a region corresponding to the upper body close to the face region, and a lower body close to the upper body. An image capturing apparatus for identifying a peripheral area of a face of a specific person by comparing color arrangements of the second image information and the additional information using color information in an area corresponding to. The imaging device according to claim 1 or 2,
The specific person information recording unit records a plurality of feature information including color difference information and luminance information as the additional information,
The second object identification unit compares the first additional information using color difference information in the second image information and the second additional information using luminance information in the second image information. Device for comparing the areas around the face of a specific person.

Images