JP2010178259A - Digital camera - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a digital camera for detecting the person's expression to decide whether or not images should be recorded, which can prevent an image of expression not preferred by a photographer from being recorded by a subject person. <P>SOLUTION: In the digital camera, in the case of person A, image recording is permitted when any one of "10% degree of smile" except for "grief", "pleasure", "anger", "surprise" and "wink" is detected and the view line is directed to a camera, and in the case of person C, image recording is not permitted even if any of the emotional expressions is detected unless the person winks with the view line directed to the camera. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to a digital camera that detects the facial expression of a person and determines whether image recording is possible.

  2. Description of the Related Art Conventionally, an apparatus capable of evaluating a person's smile level in image data is known (for example, Patent Document 1). In addition, a camera (for example, Patent Document 2) that detects a person's emotional expression or gaze direction and records an image when a predetermined expression is obtained, or a device capable of detecting the presence or absence of blinking of a person (For example, patent document 3) is also known.

JP 2004-46591 A JP 2001-51338 A JP 2001-338303 A

  Some people look good when they shoot, and some people do not. Also, what kind of facial expression the photographer wants to take varies depending on the subject person, and there is a feeling that, for example, “I do not want to take only the sadness of this person”. For this reason, if the image recording is permitted only when a certain facial expression is obtained, an image with a facial expression not desired by the photographer is recorded depending on the subject person.

A digital camera according to the present invention includes a setting unit that sets for each person whether or not to record the image data when the person in the image data has a predetermined expression, an imaging unit that obtains image data by imaging, The specifying means for specifying a person in the image data, the expression detecting means for detecting the expression of the person in the image data, and the person in the image data are specified and specified when the predetermined expression is detected. Control means for determining whether or not image data can be recorded based on the setting contents of the setting means for the person.
The digital camera according to another aspect of the invention is an image capturing unit that obtains image data by imaging, a facial expression detection unit that detects a facial expression of a person in the image data, and a continuous shot ( And a control means for stopping continuous shooting (moving image shooting) when a predetermined facial expression is no longer detected.

  According to the present invention, in a digital camera that detects the facial expression of a person and determines whether or not to record an image, it is possible to prevent an image of a facial expression that the photographer does not like from being recorded by the subject person.

1 is a control block diagram of a digital camera according to an embodiment of the present invention. The figure which shows an example of the image recording availability setting by a facial expression. The flowchart which shows the process sequence at the time of facial expression photography mode.

An embodiment of the present invention will be described with reference to FIGS.
FIG. 1 is a control block diagram of a digital camera (imaging device) in the present embodiment. The digital camera refers to all devices having a digital camera function (for example, including a mobile phone with a camera function).

  In FIG. 1, the subject luminous flux that has passed through the photographic lens 1 is imaged by the imaging device 2, and the imaging signal is input to the image processing unit 3. The image processing circuit 3a constituting the image processing unit 3 performs various processes on the input imaging signal to generate image data. The image data is displayed on a liquid crystal monitor 4 provided on the back of the camera or the like through processing by the display circuit 3b. When the shooting mode is set, the above imaging and image display are repeated, and so-called live view display (through image display) is performed. When an imaging instruction is given, imaging is performed again, and the generated image data is recorded on the recording medium 5 such as a memory card via the recording / reproducing circuit 3c.

  On the other hand, in the reproduction mode, the image data recorded on the recording medium 5 can be read out by the recording / reproduction circuit 3c and displayed on the liquid crystal monitor 4 through the processing by the image processing circuit 3a and the display circuit 3b.

  The CPU 6 controls the image processing unit 3 and a circuit (not shown) in response to an input from the operation unit 7. The operation unit 7 includes a power button, a release button, various operation members used for reproduction operation, information input, and the like.

Next, face recognition and facial expression recognition will be described.
The camera has a known face recognition function, and when a person is present as a subject in the image data, the position and size of the face area of the person are detected, and control is performed so that the face portion is focused and exposed. . A plurality of faces can be recognized in one image data, and a face recognition frame corresponding to the face size is displayed at the face position in the live view image. This is notified to the user.

  Further, in face recognition, it is possible to specify who is the person in the image data by performing a database search based on the recognized face feature information. In order to make this possible, the camera is provided with a database for face recognition, and in a predetermined face registration mode, image data obtained by imaging a person, facial feature points obtained from the image data, and the person Information (names, etc.) can be registered for a plurality of people in association with each other. The facial feature information includes feature points such as eyebrow, eye, nose, lip end points, face contour points, head apex, lower chin end points, and relative distances between the feature points. Then, the facial feature information obtained from the image data at the time of imaging is compared with the feature information in the database, and the person is specified when the similarity is found to exceed a predetermined threshold.

  Expression recognition is a function for recognizing facial expressions in the image data recognized by the face recognition. In this embodiment, emotional expressions such as “smile”, “joy”, “sadness”, “anger”, and “surprise” can be detected as facial expressions. In particular, with regard to “smile”, the level (hereinafter referred to as smile level) can be recognized in several stages from a slight smile to a big smile. Level detection may be performed for facial expressions other than smiles (joy level, sadness level, anger level, surprise level, etc.).

  Furthermore, “wink” can be detected as an action of the subject person, and it can also be detected whether or not the line of sight faces the camera (lens). Detection of these facial expressions, winks, and lines of sight can be performed using, for example, the techniques of the literature shown in the background art.

  Each detection result is reflected in the presence or absence of image recording. The camera repeats imaging in the imaging mode and performs face recognition on the image data obtained each time. When a human face is recognized in the image data, facial expression recognition is performed, and imaging, facial recognition, and facial expression recognition are repeated until it is detected that the person has a preset facial expression. When it is detected that the set facial expression is reached, image data captured with the facial expression is recorded in the recording medium 5 as an image file. As a result, an image of the facial expression desired by the photographer can be obtained.

  In this embodiment, a facial expression that determines whether or not image recording is possible can be set for each person as a subject, and the setting contents can be registered in the face recognition database. This is called image recording enable / disable setting, and details of the setting method will be described later.

  FIG. 2 shows an example of image recording permission / inhibition setting. In this example, information for three persons is registered. In the figure, a circle indicates “selected” (recording is possible), a cross indicates “non-selection” (recording is not possible), and the smile degree percentage indicates a range where a smile is recognized. For example, “smile degree of 10%” means a smile expression of 10% or more, and image recording is permitted if even a little smile. In the case of “smile degree of 50%”, image recording is not permitted with a slight smile level, but is permitted for the first time with a moderate smile level. “Smile level 0%” means that smile is not selected (images cannot be recorded with a smile). Smiles with different percentages are positioned as different facial expressions even with the same smile.

  Unlike other facial expressions of emotion, “wink” can be considered as a signal for permission to shoot or a shooting prompt from the subject person. Since we do not want to acquire a winking face image separately, if a wink is detected, the image is taken again after a predetermined time (for example, 3 to 5 seconds), and the resulting image data is recorded. It is desirable to do. “Gaze” means that the subject person is facing the camera, and when this is selected, when another selected facial expression or AND condition between “wink” and “gaze” is satisfied Image recording is allowed. When “line of sight” is not selected, image recording is permitted when any of the selected facial expressions is detected regardless of the direction of the line of sight.

  For example, in the case of the person A, any one of “smile degree” other than “sadness”, “joy”, “anger”, “surprise”, “wink” is detected, and the line of sight is facing the camera. Image recording is permitted when is detected. For those who don't usually laugh a little, “smile degree 10%” that allows image recording even with a little laugh may be effective. Also, for those who do not want to take the expression of “sadness”, it is better not to select “sadness”.

  In the case of the person B, an image is recorded when any expression of “smiling degree 50%”, “sadness”, “anger”, “wink” is detected and the line of sight is facing the camera. Is allowed. In the case of the person C, image recording is not permitted even if any emotional expression is detected unless the line of sight is directed toward the camera and winking is performed. This setting is effective when leaving the facial expression to the subject.

  It is desirable that the image recording enable / disable setting as shown in FIG. 2 can be performed by a simple operation while looking at the liquid crystal monitor 4. For example, when an image of a person registered for face recognition is displayed on the liquid crystal monitor 4, if setting input can be performed on the person, appropriate settings can be made while looking at the face of the target person. For each selection item, a circle and a cross may be alternately switched by a predetermined operation, or selection / non-selection may be determined depending on whether or not a check box corresponding to each selection item is checked. The percentage of smile level can be selected by bar display or numeric input. Depending on the display area of the liquid crystal panel 4, a plurality of images may be displayed on one screen so that settings for a plurality of people can be performed.

Here, the photographing function that determines whether or not image recording is possible according to the above-described setting of whether or not to record an image based on a facial expression is referred to as a “facial expression photographing mode”.
FIG. 3 is a flowchart showing an example of a processing procedure when the facial expression photographing mode is set. This program is activated in accordance with the setting of the facial expression shooting mode or when some operation is performed after the setting. First, in step S1, an image is taken to obtain image data. In step S2, face recognition is performed on the obtained image data, and it is determined whether or not a face is detected in step S3. If denied, the process returns to step S1, and if affirmed, database search for specifying a person is performed in step S4. In step S5, it is determined whether or not there is corresponding data in the database. If affirmative, that is, if a person is specified, the process proceeds to step S6. In step S6, setting information relating to whether or not the corresponding person can record an image is acquired from the database. This setting information has been described with reference to FIG. If step S5 is negative, that is, if the data of the person is not in the database, default setting information predetermined in step S7 is acquired.

  In step S8, facial expression recognition is performed on the image data, and a human facial expression, line-of-sight information, presence / absence of a wink, and the like are acquired. In step S9, whether or not image recording is possible is determined based on the information on whether or not to set image recording acquired in step S6 or S7 and the facial expression recognition result in step S8. For example, in the example of FIG. 2, the person in the image is A, and 10% or more of “smile”, “joy”, “anger”, “surprise”, “wink” is detected, and the line of sight is the camera Is detected, the image recording permission is determined. Similarly, when the person is A and a facial expression of “sadness” is detected, or when it is detected that the line of sight is facing other than the camera, it is determined that the image cannot be recorded.

  If the image recording is permitted, the process proceeds from step S10 to step S11, and it is determined whether or not a wink is detected by facial expression recognition in step S8. If negative, the image data is recorded on the recording medium 5 in step S14, and the process ends. If step S11 is affirmed, the process waits for a predetermined time T1 in step S12, and then performs new imaging in step S13 and proceeds to step S14. The reason for providing this waiting time is as described above.

On the other hand, if it is determined in step S10 that image recording is not possible, the process proceeds to step S15 to determine whether or not a predetermined time T2 has elapsed since the first imaging. If the result is negative, the process returns to step S1, and the processing after imaging is repeated. If step S15 is affirmed, that is, if a facial expression or line of sight for which image recording is permitted is not detected even after a predetermined time has passed, support measures are taken in step S16. This may be based on the setting contents of the image recording enable / disable setting, for example, issuing a voice warning such as “Please laugh” or “Do not laugh”. In addition, when the subject person does not readily turn his gaze toward the camera, an audio warning such as “Please turn here” may be given, or the AF auxiliary light etc. blinks to guide the gaze naturally to the camera. It may be.
Thereafter, the process returns to step S1.

  Although an example in which single frame shooting is performed has been described above, the present invention can also be applied to continuous shooting and moving image shooting. Continuous shooting (moving image shooting) may be continued while it is determined in step S10 that image recording is permitted, and continuous shooting (moving image shooting) may be stopped when it is determined that image recording is not possible.

  The facial expression to be processed is not limited to the above-mentioned “smile”, “joy”, “sadness”, “anger”, “surprise”, etc. It may be one that can detect only one facial expression. Further, the user may be able to add or delete facial expressions to be processed. In the case of adding, for example, an image of a favorite expression is selected from recorded images taken by the photographer in the past, or a new image of a favorite expression is taken and registered as a template image. A facial expression recognition program extracts facial features from a template image, stores them as facial expression data, and detects a facial expression based on the facial expression data to detect a new facial expression (the facial expression of the template image). Is possible. The new facial expression can be handled in the same way as a facial expression such as “joy” prepared in advance, and it is possible to determine whether or not to record an image by detecting the facial expression. Of course, a new facial expression can be selected / deselected as a facial expression item in the image recording enable / disable setting of FIG.

  In addition, a camera maker or the like may publish various facial expression data on a website, and the user can download and add it. Alternatively, images of various facial expressions may be prepared on the website, and a user may download a favorite image and use it as the template image. For example, a trendy facial expression may be prepared for each age, and when an age is entered, several images of fashionable facial expressions for that age may be displayed, and a favorite one can be downloaded from the images.

  Furthermore, it may be possible to register a fashionable facial expression for each country or region in consideration of the fact that fashionable facial expressions differ depending on the country or region. In this case, for example, a GPS receiver can be provided in the camera, and a popular facial expression can be automatically called based on the acquired position information, and the facial expression can be used in the above processing.

  Moreover, you may change the smile degree with respect to each person based on the positional information acquired with the GPS receiver. For example, at an amusement spot such as an amusement park where the number and degree of smiles naturally increase, the smile level in the image recording permission / inhibition setting may be set lower than usual.

  When there are a plurality of persons with different smile levels in the image data, “main character” can be selected by some operation, and processing may be performed based on the facial expression of the main character. Alternatively, the camera may automatically determine the leading role (for example, the person with the highest smile level is regarded as the leading role).

DESCRIPTION OF SYMBOLS 1 Shooting lens 2 Image pick-up element 3 Image processing part 4 Liquid crystal monitor 5 Recording medium 6 CPU

Claims (12)

Setting means for setting for each person whether or not to record the image data when the person in the image data has a predetermined expression;
Imaging means for obtaining image data by imaging;
Identifying means for identifying a person in the image data;
Facial expression detection means for detecting a facial expression of a person in the image data;
Control means for determining whether or not image data can be recorded based on the setting contents of the setting means for the specified person when a person in the image data is specified and the predetermined facial expression is detected. A digital camera characterized by   The digital camera according to claim 1, wherein the setting unit permits input of whether or not the recording is possible in a state where an image of a person is displayed on a display device, and performs setting based on the input value.   3. The digital camera according to claim 1, wherein whether or not recording is possible can be set for a plurality of facial expressions for each person, and the facial expression detection unit can detect the plurality of facial expressions. .   4. The expression warning means according to any one of claims 1 to 3, further comprising a facial expression warning unit that warns a person in the image data when a facial expression set to be recordable is not detected for a predetermined time. Digital camera. Further comprising line-of-sight detection means for detecting whether or not a person in the image data is looking at the camera;
The control means permits recording of image data when a facial expression that is set to be recordable is detected for the identified person and when it is detected that the person is looking at the camera. The digital camera according to claim 1, wherein the digital camera is a digital camera.   6. The digital camera according to claim 5, further comprising line-of-sight warning means for giving a warning when it is not detected for a predetermined time that a person in the image data is looking at the camera. Apart from the facial expression, further comprising an action detection means for detecting a predetermined action of a person in the image data,
The digital camera according to any one of claims 1 to 6, wherein when the predetermined operation is detected, the control unit permits recording of image data regardless of a facial expression.   8. The digital camera according to claim 7, wherein when the predetermined operation is detected, the control means performs imaging after a predetermined time has elapsed from the detection and records image data obtained thereby. .   9. The digital camera according to claim 7, wherein the predetermined operation is a wink. The facial expression detection means is capable of detecting a new facial expression based on facial expression data by storing facial expression data obtained from predetermined recorded image data,
The digital camera according to any one of claims 1 to 9, wherein the new facial expression can be a target of whether or not recording is possible in the setting means. Imaging means for obtaining image data by imaging;
Facial expression detection means for detecting a facial expression of a person in the image data;
2. A digital camera comprising: control means for starting continuous shooting when a predetermined facial expression is detected by the facial expression detection means, and stopping the continuous shooting when a predetermined facial expression is no longer detected. Imaging means for obtaining image data by imaging;
Facial expression detection means for detecting a facial expression of a person in the image data;
A digital camera comprising: control means for starting moving image shooting when a predetermined facial expression is detected by the facial expression detecting means, and stopping the moving image shooting when a predetermined facial expression is no longer detected.

Images