JP5359318B2 - Digital camera - Google Patents

Description

  The present invention relates to a digital camera that detects a facial expression of a subject person and determines whether to permit image recording based on the result.

  2. Description of the Related Art A camera that detects a person's emotional expression or gaze direction and records an image when a predetermined expression is obtained is known (for example, see Patent Document 1).

JP 2001-51338 A

  However, depending on the subject person, it may not be possible to obtain a predetermined expression, and image recording may not be permitted indefinitely.

The digital camera according to the present invention includes an imaging unit that captures an image of a subject and generates image data, a recognition unit that recognizes a human face and its expression in the image data, and a face recognized by the recognition unit has a specific expression. A recording permission unit that permits image recording, a control unit that records image data when image recording is permitted, and a recording permission unit that repeats imaging until the image recording is permitted by the recording permission unit. Request means for requesting the subject person to change the facial expression when image recording is not permitted, notification means for notifying the level of facial expression in the image data when image recording is permitted, Database means for registering the face feature information and the person's name in association with each other, and the recording permission means is a method in which all faces in the image data recognized by the recognition means are identified by a specific table. The image recording is permitted in the case, and the request unit extracts the feature information of the face of the person who is not a specific facial expression from the image data when the image recording is not permitted by the recording permission unit, and based on the feature information The name of the person is searched from the database means, and based on the search result, a person who is not a specific expression is named and a change of the expression is requested.

  According to the present invention, it is possible to quickly obtain a desired image by the camera supporting that a person who does not have a predetermined expression becomes a predetermined expression.

1 is a block diagram of a digital camera according to an embodiment of the present invention. The flowchart which shows an example of a face registration procedure. The figure which shows an example of face registration data. The flowchart which shows an example of the process sequence at the time of smile recognition mode setting. The flowchart which shows the other process sequence at the time of smile recognition mode setting.

An embodiment of the present invention will be described with reference to FIGS.
FIG. 1 is a control block diagram of the digital camera in the present embodiment. The digital camera refers to all devices having a digital camera function (for example, including a mobile phone with a camera function).

  In FIG. 1, the subject luminous flux that has passed through the photographic lens 1 is imaged by the imaging device 2, and the imaging signal is input to the image processing unit 3. The image processing circuit 3a constituting the image processing unit 3 performs various processes on the input imaging signal to generate image data. The image data is displayed on a liquid crystal monitor 4 provided on the back of the camera or the like through processing by the display circuit 3b. When the shooting mode is set, the above imaging and image display are repeated, and so-called live view display (through image display) is performed. When an imaging instruction is given, imaging is performed again, and the generated image data is recorded on the recording medium 5 such as a memory card via the recording / reproducing circuit 3c.

  On the other hand, in the reproduction mode, the image data recorded on the recording medium 5 can be read out by the recording / reproduction circuit 3c and displayed on the liquid crystal monitor 4 through the processing by the image processing circuit 3a and the display circuit 3b.

  In addition to the image processing unit 3, a camera control unit 7, an operation unit 8, a microphone 9, a speaker 10, and the like are connected to the CPU 6. The camera control unit 7 executes basic operations related to imaging such as exposure control in response to a command from the CPU 6. Further, the CPU 6 can record the voice input via the microphone 9 and reproduce it from the speaker 10. The operation unit 8 includes a power button, a release button, various operation members used for reproduction operation, information input, and the like.

Next, face recognition and smile recognition will be described.
The camera has a known face recognition function. When a person is present as a subject in the image data in the face recognition mode, the position and size of the person's face area are detected, and the face portion is focused and exposed. Perform such control. A plurality of faces can be recognized in one image data, and a face recognition frame corresponding to the face size is displayed at the face position in the live view image. This is notified to the user.

  Further, in face recognition, it is possible to specify who is the person in the image data by performing a database search based on the recognized face feature information. In order to make this possible, the camera is provided with a database for face recognition, and in a predetermined face registration mode, image data obtained by imaging a person, facial feature information obtained from the image data, and the person Information (names, etc.) can be registered for a plurality of people in association with each other.

An example of a face registration procedure in the face registration mode is shown in FIG.
When the face registration mode is set, the program is started by the CPU 6. First, in step S 1, a message prompting the person to take an image is displayed on the liquid crystal monitor 4, and a release operation is waited in step S 2. When the photographer puts a person to be registered into the composition and performs a release operation, the image is taken in step S3 to obtain image data. In step S4, face recognition is performed to extract face feature information of the subject. The facial feature information includes feature points such as eyebrow, eye, nose, lip end points, face contour points, head apex, lower chin end points, and relative distances between the feature points. The captured image may be displayed on a liquid crystal monitor.

  In step S5, a request is made to input the name of the person imaged, and input is waited for in step S6. As an input method, a person to be registered or a photographer may input characters on a predetermined input screen, or may be input by voice. You may make it input in both a character and an audio | voice. In the case of voice input, a voice recognition function is built into the camera, and voice data input from the microphone 9 is analyzed to grasp the contents. It is desirable that the same name is not registered twice.

  When the name is input, the name is displayed on the liquid crystal monitor 4 at step S7 to ask whether it is OK or not. In the case of NG, the name can be input again. In the case of OK, the process proceeds to step S8, and the captured face image, face feature information, and the input name are associated with each other and stored (registered) in the database.

  Data registration for one person is completed in the above process, and the user is asked whether the process is to be ended or continued in step S9. If there is another person to be registered, the operator selects “Continue”. In this case, the operator returns to step S1. When “Exit” is selected, the process ends. FIG. 3 shows an example of registration data in the database, in which information for three people is registered.

  Smile recognition is a function that detects whether or not the face in the image data recognized by the face recognition is a smile and, in the case of a smile, detects the degree in several stages. The degree of smile is called smile level, smile level, smile level, etc., but in the following explanation, it is unified to “smile level”.

  In the present embodiment, the “smile recognition mode” can be set as the shooting mode. This is used when an image of a person is to be taken with a smile, and image recording is permitted only when the smile level of the subject person reaches a reference level or higher. The reference level can be selected in advance, and if the level is set high, image recording is not permitted with a smile level.

FIG. 4 shows an example of a processing procedure when the smile recognition mode is set.
When “smile recognition mode” is selected in a state where data is registered in the database, the CPU 6 starts the program of FIG. In step S21, imaging is performed to obtain image data. In step S22, face recognition is performed on the obtained image data, and it is determined whether or not a face area is recognized in step S23. If the face area is not recognized, the process returns to step S21. If the face area is recognized, the process proceeds to step S24.

  In step S24, smile recognition is performed on the recognized face, and based on the result, it is determined whether or not it is a smile in step S25. If it is not a smile, the process proceeds to step S26, and a smile request is made to the subject person. For example, it may be possible to send a voice message such as “Please laugh” through the speaker 10. This increases the likelihood that a person who has not laughed will laugh. Then, it returns to step S21.

  On the other hand, if it is recognized that the subject person is smiling, the process proceeds to step S27 to determine whether the smile level is equal to or higher than a preset reference level. If not, the process proceeds to step S28, and the subject person is requested to increase the smile level. For example, it may be possible to send a voice message such as “Smile is not reaching the standard level” or “Please laugh more” through the speaker 10. Thereby, possibility that a smile level will become more than a reference level increases. Then, it returns to step S21.

  When step S27 is affirmed, that is, when the smile level is equal to or higher than the reference level, the process proceeds to step S29 to notify the smile level. For example, a voice message such as “smile level is 70%” may be sent through the speaker 10. In step S30, the latest image data is recorded on the recording medium 5, and then the process ends.

  In this way, when the subject person is not laughing or the smile level has not reached the standard level, a smile request is made to that person, so a photo with the desired smile level can be obtained quickly, especially not much This is effective when shooting a person who does not laugh or when the reference level is set high.

  Note that the smile level need not be particularly taken into consideration (the steps S27 and S28 are not necessary).

Next, a smile recognition mode for a plurality of persons such as a group photo will be described with reference to FIG. Steps similar to those in FIG. 4 are given the same step numbers.
In step S51, it is determined based on the result of smile recognition in step S24 whether or not everyone is smiling. If there is even one person who is not smiling, the process proceeds to step S52, and a search is performed in the above-described database to identify who is not laughing.

  The person identification is performed by comparing the feature data of the face of the person obtained from the image data with the face data registered in the database described above, and the similarity between the two data exceeds a predetermined threshold value. When something is found, the person is specified. As a result, the name of the person who is not laughing is found. In step S53, a smile request is issued to the person using the name. For example, a voice message such as “Mr. XXXX, Mr. YYYYY, please laugh” may be sent through the speaker 10. By making a smile request by name in this way, the person who is named (not laughing) is more likely to laugh. Then, it returns to step S21.

  If a person who is not registered in the database is the subject, or if accurate facial feature data cannot be obtained from the image data, the person cannot be identified (the name of the person who is not laughing cannot be determined). In this case, a voice message such as “Please laugh” or “There are people who are not laughing” may be issued. If there is a mixture of people whose names are known and people who are not known, “There are people who are not laughing” or “XXXX, please laugh. There are other people who are not laughing.”

  On the other hand, if everyone is smiling, the process proceeds to step S54 where the smile levels of the faces in the image data are compared, the face with the highest smile level (best smile) is selected, and the person with the best smile is identified by database search. . In step S55, it is determined whether or not it has been specified (whether the name has been found). If it has been found, the name and the smile level are notified. For example, it may be possible to issue a voice message such as “Mr. XXXX, Best Smile” or “Best Smile is Mr. XXXX”. This allows all photographers and non-photographers to know who is the best smile. In step S30, the image data is recorded and the process ends.

  Although the smile level is not considered in FIG. 5, steps corresponding to steps S27 and S28 in FIG. 4 may be added. In other words, it is determined whether or not the smile level of all the members is higher than the reference level, and if the result is affirmative, image recording is permitted. May be.

  Further, in the notification process in steps S29 and S56 of FIGS. 4 and 5, the name of the subject person (in the case of step S56, the names of all persons) may be notified. This can be performed not only in the “smile recognition mode” but also in the “face recognition mode” described above.

  4 and 5, information such as the name of the subject person, the smile level, the reference level, and the name of the person with the best smile may be stored in association with the image data when the image is recorded. For example, it is conceivable to describe these pieces of information as header information (Exif information or the like) of the image file. According to this, it is possible to easily classify and search image data (image file) based on the above information.

  Although described as “smile”, the same control is possible with other facial expressions (“sadness”, “anger”, “surprise”, etc.).

DESCRIPTION OF SYMBOLS 1 Shooting lens 2 Image pick-up element 3 Image processing part 4 Liquid crystal monitor 5 Recording medium 6 CPU
7 Camera control unit 8 Operation unit 9 Microphone 10 Speaker

Claims (6)

Imaging means for imaging a subject and generating image data;
A recognition means for recognizing a person's face and its expression in image data;
Recording permission means for permitting image recording when the face recognized by the recognition means is a specific facial expression;
Control means for repeating imaging by the imaging means until image recording is permitted by the recording permission means, and recording image data when image recording is permitted;
Request means for requesting the subject person to change the facial expression when image recording is not permitted by the recording permission means;
Informing means for informing the level of facial expression in the image data when the image recording is permitted;
Database means for registering the facial feature information of the person and the name of the person in association with each other,
The recording permission unit permits image recording when all the faces in the image data recognized by the recognition unit have a specific facial expression, and the request unit, when image recording is not permitted by the recording permission unit, The feature information of the face of a person who is not a specific expression is extracted from the image data, the name of the person is searched from the database means based on the feature information, and the person who is not the specific expression is searched based on the search result A digital camera characterized by requesting a change in facial expression. The informing means searches the database means for the name of the person with the highest facial expression level in the image data when the image recording is permitted, and informs the name. Item 2. The digital camera according to Item 1.   The digital camera according to claim 1, wherein the request unit is a sound generation unit.   The name of the person in the image data and the level of the facial expression are stored in association with the image data when the image recording is permitted. Digital camera.   The digital camera according to claim 1, wherein the specific facial expression is a smile.   The digital camera according to claim 1, wherein the specific facial expression is a smile of a predetermined level or more set in advance.

Images