JP5386880B2 - Imaging device, mobile phone terminal, imaging method, program, and recording medium - Google Patents

Description

  The present invention relates to an imaging device, a mobile phone terminal, an imaging method, a program, and a recording medium, and particularly to an imaging device, a mobile phone terminal, an imaging method, a program, and a recording medium that can capture a desired image more easily.

In an imaging device such as a digital camera, an unsatisfactory photograph may be taken because the timing of pressing the shutter button is not appropriate. Therefore, in recent years, a human face is detected from an image, a luminance difference between two pixels in the face image is learned as a feature amount, and whether or not a predetermined area in the input image is a face based on the feature amount is determined. There is a face detection technique in which an estimated value indicating the image is calculated, and it is finally determined whether or not the image in the region is a face from one or more estimated values. Furthermore, for the purpose of capturing an image that is more satisfying for the subject or the photographer, a person's face is detected from the image before recording the image obtained by the imaging on the recording medium, and the facial expression is detected. An imaging apparatus has been proposed that calculates the degree of proximity to a specific facial expression, and notifies the subject of notification information according to the calculation result (see Patent Document 1).
JP 2008-42319 A

  However, in the above technique, when the recognition of a specific facial expression, such as a smile, is a shutter condition or the like, the smile level can be said to be the amount of change in facial expression when laughing at normal times. Because it corresponds to the average facial expression change so that it can be applied to the subject, if the person who has a characteristic smile becomes the subject, it is necessary to change the threshold of smile level, which is the shutter condition for smile shooting was there. In some cases, the threshold value of the smile level, which is the shutter condition, can be selected in several stages. In this case, however, it is necessary for the photographer to manually change the setting according to the subject. Thus, conventionally, there has been a problem in terms of operability.

  The present invention has been made in view of such circumstances, and an object of the present invention is to more easily capture a desired image by referring to a threshold value of a specific facial expression for each individual.

  An image pickup apparatus according to the present invention is an image pickup apparatus including an image pickup unit for picking up an image, and stores personal image data necessary for face recognition of a person and a threshold value of a specific facial expression for each person. Means for detecting face image data from image data obtained by imaging, and comparing the detected face image data with the face image data stored in the personal data storage means A face recognizing means for determining whether or not to calculate a specific expression level from the image data obtained by the imaging, and if there is matching face image data by the face recognizing means, the matched face image data A specific facial expression level threshold is referred to from the personal data storage means, and if the calculated specific facial expression level exceeds the specific facial expression level threshold of the individual, a specific table A specific facial expression recognition unit that determines that, when it is determined that the a specific expression, characterized in that it comprises a storage means for storing image data obtained by the imaging.

  A cellular phone terminal according to the present invention includes the imaging device according to the present invention.

  The imaging method according to the present invention includes a step of storing facial image data necessary for personal face recognition, a threshold value of a specific facial expression for each individual in a personal data storage unit, and image data obtained by imaging, A step of detecting face image data; a step of comparing the detected face image data with the face image data stored in the personal data storage means to determine whether they match; and If there is matching facial image data in the step of calculating a specific facial expression from the image data and determining whether or not they match, the specific facial expression threshold of the matched facial image data is obtained from the personal data storage means. If the calculated specific facial expression level exceeds the specific facial expression level threshold of the individual, it is determined that the facial expression is a specific facial expression. And-up, when it is determined that the a specific expression, characterized in that it comprises the steps of: storing image data obtained by the imaging.

  The image capturing program according to the present invention includes a process for storing face image data necessary for individual face recognition, a threshold value of a specific facial expression for each individual in a personal data storage unit, and image data obtained by image capturing. Obtained by the process of detecting face image data, the process of comparing the detected face image data with the face image data stored in the personal data storage means to determine whether they match, and the imaging If there is matching facial image data in the step of calculating a specific facial expression from the image data and determining whether or not they match, the specific facial expression threshold of the matched facial image data is obtained from the personal data storage means. A process of determining that the calculated specific facial expression is a specific facial expression when the calculated specific facial expression exceeds a specific facial expression threshold of the individual; If it is determined that the serial is a specific facial expression, characterized in that to execute a process of storing the image data obtained by the imaging, to the computer.

  The recording medium according to the present invention is a computer-readable recording medium for recording the processing of the imaging program of the present invention.

  According to the present invention, a desired image can be captured more easily by referring to the threshold value of the specific facial expression for each individual.

  Embodiments of the present invention will be described below in detail with reference to the drawings. The embodiments described below are preferred embodiments of the present invention, and thus various technically preferable limitations are given. However, the scope of the present invention is particularly limited in the following description. As long as there is no description of the effect, it is not restricted to these aspects.

  The configuration of the imaging apparatus 11 according to the embodiment of the present invention will be described with reference to FIG. FIG. 1 is a block configuration diagram of an imaging apparatus 11 according to the present embodiment. The imaging device 11 includes a CPU (Central Processing Unit) 12, a ROM (Read Only Memory) 14, a RAM (Random Access Memory) 15, a personal data storage unit 16, an imaging unit 17, a face detection unit 18, a face recognition unit 19, and a specification. The facial expression recognition unit 20 and the storage unit 21 are connected via a bus 13.

  The ROM 14 is a read only memory that stores various control programs to be executed by the CPU 12 and fixed data used in the apparatus. The RAM 15 is a Random Access Memory that stores data temporarily required for the CPU 12 to execute a program.

  The personal data storage unit 16 is a non-volatile memory, and stores face image data necessary for personal face recognition and a threshold for each individual corresponding to the face image data (hereinafter also referred to as personal data). The threshold value is a value serving as a reference when determining whether or not a specific facial expression is described later. Although not particularly illustrated, an average specific facial expression that can be applied to many subjects is determined in order to cope with a case where a facial image of a person not stored in the personal data storage unit 16 is included. The threshold value can also be stored in advance.

  As an example of the threshold value, a smile degree threshold value can be considered. FIG. 2 is a diagram illustrating an example of registration of a smile level threshold according to the present embodiment. As shown in FIG. 2, a name, which is information for identifying an individual, a smile level threshold for each individual, and face image data for face recognition are registered. The smile degree threshold value can be set to low (smile), medium or high (smile). The smile degree threshold value is a shutter determination criterion, and the captured image data is stored in the storage unit 21.

  The imaging unit 17 includes a lens, an imaging device, an AFE (analog front end) circuit, and the like, and performs imaging processing. It is also possible to continue capturing an image of the subject (photographed person) with the imaging unit 17 to acquire a moving image of the subject and display it on a display unit (not shown). In this case, it is also possible to perform a specific facial expression recognition process described later on the moving image of the subject at regular time intervals. Accordingly, when it is determined that the facial expression is a specific facial expression, the image data of the captured subject can be stored in the storage unit 21 together with the operation of releasing the shutter.

  The face detection unit 18 determines whether the image data input from the imaging unit 17 includes a face, and performs processing for detecting the face image data.

  The face recognition unit 19 compares the face image data detected by the face detection unit 18 with the face image data stored in the personal data unit 16, and determines whether they match (is the same person). Do.

  The specific facial expression recognition unit 20 performs processing for determining the facial expression of the face image data detected by the face detection unit 18 and determining whether the facial expression data exceeds the above-described threshold value. For example, a smile can be applied as the specific facial expression. If it is determined that the expression is a specific facial expression, the image data is stored in the storage unit 21. When the imaging unit 17 continues to image the subject (photographed person), it is determined to release the shutter when it is determined that the face image data included in the image data obtained by imaging is a specific facial expression. Then, an imaging operation (flash, focus adjustment, etc.) command is performed, and the captured image data is stored in the storage unit 21.

  Next, the operation process of this embodiment will be described. FIG. 3 is a flowchart showing an operation process of the imaging apparatus according to the present embodiment. A case where a specific facial expression is a smile and an image is captured in a smile shooting mode will be described as an example. The smile shooting mode is stored when a face image of a person is included in the captured image and a specific facial expression level (smile level in the present embodiment) calculated from the face image exceeds a threshold value. The image is stored in the unit 21. Here, as described above, in order to cope with a case where a face image of a person not stored in the personal data storage unit 16 is included, an average stored in advance so that it can be applied to many subjects. A threshold value for determining a specific facial expression may be set as an initial setting.

  First, the photographer registers the face image data of the person who is the person to be photographed and the smile degree threshold value that is the shutter condition at the time of smile photographing in the personal data unit 16 of the imaging device 11 (step S1 in FIG. 3). As the face image data, image data captured using the imaging unit 17 provided in the imaging device 11 may be registered, or may be registered from an external device by connecting to a network from a network connection unit (not shown). The smile degree threshold value may be set in three stages of “low, medium, high” as shown in FIG. 2, for example. When registering using the imaging unit 17, it is also possible to register the appropriate step value by shooting the subject several times.

  Next, imaging is started (step S2), and image data is obtained from the imaging unit 17 (step S3). The image data obtained from the imaging unit 17 is input to the face detection unit 18 (step S4). The face detection unit 18 determines whether there is a face in the input image data, and detects the face image data (step S5).

  When a face is detected, the detected face image data is input to the face recognition unit 19 (step S6). Note that the input image data and the position and size information of the detected face image portion in the input image data may be input to the face recognition unit 19. That is, it is only necessary that the face recognition unit 19 can input data that allows comparison of face images.

  The face recognition unit 19 compares the detected face image data with an individual face image registered in advance in the personal data unit 16 and registers the face image data according to whether or not the face image data matches the face image data registered in the personal data unit. It is determined whether the person has been made (step S7).

  If it matches the face of the person registered in the personal data unit 16 (Yes in step S7), the smile level threshold value is temporarily changed to the smile level threshold value of the person registered in the personal data unit 16 (step S7). S8).

  On the other hand, if the person is not registered in the personal data section 16 (step S7 / No), the process proceeds to step S9 with the currently set smile degree threshold.

  The specific facial expression recognition unit 20 determines the smile level of the detected face image data. A smile level is calculated from the detected face image data, and it is determined whether or not the calculated smile level exceeds a smile level threshold value (step S9).

  When the threshold value is exceeded (step S9 / Yes), the imaging unit 17 is instructed to release the shutter, and the image data is stored in the storage unit 21 (step S10). On the other hand, when the threshold value is not exceeded (step S9 / No), the face image data in the image data is detected again (step S5). When the image is continuously input from the imaging unit 17, the latest image data is detected.

  According to the above embodiment, when the person to be photographed is a person whose face image is registered in advance, the smile degree threshold that is the shutter condition is set as a threshold for each individual registered in advance, and the smile degree is determined. Shooting with a smile is possible.

As another embodiment of the present invention, when it is determined in the process (step S7 in FIG. 3) performed by the face recognition unit 19 that the face image data matches the face image data previously registered in the personal data unit, It is conceivable to notify the effect. Although various notification methods can be considered, for example, it is possible to display on a display unit so that a photographer can easily confirm. FIG. 4 is a diagram for explaining that the face matching is notified to the display unit 22 of the imaging apparatus according to the present embodiment. For the notification, there are various input input methods for allowing the photographer to permit, but for example, it may be possible to input the permission by pressing a predetermined button from the operation unit 23 in FIG. When the notification is permitted, the specific facial expression recognition unit 20 determines whether the facial expression data is a specific facial expression with reference to the threshold value of the matched face image data.

  As another embodiment, when the photographed person matches the person registered in the personal data unit 16, the photographed person is notified according to the registered smile degree threshold value, and the photographed person is smiled. The degree of the state can be promoted. For example, if the subject is a person who is not judged to smile unless the threshold of smile level is high, use a display function that is suitable for the subject and make a notification such as “Please laugh as much as you want” Is possible. In addition, the notification method may be sound or lighting.

  As another embodiment, a mobile phone terminal may be provided with the above-described imaging device.

  Note that the program executed by the CPU for the processing shown in the flowchart of FIG. 3 and the operation of the above-described embodiment constitutes a program according to the present invention. As a recording medium for recording the program, a semiconductor storage unit, an optical and / or magnetic storage unit, or the like can be used. By using such a program and recording medium in a system having a configuration different from that of each of the above-described embodiments and causing the CPU to execute the program, substantially the same effects as those of the present invention can be obtained.

  Although the present invention has been specifically described above based on the preferred embodiments, it is needless to say that the present invention is not limited to the above-described ones and can be variously modified without departing from the gist thereof.

It is a block block diagram of the imaging device which concerns on embodiment of this invention. It is a figure which shows an example of the registration content of the personal data part which concerns on embodiment of this invention. It is a flowchart which shows the operation | movement process of the imaging device which concerns on embodiment of this invention. It is a figure for demonstrating performing notification of face coincidence to the display part 22 of the imaging device which concerns on embodiment of this invention.

Explanation of symbols

11 Imaging device 12 CPU
13 bus 14 ROM
15 RAM
DESCRIPTION OF SYMBOLS 16 Personal data memory | storage part 17 Imaging part 18 Face detection part 19 Face recognition part 20 Specific expression recognition part 21 Storage part 22 Display part 23 Operation part

Claims (8)

An imaging device comprising an imaging means for imaging an image,
Personal data storage means for storing face image data necessary for personal face recognition, and threshold values having a plurality of levels of smile levels identified from smiles for each individual;
Face detection means for detecting face image data from image data obtained by imaging;
A face recognition means for comparing the detected face image data with the face image data stored in the personal data storage means to determine whether they match,
A smile level is calculated from the image data obtained by the imaging, and when there is a matching face image data by the face recognition means, a threshold having a plurality of levels of the smile level of the matched face image data is set to the personal data. A specific facial expression recognizing unit that refers to the storage unit and determines that the smile is a smile when the calculated smile level exceeds any one of a plurality of threshold levels set for the individual smile levels. ,
An imaging apparatus comprising: storage means for storing image data obtained by imaging when it is determined that the smile is detected. Means for notifying that when the face recognition means determines that the face image data match,
Input means for inputting whether or not to permit the notification,
The said specific facial expression recognition means judges whether it is a smile with reference to the threshold value of the said corresponding face image data individual, when the notification is permitted. Imaging device. When the specific facial expression recognition means determines that face image data matching the face image data detected from the image data obtained by the imaging by the face recognition means is not stored in the personal data storage means, The imaging apparatus according to claim 1, wherein a plurality of threshold values for a predetermined smile level are referred to.   The imaging according to any one of claims 1 to 3, further comprising notification means for performing notification according to a threshold value of the matched face image data individual or a plurality of threshold values of the predetermined smile degree. apparatus.   A mobile phone terminal comprising the imaging device according to claim 1. Storing face image data necessary for personal face recognition and threshold values having a plurality of levels of smile levels identified from smiles for each individual in a personal data storage means;
Detecting face image data from image data obtained by imaging;
Comparing the detected face image data with the face image data stored by the personal data storage means to determine whether they match,
If there is matching face image data in the step of calculating smile level from the image data obtained by the imaging and determining whether or not they match, the threshold value having a plurality of stages of the smile level of the matched face image data Is determined from the personal data storage means, and when the calculated smile level exceeds any one of a plurality of threshold levels set for the individual smile level, it is determined that the smile level is a smile. When,
And a step of storing image data obtained by the imaging when the smile is determined. Processing for storing facial image data necessary for personal face recognition and threshold values having a plurality of levels of smile levels identified from smiles for each individual in personal data storage means;
Processing for detecting face image data from image data obtained by imaging;
A process of comparing the detected face image data with the face image data stored by the personal data storage means to determine whether they match,
If there is matching face image data in the step of calculating smile level from the image data obtained by the imaging and determining whether or not they match, the threshold value having a plurality of stages of the smile level of the matched face image data Is determined from the personal data storage means, and when the calculated smile level exceeds any one of a plurality of threshold levels set for the individual smile level, it is determined that the smile level is a smile. When,
An imaging program that causes a computer to execute processing for storing image data obtained by imaging when it is determined that the user is smiling.   A computer-readable recording medium on which the imaging program according to claim 7 is recorded.

Images