JP2010028341A - Electronic still camera - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To include an image with expression at a higher level in a photographed image. <P>SOLUTION: This electronic still camera is provided with: a smiling face level data storage part 33 which stores a plurality of smiling face levels gradually expressing degrees of smiling faces; a smiling face level setting part 41 which sets a first smiling face level value indicating degrees of smiling faces and a second smiling face level value with a degree lower than that of the first smiling face level value; a smiling face level determination part 23 which determines that a smiling face level of an object corresponds to the first and the second smiling face level values, respectively, generates a first output signal when it is determined that the smiling face level corresponds to the first smiling face level value, and generates a second output signal when it is determined that the smiling face level corresponds to the second smiling face level value; and an automatic continuous photography part 24 which starts continuous photography according to the first output signal and ends the continuous photography according to the second output signal. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to an electronic still camera having a function of automatically photographing by recognizing facial expressions of a subject.

  Conventionally, the face of the subject is detected, it is determined whether the facial expression is in a predetermined state (for example, joy, sadness, anger, or surprise), and the image is automatically taken when the predetermined facial expression is obtained. There is known a camera that performs the above (see, for example, Patent Document 1).

JP 2001-51338 A

  Although there are various human facial expressions, for example, when focusing on laughter, it usually starts with a smile, gradually becomes a big laughter as the facial expression relaxes, and the process of laughing gradually goes away. Therefore, facial expressions change continuously with the progress of laughter. With the camera of Patent Document 1, since automatic photographing is performed when the facial expression is in a predetermined state, only a certain level of facial expression can be photographed. Therefore, a face with a richer facial expression is photographed. There is a problem of losing opportunities.

(1) An electronic still camera having a facial expression photographing mode according to the first aspect of the present invention includes an imaging means for imaging a subject to generate a subject image, and a plurality of facial expression levels each representing a degree of facial expression step by step. Expression level data storage means for storing expression level data indicating facial features at the expression level, a first expression level indicating the degree of facial expression, and a second expression having a lower expression level than the first expression level Expression level setting means for setting the expression level, expression detection means for detecting the expression of the face of the subject from the subject image, and outputting expression data representing the expression, expression detection based on the expression data and expression level data It is determined that the facial expression of the subject detected by the means corresponds to the first expression level and the second expression level, respectively, and the first expression is determined based on the determination that the expression corresponds to the first expression level. A determination signal is generated, and expression level determination means for generating a second determination signal based on the determination that the second determination signal corresponds to the second expression level; and continuous shooting is started in response to the first determination signal; And automatic continuous photographing means for ending the continuous photographing in response to the signal.
(2) According to the invention of claim 2, in the electronic still camera of claim 1, when the expression level determination means has passed a predetermined time without generating the first determination signal, the expression photographing mode Is automatically canceled.
(3) According to a third aspect of the present invention, in the electronic still camera according to the first or second aspect, images picked up by continuous shooting are automatically stored in a common folder.
(4) According to a fourth aspect of the present invention, in the electronic still camera according to any one of the first to third aspects, the continuous shooting includes continuous shooting and moving image shooting for continuously capturing a predetermined number of still images. It is characterized by including at least one of these.
(5) The invention of claim 5 is the electronic still camera according to claim 4, wherein the continuous shooting includes continuous shooting, and sets the number of shots for setting the number of still images taken by continuous shooting. And an automatic continuous photographing means is characterized in that continuous photographing is performed with a set number of images.
(6) The invention of claim 6 is the electronic still camera according to claim 5, wherein still images captured by continuous shooting are automatically reproduced in descending order of expression level.

  According to the electronic still camera of the present invention, the continuous shooting is automatically started at the first facial expression level, and the continuous shooting is automatically ended at the second setting level. It is possible to include an image of a facial expression with a level higher than the first facial expression level.

Hereinafter, an electronic still camera according to an embodiment of the present invention will be described with reference to FIGS.
FIG. 1 is a block diagram showing a configuration of an electronic still camera according to an embodiment of the present invention.

  As shown in FIG. 1, the electronic still camera includes a photographing lens 11, an aperture 12, an image sensor 13, a buffer memory 14, an image processing unit 15, an image recording unit 16 such as a memory card, and a control unit 20. The electronic still camera includes a face detection unit 21, a smile detection unit 22, a smile level determination unit 23, an automatic continuous shooting unit 24, and a liquid crystal display 25. Further, the electronic still camera includes a face data storage unit 31, an expression data storage unit 32, a smile level data storage unit 33, a face registration data storage unit 34, a smile shooting mode setting unit 40, a smile level setting unit 41, and a number of shots setting unit. 42 and a smile shooting mode canceling unit 43.

  The photographing lens 11 includes a plurality of lenses including a zoom lens and a focus lens, and forms a subject image on the image sensor 13. In FIG. 1, for the sake of simplicity, the taking lens 11 is shown as a single lens. The imaging element 13 generates an image signal by photoelectrically converting incident light L1 from the photographing lens 11.

  The image signal output from the image pickup device 13 is sent to the image processing unit 15 via the buffer memory 14, where predetermined various image processing is performed. At the stage before the start of shooting, the image signal from the image sensor 13 is sent to the liquid crystal display 25 via the buffer memory 14, the image processing unit 15, and the control unit 20, and displayed as a through image. At the photographing stage, the image signal from the image sensor 13 is recorded in the image recording unit 16 via the buffer memory 14 and the image processing unit 15.

  The face detection unit 21 inputs through image data sent from the image sensor 13 via the buffer memory 14, the image processing unit 15, and the control unit 20, and the through image data is stored in the face data storage unit 31 in advance. By comparing with existing face data, a human face in the through image is detected. The detected face area image is sent from the face detection unit 21 to the control unit 20. For the face detection, for example, a detection method disclosed in Japanese Patent Laid-Open No. 2001-16573 can be used. In this detection method, feature points are extracted from an input image to detect the face area, face size, etc. of the subject. The feature points include end points of eyebrows, eyes, nose, lips, and face contour points, such as the head vertex and the lower end point of the chin.

  The face data storage unit 31 stores, for example, feature point data relating to the shape of eyebrows, eyes, nose, and lips.

  As other face detection, for example, a detection method disclosed in JP-A-2005-157679 can be used. In this detection method, first, a luminance difference between two pixels in an input image is learned as a feature amount, and an estimated value indicating whether or not a face exists in a predetermined region in the input image based on the feature amount. And when the estimated value is 1 or more, it is determined that a face exists in the predetermined area.

  The smile detection unit 22 inputs a partial image of a predetermined area detected as a face area by the face detection unit 21 from the control unit 20 and detects that the face of the partial image is a smile. That is, the smile detection unit 22 determines that the face of the partial image is a smile by comparing the facial expression of the partial image with various facial expression data stored in advance in the facial expression data storage unit 32. . For the smile detection, for example, a detection method disclosed in Japanese Patent Application Laid-Open No. 2008-42319 can be used. In this detection method, for example, the type of facial expression is determined based on which one of the two facial expressions of a smile and a normal facial expression is close.

  The expression data storage unit 32 stores data related to facial expressions such as a laughing face, a crying face, an angry face, and a surprised face.

  The smile level determination unit 23 inputs the smile image detected by the smile detection unit 22 and smile level data stored in advance in the smile level data storage unit 33 via the control unit 20 and determines that it is a smile. The degree of smile of the selected partial image is determined. As a method for determining the degree of smile, for example, a method disclosed in Japanese Patent Application Laid-Open No. 2004-46591 can be used. This determination method evaluates the degree of smile for the smile of the input partial image, and calculates a smile evaluation value. When evaluating smiles, calculate the evaluation points for the shape of each element of the eyebrows, pupils, and lips by referring to the smile data stored in advance. Value.

  FIG. 2 is a diagram schematically showing a smile level of the electronic still camera according to the present embodiment. 100 indicates the smile level, that is, the smile level is 20%, and 101, 102, and 103 indicate the smile levels of 40%, 60%, and 80%, respectively. In this way, as the smile level increases to 20%, 40%, 60%, and 80%, the smile expression increases from the smile level to the big smile level.

  For example, the smile level data storage unit 33 has a smile level (smile level) of 10%, 20%, 30, 40%, 50%, 60%, 70%, 80%, 90%, 100%. Information representing the characteristics of each smile and smile level data are stored for each smile level.

  The smile level setting unit 41 sets a first smile level value for starting automatic continuous shooting and a second smile level value for ending automatic continuous shooting, respectively, by an external operation. In this setting, the first smile level value is set so that the degree of smile is higher than the second smile level value. For example, the first smile level value is set to 80% and the second smile level value is set to 60%. The setting by the smile level setting unit 41 may be performed using a percentage value, or may be set with reference to a sample image as shown in FIG. 2 displayed on the liquid crystal display 25.

  When the first smile level value and the second smile level value are set by the smile level setting unit 41, the control unit 20 stores the smiles corresponding to the first and second smile level values from the smile level data storage unit 33, respectively. The level data is read and the data is sent to the smile level determination unit 23.

  The smile level determination unit 23 compares the smile data from the smile detection unit 22 with smile level data corresponding to the first smile level value, and the smile data becomes smile level data corresponding to the first smile level value. A first determination signal is output to the control unit 20 when it corresponds. Similarly, the smile level determination unit 23 compares the smile data from the smile detection unit 22 with smile level data corresponding to the second smile level value, and the smile data corresponding to the second smile level value. A second determination signal is output to the control unit 20 when it corresponds to the level data.

  The automatic continuous photographing unit 24 outputs an operation signal to the photographing lens 11, the diaphragm 12, and the image sensor 13 in accordance with an output signal issued from the control unit 20 based on the first determination signal from the smile level determination unit 23. Then, the continuous shooting of the subject is started, and the shooting of the subject is ended according to the output signal issued from the control unit 20 based on the second determination signal from the smile level determination unit 23.

  The liquid crystal display 25 is provided on the back side of the electronic still camera, and is used for displaying a through image and reproducing a recorded image. The liquid crystal display 25 has a variable screen angle and direction, and the screen can be directed toward the subject. The liquid crystal display 25 may display not only the image but also the smile level value described above.

  The smile shooting mode setting unit 40 is an operation unit for setting ON / OFF a mode for executing automatic continuous shooting according to the degree of smile of the subject person.

  The number of shots setting unit 42 is an operation unit for setting the number of still images to be shot in one automatic continuous shooting. The number of still images for continuous shooting should be selected so that it can be shot within the duration of laughter. Therefore, when the smile shooting mode is set, the number of shots within the standard laughter duration is displayed on the liquid crystal display 25 or the like, for example, 5 or 10 and the number of shots is set by the user. To select the desired number.

  The smile shooting mode cancel unit 43 automatically cancels the smile shooting mode when a predetermined time elapses without the smile expression of the subject person reaching the first smile level value. The cancellation signal is sent from the smile shooting mode cancellation unit 43 to the control unit 20, and the control unit 20 turns the smile shooting mode OFF. The smile shooting mode canceling unit 43 can prevent the user from wasting time.

  The face registration data storage unit 34 adds an index to a face image of a specific person and registers it as data. The registered data may be one normal face image of the person or a plurality of smile images with different smile levels. The smile level determination unit 23 determines the smile level of the registered person using the registration data, but since the characteristics of the face and facial expression are known, a reliable determination result can be obtained quickly.

  The smile shooting mode of the electronic still camera configured as described above operates according to the flowchart shown in FIG.

  Referring to FIG. 3, in step S <b> 11, the user sets the smile shooting mode ON by operating the smile shooting mode setting unit 40. In step S12, a first smile level value for starting automatic continuous shooting by the operation of the smile level setting unit 41, for example, 80%, and a second smile level value for ending automatic continuous shooting, for example, 60%. Set up. In step S13, the number of still images to be shot in one continuous shooting is set by operating the shooting number setting unit 42.

  In step S14, the face detection unit 21 detects the face of the subject. In step S15, the smile detection unit 22 compares the face image of the subject with the expression data from the expression data storage unit 32 and detects that it is a smile. The smile data that represents is output. In step S <b> 16, the smile level determination unit 23 determines that the smile data from the smile detection unit 22 has reached the first smile level value 80% or more set by the smile level setting unit 41 and performs the first determination. Generate a signal. This determination is performed by comparing smile data from the smile detection unit 22 with smile level data corresponding to the first smile level value of 80%.

  In step S <b> 17, the control unit 20 activates the automatic continuous shooting unit 24 based on the first determination signal from the smile level determination unit 23 and starts continuous shooting. In step S18, the smile level determination unit 23 determines that the smile data from the smile detection unit 22 has become equal to or less than the second smile level value 60% set by the smile level setting unit 41, as in step S16. Then, a second determination signal is generated. In step S <b> 19, the control unit 20 sends a signal instructing the end of continuous shooting to the automatic continuous shooting unit 24 based on the second determination signal of the smile level determination unit 23. The continuously shot images are automatically saved in the image recording unit 16.

Next, the operation from step S15 to step S19 will be described in detail with reference to FIG.
FIG. 4 is a graph showing a smile curve as a transition of the smile level from step S15 to step S19 in FIG. 3, where the vertical axis represents the smile level and the horizontal axis represents time. In general, the expression of human laughter changes like this smile curve.

  At time t = 0, smile detection in step S15 is performed, and the smile level increases as time passes. At time t1, the smile level reaches the first smile level value of 80% (point A in the figure), and continuous shooting in step S17 starts. Furthermore, the smile level rises as time passes, and reaches the highest level (point B in the figure) at time t2. After that, the smile level decreases with the passage of time. At time t3, the smile level reaches the second smile level value of 60% (point C in the figure), and the continuous shooting in step S19 ends. That is, the time from the time t1 to the time t3 is the time during which continuous shooting is performed.

  A person's highest level of smile is very likely to be included in a plurality of images taken from time t1 to time t3. Therefore, by using the electronic still camera according to the present embodiment in the smile shooting mode, the chance of shooting the highest level of smile can be reliably captured.

Variations of the electronic still camera of the present invention are as follows.
(1) Other facial expressions In the above embodiment, smiles are detected in facial expressions and automatic continuous shooting is started. However, other facial expressions such as a crying face, an angry face, a surprised face, and a sad face are used. The automatic facial shooting may be started by detecting the facial expression.
(2) Setting Smile Level Value In the above embodiment, the smile level setting unit 41 manually sets both the first and second smile level values. However, the smile level setting unit 41 automatically sets the second smile level value to a value lower than the first smile level value after the user manually sets the first smile level value. Also good. Furthermore, the smile level setting unit 41 may automatically set both the first and second smile level values when the smile shooting mode is set.

(3) Movie shooting In the above embodiment, the continuous shooting has been described for the case of shooting a plurality of still images by continuous shooting, but moving image shooting may be used instead of continuous shooting. In this case, the shot number setting unit 42 sets the shooting time instead of the number of shot frames.
(4) Setting the number of shots In the electronic still camera of the above-described embodiment, the number of shots is set by selecting a desired number from the number displayed on the liquid crystal display 25, for example, five or ten. there were. Instead, the number of shots setting unit 42 may be configured so that an arbitrary number of shots can be manually set. In this case, when the remaining memory capacity is small, the number of shots can be set to be small accordingly.

(5) Saving in a common folder In the above embodiment, continuously shot images are automatically saved in the image recording unit 16, but at this time, one continuously shot image is set to be saved in a common folder. . Thereby, the image search time can be shortened. Even in the case of moving images, a series of moving images is stored in one folder.
(6) Reproduction of Captured Image The electronic still camera according to the above-described embodiment may be automatically and sequentially reproduced from an image with a high smile level when reproducing continuously photographed images stored in one common folder. In this case, the level of the image can be easily recognized by displaying the smile level value together with the image. The display may be performed on the liquid crystal display 25 provided in the electronic still camera of the present embodiment, or may be performed on another playback device.
The present invention is not limited to the embodiments described above as long as the characteristics are not impaired.

It is a block diagram which shows the structure of the electronic still camera which concerns on embodiment of this invention. It is a figure which shows typically the smile level in the electronic still camera which concerns on embodiment. It is a flowchart showing operation | movement of the continuous imaging | photography by the smile imaging | photography mode of the electronic still camera which concerns on embodiment. It is a graph which shows the smile curve which is transition of the smile level from step S15 of FIG. 3 to step S19.

Explanation of symbols

20: Control unit 21: Face detection unit 22: Smile detection unit 23: Smile level determination unit 24: Automatic continuous shooting unit 31: Face data storage unit 32: Facial expression data storage unit 33: Smile level data storage unit 34: Face registration data Storage unit 40: Smile shooting mode setting unit 41: Smile level setting unit 42: Number of shots setting unit 43: Smile shooting mode canceling unit

Claims (6)

Imaging means for imaging a subject to generate a subject image;
Facial expression level data storage means for storing facial expression level data indicating the facial features of each facial expression level for each of a plurality of facial expression levels expressing the degree of facial expression in stages;
Facial expression level setting means for setting a first facial expression level indicating the facial expression level and a second facial expression level having a lower facial expression level than the first facial expression level;
Facial expression detection means for detecting facial expression of the subject from the subject image and outputting facial expression data representing the facial expression;
Based on the facial expression data and the facial expression level data, it is determined that facial expressions of the subject detected by the facial expression detection means correspond to the first facial expression level and the second facial expression level, respectively. Facial expression level determination means for generating a first determination signal by determining that the expression level corresponds to the expression level, and generating a second determination signal by determining that the expression level corresponds to the second expression level;
An electronic still camera having an expression photographing mode, comprising: automatic continuous photographing means for starting continuous photographing in response to the first determination signal and ending the continuous photographing in response to the second determination signal . The electronic still camera according to claim 1.
An electronic still camera, wherein the facial expression photographing mode is automatically canceled when a predetermined time elapses without the facial expression level determination means generating the first determination signal. The electronic still camera according to claim 1 or 2,
An electronic still camera, wherein images captured by the continuous shooting are automatically saved in a common folder. The electronic still camera according to any one of claims 1 to 3,
The electronic still camera is characterized in that the continuous shooting includes at least one of continuous shooting and moving image shooting for continuously capturing a predetermined number of still images. The electronic still camera according to claim 4.
The continuous shooting includes the continuous shooting,
A number-of-photographing number setting means for setting the number of still images picked up by the continuous shooting,
The electronic still camera according to claim 1, wherein the automatic continuous photographing means performs the continuous photographing with the set number of sheets. The electronic still camera according to claim 5.
An electronic still camera, wherein still images captured by the continuous shooting are automatically reproduced in descending order of the expression level.

Images