JP2010187109A - Electronic still camera - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To easily recognize an image with high evaluation out of a plurality of smile subject images. <P>SOLUTION: Attribute data including a smile level of a subject, which is evaluated by a face expression level evaluation section 25, and name of the subject which is input by an identification data input section 35, are related to subject image data and recorded in a memory 16a. A ranking determination section 28 determines the ranks of a plurality of subject image data read out from the memory 16a based on smile levels. The thumbnail images of the plurality of subject image data to which ranks and names are respectively added are displayed on a display 42 as a list. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to an electronic still camera having a function of evaluating a facial expression level of a person in a subject image.

  2. Description of the Related Art Conventionally, there has been known an image evaluation apparatus having a function of evaluating a facial expression of a person in a photographed image and displaying a plurality of photographed images arranged in order of good facial expression based on the evaluation (for example, Patent Literature 1).

JP 2004-46591 A

  However, in the apparatus of the cited document 1, evaluation values of facial expressions are given for each captured image, and these captured images are displayed side by side in the order of good facial expression. Usually, a plurality of images are arranged two-dimensionally. In the case of display, there is a problem that it is difficult to know which image has a higher evaluation value by comparing the images.

(1) An electronic still camera according to a first aspect of the present invention is an imaging means for outputting subject image data, and a facial expression detection means for detecting that the facial expression of the subject person in the subject image data is a predetermined facial expression. Level evaluation means for evaluating the facial expression level of the subject person, input means for inputting personal identification information representing the subject person, storage means for storing the subject image data in association with the personal identification information and the facial expression level, Ranking determination for determining a ranking of a plurality of read subject image data based on a reading unit that reads a plurality of subject image data from a storage unit and a facial expression level associated with the subject image data read by the reading unit And display means for displaying a list of thumbnail images of the plurality of read subject image data. It is in each of the listed thumbnail images, and a ranking and personal identification information and displaying appended.

  According to the electronic still camera of the present invention, the ranking of a plurality of subject image data is determined based on the facial expression level of the subject image data, and the ranking is also displayed together with the thumbnail images of these subject image data. This makes it easy to grasp which image has a high evaluation level.

It is a block diagram which shows the structure of the electronic still camera which concerns on embodiment of this invention. It is a figure which shows typically the display screen by which the thumbnail list display of the electronic still camera concerning this Embodiment was displayed. It is a 1st modification of the electronic still camera which concerns on this Embodiment, and is a figure which shows typically the display screen by which the thumbnail list was displayed. It is a 2nd modification of the electronic still camera which concerns on this Embodiment, and is a figure which shows typically the display screen by which the thumbnail list was displayed. It is a 3rd modification of the electronic still camera which concerns on this Embodiment, and is a figure which shows typically the display screen by which the thumbnail list was displayed.

Hereinafter, an electronic still camera according to an embodiment of the present invention will be described with reference to the drawings.
FIG. 1 is a block diagram showing a configuration of an electronic still camera according to an embodiment of the present invention.
As shown in FIG. 1, the electronic still camera includes a photographic lens 11, an aperture 12, an image sensor 13, a buffer memory 14, and an image processing unit 15. Further, the digital camera includes an image recording unit 16, a CPU (Central Processing Unit) 17, a ROM (Read Only Memory) 18, a bus 19, an image reading unit 20, an identification data input unit 35, a facial expression setting unit 36, and a priority selection. Unit 37, display control unit 41, display 42, and image selection unit 43.

  The image processing unit 15, the image recording unit 16, the CPU 17, the ROM 18, the image reading unit 20, the identification data input unit 35, the facial expression setting unit 36, the priority selection unit 37, the display control unit 41, and the image selection unit 43 are connected to the bus 19. Are connected to each other.

The CPU 17 functions as a face detection unit 21, a facial expression detection unit 23, a facial expression level evaluation unit 25, an automatic photographing unit 27, and a ranking determination unit 28.
The ROM 18 functions as a face data storage unit 22, a facial expression data storage unit 24, a facial expression level data storage unit 26 and a face registration data storage unit 29.

  The photographing lens 11 includes a plurality of lenses including a zoom lens and a focus lens, and forms a subject image on the image sensor 13. In FIG. 1, for the sake of simplicity, the taking lens 11 is shown as a single lens. The imaging element 13 generates an image signal by photoelectrically converting the subject light L1 from the photographing lens 11.

  The image signal output from the image pickup device 13 is sent to the image processing unit 15 via the buffer memory 14, where predetermined various image processing is performed. At the stage before the start of photographing, the image signal from the image sensor 13 is sent to the display 42 via the buffer 19 via the buffer memory 14 and the image processing unit 15 and displayed as a through image. In the photographing stage, the image signal from the image sensor 13 is recorded in the image recording unit 16 via the bus 19 via the buffer memory 14 and the image processing unit 15.

  The image recording unit 16 records the image data and attribute data (tag information) related to the image data in association with each other in the nonvolatile memory (storage medium) 16a. The image data includes a subject image obtained by photographing the subject with the photographing lens 11, aperture 12, and image sensor 13, and a subject image input to the image recording unit 16 from an external device.

  The image reading unit 20 is an operation unit that searches for image data recorded in the memory 16a of the image recording unit 16 and reproduces an image based on attribute data.

  The face detection unit 21 inputs subject image data and compares it with face data stored in advance in the face data storage unit 22 to detect the face of the subject person in the image. When there are a plurality of subjects, the face of each person is detected. The face data storage unit 22 stores, for example, feature point data relating to the shape of eyebrows, eyes, nose, and lips.

  For the face detection, for example, a detection method disclosed in Japanese Patent Laid-Open No. 2001-16573 can be used. In this detection method, feature points are extracted from an input image to detect the face area, face size, etc. of the subject. The feature points include end points of eyebrows, eyes, nose, lips, and face contour points, such as the head vertex and the lower end point of the chin.

  As other face detection, for example, a detection method disclosed in JP-A-2005-157679 can be used. In this detection method, first, a luminance difference between two pixels in an input image is learned as a feature amount, and an estimated value indicating whether or not a face exists in a predetermined region in the input image based on the feature amount. And when the estimated value is 1 or more, it is determined that a face exists in the predetermined area.

The facial expression detection unit 23 compares the facial expression of the image of the face area detected by the face detection unit 21 with predetermined facial expression data stored in advance in the facial expression data storage unit 24, so that It is detected that the facial expression of the subject person corresponds to a predetermined facial expression. The facial expression detection unit 23 detects a facial expression for the face of a predetermined person when there are a plurality of subject persons.
There are various types of facial expressions such as a smiling face, a crying face, an angry face, a surprised face, and a sleeping face, and the facial expression data storage unit 24 stores the plurality of facial expression data. For example, when detecting that the facial expression of the subject person in the subject image is a smile, the facial expression detection unit 23 outputs a detection signal to that effect.

  When the facial expression detection unit 23 detects a facial expression of an image in the face area, a detection method disclosed in Japanese Patent Application Laid-Open No. 2008-42319 can be used. In this detection method, for example, in order to detect a smile, the type of facial expression is determined based on whether the face of the detected image is close to the two facial expressions of a smile and a normal face. .

  The facial expression level evaluation unit 25 compares the facial expression of the image of the facial area detected by the face detection unit 21 with a plurality of facial expression level data stored in advance in the facial expression level data storage unit 26, thereby It is evaluated which facial expression level the facial expression of the subject person in the image of the region corresponds to. The facial expression level evaluation unit 25 outputs an evaluation signal corresponding to the facial expression level of the subject person. The image recording unit 16 records subject image data and attribute data (a type of facial expression and its facial expression level) based on the evaluation signal in the memory 16a as one image file.

  The facial expression level data storage unit 26 stores each smile when the facial expression level (facial expression level), for example, the smile level (smile level) is 10%, 20%, 30%,..., 100%. Smile level data (for example, the interval between feature points) representing each feature is stored for each smile level. As the smile level increases, the smile level increases from smile level to big smile level. Similarly, the facial expression level data storage unit 26 stores level data for crying faces, angry faces, and the like.

  As a method for determining the degree of smile, for example, a method disclosed in Japanese Patent Application Laid-Open No. 2004-46591 can be used. This determination method evaluates the degree of smile for the smile of the input partial image, and calculates a smile evaluation value. When evaluating smiles, calculate the evaluation points for the shape of each element of the eyebrows, pupils, and lips by referring to the smile data stored in advance. Value.

  The automatic photographing unit 27 performs automatic photographing of the subject by outputting operation signals to the photographing lens 11, the diaphragm 12, and the image sensor 13 in accordance with the detection signal from the facial expression detecting unit 23. In this automatic photographing, one image is captured every time the facial expression detection unit 23 detects a predetermined facial expression, for example, a smile.

  The ranking determining unit 28 ranks the subject image data read from the memory 16a by the image reading unit 20. That is, the ranking determination unit 28 ranks a plurality of subject image data based on attribute data added to each subject image data, for example, level values of smile levels 80% and 60%. And the signal showing a ranking result is output.

  The face registration data storage unit 29 stores characteristic face information based on face images of registered persons registered in the camera as registration data. The characteristic face information is, for example, characteristic amount data of feature points such as eyebrows, eyes, nose, and lips.

  The identification data input unit 35 is an operation unit for inputting personal identification data representing a subject person, for example, the name or nickname of the person as attribute data in the image file recorded in the memory 16a. Due to the input of the personal identification data, the attribute data includes the type of facial expression of the subject person, the facial expression level (for example, smile level: 80%), and the personal name (for example, Mr. A). Then, the image recording unit 16 records the subject image data and the attribute data in the memory 16a in association with each other.

  The facial expression setting unit 36 is an operation unit for setting the type of facial expression of the subject person from the outside, and can set a plurality of facial expressions including a smile. For example, if the facial expression setting is a smile, the facial expression detection unit 23 detects a smile, generates a detection signal, and automatic shooting is performed. If the facial expression setting unit 36 sets a smile, the image reading unit 20 reads smile subject image data from the subject image data recorded in the memory 16a.

  The prioritizer selection unit 37 ranks the first selection unit and the lowest rank that select the subject person of the highest-ranked subject image data for the plurality of subject image data ranked by the ranking determination unit 28. A second selection unit for selecting a subject person in the subject image data, and is configured to be switchable between the first selection unit and the second selection unit.

As a result, when there are a plurality of subject persons in the subject image data, and the person selected by the first selection unit (rank: highest) exists among the plurality of subject persons, the person is designated as another person. Automatic photographing can be performed by detecting facial expressions with priority. Similarly, when a person (rank: lowest) selected by the second selection unit exists among a plurality of subject persons, the facial expression is detected with priority on the person over other persons, and automatic shooting is performed. be able to.
For the subject person of the subject image data ranked at the top and the bottom, face registration data representing facial features is created from the subject image data, and this face registration data is recorded in the face registration data storage unit 29.

  The display control unit 41 controls display on the display 42. That is, the display control unit 41 displays the through image on the display 42, displays the subject image temporarily recorded in the buffer memory 14, and reproduces and displays the subject image stored in the memory 16a of the image recording unit 16. Controls the display mode of the hour. For example, the display control unit 41 displays only the subject image on the display 42 or displays the subject image with the attribute information added. Further, the display control unit 41 controls the display mode of thumbnail images of subject image data, for example, the display of thumbnail images on the screen and the rank display based on the signal from the ranking determination unit 28.

  The display 42 displays a through image, a subject image temporarily recorded in the buffer memory 14, and a subject image stored in the memory 16a according to a command from the display control unit 41, and displays attribute information related to the subject image. Display. In addition, the display 42 displays a list of thumbnail images according to instructions from the display control unit 41.

  The image selection unit 43 is an operation unit that selects an arbitrary thumbnail image from a plurality of thumbnail images displayed as a list on the display 42. For example, in the case of a display screen (touch panel) with a pressure sensor, an arbitrary thumbnail image is selected. A selection operation is possible by touching.

The electronic still camera configured as described above operates as follows.
First, “smile” is set as a predetermined facial expression by the facial expression setting unit 36. The face detection unit 21 detects the face of the subject person in the through image by comparing the through image with the face data in the face data storage unit 22.

  The facial expression detection unit 23 compares the facial expression of the image of the face area detected by the face detection unit 21 with predetermined facial expression data stored in advance in the facial expression data storage unit 24, thereby It is detected that the facial expression of the subject person is a smile. The facial expression detection unit 23 outputs a detection signal based on this smile detection, and the automatic photographing unit 34 performs automatic photographing of a subject in which a smile is detected. The subject image data obtained by the automatic photographing is recorded in the memory 16a together with attribute data indicating that the image data is a smile detected.

The facial expression level evaluation unit 25 compares the smile level of the subject person in the subject image data obtained by automatic photographing with the degree of smile (smile) compared to the smile level data stored in the facial expression level data storage unit 26. Level) and a determination signal representing the evaluated smile level is output.
The image recording unit 16 records subject image data and attribute data related to the image data (for example, smile level 90%) in the memory 16a in association with each other.

  The image reading unit 20 reads predetermined subject image data from the memory 16a. When “smile” is set as the predetermined facial expression by the facial expression setting unit 36, the image reading unit 20 searches using the attribute data (for example, smile level 90%), and the subject image of the smiling subject person Read data.

  A reproduced image based on the read subject image data is displayed on the display 42, and personal identification data representing the subject person in the reproduced image, for example, the person name “Mr. A” is input by the identification data input unit 35. As a result, “Mr. A” data is added to the attribute data of the read subject image data in addition to the smile level 90% data. Then, the image recording unit 16 records subject image data and attribute data including smile level data and person name data in the memory 16a.

By repeating the above-described operation, attribute data is created for a plurality of subject image data, and the plurality of subject image data and attribute data are recorded in the memory 16a.
Subsequently, an operation for determining a ranking based on the smile level for the plurality of subject image data is performed. The image reading unit 20 searches for “smile” using the attribute data, and reads a plurality of subject image data from the memory 16a.

  The ranking determining unit 28 ranks the read subject image data on the basis of smile level 90% and 80% level values. For example, when there are five subject image data related to attribute data with smile level values of 90%, 80%, 80%, 70%, and 60%, the rankings are ranked first, second, 2nd, 4th and 5th.

In this way, thumbnail images of a plurality of subject image data for which ranking has been determined are displayed as a list on the display 42.
FIG. 2 is a diagram schematically showing a screen of the display 42 displayed as a thumbnail list. In FIG. 2, five thumbnail images are shown together with the attribute data. When attention is paid to the thumbnail image in the upper left, the thumbnail image 101, the ranking “first” 111, and the person name “Mr. A” 121. “90%” 131 which is a smile level is displayed. Similarly, the thumbnail image 102, “second place” 112, “Mr. B” 122, and “80%” 132 are displayed in the thumbnail image at the upper center of the screen. The same applies to the other three thumbnail images, although the reference numerals are omitted.

  In FIG. 2, five thumbnail images are arranged and displayed in the ranking order. However, since the ranking is added to each of the five thumbnail images, it is not always necessary to arrange them in the ranking order. For example, a plurality of thumbnail images may be arranged and displayed in the shooting order. In FIG. 2, the smile level is displayed as a numerical value. However, the smile level may be displayed as a graph, for example, by the bar length of a bar graph.

The electronic still camera according to the present embodiment has the following effects.
(1) Since a plurality of thumbnail images are displayed in a list with a ranking of the smile of the subject person, an image with a high smile evaluation can be recognized at a glance.
(2) Since a plurality of thumbnail images are displayed in a list with a ranking of the person's smile and the name of the person, it is possible to recognize an image of a person with a high smile evaluation at a glance.
(3) Since multiple thumbnail images are displayed in a list with the smile ranking of the subject person, the name of the person, and the smile level, the image of the subject person with the highest smile rating can be recognized and the smile can be done at the same time. You can also know the level.

The electronic still camera according to the present embodiment can also perform the following operations.
(1) Enlarged display of an image By selecting an arbitrary thumbnail image from among a plurality of thumbnail images displayed as a list on the display 42 by the image selection unit 43, the thumbnail image can be enlarged and displayed on the entire screen. it can. At this time, the enlargement display time may be changed according to the smile level, and the enlargement display time may be set longer as the smile level is higher. For example, if the subject image has a smile level of 80%, the enlarged display time is lengthened to 5 seconds, and if the subject image has a smile level of 40%, the enlarged display time is shortened to 2 seconds. In the case of a slide show, a subject image with a high smile level has a long display reproduction time and the display size is expanded, and a subject image with a low smile level has a short display reproduction time and a reduction in display size. Good.

(2) List display of thumbnail images of the same person By selecting an arbitrary person name by the image selection unit 43 from a plurality of thumbnail images displayed as a list with a smile ranking on the display 42, thumbnail images of the person are displayed. You can switch to the thumbnail list display that consists of At this time, the display size of each thumbnail image may be changed according to the smile level, and the display size may be set larger as the smile level is higher.

(3) List display of thumbnail images at the same level By selecting an arbitrary smile level by the image selection unit 43 from among a plurality of thumbnail images displayed as a list with a smile ranking on the display 42, it is equal to the smile level. It is possible to switch to a thumbnail list display composed of level thumbnail images.

(4) The priority shooting ranking determining unit 28 of the person selected based on the smile ranking ranks the plurality of subject image data based on the smile level, and when the highest level and the lowest level are determined, At the time of the next shooting, it is possible to perform automatic shooting by detecting a smile with priority given to the subject person of the subject image data ranked highest or the subject person of the subject image data ranked lowest.
First, the priority selection unit 37 selects the subject person of the subject image data ranked at the top as the priority. At the time of shooting, when there are a plurality of subject persons in the through image, and there is a priority person, the facial expression detection unit 23 uses the face registration data recorded in the face registration data storage part 29 to give priority. Is detected and a smile detection operation is started. When the facial expression detection unit 23 detects the priority person's smile, the automatic photographing unit 27 performs automatic photographing.

  For the priority person, smile detection and automatic photographing are performed with priority over other subject persons. The facial expression detection unit 23 only detects the priority person's smile, does not detect the smiles of other subject persons, and performs shooting when the priority person's smile is detected. The smiles of all the people in the image are detected, but the priority is taken when the smile is detected once, and the other subjects are photographed for the first time when the smile is detected three times. There is a case to make it.

  The above explanation is the case where the subject person of the subject image data ranked at the top is selected as a priority person, and the same applies when the subject person of the subject image data ranked at the bottom part is selected as a priority person. is there. Note that the priority selection unit 37 can freely switch between the selection of the subject person of the subject image data ranked at the top and the selection of the subject person of the subject image data ranked at the bottom.

Hereinafter, modifications of the electronic still camera of the present embodiment will be described.
In the above embodiment, the thumbnail images of the plurality of subject image data for which the ranking has been determined are displayed with the order of “first place”, “second place”, etc., as shown in FIG. Are listed. The following modified examples shown in FIGS. 3 to 5 relate to the display mode of the thumbnail list display without ranking, and the same display contents are given the same reference numerals.

  FIG. 3 shows a first modification of the thumbnail list display. Instead of the ranking display shown in FIG. 2, the display size of the thumbnail image is changed according to the smile level, and the thumbnail of the smile level “90%” 131 is displayed. The image 101a is displayed in the largest size, the thumbnail image 102a with the smile level “80%” 132 is set to the next largest size, and the size is displayed smaller as the smile level becomes lower. When displayed in this way, a thumbnail image with a high ranking can be intuitively identified.

FIG. 4 shows a second modification of the thumbnail list display, in which thumbnail images are scrolled in order of rank instead of the ranking display shown in FIG. Each thumbnail image is scroll-displayed one after another from the bottom of the screen, as indicated by the arrows in the figure.
FIG. 5 shows a third modification of the thumbnail list display, in which the ranking is also known by arrangement of thumbnail images instead of the ranking display shown in FIG. In FIG. 5, the top left position of the screen is the first position, and the thumbnail images are arranged in rank order as indicated by the arrows in the figure.

In the above embodiment, the electronic still camera that automatically shoots when a facial expression such as a smile is detected has been described. However, the electronic still camera that does not have the automatic photographing unit 27 and shoots at the intention of the photographer is also described. The present invention is applicable. Even in the case of this manual photographing, the facial expression detection unit 23 detects a smile, and the facial expression level evaluation unit 25 evaluates the smile level.
In the above embodiment, the electronic still camera has been described. However, the present invention can also be applied to application software (image browsing / editing software).
The present invention is not limited to the embodiments described above as long as the characteristics are not impaired.

16: Image recording unit 16a: Memory 20: Image reading unit 23: Facial expression detection unit 24: Facial expression data storage unit 25: Facial expression level evaluation unit 26: Facial expression level data storage unit 27: Automatic photographing unit 28: Ranking determination Unit 35: Identification data input unit 37: Priority selection unit 41: Display control unit 43: Image selection unit 101, 102: Thumbnail image

Claims (9)

Imaging means for outputting subject image data;
Facial expression detection means for detecting that the facial expression of the subject person in the subject image data is a predetermined facial expression;
Level evaluation means for evaluating the facial expression level of the subject person;
Input means for inputting personal identification information representing the subject person;
Storage means for storing the subject image data in association with the personal identification information and the facial expression level;
Reading means for reading a plurality of the subject image data from the storage means;
Ranking determining means for determining a ranking of the plurality of read subject image data based on the facial expression level associated with the subject image data read by the reading means;
Display means for displaying a list of thumbnail images of the plurality of the read subject image data,
The electronic still camera, wherein the display means additionally displays the ranking and the personal identification information on each thumbnail image displayed in the list. The electronic still camera according to claim 1.
An electronic still camera, further comprising automatic photographing means for automatically photographing according to a detection output of the facial expression detecting means. The electronic still camera according to claim 1.
The electronic still camera, wherein the personal identification information is at least one of a name and a nickname of the subject person. The electronic still camera according to claim 1.
The electronic still camera, wherein the display means displays the face expression level as a numerical value or a graph in addition to each of the thumbnail images displayed in the list. The electronic still camera according to claim 1.
A selection means for selecting an arbitrary thumbnail image from the plurality of thumbnail images displayed in the list;
The electronic still camera, wherein the display means enlarges and displays the thumbnail image selected by the selection means and changes an enlargement display time according to the smile level. The electronic still camera according to claim 1.
When arbitrary personal identification information is selected from the personal identification information additionally displayed on the thumbnail images displayed in the list by the selection means,
The electronic still camera, wherein the display means displays a list of thumbnail images of subject image data having the personal identification information selected by the selection means. The electronic still camera according to claim 4.
When an arbitrary facial expression level is selected from the facial expression levels additionally displayed on the thumbnail images displayed in the list by the selection unit,
The electronic still camera, wherein the display means displays a list of thumbnail images of subject image data having a facial expression level equal to the facial expression level selected by the selection means. The electronic still camera according to claim 2,
A first priority selection means for selecting a subject person of the subject image data determined to be ranked first by the ranking determination means as a priority;
The facial expression detection means has a priority when there are a plurality of subject persons in the subject image data and a priority selected by the first priority selection means exists in the plurality of subject persons. Detecting that the person's facial expression is a predetermined facial expression over the other subject person in the subject image data,
The electronic still camera according to claim 1, wherein the automatic photographing unit automatically photographs in accordance with a detection output of the facial expression detection unit. The electronic still camera according to claim 2,
A second priority selection means for selecting a subject person of the subject image data determined as the lowest ranking by the ranking determination means as a priority;
The facial expression detection means has priority when a plurality of subject persons exist in the subject image data and a priority selected by the second priority selection means exists in the plurality of subject persons. Detecting that the person's facial expression is a predetermined facial expression over the other subject person in the subject image data,
The electronic still camera according to claim 1, wherein the automatic photographing unit automatically photographs in accordance with a detection output of the facial expression detection unit.

Images