JP5353293B2 - Image processing apparatus and electronic still camera - Google Patents

Description

  The present invention relates to an image processing apparatus that performs image processing, and an electronic still camera including the image processing apparatus.

  2. Description of the Related Art Conventionally, cameras that detect a subject's smile and automatically capture an image are known. For example, the camera disclosed in Patent Document 1 determines the image signal when the facial expression detected from the image signal is a smile during the period until the image signal obtained by imaging is recorded on the recording medium. Automatically record on the recording medium.

  However, it is difficult to acquire a preferable smile image of the subject. For example, if the subject is not good at making a smile, or if the detection condition is not satisfied even if a smile is made, imaging is not performed and a smile image cannot be acquired. On the other hand, even if a smile image can be acquired, there is a case where a desired smile image is not acquired unexpectedly if confirmed.

  Therefore, an object of the present invention is to provide an image processing apparatus and an electronic still camera that can easily acquire a desired smile image.

An image processing apparatus as an example of the present invention includes a face detection unit that detects a face from a first image, an image storage unit that can store a plurality of images obtained by photographing a person's smile, and an image stored in the image storage unit from an image selection section for selecting a different second image from the first image, as a reference evaluation value smile evaluation value indicating the degree of smile obtained from the second image, the detected face from the first image and a correcting section that corrects, based on the criteria evaluation value.

  According to the image processing apparatus and the electronic still camera of the present invention, a desired smile image can be easily acquired.

It is a block diagram which shows the structure of the electronic still camera 1 in 1st Embodiment of this invention. It is a figure which shows the example of the display part 11 at the time of imaging of a best smile image. 7 is a flowchart illustrating an operation of the control unit 7 during imaging in the smile mode in the electronic still camera 1. It is a flowchart which shows operation | movement of the control part 7 at the time of reproduction | regeneration in the electronic still camera 1 of 2nd Embodiment. It is a figure which shows the example of the display part 11 at the time of reproduction | regeneration in the electronic still camera 1 of 2nd Embodiment.

(First embodiment)
Hereinafter, a first embodiment of the present invention will be described with reference to the drawings. In this embodiment, an electronic still camera will be described as an example of the image processing apparatus of the present invention. FIG. 1 is a block diagram showing a configuration of an electronic still camera 1 according to the first embodiment of the present invention. As shown in FIG. 1, an electronic still camera 1 includes an imaging lens 2, an imaging device 3, an A / D conversion unit 4, a buffer memory 5, an image processing unit 6, a control unit 7, a display unit 11, a memory 12, and an operation unit. 13, a recording I / F unit 16, a recording medium 17, and a bus 18.

  The imaging lens 2 forms a subject image on the imaging surface of the imaging element 3. In addition to the imaging lens 2, an imaging optical system including a plurality of lenses such as a zoom lens and a focus lens may be used. The imaging element 3 photoelectrically converts the subject light that has passed through the imaging lens 2 and outputs analog image signals corresponding to R, G, and B colors. In the photographing mode, the image sensor 3 performs thinning readout at predetermined intervals to obtain a through image. Thereafter, the image sensor 3 acquires the main image when the release button 14 is fully pressed. Further, the image pickup device 3 automatically acquires the main image when the facial expression identifying unit 9 described later identifies a smile from the through image. Hereinafter, a mode in which a main image is automatically acquired when a smile is identified will be described as a “smile mode”.

  An image signal output from the image sensor 3 is input to the A / D converter 4. The A / D converter 4 performs A / D conversion on the analog image signal output from the image sensor 3 and converts the analog image signal into a digital image signal. The digital image signals are collected into one frame and recorded in the buffer memory 5 as image data. The buffer memory 5 temporarily records image data in the pre-process and post-process of image processing by the image processing unit 6.

  The image processing unit 6 performs image processing on the image data stored in the buffer memory 5. Examples of the image processing include well-known white balance adjustment, color interpolation, gradation conversion processing, and contour enhancement processing. The image processing unit 6 also executes a process of compressing in JPEG (Joint Photographic Experts Group) format or the like before recording an image file on the recording medium 17, and a process of decompressing and restoring the compressed data. Further, the image processing unit 6 corrects the face detected by the face detection unit 8 described later. For example, the image processing unit 6 performs coordinate conversion and interpolation calculation using the pixel position coordinates, and changes the shape of the eyebrows, eyes, and mouth.

  The control unit 7 performs overall control of the electronic still camera 1 according to a predetermined sequence program, and executes various calculations (AF, AE, etc.) necessary for imaging. The control unit 7 has functions such as a face detection unit 8, a facial expression identification unit 9, and a face identification unit 10. The face detection unit 8 extracts feature points from the through image, the main image, and the image recorded on the recording medium 17 to detect the face, the size of the face, and the like. For example, the face detection unit 8 extracts a face by a feature point extraction process described in Japanese Patent Application Laid-Open No. 2001-16573. Examples of the feature points include eyebrow, eye, nose, and lip end points, face contour points, head apexes, and chin lower end points.

  The facial expression identification unit 9 identifies the facial expression detected by the face detection unit 8. For example, as shown in JP-A-8-249453 and JP-A-10-255043, the facial expression identification unit 9 has a total of six basic types of facial expressions such as anger, disgust, fear, sadness, smile, and surprise. Consider facial expressions, process time-series images from non-facial expressions to basic facial expressions, and identify facial expressions.

  The face identification unit 10 generates face recognition data that is feature amount information indicating the feature amount of a person's face. For example, when the “person registration mode” in which the person in the image is registered in advance is set, the face identification unit 10 generates face recognition data according to the following procedure. First, the control unit 7 controls the image sensor 3 to acquire a through image. Next, the control unit 7 matches the eyes and mouth of the guide displayed on the display unit 11 with the eyes and mouth of the subject to be registered, and acquires an image of the subject to be registered by the image sensor 3. The face detection unit 8 extracts facial feature points from this image. Then, the face identification unit 10 generates face recognition data from the position of the feature point detected by the face detection unit 8, the relative distance between the feature points, and the like. The generated face recognition data is stored in the memory 12.

  Further, the face identifying unit 10 determines whether or not the subject having the face detected by the face detecting unit 8 in the “face recognition photographing mode”, that is, the subject at the time of imaging matches the subject registered in advance. . For example, the face identification unit 10 calculates the position of the feature point detected by the face detection unit 8, the relative distance between the feature points, and the like. Next, the face identification unit 10 compares the above calculation result with the face recognition data stored in the memory 12, and determines the similarity between the face of the subject registered in advance and the face of the subject at the time of imaging. Ask. Then, the face identification unit 10 determines that the subject at the time of imaging matches the subject registered in advance when the similarity is greater than the threshold value.

  Further, the control unit 7 calculates a smile evaluation value indicating the degree of smile based on the smile evaluation reference table stored in advance in the memory 12. For example, the control unit 7 refers to a smile evaluation criterion table as shown in Japanese Patent Application Laid-Open No. 2004-46591, and calculates evaluation values related to face components such as eyebrows, eyes, and mouth shapes. . And the control part 7 adds each evaluation value, and calculates a smile evaluation value. The smile evaluation criteria table is based on the element area for storing data representing facial components (for example, eyebrows, eyes, mouth), the evaluation value of each corresponding component, and the state in which each component is flat. And an evaluation value area for storing an angle. For example, for the eyebrows, the evaluation value increases as the eyebrow lowers. As for the eyes, the evaluation value increases as the corners of the eyes decrease. For the mouth, the evaluation value increases as the mouth angle increases. The smile evaluation value is 100 points, and the closer to a full smile, the higher the score.

  The display unit 11 displays various images under the control of the control unit 7. Various images displayed on the display unit 11 include a through image, a main image, an image recorded on the recording medium 17, a menu image, and the like. The operation unit 13 includes a release button 14, a cross key 15, and the like. The release button 14 is operated by the user during imaging. The cross key 15 is operated on the above menu image or the like. The state of the release button 14 and the cross key 15 is detected by the control unit 7, and a sequence based on the detected button state is executed. The recording I / F unit 16 includes a connector for connecting the recording medium 17. When the recording I / F unit 16 and the recording medium 17 are connected, data writing / reading with respect to the recording medium 17 is executed. The bus 18 connects the buffer memory 5, the image processing unit 6, the control unit 7, the display unit 11, the memory 12, and the recording I / F unit 16, thereby executing input / output of data and signals. The output of the A / D conversion unit 4 is connected to the image processing unit 6 via the bus 18.

  By the way, conventionally, an electronic still camera that automatically acquires a smile image when a smile of a subject is detected is known. However, for example, if the subject is not good at making a smile, or if the detection condition is not satisfied even if a smile is made, imaging is not performed and a smile image cannot be acquired. On the other hand, even if a smile image can be acquired, there is a case where a desired smile image is not acquired unexpectedly if confirmed. Also, even if the subject is the same subject, there are users who like a smile image whose subject is a full smile, while there are users who like a smile image that the subject smiles. .

  Therefore, the electronic still camera 1 of the present embodiment stores a desired smile image (hereinafter referred to as the best smile image) and a smile evaluation value of the best smile image in the memory 12 in advance. The electronic still camera 1 corrects the face in the main image so that the smile evaluation value of the main image approaches the smile evaluation value of the best smile image. Thereby, for example, even an image that does not like a smile can be brought close to a desired smile image.

  In addition, the smile evaluation value of a preferable smile image is naturally different for different subjects. Therefore, in order to obtain a higher-quality smile image, it is desirable that the subject of the pre-registered best smile image and the subject at the time of imaging are the same person.

  Next, acquisition of the best smile image in the “person registration mode” will be described. First, the control unit 7 controls the image sensor 3 to acquire a through image. At this time, the control unit 7 may display a message such as “Please laugh!” On the display unit 11. Further, the electronic still camera 1 may be provided with a speaker (not shown) to output a sound such as “Please laugh!”. And the control part 7 will acquire the best smile image, when the release button 14 is pressed fully. Hereinafter, the best smile image acquired in advance in the “person registration mode” will be described with reference numeral 21. The control unit 7 calculates the smile evaluation value T of the best smile image 21. Moreover, the control part 7 displays the best smile image 21 and the smile evaluation value T (code | symbol 22) on the display part 11, as shown in FIG. Note that the control unit 7 may display the level of the smile evaluation value T instead of displaying the numerical value of the smile evaluation value T. Thereby, the user can visually confirm the degree of smile. Thereafter, the control unit 7 accepts a selection as to whether or not the best smile image 21 may be registered via the cross key 15 from the user. When registering the best smile image 21, the control unit 7 stores the best smile image 21 and the smile evaluation value T in the memory 12.

  FIG. 3 is a flowchart showing the operation of the control unit 7 during imaging in the smile mode in the electronic still camera 1. Hereinafter, for the sake of convenience, description will be made on the premise of a scene in which one person is imaged. In the first embodiment, the best smile image 21 and the smile evaluation value T are stored in the memory 12 in advance, and the subject of the best smile image 21 and the subject at the time of imaging are the same person.

  In step S101, the control unit 7 starts the smile mode. For example, the control unit 7 controls the imaging device 3 to start acquiring a through image.

  In step S102, the face detection unit 8 detects a face from the through image.

  In step S103, the facial expression identification unit 9 determines whether the face detected in step S102 is a smile. When it is a smile (when determination of step S103 becomes YES), it progresses to step S104. On the other hand, when it is not a smile (when determination of step S103 becomes NO), it returns to step S102.

  In step S104, the control unit 7 controls the image sensor 3 to acquire the main image PI.

  In step S105, the face detection unit 8 detects a face from the main image PI. Then, the control unit 7 calculates the position information of the pixels constituting the eyebrow, eye, and mouth areas based on the end points of the eyebrows, eyes, and mouth, and stores the calculated position information of the pixels in the buffer memory 5. To do.

  In step S106, the control unit 7 calculates the face smile evaluation value U detected from the main image PI in step S105. Note that the control unit 7 may display the main image PI and the smile evaluation value U on the display unit 11. The control unit 7 may display the level of the smile evaluation value U instead of displaying the numerical value of the smile evaluation value U.

  In step S <b> 107, the control unit 7 reads the smile evaluation value T from the memory 12.

  In step S108, the control unit 7 controls the image processing unit 6 to correct the main image PI based on the smile evaluation value T read in step S107. Hereinafter, the corrected main image PI is referred to as a main image PI ′, and the smile evaluation value of the main image PI ′ is referred to as a smile evaluation value U ′. Here, the image processing unit 6 determines the absolute value of the difference between the smile evaluation value U ′ and the smile evaluation value T to be smaller than the absolute value of the difference between the smile evaluation value U and the smile evaluation value T. The image PI is corrected. Thereby, the main image PI ′ becomes a smile image closer to the user's preference than the main image PI, and it is possible to prevent an unnatural smile on the subject in the main image PI ′ due to excessive correction. .

  For example, the smile evaluation value T is 85 points. The smile evaluation value U is set to 40 points. In this case, the image processing unit 6 corrects the main image PI so that 40 points <smile evaluation value U ′ ≦ 100 points. At this time, it is most desirable to correct the smile evaluation value U ′ to be equal to the smile evaluation value T, that is, 85 points, but an improvement effect can be expected as long as it approaches at least 85 points.

  Next, the control unit 7 reads the position information of the pixels constituting the eyebrow, eye, and mouth regions of the main image PI stored in the buffer memory 5 in step S105. The control unit 7 reads angle information corresponding to the corrected evaluation values of the eyebrows, eyes, and mouth from the smile evaluation reference table. Then, the image processing unit 6 performs coordinate conversion and interpolation calculation based on the read position information and angle information, and changes the shape of the eyebrows, eyes, and mouth of the main image PI to create the main image PI ′. . At this time, for the eyebrows or eyes, the higher the evaluation value, the lower the eyebrows or the corner of the eyes, and for the mouth, the higher the evaluation value, the higher the mouth corner.

  The smile evaluation reference table may store not only angle information but also a reference image in which the shape of each component is recorded. For example, for eyebrows and eyes, a higher evaluation value is associated with a reference image in which the bottom of the eyebrows or the bottom of the eyes is lowered. As for the mouth, a higher evaluation value is associated with a reference image with a higher mouth corner. Then, the image processing unit 6 changes the shape of the eyebrows, eyes, and mouth of the main image PI based on the reference image of the eyebrows, eyes, and mouth corresponding to the smile evaluation value U ′, and creates the main image PI ′. May be.

  In step S <b> 109, the control unit 7 displays the main image PI ′ and the smile evaluation value U ′ on the display unit 11. Thereby, the user can confirm the performance of the main image PI ′.

  In step S <b> 110, the control unit 7 receives a selection as to whether or not to record the main image PI ′ from the user via the cross key 15. In the case of recording (when the determination in step S110 is YES), the process proceeds to step S111. On the other hand, when not recording (when determination of step S110 becomes NO), it returns to step S108 and repeats correction | amendment until the main image PI 'which becomes a desired smile is obtained.

  In step S111, the control unit 7 records the main image PI 'as an image file on the recording medium 17, and ends a series of processes. The control unit 7 preferably records the smile evaluation value U ′ as information such as a tag in the image file together with the main image PI ′. Further, the control unit 7 may record the main image PI and the smile evaluation value U together with the main image PI ′ and the smile evaluation value U ′.

  In the first embodiment, an example is shown in which the subject of the best smile image 21 and the subject at the time of imaging are the same person, but they may not be the same person. When the subject of the best smile image 21 and the subject at the time of imaging are not the same person, the improvement effect can be expected, although the quality of the smile is not as good as when the subject is the same person. In addition, since it is not necessary to register the best smile image every time the subject is changed and imaged, the imaging becomes easy for the user.

  Further, whether or not to correct may be determined depending on whether or not the subject of the best smile image 21 and the subject at the time of imaging are the same person. In this case, after step S105, the face identifying unit 10 determines whether or not the subject of the main image PI matches the subject of the best smile image 21. If it is determined that they do not coincide with each other, the processing after step S106 may not be performed.

  In the first embodiment, the example in which the main image PI acquired in step S104 is corrected in step S108 has been described, but the present invention is not limited thereto. For example, the correction may be made only when the main image PI that gives a desired smile is not obtained in step S104. In this case, the control unit 7 determines whether or not a main image PI that provides a desired smile is obtained after step S104. When the main image PI that provides a desired smile is obtained, the main image PI is recorded in step S111 without performing the processing from step S105 to step S110. On the other hand, when the main image PI that gives a desired smile is not obtained, the processing from step S105 to step S111 may be performed. Thereby, a desired smile image can be obtained more easily.

  In some cases, a smile is not detected in step S103, and the main image PI is not acquired in step S104. Therefore, a configuration may be adopted in which the smile detection level in step S103 is automatically lowered when a predetermined time elapses after the smile mode is started in step S101. This makes it easy to avoid a situation where the user cannot capture images even though the electronic still camera 1 is held. If the smile detection level is lowered, there is a high possibility that a main image PI having a low smile evaluation value will be acquired. However, since it is corrected by the image processing unit 6 thereafter, the final performance of the main image PI ′ is obtained. Is no problem.

  The number of best smile images to be pre-registered may be one or more. The smaller the number of best smile images to be registered in advance, the more the capacity occupied in the memory 12 can be saved. In addition, smile images with various variations can be created as the number of best smile images increases.

  Here, a procedure in which the control unit 7 selects a best smile image as a correction reference when a plurality of images are stored as the best smile image in the memory 12 will be described. For example, the control unit 7 automatically selects any one of the plurality of images or selects any one of the plurality of images based on a user instruction. When selecting automatically, the control part 7 selects an image with the highest smile evaluation value as a best smile image. This is because an image having the highest smile evaluation value is generally likely to be a preferable smile image.

  Next, when selecting based on a user instruction, the control unit 7 displays a thumbnail image list of a plurality of images on the display unit 11. Then, the control unit 7 accepts selection of any thumbnail image from the user via the cross key 15, and sets the image corresponding to the selected thumbnail image as the best smile image. As a result, the best smile image can be selected according to the subjective criteria of the user, so-called preference.

  In the first embodiment, the best smile image is acquired in the “person registration mode”. However, the present invention is not limited to this. For example, the best smile image may be selected from the images recorded on the recording medium 17. When a plurality of images are recorded on the recording medium 17, as described above, the control unit 7 automatically selects one image from the images recorded on the recording medium 17, or according to a user instruction. One image may be selected from a plurality of images based on the image.

  As described above, the electronic still camera 1 according to the first embodiment corrects the face in the main image based on the smile evaluation value of the best smile image. Therefore, according to the electronic still camera 1 of the first embodiment, a desired smile image can be obtained.

  Further, since the electronic still camera 1 of the first embodiment identifies the smile of the subject and automatically acquires the main image, it is possible to eliminate the possibility of missing a photo opportunity.

(Second Embodiment)
Hereinafter, a second embodiment of the present invention will be described. In the first embodiment, an example in which the face in the main image is corrected at the time of imaging has been described. In the second embodiment, an example of correcting the face in the main image recorded on the recording medium 17 during reproduction will be described. Moreover, in 1st Embodiment, the example which acquires the best smile image previously in person registration mode was shown. The second embodiment shows an example in which the best smile image is selected from the images recorded on the recording medium 17. Here, the configuration of the electronic still camera 1 in the following embodiment is the same as that of the electronic still camera 1 of the first embodiment shown in FIG.

  FIG. 4 is a flowchart showing the operation of the control unit 7 during reproduction in the electronic still camera 1 of the second embodiment. For convenience, the description will be made on the assumption that an image of one person is captured.

  In step S201, the control unit 7 starts the reproduction mode. For example, the control unit 7 displays a menu image 41 on which a plurality of items relating to the retouch function and an item of “best smile retouch” are displayed on the display unit 11 (see FIG. 5A). Here, “best smile retouch” is an item indicating a function of correcting the recorded main image so as to approach the best smile image.

  In step S <b> 202, the control unit 7 accepts selection of the “best smile retouch” item from the user via the cross key 15. If the “Best Smile Retouch” item is selected, the control unit 7 will make a smile based on the “Best Smile image. A person with a slightly stiff expression will also have a pleasant smile. Or the like may be displayed on the display unit 11.

  In step S203, the control unit 7 reads a smile evaluation value from information such as a tag recorded in the image file of the recording medium 17, and reads an image having a smile evaluation value equal to or greater than a threshold value as a smile image. Then, the control unit 7 displays a list of thumbnail images corresponding to the read images in the thumbnail image display area 42 of the display unit 11 as shown in FIG. Further, the control unit 7 displays a message such as “Please select the best smile image” on the display unit 11. When the smile evaluation value is not recorded on the recording medium 17, the face detection unit 8 detects a face from the image recorded on the recording medium 17. Then, the facial expression identification unit 9 identifies facial expressions from the face detected by the face detection unit 8 and reads a smile image.

  In step S204, the control unit 7 accepts selection of any image from the list of thumbnail images displayed in step S203 via the cross key 15 from the user. Then, the control unit 7 sets the image corresponding to the selected thumbnail image as the best smile image. As a result, the best smile image can be selected according to the subjective criteria of the user, so-called preference.

  In step S205, the control unit 7 performs the same process as in step S203, and displays a list of thumbnail images corresponding to the smile images in the thumbnail image display area 42 of the display unit 11 (see FIG. 5 (3)). Further, the control unit 7 displays a message such as “Please select an image to correct” on the display unit 11.

  In step S206, the control unit 7 accepts selection of any image from the list of thumbnail images displayed in step S205 via the cross key 15 from the user. Then, the control unit 7 sets an image corresponding to the selected thumbnail image as a correction target image.

  Steps S207 to S210 respectively correspond to steps S108 to S111 in FIG. FIG. 5 (4) is a diagram illustrating a display example of the display unit 11 in step S209.

  In the second embodiment, an example in which selection of the best smile image is received from the user via the cross key 15 in step S204 is shown, but the present invention is not limited to this. For example, the control unit 7 may select any one of a plurality of images automatically recorded on the recording medium 17. For example, the control unit 7 reads the smile evaluation value from information such as a tag recorded in the image file of the recording medium 17 and sets the image having the highest smile evaluation value as the best smile image. This is because an image having the highest smile evaluation value is generally likely to be a preferable smile image. In this case, the operation for the user to select the best smile image can be omitted.

  In step S203, a list of thumbnail images of smile images whose smile evaluation value is greater than or equal to the threshold value is displayed on the display unit 11. However, the present invention is not limited to this. For example, first, the control unit 7 accepts input of a numerical value of a smile evaluation value or a keyword such as a big smile or a smile through the cross key 15 from the user. Then, the control unit 7 extracts images recorded on the recording medium 17 in accordance with the input numerical values and keywords, and displays a list of thumbnail images of the extracted images on the display unit 11. Alternatively, the control unit 7 classifies the images recorded in advance on the recording medium 17 according to the smile evaluation value, and displays a list of thumbnail images of the classified images on the display unit 11. Thereafter, the control unit 7 may be configured to accept selection of any thumbnail image from the user via the cross key 15 again. Thereby, the user can easily select the best smile image.

As described above, the electronic still camera 1 of the second embodiment corrects the face in the main image recorded on the recording medium 17 based on the smile evaluation value of the best smile image. Therefore, according to the electronic still camera 1 of the second embodiment, a desired smile image can be obtained as in the first embodiment.
(Supplement of embodiment)
In the first embodiment and the second embodiment, an example of an electronic still camera is shown, but the present invention is not limited to this. For example, a program installed in a computer can be provided with functions shown in a procedure for calculating smile evaluation values and a procedure for correcting eyebrow, eye, and mouth shapes in the first and second embodiments. In this case, it is possible to store the image processing program in a recording medium in advance and execute or install the image processing program after connecting to the computer.

  In 1st Embodiment and 2nd Embodiment, in order to calculate a smile evaluation value, although the example which calculates the evaluation value regarding the shape of eyebrows, eyes, and a mouth was shown, it is not restricted to this, The structure which has a correlation with a smile Any element can be used as long as it is an element.

  At the time of registration of the best smile image in the first embodiment, FIG. 2 shows an example in which the smile evaluation value T (reference numeral 22) is displayed on the display unit 11. However, the present invention is not limited to this. For example, the evaluation values of the eyebrows, eyes, and mouth may be displayed individually.

  In step S108 of the first embodiment, the absolute value of the difference between the smile evaluation value U ′ and the smile evaluation value T is set to be smaller than the absolute value of the difference between the smile evaluation value U and the smile evaluation value T. Although the example which correct | amends image PI was shown, it is not restricted to this. For example, the absolute value of the difference between each evaluation value of the main image PI ′ and each evaluation value of the best smile image 21 is the absolute value of the difference between each evaluation value of the main image PI and each evaluation value of the best smile image 21. Individual corrections are made so as to be smaller than the respective values. Thereby, correction with higher accuracy becomes possible.

  For example, each evaluation value of the best smile image 21 is 30 points for the eyebrows, 20 points for the eyes, and 35 points for the mouth. When the evaluation values of the main image PI are 15 points for the eyebrows, 15 points for the eyes, and 10 points for the mouth, the evaluation values of the main image PI ′ are corrected so as to satisfy the following inequalities.

15 points <corrected eyebrow evaluation value <45 points 15 points <corrected eye evaluation value <25 points 10 points <corrected mouth evaluation value <60 points At this time, each evaluation of the main image PI ' Although it is most preferable to correct the value so as to be equal to each evaluation value of the best smile image 21, an improvement effect can be expected as long as at least the evaluation value of the best smile image 21 is approached. The same may be applied to step S207 of the second embodiment.

  In step S108 of the first embodiment, an example is shown in which the smile evaluation value U is lower than the smile evaluation value T. However, the present invention is not limited to this, and the present invention can also be applied when the smile evaluation value U is higher than the smile evaluation value T. Thereby, when the subject in the main image is overly laughing, it can be corrected to a desired smile. The same applies to step S207 of the second embodiment.

  In step S108 of the first embodiment, an example is shown in which the eyebrows, eyes, and mouth are corrected in order to approximate the smile evaluation value of the best smile image, but this is not a limitation. For example, only the shape of at least one of eyebrows, eyes, and mouth may be corrected. The same applies to step S207 of the second embodiment.

  In step S108 of the first embodiment, an example in which each shape is corrected based on the eyebrow, eye, and mouth angle information of the main image PI has been described, but the present invention is not limited thereto. For example, each shape may be corrected based on the position information of the eyebrows, eyes, and mouth of the main image PI. Hereinafter, the case where the coordinates of the corners of the eyes and the eyes are used will be described as an example. For example, the coordinates of the corners of the eyes with respect to the eyes are recorded in the memory 12 in advance as reference coordinates. It can be said that if a straight line in which the corners of the eyes are lowered when the eyes and the corners of the eyes are connected, it is a smile. In addition, the larger the slope of the straight line connecting the eyes and the corners of the eyes, the lower the corners of the eyes, and the higher the smile evaluation value. In step S108, the control unit 7 obtains the coordinates of the two points of the subject's eye and the corners in the main image PI. For example, when it is desired to increase the smile evaluation value, the image processing unit 6 performs coordinate conversion of the corners of the eyes so that the corners of the eyes are lowered based on the reference coordinates. The same applies to eyebrows and mouths. The same applies to step S207 of the second embodiment.

  In step S108 of the first embodiment, the example in which the main image PI 'is created by changing the shape of each component of the main image PI has been shown, but each component may be replaced. For example, a table in which the reference images of the components having different shapes and the evaluation values are associated with each other is stored in the memory 12 in advance. Then, the image processing unit 6 uses the reference image of each component corresponding to the evaluation value closer to the evaluation value of each component of the best smile image than the evaluation value of each component of the main image PI. Replace each component of the PI. Furthermore, a table in which each component extracted from the best smile image and an evaluation value of each component are associated with each other may be stored in the memory 12 in advance. Then, the image processing unit 6 may replace each component in the main image PI with each component stored in the table. The same applies to step S207 of the second embodiment.

  Although an example in which image correction is performed in step S108 of the first embodiment has been described, it is not always necessary to perform image correction in step S108. For example, the processing after step S108 may not be performed.

  In the second embodiment, an example in which correction is performed so as to approach the best smile image is shown, but it is not always necessary to be the best smile image. For example, an image of an angry face or an image of a crying face may be used.

  In step S205 of the second embodiment, an example is shown in which a list of thumbnail images corresponding to smile images is displayed, but the present invention is not limited to this. For example, a list of thumbnail images corresponding to images including a person's face may be displayed regardless of smile evaluation values, or thumbnail images corresponding to images whose smile evaluation values are smaller than a threshold value, that is, images that are not smiling. A list may be displayed.

  In the first embodiment and the second embodiment, the scene that captures one person has been described. However, even when capturing a scene that includes a plurality of persons, the best smile image for each subject that has been registered in advance. Each subject can be corrected based on the smile evaluation value.

  In the smile mode of the first embodiment and the second embodiment, a smile evaluation value serving as a reference for acquiring the main image may be set for each subject registered in advance. Thus, a person who usually does not laugh very much can be set to automatically acquire the main image by identifying it as a smile even if he laughs a little.

  In the electronic still camera 1 of the first embodiment and the second embodiment, when capturing a scene including a plurality of persons such as a group photo, the main image can be acquired with priority given to the smile of a pre-registered subject. A function of “registered person priority” may be provided.

DESCRIPTION OF SYMBOLS 1 ... Electronic still camera, 3 ... Image sensor, 6 ... Image processing part, 7 ... Control part, 11 ... Display part, 12 ... Memory, 13 ... Operation part

JP 2008-042319 A

Claims (9)

A face detection unit for detecting a face from the first image;
An image storage unit capable of storing a plurality of images of a person's smile;
An image selection unit for selecting a second image different from the first image from the images stored in the image storage unit;
; And a correcting unit for correcting, based the extent of the resulting the smile from the second image as a reference evaluation value smile evaluation value indicating the detected face from the first image to the reference evaluation value An image processing apparatus. The image processing apparatus according to claim 1.
An evaluation value calculation unit for calculating a smile evaluation value indicating the degree of smile on the face detected from the first image;
The image processing apparatus, wherein the correction unit corrects the face detected from the first image so that the smile evaluation value calculated by the evaluation value calculation unit approaches the reference evaluation value. The image processing apparatus according to claim 1.
An evaluation value calculation unit for calculating a smile evaluation value indicating the degree of smile on the face detected from the first image;
The correction unit corrects the face detected from the first image so that the smile evaluation value calculated by the evaluation value calculation unit and the reference evaluation value are the same. apparatus. The image processing apparatus according to any one of claims 1 to 3,
An information storage unit for storing feature amount information indicating a feature amount of a smile of the second image;
A face identifying unit that determines whether or not the feature amount extracted from the face detected from the first image substantially matches the feature amount information stored in the information storage unit;
The correction unit corrects the face detected from the first image when it is determined that the feature amount of the face detected from the first image by the face identification unit substantially matches the feature amount information. An image processing apparatus. The image processing apparatus according to claim 1, wherein:
At least one of the smile evaluation value obtained from the first image, the smile evaluation value obtained from the first image corrected by the correction unit, or the reference evaluation value is numerically displayed, or An image processing apparatus further comprising an evaluation value display unit for displaying a level. The image processing apparatus according to any one of claims 1 to 5 ,
An image extraction unit that extracts an image including a smile from the image stored in the image storage unit;
The image selection unit selects an image to be used as the second image from the images extracted by the image extraction unit. The image processing apparatus according to claim 6 .
The evaluation value calculation unit obtains a smile evaluation value in the image extracted by the image extraction unit;
The image selection unit selects, as the second image, an image having the highest smile evaluation value calculated by the evaluation value calculation unit from among the images extracted by the image extraction unit. . The image processing apparatus according to any one of claims 1 to 7 ,
An image extraction unit for extracting an image including a smile from the image stored in the image storage unit;
An image display unit that displays a list of images extracted by the image extraction unit;
The image processing device, wherein the image selection unit selects an image to be used as the second image from images displayed on the image display unit. The image processing apparatus according to any one of claims 1 to 8 ,
An electronic still camera comprising: an image acquisition unit that acquires the first image.

Images