JP5640388B2 - Image processing apparatus, imaging apparatus, and image processing program - Google Patents

Description

  The present invention relates to an image processing device, an imaging device, and an image processing program.

  Conventionally, various techniques have been considered in order to obtain an image with a good shooting state such as a low camera shake. For example, the following electronic camera disclosed in Patent Document 1 discloses a technique for continuously capturing a subject to generate a plurality of images, evaluating the quality of the generated images, and storing only one image. Has been.

JP 11-136557 A

  By the way, in recent years, a person recognition technique for recognizing a person included in a subject, a face recognition technique for recognizing a person's face, and the like have been advanced. Is required to utilize.

  The present invention has been made in view of the above problems, and an object of the present invention is to obtain an image in a good shooting state using subject recognition technology.

An image processing apparatus according to the present invention includes an acquisition unit that acquires a plurality of images including similar subjects, a face detection unit that detects a human face for each of the plurality of images, and each of the plurality of images. The calculation unit that calculates a plurality of evaluation values related to the facial expression of the person, the plurality of images are compared based on the evaluation value, and the person is preferably captured from the plurality of images based on the comparison result. A selection unit that selects an image that can be estimated to be present, and the calculation unit detects, for each of the plurality of images, a first evaluation value that evaluates a degree of smile of the person as the evaluation value. The number of persons detected by the unit, the direction of the line of sight of the person, the direction of the face of the person, the area ratio of the face of the person in the entire image, the estimated age of the person, and the shooting time Less Calculating at least one second evaluation value evaluated based on one and a third evaluation value evaluating the degree of blinking of the person, and the selection unit is configured to calculate the first evaluation value. When the first evaluation value is equal to or greater than the predetermined threshold value, the person is preferably captured from the plurality of images based on the first evaluation value. If an image that can be presumed to be selected is selected and cannot be selected, the priority of the second evaluation value is set higher than the third evaluation value, and the person is preferably captured from the plurality of images. And when the first evaluation value is less than a predetermined threshold, the priority of the third evaluation value is made higher than the second evaluation value, From the plurality of images, it can be estimated that the person is preferably captured. You select the image.

  Moreover, you may provide the recording part which records only the image selected by the said selection part among the said some images.

  The imaging device of the present invention includes an imaging unit that captures a subject image and generates an image, and any one of the image processing devices described above, and the acquisition unit acquires the plurality of images from the imaging unit.

  When the imaging start instruction is given, the imaging unit continuously captures the subject image to generate the plurality of images, and the acquisition unit is generated by continuous imaging by the imaging unit. A plurality of images may be acquired.

  A setting unit configured to set a continuous shooting mode for continuously capturing the subject image by the imaging unit; and a recording unit configured to record a plurality of images generated by shooting in the continuous shooting mode. When an imaging start instruction is performed in a state where the continuous shooting mode is set by the setting unit, the imaging unit continuously captures the subject image and generates the plurality of images. The plurality of images generated by continuous imaging by the imaging unit are acquired, and the recording unit records the image selected by the selection unit as a representative image when grouping and recording the plurality of images. You may do it.

  In addition, a configuration obtained by converting the configuration related to the above invention into an image processing program for realizing image processing for image data to be processed is also effective as a specific aspect of the present invention.

  According to the present invention, an image in a good shooting state can be obtained using subject recognition technology.

It is a functional block diagram of the digital camera of this embodiment. It is a flowchart which shows operation | movement of the control part 7 when an imaging instruction | indication is performed. It is a figure explaining face detection. It is a figure explaining each evaluation value and a face database. It is an example of the table regarding a smile degree. It is another flowchart which shows operation | movement of the control part 7 when an imaging instruction | indication is performed.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. Hereinafter, as an example, a digital camera will be described as an example.

  FIG. 1 is a block diagram showing a configuration of a digital camera 1 in the present embodiment. As shown in FIG. 1, the digital camera 1 includes an imaging lens 2, an imaging device 3, an A / D conversion unit 4, a buffer memory 5, an image processing unit 6, a control unit 7, a display unit 8, an operation unit 9, and a memory 10. A recording I / F unit 11, a recording medium 12, a connection unit 13, and a bus 14.

  The imaging lens 2 forms a subject image on the imaging surface of the imaging element 3. The imaging lens 2 may be an imaging optical system including a plurality of lenses such as a zoom lens and a focus lens. The imaging element 3 photoelectrically converts the subject light that has passed through the imaging lens 2 and outputs analog image signals corresponding to R, G, and B colors. The imaging device 3 performs thinning-out reading at predetermined intervals during non-imaging, and acquires a through image (image for composition confirmation). Thereafter, the imaging device 3 acquires the main image when an imaging instruction is given.

  An image signal output from the image sensor 3 is input to the A / D converter 4. The A / D conversion unit 4 performs A / D conversion on the analog image signal output from the image sensor 3 and changes it to a digital image signal. The digital image signals are collected into one frame and recorded in the buffer memory 5 as image data. The buffer memory 5 temporarily records image data in the pre-process and post-process of image processing by the image processing unit 6.

  The image processing unit 6 performs image processing on the image data recorded in the buffer memory 5. Examples of the image processing include well-known white balance adjustment, color interpolation, gradation conversion processing, and contour enhancement processing. The image processing unit 6 also executes a process of compressing in JPEG (Joint Photographic Experts Group) format or the like before recording an image file on the recording medium 12, and a process of decompressing and restoring the compressed data.

  The image processing unit 6 performs a trimming process for cutting out a part of the image and generating a new image when creating a face database, which will be described later.

  The control unit 7 performs overall control of the digital camera 1 according to a predetermined sequence program, and executes various calculations (AF, AE, etc.) necessary for imaging. The control unit 7 has functions of a face detection unit 15 and an evaluation value calculation unit 16.

  The face detection unit 15 extracts feature points from the through image, the main image, and the image recorded on the recording medium 12, and detects the face area, the size of the face area (face area), and the like. Examples of the feature points include eyebrow, eye, nose, lip end points, face contour points, head apexes, chin lower end points, and the like. Then, the face detection unit 15 specifies the position information of the face area. For example, the face detection unit 15 calculates the X coordinate and Y coordinate of the pixels included in the face area, with the horizontal direction of the image as the X axis and the vertical direction as the Y axis. Furthermore, the face detection unit 15 determines the detected facial expression, age, and the like of the person by the arithmetic processing using the feature points described above. Details of the evaluation value calculation unit 16 will be described later.

  The display unit 8 displays various images under the control of the control unit 7. Various images displayed on the display unit 8 include a through image for composition confirmation at the time of shooting, a main image, an image recorded on the recording medium 12, a menu image, and the like. The display unit 8 includes a touch panel 17. The touch panel 17 detects a position where a finger or a stylus is touched on the panel, a dragged direction, and the number of times the drag is performed. The state of the touch panel 17 is detected by the control unit 7. Note that the touch panel 17 is configured by a transparent panel having a size equivalent to that of the display unit 8 because the touch panel 17 is stacked on the entire surface of the display unit 8. Moreover, in the example of this embodiment, the touch panel 17 is comprised with the electrostatic capacitance type panel which senses the electrical signal by static electricity. Not only the capacitance type, but also a resistive film type panel that detects a change in voltage due to pressure may be used.

  The operation unit 9 includes a release button (not shown), a cross key, and the like. The operation state of the operation unit 9 is detected by the control unit 7 and a sequence based on the detected state of the operation unit 9 is executed.

  The recording I / F unit 11 includes a connector for connecting the recording medium 12. When the recording I / F unit 11 and the recording medium 12 are connected, data writing / reading is executed on the recording medium 12.

  The connection unit 13 includes input / output terminals and the like, and is connected to an external device of the digital camera 1 to input / output image data and the like. The bus 14 connects the buffer memory 5, the image processing unit 6, the control unit 7, the display unit 8, the memory 10, and the recording I / F unit 11, thereby executing input / output of data and signals.

  The digital camera 1 having the above-described configuration has an automatic smile mode for automatically performing continuous imaging (so-called continuous shooting) in accordance with a shooting instruction and acquiring an image in which a person included in a subject is captured with a preferable facial expression. The automatic smile mode may be provided in parallel with an existing scene mode such as a portrait mode or a landscape mode, or may be provided as a mode that can be set in addition to the existing scene mode. The automatic smile mode may be set manually by a user operation via the operation unit 9 or the touch panel 17, or may be automatically set by the control unit 7. In the case of manual setting by user operation, a dedicated button or the like may be provided on the operation unit 9 or a menu screen or the like may be displayed on the display unit 8.

  With reference to the flowchart shown in FIG. 2, the operation of the control unit 7 when a shooting instruction is performed in the state where the automatic smile mode is set to ON will be described.

  In step S1, the control unit 7 determines whether or not a shooting instruction has been issued. The control unit 7 stands by until a shooting instruction is given, and proceeds to step S2 when determining that the shooting instruction has been given.

  Note that the shooting instruction may be performed based on a user operation via the release button 18 or the touch panel 17 or may be automatically performed by the control unit 7. The case where the shooting instruction is automatically issued by the control unit 7 means that the subject is in a predetermined state (face orientation is front, smile is recognized, etc.) by self-timer shooting or subject recognition performed in advance. This includes automatic shooting that is sometimes performed.

  In step S <b> 2, the control unit 7 controls the imaging lens 2, the imaging device 3, the A / D conversion unit 4, and the like to continuously capture a subject image and generate a plurality of images. The control unit 7 temporarily records the generated plurality of images in the buffer memory 5.

  The control unit 7 automatically performs continuous shooting regardless of the setting in the electronic camera 1. The shooting conditions at the time of continuous shooting are set in the same manner as in the known technique. However, unlike continuous shooting, which is conventionally performed to obtain an image with a good shooting state with respect to focus and camera shake, the purpose of the continuous shooting in this embodiment is to obtain an image with a good expression of the subject. And Therefore, it is not preferable that the shooting interval in continuous shooting is too short. Therefore, for example, continuous shooting may be performed at a shooting interval of about 3 to 5 frames per second. Further, the number of shots is predetermined (for example, 5 frames).

  In step S3, the control unit 7 performs face detection for each of the plurality of images generated in step S2. The face detection unit 15 performs face detection for each of the images generated in step S2. The specific method of face detection is the same as that of the known technique, and thus the description thereof is omitted.

  Note that the control unit 7 may perform face detection on other images using the result of face detection performed on an arbitrary image among the plurality of images generated in step S2. If face detection is performed prior to the shooting instruction described in step S1, face detection may be performed on each of the plurality of images generated in step S2 using the face detection result. . That is, when face detection is performed for a through image, the control unit 7 may perform face detection in each of the plurality of images generated in step S2 based on the result of face detection in the through image. When face detection is performed on a through image, it is preferable to display a frame on the detected face portion in the through image displayed on the display unit 8, for example, as shown in FIG.

In step S4, the control unit 7 calculates each evaluation value related to the facial expression for each of the faces detected in step S3. Each evaluation value calculated by the evaluation value calculation unit 16 includes the following evaluation values.
(1) Smile level The evaluation value calculation unit 16 detects a smile level for each of the faces detected in step S3 using a known technique (for example, facial expression recognition, pattern matching, etc.). In this embodiment, as an example, smile levels are shown in four stages from 0 (no smile) to 3 (big laugh).
(2) Gaze direction The evaluation value calculation unit 16 detects the gaze direction (front, diagonal, sideways, etc.) for each of the faces detected in step S3 using a known technique, and an evaluation value indicating the detection result. Is calculated. In the present embodiment, the direction of the line of sight is indicated by four types: 0 (front of line of sight), 1 (left direction of line of sight), 2 (right direction of line of sight), and 3 (line of sight line cannot be detected).
(3) Face direction The evaluation value calculation unit 16 detects the face direction (front, diagonal, sideways, etc.) using a known technique for each of the faces detected in step S3, and an evaluation value indicating the detection result. Is calculated. In this embodiment, the face direction is indicated by four types: 0 (front direction facing the face), 1 (left direction facing the face), 2 (right direction facing the face), and 3 (undetectable).
(4) Face Area The evaluation value calculation unit 16 obtains the area of the face part for each of the faces detected in step S3.
(5) Age Evaluation value calculation unit 16 estimates the age of the person using a known technique for each of the faces detected in step S3, and calculates an evaluation value indicating the estimation result.
(6) The degree of blinking The evaluation value calculation unit 16 detects the degree of blinking using a known technique (for example, facial expression recognition) for each of the faces detected in step S3, and calculates an evaluation value indicating the detection result. To do. In the present embodiment, the degree of blinking is indicated by four types: 0 (no blinking), 1 (some blinking), 2 (both eyes blinking), and 3 (no blinking detection).

  In step S5, the control unit 7 creates a face database based on the face detection performed in step S3 and each evaluation value calculated in step S4. First, the control unit 7 cuts out a face part image for each of the plurality of images generated in step S2 based on the result of the face detection performed in step S3. For example, as illustrated in FIG. 3, when five persons are included, for each of the plurality of images generated in step S <b> 2, a face part image is cut out to generate a thumbnail image. Then, a database is created by associating each thumbnail image with each evaluation value calculated in step S4. FIG. 4 illustrates a database of persons shown in FIG.

  Note that the thumbnail image described above may be performed before the calculation of each evaluation value described in step S4. That is, a thumbnail image is generated from each of the plurality of images generated in step S2 based on the result of the face detection performed in step S3, and the evaluation values described in step S4 are calculated based on the generated thumbnail image. May be performed. By adopting such a configuration, it can be expected to shorten the processing time.

  In step S6, the control unit 7 determines whether or not face designation is performed. If the control unit 7 determines that face designation is performed, the process proceeds to step S7. On the other hand, if it is determined that no face is designated, the control unit 7 proceeds to step S8 described later.

  The face designation is a function for designating a specific person in advance. For example, by registering the face image of the specific person in advance, the image of the person can be reliably acquired. The face designation is performed in advance by a user operation via the operation unit 9 or the touch panel 17. For example, as described with reference to FIG. 3, in the through image displayed on the display unit 8, in the state where a frame is displayed on the detected face portion, the user can select any person with the touch panel 17. It is good also as a structure which can designate. Also, face designation may be performed for a plurality of persons, or face designation may be performed for a plurality of persons with priorities.

  In step S7, the control unit 7 selects an image in which the face designated as a face is a smile from a plurality of images based on the database created in step S5. Whether a specific person with a face specified is included in an image is calculated by calculating the position of the facial feature point, the size of each facial part, the relative distance of each feature point, etc. It can be determined by comparing the information based on the image of the specified person and calculating the similarity.

  Based on the smile level calculated in step S4 and the like, the control unit 7 selects an image in which the face designated as a smile is selected from the plurality of images generated in step S2, and selects a plurality of selected images (only one image). Only in some cases).

  In step S8, the control unit 7 calculates the smile level. Based on the smile level calculated in step S4, the control unit 7 sets the image for each of the plurality of images generated in step S2 (however, in the case where selection is performed in step S7). The degree of smile, which is an evaluation value representative of, is calculated. The evaluation value calculation unit 16 calculates the smile level using the following equation.

(Smiling degree) = (Sum of smiling coefficients) ÷ (Number of detected faces) (Equation 1)
In Equation 1, the smile coefficient is a coefficient determined for each level with respect to the smile level calculated in step S4.

  FIG. 5 illustrates a table indicating the relationship between the smile level and the smile coefficient. As shown in FIG. 5, the smile coefficient increases as the smile level described in step S4 increases.

  In step S9, the control unit 7 determines whether or not the smile level is equal to or greater than a threshold value. If the controller 7 determines that the smile level is equal to or greater than the threshold, the process proceeds to step S10. On the other hand, if it determines with the smile degree not being more than a threshold value, the control part 7 will progress to step S10 mentioned later.

  The control unit 7 compares the representative value (average value, maximum value, minimum value, etc.) of each smile degree obtained in step S8 with a predetermined threshold value, and if representative value ≧ threshold value, the process proceeds to step S10. If representative value <threshold, the process proceeds to step S11.

  In step S10, the control unit 7 compares the plurality of images generated in step S2 (however, in the case where selection is performed in step S7), the image having the highest evaluation value regarding smiles. Is selected as the recorded image.

First, the control unit 7 compares the smile degree calculated in step S8 with respect to a plurality of images generated in step S2 (however, in the case where selection is performed in step S7, the image after selection). Then, an image having the maximum smile degree is selected as a recorded image. When the recorded image cannot be selected by comparing the smile levels, the control unit 7 repeats the comparison according to the following priority order based on each evaluation value calculated in step S4.
(1) Number of face detections (2) Gaze direction (3) Face direction (4) Face area (5) Age (6) Blink degree Among the evaluation values of (1) to (6) described above, (1) Each evaluation value other than the number of detected faces has an evaluation value for each person included in the image (a part of persons when face designation is performed in step S6). For this reason, representative values (average value, maximum value, minimum value, etc.) of evaluation values regarding a plurality of persons may be obtained and compared. Further, when a plurality of persons have priorities, weighting or the like may be performed.

In step S11, the control unit 7 compares the plurality of images generated in step S2 (however, in the case where sorting is performed in step S7), the evaluation value regarding blinking is the highest. Select an image as the recorded image.
First, the control unit 7 compares the degree of blinking calculated in step S4 with respect to the plurality of images generated in step S2 (however, in the case where selection is performed in step S7, the image after selection). Then, an image with the smallest blink degree is selected as a recorded image.

  Note that the degree of blinking has an evaluation value for each person included in the image (a part of persons when face designation is performed in step S6). For this reason, representative values (average value, maximum value, minimum value, etc.) of evaluation values regarding a plurality of persons may be obtained and compared. Further, when a plurality of persons have priorities, weighting or the like may be performed.

  In step S12, the control unit 7 records the recording image selected in step S10 or step S11 on the recording medium 12 via the recording I / F unit 11, and ends the series of processes.

  Note that when recording a recorded image in step S12, a preview image for confirming shooting may be generated from the recorded image selected in step S10 or step S11 and displayed on the display unit 8.

  The processing performed in the above-described flowchart can also be used when executing a so-called continuous shooting mode in which continuous shooting is performed. Hereinafter, an operation of the control unit 7 when a shooting instruction is performed in a state where the continuous shooting mode is set will be described with reference to a flowchart shown in FIG.

In step S21, the control unit 7 determines whether or not a shooting instruction has been issued , as in step S1 of the flowchart of FIG. The control unit 7 waits until a shooting instruction is given, and proceeds to step S22 when determining that the shooting instruction has been given.

  In step S <b> 22, the control unit 7 controls the imaging lens 2, the imaging device 3, the A / D conversion unit 4, and the like to continuously capture a subject image and generate a plurality of images. The control unit 7 temporarily records the generated plurality of images in the buffer memory 5. Note that the shooting conditions during continuous shooting are set in the same manner as in the known art. However, unlike step S2 in the flowchart of FIG. 2, continuous shooting is performed at a shooting interval according to the setting.

  In step S23 to step S29, the control unit 7 performs the same processing as in step S3 to step S19 in the flowchart of FIG.

  In step S30, the control unit 7 performs the same process as in step S10 in the flowchart of FIG. 2, and selects a representative image instead of the recorded image.

  In step S31, the control unit 7 performs processing similar to that in step S11 in the flowchart of FIG. 2, and selects a representative image instead of the recorded image.

  In step S32, the control unit 7 groups the plurality of images generated in step S22, records them on the recording medium 12 via the recording I / F unit 11, and ends the series of processes. At this time, the control unit 7 records the image selected in step S30 or step S31 as a representative image of the group.

  Note that when recording a recorded image in step S12, a preview image for confirming shooting may be generated from the recorded image selected in step S10 or step S11 and displayed on the display unit 8.

  As described above, according to the present embodiment, a plurality of images are acquired, and human face detection is performed for each of these images. Then, for each of the plurality of images, calculate at least one kind of evaluation value related to the facial expression of the person, compare the plurality of images based on the calculated evaluation value, and based on the comparison result, from the plurality of images, Select an image that can be presumed that a person is preferably captured.

  Therefore, it is possible to obtain an image in a good shooting state using the subject recognition technique. In recent years, with the increase in recording media capacity, camera performance, and speed, a large number of similar images are taken, and all images are recorded, which increases recording time and data volume. An increase was occurring. As a result, the load of the user's sorting work has been increased. On the other hand, according to the present embodiment, it is possible to efficiently obtain an image in a good shooting state.

  Further, according to the present embodiment, an evaluation value obtained by evaluating the degree of smile of a person for each of a plurality of images is calculated as the evaluation value. Therefore, an image in which a person is smiling can be obtained.

  Further, according to the present embodiment, as the evaluation value, the number of persons detected by the detection unit, the direction of the line of sight of the person, the direction of the face of the person, and the person who occupies the entire image as the evaluation values At least one type of evaluation value is calculated based on at least one of the area ratio of the face, the estimated age of the person, the degree of blinking of the person, and the shooting time. Therefore, it is possible to obtain an image in a good shooting state according to the state of the person.

  Further, according to the present embodiment, as the evaluation value, the first evaluation value obtained by evaluating the degree of smile of the person, the number of persons detected by the detection unit for each of the plurality of images, and the line of sight of the person At least one type based on at least one of the orientation, the orientation of the person's face, the area ratio of the person's face in the entire image, the estimated age of the person, the degree of blinking of the person, and the shooting time The second evaluation value is calculated. First, based on the first evaluation value, an image that can be estimated that a person is preferably reflected is selected from a plurality of images, and when selection is not possible, a plurality of images are calculated based on the second evaluation value. From the image, an image that can be estimated that a person is preferably captured is selected. Therefore, even if the facial expression of the person is not a smile, an image with a good shooting state can be obtained.

  Moreover, according to this embodiment, only the image selected by the selection part among several images is recorded. Therefore, it is possible to prevent wasteful use of the recording capacity of the recording unit.

  In addition, according to the present embodiment, a setting unit that sets a continuous shooting mode in which a subject image is continuously captured by the imaging unit, and a recording unit that groups and records a plurality of images generated by shooting in the continuous shooting mode When the shooting start instruction is performed in a state where the continuous shooting mode is set by the setting unit, the imaging unit continuously captures the subject image to generate a plurality of images, and the recording unit When grouping and recording images, the image selected by the selection unit is recorded as a representative image. Therefore, when a plurality of images generated by shooting in the continuous shooting mode are grouped and recorded, a suitable image can be recorded as a representative image.

  In the present embodiment, an example is shown in which a plurality of images are generated in step S2 of the flowchart of FIG. 2 and the processes after step S3 are performed on these images. However, the present invention is not limited to this example. For example, two images are generated in step S2, the processing from step S3 to step S10 is performed on the two images, and one image is selected. Even if the process of generating one image again and selecting one of the selected image and the newly generated image is repeated a predetermined number of times, the same effect as in the present embodiment can be obtained. Can do. In this case, since only two images are temporarily recorded in the buffer memory 5 at the maximum, the capacity of the buffer memory 5 can be suppressed.

  Moreover, in this embodiment, each evaluation value in step S4 of the flowchart of FIG. 2 is an example, and the present invention is not limited to this example. Further, the calculation timing of each evaluation value is not limited to this example. For example, in step S8, only the evaluation value necessary for calculating the smile level is calculated in step S4, and when a recorded image cannot be selected based on the smile level when selecting a recorded image in step S10, a new It is good also as a structure which calculates an evaluation value.

  In the present embodiment, the facial expression of a person has been described mainly with a smile, but the present invention is not limited to this example. For example, it is good also as a structure which performs various processes regarding a crying face and an angry face.

  In the present embodiment, an example is shown in which each evaluation value is independently compared when selecting a recorded image in step S10 of the flowchart of FIG. 2, but the present invention is not limited to this example. For example, a configuration may be adopted in which each evaluation value is added with a predetermined weight to calculate a representative evaluation value, and the recorded image is selected by comparing the representative evaluation values.

  In the present embodiment, an example in which a recorded image is selected in step S10 of the flowchart of FIG. 2 is shown, but the present invention is not limited to this example. For example, in step S10, a plurality of images may be ranked for evaluation. Such information may be recorded as tag information for each image.

  In this embodiment, the case where a series of processing is performed at the time of imaging is described as an example. However, the same processing as that described in this embodiment is performed on an image recorded on the recording medium 12. Also good. That is, when a reproduction target image is selected, the processing after step S3 in the flowchart of FIG. 2 may be performed to select a recorded image. In addition, when a reproduction target image is selected, the processing after step S23 in the flowchart of FIG. 6 may be performed to select a representative image.

  In the present embodiment, the digital camera 1 has been described as an example, but the present invention is not limited to this example. For example, the present invention can be similarly applied to a digital camera having a configuration other than that illustrated in FIG. 1, such as a digital camera without a touch panel. Further, the present invention can be similarly applied to a digital camera capable of capturing a moving image. Furthermore, the present invention can be similarly applied to a digital camera mounted on a mobile phone or the like.

  Moreover, it is good also as a structure which implement | achieves a part or all of the process which the control part 7 of this embodiment performed with a computer. In this case, an image processing program for executing part or all of the processing (FIGS. 2 and 6) performed by the control unit 7 of the present embodiment is stored in a computer-readable recording medium. By reading the processing program from the recording medium and executing it on a computer, the same processing can be performed to obtain the same effect as the present embodiment.

DESCRIPTION OF SYMBOLS 1 ... Digital camera, 3 ... Image sensor, 6 ... Image processing part, 7 ... Control part, 8 ... Display part, 15 ... Face detection part 15 ... Evaluation value calculation part

Claims (6)

An acquisition unit for acquiring a plurality of images including similar subjects;
A face detection unit for detecting a person's face for each of the plurality of images;
For each of the plurality of images, a calculation unit that calculates a plurality of evaluation values related to the facial expression of the person,
A selection unit that compares the plurality of images based on the evaluation value, and selects an image that can be estimated that the person is preferably reflected from the plurality of images based on a comparison result ;
The calculation unit, as the evaluation value, a first evaluation value that evaluates the degree of smile of the person, the number of persons detected by the detection unit for each of the plurality of images, At least one kind of evaluation evaluated based on at least one of the direction of the line of sight, the direction of the face of the person, the area ratio of the face of the person in the entire image, the estimated age of the person, and the shooting time Calculating a second evaluation value and a third evaluation value for evaluating the degree of blinking of the person;
The selection unit compares the first evaluation value with a predetermined threshold value,
If the first evaluation value is greater than or equal to a predetermined threshold, first, based on the first evaluation value, select an image that can be estimated that the person is preferably reflected from the plurality of images, If the selection is not possible, the priority of the second evaluation value is set higher than the third evaluation value, and an image that can be presumed that the person is preferably reflected is selected from the plurality of images. ,
When the first evaluation value is less than a predetermined threshold, the priority of the third evaluation value is set higher than the second evaluation value, and the person is preferably captured from the plurality of images. Images that can be estimated to be
An image processing apparatus. The image processing apparatus according to claim 1 .
An image processing apparatus comprising: a recording unit that records only the image selected by the selection unit among the plurality of images. An imaging unit that captures a subject image and generates an image;
The image processing apparatus according to claim 1 or 2 ,
The acquisition unit is configured to acquire the plurality of images from the imaging unit. The imaging device according to claim 3 .
When the imaging start instruction is given, the imaging unit continuously captures the subject image to generate the plurality of images,
The acquisition unit acquires a plurality of images generated by continuous imaging by the imaging unit. The imaging device according to claim 3 .
A setting unit for setting a continuous shooting mode in which the subject image is continuously captured by the imaging unit;
A recording unit that groups and records a plurality of images generated by shooting in the continuous shooting mode;
When the imaging unit is instructed to start shooting while the continuous shooting mode is set by the setting unit, the imaging unit continuously captures the subject image and generates the plurality of images,
The acquisition unit acquires the plurality of images generated by continuous imaging by the imaging unit,
The recording unit records an image selected by the selection unit as a representative image when the plurality of images are grouped and recorded. An image processing program for performing image processing on an image to be processed by a computer,
An acquisition step of acquiring a plurality of images including similar subjects as the image to be processed;
For each of the plurality of images, a face detection step for detecting a human face;
For each of the plurality of images, a calculation step of calculating a plurality of evaluation values related to the facial expression of the person;
A selection step of comparing the plurality of images based on the evaluation value, and selecting an image that can be estimated that the person is preferably reflected from the plurality of images based on a comparison result ;
In the calculation step, as the evaluation value, a first evaluation value that evaluates the degree of smile of the person, the number of persons detected in the detection step for each of the plurality of images, At least one kind of evaluation evaluated based on at least one of the direction of the line of sight, the direction of the face of the person, the area ratio of the face of the person in the entire image, the estimated age of the person, and the shooting time Calculating a second evaluation value and a third evaluation value for evaluating the degree of blinking of the person;
In the selection step, the first evaluation value is compared with a predetermined threshold value,
If the first evaluation value is greater than or equal to a predetermined threshold, first, based on the first evaluation value, select an image that can be estimated that the person is preferably reflected from the plurality of images, If the selection is not possible, the priority of the second evaluation value is set higher than the third evaluation value, and an image that can be presumed that the person is preferably reflected is selected from the plurality of images. ,
When the first evaluation value is less than a predetermined threshold, the priority of the third evaluation value is set higher than the second evaluation value, and the person is preferably captured from the plurality of images. Images that can be estimated to be
An image processing program characterized by that.

Images