JP5050926B2 - Image output method, image output apparatus, and image output program - Google Patents

Description

  The present invention relates to an image output method, an image output apparatus, and an image output program, and more particularly to an image output method, an image output apparatus, and an image output program for outputting image data including a person image.

With the rapid spread of digital still cameras, users own a lot of image data. For this reason, how to manage a large number of image data is a big problem. In order to deal with such a problem, there has been proposed one that detects a human face from image data, evaluates the image quality of each person's face, and presents the result (see Patent Document 1). In this document, an evaluation value is calculated for each face of a person included in each image data, and the quality of the image data is presented by the average value. By doing so, it is possible to select image data with a good image quality of the face of the photographed person without actually confirming each image data, for example, to be printed.
JP 2006-252327 A

  However, the user does not always expect good image quality for all persons included in the image data. That is, even if the image quality of the faces of some persons included in the image data is poor, the user may think that the image data as a whole is good. For example, when a specific person is photographed with the aperture opened, the person present in the background is blurred and photographed, so the evaluation value based on the average of the evaluation of each face is low. In such a case, the image data is evaluated low even though the person existing in the background can be blurred as intended by the user.

  The present invention has been made in view of the above problems, and provides an image output method, an image output apparatus, and an image output program capable of evaluating image data in accordance with the image quality of an important person.

  In order to solve the above problem, when a face included in an image indicated by the image data is detected and the face detection unit detects a plurality of faces, a face evaluation value for each face is calculated based on the image quality of each face. Output. Then, an image evaluation value indicating an evaluation result of the image data is output based on the face evaluation value of which the face evaluation value is a predetermined value or more among the face evaluation values of each face. By doing so, it is possible to prevent the overall evaluation of the image data from deteriorating by chance based on a person's face or the like captured in the background in a blurred state. For example, the image data may be evaluated by limiting to the maximum face evaluation value among the face evaluation values of each face. In many cases, it can be estimated that shooting is performed with focus adjustment and exposure adjustment for a single person, and even if the entire image data is evaluated only for the best one, it is sufficient. You can evaluate the performance of shooting.

  The face evaluation value of each face is not limited to outputting the image evaluation value based only on the maximum face evaluation value, and is the same as the maximum face evaluation value among the face evaluation values of each face. The image evaluation value may be output based on a plurality of the face evaluation values belonging to the group. That is, when a group photo of several groups is taken, it can be estimated that focus adjustment and exposure adjustment are performed with emphasis on the entire group. Therefore, the image evaluation is performed based on the face evaluation value of the entire group. It is desirable to output a value.

  As a preferred configuration for indicating the evaluation result of the image data, an image indicating the face corresponding to the face evaluation value limited to output the image evaluation value is displayed. May be highlighted in comparison with other areas. In this way, it is possible to easily confirm on which face the image data has been evaluated. For example, it is possible to easily recognize whether or not a good image evaluation value is presented based on the image quality of the intended person's face.

  Furthermore, when outputting the face evaluation value for each face, it is desirable to output the face evaluation value based on the image quality related to the eyes of the face. According to the image quality of the eyes, the overall image quality of the face can be accurately evaluated. By the way, it can be considered that the image data to be printed in the past has achieved image quality that satisfies the user to some extent. Therefore, when displaying a list of images indicated by a plurality of the image data, the image data that has been a print target in the past may be indicated.

  Furthermore, the technical idea of the present invention can be implemented not only by a specific image output method but also by an image output apparatus. That is, the present invention can be specified as an image output apparatus having means corresponding to each step performed by the above-described image output method. Of course, when the above-described image output device reads a program to realize each of the above-described means, the technical features of the present invention can be applied to a program for executing a function corresponding to each of the means and various recording media on which the program is recorded. It goes without saying that the idea can be embodied. Needless to say, the image output apparatus of the present invention can be distributed not only by a single apparatus but also by a plurality of apparatuses. For example, each unit included in the image output apparatus can be distributed in both the printer driver executed on the personal computer and the printer. It is also possible to include each unit of the image output apparatus of the present invention in a printing apparatus such as a printer.

Hereinafter, embodiments of the present invention will be described in the following order.
1. Configuration of image evaluation device:
2. Image evaluation process flow:
3. Variations:
3-1. Modification 1:
3-2. Modification 2:

1. Configuration of Image Evaluation Apparatus FIG. 1 shows the configuration of a computer that specifically implements an image evaluation apparatus as an image output apparatus according to an embodiment of the present invention. In FIG. 1, a computer 10 includes a CPU 11, a RAM 12, a ROM 13, a hard disk drive (HDD) 14, a general purpose interface (GIF) 15, a video interface (VIF) 16, an input interface (IIF) 17, and a bus 18. The bus 18 implements data communication between the elements 11 to 17 constituting the computer 10, and communication is controlled by a chip set (not shown). The HDD 14 stores program data 14a for executing various programs including an operating system (OS), and the CPU 11 executes calculations according to the program data 14a while expanding the program data 14a in the RAM 12. .

  Further, the HDD 14 stores an image data ID input by a digital still camera or a scanner, and a number of face templates 14b and eye templates 14c used in pattern matching described later. Further, the HDD 14 stores a face detection database 14e generated by an image evaluation process described later. The GIF 15 provides an interface conforming to the USB standard, for example, and connects an external printer 20 and a digital still camera 30 to the computer 10. A plurality of image data IDs are also stored in a flash memory (not shown) of the digital still camera 30. The VIF 16 connects the computer 10 to an external display 40 and provides an interface for displaying an image on the display 40. The IIF 17 connects the computer 10 to an external keyboard 50a and a mouse 50b, and provides an interface for the computer 10 to acquire input signals from the keyboard 50a and the mouse 50b.

  FIG. 2 shows a software configuration of a program executed in the computer 10. In the figure, an operating system (OS) P1, an image evaluation application P2, and a printer driver P3 are executed. The OS P1 provides an interface between the programs, and the printer driver P3 executes a process for controlling the printer 20. The image evaluation application P2 includes an evaluation image selection unit P2a, a face detection unit P2b, a face organ detection unit P2c, a face evaluation value calculation unit P2d, an evaluation target limiting unit P2e, an image evaluation value calculation unit P2f, an evaluation result output unit P2g, and a print And an output unit P2h. Details of processing executed by each of the modules P2a to P2h constituting the image evaluation application P2 will be described together with a flow of image evaluation processing described later.

2. Flow of Image Evaluation Processing FIG. 3 shows the flow of image evaluation processing. In step S100, the evaluation image selection unit P2a displays a predetermined UI screen on the display 40, and accepts an operation for selecting an image evaluation target range via the keyboard 50a and the mouse 50b. For example, the directory structure of the HDD 14 or the digital still camera 30 may be displayed, and a plurality of image data IDs to be subjected to image evaluation may be selected. Based on this, an image data ID to be subjected to image evaluation may be searched.

  In step S110, the evaluation image selection unit P2a acquires the plurality of image data IDs selected in step S100 and outputs them to the face detection unit P2b. In step S120, the face detection unit P2b sequentially performs face detection on the plurality of image data IDs output from the evaluation image selection unit P2a, and stores the face detection results in the face detection database 14e. Note that the face detection in step S120 can be performed by, for example, a known pattern matching method. A number of face templates 14b are compared with images in the comparison area formed in the image data ID, and it is determined that there is a face in the comparison area where the similarity with the face template 14b is high. By shifting the position and size of the comparison area, faces included in each part of the image data ID can be sequentially detected.

  FIG. 4 shows the result of face detection and the data stored in the face detection database 14e. In the figure, the face position (center of gravity coordinates of the face area FA) and size (width of the face area FA) detected for each of the plurality of image data IDs are stored. For the image data ID from which a plurality of faces are detected, the face position and size are stored for each face. When the face detection database 14e can be generated as described above, the face organ detection unit P2c performs eye detection on each face detected in step S130. Again, the eyes can be detected by performing a known pattern matching method using the eye templates 14c for the left and right eyes. It should be noted that the area where the eyes exist can be narrowed down according to the face detection result stored in the face detection database 14e.

  FIG. 5 shows an example of the eye detection result. In the figure, the faces of a plurality of persons are detected, and the left and right eyes are detected in each face. In the present embodiment, the face template 14b and the eye template 14c have a rectangular shape, and an eye area EA including the left and right eyes and the periphery thereof is detected inside the face area FA detected in a rectangular shape. When the image data ID includes a plurality of faces, a plurality of face areas FA, FA,... Are detected, and left and right eye areas EA, EA,. Will be. Depending on the face angle and photographing quality, only one eye area EA, EA... May be detected in each face area FA, FA.・ In some cases, it may not be detected.

  When the eye areas EA, EA,... Are detected as described above, the face evaluation value calculation unit P2d calculates the sharpness S for the area inside each eye area EA, EA,. The sharpness S may be an index that can be used to determine whether or not an image is clear. In the present embodiment, the accumulation of luminance differences between adjacent pixels is used as the sharpness S. The sharpness S is calculated by accumulating the difference between the luminance of the pixel of interest and the luminance of the pixel adjacent to the pixel of interest for the entire eye area EA, EA,. If the definition S is large, it can be determined that the image quality related to the eyes is good, and if the definition S is small, it can be determined that the image quality related to the eyes is not good.

  In step S150, when the left and right eye areas EA and EA exist in the same face area FA, the sharpness S is calculated for each, and the average value is defined as the sharpness S for the face area FA. . On the other hand, when only one eye area EA exists in the same face area FA, the definition S for each eye area EA is used as the definition S for the face area FA as it is. Further, when any eye area EA does not exist (cannot be detected) in the same face area FA, the definition S for the face area FA is set to zero. The definition S corresponds to the face evaluation value of the present invention. When the definition S for each face area FA can be calculated, the definition S is added to the face detection database 14e.

  FIG. 6 shows the face detection database 14e in which the sharpness S is additionally written. In the figure, the definition S is stored for each face area FA detected in each image data ID. In step S160, the evaluation target limiting unit P2e detects the face area FA with the highest definition S among the face areas FA included in each image data ID. In step S170, the image evaluation value calculation unit P2f sets the highest definition S as the image evaluation value E of each image data ID. That is, the sharpness S of the face included in each image data ID with good image quality is set as the image evaluation value E of each image data ID. In other words, in steps S160 to S170, the sharpness S to be considered in calculating the image evaluation value E is limited to that having good image quality. In this way, it is possible to evaluate the entire image data ID based on the definition S of the face of the person most important when photographing the image data ID. On the other hand, since it is possible to estimate that the face of a person who is not regarded as most important at the time of photographing the image data ID is inferior in image quality (the sharpness S is small), the image evaluation value E is used for those having a small sharpness S. Not to consider.

  FIG. 7 shows a face detection database 14e in which an image evaluation value E is additionally recorded for each image data ID. As shown in the figure, an image evaluation value E is stored for each image data ID. Note that 0 is stored as the image evaluation value E for the image data ID for which no face was detected or the image data ID for which no face was detected even if a face was detected. In step S180, the evaluation result output unit P2g refers to the face detection database 14e and outputs a UI screen for notifying the user of the image evaluation result to the display 40.

  FIG. 8 shows an example of the UI screen. In the figure, thumbnails of a plurality of image data IDs selected in step S100 are displayed as a list, and an image evaluation value E is displayed below each thumbnail. In this way, the user can easily grasp the outline of the composition of each image data ID and the image quality of each image data. Further, in each thumbnail, a rectangular frame F indicating the outer shape of the face area FA including the eye area EA having the highest definition S adopted as the image evaluation value E is displayed. In other words, the face of the person used to calculate the image evaluation value E is highlighted than the face not used to calculate the image evaluation value E. Therefore, it is possible to easily grasp which person's face the displayed image evaluation value E is based on. Therefore, it can be easily determined whether or not the person photographed with the highest image quality is the intended person. In general, since a face photographed small is inferior in resolution, the sharpness S of a face photographed small is evaluated to be small (0001.jpg). However, even when the face is photographed large, the sharpness S may be smaller than the face photographed small (0003.jpg) when the exposure and focus are not correct.

  FIG. 9 shows another example of the UI screen. In the figure, the thumbnails are rearranged in descending order of the image evaluation value E, and a rectangular frame F is displayed in each thumbnail. In addition, the recommended print size for each thumbnail is shown. If the image evaluation value E is large, a person to be emphasized can be clearly printed even if the image is enlarged. Therefore, the image evaluation value E is determined as a threshold value based on a threshold value corresponding to each print size. When the image evaluation value E exceeds a threshold value corresponding to each print size, a message indicating that printing with a corresponding print size is recommended is displayed. In general, the larger the print size, the higher the sharpness S required. Therefore, the larger the print size, the narrower the image data ID recommended to be printed is on the larger image evaluation value E side. The UI screen is provided with a print execution button. When the thumbnail is selected and the print execution button is clicked, the print output unit P2h displays the image data ID corresponding to the selected thumbnail. The data is output to the driver P3, and printing is executed by the printer 20.

3-1. Modification 1
In the above, by using the image with the highest definition S as the image evaluation value E, the evaluation is performed based on the image quality of the face of the person most important at the time of shooting. Is not necessarily single. That is, in the case of a group photo or the like, the entire group composed of a plurality of persons is emphasized. In such a case, it is desirable to calculate the image evaluation value E based on the definition S for the entire group without limiting the evaluation target to the highest definition S.

  FIG. 10 schematically shows how the image evaluation value E is calculated in this modification. In the figure, the sharpness S for a plurality of human faces detected in a single image data ID is plotted in the vertical axis direction. In this modification, the attention group most important at the time of shooting is estimated based on the definition S of the faces of a plurality of persons, and the average value of the definition S of the faces belonging to the attention group is calculated as an image evaluation value. Used as E. Here, first, an attention group including a face having the highest definition S is specified. In the present embodiment, it is determined that a face having a sharpness S close to the highest definition S belongs to the attention group, and a face having a definition S that is not smaller than the predetermined threshold T1 than the highest definition S. It belongs to the attention group. Note that the threshold value T1 may be determined by a percentage of the highest definition S or may be determined by an absolute value. Further, the threshold value T1 may be determined based on a statistical index such as a variance or standard deviation of the image evaluation value E.

  When the attention group can be identified as described above, the average value of the sharpness S of the faces belonging to the attention group is calculated, and the average value is set as the image evaluation value E of the entire image data ID. That is, the evaluation target limiting unit P2e limits the definition S to be considered when calculating the image evaluation value E to the faces belonging to the target group. In this way, the image evaluation value E can be calculated based on the sharpness S of the entire group of interest including the face having the highest sharpness S. Therefore, the image quality of the entire group emphasized at the time of shooting is determined for each image. Data IDs can be compared.

  FIG. 11 shows an example of a UI screen displayed in step S180 of this modification. As shown in the figure, thumbnails are displayed, and rectangular frames F, F,... Indicating the outer shapes of a plurality of face areas FA, FA,. By displaying such rectangular frames F, F..., It is possible to easily determine whether or not a good image quality is obtained for the intended group.

3-2. Modification 2
FIG. 12 shows the flow of printing processing according to this modification. In step S200, the printer driver P3 causes the printer 20 to print the image data ID. In step S210, a predetermined UI screen is displayed on the display 40, and it is accepted whether the user is satisfied with the print result. Here, if it is accepted that the user is not satisfied, the processing is terminated as it is. On the other hand, when satisfaction is accepted, the print history database 14f is updated in step S220. Here, the identification information (file name and the like) of the image data ID that has been printed in step S200 and the print size are added to the print history database 14f.

  FIG. 13 shows an example of a UI screen displayed in step S180 of this modification. In this modification, the evaluation result output unit P2g refers to the print history database 14f together with the face detection database 14e to display the UI screen. In the UI screen, the thumbnails are listed in descending order of the image evaluation value E. For the image data ID stored in the print history database 14f that the print result is satisfied, the outer frame of the corresponding thumbnail is highlighted (emphasis by color, emphasis by the thickness of the frame) and the print result. The print size when satisfied is displayed. In this way, the user can know what print size is actually satisfied when printing. For image data IDs having an image evaluation value E greater than the emphasized image data ID, it can be predicted that a satisfactory print result will be obtained if printing is performed with the same print size, and it is determined whether or not to execute printing. It can be made easier.

It is a block diagram which shows the hardware constitutions of an image evaluation apparatus. It is a block diagram which shows the software structure of an image evaluation apparatus. It is a flowchart which shows the flow of an image evaluation process. It is a figure which shows image data and a face detection database. It is a figure which shows an example of the detection result of eyes. It is a figure which shows a face detection database. It is a figure which shows a face detection database. It is a figure which shows an example of UI screen. It is a figure which shows another example of UI screen. It is a figure which shows a mode that the image evaluation value concerning a modification is calculated. It is a figure which shows UI screen concerning a modification. It is a flowchart of the printing process concerning a modification. It is a figure which shows UI screen concerning a modification.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 10 ... Computer, 11 ... CPU, 12 ... RAM, 13 ... ROM, 14 ... HDD, 14a ... Program data, 14b ... Face template, 14c ... Eye template, 14e ... Face detection database, 14f ... Print history database, 15 ... GIF 16 ... VIF, 17 ... IIF, 18 ... bus, 20 ... printer, 40 ... display, 50a ... keyboard, 50b ... mouse, P1 ... OS, P2 ... image evaluation application, P2a ... evaluation image selection unit, P2b ... face detection , P2c ... Face organ detection unit, P2d ... Face evaluation value calculation unit, P2e ... Evaluation target limiting unit, P2f ... Image evaluation value calculation unit, P2g ... Evaluation result output unit, P2h ... Print output unit, P3 ... Printer driver, ID: Image data.

Claims (7)

An image output method for evaluating image data,
Detecting a face included in the image indicated by the image data;
When multiple faces are detected , the eyes included in each face are detected, when both eyes are detected in the face, the average value of the sharpness of both eyes is calculated, and when one eye is detected in the face , Calculate the sharpness of one eye, calculate a face evaluation value for each face based on the calculated sharpness ,
An image output method for outputting an image evaluation value indicating an evaluation result of the image data based on the face evaluation value of which the face evaluation value is a predetermined value or more among the face evaluation values of each face.   The image output method according to claim 1, wherein an image evaluation value indicating an evaluation result of the image data is output based on the maximum face evaluation value among the face evaluation values of each face.   An image evaluation value indicating an evaluation result of the image data is output based on a plurality of the face evaluation values belonging to the same group as the largest face evaluation value among the face evaluation values of each face. The image output method according to claim 1.   An image indicated by the image data is displayed, and an area indicating each face corresponding to the face evaluation value limited in calculating the image evaluation value in the display is highlighted than the other areas. The image output method according to any one of claims 1 to 3. With displaying a list of images represented by a plurality of said image data, that points to the image picture data indicates that is a print in the past target in the display the preceding claims, characterized in any one of claims 4 The image output method described. An image output device for evaluating image data,
Face detection means for detecting a face included in an image indicated by the image data;
When multiple faces are detected , the eyes included in each face are detected, when both eyes are detected in the face, the average value of the sharpness of both eyes is calculated, and when one eye is detected in the face A face evaluation unit that calculates the sharpness of one eye and calculates a face evaluation value of each face based on the calculated sharpness ;
Image evaluation means for outputting an image evaluation value indicating an evaluation result of the image data based on the face evaluation value of which the face evaluation value is a predetermined value or more among the face evaluation values of each face. An image output device. A computer-readable image output program for causing a computer to execute a function of evaluating image data,
A face detection function for detecting a face included in an image indicated by the image data;
When multiple faces are detected , the eyes included in each face are detected, when both eyes are detected in the face, the average value of the sharpness of both eyes is calculated, and when one eye is detected in the face A face evaluation function that calculates the sharpness of one eye and calculates a face evaluation value of each face based on the calculated sharpness ;
Causing a computer to execute an image evaluation function for outputting an image evaluation value indicating an evaluation result of the image data based on the face evaluation value of which the face evaluation value is a predetermined value or more among the face evaluation values of the faces. An image output program readable by a computer.

Images