JP2010224677A - Image evaluation method, image evaluation program and printing device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a method for comprehensively evaluating the quality of an image specially for an image obtained by photographing a small number of persons. <P>SOLUTION: The image evaluation method includes a face detection step of detecting a face image from an image represented by input image data; a face number determination step of determining whether the number of detected face images is a specific number or less; and an evaluation execution step of performing, for each face image, evaluation of a position within the image data, photogenic evaluation of face (S170), evaluation of face direction (S180), and evaluation of a predetermined defect condition (S190), calculating a comprehensive evaluation result based on each evaluation result (S200), and outputting the comprehensive evaluation result to the outside. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to an image evaluation method, an image evaluation program, and a printing apparatus for evaluating an image.

  A user freely performs photographing using an imaging device such as a digital still camera (DSC), and stores a large number of photographing results (image data) in a predetermined memory.

  Here, when a person is included in the captured image, the image is evaluated by comparing the position information of the person's face with the portrait composition or the commemorative photographing composition, and when the captured image does not include the person. An imaging device is known that detects line segment elements and subjects in an image, evaluates an image by comparing the line segment elements and the subject with three-division composition data, and outputs the evaluation result to a screen (Patent Literature). 1).

Japanese Patent Laid-Open No. 2001-167253

  For the majority of DSC users, for example, when selecting an image to be printed at a DPE shop in the city or selecting an image to be printed with a home printer, each of the images taken in and stored in memory is good. It is difficult to objectively judge what is wrong. In addition, since the employees of the DPE shop are mostly not experts in photo evaluation, it is difficult to evaluate (suggest) good or bad image data brought in by the user. Here, there are very many cases where a subject to be photographed by a user is a person such as a family member, a friend or a lover. In addition, when photographing a person in this way, the number of people included in one photograph is almost a small number (for example, about 1 to 3 people).

  In other words, it is desired to establish a technique for evaluating the quality of an image by specializing in an image obtained by photographing a small number of people. However, since the above document only evaluates the position of the face in the case of a human image, the human image cannot be evaluated comprehensively from many viewpoints. Since even landscape images that are not included are evaluated individually, they do not satisfy the needs of users who want a comprehensive evaluation of a human image of a small number of people.

  The present invention has been made in view of the above problems, and an object of the present invention is to provide an image evaluation method, an image evaluation program, and a printing apparatus capable of accurately and comprehensively evaluating an image obtained by photographing a small number of persons. And

  In order to achieve the above object, an image evaluation method according to the present invention includes a face detection step of detecting a face image from an image represented by input image data, and whether the number of detected face images is a specific number or less. A face number determination step for determining whether or not the number of the face images is equal to or less than a specific number, the position evaluation in the image data, the evaluation of the appearance of the face image, and the face An evaluation execution step of performing an evaluation of a direction and an evaluation of a predetermined defect condition, calculating a comprehensive evaluation result based on each evaluation result, and outputting the comprehensive evaluation result to the outside. According to the present invention, it is possible to limit the image data including a specific number of face images or less to perform comprehensive evaluation on the face image included in the image data and present the comprehensive evaluation result to the user. . The specific number mentioned here is a number that is small to some extent, for example, about 1 to 3.

  In the evaluation execution step, the composition of the image represented by the image data is determined when the number of the face images is equal to or less than a specific number and the ratio of the area of the face image in the image represented by the image data is equal to or less than a predetermined value. While evaluating, the said comprehensive evaluation result may be calculated based on the evaluation result of the said composition and said each evaluation result other than the evaluation result of the said composition. According to this configuration, when the area of an image (background image) other than the face image is large to some extent in the image, the background composition is also evaluated, and the evaluation result for the background composition and each evaluation result for the face image Based on this, a comprehensive evaluation result can be obtained. In the evaluation execution step, as an evaluation of the composition, whether or not the composition corresponds to the line perspective method, whether or not the composition corresponds to the air perspective method, and whether the composition includes a horizontal line. At least one of the negative evaluations may be evaluated.

Further, in the evaluation execution step, as the evaluation of the facial reflection, an evaluation may be performed on at least one element among the brightness of the face, the expression, and the sharpness of the face.
Further, in the evaluation execution step, as the evaluation of the face orientation, it may be evaluated whether or not a space of a predetermined size exists in the direction in which the face faces in the image represented by the image data.
In the evaluation execution step, evaluation of at least one element of exposure and blur may be performed as the evaluation of the defect condition.

  The technical idea of the present invention can be realized by methods other than the image evaluation method. For example, the apparatus (printing device, computer, imaging device, or the like) executes an invention of a device (printing device, computer, imaging device, or the like) that executes the image evaluation method, or processing executed in the image evaluation method The invention of the image evaluation program can also be grasped.

It is the block diagram which showed the structure of the computer roughly. It is the flowchart which showed the image evaluation process. It is the figure which showed an example of the face image detection result. It is the flowchart which showed the detail of the background composition evaluation process. It is the flowchart which showed the detail of the face position evaluation process. It is the figure which showed a mode that the face area overlapped on the lattice point of the dividing line. It is the figure which showed a mode that the face area | region overlapped on each lattice point in diagonal relationship. It is the figure which showed a suitable example of parent-child arrangement | positioning. It is the figure which showed a suitable example of friend arrangement | positioning. It is the flowchart which showed the detail of the facial appearance evaluation process. It is the flowchart which showed the detail of the face direction evaluation process. It is the figure which showed a mode that face direction was determined. It is the figure which showed the case where there is a predetermined space in the point where the face turned, and the case where there is no. It is the flowchart which showed the detail of the defect condition evaluation process.

1. FIG. 1 schematically shows a configuration of a computer 10 as an example of an image evaluation apparatus according to the present embodiment. The image evaluation device is a specific device for executing the image evaluation method. The computer 10 includes a CPU 11, an internal memory 12 including a ROM and a RAM, a hard disk drive (HDD) 13, a general-purpose interface (GIF) 14, a video interface (VIF) 15, and an input interface (IIF) 16. . Each of these elements 11 to 16 realizes communication by a bus 17. The HDD 13 stores program data P for executing various programs (such as an image evaluation program and various drivers) including an operating system (OS), and the CPU 11 develops the program data P while expanding the program data P into the RAM. Arithmetic processing according to P is performed.

  The HDD 13 stores image data D input from an image input device such as the DSC 20. In the present embodiment, the image data D is image data representing a photographic image. The GIF 14 provides an interface conforming to the USB standard, for example, and connects the DSC 20 and the printer 30 to the computer 10. The GIF 14 also has a function of a card interface connected to a card slot (not shown), and can input image data D stored in the memory card 40 from the memory card 40 inserted in the card slot. The VIF 15 connects the computer 10 to an external display 50 and provides an interface for displaying an image on the display 50. The IIF 16 connects the computer 10 to an external input operation unit (keyboard or mouse) 60 and provides an interface for the computer 10 to acquire an input signal from the input operation unit 60.

  In FIG. 1, an image evaluation processing unit 121, a display processing unit 122, and a print processing unit 123 are shown as functional blocks realized by the program data P in the internal memory 12. The image evaluation processing unit 121 includes a face image detection unit 121a, a face number determination unit 121b, a face position evaluation unit 121c, a facial appearance evaluation unit 121d, a face orientation evaluation unit 121e, and a defect condition evaluation unit as program modules. 121f, a background composition evaluation unit 121g, and a comprehensive evaluation unit 121h. The functions of these units will be described later.

  The display processing unit 122 is a display driver that controls the display 50 to display a predetermined user interface (UI), an evaluation result, and the like on the screen of the display 50. The print processing unit 123 generates print data by performing predetermined image processing on the image data D selected as a print target, and controls the printer 30 to execute image printing based on the print data. This is a printer driver.

2. Image Evaluation Process FIG. 2 shows an image evaluation process executed by the computer 10 in a flowchart. This process is mainly realized by the function of the image evaluation processing unit 121.
In step S (hereinafter, step notation is omitted) 100, the image evaluation processing unit 121 acquires the image data D and evaluates the image data D related to one image from the acquired image data D. Set as target image data D. That is, the image evaluation processing unit 121 reads (acquires) image data D from a predetermined storage medium such as the HDD 13, the memory in the DSC 20, or the memory card 40 when the user instructs the start of the image evaluation processing via the UI. To do). The user designates from which storage medium the image data D is acquired. Further, the acquired image is the total number of images stored in the storage medium at that time, the folder unit in the storage medium, the predetermined number unit of the predetermined number, or the image instructed by the user. There is. If the number of acquired images is one, the acquired image is set as an evaluation target.

  In S110, the face image detection unit 121a detects a face image from the image (target image) represented by the image data D set in S100. The detection of a face image is a process of detecting a rectangular face region that is assumed to include at least each rectangle of a predetermined facial organ (eyes, nose, or mouth) in the target image. The detection of the face area is performed using a known detection method such as a pattern matching method using each template of eyes and mouth. The face image detection unit 121a can employ any technique other than pattern matching as long as it can detect a face area. For example, in a rectangular area (detection target area) set in the target image, various information of the image (for example, luminance information, edge amount, contrast, etc.) is input, and whether or not the detection target area corresponds to a face area is determined. It is possible to detect a face area by using a pre-learned neural network that outputs information indicating whether or not the face area is detected for each detection target area using a support vector machine. May be.

  FIG. 3 is a diagram illustrating an example of the detection result in S110. In the example of FIG. 3, since the person P1 and the person P2 are included in the target image (image data D), the face area Fd1 corresponding to the face image of the person P1 and the face area Fd2 corresponding to the face image of the person P2 And have been detected. The face areas Fd1 and Fd2 are rectangular areas each including all images of both eyes, nose and mouth. The sizes of the detected face areas Fd1 and Fd2 are different for the faces of the persons P1 and P2. The face image detection unit 121a identifies the face areas Fd1 and Fd2 by the coordinates of the four vertices of the face area.

  In S120, the face number determination unit 121b determines whether one or more face areas have been detected by the process in S110. If no face area has been detected, the flowchart ends. If a plurality of pieces of image data D have been acquired from the predetermined storage medium, image data D to be evaluated is newly set from the acquired image data D (S100). repeat.

  On the other hand, when one or more face areas are detected from the target image, the face number determination unit 121b determines whether or not the number of face areas detected from the target image is equal to or less than a specific number in S130. To do. The specific number is a numerical value set in advance in the program and is a relatively small integer of “1” or more. The specific number is “3”, for example. If the number of face areas is less than or equal to the specific number, the process proceeds to S140, and if the number of face areas exceeds the specific number, as in the case of “No” determination in S120, The flowchart is terminated. Such processing of S110 to S130 corresponds to a face detection step and a face number determination step.

After S140, evaluation over a plurality of items is performed with respect to the target image, particularly the face area. The processing after S140 corresponds to an evaluation execution step. In other words, in the present embodiment, image evaluation is performed only for images in which a small number of people such as about 1 to 3 are shown.
In S140, the image evaluation processing unit 121 determines whether or not the ratio (area ratio) of the face area in the target image is equal to or less than a predetermined threshold value. For example, the process proceeds to S150 (background composition evaluation process). If the ratio of the face area exceeds the threshold value, S150 is skipped and the process proceeds to S160 (face position evaluation process). When there are a plurality of face areas, the ratio of the total area of each face area to the size of the target image is calculated and determined. The size of the face area can be calculated based on the coordinate position of the vertex of the face area in the target image.

  In S150, the background composition evaluation unit 121g evaluates the composition of the target image. That is, when various image evaluations are performed only on an image showing a small number of people, if the size of the face in the image is small to some extent, the background composition is also evaluated. The background composition evaluation unit 121g evaluates whether or not the target image is a composition including a horizontal line, evaluates whether or not it is a composition corresponding to the line perspective method, and determines whether or not it is a composition corresponding to the air perspective method. Are evaluated.

  FIG. 4 is a flowchart showing details of the background composition evaluation process in S150. In S151, the background composition evaluation unit 121g detects an edge in the target image. At this time, the region to be subjected to edge detection may be the entire target image or a region obtained by removing the face region from the target image. For example, the edge detection is performed by applying a filtering process using a second-order differential filter (Laplacian filter) to each pixel of the target image to calculate an edge amount, and comparing the edge amount with a predetermined threshold value. Whether or not each pixel is an edge is detected. For edge detection, a primary differential filter (Prewitt filter, Sobel filter, Roberts filter, Kirsh filter, etc.) may be used instead of the secondary differential filter.

  In S152, the background composition evaluation unit 121g determines whether or not the target image is a composition including a horizontal line. The background composition evaluation unit 121g extracts a line segment in the target image based on each edge detected in S151, and determines whether or not any of the extracted line segments corresponds to a horizontal line. For example, the background composition evaluation unit 121g includes a horizontal line if there is a line segment having a length equal to or greater than a certain reference in the target image and substantially perpendicular to the vertical direction of the target image. Judge as composition. In the present embodiment, when the target image is a landscape (landscape) image, the short side direction of the image is the vertical direction, and when the target image is a portrait (portrait) image, the long side direction of the image is The vertical direction. The term “substantially perpendicular” refers to an angle within a predetermined angle range centered on 90 °.

  In S153, the background composition evaluation unit 121g determines whether the target image is a composition corresponding to the line perspective method, a composition corresponding to the air perspective method, or a composition of any perspective method. . In this case, the background composition evaluation unit 121g detects a vanishing point of the line segment in the target image. For example, the background composition evaluation unit 121g analyzes whether or not the end of a predetermined number or more of the line segments in the target image has converged to a certain point (or one small region) in the image. It is assumed that the vanishing point can be detected when there is such convergence, and as a result, it is determined that the background in the target image has a composition corresponding to the linear perspective method. In addition, the background composition evaluation unit 121g adds up the edge amounts in the entire area targeted for edge detection. When the total amount of edges is equal to or less than a predetermined threshold value (first threshold value for edge amount comparison), the target image is an image close to a blurred image, that is, an image having a composition corresponding to the air perspective method. Judge that there is.

  In S154, the background composition evaluation unit 121g gives a first evaluation score for the target image based on the determination results in S152 and S153. First, when it is determined in S152 that the composition includes a horizontal line, a predetermined high score is recognized as the first evaluation score for the target image. The score mentioned here is a relative score within a full score (for example, a maximum of 10 points) for each evaluation process (background composition evaluation process, face position evaluation process, facial image evaluation process, face orientation evaluation process, defect condition evaluation process). This means a score, and a specific score is determined in advance for each determination result in each evaluation process. Even when it is determined in S153 that the composition corresponds to the line perspective method or the composition corresponding to the air perspective method, the background composition evaluation unit 121g recognizes a predetermined high score as the first evaluation score. Furthermore, for a composition that includes a horizontal line and falls within the line perspective method, a higher score is recognized as the first evaluation point.

  On the other hand, a predetermined low score (or no score) is recognized as a first evaluation score for a target image that does not include a horizontal line and does not correspond to any of the above perspective methods. The background composition evaluation unit 121g records the first evaluation point thus identified in a predetermined area of the internal memory 12. The background composition evaluation unit 121g determines whether the target image necessarily includes a horizontal line, whether the composition corresponds to the line perspective method, and whether the composition corresponds to the air perspective method. It is not necessary to make this determination, and at least a part of the determination may be made.

Following S140 or following S150, the face position evaluation unit 121c evaluates the position of the face image in the target image in S160.
FIG. 5 is a flowchart showing details of the face position evaluation process in S160. In S161, the face position evaluation unit 121c branches the process depending on whether the number of face areas detected from the target image is one or more. When the number of face areas is 1 (that is, when the person is a target image in which one person is photographed), the process proceeds to S162, and the photographing range of the person is determined. The determination of the shooting range of a person means that the shooting range of the person in the target image is “whole body”, “bust up”, “whole face only (including neck)”, “part of face” It is an operation for determining which one corresponds. In other words, since the balance of the size of the human face and the body other than the face is roughly determined, the face position evaluation unit 121c determines that the shooting range of the person is “in accordance with the position and size of the face area in the target image. It is determined (guessed) whether it corresponds to “whole body”, “bust up”, “whole face only (including neck)”, or “part of face”.

  In S163, the face position evaluation unit 121c determines whether or not the face area of the person is placed at an appropriate position according to the shooting range of the person in the target image. For example, when the shooting range of a person is one of “whole body”, “bust up”, or “whole face only (including neck)”, whether or not the face area is located in the central area of the target image To determine. The central region is a circular region having a predetermined size including the center of the target image. In addition, when the shooting range of a person is one of “entire body”, “bust up”, or “whole face only (including neck)”, the grid points of the dividing lines when the target image is divided into three parts vertically and horizontally It is determined whether or not the face areas overlap.

  FIG. 6 shows a state in which the face region Fd overlaps one grid point among the grid points of the dividing line. In FIG. 6, for the sake of convenience, dividing lines dL (chain lines) and lattice points dP (black circles) are shown. However, such dividing lines dL and lattice points dP do not actually appear in the target image (image data D). No (same for FIG. 7). When the face region Fd overlaps such a grid point, the face position evaluation unit 121c determines whether or not the eye (right eye or left eye) region in the face region Fd further overlaps the grid point. To do.

  In S166 after S163, the face position evaluation unit 121c gives a second evaluation score for the face image of the target image based on the determination result in S163. First, when it is determined that the face area is located in the central area of the target image, or when it is determined that the face area overlaps the lattice points, a predetermined high score is recognized as the second evaluation score. If it is determined that the eye (right eye or left eye) region in the face region overlaps the grid point, a higher score is recognized as the second evaluation score. On the other hand, a predetermined low score (or no score) is given to the face area determined not to exist in the central area or on the grid point in S163, or to the face area not determined in S163. Is certified as the second evaluation point.

  On the other hand, when the number of face areas is plural (2 or 3), the face position evaluation unit 121c proceeds from S161 to S164, and specifies a person arrangement pattern. The specification mentioned here means specification of the number of persons and specification of the arrangement relationship of the persons in the specified number of persons. More specifically, it is specified whether it is “2 people” or “3 people”, and if it is “3 people”, that “3 people” is either “parent-child arrangement” or “friend arrangement”. Identify whether it applies. “Parent-child arrangement” refers to an arrangement in which only one of three persons has a face area position (height position) that is lower by a predetermined amount in the vertical direction than the other two persons. If it is “three people” and not “parent-child arrangement”, the face position evaluation unit 121c identifies “friend arrangement”.

  In S165, the face position evaluation unit 121c determines whether or not each person's face area is arranged at an appropriate position according to the arrangement pattern specified in S164. For example, when the arrangement pattern is “two people”, the grid points of the dividing line when the vertical and horizontal directions of the target image are each divided into three, and two grid points in a diagonal relation on the central grid block It is determined whether or not each face area overlaps.

  FIG. 7 shows a state in which the face areas Fd and Fd overlap each other on the two lattice points having the above diagonal relationship. Further, the face position evaluation unit 121c determines that the center of gravity of the two face areas is equal to each other when the arrangement pattern is “two people” and the face areas do not overlap on the two lattice points having the diagonal relationship. It is determined whether or not the center of the line segment connecting is overlapped on any grid point (is near any grid point). Further, when the arrangement pattern is “three people” and “parent-child arrangement”, the face position evaluation unit 121c determines the face area located in the middle in the positional relationship of the face areas in the direction perpendicular to the vertical direction (left and right direction). It is determined whether the height position is the lowest of the three face regions.

  FIG. 8 illustrates a case where the height position of the face region Fd located in the middle is the lowest when the arrangement pattern is “three people” and “parent-child arrangement”. That is, the arrangement as shown in FIG. 8 is presumed to be an arrangement in which there are children between parents, and such an arrangement can be said to be preferable. In addition, when the arrangement pattern is “three people” and “friend arrangement”, the face position evaluation unit 121c determines whether the height positions of the three face areas are substantially the same. When the arrangement pattern is “three people” and “friend arrangement”, the sizes of the three face areas are large, medium, and small, and the face areas are arranged in one direction in the order of large, medium, and small. It is determined whether or not (perspective arrangement).

  FIG. 9 shows an example in which each face area Fd, Fd, Fd is a perspective arrangement when the arrangement pattern is “three people” and “friend arrangement”. In S166 after S165, the face position evaluation unit 121c gives a second evaluation score for the face image of the target image based on the determination result in S165. That is, when it is determined that the arrangement pattern is “two people” and the face areas overlap each other on the two lattice points having the above diagonal relationship, the arrangement pattern is “two people” and two If it is determined that the center of the line segment connecting the centroids of the face area overlaps on any grid point (near any grid point), the placement pattern is “3 people” and “parent-child placement” If it is determined that the height position of the face area located in the middle is the lowest, it is determined that the arrangement pattern is “three people” and “friend arrangement”, and the height positions of the three face areas are substantially the same. If the arrangement pattern is “3 people” and “friend arrangement”, and each face area is determined to be a perspective arrangement, if any one of them is determined, a predetermined value corresponding to each determination result is obtained. The high score is recognized as the second evaluation score, otherwise It is certified predetermined low score (or no score) as the second evaluation points. The face position evaluation unit 121 c records the second evaluation point thus identified in a predetermined area of the internal memory 12.

  In S170, the facial appearance evaluation unit 121d evaluates the facial appearance of the facial image in the target image. The facial image evaluation unit 121d performs facial brightness evaluation, facial expression evaluation, and facial sharpness evaluation as facial evaluation.

  FIG. 10 is a flowchart showing details of the facial image evaluation process in S170. In S171, the facial appearance evaluation unit 121d evaluates the contrast (contrast) of the face region in the target image. Specifically, the facial appearance evaluation unit 121d generates a histogram (for example, a luminance distribution in the luminance range 0 to 255) of the luminance of each pixel in the region for one facial region, and the histogram width (maximum luminance and minimum luminance). Difference). If the width is equal to or greater than a predetermined threshold value, the face area subjected to the brightness evaluation is determined to have high contrast. Such generation of the histogram and determination of the width are performed for each face area when there are a plurality of face areas.

  In S172, the facial appearance evaluation unit 121d evaluates the facial expression of the face area in the target image. Specifically, the facial appearance evaluation unit 121d determines whether the facial expression in one face area is a smile. The facial appearance evaluation unit 121d has a known facial expression determination function, and can determine whether the face is a smile based on an image (for example, mouth shape) in the face area. Such facial expression determination is performed for each face area when there are a plurality of face areas. The determination of whether or not the face is a smile may be performed by the face image detection unit 121a together with the detection of the face area in S110. If the face image detection unit 121a has already determined whether or not the face area is a smile, in S172, it is evaluated whether or not the face area is determined to be a smile.

  In S173, the facial appearance evaluation unit 121d evaluates the sharpness (sharpness) of the face area in the target image. The sharpness index varies, but here the facial appearance evaluation unit 121d calculates the edge amount for one face area. As described above, the edge amount calculation is performed by applying a predetermined filter to each pixel in the target region. The face appearance evaluation unit 121d sums the edge amounts in the face area that is the target of the edge amount calculation. When the total amount of edges is equal to or greater than a predetermined threshold value (second threshold value for edge amount comparison), it is determined that the face in the face area subject to edge amount calculation is a sharp face. . Such sharpness determination is performed for each face area when there are a plurality of face areas.

  In S174, the facial appearance evaluation unit 121d gives a third evaluation score for the face area of the target image based on the determination results in S171 to S173. Specifically, the facial appearance evaluation unit 121d makes positive determinations (a determination result of high contrast, a determination result of smile, and a determination result of sharp face) in the three types of determinations of S171 to S173 for the face area. If so, the predetermined highest score for the face area is recognized as the third evaluation score. Moreover, a low score is recognized as the third evaluation score as the number of positive determination results in the three types of determinations decreases. When there are a plurality of face areas in the target image, the facial appearance evaluation unit 121d first assigns a third evaluation point for each face area, and calculates an average score of the third evaluation points of each face area in the target image. One third evaluation point for a plurality of face regions is recognized.

  The facial appearance evaluation unit 121 d records the third evaluation point thus identified in a predetermined area of the internal memory 12. Note that the facial appearance evaluation unit 121d does not necessarily have to determine all of the brightness of the face, the expression, and the sharpness of the face for the face area in the target image, and may make at least a partial determination.

In S180, the face direction evaluation unit 121e evaluates whether or not a space of a predetermined size exists in the direction in which the face of the face area faces in the target image as the face direction evaluation. This is because it can be said that when a space of a predetermined size exists in the direction in which the face faces in the target image, the composition gives a good impression to the person who sees the photograph.
FIG. 11 is a flowchart showing details of the face orientation evaluation process in S180. In S181, the face orientation evaluation unit 121e determines the face orientation of the face area in the target image.

  12A and 12B show how the face direction evaluation unit 121e determines the face direction. The face direction evaluation unit 121e has a right eye rectangular area (right eye area Ar), a left eye rectangular area (left eye area Al), and a mouth rectangular area (mouth area Am) in the face area Fd detected during the face detection. The orientation of the face is determined by analyzing the positional relationship. The face direction evaluation unit 121e defines a straight line that is orthogonal to the line segment M connecting the center of gravity of the right eye region Ar and the center of gravity of the left eye region Al and passes through the center of gravity of the mouth region Am as the center axis L of the face. The face orientation evaluation unit 121e determines the face orientation based on the size of the length m1 of the right eye segment and the length m2 of the left eye segment when the segment M is divided by the central axis L. That is, if the line segment m1 on the right eye side = line segment m2 on the left eye side, it can be determined that the face is oriented in a direction perpendicular to the image plane (front-facing) (see FIG. 12A). Further, if the line segment m1 on the right eye side> the line segment m2 on the left eye side, the face can be determined to be “rightward” (see FIG. 12B), and the line segment m1 on the right eye side <the line segment m2 on the left eye side. If so, the face can be determined to be “leftward”.

  If the face is determined to be “rightward”, the person in the target image is facing the left side of the person, and if the face is determined to be “leftward”, the person is facing the right side of the person. Become. The face direction evaluation unit 121e may perform such rightward / leftward determination by comparing the width Wr of the right eye region Ar and the width Wl of the left eye region Al (see FIG. 12). In other words, if the face has a swing, the size of the detected right eye area Ar or left eye area Al changes depending on the degree of the swing, so if width Wr = width Wl, the face is determined to face front. If width Wr> width Wl, the face can be determined to face right, and if width Wr <width Wl, the face can be determined to face left.

  In S182, the face direction evaluation unit 121e determines whether there is a space of a predetermined size in the direction in which the face faces in the target image according to the determination result of the face direction. For example, when the face orientation evaluation unit 121e determines that the face orientation is “right”, a space having a predetermined size including an extension line of the line segment M is present in a region on the right side of the face region in the target image. It is determined whether or not it exists. The space of a predetermined size mentioned here means a block of a region that does not include a person image in the target image and has a size that is equal to or larger than a predetermined ratio with respect to the area of the target image.

  FIG. 13A shows a case where the face direction of the face area Fd in the target image is rightward (upward diagonally right), and a space E of a predetermined size including an extension line of the line segment M exists on the right side of the face area Fd. Is illustrated. In the example of FIG. 13A, since there is a predetermined space ahead of the face of the face area (particularly the line of sight), the face direction evaluation unit 121e determines that a space of a predetermined size exists in the direction in which the face faces in the target image. judge. On the other hand, when the target image is an image having no space E having a predetermined size including an extension line of the line segment M as shown in FIG. Then, it is determined that there is no space of a predetermined size in the direction in which the face faces in the target image. Note that the orientation of the face for determining whether or not there is a space of a predetermined size includes not only the difference between the right direction and the left direction but also the inclination of the face (the inclination of the central axis L with respect to the vertical direction of the target image). to decide.

  In S183, the face orientation evaluation unit 121e gives a fourth evaluation score for the face area of the target image based on the determination result in S182. That is, when the face direction evaluation unit 121e determines that there is a space of a predetermined size in the direction in which the face faces in the target image, the face direction evaluation unit 121e recognizes a predetermined high score for the face area as the fourth evaluation point. On the other hand, when it is evaluated that the face of the face area is facing forward, or when it is not determined that a space of a predetermined size exists in the direction in which the face faces in the target image, a predetermined score lower than the high score Is certified as the fourth evaluation point. When there are a plurality of face areas in the target image, the face direction evaluation unit 121e determines the face direction in units of face areas in S181. In S182, only when the face orientations of all the face areas are the same, it is determined whether there is a space of a predetermined size including an extension line of each line segment M of each face area. It is determined whether or not there is a space of a predetermined size in the direction in which the face faces in the target image.

  The face direction evaluation unit 121e records the fourth evaluation point thus identified in a predetermined area of the internal memory 12. The face orientation determination may be performed by the face image detection unit 121a together with the detection of the face area in S110. When the face image detection unit 121a has already determined the face orientation, the face orientation evaluation unit 121e performs the process of S182 using the face orientation determination result by the face image detection unit 121a.

In S190, the defect condition evaluation unit 121f evaluates the degree of exposure and the degree of blur for the face area in the target image as the defect condition evaluation.
FIG. 14 is a flowchart showing details of the failure condition evaluation process in S190. In S191, the defect condition evaluation unit 121f determines the degree of exposure in the face area. Although there are various methods for determining the degree of exposure, for example, the facial appearance evaluation unit 121d generates a histogram of luminance relating to one face area and calculates the number of samples (for generating the histogram from the maximum luminance in the histogram). The luminance on the low luminance side by a predetermined number percent (for example, 5%) of the number of used pixels) is set as the reference luminance. If the reference luminance is greater than a predetermined threshold value (first threshold value for exposure determination), it is determined that the face in the face area is overexposed. On the other hand, when the reference luminance is equal to or lower than the second threshold value for exposure determination (however, the first threshold value for exposure determination> second threshold value for exposure determination), the face in the face area is underexposed. Is determined. Such determination of the degree of exposure is performed for each face area when there are a plurality of face areas.

  In S192, the failure condition evaluation unit 121f determines whether or not the face area is blurred. The blur determination is basically the same as the sharpness determination described above. In other words, the defect condition evaluation unit 121f sums the edge amounts in the face area, and the sum of the edge amounts is a predetermined threshold value (a third threshold value for edge amount comparison. However, the second threshold value for edge amount comparison). Value> the third threshold value for edge amount comparison.) If it is smaller, it is determined that the face in the face area is blurred. Such blur determination is performed for each face area when there are a plurality of face areas.

  In S193, the failure condition evaluation unit 121f gives a fifth evaluation score for the face area of the target image based on the determination results in S191 and S192. Specifically, the failure condition evaluation unit 121f determines that both of the two types of determinations of S191 and S192 are defective conditions for one face area (determination that the image is overexposed or underexposed and is blurred. ), The predetermined lowest score for the face region is recognized as the fifth evaluation score. Further, a higher score is recognized as the fifth evaluation score as the number of determination results indicating that the condition is a failure condition in the two types of determinations decreases. When there are a plurality of face areas in the target image, the failure condition evaluation unit 121f first assigns a fifth evaluation point for each face area, and calculates the average score of the fifth evaluation points of each face area in the target image. It is recognized as a fifth evaluation point for a plurality of face regions.

  The defect condition evaluation unit 121f records the fifth evaluation point thus recognized in a predetermined area of the internal memory 12. The defect condition evaluation unit 121f does not necessarily have to determine both exposure and blur for the face area in the target image, and may perform at least a part of the determination.

  In S200, the comprehensive evaluation unit 121h determines the first to fifth evaluation points certified by the evaluation processes of S150 to S190 (if S150 has not been executed, the first evaluation is authorized by the evaluation processes of S160 to S190. A comprehensive evaluation score is calculated based on the second to fifth evaluation scores. The comprehensive evaluation unit 121h calculates, for example, the ratio (score rate) of the total value of each evaluation point to the total value of the full marks in each evaluation process as the comprehensive evaluation point. Alternatively, the comprehensive evaluation unit 121f may add the weights after giving a predetermined weight to each evaluation score, and may use this total (total score rate) as a comprehensive evaluation score.

  In S210, the image evaluation processing unit 121 outputs a comprehensive evaluation score to the outside. Specifically, the image evaluation processing unit 121 notifies the comprehensive evaluation score to the display processing unit 122 and causes the display processing unit 122 to display the comprehensive evaluation score on the screen of the display 50. In this case, the display processing unit 122 may display the overall evaluation score concerning the image together with the thumbnail image of the target image. When the image evaluation processing unit 121 repeatedly executes the flowchart of FIG. 2 for a plurality of target images, the image evaluation processing unit 121 associates the thumbnails of the plurality of target images with the overall evaluation points of the plurality of target images on the screen of the display 50. May be displayed. Further, the image evaluation processing unit 121 may display not only the comprehensive evaluation score but also the first to fifth evaluation points on the screen of the display 50.

3. Summary As described above, according to the image evaluation process of the present embodiment, at least a position evaluation of a face image, brightness, darkness, facial expression, and sharpness is targeted only for an image in which a small number of persons of about 1 to 3 are copied. Evaluation of the presence of space related to face orientation, evaluation of bad conditions such as overexposure / underexposure of face and blurring, etc., and calculating overall evaluation score based on each evaluation result And the overall evaluation score is presented to the user. That is, it is possible to present an objective and comprehensive evaluation of a small number of person photographs that are usually taken frequently by the majority of DSC users. In addition, if the area of the background area other than the face area is large to some extent in the target image showing a small number of people, the background composition is also evaluated, and the overall evaluation score including the evaluation result of such composition is calculated. Then. Therefore, an accurate and comprehensive evaluation can be presented to the user for various small-sized person photographs.

  The computer 10 that executes the image evaluation process may be, for example, a personal computer used by a user, or a computer that controls a commercial print minilab machine (so-called minilab machine) placed in a DPE shop. . The image evaluation process may be realized by the printer 30. In this case, a configuration in which the controller incorporated in the printer 30 executes the process according to the program data P (the process shown in FIG. 2) and displays the overall evaluation score on the display unit (liquid crystal panel) provided in the printer 30. And In addition, the order of each evaluation process shown to S150-S190 of FIG. 2 is not limited to the process order mentioned above.

DESCRIPTION OF SYMBOLS 10 ... Computer, 11 ... CPU, 12 ... Internal memory, 13 ... HDD, 20 ... DSC, 30 ... Printer, 40 ... Memory card, 50 ... Display, 60 ... Input operation part, 121 ... Image evaluation processing part, 121a ... Face Image detection unit, 121b ... Face number determination unit, 121c ... Face position evaluation unit, 121d ... Face reflection evaluation unit, 121e ... Face orientation evaluation unit, 121f ... Defective condition evaluation unit, 121g ... Background composition evaluation unit, 121h ... Comprehensive evaluation 122, display processing unit, 123 ... print processing unit

Claims (8)

A face detection step of detecting a face image from the image represented by the input image data;
A face number determination step of determining whether or not the number of detected face images is equal to or less than a specific number;
When it is determined that the number of face images is equal to or less than a specific number, the position of the face image in the image data, the evaluation of the appearance of the face, the evaluation of the face direction, and predetermined defect conditions are determined. An image evaluation method comprising: an evaluation execution step of performing an evaluation, calculating a comprehensive evaluation result based on each evaluation result, and outputting the comprehensive evaluation result to the outside.   In the evaluation execution step, the composition of the image represented by the image data is determined when the number of the face images is equal to or less than a specific number and the ratio of the area of the face image in the image represented by the image data is equal to or less than a predetermined value. The image evaluation method according to claim 1, wherein the comprehensive evaluation result is calculated based on the evaluation result of the composition and the evaluation results other than the evaluation result of the composition.   In the evaluation execution step, as an evaluation of the composition, an evaluation whether or not the composition corresponds to the line perspective, an evaluation whether or not the composition corresponds to the air perspective, and whether or not the composition includes a horizontal line. The image evaluation method according to claim 2, wherein at least one of the evaluations is performed.   4. The evaluation according to claim 1, wherein, in the evaluation execution step, as the evaluation of the facial appearance, an evaluation is performed on at least one element among facial contrast, facial expression, and facial sharpness. Image evaluation method.   In the evaluation execution step, as the evaluation of the face orientation, it is evaluated whether or not a space of a predetermined size exists in a direction in which the face faces in the image represented by the image data. The image evaluation method according to claim 4.   6. The image evaluation method according to claim 1, wherein, in the evaluation execution step, at least one element of exposure and blur is evaluated as the evaluation of the defect condition. An image evaluation program for causing a computer to perform image evaluation,
A face detection function for detecting a face image from an image represented by input image data;
A face number determination function for determining whether the number of detected face images is equal to or less than a specific number;
When it is determined that the number of face images is equal to or less than a specific number, the position of the face image in the image data, the evaluation of the appearance of the face, the evaluation of the face direction, and predetermined defect conditions are determined. An image evaluation program characterized by performing an evaluation, calculating a comprehensive evaluation result based on each evaluation result, and executing an evaluation execution function for outputting the comprehensive evaluation result to the outside. A face detection unit that detects a face image from the image represented by the input image data;
A face number determination unit for determining whether or not the number of detected face images is equal to or less than a specific number;
When it is determined that the number of face images is equal to or less than a specific number, the position of the face image in the image data, the evaluation of the appearance of the face, the evaluation of the face direction, and predetermined defect conditions are determined. A printing apparatus comprising: an evaluation execution unit that performs evaluation and calculates a comprehensive evaluation result based on each evaluation result and outputs the comprehensive evaluation result to the outside.

Images