JP5825172B2 - Image determination apparatus, image determination method, and computer program for image determination - Google Patents

Description

  The present invention relates to, for example, an image determination apparatus, an image determination method, and an image determination computer program that determine the image quality of a plurality of images including the same subject and select images with good image quality.

  By continuously shooting the subject, a plurality of images showing the subject may be generated, and the user may select the most favorite image from the plurality of images. In particular, with the recent spread of digital cameras equipped with a continuous shooting function, it is no longer necessary to print all images on a film, so that the user can easily select such images. There has also been proposed a technique for automatically selecting a good image from a plurality of images obtained by continuously photographing a subject (see, for example, Patent Documents 1 and 2).

  For example, in Patent Document 1, the quality of image data is evaluated in an evaluation area partially provided on the screen, and image data with high evaluation is selected from the image data subjected to the quality evaluation. A technique for recording is disclosed.

  Patent Document 2 discloses a technique for determining whether or not the image data is acceptable based on at least one of the focus degree and the degree of blurring in the image data, and displaying only the image data for which the pass determination has been made. Has been.

JP 2000-209383 A JP 2006-31340 A

  However, like a portrait photograph, depending on the type of subject, only a part of the image may be important and the other parts may not be important. Therefore, even if the image quality is evaluated based on an image quality index obtained from the entire image, such as the degree of focusing on the entire image, the quality of the image may not be properly determined.

  Further, if the subject is not specified, the device that determines the quality of the image cannot know in advance in which position on the image the important part of the subject appears when determining the quality of the image. . For example, there may be a case where the subject is a paper surface and a part of a character string written on the paper surface is information to be left as an image. However, only a user who wants to take a picture of the page knows where in the image a part of the character string appears. Therefore, an important part of the subject may not be shown in the specific part on the preset image. For this reason, even if the quality of the entire image is determined using only the data of the specific part, the quality of the image may not be determined appropriately.

  Accordingly, the present specification provides an image determination apparatus capable of selecting an image with good image quality from among a plurality of images in which the subject is captured, regardless of the position of the subject on the image or the type of the subject. Objective.

  According to one embodiment, an image determination device is provided. This image determination apparatus is configured to divide each of a plurality of images showing the same subject into a plurality of partial regions, and for each of the plurality of images, the subject of each subject in the partial region. A sharpness calculation unit for determining the sharpness of the image of the part, and a reference for setting the highest sharpness of the partial areas of the plurality of images as the reference sharpness of the partial areas for each of the partial areas For each of the plurality of images, the sharpness determination unit calculates an evaluation value that increases as the difference between the reference sharpness for each partial region and the sharpness of the partial region of the image decreases, and based on the evaluation value An evaluation unit that selects an image with good image quality from a plurality of images and outputs identification information representing the selected image.

The objects and advantages of the invention will be realized and attained by means of the elements and combinations particularly pointed out in the appended claims.
It should be understood that both the foregoing general description and the following detailed description are exemplary and explanatory and are not restrictive of the invention as claimed.

  The image determination apparatus disclosed in this specification can select an image with good image quality from among a plurality of images in which the subject is captured, regardless of the position of the subject on the image or the type of the subject.

It is a block diagram of the imaging device with which the image determination apparatus was incorporated. It is a block diagram of the image determination apparatus by 1st Embodiment. It is an example of a Laplacian filter. It is a figure which shows an example of the histogram of the edge intensity | strength about one partial area | region. (A) represents an example of the sharpness calculated | required for every partial area | region of each of the some image in which the same subject was reflected. (B) represents the reference sharpness for each partial region determined from each image shown in (a). It is an example of a deduction function table representing a deduction function D (α). It is an operation | movement flowchart of the image determination process by 1st Embodiment. It is a block diagram of the image determination apparatus by 2nd Embodiment. It is an operation | movement flowchart of an exclusion area | region specific process. (A) is a figure showing an example of a subject including a part which is not clear. (B) is a figure which shows the partial area | region set with respect to the image which image | photographed the to-be-photographed object shown to (a). (C) is an enlarged view of a region surrounded by a frame line shown in (b). It is an operation | movement flowchart of the image determination process by 2nd Embodiment. It is a block diagram of the computer which operate | moves as an image determination apparatus by the computer program which implement | achieves the function of each part of the image determination apparatus by each embodiment or its modification.

Hereinafter, an image determination apparatus according to some embodiments will be described with reference to the drawings.
This image determination apparatus determines an image with good image quality among a plurality of images in which the same subject is captured. For this purpose, the image determination apparatus divides each image into a plurality of partial areas, and obtains the sharpness of the image of the subject in the partial area for each partial area. And this image determination apparatus makes the highest sharpness among the sharpness of the partial area of a some image for every partial area as the reference | standard sharpness of the partial area. An image obtained by collecting and synthesizing all partial areas corresponding to the reference sharpness corresponds to a composition of partial areas that are in focus among a series of images in which the same subject is captured. Therefore, it is estimated that an image having a lower degree of reduction in sharpness from the reference sharpness of each partial region has less blur over the entire image. Therefore, this image determination device obtains an evaluation value that increases as the degree of decrease in the sharpness of the image from the reference sharpness for each partial region becomes smaller for each image, and an image with good image quality based on the evaluation value. Select.

  In this embodiment, the image to be processed is a gray image in which each pixel has only a luminance value. Alternatively, the image to be processed may be a color image expressed in the RGB color system. In this case, after the color image is converted into the HSV color system, the processing described below may be executed for the luminance component of each pixel.

  FIG. 1 is a schematic configuration diagram of an imaging apparatus in which an image determination apparatus according to one embodiment is incorporated. The imaging device 1 is an imaging device that can generate a plurality of images of the same subject by continuously shooting the subject at a predetermined shooting interval (for example, 3 to 5 frames / second). A mobile phone equipped with a camera having a continuous shooting function or a digital camera. As illustrated in FIG. 1, the imaging device 1 includes a camera module 2, an operation unit 3, a display unit 4, a storage unit 5, an image determination device 6, and a control unit 7. Furthermore, the imaging device 1 may have an interface circuit (not shown) according to a serial bus standard such as a universal serial bus in order to connect the imaging device 1 to another device such as a computer or a television. The control unit 7 and other units of the imaging device 1 are connected by, for example, a bus.

  The camera module 2 includes an image sensor having an array of solid-state imaging elements arranged in a two-dimensional manner, and an imaging optical system that forms an image of a subject on the image sensor. And the camera module 2 produces | generates the image which image | photographed the object with the aperture diameter and shutter speed notified from the control part 7. FIG. In the present embodiment, the camera module 2 continuously shoots the subject with the aperture diameter and shutter speed adjusted so as to obtain an appropriate exposure amount, and generates an image every time the image is taken. And the camera module 2 memorize | stores each image in the memory | storage part 5 with the imaging | photography time.

  The operation unit 3 includes, for example, various operation buttons or dial switches for the user to operate the imaging device 1. The operation unit 3 transmits to the control unit 7 a control signal for setting the shutter speed, the aperture diameter, or the like, or a control signal for starting shooting or focusing, in accordance with a user operation.

  The display unit 4 includes, for example, a display device such as a liquid crystal display device, and displays various types of information received from the control unit 7 or an image generated by the camera module 2 or a reduced image thereof. Note that the operation unit 3 and the display unit 4 may be integrally formed using, for example, a touch panel display.

  The storage unit 5 includes, for example, a readable / writable volatile or nonvolatile semiconductor memory circuit. The storage unit 5 stores the image received from the camera module 2. The storage unit 5 passes the image to the image determination device 6 in response to a read request from the image determination device 6. Further, the storage unit 5 may store various data used in the image determination process executed by the image determination device 6 or intermediate calculation results. Further, the storage unit 5 erases images other than the image determined to have the best image quality among a plurality of images in which the same subject is captured by the image erasing command from the control unit 7. Furthermore, when each function of the image determination device 6 is realized by a computer program executed on a processor of the control unit 7, the computer program may be stored.

  The image determination device 6 selects an image having the best image quality from a plurality of images obtained by continuously photographing the same subject. Then, the image determination device 6 notifies the control unit 7 of the identification information of the selected image. The details of the image determination device 6 will be described later.

  The control unit 7 includes at least one processor and its peripheral circuits, and controls the entire imaging apparatus 1. For example, the control unit 7 sets the shutter speed or the aperture diameter according to the setting signal received from the operation unit 3 and the exposure amount of the subject. Further, the control unit 7 may adjust the color balance for each image, or execute processing such as edge enhancement or contrast enhancement. Further, the control unit 7 outputs an image determined to be the best image quality by the image determination device 6 among a plurality of images including the same subject, or causes the display unit 4 to display the image. Alternatively, the control unit 7 may notify the storage unit 5 of an instruction to delete other than the image determined to have the best image quality among a plurality of images in which the same subject is captured.

Hereinafter, each component of the image determination apparatus 6 will be described. In FIG. 2, the block diagram of the image determination apparatus 6 is shown. The image determination device 6 includes a dividing unit 11, a sharpness calculation unit 12, a reference sharpness determination unit 13, and an evaluation unit 14.
These units included in the image determination device 6 are formed as one integrated circuit in which circuits corresponding to the units are integrated. In addition, these units included in the image determination device 6 may be formed as separate circuits.

The image determination device 6 reads a plurality of images showing the same subject from the storage unit 5. For example, the image determination device 6 checks the shooting time of each image, and the time difference from the earliest shooting time to the latest shooting time is the same for a plurality of images within a predetermined period in which it can be considered that the same subject was shot. It is determined that the subject is captured. For example, the predetermined period can be a period obtained by multiplying the number of frames that can be captured by one continuous shooting of the camera module 2 by the shooting interval at the time of continuous shooting.
Alternatively, the image determination device 6 determines that the same subject is captured in the image selected by operating the operation unit 3 while confirming the image displayed on the display unit 4 by the user, and the selected image is selected. The read image may be read from the storage unit 5.
The image determination device 6 sequentially passes the read images to the dividing unit 11.

  The dividing unit 11 divides each image showing the same subject into a plurality of partial areas. The dividing unit 11 divides the image so that each partial area has a size sufficient to calculate the sharpness of the partial area. The divided areas set for each image are the same. For example, the dividing unit 11 divides each image into partial regions each having a rectangular shape of 3 × 3.

  For example, the dividing unit 11 assigns identification numbers to the partial areas in the raster scan order from the upper left partial area, and passes each partial area to the sharpness calculation unit 12 together with the identification number of the partial area.

  The sharpness calculation unit 12 calculates the sharpness of the image of the subject in each partial area. For example, in this embodiment, the ratio of the number of pixels corresponding to the edge of the subject to the total number of images in which a part of the subject included in the partial area is captured is defined as the sharpness. For this purpose, the sharpness calculation unit 12 obtains the edge strength for each pixel by applying a Laplacian filter to each pixel in the partial region.

FIG. 3 is an example of a Laplacian filter used in the present embodiment. In this embodiment, the Laplacian filter 300 is a filter having horizontal 3 × vertical 3 pixels, and provides secondary differential values in the vertical direction and the horizontal direction. In the present embodiment, the Laplacian filter 300 outputs a value represented by the following equation.
L (x, y) = 4p (x, y) -p (x-1, y) -p (x + 1, y) -p (x, y-1) -p (x, y + 1)
p (x, y) is the pixel value of the horizontal coordinate x and the vertical coordinate y, and L (x, y) is the edge intensity that is the output value of the Laplacian filter 300 for the pixel of the coordinate (x, y). Represents.
The usable Laplacian filter may be a Laplacian filter having 5 × 5 pixels. In addition, the sharpness calculation unit 12 may obtain the edge strength for each pixel by applying a first-order differential filter for each pixel in the partial region. For example, the sharpness calculation unit 12 calculates a horizontal edge strength by applying a horizontal sobel filter for each pixel, and calculates a vertical edge strength by applying a vertical sobel filter. . Then, for each pixel, the sharpness calculation unit 12 may use the larger of the horizontal edge intensity absolute value and the vertical edge intensity absolute value as the edge intensity of the pixel.

p_max及びp_minは、それぞれ、注目する部分領域が属する画像全体の画素の値の最大値及び最小値である。またC_maxは、注目する部分領域が属する画像のコントラストが取り得る最大値であり、例えば、画素値が取り得る最大値と最小値の差である。例えば、画素値が8ビットで表される場合、C_maxは255となる。なお、注目する部分領域に、壁または空のような一様な被写体が写っている場合において、過度にエッジ強度が増幅されることを防ぐため、(p_max-p_min)が所定の下限値C_low以下となる場合、ゲイン値gを(C_max/C_low)とする。ただし、C_lowは、例えば、C_maxの1/4に設定される。
なお、鮮鋭度算出部１２は、簡単化のため、及び、エッジ強度を表すためのビット数を削減するために、エッジ強度の絶対値が画素値の最大値（例えば、255）を超える場合には、エッジ強度の絶対値をその最大値とする。 The sharpness calculation unit 12 normalizes the edge strength of each pixel with the contrast of the entire image in order to make the value independent of the contrast of the image. For example, the sharpness calculation unit 12 normalizes the edge strength by multiplying the edge strength of each pixel in the partial region by the gain value g calculated by the following equation.

p _max and p _min are the maximum value and the minimum value of the pixel values of the entire image to which the partial region of interest belongs, respectively. C _max is the maximum value that can be taken by the contrast of the image to which the partial region of interest belongs, and is, for example, the difference between the maximum value and the minimum value that the pixel value can take. For example, when the pixel value is represented by 8 bits, C _max is 255. Note that (p _max -p _min ) is a predetermined lower limit value to prevent excessive edge intensity amplification when a uniform subject such as a wall or sky appears in the target partial area. When C _low or less, the gain value g is set to (C _max / C _low ). However, C _low is set to 1/4 of C _max , for example.
Note that the sharpness calculation unit 12 is used when the absolute value of the edge strength exceeds the maximum value (for example, 255) of the pixel value in order to simplify and reduce the number of bits for representing the edge strength. Uses the absolute value of the edge strength as its maximum value.

ここで、h(j)は、エッジ強度の絶対値がjとなる画素の数を表す。またp_maxは、エッジ強度の絶対値の最大値である。そしてAは、被写体が写っている画素の正規化エッジ強度の絶対値の下限値であり、例えば、様々な被写体が写ったサンプル画像を解析することにより予め実験的に定められる。例えば、ゲイン値gが1であれば、Aは32に設定される。またゲイン値gが1より大きく、かつ2以下であれば、Aは64に設定される。さらに、ゲイン値gが2より大きければ、Aは96に設定される。 Next, the sharpness calculation unit 12 obtains a histogram of edge strength by obtaining the frequency of pixels for each absolute value of edge strength for each partial region. The sharpness calculation unit 12 refers to the histogram and sets the ratio of the number of pixels in which the edge of the subject is captured to the number of pixels in which the subject is captured as the sharpness of the partial region. That is, the sharpness is calculated according to the following equation.

Here, h (j) represents the number of pixels for which the absolute value of the edge strength is j. Further, p _max is the maximum absolute value of the edge strength. A is a lower limit value of the absolute value of the normalized edge intensity of the pixel in which the subject is photographed, and is experimentally determined in advance, for example, by analyzing sample images in which various subjects are photographed. For example, if the gain value g is 1, A is set to 32. If the gain value g is greater than 1 and less than or equal to 2, A is set to 64. Further, if the gain value g is greater than 2, A is set to 96.

  FIG. 4 is a diagram showing an example of an edge intensity histogram for one partial region. In FIG. 4, the horizontal axis represents the absolute value of the edge strength, and the vertical axis represents the number of pixels. A histogram 400 is a histogram representing the number of pixels for each absolute value of edge strength. The value of the denominator in equation (2) is the total number of pixels for each absolute value of the edge strength included in the section 401. On the other hand, the numerator value of the equation (2) is the number of pixels having the maximum value Max of the absolute value of the edge strength.

  The sharpness calculation unit 12 may calculate the sharpness of each partial region according to any of various other methods for obtaining the sharpness of an image. For example, the sharpness calculation unit 12 obtains a frequency spectrum included in the partial region by performing fast Fourier transform or wavelet transform on the partial region. Then, the sharpness calculation unit 12 may use the average absolute value of the spectrum values in a specific frequency band higher than a predetermined threshold as the sharpness.

  The sharpness calculation unit 12 stores the sharpness of each partial area in the storage unit 5 together with an identification number representing the partial area and an identification number of an image including the partial area.

  The reference sharpness determination unit 13 sets, for each partial region, the highest sharpness among the sharpnesses of the partial regions of a plurality of images in which the same subject is captured as the reference sharpness for the partial region. That is, the reference sharpness represents the sharpness of the image of the subject that is in the most focused state in the series of images for the partial region corresponding to the reference sharpness.

FIG. 5A illustrates an example of the sharpness obtained for each partial region of each of a plurality of images in which the same subject is captured. FIG. 5B shows the reference sharpness for each partial region determined from each image shown in FIG. In this example, there are four images.
As shown in FIG. 5A, each of the images 501 to 504 is set to 3 × 3 partial areas, and the numerical values shown in the partial areas are the values of the partial areas. Represents sharpness. FIG. 5B shows the reference sharpness for each partial region. For example, focusing on the upper left partial area, the sharpness of the upper left partial area 511 of the image 501 is the largest, 0.9 among the images 501 to 504. Accordingly, the reference sharpness of the upper left partial region 521 in FIG. 5B is 0.9. Similarly, focusing on the central partial area, the sharpness of the central partial area 512 of the image 502 among the images 501 to 504 is 0.8, which is the largest. Accordingly, the reference sharpness of the central partial region 522 in FIG. 5B is 0.8.

  The reference sharpness determination unit 13 notifies the evaluation unit 14 of the reference sharpness for each partial region.

S(k)は、画像上のk番目の部分領域の鮮鋭度であり、R(k)は、k番目の部分領域の基準鮮鋭度である。なお、部分領域の序列は、例えば、左上端からラスタスキャン順に設定される。Mは、一つの画像に設定される部分領域の総数であり、図５（ａ）及び図５（ｂ）に示された例では、M=9である。そして減点関数D(α)は、変数αが大きいほど、負の大きな値を出力する単調減少関数である。減点関数D(α)は、例えば、αと出力値との関係を規定するテーブルによって表される。 The evaluation unit 14 calculates an evaluation value that increases as the difference between the reference sharpness for each partial region and the sharpness of the image is smaller for each of a plurality of images in which the same subject is captured. For example, the evaluation unit 14 calculates the evaluation value E of each image according to the following equation.

S (k) is the sharpness of the kth partial area on the image, and R (k) is the reference sharpness of the kth partial area. For example, the order of the partial areas is set in the raster scan order from the upper left corner. M is the total number of partial areas set in one image, and M = 9 in the example shown in FIGS. 5A and 5B. The deduction function D (α) is a monotonously decreasing function that outputs a larger negative value as the variable α is larger. The deduction function D (α) is represented by, for example, a table that defines the relationship between α and the output value.

  FIG. 6 is an example of a deduction function table representing the deduction function D (α). A representative value of α (= R (k) −S (k)) is shown in each column of the left column of the deduction function table 600, and each column of the right column has α of the adjacent column. The deduction value corresponding to the representative value is shown. For example, if the representative value is “0.2”, the deduction point value is “−0.1”. When the actual α value is different from the representative value shown in each column in the left column of the deduction function table 600, the evaluation unit 14 selects the closest representative value from the representative value. select. Then, the evaluation unit 14 sets the deduction value corresponding to the selected representative value as the output value of the deduction function D (α). For example, if the actual value of α is “0.28”, the representative value “0.3” in the deduction function table 600 is closest to the value of α. Therefore, the evaluation unit 14 sets the deduction point value “−0.2” corresponding to the representative value “0.3” as the output value of the deduction point function D (α).

Note that, according to the modification, the evaluation unit 14 may calculate the evaluation value E according to the following equation instead of the equation (3).

  As is clear from the equations (3) and (4), the evaluation value E increases as the sharpness of each partial region is closer to the reference sharpness. In other words, the evaluation value E increases as the overall degree of blur from the state in which each partial area is in focus is the smallest in a series of images in which the same subject is captured. Therefore, the evaluation value E is higher when the overall sharpness is higher than that of the image with high sharpness only in the specific partial region. As is clear from the equations (3) and (4), the evaluation value E is a value calculated without depending on the position of the subject.

  The evaluation unit 14 selects an image having the highest evaluation value from among a plurality of images as an image having the best image quality. Then, the evaluation unit 14 notifies the control unit 7 of the identification number of the selected image.

FIG. 7 is an operation flowchart of image determination processing executed by the image determination apparatus 6.
The image determination device 6 reads a plurality of images showing the same subject from the storage unit 5 (step S101). The dividing unit 11 divides each image into a plurality of partial areas (step S102). Then, the dividing unit 11 passes each partial area to the sharpness calculating unit 12 together with the identification number.

  The sharpness calculation unit 12 calculates the sharpness for each partial region (step S103). Thereafter, the sharpness calculation unit 12 outputs the sharpness for each partial region to the reference sharpness determination unit 13 and the evaluation unit 14. For each partial region, the reference sharpness determination unit 13 sets the highest value of the sharpness of the partial region of each image as the reference sharpness of the partial region (step S104). Then, the reference sharpness determination unit 13 notifies the evaluation unit 14 of the reference sharpness for each partial region.

The evaluation unit 14 obtains a difference between the reference sharpness for each partial region and the sharpness of the image for each image, thereby increasing the evaluation value as the difference between the reference sharpness and the sharpness for each partial region is smaller. Is calculated (step S105). Then, the evaluation unit 14 determines that the image having the highest evaluation value has the best image quality, and selects the image (step S106). Then, the image determination device 6 outputs the identification number of the selected image to the control unit 7.
Thereafter, the image determination device 6 ends the image determination process.

  As described above, this image determination device obtains the sharpness for each partial area of the image, and for each partial area, the highest sharpness in the series of images and the sharpness of the image. The evaluation value is obtained by the difference of. Therefore, the evaluation value is higher for an image with a high overall sharpness than for an image with a very high sharpness in some partial areas. The evaluation value does not depend on the position of the subject on the image. Therefore, this image determination apparatus can select an image having the best image quality from among a plurality of images showing the same subject, regardless of the position and type of the subject.

Next, an image determination apparatus according to the second embodiment will be described.
Generally, the user tries to focus on a subject near the center of the shooting range. Therefore, if the sharpness of the subject is uniform throughout and is focused on the portion of the subject near the center of the shooting range, the sharpness of the partial area near the center of the image is It tends to be higher than the sharpness of the partial area at the edge of the image. However, if a part of the subject is not clearer than the other parts in the first place, the sharpness of the partial area in which a part of the subject is reflected is low. Therefore, the image determination apparatus according to the second embodiment does not use a partial area in which the sharpness is lower than the sharpness of the surrounding partial areas in each of a plurality of images in which the same subject is captured for image quality evaluation. Thus, the evaluation value is obtained more appropriately.

FIG. 8 shows a configuration diagram of an image determination apparatus 61 according to the second embodiment. The image determination device 61 includes a dividing unit 11, a sharpness calculation unit 12, a reference sharpness determination unit 13, an evaluation unit 14, an alignment unit 15, and an excluded area specifying unit 16.
These units included in the image determination device 61 are formed as one integrated circuit in which circuits corresponding to the units are integrated. Each of these units included in the image determination device 61 may be formed as a separate circuit.

  In FIG. 8, each constituent element included in the image determination device 61 is assigned the same reference number as the corresponding constituent element included in the image determination device 6 according to the first embodiment shown in FIG. 2. The image determination device 61 according to the second embodiment is different from the image determination device 6 according to the first embodiment in that it includes an alignment unit 15 and an excluded area specifying unit 16. Therefore, in the following, the positioning unit 15, the excluded area specifying unit 16, and related parts will be described.

  The alignment unit 15 obtains a positional deviation amount between the positions of the subject images shown in each of the plurality of images. Then, the alignment unit 15 aligns the image of the subject on each image by moving the image of the subject in each image so as to cancel out the displacement.

  The alignment unit 15 aligns any one of the plurality of images, for example, the first captured image as a reference, and aligns the other images with the reference image. For this purpose, the alignment unit 15 executes the following processing for each set of a reference image and other images.

  In order to obtain the amount of displacement, the alignment unit 15 extracts at least one feature point on the reference image, for example. For example, the alignment unit 15 sets a plurality of points detected by applying a corner detector to the reference image as feature points. The alignment unit 15 can use, for example, a Harris detector as such a corner detector. The alignment unit 15 may use another feature point extraction detector to extract feature points from the reference image. For example, a Moravec detector, a Smallest Univalue Segment Assymilating Nucleus (SUSAN) detector, a Kanade-Lucas-Tomasi (KLT) Tracker, or a Scale-invariant feature transform (SIFT) detector is used as such a detector. May be.

  Next, for each feature point extracted from the reference image, the alignment unit 15 sets a predetermined area centered on the feature point as a template. For example, the alignment unit 15 performs template matching while changing the relative position between the template and the other image to obtain the similarity, and the center position of the region where the similarity is the maximum is used as a reference. It is obtained as a feature point on the other image corresponding to the feature point on the image. At that time, the alignment unit 15 sets the corresponding pixel on the other image as the first search point for the feature point of interest. Then, the alignment unit 15 obtains the similarity for the search point and its neighboring 8 neighboring pixels or 24 neighboring pixels, and sets the pixel having the maximum similarity as the next search point. The alignment unit 15 may repeat the above processing until the search point stops moving, and may use the finally obtained search point as a feature point. Note that the alignment unit 15 calculates, for example, a normalized cross-correlation value between the template and the region to be compared on the other image as the similarity. Alternatively, the alignment unit 15 may use the reciprocal of the sum Δ of absolute values of pixel value differences between corresponding pixels of the template and the comparison region on the other image, or 1 / (1 + Δ) as the similarity. .

  The alignment unit 15 sets the feature point on the other image and the corresponding point on the reference image as the same part of the subject only when the maximum value of the similarity is equal to or greater than a predetermined threshold. A set of corresponding feature points may be used. In this case, if the maximum value of the similarity is less than the threshold value, the alignment unit 15 determines that the feature point matching the feature point corresponding to the template does not exist in the other image, and sets the feature point as the feature point. You may exclude from the search object of this group. The higher the predetermined threshold value is set, the more the alignment unit 15 can improve the certainty that the set of feature points corresponds to the same part of the subject. For example, the predetermined threshold is set to a value obtained by multiplying the maximum value that the similarity can take by 0.8 to 0.9. Alternatively, the alignment unit 15 may increase the predetermined threshold as the number of feature points extracted from the reference image increases. Thereby, the alignment unit 15 can extract only a set of feature points that are highly likely to correspond to the same part when the number of feature points extracted from the reference image is large. Further, even if the number of feature points extracted from the reference image is small, the alignment unit 15 can extract a sufficient number of feature point sets for alignment of the image of the subject on both images.

  Next, the alignment unit 15 is based on the obtained set of feature points, and the coordinate conversion parameter for matching the position of the subject in the other image with the position of the subject in the reference image. Is calculated. The set of coordinate transformation parameters can be, for example, a set of affine transformation coefficients applied to each pixel of the other image.

  In this case, the alignment unit 15 determines a set of coordinate conversion parameters by, for example, the least square method. That is, the alignment unit 15 converts the coordinates of the feature points of the other image according to the affine transformation for each of the plurality of feature point sets, using the coefficients of the affine transformations included in the set of coordinate transformation parameters as variables. And the alignment part 15 calculates | requires the average value of the square of the distance between the feature points after conversion about each set of feature points. The alignment unit 15 obtains a set of coordinate conversion parameters that minimizes the mean square value.

  The alignment unit 15 may obtain a set of coordinate conversion parameters using any of various other alignment methods.

The alignment unit 15 affine-transforms the coordinates of each pixel of the other image using the obtained set of coordinate transformation parameters. Thereby, the alignment part 15 can correct | amend each image so that the position of the image of the to-be-photographed object reflected in each image may correspond.
The alignment unit 15 outputs each image in which the image of the subject is aligned to the dividing unit 11.

  The dividing unit 11 divides each image in which the image of the subject is aligned by the alignment unit 15 into a plurality of partial areas. In this embodiment, each image is divided into, for example, horizontal 16 × vertical 12 partial areas. Then, the dividing unit 11 outputs each partial region to the sharpness calculating unit 12. The sharpness calculation unit 12 calculates the sharpness of each partial area. Then, the sharpness calculation unit 12 outputs the sharpness of each partial region to the reference sharpness determination unit 13. Then, the reference sharpness determination unit 13 sets, for each partial region, the highest sharpness among the sharpnesses of the partial regions of the plurality of images as the reference sharpness of the partial region. The reference sharpness determination unit 13 outputs the reference sharpness of each partial region to the exclusion region specifying unit 16.

The excluded area specifying unit 16 specifies a partial area that is not used for calculating the evaluation value as an excluded area.
FIG. 9 is an operation flowchart of an excluded area specifying process executed by the excluded area specifying unit 16.
The excluded area specifying unit 16 sets any of the partial areas not set as the focused partial area as the focused partial area (step S201). For example, the excluded area specifying unit 16 sets a partial area to be noted in the raster scan order.
The excluded area specifying unit 16 determines whether or not the reference sharpness is less than the excluded reference value for the partial area of interest (step S202). The exclusion reference value is an upper limit value of the sharpness of a partial area where a non-clear subject is captured, and when the sharpness is calculated according to equation (2), the exclusion reference value is set to 0.3, for example.

When the reference sharpness is less than the exclusion reference value (Yes at Step S202), the exclusion area specifying unit 16 sets the partial area of interest as an exclusion candidate area (Step S203).
When the reference sharpness is equal to or greater than the exclusion reference value (No at Step S202), or after Step S203, the exclusion area specifying unit 16 determines whether or not all the partial areas have been set as the partial areas of interest (Step S202). S204). If any of the partial areas is not set as the focused partial area (step S204—No), the excluded area specifying unit 16 repeats the processes after step S201. On the other hand, if all the partial areas are set as the partial areas of interest (step S204—Yes), the excluded area specifying unit 16 performs a labeling process on the excluded candidate areas. Thereby, the exclusion area specifying unit 16 groups the exclusion candidate areas adjacent to each other into one exclusion candidate group (step S205).

The exclusion area specifying unit 16 determines whether or not the exclusion candidate group is surrounded by a sharp partial area (step S206). If the exclusion candidate group is surrounded by a sharp partial area (step S206-Yes), it is assumed that the part of the subject shown in the exclusion candidate group is also focused, so that the part itself is not clear. This is presumed to be the reason why the sharpness of the exclusion candidate group is lower than the surroundings. Therefore, the exclusion area specifying unit 16 sets a partial area in the exclusion candidate group as an exclusion area (step S207). Specifically, if the reference sharpness of all partial areas adjacent to the exclusion candidate group is higher than the focus reference value, the exclusion area specifying unit 16 determines that the exclusion candidate group is surrounded by a sharp partial area. judge. In addition, when one end of the exclusion candidate group is in contact with the image end, the exclusion area specifying unit 16 determines that the reference sharpness is higher than the focus reference value for all partial areas adjacent to the end of the exclusion candidate group other than the one end. If it is high, it is determined that the exclusion candidate group is surrounded by a sharp partial region. The focus reference value is set to a value higher than the exclusion target reference value, for example, 0.6.
On the other hand, if the exclusion candidate group is not surrounded by a sharp partial region (No in step S206), the exclusion region specifying unit 16 does not set each partial region in the exclusion candidate group as an exclusion region (step S208). After step S207 or S208, the excluded area specifying unit 16 ends the excluded area specifying process. When there are a plurality of exclusion candidate groups, the exclusion region specifying unit 16 performs the processes of steps S206 to S208 for each exclusion candidate group.

FIG. 10A is a diagram illustrating an example of a subject including an unclear part. FIG. 10B is a diagram showing a partial region set for the image in which the subject shown in FIG.
A subject 1000 shown in FIG. 10A is a paper surface on which a character string 1001 and a photograph 1002 surrounded by the character string are described. In the subject 1000, the photograph 1002 is less sharp than the surrounding character string 1001. In FIG. 10B, an image 1010 showing the subject 1000 is shown in units of partial areas, and a photograph 1002 is shown in an area surrounded by a frame line 1011.

  FIG. 10C is an enlarged view of the region surrounded by the frame line 1012 shown in FIG. 10B, and shows the reference sharpness of each partial region included in the frame line 1012. The reference sharpness of each partial area in the frame line 1011 in which the photograph 1002 is shown is lower than the exclusion reference value (for example, 0.3). On the other hand, since a part of the character string 1001 is shown in each partial area around the frame line 1011, the reference sharpness of these partial areas is higher than the focus reference value (for example, 0.6). . Therefore, in this example, each partial area within the frame line 1011 is an excluded area.

  The exclusion region specifying unit 16 notifies the evaluation unit 14 of the identification number of the partial region that is the exclusion region.

  The evaluation unit 14 calculates an evaluation value for each image according to the formula (3) or the formula (4), excluding the partial area that is the excluded area. Then, the evaluation unit 14 determines that the image having the highest evaluation value is the image having the best image quality.

FIG. 11 is an operation flowchart of image determination processing executed by the image determination apparatus 61 according to the second embodiment.
The image determination device 61 reads a plurality of images showing the same subject from the storage unit 5 (step S301). The alignment unit 15 corrects each image so that the position of the image of the subject in each image matches (step S302). The dividing unit 11 divides each corrected image into a plurality of partial areas (step S303). Then, the dividing unit 11 passes each partial region to the sharpness calculating unit 12.

  The sharpness calculation unit 12 calculates the sharpness for each partial region (step S304). Thereafter, the reference sharpness determination unit 13 sets, for each partial region, the highest value among the respective sharpnesses of the plurality of images as the reference sharpness of the partial region (step S305). Then, the reference sharpness determination unit 13 outputs the reference sharpness for each partial region to the excluded region specifying unit 16 and the evaluation unit 14. The excluded area specifying unit 16 specifies an excluded area based on the reference sharpness of each partial area (step S306). Then, the excluded area specifying unit 16 notifies the evaluation unit 14 of the identification number of the excluded area.

For each image, the evaluation unit 14 obtains the difference between the reference sharpness of each partial region other than the excluded region and the sharpness of the image, so that the difference between the reference sharpness and the sharpness for each partial region is smaller. A higher evaluation value is calculated (step S307). The evaluation unit 14 determines whether or not the highest evaluation value Eh is higher than a predetermined threshold value Th (step S308). If the highest evaluation value Eh is higher than the threshold value Th (step S308—Yes), the image determination device 6 outputs the identification number of the image determined to have the best image quality corresponding to the evaluation value Eh to the control unit 7. (Step S309). On the other hand, when the evaluation value Eh is equal to or less than the threshold value Th (step S308-No), the image determination device 6 outputs a determination result that there is no image with good image quality to the control unit 7 (step S310). Note that the threshold value Th is set in advance based on, for example, evaluation values calculated for a plurality of sample images with good image quality and evaluation values calculated for a plurality of sample images with poor image quality.
After step S309 or S310, the image determination device 61 ends the image determination process.

  As described above, the image determination apparatus according to the second embodiment calculates the evaluation value of each image except for a partial area in which a part of a subject that is originally not clear is shown. Therefore, this image determination apparatus can determine an image with good image quality more appropriately.

  In addition, according to the modification of 2nd Embodiment, the position alignment part 15 may be abbreviate | omitted. In particular, if the shooting interval is sufficiently short, the position where the subject appears in each image hardly changes. Therefore, in each image, it can be considered that the same part of the subject appears in the partial region at the same position. .

  Similarly to the evaluation unit according to the second embodiment, the evaluation unit according to the first embodiment performs the processing of Steps S308 to S310, and only when the highest evaluation value is higher than the threshold value Th. An image corresponding to a high evaluation value may be selected as an image having the best image quality.

  According to the modification of each embodiment described above, the evaluation unit may select all images having an evaluation value higher than the threshold value Th as having good image quality. In this case, the image determination device notifies the identification number of each selected image to the control unit of the imaging device.

  The function of each unit of the image determination device according to the above-described embodiment or its modification may be realized by a computer program executed on a processor. Such a computer program may be provided in a form recorded on a computer-readable recording medium such as a magnetic recording medium or an optical recording medium.

FIG. 12 is a configuration diagram of a computer that operates as an image determination device when a computer program that realizes the functions of the respective units of the image determination device according to the above-described embodiment or its modification is operated.
The computer 100 includes a user interface unit 101, a communication interface unit 102, a storage unit 103, a storage medium access device 104, and a processor 105. The processor 105 is connected to the user interface unit 101, the communication interface unit 102, the storage unit 103, and the storage medium access device 104 via, for example, a bus.

  The user interface unit 101 includes, for example, an input device such as a keyboard and a mouse, and a display device such as a liquid crystal display. Alternatively, the user interface unit 101 may include a device such as a touch panel display in which an input device and a display device are integrated. For example, the user interface unit 101 outputs an operation signal for starting an image determination process to the processor 105 in accordance with a user operation.

The communication interface unit 102 may include a communication interface for connecting the computer 100 to an imaging device (not shown) capable of continuous shooting and a control circuit thereof. Such a communication interface can be, for example, Universal Serial Bus (Universal Serial Bus, USB).
Furthermore, the communication interface unit 102 may include a communication interface for connecting to a communication network according to a communication standard such as Ethernet (registered trademark) and a control circuit thereof.
In this case, the communication interface unit 102 acquires a plurality of images showing the same subject from other devices connected to the communication network, and stores these images in the storage unit 103. Further, the communication interface unit 102 may output an image selected by the processor 105 as having good image quality to another device via the communication network.

  The storage unit 103 includes, for example, a readable / writable semiconductor memory and a read-only semiconductor memory. The storage unit 103 stores a computer program for executing an image determination process executed on the processor 105, an intermediate calculation result obtained in the middle of the image determination process, and the like. The storage unit 103 stores an image received from the communication interface unit 102.

  The storage medium access device 104 is a device that accesses a storage medium 106 such as a magnetic disk, a semiconductor memory card, and an optical storage medium. For example, the storage medium access device 104 reads a computer program for image determination processing that is executed on the processor 105 stored in the storage medium 106 and passes the computer program to the processor 105. In addition, the storage medium access device 104 may write an image selected by the processor 105 as having good image quality from among a plurality of images in which the same subject is captured, to the storage medium 106.

  The processor 105 executes the image determination processing computer program according to the above-described embodiment or modification, thereby selecting an image with good image quality from among a plurality of images in which the same subject is captured. Then, the processor 105 outputs the selected image to another device via the communication interface unit 102.

  All examples and specific terms listed herein are intended for instructional purposes to help the reader understand the concepts contributed by the inventor to the present invention and the promotion of the technology. It should be construed that it is not limited to the construction of any example herein, such specific examples and conditions, with respect to showing the superiority and inferiority of the present invention. Although embodiments of the present invention have been described in detail, it should be understood that various changes, substitutions and modifications can be made thereto without departing from the spirit and scope of the present invention.

The following supplementary notes are further disclosed regarding the embodiment described above and its modifications.
(Appendix 1)
A dividing unit that divides each of a plurality of images of the same subject into a plurality of partial areas;
For each of the plurality of images, a sharpness calculation unit that calculates the sharpness of the image of the part of the subject that is reflected in the partial area for each partial area;
For each of the plurality of partial regions, a reference sharpness determination unit that sets the highest sharpness among the sharpnesses of the partial regions of the plurality of images as the reference sharpness of the partial region;
For each of the plurality of images, an evaluation value that is higher as the difference between the reference sharpness for each partial region and the sharpness of the image is smaller is calculated. An evaluation unit that selects an image with good image quality from the image and outputs identification information representing the selected image;
An image determining apparatus.
(Appendix 2)
Of the plurality of partial areas, a partial area whose reference sharpness is less than an exclusion reference value indicating that the part of the subject is unclear is extracted, and the extracted partial area is more than the exclusion reference value. If it is surrounded by a partial region having a reference sharpness higher than a high predetermined reference value, further comprising an excluded region specifying unit for setting the extracted partial region as an excluded region;
For each of the plurality of images, the evaluation unit calculates the difference between the reference sharpness and the sharpness of the image for a partial region other than the excluded region of the plurality of partial regions. The image determination apparatus according to appendix 1, wherein
(Appendix 3)
The image evaluation apparatus according to appendix 1 or 2, wherein the evaluation unit selects an image having the highest evaluation value from the plurality of images.
(Appendix 4)
The image evaluation apparatus according to appendix 1 or 2, wherein the evaluation unit selects an image having the evaluation value higher than a predetermined threshold among the plurality of images.
(Appendix 5)
The evaluation unit calculates a deduction value having a smaller value as the difference between the reference sharpness and the sharpness is larger for each partial region, and the sum of the deduction values of each partial region is used as the evaluation value. The image determination apparatus according to any one of appendices 1 to 4.
(Appendix 6)
The evaluation unit calculates a deduction value having a smaller value as the difference between the reference sharpness and the sharpness is larger for each partial region, and the sum of the deduction point value and the sharpness for each partial region is calculated. The image determination apparatus according to any one of appendices 1 to 4, wherein a total is the evaluation value.
(Appendix 7)
Divide each of multiple images of the same subject into multiple partial areas,
For each of the plurality of images, obtain the sharpness of the image of the part of the subject that is reflected in the partial area for each partial area,
For each of the plurality of partial regions, the highest sharpness among the sharpnesses of the partial regions of the plurality of images is the reference sharpness of the partial region,
For each of the plurality of images, an evaluation value that is higher as the difference between the reference sharpness for each partial region and the sharpness of the image is smaller is calculated. An image with good image quality is selected from, and identification information representing the selected image is output.
An image determination method including the above.
(Appendix 8)
Divide each of multiple images of the same subject into multiple partial areas,
For each of the plurality of images, obtain the sharpness of the image of the part of the subject that is reflected in the partial area for each partial area,
For each of the plurality of partial regions, the highest sharpness among the sharpnesses of the partial regions of the plurality of images is the reference sharpness of the partial region,
For each of the plurality of images, an evaluation value that is higher as the difference between the reference sharpness for each partial region and the sharpness of the image is smaller is calculated. An image with good image quality is selected from, and identification information representing the selected image is output.
An image determination computer program for causing a computer to execute the above.

DESCRIPTION OF SYMBOLS 1 Imaging device 2 Camera module 3 Operation part 4 Display part 5 Memory | storage part 6, 61 Image determination apparatus 7 Control part 11 Division | segmentation part 12 Sharpness calculation part 13 Reference | standard sharpness determination part 14 Evaluation part 15 Positioning part 16 Exclusion area specific | specification part DESCRIPTION OF SYMBOLS 100 Computer 101 User interface part 102 Communication interface part 103 Storage part 104 Storage medium access apparatus 105 Processor 106 Storage medium

Claims (6)

A dividing unit that divides each of a plurality of images of the same subject into a plurality of partial areas;
For each of the plurality of images, a sharpness calculation unit that calculates the sharpness of the image of the part of the subject that is reflected in the partial area for each partial area;
For each of the plurality of partial regions, a reference sharpness determination unit that sets the highest sharpness among the sharpnesses of the partial regions of the plurality of images as the reference sharpness of the partial region;
For each of the plurality of images, an evaluation value that is higher as the difference between the reference sharpness for each partial region and the sharpness of the image is smaller is calculated. An evaluation unit that selects an image with good image quality from the image and outputs identification information representing the selected image;
An image determining apparatus. Of the plurality of partial areas, a partial area whose reference sharpness is less than an exclusion reference value indicating that the part of the subject is unclear is extracted, and the extracted partial area is more than the exclusion reference value. If it is surrounded by a partial region having a reference sharpness higher than a high predetermined reference value, further comprising an excluded region specifying unit for setting the extracted partial region as an excluded region;
For each of the plurality of images, the evaluation unit calculates the difference between the reference sharpness and the sharpness of the image for a partial region other than the excluded region of the plurality of partial regions. The image determination apparatus according to claim 1, wherein:   The evaluation unit calculates a deduction value having a smaller value as the difference between the reference sharpness and the sharpness is larger for each partial region, and the sum of the deduction values of each partial region is used as the evaluation value. The image determination apparatus according to claim 1 or 2.   The evaluation unit calculates a deduction value having a smaller value as the difference between the reference sharpness and the sharpness is larger for each partial region, and the sum of the deduction point value and the sharpness for each partial region is calculated. The image determination apparatus according to claim 1, wherein a total is the evaluation value. Divide each of multiple images of the same subject into multiple partial areas,
For each of the plurality of images, obtain the sharpness of the image of the part of the subject that is reflected in the partial area for each partial area,
For each of the plurality of partial regions, the highest sharpness among the sharpnesses of the partial regions of the plurality of images is the reference sharpness of the partial region,
For each of the plurality of images, an evaluation value that is higher as the difference between the reference sharpness for each partial region and the sharpness of the image is smaller is calculated. An image with good image quality is selected from, and identification information representing the selected image is output.
An image determination method including the above. Divide each of multiple images of the same subject into multiple partial areas,
For each of the plurality of images, obtain the sharpness of the image of the part of the subject that is reflected in the partial area for each partial area,
For each of the plurality of partial regions, the highest sharpness among the sharpnesses of the partial regions of the plurality of images is the reference sharpness of the partial region,
For each of the plurality of images, an evaluation value that is higher as the difference between the reference sharpness for each partial region and the sharpness of the image is smaller is calculated. An image with good image quality is selected from, and identification information representing the selected image is output.
An image determination computer program for causing a computer to execute the above.

Images