JP2015184760A - Electronic device and similarity determination program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To improve determination accuracy of similarity between images taking even shooting date and time into account.SOLUTION: A similarity determination unit 18 of a digital camera comprises: an image acquisition unit 20 that acquires a plurality of images along with information on each shooting date and time of the images; a characteristic-amount calculation unit 21 that calculates each amount of characteristic of the plurality of images acquired by the image acquisition unit 20; an evaluation value calculation unit 22 that calculates an evaluation value expressing similarity between the images on the basis of each amount of characteristic of at least two images of similarity determination object calculated by the characteristic-amount calculation unit 21; a shooting interval calculation units 23 that calculates a shooting interval between each shooting date and time of at least two images of similarity determination object; and a determination unit 24 that determines similarity between at least two images of similarity determination object using a similarity determination characteristic, which is defined by the shooting interval and an evaluation threshold and causes the images to be hardly determined similar as the shooting interval gets longer.

Description

  The present invention relates to an electronic device and a similarity determination program executed on a computer included in the electronic device.

  An image classification method for classifying images that are similar to each other into a predetermined number of groups is known (for example, Patent Document 1). In Patent Document 1, a plurality of images are classified into coarse groups using hierarchical clustering based on imaging date and time, and then images of each coarse group are classified into final groups using hierarchical clustering based on image feature values. .

JP 2009-134411 A

  In the method described in Patent Document 1, since the images are not classified considering both the imaging date and time and the image feature amount at the same time, the determination accuracy of whether the images are similar may be low. For example, even if an image feature amount is an image that should be classified into the same final group soon, if it is classified into a different coarse group according to the classification based on the shooting date and time, it will be classified into a different final group.

An electronic apparatus according to an aspect of the present invention includes an image acquisition unit that acquires a plurality of images together with information on each shooting date and time, a feature amount calculation unit that calculates the feature amounts of the plurality of images acquired by the image acquisition unit, and An evaluation value calculation unit that calculates an evaluation value that represents the degree of similarity between images based on the respective feature amounts of at least two similarity determination target images calculated by the feature amount calculation unit, and at least two similarity determination target images. Similarity that is defined by the shooting interval calculation unit that calculates the shooting interval between each shooting date and time, and the shooting interval and the evaluation threshold, and that the longer the shooting interval, the harder it is determined that the images are similar A similarity determination unit that determines the similarity between at least two similarity determination target images using the determination characteristics.
The similarity determination program according to another aspect of the present invention calculates an image acquisition procedure for acquiring a plurality of images together with information on each shooting date and a feature amount of the plurality of images acquired by the image acquisition procedure. A feature amount calculating procedure, an evaluation value calculating procedure for calculating an evaluation value representing similarity between images based on the respective feature amounts of at least two similarity determination target images calculated by the feature amount calculating procedure, and at least two similarities It is a characteristic defined by the shooting interval calculation procedure for calculating the shooting interval between each shooting date and time of the sex determination target image, and the shooting interval and the evaluation threshold, and it is determined that the images are more similar as the shooting interval is longer A similarity determination procedure for determining similarity between at least two similarity determination target images is performed by the computer using the similarity determination characteristic that is less likely to be performed.

  According to the present invention, it is possible to determine the similarity / dissimilarity of images in consideration of the shooting date and time, and the accuracy of classification is improved.

It is a block diagram which shows the structural example of the electronic device by the 1st to 4th embodiment of this invention. It is a functional block diagram regarding a similarity determination part. It is a figure used for description of the similarity determination part in the electronic device by the 1st Embodiment of this invention. It is a flowchart regarding the similarity determination part in the electronic device by the 1st Embodiment of this invention. It is a flowchart regarding the image classification | category part in the electronic device by the 1st-4th embodiment of this invention. It is a figure used for description of an image classification part. It is a figure used for description of the similarity determination part in the electronic device by the 2nd Embodiment of this invention. It is a flowchart regarding the similarity determination part in the electronic device by the 2nd Embodiment of this invention. It is a figure used for description of the similarity determination part in the electronic device by the 3rd Embodiment of this invention. It is a figure used for description of the similarity determination part in the electronic device by the 4th Embodiment of this invention. It is a flowchart regarding the similarity determination part in the electronic device by the 4th Embodiment of this invention. It is a figure used for description of the modification of the 4th Embodiment of this invention. It is a figure used for description of the modification of an image classification part. It is a figure which shows the provision form of the computer program product for the electronic device by this invention to perform the classification process of a similar image.

(First embodiment)
FIG. 1 is a block diagram showing a configuration example of an electronic device according to the first embodiment of the present invention. FIG. 1 illustrates the configuration of the digital camera 1, which includes an imaging unit 10, an image processing unit 11, a CPU 12, a storage unit 13, and a display unit 14. The image processing unit 11, the CPU 12, the storage unit 13, and the display unit 14 are connected via a bus 15.

The imaging unit 10 includes a photographing optical system, an imaging element, and the like, and captures a subject image. Then, an image signal obtained by imaging is output to the image processing unit 11.
The image processing unit 11 records the image signal output from the imaging unit 10 in the storage unit 13 via the bus 15 and performs image processing such as white balance adjustment and image compression processing to generate image data. The generated image data is recorded in a memory unit (not shown) or the like that is detachably attached to the storage unit 13 and the digital camera 1. The image data recorded in the storage unit 13 or the like is associated with information related to the imaging date and time.

  The CPU 12 controls the entire digital camera 1. The CPU 12 functions as the image classification unit 17 by executing the image classification program 16 stored in the storage unit 13. The image classification unit 17 classifies the plurality of input image data into a plurality of groups, and outputs information related to the image classification result.

  The storage unit 13 includes an SDRAM and a flash memory. The SDRAM is a volatile memory, and is used as a work memory for the CPU 12 to develop a program when the program is executed, and as a buffer memory for temporarily recording data. The flash memory is a non-volatile memory in which data of programs such as the image classification program 16 executed by the CPU 12 and various parameters read during program execution are recorded.

  The display unit 14 is, for example, a liquid crystal monitor provided on the back surface of the digital camera 1, and displays a setting screen of the digital camera 1.

  The image classification unit 17 includes a similarity determination unit 18 that determines whether two input image data are similar, and uses hierarchical clustering based on the determination result of the similarity determination unit 18. A plurality of image data is grouped without any problem. In the digital camera 1, for example, as shown in Patent Document 1, image data is grouped by the image classification unit 17 when a slide show display of image data is performed.

  FIG. 2 is a functional block diagram of the similarity determination unit 18. As shown in FIG. 2, the similarity determination unit 18 includes an image acquisition unit 20, a feature amount calculation unit 21, an evaluation value calculation unit 22, a shooting interval calculation unit 23, a determination unit 24, and a threshold setting unit 25. Have The similarity determination unit 18 shown in FIG. 2 receives the image A and the image B, and outputs a determination result as to whether or not the image A and the image B are similar. In the following description, the image A and the image B may be referred to as an input image A and an input image B.

  The image acquisition unit 20 acquires the image data of the input image A and the input image B together with information regarding the imaging date and time associated with the image data. The image acquisition unit 20 outputs the acquired image data to the feature amount calculation unit 21 and outputs information related to the imaging date and time to the imaging interval calculation unit 23.

The feature amount calculation unit 21 calculates the image feature amount of each of the input image A and the input image B. The image feature amount includes, for example, various image statistics (color, texture, shape, etc.) defined in ISO / IEC15938-3 (mpeg visual description). The feature quantity calculation unit 21 calculates various feature quantities using, for example, a known edge detection process, contour extraction process, and the like, and generates feature quantity vectors V _A and V _B for the input image A and the input image B, respectively. To do.

The evaluation value calculation unit 22 calculates an evaluation value E _AB representing the similarity between the feature amount vectors V _A and V _B generated by the feature amount calculation unit 21. For example, the evaluation value calculation unit 22 calculates the cosine similarity of the feature amount vectors V _A and V _B as the evaluation value E _AB .

The shooting interval calculation unit 23 uses (Equation 1) for the interval between the shooting date and time t _A associated with the input image _A and the shooting date and time t _B associated with the input image B, that is, the shooting interval T _AB. To calculate.
T _AB = | t _A −t _B | (Formula 1)

  The determination unit 24 determines whether the input image A and the input image B are similar using the similarity determination characteristic diagram defined by the shooting interval and the evaluation threshold as shown in FIG.

The similarity determination characteristic diagram shown in FIG. 3 is a coordinate system in which the vertical axis represents the evaluation value E _AB and the horizontal axis represents the imaging interval T _AB . The coordinate region of the similarity determination characteristic diagram includes a similarity region that outputs a determination result that the input image A and the input image B are similar, and the input image A and the input image B by the upwardly rising boundary line 40. The region is divided into dissimilar regions that output a determination result of dissimilarity.

The boundary line 40 is a linear function that expresses a similarity determination threshold value E _th that has a positive slope and uses the shooting interval T _AB as an argument. The threshold value E _th indicated by the boundary line 40 increases as the imaging interval T _AB increases. When the shooting interval T _AB is short, for example, when the same subject is shot a plurality of times in a short time by changing the setting of the digital camera 1 or when continuous shooting is performed, the threshold E _th is set to be small. It is easy to determine that the image A and the input image B are similar. On the other hand, when the shooting interval T _AB is long, there is a high possibility that a different subject is shot or shot at a different place, so the threshold E _th is set large, and the input image A and the input image B are It is difficult to determine that they are similar.
That is, the similarity determination characteristic defined by the boundary line 40 is a characteristic that it is difficult to determine that the images are similar as the shooting interval is longer.

When the shooting interval T _AB calculated by the shooting interval calculation unit 23 is input, the threshold setting unit 25 outputs a threshold E _th corresponding to the shooting interval T _AB to the determination unit 24 based on a function representing the boundary line 40. To do. The determination unit 24 determines whether or not the evaluation value E _AB calculated by the evaluation value calculation unit 22 is greater than or equal to the threshold value E _th set by the threshold value setting unit 25. When the evaluation value E _AB is equal to or greater than the threshold value E _th , that is, within the similar region, the determination unit 24 determines that the input image A and the input image B are similar. When the image evaluation value E _AB is less than the threshold value E _th , that is, within the dissimilar region, the determination unit 24 determines that the input image A and the input image B are dissimilar.
In other words, the upper and lower two regions where the coordinate position of the intersection between the evaluation value E _AB calculated by the evaluation value calculation unit 22 and the shooting interval T _AB calculated by the shooting interval calculation unit 23 is partitioned by the boundary line 40 in FIG. Similarity / dissimilarity between images is determined depending on where the image exists.

In the similarity determination unit 18, the threshold setting unit 25 sets the threshold E _th based on the shooting interval T _AB , and the determination unit 24 compares the evaluation value E _AB representing the similarity of the image feature amount with the threshold E _th. Thus, it is possible to determine whether or not the images are similar in consideration of both the image feature amount and the imaging date and time. Therefore, it is possible to determine whether the images are similar in consideration of the shooting date and time, and the accuracy of classification is improved.

  FIG. 4 is a flowchart regarding the similarity determination unit 18. The flowchart shown in FIG. 4 is executed by the CPU 12 executing the image classification program 16, and starts executing when two images are input as the input image A and the input image B.

In step S300, the CPU 12 acquires two images input using the image acquisition unit 20, that is, the imaging dates and times t _A and t _B stored in association with the input image A and the input image B, respectively. The interval calculation unit 23 is used to calculate the shooting interval T _AB .

In step S <b> 301, the CPU 12 uses the feature amount calculation unit 21 for the two images that the image acquisition unit 20 has acquired image data, that is, the input image A and the input image B, respectively, and uses the feature amount vectors V _A and V _B. Is calculated.

In step S302, the CPU 12 uses the evaluation value calculation unit 22 to evaluate the evaluation value E _{AB indicating} the similarity between the input image A and the input image B based on the feature amount vectors V _A and V _B calculated in step S301. Is calculated.

In step S303, the CPU 12 uses the threshold setting unit 25 to set the threshold E _th based on the shooting interval T _AB calculated in step S300.

In step S304, the CPU 12 uses the determination unit 24 to determine whether or not the evaluation value E _AB calculated in step S302 is greater than or equal to the threshold value E _th calculated in step S303. If the evaluation value E _AB is greater than or equal to the threshold value E _th , the CPU 12 makes an affirmative decision in step S304 and proceeds to the process of step S305. If the evaluation value E _AB is less than the threshold value E _th , the CPU 12 makes a negative decision in step S304. The process proceeds to step S306.

In step S305, the CPU 12 outputs a determination result that the input image A and the input image B are similar, and ends the process of FIG. For example, the CPU 12 returns TRUE as the output result of the similarity determination unit 18.
In step S306, the CPU 12 outputs a determination result that the input image A and the input image B are dissimilar, and ends the process of FIG. For example, the CPU 12 returns FALSE as an output result of the similarity determination unit 18.

  An example of the operation of the image classification unit 17 will be described with reference to FIGS. FIG. 5 is an example of a flowchart relating to the image classification unit 17.

  FIG. 6 shows a specific example of image data classified by the image classification unit 17. FIG. 6 illustrates four pieces of image data IMG [1], IMG [2], IMG [3], and IMG [4] that are different in imaging date and time.

  In step S500 of FIG. 5, the CPU 12 converts the input unclassified n pieces of image data IMG [1], IMG [2],..., IMG [n] as illustrated in FIG. Sort based on the imaging date and time associated with.

  In step S501 in FIG. 5, the CPU 12 initializes the number of repetitions i of steps S503 to S507 described below to 1.

  In step S502, the CPU 12 generates a group including the first image data. In other words, the CPU 12 classifies the first image data into the first group and the first group. For example, IMG [1] in FIG. 6 is classified into the first group. The CPU 12 associates information indicating the first group with the first image data.

  In step S503, the CPU 12 uses the similarity determination unit 18 to perform similarity determination for the i-th image data and the i + 1-th image data among the n pieces of image data sorted in step S500. Since i = 1 is set at the timing immediately after the completion of the processing in step S502, it is determined whether or not the first image, for example, IMG [1], is similar to the second image, for example, IMG [2]. To do.

  In step S504, the CPU 12 determines whether or not it is determined in the similarity determination in step S503 that the i-th image data and the i + 1-th image data are similar. If it is determined in step S503 that the images are similar, the CPU 12 proceeds to step S505. If it is determined that they are not similar, the CPU 12 proceeds to step S506. For example, if it is determined that IMG [1] and IMG [2] are similar, the process proceeds to step S505. If it is determined that IMG [1] and IMG [2] are dissimilar, step S505 is performed. The process proceeds to S506.

In step S505, the CPU 12 includes the (i + 1) th image data in the group including the i-th image data. For example, IMG [2] is included in the first group including IMG [1].
The CPU 12 associates information indicating that the group is a classified group with the (i + 1) th image data.

In step S506, the CPU 12 generates a group including the (i + 1) th image data. In other words, the CPU 12 classifies the i + 1-th image data into a different group from the i-th image data. For example, IMG [2] is classified into a new group and a second group.
The CPU 12 associates information indicating that the group is a classified group with the (i + 1) th image data.

In step S507, the CPU 12 increases i by one.
In step S508, the CPU 12 determines whether i is larger than n-1. If i is n−1 or less, the CPU 12 returns to step S503, and if i is greater than n−1, the process proceeds to step S509.

  CPU12 repeats the process from step S503 to step S507 n-1 times. That is, the processes in steps S501, S507, and S508 are repeated until it is determined whether or not the (n-1) th image and the nth image are similar.

  In step S509, the CPU 12 associates the classification result with each image data, and ends the process of FIG. For example, when the image data IMG [1] is classified into the first group and IMG [2] to IMG [4] are classified into the second group, the image data IMG [1] represents the first group as additional information. The information is associated, and the image data IMG [2] to IMG [4] are associated with information representing the second group as additional information.

  In the example of FIG. 6, in the second step S503, similarity determination is performed for IMG [2] and IMG [3]. When it is determined that IMG [2] and IMG [3] are similar, IMG [3] is included in the group including IMG [2], and IMG [2] and IMG [3] are determined to be dissimilar If so, create a new group containing IMG [3].

  For example, if IMG [1] and IMG [2] are determined to be similar in the first step S503, and IMG [2] and IMG [3] are determined to be similar in the second step S503, IMG [ 2] is included in the same first group as IMG [1], so IMG [3] is also included in the same first group as IMG [1] and IMG [2].

  When it is determined that IMG [1] and IMG [2] are dissimilar in the first step S503, and IMG [2] and IMG [3] are determined to be similar in the second step S503, IMG [1 ] Is included in the first group, IMG [2] is included in the second group, and IMG [3] is included in the same second group as IMG [2].

  When it is determined that IMG [1] and IMG [2] are similar in the first step S503, and IMG [2] and IMG [3] are determined to be dissimilar in the second step S503, IMG [2 ] Is included in the same first group as IMG [1], and IMG [3] is included in a new group, the second group, which is different from IMG [1] and IMG [2]. If it is determined in the third step S503 that IMG [3] and IMG [4] are similar, IMG [4] is included in the same second group as IMG [3].

  If it is determined that IMG [1] and IMG [2] are dissimilar in the first step S503, and IMG [2] and IMG [3] are determined to be dissimilar in the second step S503, IMG [ 1] is included in the first group, IMG [2] is included in the second group, and IMG [3] is included in a third group that is different from both IMG [1] and IMG [2].

According to the first embodiment described above, the following operational effects are obtained.
(1) The digital camera 1 that is an electronic apparatus according to the first embodiment of the present invention includes the following functional units that are executed by the CPU 12 executing the image classification program 16. That is, the electronic device includes an image acquisition unit 20 that acquires a plurality of images together with information on each shooting date, a feature amount calculation unit 21 that calculates a feature amount of each of the plurality of images acquired by the image acquisition unit 20, and a feature. An evaluation value calculation unit that calculates an evaluation value E _AB that represents the degree of similarity between images based on the feature amounts of at least two similarity determination target images calculated by the amount calculation unit, and at least two similarity determination target images. The characteristics defined by the shooting interval calculator 23 for calculating the shooting interval T _AB between the respective shooting dates and times, the shooting interval T _AB and the evaluation threshold value E _AB, and the longer the shooting interval, the more similar the images. Using similarity determination characteristics (in other words, similarity determination characteristics in which the similarity determination of an image becomes more severe as the shooting interval is longer). And a similarity determination unit 24 that determines the similarity between the similarity determination target images.
Therefore, the digital camera 1 that is the electronic apparatus according to the first embodiment of the present invention determines similarity / dissimilarity between two images in consideration of the shooting interval of two images that are similarity determination targets. It is possible to improve the accuracy of determination of similarity / dissimilarity for a plurality of images within a longer period than that of a conventional electronic device.

(2) The digital camera 1 can also be used as the image classifying unit 17 that classifies a plurality of images based on a plurality of results of similarity determination repeatedly performed between different images by the CPU 12 executing the image classification program 16. Function. The image classification unit 17 determines whether or not the pair of all two adjacent images sorted based on the shooting date and time among the plurality of images is similar by the similarity determination unit 18, and the determination results thereof Based on the above, the plurality of images are classified into one or more groups. Therefore, the digital camera 1 can classify images at a high speed because the number of times of determining whether or not two images are similar to each other is smaller than a method using hierarchical clustering.

(Second Embodiment)
An electronic device according to the second embodiment of the present invention will be described. The electronic device according to the second embodiment is the same as the first embodiment in that it is a digital camera 1 having the configuration shown in FIG. In the second embodiment, among the functional blocks of the similarity determination unit 18, the process of the determination unit 24 is different from that of the first embodiment. In the second embodiment, the determination unit 24 determines whether or not the input image A and the input image B are similar using a similarity determination characteristic diagram as illustrated in FIG.

In the similarity determination characteristic diagram shown in FIG. 7, when the shooting interval T _AB is equal to or shorter than the predetermined time T1, it is determined that the input image A and the input image B are similar regardless of the evaluation value, and the shooting interval is predetermined. When the time is T2 or longer, it is determined that the input image A and the input image B are dissimilar regardless of the evaluation value. If the shooting interval T _AB is greater than T1 and less than the predetermined time T2, it is determined whether or not they are similar due to the magnitude relationship between the evaluation value E _AB and the threshold value E _th as in the first embodiment. Similar to the boundary line 40 in the first embodiment, the boundary line 60 at T1 <T _AB <T2 has a positive slope, and represents the similarity determination threshold value E _th using the shooting interval T _AB as an argument. It is a function.

  The predetermined time T1 is a shooting interval between image data generated by continuous shooting, for example, and is, for example, several seconds to several tens of seconds. The predetermined time T2 is, for example, several tens of seconds to several hundreds of seconds.

  FIG. 8 is a flowchart relating to the similarity determination unit 18 in the second embodiment. Of the operations shown in FIG. 8, the same operations as those in FIG. 4 are denoted by the same step numbers as those in FIG. The flowchart shown in FIG. 8 differs from FIG. 4 in that steps S700 and S701 are inserted between steps S300 and S301.

In step S700, the CPU 12 determines whether or not the shooting interval T _AB calculated in step S300 is equal to or longer than a predetermined time T1. When the shooting interval T _AB is equal to or shorter than the predetermined time T1, the CPU 12 proceeds to the process of step S305. When the shooting interval T _AB is longer than the predetermined time T1, the CPU 12 proceeds to the process of step S701.

In step S701, the CPU 12 determines whether or not the shooting interval T _AB calculated in step S300 is equal to or longer than a predetermined time T2. When the shooting interval T _AB is equal to or longer than the predetermined time T2, the CPU 12 proceeds to the process of step S306. When the shooting interval T _AB is less than the predetermined time T2, the CPU 12 proceeds to the process of step S301.

As described above, in the second embodiment, in the operation of the similarity determination unit 18, the processing from step S301 to step S304 is performed when the shooting interval _TAB is equal to or shorter than the predetermined time T1 and when it is equal to or longer than the predetermined time T2. By omitting, processing time can be shortened.

According to the second embodiment described above, the following operational effects can be obtained.
(1) As shown in FIG. 7, the determination unit 24 of the digital camera 1 which is an electronic apparatus according to the second embodiment of the present invention has a first shooting interval regardless of the magnitude of the calculated evaluation value. The similarity determination characteristic diagram is determined so that images are similar when the value is equal to or less than the predetermined value, and images are determined to be dissimilar when the value is equal to or greater than the second predetermined value (> first predetermined value). have. Specifically, the determination unit 24 determines that the first image and the second image, for example, the input image A and the input image B, when the shooting interval T _AB calculated by the shooting interval calculation unit 23 is equal to or shorter than a predetermined time T1. It is determined that they are similar, and when the shooting interval T _AB is equal to or longer than the predetermined time T2, it is determined that the first image and the second image are dissimilar. Therefore, in the digital camera 1 according to the second embodiment of the present invention, the processing time for determining and classifying whether or not the first image and the second image are similar in consideration of the shooting date and time is also included. Can be made shorter than in the first embodiment.

(Third embodiment)
An electronic apparatus according to the third embodiment of the present invention will be described. The electronic device according to the third embodiment is the same as the first embodiment in that it is a digital camera 1 having the configuration shown in FIG. In the third embodiment, the threshold setting unit 25 sets two thresholds, the first threshold E ₁ and the second threshold E ₂ based on the imaging interval T _AB , and the determination unit 24 sets the first threshold. E ₁ and the point where the input image a using a second threshold value E ₂ and the input image B to determine whether similar differs from the first embodiment.

FIG. 9 shows a similarity determination diagram in the third embodiment. This similarity determination diagrams are shown and the second boundary line 81 representing the first boundary line 80 and the second threshold value E ₂ representing the first threshold value E ₁ of the threshold setting unit 25 sets the. In the similarity determination characteristic diagram shown in FIG. 9, whether or not the similar region, the dissimilar region, the input image A, and the input image B are similar by the first boundary line 80 and the second boundary line 81 is determined. It is divided into a gray area for which the determination is suspended.

Determining unit 24, when the evaluation value E _AB is the first threshold value E ₁ or more, it is determined that the input image A and the input image B are similar, if the evaluation value E _AB is the second threshold value E ₂ or less, When it is determined that the input image A and the input image B are dissimilar and the evaluation value E _AB is greater than the second threshold E _{2 and} less than the first threshold E _1, whether or not the input image A and the input image B are similar This decision is suspended.

According to the third embodiment described above, the following operational effects can be obtained.
(1) The determination unit 24 of the digital camera 1 which is an electronic apparatus according to the third embodiment of the present invention, as shown in FIG. 9, the first characteristic defined by the evaluation threshold for the shooting interval, and the shooting interval The second characteristic defined by the evaluation threshold value for is provided. Specifically, the threshold setting unit 25 sets the first threshold E ₁ and the second threshold E ₂ based on the shooting interval T _AB , and the determination unit 24 sets the first threshold E ₁ and the second threshold E. ₂ is used to determine whether the input image A and the input image B are similar. That is, the similarity determination unit 24 calculates the evaluation value calculated based on the feature amounts of the first image and the second image when the evaluation value is equal to or higher than the evaluation threshold E ₁ calculated based on the boundary line 80 that is the first characteristic. It is determined that the first image and the second image are similar. The first image and the evaluation value calculated by the characteristic amount of the second image, when the evaluation threshold E ₂ less calculated based on the boundary line 81 is a second characteristic, the first and second images Are determined not to be similar. The first image and the evaluation value calculated by the characteristic amount of the second image is less than rated threshold E _1, and when the calculated larger evaluation threshold E ₂ based on the second characteristic, similar / dissimilar The judgment result is not output.
Therefore, the digital camera 1 which is the electronic apparatus according to the third embodiment of the present invention has an image with no similarity that is classified between the two threshold values determined by the boundary lines 80 and 81. By suspending the similarity determination for, it is possible to further improve the determination accuracy of whether or not the image is similar to the conventional electronic device.

  For example, three images D1, D2, and D3 obtained by continuous shooting will be described as follows. The evaluation value E12 indicating the similarity between the images D1 and D2 by continuous shooting and the evaluation value E23 indicating the similarity between the images D2 and D3 by continuous shooting often show substantially the same value. . However, when the similarity / dissimilarity by one boundary line 40 is determined as in the first embodiment, there is a possibility that the images D1 and D2 are determined to be similar and the images D2 and D3 are determined to be dissimilar. The third embodiment can eliminate the absurdity that a plurality of images obtained by continuous shooting are thus classified as similar and dissimilar.

(Fourth embodiment)
An electronic apparatus according to the fourth embodiment of the present invention will be described. The electronic device according to the fourth embodiment is the same as the first embodiment in that it is a digital camera 1 having the configuration shown in FIG. In the fourth embodiment, as in the second embodiment, the determination unit 24 is similar to the input image A and the input image B regardless of the evaluation value when the shooting interval T _AB is equal to or shorter than the predetermined time T1. If the shooting interval is equal to or longer than the predetermined time T2, it is determined that the input image A and the input image B are dissimilar regardless of the evaluation value. The determination unit 24, when the object distance is greater than the predetermined time period T2 than the predetermined time T1, as in the third embodiment, when the evaluation value E _AB is E ₁ or more first threshold input image A an input image B is determined to be similar to the evaluation value E _AB is when the second threshold value E ₂ or less is determined that the input image a and the input image B to be dissimilar, the evaluation value E _AB is the second threshold value E ₂ when greater than the first less than the threshold value E ₁ is the input image B and the input image a to hold determines whether similar.

An example of the similarity determination characteristic diagram in the fourth embodiment is shown in FIG. In FIG. 10, the region where the shooting interval T _AB is equal to or less than the predetermined time T1 and the region where the first threshold line 90 or more related to the first threshold is a similar region, and the region where the shooting interval T _AB is equal to or longer than the predetermined time T2; The area below the second boundary line 91 related to the second threshold is a dissimilar area, and the area sandwiched between the first boundary line 90 and the second boundary line 91 is a gray area.

  FIG. 11 is a flowchart regarding the similarity determination unit 18 that performs similarity determination based on the similarity determination characteristic diagram of FIG. Among the operations shown in FIG. 11, the same operations as those in FIG. 4 and FIG. 8 are denoted by the same step numbers as those in FIG. 4 and FIG. The flowchart shown in FIG. 11 is different from FIG. 8 in that step S303 is replaced with step S1000, and step S304 is replaced with step S1001 and step S1002.

In step S1000, CPU 12 uses the threshold setting unit 25, based on the photographing interval _{T AB} calculated in step S300, the set first threshold value _{E 1} and the second threshold value _{E 2.}

In step S1001, CPU 12 determines using the determination unit 24, evaluation value _{E AB} calculated in step S302 as to whether or not the first threshold value _{E 1} or more, which is calculated in step S1000. CPU12, when the evaluation value _{E AB} is the first threshold value _{E 1} or more, an affirmative decision in step S1001, the process proceeds to the step S305, when the evaluation value _{E AB} is first less than the threshold value _{E 1,} a negative step S1001 Determination is made and the process proceeds to step S1002.

In step S1002, CPU 12 determines using the determination unit 24, evaluation value _{E AB} calculated in step S302 is whether the second threshold value _{E 2} or less calculated in step S1000. CPU12, when the evaluation value _{E AB} is the second threshold value _{E 2} or less, an affirmative decision in step S1002, the process proceeds to the step S306, when the evaluation value _{E AB} is the second larger threshold value _{E 2,} the step S1002 A negative determination is made, and the processing of FIG. 11 ends.

According to the fourth embodiment described above, the following operational effects can be obtained.
(1) In the digital camera 1 which is an electronic device according to the fourth embodiment of the present invention, when the shooting interval T _AB calculated by the shooting interval calculation unit 23 is equal to or shorter than the predetermined time T1, the first image and the second image It is determined that the image, for example, the input image A and the input image B are similar, and the first image and the second image are determined to be dissimilar when the shooting interval T _AB is equal to or longer than the predetermined time T2. Therefore, the digital camera 1 according to the fourth embodiment of the present invention achieves the same operational effects as the third embodiment and determines whether the input image A and the input image B are similar to each other. The determination can be made faster than in the embodiment.

The following modifications are also within the scope of the present invention, and one or a plurality of modifications can be combined with the above-described embodiment.
(Modification 1)
The first boundary line 90 and the second boundary line 91 shown in FIG. 10 may not be straight lines. For example, it may be a curved line such as a first boundary line 90a and a second boundary line 91a illustrated in FIG. 12, or may be a line represented by an arbitrary function. When the shooting interval T _AB is at or near the predetermined time T1, the probability that the input image A and the input image B are similar is higher than when the shooting interval T _AB is at or near the predetermined time T2. The inclination of the first boundary line 90a with respect to the shooting interval axis in the vicinity of the predetermined time T1 is made gentler (flat) than that in the vicinity of the predetermined time T2, so that it is easy to determine that they are similar. When the shooting interval T _AB is at or near the predetermined time T2, there is a higher probability that the input image A and the input image B are dissimilar than when the shooting interval T _AB is at or near the predetermined time T1. The inclination of the second boundary line 90b in the vicinity of the predetermined time T2 with respect to the shooting interval axis is made gentler (flat) than that in the vicinity of the predetermined time T1, thereby making it easier to determine that the dissimilarity is not obtained.
Note that the boundary line 40 shown in FIG. 3, the boundary line 60 shown in FIG. 7, and the first boundary line 90 and the second boundary line 91 shown in FIG. 9 are similarly expressed by arbitrary functions. May be.
Further, the similarity determination diagram may be defined in the form of a lookup table without using a function.

(Modification 2)
In the above embodiment, the threshold setting unit 25 sets the threshold Eth based on the shooting interval TAB. However, the threshold setting unit 25 may set the threshold Eth using another method. For example, the user may manually input the positions on the similarity determination characteristic diagram at both ends of the boundary line 40 shown in FIG. 3 and the boundary line 60 shown in FIG.

(Modification 3)
And record the shooting date and time each time the image data is generated in the storage unit 13, calculates the average value of the interval of imaging last 10 minutes, the first threshold value E ₁ and the second threshold value E on the basis of the average photographing interval ₂ may be set. For example, as in the after continuous shooting is performed, if the average photographing interval is shorter than the predetermined time T1, it is determined that more likely to perform continuous shooting first threshold value E ₁ value You may decide to perform correction which lowers. The average photographing interval of the predetermined time, if longer than 1 day for example, may be possible to correct to increase the second threshold value E _2.

(Modification 4)
The threshold value _Eth may be set as follows. It is assumed that user information including user identification information, information on settings of the digital camera 1 specified by the user, and a threshold value _Eth specified by the user is stored in the storage unit 13 in advance in the electronic device. Here, the information regarding the setting of the digital camera 1 includes an AF mode, a scene mode, lens information, an exposure time, and the like. The CPU 12 acquires information related to the settings of the digital camera 1 at the time of image data generation and records the information in the storage unit 13 as history information. When determining whether or not the image data is similar, the CPU 12 searches for the most frequent setting of the digital camera 1 in the history information from the shooting date and time of the image data to a predetermined time before. Then, the user information database is searched based on the information relating to the setting of the most frequent digital camera 1, and the threshold value _Eth is set to the value of the threshold value _Eth included in the searched user information. By doing in this way, the similarity determination adapted to the user's individuality can be performed.

(Modification 5)
The method of classifying images into groups by the image classification unit is not limited to the method shown in FIG. For example, classification into groups may be performed according to the procedure shown in FIG. One image data is selected from n image data to be classified, and it is determined whether or not the selected image data is similar to other image data, and the image determined to be similar Classify the data into the same group as the selected image data. For example, image data IMG [1] is selected, and it is determined whether IMG [2], IMG [3], and IMG [4] are similar to IMG [1]. As a result of the determination, it is determined that IMG [1] and IMG [2] are similar, IMG [1] and IMG [3] are dissimilar, and IMG [1] and IMG [4] are similar. If so, the CPU 12 classifies IMG [2] and IMG [4] into the same group as IMG [1]. Then, the CPU 12 generates a group of only IMG [3] and ends the image classification. By doing in this way, similarity determination can be performed even for two image data that are separated in time, and the accuracy of grouping similar images can be improved.

(Modification 6)
In the above-described embodiment, the evaluation value E _AB is described as being higher as the image feature amount is similar. However, a feature amount calculation method is used such that the evaluation value E _AB is lower as the image feature amount is similar. May be. For example, a value obtained by subtracting the cosine similarity of the feature amount vectors V _A and V _B from the maximum value of the cosine similarity may be calculated as the evaluation value E _AB . In the first embodiment, when a feature amount that decreases as the image feature amount is similar is used, the threshold value _Eth is changed so that the boundary line 40 in FIG. It is preferable to do.

(Modification 7)
In the above description, the case where the present invention is applied to the digital camera 1 has been described. However, the present invention is not limited to an electronic device including a computer that can execute the image classification program 16. It can also be applied to smartphones, personal computers, and the like.

(Modification 8)
In the above embodiment, the example in which the image classification program 16 is stored in the storage unit 13 in advance has been described. These programs can be provided to the computer through various recording media such as CD-ROM, DVD, USB memory, and data signals such as the Internet. FIG. 14 shows the state. The computer 1100 is provided with a program via, for example, a CD-ROM 1101. The computer 1100 has a connection function with the communication line 1102. A server computer 1103 is a computer that provides the program, and stores the program in a recording medium such as a hard disk 1104. The communication line 1102 is a communication line such as the Internet or a dedicated communication line. The server computer 1103 reads the program using the hard disk 1104 and transmits the program to the computer 1100 via the communication line 1102. That is, the program is transmitted as a data signal on a carrier wave and transmitted via the communication line 1102. Thus, the program can be supplied as a computer-readable computer program product in various forms such as a recording medium and a data signal.

  Such a similarity determination program determines the similarity between images in consideration of the shooting interval. That is, it is calculated by an image acquisition procedure for acquiring a plurality of images together with information on each shooting date, a feature amount calculation procedure for calculating a feature amount of each of the plurality of images acquired by the image acquisition procedure, and a feature amount calculation procedure. An evaluation value calculation procedure for calculating an evaluation value representing the similarity between images based on the respective feature amounts of at least two similarity determination target images, and a shooting interval between the respective shooting dates and times of the at least two similarity determination target images Using the similarity determination characteristic that is defined by the shooting interval calculation procedure for calculating the image, and the shooting interval and the evaluation threshold value, and the longer the shooting interval, the harder it is determined that the images are similar. Based on a similarity determination procedure for determining similarity between two similarity determination target images and a plurality of results of similarity determination repeatedly performed between different images, To perform a classification procedure to classify the image number into the computer.

(Modification 9)
In the electronic device according to the above-described embodiment, the image classification unit 17 groups a plurality of image data based on the determination result of the similarity determination unit 18. However, even when processing other than grouping of a plurality of image data is performed based on the determination result of the similarity determination unit 18, the accuracy of determination by the similarity determination unit 18 is not changed. That is, the present invention can also be applied to a case where processing other than grouping of a plurality of image data is performed based on the determination result of the similarity determination unit 18. The same applies to the similarity determination program in Modification 8.

Although various embodiments and modifications have been described above, the present invention is not limited to these contents. Other embodiments conceivable within the scope of the technical idea of the present invention are also included in the scope of the present invention.
Therefore, instead of the similarity / dissimilarity determination between a pair of images, for example, an electronic device that performs a similarity / dissimilarity determination between a pair of image groups constituted by a plurality of images is also within the scope of the present invention.

1 Digital Camera 10 Imaging Unit 11 Image Processing Unit 12 CPU
DESCRIPTION OF SYMBOLS 13 Memory | storage part 16 Image classification program 17 Image classification part 18 Similarity determination part 20 Image acquisition part 21 Feature-value calculation part 22 Evaluation value calculation part 23 Shooting interval calculation part 24 Determination part 25 Threshold setting part

Claims (10)

An image acquisition unit for acquiring a plurality of images together with information of each shooting date and time;
A feature amount calculation unit for calculating feature amounts of a plurality of images acquired by the image acquisition unit;
An evaluation value calculation unit that calculates an evaluation value that represents the degree of similarity between images based on the respective feature amounts of the at least two similarity determination target images calculated by the feature amount calculation unit;
A shooting interval calculation unit that calculates a shooting interval between the respective shooting dates and times of the at least two similarity determination target images;
A characteristic defined by a shooting interval and an evaluation threshold, and using a similarity determination characteristic that makes it difficult to determine that images are more similar as the shooting interval is longer, between the at least two similarity determination target images A similarity determination unit for determining the similarity of
Electronic equipment comprising. The electronic device according to claim 1,
An electronic apparatus further comprising a classification unit that classifies a plurality of images based on a plurality of results of similarity determination repeatedly performed between different images by the similarity determination unit. The electronic device according to claim 1 or 2,
The similarity determination unit, based on the evaluation threshold calculated using the similarity determination characteristic for the calculated shooting interval and the evaluation value calculated by the feature amount of the at least two images, An electronic device that determines similarity between at least two similarity determination target images. The electronic device according to any one of claims 1 to 3,
The similarity determination characteristic is one characteristic defined as an evaluation threshold for an imaging interval,
The similarity determination unit is configured to determine the at least two similarity determinations when the evaluation value calculated based on the feature amount of the at least two similarity determination target images is equal to or greater than the evaluation threshold calculated based on the one characteristic. An electronic device that determines that the target images are similar. The electronic device according to any one of claims 1 to 3,
The similarity determination characteristic is a first characteristic defined by an evaluation threshold for an imaging interval, and a second characteristic defined by an evaluation threshold for an imaging interval,
The similarity determination unit
When the evaluation value calculated based on the feature amount of the at least two similarity determination target images is equal to or greater than the evaluation threshold calculated based on the first characteristic, the at least two similarity determination target images are similar to each other. It is determined that
When the evaluation value calculated based on the feature amount of the at least two similarity determination target images is equal to or less than the evaluation threshold calculated based on the second characteristic, the at least two similarity determination target images are similar. It is determined that
The evaluation value calculated from the feature amount of the at least two similarity determination target images is less than the evaluation threshold calculated based on the first characteristic, and the evaluation calculated based on the second characteristic An electronic device that does not output similar / dissimilar judgment results when the threshold value is exceeded. The electronic device according to any one of claims 1 to 5,
The similarity determination characteristic is an electronic apparatus in which the evaluation threshold is defined as a function of the photographing interval. The electronic device according to any one of claims 1 to 6,
Regardless of the magnitude of the calculated evaluation value, the similarity determination characteristic is that images are similar when the shooting interval is equal to or less than a first predetermined value, and the shooting interval is less than the first predetermined value. An electronic device having characteristics determined to determine that images are dissimilar if they are longer than a second predetermined value. The electronic device according to claim 7,
The first characteristic is a characteristic used for similarity determination, and when the shooting interval is at or near the second predetermined value, the images are more similar to each other. Is a characteristic that makes it easier to determine that
The second characteristic is a characteristic used for determining dissimilarity, and the image is captured when the photographing interval is at or near the second predetermined value than when the photographing interval is at or near the first predetermined value. An electronic device having characteristics that make it easier to determine that they are dissimilar to each other. An image acquisition procedure for acquiring a plurality of images together with information on each shooting date and time,
A feature amount calculation procedure for calculating feature amounts of a plurality of images acquired by the image acquisition procedure;
An evaluation value calculation procedure for calculating an evaluation value representing the degree of similarity between images based on the respective feature amounts of at least two similarity determination target images calculated by the feature amount calculation procedure;
A shooting interval calculation procedure for calculating a shooting interval between the respective shooting dates and times of the at least two similarity determination target images;
A characteristic defined by a shooting interval and an evaluation threshold, and using a similarity determination characteristic that makes it difficult to determine that images are more similar as the shooting interval is longer, between the at least two similarity determination target images A similarity determination program for causing a computer to execute a similarity determination procedure for determining the similarity of The similarity determination program according to claim 9,
A similarity determination program for causing a computer to further execute a classification procedure for classifying a plurality of images based on a plurality of results of similarity determination repeatedly performed between different images by the similarity determination procedure.

Images