JP2009217538A - Image evaluating device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image evaluating device capable of clustering images according to the roughness and fineness at a photographing time, thereby simultaneously displaying a set of images expressing the same event of close photographing time. <P>SOLUTION: The image evaluating device 100 is composed of: a database 103 in which the images and photographing time information on the images are stored in association; a clustering execution part 104 which reads the images and photographing time information from the database 103, executes clustering process of generating, according to the photographing time information, a cluster which is composed of close photographing times and to which images equal to or less than a threshold belong, and associates the cluster to the images to store in the database 103; and an image display part 107 which reads the images by each cluster from the database 103 and displays the images. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to an image evaluation apparatus.

Conventionally, a set of images is clustered based on the time when the images were created or the times when the images were taken, and the images close to the shooting times are grouped together. A method has been proposed in which “group” is regarded as a group belonging to “same event” and images of this group are displayed together (for example, see Patent Documents 1 to 3). Among these, for example, in the invention described in Patent Document 1, a process is performed in which a fixed threshold is given to the photographing time interval and grouping is performed for an image group equal to or less than the threshold. In the invention described in Patent Document 2, it is proposed to perform a clustering process by setting a threshold for the primary difference of the shooting interval, changing the threshold of the shooting interval according to the time density. Furthermore, in the invention described in Patent Document 3, after performing clustering at the shooting time, a cluster smaller than a predetermined number is merged with an adjacent cluster, and a large cluster is generated for an area where the shooting interval is sparse. It is proposed to perform the process.
JP 2007-193592 A JP 2007-094762 A JP 2007-199923 A

  However, in the conventional techniques described in Patent Documents 1 to 3 described above, in any case, it is assumed that a shooting threshold (or its primary difference) has a fixed threshold value, and changes depending on various situations. There was a problem that it was difficult to cope with the shooting interval.

  The present invention has been made in view of such a problem, and can cluster images according to the density of shooting times, and as a result, display a set of images representing the same event with close shooting times at a time. An object of the present invention is to provide an image evaluation apparatus capable of performing the above.

  In order to solve the above problems, an image evaluation apparatus according to the present invention includes a storage unit (for example, the database 103 in the embodiment) in which an image and shooting time information of the image are associated and stored, and an image from the storage unit. Then, a clustering process is performed to generate a cluster that includes images having similar shooting times and whose number of images belongs to a threshold value or less based on the shooting time information. And a clustering execution unit that associates the clusters and stores them in the storage unit.

  Such an image evaluation apparatus preferably includes an image display unit that reads and displays an image for each cluster from the storage unit.

  At this time, it is preferable that the clustering execution unit controls the number of clusters to be generated so that the number of images belonging to each cluster is equal to or less than the maximum number of display images that can be displayed on the image display unit.

  In addition, the clustering execution unit re-executes the clustering process on a cluster in which the number of images belonging to the generated cluster exceeds the maximum number of display images that can be displayed on the image display unit, and a plurality of the clusters are included. It is preferable that the number of assigned images of all the clusters finally generated is divided below the maximum number of displayed images.

Furthermore, the clustering execution unit sets the number of clusters to be generated when the number of images to be processed is m, the maximum number of display images that can be displayed on the image display unit is n, and ceiling (·) is rounded up. C = ceiling (m / n)
It is preferable to determine by.

In addition, the clustering execution unit re-executes the clustering process for a cluster in which the number of images to which it belongs exceeds the maximum number of display images that can be displayed on the image display unit. When the number of images is k, the maximum number of display images that can be displayed on the image display unit is n, and ceiling (·) is rounded up, the number of clusters c _k generated by dividing the cluster is expressed by the following equation c _k = ceiling (k / n)
It is preferable to determine by.

  In addition, the clustering execution unit assigns n when the maximum number of display images that can be displayed on the image display unit is n and, as a result of the clustering process, when a plurality of clusters having the number of images belonging to n / 2 or less are adjacent to each other. The clusters are preferably combined into one cluster so that the number of images to be processed is n or less.

  When the image evaluation apparatus according to the present invention is configured as described above, images can be clustered according to the density of shooting times, and as a result, a set of images representing the same event with a short shooting time can be displayed at a time. It becomes possible.

  Hereinafter, preferred embodiments of the present invention will be described with reference to the drawings. First, the configuration of the image evaluation apparatus 100 according to the present embodiment will be described with reference to FIG. As shown in FIG. 1, the image evaluation apparatus 100 includes a control unit 101 that controls the entire apparatus, a database 103 (storage unit) that stores an image and shooting time information of the image associated with each other, and a control unit 100. CPU 102 that performs image evaluation processing according to the instruction, a clustering execution unit 104 that generates a cluster composed of images of the database 103 that are close in shooting time (executes clustering processing) on the CPU 102, and results of clustering processing And a work area (Work RAM) 105 for temporarily storing intermediate calculations and the like, a clustering start switch (S / W) 106 for giving a clustering start instruction to the control unit 101, and an image display for displaying an image based on the clustering result Part 107.

  The database 103 has a data structure as shown in FIG. 2, and an image column 103a for storing an image, a shooting time information column 103b for storing shooting time information of the image, and a result of clustering described later. The cluster number column 103c is stored, and one image is managed as one record. For each image file, an image column 103a for managing image data itself and a shooting time information column 103b for storing attribute information areas for storing attribute information such as header areas and shooting time information are stored in one file. You may make it comprise as. As such a data structure, Exif (Exchangeable image file format) standardized by the Japan Electronics Industry Promotion Association (JEIDA) is known.

(Image evaluation processing)
The image evaluation process performed by the image evaluation apparatus 100 will be described with reference to the flowchart of FIG. When the user presses the clustering start switch 106, a clustering start instruction is transmitted to the control unit 101 (step S100). When receiving the clustering start instruction, the control unit 101 issues a clustering start command to the CPU 102 (step S200). When the CPU 102 receives the clustering start command, the CPU 102 sets the shooting time information associated with the image stored in the database 103 and stored in the shooting time information column 103b to the file name of the image stored in the image column 103a (for example, , 0001.jpg, etc.) and all data are read (step S300). The entire shooting time information associated with the file name of this image is called a “shooting time information data set”. Note that various data such as the photographing time information data set, the calculation results in clustering described below, and the clustering results are temporarily stored in the work area 105, and the data between the units is used by each unit. Is delivered.

  Next, the CPU 102 passes this imaging time information data set to the clustering execution unit 104 and issues a clustering execution command to the clustering execution unit 104 (step S400). The clustering execution unit 104 executes clustering processing based on the given shooting time information data set. That is, as shown in FIG. 7, the images are classified based on the shooting time information data set, and a plurality of clusters are generated by combining the images having the same shooting time. The number of generated clusters and a calculation result (hereinafter referred to as a clustering result) indicating to which cluster the image associated with each shooting time information belongs are returned to the CPU 102 (step S500). The CPU 102 acquires the crystallizing result (step S600), assigns cluster numbers to all the generated clusters in ascending order based on the received clustering result, and belongs to each image in the database 103 for each image. The cluster numbers of the clusters are associated and stored in the cluster number column 103c of the database 103 (step S700). For example, in this embodiment, as shown in FIG. 7, the image is clustered into five clusters, and each cluster is assigned a cluster number of 1 to 5.

  After the above clustering processing (steps S400 to S700) is completed, the CPU 102 notifies the control unit 101 to that effect. When the control unit 101 receives a clustering processing end notification from the CPU 102, the control unit 101 accesses the database 103, reads all the image data including the image and shooting time information, and the cluster number, and sends each cluster to the image display unit 107. Each time, an instruction and each image are transmitted to sequentially display a set of images clustered in the same cluster (step S800).

  Upon receiving the clustering result display command and the image data from the control unit 101, the image display unit 107 sequentially displays each image set as a clustering result according to a predetermined display format (screen layout) (step S900). ). For example, when the maximum display number n = 2, the image is displayed according to the screen layout as shown in FIG. That is, two images 20 are displayed side by side on the left and right of the screen 10. The above is the operation of the image evaluation process by the image evaluation apparatus 100.

  FIG. 8B shows an example of the screen layout when n = 3, and FIG. 8C shows an example of the screen layout when n = 5. In the above, the screen layout is determined according to the maximum image display number n, but the image layouts shown in FIGS. 8A to 8C may be used according to the cluster size of each cluster. For example, when the maximum image display number n = 5, a cluster with a cluster size of 2 displays an image using the screen layout shown in FIG. 8A, and a cluster with a cluster size of 3 An image is displayed using the screen layout shown in FIG. 8B, and the clusters having cluster sizes of 4 and 5 are displayed using the screen layout shown in FIG. As described above, by displaying the image by changing the screen layout according to the cluster size, it is possible to efficiently display the image on the image display unit 107. Here, the image display method follows the format (size and position) defined in the selected layout.

(Clustering process)
Next, the procedure of the clustering process (step S500) performed by the clustering execution unit 104 based on the photographing time information will be described based on the flowchart of FIG. The clustering execution unit 104 acquires shooting time information data sets from the CPU 102 (step S510), and executes hierarchical clustering, which is a kind of agglomerative clustering, for these (step S520). For this hierarchical clustering, a conventionally known method can be used. As a result of the hierarchical clustering, a system diagram as shown in FIG. 5 is created for the image corresponding to the feature amount.

  Next, the number of clusters to be generated is determined. First, when the maximum number of displayed images on the image display unit 107 is n and the number of images read from the database 103 is m, the number of clusters to be generated c is determined by the following equation (1). Here, ceiling (·) means a round-up operation. By this processing, for example, when the maximum number of displayed images n = 2 and the number of images m = 8, the number of clusters c = 4 is calculated.

c = ceiling (m / n) (1)

  Based on the number of clusters c, the system diagram is cut so that the number of clusters obtained is c, and a clustering result is obtained. For example, FIG. 6 shows the clustering result when the system diagram is cut so that c = 4 when the number of images m = 8 as described above (step S530).

Here, in FIG. 6, a cluster composed of {0006.jpg, 0007.jpg, 0008.jpg} has a cluster size in which the number of images, that is, the cluster size k exceeds the maximum number n = 2 of displayed images of the image display unit 107. It has become. The clustering execution unit 104 determines whether or not the generated cluster size k is equal to or smaller than the maximum number of displayed images n (step S540). Clustering (system diagram cutting) is performed (step S550). The number of clusters _ck generated at that time is determined by the following equation (2).

c _k = ceiling (k / n) (2)

In the case of the example shown in FIG. 6, since k = 3 and n = 2, c _k = ceiling (3/2) = 2. Accordingly, since one of the four clusters is divided into two, the total number of clusters c to be generated is 5 at this point. The process of step S550 is repeated until the cluster size of all clusters finally becomes n or less. FIG. 7 shows an example of the clustering result finally obtained.

(Cluster join processing)
In the above embodiment, in the clustering process, clustering is repeated to further divide the cluster so that the cluster size k is equal to or less than the maximum display image number n. However, the cluster size of images belonging to temporally adjacent clusters is In the case where n / 2 or less is equal to or smaller than a certain threshold value, adjacent clusters whose cluster size is equal to or smaller than the threshold value may be merged to generate a cluster having a cluster size close to n. By this combination, a large number of screens with a small number of images are not displayed on the image display unit 107, the number of screens can be reduced and the layout can be improved, and the user can easily browse. In addition, when the cluster size of the cluster and its adjacent clusters is less than or equal to the threshold value, when three or more clusters with the cluster size equal to or less than the threshold value continue, the clusters closest in time are combined.

  Further, although hierarchical clustering is used as a clustering method in this embodiment, any conventionally known method may be used as the clustering method as long as the number of clusters to be generated can be controlled, for example, hierarchical clustering. Instead of this, an algorithm such as K-means clustering or a self-organizing map may be used.

  As described above, since it has been conventionally assumed that the shooting interval has a fixed threshold, it is difficult to cope with the shooting interval that changes due to various situations. However, according to the image evaluation apparatus of this embodiment, the shooting time Images can be clustered according to the density. At this time, if there is no restriction on the cluster size, for example, when considering displaying the clustering result on one screen, clustering corresponding to time (that is, clustering grouped for each image according to the shooting event). Even when the image is displayed, the result is limited by the number of images that can be output simultaneously by the image output device, so that it is possible to display the image ignoring the clustering result. However, in the image evaluation apparatus according to the present embodiment, clusters that are adjacent in time are combined by repeating clustering so that the cluster size is equal to or smaller than the maximum number of displayed screens or close to the maximum number of displayed images. By doing this, it is possible to display a set of images representing the same event with a short shooting time at a time.

It is a block diagram showing the structure of the image evaluation apparatus of this embodiment. It is explanatory drawing which shows the data structure of a database. It is a flowchart showing the image evaluation process sequence performed with the image evaluation apparatus of this embodiment. It is a flowchart showing the clustering process sequence in a clustering execution part. It is a schematic diagram showing the systematic diagram obtained as a result of hierarchical clustering. It is a schematic diagram showing the result of having cut | disconnected the system | strain diagram by cluster size c = 4. It is a schematic diagram showing the example of the clustering result finally obtained. It is a schematic diagram which shows the example of the screen displayed on an image display part according to the screen layout for image display, (a) is a screen for two image display, (b) is for three image display (C) is a screen for displaying five images.

Explanation of symbols

DESCRIPTION OF SYMBOLS 100 Image evaluation apparatus 101 Control part 103 Database (memory | storage part)
104 Clustering execution unit 107 Image display unit

Claims (7)

A storage unit in which an image and shooting time information of the image are associated and stored;
Clustering that reads out the image and the shooting time information from the storage unit and generates a cluster that is composed of images that have a close shooting time based on the shooting time information and in which the number of belonging images is equal to or less than a threshold value A clustering execution unit that executes processing and associates the cluster with the image and stores the image in the storage unit;
An image evaluation apparatus.   The image evaluation apparatus according to claim 1, further comprising an image display unit that reads and displays the image for each cluster from the storage unit. The clustering execution unit
The image evaluation apparatus according to claim 2, wherein the number of the clusters to be generated is controlled so that the number of images belonging to each of the clusters is equal to or less than a maximum number of display images that can be displayed on the image display unit. The clustering execution unit
Of the generated clusters, the clustering process is performed again on a cluster in which the number of images to which it belongs exceeds the maximum number of displayed images that can be displayed on the image display unit, and the cluster is divided into a plurality of clusters. 4. The image evaluation apparatus according to claim 2, wherein the number of images belonging to all clusters finally generated is set to be equal to or less than the maximum number of displayed images. The clustering execution unit
When the number of images to be processed is m, the maximum number of display images that can be displayed on the image display unit is n, and ceiling (·) is rounded up, the number c of clusters to be generated is expressed by the following equation c = Ceiling (m / n)
The image evaluation apparatus according to any one of claims 2 to 4, which is determined by: The clustering execution unit
When the clustering process is performed again on a cluster in which the number of images to which it belongs exceeds the maximum number of display images that can be displayed on the image display unit,
When the number of images belonging to the cluster exceeding the maximum number of displayed images is k, the maximum number of display images that can be displayed on the image display unit is n, and ceiling (·) is rounded up, the cluster is The number c _{k of} the clusters generated by division is expressed by the following formula: c _k = ceiling (k / n)
The image evaluation apparatus according to claim 4 or 5, which is determined by The clustering execution unit
When the maximum number of displayable images that can be displayed on the image display unit is n, the result of the clustering process is that when a plurality of clusters that belong to n / 2 or less are adjacent to each other, the number of images that belong is n. The image evaluation apparatus according to any one of claims 2 to 6, wherein the clusters are combined to form one cluster so as to satisfy the following conditions.

Images