JP2006344155A - Device, method, and program for determining image display position, and computer readable recording medium with program recorded thereon - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To arrange similar images close to one another when determining the display position of each image in order to display images in a list. <P>SOLUTION: An image device list display device 1 is a device for displaying a plurality of thumbnail images in a list and includes an image display position determining part 10 for determining the display position of each thumbnail image in the images of the list. The image display position determining part 10 includes a characteristic value calculating part 12 for calculating coordinates in an original image of a domain block closest to an image similar to a range block in the original image as a first characteristic value representing characteristics of the original image, a characteristic vector generating part 13 for generating a characteristic vector including the first characteristic value calculated about each original image as an element for each original image, and an image arranging part 15 for using the characteristic vector to learn a self-organization map and for determining the display position of each thumbnail image in the images of the list on the basis of the position of an element belonging to each original image in the learned self-organization map. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to an apparatus for determining a display position of each thumbnail image when displaying a list of a plurality of thumbnail images, a method used in the apparatus, a program for realizing the apparatus, and a recording medium recording the same Is.

  With the recent development of broadband and the spread of various digital media, the number of images owned by individuals has increased dramatically. Along with this, there is a need to search for a desired image from an enormous amount of images, but it is a heavy burden to perform manually. For this reason, there is an increasing need for a method of automatically selecting a desired image or an image similar to the image from given images and presenting it to the user.

  By the way, about the technique which searches a similar image automatically, research has been actively performed conventionally, and very many methods are proposed. In general, these methods require a user to provide a question image in advance in order to perform a search.

  However, there is a possibility that the user does not hold an image similar to the image to be searched. In such a case, it is not possible to search for a similar image by these methods. In addition, there are many cases where the search intention of what kind of desired image is in the initial stage of search is not clear to the user. In such a case, it can be considered that the user desires to browse many images by trial and error. Therefore, it is necessary to collect many similar images and present the result in a form that is easy for the user to understand, that is, to visualize the result.

  From such a viewpoint, research has been conducted on a technique for displaying similar images close to each other, and such techniques are described in Non-Patent Documents 1 to 3. In the methods described in these Non-Patent Documents 1 to 3, image features are extracted as numerical values, and image features are expressed by feature vectors having these numerical values (hereinafter referred to as “characteristic values”) as elements. Then, learning of the self-organizing map is performed using this feature vector, and the arrangement of the images is determined depending on which element of the self-organizing map after learning belongs to.

Here, in the method of Patent Document 1, an edge histogram standardized by MPEG (Moving Picture Expert Group) -7 is used as the characteristic value. On the other hand, in the method of Patent Document 2, an average value of RGB components of each pixel included in an image is used as the above characteristic value. Moreover, in the method of patent document 3, the value obtained from a Gabor filter is used as said characteristic value.
Laaksonen, J., Koskela, M., Oja, E., “PicSOM-self-organizing image retrieval with MPEG-7 content descriptors,” IEEE Transactions on Neural Networks, v 13, n 4, July 2002, p 841-853 . Bin Zhu, Ramsey, M., Hsinchun Chen, “Creating a large-scale content-based airphoto image digital library,” IEEE Transactions on Image Processing, v 9, n 1, Jan. 2000, p 163-167. Laaksonen, J., Koskela, M., Laakso, S., Oja, E., “Self-organising maps as a relevance feedback technique in content-based image retrieval,” Pattern Analysis and Applications, v 4, n 2-3 , 2001, p 140-152.

However, the methods shown in the above non-patent documents 1 to 3 are in a research stage and have a problem that they are still impractical in determining similarity. Specifically, the methods described in Non-Patent Documents 1 to 3 classify a plurality of similar images (that is, a plurality of images included in a single similar image group) based on the similarity. Although possible, it is not suitable for classifying (clustering) many images included in a plurality of similar image groups into individual similar image groups. For example, it is possible to classify (clustering) a plurality of images in which similar patterns or the like are drawn based on the similarity, but for many images obtained by imaging a plurality of subjects from various angles for each subject, the same It cannot be accurately classified (clustered) into an image group including only the subject. For this reason, there is a need for an improved method for displaying similar images close to each other.

  The present invention has been made in view of the above problems, and its purpose is to accurately determine similarity when determining the display position of each image in order to display a list of images, and to which position the user is An object of the present invention is to provide an image display position determining apparatus and an image display position determining method capable of arranging thumbnail images so that it is easy to grasp what kind of image is displayed.

  As a result of repeated studies in view of the above problems, the present inventor has found that the IFS parameter value used in the fractal image coding method can be suitably used as a characteristic value representing the characteristic of an image, and completes the present invention. It came.

  That is, the image display position determination device according to the present invention is an image display position determination device that determines a display position of each thumbnail image in a list image when a plurality of images are displayed as a list of thumbnail images. An image acquisition unit that acquires an original image that is an image, a range block creating unit that creates a range block by dividing the original image for each original image, and a plurality of partial images for each original image while allowing overlapping Domain block creating means for creating a plurality of domain blocks by cutting out, and in each original image, the coordinates of the domain block most similar to the range block in the original image A coordinate calculation means for calculating the characteristic value; and a first characteristic calculated for each original image by the coordinate calculation means. A feature vector generation unit that generates a feature vector including a value as an element for each original image, and learning of a self-organizing map using the above-described feature vector generated for a plurality of original images, and self-organization after learning Self-organizing mapping means for determining the position of the element to which each original image belongs in the map, and the display position of each thumbnail image in the list image based on the position of the element to which each original image belongs in the self-organizing map And an image arrangement means for determining the image quality.

  In learning of the self-organizing map, any one of a plurality of feature vectors is arbitrarily selected and given as an input vector, thereby updating a weight vector of each arranged element. Here, the elements to be updated are an element (victory element) having a weight vector having the highest similarity (close distance) to the input vector, and an element in the vicinity thereof (neighboring element). The update amount of the neighboring element is usually set smaller than the update amount of the winning element. By repeating such learning, adjacent elements in the self-organizing map have similar weight vectors.

  In the learning of the self-organizing map described above, a feature vector including a characteristic value indicating the characteristic of the image is used as an element, and the image is attributed to the element of the self-organizing map after learning based on the feature vector. Clustering can be performed. As a result, images belonging to adjacent elements are images having similar feature vectors, that is, similar images. Therefore, similar images are arranged close to each other by the image arrangement means. As described above, the image display position determination device according to the present invention can arrange an image so that the user can easily grasp what image is displayed at which position.

  Here, in the present invention, attention is paid to self-similarity in an image as a characteristic value indicating the characteristic of the image. Note that the above "domain block most similar to the range block" means that the domain block is adjusted so that it is the same size as the range block, and further rotated as necessary between the range block and the range block. The dissimilarity based on the difference in luminance of each pixel is the smallest compared to other domain blocks.

  In general, when the images are similar (for example, the included subjects are the same), the position of the domain block most similar to a certain range block tends to be similar. Therefore, the similarity of images can be determined based on the position in the image where a domain block similar to the range block exists. Thereby, many images included in a plurality of similar image groups can be accurately classified when they are classified (clustered) into individual similar image groups.

  Such self-similarity tends to be relatively preserved even when an image is compressed by various compression methods and includes noise. Therefore, similarity can be accurately determined for images of various qualities.

  Further, the image display position determination device includes range block luminance calculation means for calculating an average value of luminance of each pixel included in the range block as a second characteristic value representing the characteristic of the original image in each original image. In addition, it is preferable that the feature vector generated by the feature vector generation unit further includes, as an element, the second characteristic value calculated for each original image by the range block luminance calculation unit.

  According to the above configuration, in addition to the above-described self-similarity in an image, the similarity of images can be determined based on the luminance of a specific area in the image. Therefore, the determination of the similarity of images can be made more accurate.

  Further, the image display position determination device calculates an average value of luminance of each pixel included in a domain block most similar to the range block as a third characteristic value representing the characteristic of the original image in each original image. Domain block luminance calculating means, and the feature vector generating means adds the third characteristic value calculated for each original image by the domain block luminance calculating means to the feature vector generated for each original image. It is preferable to further include.

  According to the above configuration, the similarity of images can be determined based on the luminance of the domain block in addition to the position information of the domain block most similar to the range block. Therefore, the determination of the similarity of images can be made more accurate.

  In addition, the image display position determination device is configured such that, in each original image, (a) the range block and (b) a size that is based on a domain block that is most similar to the range block and that is the same size as the range block. Dissimilarity calculating means for calculating the dissimilarity with the adjustment domain block as a fourth characteristic value representing the characteristics of the original image, further comprising the feature vector generated by the feature vector generating means for each original image It is preferable that the fourth characteristic value calculated for each original image by the dissimilarity calculating means is further included as an element.

  According to the above configuration, in addition to the position information of the domain block that is most similar to the range block, the characteristics of the image are expressed using as an index how much the range block differs from the similar domain block. Will be able to. Therefore, the determination result becomes more accurate by determining the image based on this dissimilarity.

  Further, the domain block creating means may create a domain block having the same size as the range block.

  According to the above configuration, since the size of the range block and the domain block is the same, when determining the domain block most similar to the range block, a domain block congruent with the range block may be determined. When obtaining such a domain block, it is not necessary to perform conversion processing for enlargement or reduction of the block, so that the amount of calculation can be reduced. Therefore, the display position of the thumbnail image can be determined quickly.

  Further, the range block creating means creates a range block having a size substantially proportional to the size of each original image, and the domain block creating means has a number of pixels substantially proportional to the size of each original image. It is preferable to create a domain block by cutting out a partial image having a size substantially proportional to the size of each original image while shifting.

  By making the size of the range block proportional to the size of the original image, the proportion of the range block in the original image is always constant. Therefore, the number of range blocks created from different-size original images is always constant.

  Further, by making the shift amount at the time of clipping proportional to the size of the original image, the number of domain blocks obtained is also constant. Further, by making the size of the domain block proportional to the size of the image, the proportion of the domain block in the original image becomes constant.

  From the above, even if the size of the image used for input is not constant, the conditions of the obtained range block and domain block are constant, so that similarity determination independent of the image size can be performed. Become.

  Further, the feature vector generated for each original image includes the characteristic value calculated for a plurality of different range blocks as an element, and the self-organizing mapping means searches for a winning element in learning of the self-organizing map. It is preferable to weight the characteristic values calculated for each range block included in the feature vector based on the position of the range block in the original image.

  According to the above configuration, the feature vector includes a plurality of characteristic values for different range blocks. Here, in order to determine the winning element in the self-organizing map, when calculating the dissimilarity between the input vector including the characteristic value and the weight vector, a weight based on the range block is given to the characteristic value. Thus, the dissimilarity of a specific range block can be evaluated with priority. Therefore, similarity can be determined more accurately by preferentially evaluating the dissimilarity for a region considered to be important.

  Moreover, it is preferable that the said weighting is a thing according to the distance between the position of a range block, and the center of an image.

  Usually, in an image, the subject tends to be placed near the center of the image. Therefore, according to the above-described configuration, it is possible to focus on the evaluation of the dissimilarity with respect to the range block that is important in the similarity determination, so that the similarity determination can be performed more accurately.

  Further, the image acquisition unit further acquires subject position data indicating the position of the subject in the image, and the weighting is between the position of the range block and the position of the subject in the image indicated by the subject position data. It may be in accordance with the distance.

  According to the above-described configuration, since the dissimilarity evaluation can be focused on the range blocks around the subject that should be regarded as important in the similarity determination, the similarity determination becomes more accurate.

  Further, the original image is created by imaging, an imaging environment data acquisition unit that acquires imaging environment data indicating an environment when the original image is captured, and the original image based on the imaging environment data Non-hierarchical classification means for classifying the image into a plurality of non-hierarchical groups, wherein the self-organizing mapping means learns the self-organizing map and determines the position of the element to which the original image belongs. The image placement unit divides the list image into partial areas corresponding to the group, and the display position of the thumbnail image in the partial area is determined by the element to which the original image belongs in the self-organizing map. It is preferable to determine the display position of each thumbnail image in the list image by determining for each group based on the position.

  Usually, when an image is captured, the environment in which the image is captured and the subject included in the captured image often have a relationship. That is, there is a high possibility that images taken in the same environment include the same subject. Therefore, by classifying the captured images in advance according to the imaging environment, the captured images are roughly classified according to the contents. Therefore, even if an image that cannot be effectively classified only by learning the self-organizing map using the feature vector described above is included, an image that is not similar can be obtained by classifying such an image based on the imaging environment. Can be avoided from being placed in close proximity.

Another image display position determining apparatus according to the present invention is an image display position determining apparatus that determines a display position of each thumbnail image in a list image when a plurality of images are displayed as a list by thumbnail images. In order to solve the problem, an image acquisition unit that acquires an original image that is the plurality of images, a characteristic value calculation unit that calculates an IFS parameter value for each original image by a fractal image encoding method, and a plurality of original images Image selection means for sequentially selecting a combination of two original images, and two original images for which the combination has been selected by the image selection means, based on the IFS parameter value. The error value calculation means for calculating the similarity and the display position of each thumbnail image in the list image indicate the distance in the list image between the thumbnail images. There, characterized in that it comprises a mapping means for determining to be a distance corresponding to the dissimilarity between the original image corresponding to the thumbnail image.

  According to the above configuration, the dissimilarity between images is reflected in the distance between thumbnail images in the list image in which the thumbnail images are arranged. Therefore, similar original images are arranged at positions closer to each other by the thumbnail images. As described above, the image display position determination device of the present invention can arrange an image so that the user can easily grasp what image is displayed at which position.

  Here, in the present invention, IFS parameter values calculated by the fractal image encoding method are used as characteristic values indicating the characteristics of the image, focusing on self-similarity in the image. Thus, by using the IFS parameter value, many images included in a plurality of similar image groups can be accurately classified when they are classified (clustered) into individual similar image groups.

  Such self-similarity tends to be relatively preserved even when an image is compressed by various compression methods and includes noise. Therefore, similarity can be accurately determined for images of various qualities.

  Further, the image display position determination device reconstructs a reconstructed image by reconstructing the other original image using the IFS parameter value of one original image of the two original images selected by the image selection unit. Preferably, the image forming apparatus further includes a configuration unit, and the error value calculation unit calculates a dissimilarity between the two original images based on a luminance error value between the other original image and the reconstructed image.

  According to the above configuration, the dissimilarity between the images can be calculated based on the luminance error value in the images before and after reconstruction.

  By the way, each unit of the image display position determination device may be realized by hardware, or may be realized by causing a computer to execute a program. Specifically, the program according to the present invention is a program that causes a computer to operate as each of the above means, and the program is recorded on a recording medium according to the present invention.

  When these programs are executed by a computer, the computer operates as each unit of the image display position determination device. Therefore, it is possible to accurately determine the similarity and arrange the images so that the user can easily grasp what image is displayed at which position.

  An image display position determination method according to the present invention is an image display position determination method for determining a display position of each thumbnail image in a list image when a plurality of images are displayed as a list of thumbnail images. An image acquisition process for acquiring an original image that is an image, a range block creation process for creating a range block by dividing the original image for each original image, and a plurality of partial images for each original image while allowing overlapping A domain block creating step of creating a plurality of domain blocks by cutting out, a first block representing the characteristics of the original image, in each original image, the coordinates of the domain block most similar to the range block in the original image A coordinate calculation step for calculating as a characteristic value, and a first characteristic calculated for each original image in the coordinate calculation step A feature vector generation step for generating a feature vector for each original image, and learning of a self-organizing map using the above-described feature vectors generated for a plurality of original images, and a self-organizing map after learning The self-organizing mapping step for determining the position of the element to which each original image belongs, and the display position of each thumbnail image in the list image based on the position of the element to which each original image belongs in the self-organizing map. And an image arrangement step to be determined.

  For learning self-organizing maps, feature vectors containing characteristic values indicating the characteristics of the images are used as elements, and images are clustered by assigning images to the elements of the self-organizing map after learning based on the feature vectors. It can be carried out. As a result, images belonging to adjacent elements are images having similar feature vectors, that is, similar images. Therefore, similar images are arranged close to each other in the position process. As described above, the image display position determination device according to the present invention can arrange an image so that the user can easily grasp what image is displayed at which position.

  Here, in the present invention, the self-similarity in the image is used as the characteristic value indicating the characteristic of the image. In general, when the images are similar (for example, the included subjects are the same), the positions of the domain blocks that are similar to a certain range block tend to be similar. Therefore, the similarity of images can be determined based on the position of the domain block most similar to the range block. Such a similarity relationship tends to be relatively preserved even when an image is compressed by various compression methods and includes noise. Therefore, similarity can be accurately determined for images of various qualities.

  Another image display position determination method according to the present invention is an image display position determination method for determining a display position of each thumbnail image in a list image when a plurality of images are displayed as a list of thumbnail images. In order to solve the problem, an image acquisition step of acquiring the original images as the plurality of images, a characteristic value calculation step of calculating an IFS parameter for each original image by a fractal image encoding method, and a plurality of original images An image selection step for sequentially selecting a combination of two original images, and two dissimilarities between the two original images based on the IFS parameter for the two original images selected in the image selection step. The error value calculation step for calculating the distance between the thumbnail images and the display position of each thumbnail image in the list image. Characterized in that it includes a mapping step of determining to be a distance corresponding to the dissimilarity between the original image corresponding to the thumbnail image.

  According to the above configuration, the dissimilarity between images is reflected in the distance between thumbnail images in the list image in which the thumbnail images are arranged. Therefore, similar original images are arranged at positions closer to each other by the thumbnail images. As described above, the image display position determination device of the present invention can arrange an image so that the user can easily grasp what image is displayed at which position.

  Here, in the present invention, IFS parameter values calculated by the fractal image encoding method are used as characteristic values indicating the characteristics of the image, focusing on self-similarity in the image. Thus, by using the IFS parameter value, many images included in a plurality of similar image groups can be accurately classified when they are classified (clustered) into individual similar image groups.

  Such self-similarity tends to be relatively preserved even when an image is compressed by various compression methods and includes noise. Therefore, similarity can be accurately determined for images of various qualities.

  As described above, the image display position determination device according to the present invention includes an image acquisition unit that acquires an original image, a range block generation unit that generates a range block by dividing the original image for each original image, and each element Domain block creation means for creating a plurality of domain blocks by cutting out a plurality of partial images while allowing overlap with each other in the image, and in each original image, in the original image of the domain block that is most similar to the above range block For each original image, a coordinate calculation means for calculating the coordinates of the original image as a first characteristic value representing the characteristics of the original image, and a feature vector including the first characteristic value calculated for each original image by the coordinate calculation means as an element A self-organizing map using the feature vector generating means for generating the original image and the feature vector generated for a plurality of original images. Self-organizing mapping means for determining the position of the element to which each original image belongs in the self-organizing map after learning, and based on the position of the element to which each original image belongs in the self-organizing map And image arrangement means for determining the display position of each thumbnail image in the list image.

  In addition, as described above, another image display position determination device according to the present invention includes a characteristic value calculation unit that calculates an IFS parameter value for each original image by a fractal image encoding method, and a plurality of original images. The dissimilarity between the two original images is determined based on the IFS parameter value for the image selection means for sequentially selecting the combination of the two original images and the two original images for which the combination is selected by the image selection means. The error value calculation means for calculating, and the display position of each thumbnail image in the list image, the distance in the list image between the thumbnail images is a distance according to the dissimilarity between the original images corresponding to the thumbnail image And mapping means for determining to be.

  In addition, as described above, the image display position determination method according to the present invention creates the range block by dividing the original image for each original image by acquiring the original image as the plurality of images. The range block creation step, the domain block creation step of creating a plurality of domain blocks by cutting out a plurality of partial images while allowing each other to overlap each other, and the most similar to the above range block in each original image A coordinate calculation step for calculating the coordinates of the domain block in the original image as a first characteristic value representing the characteristic of the original image, and the first characteristic value calculated for each original image in the coordinate calculation step as an element Generating a feature vector for each original image, and a feature vector generated for a plurality of original images. Is used to learn a self-organizing map, determine a position of an element to which each original image belongs in the self-organizing map after learning, and each original image in the self-organizing map. And an image arrangement step for determining the display position of each thumbnail image in the list image based on the position of the element to which it belongs.

  Further, another image display position determination method according to the present invention, as described above, for each original image, a characteristic value calculation step of calculating an IFS parameter by a fractal image encoding method, and a plurality of original images, An image selection step for sequentially selecting a combination of two original images, and an error value for calculating the dissimilarity between the two original images based on the IFS parameter for the two original images for which the combination was selected in the image selection step The calculation step and the display position of each thumbnail image in the list image are determined such that the distance between the thumbnail images in the list image is a distance corresponding to the dissimilarity between the original images corresponding to the thumbnail images. And a mapping process.

  Therefore, as described above, when determining the display position of each image in order to display a list of images, the similarity is accurately determined, and the user grasps what image is displayed at which position. There is an effect that thumbnail images can be arranged easily.

[Embodiment 1]
An embodiment of the image display position determining apparatus according to the present invention will be described below with reference to FIGS. First, an outline of the present embodiment will be described.

〔Overview〕
The image list display device 1 of this embodiment is a device for displaying a list of thumbnail images of a plurality of captured images. Here, when the captured images are displayed as a list, the image list display device 1 determines the display position of the thumbnail image based on the self-organizing map created for each imaging date and time. Note that, as input data (feature vector) when creating a self-organizing map, an IFS parameter value calculated for each image in the fractal image coding method or the like is used. As a result, the plurality of captured images are arranged so that the images belonging to the same date group are grouped together, and similar images are close to each other in the group. The overall image of such an image list display device 1 will be described as follows.

〔Overall picture〕
FIG. 1 is a functional block diagram of the image list display device 1. The image list display device 1 generally includes an image display position determination unit (image display position determination device) 10, a thumbnail image generation unit 21, a list image generation unit 22, and a display unit 20.

  The image display position determination unit 10 is for determining the display position of each thumbnail image when displaying a list of thumbnail images of a plurality of captured images. The thumbnail image generation unit 21 is for generating a thumbnail image corresponding to the captured image. Normally, the thumbnail image is a reduced image of the captured image (original image), but is not necessarily limited to this, and the original image may be used as it is. The list image generation unit 22 is for generating list image data displaying a list of thumbnail images.

  The display unit 20 is for displaying a list image based on the list image data generated by the list image generation unit 20. The display unit 20 is not particularly limited as long as it can display an image, and may be any of a display device such as a CRT display or a liquid crystal display, a projection device such as a projector, or a printing device such as a printer. May be.

  In detail, the image display position determination unit 10 includes an image data acquisition unit (image acquisition unit, imaging environment data acquisition unit) 11 with an imaging date and time data, a characteristic value calculation unit 12, and a feature vector generation unit (feature vector generation unit) 13. A self-organizing mapping unit (self-organizing mapping unit) 14, an image arrangement unit (image arrangement unit) 15, and a non-hierarchical classification unit (non-hierarchical classification unit) 16.

  The image data acquisition unit 11 with imaging date / time data is for acquiring image data of a plurality of images captured by a digital camera or the like together with the imaging date / time data.

  In recent years, the Exif (Exchange Image File Format) format has been used as an image file standard for digital cameras. According to this format, in the image file including the image data of the captured image, various tag information related to the environment and conditions when the image is captured is recorded at the same time. Examples of the tag information include image size, compression mode, imaging date / time, color space information, spectral sensitivity, luminance, exposure correction, subject distance, subject position, and the like.

  The image list display device 1 of the present embodiment uses imaging date / time data included as tag information in an image file. Therefore, in the present embodiment, the acquisition of the image data and the acquisition of the imaging environment data is realized by the single image data acquisition unit 11 with the imaging date / time data, but the present invention is not limited to this, and the image data Acquisition and acquisition of imaging environment data may be realized by another member. Also, the imaging environment data is not limited to the imaging date and time, and for example, a suitable piece of the tag information may be used.

  The hardware configuration of the image data acquisition unit (image acquisition unit) 11 with imaging date / time data is not particularly limited as long as image data can be acquired, and data can be read from a fixed disk or a removable disk. An input / output interface, an input / output interface that can read data from another communication terminal device via a LAN, or an input interface that can import image data and imaging date / time data from a digital camera or the like may be provided. .

  The characteristic value calculation unit 12 is for calculating characteristic values representing various characteristics of an image for each image whose image data has been acquired by the image data acquisition unit 11 with imaging date and time data. Here, the characteristic value representing the characteristic of the image is not particularly limited, and general luminance values and various commonly used parameter values used in the field of pattern recognition can be used. The image list display device 1 uses an IFS parameter value calculated in the fractal image encoding method. Details of the characteristic value will be described later.

  The feature vector generation unit 13 generates a feature vector including various characteristic values obtained for each image by the characteristic value calculation unit 12, and uses this as an index that comprehensively represents the features of each image. One feature vector is generated for each input image.

  The non-hierarchical classification means (non-hierarchical classification unit) 16 is for classifying images into non-hierarchical groups based on imaging environment data (imaging date / time data) attached to the image data.

  The self-organizing mapping means (self-organizing mapping section) 14 learns the self-organizing map for each group classified by the non-hierarchical classification means (non-hierarchical classification section) 16. Here, image feature vectors are used as input data for learning. Then, it is determined to which element in the self-organizing map after learning the image belongs.

  The image placement unit (image placement unit) 15 is for determining the display position of the thumbnail image based on the position of the element to which the image belongs in the self-organizing map and the group.

  The characteristic value calculation unit 12, the non-hierarchical classification unit (non-hierarchical classification unit) 16, the self-organizing mapping unit (self-organizing mapping unit) 14, and the image arrangement unit (image arrangement unit) 15 are a CPU or the like. It is a functional block realized by an arithmetic device executing a program code stored in a storage device such as a ROM or a RAM.

  Next, a schematic operation of the image list display device 1 will be described. FIG. 2 is a process diagram showing the operation of the image list display device 1. FIG. 3 is a diagram showing a scheme of a display method of the image list display device.

  First, the image data acquisition unit 11 with imaging date / time data acquires image data of the captured image together with the imaging date / time data using a fixed disk, a removable disk, or a LAN (S1). The acquired image data is sent to the characteristic value calculation unit 12 and the thumbnail image generation unit 21. The acquired imaging date / time data is sent to the non-hierarchical classification unit 16.

  Next, the non-hierarchical classification unit 16 classifies the captured images into a plurality of groups based on the imaging date / time data (S2). The correspondence between the group and the image (which image belongs to which group) is sent to the self-organizing mapping unit 14. On the other hand, the characteristic value calculation unit 12 calculates various characteristic values for each image based on the fractal encoding method (S3). The calculated characteristic value is sent to the feature vector generation unit 13. Note that the execution order of steps S2 and S3 may be reversed.

  Next, the feature vector generation unit 13 generates a feature vector for each image including various characteristic values calculated for each image as elements (S4).

  Subsequently, the self-organizing mapping unit 14 learns the self-organizing map for each group created in step S2 (S5). At this time, the feature vector generated for each image in step S4 is used as input data (input vector) of the self-organizing map.

  Then, it is determined which element the feature vector of each image is closest to the weight vector of the elements included in the created self-organizing map, and determines which element each image belongs to (S6). . This process is performed for all groups. Then, the positional relationship information of the element to which the image belongs in the self-organizing map for each group is sent to the image placement unit 15.

  Subsequently, the image placement unit 15 determines the display position of the image based on the position information of the element to which the image belongs and the group (S7). Information on the determined display position of the image is sent to the list image generation unit 22.

  In addition, the thumbnail image generation unit 21 generates a thumbnail image of each image based on the image data acquired by the image acquisition unit 11 with imaging date and time data. The generated thumbnail image data is sent to the list image generation unit 22. Then, the list image generation unit 22 generates a list image in which the thumbnail images of each image are arranged at the display position determined by the image arrangement unit 15. The list image data is sent to the display unit 20, and the display unit 20 displays the list image (S8).

  Through the above processing, it is possible to display a list image in which captured images are grouped by date and time, and similar images are arranged close to each other in the same date and time group. Hereinafter, the non-hierarchical classification unit 16, the characteristic value calculation unit 12, the self-organizing mapping unit 14, and the image arrangement unit 15 will be described in detail.

[Non-hierarchical classification, non-hierarchical classification process]
The non-hierarchical classification unit 16 is for classifying images into groups based on the imaging environment data.

  Usually, when traveling and taking a picture, the photograph is often taken while sequentially moving a plurality of points. In this case, the same subject is photographed at substantially the same time (or day). Conversely, photographs taken at approximately the same time tend to include the same subject. Therefore, if the images are classified into a plurality of groups for each shooting date and time, it is considered that the images included in the same group are more likely to contain the same subject.

  For the above reason, the non-hierarchical classification unit 16 classifies the captured image based on the date and time. As a specific method for classifying into groups, for example, each day may be classified into separate groups, or may be further classified into separate groups in the morning / afternoon. You may classify into separate groups by time. Further, the above reference may be adjusted according to the number of acquired images.

  In general, the dates and times when images are captured are not uniformly distributed and tend to be unevenly distributed at specific times or days. For example, there may be many weekends or few late nights. Therefore, instead of setting the threshold value used when classifying into groups at regular intervals as described above, the threshold value may be set dynamically according to the distribution of the captured date and time. In order to realize this, clustering may be performed by a well-known and commonly used method using the imaging date and time as an explanatory variable. Note that as a result of classification by the non-hierarchical classification unit 16, the images are classified into non-hierarchical groups. Then, the data of the correspondence relationship between the group and the image is sent to the self-organizing mapping unit 14.

[Characteristic Value Calculation Unit, Characteristic Value Calculation Step / Feature Vector Generation Unit, Feature Vector Generation Step]
The characteristic value calculation unit 12 calculates various parameter values such as IFS parameter values used for fractal image encoding as image characteristic values for all captured images. The inventor of the present invention has found that IFS parameters used for image encoding and similar parameters can be suitably used to express image characteristics.

  The fractal image encoding method is a method proposed by Jacquin. The details are described in (AE Jacquin, "Image coding based on fractal theory of iterated contractive iamge transformation," IEEE Trans. On Image Process., Vol. 1, pp.18-30, Jan 1992.) This is incorporated herein. Further, a known improvement may be added to this. Hereinafter, a method for calculating an image characteristic value using the fractal image encoding method will be described.

  FIG. 4 is a functional block diagram showing a detailed configuration of the characteristic value calculation unit 12. The characteristic value calculator 12 includes a range block generator (range block generator) 31, a domain block generator (domain block generator) 32, an IFS parameter calculator (coordinate calculator) 33, and a range block luminance calculator (range block). (Brightness calculating means) 34, domain block luminance calculating section (domain block luminance calculating means) 35, and dissimilarity calculating section (dissimilarity calculating means) 36.

  FIG. 5 is a diagram showing a scheme of a fractal image encoding method. Here, it is assumed that the image f for which the IFS parameter value is calculated is composed of N × M pixels. In the image f, the luminance at coordinates (x, y) (x = 1, 2,..., N; y = 1, 2,..., M) is defined as f (x, y).

(Procedure 1)
First, the image f is divided into B × B pixel divided images R _i (i = 1, 2,..., N; n = [N / B] × [M / B]) that do not overlap each other. If the result of division does not result in an integer, the decimal part is rounded down. This divided image R _i is called a range block. In addition, the luminance value of the pixel (x ′, y ′) (x ′ = 1, 2,..., B; y ′ = 1, 2,..., B) included in the range block R _i is expressed as f _Ri (x ′ , Y ′).

(Procedure 2)
Next, a partial image D _j (j = 1, 2,..., M; m = [(N) of 2B × 2B pixels while being shifted from the image f by δ _{h in} the horizontal direction and δ _{v in} the vertical direction while allowing overlap. −2B + 1) / δ _h ] × [(M−2B + 1) / δ _v ]). This partial image D _j is called a domain block. Further, the luminance value of the pixel (x ′, y ′) (x ′ = 1, 2,..., B; y ′ = 1, 2,..., B) included in the domain block D _j is expressed as f _Dj (x ′ , Y ′).

(Procedure 3)
Then, in order to obtain a domain block D _k (1 ≦ k ≦ m) that gives the best approximation for each range block R _i (in other words, closest to the similar shape of each range block R _i ), Steps 3-1 and 3-2 are performed for each range block.

(Procedure 3-1)
First, a reduced domain block (size adjustment domain block) D ′ _j obtained by reducing each domain block D _j to B × B pixels is obtained. Here, the luminance value of the pixel (x ′, y ′) (x ′ = 1, 2,..., B; y ′ = 1, 2,..., B) included in the reduced domain block D ′ _j is expressed as f _D. Let _'j (x', y '). At this time, f _D ′ _j (x ′, y ′) is obtained by the conversion equation shown below.

  However,

  Here, θ (θ = {0, π / 2, π, 3π / 2 [rad]}) represents a rotation angle. Further, λ (λ = {1, −1}) represents an inversion coefficient. e and f represent values for correcting a shift in coordinates caused by rotation or inversion. s and o represent the scaling coefficient and offset coefficient of the luminance conversion value, respectively. The specific values of θ, λ, s, and o are values for minimizing the error E defined by the following equation.

The error E defined by the above equation (4) is based on the difference between the luminance value of each pixel included in the range block R _i and the luminance value of each pixel included in the reduced domain block D ′ _j . , The dissimilarity between the range block R _i and the reduced domain block D _j is shown.

(Procedure 3-2)
Among the reduced domain blocks D ′ _j obtained in the procedure 3-1, the reduced domain block D ′ _opt (D ′ _opt ∈ {D ′) that minimizes the error E of the equation (4) for the range block R _i . ₁ , D ′ ₂ ,..., D ′ _m }). Then, parameters x _D , y _D (hereinafter referred to as position parameters) representing the horizontal method and vertical coordinates of the domain block D _opt giving the found reduced domain block D ′ _opt , parameters representing rotation / inversion at this time theta, lambda, and scaling parameters s of luminance conversion value, o is the IFS parameter for range block _{R i.} Thus, the calculation of the IFS parameter value for each range block in a certain image is completed.

Here, all of the above IFS parameter values obtained for each range block may be used as image characteristic values. However, in the present embodiment, from the viewpoint of reducing the amount of calculation, among the above IFS parameters, the domain block The values of the position parameters x _D and y _D indicating the coordinates in the image are used as characteristic values. The values of the position parameters x _D and y _D correspond to the first characteristic value in the claims.

One set of position parameters x _D and y _D is obtained for each range block. Here, in the present embodiment, the position parameters x _D and y _D are obtained for all the n range blocks included in each image, but the present invention is not limited to this, and the n number of range blocks are determined. Among the range blocks, the position parameters x _D and y _D may be obtained only for a predetermined number of range blocks at a predetermined position. In this case, the “predetermined position” may be any suitable position in the image including, for example, the center of the image or the peripheral edge of the image. Similarly, the “predetermined number” can be a suitable number from 1 to n. However, when the position parameters x _D and y _D are obtained only for a part of the range blocks, the above “predetermined position” and “predetermined number” need to be unified among all images.

Further, when the captured image is a color image, the luminance of each pixel included in the image is decomposed into three color components Y, Cb, and Cr, and the above procedure is executed for each color component. It is preferable to obtain the values of the position parameters x _D and y _D for each color component. In this case, for each range block, three sets of position parameters x _D and y _D for each color component are obtained.

  Of the above-described procedures, procedure 1 is performed by the range block creation unit 31, procedure 2 is performed by the domain block creation unit 32, and procedure 3 is performed by the IFS parameter calculation unit 33.

In this embodiment, as a preferred embodiment, the following characteristic values are further calculated in addition to the values of the position parameters x _D and y _D.

First, the average value f ^ave _Ri of the luminance of each pixel included in the above range block R _i, is calculated for each range block. The value of the average value f ^ave _Ri of the luminance of pixels included in the range block R _i can be calculated by the following equation.

Further, an average value f ^ave _Dopt of the luminance of each pixel included in each domain block D _opt obtained for each range block R _i is calculated. Note that the value of the average value f ^ave _Dopt of the luminance of each pixel included in the domain block D _opt can be calculated by the following equation.

Further, using the error E obtained in the above equation (4), the value of _ERMS defined by the following equation is calculated for each range block.

Here, the value of f ^ave _Ri corresponds to the second characteristic value in the claims, and is calculated by the range block luminance calculation unit 34. The value of f ^ave _Dopt corresponds to the third characteristic value in the claims, and is calculated by the domain block luminance calculation unit 35. Further, the value of _ERMS corresponds to the fourth characteristic value in the claims, and is calculated by the dissimilarity calculation unit 36.

  When the captured image is a color image, the luminance of each pixel included in the image is decomposed into three color components Y, Cb, and Cr, and the above-described range blocks and domain blocks having the respective color components are described above. Calculate the characteristic value. In this case, three sets of the above three characteristic values are obtained for a certain range block.

In this embodiment, the values of f ^ave _Ri , f ^ave _Dopt , and E _RMS are obtained for all range blocks. However, as described above, only some of the range blocks may be obtained.

The use of f ^ave _Ri , f ^ave _Dopt , and E _RMS as described above (one set for a monochrome image and three sets for a color image) have the following advantages. That is, when only the value of the position parameter of the domain block is adopted as the image characteristic value, images having similar positional relationships between the range block and the domain block are similar images in the self-organizing mapping process described later. It is recognized that there is. However, even if the positional relationship between the range block and the domain block is similar between two certain images, the luminance of the range block and the luminance of the domain block may be different between such images. Such images are not actually similar to each other, but are considered similar when only the position parameter value is adopted.

On the other hand, the above-mentioned problem can be solved by adopting the luminance values f ^ave _Ri and f ^ave _Dopt of each range block and each domain block as the characteristic value of the image.

Similarly, E _RMS is an index representing the degree of dissimilarity between the luminance value of each pixel included in each range block and the luminance value of the pixel of the reduced domain block corresponding thereto. Therefore, by adopting this _ERMS , it is possible to express to what extent each range block is not similar to the domain block (that is, dissimilarity) as a characteristic of the image.

In this way, by using f ^ave _Ri , f ^ave _Dopt , and E _RMS as image characteristic values, the similarity of images can be estimated more accurately.

  By the way, in the step of calculating the above five characteristic values, the size of the range block and the size of the domain block may be the same from the viewpoint of reducing the amount of calculation. As a result, the above-described reduced domain block can be generated without changing the size of the domain block. Specifically, the above equation (2) is changed to the following equation.

  Moreover, said Formula (6) is changed into the following formula.

In addition, when the resolution (number of vertical and horizontal pixels) of the captured image differs depending on the image, the range block for each image is used so that the position parameters x _D and y _D do not depend on the resolution of the image. In addition, the size of the domain block may be changed. Specifically, when the size of the image _{f i} is the longitudinal _{X i,} transverse _{Y i,} the size of the range block, _{vertical: X} i / H, the _horizontal: the _Y i / V (each unit is a pixel). However, H and V are constants. By doing in this way, even if the resolution of the captured image differs, the number of range blocks obtained is always H × V. Similarly, when the size of the domain block is twice as large as the range block, the length is 2X _i / H and the width is 2Y _i / V (units are pixels). When the size is the same as the range block, the length is X _i / H, the _horizontal: the _Y i / V (each unit is a pixel). Further, the shift amounts δ _h and δ _v when cutting out the domain block are also proportional to the size of the image. As a result, the number of domain blocks obtained is also constant, and the relationship of domain blocks to the entire image is always constant regardless of whether the image size is large or small. As described above, by adjusting the range block and the domain block according to the size of the image, the similarity can be determined without depending on the size (resolution) of the image. If the result of division is not an integer, a suitable method such as rounding down or rounding up may be used.

  Further, a characteristic value obtained by standardizing each calculated value based on an average value or a standard deviation may be used so that the similarity can be appropriately determined in the subsequent self-organizing mapping process.

  The characteristic values calculated for each range block for each image as described above are transmitted to the feature vector generation unit 13. The feature vector generation unit 13 generates a feature vector having each characteristic value included in the set of characteristic values calculated for each image as an element. As an example, for example, when the captured image is a monochrome image and the number of range blocks obtained by division is 4 × 4 = 16, the obtained characteristic value is 5 × 16 = 80. Therefore, the feature vector of each image is 80 dimensions. If the captured image is a color image and the range block conditions are the same as described above, the obtained characteristic value is 5 × 16 × 3 = 240. Therefore, the feature vector of each image is 240 dimensions.

[Self-organizing mapping means, self-organizing mapping process]
For each group created by the non-hierarchical classification unit 16, the self-organizing mapping unit 14 clusters images belonging to the group using a self-organizing map.

  The self-organizing map is an unsupervised competitive learning type neural network model proposed by Kohonen. Details thereof are described in (T. Kohonen, “Self-organization and associative memory,” Springer-Verlag, New York, 1989.), which is incorporated herein by reference. Further, a known improvement may be added to this.

  FIG. 6 is a diagram showing a scheme of a self-organizing map. In the self-organizing map, an element group arranged in two dimensions is usually used. As shown in FIG. 6, the configuration is composed of an element group layer (map layer) and a data input layer (input layer). It has become. Here, the data input layer is coupled to all the elements existing in the element group layer, and the input data can be given to all the elements. In addition, a feature vector having the characteristic value calculated by the characteristic value calculation unit 12 as an element is used as input data. Hereinafter, a feature vector used for input is referred to as an input vector and is represented by x.

An element k in the element group layer (k is a number for identifying an element on the map, and the total number of elements is K) has a weight vector m _k . Here, the weight vector m _k is the same dimension as the input vector, that is, a vector having the same number of elements.

  Hereinafter, learning of the self-organizing map will be specifically described. Unless otherwise specified, it is assumed that each process is performed by the self-organizing mapping unit 14.

  First, an arbitrary random value is given to each element of the weight vector for all elements. Then, one of the images belonging to the group is selected at random, the feature vector of the image is given as an input vector, and the self-organizing map is learned.

The learning algorithm of the self-organizing map includes (a) similarity matching for finding an element (winning element) having a weight vector most similar to the input vector x, and (b) the weight vector m _k and the weights of neighboring elements. And an update procedure for bringing the vector close to the input vector x.

In similarity matching, usually, the Euclidean distance between the input vector x and the weight vector m _{k of} each element is calculated, and the element having the smallest weight vector _mc is determined as the winning element. This can be expressed mathematically as:

However, the x-m _c that enclosed in double vertical bars represent the Euclidean distance between the weight vector m _c victory element and the input vector x, similarly, the x-m _k that enclosed in double vertical bars, input It represents the Euclidean distance between the vector x and the weight vector m _k .

Then, the update procedure, and the weight vector m _c of the winning element, the weight vector m _k of elements (in the vicinity elements) in the vicinity of the winner element in the layer of the element group is updated as close to the input vector x. This can be expressed by the following formula.

Here, _Nc represents a subset having the winning elements and neighboring elements in the element group layer as elements, and α represents a positive learning parameter. Note that _Nc changes depending on the number of updates, and is normally set to decrease as the number of updates increases. In addition, α changes depending on the number of updates and the distance from the winning element, and is normally set to decrease as the number of updates and the distance increase.

By repeating this learning, a weight vector group m _k (k = 1, 2,..., K) reflecting the distribution of the input vector x (that is, the feature vector of the image) is generated. In other words, the weight vector group m _k is a prototype obtained by quantizing the distribution of feature vectors.

  In addition, when the weight vector of the winning element is updated, the weight vector of the neighboring element is also updated. Therefore, in the element group arranged two-dimensionally, adjacent elements have similar weight vectors. Thereby, the self-organizing map after learning forms a set of prototypes reflecting the phase of the input data space (that is, the feature vector space of the image).

  Finally, the feature vector of each image belonging to the group is sequentially given as an input vector to the self-organizing map after learning to determine which element each image belongs to. Here, the determination method of the belonging element is the same as the determination method of the winning element. In addition, when multiple images belong to the same element, an image having a feature vector that is closest to the weight vector of the element is placed in that element, and the remaining images are placed in the vicinity thereof. To do. Thereby, it can be avoided that a plurality of images are arranged on the same element.

  The above processing is performed for each group, and data representing the positional relationship of the belonging elements of the image in the self-organizing map is transmitted to the image placement unit 15.

  In the present embodiment, as a preferred embodiment, weighting is performed on the plurality of characteristic values in similarity matching for obtaining a winning element. Specifically, weighting is performed on each set of characteristic values calculated for each range block based on the position of the range block. Along with this, Expression (8) in the similarity matching described above is changed as follows.

However, T represents transposition of a vector, and W represents a diagonal matrix whose diagonal component is a weighting factor. The above formula (8 ') is, when calculating the sum of the squares of the difference of each component of the input vector x and weight vector m _k (i.e., the square of the distance between the input vector x and weight vector m _k), This means that each squared value is multiplied by a weighting factor based on the position of the range block, and an element having a weight vector with the smallest sum is obtained as a winning element. Note that five (15 in the case of a color image) characteristic values obtained for the same range block are multiplied by the same weight coefficient.

Here, the diagonal component (that is, the weighting factor) of the diagonal matrix W can be a value based on the distance from the center of the image to the range block. For example, when there are 4 × 4 = 16 range blocks in each image, the coordinates (x _i , y _i ) of each range block R _i (i = 1, 2,..., 16) are defined as shown in FIG. The weight w _i may be set based on a Gaussian distribution represented by the following equation.

  However, (sigma) is arbitrary suitable values.

  In the above case, the feature vector of each image (ie, the input vector x) is

(However, in each element of the above vector, the numerical value on the right shoulder represents the number of the range block), the diagonal matrix W in the equation (8 ′) is 80 rows × 80 columns, and FIG. It becomes like this.

The weighting based on the position of the range block with respect to the characteristic value obtained for each range block has the following advantages. Normally, there is a tendency that the subject to be imaged is included in the center of the image. Therefore, when judging the similarity of images, it is preferable to focus on the vicinity of the center of an image where there is a high possibility that a subject exists. According to the above configuration, the range block is the value of the higher weight coefficients w _i close to the center of the image increases, in the determination of the winning element, so that the characteristic values for the central area of the image is more important. As a result, the similarity can be determined more accurately.

  Further, the weight may be given based on the distance from the subject position to the range block position, not the distance from the center of the image to the range block position. As described above, in the Exif format, data on the subject position can be included in the image file. Therefore, if subject position data is acquired as imaging environment data, weighting can be performed based on the distance from the position indicated by the subject position data to the range block. By doing so, the similarity can be determined more accurately.

[Image placement section, image placement process]
The image arrangement unit 15 determines the display position of the thumbnail image when displaying the list. At this time, the image arrangement unit 15 determines a position based on the group created by the non-hierarchical classification unit 16 and the self-organization map for each group created by the self-organization mapping unit 14. In determining the position, the images belonging to the same group may be grouped, and the positional relationship determined based on the self-organizing map in each group may be used. Specifically, for example, it can be performed as follows.

  First, a virtual display area for displaying a list of images is set, and the display area is divided by the number of groups. Then, an image is arranged in each divided display area so as to have a positional relationship based on the self-organizing map.

  Then, the image arrangement unit 15 creates list image data in which the images are arranged in this way. Here, the images displayed as a list may be captured images as they are, or may be images whose sizes are reduced so as to fit in the display area. In other words, the thumbnail images used when displaying the list may be the original images as they are or may be reduced (or enlarged). The list image data created in this way is sent to the display unit 20.

  As described so far, the image list display device 1 of the present embodiment includes the image data acquisition unit 11 with imaging date / time data that acquires image data accompanied by imaging date / time data indicating the date and time when the image was captured, and imaging. A non-hierarchical classification unit 16 that classifies images into groups based on the time of day, a characteristic value calculation unit 12 that calculates a characteristic value that represents the characteristics of the image for each image, and a feature vector that includes this characteristic value as an element for each image A feature vector generation unit 13 to generate a self-organization map for each classified group based on the above-described feature vector, and to which element in the generated self-organization map the image belongs Based on the self-organizing mapping unit 14 to be determined, the position of the belonging element, and the group, the image layout for determining the display position of the image in the virtual display area And parts 15, and a.

  According to the self-organizing map, images having similar feature vectors are arranged at close positions. Therefore, it is possible to obtain a list image in which similar images are displayed at close positions. Furthermore, since images are classified into groups according to the shooting date and time, images belonging to the same group are more likely to contain the same subject. Here, since the images belonging to the same group are arranged as a group, it is possible to obtain a list image in which similar images are gathered and displayed. Therefore, the user can easily grasp where and what images exist when listing a large number of captured images.

[Modifications / Others]
In the above embodiment, an example in which imaging date / time data is used as imaging environment data has been described. However, instead of the imaging date / time data, imaging location data relating to the imaging location may be used. FIG. 9 is a diagram showing the distribution of imaging locations.

  In recent years, measurement information by GPS (Global Positioning System) can be stored as tag information in an image file storing image data. According to GPS, it is possible to measure the latitude and longitude of the place where the image was taken, the moving speed of the photographer, the traveling direction, and the like. Of these, images can be classified using the latitude and longitude of the imaged location.

  In this case, the image data acquisition unit 11 acquires the latitude and longitude of the place where each image was captured as imaging environment data, and the non-hierarchical classification unit 16 classifies the images into a plurality of groups based on the acquired latitude and longitude. To do. Here, as a method of classifying into a plurality of groups, various well-known various methods that can be classified into non-hierarchical clusters can be used, and for example, a k-means method or the like can be used. Thereby, the place where the image was captured can be classified into a plurality of groups as shown in FIG. Then, the group to which the imaged location belongs may be the group to which the image belongs.

  Usually, when a famous building is imaged or a landscape is imaged from an observation deck or the like, a plurality of images are often captured at substantially the same place. Therefore, the imaging points are not uniformly distributed at every latitude and longitude, but tend to be unevenly distributed in a specific region. At the same time, there is a high possibility that the same subject is included in images taken at substantially the same place. Therefore, by classifying the images into a plurality of groups based on the imaging location, it is possible to more accurately place similar images close to each other.

  In the image display position determination device, the image data acquired to display a list is not limited to captured image data, and may be artificially generated image data. Furthermore, the image data is not limited to still image data, and may be moving image data. In this case, for example, moving image indexing can be performed by extracting still image data corresponding to the first frame of the moving image and performing the above-described processes on the extracted still image data.

  Moreover, in said embodiment, although the image acquisition part and the imaging environment data acquisition part were comprised as the integrated image data acquisition part 11 with imaging environment data, these may be separate. In this case, the imaging environment data is not limited to that attached to the image data. For example, the image acquisition unit includes an input interface that captures data from a scanner, and the imaging environment data acquisition unit includes an input interface that captures data from a keyboard, a mouse, or the like. (Date / time data, imaging location data, etc.) may be separately acquired by user input.

  Finally, each block of the image display position determination unit 10, in particular, the characteristic value calculation unit 12, the non-hierarchical classification unit 16, the self-organizing mapping unit 14, and the image arrangement unit 15 may be configured by hardware logic. However, it may be realized by software using a CPU.

  When implemented by software using a CPU, the image display position determination unit 10 includes a CPU (central processing unit) that executes instructions of a control program that implements each function, a ROM (read only memory) that stores the program, and the above A RAM (random access memory) for expanding the program, a storage device (recording medium) such as a memory for storing the program and various data, and the like are provided. A recording medium in which a program code (execution format program, intermediate code program, source program) of a control program of the image display position determination unit 10 which is software for realizing the functions described above is recorded so as to be readable by a computer is recorded on the image display. The object of the present invention can also be achieved by supplying to the position determination unit 10 and reading and executing the program code recorded on the recording medium by the computer (or CPU or MPU).

  Examples of the recording medium include tapes such as magnetic tapes and cassette tapes, magnetic disks such as floppy (registered trademark) disks / hard disks, and disks including optical disks such as CD-ROM / MO / MD / DVD / CD-R. Card system such as IC card, IC card (including memory card) / optical card, or semiconductor memory system such as mask ROM / EPROM / EEPROM / flash ROM.

  Further, the image display position determination unit 10 may be configured to be connectable to a communication network, and the program code may be supplied via the communication network. The communication network is not particularly limited. For example, the Internet, intranet, extranet, LAN, ISDN, VAN, CATV communication network, virtual private network, telephone line network, mobile communication network, satellite communication. A net or the like is available. Also, the transmission medium constituting the communication network is not particularly limited. For example, even in the case of wired such as IEEE 1394, USB, power line carrier, cable TV line, telephone line, ADSL line, etc., infrared rays such as IrDA and remote control, Bluetooth ( (Registered trademark), 802.11 wireless, HDR, mobile phone network, satellite line, terrestrial digital network, and the like can also be used. The present invention can also be realized in the form of a computer data signal embedded in a carrier wave in which the program code is embodied by electronic transmission.

[Embodiment 2]
Another embodiment of the image display position determining apparatus according to the present invention will be described below with reference to FIGS. Note that a description of the same parts as those in the first embodiment will be omitted. First, an outline of the present embodiment will be described.

〔Overview〕
The image list display device 2 of the present embodiment is a device for displaying a list of thumbnail images of a plurality of images. In this embodiment, when displaying a list of captured images, first, two images are extracted from a plurality of images, and the dissimilarity between the images is calculated. The dissimilarity is calculated for all image combinations. Thereafter, a plurality of images are arranged in the two-dimensional display area. At this time, it arrange | positions so that the distance of each thumbnail image in a display area may become the distance according to the calculated dissimilarity. Thereby, dissimilar images are arranged at positions separated from each other, and similar images are arranged close to each other. Note that the IFS parameter value calculated by the above-described fractal image encoding method is used to calculate the dissimilarity between images.

〔Overall picture〕
FIG. 16 is a functional block diagram of the image list display device 2. The image list display device 2 schematically includes an image display position determination unit (image display position determination device) 110, a thumbnail image generation unit 21, a list image generation unit 22, and a display unit 20.

  The image display position determination unit 110 is for determining the display position of each thumbnail image when displaying a list of thumbnail images of a plurality of images. The thumbnail image generation unit 21, the list image generation unit 22, and the display unit 20 are the same as those in the first embodiment.

  Specifically, the image display position determination unit 110 includes an image data acquisition unit (image acquisition unit) 111, a characteristic value calculation unit (characteristic value calculation unit) 112, a dissimilarity calculation unit (dissimilarity calculation unit) 113, mapping Part (mapping means) 114 is provided.

  The image data acquisition unit (image acquisition unit) 111 is for acquiring image data of a plurality of images together with imaging date / time data. The type of image to be acquired is not particularly limited, and may be, for example, an image captured by a digital camera, an image captured by a scanner, or an artificially generated image. Good.

  The hardware configuration of the image data acquisition unit (image acquisition unit) 111 is not particularly limited as long as image data can be acquired, and input / output capable of reading data from a fixed disk or a removable disk. An interface, an input / output interface that can read data from another communication terminal device via a LAN, an input interface that can read image data from a digital camera, a scanner, or the like may be provided.

The characteristic value calculation unit (characteristic value calculation means) 112 is for calculating an IFS parameter value by fractal coding for each image whose image data has been acquired by the image data acquisition unit 111. The IFS parameters calculated here are x _D , y _D , θ, λ, s, o, e, and f in the first embodiment described above.

  The dissimilarity calculating unit (dissimilarity calculating means) 113 is for calculating the dissimilarity between images for the acquired images. Here, the dissimilarity is calculated for all image combinations. For example, when there are three images A, B, and C, the dissimilarity is calculated between AB, between BC, and between AC.

  The mapping unit (mapping means) 114 is for determining the arrangement of thumbnail images based on the above dissimilarity. More specifically, the display position of each thumbnail image is determined so that the distance between the thumbnail images in the list image is a distance corresponding to the above dissimilarity. For example, assuming that there are three images A, B, and C and their respective thumbnail images a, b, and c, the distance between thumbnail images a and b in the list image, the distance between bc, and a The relationship between the distances between -c and the dissimilarity between images A and B, the dissimilarity between B and C, and the dissimilarity between A and C are matched as much as possible.

  The characteristic value calculation unit 112, the dissimilarity calculation unit 113, and the mapping unit 114 described above are realized by an arithmetic device such as a CPU executing program code stored in a storage device such as a ROM or RAM. It is a functional block.

  Next, a schematic operation of the image list display device 2 will be described. FIG. 18 is a process diagram illustrating the operation of the image list display device 2.

  First, the image data acquisition unit 111 acquires image data from a disk device or a digital camera (S11). The acquired image data is sent to the characteristic value calculation unit 112 and the thumbnail image generation unit 21.

Next, the characteristic value calculation unit 112 calculates an IFS parameter value for each image from which image data has been acquired based on the fractal encoding method (S12). The calculation method of the IFS parameter value is the same as that in the first embodiment described above. In this embodiment, for each image, specific values of x _D , y _D , θ, λ, s, o, e, and f are given for each range block. Calculate the value. The IFS parameter value calculated for each image is sent to the dissimilarity calculation unit 113.

  The dissimilarity calculation unit 113 calculates the dissimilarity between the images based on the IFS parameter value (S13). This dissimilarity is calculated for all image combinations. Details of the method of calculating the dissimilarity will be described later. The calculated dissimilarity is sent to the mapping unit 114.

  Next, the mapping unit 114 determines the display position of the thumbnail images so that the distance between the thumbnail images becomes a distance according to the dissimilarity (S14). Details of the display position determination method will be described later. Information on the determined display position is transmitted to the list image generation unit 22.

On the other hand, the thumbnail image generation unit 21 creates a thumbnail image of each acquired image. The generated thumbnail image data is sent to the list image generation unit 22. Then, the list image generation unit 22 generates a list image in which the thumbnail images of the respective images are arranged at the display positions determined by the mapping unit 14 (S15). The list image data is sent to the display unit 20, and the display unit 20 displays the list image (S16).
With the above processing, it is possible to display a list image in which thumbnail images of similar images are arranged close to each other. Hereinafter, the dissimilarity calculation unit 113 and the mapping unit 114 will be described in detail.

[Dissimilarity calculation unit / dissimilarity calculation process]
The dissimilarity calculation unit 113 calculates the dissimilarity between the images based on the IFS parameter value. The number of IFS parameter value (characteristic value) with each image in this case, eight the number of (i.e. _{x D, y D, θ,} λ, s, o, e, and f) × range block. Using the characteristic value of such an image, the dissimilarity corresponding to the distance between the thumbnail images is calculated in the subsequent mapping step. As described above, the number of characteristic values possessed by each image is plural. There are pieces. Therefore, when calculating the dissimilarity, it is necessary to reduce the dimension from a plurality of characteristic values (that is, a multidimensional vector) to a dissimilarity composed of a single component (that is, a scalar).

  As the method, for example, as in Embodiment 1 described above, a feature vector including each characteristic value as an element is generated, and the dissimilarity between the images is calculated by calculating the Euclidean distance between the feature vectors. May be. However, in this embodiment, a more advanced method described below is used as a preferred embodiment.

  FIG. 17 is a functional block diagram showing details of the dissimilarity calculation unit 113. FIG. 19 is a process diagram showing details of the dissimilarity calculation process. As shown in FIG. 17, the dissimilarity calculation unit 113 includes a characteristic value database 130 provided in a storage unit (not shown), an image selection unit (image selection unit) 131, a reconstruction unit (reconstruction unit) 132, an error value. A calculation unit (error value calculation unit) 133 and an error value averaging unit (error value averaging unit) 134 are provided.

  The characteristic value database 130 stores an IFS parameter value (characteristic value) of each image so that it can be extracted for each image. The characteristic value database 130 is constituted by a RAM or the like.

  The image selection unit 131 is for selecting two images from all the images and acquiring the IFS parameter value of each image from the characteristic value database 130. Note that the image selection unit 131 sequentially selects combinations of two images so that all combinations are selected.

  The reconstruction unit 132 is configured to reconstruct the other image using the IFS parameter value included in one of the two images selected by the image selection unit 131 to create a reconstructed image. is there. Specifically, the second image is reconstructed using the IFS parameter value of the first image to create a second reconstructed image. Conversely, the first image may be reconstructed using the IFS parameter value of the second image to create the first reconstructed image.

  The error value calculation unit 133 calculates an error value between the pre-reconstruction image and the reconstructed image by comparing the luminance values of the pre-reconstruction image and the reconstructed image. The calculation of the error value is performed for each of the first image and the second image. The error value averaging unit 134 is for calculating the dissimilarity by averaging the two types of error values calculated by the error value calculation unit 133.

  The image selection unit 131, the reconstruction unit 132, the error value calculation unit 133, and the error value averaging unit 134 execute program codes stored in a storage device such as a ROM or RAM by an arithmetic device such as a CPU. It is a functional block realized by this.

  In such a dissimilarity calculation unit 113, first, the image selection unit 131 selects two images, and acquires IFS parameter values of the respective images from the characteristic value database 130 (S131). Hereinafter, the two images are referred to as image A and image B, respectively. The acquired IFS parameter values of the image A and the image B are sent to the reconstruction unit 132 together with the image data.

The reconstruction unit 132 that has acquired the IFS parameter value first reconstructs the image B using the IFS parameter value of the image A. Specifically, a domain block corresponding to a certain range block of the image B is determined based on the IFS parameter value (x _D , y _D ) of the image A. Then, the corresponding domain block is affine transformed using the IFS parameter values (θ, λ, s, o, e, f) of the image A, and the transformed domain block is projected onto the range block (S133). This affine transformation is performed using the equations (1) to (3) of the first embodiment described above.

  Next, it is determined whether or not affine transformation has been performed for all range blocks (S133). If not performed for all range blocks, the process returns to step S132 and the same processing is performed for the next range block. Do. On the other hand, if the affine transformation has been performed for all range blocks, the process proceeds to the next step S134. The above affine transformation may be performed only once for each range block, but it is preferable to repeat steps S132 and S133 about 8 to 10 sets. The processing in steps S132 and S133 is called reconstruction.

When an image created by reconstructing the image B based on the IFS parameter value of the image A in this way is an image B ′, the error value calculation unit 133 calculates the luminance between the image B and the image B ′. An error value is calculated (S134). Specifically, an error value is calculated by taking the sum of squares of differences in luminance values of pixels included in the image and dividing the result by the number of pixels to obtain a square root. That is, the image B and the image B ′ are M × N pixels, the luminance value at the coordinates (x, y) of the image B is f _B (x, y), and the luminance value at the coordinates (x, y) of the image B ′ is If f _{B ′} (x, y), the error value E (A → B) is

It can be expressed as.

  In this way, the image B is reconstructed based on the IFS parameter value of the image A, and the luminance error value of the image B before and after the reconstruction is set to E (A → B). Here, an error value E (A → B) when the image B is reconstructed based on the IFS parameter value of the image A, and an error value E when the image A is reconstructed based on the IFS parameter value of the image B (B → A) does not necessarily have the same value. Therefore, it is preferable to calculate error values in both directions.

  Therefore, it is determined whether or not the calculation of the error value has been performed in both directions (S135). If the error value has not been calculated in both directions, the process returns to step S132 and the error value E (B → A) is obtained in the same manner. Is calculated. When both the error value E (A → B) and the error value E (B → A) are calculated, the process proceeds to the next step.

  Next, the error value averaging unit 134 averages the error values E (A → B) and E (B → A) to calculate E (A⇔B). This can be expressed as a formula:

It becomes. Then, this E (A⇔B) is set as the dissimilarity between the images AB.

  As described above, the dissimilarity between the images A and B can be calculated. The dissimilarity is calculated between all the images. That is, when there are only acquired images A, B,..., Z, E (A⇔B), E (A⇔C), E (A⇔D),. In addition, dissimilarities are calculated for all image combinations. Therefore, it is determined whether or not dissimilarities have been calculated for all images (S137). Note that the calculated dissimilarity is sent to the mapping unit 114.

As described above, the dissimilarity calculating unit 113 reconstructs the other image based on the IFS parameter value of one image, and calculates the dissimilarity based on the luminance error value in the images before and after the reconstruction. . Here, when reconstructing the other image based on the IFS parameter value of one image, it is not always necessary to reconstruct the other image based on all the IFS parameter values of the one image. For example, to explain using the above examples of images A and B, when performing affine transformation in image B, θ, λ, s, o, excluding x _D and y _D among IFS parameters necessary for affine transformation. e, used as the image a for f, x _D, it may be used as the image B for y _D. In other words, when the image B is reconstructed, it may be based on some IFS parameters of the image A.

[Mapping unit / Mapping means]
The mapping unit 114 determines the position of the thumbnail image in the list image. At this time, the distance between the thumbnail images on the list image is set to be a distance corresponding to the dissimilarity.

Specifically, as shown in FIG. 20, the mapping unit 114 sets a two-dimensional virtual display area, and further sets coordinates for arranging each thumbnail image. Here, the distance between the image A and the image B on the virtual display area is E ^* (A⇔B). Note that E ^* (A⇔B) can be calculated based on the coordinates in the virtual display area of image A and image B. In this case, if there are only acquired images A, B,..., Z, the position of the image A is Err _A expressed by the following equation:

It is desirable that the position be a position that minimizes. The image B, C, ..., Z _likewise, Err _B, Err c, ..., it is desirable is a position that minimizes the Err _Z.

  As an algorithm for obtaining such an arrangement, various known and commonly used algorithms such as a random search, a hill-climbing method, an annealing method, and a genetic algorithm can be used. For example, when using the hill-climbing method, first, each image is randomly arranged on the virtual display area as an initial state.

Next, the Err _A is calculated for each of the case where the image A is moved to the coordinates of the upper, lower, left, right, upper left, upper right, lower left, and lower right, and the case where the image A is not moved. _The position where the value of _A is the smallest is taken as the position of the next stage image A. The same processing is performed on the image B to the image Z. By repeating such processing a plurality of sets, the position of each image approaches the position reflecting the degree of dissimilarity. Then, this process ends when each image stops moving on the virtual display area. Thereby, the position of each thumbnail image on the virtual display area can be determined.

  An embodiment of the present invention will be described with reference to FIGS. In this example, the following experiment was conducted in order to verify the usefulness of the present invention. In Examples 1 to 3 below, the configuration of Embodiment 1 described above is basically used.

[Example 1]
In this embodiment, an example using an artificially created image is shown. As test images, 20 types of images of 128 × 128 pixels and monochrome 256 gradations obtained from COLUMBIA UNIVERSITY IMAGE LIBRARY (COIL-20) shown in FIG. 10 and objects included in each of these images are 10 °, 20 An image rotated by 70 ° was prepared. These 160 images were input to the image list display device.

In this embodiment, since an artificially created image is used, the image is not classified according to the imaging date and time. Further, the above-described IFS parameters x _D and y _D are used as the characteristic values of each image. Further, in determining the winning element of the self-organizing map, no weight based on the position of the range block was given.

  FIG. 11 is a diagram showing a result of displaying a list of images based on the above conditions. In the figure, images including the same object or similar objects are displayed together. From this, it was verified that the determination of similar images was performed with high accuracy, and that similar images were appropriately arranged close to each other. Note that the high-accuracy determination of the similar image is considered to be caused by the fact that the IFS parameter used as the feature vector in this embodiment is robust against the change of the image accompanying the rotation of the object.

[Example 2]
In this embodiment, an example using a natural image captured by a digital camera is shown. The test image was captured over 4 days, and 267 24-bit color images with an image size of 512 × 384 pixels or 567 × 426 pixels were used.

In the present embodiment, the images are classified into groups divided every hour based on the imaging date and time. As the characteristic values, the values of the IFS parameters x _D and y _D were used as in the first embodiment. Here, the range block and the domain block have a size proportional to the captured image size. Further, since the present embodiment uses a color image _was determined x D, the value of _{y D} color components Y, Cb, each Cr. The number of elements arranged in the self-organizing map was 3 × 11, and no weight based on the position of the range block was given in the determination of the winning element.

  FIG. 12 is a diagram showing a part of the result of displaying a list of images based on the above conditions. In the figure, many images similar to each other are collected in each group. In addition, FIG. 13 shows the result of selecting and enlarging three groups. As shown in FIG. 13, it can be seen that similar images are displayed together in the same group. Further, since this tendency was not related to the size (resolution) of the input image, it was verified that it was effective even for images having different sizes (resolution).

Example 3
In this example, 101 images taken in one day among the images used in Example 2 were used. The characteristic value, IFS parameter _x D, in addition to the _{y D,} above ^{_{f ave Ri, f ave Dopt,}} E RMS was also used. Further, when determining the winning element in learning of the self-organizing map, as described in the above-described embodiment, each characteristic value is given a weight based on the distance from the center of the image to the range block. . The number of range blocks is 4 × 4 = 16, and σ in equation (10) is 1.

FIG. 14 is a diagram showing images displayed as a list. For reference, the results displayed as a list under the conditions of Example 2 are shown in FIG. Comparing FIG. 14 with FIG. 15, it can be seen that similar images are arranged closer to each other. Therefore, it has been verified that the accuracy of determination of similar images becomes more accurate by adding f ^ave _Ri , f ^ave _Doppt , and E _RMS as characteristic values and performing weighting when determining a winning element.

  The present invention is not limited to the above-described embodiments and examples, and various modifications can be made within the scope of the claims. That is, embodiments obtained by combining technical means appropriately modified within the scope of the claims are also included in the technical scope of the present invention.

  According to the present invention, it is possible to cluster digital images taken by an individual with a digital camera or the like based on the similarity of images and obtain a list image in which similar images are arranged close to each other. Therefore, it can be suitably used for personal photo album utilities and the like.

1, showing an embodiment of the present invention, is a functional block diagram illustrating a main configuration of an image list display device. FIG. BRIEF DESCRIPTION OF THE DRAWINGS FIG. 1 is a process diagram illustrating a processing process of an image list display device according to an embodiment of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS FIG. 1, showing an embodiment of the present invention, is a diagram illustrating a scheme of a display method of an image list display device. 1, showing an embodiment of the present invention, is a functional block diagram illustrating a detailed configuration of a characteristic value calculation unit in FIG. 1. FIG. 3 is a diagram illustrating a fractal image encoding scheme according to an embodiment of the present invention. FIG. 3 is a diagram illustrating a scheme of a self-organizing map according to an embodiment of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS FIG. 1 illustrates an embodiment of the present invention, and is a diagram illustrating an example of range block division and coordinates of each range block. It is a figure which shows one Embodiment of this invention and shows the example of the diagonal matrix W used for weighting. FIG. 4 is a diagram illustrating an embodiment of the present invention and a group based on a distribution of imaging locations and imaging locations. 3 is a diagram illustrating a monochrome image group used in Example 1. FIG. It is a figure which shows the result of having performed the list display in Example 1. FIG. It is a figure which shows a part of result of having performed the list display in Example 2. FIG. It is the figure which extracted a part of result of having performed the list display in Example 2, and expanded and displayed it. In Example 3, it is a figure which shows the result of the list display at the time of weighting while increasing a characteristic value. In Example 3, it is a figure which shows the result of having performed the list display on the conditions of Example 2. FIG. 1, showing an embodiment of the present invention, is a functional block diagram illustrating a main configuration of an image list display device. FIG. FIG. 17, showing an embodiment of the present invention, is a functional block diagram illustrating a detailed configuration of a dissimilarity calculation unit in FIG. 16. BRIEF DESCRIPTION OF THE DRAWINGS FIG. 1 is a process diagram illustrating a processing process of an image list display device according to an embodiment of the present invention. FIG. 19 illustrates an embodiment of the present invention and is a diagram illustrating a detailed processing process of step S13 in FIG. 18. FIG. 3 is a diagram illustrating a virtual display area according to an embodiment of the present invention.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Image list display apparatus 2 Image list display apparatus 10 Image display position determination part (image display position determination apparatus)
11 Image data acquisition unit with imaging date and time data (image acquisition unit, imaging environment data acquisition unit)
12 characteristic value calculation unit 13 feature vector generation unit (feature vector generation means)
14 Self-organizing mapping unit (self-organizing mapping means)
15 Image placement unit (image placement means)
16 Non-hierarchical classification (non-hierarchical classification means)
31 Range block creation unit (range block creation means)
32 Domain block creation part (domain block creation means)
33 IFS parameter calculation unit (coordinate calculation means)
34 Range block luminance calculation unit (range block luminance calculation means)
35 Domain block luminance calculation section (domain block luminance calculation means)
36 Dissimilarity calculation unit (dissimilarity calculation means)
110 Image display position determination unit (image display position determination device)
111 Image data acquisition unit (image acquisition unit)
112 characteristic value calculation unit (characteristic value calculation means)
113 Dissimilarity calculation unit 114 Mapping unit (mapping means)
131 Image selection unit (image selection means)
132 Reconfiguration unit (reconfiguration means)
133 Error value calculation unit (error value calculation means)

Claims (16)

An image display position determination device for determining a display position of each thumbnail image in a list image when displaying a list of a plurality of images as thumbnail images,
An image acquisition unit for acquiring an original image as the plurality of images;
For each original image, a range block creating means for creating a range block by dividing the original image,
For each original image, a domain block creating means for creating a plurality of domain blocks by cutting out a plurality of partial images while allowing overlap with each other,
In each original image, coordinate calculation means for calculating the coordinates in the original image of the domain block most similar to the range block as a first characteristic value representing the characteristics of the original image;
Feature vector generation means for generating, for each original image, a feature vector including the first characteristic value calculated for each original image by the coordinate calculation means as an element;
Self-organizing mapping means for learning a self-organizing map using the feature vectors generated for a plurality of original images and determining the position of an element to which each original image belongs in the self-organizing map after learning; ,
Image placement means for determining the display position of each thumbnail image in the list image based on the position of the element to which each original image belongs in the self-organizing map;
An image display position determining device comprising: In each original image, further comprising range block luminance calculation means for calculating an average value of luminance of each pixel included in the range block as a second characteristic value representing the characteristic of the original image,
The feature vector generation means further includes, as an element, the second characteristic value calculated for each original image by the range block luminance calculation means in the feature vector generated for each original image. Item 2. The image display position determination device according to Item 1. In each original image, further comprising a domain block luminance calculating means for calculating an average value of luminance of each pixel included in the domain block most similar to the range block as a third characteristic value representing the characteristic of the original image,
The feature vector generation means further includes, as an element, the third characteristic value calculated for each original image by the domain block luminance calculation means in the feature vector generated for each original image. Item 3. The image display position determination device according to Item 1 or 2. In each original image,
(A) the above range block;
(B) a size-adjusted domain block based on a domain block most similar to the range block and having the same size as the range block;
Further comprising a dissimilarity calculating means for calculating the dissimilarity as a fourth characteristic value representing the characteristics of the original image,
The feature vector generation means further includes a fourth characteristic value calculated for each original image by the dissimilarity calculation means as an element in the feature vector generated for each original image. Item 4. The image display position determination device according to any one of Items 1 to 3.   5. The image display position determining apparatus according to claim 1, wherein the domain block creating unit creates a domain block having the same size as the range block. 6. The range block creating means creates a range block having a size substantially proportional to the size of each original image,
The domain block creation means creates a domain block by cutting out a partial image having a size substantially proportional to the size of each original image while shifting by the number of pixels substantially proportional to the size of each original image. The image display position determination apparatus according to claim 1, wherein the image display position determination apparatus is a display apparatus for determining an image display position. The feature vector generated for each original image includes the characteristic value calculated for a plurality of different range blocks as an element,
When the self-organizing mapping means searches for a winning element in learning of the self-organizing map, the original image of the range block is calculated with respect to the characteristic value calculated for each range block included in the feature vector. The image display position determination apparatus according to claim 1, wherein weighting is performed based on a position within the image display position.   8. The image display position determining apparatus according to claim 7, wherein the weighting is in accordance with a distance between the position of the range block and the center of the image. The image acquisition unit further acquires subject position data indicating the position of the subject in the image,
8. The image display position determination according to claim 7, wherein the weighting is in accordance with a distance between the position of the range block and the position of the subject in the image indicated by the subject position data. apparatus. The original image is created by imaging,
An imaging environment data acquisition unit that acquires imaging environment data indicating an environment when the original image is captured;
Non-hierarchical classification means for classifying the original image into a plurality of non-hierarchical groups based on the imaging environment data,
The self-organizing mapping means performs learning of the self-organizing map and determination of the position of the element to which the original image belongs for each group,
The image arrangement means divides the list image into partial areas corresponding to the group, and the display position of the thumbnail image in the partial area is determined for each group based on the position of the element to which the original image belongs in the self-organizing map. The image display position determination device according to claim 1, wherein the display position of each thumbnail image in the list image is determined by determining. An image display position determination device for determining a display position of each thumbnail image in a list image when displaying a list of a plurality of images as thumbnail images,
An image acquisition unit for acquiring an original image as the plurality of images;
For each original image, characteristic value calculating means for calculating an IFS parameter value by a fractal image encoding method,
Image selecting means for sequentially selecting a combination of two original images from a plurality of original images;
Error value calculating means for calculating the dissimilarity between the two original images based on the IFS parameter value for the two original images selected by the image selecting means;
Mapping means for determining a display position of each thumbnail image in the list image so that a distance in the list image between the thumbnail images is a distance according to the dissimilarity between the original images corresponding to the thumbnail images; ,
An image display position determining device comprising: Reconstructing means for reconstructing a reconstructed image by reconstructing the other original image using the IFS parameter value of one of the two original images selected by the image selecting means;
12. The error value calculating means calculates a dissimilarity between the two original images based on a luminance error value between the other original image and the reconstructed image. The image display position determination device described.   The image display position determination program for operating a computer as each means of any one of Claims 1-12.   A computer-readable recording medium on which the image display position determination program according to claim 13 is recorded. An image display position determination method for determining a display position of each thumbnail image in a list image when displaying a list of a plurality of images as thumbnail images,
An image acquisition step of acquiring an original image that is the plurality of images;
For each original image, a range block creation step of creating a range block by dividing the original image,
For each original image, a domain block creating step for creating a plurality of domain blocks by cutting out a plurality of partial images while allowing overlap with each other,
In each original image, a coordinate calculation step for calculating the coordinates in the original image of the domain block most similar to the range block as a first characteristic value representing the characteristics of the original image;
A feature vector generation step for generating, for each original image, a feature vector including the first characteristic value calculated for each original image in the coordinate calculation step as an element;
A self-organizing mapping step of learning a self-organizing map using the feature vectors generated for a plurality of original images and determining the position of an element to which each original image belongs in the self-organizing map after learning. ,
An image placement step for determining a display position in the list image of each thumbnail image based on the position of the element to which each original image belongs in the self-organizing map;
The image display position determination method characterized by including. An image display position determination method for determining a display position of each thumbnail image in a list image when displaying a list of a plurality of images as thumbnail images,
An image acquisition step of acquiring an original image that is the plurality of images;
For each original image, a characteristic value calculating step for calculating an IFS parameter by a fractal image encoding method,
An image selection step of sequentially selecting a combination of two original images from a plurality of original images;
An error value calculating step of calculating a dissimilarity between the two original images based on the IFS parameter for the two original images for which the combination is selected in the image selection step;
A mapping step for determining a display position of each thumbnail image in the list image so that a distance in the list image between the thumbnail images is a distance according to the dissimilarity between the original images corresponding to the thumbnail images; ,
The image display position determination method characterized by including.