JP2009169533A - Image arrangement data generation device and image arrangement data generation method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To generate image arrangement data for displaying image map which facilitates an image retrieval. <P>SOLUTION: An image arrangement processing part 2004 retrieves featured value vectors on an SOM model whose Euclidian distance to featured value vectors extracted from an input image by a featured value extraction part 2003 is the minimum, and generates image arrangement information 2011 in which the image identification information of the input image is recorded at lattice points corresponding to the retrieved featured value vectors. An image rearrangement processing part 2006 applies consecutive sequences to the image identification information and "emptiness" information meaning that any image is not arranged to the image arrangement information 2011 while spirally tracing the lattice points, and rearranges the image by successively re-associating the image identification information and the "emptiness" information with the lattice points according to the applied sequences while spirally tracing the lattice points, and generates image arrangement data configured of the set of the pairs of the image identification information and the lattice point coordinates. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to a technical field for image search / browsing, and more particularly, to an apparatus and method for generating image arrangement data for reducing and displaying a large number of images on a screen in an easily searchable manner.

  When a large number of images are reduced on the screen and a target image is searched for on the screen, it is not easy to find the target image if the images are arranged randomly. It is desirable to control the arrangement of images so that a target image can be easily found.

  For example, a method of arranging images with high similarity so as to be close to each other using a self-organizing map (SOM) can be considered (see Patent Document 1). In this way, the user can search the target image more efficiently by tracing the images with high similarity.

  In addition, the multidimensional feature values of the image are aggregated into a two-dimensional feature value, the two-dimensional feature value is regarded as a metric space, the images are clustered, and a partial area of the screen is assigned to each cluster, and the image is reduced and displayed. A method is conceivable (see Patent Document 2). Since images having similar feature amounts are arranged in the same partial area, the target image can be searched for more efficiently than in the case where images are displayed in random order.

Japanese Patent No. 3614235 JP 2005-235041 A

  For example, when images are arranged using a self-organizing map, a plurality of images may be arranged in duplicate at the same position. In Patent Document 1, a depth is added to a two-dimensional space, and images arranged at the same position in the two-dimensional space are rearranged by shifting the depth so that an image arranged at the same position in the two-dimensional space is changed. It doesn't overlap and disappear. However, such a method cannot be employed for the purpose of displaying a simple image map in which reduced images (thumbnails) of images are arranged two-dimensionally.

  In addition, if images overlap as a result of placing images at two-dimensional space grid points using a self-organizing map, the images are rearranged between the grid points to avoid the overlap. It is also possible to do. However, such a rearrangement is difficult if many images are arranged at the same grid point. In addition, the interval between the grid points is selected so that the thumbnails of the images arranged at the adjacent grid points do not overlap, but in order to rearrange the images between the grid points, processing such as reducing the size of the thumbnails is necessary. As a result, the processing cost increases.

  Accordingly, an object of the present invention is to provide an improved apparatus and method for generating image arrangement data for displaying an image map arranged two-dimensionally so that thumbnails of a large number of images can be efficiently searched. It is to provide.

An image arrangement data generation device according to the invention of claim 1 is provided.
An image input means for inputting an image;
Feature quantity extraction means for extracting the feature quantity of each image input by the image input means;
Based on the feature amount extracted by the feature amount extraction unit, each image input by the image input unit is arranged at a grid point in the two-dimensional space so that a similar image is closer, and the image is arranged at the grid point. Image arrangement information generating means for generating image arrangement information in which image identification information of an image or “empty” information meaning that no image is arranged at a grid point is recorded in association with the grid point;
A first process for assigning a sequential order to image identification information and “empty” information while tracing lattice points in a predetermined order with respect to the image arrangement information generated by the image arrangement information generating unit; and the predetermined order The image is rearranged by following the grid points and re-allocating the image identification information and the “empty” information to the grid points in order according to the order given in the first process. Image relocation processing means for performing a second process of generating image arrangement data consisting of a set of pairs of image identification information and position information of lattice points associated with the image identification information;
It is characterized by having.

  According to a second aspect of the present invention, in the image arrangement data generating apparatus according to the first aspect of the invention, the image rearrangement processing means follows lattice points in a spiral shape in the first and second processes. .

  According to a third aspect of the present invention, in the image arrangement data generating device according to the first aspect of the invention, the image rearrangement processing means performs raster scan for each divided region of the two-dimensional space in the first and second processes. It is to follow the lattice points in order.

According to a fourth aspect of the present invention, in the image arrangement data generating apparatus according to the first aspect of the present invention, each image input by the image input unit is based on the feature amount extracted by the feature amount extraction unit. It further has a clustering means for classifying into a plurality of clusters,
The image arrangement information generating means performs arrangement of each image on a grid point in a two-dimensional space to generate image arrangement information for each cluster,
The image rearrangement processing means performs the first processing for each cluster, and in the second processing, the image for each cluster is traced in the predetermined order from the starting lattice point set for each cluster. The image information is rearranged by associating the identification information and the “empty” information with the grid points in order according to the order given in the first process.

  According to a fifth aspect of the present invention, in the image arrangement data generating apparatus according to the fourth aspect of the invention, the image rearrangement processing means traces lattice points in a spiral shape in the first and second processes. .

  According to a sixth aspect of the present invention, in the image arrangement data generating apparatus according to the fourth aspect of the invention, the image rearrangement processing means follows the grid points in the raster scan order in the first and second processes. .

  According to a seventh aspect of the present invention, in the image arrangement data generating device according to the fourth aspect of the invention, the image rearrangement processing unit assigns an order only to the image identification information in the first processing, and the first In the second process, the image is rearranged by re-associating only the image identification information with the lattice points.

An image arrangement data generation method according to the invention of claim 8 is provided.
An image input process for inputting an image;
A feature amount extraction step of extracting a feature amount of each image input by the image input step;
Based on the feature amount extracted in the feature amount extraction step, each image input in the image input step is arranged at a grid point in the two-dimensional space so that a similar image is closer, and the image is placed at the grid point. An image arrangement information generating step for generating image arrangement information in which image identification information of an image or “empty” information meaning that no image is arranged at a grid point is recorded in association with the grid point;
A first process for assigning a sequential order to image identification information and “empty” information while following lattice points in a predetermined order with respect to the image arrangement information generated in the image arrangement information generation step; and the predetermined order The image is rearranged by following the grid points and re-allocating the image identification information and the “empty” information to the grid points in order according to the order given in the first process. An image rearrangement process step of performing a second process of generating image arrangement data composed of a set of pairs of image identification information and position information of lattice points associated with the image identification information;
It is characterized by having.

  According to a ninth aspect of the present invention, in the image arrangement data generation method according to the eighth aspect of the invention, the image rearrangement processing step traces lattice points spirally in the first and second processes. .

  According to a tenth aspect of the present invention, in the image arrangement data generation method according to the eighth aspect of the invention, the image rearrangement processing step performs a raster scan for each divided region of the two-dimensional space in the first and second processes. It is to follow the lattice points in order.

The invention according to claim 11 is the image layout data generation method according to claim 8,
A clustering step of classifying each image input in the image input step into a plurality of clusters based on the feature amount extracted in the feature amount extraction step;
The image arrangement information generation step performs arrangement of the image at a lattice point in the two-dimensional space for each cluster to generate image arrangement information for each cluster,
In the image rearrangement processing step, the first processing is performed for each cluster, and in the second processing, the image for each raster is obtained while tracing the lattice points in the predetermined order from the starting lattice points set for each cluster. The image information is rearranged by associating the identification information and the “empty” information with the grid points in order according to the order given in the first process.

  According to a twelfth aspect of the present invention, in the image arrangement data generation method according to the eleventh aspect of the invention, the image rearrangement processing step traces lattice points spirally in the first and second processes. .

  According to a thirteenth aspect of the present invention, in the image arrangement data generation method according to the eleventh aspect of the invention, the image rearrangement processing step follows the grid points in the raster scan order in the first and second processes. .

  According to a fourteenth aspect of the present invention, in the image arrangement data generation method according to the eleventh aspect of the invention, the image rearrangement processing step assigns an order only to image identification information in the first processing, and the first In the second process, only the image identification information is associated with the grid points again.

  According to the invention described in any one of claims 1 to 3 or claims 8 to 10, when a plurality of images are arranged in duplicate at the same grid point in the image arrangement information generation means or process, the image rearrangement processing means or process The images are rearranged so as not to overlap. The arrangement relationship of the images does not change so much before and after the rearrangement. Therefore, when an image map is displayed according to the image arrangement data generated by the invention according to claims 1 to 3 or claims 8 to 10, thumbnails of similar images are arranged at a short distance, and a plurality of images are displayed. Since the thumbnails do not overlap, the target image can be efficiently found on the image map.

  According to the invention described in claims 4 to 6 or claims 11 to 13, images are arranged for each cluster by the image arrangement information generating means or process, but a plurality of images in the cluster overlap with the same grid point. When the images are arranged, the images are rearranged by the image rearrangement processing means or process so that the images do not overlap in the cluster. The arrangement relationship of the images in the cluster does not change so much before and after the rearrangement. Further, the rearrangement of the images in each cluster is performed while tracing the lattice points from the starting point lattice points set for each cluster. Therefore, when an image map is displayed according to the image arrangement data generated by the invention according to any one of claims 4 to 6 or claims 11 to 13, thumbnails of images are arranged for each cluster, and thumbnails of similar images are arranged in the cluster. Are arranged at a close distance, and thumbnails of a plurality of images do not overlap, so that a target image can be found efficiently on the image map.

  According to the invention described in claim 7 or 14, images are arranged for each cluster by the image arrangement information generating means or process, but when a plurality of images in the cluster are arranged at the same grid point, Images in the cluster are rearranged so as not to overlap by the rearrangement processing means or process. However, since the images in the cluster are rearranged in a dense state, the arrangement relationship of the images in the cluster changes considerably before and after the rearrangement. Further, the rearrangement of the images in each cluster is performed while tracing the lattice points from the starting point lattice points set for each cluster. Therefore, when the image map is displayed according to the image arrangement data generated by the invention according to claim 7 or 14, the thumbnails of the images are arranged in a clustered manner for each cluster, and the thumbnails of a plurality of images overlap. Therefore, the target image can be efficiently found on the image map. Further, since the images of each cluster are arranged densely, more thumbnails of images can be displayed on the image map.

  FIG. 1 is a block diagram showing an example of an image processing system to which the present invention is applied. The image processing system includes an image storage 1000, an image arrangement data server 1001, an image distribution server 1002, and a client 1003.

  The image storage 1000 is a device that stores a large number of images. The image arrangement data server 1001 generates image arrangement data used when a large number of images stored in the image storage 1000 are reduced and displayed on the client 1003, and the image arrangement data is sent to the client in response to a request from the client 1003. This is a device that transmits to 1003. The image distribution server 1002 is a device that transmits an image stored in the image storage 1000 or a reduced image (thumbnail) thereof to the client 1003 in response to a request from the client 1003. A client 1003 is a device for browsing images, and includes an image display device and other user interface means. Specifically, the client 1003 is, for example, a personal computer on which a client application operates.

  The image layout data server 1001 determines the layout of each image based on the input feature amount of the image input unit 1005 that inputs the image stored in the image storage 1000, and image identification information (for example, a file name). The image arrangement data generating unit 1006 that generates image arrangement data including the position information, the image arrangement data storage unit 1007 that stores the generated image arrangement data, and the image arrangement data storage unit 1007 in response to a request from the client 1003 An image arrangement data request processing unit 1008 for transmitting the stored image arrangement data to the client 1003 is provided.

  Here, operations related to image browsing on the client 1003 will be described. The client 1003 receives image arrangement data from the image arrangement data server 1001 by transmitting an image arrangement data request to the image arrangement data server 1001. The client 1003 transmits a thumbnail distribution request including all image identification information in the image arrangement data to the image distribution server 1002. The image distribution server 1002 reads an image corresponding to each piece of image identification information included in the thumbnail distribution request from the image storage 1000, creates the thumbnail, and transmits the thumbnail to the client 1003. If image thumbnails are also stored in the image storage 1000, the image distribution server 1002 reads the thumbnails from the image storage 1000 and transmits them to the client 1003.

  The client 1003 displays on the screen of the image display device an image map as schematically shown in FIG. 2 in which the thumbnails received from the image distribution server 1002 are arranged according to the position information in the image arrangement data. In FIG. 2, each hatched rectangle is a thumbnail of one image. As will be described later, in the image map, thumbnails of similar images are arranged at positions close to each other, and the thumbnails of a plurality of images do not overlap at the same position. Therefore, the user of the client 1003 wants to view images on the screen. Can be found efficiently.

  When the user designates a thumbnail of an image to be viewed on the image map, the client 1003 transmits an image distribution request including image identification information corresponding to the image to the image distribution server 1002. The image distribution server 1002 reads an image corresponding to the image identification information included in the received image distribution request from the image storage 1000 and transmits the image to the client 1003. The client 1003 displays the received image on the screen of the image display device, and the user can view the image.

  Next, a detailed embodiment of the image arrangement information generation unit 1006 in the image arrangement data server 1001 will be described.

<First Embodiment of Image Arrangement Data Generation Unit 1006>
FIG. 3 is a block diagram for explaining the first embodiment of the image arrangement data generation unit 1006. An image arrangement data generation unit 1006 according to this embodiment uses a self-organizing map (SOM) for image arrangement processing, and includes an SOM model generation unit 2001, an SOM model storage unit 2002, a feature amount extraction unit 2003, an image, and the like. The image processing apparatus includes an arrangement processing unit 2004, an image arrangement information storage unit 2005, an image rearrangement processing unit 2006, and a control unit 2007.

  First, a configuration related to an operation of generating an SOM model used for image arrangement will be described.

  Under the control of the control unit 2007, m sample images stored in the image storage 1000 are input through the image input unit 1005. Each input sample image is sent to the feature amount extraction unit 2003 via the control unit 2007. The feature quantity extraction unit 2003 extracts n-dimensional feature quantities such as saturation, hue, luminance, and texture from each sample image, and outputs an n-dimensional feature quantity vector composed of these feature quantities. This feature vector is input to the SOM model generation unit 2001 via the control unit 2007. Finally, m feature quantity vectors are input to the SOM model generation unit 2001.

  In the SOM model generation unit 2001, an SOM learning algorithm is executed using the input m feature quantity vectors, and an SOM model 2010 is generated on the SOM model storage unit 2002.

  The SOM model 2010 is data in which m feature amount vectors are arranged without overlapping with lattice points in a two-dimensional space so that the smaller the Euclidean distance between them, the closer. More specifically, the SOM model 2010 has, for example, an entry corresponding to a lattice point in a two-dimensional space, and table-structured data in which feature quantity vectors arranged at the lattice point are written in the entry corresponding to the lattice point. It can be. FIG. 4 is a conceptual diagram of the SOM model 2010, in which circles in the figure indicate lattice points in a two-dimensional space, and arrow lines in the circles indicate feature vectors arranged at the lattice points.

  Note that once the SOM model is generated, the SOM model generation operation is not executed unless there is a situation where the SOM model needs to be regenerated. In addition, an SOM model prepared in advance may be stored in the SOM model storage unit 2002. In this case, it is not necessary to provide the SOM model generation unit 2001.

  Next, an image arrangement data generation operation and a related configuration will be described. FIG. 5 shows a schematic processing flow of the image arrangement data generation operation.

  Step S101: One image stored in the image storage 1000 is input to the image arrangement data generation unit 1006 by the image input unit 1005 together with the image identification information (for example, file name). Note that the image to be input may be the entire image in the image storage 1000 or the partial image in the image storage 1000 narrowed down by, for example, keyword search or category designation. In any case, an image that can be browsed by the client 1003 is an input target.

  Step S102: The input image is transferred to the feature amount extraction unit 2003 through the control unit 2007. The feature quantity extraction unit 2003 extracts n-dimensional feature quantities such as the saturation, hue, luminance, and texture of the image, and outputs an n-dimensional feature quantity vector composed of these feature quantities. The extracted feature quantity is of the same type as that extracted when the SOM map is generated.

  Step S103: The feature quantity vector output from the feature quantity extraction unit 2003 is transferred to the image arrangement processing unit 2004 through the control unit 2007 together with the image identification information of the image. The image arrangement processing unit 2004 searches for a feature vector having the shortest Euclidean distance from the feature vector extracted from the image from the feature vector group on the SOM model 2010, and finds a lattice point where the feature vector is arranged. The image arrangement information 2011 is determined in the image arrangement information storage unit 2005 by determining the lattice point where the image is arranged and recording the image identification information of the image in association with the lattice point. Specifically, the image arrangement information 2011 has, for example, a table structure having an entry associated with each grid point in a two-dimensional space in a one-to-one manner. Image identification information of an image arranged at a certain grid point is It is written in the entry corresponding to the lattice point. The content of the entry corresponding to the grid point where no image is arranged is information indicating that no image is arranged (hereinafter referred to as “empty” information).

  If the feature quantity vectors of a plurality of images are very similar, the feature quantity vectors on the SOM model having the smallest Euclidean distance from each feature quantity vector are the same, and the plurality of images are duplicated and the same. May be placed at grid points. In this case, the image identification information of the plurality of images is written in the entry of the same lattice point.

  Step S104: The control unit 2007 determines whether or not the last image to be input has been input. If there is an image to be input, the control unit 2007 requests the image input unit 1005 to input the next image. Control is performed to execute the processing of steps S101 to S103 on the image. When it is determined that the last image has been input, the control unit 2007 instructs the image rearrangement processing unit 2006 to execute the image rearrangement process (the process of step S105).

  Step S105: When images are arranged using the SOM model, as described above, two or more images may be arranged in duplicate at the same grid point. When an image map is displayed according to such image arrangement information, the thumbnails of two or more images are overlapped, and only the thumbnail of the image displayed at the top is invisible. The image rearrangement process is a process for generating final image arrangement data by rearranging images so as to eliminate such overlapping of thumbnails of a plurality of images.

  In addition, after completing the above series of processing, when a new image to be viewed is added to the image storage 1000, or some images accumulated in the image storage 1000 are newly added to the viewing target. In this case, the image arrangement information 2011 in the image arrangement information storage unit 2005 is updated by performing the processing of steps S101 to S104 for the new browsing target image. Then, the image rearrangement process of step S105 is executed on the updated image arrangement information 2011, and updated image arrangement data is generated. Note that the arrangement relationship of the images on the image map displayed according to the updated image arrangement data does not change much from the image arrangement relationship before the update, and thus the user can use the knowledge of the image arrangement before the update.

  Next, the image rearrangement process in step S105 will be described. FIG. 6 shows a schematic processing flow of the image rearrangement process. In step S106, a process (process A) is performed on the image arrangement information 2011, with a sequential order added to the image identification information and the “empty” information while following the lattice points in a predetermined order. In step S107, the image points are rearranged by re-associating the image identification information and the “empty” information with the lattice points in order according to the order given thereto while tracing the lattice points in the same order as in the process A. A process (process B) is performed to generate final image arrangement data composed of a set of pairs of later image identification information and coordinates (position information) of lattice points associated with the image identification information. Hereinafter, the process A and the process B will be specifically described.

(First embodiment)
According to the first embodiment, in the process A, the image arrangement information 2011 continues to the recording information corresponding to the lattice points while tracing the lattice points spirally starting from the lattice point at the center of the two-dimensional space. Processing to assign the order is performed. Here, the recording information corresponding to the lattice point is image identification information of one or more images arranged at the lattice point, or “empty” information meaning that no image is arranged.

  After such ordering, in processing B, the image identification information and “empty” information are given to the process while tracing the lattice points in a spiral shape starting from the lattice point at the center of the two-dimensional space as in the ordering. The image is rearranged by re-associating with the lattice points in the order in which they are arranged, and the final image arrangement data comprising a set of image identification information after the rearrangement and the coordinates of the coordinates of the lattice points with which the images are associated Is generated.

  For example, it is assumed that the image arrangement information 2011 has contents as shown in FIG. In FIG. 7, circles correspond to lattice points, and the inside of each circle indicates recording information corresponding to the lattice points. That is, alphabets such as “a” and “b” inside the circle indicate image identification information, and a character “empty” inside the circle indicates “empty” information.

  Here, if the lattice point on which the image having the image identification information a is arranged is the lattice point at the center of the two-dimensional space, the lattice points are spirally arranged in the order indicated by the arrow line from the lattice point as the starting point. As shown in FIG. 7, a sequential order is given to the recording information (image identification information or “empty” information) corresponding to each lattice point.

  The image is rearranged by associating the image identification information and the “empty” information with the lattice points in order according to the order given to the lattice points in a spiral shape starting from the lattice point at the center of the two-dimensional space. The result is as shown in FIG. That is, f in the image identification information e and f of two images arranged in duplicate on the same grid point is re-associated with the next grid point, and the “empty” information ahead is also one by one. Since the images are sequentially fed, the image identification information g is also associated with the next grid point. A set of pairs of image identification information and grid point coordinates (position information) after rearranging images in this way is generated as final image arrangement data, and this is stored in the image arrangement data storage unit 1007.

  As can be seen by comparing FIG. 7 and FIG. 8, the image rearrangement process causes the image arrangement to be spiraled by the amount of the overlapping image, but the overall image arrangement characteristic before the image rearrangement is characteristic. Is almost maintained.

  An example of the processing flow of processing A and processing B is shown in FIGS. 9 and 10, respectively.

  First, the process A shown in FIG. 9 will be described. A lattice point at the center of the two-dimensional space is set as a starting lattice point (step S111), and it is checked whether or not the recording information of the lattice point is image identification information (step S112). When an image is arranged at the lattice point, the recorded information is image identification information (for example, a file name), and is added to the queue 1 (step S113). When a plurality of images are arranged at the lattice points, a plurality of pieces of image identification information are recorded, so that each piece of image identification information is added to the queue 1 in order. If no image is arranged at the grid point, the recorded information, that is, “empty” information is added to the queue 1 (step S114).

  It is determined whether or not the grid point is the last grid point (step S115). If the grid point is not the last grid point (No in step S115), a grid point moved by one spiral from the grid point is set (step S116), and the process from step S112 is executed again on the grid point. To do. Similar processing is performed up to the last grid point.

  In the example of FIG. 7, the contents of the queue 1 after processing are as shown in FIG. The number of each entry in the queue 1 corresponds to the order given to the image identification information or “empty” information recorded in each entry.

  Next, processing B shown in FIG. 10 will be described. First, 1 is set to a lattice point counter that counts the number of lattice points that have spirally traced from the central lattice point (step S121). A lattice point counter value of 1 means that the lattice point is the central lattice point that is the starting point. Since the order in which the lattice points are traced is determined, the lattice points can be identified from the value of the lattice point counter, and the coordinates (position information) of the lattice points can be calculated.

  One piece of data is extracted from the head of the queue 1 (step S122), and it is determined whether or not the data is image identification information (step S123).

  If it is image identification information (step S123, Yes), the coordinates of the lattice point are calculated based on the value of the lattice point counter (step S124), and a pair of the lattice point coordinates (position information) and the image identification information. Is added to the queue 2 (step S125). If the data extracted from the queue 1 is “empty” information (No at Step S123), Steps S124 and S125 are skipped.

  The value of the grid point counter is incremented by 1 (step S127), the process returns to step S122, the next data is extracted from the queue 1, and the same processing is performed. When the above processing is performed up to the last data in the queue 1 (step S126, Yes), the processing is terminated.

  In the example of FIG. 7, the contents of the queue 2 are as shown in FIG. As is clear from the comparison between FIG. 12 and FIG. 8, the contents of the queue 2 are composed of a set of pairs of image identification information after rearranging the images and their lattice point coordinates as shown in FIG. The contents of the queue 2 are stored in the image arrangement data storage unit 1007 as final image arrangement data.

  Note that the process A in FIG. 9 and the process B in FIG. 10 can be integrated. In other words, the lattice points are traced spirally from the central lattice point, and each image identification information is assigned to the image identification information and “empty” information recorded in correspondence with each lattice point while giving a consecutive number. The coordinates of the grid point corresponding to the number assigned are calculated, and the grid point coordinates are paired with the image identification information and stored in the queue. “Empty” information is not stored in the queue. In the case of such processing, the contents of the queue are directly used as final image arrangement data after rearrangement.

(Second embodiment)
According to the second embodiment, in the process A, the image arrangement information 2011 is subjected to a process of assigning a sequential order to the image identification information and the “empty” information as in the first embodiment. However, in this second embodiment, the two-dimensional space is divided into a plurality of areas, for example, vertically and horizontally, and image identification information and A continuous order is assigned to the “empty” information. Then, for each divided region, the image identification information and the “empty” information are sequentially associated with the lattice points in the order given to the image while tracing the lattice points in the raster scan order from the lattice point in the upper left corner. Are rearranged to generate final image arrangement data.

  For example, it is assumed that the image arrangement information 2011 has contents as shown in FIG. 13 in a certain divided area. As is clear from the comparison between FIG. 13 and FIG. 7, the image arrangement in the divided area is the same as that in FIG.

  Here, as the grid points are traced from the grid point in the upper left corner of the divided area in the raster scan order, the recording information corresponding to each grid point, that is, the image identification information or “empty” information is given a sequential order as shown in FIG. To do.

Next, in process B, the image is re-established by associating the image identification information and the “empty” information with the lattice points in the order given to them while tracing the lattice points in the raster scan order from the lattice point in the upper left corner. The arrangement result is as shown in FIG.
As can be seen by comparing FIG. 14 and FIG. 13, the rearrangement process sequentially forwards the image layout by the amount of the redundantly arranged images in the raster scan order. Is almost maintained.

  The processing A and the processing B for each divided region described above can be executed according to the processing flow as shown in FIGS. 9 and 10 as in the first embodiment. However, in the processing flow shown in FIG. 9, the grid point of the upper left corner of the divided region is set in step S111, and the grid point is moved to the next grid point in the raster scan order in step S116. By recording pairs of grid point coordinates and image identification information in the queue 2 according to the processing flow of FIG. 10 for all the divided areas, final image arrangement data is obtained in the queue 2.

  Of course, as in the case of the first embodiment, the processing as shown in FIG. 9 and the processing as shown in FIG. 10 can be integrated.

  As is clear from the above description, the present embodiment corresponds to an embodiment of the invention according to claims 1 to 3. The “image input unit” corresponds to the image input unit 1005, and the “feature amount extraction unit” corresponds to the feature amount extraction unit 2003. The “image arrangement information generation unit” is realized by the image arrangement processing unit 2004 and the SOM map 2010. “Image rearrangement processing means” corresponds to the image rearrangement processing 2006.

<Second Embodiment of Image Arrangement Data Generation Unit 1006>
FIG. 15 is a block diagram for explaining a second embodiment of the image arrangement data generation unit 1006. The image arrangement data generation unit 1006 according to this embodiment includes a feature amount extraction unit 3001, an image arrangement processing unit 3002, an image arrangement information storage unit 3003, an image rearrangement processing unit 3004, and a control unit 3005.

  The processing flow of the image arrangement data generation operation in the present embodiment can be expressed as shown in FIG. 5 as in the first embodiment. However, the processing content of step S103 is different from that of the first embodiment. Hereinafter, the operation will be described along the processing flow of FIG.

  Step S101: One image stored in the image storage 1000 is input to the image arrangement data generation unit 1006 by the image input unit 1005 together with the image identification information (for example, file name). As described above, the input target image may be the entire image in the image storage 1000 or may be a partial image in the image storage 1000 narrowed down by, for example, keyword search. . In any case, an image that can be viewed by the client 1003 is an input target.

  Step S102: The input image is transferred to the feature amount extraction unit 3001 through the control unit 3005. The feature amount extraction unit 3001 extracts and outputs n-dimensional feature amounts such as saturation, hue, luminance, texture, and the like of the image.

  Step S103: The n-dimensional feature value output from the feature value extraction unit 3001 is transferred to the image arrangement processing unit 3002 through the control unit 3005 together with the image identification information of the image. In the image arrangement processing unit 3002, the input n-dimensional feature values are aggregated into two-dimensional feature values x and y by principal component analysis, and the values of the feature values x and y are directly replaced with two-dimensional coordinates, Image arrangement information 3006 in which the image is arranged at grid points is generated in the image arrangement information storage unit 3003. Specifically, the image arrangement information 3006 has, for example, a table structure having an entry associated with each lattice point in the two-dimensional space in a one-to-one manner. Image identification information of an image arranged at a certain lattice point is It is written in the entry corresponding to the lattice point. The content of the entry corresponding to the grid point where no image is arranged is information ("empty" information) indicating that no image is arranged.

  FIG. 16 is an explanatory diagram of image arrangement processing in the present embodiment. For the images 1 to 6, n-dimensional feature quantities 1 to n as shown in FIG. 16A are extracted. These feature quantities are obtained by principal component analysis as shown in FIG. 16B. x and y. Then, as shown in FIG. 16C, the images 1 to 6 are arranged at the grid points of the two-dimensional coordinates indicated by the feature amounts x and y.

  Also in this embodiment, when the characteristics of a plurality of images are very similar, the plurality of images are overlapped and arranged at the same grid point. In this case, the image identification information of the plurality of images is written in the entry of the same lattice point.

  Step S104: The control unit 3005 determines whether or not the last image to be input has been input. If there is an image to be input, the control unit 3005 asks the image input unit 1005 to input the next image, and Control is performed to execute the processing of steps S101 to S103 on the image. When it is determined that the last image has been input, the control unit 3005 instructs the image rearrangement processing unit 3004 to execute the image rearrangement process (the process of step S105).

  Step S105: The image rearrangement processing unit 3004 performs image rearrangement processing on the image arrangement information 3006 to generate final image arrangement data in the image arrangement data storage unit 1007. Since this image rearrangement process is the same as the process performed by the image rearrangement processing unit 2006 in the first embodiment, description thereof will not be repeated.

  In addition, after completing the above series of processing, when a new image to be viewed is added to the image storage 1000, or some images accumulated in the image storage 1000 are newly added to the viewing target. In this case, the image arrangement information 3006 in the image arrangement information storage unit 3003 is updated by performing the processing of steps S101 to S103 for the new browsing target image. Then, the image rearrangement process in step S105 is executed on the updated image arrangement information 3006, and updated image arrangement data is generated.

  As is clear from the above description, the present embodiment corresponds to an embodiment of the invention according to claims 1 to 3. The “image input unit” corresponds to the image input unit 1005, the “feature amount extraction unit” corresponds to the feature amount extraction unit 3001, the “image arrangement information generation unit” corresponds to the image arrangement processing unit 3002, and “ “Image rearrangement processing means” corresponds to the image rearrangement processing 3004.

<Third Embodiment of Image Arrangement Data Generation Unit 1006>
FIG. 17 is a block diagram for explaining a third embodiment of the image arrangement data generation unit 1006. The image arrangement data generation unit 1006 according to this embodiment uses a self-organizing map (SOM) for image arrangement processing, and includes an SOM model generation unit 4001, an SOM model storage unit 4002, a feature amount extraction unit 4003, clustering. A unit 4004, an image arrangement processing unit 4005, an image arrangement information storage unit 4006, an image rearrangement processing unit 4007, and a control unit 4008.

  First, an operation for generating an SOM model used for image arrangement and a related configuration will be described.

  Under the control of the control unit 4008, m sample images stored in the image storage 1000 are input through the image input unit 1005. Each input sample image is sent to the feature amount extraction unit 4003 via the control unit 4008. The feature quantity extraction unit 4003 extracts n-dimensional feature quantities such as saturation, hue, luminance, and texture from each sample image, and outputs an n-dimensional feature quantity vector composed of these feature quantities. This feature vector is input to the SOM model generation unit 4001 and the clustering unit 4004 via the control unit 4008.

  The clustering unit 4004 performs clustering to classify the sample image into several clusters (categories) using the partial dimension feature quantity of the input feature quantity vector of each sample image, and information on the cluster to which each sample image belongs Is output. The information of the cluster belonging to this is input to the SOM model generation unit 4001 via the control unit 4008. Here, the partial dimension feature quantity of the n-dimensional feature quantity vector output from the feature quantity extraction unit 4003 is used for clustering. However, another feature quantity dedicated to clustering is extracted, and this feature quantity is used. Clustering may be performed.

  Finally, the n-dimensional feature vector and the belonging cluster information of m sample images are input to the SOM model generation unit 4001. In the SOM model generation unit 4001, for each cluster, a SOM learning algorithm is executed using the feature vector of the sample image belonging to the cluster, and a plurality of SOM models 4010 for each cluster are generated on the SOM model storage unit 4002. .

  Each SOM model 4010 for each cluster is data in which feature vector of a plurality of sample images belonging to a cluster is arranged without overlapping with a lattice point in a two-dimensional space so that the feature vectors of the sample images are closer to each other. It is. More specifically, each SOM model 4010 has, for example, an entry corresponding to a lattice point in a two-dimensional space, and the table structure data in which the feature amount vector arranged at the lattice point is written in the corresponding entry. Can do.

  Note that once the SOM model 4010 is generated, the SOM model generation operation is not executed unless there is a situation where the SOM model needs to be regenerated. In addition, a cluster-specific SOM model prepared in advance may be stored in the SOM model storage unit 4002. In this case, the SOM model generation unit 4001 need not be provided.

  Next, a configuration related to the image arrangement data generation operation will be described. FIG. 18 shows a schematic processing flow of the image arrangement data generation operation.

  Step S201: One image stored in the image storage 1000 is input to the image arrangement data generation unit 1006 by the image input unit 1005 together with the image identification information (for example, file name). As described above, the input target image may be the entire image in the image storage 1000 or may be a partial image in the image storage 1000 narrowed down by keyword search or the like. It is. In any case, an image that can be viewed by the client 1003 is an input target.

  Step S202: The input image is transferred to the feature amount extraction unit 4003 through the control unit 4008. The feature quantity extraction unit 4003 extracts n-dimensional feature quantities such as the saturation, hue, luminance, and texture of the image, and outputs an n-dimensional feature quantity vector composed of these feature quantities. The extracted feature quantity is of the same type as that used for SOM map generation. This feature vector is transferred to the image arrangement processing unit 4005 and the clustering unit 4004 through the control unit 4008 together with the image identification information of the image.

  Step S203: The clustering unit 4004 obtains a cluster to which the image belongs by clustering using the partial dimension feature quantity of the input n-dimensional feature quantity vector. The information of the cluster belonging to this is input to the image arrangement processing unit 4005 through the control unit 4008. As in the case of generating the SOM model, another feature quantity dedicated to clustering may be extracted by the feature quantity extraction unit 4003, and the clustering unit 4004 may use this feature quantity to obtain the belonging cluster.

  Step S204: The image arrangement processing unit 4005 performs image arrangement processing for each cluster, and generates a plurality of image arrangement information 4011 for each cluster in the image arrangement information storage unit 4006. That is, a feature vector having the shortest Euclidean distance from the feature vector extracted from the image is searched from the feature vector group on the SOM model 4010 corresponding to the cluster to which the image belongs, and the feature vector is arranged. The lattice point is determined as the lattice point for arranging the image, and the image identification information of the image is recorded in association with the lattice point in the image arrangement information 4011 corresponding to the belonging cluster. Specifically, each cluster image arrangement information 4011 has, for example, a table structure having an entry associated with each grid point in a two-dimensional space on a one-to-one basis, and an image of an image arranged at a grid point. The identification information is written in the entry corresponding to the lattice point. The content of the entry corresponding to the grid point where no image is arranged is information ("empty" information) indicating that no image is arranged. Note that since the difference in image feature amounts in the same cluster is small, the width of the two-dimensional space in which images in individual clusters are arranged is narrower than the entire two-dimensional space in which images of all clusters are arranged.

  In each cluster, when the feature vectors of a plurality of images are very similar, the feature vectors on the SOM model having the smallest Euclidean distance from each feature vector are the same, and the plurality of images overlap. Arranged at the same grid point. In this case, the image identification information of the plurality of images is written in the entry of the same lattice point.

  Step S205: The control unit 4008 determines whether or not the last image to be input has been input. If there is an image to be input, the control unit 4008 asks the image input unit 1005 to input the next image, and Control for executing the processing of steps S201 to S204 on the image is performed. When it is determined that the last image has been input (No at Step S205), the control unit 4008 instructs the image rearrangement processing unit 4007 to execute the image rearrangement process (the process at Step S206).

  Step S206: When images are arranged using the SOM model, as described above, two or more images may be arranged in duplicate at the same grid point. When an image map is displayed in accordance with image arrangement information in which such images overlap, two or more image thumbnails are overlapped, and only the thumbnail of the image displayed at the top is invisible. The image rearrangement process is a process for generating final image arrangement data by rearranging images so as to eliminate such overlapping of thumbnails of a plurality of images.

  In addition, after completing the above series of processing, when a new image to be viewed is added to the image storage 1000, or some images accumulated in the image storage 1000 are newly added to the viewing target. In this case, the cluster-by-cluster image arrangement information 4011 in the image arrangement information storage unit 4006 is updated by performing the processing of steps S201 to S204 for these new browsing target images. Then, the image rearrangement process of step S206 is executed on the updated image arrangement information 4011, and updated image arrangement data is generated.

  Next, the image rearrangement process (step S206) of the image rearrangement processing unit 4007 will be described. FIG. 19 shows a schematic processing flow of image rearrangement processing.

  First, one cluster is selected (step S211). For the image arrangement information 4011 corresponding to the selected cluster, processing (Processing A) is performed in which the image identification information corresponding to the lattice points and the “empty” information are given a sequential order while tracing the lattice points in a predetermined order. (Step S212). Next, one lattice point on the two-dimensional space where the images of all clusters are arranged is set as the starting lattice point of the cluster (step S213). Next, the image identification information and the “empty” information are rearranged in the lattice points according to the order given thereto while tracing the lattice points from the starting lattice point in the same order as in the process A, and the final image arrangement data is obtained. A generation process (process B) is performed (step S214). It is checked whether there is an unprocessed cluster (step S215). If there is, the process returns to step S211 to execute processing for the next cluster. When the processing is completed for all clusters in this manner, final image arrangement data for images belonging to all clusters is generated.

  Hereinafter, the process A and the process B will be specifically described.

(First embodiment)
According to the first example, as in the case of the first example according to the first embodiment, in the processing A, the order is performed while tracing the lattice points spirally from the central lattice point. Similarly, in the process B, the rearrangement of images is performed while following the lattice points in a spiral shape.

  In the first embodiment, the process A for each cluster can be executed according to the process flow shown in FIG. The process B can be executed according to the process flow shown in FIG. However, in step S124 in FIG. 10, the lattice point coordinates (position information) in the two-dimensional space are calculated based on the starting lattice point of the cluster and the value of the lattice point counter.

  Although not shown in FIG. 10, when the process B for one cluster is completed, the contents of the queue 1 used in the process A for the cluster are cleared. The contents of the queue 2 at the stage when the process B for the last cluster is completed are the final image arrangement data.

  FIG. 20 schematically shows final image arrangement data (the contents of the queue 2).

  FIG. 21 is a schematic diagram showing a state of image rearrangement in the present embodiment. Here, the images are classified into three clusters. The starting grid points of each cluster are set so that the thumbnails (hatched rectangles) of the images of each cluster do not overlap on the image map, and the images in the clusters are rearranged in a spiral shape from this starting grid point. .

  This is a method for setting the starting grid point of each cluster. However, if the distance between the center of the cluster is larger than the distance between the center of the cluster and the image farthest from the cluster, thumbnails of images of different clusters will not overlap. . Therefore, for example, the distance from the cluster center to the furthest image can be calculated from the number of images in the cluster, and the starting grid point of each cluster can be set so that the distance between the centers of each cluster is larger than that distance. it can. Such a method has an advantage that it can flexibly cope with fluctuations in the image arrangement range in each cluster. However, it is possible to adopt a method of fixing the starting lattice points of each cluster in advance with a sufficient interval so as not to cause overlapping of images between clusters.

(Second embodiment)
According to the second example, as in the case of the second example according to the first embodiment, in the process A, for each cluster, the grid points are arranged in raster scan order from the grid point at the upper left corner of the two-dimensional space. The order is given while tracing. Similarly, in the process B, the image rearrangement is performed while following the lattice points in the raster scan order.

  In the second embodiment, the process A for each cluster can be executed according to the process flow shown in FIG. However, the upper left corner lattice point is set in step S111, and the lattice point is moved to the next lattice point in the raster scan order in step S116.

  The process B can be executed according to the process flow shown in FIG. However, in step S124 in FIG. 10, the lattice point coordinates (position information) in the two-dimensional space are calculated based on the starting lattice point of the cluster and the value of the lattice point counter.

  Although not shown in FIG. 10, when the process B for one cluster is completed, the contents of the queue 1 used in the process A for the cluster are cleared. The contents of the queue 2 at the stage when the process B for the last cluster is completed are the final image arrangement data.

  In the above two embodiments, the sequential order is given to the image identification information and the “empty” information in the process A (step S212 in FIG. 19), but the third embodiment and the fourth embodiment described below. Then, in process A, the order is given only to the image identification information.

(Third embodiment)
According to the third example, as in the case of the first example according to the first embodiment, in the process A, an order is given while tracing the lattice points spirally from the central lattice point. No order is given to “empty” information. Similarly, in the process B, the rearrangement of images is performed while following the lattice points in a spiral shape.

  In the third embodiment, the process A for each cluster can be executed according to the process flow shown in FIG. However, step S14 is skipped. The process B can be executed according to the process flow shown in FIG. However, in step S124 in FIG. 10, the lattice point coordinates (position information) in the two-dimensional space are calculated based on the starting lattice point of the cluster and the value of the lattice point counter.

  Although not shown in FIG. 10, when the process B for one cluster is completed, the contents of the queue 1 used in the process A for the cluster are cleared. The contents of the queue 2 at the stage when the process B for the last cluster is completed are the final image arrangement data.

  Assume that the image arrangement information of a certain cluster has contents as shown in FIG. 22, and the lattice point where the image of the image identification information a is arranged is the center lattice point. By the process A, the order as shown in FIG. 22 is given to the image identification information. Therefore, by processing B, the images are rearranged in a dense form without gaps as shown in FIG.

  As described above, according to this embodiment, unlike the first embodiment, the arrangement relationship of images before and after rearrangement is considerably different. However, since an image is arranged for each cluster, even if there is such a change in the arrangement relationship of the images, there is no big problem in searching for the target image. In addition, since the images are concentrated, more thumbnails of images can be displayed on the image map.

(Fourth embodiment)
According to the fourth example, as in the case of the second example according to the first embodiment, in the process A, the grid points are traced in the raster scan order from the grid point at the upper left corner of the two-dimensional space of each cluster. However, the ordering is performed, but no order is given to the “empty” information. Similarly, in the process B, the image rearrangement is performed while following the lattice points in the raster scan order.

  In the fourth embodiment, the process A for each cluster can be executed according to the process flow shown in FIG. However, the upper left corner lattice point is set in step S111, and the lattice point is moved to the next lattice point in the raster scan order in step S116. Step S14 is skipped.

  The process B can be executed according to the process flow shown in FIG. However, in step S124 in FIG. 10, the lattice point coordinates (position information) in the two-dimensional space are calculated based on the starting lattice point of the cluster and the value of the lattice point counter.

  Although not shown in FIG. 10, when the process B for one cluster is completed, the contents of the queue 1 used in the process A for the cluster are cleared. The contents of the queue 2 at the stage when the process B for the last cluster is completed are the final image arrangement data.

  Also in this embodiment, as in the case of the third embodiment, the images are rearranged in a dense manner by the process B, so that the image arrangement is considerably different before and after the rearrangement. However, since an image is arranged for each cluster, even if there is such a change in the arrangement relationship of the images, there is no big problem in searching for the target image. Also, more thumbnails of images can be displayed on the image map.

  As is clear from the above description, this embodiment corresponds to an embodiment of the invention according to claims 4 to 7. The “image input unit” corresponds to the image input unit 1005, the “feature amount extraction unit” corresponds to the feature amount extraction unit 4003, and the “clustering unit” corresponds to the clustering unit 4004. The “image arrangement information generating unit” is realized by the image arrangement processing unit 4005 and the SOM model 4010 for each cluster. The “image rearrangement processing unit” corresponds to the image rearrangement processing 4007.

<Fourth Embodiment of Image Arrangement Data Generation Unit 1006>
FIG. 24 is a block diagram for explaining a fourth embodiment of the image arrangement data generation unit 1006. The image arrangement data generation unit 1006 according to this embodiment includes a feature amount extraction unit 5001, a clustering unit 5002, an image arrangement processing unit 5003, an image arrangement information storage unit 5004, an image rearrangement processing unit 5005, and a control unit 5006. .

  The processing flow of the image layout data generation operation according to the present embodiment can be shown as in FIG. 18 as in the third embodiment. However, the processing content of step S204 is different from that of the third embodiment. Hereinafter, the operation will be described along the processing flow of FIG.

  Step S201: One image stored in the image storage 1000 is input to the image arrangement data generation unit 1006 by the image input unit 1005 together with the image identification information (for example, file name). As described above, the input target image may be the entire image in the image storage 1000 or may be a partial image in the image storage 1000 narrowed down by keyword search or the like. It is. In any case, an image that can be viewed by the client 1003 is an input target.

  Step S202: The input image is transferred to the feature amount extraction unit 5001 through the control unit 5006. The feature quantity extraction unit 5001 extracts n-dimensional feature quantities such as the saturation, hue, luminance, and texture of the image, and outputs an n-dimensional feature quantity vector composed of these feature quantities. This feature vector is transferred to the image arrangement processing unit 5003 and the clustering unit 5002 through the control unit 5006 together with the image identification information of the image.

  Step S203: The clustering unit 5002 obtains a cluster to which the image belongs by clustering using a partial dimension feature quantity of the input n-dimensional feature quantity vector. The information of the cluster belonging to this is input to the image arrangement processing unit 5003 through the control unit 5006. Note that another feature amount dedicated to clustering may be extracted by the feature amount extraction unit 5001, and the clustering unit 5002 may use this feature amount to obtain the belonging cluster.

  Step S204: The image arrangement processing unit 5003 performs image arrangement processing for each cluster, and generates a plurality of pieces of image arrangement information 5011 for each cluster in the image arrangement information storage unit 5004. That is, the image arrangement processing unit 5003 aggregates the input n-dimensional feature values into two-dimensional feature values x and y by principal component analysis, and directly replaces the values of the feature values x and y with two-dimensional coordinates. The grid point of the coordinate is determined as the grid point at which the image is arranged, and the image identification information of the image is recorded in association with the grid point in the image arrangement information 5011 corresponding to the belonging cluster. Specifically, each cluster image arrangement information 5011 has, for example, a table structure having an entry associated with each grid point in a two-dimensional space on a one-to-one basis, and an image of an image arranged at a grid point. The identification information is written in the entry corresponding to the lattice point. The content of the entry corresponding to the grid point where no image is arranged is information indicating that no image is arranged (hereinafter referred to as “empty” information). Note that since the difference in image feature amounts in the same cluster is small, the width of the two-dimensional space in which images in individual clusters are arranged is narrower than the entire two-dimensional space in which images of all clusters are arranged.

  In addition, in each cluster, when the feature-value vector of a some image is very similar, a some image may overlap and be arrange | positioned at the same lattice point. In this case, the image identification information of the plurality of images is written in the entry of the same lattice point.

  Step S205: The control unit 5006 determines whether or not the last image to be input has been input. If there is an image to be input, the control unit 5006 requests the image input unit 1005 to input the next image, and Control for executing the processing of steps S201 to S204 on the image is performed. When it is determined that the last image has been input (No at Step S205), the control unit 5006 instructs the image rearrangement processing unit 5005 to execute an image rearrangement process (the process at Step S206).

  Step S206: The image rearrangement processing unit 5005 performs image rearrangement processing. Since this image rearrangement process is the same as the process by the image rearrangement processing unit 4007 of the third embodiment, the description thereof will not be repeated.

  In addition, after completing the above series of processing, when a new image to be viewed is added to the image storage 1000, or some images accumulated in the image storage 1000 are newly added to the viewing target. In this case, the cluster-by-cluster image arrangement information 5011 in the image arrangement information storage unit 5004 is updated by performing the processing of steps S201 to S204 for these new browsing target images. Then, the image rearrangement process in step S206 is executed on the updated image arrangement information 5011, and updated image arrangement data is generated.

  As is clear from the above description, this embodiment corresponds to an embodiment of the invention according to claims 4 to 7. The “image input unit” corresponds to the image input unit 1005, the “feature amount extraction unit” corresponds to the feature amount extraction unit 5001, the “clustering unit” corresponds to the clustering unit 5002, and the “image arrangement information generation unit”. "Corresponds to the image rearrangement processing unit 5003, and" image rearrangement processing means "corresponds to the image rearrangement processing 5005.

  It is obvious that the image arrangement data generation unit 1006 and the image input unit 1005 according to each embodiment described above can be realized by a program using computer hardware resources. Such a program, that is, a program that causes a computer to function as each component of the image arrangement data generation unit 1006 and the image input unit 1005 is also included in the present invention. Various information recording (storage) media that can be read by a computer, such as a semiconductor storage element, a magnetic disk, an optical disk, and a magneto-optical disk, in which such a program is recorded are also included in the present invention.

  Further, the description of the image layout data generation processing in each embodiment described above is also the description of the processing procedure of the image layout data generation method according to the inventions according to claims 8 to 14. Therefore, the description regarding the embodiment of the image arrangement data generation method of the present invention will not be repeated.

  In the above description, an orthogonal coordinate system lattice point is assumed as a lattice point on which an image is arranged. However, a polar coordinate system lattice point can also be used.

1 is a block diagram of an image processing system according to an embodiment of the present invention. It is a figure which shows typically the image map which has arrange | positioned the thumbnail of the image according to image arrangement data. It is a block diagram which shows 1st Embodiment of an image arrangement | positioning data generation part. It is a conceptual diagram of an SOM map. It is a figure which shows the schematic processing flow of the image arrangement | positioning data generation operation | movement in 1st Embodiment of an image arrangement | positioning data generation part. It is a figure which shows the processing flow of an image rearrangement process. It is explanatory drawing of order provision to image identification information and "empty" information. It is explanatory drawing of image rearrangement. It is a figure which shows the processing flow of the process A which puts order to image identification information and "empty" information. It is a figure which shows the processing flow of the process B which rearranges an image and produces | generates final image arrangement data. It is a figure which shows the content of the queue 1 after the process A. FIG. It is a figure which shows the content of the queue 2 after the process B. FIG. It is explanatory drawing of order provision to image identification information and "empty" information. It is explanatory drawing of image rearrangement. It is a block diagram which shows 2nd Embodiment of an image arrangement | positioning data generation part. It is explanatory drawing of the aggregation to the two-dimensional feature-value by a principal component analysis of an n-dimensional feature-value, and the image arrangement | positioning by a two-dimensional feature-value. It is a block diagram which shows 3rd Embodiment of an image arrangement | positioning data generation part. It is a figure which shows the schematic processing flow of the image arrangement | positioning data generation operation | movement in 3rd Embodiment of an image arrangement | positioning data generation part. It is a figure which shows the schematic process flow of an image rearrangement process. It is explanatory drawing of the content (final image arrangement data) of the queue 2. FIG. It is explanatory drawing of the rearrangement of the image of each cluster. It is explanatory drawing of order provision to image identification information. It is explanatory drawing of image rearrangement. It is a block diagram which shows 4th Embodiment of an image arrangement | positioning data generation part.

Explanation of symbols

1000 Image storage 1001 Image arrangement data server 1002 Image distribution server 1003 Client 1005 Image input unit 1006 Image arrangement data generation unit 1007 Image arrangement data storage unit 1008 Image arrangement data request processing unit 2001 SOM model generation unit 2002 SOM model storage unit 2003 Features Extraction unit 2004 Image arrangement processing unit 2005 Image arrangement information storage unit 2006 Image rearrangement processing unit 2007 Control unit 2010 SOM model 2011 Image arrangement information 3001 Feature amount extraction unit 3002 Image arrangement processing unit 3003 Image arrangement information storage unit 3004 Image rearrangement processing Unit 3005 control unit 3006 image arrangement information 4001 SOM model generation unit 4002 SOM model storage unit 4003 feature quantity extraction unit 4004 clustering unit 4005 image arrangement processing unit 4006 Image arrangement information storage unit 4007 Image rearrangement processing unit 4008 Control unit 4010 Cluster-specific SOM model 4011 Cluster-specific image arrangement information 5001 Feature quantity extraction unit 5002 Clustering unit 5003 Image arrangement processing unit 5004 Image arrangement information storage unit 5005 Image rearrangement Processing unit 5006 Control unit 5011 Image arrangement information for each cluster

Claims (14)

An image input means for inputting an image;
Feature quantity extraction means for extracting the feature quantity of each image input by the image input means;
Based on the feature amount extracted by the feature amount extraction unit, each image input by the image input unit is arranged at a grid point in the two-dimensional space so that a similar image is closer, and the image is arranged at the grid point. Image arrangement information generating means for generating image arrangement information in which image identification information of an image or “empty” information meaning that no image is arranged at a grid point is recorded in association with the grid point;
A first process for assigning a sequential order to image identification information and “empty” information while tracing lattice points in a predetermined order with respect to the image arrangement information generated by the image arrangement information generating unit; and the predetermined order The image is rearranged by following the grid points and re-allocating the image identification information and the “empty” information to the grid points in order according to the order given in the first process. Image relocation processing means for performing a second process of generating image arrangement data consisting of a set of pairs of image identification information and position information of lattice points associated with the image identification information;
An image arrangement data generation device characterized by comprising:   2. The image arrangement data generation apparatus according to claim 1, wherein the image rearrangement processing unit traces lattice points in a spiral shape in the first and second processes.   2. The image arrangement data generation apparatus according to claim 1, wherein the image rearrangement processing unit traces the grid points in the raster scan order for each divided region of the two-dimensional space in the first and second processes. Clustering means for classifying each image input by the image input means into a plurality of clusters based on the feature quantity extracted by the feature quantity extraction means,
The image arrangement information generating means performs arrangement of each image on a grid point in a two-dimensional space to generate image arrangement information for each cluster,
The image rearrangement processing means performs the first processing for each cluster, and in the second processing, the image for each cluster is traced in the predetermined order from the starting lattice point set for each cluster. The image arrangement according to claim 1, wherein the image is rearranged by re-associating the identification information and the “empty” information with the grid points in order according to the order given by the first processing. Data generator.   5. The image arrangement data generation apparatus according to claim 4, wherein the image rearrangement processing means traces lattice points in a spiral shape in the first and second processes.   5. The image arrangement data generation apparatus according to claim 4, wherein the image rearrangement processing unit traces the grid points in the raster scan order in the first and second processes.   The image rearrangement processing unit rearranges an image by assigning an order only to the image identification information in the first process and re-associating only the image identification information with the lattice points in the second process. The image arrangement data generation device according to claim 4, wherein: An image input process for inputting an image;
A feature amount extraction step of extracting a feature amount of each image input by the image input step;
Based on the feature amount extracted in the feature amount extraction step, each image input in the image input step is arranged at a grid point in the two-dimensional space so that a similar image is closer, and the image is placed at the grid point. An image arrangement information generating step for generating image arrangement information in which image identification information of an image or “empty” information meaning that no image is arranged at a grid point is recorded in association with the grid point;
A first process for assigning a sequential order to image identification information and “empty” information while following lattice points in a predetermined order with respect to the image arrangement information generated in the image arrangement information generation step; and the predetermined order The image is rearranged by following the grid points and re-allocating the image identification information and the “empty” information to the grid points in order according to the order given in the first process. An image rearrangement process step of performing a second process of generating image arrangement data consisting of a set of pairs of image identification information and lattice point position information associated with the image identification information;
An image layout data generation method characterized by comprising:   9. The image arrangement data generation method according to claim 8, wherein the image rearrangement processing step traces lattice points spirally in the first and second processes.   9. The image arrangement data generation method according to claim 8, wherein the image rearrangement processing step traces the grid points in the raster scan order for each divided region of the two-dimensional space in the first and second processes. A clustering step of classifying each image input in the image input step into a plurality of clusters based on the feature amount extracted in the feature amount extraction step;
The image arrangement information generation step performs arrangement of the image at a lattice point in the two-dimensional space for each cluster to generate image arrangement information for each cluster,
In the image rearrangement processing step, the first processing is performed for each cluster, and in the second processing, the image for each raster is obtained while tracing the lattice points in the predetermined order from the starting lattice points set for each cluster. 9. The image arrangement according to claim 8, wherein the image is rearranged by re-associating the identification information and the “empty” information with the lattice points in order according to the order given by the first processing. Data generation method.   12. The image arrangement data generation method according to claim 11, wherein the image rearrangement processing step traces lattice points spirally in the first and second processes.   12. The image arrangement data generation method according to claim 11, wherein the image rearrangement processing step traces the grid points in the raster scan order in the first and second processes.   The image rearrangement processing step rearranges the image by assigning an order only to the image identification information in the first processing and re-associating only the image identification information with the grid points in the second processing. The image arrangement data generation method according to claim 11.

Images