JP2009217828A - Image retrieval device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image retrieval method and device efficiently and quickly retrieving an object image and to provide a retrieval method and device having high retrieval accuracy. <P>SOLUTION: Since a personal computer 10 retrieves an image similar to an image for retrieval from the inside of an image database having directory structure constituted of one or more folders, and has a display picture for displaying the retrieved image, the image is efficiently retrieved by using such directory structure. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to a technique for efficiently retrieving images included in an image database.

  An image acquired using a digital still camera or a scanner is stored in an image database such as an electronic album. By the way, in the case of a character database, a search can be performed using a keyword, so even a huge amount of data can be searched in a relatively short time. On the other hand, in the case of an image database, the search target is an image.

  In contrast, for example, Japanese Patent Laid-Open No. 10-275161 discloses a technique for efficiently searching for an image using a keyword stored in association with the image. However, according to such a conventional technique, when an image is registered in the image database, it is necessary to register a corresponding keyword. Therefore, for example, when an enormous amount of images already exist, before registering in the image database, the user needs to classify them and assign an appropriate keyword for each image. Become.

  On the other hand, a technique for searching for an image similar to the search target image is disclosed in, for example, Japanese Patent Laid-Open No. 10-260983. According to such a conventional technique, an image is divided into blocks, and the similarity is determined using each feature amount as a matrix. However, there is a problem that the amount of calculation becomes enormous and takes time because the image is divided into blocks and the respective feature amounts are found.

  Further, Japanese Patent Application Laid-Open No. 10-254901 discloses a technique in which a part of an image is designated in advance and its attribute is detected to obtain the similarity of the images. However, in order to detect an attribute, there exists a problem that a complicated calculation is required and takes time. In addition, in order to search for an image, since it is premised that an attribute is detected in advance in each part of all images in the image database, there is a problem that it takes time and effort. In addition, when the attribute of the search target image is not known, it takes time to obtain this.

  SUMMARY OF THE INVENTION The main object of the present invention is to provide an image search device, an image search method, and an image display device capable of displaying a specific image, which efficiently search for an image contained in an image database. There is.

  More specifically, a first object of the present invention is to provide an image search method and apparatus capable of efficiently and quickly searching for a target image, and to provide a search method and apparatus with high search accuracy. .

  Furthermore, the second object of the present invention is to provide an image search method and apparatus capable of learning the search contents according to the user's preference through operations such as image search and browsing, and performing a more accurate search.

  Furthermore, a third object of the present invention is to provide a technique capable of facilitating image search on a network such as the Internet and uploading image data extracted by the image search. is there.

  Furthermore, a fourth object of the present invention is to make a search, even if the search target image is ambiguous, it can be quickly searched from the image database, and the classification information of the image database can be easily used, and comments relating to the image can be obtained. It is to provide a technology that makes it easy to input keywords.

  Furthermore, a fifth object of the present invention is to provide a technique for quickly browsing an outline of an image in an image database and quickly searching for an ambiguous image from the image database.

  In order to achieve such an object, an image search apparatus of the present invention is searched with search means for searching for an image similar to a search target image from an image database having a directory structure composed of one or a plurality of folders. And display means for displaying the image.

  An image search apparatus according to the present invention includes search means for searching for an image from an image database, and display means for displaying the searched image. The search means is the same or in a folder close on the time axis. An image having a high degree of similarity at the edge of the image is searched for, and the display means displays a composite image obtained by overlapping the edges of the image and joining a plurality of images together.

  The image search apparatus of the present invention includes search means for searching for an image similar to the search target image from the image database, and display means for displaying the searched image. The search means selects the image. A similar image is searched for using frequency as one of the feature amounts.

  The image search method of the present invention includes a step of storing an image selection frequency in an image database, and a step of searching for an image similar to a search target image with the frequency of image selection as one of feature quantities. It is characterized by.

  The image search apparatus of the present invention records search means for searching for an image similar to the search target image, display means for displaying the searched image, and history of the displayed image from the image database. Display control means for restricting display of the image displayed once.

  The image search method of the present invention records a step of searching for an image similar to the search target image in the image database, a step of displaying the searched image as a candidate image, and a history of the displayed image. And a step of excluding the displayed image from the candidate image.

  An image search apparatus according to the present invention includes a unit that classifies images in an image database into categories, a search unit that searches for images similar to a search target image based on category information in the image database, and a searched image Display means for displaying the image, wherein the display means displays an image representing each category before displaying the image searched by the search means.

  The image search method of the present invention includes a step of classifying images in an image database for each category and a step of displaying an image representative of the category.

  The image search apparatus of the present invention includes search means for searching for an image similar to the search target image from the image database, and display means for displaying the searched image, wherein the display means is an image representing a color. And the search means searches for an image including a color similar to the color corresponding to the image displayed by the display means.

  The image search method of the present invention includes a step of displaying an image representing a color, a step of searching an image containing a color similar to the color corresponding to the displayed image from the image database, and the searched image And a step of displaying.

  The image search apparatus of the present invention includes search means for searching for an image similar to the search target image from the image database, and display means for displaying the searched image. The display means is stored in the image database. An image not included is acquired and displayed, and the search means searches for an image similar to the image displayed by the display means.

  The image search method of the present invention includes the steps of acquiring and displaying an image, searching the image database for an image similar to the acquired image, and displaying the searched image. Features.

  The image search apparatus according to the present invention includes search means for searching for an image similar to the search target image from the image database, and retouch means for collectively retouching displayed images.

  The image search method of the present invention includes a step of searching for an image in an image database, and a step of collectively retouching the searched images.

  An image search apparatus according to the present invention includes a display unit that displays an image, a storage unit that stores a specified position when a position is specified on the image displayed on the display unit, and the storage unit Search means for searching for an image in the image database based on the position stored in the image database.

  The image search apparatus of the present invention is characterized by comprising storage means for storing information relating to a hard copy of an image in an image database, and search means for searching for an image based on the information relating to the hard copy.

  An image search apparatus according to the present invention includes an input unit capable of inputting a character keyword, and a setting unit configured to set an image feature amount and a network address related to the keyword based on the character keyword input by the input unit. It is characterized by having.

  The image search apparatus according to the present invention includes an extraction unit that extracts a feature amount of a specific image, a determination unit that determines a character keyword including the feature amount extracted by the extraction unit, and a network address related to the determined character keyword. It is characterized by having a means for acquiring an image and a determination means for extracting a feature amount of the acquired image and determining whether or not it includes a feature of a specific image.

  The image search apparatus according to the present invention is an image search apparatus that preferentially searches for images in the same category when searching for an image in an image database, and adds a category common to one or more images It has the means.

  The image search apparatus according to the present invention is characterized by having accompanying means for attaching sound information to an image in an image database, and search means for searching for an image based on the sound information. An image display apparatus according to the present invention includes a selection unit that selects an image in an image database, an ordering unit that orders the images in the image database with the selected image as a head, and a display unit that displays the images according to the ordering. It is characterized by having.

  According to the image search apparatus of the present invention, search means for searching for an image similar to a search target image from an image database having a directory structure composed of one or a plurality of folders, and the searched image are displayed. Since the display means is included, an image can be searched efficiently by using such a directory structure. According to the image search apparatus of the present invention, the image search apparatus includes a search unit for searching for an image from the image database, and a display unit for displaying the searched image, wherein the search unit is the same or close on the time axis. The image is searched for an image having a high similarity at the edge of the image, and the display means displays a composite image obtained by joining the plurality of images by overlapping the edges of the image. When it is desired to synthesize panoramic images using the images in the database, the search means automatically searches for connected images, so that the images can be synthesized efficiently.

  According to the image search device of the present invention, the image search apparatus includes search means for searching for an image similar to the search target image from the image database, and display means for displaying the searched image. Since similar images are searched using the selected frequency as one of the feature amounts, for example, images that are frequently used among images in the image database are considered to have common features. You can search for images.

  According to the image search method of the present invention, the step of storing the selection frequency of the image in the image database, and the step of searching for an image similar to the search target image with the frequency of selecting the image as one of the feature amounts. Therefore, for example, images that are frequently used among the images in the image database are considered to have common characteristics. Therefore, if the frequency is used, the images can be searched more efficiently.

  According to the image search device of the present invention, a search unit for searching for an image similar to the search target image, a display unit for displaying the searched image, and a history of the displayed image are recorded from the image database. And a display control unit that restricts the display of the image once displayed, so that the image displayed once by the display unit is regarded as an image that the user has already excluded from the image, thereby enabling more efficient image search. It is possible.

  According to the image search method of the present invention, a step of searching for an image similar to the search target image in the image database, a step of displaying the searched image as a candidate image, and a history of the displayed image are recorded. And the step of excluding the image displayed once from the candidate image, the image displayed once is considered as an image that the user has already excluded from the consideration, thereby enabling more efficient image search. . According to the image search device of the present invention, the means for classifying the images in the image database into categories, the search means for searching for images similar to the search target image based on the category information from the image database, and the search Display means for displaying an image, and the display means displays an image representative of each category before displaying the image searched by the search means. The user can immediately recognize what category the image is in, thereby enabling efficient image retrieval.

  According to the image search method of the present invention, the method includes a step of classifying images in the image database for each category and a step of displaying an image representative of the category. The user can immediately recognize what kind of category contains the image, thereby enabling efficient image retrieval.

  According to the image search apparatus of the present invention, the image search apparatus includes search means for searching for an image similar to the search target image from the image database, and display means for displaying the searched image. An image can be displayed, and the search means searches for an image including a color similar to the color corresponding to the image displayed by the display means. Is possible.

  According to the image search method of the present invention, a step of displaying an image representing a color, a step of searching an image containing a color similar to the color corresponding to the displayed image from the image database, and a search are performed. And a step of displaying the displayed image, it is possible to efficiently search for the image by using the color.

  According to the image search device of the present invention, the image search apparatus includes search means for searching for an image similar to the search target image in the image database, and display means for displaying the searched image, wherein the display means includes the image When an image not included in the database is acquired and displayed, and the search means searches for an image similar to the image displayed by the display means, for example, when the user cannot point out the characteristics of the image to be searched However, if there is a similar image outside the image database, it can be used for more efficient image retrieval.

  According to the image search method of the present invention, the method includes a step of acquiring and displaying an image, a step of searching for an image similar to the acquired image from the image database, and a step of displaying the searched image. Therefore, for example, even when the user cannot point out the characteristics of the image to be searched, if there is an image that is very similar outside the image database, it is possible to perform more efficient image search by using this.

  According to the image search apparatus of the present invention, the image search apparatus includes search means for searching for an image similar to the search target image and retouch means for collectively retouching displayed images from the image database. Retouch similar images at once. According to the image search method of the present invention, the method includes a step of searching for an image in the image database and a step of retouching the searched images at a time, so that similar images that need to be retouched can be retouched in a batch.

  According to the image search device of the present invention, a display unit that displays an image, a storage unit that stores a specified position when a position is specified on the image displayed on the display unit, Since it has a search means for searching for an image in the image database based on the position stored in the storage means, it is possible to perform a more efficient search using a specified position such as a face of a person image, for example. .

  According to the image retrieval apparatus of the present invention, the image retrieval apparatus includes a storage unit that stores information relating to a hard copy of an image in an image database, and a retrieval unit that retrieves an image based on the information relating to the hard copy. Images can be divided according to the number of times and searches can be performed more efficiently.

  According to the image search apparatus of the present invention, an input unit capable of inputting a character keyword, and a setting unit that sets an image feature amount and a network address related to the keyword based on the character keyword input by the input unit. Therefore, a more efficient image search can be performed using the character keyword.

  According to the image search device of the present invention, an extraction unit that extracts a feature amount of a specific image, a determination unit that determines a character keyword including the feature amount extracted by the extraction unit, and a network related to the determined character keyword Since it has a means for acquiring an image from an address and a determination means for extracting the feature amount of the acquired image and determining whether or not the characteristic of the specific image is included, a character keyword is not added to the image in advance. However, since the determining means determines the character keyword, it is possible to perform more efficient image search.

  According to the image search device of the present invention, when searching for an image in the image database, the image search device preferentially searches for an image in the same category, and a category common to one or a plurality of images is added. For example, if the user puts favorite images together in the same category and adds such a category, the subsequent search can be performed efficiently.

  According to the image search device of the present invention, the image search apparatus includes the accompanying means for attaching the sound information to the image of the image database and the search means for searching for the image based on the sound information. ”,“ Fukei ”, etc., if related sounds are used, more efficient image search can be performed using this.

  According to the image display device of the present invention, the selecting means for selecting an image in the image database, the ordering means for ordering the images in the image database with the selected image as the head, and the display means for displaying the images according to the ordering. Therefore, images that are highly related to the selected image are sequentially displayed, and the image can be viewed more efficiently.

  According to the present invention, it is possible to efficiently search for an image included in an image database.

It is a figure for demonstrating this Embodiment. It is a figure which shows an example of the directory structure of this Embodiment. It is a figure which shows the modification of a directory structure. It is a figure which shows the example of a screen display of the personal computer 10 of FIG. FIG. 5 is a diagram illustrating an example of a screen on which a similar image is displayed when the upper left image in FIG. 4 is selected as a key image. It is a figure which shows the example of the display screen which provided the image acquisition button. It is a figure which shows the example of the display screen which provided the retouch button. It is a figure which shows the example clustered by the search coordinate. It is a figure which shows the display screen as an example which performed the position designation | designated in an image. It is a figure which shows the example by which the searched image was displayed. It is a figure which shows a search map. 2 is a diagram showing a display screen of a personal computer 10. FIG. It is a figure which shows matching with a search keyword, the feature-value of an image, and related URL. It is a figure which shows the example of the image search using an image file. It is the figure which showed the example in which an image is displayed on a display screen. It is a figure for demonstrating 6th Embodiment.

  Embodiments according to the present invention will be described below with reference to the drawings.

  FIG. 1 is a diagram for explaining the present embodiment. In FIG. 1, a personal computer 10 is connected to a server 20 having a large storage capacity via a network N. On the other hand, a film scanner 30 installed in a laboratory, for example, is also connected to the server 20 via the network N. In the server 20, an image database is constructed. Note that the image database is not necessarily constructed in the server 20 and may be constructed in the hard disk of the personal computer 10. In such a case, the image data from the film scanner 30 can be input to the personal computer 10 via a storage medium such as a CD-R.

  An aspect of registering an image in the image database of the server 20 will be described. In the laboratory, the negative film obtained by the user photographing with the silver salt camera is converted into image data by the film scanner 30. Such image data is transmitted to the server 20 via the network N, and is registered in the image database as an image.

  Here, when an image is registered, if it is stored as it is, it takes time to search for a specific image later. Therefore, in this embodiment, a directory structure is formed to store data relating to images in a hierarchical manner.

  FIG. 2 shows an example of such a directory structure. In FIG. 2, for example, a directory named “199801001.001” is formed in the upper layer directory. This is because the name is composed of (year letter 4 characters) + (month 2 characters) + (day 2 characters) + (number 3 characters), that is, the first negative read on January 1, 1998 It is a film.

  Therefore, the registration date of the image can be read from this directory name. All folders (KN001001.JPG ~) related to images in the same negative film are stored in this directory.

  An image captured by a digital still camera or the like can also be stored in the directory structure in the same manner. In such a case, since a recording medium such as a memory card corresponds to the negative film, one upper layer directory is formed for each memory card. Each directory in such an upper layer directory is hereinafter referred to as a lot.

  Further, in this directory, a directory (HTML) for placing an HTML file indicating a list of grouped images is formed. In the present embodiment, the image format corresponds to jpeg and FlashPix.

  FIG. 3 is a diagram showing a modification of the directory structure. In the case of the example according to FIG. 3, directories named “Album_01B”, “Album_04B”, and Album_16B ”are formed in the upper directory, which store images of 1Base, 4Base, and 16Base, respectively. It means that. The directories below it are the same as those shown in FIG.

  Next, an aspect of searching for a specific image from the image database constructed in this way will be described. FIG. 4 is a diagram showing a screen display example of the personal computer 10 of FIG. When the user starts the image search software, the display shown in FIG. 4 appears on the screen as an initial screen.

  The plurality of images displayed here are images representative of each cluster, for example, by statistically classifying the images in the image database so as to have a predetermined number of clusters. The representative image may be, for example, the first image of each lot.

  The user selects an image that seems to be similar to the image he / she wants to find (for example, a mountain in the landscape or a person), and uses the mouse (not shown) or the like. Click. At this time, the selected image is hereinafter referred to as a key image.

  The personal computer 10 searches the image database for an image similar to the key image. More specifically, the similarity is calculated by obtaining the relative distance between the feature amount corresponding to the selected image and the feature amount of the other image. Here, the coordinate axes are set so that the similarity is higher as the distance is shorter. The image feature data file is recorded on the hard disk of the personal computer 10 or the like. In addition to general features such as color and frequency distribution, the feature amount includes, for example, a position on the database or a folder name corresponding to the date as an amount representing time, and the date and time when the image was registered, or the image The date and time when the image was taken, and in some cases the date and time when the image was corrected, are also taken into consideration in the search. That is, an image closer to the date and time when the selected image is taken or registered is preferentially searched.

  This is because the time factor is often the key when searching with an electronic album. Among images of the same lot as the key image, the predetermined number of images in descending order of similarity tends to increase the time factor particularly when the similarity is increased.

  The personal computer 10 displays thumbnail images on the display in descending order of similarity. In FIG. 4, when the [Lot] button BL on the right side of the image is clicked, the personal computer 10 can switch the screen and display all the images of the same lot. In such a case, the similarity may be examined within the same lot, and the images may be rearranged and displayed in descending order of similarity. As a result, the user can reach the desired image more quickly.

  FIG. 5 shows an example of a screen displaying the similar image when the upper left image in FIG. 4 is selected as the key image. In the example of FIG. 5, the personal computer 10 searches the image database for an image in which two persons are photographed as main subjects as a similar image.

  If the image desired by the user is not obtained in the first search, the user can continue the search. For example, in the example shown in FIG. 5, when the image is selected by clicking on the lower right image, an image similar to that (for example, an image including a sea landscape image) is displayed on the screen.

  At this time, the personal computer 10 can store a history of displaying images, and can prevent images displayed once during the search process from being displayed. An image that has not been selected even though the user browsed it once is regarded as an image to be searched and is prevented from being displayed again, thereby achieving a quick search. The same operation may be repeated until the image to be searched for is found. Such history may be reset by clicking a reset button (not shown).

  As other functions, the following functions may be added to the personal computer 10. Both are auxiliary functions to facilitate searching.

  (1) When the [Info] button BI on the right side of the image is clicked, the original image is displayed in a large size, not a thumbnail, and information about the image such as the shooting date and size and the size is also displayed in text. For example, if the image is recorded on a film containing an IX240 cartridge (so-called APS film), the information can be obtained from the information written in the magnetic recording layer of the film or written in the header of the image file. It can also be obtained from information.

  (2) When the [Key Image History Display] button BK is clicked, a list of key images selected so far is displayed, and the state at that time can be returned.

  (3) Click the [Category Display] button BC to return to the initial screen.

  (4) When the “lot representative display” button BP is clicked, several images representing the lot are displayed in time series as a start screen.

  Thus, according to the present embodiment, since the position on the time axis is added to the feature amount, it is possible to search for an image to be searched at high speed. Further, since the image database has a directory structure corresponding to the time axis, the position on the time axis can be easily obtained.

  Furthermore, according to the present embodiment, even if the user cannot specifically specify the image to be searched for, for example, there was an image taken at the same time, or there were three people in the image, Even if it is vague information that a photograph should have been taken against the back of a mountain, the personal computer 10 can search for a similar image from the image database, so that an efficient search is possible. In addition, there is an advantage that it is not necessary to previously assign an attribute or the like to the image when performing such a search.

  If the image data on the recording medium is copied to the hard disk on the personal computer for each directory structure, the access speed can be increased and the search speed can be increased.

  Furthermore, the creation of an image database is not limited to using a laboratory service, and an individual user creates a database with a similar structure using a personal computer at home using image search software. It is also possible.

  As a modification of the present embodiment, a menu bar is displayed when the [Category Display] button BC is clicked on the initial screen shown in FIG. 4, and the representative image displayed on the initial screen can be changed. It is done.

  In such a case, as a default, as in the above-described embodiment, a representative image of each cluster classified statistically is displayed, and when [Color Image] is selected, a color picker (not shown) is displayed on the initial screen, The user may be able to select a dominant color in the image he wants to search for.

  For example, “green” color is generally considered dominant in summer mountain scenery, and “blue” color is considered dominant in sea scenery. Therefore, the image can be narrowed down to some extent by selecting the color. In this way, a search is performed using the color-related feature amount, and the higher the control ratio of the selected color (that is, the higher the proportion of the selected color), the higher the similarity, and the result is displayed on the screen. Is displayed.

  In the above modification, the color is selected using the color picker. Instead, words representing the contents of the image such as [mountain] [sea] can be selected, and the images representing the respective words are displayed. May be displayed. If a keyword is added in advance, a corresponding image may be displayed, or an image representing [mountain] [sea] is determined in advance, and an image having a high similarity to the image is displayed. It doesn't matter. Note that the personal computer 10 can determine the category to which the image should belong based on the content of the image or the keyword if the keyword is attached to the image.

  As described above, according to the present modification, the user can select the initial screen that most closely matches the image he / she wants to search for, so that a desired image can be found more quickly.

  As another modified example, an image that is not registered in the database can be taken in as an image displayed on the initial screen shown in FIG. For example, an [Image Import] button BT as shown in FIG. 6 is provided, and when this button BT is clicked, an image selection dialog box is displayed, and an image outside the database (may be obtained via the Internet) is selected. Like that. If there is no thumbnail image of this image, a thumbnail may be automatically created, or a part of the image may be trimmed so that the selected image can be displayed.

  According to this modification, since an image outside the database can be selected as a start image option, the user can select an initial screen that most closely matches the image he / she wants to find, and thus can be found earlier.

  Next, a second embodiment will be described. First, when the image search software is activated on the personal computer 10, an initial screen shown in FIG. 7 is displayed. When the user clicks the [Retouch] button BR, a menu bar is displayed, and items such as “panorama” and “red-eye correction” are displayed (not shown). Here, when the user selects “panorama”, the personal computer 10 performs the following processing.

  The personal computer 10 searches for a combination of images having a high degree of similarity in feature amounts related to the edges of the top, bottom, left, and right edges of the image within the same lot (the possibility that the shooting location is different if the lot is different). At this time, the similarity between the left edge of the target image and the right edge of the other image, the right edge and the left edge, the upper edge and the lower edge, and the similarity between the lower edge and the upper edge are respectively obtained, and the image whose similarity is higher than a predetermined threshold value Are searched as images constituting a panoramic image.

  The personal computer 10 performs pattern matching on the end portions having a high degree of similarity, finds an overlapping position, and combines the two images together. Since pattern matching is already known, its details are omitted. The synthesized image is displayed on the screen.

  On the other hand, when the user selects “red-eye correction”, the personal computer 10 performs the following processing. First, the user searches the database for one red-eye image. Next, when the user selects “red-eye correction”, the personal computer 10 determines whether there is a further red eye for an image having a high degree of similarity to the red-eye image. to correct. Furthermore, the personal computer 10 displays an image corrected for red eyes.

  As described above, according to the present embodiment, it is possible to automatically search for images that need to be selected and process them in a batch. Note that the batch processing of images is not limited to red-eye, and for example, it is possible to change the color of clothes of a photographed person so as to be unified.

  Next, a third embodiment will be described. In such an embodiment, according to the user's operation of clicking on an image, a part of the image that is considered to be characteristic for the user is determined and used for subsequent searches. More specifically, first, the user clicks on a part that seems to be important in the key image, and the location is stored as important information together with the file. The key image can be selected many times and gradually approaches the image desired by the user. Further, in order to extract the feature amount of the image, the complexity of the image and the color distribution are used.

  First, as a first step, the personal computer 10 displays a key image. The images already in the image database are clustered and a representative image is displayed. FIG. 8 shows the relationship between the images plotted on the search coordinates and clustered. The vertical axis represents the color of the image, and the horizontal axis represents the complexity of the image. The complexity of an image can be evaluated with a specific spatial frequency component, and the color of an image can be evaluated with a hue. When each image is analyzed to determine the color and complexity, and plotted on the search coordinates, it can be divided into, for example, five groups.

  By doing so, it becomes possible to perform clustering by analyzing the images in the image database. Even when a new image is registered in the database, it is only necessary to follow similar clustering, and no new attribute input is required. I'm sorry. Here, attributes refer to flowers, people, shooting data, and the like.

  Next, as a second step, an image is designated. Specifically, the user is allowed to designate a part considered to be most important. In this example, the person's face on the first screen is specified. The X mark displayed on the image of FIG. 9 is the position.

  This process can be efficiently learned by using the key image (the image closest to the image desired by the user). Of course, it is also possible to specify the attention point in order in another registration mode without using both.

  Further, as a third step, coordinates are recorded. The xy coordinates for this image are captured and recorded. This result is added to the data of the file or recorded in a separate file. Note that the personal computer 10 may automatically store information such as the purpose of use, the user, and the use account in association with the image whose position is specified.

  When it is repeatedly specified, a plurality of xy coordinates are generated, but all of them are recorded. As a result, the degree of concentration of a plurality of images indicates the importance in the image. If a plurality of position data are close to each other, the attention range is considered to be narrow. Moreover, if it is dispersed, it is considered wide.

  As a modification of the third embodiment, when a special place (for example, a corner) in an image is designated, other information can be given instead of giving the recognition of the priority position. For example, when a corner is designated, it is possible to search more conveniently if it is determined that the entire screen is the attention range, and that the attention range is half the size of the screen within a certain range in the center.

  Further, as a fourth step, the characteristics of the coordinate peripheral image are examined and the result is displayed. Analyzes the image around the point of interest and examines the color and complexity around the specified area. Even if the subject is the same, the subject may look different, for example, when the subject is photographed with a wide-angle lens and when it is photographed with a telephoto lens. Through such processing, the same subject can be easily recognized and searched.

  FIG. 10 is a diagram illustrating an example in which a searched image is displayed. In each image, the positions where the target pixel has already been designated are indicated by ☆ (star) marks. The user may specify the same location or a different location. If it is the same position, it becomes clear that the attention point of the image is that place, and if it is a different position, it can be determined that the entire image is viewed.

  As a fifth step, the search can be repeated. In the image displayed in FIG. 10, the user first selects an image he / she thinks, gradually approaches the desired image while specifying the position, and finally reaches the image. It is good to make it. In such a case, a search map can be used to inform the user of where the current search is being made.

  FIG. 11 shows a search map. For example, if the first key image is G1, the next key image is G2, the next key image is G3, the next key image is G4, and the images are connected by arrows, In some cases, the image GT may be surrounded. In this case, it can be seen that there is a high possibility that the image desired by the user is within the region surrounded by the arrow. On the other hand, when all the images are not searched but the key images overlap, a warning can be displayed as the search has entered a closed loop.

  As a modification of the present embodiment, it is conceivable to record the number of hard copies of an image. Since image retrieval is often performed to obtain a hard copy in the end, for example, an image with a large number of hard copies is considered to be an important image for the user, and it is more efficient if it is recorded. The possibility of performing a search increases. In addition, the hard copy can be reduced in priority as a search target, and by changing the display order, the convenience of the user can be achieved during the search.

  Furthermore, if information on the time of image creation or modification is always used to display an image of a close time, the user can easily recover the memory, and the search can proceed efficiently. For this, it is conceivable that an image is taken in one directory (one film or one memory in a digital camera) for a directory structure.

  As described above, according to the present embodiment, the search is performed based on the content specified in the position of the image, so that the search can be gradually performed with higher accuracy while being used. In addition, by letting the user specify a location that seems to be important in the image, the point of interest in the image becomes clear, and a highly accurate search can be performed.

  Furthermore, since the position designation is performed together with the designation of the key image for the image retrieval, no special effort is required, so that the retrieval can be simplified. In an image in which an attention point has already been specified, the information is displayed at the same time, so that the user can specify the second attention point when there are a plurality of attention points. In addition, the user can grasp whether or not attention has already been paid while browsing and searching the screen.

  According to this embodiment, when searching for a plurality of people or a plurality of purposes, it is possible to search for an image suitable for each. Further, by displaying the coordinates of the key image used for the search, the user can visually grasp how the search is performed. Further, by leaving a hard-copy record, it is possible to facilitate a search for hard copy and to make a hard copy of only necessary images. In addition, the image search accuracy can be improved.

Next, a fourth embodiment will be described. FIG. 12 is a diagram showing a display screen of the personal computer 10 and shows an example of searching for an image in the “image database” folder of the server 20 using a search keyword. As shown in FIG. 13, the personal computer 10 stores the search keyword in association with the feature amount and the associated URL. In the example shown in FIG. 13, for the keyword “sky”, a feature regarding the color of L ^* , a ^* , b ^* and a feature regarding the location where the color exists are given. The URL is used to search a home page on the Internet related to “sky”. In the example of FIG. 13, L ^* = 75, a ^* = − 7, b ^* = 35, the upper 30% in the vertical direction of the image, and the 100% area in the horizontal direction.

  When the user presses the search button, the personal computer 10 analyzes the image data in the “image database” folder and determines whether there is an image area that satisfies the feature amount of FIG. Images that satisfy the condition are sequentially displayed on the screen. The URL related to “empty” is displayed in the lower row. When the user selects this display, the personal computer 10 connects to the URL and downloads an HTML file, image data, etc., and displays the image in another window (not shown) or in the lower row.

  When the user selects the URL button on the browsing screen, the personal computer 10 sequentially connects to the URLs in Table 1 and downloads the image data. Further, for the downloaded image, an image satisfying the feature amount condition is searched and displayed on the screen. Similarly, the URL registered in FIG. 13 is displayed in the lower row.

  At the time of downloading, the personal computer 10 creates a folder with the URL name, and stores an HTML file, image data, and the like there. Then, a search is performed on the images in this folder.

  FIG. 14 is a diagram illustrating an example of an image search using an image file. First, the personal computer 10 reads initial image data from the server 20 and displays it on the screen. Here, when the user presses the search button, the feature amount of the image is extracted and collated with the registered keyword feature amount. The image data is downloaded from the URL of the keyword including the feature quantity, searched, and displayed on the screen.

  When the search folder is selected, the personal computer 10 extracts the feature amount from the image in the folder, and displays the image including the feature amount of the initial image on the screen. It is also possible to select a part of the initial image and perform a search using the feature amount (color, coordinates) of the selected part.

  It is also possible to prepare several templates as in the example shown in FIG. 13 so that the user can edit them. In addition, a network address such as an IP address can be set instead of the URL in FIG. Furthermore, the feature amount can have a width before and after the set numerical value.

  As described above, according to the present embodiment, image data can be downloaded from the Internet and used in response to a user's request, so that a wider search is possible.

  Next, a fifth embodiment will be described. In the search process, images similar to the key image are sequentially displayed on the personal computer 10. For example, when there are many similar images, these are searched one after another as a similar image and displayed on a narrow screen. If it is displayed on the screen, the target image is not easily reached, and it is expected that it takes time to search. In the fifth embodiment, this problem is alleviated as follows.

  In the present embodiment, as in the above-described embodiment, a plurality of images similar to the key image are sequentially selected from the image database and displayed, and one image is selected from the images. A plurality of images to be displayed next are determined. This is repeated until all images are displayed.

  The personal computer 10 determines the display order of images as follows. To obtain similarities of images in the database that have not yet been displayed with respect to the selected image and arrange them in descending order of similarity, for example, every second image is not displayed. As shown, the image is thinned out. That is, for example, only one of almost the same images, such as images continuously photographed at the same place, is displayed, thereby enabling high-speed browsing. For the calculation of the similarity, the color of the image content, the frequency characteristics, the structure, the photographing time, etc. can be used as in the above-described embodiment.

  Furthermore, the personal computer 10 can categorize the images in the database based on, for example, the directory name, and display the images for each category. When images are sequentially displayed and an image similar to the image to be searched for is displayed, it is possible to switch to the image search mode based on the images. At that time, an image that has been overlooked is also retrieved by performing a search including the previously displayed image.

  Further, the personal computer 10 can switch the display screen as follows. For example, a plurality of images can be displayed, and a plurality of images based on the images can be displayed each time the image is clicked. Further, the images are switched and displayed one after another at any time (see FIG. 15A), all are switched at once, or the images are displayed to flow (FIG. 15B). See also)). Such a display can also be realized by Java (registered trademark).

  In this case, the display interval can be arbitrarily adjusted by the user, the display can be temporarily stopped, and frame advance display can be performed on the previous image and the subsequent image. As shown in FIG. 15, the display number of the image can be additionally displayed at the lower right of the image so as to know how far the display has progressed. Such a number may be stored as a history in association with an image. In addition, a scroll bar or the like may be provided below the image, and the currently viewed image may be displayed around the entire image. Furthermore, it is also possible to display a part of the image by overlapping it and shifting it horizontally or vertically. In addition, the last displayed image may be displayed as “LAST”, for example, so that the user can recognize that it is the last.

  According to the present embodiment, images in the image database can be browsed quickly, and a display that is easy for the user to browse can be provided, which is more user-friendly.

  By the way, in the image database, if a keyword or the like is attached to the image, the image can be searched more efficiently. However, inputting a keyword or the like takes a lot of time and effort when there are many images, and the content of the keyword itself may not be known by the photographer himself / herself. In addition, even if a random keyword is attached, it may not be used in a later search. Also, inputting using a keyboard is usually a difficult task for a person who is not familiar with the keyboard, and there is a problem that it takes time. Therefore, in the sixth embodiment described below, this problem is alleviated as follows.

  FIG. 16 is a diagram for explaining the sixth embodiment. As shown in FIG. 16, the personal computer 10 selects and displays a plurality of images from the image database. The image is displayed one by one from the images clustered by the feature amount of the image information, for example, an image in which the photographing time is shifted every certain period, for example, or a random number is used as appropriate. A simple image may be displayed.

  Next, the user puts a check mark on a favorite image in the display image. More specifically, from the displayed images, an image that fits the theme that the user thought (for example, any theme such as “candidate New Year's card” or “snap during travel”), so-called “favorite” Click the check box CB on the right side of the image with a mouse or the like to add a check mark. In the example of FIG. 16, a check mark is added to the lower center image and the lower right image. The personal computer 10 registers such an image in the selected category. Furthermore, the images may be displayed in the display order used for high-speed browsing.

  Further, the user clicks an image close to the theme from among the displayed images, and causes the personal computer 10 to display a plurality of images having high similarity to the image, images close to the time, etc., or randomly. New images can be displayed. By repeating these steps, you can check your favorite images and images that match your theme.

  In order to confirm the checked image, the personal computer 10 can display a list of images, delete an image, or register additional images according to a user request. The user registers the images classified into this category in a batch by using comments, keywords, audio information, and the like. In addition, for example, “new year's card” or the like may be registered individually in the image.

  The voice information can be input by a microphone (not shown), converted into characters in the personal computer 10 and recorded together with the image in the image database, or the voice itself may be recorded together with the image. In this case, the similarity between images can be determined by analyzing the frequency of recorded sound and determining whether the frequency characteristics are similar.

  At the time of image search, a category newly created by the user as described above can also be used as one category for image search. For example, in the case of a search function that uses images with the same category as one index of approximation, images in the selected category are also used as one index of approximation.

  Also, images registered across multiple selected categories are considered important by the user and may have a high utility value, so it is recommended to increase the search priority based on category information. . When a face of a specific person is registered as the selected category, an image including the same or similar face is registered in the category, and a category (namely, album) for each person is completed.

  As described above, according to the present embodiment, since the user can add a new category in addition to the category automatically determined by the personal computer 10 in the image database, it is easier to search for each user. It has become a thing.

  The present invention has been described above with reference to the embodiments. However, the present invention should not be construed as being limited to the above-described embodiments, and can be modified or improved as appropriate. For example, image search software that executes the image search method according to the above-described embodiment can be recorded on a recording medium such as an FD or a CD and installed in the personal computer 10.

10 PC 20 Server 30 Film scanner

Claims (69)

A search unit that searches for an image similar to the search target image from an image database having a directory structure including one or a plurality of folders, and a display unit that displays the searched image Image search device. The search means includes means for obtaining a relative position on a time axis from a search target image when searching for an image, and searches for a similar image using the relative position as one of feature quantities. The image search device according to claim 1. The relative position on the time axis from the search target image is obtained from at least one of a folder name, a time when the image is registered, a time when the image is captured, and a time when the image is corrected. Image search device. 4. The image search apparatus according to claim 2, wherein the relative position on the time axis from the search target image is obtained via data from data stored in one or a plurality of recording media. 5. The search unit according to claim 2, wherein when searching for an image, all or part of images included in the same folder or a folder close in time axis are candidates for similar images. The image search apparatus described. The image search apparatus according to claim 2, wherein the display unit displays an image included in a folder that is the same or close to the time axis as an image that is a candidate for a similar image. A means for extracting a plurality of representative images from images included in the folder is obtained, and as a first step of search, a similarity with this representative image is obtained. The image search apparatus according to claim 1, wherein all images are searched for by obtaining similarity. Search means for searching for an image from the image database;
Display means for displaying the searched image,
The search means searches for images with high similarity at the edges of the images in images in the same or near folder on the time axis,
The image search apparatus, wherein the display means displays a composite image obtained by overlapping a plurality of images by overlapping end portions of images. Search means for searching for an image similar to the search target image from the image database;
Display means for displaying the searched image,
The image search apparatus is characterized in that the search means searches for similar images using the frequency of selecting an image as one of the feature quantities. Storing the frequency of image selection in the image database;
And a step of searching for an image similar to the search target image, with the frequency of selecting the image as one of the feature amounts. Search means for searching for an image similar to the search target image from the image database;
Display means for displaying the searched image;
An image search apparatus comprising: a display control unit that records a history of displayed images and restricts the display of the images displayed once. The image search apparatus according to claim 11, wherein the display control unit has a function of resetting the history. Searching for an image similar to the search target image in the image database;
Displaying the searched images as candidate images;
And a step of recording a history of the displayed image and excluding the image once displayed from the candidate image. The image search method according to claim 13, further comprising a step of resetting the history. Means for classifying the images in the image database into categories,
Search means for searching for an image similar to the search target image based on the category information in the image database;
Display means for displaying the searched image,
The display unit displays an image representing each category before displaying the image searched by the search unit. The image search apparatus according to claim 15, wherein the display unit sets a statistically classified cluster as a category. The image search device according to claim 15 or 16, wherein the display unit classifies the images into categories based on keywords added to the images. The image search apparatus according to claim 15, wherein the display unit classifies the images into categories based on the contents of the images. Classifying the images in the image database by category;
And displaying an image representative of the category. The image search method according to claim 19, further comprising a step of categorizing the statistically classified clusters. 21. The image search method according to claim 19, further comprising a step of classifying the image into categories based on a keyword added to the image. The image search method according to any one of claims 19 to 21, wherein the images are classified into categories based on the contents of the images. Search means for searching for an image similar to the search target image from the image database;
Display means for displaying the searched image,
The display means can display an image representing a color, and the search means searches for an image including a color similar to the color corresponding to the image displayed by the display means. Image search device. The image search apparatus according to claim 23, wherein the display means displays a color picker as an image representing a color. Displaying an image representing the color;
Searching an image database for an image containing a color similar to the color corresponding to the displayed image;
And a step of displaying the searched image. The image search method according to claim 25, further comprising a step of displaying a color picker as an image representing a color. Search means for searching for an image similar to the search target image from the image database;
Display means for displaying the searched image,
The image search apparatus characterized in that the display means acquires and displays an image not included in the image database, and the search means searches for an image similar to the image displayed by the display means. 28. The image search apparatus according to claim 27, wherein the display means displays the acquired image after reducing or trimming. Acquiring and displaying an image;
Searching the image database for an image similar to the acquired image;
And a step of displaying the searched image. 30. The image search method according to claim 29, further comprising a step of displaying the input image after reducing or trimming. Search means for searching for an image similar to the search target image from the image database;
An image search apparatus comprising: retouching means for collectively retouching displayed images. Searching for an image in an image database;
And a step of collectively retouching the searched images. Display means for displaying an image;
Storage means for storing the designated position when position designation is performed on the image displayed on the display means;
An image search apparatus comprising: search means for searching for an image in an image database based on the position stored in the storage means. The specified position is a part that is observed by an observer who observes the image displayed on the display means,
The image search apparatus according to claim 33, further comprising coordinate determination means for obtaining and storing coordinates of a designated position. 35. The image search apparatus according to claim 33 or 34, wherein the designated position is indicated on the image by the display means. 36. The image search apparatus according to claim 33, wherein at least one of a purpose of use, a user, and a use account is stored in association with an image whose position is designated. 37. The image search apparatus according to any one of claims 33 to 36, which illustrates which position in the coordinates of the search space corresponds to an image designated in the past. Storage means for storing information relating to hard copy for the images in the image database;
An image search apparatus comprising: search means for searching for an image based on information relating to the hard copy. 39. The image search apparatus according to claim 38, further comprising display means for preferentially displaying an image that has not been hard copied. 40. The image search apparatus according to claim 33, wherein the images in the image database are grouped in order of image creation or modification time. An input means for inputting character keywords;
An image search apparatus comprising: a setting unit that sets a feature amount of an image and a network address related to the keyword based on the character keyword input by the input unit. Extracting means for extracting the feature amount of the specific image;
Determining means for determining a character keyword including the feature amount extracted by the extracting means;
Means for obtaining an image from a network address associated with the determined character keyword;
An image search apparatus comprising: a determination unit that extracts a feature amount of an acquired image and determines whether or not the feature of the specific image is included. 43. The image search apparatus according to claim 41, wherein the feature amount of the image includes information on a color and a region in the image. An image search device that preferentially searches for an image in the same category when searching for an image in an image database,
An image search apparatus comprising an adding means for adding a category common to one or a plurality of images. 45. The image search device according to claim 44, wherein the adding means forms a new category by selecting one or a plurality of images. 46. The image search apparatus according to claim 44 or 45, further comprising a sorting unit that judges contents of an image and sorts the image into categories. 47. The image search apparatus according to claim 44, further comprising display means for displaying images of the same category. 48. The image search apparatus according to claim 44, wherein a category is newly formed by selecting an image in the same category displayed by the display means. 49. The image search apparatus according to claim 44, further comprising an adding unit that adds a comment to images of the same category. The image search device according to any one of claims 44 to 49, wherein when an image common to different categories is included, the search priority of the image is increased. 51. The image search apparatus according to claim 44, wherein when a person is included in an image, images including a face image of a specific person are classified into the same category. 52. The image search apparatus according to claim 51, wherein an image including a face image similar to the face image of the specific person is also included in the same category. An image search apparatus comprising: an accompanying unit that associates sound information with an image in an image database; and a search unit that searches for an image based on the sound information. 54. The image search apparatus according to claim 53, wherein the search means searches the image by converting the sound information into a character keyword. The image search apparatus according to claim 53 or 54, wherein the search means searches for an image based on an approximation of a frequency characteristic included in the sound information. Having a selection means for selecting an image;
The image search apparatus according to any one of claims 53 to 55, wherein the accompanying means attaches common sound information to the image selected by the selecting means. A selection means for selecting an image in the image database;
An ordering means for ordering the images in the image database with the selected image at the top;
An image display device comprising display means for displaying images according to ordering. 58. The image display apparatus according to claim 57, wherein the ordering unit sets an image having a higher degree of similarity to the selected image in an order closer to the selected image. 59. The image display apparatus according to claim 57 or 58, wherein the ordering unit sets an order in which an image having a shorter photographing time with respect to the selected image is closer to the selected image. The image display device according to any one of claims 57 to 59, wherein the ordering unit sets an image closer in content to the selected image so as to be closer to the selected image. 61. The image display device according to claim 57, wherein the display means displays the images ordered by the ordering means for each predetermined order. 62. The image display device according to claim 57, wherein the display means does not display an image once displayed. 63. The image display device according to claim 57, wherein the display unit sequentially displays the ordered images at predetermined time intervals. 64. The image display device according to claim 63, further comprising changing means for changing the predetermined time interval. 65. The image display device according to claim 63, further comprising interruption means for interrupting sequential display of images. 66. The image display device according to claim 57, wherein the display unit displays information indicating a relationship between a displayed image and an image to be displayed. 67. The image display device according to claim 57, further comprising search means for searching for a similar image based on the image displayed by the display means. 68. The image display apparatus according to claim 57, further comprising means for informing that the last ordered image is displayed. 70. The image display device according to claim 57, further comprising storage means for storing a display order of images displayed by the display means.

Images