JP4313738B2 - Image search apparatus and method, program thereof, and recording medium - Google Patents

Description

  The present invention relates to an image search apparatus and method for searching for an image existing on the Web, a program thereof, and a recording medium.

  2. Description of the Related Art Conventionally, an image search device is a system for searching for images (image data and image information) that operate on a computer, and refers to a technique for presenting an image based on information input from another computer. . As an example of such an image search apparatus, FIG. 6 shows a system configuration example of an image search apparatus according to the prior art.

  The image search device 20 shown in FIG. 6 includes an image index 21, an image search unit 22, a client output unit 23, and a client input reception unit 24. Further, the image search device 20 is connected to a network 4 such as the Internet, and the client 2 is also connected via the network 4.

  In the image search apparatus according to the prior art shown in FIG. 6, there are two types of search methods: a character string image search and a similar image search.

  First, in the case of character string image search, the image index 21 automatically generates character string information to be associated with an image from text on a page linked to an image on the Web (network 4), and stores this information. ing.

  A search character string accompanying an image search request from the client 2 is transferred from the client input receiving unit 24, and the character string exists in the character string information associated with the image stored in the image index 21 for the character string. Whether or not to do so is determined by the image search unit 22.

  The image search unit 22 transfers a screen that presents a list of images in which the search character string exists in the character string information to the client output unit 23. This method is used for a general Web image search service.

  Next, FIG. 7 shows a flowchart for the Web page collection robot (Web page collection means) according to the prior art to create an image index.

  A web page collection robot according to the prior art acquires a web page from a URI (uniform resource identifier) which is a seed for collecting a web page called a seed.

  First, collection of web pages is started in step 20: S20, and in step 21: S21, the seed web page is acquired from the network using a protocol such as HTTP (Hypertext Transfer Protocol).

  Next, in step 22: S22, the Web page is determined in order to classify whether the content of the acquired Web page is text or an image. If it is determined that the image is an image (YES), in step 27: S27, after temporarily storing the image, the image feature amount and text information are extracted and stored.

  On the other hand, if it is determined No in step 22: S22, a character string described in HTML (Hypertext Markup Language) or the like is analyzed in step 23: S23, and a URI to which the Web page links is obtained. To extract tag elements. The linked URI is used at the next collection.

  Next, in step 24: S24, text information obtained by removing the tag from the Web page is extracted.

  Next, in step 25: S25, the extracted text information is stored.

  Thereafter, in step 26: S26, it is determined that all Web pages to be collected have been acquired. If YES, the process ends. If NO, the process returns to step 21: S21 again.

In a similar image search, which is a different conventional technique, image feature amounts (texture, color, etc.) are extracted from an image in advance and stored in an image index. The image search unit extracts the feature amount of the image transferred from the client input reception unit, compares the feature amount with the feature amount stored in the image index, and automatically selects an image that is assumed to be similar. And a screen presenting a list of the images is transferred to the client output unit. (See Patent Documents 1 to 4)
JP 2000-250943 A JP-A-10-240765 JP 11-025271 JP-A-11-224256

  In the above-described conventional image search apparatus, when similar image search is performed using an image of a search result of character string image search as an input, search for various sizes and image quality as in a Web page on the Internet is performed. In order to do this, a large amount of data in which the number of dimensions of the image feature amount used for the similar image search is several hundred dimensions is required, and the amount of calculation has been enormous.

  In addition, there is a problem that the cost associated with an increase in the amount of calculation for automatically determining similar images is enormous, and it is difficult to construct a system that maintains a practical speed.

  The present invention has been made in view of these problems, and an object of the present invention is to classify information stored in an image index in advance with text information and image feature amounts of Web pages including images. Another object of the present invention is to provide an image search apparatus and method, a program thereof, and a recording medium that can limit the search target of the image search and reduce the calculation amount related to the image search.

In order to solve the problem, the present invention according to claim 1 is a client information terminal for issuing an image search request in an image search apparatus for searching for an image contained in a Web page in response to an image search request. Web page collection means for collecting information constituting the web page and extracting images and character strings; and the images and character strings extracted by the web page collection means in advance for classification. Category classification means for associating with set index information for each category, and image index information storage for classifying and storing similar images similar to the image for each category together with the image as an index for search And the image index information storage means, and the character string classified for each category is displayed as the similar image. The character string index information storage means for storing as an index for searching in association with the character string index information storage means, using the character string included in the image search request as a clue, and the character string index information storage means In the meaning of the character string retrieved from the image index information storage means
Character string image search means for presenting the similar image as a plurality of corresponding candidates to the client information terminal, and the similar image selected from the similar images presented to the client information terminal Inquires of the image index information storage means to acquire the image feature amount, and inquires of the character string index information storage means according to the selected similar image to acquire the category information. The image feature amount and the category information And a similar image search means for searching and acquiring an image stored in the image index information storage means and presenting it to the client information terminal again.

According to a second aspect of the present invention, there is provided an image search method for processing an image search apparatus for searching for an image contained in a Web page in response to an image search request, wherein the image search apparatus is used by a client information terminal. The step of issuing the image search request, the step of collecting information constituting the Web page by the Web page collection unit and extracting the image and the character string, and the image extracted by the Web page collection unit by the category classification unit And a step of associating the character string with index information for each category set in advance for classification, and a similar image similar to the image as an index for searching by the image index information storage means together with the image A step of classifying and storing each, and character string index information storage means A step of storing the character string classified for each category in association with the similar image as an index for searching in association with the image index information storage unit; The character string index information storage means is queried using the character string contained therein as a clue, and the character string index information storage means becomes a plurality of candidates corresponding to the meaning of the character string retrieved from the image index information storage means. A step of presenting a similar image to the client information terminal, and an image obtained by inquiring the image index information storage unit according to the similar image selected from the similar images presented to the client information terminal by the similar image retrieval unit. Get the feature value and also the selected similarity The query character string index information storage unit acquires category information, acquires the image feature amount and the category information as a search condition the image index information storage means searches the stored image according to the image, Again presenting to the client information terminal.

According to a third aspect of the present invention, there is provided an image search program to be executed by an image search device for searching for an image included in a Web page in response to an image search request. The step of issuing the image search request by the terminal, the step of collecting the information constituting the Web page by the Web page collection unit and extracting the image and the character string, and the Web page collection unit extracted by the category classification unit A step of associating the image and the character string with index information for each category set in advance for classification; and a similar image similar to the image by the image index information storage means as an index for searching together with the image Categorizing and storing each category, and character string index information Storing the character string classified for each category in association with the similar image and storing it as an index for search in association with the image index information storage unit by a storage unit; and the image search by a character string image search unit Querying the character string index information storage means using the character string included in the request as a clue, and a plurality of candidates corresponding to the meaning of the character string retrieved from the image index information storage means by the character string index information storage means Presenting the similar image to the client information terminal, and inquiring the image index information storage unit according to the similar image selected from the similar images presented to the client information terminal by the similar image search unit To obtain the image feature amount and select Is in accordance with the similar image query to retrieve the category information to the character string index information storage means has, by searching an image stored in the image index information storage means the image feature amount and the category information as a search condition Obtaining and presenting it again to the client information terminal.

According to a fourth aspect of the present invention, there is provided a recording medium storing an image search program to be executed by an image search device for searching for an image included in a web page in response to an image search request. A step of issuing an image search request to a device by a client information terminal; a step of collecting information constituting the Web page by a Web page collection unit to extract an image and a character string; and a Web page by a category classification unit A step of associating the image extracted by the collecting means and the character string with index information for each category set in advance for classification; and a similar image similar to the image by the image index information storage means for searching A step of classifying and storing together with the image as an index for each category; A column index information storage unit linked with the image index information storage unit, and storing the character strings classified for each category in association with the similar images as a search index; and a character string image search unit A plurality of characters corresponding to the meaning of the character string retrieved from the image index information storage means by the character string index information storage means by querying the character string index information storage means using the character string included in the image search request as a clue. Presenting the similar image as a candidate for the client information terminal to the client information terminal, and storing the image index information according to the similar image selected from the similar images presented to the client information terminal by the similar image search means acquiring an image feature quantity contact the unit Also query acquires category information to the character string index information storage means according to the selected similar image, stored in the image index information storage means the image feature amount and the category information as a search condition Retrieving and obtaining an image, and again presenting the image to the client information terminal.

  According to the present invention, it is possible to classify in advance the information stored in the image index with the text information and the image feature amount of the Web page including the image, it is possible to limit the search target of the image search, and the image search It is possible to provide an image search apparatus and method that can reduce the amount of calculation, a program thereof, and a recording medium.

  In the present invention, for an image (image data, image information) on the Web (network), whether or not a predetermined character string (character string information) exists in a database that stores text information associated with the image. Image search apparatus and method, a program and a record thereof, which are a combination of a character string image search program that searches for an image by determining the image and a similar image search program that searches for a similar image from a predetermined image It is a medium.

  FIG. 1 is a configuration diagram for explaining the overall configuration according to an embodiment of an image search apparatus of the present invention.

  FIG. 1 shows an image search apparatus 1, and the image search apparatus 1 is connected to a network 3 and a network 4 such as the Internet (www) and a client (client information) connected via the network 4. Terminal) 2 is connected.

  The image search apparatus 1 includes a client input receiving unit 10 for receiving an input from the client 2 connected to the network 4 therein, a client output unit 9 for outputting information to the client 2, and a network 3 A web page collection robot (web page collection means) 11 for connecting and collecting web pages, and a category classification unit (category classification means) for classifying the web pages collected by the web page collection robot 11 into predetermined categories. 12, an image index (image index information storage means) 5 for storing information classified by the category classification unit 12, and a character string index (character string index information storage means) operating in conjunction with the image index 5 ) 6, a similar image image search unit (similar image search means) 7, String image search section includes a (character string image retrieval unit) 8, a.

  An image search is executed in response to an image search request issued from the client 2, using the image search apparatus 1 of the present invention having such a configuration. The client 2 is constituted by a search request for performing an image search or an information terminal for acquiring the searched image and displaying the image. An image includes image data and image information for forming an image.

  An image search request issued from the client 2 is received by the client input receiving unit 10 via the network 4. The image search request received by the client input receiving unit 10 is sent to the character string index 6 and the image index 5 through the character string image search unit 8 and the similar image image search unit 7. Make an inquiry.

  In response to this inquiry, the character string index 6 is based on a character string related to the requested image, and a plurality of candidate similar images related to the meaning of the character string are linked to each other. Search from inside to get.

  The acquired plurality of similar images are presented to the client 2 via the client output unit 9, and are selected by an operator (not shown) who operates the client 2, and this selection instruction is input. Based on the selected similar image, the similar image search unit 7 searches and acquires the corresponding image from the image index 5. The image acquired by the similar image search unit 7 is presented to the client 2 via the client output unit 9.

  The character string information and image information (including similar images) stored in the image index 5 and the character string index 6 are information collected via the network 3 such as the Internet. Such information collection is executed by the web page collection robot 11. Information collection by the Web page collection robot 11 may be performed each time an image is requested from the client 2 or may be collected at a predetermined cycle.

  The information collected by the web page collection robot 11 is classified for each preset category by the category classification unit 12. Image information classified for each category is assigned index information (index information) indicating the category into which the image information is classified, and similarly, index information indicating its own category is also assigned to the character string information. Thereafter, the image index 5 and the character string index 6 are organized, stored, and saved.

  FIG. 2 shows a flowchart until the Web page collection robot 11 and the category classification unit 12 create information to be stored in the character string index 6 and the image index 5 according to the embodiment of the image search apparatus of the present invention. .

  First, in step 1: S1, collection of image information is started from a Web page existing on the network 3 such as the Internet.

  Next, in step 2: S2, Web page information is acquired from a Web page on the Internet. In this information acquisition, the Web page collection robot 11 acquires Web page information from a URI (uniform resource identifier), which is a seed for collecting Web pages called seeds. The web page collection robot 11 uses the protocol such as HTTP (Hypertext Transfer Protocol) for the seed web page, and acquires the target information in the HTML format from the web page on the network 3 (Internet).

  Next, in step 3: S3, HTML acquired from the Web page is analyzed, and link information is extracted. The link information includes a pointer to another HTML described by a URI and a link to an image.

  Next, in step 4: S4, all of the links obtained as a result of the HTML analysis acquired from the Web page whose link destination is an image are temporarily stored. Further, the image feature amount is extracted from the image data temporarily stored in this way. The extracted image feature amount is stored in the image index 5. The image index 5 can make an inquiry using the URI of the image as a primary key.

  Next, in step 5: S5, text information is extracted from the input HTML, and stored in the character string index 6 in next step 6: S6. Note that the text information here means text extracted from HTML or text in the vicinity of a tag to which an image is linked. In the present invention, the character string information to be stored is not limited.

  Next, in step 7: S7, the category to which the URI belongs is automatically determined based on the extracted image feature quantity and text information, and stored in the database of the character string index 6. There are various methods for performing category classification, but classification using machine learning such as Naive Bayes, c4.5, and SVM may be performed.

  In that case, in order to create a learning device, data to which category information is given in advance is required. In other cases, clustering using EM or K-Means may be performed for classification.

  Further, the category information is not limited, but the category in the case of giving data with category information given in advance may be classified into (idol, landscape, vehicle, food) and the like. For those that could not be classified, a category of “no classification” is created.

  In the Web image search described above, for example, regarding the classification, if the original page to which the image is linked is the parent page, the parent page is classified, and the parent page is classified for the images belonging to the Web page. A category may be used, or the image itself may be classified into categories using text around the image.

  In the case of classification on the parent page, images belonging to the parent page may be input as the information for classification. The category information automatically classified here is stored in the character string index 6.

  Next, in step 8: S8, it is determined whether or not the processing from step 2: S2 to step 7: S7 has been completed for all Web pages to be collected. Here, if it is determined that all the processes have been completed (YES), all the processes relating to the data collection of the Web page are completed in step 9: S9. On the other hand, if it is determined that the process has not ended (NO), the process returns to step 2: S2 and is repeated.

  FIG. 3 is an explanatory diagram for explaining the configuration of the database 15 of data stored and stored in the character string index 6 according to the embodiment of the image search apparatus of the present invention.

  The database 15 shown in FIG. 3 associates an image URI, category information classified using the image URI as a primary key, a URI of a parent page, and other text information (1). Have been organized and memorized. The other text information (1)... (N) means the title of the Web page or text near the link.

  FIG. 4 is an explanatory diagram for explaining the configuration of the database 16 of data stored and stored in the image index 5 according to the embodiment of the image search device of the present invention.

  The database 16 stored in the image index 5 includes various image features such as an image URI, a color histogram associated with the image URI as a primary key, a texture, and FEATURE (n). The amount is preserved.

  FIG. 5 shown next is a flowchart of the image search method according to the embodiment of the image search apparatus of the present invention. This image search method is executed in the image search apparatus 1 shown in FIG.

  First, in step 10: S10, an image search is started.

  Next, in step 11: S11, an inquiry is made to the character string index 6 using the character string related to the image request from the client 2 transferred from the client input receiving unit 10 as a clue. In response to this inquiry, the character string index 6 searches the image stored in the image index 5 and presents the image to the client 2 through the client output unit 9. Note that the image retrieved here and presented to the client 2 is a similar image similar to the meaning of the character string, and a plurality of similar images are presented to the client 2 according to the content meant by the character string.

  Next, in step 12: S12, the client 2 is operated to operate the similar image closest to the desired image requested to be searched from the images (similar images) output from the client output unit 9 and presented to the client 2. The operator selects and inputs.

  Next, in step 13: S13, an image feature quantity is acquired by inquiring of the image index 5 from the URI of the presented image. Further, the category information is obtained by inquiring the character string index 6 from the presented image URI.

  Next, in step 14: S14, if the category information is not “no classification” (NO) but an existing (YES) category as a result of the inquiry to the character string index 6, then the process goes to step 16: S16. Processing proceeds.

  In this step 16: S16, category information is added to the search conditions for the similar image search by the similar image search unit 7. For example, if the category information is “vehicle”, the vehicle category is searched, and the image searched from the image index 5 is presented to the client 2.

  On the other hand, in step 15: S15, when the category information is blank or “no classification” (NO), a similar image search is performed from the image index 5 without particularly adding category information, and the client output unit 9 shows the searched image to the client 2.

  Next, in step 17: S17, all of the image presentation to the client 2 is completed.

  According to the embodiment of the present invention described above, in an image search apparatus for searching for an image included in a Web page in response to an image search request, a client information terminal for issuing an image search request, and a Web page Web page collection means for collecting constituent information and extracting images and character strings, and image and character strings extracted by the Web page collection means are converted into index information for each category set in advance for classification. In association with category classification means for associating, image index information storage means for classifying and storing similar images similar to images for each category as an index for search, and image index information storage means, To store character strings classified by category in association with similar images as an index for search To retrieve a similar image corresponding to the character string included in the character string index information storage means and the character search request from the image index information storage means via the character string index information storage means, and present the similar image to the client information terminal The image stored in the image index information storage unit is retrieved and acquired in accordance with the similar image selected from the similar image displayed on the client information terminal and the similar image presented on the client information terminal, and again to the client information terminal An image search apparatus comprising similar image search means for presenting can be provided.

  Further, it is possible to provide an image search apparatus that includes the similar image selected by the client information terminal and the category information stored in the character string index information storage unit as search conditions of the similar image search unit.

  The image search apparatus according to the embodiment of the present invention assumes that an image search program that operates on a Web server on the Internet is stored in a recording medium such as a hard disk and executed. Therefore, the client side accesses the image search apparatus 1 using a Web browser via the network, and searches for an image through the client input reception unit 10 and the client output unit 9. This image retrieval method can also be applied to an independent stand-alone computer system that is not connected to the network 3 or the like.

  In the image search device according to the embodiment of the present invention, the information processing executed by the image search device and the operation of the constituent elements are replaced with a computer program, and a recording medium such as a CD, DVD, or FD is constructed. May be recorded. By incorporating these recording media into a computer system, or downloading a program recorded on the recording medium to a computer system via a communication line, or installing from the recording medium and operating the computer system with the program, Needless to say, it can function as an image retrieval apparatus and a means for realizing an image retrieval method, and by using such a recording medium, it is possible to improve its distribution.

  As described above, according to the embodiment of the present invention described above, the information stored in the image index can be classified in advance with the text information and the image feature amount of the Web page including the image. It is possible to provide an image search apparatus and method, a program, and a recording medium that can limit the target and reduce the amount of calculation related to the image search.

The block diagram for demonstrating the whole structure based on embodiment of the image search device of this invention is shown. The flowchart until the web page collection robot 11 and the category classification | category part 12 create the information memorize | stored in the character string index 6 and the image index 5 based on embodiment of the image search device of this invention is shown. An explanatory view for explaining a configuration of a database 15 of data stored and stored in a character string index 6 according to an embodiment of an image search device of the present invention is shown. An explanatory view for explaining a configuration of a database 16 of data stored and stored in an image index 5 according to an embodiment of an image search device of the present invention is shown. 3 shows a flowchart of an image search method according to an embodiment of an image search apparatus of the present invention. The system structure of a prior art is shown. 6 shows a flowchart for creating an image index by a prior art web page collection robot.

Explanation of symbols

1 Image search device 2 Client 3 Network (Internet)
DESCRIPTION OF SYMBOLS 4 Network 5 Image index 6 Character string index 7 Similar image image search part 8 Character string image search part 9 Client output part 10 Client input reception part 11 Web page collection robot (Web page collection means)
12 Category classification part (category classification means)

Claims (4)

In an image search apparatus for searching for an image included in a web page in response to an image search request,
A client information terminal for issuing the image search request;
Web page collection means for collecting information constituting the Web page and extracting images and character strings;
Category classification means for associating the image and the character string extracted by the Web page collection means with index information for each category set in advance for classification;
Image index information storage means for classifying and storing similar images similar to the image as an index for searching together with the image for each category;
Character string index information storage means for linking with the image index information storage means and storing the character strings classified for each category in association with the similar images as an index for search;
A plurality of characters corresponding to the meaning of the character string retrieved from the image index information storage means by the character string index information storage means by querying the character string index information storage means using the character string included in the image search request as a clue. A character string image search means for presenting the similar image as a candidate to the client information terminal;
The image index information storage means is inquired according to the similar image selected from the similar images presented to the client information terminal to acquire the image feature amount, and the image information is acquired according to the selected similar image. Inquires the character string index information storage means to acquire category information, searches and acquires the image stored in the image index information storage means using the image feature quantity and the category information as a search condition , and again the client information terminal Similar image search means for presenting to
An image search apparatus comprising:
In an image search method processed by an image search device for searching for an image included in a web page in response to an image search request,
By the image search device,
Issuing the image search request by a client information terminal;
Collecting information constituting the web page by a web page collection means and extracting an image and a character string;
Associating the image and the character string extracted by the web page collection means by category classification means with index information for each category set in advance for classification;
Classifying and storing similar images similar to the image by the image index information storage means together with the image for each category as an index for searching;
Correlating with the image index information storage means by a character string index information storage means, and storing the character strings classified for each category in association with the similar images as a search index;
The character string image search means inquires of the character string index information storage means for a character string included in the image search request, and the character string retrieved from the image index information storage means by the character string index information storage means. Presenting the similar image as a plurality of candidates corresponding to the meaning of the client information terminal;
The image index information storage means is inquired according to the similar image selected from the similar images presented to the client information terminal by the similar image search means to acquire the image feature amount, and the selected similarity Inquires the character string index information storage means according to the image to obtain category information, retrieves and obtains the image stored in the image index information storage means using the image feature amount and the category information as a search condition , Again presenting to the client information terminal;
An image search method characterized by comprising:
In an image search program to be executed by an image search device for searching for an image included in a web page in response to an image search request,
In the image search device,
Issuing the image search request by a client information terminal;
Collecting information constituting the web page by a web page collection means and extracting an image and a character string;
Associating the image and the character string extracted by the web page collection means by category classification means with index information for each category set in advance for classification;
Classifying and storing similar images similar to the image by the image index information storage means together with the image for each category as an index for searching;
Linking with the image index information storage means by a character string index information storage means, and storing the character strings classified for each category in association with the similar images as an index for searching;
The character string image search means inquires of the character string index information storage means for a character string included in the image search request, and the character string retrieved from the image index information storage means by the character string index information storage means. Presenting the similar image as a plurality of candidates corresponding to the meaning of the client information terminal;
The image index information storage unit is inquired according to the similar image selected from the similar images presented to the client information terminal by the similar image search unit to acquire the image feature amount, and the selected similarity Inquires the character string index information storage means according to the image to obtain category information, retrieves and obtains the image stored in the image index information storage means using the image feature amount and the category information as a search condition , Again presenting to the client information terminal;
An image search program characterized in that
In a recording medium on which an image search program to be executed by an image search device for searching for an image included in a web page in response to an image search request is recorded.
In the image search device,
Issuing the image search request by a client information terminal;
Collecting information constituting the web page by a web page collection means and extracting an image and a character string;
Associating the image and the character string extracted by the web page collection means by category classification means with index information for each category set in advance for classification;
Classifying and storing similar images similar to the image by the image index information storage means together with the image for each category as an index for searching;
Linking with the image index information storage means by a character string index information storage means, and storing the character strings classified for each category in association with the similar images as an index for searching;
The character string image search means inquires of the character string index information storage means for a character string included in the image search request, and the character string retrieved from the image index information storage means by the character string index information storage means. Presenting the similar image as a plurality of candidates corresponding to the meaning of the client information terminal;
The image index information storage unit is inquired according to the similar image selected from the similar images presented to the client information terminal by the similar image search unit to acquire the image feature amount, and the selected similarity Inquires the character string index information storage means according to the image to obtain category information, retrieves and obtains the image stored in the image index information storage means using the image feature amount and the category information as a search condition , Again presenting to the client information terminal;
A computer-readable recording medium on which an image search program is recorded.

Images