JPH0248772A - Analogous picture retrieving system - Google Patents

Abstract

PURPOSE:To realize retrieval to meet the sense of a man by using an example picture, and grasping all its features, and determining the degree of analogy when picture data is retrieved at a terminal. CONSTITUTION:When a user inputs an input example picture 34 to the inputting part 31 of the terminal 3, a feature extracting part 23 extracts an element to show the feature of the picture from an input data storage part 22 through a control part 21 similarly to the time of storage. After extraction is completed, and analogical degree calculating part 24 is started, and after the calculation of the degree of analogy is completed, in analogical degree determining part 25 determines analogous picture candidate order for the input example picture on the basis of the result of the calculation by the analogical degree calculating part 24. Then, it takes out necessary picture data from a data storage part 210, and displays it on the displaying part 33 of the terminal 3. Thus, the retriev al to be adapted to picture contents and to meet the sense of a man can be realized.

Description

[Detailed description of the invention] (Technical field to which the invention pertains) The present invention relates to a similar image search method targeting the content of images in a system for searching necessary images from an image database in response to a request from a terminal. .

(Prior Art) Conventionally, in similar image retrieval systems of this kind, the retrieval keys are keywords such as words and traffic lights, and the image database is composed of keyword classifications determined at the time of construction.

By the way, when an image database is large in size and the content expressed is complex, the classifications prepared in advance in the image database are not sufficient for classification, and the images are often stored in positions that do not match the image content, so keywords such as words and signals are necessary. In many cases, it is not possible to search for the desired image in one go, and it is necessary to reconsider the keywords such as words and traffic lights that are thought to contain the desired image, and to search multiple times. There was a problem in that it was not possible to perform a search based on the content of images that matched the image.

(Objective of the invention) The present invention aims to solve the following points,
By inputting an example image from a terminal, grasping the image content to be searched, and narrowing down the search accurately, images similar to the example image content are presented to the terminal from the image database according to the candidate ranking, so that the user can search according to the image content. The goal is to realize a search that matches the senses of people.

(Structure of the Invention) (Characteristics of the Invention and Differences from the Prior Art) In the present invention, when an image database creator inputs an image for storage, the features of the input image are extracted, and image data is created based on the features. An index of both storage parts of image features is created and stored in an index file, and is also displayed to the image database creator for confirmation, and an image database is created. Next, in the search process, when the user inputs an input example image, its features are extracted in the same way as when storing them, and based on this, similarity is calculated using the feature parameters from the feature storage section while referring to the index file. At this time, a plurality of indexes of image data having the corresponding feature parameters are searched, and an evaluation scale indicating the similarity between the input exemplary image and each candidate image data is calculated. Then, the similar image candidate ranking for the input example image is determined, the candidate images are retrieved from the image data storage section with reference to the index file, and the necessary images are displayed on the user terminal.

Since the conventional technology uses a search method using keywords such as words and signals, it cannot adequately handle the large scale of the image database and the large content of expression.

The present invention differs in that it is possible to search for a desired image using input example images without using such keywords such as words or signals.

(Embodiment) The figure is a block diagram showing the configuration of an embodiment of the system of the present invention. 11 of the image database creation device 1 is the storage image 1
4, an input section 12, a control section for controlling communication with the image database system, 13, a display section for checking stored images, and 14, an image for storage input by the image database creator using the input section 11.

21 of the image database system 2 is a control unit that controls the overall operation of this system; 22 is an input data storage unit that stores the storage images 14 and input example images 34; 23 is an input image stored in the input data storage unit; 24 is a similarity calculation unit between the stored image and the input example image; 25 is a similarity determination unit that determines similar image candidates for the input example image based on the results calculated by the similarity calculation unit 24; 26 is an index creation section, 27 is an index file, 28 is an image feature storage section, 29 is an image data storage section, and 210 is an output data storage section for extracting candidate images.

Further, 31 of the terminal 3 is an input section through which the user inputs the input example image 34, 32 is a control section that controls communication with the image database system 2, 33 is a display section that displays candidate images, and 34 is a section where the user inputs the input example image 34. This is an example image of input input using the input unit 31.

Next, the operation of this embodiment will be described. The operation of this system can be roughly divided into two parts. One is image database creation processing, and the other is search processing. The above two will be explained in order.

First, a two-image database creator inputs the storage image 14 into the input section 11 of the image database creation device 1. The control unit 12 calls the image database system 2 and transfers the input image from the input unit 11 to the input data storage unit 22 via the control unit 21. After that, the control unit 21 activates the feature extraction unit 23.

The feature extraction unit 23 receives elements a i (i=1. 2, 3...; i indicates the identifier of the image feature).

When the extraction is completed, the control unit 21 is notified of the extraction.

When the control unit 21 receives the notification, it controls the index creation unit 26.
Start. The index creation unit 26 stores images 14 for storage.
An index for the image data storage section 29 and the image time Wi storage section 28 is created based on the element ai representing the feature obtained from the index file 27.

When the storage is completed, the controller 21 is notified of the storage.

The control unit 21 follows the contents of the index file 27,
The image data and the element ai representing its characteristics are stored in the image data storage section 29 and the image feature storage section 28, respectively.

At the same time, the control unit 21 calls the image database creation device 1 and displays it on the display unit 13 via the control unit 12. The image database creator looks at the display on the display unit 13 and
For example, when a confirmation instruction is input to the input section 11, the control section 1
2 calls the image database 2 and notifies the control unit 21. When the control unit 21 receives the confirmation instruction, the series of processing ends.

Next, in the search process, the user inputs the input example image 34 to the terminal 3.
input to the input section 31 of. The control unit 32 calls the image database system 2 and sends the input image to the input unit 31.
The input data is then transferred to the input data storage section 22 via the control section 21. After that, the control unit 21 starts the feature extraction 23.

The feature extraction unit 23 extracts elements a i (i=1
, 2, 3. ) is extracted in the same way as when accumulating.

When the extraction is completed, the control unit 21 is notified of the extraction.

Upon receiving the notification, the control unit 21 activates the similarity calculation unit 24. The similarity calculation unit 24 refers to the index file 27 via the control unit 21 based on the element ai representing the feature acquired from the input example image 34, and calculates the image feature storage unit 28.
The feature parameters are obtained from the above, and these feature parameters are compared with the element al representing the feature. An evaluation scale (similarity degree) that searches for multiple indexes of image data having the corresponding feature parameters and indicates the state of similarity between the input example image 34 and each candidate image data S J F”ly2+:L・
...;j is the identifier of the candidate image). Here, s
Although various types of expressions can be considered for j, for example, weight mji(j''L2
t3+...;i=1, 2, 3. ), and s
j = mjlal + mj2a2+ (j”
L2e3+...) may be considered.

The degree of similarity sj is expressed numerically from most similar to least similar (for example, from 0 to 1), and is used to rank the degree of similarity.

When the similarity calculation is completed, the controller 21 is notified.

When the control unit 21 receives the notification, it activates the similarity determination unit 25. The similarity determination unit 25 determines the ranking of similar image candidates for the input example image 34 based on the results calculated by the similarity calculation unit 24.

According to this determination, the index file 27 is referred to via the control section 21, candidate images are sequentially extracted by the image data storage section 29, and after adjusting the output format, are transferred to the output data storage section 210.

When the transfer is completed, the control unit 21 receives a request from the terminal 3 (for example, displays only the first candidate, displays the nth candidate (n=1.2°3, .
. When the control section 32 receives the image data, it displays it on the display section 33.

After the above two processes, the user finds the desired image among the candidate images, and by changing the input example image 34, the user can extract the desired image and similar images.

Note that the input example image 34 includes not only images but also words.

Information may be limited by including symbols.

(Effects of the Invention) As explained above, the present invention uses an example image when a terminal searches for image data to grasp all the characteristics of the example image, and calculates the degree of similarity using an evaluation scale obtained from a plurality of characteristics. Because it can be determined, any image can be quantitatively classified using an evaluation scale.

This method has the advantage of being able to perform searches that match the human senses based on the content of the image.

[Brief explanation of the drawing]

The figure is a block diagram showing the configuration of an embodiment of the present invention. 1... Image database creation device, 2... Image database system, 3... Terminal, 11...
Input section, 12... Control section, 13... Display section, 14.
... Image for storage, 21... Control unit, 22... Input data storage unit, 23... Feature extraction unit, 24... Similarity calculation unit, 25... Similarity determination unit, 26. ... Index creation section, 27 ... Index file, 28
... image feature storage unit, 29... image data storage unit,
210... Output data storage section, 31... Input section, 3
2... Control unit, 33... Display unit, 34... Input example image. Patent applicant Nippon Telegraph and Telephone Corporation

Claims (1)

[Claims] An image retrieval system that stores and searches images includes a first means for extracting features indicating the content of the image from an input image, and a second means for creating an index file based on the features obtained by the first means. 2, and the first means
a third means for accumulating the characteristics of the input image and the storage image obtained by the means based on the index file; and a third means for accumulating the characteristics of the input image and the image for accumulation based on the index file; a fourth means for calculating an evaluation scale from the features of the input image obtained by the first means for evaluation; and a fifth means for determining the degree of similarity from the evaluation scale obtained by the fourth means. It is characterized by comprising means for ranking candidates based on the degree of similarity determined by the fifth means, and searching for a necessary image from an image database according to the classification and ranking of candidates based on the degree of similarity. Similar image search method.