JPH11219365A - Image retrieving device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To easily retrieve target image data by determining whether or not an image belongs to the category of an inputted key word from clustering and image characteristics obtained by a clustering neural net part and an image characteristic extraction part, and displays the image when it belongs to the category. SOLUTION: On a secondary storage 5, an image data base(DB) 6 stores a large amount of image data, an image read-in part 7 reads in an image, and a composition information extraction part 8 extracts composition information from the image. The clustering net(NN) part 9 performs clustering on the basis of the composition information of the image, the image characteristic extraction part 10 extracts image characteristic information on a color phase, luminance, etc., from the image, and a decision NN part 11 determines whether or not the given image belongs to the category of a specified key word. An image display part 12 displays the image on a display. Here, the image DB 6 can be provided as a memory in the device.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image retrieval apparatus, and more particularly to a technique for automatically retrieving a target image from a large amount of image data.

[0002]

2. Description of the Related Art In recent years, with the spread of the Internet, digital cameras, and the like, opportunities to come into contact with our digital images have been more and more increased. In addition, due to price reduction and large capacity storage, an environment has become available in which not only offices but also individuals can store a large amount of image data.

Therefore, an environment for efficiently handling digital images with respect to such a large amount of image data is required, and an image retrieval apparatus capable of easily presenting image data intended by a user has attracted attention. Have been.

FIG. 8 is an explanatory diagram showing the configuration of a conventional image search device. As shown in the figure, a conventional image search device includes a central processing unit 20, a display device 21 such as a monitor for displaying a color image, an input device 22 such as a keyboard and a mouse, and temporarily stores a program or data being processed. , And a secondary storage device 24 for storing programs, data, and the like. Also,
The secondary storage device 24 includes an image database (image DB) 25 that stores a large amount of image data, an image reading unit 26 that reads one piece of image data, an image display unit 27 that displays an image on a display, and a search Keyword registration unit 28 for registering a keyword to be used in association with an image
And a keyword table 29 for securing the keywords registered with the image data in association with each other, and a keyword check unit 30 for searching for a keyword specified by the user
It consists of

Next, the process of image retrieval using a keyword in such a conventional image retrieval apparatus will be described with reference to the flowcharts of FIGS.

In order to search for a large number of images by using a keyword, it is necessary to register a keyword associated with an image for each image. Therefore, the image registration process will be described with reference to FIG.

To register an image, first, an image DB storing a large amount of image data is accessed (step S2).
0). Then, one image to be registered is loaded into the memory (step S21).

Next, an arbitrary number of keywords associated with the image are registered (step S22).

Finally, the captured image is associated with the keyword registered in the image and registered in the table (step S23).

Thus, in an environment in which a registered keyword table exists, an image can be searched for using a keyword.

Next, referring to FIG. 10, a process of searching for a registered image will be described. First, the user inputs a keyword associated with a desired image (step S2).
4). Thereby, the keyword table is read (step S25).

Next, the presence or absence of a keyword specified by the user in step S24 is checked in the registered keyword table (step S26).

When the designated keyword is found, the image DB is accessed using the index of the image (step S27).

Finally, the retrieved image is fetched and displayed on the display (step S28).

[0015]

However, such a conventional apparatus has the following problems.

First, there is a problem that a task of inputting a keyword to an image is troublesome. Another problem is that the keyword attached to an image cannot be specified well during a search because the user forgets which keyword has been assigned to which image over time. In particular, since it is very difficult to associate a keyword assigned by another person, it is difficult to find an appropriate keyword.

Therefore, the retrieval accuracy in this keyword-based image retrieval, which is the mainstream of the current image retrieval, is never satisfactory.

Accordingly, an object of the present invention is to provide an image retrieval apparatus capable of easily retrieving target image data from a large amount of image data.

[0019]

In order to solve this problem, an image search apparatus according to the present invention comprises: an input device for inputting a search target image as a predetermined keyword; an image database in which image data is stored; An image reading unit that reads an image to be detected extracted from an image database, a composition information extraction unit that extracts composition information from an image, and a composition information of the image extracted by the composition information extraction unit.
An image was input from a clustering neural network unit that performs clustering by a neural network, an image characteristic extraction unit that extracts image characteristic information of an image, and clustering and image characteristics obtained by the clustering neural network unit and the image characteristic extraction unit. It is characterized by having a discrimination neural network unit for discriminating whether or not the image belongs to the keyword category, and an image display unit for displaying the image when it belongs to the keyword category.

Thus, even when the composition of the image is different, it is possible to easily search for the target image data from a large amount of image data.

[0021]

DESCRIPTION OF THE PREFERRED EMBODIMENTS The invention according to claim 1 of the present invention provides an input device for inputting a search target image as a predetermined keyword, an image database in which image data is stored, and extraction from the image database. An image reading unit that reads the detected image that has been detected, a composition information extraction unit that extracts composition information from the image, and a clustering neural that performs clustering by a neural network based on the composition information of the image extracted by the composition information extraction unit. A net unit, an image characteristic extracting unit for extracting image characteristic information of the image,
A clustering neural network unit and a discriminating neural network unit for judging whether or not an image belongs to the category of the input keyword from the clustering and image characteristics obtained by the image feature extracting unit, and a case where the image belongs to the keyword category In addition, the present invention provides an image search device having an image display unit for displaying the same, and even when the composition of the images is different, it is possible to easily search for the target image data from a large amount of image data. Having.

[0022] The invention described in claim 2 of the present invention provides:
According to the first aspect of the present invention, the image database is an image search device in the device, and even when the composition of the images is different, the target image data can be easily searched from a large amount of image data. It has the effect of becoming.

According to a third aspect of the present invention, in the first aspect of the present invention, the image database is an image search device which is a plurality of images scattered on a network.
Even when the composition of the images is different, there is an effect that the target image data can be easily searched from a large amount of image data.

According to a fourth aspect of the present invention, in the first, second or third aspect of the present invention, the clustering neural network unit is an image search device in which a user learns a composition pattern by a user. It is possible to search for an accurate image for the purpose of (1).

Hereinafter, an embodiment of the present invention will be described with reference to FIG.
This will be described with reference to FIG. First, a neural network (NN) used in the embodiment of the present invention will be briefly described.

The NN connects a large number of models of nerve cells (neurons) of the human brain to form a network, and constructs an information system by transmitting information between the neurons. Further, by applying teacher data to certain input data and performing a learning process called back propagation (BP method), it is possible to construct a nonlinear mapping in the NN.

One of the important functions of the NN, the hierarchical NN
The learning function in the back propagation method will be described with reference to FIG.

The learning of the NN is performed by changing the weight between neurons. So, at the start of learning,
Initial values are given to those weights (step S10).

Next, input data of an arbitrary dimension is given to the NN (step S11). Thereby, information is propagated between neurons constituting the NN (step S1).
2).

Next, an ideal value for the current input data is given as teacher data (step S13). Then
The neuron information is updated in the reverse order using the difference between the NN output data and the teacher data (BP method) (step S14).

Then, from step S11 to step S1
4 is repeated a specified number of times N (step S1).
5) When the number of times of learning exceeds N, learning ends.

Next, the learning of the clustering NN for performing clustering based on the composition information of the image used in the embodiment will be described with reference to FIGS.

FIG. 2 is an explanatory diagram showing a clustering process based on image composition information, and FIG. 3 is a flowchart showing a learning process procedure of the clustering NN.

In order to construct a clustering NN,
It is necessary to create teacher data. Therefore, clustering using the composition information of the image is performed, and the group name or the representative value is used as teacher data when constructing the clustering NN. Therefore, as shown in FIG.
Clustering is performed on the images belonging to a certain keyword in B based on the composition information of the images. The clustering algorithm uses a generally used neighborhood method, k-means method, or the like. In the example of FIG. 2, images belonging to a certain keyword category stored in the image DB are group A, group B, group C,
Clustered into group D. Therefore, when learning the clustering NN, the group name or the representative value of the group is used as the teacher data for the composition information of a certain image.

As the composition information of the image, the area is divided by an area dividing method based on changes in hue, texture, etc., and the contour length of the area, the curve ratio, the relation with the adjacent area, the moment of inertia of the area And a description that can express the composition of the image such as the moment of the center of gravity.

Referring to FIG. 3, a description will be given of a learning process of the clustering NN. In the learning process of the clustering NN, first, an image included in a certain keyword category is read from the image DB (step S16).

Next, composition information is extracted from the read image (step S17). Then, composition information is given to the NN as input data, and NN learning is performed using the group name or the representative value of the clustering result as the teacher data (step S18).

When learning is performed a specified number of times in this way,
A clustering NN is constructed (step S19).

The clustering NN has a function of, when a certain image is given, outputting a group name or a representative value when clustering is performed based on the composition information.

Next, a method of constructing a discrimination NN for judging whether or not an arbitrary image is an image including a certain subject will be described with reference to FIG. Here, FIG. 4 is an explanatory diagram showing, as an example, a procedure for constructing a discrimination NN for discriminating an image belonging to the keyword category of “sea”.

The input data of the discrimination NN is obtained from the output data of the clustering NN of the images belonging to the keyword category of "sea" and the image characteristics such as hue, luminance, texture (distribution of fine uniform patterns, etc.). Information. The clustering NN outputs a composition pattern to which a currently input image belongs, among a plurality of composition patterns of an image belonging to the keyword “sea”. For the input data, whether the image is an image of “sea” or an image that does not belong to the keyword of “sea” is given as teacher data. By repeating this learning, the "sea" of multiple different compositions
Can be determined.

Based on the above description of the NN,
This embodiment will be described. FIG. 5 is a block diagram showing an image search device according to an embodiment of the present invention, FIG. 6 is a flowchart showing an image search procedure by the image search device of FIG. 5, and FIG. It is explanatory drawing which shows an example.

As shown in the figure, the image retrieval apparatus according to the present embodiment includes a central processing unit 1, a display device 2 for displaying a color image such as a monitor, an input device 3 such as a keyboard and a mouse, It comprises a main storage device 4 for temporarily storing programs and data, and a secondary storage device 5 for storing data and programs. The secondary storage device 5 includes an image DB 6 that stores a large amount of image data,
An image reading unit 7 for reading an image; a composition information extracting unit 8 for extracting composition information from the image; a clustering NN unit 9 for performing clustering based on the composition information of the image; an image for extracting image characteristic information such as hue and luminance from the image Characteristic extraction unit 1
0, a discrimination NN unit 11 for discriminating whether or not a given image belongs to a category of a designated keyword, and an image display unit 12 for displaying an image on a display. Note that the image DB 6 can be provided as a memory inside the apparatus, but it can also apply a plurality of images scattered on a network such as the Internet.

Next, referring to FIG. 6, the procedure of an image search by such an image search apparatus will be described in detail.

When a user inputs an image to be searched for as a "keyword" (step S1), a clustering NN unit for clustering images belonging to the category of the keyword based on composition information is read (step S1).
2).

Next, the search target image is checked (step S3), and if there is no target image, the process ends (step S3).

If there is image data to be searched in the image DB 6, one image is read by the image reading section 7 (step S4).

Then, composition information is extracted from the read image (step S5), and the extracted composition information is input to the clustering NN unit 9 (step S6).

At the same time, image characteristics are extracted from the read image (step S7).

Then, the clustering N in step S6
Based on the output data in the N unit 9 and the image characteristic information extracted in step S7, the image is input to the determination NN unit 11 for determining whether the image belongs to the category of the keyword (step S8).

When the discrimination NN unit 11 determines that the image belongs to the category of the designated keyword, the image is displayed on the display (step S9).

Next, a specific example of such an image search is shown in FIG.
This will be described with reference to FIG. Here, it is assumed that the user pays attention to the sea and inputs "sea" as a keyword.

Then, the image search device according to the present embodiment
The image DB 6 to be searched is read, and the images are checked using the clustering NN unit 9 and the discrimination NN unit 11. As a result, if it is determined that the loaded image belongs to the keyword category of "sea",
Display on the display. At this time, an image having not only one composition but also a plurality of compositions such as composition A, composition B, and composition C clustered based on the composition information is displayed. In addition, depending on the keyword, there may be only one composition, and in this case, one image is displayed. In addition, by performing additional learning of the clustering NN on its own, a search corresponding to an arbitrary number of composition patterns can be performed.

[0054]

As described above, according to the present invention, even when the composition of an image is different, it is possible to easily search for a target image data from a large amount of image data. Is obtained.

Further, by constructing the clustering neural network unit by learning the composition pattern by the user, an advantageous effect that it is possible to search for an accurate image special for each user's purpose is obtained.

[Brief description of the drawings]

FIG. 1 is an explanatory diagram showing a learning function in a back propagation method of a hierarchical neural network.

FIG. 2 is an explanatory diagram showing a clustering process based on image composition information.

FIG. 3 is a flowchart showing a learning processing procedure of the clustering neural network.

FIG. 4 is an explanatory diagram showing a procedure for constructing a discrimination neural network for discriminating an image belonging to a category of a predetermined keyword;

FIG. 5 is a block diagram showing an image search device according to an embodiment of the present invention.

FIG. 6 is a flowchart showing an image search procedure by the image search device in FIG. 5;

FIG. 7 is an explanatory diagram showing a specific example of an image search by the image search device in FIG. 5;

FIG. 8 is an explanatory diagram showing a configuration of a conventional image search device.

FIG. 9 is a flowchart showing an image registration process in the image search device of FIG. 8;

FIG. 10 is a flowchart showing a search process of a registered image in the image search device of FIG. 8;

[Explanation of symbols]

 Reference Signs List 3 input device 6 image database 7 image reading unit 8 composition information extracting unit 9 clustering neural network unit 10 image characteristic extracting unit 11 discriminating neural network unit 12 image display unit

──────────────────────────────────────────────────の Continued on the front page (51) Int.Cl. ⁶ Identification code FIG06F 15/70 330Q

Claims (4)

[Claims] An input device for inputting a search target image as a predetermined keyword; an image database storing image data; an image reading unit for reading a detection target image extracted from the image database; A composition information extraction unit that extracts composition information from the image; a clustering neural network unit that performs clustering by a neural network based on the composition information of the image extracted by the composition information extraction unit; and image characteristic information of the image. A clustering neural network and a clustering and image characteristic obtained by the image characteristic extracting unit, and a discrimination neural that determines whether or not the image belongs to the category of the input keyword. The net part and the image belongs to the keyword category And an image display unit for displaying the image when the search is performed. 2. The image retrieval apparatus according to claim 1, wherein said image database is provided in said apparatus. 3. The image database according to claim 1, wherein the image database is a plurality of images scattered on a network.
The image search device according to the above. 4. The image retrieval apparatus according to claim 1, wherein the clustering neural network is configured to allow a user to learn a composition pattern.