JP3500930B2 - Keyword assignment method and keyword automatic assignment device - Google Patents

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a keyword assigning method for automatically assigning a keyword to an object registered in a storage unit, and an automatic keyword assigning apparatus using this keyword assigning method.

[0002]

2. Description of the Related Art With the rapid spread of computers, electronic information handled by people has been rapidly increasing. Such a large amount of electronic information is generally stored together with a keyword, and by designating this keyword, desired electronic information can be searched.

As a method of adding a keyword to such electronic information, there is a method described in Japanese Patent Laid-Open No. 6-295318. In this keyword assigning method, for example, when assigning a keyword to image data of a human face photograph and registering it, the following is performed.

In advance, "face length" and "round face" of a human face
Register various keyword candidates such as. Further, for each keyword candidate, the fitness calculation information and the threshold value used when calculating the fitness are registered. For example, when the keyword candidate is "face length", the fitness calculation information is "contour aspect ratio". Next, one keyword candidate “face length”, the fitness calculation information and the threshold value related to this keyword candidate “face length” are read out, and the suitability of the keyword candidate “face length” is read from the feature amount of the image data to which the keyword is attached. The compatibility is calculated based on the calculation information. Then, if the obtained matching degree is equal to or larger than the threshold value of the keyword candidate “face length”, it is determined that the keyword candidate “face length” is a keyword to be added to the image data. Hereinafter, the same operation is performed for all the registered keyword candidates, and the keywords to be given are determined. In this way, the objective keyword assignment is performed regardless of the subjectivity of the registration worker.

As another keyword assigning method, there is a method described in Japanese Patent Application Laid-Open No. 8-329096.
In this keyword assigning method, for example, pleasure-discomfort and strength-
It defines a two-dimensional keyword map that has two axes, weak and weak. And when registering a keyword,
The keywords to be registered are arranged manually at points on the keyword map. By doing so, in the case of ambiguous search for registered image data, if the designated keyword and the distance on the axis of the keyword map are designated, the position within the designated distance is centered on the position of the designated keyword on the keyword map. The image data is searched using the keyword as a search condition.

[0006]

However, in the keyword assigning method described in Japanese Patent Laid-Open No. 6-295318, the degree of relevance among all keyword candidates is unknown. Therefore, there is a problem that it is not possible to specify one keyword at the time of image search and perform an ambiguous search in which an image search is performed based on the specified keyword and a similar keyword located near the specified keyword. Further, the calculation of the goodness of fit based on the goodness of fit calculation information and the comparison with the threshold value have to be repeated for the number of registered keyword candidates, which causes a problem that the image data registration work takes time. Further, it is necessary to set and register appropriate fitness calculation information and threshold values for each keyword candidate in advance, and although the keyword assignment work itself is automatically performed, it is troublesome and difficult to construct a keyword assignment system. There is also a problem that it involves a lot of work.

Further, in the keyword assigning method described in Japanese Patent Laid-Open No. 8-329096, the keywords can be positioned in a certain two-dimensional space, and the degree of association between the keywords can be defined. Therefore,
As described above, it is effective for keyword selection during fuzzy search. However, humans associate the keyword with the image data to which the keyword should be added. Further, as described above, the placement of keywords on the keyword map must be performed manually, which is very troublesome.

Therefore, an object of the present invention is to provide a keyword assigning method capable of easily and automatically assigning a keyword, and an automatic keyword assigning apparatus using this keyword assigning method.

[0009]

In order to achieve the above object, the invention according to claim 1 is a keyword assigning method for automatically assigning a keyword to an object registered in a storage unit, which comprises a plurality of objects and these objects. A two-dimensional or more semantic space in which the keywords assigned to the object are arranged, the feature amount of the registration target object is extracted, and the position of the registration target object in the semantic space is determined based on the feature amount. Obtained, from the keywords arranged in the semantic space, select a keyword located near the position of the registration target object,
The feature is that it is given to the registration target object.

According to the above configuration, a two-dimensional or more semantic space in which a plurality of objects and keywords assigned to these objects are arranged can be easily obtained by multivariate analysis such as quantification theory III. it can. Therefore, only by exemplifying a plurality of objects to which the keyword is added, the keyword suitable for the registration target object is easily and automatically selected and added.

According to a second aspect of the present invention, in the keyword assigning method according to the first aspect of the present invention, the normal correlation analysis is performed on the position and the feature amount of the plurality of objects in the semantic space, The correlation between the position in the above-mentioned semantic space and the feature amount regarding a plurality of objects is obtained,
The position of the registration target object in the semantic space is obtained based on the feature amount of the registration target object using the correlation.

According to the above configuration, by using the correlation between the position in the semantic space and the feature amount regarding a plurality of objects obtained by the canonical correlation analysis, the feature amount of the registration target object is used to The correct position of the registration target object in the above semantic space is obtained.

According to a third aspect of the invention, in the keyword assigning method of the first aspect of the invention, the semantic space is a semantic space for an object in which the plurality of objects are arranged, and a semantic space for the object. It is prepared separately from the semantic space for the keyword that has the same spatial structure as the semantic space and the keywords assigned to the plurality of objects are arranged, and the registration target object in the semantic space for the object Select n objects located near the position,
Based on the position of each of the selected n objects in the meaning space for the object, a keyword located in the vicinity of the registration target object is selected using the meaning space for the keyword. There is.

According to the above configuration, by using the semantic space for the object having the same spatial structure and the semantic space for the keyword, the keyword located near the position of the registration target object can be easily selected. It

Further, in the invention according to claim 4, in the keyword assigning method of the invention according to claim 1, the semantic space is a semantic space for an object in which the plurality of objects are arranged, and a semantic space for the object. It is prepared separately from the semantic space for the keyword that has the same spatial structure as the semantic space and the keywords assigned to the plurality of objects are arranged, and the registration target object in the semantic space for the object Select n objects located near the position,
Based on the positions of the selected n objects in the semantic space for the object, the attention position in the semantic space for the object is obtained, and the semantic space for the keyword is used based on the attention position. hand,
The feature is that a keyword located near the registration target object is selected.

According to the above construction, by using the semantic space for the object having the same spatial structure and the semantic space for the keyword, the keyword located near the position of the registration target object can be easily selected. It

The invention according to claim 5 is an automatic keyword assigning device for automatically assigning a keyword to an object to be registered in a storage unit, wherein a plurality of objects and keywords assigned to these objects are arranged. A semantic space storing means for storing information of the generated two or more-dimensional semantic space, a feature quantity extracting means for extracting a feature quantity of a registration target object, and the semantic space of the registration target object based on the feature quantity An object position calculating means for obtaining the upper position, a keyword selecting means for selecting a keyword located in the vicinity of the position of the registration target object among the keywords arranged in the semantic space, and the selected It is characterized by including a keyword assigning means for assigning a keyword to the registration target object. .

According to the above configuration, the two-dimensional or more semantic space stored in the semantic space storage unit is a multivariate such as quantification theory III based on a plurality of objects and keywords assigned to these objects. It can be easily obtained by analysis. Therefore, only by preparing a plurality of objects to which the keyword is added, based on the position of the registration target object in the semantic space, a keyword suitable for the registration target object from among the keywords arranged in the semantic space. Is easily and automatically selected.

According to a sixth aspect of the present invention, in the automatic keyword assigning apparatus of the fifth aspect of the invention, a regular correlation analysis is performed on the above-mentioned position in the semantic space and the feature amount regarding a plurality of objects. And a correlation calculating means for obtaining a correlation between the position in the semantic space and the feature amount for the plurality of objects, the object position calculating means, using the correlation, the feature amount of the registration target object. It is characterized in that the position of the registration target object in the semantic space is obtained based on.

According to the above configuration, by using the correlation between the position in the semantic space and the feature amount regarding the plurality of objects obtained by the normal correlation analysis, the feature amount of the registration target object is used to The correct position of the registration target object in the above semantic space is obtained.

[0021] According to a seventh aspect of the invention, in the keyword automatic assigning device of the fifth aspect of the invention, the semantic space information is a semantic space for an object in which the plurality of objects are arranged, The object has the same space structure as the object semantic space and is stored separately in the keyword semantic space in which the keywords assigned to the plurality of objects are arranged. Based on neighboring object extraction means for extracting n objects located in the vicinity of the position of the registration target object in the semantic space, and the position of each of the extracted n objects in the semantic space for the object. Then, using the semantic space for the above keywords, It is characterized in that it is provided with a neighborhood keyword extracting means for extracting a keyword located.

According to the above configuration, the keyword located near the position of the registration target object can be easily selected by associating the semantic space for the object having the same spatial structure with the semantic space for the keyword. To be done.

The invention according to claim 8 is the automatic keyword assigning device of the invention according to claim 5, wherein the information on the semantic space is a semantic space for an object in which the plurality of objects are arranged, The object has the same space structure as the object semantic space and is stored separately in the keyword semantic space in which the keywords assigned to the plurality of objects are arranged. Based on neighboring object extraction means for extracting n objects located near the position of the registration target object in the semantic space, and the positions of the extracted n objects in the semantic space for the object. ,
Attention position calculation means for obtaining an attention position in the semantic space for the object, and a neighborhood keyword for extracting a keyword located in the vicinity of the registration target object based on the attention position and using the meaning space for the keyword It is characterized by having extraction means.

According to the above construction, the meaning space for the object having the same spatial structure and the meaning space for the keyword are associated with each other, so that the keyword located near the position of the object to be registered is easily selected. To be done.

According to a ninth aspect of the program storage medium of the present invention, a feature space of a registration target object is extracted, and a two-dimensional or more semantic space in which a plurality of objects and keywords assigned to these objects are arranged. The position of the registration target object on the above is obtained based on the feature amount, and from among the keywords arranged in the semantic space, the keyword located near the position of the registration target object is selected, It is characterized in that a program to be added to the registration target object is stored.

According to the above configuration, the position of the registration target object in the semantic space in which the plurality of objects and the keywords assigned to these objects are arranged is obtained based on the feature amount of the registration target object. As a result, the keyword existing near the position on the semantic space of the registration target object can be easily obtained as a keyword suitable for the registration target object.

The invention according to claim 10 is the invention according to claim 9.
In the program storage medium of the invention according to the above, a regular correlation analysis is performed on the positions in the semantic space and the feature amount regarding the plurality of objects, and the position and the feature amount in the meaning space regarding the plurality of objects are It is characterized by storing a program for obtaining a correlation and using the correlation to obtain the position of the registration target object in the semantic space based on the feature amount of the registration target object.

According to the above configuration, when the position of the registration target object in the semantic space is calculated based on the feature amount of the registration target object, the correlation obtained by the normal correlation analysis is used. In this way, the correct position of the registration target object in the semantic space is obtained based on the feature amount of the registration target object.

Here, the above "object" means
It means electronic information (for example, text data, image data, voice data) that is the subject to be stored by an information processing device such as the present search device, or a folder or the like in which these are collected.

[0030]

BEST MODE FOR CARRYING OUT THE INVENTION The present invention will be described in detail below with reference to the embodiments shown in the drawings. FIG. 1 is a functional block diagram of a search device equipped with the automatic keyword assigning device of the present embodiment. In this embodiment, image data is used as the “object”.

Image data to be stored in the image data storage unit 12, search conditions, various instructions, etc. are input from the input unit 1. The keyword registration processing unit 2 registers the keyword, which is one of the search conditions input from the input unit 1, and the coordinates in the keyword space in the keyword storage unit 3 in association with each other. The image data search processing unit 4 searches the image data stored in the image data storage unit 12 for image data that meets the above search conditions, using the keyword set from the input unit 1 as a search condition. Then, the candidate image is displayed on the display unit 5 based on the searched image data.

The dictionary creation processing unit 6 determines, based on the sample data showing the addition state of the keyword to the example image,
An image dictionary and a keyword dictionary are constructed and stored in the dictionary storage unit 8 as described later in detail. The axis addition unit 7
Adds a new axis to each constructed dictionary. The vector conversion coefficient creation processing unit 9 calculates the coefficient matrix A1 used when converting the n-dimensional coordinate vector of the image in the image dictionary into the k-dimensional composite variable vector f and the p-dimensional feature vector of the image by k.
A coefficient matrix A2 used for conversion into the dimensional composite variable vector g is obtained and stored in the vector conversion coefficient storage unit 10.

The keyword automatic setting processor 14 converts the p-dimensional feature vector of the registration target image extracted by the feature extractor 15 into the k-dimensional composite variable vector g by the vector converter 16.
Convert to. Then, the neighborhood image of the registration target image is obtained from the association with the k-dimensional composite variable vector f obtained by converting the n-dimensional coordinate vector in the image dictionary of all the images registered in the image data storage unit 12. . Then, based on the coordinate vector of this neighborhood image in the image dictionary, a keyword suitable for the registration target image is set as described later. The additional information input processing unit 17 manually inputs a keyword and other additional information.

The image data registration processing unit 11 includes the keyword automatic setting processing unit 14 and the additional information input processing unit 17.
Is controlled to automatically or manually add a keyword to the image data input from the input unit 1 and register it in the image data storage unit 12. Further, the additional information change processing unit 13 controls the keyword automatic setting processing unit 14 and the additional information input processing unit 17 to automatically or manually change the keyword added to the image data of the image to be changed. The image data storage unit 12 is updated.

2 and 3 are diagrams showing a hardware configuration for realizing the functions of FIG. As shown in FIG. 2, the present search device has a CRT (cathode ray tube) 22, a keyboard 23, and a mouse 24 centered on a control device 21 having a CPU (central processing unit) mounted therein and controlling the operation of the entire search device. , Floppy disk drive 25b, hard disk device 26, printer 27, scanner 28, CD-R
It is composed of an OM drive 29b, a speaker 30, a microphone 31, and the like.

On the CRT 22, the retrieved images and
Various information necessary for operation such as character information is displayed. The keyboard 23 and the mouse 24 are used for various input operations and instruction operations. The floppy disk 25a is attached to the floppy disk drive 25b of the control device 21 to store and reproduce data. Hard disk device 2
Reference numeral 6 stores image data and keywords. Printer 2
Reference numeral 7 outputs an image based on the image data, a drawing created by image organization, or the like on a sheet. The scanner 28 reads an image on a sheet-shaped document and outputs image data. The CD-ROM 29a stores image data in a predetermined format, and a CD-ROM drive 29
Attached to b, the image data is played back. Speaker 30
Outputs a voice, and the microphone 31 inputs a voice. Here, the image data read by the scanner 28 and the CD-ROM drive 29b is stored in the hard disk device 26. In addition, the scanner 28, CD-R
The OM drive 29b, the speaker 30, and the microphone 31 may be built in the control device 21 and integrated.

FIG. 3 represents FIG. 2 in a block diagram centering on the CPU mounted in the control device 11. The CPU 35 is, for example, an Intel product number i8.
0486DX or the like is used. The CPU 35 has a ROM (Read Only Memory) 38 in which a program for controlling the search device is stored and a RAM (Random Access Memory) in which various data and programs are stored in the CPU 35. 39, CR image or text
A display control circuit 40 for displaying at T22, a keyboard control circuit 41 for controlling transfer of input from the keyboard 23, a mouse control circuit 42 for controlling transfer of input from the mouse 24,
Floppy disk drive control circuit 43 for controlling floppy disk drive 25b, hard disk control circuit 44 for controlling hard disk device 26, printer control circuit 45 for controlling output operation of printer 27, scanner control circuit for controlling image input operation of scanner 28 Four
6. CD-RO for controlling the CD-ROM drive 29b
The M drive control circuit 47, the speaker control unit 48 that controls the audio output of the speaker 30, and the microphone 3
Microphone control circuit 4 for controlling voice input from 1
9 is connected.

The CPU 35 is connected with a clock 37 for generating a reference clock necessary for operating the present search device, and further, an expansion for connecting various expansion boards via the data bus 36. The slot 50 is connected. Here, the expansion slot 50 has a SCSI
The board is connected, and the floppy disk drive 25b and the hard disk device 2 are connected via this SCSI board.
6, the scanner 28, the CD-ROM drive 29b, etc. may be connected.

That is, in the present embodiment, the input unit 1 is used as the keyboard 23, the mouse 24, the hard disk device 26, the scanner 28 and the CD-ROM drive 2.
9b, the image data storage unit 6 and the dictionary storage unit 8 are configured by the floppy disk 25a and the hard disk device 26, and the keyword storage unit 3 and the vector conversion coefficient storage unit 10 are configured by the RAM 39 and the like, and the display unit 5 is used. Is composed of a CRT 22, and a keyword registration processing unit 2, an image data search processing unit 4, a dictionary creation processing unit 6, a vector conversion coefficient creation processing unit 9, an image data registration processing unit 11, an additional information change processing unit 13, and a keyword automatic setting. The processing unit 14 and the additional information input processing unit 17 are constituted by the CPU 35.

In the present embodiment, as described above,
The image data storage section 12 is stored in the floppy disk 25a.
Although the hard disk device 26 is used, another storage medium such as a magneto-optical disk device may be used. Although the image data input section of the input section 1 is composed of the hard disk device 26, the scanner 28 and the CD-ROM drive 29b, other input devices such as a still video camera and a digital camera may be used. Further, instead of the printer 27, another output device such as a digital copying machine may be used.

Further, in the present search device, there are performed a keyword registration process, a dictionary creation process, a vector conversion coefficient creation process, a keyword automatic setting process, an additional information input process, an image data registration process, an additional information change process, an image data search process, etc. The program is stored in the ROM 38. However, part or all of the program may be stored in an external storage medium such as the floppy disk 25a or the hard disk device 26, and the program may be read into the RAM 39 as needed.

The search device having the above configuration operates as follows. FIG. 4 is a flowchart of a basic processing operation executed under the control of the CPU 35 based on the program stored in the ROM 38. The basic processing operation of this search device will be described below with reference to FIGS. 1 and 4.

When the power of the present search device is turned on and the program is started, the basic processing operation is started.

In step S1, initialization of flags and registers necessary for the following processing and initial setting processing such as initial screen display on the display unit 5 are performed.

If it is determined in step S2 that the job menu "search" is selected from the initial screen displayed on the display section 5, the process proceeds to step S3, and if not, the process proceeds to step S4.

In step S3, a search process for searching for desired image data from the image data storage unit 12 and displaying it on the display unit 5 is executed in accordance with the selected job menu "search". After that, the process proceeds to step S17. still,
The search processing is performed by the image data search processing unit 4.

If it is determined in step S4 that the job menu "create dictionary" is selected from the initial screen displayed on the display section 5, the process proceeds to step S5, and if not, the process proceeds to step S6.

In step S5, the image dictionary (image space) and the keyword dictionary (keyword space) are set based on the example image to which the keyword is given in advance in accordance with the selected job menu "create dictionary". A dictionary creation process is performed in which the coordinates on the image dictionary and the coordinates on the keyword dictionary of the assigned keyword are obtained and stored in the dictionary storage unit 8. After that, the process proceeds to step S17. still,
The dictionary creation processing is performed by the dictionary creation processing unit 6.

If it is determined in step S6 that the job menu "keyword registration" is selected from the initial screen displayed on the display section 5, the process proceeds to step S7, and if not, the process proceeds to step S8.

In step S7, according to the selected job menu "keyword registration", a keyword registration process of registering the keyword input from the input unit 1 in the keyword storage unit 3 together with the coordinate value in the keyword space is executed. . After that, the process proceeds to step S17. still,
The keyword registration processing is performed by the keyword registration processing unit 2.

If it is determined in step S8 that the job menu "Create vector conversion coefficient" is selected from the initial screen displayed on the display section 5, the process proceeds to step S9.
If not, the process proceeds to step S10.

In step S9, the n-dimensional coordinate vector of each image stored in the created dictionary and the p extracted from the image data of each image according to the selected job menu "Create vector conversion coefficient" are selected. A regular correlation analysis with the dimensional feature amount vector is performed, and a vector conversion coefficient creating process for obtaining regular correlation coefficient matrices A1 and A2 described later is executed. After that, the process proceeds to step S17. Here, the positive correlation coefficient matrix A1 is a vector conversion coefficient for converting an n-dimensional coordinate vector into a k-dimensional composite variable vector f, and the normal correlation coefficient matrix A2 is a p-dimensional feature vector k. It is a vector conversion coefficient for converting into a dimensional composite variable vector g. The vector conversion coefficient creation processing is performed by the vector conversion coefficient creation processing unit 9.

If it is determined in step S10 that the job menu "data registration" has been selected from the initial screen displayed on the display section 5, the process proceeds to step S11, and if not, the process proceeds to step S12.

In step S11, the image data input from the input unit 1 is fetched according to the selected job menu "data registration", and a keyword is added to this image data to be registered in the image data storage unit 12. Data registration processing is executed. After that, the process proceeds to step S17. The image data registration processing is performed by the image data registration processing unit 11.

If it is determined in step S12 that the job menu "change additional information" has been selected from the initial screen displayed on the display section 5, the process proceeds to step S13, and if not, the process proceeds to step S14.

In step S13, the image data storage unit 1 is operated in accordance with the selected job menu "additional information change".
The image data registered in No. 2 is taken in, and the additional information changing process for changing the additional information such as the keyword added to this image data is executed. After that, the process proceeds to step S17. The additional information change processing is performed by the additional information change processing unit 13.

If it is determined in step S14 that the job menu "other menu" is selected from the initial screen displayed on the display section 5, the process proceeds to step S15, and if not, the process proceeds to step S16.

In step S15, other menu processing such as printout of the search image is executed according to the selected job menu "other menu". After that, the process proceeds to step S17.

In step S16, it is determined whether or not "end" has been selected from the initial screen displayed on the display section 5. As a result, if "end" is selected, the basic processing operation is ended, while if not, the process proceeds to step S17.

At step S17, other processing is executed. After that, the process returns to the step S2 to shift to the next menu process. Then, when "end" is selected in step S16, the basic processing operation ends.

The "dictionary creating process", "keyword registering process", "vector conversion coefficient creating process", "image data registering process" and "additional information changing process" will be described in detail below. "Search processing" and "Other menu processing"
Since these are not directly related to the present invention, detailed description thereof will be omitted.

FIG. 5 is a flowchart of a dictionary creation processing subroutine executed by the dictionary creation processing unit 6 in step S5 in the basic processing operation shown in FIG. Hereinafter, the dictionary creation processing operation will be described in detail with reference to FIG. When it is determined in step S4 of the main routine of the basic processing operation shown in FIG. 4 that the job menu "create dictionary" is selected, the dictionary creation processing subroutine starts.

Here, the dictionary is a correlation dictionary of a plurality of images and keywords assigned to each image, and an image dictionary in which coordinate vectors of each image in an n-dimensional semantic space are registered, and the above-mentioned dictionary. It is composed of a keyword dictionary in which coordinate vectors of each keyword in the semantic space are registered. The dictionary is constructed based on sample data obtained by giving a plurality of exemplary images with a plurality of keywords for each exemplary image by a questionnaire.

FIG. 6 shows an example of the sample data. In FIG. 6A, the keyword is added to the example image (1)
It indicates whether or not (0). On the other hand, Fig. 6 (b)
Indicates the number of persons who have given the keyword to the example image. In addition to this, it does not matter whether the keyword is added (1) or not (0) to the exemplary image a threshold number of times or more, or the keyword may be weighted for aggregation. It should be noted that the sample data also includes the feature amount of each example image and is used in the subsequent vector conversion coefficient creation processing.

In step S21, it is determined whether or not to construct the entire dictionary by the dialogue with the user. As a result, if the entire dictionary is constructed, the process proceeds to step S22, and if not, the process proceeds to step S23.

In step S22, the entire dictionary is constructed as will be described later.

In step S23, it is determined whether or not to add the axis to the dictionary by the dialogue with the user. As a result, if an axis is to be added, the process proceeds to step S24, and if not, the dictionary creation process subroutine is terminated and the process returns to the main routine of the basic process operation shown in FIG.

In step S24, axis addition processing is executed as will be described later in detail. After that, the dictionary creation processing subroutine is terminated, and the processing returns to the main routine of the basic processing operation shown in FIG.

FIG. 7 is a flowchart of the dictionary construction process subroutine executed in step S22 of the dictionary creation process subroutine shown in FIG. Less than,
The construction processing operation of the entire dictionary will be described in detail with reference to FIGS. 1 and 7. When "build entire dictionary" is designated in step S21 in the dictionary creating processing subroutine shown in FIG. 5, the entire dictionary building processing subroutine starts.

In step S31, the sample data is read from the input section 1 and stored in the internal memory or the like. Then, all the exemplary images and the keywords assigned to the respective exemplary images are captured as a pair.

At step S32, each keyword is set as a category, and image data of each image is set as a sample.
Analysis by quantification theory III is performed.

Here, the quantification theory III is one of multivariate analysis, and both the sample and the category in the n-dimensional semantic space are determined based on the correspondence of the sample to various categories. Quantify as coordinates. That is,
Given the quantity x _i ⁽ⁿ⁾ to sample i and y to category j
_When giving a quantity of _j ⁽ⁿ⁾ , x _i ⁽ⁿ⁾ and y _j ⁽ⁿ⁾ that maximize the correlation coefficient between x _i ⁽ⁿ⁾ and y _j ⁽ⁿ⁾ are obtained.

In step S33, the category score y _j up to the n-th component is calculated from the analysis result by the above quantification theory III.
⁽ⁿ⁾ and the sample score x _i ⁽ⁿ⁾ are obtained. here,
n is a number smaller than the smaller one of the number of categories and the number of samples.

In step S34, each category score is normalized so that the maximum value of the category score is "1" and the minimum value thereof is "-1" for each component.

In step S35, each sample score is normalized so that the maximum value of the sample score is "1" and the minimum value thereof is "-1" for each component.

In step S36, the coordinate vector in the n-dimensional meaning space (keyword space) of each keyword obtained from the normalized value of each category score obtained in step S34 is stored in the dictionary storage unit 8 as a keyword dictionary. Is stored.

In step S37, the coordinate vector in the n-dimensional meaning space (image space) of each image obtained from the normalized value of each sample score obtained in step S35 is used as the image dictionary in the dictionary storage unit 8 described above. Stored in. After that, the construction process subroutine for the entire dictionary is ended, and the process returns to the main routine of the basic processing operation shown in FIG.

The keyword n-dimensional meaning space (keyword space) constructed as described above and the image n-dimensional meaning space (image space) have a high correlation. Therefore, for any image in the image space, a keyword suitable for the image is arranged near the coordinates in the keyword space corresponding to the coordinates in the image space of the image. On the contrary, for an arbitrary keyword in the keyword space, an image suitable for the keyword is arranged near the coordinates in the image space corresponding to the coordinates of the keyword in the keyword space.

That is, in the present embodiment, the semantic space storage means is constructed by the dictionary storage unit 8.

In this way, the coordinate vector of the keyword in the n-dimensional meaning space (keyword space) is registered in the dictionary storage unit 8 as a keyword dictionary, and the coordinate vector of the image in the n-dimensional meaning space (image space) is registered. By registering as an image dictionary, the degree of association between keywords and images can be set. Therefore,
A similar keyword of a certain keyword or a similar image of a certain image can be easily obtained by looking up the keyword dictionary or the image dictionary. Furthermore, it is possible to search for a keyword suitable for giving to a new image.

FIG. 8 is a flowchart of the axis addition processing subroutine executed by the axis addition section 7 of the dictionary creation processing section 6 in step S24 of the dictionary creation processing subroutine shown in FIG. Hereinafter, the axis addition processing operation will be described in detail with reference to FIGS. 1 and 8. Figure 5
When "addition of axis" is designated in the step S23 in the dictionary creation processing subroutine shown in (4), the axis addition processing subroutine is started.

In step S41, the keyword storage unit 3
A list of registered keywords registered in is displayed on the display unit 5.

In step S42, two keywords to be placed opposite to each other are selected from the registered keywords by the dialogue with the user.

In step S43, a two-axis map is created which is a combination of the two axes in which the distances between the selected keywords are the longest in the keyword space. Then, all registered keywords are projected on a straight line connecting both selected keywords on the two-axis map, and the coordinates on the straight line of all registered keywords are obtained.

In step S44, the coordinate value of each registered keyword is set so that the maximum value is "1" and the minimum value is "-1" among the coordinates on the straight line of all registered keywords obtained in step S43. Is normalized.

In step S45, all registered keywords are displayed on the display unit 5 in the order of the normalized coordinate values.

In step S46, the axis name having the above straight line as a new coordinate axis is set by the interaction with the user.
The "+" label and "-" label of the new axis are set.

In step S47, the set new axis information and the coordinates of all registered keywords on the new axis are additionally registered in the dictionary storage unit 8. After that, the axis addition processing subroutine is ended, and the process returns to the main routine of the basic processing operation shown in FIG.

Here, the user cannot know the specific meaning of the n axes obtained as a result of the analysis based on the quantification theory III. However, by freely setting the new axis as described above by the user, the registered keyword can be arranged on the axis from which the user can understand the meaning.

FIG. 9 is a flowchart of the keyword registration processing subroutine executed by the keyword registration processing section 2 in step S7 in the basic processing operation shown in FIG. Hereinafter, the keyword registration processing operation will be described in detail with reference to FIG. When it is determined that the job menu "keyword registration" is selected in step S6 of the main routine of the basic processing operation shown in FIG. 4, the keyword registration processing subroutine is started.

In step S51, the input keyword input by the user from the input section 1 is fetched.

In step S52, the image data storage unit 1
The registered images are displayed as a list on the display unit 5 based on the image data registered in 2.

In step S53, an image suitable for the input keyword is selected from the registered images displayed in the list by the dialogue with the user. At least one selected image may be selected, and a plurality of selected images may be selected.

In step S54, the coordinate vector in the image dictionary of the image selected in step S53 is read from the dictionary storage unit 8, and the image space of the selected image is read.
The average coordinates on the (image dictionary) are calculated. The calculation of the average coordinates is performed by obtaining a vector whose element value is a value obtained by simply averaging each element value of the read coordinate vector. Since the average coordinates in the image space are associated with the same coordinates in the keyword space, which is the same meaning space, they can be regarded as the coordinates of the input keyword in the keyword space.

In step S55, the coordinate vector of the average coordinates obtained in step S54 is added to the input keyword and additionally registered in the keyword storage unit 3. Further, position information of the input keyword in the keyword space is additionally registered in the dictionary storage unit 8.

At step S56, it is determined by dialogue with the user whether or not the keyword registration is completed. As a result, if it has not been completed, the procedure returns to step S51 to move to the input of the next keyword, and if it has been completed, the procedure returns to the main routine shown in FIG.

Thus, the user can determine the position of the keyword to be registered in the keyword space based on the position of the selected image in the image space by a simple operation of simply selecting an image suitable for the keyword to be newly registered. It is automatically obtained and registered at the position in the keyword space. When the keyword dictionary is expanded by adding new keywords to the keyword dictionary as described above, the keywords are already registered due to the influence of the coordinates of the additionally registered keywords. It becomes necessary to change the coordinates of the keywords. In that case, by executing the "construction process of the entire dictionary" again, it is possible to construct the keyword dictionary in which the latest judgment standard of the keyword setting of the user is reflected.

By the way, since the image dictionary (image space) and the keyword dictionary (keyword space) constructed as described above are the same n-dimensional semantic space, they have a high correlation. Therefore, if the position of the image to be newly registered in the image dictionary is known, a keyword near the position in the keyword dictionary corresponding to the position in the image dictionary is automatically set as a keyword suitable for the newly registered image. Can be given. Therefore, in order to realize such automatic keyword assignment, it is necessary to know the position in the image dictionary of the image to be newly registered.

On the other hand, there are characteristic quantities such as color, shape, texture, etc. as values representing the characteristics of the image. Therefore, if the relationship between the image feature amount and the position in the image dictionary is known, the position of the newly registered image in the image dictionary can be known by obtaining the feature amount of the newly registered image. it can. Below, the relationship between the position of the image in the image dictionary and the feature amount,
That is, the analysis of the correspondence relationship between the two quantitatively measured variable groups will be described.

Two sets of variables X (X ₁ , X ₂ , ..., X _n ), Y (Y ₁ ,
A canonical correlation analysis is a method for analyzing the correlation structure between Y ₂ , ..., Y _p ). According to this canonical correlation analysis, the n-dimensional coordinate vector of the r images on the image dictionary is X, and its composite variable matrix is F (= A1 · X ^T ). On the other hand, the p-dimensional feature vector is Y, and its composite variable matrix is G (= A2.
Y ^T ). Then, the canonical correlation coefficient matrices A1 and A2 are determined so as to increase the correlation between the composite variable matrices F and G. As a result, the space of the (r × k) composite variable matrix F and the space of the (r × k) composite variable matrix G that are associated with each other with high correlation are constructed, and the composite variable based on the image feature vector. Given a vector in the space of the matrix G, the coordinate vector in the image dictionary of the same image can be obtained via the corresponding vector in the space of the composite variable matrix F. That is, the canonical correlation coefficient matrix A1 is a vector conversion coefficient for converting an n-dimensional coordinate vector into a k-dimensional synthetic variable vector f, and the canonical correlation coefficient matrix A2 is a p-dimensional feature vector k-dimensional. Is a vector conversion coefficient for converting into the combined variable vector g of.

FIG. 10 is a flow chart of a vector conversion coefficient creation subroutine executed by the vector conversion coefficient creation processing section 9 in step S9 in the basic processing operation shown in FIG. Hereinafter, FIG. 1 and FIG.
0, the vector conversion coefficient creation processing operation will be described in detail. When it is determined in step S8 of the main routine of the basic processing operation shown in FIG. 4 that the job menu "vector conversion coefficient creation" is selected, the vector conversion coefficient creation processing subroutine is started.

In step S61, the n-dimensional coordinate vector of each image in the image dictionary is read from the dictionary storage unit 8. On the other hand, the p-dimensional feature vector of the corresponding image is read from the internal memory. Then, a canonical correlation analysis is performed between the read coordinate vector of each image and the corresponding feature vector.

In step S62, the canonical correlation coefficient matrix A1 (n × k) for coordinate vector conversion obtained as a result of the canonical correlation analysis in step S61 and the canonical correlation coefficient matrix for feature vector conversion are obtained. A 2 (p × k) is stored in the vector conversion coefficient storage unit 10.

In step S63, the coordinate vectors of the r images stored in the image dictionary are multiplied by the canonical correlation coefficient matrix A1 and converted into a k-dimensional composite variable vector f. Then, the obtained composite variable vector f is stored in the vector conversion coefficient storage unit 10.

In step S64, the feature vectors of the r images are multiplied by the canonical correlation coefficient matrix A2, and g-transformed into a k-dimensional composite variable vector.

In step S65, the maximum value and the minimum value for each dimension of the k-dimensional composite variable vector g calculated in step S64 are calculated and stored in the vector conversion coefficient storage unit 10. After that, the vector conversion coefficient creation processing operation is terminated and the process returns to the main routine shown in FIG.

That is, in this embodiment, the correlation calculating means is constituted by the steps S61 and S62 in the vector conversion coefficient creating processing subroutine.

Thus, the p-dimensional feature vector of the image is k
Since the canonical correlation coefficient matrix A2 to be converted into the dimensional synthetic variable vector g is obtained, the position of the image in the image dictionary can be estimated. Therefore, as mentioned above,
It becomes possible to automatically assign a keyword suitable for an image to be newly registered. Hereinafter, the image data registration processing including the keyword automatic addition operation will be described.

FIG. 11 shows the above-mentioned image data registration processing unit 1 in step S11 in the basic processing operation shown in FIG.
6 is a flowchart of an image data registration processing subroutine executed by the first embodiment. The image data registration processing operation will be described in detail below with reference to FIGS. 1 and 11.
When it is determined in step S10 in the main routine of the basic processing operation shown in FIG. 4 that the job menu "data registration" is selected, the image data registration processing subroutine is started.

In step S71, it is determined by dialogue with the user whether or not new image data is to be registered. As a result, in the case of new image data, step S72
Otherwise, to step S73.

In step S72, an image data fetching process is executed as will be described later. After that, the process proceeds to step S74.

In step S73, existing image data is selected and read.

In step S74, it is determined by dialogue with the user whether or not a keyword is automatically added.
As a result, if the automatic keyword assignment is performed, the process proceeds to step S75, and if not, the process proceeds to step S77.

In step S75, the keyword automatic setting processing section 14 is controlled to execute the keyword automatic setting processing as will be described later in detail.

At step S76, the keyword set at step S75 is set as one of the additional information of the image data fetched at step S72 or step S73.

In step S77, it is determined whether or not the additional information is to be manually input by the interaction with the user. As a result, when manually inputting, the process proceeds to step S78,
Otherwise, it proceeds to step S79.

In step S78, the additional information input processing section 17 is controlled and the additional information input processing is executed, as will be described later in detail.

In step S79, the captured image data is registered in the image data storage section 12 in association with the set additional information. After that, the image data registration processing operation is terminated and the process returns to the main routine shown in FIG.

In this way, the user only selects the job menu "data registration" from the input section 1 and specifies "keyword automatic setting", and the keyword suitable for the image to be registered is automatically set, and the corresponding keyword is set. It is added to the image data and registered.

FIG. 12 is a flow chart of the image data import processing subroutine executed in step S72 in the image data registration processing subroutine shown in FIG. Hereinafter, the image data import processing operation will be described in detail with reference to FIGS. 1 and 12. Figure 1
In step S71 of the image data registration processing subroutine shown in FIG. 1, if it is determined to register new image data, the image data import processing subroutine starts.

In step S81, new image data is fetched from the input unit 1 (image input device such as scanner 28 or CD-ROM drive 29b or image data storage medium such as hard disk device 26).

In step S82, a storage destination such as a work memory for storing the above-mentioned captured image data is designated.

In step S83, the captured image data is stored in the working memory with the image data name set by the user from the input section 1 being given. After that, the image data import processing subroutine is ended, and the process returns to the image data registration processing subroutine shown in FIG.

FIG. 13 is a flowchart of the keyword automatic setting processing subroutine executed by the keyword automatic setting unit 14 in step S75 in the image data registration processing subroutine shown in FIG. Hereinafter, the keyword automatic setting processing operation will be described in detail with reference to FIGS. 1 and 13. In step S74 of the image data registration processing subroutine shown in FIG.
When it is determined that the keyword is automatically added, the keyword automatic setting processing subroutine starts.

In step S91, the feature quantity extraction unit 15 selects from the image data captured in step S72 or step S73 of the image data registration processing subroutine shown in FIG. Is extracted, and a p-dimensional feature vector is obtained.

In step S92, vector conversion processing is executed as described later in detail. By this vector conversion process, the p-dimensional feature vector extracted in step S91 is converted into the k-dimensional composite variable vector g.

In step S93, a neighborhood image extraction process is executed as will be described later in detail. By this neighborhood image extraction processing, the neighborhood image of the registration target image is extracted based on the k-dimensional composite transformation vector g transformed in step S92.

In step S94, the image neighborhood keyword extraction processing is executed as will be described later. By this image neighborhood keyword extraction processing, image neighborhood keywords close to the neighborhood image extracted in step S93 are extracted using the image dictionary.

In step S95, duplicate keywords are removed from the image neighborhood keywords extracted in step S94, and the keywords are set as setting keywords. After that, the keyword automatic setting process subroutine is ended, and the process returns to the image data registration process subroutine shown in FIG.

That is, in the present embodiment, the object position calculating means is constituted by the step S92 in the keyword automatic setting processing subroutine, and the keyword selecting means is constituted by the steps S93 to S95 to extract the neighboring objects. Means above step S93
And the above-mentioned neighborhood keyword extracting means is constituted by the steps S94 and S95, and the keyword assigning means is constituted by the step S in the image data registration processing subroutine.
It is composed of 76 and S79.

FIG. 14 is a flowchart of the vector conversion processing subroutine executed by the vector conversion unit 16 in step S92 in the keyword automatic setting processing subroutine shown in FIG. The vector conversion processing operation will be described in detail below with reference to FIG. In step S91 of the keyword automatic setting processing subroutine shown in FIG. 13, when the p-dimensional feature vector of the registration target image data is extracted, the vector conversion processing subroutine starts.

In step S 101, the extracted p-dimensional feature vector is added to the feature vector conversion canonical correlation coefficient matrix A stored in the vector conversion coefficient storage unit 10.
It is multiplied by 2 (p × k) and k-dimensional composite variable vector g
Is converted to.

In step S102, the respective element values (values of the respective dimensions) of the k-dimensional composite variable vector g obtained from the feature vector in the above step S101 are stored in the vector conversion coefficient storage unit 10. The scale is changed using the maximum value and the minimum value for each dimension of the k-dimensional composite transformation vector g obtained from the feature vector of the registered image. In this way, the composite variable vector g of the registration target image is arranged at the corresponding position in the space of the composite variable matrix G. After that, the vector conversion processing operation is terminated, and the processing returns to the keyword automatic setting processing subroutine shown in FIG.

FIG. 15 is a flowchart of the neighborhood image extraction processing subroutine executed in step S93 in the keyword automatic setting processing subroutine shown in FIG. Hereinafter, the neighboring image extraction processing operation will be described in detail with reference to FIG. In step S92 of the keyword automatic setting processing subroutine shown in FIG. 13, when the p-dimensional feature vector of the registration target image data is converted into the k-dimensional composite conversion vector g, the neighborhood image extraction processing subroutine is started.

In step S111, one composite conversion vector f is converted from the r k-dimensional composite variable vectors f converted from the coordinate vectors of the r registered images and stored in the vector conversion coefficient storage unit 10. Is read.

In step S112, the k-dimensional composite conversion vector g converted from the feature vector of the registration target image.
And the distance to the k-dimensional composite variable vector f read in step S111 is calculated. The calculation of this distance is performed by calculating the sum of squares of the differences between the element values.

In step S113, it is determined whether or not the distance has been calculated for the combined transformation vector f of all the registered images stored in the vector transformation coefficient storage unit 10. As a result, if not calculated, the process returns to step S111 to move to the calculation of the distance to the next combined conversion vector f, and if calculated, the process proceeds to step S114.

In step S114, q composite transformation vectors f are obtained in the order of decreasing distances calculated above, and q registered images relating to the obtained composite transformation vectors f are obtained as neighboring images. After that, the neighborhood image extraction processing operation is terminated, and the processing returns to the keyword automatic setting processing subroutine shown in FIG.

FIG. 16 is a flowchart of the image neighborhood keyword extraction processing subroutine executed in step S94 in the keyword automatic setting processing subroutine shown in FIG. Hereinafter, the image vicinity keyword extraction processing operation will be described in detail with reference to FIG. In step S93 of the keyword automatic setting processing subroutine shown in FIG. 13, when the neighborhood image is extracted, the image neighborhood keyword extraction processing subroutine is started.

In step S121, the coordinate vector on the image space of one of the extracted q neighboring images is read from the dictionary storage unit 8.

In step S122, the distance between the read n-dimensional coordinate vector in the image space and the n-dimensional coordinate vector in the keyword space of all the registered keywords stored in the dictionary storage unit 8 ( The sum of squares of each element value is calculated and stored in the internal memory or the like.

In step S123, h keywords are obtained in the order of decreasing distances calculated above.

In step S124, it is determined whether or not h keywords have been obtained for each of the q extracted image vectors of the neighboring images. As a result, if it has not been obtained, the process returns to the step S121 to shift to the process for the next registered keyword. On the other hand, if so, the process proceeds to step S125.

In step S125, q × h obtained as described above
Each keyword is stored in the internal memory or the like as an image neighborhood keyword. After that, the image neighborhood keyword extraction process is terminated, and the process returns to the keyword automatic setting process subroutine.

Then, as described above, when the keyword for setting is confirmed by removing the duplication from the extracted image neighborhood keyword, the keyword automatic setting process is terminated,
The process returns to the image data registration processing subroutine. Then, the keyword automatically set as described above is set as one of the additional information, and when it is determined that the additional information is manually input, the additional information input process is executed.

FIG. 17 is a flowchart of the additional information input processing subroutine executed by the additional information input processing unit 17 in step S78 in the image data registration processing subroutine shown in FIG. Below, Figure 1
The additional information input processing operation will be described in detail with reference to FIG. When it is determined in step S77 of the image data registration processing subroutine shown in FIG. 11 that "manual input of additional information" is designated, the additional information input processing subroutine starts.

In step S131, the additional information input menu is displayed on the display section 5.

If it is determined in step S132 that the additional information input menu "add keyword" is selected, the process proceeds to step S133, and if not, the process proceeds to step S134.

In step S133, the keyword addition process described later in detail is executed according to the selected additional information input menu "keyword addition". After doing so,
It proceeds to step S136.

If it is determined in step S134 that the "additional information" is selected in the additional information input menu, the process proceeds to step S135. If not, the process proceeds to step S136.

In step S135, the additional information input from the input unit 1 is fetched according to the selected additional information input menu "Additional information" and stored in the register as one of the additional information of the image data. It

In step S136, it is determined whether or not the additional information input processing is to be ended by the dialogue with the user.
As a result, if it is ended, the additional information input processing subroutine is ended and the process returns to the image data registration processing subroutine shown in FIG. On the other hand, if the process is not finished, the process returns to step S132 to move to the next menu process.

Thus, the keywords other than the keywords set in step S75 in the image data registration processing subroutine and the additional information other than the keywords such as the creator name of the data, the creation date, the registration date, the size, the comment, etc. Entered manually by.

FIG. 18 is a flowchart of the keyword addition processing subroutine executed in step S133 of the additional information input processing subroutine shown in FIG. The keyword addition processing operation will be described in detail below with reference to FIGS. 1 and 18. When it is determined in step S132 of the additional information input processing subroutine shown in FIG. 17 that the additional information input menu "add keyword" is selected, the keyword addition processing subroutine starts.

In step S141, the keywords registered in the keyword storage unit 3 are listed on the display unit 5.

At step S142, a specific keyword is selected from the displayed keywords by an interaction with the user.

In step S143, the selected keyword is set as one of the additional information and stored in the register.

In step S144, it is determined by dialogue with the user whether or not the keyword addition processing is to be ended. As a result, if it is not completed, the process returns to step S142 to select the next keyword. On the other hand, if it is finished, the keyword addition processing subroutine is finished, and FIG.
The process returns to the additional information input processing subroutine shown in.

Then, as described above, when it is determined that the additional information input process is to be ended by the dialog with the user, the process returns to the image data registration process subroutine. And
The additional information set in steps S75, S76 and S78 is stored in the image data storage unit 12 together with the image data to be registered.

FIG. 19 shows the additional information change processing unit 13 in step S13 of the basic processing operation shown in FIG.
8 is a flowchart of an additional information change processing subroutine executed by. The additional information change processing operation will be described in detail below with reference to FIGS. 1 and 19. When it is determined in step S12 of the main routine of the basic processing operation shown in FIG. 4 that the "additional information change" job menu is selected, the additional information change processing subroutine is started.

In step S151, a list of registered images is displayed on the display unit 5 based on the image data registered in the image data storage unit 12.

In step S152, the image data to be changed is selected from the displayed images by the interaction with the user.

In steps S153 to S157, FIG.
Similar to steps S74 to S78 in the image data registration processing subroutine shown in FIG. 7, determination of whether or not to automatically add a keyword, execution of keyword automatic setting processing, setting as additional information of setting keyword, manual input of additional information And the additional information input process is executed. Incidentally, the execution of the keyword automatic setting process is performed by referring to FIG.
This is performed according to the keyword automatic setting processing subroutine shown in. In that case, the “registration target image” becomes the “change target image”. The additional information input process is executed according to the additional information input process subroutine shown in FIG.

In step S158, the additional information of the selected image data stored in the image data storage section 12 is rewritten with the additional information set as described above. After doing that,
The additional information change processing operation is terminated and the process returns to the main routine shown in FIG.

In this way, the user can change the keyword assigned to the registered image to a more suitable keyword simply by selecting "additional information change" of the job menu from the input section 1 and designating "keyword automatic setting". You can do it.

FIG. 20 shows the concept of the keyword automatic setting processing shown in FIG. The dictionary storage unit 8 stores coordinate vectors of r registered images in the n-dimensional image space. Further, a p-dimensional feature vector is obtained from the feature amounts of the r images. Then, the n-dimensional coordinate vector is converted into a k-dimensional composite variable vector f by the canonical correlation coefficient matrix A1, while the p-dimensional feature vector is converted into a k-dimensional composite variable vector g by the canonical correlation coefficient matrix A2. To be done. By doing so, the n-dimensional coordinate vector and the p-dimensional feature vector can be associated with each other in the space of the same (r × k) composite variable matrices F and G.

On the other hand, when the p-dimensional feature vector of the registration target image is converted into the k-dimensional composite variable vector g by the canonical correlation coefficient matrix A2, the position of the registration target image on the composite variable matrix G space is known. Then, as described above, since the n-dimensional coordinate vector and the p-dimensional feature vector can be associated in the space of the synthetic variable matrices F and G of the same size, the synthetic variable vector g of the registration target image is synthesized. By comparing the distance between the position in the variable matrix G space and the position in the combined variable matrix F space of the combined variable vector f of each registered image, q registered images similar to the registration target image (neighboring image ) Can be obtained.

Thereafter, the positions in the keyword space (keyword dictionary) corresponding to the positions in the image space (image dictionary) of the q neighboring images thus obtained and the positions in the keyword space of the registered keywords are set. By comparing the distances, h neighboring keywords of each neighboring image are obtained. In this way, q × h keywords suitable for giving to the image to be registered are automatically set.

As described above, in the present embodiment, the dictionary creation processing unit 6 uses the image data of the exemplary image as a sample based on the plurality of exemplary images and the keyword given to each exemplary image, and The analysis is carried out according to the quantification theory type III with the category of. From this analysis result, the sample score x _i ⁽ⁿ⁾ up to the n-th component is normalized to the maximum value of “1” and the minimum value to “−1”, while the category score y _j ⁽ⁿ⁾ is set to the maximum value. The minimum value is normalized to "1" to "-1". Then, the coordinate vector in the image space having the obtained normalized value of the sample score x _i ⁽ⁿ⁾ as an element value is stored in the dictionary storage unit 8 as an image dictionary. On the other hand, the coordinate vector in the keyword space having the normal value of the category score y _j ⁽ⁿ⁾ as the element value is stored in the dictionary storage unit 8 as the keyword dictionary.

The keywords positioned on the keyword dictionary thus constructed are arranged on n meaning scales in the n-dimensional meaning space. Therefore, according to the present embodiment, the keyword space can be defined, and the position of a certain keyword in the keyword space (the degree of association with another keyword) can be automatically set.
Therefore, it is possible to perform a fuzzy search using the degree of association between the keywords.

As described above, according to the above-described embodiment, it is only necessary to add a keyword to a plurality of exemplary images and present them.
It is possible to automatically define a keyword space and automatically set the degree of association between one keyword and another keyword based on this keyword space. Therefore, JP-A-8
The user is freed from the troublesome work of manually arranging keywords on the keyword map (that is, setting the degree of association of other keywords with the keyword) as in the keyword assigning method disclosed in Japanese Patent No. 329096. It is possible.

In that case, an image dictionary and a keyword dictionary for each field or application can be constructed by selecting an image according to the field of application or application as the above-mentioned exemplary image.

Further, the dictionary creation processing section 6 has an axis adding section 7 for setting axes other than the automatically set n axes in the image dictionary or the keyword dictionary through a dialogue with the user. There is. When two keywords to be arranged on opposite poles are selected, the axis adding unit 7 sets a straight line connecting both selected keywords on the two-axis map where the distance of the selected keyword is the longest as a new axis, and the entire axis is on this axis. The registered keywords are projected to obtain the coordinates on the new axis of all the keywords.

Therefore, according to the above embodiment, it is possible to additionally set the axis having the new evaluation scale designated by the user in the keyword space. By doing so, in addition to the n axes whose specific meanings are difficult to understand, which are obtained as a result of the analysis by the above-mentioned quantification theory III, it is possible to set axes whose meanings are understandable to the user.

The image dictionary (n-dimensional meaning space: image space) constructed as described above and the keyword dictionary (n-dimensional meaning space: keyword space) have a high correlation. Therefore, a keyword suitable for the image is arranged near the coordinates on the keyword dictionary corresponding to the coordinates on the image dictionary of a certain image. The opposite is also true. Therefore, the keyword registration processing unit 2 selects an image suitable for the registration target keyword from the image data storage unit 12 through a dialogue with the user, and obtains an average coordinate of the selected image in the image space. Then, the average coordinates in the image space are additionally registered in the keyword dictionaries of the keyword storage unit 3 and the dictionary storage unit 8 together with the registration target keyword as coordinates in the keyword dictionary of the registration target keyword.

Therefore, according to the above-described embodiment, a new keyword is stored in the keyword dictionary constructed by the above-mentioned exemplary image together with the coordinate vector by a simple process of selecting an image suitable for the keyword to be registered from the registered images. You can register.

As described above, the image dictionary (image space)
And the keyword dictionary (keyword space) have the same n-dimensional meaning space, and thus have a high correlation with each other. Therefore, a keyword suitable for the registration target image is registered near the position in the keyword dictionary corresponding to the position in the image dictionary of the registration target image. Therefore, if the position of the registration target image in the image dictionary is known, the keyword suitable for the registration target image can be obtained.

The vector conversion coefficient creation processing section 9 performs a positive correlation analysis between the n-dimensional coordinate vector of each registered image read from the image dictionary and the p-dimensional feature vector of the registered image. Then, a normal correlation coefficient matrix A1 (n × k) for coordinate vector conversion and a normal correlation coefficient matrix A2 (p × k) for feature vector conversion are obtained and stored in the vector conversion coefficient storage unit 10. Store. Further, the n-dimensional coordinate vectors of the r registered images registered in the image dictionary are multiplied by the normal correlation coefficient matrix A1 (n × k) to obtain a k-dimensional composite variable vector f. Also, the p-dimensional feature vectors of the r registered images are multiplied by the normal correlation coefficient matrix A2 (p × k) to obtain a k-dimensional composite variable vector g. The composite variable matrix F and the composite variable matrix G thus obtained are stored in the vector conversion coefficient storage unit 10. Here, the composite variable matrix F
And the synthetic variable matrix G have a high correlation. Therefore, it is possible to obtain the coordinate vector of the registration target image in the image dictionary from the feature vector of the registration target image.

In the keyword automatic setting processing unit 14, the feature amount extraction unit 15 extracts the feature amount of the registration target image or the change target image to obtain a p-dimensional feature vector, and the vector conversion unit 16 performs the canonical phase relation. The number matrix A2 is multiplied and converted into a k-dimensional composite variable vector g, and the scale is changed. Then, based on the distance between the composite variable vector g of the target image and the composite variable vector f of the r registered images stored in the vector conversion coefficient storage unit 10, q similar to the target image is obtained.
Obtain the registered images (neighboring images). And the obtained q
Based on the distance between the coordinate vector in the image space (image dictionary) of each neighboring image and the coordinate vector in the keyword space (keyword dictionary) of the registered keyword, obtain h neighboring keywords of each neighboring image. I have to. Therefore, q × h keywords suitable for being added to the registration target image or the change target image can be automatically set.

That is, according to the above embodiment, the user can be relieved from the trouble of manually assigning the keyword to the registration target image or the change target image. Therefore,
It is possible to efficiently register a new image and change additional information of an already registered image.

At this time, a plurality of exemplary images are provided with keywords in advance, and the dictionary creation processing unit 6 automatically constructs an image dictionary and a keyword dictionary and stores them in the dictionary storage unit 8. Only then, a keyword suitable for the registration target image or the change target image is automatically set.
Therefore, unlike the keyword assigning method disclosed in Japanese Patent Laid-Open No. 6-295318, it is not necessary to set and register the matching calculation information and the threshold value for each keyword candidate.

When the keyword is automatically set, the keyword automatic setting processing unit 14 in the above embodiment obtains h number of image neighborhood keywords of each neighboring image of the registration target image or the change target image, and registers the registration target image. The q × h keywords appropriate to be assigned to the are set. However, the present invention is not limited to this, and a keyword suitable for the registration target image or the change target image may be set as follows.

FIG. 21 is a flowchart different from FIG. 13 of the keyword automatic setting processing subroutine executed by the keyword automatic setting processing unit 14 in step S75 in the image data registration processing subroutine shown in FIG. Hereinafter, according to FIG. 1 and FIG.
This keyword automatic setting processing operation will be described. Figure 1
In step S74 of the image data registration processing subroutine shown in FIG. 1, if it is determined that a keyword is automatically added, this keyword automatic setting processing subroutine starts.

In steps S161 to S163, as shown in FIG.
Similar to steps S91 to S93 in the keyword automatic setting processing subroutine shown in FIG. 5, p feature quantity extraction by the feature quantity extraction unit 15, vector conversion processing by the vector conversion unit 16, and neighborhood image extraction processing are performed. To be executed. Then, the image near the registration target image or the change target image is extracted.

In step S164, the central coordinate vector of the neighboring image in the dictionary space is obtained based on the coordinate vectors of the extracted q neighboring images in the dictionary space. This center coordinate vector is a vector whose element is the average value of the corresponding element values of the coordinate vectors of all the neighboring images.

At step S165, h'keywords close to the coordinate vector on the keyword space corresponding to the central coordinate vector on the dictionary space obtained at step S164 are selected and fixed as the setting keywords. After that, the keyword automatic setting process subroutine is ended, and the process returns to the image data registration process subroutine shown in FIG.

That is, in this embodiment, the object position calculating means is controlled by the step S162 in the keyword automatic setting processing subroutine shown in FIG.
In step S16, the keyword selection means
3 to S165, the neighborhood object extraction means is constructed in step S163, the attention position calculation means is constructed in step S164, and the neighborhood keyword extraction means is constructed in step S165.

FIG. 22 shows the concept of the keyword automatic setting process shown in FIG. In this keyword automatic setting process,
Similar to the keyword automatic setting process shown in FIG. 13, q registered images (neighboring images) similar to the registration target image are obtained.

The position in the keyword space (keyword dictionary) corresponding to the central position in the image space (image dictionary) of the q neighboring images thus obtained and the position in the keyword space of the registered keyword are Based on the distance, h'keywords near the center position are obtained. Thus
That is, h'keywords suitable for giving to the image to be registered are automatically set.

In the above embodiment, when the coordinates in the keyword space corresponding to the combined variable vector g relating to the feature quantity of the image to be registered are to be obtained, the neighboring image extraction processing subroutine and the image neighboring keyword extraction processing subroutine are used. , The composite variable matrices G and F of registered images are used, but the present invention is not limited to this. That is, for example, the correspondence relationship between the synthesis variable vectors relating to the feature quantities of some representative images and the coordinates in the keyword space is obtained in advance using the synthesis variable matrices G and F of the registered images. Then, using this correspondence relationship, the coordinates in the keyword space corresponding to the combined variable vector g relating to the feature amount of the registration target image may be obtained. In addition, a conversion matrix from the synthetic variable vector relating to the image feature amount to the keyword space is created in advance, and the coordinate in the keyword space corresponding to the synthetic variable vector g relating to the feature amount of the registration target image is calculated using this conversion matrix. You may ask.

Further, in the above embodiment, the case where the keyword is added to the image data has been described as an example.
However, the present invention is not limited to this, and is also applicable to a case where a keyword is added to audio data or multimedia data in which audio data is combined with image data. The point is that the data need only be data capable of extracting a feature amount that represents the feature of the content of the data.

The algorithms of the dictionary creating process operation, the keyword registering process operation, the vector conversion coefficient creating process operation, the image data registering process operation, and the additional information changing process operation are not limited to those in the above flow chart.

[0193]

As is clear from the above, the keyword assigning method of the invention according to claim 1 prepares a two-dimensional or more semantic space in which a plurality of objects and assigned keywords are prepared, and the object to be registered is registered. Is extracted, and a keyword in the vicinity of the position in the semantic space of the registration target object obtained based on the feature amount is selected and assigned to the registration target object. By using the above-mentioned semantic space that can be easily obtained by multivariate analysis such as quantification theory III by simply exemplifying the object of (3), a keyword suitable for the registration target object can be easily and automatically selected and added. Therefore, at the time of object registration, the user can be freed from the extremely troublesome work of selecting and assigning a suitable keyword, and the object registration work can be performed efficiently.

Further, if a semantic space in which a plurality of objects corresponding to fields of use and uses and keywords assigned to these objects are arranged is prepared, appropriate keywords are assigned according to the fields of use and uses. be able to.

Further, in the keyword assigning method of the invention according to claim 2, the correlation between the position on the semantic space and the feature amount regarding a plurality of objects is obtained by using the normal correlation analysis, and the correlation is used by using the correlation. Since the position in the semantic space is obtained based on the feature amount of the registration target object, the position of the registration target object in the semantic space can be correctly obtained from the feature amount of the registration target object. Therefore, a more suitable keyword can be added to the registration target object.

In the keyword assigning method of the invention according to claim 3, the semantic space is prepared separately for the object semantic space and the keyword semantic space, and the registration target in the object semantic space is prepared. N objects are selected from the vicinity of the position of the object, and the keywords in the vicinity of the registration target object are selected from the positions of the n objects in the semantic space for the object using the semantic space for the keyword. Since the selection is performed, the semantic space for the object having the same spatial structure and the semantic space for the keyword are associated with each other, and the keyword located near the registration target object can be easily selected.

Further, in the keyword assigning method of the invention according to claim 4, the semantic space is prepared separately for the object semantic space and the keyword semantic space, and the registration target in the object semantic space is prepared. N objects are selected from the vicinity of the position of the object, an attention position is obtained from the positions of the n objects in the semantic space for the object, and the semantic space for the keyword is used based on the attention position. Since the keywords near the registration target object are selected, the semantic space for the object having the same spatial structure and the semantic space for the keyword are associated with each other, and the keywords located near the registration target object can be easily identified. Can be elected.

Further, the keyword automatic assigning apparatus of the invention according to claim 5 stores in the semantic space storage means information on a two or more-dimensional semantic space in which a plurality of objects and keywords assigned to these objects are arranged. Then, the characteristic amount of the registration target object is extracted by the characteristic amount extraction means, the position of the registration target object in the semantic space is obtained by the object position calculation means based on the characteristic amount, and the registration target by the keyword selection means. Since a keyword near the position of the object is selected and the selected keyword is added to the registration target object by the keyword adding means, the quantification theory is based on the plurality of objects and the added keyword.
By using the semantic space that is easily obtained by multivariate analysis such as class III, a keyword suitable for the registration target object can be easily and automatically selected. Therefore, at the time of object registration, the user can be freed from the extremely troublesome work of selecting and assigning a suitable keyword, and the object registration work can be performed efficiently.

Further, if the semantic space storing means stores the information of the semantic space in which a plurality of objects corresponding to the fields of use and applications and keywords assigned to these objects are stored, It is possible to add an appropriate keyword according to the purpose.

Further, in the keyword automatic assigning device of the invention according to claim 6, the correlation calculating means obtains the correlation between the position on the semantic space and the feature amount regarding the plurality of objects by the normal correlation analysis, and Since the object position calculating means obtains the position of the registration target object in the semantic space based on the feature amount of the registration target object using the correlation, the object position calculation unit determines the registration target object from the feature amount of the registration target object. The position in the above semantic space can be correctly obtained. Therefore, a more suitable keyword can be added to the registration target object.

Further, the automatic keyword assigning device of the invention according to claim 7 stores the information on the semantic space separately into a semantic space for an object and a semantic space for a keyword,
The keyword selecting means uses the neighboring object extracting means for extracting n objects in the vicinity of the registration target object in the object semantic space and the meaning space for the keyword from the position of each of the n objects. Since it has a neighborhood keyword extracting means for extracting a keyword near the registration target object, the meaning space for the object having the same spatial structure and the meaning space for the keyword are made to correspond to each other, and the position near the registration target object is set. You can easily select the keywords you are doing.

The automatic keyword assigning apparatus of the invention according to claim 8 stores the information on the semantic space separately into a semantic space for objects and a semantic space for keywords,
The keyword selecting means includes neighboring object extracting means for extracting n objects in the vicinity of the registration target object in the semantic space for the object, and attention position calculating means for obtaining attention positions from the positions of the n objects. Based on the position of interest, since there is a neighborhood keyword extracting means for extracting a keyword near the registration target object using the meaning space for the keyword, the meaning space for the object having the same spatial structure and the meaning for the keyword A keyword located near the registration target object can be easily selected in association with the space.

Further, the program storage medium of the invention according to claim 9 extracts a feature amount of an object to be registered, and has a two-dimensional or more semantic space in which a plurality of objects and keywords assigned to these objects are arranged. Since the program stores the position of the registration target object above based on the feature amount, selects a keyword near the position of the registration target object, and assigns it to the registration target object, the registration target object is stored. The position on the semantic space can be obtained from the feature amount of. Therefore, it is possible to easily obtain a keyword suitable for giving from the position of the registration target object in the semantic space.

According to a tenth aspect of the program storage medium of the present invention, the correlation between the position in the semantic space and the feature amount regarding a plurality of objects is obtained by a normal correlation analysis,
Since a program for finding the position in the semantic space based on the feature amount of the registration target object using this correlation is stored, the feature amount of the registration target object in the semantic space of the registration target object is stored. The position can be calculated correctly. Therefore, a more suitable keyword can be added to the registration target object.

[Brief description of drawings]

FIG. 1 is a functional block diagram of a search device equipped with an automatic keyword assigning device of the present invention.

FIG. 2 is a diagram showing a device configuration that realizes the functions shown in FIG.

FIG. 3 is a hardware CP that realizes the functions shown in FIG.
It is a block diagram centering on U.

4 is a flowchart of a basic processing operation executed under the control of the CPU in FIG.

5 is a flowchart of a dictionary creation processing subroutine executed by a dictionary creation processing unit in FIG.

FIG. 6 is an explanatory diagram of sample data.

FIG. 7 is a flowchart of an entire dictionary construction processing subroutine that is executed during the dictionary creation processing shown in FIG.

8 is a flowchart of an axis addition processing subroutine executed by an axis addition unit in FIG.

9 is a flowchart of a keyword registration processing subroutine executed by a keyword registration processing unit in FIG.

10 is a flowchart of a vector conversion coefficient creation processing subroutine executed by the vector conversion coefficient creation unit in FIG.

11 is a flowchart of an image data registration processing subroutine executed by the image data registration processing unit in FIG.

FIG. 12 is a flowchart of an image data import process subroutine executed during the image data registration process shown in FIG. 11.

13 is a flowchart of a keyword automatic setting processing subroutine executed during the image data registration processing shown in FIG.

14 is a flowchart of a vector conversion processing subroutine executed by the vector conversion unit in FIG.

15 is a flowchart of a neighborhood image extraction processing subroutine executed during the keyword automatic setting processing shown in FIG.

16 is a flowchart of an image neighborhood keyword extraction processing subroutine executed during the keyword automatic setting processing shown in FIG.

17 is a flowchart of an additional information input processing subroutine executed by an additional information input processing unit in FIG.

18 is a flowchart of a keyword addition processing subroutine executed during the additional information input processing shown in FIG.

19 is a flowchart of an additional information change processing subroutine executed by an additional information change processing unit in FIG.

FIG. 20 is a diagram showing the concept of the keyword automatic setting process shown in FIG. 13.

FIG. 21 is a flowchart of a keyword automatic setting processing subroutine different from FIG.

22 is a diagram showing the concept of the keyword automatic setting process shown in FIG. 21. FIG.

[Explanation of symbols]

1 ... Input unit, 2 ... Keyword registration processing unit, 3 ... Keyword storage unit, 4 ... Image data search processing unit, 5 ... Display unit,
6 ... Dictionary creation processing unit, 7 ... Axis addition unit,
9 ... Vector conversion coefficient creation processing unit, 11 ... Image data registration processing unit, 12 ... Image data storage unit, 1
3 ... Additional information change processing unit, 14 ... Keyword automatic setting processing unit, 15 ... Feature amount extraction unit, 1
6 ... Vector conversion unit, 17 ... Additional information input processing unit.

─────────────────────────────────────────────────── ─── Continued Front Page (56) References Naoto Honma, Spatial Representation of Keyword in Information Retrieval, IEICE Technical Report (NC94-138), March 18, 1995, Vol. 94, No. 563, p. 187-194 Naoto Honma, Spatial representation of keywords and their layering, IEICE Technical Report (NC96-183), March 18, 1997, Vol. 96, No. 584, p. 213-220 (58) Fields surveyed (Int.Cl. ⁷ , DB name) G06F 17/30 JISST file (JOIS)

Claims (10)

(57) [Claims] 1. A keyword assigning method for automatically assigning a keyword to an object to be registered in a storage unit, comprising a two-dimensional or more semantic space in which a plurality of objects and keywords assigned to these objects are arranged. Prepare and extract the feature amount of the registration target object, obtain the position of the registration target object in the semantic space based on the feature amount, and select the registration target among the keywords arranged in the semantic space. A keyword assigning method, characterized in that a keyword located near the position of an object is selected and assigned to the registration target object. 2. The keyword assigning method according to claim 1, wherein a regular correlation analysis is performed on the position and the feature amount of the plurality of objects in the semantic space, and the feature amount is calculated in the semantic space of the plurality of objects. A keyword assigning method characterized in that the position of the registration target object in the semantic space is calculated based on the feature amount of the registration target object using the correlation. . 3. The keyword assigning method according to claim 1, wherein the semantic space has a semantic space for an object in which the plurality of objects are arranged, and the same spatial structure as the semantic space for the object. , The keywords assigned to the plurality of objects are separately prepared in the keyword semantic space in which the keywords are arranged, and the keywords are located in the vicinity of the position of the registration target object in the object semantic space. n objects are selected, and the objects are located in the vicinity of the registration target object using the keyword semantic space based on the positions of the selected n objects in the object semantic space. A keyword assigning method characterized by selecting a keyword. 4. The keyword assigning method according to claim 1, wherein the semantic space has a semantic space for an object in which the plurality of objects are arranged, and the same spatial structure as the semantic space for the object. , The keywords assigned to the plurality of objects are separately prepared in the keyword semantic space in which the keywords are arranged, and the keywords are located in the vicinity of the position of the registration target object in the object semantic space. n objects are selected, based on the positions of the selected n objects in the object semantic space, a position of interest in the object semantic space is determined, and based on the position of interest, Located near the registration target object using the semantic space for the keyword Keyword assignment method which is characterized in that to elect a keyword. 5. A keyword automatic assigning device for automatically assigning a keyword to an object registered in a storage unit, which is a two-dimensional or more semantic space in which a plurality of objects and keywords assigned to these objects are arranged. Information storing means stores a meaning space storing means, a feature amount extracting means for extracting a feature amount of a registration target object, and an object position calculation for obtaining the position of the registration target object in the meaning space based on the feature amount. Means, a keyword selecting means for selecting a keyword located in the vicinity of the position of the registration target object from among the keywords arranged in the semantic space, and assigning the selected keyword to the registration target object An automatic keyword assigning device, which is provided with a keyword assigning means. 6. The automatic keyword assigning device according to claim 5, wherein a regular correlation analysis is performed on the position and the feature amount of the plurality of objects in the semantic space to obtain the semantic space of the plurality of objects. The object position calculating means is provided with a correlation calculating means for obtaining a correlation between the upper position and the feature amount, and the object position calculating means uses the correlation, based on the feature amount of the registration target object, An automatic keyword assigning device, characterized in that the position in the semantic space is obtained. 7. The automatic keyword assigning device according to claim 5, wherein the semantic space information is a semantic space for an object in which the plurality of objects are arranged, and the same spatial structure as the semantic space for the object. And the keywords assigned to the plurality of objects are separately stored in the semantic space for keywords in which the keywords are arranged. The keyword selection means is configured to store the object to be registered in the semantic space for the objects. Based on the position of each of the extracted n objects in the semantic space for the object, the proximity space for the keyword is extracted based on the position of each of the extracted n objects. To search for keywords located near the above registration target object. Keywords automatic application device, characterized in that it comprises a proximity keyword extracting means for output. 8. The automatic keyword assigning device according to claim 5, wherein the information on the semantic space is a semantic space for an object in which the plurality of objects are arranged, and the same spatial structure as the semantic space for the object. And the keywords assigned to the plurality of objects are separately stored in the semantic space for keywords in which the keywords are arranged. The keyword selection means is configured to store the object to be registered in the semantic space for the objects. In the semantic space for the object based on the positions in the semantic space for the object of the n objects that are extracted in the vicinity of the position n. Attention position calculation means for obtaining the attention position in the Te, by using the semantic space for the keyword, an automatic application device which comprising the near keyword extracting means for extracting a keyword that is located in the vicinity of the object to be registered. 9. A feature quantity of a registration target object is extracted, and a plurality of objects and keywords assigned to these objects are arranged based on the feature quantity.
The position of the registration target object in the semantic space of dimension or more is obtained, and among the keywords arranged in the semantic space, the keyword located near the position of the registration target object is selected, and the registration is performed. A program storage medium, which stores a program to be given to a target object. 10. The program storage medium according to claim 9, wherein a regular correlation analysis is performed on the position and the feature amount of the plurality of objects in the semantic space, and in the semantic space of the plurality of objects. A program for obtaining the correlation between the position and the feature amount, and using the correlation, determining the position of the registration target object in the semantic space based on the feature amount of the registration target object. Characteristic program storage medium.