JPH04237377A - Image retrieving system using arrangment and color attribute of keyword - Google Patents

Abstract

PURPOSE:To retrieve an image by directly describing the specification of the position and positional relation of a keyword by an image held by a user without using character language. CONSTITUTION:A keyword inputted/displayed from a keyboard 1 is specified by a pointing device 5 and optionally arranged on a display screen. A keyword position relation computing part 11 finds out the mutual positional relation of all arranged keywords based upon the arrangement information of the keywords on the display screen. All keywords, the positions of respective keywords on the screen and the mutual positional relation of respective keywords are sent to a data base management part 12 as retrieving keys. In image retrieval using positional information in an image data base, a retrieving method addaptive to human's image structure is offered.

Description

[Detailed description of the invention]

0001

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image retrieval method using natural language.

0002

[Prior Art] Conventionally, as a search method for multimedia databases containing figures, tables, and photographs, a search keyword that reflects the image content is assigned to the image in advance, and the image is searched by keyword search to express the structure of the image. There are various search methods, such as searching for images with the same or similar structure using an abstract image as a search key, and classifying and searching target images using feature parameters such as image pixel run length.

[0003] When a human recognizes an image, a vague image of the object and its positional relationship is formed in the memory, which is the key to the memory. Similarly, when searching for an image with a certain image, the image often contains extremely vague information. When searching for images based on this information, it is effective to specify the name, color, size, position, and positional relationship of the object in the image with other objects. Expressing everything in natural language is not necessarily efficient. Particularly in the description of position and positional relationships, such information is not originally linguistic information, but it is arbitrarily written as ``above, right,...
”, it corresponds to written language. Retrieving such information through written language is inefficient and may increase ambiguity.

0004

[Problems to be Solved by the Invention] The present invention has been made to solve the problems of the prior art described above.
To provide an image search method that allows a user to directly specify the position and positional relationship of keywords using his or her own image without using written language.

[0005]

[Means for Solving the Problem] In the present invention, at least one or more keywords expressed in natural language are directly added to an image, and the keyword and position given as a search key, and the position between each keyword are provided. The present invention is characterized in that it recognizes the relationship, searches for images that have a keyword relationship that matches these search keys in position, and that the relative positional relationship between the keywords is within a predetermined range.

[0006]

DESCRIPTION OF THE PREFERRED EMBODIMENTS Examples of the present invention will be explained in detail below with reference to the drawings.

FIG. 1 is a block diagram of an embodiment according to the invention.

As shown in FIG. 1, the configuration of this embodiment includes a keyboard 1, an image data file 2, and a screen display controller 3.
, display means 4, pointing device 5 such as a mouse, pointing ON/OFF detection section 6, pointing position comparator 7, character code memory 8, character pattern display position memory 9, keyword storage section 10, and keyword position relationship calculation section 11 , a database management section 12 , and a mode changeover switch 13 .

The keyboard 1 is used to input character codes.

[0010] Image data file 2 stores image data to which an internal key has been assigned.

The screen display controller 3 displays character patterns and pointing cursor patterns on the display means 4.

Further, image data is read from the image data file 2 and displayed on the display means 4 based on an internal key for specifying image data.

The pointing device 5 is used to input pointing cursor coordinate information (x, y) and pointing ON/OFF information.

[0014] The pointing ON/OFF detection section 6
Upon receiving the coordinate information and pointing ON/OFF information from the pointing device, the pointing cursor position information is sent to the screen display controller 3, and in accordance with the pointing ON/OFF information, the pointing position comparator 7 starts the comparison process. Send a signal to finish.

The pointing position comparator 7 compares the pointing position with the display position information of the keyword character corresponding to each image stored in the character display position memory 9, and selects a character pattern whose position matches with the character code memory. 8 and sends it to the screen display controller 3, and also sends the character code and position coordinate information of the pointing cursor after the pointing position comparison process has started to the keyword positional relationship calculation unit 11.

The keyword positional relationship calculation unit 11 determines the relative positional relationship between at least one or more keywords sent from the pointing position comparator 7, and searches for all keywords, keyword positions, and keyword positional relationships. It is sent to the database management unit 12 as a foreign key for use.

The database management section 12 receives an internal key specifying the image data sent from the mode changeover switch 13, and stores it in association with an external key received from the keyword positional relationship calculation section 11. Further, if an instruction signal to switch to the search mode is received from the mode changeover switch 13, an internal key corresponding to the external key is sent to the screen display controller 3.

When the mode change switch 13 receives a specific character code for specifying a mode from the keyboard 1, it sends a mode change signal to the database management section 12, and thereafter input from the keyboard 1 is transferred to the screen display controller 3.
send to

The above operation will be explained as follows.

When a character code is input from the keyboard 1, the mode change switch 13 determines whether the code is for a mode change instruction, and if it is a code for a mode change instruction, the mode change signal is
It is sent to the database management section 12.

[0021] In the following, the keyword assignment mode and search mode will be explained separately.

(Keyword Assignment Mode) After the mode is switched, the character code is determined to be a keyword and sent to the screen display controller 3 until the mode switching code is input again. However, if the character code is an internal key that specifies image data in the image data file 2, the internal key is sent to the database management section 12, and
It is also sent to the screen display controller 3. Upon receiving the internal key, the screen display controller 3 reads the image data corresponding to the internal key from the image data file 2 and displays it on the display means 4. The keywords inputted thereafter are displayed in a specific area on the display means 4, and the keywords are displayed in a specific area on the display means 4.
This process is repeated until a specific character code is input, such as keyword 2, . . . , keyword n. At this time, information on the character code and character display position of each keyword is stored in the character code memory 8 and character display position memory 9 in the keyword storage section 10, respectively.

[0023] When the input of the keyword from the keyboard 1 is completed, the coordinate information and pointing ON/OFF information from the pointing device 5 are changed to the pointing ON/OFF information.
The signal is taken into the OFF detection section 6. Pointing OFF
If so, the pointing cursor pattern is sequentially displayed on the display means 4 via the screen display controller 3. When pointing is ON, an activation signal and pointing cursor position information are sent to the pointing position comparator 7.

The pointing position comparator 7 that receives the activation signal compares the coordinate information of the pointing cursor with the position in the character display position memory 9, checks whether a keyword exists at the same position, and if it exists, displays that character. The character code is taken out from the character code memory 8, and the character code is sent to the screen display controller 3 together with the pointing position information to be displayed on the display means 4, superimposed on the image already displayed. As a result, on the display means 4, each keyword input from the keyboard 1 is placed at an arbitrary position above the displayed image. At this time, the positional information of the placed keyword is sent from the pointing position comparator 7 to the keyword positional relationship calculating section 11 together with the character code, and is stored therein. The operation of arranging the keywords is repeated until a specific end signal is sent from the pointing device 5, whereby a plurality of keywords are arranged on the display means 4. When an end signal is sent from the pointing device 5, the keyword placement operation ends. Next, the keyword positional relationship calculation unit 11 calculates the early exit positional relationships between all keywords, such as vertical, horizontal, direction, etc., from the already accumulated positional information of each keyword, and calculates the positional relationships of all keywords and each keyword. , the mutual positional relationship between the keywords is sent to the database management unit 12 as a search external key.

The database management section 12 associates and stores the internal key specifying the image data sent from the mode changeover switch 13 and the external key received from the keyword positional relationship calculation section 11.

The series of operations described above, consisting of reading and displaying image data using internal keys, arranging keywords, and creating and storing external keys, is repeated until the mode is switched.

(Search Mode) After the mode is switched, the character code is determined to be a keyword and sent to the screen display controller 3 until the mode switching code is input again. Keywords input thereafter are displayed in a specific area on the display means 4, and the keyword 1 is displayed in a specific area on the display means 4.
, keyword 2, . . . , keyword n, this process is repeated until a specific character code is input. At this time, information on the character code and character display position of each keyword is stored in the character code memory 8 and character display position memory 9 in the keyword storage section 10, respectively.

When the input of the keyword from the keyboard 1 is completed, the coordinate information and pointing ON/OFF information from the pointing device 5 are changed to the pointing ON/OFF information.
The signal is taken into the OFF detection section 6. Pointing OFF
If so, the pointing cursor pattern is sequentially displayed on the display means 4 via the screen display controller 3. When pointing is ON, an activation signal and pointing cursor position information are sent to the pointing position comparator 7.

The pointing position comparator 7 that receives the activation signal compares the coordinate information of the pointing cursor with the position in the character display position memory 9, checks whether a keyword exists at the same position, and if it exists, displays that character. , and sends the character code along with pointing position information to the screen display controller 3.
The image is displayed on the display means 4, superimposed on the image already displayed. As a result, on the display means 4, each keyword input from the keyboard 1 is placed at an arbitrary position above the displayed image. At this time, the positional information of the placed keyword is sent from the pointing position comparator 7 along with the character code to the keyword positional relationship calculation unit 11.
is sent to and stored. The operation of arranging the keywords is repeated until a specific end signal is sent from the pointing device 5, whereby a plurality of keywords are arranged on the display means 4. When the end position signal is sent from the pointing device 5, the keyword placement operation ends. Next, the keyword positional relationship calculation unit 11 calculates the relative positional relationship between all the keywords, such as vertical, horizontal, direction, etc., from the already accumulated positional information of each keyword, and calculates the position of all keywords and each keyword. , the mutual positional relationship between the keywords is sent to the database management unit 12 as a search external key.

The database management unit 12 determines a corresponding internal key from the external key received from the keyword positional relationship calculation unit 11 and sends it to the screen display controller 1. The screen display controller 3 that received the internal key saves the image data file 2.
The image data corresponding to the internal key is read out from within and displayed on the display means 4 as an image of the search result.

[0031]

As described above, the present invention is a search method adapted to the image structure of human images in image search using position information in an image database.

[Brief explanation of the drawing]

FIG. 1 is a block diagram of an embodiment according to the invention.

[Explanation of symbols]

1 Keyboard, 2 Image data file 3 Screen display controller 4 Display means 5 Pointing device such as a mouse 6 Pointing ON/OFF detection section 7 Pointing position comparator 8 Character code and pattern memory 9 Character pattern display position memory 10 Keyword storage section 11 Keyword positional relationship calculation unit 12 Database management unit 13 Mode changeover switch

Claims (2)

[Claims] 1. An image retrieval system for identifying the position of an object constituting an image and searching for an image, comprising: means for combining a character/symbol string key representing an object with an object on the image;
means for inputting the absolute position of the object on the image combined with the key; means for calculating the relative positional relationship between at least one or more objects on the image combined with the key;
One or more character/symbol string keys related to the constituent objects of the image obtained as the output of the means, the color attribute of the object corresponding to the key, the absolute position of the object, and the data of the relative positional relationship between each object are used as image search keys. 1. An image retrieval method, comprising means for creating a structured image, and combining the key with the image and storing the key. [Claim 2] Expressing the constituent objects of an image and their color attributes as character strings and symbol strings, and creating and using the color, absolute position, and relative positional relationship between objects of the object corresponding to the character and symbol strings as search keys. In a stored image search system, a means for analyzing a relative positional relationship key of two or more objects from a character/symbol string key indicating one or more objects to be searched and a position summary key inputted from the outside; A means for determining the area on the image where the object should exist from the position outline key, a means for determining the relative positional relationship of the object to be searched from the relative positional relationship key of the object, and a search based on the outputs of the above three means. The image is searched by searching for some or all of the objects that make up the image, using a means for determining the key and a means for searching the search key part of the image using the determined search key. An image search method that uses