KR970010030B1 - Picture search system - Google Patents

Abstract

User question is inputted to system through keyboard(10), and the words question of user inputted through keyboard input unit(10) is transferred to microprocessor(20). According to the question words transferred microprocessor(20) abstracts index words from all sorts of dictionary written down to a large amount of storage(30) to move it to main memory(40). Microprocessor(20) computes the frequency for each index words, based on that, it decides the prior ranking of index words to arrange index words by prior ranking decided. Detected result outputs the photo data recorded to a large amount of storage(30) to high resolution screen(50).

Description

Photo retrieval system

1 is a block diagram of a hardware system for implementing the present invention.

2 is a diagram showing the configuration and data processing flow of a software system for realizing the present invention.

3 is the initial screen of the photo retrieval system.

4 is a part of the icon search unit screen.

5 is a screen of the natural language query input unit.

6A to 6G are flowcharts showing the function performance of each component of the photo retrieval system according to the present invention.

* Explanation of symbols for main parts of the drawings

10: keyboard 20: microprocessor

30: mass storage device 40: main memory

50: screen output device 60: mouse

The present invention relates to a system that allows a user to input a natural language query into a computer and search for a desired picture or related data thereof with a large amount of picture data stored in a PC-class computer.

In today's so-called information society, computer users are becoming more and more demanding as well as the demand for computer users, and with the development of society, the need to deal with huge amounts of data is rising. It is true.

Therefore, a database management system has already been developed and put into practical use for the purpose of providing a user with the greatest convenience by efficiently managing large amounts of information.

Most corporations and societies have made great efforts in managing and manipulating the huge size of databases requested by society.

In the present, the development of the field of processing a large amount of image information has been made rapidly.

However, queries such as Structured Query Language (SQL), Pictorial Structured Query Language (PSQL), Query-by-Pictorial-Example (QPE), and Query-By-Example (QBE), which are queries for user dialogue when retrieving image information This is not only difficult to use from the user's point of view, but also has a disadvantage of remembering the grammar of the query.

In order to improve this point, it is very important to secure smooth communication means between human computers.

In order to solve this problem, it is necessary to examine the characteristics of human behavior, which is the center of use, and, if possible, to study how to make the most of the functions of human senses.

In general, the user's computer feels fear or discomfort, so the first and most efficient user interface method to come up for practical use and acquisition of future information is to search for information with everyday language that does not require any learning from the user.

Therefore, there is an urgent need for information inspection using natural language interfaces. The development of the user interface using natural salmon has been developed mainly as a front-end-system of a database abroad, and now commercially available systems have appeared.

However, the development of natural language interface in Korea has not been achieved systematically yet.

It is possible to search the stored data only by using the formal structure instead of the general query method and the data structure dependent on each system in the database system or the data retrieval system which is widely used to date.

For this reason, it takes a long period of training and subsequent costs to use a database system or a data retrieval system that is installed by ordinary users.

SUMMARY OF THE INVENTION The present invention provides a system that makes it possible to retrieve data stored by a language used by a human being (ie, a natural language) so that a user without a special knowledge of a computer can easily access and use the system. have.

In the present invention, in order to enable the user to efficiently search for the desired information, a natural language query is analyzed to extract a phrase (TERM), which is an index word of the information retrieval system.

In addition, the present invention additionally includes a menu-type interface for constructing a stopword dictionary, an electronic home appliance, and a similar word dictionary necessary for accurate search, and mainly using a graphic based on a room metaphor other than the natural language interface. .

Here, the room metaphor is a metaphor for office environments such as clocks, calculators, printers, memories, books, etc., which are commonly found around.

The present invention also introduces a hierarchical graphical retrieval interface using graphically represented icons of the classification (mostly, mid-class, sub-class) of each photo stored.

1 is a block diagram of a hardware for realizing a photo retrieval system according to the present invention. The operation of each component is as follows.

The user query is input to the system through the keyboard 10, and the natural language query of the user input through the keyboard input device 10 is transmitted to the microprocessor 20.

According to the transmitted query word, the microprocessor 20 extracts a color language based on various dictionaries pre-stored in the mass storage device 30 and places the color language in the may memory 40.

Again, the processor 20 calculates a frequency for each index word, determines the priority of the index words according to the frequency, and sorts the index words by the determined priority. At this time, all index words are located in the main memory 40 for high-speed execution of frequency calculation.

The results detected in this way are used to output the photo data contained in the mass storage device 30 on the high resolution screen 50.

The photographic data is stored in the mass storage device 30 in a compressed form by a specific compression algorithm, and the data is restored to a data form before being compressed by the microprocessor 20 to be displayed on the high resolution screen 50. The picture is reproduced.

The four loans can be moved through the mouse input device 60, selection of objects, extraction of photo data, and the like.

2 is a diagram showing the configuration of the function execution means of the system 하는 that performs a photo search through a natural language query and the function execution order of each means according to the present invention.

Each of the above means are those realized by software stored in the microprocessor 20. That is, the microprocessor 20 may include the icon search unit 21 shown in FIG. 2, the natural language input unit 22, the search separation unit 23, the index word search unit 24, the frequency calculator 25, And a photo data output unit 26 and a screen output unit 27. The above-described function realization means realized by the microprocessor 20 will be described in detail later.

The start method of the picture retrieval system is a logo screen of the picture retrieval system according to the present invention is displayed by inputting, for example, sara3 through the keyboard 10 in a DOS prompt state. If you press any key, you will be redirected to the initial screen showing the office environment as shown in FIG.

The following describes each object for the base screen.

Bookshelf: Clicking with the mouse (60) shows an enlarged bookshelf, and the user selects a desired book and immediately changes the bookshelf mode.

· Clock: Displays the current time while zoomed in.

· Calculator: Performs basic arithmetic operations while zoomed in.

Memory: The enlarged display allows memo to be performed by the mouse (60), and can be searched by forward backward.

· Q: The screen fades out gradually, and I finish my work in the room environment and fall back to DOS state.

Book: In the room environment, the booklet changes to the mode.

The contents of the book are divided into two pages. The first screen is divided into major categories, the next screen is divided into subclasses, and the next screen displays icons representing subclasses. The user can click on these icons to search for the desired photo. Three pictures and their titles appear on a page.

Lady: The environment messages about what comes to the photo retrieval system are explained with animation and sound, and usually you ask someone to enter the picture code you want to find using the keyboard or to know about the Arc de Triomphe in Paris. As you can see, you can ask questions in natural language that are not tied to form.

In the initial screen as described above, the user can select a search method to be used.

That is, one method of searching using a classification table based on an icon or a natural language query should be selected. This is specified when the user selects a lady. If the user selects a lady, a query input dialog box is displayed on the screen. Otherwise, the icon search screen by the classification table is displayed.

As shown in Fig. 6A, the icon retrieval unit 21 searches for the classification (large classification, medium classification, small classification) selected by the user, and then extracts the identifier of the photo data.

The result of this is passed directly to the photo data extraction section 26. One of the screens output by this icon search unit 21 is shown as an example in FIG.

The natural language query input unit 22 is dedicated to moving the position of the query language composed of Korean and English inputted by the user through the keyboard 10 from the dialog box to the main memory 40.

FIG. 5 illustrates a query input screen, and FIG. 6B illustrates a process of performing a specific function of the natural query input unit 22.

The survey beak 23 first extracts only nouns in order to process the query words transmitted by the input unit 22 to the main memory 40. For this purpose, the survey is separated by using various pre-produced dictionaries. Extract only cotton yarn. For example, if there is a query that shows me about the scenery of thatched houses in Japan, the survey division extracts only the nouns of Japan, thatched houses and scenery.

The specific function execution process of the irradiation separation unit 23 is as shown in FIG. 6C.

Conventional information retrieval techniques have usually used a person to index data having a structured structure by keyword, store it in a database, and retrieve it in a specific language. However, in the form of data used in the present system, there are not only standardized data but also unstructured data.

For a detailed understanding of the system, look at the data structure of the sample slide film shown in Table 1 below.

Table 1 shows the relevant data for one slide of film used in the system of the present invention. Of course, there are pictures of each slide associated with it.

As shown in Table 1 above, the classification number, identification number, title, photo size, color, photographer, and photographing date have a typical structured database.

These data become indexes of the photo retrieval system and are used as keywords in retrieval.

The picture content field is unstructured text data and cannot be used as an index word in a general database. However, in the present system, a full-text indexing technique is used to automatically index and search words. However, the slide photograph itself is not text but unstructured data and cannot be used as an index word, but only as data provided to a user on a screen or a printer.

For the data of the type having such a data structure, the existing search system can search only the structured data structure described above.

For example, in a typical database retrieval system, a method for retrieving a photo as shown in Table 1 should use the following extremely low-language language (when using the SQL language in a relational database).

...

SELECT FROM SLIDE FILM

WHERE Title = Mexico Music and Photographer = S. W. Frank…

In particular, when the user does not have the most knowledge about the slide film to be searched, it is difficult to expect a satisfactory search result.

And even if the search result comes out, the amount is probably the amount that the user should search with his eyes and hands.

However, in this system, if a user inquires using a natural language the type of photo description desired by a user without information on a specific slide film, the related slide film and a detailed description thereof are displayed on a computer screen.

Next, a search operation in the color inner searcher 24 and the frequency calculator 25 will be described.

The search separation section 23 generates [Japan, thatched houses, landscapes] from the query "Show me about thatched landscapes in Japan" and delivers them to the index word search unit 24. The picture retrieval system as shown in FIG. The search at is done.

If nouns such as Japan, thatched houses, and landscapes are generated as the search data, items corresponding to each noun are obtained from the keyword index file created in another system. Assume that the obtained results are as shown in Table 2 below.

The meaning of the index term index entry in Japan means that the total number of cards in which the index term is used is four and that it is used once for 11060 cards, twice for 11063 cards, once for 11067 cards, and once for 11072 cards. do.

It can also be seen that it was used five times (1 + 2 = 1 + 1) overall. Cards referred to herein refer to a slide film and a description thereof associated with each sheet.

When the user's question is input, the card to be shown is the card having the highest frequency when the similarity between keywords appearing in the user's question is obtained.

Such a search is performed using index index search results, and is divided into OR and AND operations according to a method of obtaining similarity.

This operation is processed in the frequency calculator 25 performing the process as shown in FIG. 6E. First, in preparation for finding the similarity between index words, the index index search results are stored as an array of lags (records with two elements: 1. card number, 2. frequency on that card), and it is also used for each card in the record. Sorting by the number element size, the result is shown in Table 3 below, which shows the preparation process for finding similarity between index words.

Let's first look at the OR operation. If there is really a feature card that contains the user's content, then most of the index words included in the question will be included in the content of the card.

For example, if the 11060 card includes Japan, thatched houses and landscapes, the card 11060 will appear three times (one for each index), as shown in Table 2.

Therefore, if the index words included in the question are concentrated on a particular card, it can be determined that the card is the one desired by the user. Therefore, from Table 3, we can consider that the same card number is repeated or the frequency of occurrence is higher, which is higher priority as a search result.

The degree of high priority as a search result is called the weight, and an algorithm for calculating the importance of each card is shown in Table 4 below.

Each card number and their importance generated by this algorithm is stored as an array of records (two elements: 1. card number, 2. importance), which in turn is sorted by importance and made into the final search result (table 5).

Now let's look at the difference between the OR operation and the AND operation.

Looking at the result of the OR operation (Table 5), there are only 11060 and 11072 defending cards containing all three keywords in the question. We can assume that these two cards are what the user wants.

However, in the logic operation, if the index word used in the query is used even once, the card is provided as a search result. In some cases, it causes unnecessary computation time and provides the user with results that are not related to the question.

Therefore, we need a function that can sort out the cards that contain all the index words included in the query word.

Table 6 below shows the algorithms required for the logical product operation.

As described above, in the logical AND operation, in some cases, unnecessary operation time can be saved than the OR operation, but the user often needs to perform the OR operation. It is configured to select a task.

The photo data output unit 26 is responsible for extracting the relevant photo data from the photo database using the cards as a result of the function of the frequency counting unit 25. A detailed functional process thereof is shown in FIG. have.

The image storage compression algorithm used in this photo retrieval system is used to select the one that is not very complicated in order to increase the performance of the system, that is, the effect of real-time retrieval.

The basic requirement of a photo retrieval system is to be able to retrieve a large amount of photos in a very short time, and with the use of highly complex algorithms, the system degradation that comes from it is ignored, no matter how much space can be saved on the storage device. It should not be lost. Therefore, the screen output unit 27 in the present system is configured in consideration of these requirements.

As shown in FIG. 6G, the screen output unit 27 executes a job of outputting the photo data extracted from the photo data output unit 26 to the high resolution screen 50.

In this screen output unit 27, the maximum usable amount is allocated to the main memory 40 in order to reproduce the compressed picture data in real time, thereby minimizing execution time.

Create a processing module suitable for a high resolution graphics output device.

Claims (1)

Through the mass storage device 30, the main memory 40, a keyboard 10 for inputting a user's natural language query, and the keyboard 10, in which compressed photographic data are stored and dictionaries are constructed. According to the user's natural language, the index words are extracted from the dictionaries obtained in advance in the mass storage device 30, placed in the main memory 40, the frequency is calculated, the priority is determined, and the priority order is determined. A high resolution screen 50 for performing photo retrieval by sorting index words and reproducing photo data obtained in the mass storage device 30 according to the results retrieved by the microprocessor, and in the system A mouse 60 for carrying out the movement and selection of objects and extraction of photographic data; The microprocessor 20 performs a search for each category selected by the user, extracts an identifier of photo data, and displays a query word composed of Korean and English inputted by the user through the icon search unit 21 and the keyboard 10. A noun using the dictionaries to process a query transmitted to the main memory 40 by the natural language query input unit 22 that moves the position to the main memory 40 in a dialog box. On the basis of the survey separation unit 23 for extracting the bay, the index word search unit 24 and the frequency calculating unit 25 for obtaining the index and the frequency of the index word, respectively, and the cards which are the result of performing the function of the frequency calculating unit 25. The high resolution screen 50 displays the pre-data output unit 26 for extracting related photo data from the photo database and the photo data extracted from the photo data output unit 26. Photo retrieval system comprising a screen output unit 27 for outputting.