KR100918361B1 - System and method for high-speed search modeling - Google Patents

Abstract

An accelerated search modeling system and method are disclosed. The accelerated search modeling system includes a test collection generator that generates a test collection using search results for a query, a search model generator that generates a search model for determining a correct ranking according to the query from the test collection, and the generation. It includes a search model evaluation unit for evaluating the performance of the search model.

Faster search modeling, test collections, correlations, features

Description

Speeding Search Modeling System and Method {SYSTEM AND METHOD FOR HIGH-SPEED SEARCH MODELING}

The present invention relates to a fast search modeling system and method, and more particularly, to a system for building a fast search modeling by generating a test collection using a search result for a query, and generating and evaluating a search model from the test collection. And to a method.

Recently, people with various hobbies are increasing the demand to search for professional knowledge. People can acquire specialized knowledge data in specific fields such as movies, automobiles, securities, and sports by searching a database that stores information on a specific field through a search engine. For example, a person who wants to collect information about 'wine' may collect search results through the query word wine.

However, it has been difficult to create a search model for searching a database in which information on a specific field is stored. In detail, in the process of generating a conventional search model, the developer intuitively generates and tunes the search model and repeats the process of reviewing the search service planner. In other words, the search model is modeled around the developer, and after the demo is generated, the search model is modified and completed through review by the planner.

In this case, a developer may lack a knowledge or experience with specialized data, and thus an incorrect search model may be generated. Then, a wrong search result may be exposed to the query input by the user. In order to prevent such a problem, a search model may be generated by reflecting the opinion of the search planner, but it may still be a problem in efficiency due to a communication problem between the developer and the search planner.

Therefore, if the characteristics of specialized data are known, an invention that can generate a search model even if the developer level of the search model is not required.

The present invention can provide a fast search modeling system and method that can provide the correct ranking of the expert knowledge by generating a test collection using the search results for the query.

The present invention can provide a speedy search modeling system and method that can generate a more accurate search model by aligning the ranking of search results for a query term by experts or search planners for the query term.

The present invention can provide a fast search modeling system and method that can quickly modify the search model by evaluating the generated search model in real time.

According to the present invention, when the performance of the generated search model is not met, the fast search modeling system can generate a more stable and efficient search model by rearranging the ranking of search results and regenerating a test collection. And methods.

The fast search modeling system according to an embodiment of the present invention includes a test collection generation unit that generates a test collection using a search result for a query, and determines a correct ranking according to the query from the test collection. A search model generator for generating a search model and a search model evaluator for evaluating the performance of the search model by analyzing evaluation data on the generated search model.

In this case, the search model generator may generate a search model through a machine learning method.

The search model evaluator may analyze weights of each of the selected features with respect to the search result.

In addition, the search model evaluator may check the accuracy and correlation with respect to the generated search model in real time.

In accordance with an aspect of the present invention, there is provided a method of speeding up a search model, the method comprising: generating a test collection using a search result for a query, generating a search model capable of determining a correct ranking according to the query from the test collection; Analyzing evaluation data with respect to the generated search model may include evaluating the performance of the search model.

In this case, the step of generating a test collection may generate a test collection for the query by sorting the ranking of the search results.

According to the present invention, there is provided a fast search modeling system and method that can provide the correct ranking of the expertise by generating a test collection using the search results for the query.

According to the present invention, there is provided a speedy search modeling system and method that can generate a more accurate search model by sorting the ranking of search results for a query word by an expert or search planner for the query word.

According to the present invention, there is provided a speedy search modeling system and method that can quickly modify a search model by evaluating the generated search model in real time.

According to the present invention, if the performance of the generated search model is evaluated and the performance is less than the criterion, the fast search can generate a more stable and efficient search model by rearranging the ranking of the search results and regenerating the test collection. Modeling systems and methods are provided.

Hereinafter, with reference to the contents described in the accompanying drawings will be described in detail an embodiment according to the present invention. However, the present invention is not limited or limited by the embodiments. Like reference numerals in the drawings denote like elements. The speed search modeling method according to an embodiment of the present invention may be performed by the speed search modeling system.

1 is a block diagram showing the configuration of a fast search modeling system according to an embodiment of the present invention.

The accelerated search modeling system 100 according to an embodiment of the present invention may include a test collection generator 101, a search model generator 102, and a search model evaluator 103.

The test collection generator 101 may generate a test collection by using a search result for the query. In one example, the test collection generation unit 101 may generate a test collection for the query by sorting the ranking of the search results. For example, if 10 search results are derived from the query 'wine', the test collection generator 101 may generate one test collection by sorting the 10 search results for 'wine' according to the ranking. have.

In this case, the test collection may be referred to as a set of rankings in which a specific query and search results for the query are sorted. In other words, the test collection may refer to a collection including a query and a correct ranking of search results for the query. Here, the correct ranking of the search results for the query may be generated during the initial sorting process, but may also be generated through an iterative reordering process.

In this case, the test collection generation unit 101 may receive a search result for a query from the database 104. For example, the database 104 may store professional information on a specific field, such as 'flower', 'wine', 'sports', 'jae tech', 'music', and the like.

For example, the test collection generation unit 101 may receive a comment or command of an expert or a search planner having knowledge and experience in a corresponding field to which a query word belongs, to sort the ranking of search results through a user terminal. The present invention can provide a speedy search modeling system and method that can generate a more accurate search model by aligning the ranking of search results for a query term by experts or search planners for the query term.

The test collection generation unit 101 may generate a test collection for each of a plurality of queries in a specific field. Therefore, the number of test collections generated may be one or more.

As a result, according to an embodiment of the present invention, when a searcher searches by inputting a query word for a specialized field, search results in which the ranking is arranged according to the intention of the expert or search planner may be exposed to the searcher. That is, according to an embodiment of the present invention, an accurate search result for a query belonging to a specialized field may be provided to a searcher.

The process of generating the test collection is described in detail in FIGS. 2 and 3.

The search model generator 102 may generate a search model capable of determining the correct ranking according to the query word from the generated test collection. In this case, the search model generator 102 may generate a search model from the test collection by using a machine learning method. For example, the search model generator 102 may generate a search model by using a machine learning method such as linear regression, classification and regression tree, logistic regression, ListRank, Bradley-Terry Model, and Multi-Class Bradley-Terry Model. Can be.

In addition, the search model generator 102 may generate a search model by selecting at least one feature and a normalization method for the feature with respect to the search result. In this case, the feature may mean data that is a reference when sorting the ranking of the search results. That is, the search model generator 102 may generate a search model by learning which features are used to sort the ranking of search results when generating a test collection.

A process of selecting a feature to generate a search model by the search model generator 102 is described in detail with reference to FIG. 4.

The search model evaluator 103 may evaluate the performance of the generated search model. Evaluating the performance of the search model can determine whether the generated model can provide the required search results.

In this case, the search model evaluator 103 may analyze the weight of each of the selected features with respect to the search result. In other words, the analyzed weights may indicate which features have become important criteria when sorting the ranking of search results.

In addition, the search model evaluation unit 103 may check the accuracy and the correlation with respect to the generated search model in real time. That is, according to the exemplary embodiment of the present invention, the search model evaluator 103 may evaluate the performance of the search model in real time, so that the problem of the search model may be quickly identified.

At this time, if the performance of the search model does not meet the preset criteria, the test collection generator 101 may regenerate the generated test collection by rearranging the ranking of the search results. As shown in FIG. 1, a final search model 105 that can perform performance above a certain standard may be generated through iterative test collection generation, search model generation, and search model evaluation. That is, according to an embodiment of the present invention, the search model 105 that can ensure stable performance by evaluating the performance of the search model through analysis of the evaluation data can be generated. The search model evaluator 103 is described in detail through the example of FIG. 5.

2 is a diagram illustrating an example of a process of generating a test collection according to an embodiment of the present invention.

Specifically, FIG. 2 illustrates a process of sorting the search results for the query word 201. Referring to FIG. 2, a process of generating a test collection by arranging search results for a query term 'war' in the 'movie' field is illustrated. In FIG. 2, the test collection may be referred to as a set of search results 202 and 203 arranged according to rankings for the query 201 and the query 201.

As already mentioned above, the search results may be provided with search results for the query term from the database 104. As shown in FIG. 2, the query word 201 may be at least one in the movie field such as 'beauty', 'Caribbean pirate', 'Harry Potter', and 'Superman'.

The test collection generation unit 101 may generate a test collection by using a search result for the query. At this time, the test collection generation unit 101 may generate a test collection for the query by sorting the ranking of the search results.

As shown in FIG. 2, the search result 203 for 'space war' is the first place, but the search result 202 for 'X-man-last war', which is the fourth place, is ranked first by sorting the ranking of the search results. can do. The criteria for sorting the ranking of search results may vary depending on the features of the search results. For example, in the case of a search result for 'movie', the features of the search result may include the latestness, the number of images, the rating, the number of participants, the number of famous words, the document length, and the like. Movie specialists or search planners understand the features of these search results better than developers of search models.

Thus, as an example, the test collection generation unit 101 may receive an opinion or command of an expert or search planner having knowledge and experience in the corresponding field to which the query belongs, and sort the search results according to the ranking through the user terminal.

3 is a diagram illustrating another example of a process of generating a test collection according to an embodiment of the present invention.

Specifically, FIG. 3 illustrates a process of sorting the search results for the query word 301. Referring to FIG. 3, a process of generating a test collection by arranging search results for a query term 'Harry Potter' in the 'movie' field is illustrated.

As can be seen in Figure 3, it can be seen that there are three search results arranged in the ranking first place. For example, the test collection generator 101 may sort the search results 302, 303, and 304 for the query word 301 in the same ranking when it is difficult to distinguish the ranking by the ranking or when there is little difference in features. For example, the case where it is difficult to distinguish the ranking may include a case in which a similar search frequency between search results is displayed or a series form. The criteria for aligning with the same ranking may vary depending on the configuration of the system.

4 is a diagram illustrating an example of selecting a feature for generating a search model according to an embodiment of the present invention.

In this case, the search model may mean a model that abstracts the process of searching for the most relevant information for a specific query. The search model generator 102 may generate a search model capable of determining the correct ranking according to the query from the test collection. That is, the search model generator 102 may generate a search model to determine whether the ranking of the sorted search results is the correct answer ranking. In this case, the search model generator 102 may select at least one feature to generate the search model through a machine learning method. For example,

The feature selection table 400 shown in FIG. 4 may consist of a feature name 401, a description of the feature 402, and a normalization method 403 for each feature. The feature selection table 400 may vary depending on the system. As can be seen in Figure 4, the feature has been selected the freshness, the number of images, the rating, the number of participants / reviews, the number of pronouns. In one example, the search model generator 102 may additionally select a normalization method for each feature to generate a search model.

The normalization method may include initialization or log normalization. That is, when the value of the feature has a small number of digits, the feature value may be used as it is. Conversely, if the value of the feature has a large number of digits, the feature value may be used through log normalization. The criteria for selecting a normalization method may vary depending on the configuration of the system.

5 is a diagram illustrating an example of an evaluation result of the performance of a search model according to an embodiment of the present invention.

Specifically, FIG. 5 shows a learning result table 500, evaluation data 505, and analysis graph 508. The training result table 500 may include a feature name 501, a description 502 for each feature, a normalization method 503, and an importance 504. The search model evaluator 103 may analyze the weight of each of the selected features with respect to the search result. Referring to FIG. 5, it may be said that the item of importance 504 corresponds to the analyzed weight in the learning result table 500.

That is, the search model evaluator 103 may evaluate whether or not a feature has been generated by sorting the ranking of search results based on the importance item. Referring to FIG. 5, the search model evaluator 103 may evaluate that the test collection is generated by sorting the ranking of search results based on similarity, freshness, and reliable rating.

In addition, the search model evaluator 103 may check the accuracy and correlation with respect to the search model generated through the evaluation data 505 in real time. Here, the accuracy may mean the accuracy of the query word and the generated search model. The correlation may mean a correlation between a query and a search model.

The analysis graph 508 also shows the relationship between the number of test collections for the query and the correlation. Referring to FIG. 5, it can be seen that the correlation increases as the number of test collections for the query increases. In other words, the more test collections are created, the higher the correlation between the query and the search model.

6 is a flowchart illustrating a speedy search modeling method according to an embodiment of the present invention.

The fast search modeling method according to an embodiment of the present invention may generate a test collection by using a search result for a query (S601). Generating a test collection (S601) may generate a test collection for the query by sorting the ranking of the search results. As mentioned earlier, a test collection is a set of rankings in which a particular query and the search results for that query are sorted.

In other words, the test collection may refer to a collection including a query and a correct ranking of search results for the query. Here, the correct ranking of the search results for the query may be generated during the initial sorting process, but may also be generated through an iterative reordering process.

At this time, the step of generating a test collection (S601) can be sorted in the same order, if the ranking of the search results can not be distinguished. That is, the step (S601) of generating a test collection may be sorted by the same rank when the ranking cannot be distinguished because it is ambiguous to rank the search results. In addition, a test collection may be generated for each of a plurality of queries in a specific field, and the number of test collections generated may be one or more.

For example, the step of generating a test collection (S601) may receive an opinion or command of an expert or search planner having knowledge and experience in a corresponding field to which a query word belongs, to sort the ranking of search results through a user terminal. The present invention can provide a faster search modeling method that can generate a more accurate search model by aligning the ranking of the search results for the query to the expert or search planner for the query.

The speed search modeling method according to an embodiment of the present invention may generate a search model capable of determining a correct ranking according to the query word from the test collection (S602).

At this time, in step S602 of generating a search model, a search model may be generated through a machine learning method. In one example, generating the search model (S602) generates a search model by using a machine learning method such as linear regression, classification and regression tree, logistic regression, ListRank, Bradley-Terry Model, Multi-Class Bradley-Terry Model, etc. can do.

In this case, the generating of the search model (S602) may generate a search model by selecting at least one feature and a normalization method for the feature with respect to the search result. In this case, the feature may mean data that is a reference when sorting the ranking of the search results. That is, in operation S602 of generating a search model, a search model may be generated through a machine learning method by referring to a feature that is a reference when an expert or a search planner sorts the ranking of search results.

In the speed search modeling method according to an embodiment of the present invention, the performance of the search model may be evaluated with respect to the generated search model (S603).

At this time, in step S603 of evaluating the performance of the search model, the weight of each of the selected features may be analyzed for the search result. That is, in the step of evaluating the performance of the search model (S603), when the test collection is generated by sorting the search results by analyzing the weights, the expert or the search planner may determine a feature that is mainly referred to.

In this case, in the step of evaluating the performance of the search model (S603), the accuracy and the correlation may be confirmed in real time with respect to the generated search model. That is, in the step of evaluating the performance of the search model (S603), the problem of the search model may be identified in a short time by evaluating the performance of the search model in real time.

In this case, in the step S601 of generating a test collection, when the performance of the search model does not meet a preset criterion, the generated test collection may be regenerated by rearranging the ranking of the search results. That is, according to an embodiment of the present invention, a search model for evaluating the performance of the search model through the step of generating a test collection (S601) and ensuring a stable performance by generating the test collection again based on the evaluation data. Can be generated.

Parts not described in FIG. 6 may refer to FIGS. 1 to 5.

In addition, the fast search modeling method according to an embodiment of the present invention includes a computer readable medium including program instructions for performing various computer-implemented operations. The computer readable medium may include program instructions, data files, data structures, etc. alone or in combination. The medium or program instructions may be those specially designed and constructed for the purposes of the present invention, or they may be of the kind well-known and available to those having skill in the computer software arts. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tape, optical media such as CD-ROMs, DVDs, and magnetic disks, such as floppy disks. Magneto-optical media, and hardware devices specifically configured to store and execute program instructions, such as ROM, RAM, flash memory, and the like. Examples of program instructions include not only machine code generated by a compiler, but also high-level language code that can be executed by a computer using an interpreter or the like.

As described above, the present invention has been described by way of limited embodiments and drawings, but the present invention is not limited to the above-described embodiments, which can be variously modified and modified by those skilled in the art to which the present invention pertains. Modifications are possible. Accordingly, the spirit of the present invention should be understood only by the claims set forth below, and all equivalent or equivalent modifications thereof will belong to the scope of the present invention.

1 is a block diagram showing the configuration of a fast search modeling system according to an embodiment of the present invention.

2 is a diagram illustrating an example of a process of generating a test collection according to an embodiment of the present invention.

3 is a diagram illustrating another example of a process of generating a test collection according to an embodiment of the present invention.

4 is a diagram illustrating an example of selecting a feature for generating a search model according to an embodiment of the present invention.

5 is a diagram illustrating an example of an evaluation result of the performance of a search model according to an embodiment of the present invention.

6 is a flowchart illustrating a speedy search modeling method according to an embodiment of the present invention.

<Explanation of symbols for the main parts of the drawings>

100: Fast Search Modeling System

101: test collection generator

102: search model generation unit

103: search model evaluation unit

104: database

105: search model

Claims (19)

A test collection generator configured to generate a test collection using a search result for a query; A search model generator configured to generate a search model capable of determining a correct answer ranking according to the query word from the test collection through a machine learning method; And Search model evaluation unit for evaluating the performance of the generated search model Speeding search modeling system comprising a. The method of claim 1, The test collection generation unit, Speeding search modeling system, characterized in that to sort the ranking of the search results to generate a test collection for the query. The method of claim 1, The test collection generation unit, If the ranking of the search results can not be distinguished, high-speed search modeling system, it characterized in that the sorting in the same rank. The method of claim 1, The test collection, And a query between the query word and the correct answer ranking of the search results for the query word. delete The method of claim 1, The search model generation unit, Speed search modeling system characterized in that for generating a search model by selecting at least one feature (feature) and the normalization method for the feature on the search results. The method of claim 6, The search model evaluation unit, Speed search modeling system characterized in that for analyzing the weight of each of the selected feature on the search results. The method of claim 1, The search model evaluation unit, Speed search modeling system characterized in that for checking the accuracy and the correlation in real time with respect to the generated search model. The method of claim 1, The test collection generation unit, And when the performance of the search model does not satisfy a predetermined criterion, regenerating the generated test collection by rearranging the ranking of the search results. Generating a test collection using the search results for the query word; Generating a search model capable of determining a correct ranking according to the query word from the test collection through a machine learning method; And Evaluating the performance of the search model with respect to the generated search model; Speed search modeling method comprising a. The method of claim 10, The step of creating a test collection, And sorting the ranking of the search results to generate a test collection for the query. The method of claim 10, The step of creating a test collection, If the ranking of the search results can not be distinguished, high-speed search modeling method characterized in that the sorting can be arranged in the same rank. The method of claim 10, The test collection, And a query between the query and the correct answer ranking of the search results for the query. delete The method of claim 10, The step of generating a search model, Speed search modeling method characterized in that for generating a search model by selecting at least one feature (feature) and the normalization method for the feature on the search results. The method of claim 15, The step of evaluating the performance of the search model, Speed search modeling method characterized in that for analyzing the weight of each selected feature on the search results. The method of claim 10, The step of evaluating the performance of the search model, Speed search modeling method characterized in that for verifying the accuracy and correlation in real time with respect to the generated search model. The method of claim 10, The step of creating a test collection, And if the performance of the search model does not satisfy a predetermined criterion, regenerating the generated test collection by rearranging the ranking of the search results. 19. A computer readable recording medium having recorded thereon a program for executing the method of any one of claims 10 to 13 or 15 to 18.

Images