KR100421938B1 - Method of measure for retrieval efficiency in computing system - Google Patents

Abstract

1. TECHNICAL FIELD OF THE INVENTION

The present invention relates to a method for evaluating search efficiency in a computing system and a computer readable recording medium having recorded thereon a program for realizing the same.

2. The technical problem to be solved by the invention

The present invention records an evaluation method for evaluating the superiority of the proposed search algorithm more precisely and accurately by objectively evaluating the search efficiency of a ranking system to which search results are ranked in a general computing environment, and a program for realizing the same. To provide a computer-readable recording medium.

3. Summary of Solution to Invention

The present invention relates to a method for evaluating the search efficiency of a search algorithm based on the order in which a search result of a material is searched in a database, and in which order irrelevant to a related item is searched, wherein whether the searched data within a measurement range is actually related. Receiving a search pattern indicating a first step; A second step of accumulating a distance according to a ranking of non-conformance data existing in the measurement range from the input search pattern; Accumulating the distances of all data existing within the measurement range; And dividing the cumulative distance according to the ranking of the non-conforming data existing in the measurement range by the cumulative distance of all the data existing in the measuring range, and normalizing the cumulative distance according to the ranking of the non-conforming data to output a normalized evaluation scale (NDS). Including the fourth step.

4. Important uses of the invention

The present invention is used to objectively evaluate the search efficiency of the ranking system to cover the superiority of the proposed search algorithm.

Description

Method of measure for retrieval efficiency in computing system

According to the present invention, an evaluation method for evaluating the search efficiency of a search algorithm and a program for realizing the search algorithm of a data in a database in a general computing environment, and in what order are irrelevant to what is related. The present invention relates to a recording medium which can be read by.

In order to measure the search efficiency of many conventional search algorithms, the search results should be evaluated with the search results. The present invention is a technique used to evaluate the search efficiency of a system that is ranked on these search results.

Some of the conventional methods may have to use two evaluation values. Since this is not a single-value measure, there is a problem that it is difficult to cover the superiority when one value is excellent when comparing the efficiency of the two search algorithms. In addition, the method of evaluating only the number of related data retrieved within a certain range is only a rough evaluation method. In other words, the same number of related data has been retrieved, but may be in the front rank or in the back.

In general, the well-known average rank is even more inaccurate, and other methods have also limited their ability to objectively quantify the efficiency, ie, the amount of effort the user sees, for the types found.

Therefore, a method for more precisely and accurately evaluating the superiority of the proposed search algorithm is required by objectively evaluating the search efficiency of the ranking system to which the ranking is given to the search results.

The present invention has been proposed in order to meet the above-described needs, and evaluates the superiority of the proposed search algorithm more precisely and accurately by objectively evaluating the search efficiency of a ranking system which is ranked in search results under a general computing environment. It is an object of the present invention to provide a computer-readable recording medium recording an evaluation method for performing the same and a program for realizing the method.

1 is an exemplary configuration diagram of a hardware system to which the present invention is applied.

2 is a flowchart illustrating an embodiment of a method for evaluating search efficiency according to the present invention;

3 is an exemplary view of an evaluation scale execution screen according to an embodiment of the present invention.

Figure 4 is an exemplary diagram of various distance functions used in the search efficiency evaluation method of the present invention.

FIG. 5 is a flow chart of an embodiment of an automatic search pattern generation process for all cases ( _n C _r ) where r related data are searched within n ranks in the search efficiency evaluation method of the present invention. FIG.

Figure 6 is an explanatory view showing one embodiment a search pattern generating sequence from the (C _{3 6)} If the three data relating to the search in the 6 rank evaluation method of the search efficiency of the present invention.

FIG. 7 is an exemplary diagram illustrating a recursive calling process of a search pattern for a case in which three related data are searched ( ₆ C ₃ ) within a sixth rank among the methods for evaluating the search efficiency of the present invention. FIG.

8 is an exemplary explanatory diagram illustrating a recursive calling process of a search pattern for all cases ( _n C _r ) where r related data are searched within n ranks in the search efficiency evaluation method of the present invention.

* Explanation of symbols for the main parts of the drawings

11: central processing unit 12: main memory unit

13: auxiliary memory device 14: input device

15: output device 16: peripheral device

In order to achieve the above object, the present invention provides a method for evaluating the search efficiency of a search algorithm having a search result of a material in a database and an order in which irrelevant ones are searched. A first step of receiving a search pattern indicating whether or not it is actually related; A second step of accumulating a distance according to a ranking of non-conformance data existing in the measurement range from the input search pattern; Accumulating the distances of all data existing within the measurement range; And dividing the cumulative distance according to the ranking of the non-conforming data existing in the measurement range by the cumulative distance of all the data existing in the measuring range, and normalizing the cumulative distance according to the ranking of the non-conforming data to output a normalized evaluation scale (NDS). It is characterized by comprising a fourth step.

In addition, the present invention is characterized in that it further comprises a fifth step of automatically generating all search patterns that can occur under a given condition in order to analyze the characteristic difference between the existing rating scale and the normalized rating scale.

In order to achieve the above object, the present invention provides a computing system with a processor for evaluating the search efficiency of a search algorithm with a search result of some material in a database and in what order the unrelated ones are searched. A first function of receiving a search pattern indicating whether the searched material within the range is actually related; A second function of accumulating a distance according to a ranking of non-conforming material existing in the measurement range from the input search pattern; A third function of accumulating distances of all data existing within the measurement range; And dividing the cumulative distance according to the ranking of the non-conforming data existing in the measurement range by the cumulative distance of all the data existing in the measuring range, and normalizing the cumulative distance according to the ranking of the non-conforming data to output a normalized evaluation scale (NDS). A computer readable recording medium having recorded thereon a program for realizing the fourth function is provided.

In addition, the present invention provides a computer readable recording program for further realizing a fifth function of automatically generating all search patterns that can occur under a given condition, in order to analyze the characteristic differences between the existing and normalized rating scales. Provide the medium.

The present invention relates to an evaluation method for evaluating the retrieval efficiency of a retrieval algorithm with a search result of a certain material in a database and in what order the irrelevant ones are retrieved. At this time, the searched data is composed of related or inappropriate data, and the ranking is assigned to them so that the efficiency of the search algorithm can be quantified from 0 to 1 and compared with other algorithms.

In the evaluation method of the present invention, an arbitrary search result may be input and calculated, and the evaluation may be performed by embedding an automatic search pattern generation process that automatically generates all types of search results (search patterns) that can occur under a given condition. The characteristic difference from the proposed evaluation scale was used for the analysis.

The above objects, features and advantages will become more apparent from the following detailed description taken in conjunction with the accompanying drawings. Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings.

1 is an exemplary configuration diagram of a hardware system to which the present invention is applied.

As shown in FIG. 1, the hardware system to which the present invention is applied includes a central processing unit 11, a main memory device 12 connected to the central processing device 11, and an auxiliary memory device connected to the main memory device 12. 13, the input / output devices 14 and 15 connected to the central processing unit 11 and the peripheral device 16 connected to the main memory device 12 are provided.

Here, the hardware system, the central processing unit 11 for controlling and managing the overall operation of the computer, the program stored in the central processing unit 11 and stores a variety of data used during or during operation And a main memory device 12 and an auxiliary memory device 13, an input / output device 14 and 15 for inputting / outputting data to and from a user, and a peripheral device 16 for a communication interface.

The auxiliary memory device 13 stores a large amount of data, and the input / output devices 14 and 15 include a general keyboard, a display device, and a printer.

However, since the computer hardware environment having the configuration as described above is only a technique well known in the art, detailed description thereof will be omitted herein. However, in the main memory device 12 of the hardware system, an evaluation scale program for objectively evaluating the search efficiency of the ranking system, which is ranked in the search results, to more precisely and accurately evaluate the superiority of the proposed search algorithm. It is stored under the control of the central processing unit (11).

Now, the evaluation scale program of the present invention will be described in more detail.

The evaluation scale program may be calculated by receiving an arbitrary search result. The evaluation scale program may be used for evaluation by embedding an automatic search pattern generation program that automatically generates all types of search results that can occur under a given condition.

The evaluation scale program evaluates the search efficiency based on the search results and quantifies it to a normalized value between 0 and 1. To implement this, the search results must be quantified by the effort required by the queryer.

In order to see as much relevant data as desired, the queryer must view the screen up to a certain rank, which includes not only relevant data but also nonconforming data. At this time, since the effort to view the related data is natural, the search efficiency of the system that can minimize the effort to view the non-conforming data is good.

The metrics should be as good as possible, as much data as possible, as little data as possible, as low as possible, and as high as possible. The more relevant data in the measurement range is retrieved, the less relevant data is retrieved, and the inquirer's effort to see a single non-conforming data becomes larger the earlier it is located. This is because the more nonconforming data is located in the front, the more relevant data is located in the back, because the viewing effort of the interrogator increases due to the nonconforming data.

The evaluation scale program of the present invention introduces the distance concept based on the ranking of the nonconformance data within the measurement range, sums and normalizes them, and quantizes them to values between 0 and 1. This can solve the problems caused by using the existing ranking directly. In this case, the distance is inversely proportional to the ranking, and by using the distance function, the distance function can be selected according to the search efficiency pursued by a specific database. In addition, an automatic search pattern generation program for automatically generating all search patterns that can occur under a given condition is used to analyze the characteristic difference between the existing evaluation scale and the proposed evaluation scale.

2 is a flowchart illustrating a method for evaluating search efficiency according to the present invention.

The present invention receives a search pattern from the user (201), evaluates it (202 to 204) and outputs the result as a numerical value (205). At this time, it is output along with the existing evaluation scale for comparison and reference. Here, the search pattern indicates whether or not the data searched within the measurement range is actually related. An evaluation scale execution screen is shown in FIG. 3.

Here, NDS (Normalized Distance Sum) means a normalized evaluation scale value.

The meaning of the distribution type of FIG. 3 is that when the measurement range reaches the 10th position, all 5 related data were searched in the 10th position, and the positions are 1, 2, 3, 4, 10th, respectively, and the non-conforming data is 5, It means that 5 searches were made in 6th, 7th, 8th, and 9th place. The search efficiency evaluation value at this time is shown in Fig. 3. The search pattern is described in more detail as follows. A user inputs data (keywords, pictures, etc.) into a query item of a search system and stores related or similar data in a database. When the search execution button is pressed to search within, the search system outputs the search results in order of similarity. In this case, the lower the similarity, the more likely it is to be an inaccurate result from the user's point of view. Therefore, the search result is regarded as a search result up to a certain high ranking. In order to measure the efficiency of a search result, the range that the search result ranks as a higher schedule is called a measurement range. Within this measurement range is actually composed of relevant data and irrelevant data that the search system finds incorrectly. For example, if the relevant data is marked as "O" and the irrelevant data is denoted as "X" and the measurement range is 5th, the search pattern of A system is "OO × XO" (searched as related data 1,2,5). The B system can be in various forms, such as " 0 " (refer to related data 1, 2, 3). In order to evaluate the search efficiency, such a search pattern is used. The search pattern can be input through the program to calculate the search efficiency through each evaluation scale. Accordingly, the search pattern input by the user refers to data input to the program as shown in FIG. 3 for calculating the search efficiency based on the search result as described above. 3 is an example in which the first, second, third, fourth, and tenth search patterns in which the relevant data is searched are searched when the search result is measured at the 25th position.

The present invention satisfies the following four conditions on one scale in measuring the search efficiency and is a normalized value of 0 to 1.

[Condition 1] It should be possible to search a lot of related data. [Condition 2] There should be a small amount of nonconforming data searched. [Condition 3] The related data should be ranked first.

Unlike other evaluation measures, the present invention uses distances, not the number or rank of relevant / nonconforming data. The distance to the ranking of the specific data retrieved is given in Table 1 below.

ranking One 2 3 ... i ... m-2 m-1 m ... d _i m m-1 m-2 ... m-i + 1 ... 3 2 One 0

Therefore, when the rank of any data retrieved within the measurement range m is i, the distance to this is obtained by the following formula (1).

Now, add together the distance (Equation 1) for nonconforming data in the range m order of measurement. At this time, the searched data after all relevant data is searched is not regarded as inappropriate data. That is, the range l added is the ranking of the last relevant data retrieved when all related data are searched. If the relevant data is only partially searched, the range added is up to m rank. Adding the distance to the nonconforming data within this range l is as shown in Equation 2 (B) below (202).

Now, in order to normalize Equation (2) (B), the distance to all the retrieved data is divided by the value (C) (B / C). The sum of the distances for all the data retrieved is shown in Equation 3 (C) below (203).

Accordingly, the normalized evaluation scale value (NDS) obtained by dividing [Equation 2] (B) by [Equation 3] (C) (B / C) may be represented by Equation 4 below (204). ).

Since Equation 4 is included in Equation 4, the distance change according to the rank decreases at the same rate as the rank increases as shown in Equation 1. This is a very common case, but there are times when you may want to measure a different rate of reduction. In other words, it is necessary to expand to use a distance function that decreases in a straight line rather than decreasing in a straight line. However, the reduced form should be a monotonic reduction form where the distance to the rear rank must be less than the distance to the front rank, whether straight or curved.

The distance function as shown in FIG. 4 is represented by a desired function and a trigonometric function. Since these values have 0 to 1 values, first, the rank of 1 to m is normalized to a value between 0 and 1. The regular expression is shown in [Equation 5] below.

[Equation 5] is for shaping an arbitrary rank i between 1 and 10 to 0 to 1 when the measurement range is set to 10th position (m = 10). In other words, substituting i = 1 in Equation 5 above Is 0, and is 0.1 when i = 2 and 0.9 when i = 10. Where normalized rank for rank i Normalized distance from Function you want to find Apply.

The distance function is the same as the following [Equation 6], the case corresponding to [Equation 1] is the inverse of r which is the first function of [Equation 6].

Now, the NDS extended to use these distance functions is calculated as shown in Equation 7 below. Where distance function When N is due to the inverse function of r, NDS becomes as shown in Equation 4 above.

r _L : Rank of last relevant material

m: the last rank in the measuring range

If the search pattern is r related data (O) within n ranks, the number of possible patterns is mathematically _n C _r , and the measured value is measured and graphed for these patterns. It is very helpful to understand the characteristics of the evaluation scale according to the range or the number of related images. As such, a process of automatically generating patterns is illustrated in FIG. 5.

The recursive call is the key to the automatic search pattern generation process for all cases ( _n C _r ) where r related data are retrieved within the n rank of FIG. 5. For example, if you draw a pattern for ₆ C ₃ it can be seen that there is a certain rule as shown in FIG.

In Figure 6 it can be seen that the form is repeated in a certain unit. In other words, when the first "O" is generated from rank 1, p1 to p10 corresponding to G1, and if generated from rank 2, p11 to p16 corresponding to G2, and from p3 to p19 and rank 4 of G3 from rank 3 From p20 on G4.

In other words, the 20 patterns by ₆ C ₃ are largely divided into four groups as shown in FIG. 6, and each of them repeats the occurrence pattern of the population. If G1 is further subdivided, the second " 0 " becomes G11, G12, G13, and G14 generated from ranks 2, 3, 4, and 5, respectively. In this segmentation, only the last "O" is generated, and the segmentation is no longer possible.

From this it can be seen that the algorithm by the recursive call (recursive call) is suitable. If the pattern of FIG. 6 is expressed as ₆ C ₃ G1 can be expressed as (1, ₅ C ₂ ). That is, since the first " 0 " is located in the rank 1, it means that two " 0 " In this way, G2, G3, and G4 are denoted as (2, ₄ C ₂ ), (3, ₃ C ₂ ), and (4, ₂ C ₂ ), respectively. In this case, since G1 may be further subdivided, for each of G11, G12, G13, and G14, (2, ₄ C ₁ ), (3, ₃ C ₁ ), (4, ₂ C ₁ ), and (5, ₁ C _1). Is indicated by). According to this notation, FIG. 6 may be written as shown in FIG. 7.

After subdividing to the end, the last "O" can be created at a specific location. For (2, ₄ C ₁ ), (3, ₃ C ₀ ), (4, ₂ C ₀ ), (5, ₁ C ₀ ), (hereinafter referred to as 6, it can be expressed also ₀ C _0), but because it is C ₀ = 1 _n without the use of any more symbol combination (3,1) (4,1) (5,1) (6,1) . This means that the last remaining one "O" is generated at positions 3 to 6, respectively.

7 may be found from a rule for segmentation, n can be r of "O" is generalized procedure is as shown in FIG. 8 to be given a number _n C _r of the pattern that can be broken down when coming within range. That is, a recursive call is made by moving to the right in order to further refine the (i, _n C _r) in the i value is called a first increment, n and r is 1 (i, _n C _r) and then decreased and liver granular group Moving down calls i (1, _n C _r ) after the first group of i increments by 1, n decreases by 1, and r locks. This repetition of calls to the right stops when r = 1 and the repetition of calls below stops when n = r. Here, (i, _n C _r ) notation means that " 0 " is generated at the i-th position, and then pattern generation for _n C _r is repeated for subsequent regions.

The algorithm of FIG. 8 is illustrated in FIG. 5, and the segmentation of FIG. 8 (the right and the subdivisions) is represented by a loop block by the right for statement selected when r of FIG. 5 is not 1. The content corresponding to the position of the last "O" which is the right part is shown in the for statement on the left, which is selected when r = 1 of FIG. 5. The method of the present invention as described above is implemented as a program and can be read by a computer. It may be stored in a recording medium (CD-ROM, RAM, ROM, floppy disk, hard disk, magneto-optical disk, etc.).

The present invention described above is capable of various substitutions, modifications, and changes without departing from the spirit of the present invention for those skilled in the art to which the present invention pertains, and the above-described embodiments and accompanying It is not limited to the drawing.

According to the present invention as described above, it is possible to objectively and accurately measure the search efficiency through the search results given the ranking, it is possible to compare the superiority of the search efficiency of the ranking system. In other words, unlike the case where the existing method does not cover the superiority or misevaluation, it is possible to evaluate precisely and accurately. Therefore, it is possible to quantify the search performance of many proposed search algorithms including the search engines on the Internet and compare the advantages. There is.

Claims (10)

In the method of evaluating the search efficiency of the search algorithm based on the order in which the search results of the data in the database and the related and unrelated things are searched, A first step of receiving a search pattern indicating whether the searched data within the measurement range is actually related; A second step of accumulating a distance according to a ranking of non-conformance data existing in the measurement range from the input search pattern; Accumulating the distances of all data existing within the measurement range; And Dividing the cumulative distance according to the ranking of the non-conforming data existing in the measurement range by the cumulative distance of all the data existing in the measuring range, normalizing the cumulative distance according to the ranking of the non-conforming data to output a normalized evaluation scale (NDS) 4th step Search efficiency evaluation method in a computing system comprising a. The method of claim 1, The fifth step of automatically generating all search patterns that can occur under a given condition in order to analyze the characteristic differences between the existing and normalized rating scales. Search efficiency evaluation method in a computing system further comprising. The method of claim 2, The fifth step, A method for evaluating search efficiency in a computing system, which automatically generates all search patterns that can occur under a given condition by a recursive call. The method according to any one of claims 1 to 3, The normalized rating scale (NDS) is, Substantially, the search efficiency evaluation method in a computing system, characterized in that the value quantified by a normalized numerical value between zero (0) to one (1) for evaluating the search efficiency of the search algorithm. The method according to any one of claims 1 to 3, The normalized rating scale (NDS) is, A method for evaluating search efficiency in a computing system, characterized in that the value is quantified by a normalized value between 0 and 1 for evaluating the search efficiency of a search algorithm by the following equation. ( , r _L : ranking of last relevant material, m: the last rank in the measurement range) The method according to any one of claims 1 to 3, The normalized rating scale (NDS) is, A method for evaluating search efficiency in a computing system, characterized in that the value is quantified by a normalized value between 0 and 1 for evaluating the search efficiency of a search algorithm by the following equation. , r _L : ranking of last relevant material, m: the last rank in the measurement range, d _ni is the normalized distance) The method of claim 4, wherein The process of normalizing the cumulative distance according to the ranking of the nonconforming material of the fourth step, A method for evaluating search efficiency in a computing system, characterized in that the rank (r) is normalized (r _ni ) to a value between zero (0) and one (1) by the following equation. (Where i is any rank and m is the measurement range) The method of claim 7, wherein The process of normalizing the cumulative distance according to the ranking of the nonconforming material of the fourth step, A method of evaluating search efficiency in a computing system, characterized in that the distance d from the rank r _ni normalized by the following equation is normalized (d _ni ) to a value between zero (0) and one (1). . In computing systems with processors, to evaluate the search efficiency of a search algorithm based on the order in which the search results of the data in the database and in what order are irrelevant and related, A first function of receiving a search pattern indicating whether or not the data searched within the measurement range is actually related; A second function of accumulating a distance according to a ranking of non-conforming material existing in the measurement range from the input search pattern; A third function of accumulating distances of all data existing within the measurement range; And Dividing the cumulative distance according to the ranking of the non-conforming data existing in the measurement range by the cumulative distance of all the data existing in the measuring range, normalizing the cumulative distance according to the ranking of the non-conforming data to output a normalized evaluation scale (NDS) 4th function A computer-readable recording medium having recorded thereon a program for realizing this. The method of claim 9, A fifth function for automatically generating all search patterns that can occur under a given condition in order to analyze the characteristic difference between the existing rating scale and the normalized rating scale A computer-readable recording medium that records a program for further realization.