JP6261669B2 - Query calibration system and method - Google Patents

Description

  The present invention relates to a query calibration system and method for providing a calibration result for a query input by a user, and more particularly to a query calibration system and method for providing a calibration result for a user query based on a translation model.

  A user can perform a search to obtain desired information through a site such as a search engine. The user can obtain desired information by inputting a query to the query input window of the search engine through the user's terminal and confirming the output search result.

  However, when a user inputs such a query through the terminal, a query including a typographical error that is different from the original intention is input due to an input error on the keyboard of the terminal and / or a selection error of the Korean-English conversion key. May occur. In particular, when the user terminal includes a touch screen, there is a possibility that a query including a typographical error may be further input due to an erroneous touch input or the like.

  When a query including a typographical error is input to the search engine, the intended search information intended by the user is not included in the output search result, but this also leads to deterioration in search quality.

  Therefore, even if the user enters a query that includes typographical errors, the entered query is converted to the intended orthographic query so that the user can get the expected information as the search result. Thus, there is a demand for a method for providing the information to the user.

  Patent Document 1 (publication date: January 25, 2011) discloses a system and method for calibrating a user query determined to be a typographical error query based on statistical data in units of entire queries or words. .

  The information described above is merely for helping understanding, and may include content that does not form part of the prior art, or may not include that the prior art can be presented to a normal engineer.

Korean Published Patent No. 10-2011-0007743

  In one embodiment, the optimal proofreading candidate is calculated without calculating the optimal proofreading candidate by calculating the probability of conversion of the user query to the proofreading candidate based on the translation model, without limiting the edit distance between the user query and the proofreading candidate. A user query proofing system and method are provided.

  In one aspect, a proofreading information extraction unit that extracts information on at least one proofreading candidate for the query based on log information of a search result for the input query, and the proofreading candidate for the query based on the extracted information A parameter acquisition unit for acquiring at least one parameter relating to the conversion to, and a probability associated with the conversion of the query to each of the at least one calibration candidate based on the acquired parameter, There is provided a query proofreading system including a proofreading result generation unit that extracts at least one proofreading candidate from among the proofreading candidates as a proofreading result for the query based on the obtained probability.

  The proofreading information extraction unit may determine whether the query includes a typographical error using the log information.

  The proofreading information extraction unit may identify at least one proofreading candidate for the query when the query includes a typographical error.

  The log information includes a time from when the first query is input by the user until the second query is input, user click information with respect to the search result, and information on the similarity between the first query and the second query. , And at least one of the search result attributes.

  The attribute of the search result may be a content category included in the search result.

  The parameter acquisition unit includes a probability that an element included in the query is converted into an element included in each of the calibration candidates, a numerical value indicating a positional relationship of the element included in the query with respect to each element of the calibration candidate, and the query At least one of the probabilities indicating the degree of naturalness of the conversion to each of the calibration candidates may be acquired as a parameter.

  The element included in the query may be a syllable included in the query, and the element included in each of the calibration candidates may be a syllable included in each of the calibration candidates.

  When the calibration result generation unit extracts calibration candidates from the at least one calibration candidate as the calibration result, the order in which the elements included in the query are arranged and the corresponding elements of the at least one calibration candidate are The order of arrangement may be assumed to be equal.

  The at least one proofreading candidate extracted as the proofreading result may be included in a search result for the input of the query by the user.

  The acquired parameter may be plural.

  The calibration result generation unit may calculate the probability based on a double of the acquired parameter or log addition.

  There may be a plurality of calibration candidates.

  The calibration result generator generates at least one calibration candidate among the plurality of calibration candidates as a calibration result for the query based on the probability distribution calculated for each of the plurality of calibration candidates. You may exclude from proofreading candidate extraction.

  The probability may be calculated by a mathematical formula.

  The formula is

であってもよい。

It may be.

  l is the length of each proofreading candidate, m is the length of each proofreading candidate, j is the index of the proofreading candidate, and i is the index of the query.

  TR is a function indicating the probability that the i-th element included in the query is converted to the j-th element included in each of the calibration candidates.

  AL is a function indicating a numerical value indicating the positional relationship between the elements included in the query and the elements of each calibration candidate.

_PLM is a function indicating the probability of the degree of naturalness of conversion of the query into each of the calibration candidates.

  At least one of the calibration information extraction unit and the parameter acquisition unit may be realized as a distributed processing system.

  The information related to the proofreading candidates may include a typographical error-correction pair composed of the query and each proofreading candidate.

  The probability may be a conditional probability that each calibration candidate occurs when the query is input.

  In another aspect, extracting information on at least one proofreading candidate for the query based on log information of a search result for the input query, to the proofreading candidate for the query based on the extracted information Obtaining at least one parameter relating to the transformation of the query, calculating a probability associated with the transformation of the query to each of the at least one calibration candidates based on the obtained parameter, and the calculated A query proofreading method is provided that includes extracting at least one proofreading candidate from the proofreading candidates as a proofreading result for the query based on the probability.

  Extracting information about the proofreading candidates may include determining whether the query includes typographical errors using the log information.

  Extracting information about the proofreading candidates may include identifying at least one proofreading candidate for the query if the query includes typographical errors.

  The step of obtaining the parameters includes: a probability that an element included in the query is converted into an element included in each of the calibration candidates; a numerical value indicating a positional relationship of the element included in the query with respect to the element of each calibration candidate; and At least one of the probabilities indicating the degree of naturalness of conversion of the query into each of the calibration candidates may be acquired as a parameter.

  The query proofreading method may further include outputting at least one proofreading candidate extracted as the proofreading result as a search result for the input of the query by a user.

  There may be a plurality of calibration candidates.

  The step of extracting the at least one calibration candidate is based on the probability distribution calculated for each of the plurality of calibration candidates, and at least one calibration candidate is selected from the plurality of calibration candidates. The method may include a step of excluding from the calibration candidate extraction as a calibration result for.

  By calculating the probability of conversion of user queries to proofreading candidates based on the translation model and extracting the optimal proofreading candidates, the user's query and proofreading candidate can be edited regardless of the editing distance of the user's query. An optimum calibration candidate that matches the intention can be provided as a calibration result.

  By calculating the probability of conversion of user queries to proofreading candidates based on the translation model and extracting the best proofreading candidates, it is possible to improve proofreading coverage without compromising the accuracy of proofreading for user queries. it can.

It is the figure which showed the operating method of the user query proofreading system in one Embodiment. 1 is a diagram illustrating a user query proofing system in one embodiment. FIG. It is the figure which showed notionally the acquisition method of the parameter regarding conversion to the proofreading candidate of a user query in one Embodiment. 7 is pseudo code illustrating a method for obtaining a band probability and an alignment probability as parameters relating to conversion of a user query into a proofreading candidate according to an embodiment. 7 is pseudo code illustrating a method for obtaining a band probability and an alignment probability as parameters relating to conversion of a user query into a proofreading candidate according to an embodiment. It is the figure which showed the performance of the parameter acquisition method by the algorithm of FIG. 4A and FIG. 4B. It is the figure which showed the acquisition method of the LM parameter as a parameter regarding conversion to the proofreading candidate of a user query in one Embodiment. It is the conceptual diagram which showed the method of extracting the proofreading candidate as a proofreading result with respect to a user query in one Embodiment. 7 is pseudo code illustrating a method for extracting a proofreading candidate as a proofreading result for a user query according to an embodiment. 7 is a flowchart illustrating an operation method of a user query proofreading system according to an embodiment. 6 is a flowchart illustrating a method for extracting information related to at least one proofreading candidate for a user query according to an exemplary embodiment. 6 is a flowchart illustrating a method of extracting a calibration candidate as a calibration result by excluding unnecessary calibration candidates from calibration candidates for a user query according to an embodiment.

  DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, practical embodiments of the present invention will be described in detail with reference to the accompanying drawings.

  FIG. 1 is a diagram illustrating a method of operating a user query calibration system according to an embodiment.

  A user query proofreading system 100 (hereinafter referred to as a query proofing system) shown in the figure processes a user query (hereinafter referred to as a query) input from a user terminal, and provides an appropriate proofreading result for the query. It is. For example, a user can input a query through a PC or a mobile terminal (eg, a mobile phone, a smartphone, a tablet PDA, etc.), and the query proofing system 100 determines whether there is a typographical error in the query input by the user. If it is determined that there is a typographical error, a proofreading candidate as a proofreading result for the corresponding query can be provided.

  The user query input through the terminal is a keyword or a character string input to a search engine or the like in order to request execution of work such as content search and / or inquiry or information acquisition, for example. Good. The query may be composed of at least one element, and each element constituting the query may be a word or a syllable.

  The proofreading system 100 determines whether or not the inputted query is a correct character based on the log information of the search information for the query inputted by the user in the past, and identifies a proofreading candidate (group) that can be a proofreading result of the corresponding query. Then, the optimal calibration candidate among the calibration candidates (group) can be provided as the calibration result.

  The proofreading candidate determined as the proofreading result may be provided as a search result for the user query together with or separately from the user query. Or the proofreading candidate decided as a proofreading result may be contained in the search result with respect to a user query. Furthermore, a search result for a proofreading candidate determined as a proofreading result may be provided as a search result for a user query.

  Calibration results may be provided in real time for user query inputs.

  A method of identifying a calibration candidate (group) for an input query and providing a calibration result will be described in more detail with reference to FIGS.

  FIG. 2 is a diagram illustrating a user query proofing system in one embodiment. The above-described query calibration system 100 will be described in more detail with reference to FIG. Query calibration system 100 may include a processor 210. The processor 210 may be configured to execute a program required to process a user query and provide a calibration result for the query, or to process related operations. The processor 210 may include a calibration information extraction unit 220, a calibration result generation unit 230, and a parameter acquisition unit 240. Each of the calibration information extraction unit 220, the calibration result generation unit 230, and the parameter acquisition unit 240 may be realized by a different hardware configuration within the processor 210 (or outside the processor 210 unlike the figure). Good. In FIG. 2, a single processor 210 is shown, but it may be a plurality of processors and may mean at least one core in the processor. In other words, at least some of the calibration information extraction unit 220, the calibration result generation unit 230, and the parameter acquisition unit 240 may be realized in a processor 210 different from the processor 210 or in a different hardware configuration.

  Or the structure which shows the function which the processor 210 performs may be sufficient as the calibration information extraction part 220, the calibration result production | generation part 230, and the parameter acquisition part 240. In other words, each of the calibration information extraction unit 220, the calibration result generation unit 230, and the parameter acquisition unit 240 may be configured as a software module.

  The calibration system 100 may further include a communication unit 250. The communication unit 250 can transmit and receive data and information from an external server or other terminals. For example, the communication unit 250 may be configured to receive a query from a user terminal, acquire log information of a search result, and output a calibration result for the user query and the user query.

  The proofreading information extraction unit 220 can extract information on at least one proofreading candidate for the query input from the user terminal based on the log information of the search result for the query input in advance. The proofreading information extraction unit 220 may correspond to an SVM-based error typographical error-correct character candidate detector (SVM based Erata- Correct Candidate Detector).

  The proofreading information extraction unit 220 determines whether the query includes a typographical error using the log information, and identifies at least one proofreading candidate for the query when the query includes a typographical error. it can.

  The information about the proofreading candidates may include a query and a typographical error-correction pair configured as each proofreading candidate. In other words, the identified query-each proofreading candidate may be a typographical-letter-correction pair, and the identified typographical-letter-correction pair may be stored and managed in a database (not shown). .

  The log information may be log information related to a query (group) input by a user (or the like) during a predetermined period and a search result (group) based on the corresponding query (group). The log information includes the time from when the first query is input by the user until the second query to be input next is input, the user click information for the search result for the query, the first query and the second query, May include at least one of information on the degree of similarity and an attribute of a search result.

  When the log information includes a time from when the first query is input by the user until the second query to be input next is input, for example, within a predetermined time after the first query is input When the second query is input to the proofreading information, the proofreading information extracting unit 220 may determine that the first query includes (or is likely to include) a typographical error. When the second query is input after a predetermined time after the first query is input, the proofreading information extraction unit 220 may determine the first query and the second query as separate queries.

  When the log information includes user click information for the search result for the query, for example, when there is a user click for the query search result, the proofreading information extraction unit 220 does not include the typographical error. (Or a possibility that it is not likely to be included) may be determined.

  The information related to the similarity between the first query and the second query included in the log information may be an edit distance between the first query and the second query. The edit distance may be a Levenshtein distance. When the second query is input after the first query is input by the user, when the edit distance between the first query and the second query is equal to or less than a predetermined value, the calibration information extraction unit 220 , It may be determined that the first query includes (or is likely to include) a typographical error. Conversely, when the edit distance between the first query and the second query exceeds a predetermined value, the proofreading information extraction unit 220 may determine the first query and the second query as different queries.

  The attribute of the search result included in the log information may be the type or category of content included in the search result. For example, the category may be any one of a web document, music, an image, a blog, news, and personal information. When the content included in the search result for the query is only the web document, the proofreading information extraction unit 220 may determine that the corresponding query includes (or is likely to include) a typographical error.

  In identifying the proofreading candidate for the query, by considering the log information as in the above-described embodiment, it is possible to more accurately determine whether the query includes a typographical error, and a more accurate proofreading candidate for the query ( Group).

  The calibration information extraction unit 220 may be realized using a distributed processing system, for example, a Hadoop-based distributed processing system. By realizing the distributed processing system as a base, the calibration information extraction unit 220 can process a huge amount of data at high speed.

  The parameter acquisition unit 240 can acquire at least one parameter related to conversion of an input query into a calibration candidate based on the information extracted by the calibration information extraction unit 220. The parameter may be used to calculate a probability associated with the conversion of the query to each of the proofreading candidates.

  The parameter acquisition unit 240 sets the probability that an element included in the query is converted into an element included in each proofreading candidate, a numerical value indicating the positional relationship of each element included in the query with respect to each proofreading candidate element, and each proofreading candidate of the query. At least one of the probabilities indicating the degree of naturalness of conversion may be acquired as a parameter. The element included in the query may be a syllable included in the query, and the element included in each of the calibration candidates may be a syllable included in each of the calibration candidates.

  For example, the parameter acquisition unit 240 acquires parameters necessary for calculating the probability associated with the conversion of the query into each of the proofreading candidates based on an algorithm using the IBM® MODEL 2 technique. The probability and alignment parameter acquisition unit 242 and the language model parameter acquisition unit 244 may be included.

  The probability and alignment parameter acquisition unit 242 is included in the probability and the query that, for example, an element included in the query is converted into an element included in each proofreading candidate using an Expectation-Maximization (EM) algorithm. You may respond | correspond to the IBM MODEL 2 parameter EM learner (IBM MODEL 2 Parameter EM Learner) which calculates the numerical value which shows the positional relationship with respect to the element of each calibration candidate of an element.

  The language model parameter acquisition unit 244 is, for example, an LM parameter learner (LM Parameter Learner) that calculates a language model (LM) parameter as a probability indicating the degree of naturalness of conversion to each query proofreading candidate. May be. The probability and alignment parameter acquisition unit 242 and the language model parameter acquisition unit 244 may be realized using a distributed processing system, for example, a Hadoop-based distributed processing system. By realizing the distributed processing system as a base, the parameter acquisition unit 240 can process an enormous amount of data at high speed.

  Detailed operations of the probability and alignment parameter acquisition unit 242 and the language model parameter acquisition unit 244 will be described in more detail with reference to FIGS.

  The calibration result generation unit 230 can calculate the probability associated with the conversion of the query into each of at least one calibration candidate based on the parameters acquired by the parameter acquisition unit 240. For example, the calibration result generation unit 230 may calculate the probability based on the double of the parameter acquired by the parameter acquisition unit 240 or log addition. The probability associated with the conversion of a query to a proofreading candidate may be a conditional probability that each proofreading candidate occurs when a query is entered. The calibration result generation unit 230 may calculate the probability that the query is converted into each of at least one calibration candidate using the following formula (1).

Here, l is the length of each proofreading candidate, m is the length of the query , j is the index of the element included in each proofreading candidate, and i is the index of the element included in the query. TR is a function indicating the probability that the i-th element included in the query is converted to the j-th element included in each of the calibration candidates. For example, TR may be a function for calculating a band probability. AL is a function indicating a numerical value indicating the positional relationship between the elements included in the query and the elements of each calibration candidate. For example, AL may be a function for calculating the alignment probability. _PLM is a function indicating a probability indicating the degree of naturalness of conversion of each query into proofreading candidates. For example, _PLM may be a function for calculating LM parameters.

  The calibration result generation unit 230 can extract at least one calibration candidate from among the calibration candidates as a calibration result for the query based on the probability calculated for each calibration candidate. The extracted proofreading candidate is an intended (optimum) proofreading candidate, and may be a correct character query for the user query. When the calibration result generation unit 230 extracts calibration candidates from at least one calibration candidate as a calibration result, the order in which the elements included in the query are arranged and the corresponding elements of the at least one calibration candidate are arranged. You may assume that the order is equal. In other words, a monotonic alignment may be assumed between the query and the proofreading candidate.

  In addition, the calibration result generation unit 230 extracts at least one calibration candidate from among the plurality of calibration candidates as a calibration candidate extraction result as a calibration result based on the probability distribution calculated for each of the plurality of calibration candidates. May be excluded.

  The calibration result generation unit 230 may correspond to, for example, an IBM MODEL 2 decoder that generates a calibration result for a query based on an algorithm that uses the IBM MODEL 2 technique.

  Detailed operation of the calibration result generation unit 230 will be described in more detail with reference to FIGS. 7 and 8.

  The description of the technical features described above with reference to FIG. 1 can be applied to FIG.

  FIG. 3 is a diagram conceptually illustrating a method for obtaining parameters relating to conversion of a user query into a proofreading candidate according to an embodiment.

  FIG. 3 is a diagram conceptually illustrating a method of calculating a numerical value indicating the positional relationship of the elements included in the query with respect to the elements of each calibration candidate described above with reference to FIG. In FIG. 3, it is assumed that the query input by the user is “New Balance Shinchon” and the proofreading candidate is “New Balance Shinchon”. An “element” indicates a syllable constituting a query and a proofreading candidate.

  The numerical value indicating the positional relationship of the elements included in the query with respect to the elements of each calibration candidate may mean the probability that each element of the query corresponds to the position of each element of the calibration candidate. It may mean a value corresponding to an arrow. In other words, the value corresponding to each arrow may be a value (alignment probability) determined based on the above-described AL function.

  As shown in the figure, the probability and alignment parameter acquisition unit 242 performs the iteration process using the EM algorithm on the distributed system, thereby determining the positional relationship of the elements included in the query with respect to each calibration candidate element. The indicated numerical value may be calculated.

  The description of the technical features described above with reference to FIG. 1 and FIG. 2 can be applied to FIG.

  4A and 4B are diagrams illustrating pseudo code indicating a method for obtaining a band probability and an alignment probability as parameters relating to conversion of a user query into a proofreading candidate according to an embodiment. The algorithm shown in the figure may be executed by the probability and alignment parameter acquisition unit 242.

  The probability and alignment parameter acquisition unit 242 may be configured by a mapper and a reducer.

  The operation of the mapper may be shown by the following algorithm (see FIG. 4A).

Here, for the score calculated using the syllable trigram, Esti_prob as an output (k, i, j) _is, TR | may be used to calculate the (Errata i Correction _j). Also, the output esti_prob (k, i, j) value for score calculated using the Len function may be used to calculate AL (j | i, l, m) dmf. On the other hand, the esti_prob value may be an output of the reducer in the t-1 stage. The initial esti_prob value may be 1 / Len (input). In the calculation based on the above-mentioned algorithm, data stored as a perfect hash of a disk memory mapped I / O method is loaded by each of the hardoop nodes constituting the distributed system. May be done through.

  As in the output data shown in FIG. 4, the reducer may distinguish the factors necessary for calculating AL and TR into “C” and “M”, respectively (1st key). Of the output data, the information (syllable, index, length of character string) that must be the denominator may be divided, and whether it is a denominator or numerator is classified by "0", "1" (2nd key, 3rd key). Furthermore, information (syllables, indexes) that become molecules may also be classified. Of the output data, the last value of the field may be score.

  The result that results in a sum after the mapper outputs are aligned may be represented by the following algorithm that shows the operation of the reducer (see FIG. 4B).

  TR (band probability) and AL (alignment probability) corresponding to the parameters described above with reference to FIGS. 2 and 3 may be calculated by the algorithm. In addition, the calculated esti_prob (k, i, j) may be used to update the score value in the t + 1 stage, or may be stored and managed in a disk or database in a complete hash structure.

  The description of the technical features described above with reference to FIGS. 1 to 3 can be applied as it is here, and therefore, the redundant description is omitted.

  FIG. 5 is a diagram showing the performance of the parameter acquisition method based on the algorithm of FIGS. 4A and 4B.

  In FIG. 5, 24 mappers and 20 reducers were used, for a total of 15 nodes (2.2 GHz, 48 GRAM). About 37 million typographical error-correction pair data was used as input, and about 257 million model parameters were obtained as output.

  A closer look at the results confirmed that it took 11 to 14 minutes per EM stage (a total of 9 to 10 iterations were performed).

  FIG. 6 is a diagram illustrating a method for acquiring an LM parameter as a parameter related to conversion of a user query into a proofreading candidate according to an embodiment. The algorithm shown in FIG. 6 shows a method of calculating a probability (LM parameter) indicating the degree of naturalness of conversion of each of the queries described above with reference to FIG. The algorithm shown in the figure may be executed by the language model parameter acquisition unit 244. The LM parameter indicates the degree of context and / or the degree of naturalness required for a query to be converted into a proofreading candidate.

  The language model parameter acquisition unit 244 may acquire the LM parameter by performing a smoothing process and a linear interpolation process. However, the smoothing process may be performed only when the number of typographical-letter-letter pairs that must be processed is very large (eg, 1 billion or more).

  The language model parameter acquisition unit 242 may be configured with a mapper and a reducer. In obtaining the LM parameter, the denominator local SUM (local sum) calculation may be executed by, for example, generating information that becomes a denominator in the reducer key (reducer key) using the First-Priority-Key.

  In obtaining the LM parameters, statistical learning in a distributed environment may be performed using Spark.

  In the algorithm shown in the figure, the input query and its frequency were assumed to be “Akasana Hamaya”, “99”, “Akasana Hamaga”, and “1”, respectively. The estimated 10 gram probability may be calculated as the LM parameter, and the 10 gram probability may be estimated by interpolation after calculating the 8 gram probability and the 9 gram probability.

  In stage 1, the mapper may extract 8-10 grams, and the reducer may calculate the frequency sum. The step 1 may be, for example, a preprocessing process for calculating P (d | abc) for the character string “abcd”. The calculation of the LM parameter may be performed using a key structure (N = 8-10) in the form of “min (N) gram”, “N-1gram”, “Ngram”, and therefore the denominator preferred frequency sum. Can be easily executed.

  In step 2, using the result of step 1 (map = 'cat' & sort), the reducer may calculate a probability value of 8 to 10 gram. Here, the denominator of 8 gram may be assigned to sum (cnt_of_all (8 gram)).

In step 3, 10 gram probabilities may be linearly interpolated using the 8-10 gram probability values calculated in step 2. The mapper generates the same key as step 1, converts the first key (1st key) in reverse order of the string, and groups 8 to 10 grams into groups and reducers based on the “final” syllable. ). The reducer may generate final LM parameters by performing linear interpolation. For example, for the character string “Aiue”, the LM parameter P _LM (Ta | Sakaa) is a × P _LM (Ta | Sakaa), b × P _LM (Ta | Sakaa), and c × P _LM ( It may be calculated by the sum of | Here, the weight values a, b, and c may be calculated based on the method proposed in “A statistical Part-of-Speech Tagger, T. Brant et al, 2000”. The unsen probability may be calculated by 1 / sum (cnt_of_all (10gram) + cnt_of_dic (10gram)).

  Table 1 below shows the performance of the LM parameter acquisition method according to the algorithm shown in the figure. For the results in Table 1, 24 mappers and 20 reducers were used, for a total of 15 nodes (2.2 GHz, 48 GRAM).

  The description of the technical features described above with reference to FIGS. 1 to 5 can be applied here as it is, and thus the redundant description is omitted.

  FIG. 7 is a conceptual diagram illustrating a method for extracting a proofreading candidate as a proofreading result for a user query according to an embodiment. FIG. 7 shows a method of extracting an optimum calibration candidate for a query of a calibration result generation unit 230 corresponding to an IBM MODEL 2 decoder that generates a calibration result for a query based on an algorithm using the IBM MODEL 2 technique, for example. Yes.

  The optimal calibration candidate may be determined by the following mathematical formula (2).

  In other words, among the plurality of calibration candidates for the query, the calibration candidate having the maximum probability calculated based on the above formula (1) may be determined and extracted as the optimal calibration candidate.

  Monotonic alignment may be assumed between the query and the proofreading candidate. In addition, the optimal calibration candidate may be determined using a dynamic algorithm. Further, based on the probability distribution calculated for each of a plurality of calibration candidates, at least one calibration candidate among the plurality of calibration candidates may be excluded from the calibration candidate extraction as a calibration result for the query. For example, calibration candidates corresponding to intermediate probability values indicated as point values in the probability distribution may be excluded from the calibration candidate extraction process as unnecessary calibration candidates. According to the above process, even when the number of calibration candidates is large, an optimal calibration candidate can be determined at high speed.

  In step 1, the calibration result generator 230 may generate information for determining an optimal calibration candidate using the calculated TR and AL parameters for possible calibration candidates for the query.

  In step 2, the calibration result generation unit 230 may determine an optimal calibration candidate by decoding the query in syllable units. In step 2, unnecessary calibration candidates may be excluded based on, for example, a score calculated by at least one of a band probability, an alignment probability, and an LM parameter. In the example shown in the figure, “New Balance Shinchon” is determined as the optimal calibration candidate for the query “New Balance Cincheng”.

  The calibration result generation unit 230 may be realized at a speed of 1000 to 1500 TPS / core. Steps 1 and 2 will be described in more detail with reference to the algorithm of FIG. The description of the technical features described above with reference to FIGS. 1 to 6 can be applied as it is here, and therefore, the redundant description is omitted.

  FIG. 8 is a pseudo code showing a method of extracting a proofreading candidate as a proofreading result for a user query according to an embodiment. In FIG. 8, steps 1 and 2 for extracting the optimum calibration candidate described above will be described in more detail with reference to FIG.

  In step 1, the calibration result generator 230 may generate information for determining an optimal calibration candidate using the calculated TR and AL parameters for possible calibration candidates for the query. The query may be tokenized on a syllable basis and all possible AL and TR parameters may be extracted for the query based on a finite state transducer. The extracted AL and TR parameters may be stored (such as in a database or memory) and mapped to the query's syllable index.

  In step 2, the calibration result generation unit 230 may determine an optimal calibration candidate from among the calibration candidates. The output in stage 1 (input query and its AL and TR parameters) may be the input in stage 2. The score for the input calibration candidate may be calculated through the following algorithm, and one calibration candidate may be determined as the optimal calibration candidate based on the score.

  An unnecessary calibration candidate may be excluded according to the score calculated based on the band probability, the alignment probability, and the LM parameter, and the optimal calibration candidate may be determined from the calibration candidates according to the calculated score. The determined optimum proofreading candidate may correspond to the proofreading candidate extracted as the proofreading result described above, or may be provided as a search result for the user query.

  The description of the technical features described above with reference to FIGS. 1 to 7 can be applied here as it is, and therefore, redundant description is omitted.

  FIG. 9 is a flowchart illustrating a method of operating the user query proofing system in one embodiment.

  In step 910, the proofreading information extraction unit 220 may extract information on at least one proofreading candidate for the user query based on log information of a search result for the query input in advance. The extracted information about the proofreading candidates may include a typographical error-correction pair composed of the query and the proofreading candidates.

  In step 920, the parameter acquisition unit 240 may acquire at least one parameter related to conversion of the query into a proofreading candidate based on the information extracted in step 910. The bandwidth probability parameter and the alignment probability parameter for the conversion of the query into the calibration candidate may be calculated and acquired by the probability and alignment parameter acquisition unit 242 of the parameter acquisition unit 240. The LM parameter for the conversion of the query into the proofreading candidate may be calculated and acquired by the language model parameter acquisition unit 244 of the parameter acquisition unit 240.

  In operation 930, the calibration result generator 230 may calculate a probability associated with the conversion of the query into each candidate calibration group (group) based on the parameters acquired in operation 920. For example, the calibration result generation unit 230 may perform the calculation of the probability using the above-described mathematical formula (1).

  In operation 940, the calibration result generator 230 may extract at least one calibration candidate from the calibration candidates (group) as a calibration result for the query based on the probability calculated in operation 930. For example, the calibration result generation unit 230 may extract one calibration candidate as a calibration result from the calibration candidates (group) using the above-described formula (2).

  In step 950, the communication unit 250 of the query calibration system 100 may output the calibration candidate as the calibration result extracted in step 940 as a search result for the user's query input. The search result for the user's query input may include the extracted proofreading result.

  The description of the technical features described above with reference to FIG. 1 to FIG. 8 can be applied as it is here, and thus the redundant description is omitted.

  FIG. 10 is a flow diagram illustrating a method for extracting information about at least one proofreading candidate for a user query in one embodiment.

  Steps 1010 and 1020 described below may be included in step 910 described above with reference to FIG.

  In step 1010, the proofreading information extraction unit 220 may determine whether the query includes a typographical error using the log information of the search result. If the query does not include a typographical error, it is determined that the corresponding query corresponds to the correct character query, and therefore it is not necessary to provide a separate proofreading result.

  If the query includes a typographical error, in step 1020, the proofreading information extracting unit 220 may identify at least one proofreading candidate for the query. For example, the proofreading information extraction unit 220 may identify at least one typographical error-correction pair for the query-proofreading candidate as information on the proofreading candidate by identifying the proofreading candidate.

  The description of the technical features described above with reference to FIGS. 1 to 9 can be applied as it is here, and therefore, the redundant description is omitted.

  FIG. 11 is a flowchart illustrating a method for extracting a calibration candidate as a calibration result by excluding unnecessary calibration candidates from calibration candidates for a user query according to an embodiment.

  Steps 1110 and 1120 described below may be included in step 940 described above with reference to FIG.

  In step 1110, the calibration result generation unit 230 selects at least one calibration candidate among the plurality of calibration candidates as a calibration result for the query based on the probability distribution calculated in step 930 for each of the plurality of calibration candidates. Can be excluded from the proofreading candidate extraction process.

  In step 1120, the calibration result generator 230 may extract at least one calibration candidate as a calibration result for the query from among a plurality of calibration candidates from which unnecessary calibration candidates are excluded in step 1110.

  The description of the technical features described above with reference to FIGS. 1 to 10 can be applied here as it is, and therefore, redundant description is omitted.

  Below, the effect of this embodiment compared with a prior art and the technical improvement matter are demonstrated.

  In the evaluation of effects and technical improvements, the data used for learning IBM MODEL 2 was indicated by IBM_SET-7 and IBM_SET-21, respectively. IBM_SET-7 is typographical error-proofreading candidate data for a 7-day query, and IBM_SET-21 is typographical-proofreading candidate data for a 21-day query. The data used for learning the LM parameter is indicated by LM_SET-21_Q5 and LM_SET-2015_Q30, respectively. LM_SET-21_Q5 indicates data for a query with a cumulative frequency of 5 or more for a 21-day query, and LM_SET-2015_Q30 indicates data for a query with a cumulative frequency of 30 or more for the entire query up to 2015. Show. As a test set, 21,219 typographical error-> correction data were constructed by unsen random sampling. As a conventional technique for comparison with the embodiment, it is assumed that the latest learning data is used.

  Tables 2 and 3 below show whether the coverage and quality are improved when the IBM MODEL 2 learning data and LM learning data are increased. In Table 2, the use of LM_SET-2015_Q30 was fixed. In Table 3, the use of IBM_SET-21 was fixed.

  In all cases of Tables 2 and 3, increased coverage and increased accuracy were observed. Table 4 below shows the results of SEEN TEST of the prior art (AS-IS) and the embodiment (TO-BE). In order to compare the modeling power (performance) of the prior art system and the system of the embodiment, the data already used for learning (SEEN) was used for the evaluation.

  As described above, it was confirmed that the embodiment is further superior to the prior art in terms of coverage and accuracy.

  Table 5 below shows the results of UNSEEN TEST of the prior art (AS-IS) and the embodiment (TO-BE). In order to compare the modeling power (performance) of the prior art system and the embodiment system, UNSEEN data was used for the evaluation.

  As described above, in the case of the embodiment, it was confirmed that the coverage was increased by about 222% compared to the prior art, but the accuracy decrease was only 2%.

  Tables 6 and 7 below show the comparison results of the prior art and the embodiment for how much a query that is not a typographical error is recognized as an orthographic query. In Table 6, the evaluation data 1 is 5,000 data items of the shopping domain QC top regarded as a normal character. In Table 7, evaluation data 2 is 17,040 data of UNSEEN & low QC map restaurant name data regarded as normal characters.

  As described above, it has been confirmed that the positive character recognition rate of the embodiment is higher than that of the prior art.

  Table 8 below shows a comparison result between the prior art (AS-IS) and the embodiment (TO-BE) of typographical error correction.

  As described above, in the calibration coverage, it was confirmed that the embodiment is superior to the conventional technique.

  For example, the system of the embodiment has an effect of improving the calibration coverage by 2 to 3 times compared to the system of the prior art, but it has been confirmed that there is almost no or very little deterioration of the calibration accuracy.

  In addition, according to the system of the embodiment, since the proofreading coverage is wider than that of the prior art, when the user inputs a typographical lexical query, the frequency of having to re-enter the typographical query is reduced. There is an effect that it is possible to reduce the amount of data processing and calculation required for query input (client perspective) and search result processing (search server perspective) for outputting results. Furthermore, when the user's terminal is a mobile terminal, if a typographical error query is input, the frequency of having to re-input the normal character query is reduced, so that the battery saving effect of the terminal can also be achieved.

  The apparatus described above may be realized by hardware components, software components, and / or a combination of hardware and software components. For example, the devices and components described in the embodiments include, for example, a processor, a controller, an ALU (arithmic logic unit), a digital signal processor, a microcomputer, an FPGA (field programmable gate array), a PLU (programmable logic unit), a micro It may be implemented using one or more general purpose or special purpose computers, such as a processor or various devices that can execute and respond to instructions. The processing device may execute an operating system (OS) and one or more software applications running on the OS. The processing device may also respond to software execution, access data, and store, manipulate, process and generate data. For convenience of understanding, a single processing device may be described as being used, but those skilled in the art will recognize that the processing device includes multiple processing elements and / or multiple types of processing elements. But you can understand. For example, the processing device may include a plurality of processors or a processor and a controller. Also, other processing configurations such as a parallel processor are possible.

  The software may include computer programs, code, instructions, or a combination of one or more of these, configuring the processor to operate as desired, or instructing the processor independently or collectively. You may do it. Software and / or data can be interpreted based on a processing device or provide instructions or data to a processing device, any type of machine, component, physical device, virtual equipment, computer storage medium Alternatively, it may be permanently or temporarily embodied in a device, or a transmitted signal wave. The software may be distributed over computer systems connected by a network and stored or executed in a distributed manner. Software and data may be stored on one or more computer-readable recording media.

  The method according to the embodiment may be realized in the form of program instructions executable by various computer means and recorded on a computer-readable medium. The computer readable medium may include program instructions, data files, data structures, etc. alone or in combination. The program instructions recorded on the medium may be specially designed and configured for the embodiment or may be usable by those skilled in the art of computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical media such as CD-ROMs and DVDs, and magneto-optical media such as floppy disks. And hardware devices specially configured to store and execute program instructions such as ROM, RAM, flash memory, and the like. Examples of program instructions include not only machine language code such as that generated by a compiler, but also high-level language code that is executed by a computer using an interpreter or the like. The hardware device described above may be configured to operate as one or more software modules to perform the operations of the embodiments, and vice versa.

  As described above, the embodiment has been described based on the limited embodiment and the drawings, but those skilled in the art can make various modifications and variations from the above description. For example, the described techniques may be performed in a different order than the described method and / or components of the described system, structure, apparatus, circuit, etc. may be in a different form than the described method. Appropriate results can be achieved even when combined or combined, or opposed or replaced by other components or equivalents.

  Accordingly, even different embodiments belong to the appended claims as long as they are equivalent to the claims.

100: Query calibration system 210: Processor 220: Calibration information extraction unit 230: Calibration result generation unit 240: Parameter acquisition unit 250: Communication unit

Claims (19)

A proofreading information extraction unit that extracts information on at least one proofreading candidate for the query based on log information related to one or more queries input in advance ;
Based on the extracted information, the parameter acquisition unit for acquiring Rupa parameter relates to a conversion to the calibration candidate of the query, and based on the obtained parameters, the at least one calibration candidate of the query the probability associated with the conversion to respectively calculate, based on the calculated probabilities, Bei example calibration result generation unit for extracting at least one calibration candidate from among the calibration candidate as a calibration result for said query,
The parameter acquisition unit includes a probability that an element included in the query is converted into an element included in each of the calibration candidates, a positional relationship of the elements included in the query with respect to the elements of each calibration candidate, and the calibration candidates of the query A query proofreading system that acquires the degree of naturalness of conversion to each as a parameter .   The proofreading information extraction unit determines whether the query includes a typographical error using the log information, and if the query includes a typographical error, identifies at least one proofreading candidate for the query. Query calibration system as described in. The log information includes a time from when the first query is input by the user until the second query is input, user click information with respect to a search result for the query, and similarity between the first query and the second query. The query proofreading system according to claim 1, wherein the query proofreading system includes at least one of information on an attribute and an attribute of the search result.   The query proofreading system according to claim 3, wherein the attribute of the search result is a category of content included in the search result. Elements included in the query is a syllable included in the query, elements included in each of the calibration candidate syllables contained in each of the calibration candidate, in any one of claims 1 to 4 The query calibration system described. When the calibration result generation unit extracts a calibration candidate from the at least one calibration candidate as the calibration result, the order in which the elements included in the query are arranged and the corresponding element of the at least one calibration candidate There is assumed the order arranged the same as the query calibration system according to any one of claims 1 to 5. The query proofing system according to any one of claims 1 to 6 , wherein at least one proofreading candidate extracted as the proofreading result is included in a search result for an input of the query by a user. The proofreading candidates are plural,
The calibration result generation unit
Excluding at least one of the plurality of calibration candidates from the calibration candidate extraction as a calibration result for the query based on the probability distribution calculated for each of the plurality of calibration candidates; The query proofreading system according to any one of claims 1 to 7 . The probability associated with the transformation is calculated by a mathematical formula;
The formula is

And
l is the length of each proofreading candidate, m is the length of the query , j is the index of an element included in each of the proofreading candidates, i is the index of an element included in the query,
TR is a function indicating the probability that the i-th element included in the query is converted to the j-th element included in each of the calibration candidates, and AL is an element of each calibration candidate of the elements included in the query is a function indicating a number indicating the positional relationship, P _LM is the function indicating the probability of the degree of naturalness of conversion to the calibration candidates each of the query, any one of the claims 1 to 8 Query calibration system as described in. The query proofreading system according to any one of claims 1 to 9 , wherein at least one of the proofreading information extraction unit and the parameter acquisition unit is realized as a distributed processing system. The query proofreading system according to any one of claims 1 to 10 , wherein the information related to the proofreading candidate includes a typographical error-corrected character pair configured by the query and each proofreading candidate. 12. The query calibration system according to any one of claims 1 to 11 , wherein the probability associated with the transformation is a conditional probability that each calibration candidate occurs when the query is input. Extracting information about at least one proofreading candidate for the query based on log information associated with the one or more pre- entered queries;
The computer, on the basis of the extracted information, the step of acquiring the related Rupa parameters for the conversion to the calibration candidate of the query,
Stage the computer is that based on the obtained parameters, the computation of a probability associated with the conversion to each of the at least one calibration candidate of the query, and
The computer, on the basis of said calculated probability, viewed including the step of extracting at least one calibration candidate from among the calibration candidate as a calibration result for said query,
Obtaining the parameter comprises:
To the probability that an element included in the query is converted into an element included in each of the calibration candidates, the positional relationship of the element included in the query with respect to the element of each calibration candidate, and each of the calibration candidates of the query Query calibration method that obtains the degree of naturalness of transformation as a parameter . The step of extracting information about the proofreading candidates includes
The computer uses the log information to determine whether the query includes typos; and
The query proofreading method according to claim 13 , wherein the computer includes identifying at least one proofreading candidate for the query if the query includes a typographical error. Said computer, at least one calibration candidate, further comprising the step of outputting as a search result to the input of the query by a user, the query calibration method of claim 13 or 14, which is extracted as the calibration results. The proofreading candidates are plural,
Extracting the at least one proofreading candidate comprises:
The computer, on the basis of the distribution of the probability calculated for each of the plurality of calibration candidates for the calibration candidate extraction of at least one calibration candidate among the plurality of calibration candidates for the, as a calibration result for said query The query proofreading method according to any one of claims 13 to 15 , including a step of excluding from the query. The probability associated with the transformation is calculated by a mathematical formula;
The formula is

And
l is the length of each proofreading candidate, m is the length of the query , j is the index of an element included in each of the proofreading candidates, i is the index of an element included in the query ,
TR is a function indicating the probability that the i-th element included in the query is converted to the j-th element included in each of the calibration candidates, and AL is an element of each calibration candidate of the elements included in the query is a function indicating a number indicating the positional relationship, P _LM is the function indicating the probability of the degree of naturalness of conversion to the calibration candidates each of the query, any one of claims 13 to 16 Query calibration method described in. A computer program that causes a computer to execute the query proofreading method according to any one of claims 13 to 17 . A computer-readable recording medium storing the computer program according to claim 18 .

Images