KR20230152629A - Method, system, and computer readable record medium for generating reformulated query - Google Patents

Abstract

A method, system, and computer-readable recording medium for generating a reconstructed query are disclosed. The query generation method includes generating a first query candidate through query reformulation using a BERT (Bidirectional Encoder Representations from Transformer)-based machine translation model for the original query; and selecting at least some of the first query candidates as a search query.

Description

Method, system, and computer readable recording medium for generating a reformulated query {METHOD, SYSTEM, AND COMPUTER READABLE RECORD MEDIUM FOR GENERATING REFORMULATED QUERY}

The explanation below is about query reformulation (referred to as 'QR') technology to provide optimal search results.

A user may perform a search to obtain desired information through a site such as a search engine. The user can obtain the desired information by entering a query in the query input window of the search engine through the user terminal and checking the search results output for the query.

In query-term matching-based search systems, query reformulation technology is used to provide optimal search results by adding reformulated queries with similar meaning to the user's query.

For example, Korea Patent Publication No. 10-2011-0007743 (published on January 25, 2011) provides a technology that can correct user queries judged to be typos based on statistical data according to the entire query unit or word unit. This is disclosed.

We provide a method and system for generating QR candidates for user queries by additionally using the BERT (Bidirectional Encoder Representations from Transformer) model along with the SMT (statistical machine translation) model.

In a query generation method executed on a computer device, the computer device includes at least one processor configured to execute computer-readable instructions included in a memory, and the query generation method includes, by the at least one processor, an original A step of generating query candidates through query reformulation using an artificial intelligence model for the query; and selecting at least some of the query candidates as a search query by the at least one processor, wherein the step of generating the query candidates refers to a cache in which the inferred query candidates are stored. finding the query candidate corresponding to the original query in the cache; When there is no query candidate corresponding to the original query in the cache, a query comprising performing inference on the original query using the artificial intelligence model and inserting the inference result into the cache Provides a creation method.

According to one aspect, the generating step may generate the query candidate through the artificial intelligence model, which is a machine translation model that replaces the encoder part of the transformer model with the BERT model.

According to another aspect, the number of serving devices for processing queries in real time may be calculated based on at least one of the maximum number of inferences per unit time and the maximum unique query per second (UQPS).

According to another aspect, the generating step generates the query candidate through the artificial intelligence model learned through a pre-training technique using a masked language model (MLM) and next sentence prediction (NSP). can do.

According to another aspect, the query generation method further includes generating, by the at least one processor, another query candidate through query reconstruction using a statistical machine translation model for the original query. In the selection step, some of the query candidates and the other query candidates may be selected as the search query based on the prediction probability through each model.

Provided is a computer-readable recording medium on which a program for executing the query generation method on a computer is recorded.

A computer device comprising: at least one processor configured to execute computer-readable instructions included in a memory, wherein the at least one processor generates a query candidate through query reconstruction using an artificial intelligence model for an original query. procedure; And processing a process of selecting at least some of the query candidates as a search query, wherein the at least one processor refers to a cache in which the inferred query candidates are stored and selects the query candidates corresponding to the original query from the cache. and, if there is no query candidate corresponding to the original query in the cache, perform inference using the artificial intelligence model for the original query and insert the inference result into the cache. Provides computer devices that

According to embodiments of the present invention, QR quality and search quality can be improved by generating QR candidates for user queries by additionally using the BERT model along with the SMT model.

1 is a diagram illustrating an example of a network environment according to an embodiment of the present invention.
Figure 2 is a block diagram showing an example of a computer device according to an embodiment of the present invention.
Figure 3 is a flowchart showing an example of a method that can be performed by a computer device according to an embodiment of the present invention.
Figure 4 shows an example of a multi-QR generation model for QR generation in an embodiment of the present invention.
Figures 5 to 9 show examples of SMT models for QR generation in one embodiment of the present invention.
Figures 10 and 11 show an example of a BERT-based machine translation model for QR generation in an embodiment of the present invention.
Figure 12 shows an example of a BERT serving system for real-time service in one embodiment of the present invention.
Figure 13 shows an example of a cache-based BERT serving system structure in one embodiment of the present invention.

Hereinafter, embodiments of the present invention will be described in detail with reference to the attached drawings.

The query generation system according to embodiments of the present invention may be implemented by at least one computer device, and the query generation method according to embodiments of the present invention is performed through at least one computer device included in the query generation system. It can be. At this time, the computer program according to an embodiment of the present invention may be installed and driven in the computer device, and the computer device may perform the query generation method according to the embodiment of the present invention under the control of the driven computer program. . The above-described computer program can be combined with a computer device and stored in a computer-readable recording medium to execute the query generation method on the computer.

1 is a diagram illustrating an example of a network environment according to an embodiment of the present invention. The network environment in FIG. 1 shows an example including a plurality of electronic devices 110, 120, 130, and 140, a plurality of servers 150 and 160, and a network 170. Figure 1 is an example for explaining the invention, and the number of electronic devices or servers is not limited as in Figure 1. In addition, the network environment in FIG. 1 only explains one example of environments applicable to the present embodiments, and the environment applicable to the present embodiments is not limited to the network environment in FIG. 1.

The plurality of electronic devices 110, 120, 130, and 140 may be fixed terminals or mobile terminals implemented as computer devices. Examples of the plurality of electronic devices 110, 120, 130, and 140 include smart phones, mobile phones, navigation devices, computers, laptops, digital broadcasting terminals, Personal Digital Assistants (PDAs), and Portable Multimedia Players (PMPs). ), tablet PC, etc. For example, in FIG. 1, the shape of a smartphone is shown as an example of the electronic device 110. However, in embodiments of the present invention, the electronic device 110 actually communicates with other devices through the network 170 using a wireless or wired communication method. It may refer to one of various physical computer devices capable of communicating with electronic devices 120, 130, 140 and/or servers 150, 160.

The communication method is not limited, and may include not only a communication method utilizing a communication network that the network 170 may include (for example, a mobile communication network, wired Internet, wireless Internet, and a broadcast network), but also short-range wireless communication between devices. For example, the network 170 may include a personal area network (PAN), a local area network (LAN), a campus area network (CAN), a metropolitan area network (MAN), a wide area network (WAN), and a broadband network (BBN). , may include one or more arbitrary networks such as the Internet. Additionally, the network 170 may include any one or more of network topologies including a bus network, star network, ring network, mesh network, star-bus network, tree or hierarchical network, etc. Not limited.

Each of the servers 150 and 160 is a computer device or a plurality of computers that communicate with a plurality of electronic devices 110, 120, 130, 140 and a network 170 to provide commands, codes, files, content, services, etc. It can be implemented with devices. For example, the server 150 may be a system that provides a desired service (for example, a search service) to a plurality of electronic devices 110, 120, 130, and 140 connected through the network 170.

Figure 2 is a block diagram showing an example of a computer device according to an embodiment of the present invention. Each of the plurality of electronic devices 110, 120, 130, and 140 described above or each of the servers 150 and 160 may be implemented by the computer device 200 shown in FIG. 2.

As shown in FIG. 2, this computer device 200 may include a memory 210, a processor 220, a communication interface 230, and an input/output interface 240. The memory 210 is a computer-readable recording medium and may include a non-permanent mass storage device such as random access memory (RAM), read only memory (ROM), and a disk drive. Here, non-perishable large-capacity recording devices such as ROM and disk drives may be included in the computer device 200 as a separate permanent storage device that is distinct from the memory 210. Additionally, an operating system and at least one program code may be stored in the memory 210. These software components may be loaded into the memory 210 from a computer-readable recording medium separate from the memory 210. Such separate computer-readable recording media may include computer-readable recording media such as floppy drives, disks, tapes, DVD/CD-ROM drives, and memory cards. In another embodiment, software components may be loaded into the memory 210 through the communication interface 230 rather than a computer-readable recording medium. For example, software components may be loaded into memory 210 of computer device 200 based on computer programs installed by files received over network 170.

The processor 220 may be configured to process instructions of a computer program by performing basic arithmetic, logic, and input/output operations. Commands may be provided to the processor 220 by the memory 210 or the communication interface 230. For example, processor 220 may be configured to execute received instructions according to program code stored in a recording device such as memory 210.

The communication interface 230 may provide a function for the computer device 200 to communicate with other devices (eg, the storage devices described above) through the network 170. For example, a request, command, data, file, etc. generated by the processor 220 of the computer device 200 according to a program code stored in a recording device such as memory 210 is transmitted to the network ( 170) and can be transmitted to other devices. Conversely, signals, commands, data, files, etc. from other devices may be received by the computer device 200 through the communication interface 230 of the computer device 200 via the network 170. Signals, commands, data, etc. received through the communication interface 230 may be transmitted to the processor 220 or memory 210, and files, etc. may be stored in a storage medium (as described above) that the computer device 200 may further include. It can be stored as a permanent storage device).

The input/output interface 240 may be a means for interfacing with the input/output device 250. For example, input devices may include devices such as a microphone, keyboard, or mouse, and output devices may include devices such as displays and speakers. As another example, the input/output interface 240 may be a means for interfacing with a device that integrates input and output functions, such as a touch screen. The input/output device 250 may be configured as a single device with the computer device 200.

Additionally, in other embodiments, computer device 200 may include fewer or more components than those of FIG. 2 . However, there is no need to clearly show most prior art components. For example, the computer device 200 may be implemented to include at least some of the input/output devices 250 described above, or may further include other components such as a transceiver, a database, etc.

Hereinafter, specific embodiments of a method and system for generating a reconstructed query will be described.

These embodiments relate to QR technology for application to a query-term matching based search service. The goal is to view QR as a translation problem and use a machine translation model to convert it into a query suitable for search. In particular, in this embodiment, a reconstructed query (hereinafter referred to as a 'QR candidate') can be generated for the user query through a translation model that combines the SMT model and the BERT model.

Figure 3 is a flowchart showing an example of a method that can be performed by a computer device according to an embodiment of the present invention.

The computer device 200 according to this embodiment can provide a search service to clients through a dedicated application installed on the client or through access to a web/mobile site related to the computer device 200. The computer device 200 may be configured with a query generation system implemented on a computer. For example, the query generation system may be implemented in the form of a program that operates independently, or may be implemented in the form of an in-app of a specific application so that it can operate on the specific application.

The processor 220 of the computer device 200 may include at least one component for performing the query generation method according to FIG. 3 . Depending on the embodiment, components of the processor 220 may be selectively included in or excluded from the processor 220. Additionally, depending on the embodiment, components of the processor 220 may be separated or merged to express the functions of the processor 220.

The processor 220 and its components may control the computer device 200 to perform steps S310 to S330 included in the query generation method of FIG. 3. For example, the processor 220 and its components may be implemented to execute instructions according to the code of an operating system included in the memory 210 and the code of at least one program.

Here, the components of the processor 220 may be expressions of different functions performed by the processor 220 according to instructions provided by program codes stored in the computer device 200.

The processor 220 may read necessary instructions from the memory 210 where instructions related to controlling the computer device 200 are loaded. In this case, the read command may include a command for controlling the processor 220 to execute steps S310 to S330 that will be described later.

Steps S310 to S330, which will be described later, may be performed in an order different from the order shown in FIG. 3, and some of the steps S310 to S330 may be omitted or additional processes may be included.

Referring to FIG. 3, in step S310, the processor 220 may generate a first QR candidate group by reconstructing the user query using the SMT model, which is a machine translation model. The processor 220 may generate a first QR candidate group through an SMT model learned with parallel corpus training data.

In step S320, the processor 220 may generate a second QR candidate group by reconstructing the user query using a BERT-based machine translation model. While the SMT model can generate many QR candidates at a fast translation speed, it has the disadvantage of lacking translation that takes context into account. To complement this, the BERT model is applied to QR. Unlike SMT, the BERT model, a classification model, understands the meaning of words and enables natural translation that reflects the context. The BERT model can be learned using pre-training techniques using MLM (masked language model) and NSP (next sentence prediction) for deeper language understanding.

In step S330, the processor 220 searches at least a portion of the first QR candidate group and the second QR candidate group based on the translation score through the SMT model and the BERT-based machine translation model to provide a search result corresponding to the user query. You can select by query. For example, the processor 220 may use the top certain number or certain percentage of QR candidates as an actual search query based on the prediction probability through each machine translation model.

In the process of selecting a search query, the processor 220 may apply a higher weight to the translation score of the second QR candidate group than that of the first QR candidate group. Depending on the embodiment, when selecting a certain number or percentage of QR candidates based on the translation score through a search query, it is also possible to select a greater number or percentage of QR candidates from the second QR candidate group than the first QR candidate group. . Additionally, when providing search results, the processor 220 may also expose search results corresponding to the second QR candidate group higher than the search results of the first QR candidate group.

As shown in FIG. 4, the processor 220 has a multi-model structure and may include an SMT model 410 and a BERT-based machine translation model 420. The processor 220 can generate the first QR candidate group 401 through reconstruction using the SMT model 410 for the user query 40, and can also generate the first QR candidate group 401 through reconstruction using the BERT-based machine translation model 420. 2 QR candidate group 402 can be generated.

The structure of the SMT model 410 is shown in FIG. 5.

Referring to FIG. 5, the SMT model 410 provides translation results with statistical probability and includes a translation model 501 and a language model 502.

The translation model 501 converts a source language into a target language, and requires parallel corpus learning data consisting of pairs of source and target sentences.

The translation model 501 consists of an alignment model and a lexical translation model, and the alignment 601 can be extracted as shown in FIG. 6.

The language model 502 predicts the probability of a sentence and requires training data for the target language. Referring to Figure 7, when consecutive words such as 'where are we going' are given, the probability of 'are' coming after 'where' and the probability of 'we' coming after 'where are' are calculated.

The SMT model 410 can generate statistical translation results by combining the translation model 501 and the language model 502.

A QR system applying the above-mentioned SMT model 410 is shown in FIG. 8.

A translation model (501) consisting of an alignment model and a vocabulary translation model, and a language model (502) can be learned from the parallel corpus dictionary.

Referring to FIG. 8, the QR system applying the SMT model 410 goes through a tokenization process (morphological tokenization), a decoding process (SMT decoder), a result generation process (N best QR result), and heuristic logic to obtain the original By converting the source query, which is an original query, into a target query, a reconstructed query, that is, the first QR candidate group 401, can be created.

For example, as shown in Figure 9, the QR system applying the SMT model 410, when 'Naver free font' is given as the user query 901, which is the original query, 'Naver', 'free', and 'font' After tokenization as follows, through the decoding process and result generation process, 'Naver free font', 'Naver free font', and 'Naver free font' can be generated as QR candidates (902) for user inquiry (901). .

The QR system applying the SMT model (410) has the limitation of relying on the language model (502) for context, so it has the disadvantage of lacking translation that takes context into account compared to deep learning.

In this embodiment, a QR system using the BERT-based machine translation model 420 can be added to improve the quality limit of the SMT model 410.

*Figure 10 shows the BERT model structure.

Referring to FIG. 10, the BERT model (1000) receives a given sentence as input and first embeds it in token units through an input embedding layer (1001) and then tokens through a transformer layer (1002). It consists of a contextual representation structure with encoding that takes into account location information.

The transformer layer (1002) is composed of a self-attention model instead of a model such as CNN or RNN, and the BERT model (1000) embeds a language using an encoder among the transformer's encoder and decoder.

By replacing the encoder part of the transformer model with the BERT model (1000), a BERT-based machine translation model (420) can be built for application to the QR system.

The QR system applying the BERT-based machine translation model 420 is shown in Figure 11.

The BERT model, a classification model, is showing very high performance when applied to various language processing and information retrieval fields. As shown in FIG. 11, the QR system for generating QR candidates through a translation model is a transformer model and may be composed of an encoder 1110 for understanding input queries and a decoder 1120 for generating QR candidates. The encoder (1110) part can be replaced with the BERT model (1000).

The encoder 1110 can deeply understand the user query 1101, which is the original query, using the BERT model 1000, and understands the meaning of words to enable natural translation that reflects the context. The decoder 1120 is a transformer model decoder and can generate a QR candidate group 1102 for the user query 1101.

For example, as shown in Figure 11, the QR system applying the BERT-based machine translation model (420) provides a deeper query using the BERT model (1000) when 'top 10 female music rankings' is given as a user query (1101). Through language understanding, 'Top 10 female music rankings', 'Top 10 female song rankings', 'Top 10 female music rankings', and 'Top 10 female music rankings' are created as QR candidates (1102) for user queries (1101). can do.

Accordingly, the processor 220 can generate QR candidates to provide optimal search results using the BERT-based machine translation model 420 along with the SMT model 410 as a multi-QR generation model.

In the case of the BERT-based machine translation model (420), the model size is large, making it difficult to guarantee performance in real-time services.

Referring to FIG. 12, the processor 220 may further include a cache-based BERT serving system 1230 to overcome the response speed problem of the BERT-based machine translation model 420. The BERT serving system 1230 can apply a method of storing the inferred QR results of the BERT-based machine translation model 420 in a cache.

Generally, QR is performed before the search for each collection when a query comes in from integrated search, so the response speed of integrated search slows down as QR's response speed slows down. To solve this, it is possible to apply a method of adding a cache server and storing and using the inferred QR results of the BERT-based machine translation model 420 in the cache server.

Figure 13 shows an example of the BERT serving system structure in one embodiment of the present invention.

Referring to FIG. 13, the BERT serving system 1230 may include a cache server 1301, a Neural Machine Translation (NMT) producer 1302, a queue 1303, and an NMT consumer 1304.

When a user query is input, the NMT producer 1302 refers to the cache server 1301 to find a QR candidate corresponding to the user query and returns the result.

Meanwhile, the NMT producer 1302 inserts the user query into the queue 1303 when there is no QR result corresponding to the user query in the cache server 1301. When the same query is input continuously, duplicate queries may be entered in the queue 1303, so an empty value is inserted into the cache server 1301. Empty values inserted into the cache server 1301 may be temporarily maintained for a certain period of time (eg, 1 hour).

The queue 1303 functions as a message queue like Kafka, and can maintain queries received from the NMT producer 1302 and then transfer them to the NMT consumer 1304.

The NMT consumer 1304 can receive a query list from the queue 1303 and perform inference on QR candidates using the BERT-based machine translation model 420. The NMT consumer (1304) inserts the inference result for the QR candidate of the BERT-based machine translation model (420) into the cache server (1301). QR results inferred on the cache server 1301 may be updated on a regular basis (eg, 1 day).

The processor 220 can calculate the number of serving devices needed to process queries accumulating in real time. The processor 220 may determine the number of devices constituting the BERT serving system 1230 so that queries input in real time can be processed within a unit time (eg, 1 second).

For example, the processor 220 may calculate the number of serving devices based on the maximum number of inferences per unit time and the maximum unique query per second (UQPS). It is more advantageous to process n queries in a batch than to perform inference for one query n times. For example, to process 1,500 UQPS with 1 second throughput (16 queries), approximately 94 (1500/16=93.75) model server containers are required.

Therefore, in this embodiment, real-time service can be guaranteed by using a cache-based serving platform to apply the BERT model, which has a large model size and slow response speed, to the QR system.

In addition to using a cache-based serving platform, it is also possible to optimize performance by reducing the parameters of the BERT model by lowering the layers to three and applying knowledge distillation techniques for model compression.

As such, according to embodiments of the present invention, QR quality and search quality can be improved by generating QR candidates for user queries by additionally using the BERT model along with the SMT model.

The device described above may be implemented with hardware components, software components, and/or a combination of hardware components and software components. For example, the devices and components described in the embodiments include a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable gate array (FPGA), and a programmable logic unit (PLU). It may be implemented using one or more general-purpose or special-purpose computers, such as a logic unit, microprocessor, or any other device capable of executing and responding to instructions. The processing device may execute an operating system (OS) and one or more software applications running on the operating system. Additionally, a processing device may access, store, manipulate, process, and generate data in response to the execution of software. For ease of understanding, a single processing device may be described as being used; however, those skilled in the art will understand that a processing device includes multiple processing elements and/or multiple types of processing elements. It can be seen that it may include. For example, a processing device may include a plurality of processors or one processor and one controller. Additionally, other processing configurations, such as parallel processors, are possible.

Software may include a computer program, code, instructions, or a combination of one or more of these, which may configure a processing unit to operate as desired, or may be processed independently or collectively. You can command the device. The software and/or data may be embodied in any type of machine, component, physical device, computer storage medium or device for the purpose of being interpreted by or providing instructions or data to the processing device. there is. Software may be distributed over networked computer systems and stored or executed in a distributed manner. Software and data may be stored on one or more computer-readable recording media.

The method according to the embodiment may be implemented in the form of program instructions that can be executed through various computer means and recorded on a computer-readable medium. At this time, the medium may continuously store a computer-executable program, or temporarily store it for execution or download. In addition, the medium may be a variety of recording or storage means in the form of a single or several pieces of hardware combined. It is not limited to a medium directly connected to a computer system and may be distributed over a network. Examples of media include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical recording media such as CD-ROMs and DVDs, magneto-optical media such as floptical disks, And there may be something configured to store program instructions, including ROM, RAM, flash memory, etc. Additionally, examples of other media include recording or storage media managed by app stores that distribute applications, sites or servers that supply or distribute various other software, etc.

As described above, although the embodiments have been described with limited examples and drawings, various modifications and variations can be made by those skilled in the art from the above description. For example, the described techniques are performed in a different order than the described method, and/or components of the described system, structure, device, circuit, etc. are combined or combined in a different form than the described method, or other components are used. Alternatively, appropriate results may be achieved even if substituted or substituted by an equivalent.

Therefore, other implementations, other embodiments, and equivalents of the claims also fall within the scope of the claims described below.

Claims (7)

In a query generation method executed on a computer device,
The computer device includes at least one processor configured to execute computer-readable instructions contained in a memory,
The query creation method is:
Generating, by the at least one processor, a query candidate through query reformulation using an artificial intelligence model for the original query; and
Selecting at least some of the query candidates as a search query by the at least one processor
Including,
The step of generating the query candidate is,
Referring to a cache storing inferenced query candidates and finding the query candidate corresponding to the original query in the cache;
If there is no query candidate corresponding to the original query in the cache, performing inference using the artificial intelligence model for the original query and inserting the inference result into the cache
A query creation method including. According to paragraph 1,
The generating step is,
Generating the query candidate through the artificial intelligence model, which is a machine translation model that replaces the encoder part of the transformer model with the BERT model.
A query creation method characterized by . According to paragraph 1,
The number of serving devices to process queries in real time is calculated based on at least one of the maximum number of inferences per unit time and the maximum UQPS (unique query per second).
A query creation method characterized by . According to paragraph 1,
The generating step is,
Generating the query candidates through the artificial intelligence model learned using a pre-training technique using MLM (masked language model) and NSP (next sentence prediction).
A query creation method characterized by . According to paragraph 1,
The query creation method is:
Generating, by the at least one processor, another query candidate through query reconstruction using a statistical machine translation model for the original query.
It further includes,
The selection step is,
Selecting some of the query candidates and the other query candidates as the search query based on the prediction probability through each model.
A query creation method characterized by . A computer-readable recording medium on which a program for executing the query generation method of any one of claims 1 to 5 on a computer is recorded. In computer devices,
At least one processor configured to execute computer readable instructions contained in memory
Including,
The at least one processor,
A process of generating query candidates through query reconstruction using an artificial intelligence model for the original query; and
The process of selecting at least some of the above query candidates as a search query
Process it,
The at least one processor,
Refer to the cache where the inferenced query candidate is stored and find the query candidate corresponding to the original query in the cache,
If there is no query candidate corresponding to the original query in the cache, performing inference using the artificial intelligence model for the original query and inserting the inference result into the cache
A computer device characterized by a.