JP6918140B2 - Generation of query variants using a trained generative model - Google Patents

Description

Regarding the generation of query variants using a trained generative model.

Rule-based rewriting of search queries has been used within the query processing components of search systems. For example, a rule-based rewrite can generate a query rewrite by removing certain stopwords such as "the" and "a" from the query. The rewritten query can then be submitted to the search system, returning search results that respond to the rewritten query.

In addition, a set of similar queries is utilized in the search system to recommend additional queries related to, for example, submitted queries (eg, "people also search for X"). I came. Similar queries to a given query are often determined by navigational clustering. For example, for the query "funny cat pictures", submit "funny cat pictures" followed by "funny cat pictures with captions". The similar query can be determined based on the frequent submission of the similar query by the user. Therefore, a similar query for a given query is predefined.

Embodiments herein are directed to systems, methods, and computer-readable media relating to the generation of query variants for submitted queries. In many embodiments, a trained generative model is utilized to generate query variants at run time. Based on the application of query tokens to the generative model, and optionally the application of additional input features to the generative model, the generative model can be used to actively generate variants of the query. , The generative model is productive. In this way, the generative model can be used to generate variants of any query, even if the generative model was not trained based on the query. Therefore, generative models can be used to generate variants for new queries, and for so-called "tail" queries (ie, sub-threshold submit frequency and / or submit amount queries). As a result, richer query inputs can lead to more efficient identification of related results, allowing queries to be processed more effectively. For example, a query is not excluded simply because it has a low submit frequency and / or a submit amount. The improvement in efficiency can be in the rate at which the relevant results can be obtained. This is because the user does not need to resubmit the modified query if the initial query does not produce any relevant results. The disclosed embodiments make it possible to automatically test multiple query variants. Convergence of results can also be guaranteed through training of the model used to generate the mutant, thus improving efficiency by targeting query mutation generation rather than simply by simultaneous processing of multiple queries. Achieved. Therefore, the use of technical resources required to process a query, including the processing power and power consumption of a processor that implements the disclosed method, is optimized throughout the embodiments of the present invention.

In some embodiments, the generative model is a neural network model, such as a neural network model with one or more "memory layers". The storage layer contains one or more recurrent neural network (RNN) units, such as long-short-term memory (“LSTM”) units and / or gated recurrent units (“GRU”).

In some embodiments where the generative model is a neural network model with a storage layer, the generative model is a sequence-to-sequence model. For example, a sequence-to-sequence model can be one in which the tokens of the query can be applied as input to the model (eg, per token, or per combination), and the coding of the tokens is generated across layers of the network. In addition, the generated coding can be decoded across additional layers of the network, and the resulting decoding indicates a variant (directly or indirectly) of the query. For example, the resulting decoding can be applied to the softmax layer of the network to generate variants of the query. In some versions of those embodiments, the generative model has the same or similar architecture as the sequence-to-sequence neural machine translation model and is trained using query variant-specific training data. For example, query variant-specific training data is a query pair that each has a "click" on the same document (for example, to train for synonymous query variant generation), (eg, supplemental query variant generation (follow)). Based on consecutively submitted query pairs (for training for -up query variant generation) and / or original canonical query pairs (for example, for training for normalized query variant generation) obtain. Such models can optionally be pre-trained based on translation training data.

In some embodiments, the generative model is trained as a "multitasking" model in that the generative model is trained to allow the generation of any one of multiple types of query variants. In some of those embodiments, the type of query variant to be generated for a given path of the generative model can be indicated based on the type value input applied to the model in the given path. The type value input indicates the type of query variant to be generated. The types of query variants include, for example, synonymous queries, supplemental queries, generalized queries, canonical queries, language translation queries, entailment queries, specification queries, and / or clarification queries (ie, clarifications). Can include queries) provided as output to the user to prompt. Additional and / or alternative types may be defined, including those with higher or lower particle size. Training data from various types can be used when training the generation model, and each instance of the training data indicates the type of query variant for that instance and can be used as a training instance input during training. Including value input. In some of these embodiments, training a multitasking model in such a manner can take advantage of information sharing across different types of training data, thereby making the multitasking model to be trained more It can be a robust performance.

After training, the multitasking model produces variants of the first type for queries in the first pass (eg, based on applying the first type value as input) and in the second pass. For queries, it can be used to generate variants of the second type (eg, based on applying a second type value as input). Additional variants of additional types can be generated in additional paths. As described herein, the amount of additional paths generated using the multitasking model can vary from query to query. For example, the amount of additional paths can be controlled in-situ, for example, based on the variants produced in a path and / or the response to such variants. In addition, subsequent passes may generate variants of the same type as the previous variants generated in the previous path. In some of those situations, subsequent paths are the previous variant (eg, the previous variant itself and / or the response to the previous variant) and / as described herein. Alternatively, information based on other previous variants may be utilized, which may result in variants of the subsequent path being different from previous variants.

In some embodiments, multiple variants of the original query are generated utilizing a generative model, each of the plurality of variants is submitted to the search system, and the corresponding response for each of the plurality of variants. Is received. Output can be generated based on one or more of the responses and the output is provided in response to the original query. For example, the output is the "best" response (eg, indicated by the response score provided by the search system), multiple of the "best" responses, and / or when the variant is a complementary type. ) Variants and corresponding responses may be included. In this way, and in other ways, the response to the variant of the original query can be used to respond to the original query and provide output, and the output answers the original query directly. In addition, the response to the original query variant can be used to confirm / confirm the response to the original query and / or the response to other variants. For example, the accuracy of the "answer" to the original query can be determined based on whether a positive answer is provided for the variant of the original query. For example, positive similar answers (similar to the original query answer) are synonymous, generalized, and based on whether other positive answers are provided for supplemental type variants. / Or based on availability for linguistic translation type variants. In this way, and in other ways, even if an unsubstantiated / unsubstantiated response is determined, it cannot be utilized in the provided output, and / or even if it is utilized in the provided output, it is unsubstantiated. Can be flagged (for example, flagged as "potentially bullshit").

In some embodiments and / or situations, multiple responses are returned by the search system in response to the variant. In some other embodiments and / or situations, the search system provides a single response in response to the variant. In some of those embodiments, a single response is either a "answer" (eg, the response that the search system sees is an answer to a variant) or an indication that the answer is unknown. Including. In other embodiments, the indication that the answer is unknown can be the lack of any response by the search system. A search system can be a search system that operates across multiple domains, or a search system that is personalized to one or more specific domains (eg, online shopping domains). The response returned by the search system is, for example, search results (for example, snippets of content from the document and links to the document), answers (for example, content that the search system considers to be authoritative answers), images, videos, or A knowledge graph entity can be a "null" response (for example, a "no answer" response). In some situations, as an addition or alternative, the generated variant may be provided as output to the user (submitting the original query) for clarification and provided by the user in response to the prompt. Clarifying the user interface input can be used as a "response" to the variant. Such user-provided responses can be utilized to influence the generation of other variants. For example, such a user-provided response can be used to generate a context vector that is passed to the generative model in another iteration that produces the variant.

In some embodiments, multiple generative models can be generated, each of which is trained based on training data based on past query submissions of a unique group of users. For example, a first generative model can be generated based on training data based on past query submissions of users with attributes A and B. A second generative model can be generated based on training data based on past query submissions of users with attributes B and C. For submitted queries for users with attributes B and C (rather than attribute A), when the second generative model generates a variant for that user (without even selecting the first generative model). ) Can be selected for use. This is because the user attributes B and C match those used when training the second generative model. In this way, the generative model can be selected from a plurality of available generative models, and thus the selected generative model is tailored to the user's attributes. As a result, the selected generative model can be used to generate more appropriate query variants for the user. For example, for scientific researchers, very different variants can be generated compared to, for example, freelance writers.

In some embodiments, multiple generative models may be generated, each of which has a history associated with a particular attribute of the user, a particular temporal attribute, and / or other attributes. Training based on query submission Training based on data. For example, the first generative model is generated based on training data based on past query submissions related to online shopping tasks. For example, based on past query submissions being submitted to an online shopping search system, search results are shopping-centric based on the user selecting shopping content (eg, an ad) in relation to the submission. Based on that, the user can be identified based on completing the transaction following the submit. A second generative model can be generated based on training data based on past query submissions related to different specific attributes. For example, a second generative model can be generated based on training data based on past query submissions related to tasks moving to a position (eg, any position, any restaurant position, meeting position, etc.). For example, past query submits may be identified based on being submitted closer in time to the scheduled calendar input, based on being submitted before and / or during the move to a position. For a user's submitted query, the user's task can be predicted, and a generative model corresponding to the predicted task is selected to generate a variant for that submitted query. For example, if a user's calendar input and / or electronic communication indicates that the user is moving to a position (or is about to move to a position), then the second generative model in the preceding example is that model is in position. It can be selected based on what is relevant to the task to be moved. In this way, the generative model can be selected from a plurality of available generative models, and thus the selected generative model is tailored to user tasks such as the tasks that are or are expected to be involved. As a result, the selected generative model can be used to generate more appropriate query variants for the user's current task. As described above and elsewhere herein, in various embodiments, the generative model can be a multitasking model, allowing the generation of various different types of query variants. Some of those various embodiments allow the use of generative models to generate variants that extend the user query, allowing the search for multiple pathways that extend the query. Such variants may be provided simultaneously or sequentially (eg, optionally, without first issuing a query based on such variants) for presentation to the user, allowing the user to query. Allows you to explore various routes to extend. As an addition or alternative, a response to such a variant may be obtained from the search system, the response being provided for presentation to the user, and the user searching for various responses for extension to the query. to enable.

Some embodiments described herein are used to generate variants of a query submitted by a user who may have difficulty formulating a query (eg, due to physical disability). Can be done. For example, the query can be formulated by the user using gaze-guided (or other low-labor) user interface input, and query variants are generated according to the techniques described herein. In this way, query variants of a query can be generated and presented to the user without the user having to manually generate such variants.

As described herein, the generative model activates a variant of the query based on the application of the query token to the generative model and optionally the application of additional input features to the generative model. Can be used to generate. In some of those embodiments, additional input features may include attributes, temporal attributes, and / or other features related to the user who submitted the query. User-related attributes are, for example, the user's location (for example, Louisville, Kentucky, within the "Restaurant", Southeastern United States), user-related tasks (for example, cooking, car repair, travel planning), and /. Or it may include the weather at the user's location. Tasks related to the user can be tasks that the user is currently involved in or should be involved in. In some embodiments, the task is submitted by the user, for example, a user's remembered calendar entry, a user's electronic communication (eg, a chat message sent to or sent by the user or other communication). Predicted based on various signals such as past queries. Time attributes can include, for example, the current time, the current day of the week, and / or the current date. In this way, query variant generation utilizing the generative model can be personalized for the user and / or the current context based on the application of additional input features to the generative model.

In some embodiments, a generative model can be utilized to generate variants of a query, where an advertisement or other content is assigned such content to one or more of the variants. Based on, it is provided to the client device that generated the query. In some of those embodiments, the variants generated by the generative model can be tailored to the client device and / or the user of the client device using techniques as described herein. For example, a generative model may be selected based on the user's attributes and / or user-related attributes may be provided as input to the generative model and used in generating variants.

In some embodiments, the generative model is utilized to generate variants (and / or mutant tokens) of the original query at each of multiple time steps. In some of those embodiments, whether or not variants are produced and / or which variants are produced in a given time step can be based on current state characteristics. Current state features are, for example, the search system response to the original query, the search system response to the variant of the original query generated in the previous time step, the variant of the original query generated in the previous time step, the original. Can include user responses to variants of the query (eg, clarification variants provided as a prompt to the user) and / or features based on the original query. In this way, it is dynamic to generate mutations for the query during the session based on the mutants generated before the query during the session, the response to the previously generated mutants, and / or the original query. Can affect. For example, in some embodiments, one or more of such current state features may or may not produce another variant, or, as an alternative, without the generation of another variant. Can be used to determine whether a response to a previous variant (and / or the original query) should be provided instead in response to a query for. Further, for example, in some additional or alternative embodiments, as input to the generative model, one or more of such current state features influence mutation generation in time steps ( Can be applied (directly or indirectly). For example, a vector overview of current state features can be generated and applied as input to the generative model to influence the generated variants.

In some embodiments, the trained control model is directed against the generative model to influence whether the variant should be generated at each of the multiple time steps and / or the mutation generation at the time step. It is used to determine the features provided as input. For example, the trained control model can be a feedforward neural network model or a recurrent neural network (RNN) model. Current state features are applied as inputs to the trained control model and are provided to the generative model to indicate whether another variant should be generated across the model and / or to influence mutation generation. Features to be generated (eg, vector summaries of current state features and / or reward signals) (if another variant should be generated) can be generated. In this way, within the actor-critic environment, the control model can act as a "critic" and the generative model can act as an "actor". Therefore, the trained control model is used to determine whether another variant should be generated and / or the characteristics to influence such generation, based on the observed current state features. Can be done. In this way, the trained control model can control the amount of additional variants generated for a given query. The control model dynamically determines the amount of mutation generation iterations for a given query, for example, based on the variants generated in the previous iteration for a given query and / or the response to such variants. As such, such control can result in different amounts of variants being generated from query to query. Such dynamic control often produces a relatively large amount of variants (eg, greater than 5, 10 or greater than 15) and / or a relatively large amount of response to such variants. Please understand that it can be considered.

In some embodiments, the control model and / or generative model can be trained at least partially based on reinforcement learning. In some of those embodiments, the control model and the generative model are trained separately, but in combination with each other. When training control and / or generative models based on reinforcement learning, the generated variants can be submitted to the search system and the response from the search system (and optionally lack of response) can be rewarded. .. For example, a response to a query variant, which is a "answer" response, is proportional to (or so is) the quality of the response response (eg, indicated by the response score provided by the search system for the "answer" response). Rewards (otherwise associated) can be assigned. In some of those examples, when no response is provided in response to a query variant and / or when the response is considered not to be a "answer" response (eg, based on output from the search system). , Rewards are not allocated. In other words, only the final "answer" response is rewarded, and intermediate actions are updated based on such rewards (for example, in Monte Carlo Q-learning techniques). In this way, Q-function learning, or other reinforcement function learning, can be based on rewards adjusted for the responses provided by the search system interacted during reinforcement learning. In the reinforcement learning embodiments described herein, the state at a given time step is indicated by one or more of the state features (eg, as described above), and the action is a query variant. (Ie, it produces another query variant), or can provide an "answer" response. Each action in the action space can be paired with a string that defines the corresponding question or "answer" response.

Some embodiments provide methods performed by one or more processors, including receiving the original query generated based on the user's user interface input via the client device. The method further includes applying the token of the original query and one or more attributes associated with the user as input to the trained generative model. The trained generative model is a sequence-to-sequence deep neural network model with one or more storage layers. The method further comprises generating at least one variant of the original query based on the application of tokens and one or more attributes to the trained generative model. The method further comprises generating output based on at least one variant and at least one search system response to at least one variant. The method further comprises responding to the original query and providing output for presentation through the client device.

In some embodiments, the original query is received, the original query token is applied as input to the trained generative model, and the original query token is applied to the trained generative model. A method performed by one or more processors is provided that involves generating multiple variants of the query. The original query can be generated based on the user's user interface input through the client device. Each of the generated variants is different from the original query, and generating a variant involves generating a variant based on the trained parameters of a trained generative model. The trained generative model is trained to allow the generation of multiple types of query variants, and the generated variants are the first of the multiple types of query variants. Includes variants and a second variant, the second of multiple types of query variants. The method produces output based on at least one search system response to at least one of the variants and / or at least one of the variants, and in response to the original query. Further includes providing output for presentation through the client device.

Some embodiments provide methods performed by one or more processors, including receiving the original query generated based on the user's user interface input through the client device. The method is a trained generative model from multiple trained generative models, based on the trained generative model being trained based on past query submissions of a group of users with one or more attributes in common with the user. Further includes selecting a model. The method is to apply the token of the original query as input to the selected trained generative model, and generate at least one variant of the original query based on applying the token of the original query to the trained generative model. And / or generating output based on at least one search system response to at least one variant and / or at least one variant. The method further comprises responding to the original query and providing output for presentation through the client device.

In some embodiments, receiving the original query, applying the token of the original query as input to the trained generative model, and based on the input, a variant of the original query through the trained generative model. Methods are provided that are performed by one or more processors, including the generation of. The original query can be generated based on the user's user interface input through the client device. Mutants generated through a trained generative model are different from the original query, and generating a variant of a query involves producing a variant based on the trained parameters of the trained generative model. The method is based on submitting a variant of the query to the search system to determine the variant response for the variant of the query, applying additional inputs to the trained generative model, and based on the additional inputs. It further includes generating additional variants of the original query through a trained generative model. The additional input applied to the trained generative model contains at least one of the tokens of the original query and the mutant tokens of the variants of the original query. The additional variants generated are different from the variants and the original query, and generating additional variants of the original query can generate additional variants based on the trained parameters of the trained generative model. Including producing. The method further comprises determining additional variant responses for additional variants of the original query based on submitting additional variants of the original query to the search system. The method further comprises generating output based on the mutant response and / or additional mutant response, and providing output for presentation through the client device in response to the original query.

Some embodiments provide methods performed by one or more processors, including receiving the original query generated based on the user's user interface input through the client device. The method determines the expected task for the user and applies the token of the original query and one or more task attributes of the expected task for the user as input to the trained generative model. Including that further. The method further comprises generating at least one variant of the original query based on the application of tokens and one or more task attributes to the trained generative model. The method produces output based on at least one variant and / or at least one search system response to at least one variant, and for presentation through the client device in response to the original query. Further includes providing output.

Some embodiments provide methods performed by one or more processors, including receiving the original query generated based on the user's user interface input through the client device. The method further comprises determining the expected task for the user. The method further comprises selecting a trained generative model from a plurality of trained generative models based on the trained generative model being trained based on past query submissions associated with the predicted task. The method is to apply the token of the original query as input to the selected trained generative model, and generate at least one variant of the original query based on applying the token of the original query to the trained generative model. And / or generating output based on at least one search system response to at least one variant and / or at least one variant. The method further comprises providing output in response to the original query.

Various embodiments disclosed herein include a processor (eg, a central processing unit (CPU), graphics) to implement a method such as one or more of the methods described herein. It may include one or more non-temporary computer-readable storage media that store instructions that can be executed by a processing unit (GPU) and / or a tensor processing unit (TPU). Yet another various embodiment is one or more processors capable of operating to execute a stored instruction to perform a method such as one or more of the methods described herein. May include a system of one or more computers including.

It should be understood that all combinations of the above concepts and additional concepts described in more detail herein are intended as part of the subject matter disclosed herein. For example, all combinations of claims appearing at the end of this disclosure are intended as part of the subject matter disclosed herein.

FIG. 5 is a block diagram of an exemplary environment in which the embodiments disclosed herein can be implemented. It is a figure which shows an example of training a generative model according to the embodiment disclosed in this specification. It is a figure which shows an example of using a generative model to generate one or more variants of a query. FIG. 5 illustrates another example of using a generative model to generate one or more variants of a query, where the control model is used to control the generation of variants. It is a flowchart which shows the method of training the generative model according to the embodiment disclosed in this specification. It is a flowchart which shows how to use a generative model to generate one or more variants of a query. It is a flowchart showing how to use a generative model to generate one or more variants of a query, where the control model is used to control the generation of variants. FIG. 5 illustrates an exemplary graphical user interface for providing output based on variants generated according to the embodiments disclosed herein. FIG. 5 illustrates an exemplary graphical user interface for providing output based on variants generated according to the embodiments disclosed herein. It is a figure which shows the exemplary architecture of a computing device.

FIG. 1 shows an exemplary environment in which the embodiments disclosed herein can be implemented. The exemplary environment of FIG. 1 includes a client device 106, a query system 110, a search system 140, a generative model training engine 120, and a training instance engine 122. Each such system and engine can be implemented within one or more computing devices that communicate, for example, through a communication network. Communication networks can include wide area networks (WANs) such as the Internet, one or more intranets, and / or one or more bus subsystems. Optionally, the communication network may utilize one or more standard communication techniques, protocols, and / or interprocess communication techniques.

The query system 110, search system 140, generative model training engine 120, and training instance engine 122 are systems, configurations in which the techniques described herein can be performed and / or described herein. Elements and techniques are exemplary components that can interface with them. The operations performed by one or more of the systems 110, 140, and engines 120, 122 of FIG. 1 can each be distributed across multiple computer systems. In some embodiments, one or more aspects of the systems 110, 140, and engines 120, 122 may be combined as a single system and / or one or more aspects may be on the client device 106. Can be carried out at. For example, in some of those embodiments, aspects of query system 110 may be combined with aspects of search system 140.

A user of client device 106 may formulate a query through client device 106 by providing user interface input through one or more user interface input devices of client device 106. Client device 106 submits a query to query system 110. In some situations, the query is in text format. In other situations, the query can be submitted in audio and / or other formats and converted to text format by the query system 110 (or other component).

For the received query, the query system 110 generates one or more variants of the received query and causes it to provide output to the client device 106, the output being based on one or more of the variants. In some embodiments, the output comprises one or more of the variants to be provided as proposed alternative variants for user consideration. In some embodiments, as an addition or alternative, the output contains content based on one or more responses from search system 140, and the response is one or more of variants to search system 140. Based on submission. The search system 140 may determine the response based on the access of one or more resources 166 and may utilize various techniques such as conventional information retrieval techniques. Response-based content can be, for example, response-based (eg, identical to the response) graphical and / or audible "answers" or other search results. If the response-based content is provided, the query system 110 may provide the content directly to the client device 106, or the search system 140 may cause the client device 106 to provide the content. In some embodiments, optionally, the query system 110 and the search system 140 may be controlled by the same party and / or work in concert with each other. Additional and / or alternative output, such as advertisements assigned to the generated variants in one or more databases, may be provided based on the generated variants.

In FIG. 1, the query system 110 includes a mutant engine 112 and a controller engine 114. In some embodiments, one or more embodiments of the variant engine 112 and the controller engine 114 may be combined and / or implemented as separate components from the query system 110, such as the client device 106. In some embodiments, the controller engine 114 may be omitted.

The variant engine 112 utilizes one or more trained generative models 152 to generate one or more query variants for submitted queries. In some embodiments, the variant engine 112 comprises one or more CPUs, GPUs, and / or TPUs operating through a trained generative model 152. The variant engine 112 applies a token of the query as input to one of the generative models 152, and based on the input, generates variants through the generative model for submitted queries. Generate variants. In many embodiments, in generating variants, the variant engine 112 further applies additional input features as inputs to the generative model and generates variants based on the additional input features.

In some embodiments, the additional input features may include attributes, temporal attributes, and / or other features that are relevant to the user who submitted the query. For example, when generating a variant for the original query, the variant engine 112 takes the token of the original query as input to one of the generative models 152, the attributes of the user who submitted the query (eg, the user). The location, the task the user is involved in), and the temporal attributes (eg, current day of the week, current time) can be applied to generate variants through the generative model based on the input applied.

In some embodiments, as an addition or alternative, additional input features applied in a given iteration of generating a variant for the original query are variants of the original query generated in the previous iteration. Includes shape-based features and / or features based on the search system response to such variants. For example, in generating a variant for the original query, the variant engine 112 may generate a variant at each of multiple time steps. At a given time step, the variant engine 112 receives a search system response to the original query as input to one of the generative models 152, a search system for variants of the original query generated in the previous time step. Responses, variants of the original query generated in the previous time step, and / or features based on the original query can be applied. In this way, mutation generation at a given time step can be affected by previously generated mutations, responses to previously generated mutations, and / or the original query.

In some embodiments, additional or alternative additional input features applied in a given iteration that generate variants for the original query include type values. For example, in some embodiments, one of the generative models 152 is trained to allow the generation of any one of multiple types of query variants. Can be a model. In some of those embodiments, the variant engine 112 may apply a type value indicating the type of query variant to be generated as input to one of the generative models 152. Types of query variants can include, for example, synonymous queries, supplemental queries, generalized queries, normalized queries, linguistic translation queries, and / or implication queries. In some embodiments, the variant engine 112 selects different type values for each of the multiple iterations that generate the variant, thereby utilizing the same generative model to generate multiple variants of different types. Generate.

In some embodiments, multiple generative models 152 are accessible to the variant engine 112, which generates variants for the submitted query based on one or more parameters. Select one or a subset of multiple generative models 152 for. For example, multiple generative models 152 may be provided, each of which is trained based on training data based on past query submissions of a unique group of users. For example, a first generative model can be generated based on training data based on past query submissions of users with attributes A and B. A second generative model can be generated based on training data based on past query submissions of users with attributes B and C. For submitted queries for users with attributes B and C (rather than attribute A), the variant engine 112 will generate variants for that query (without even selecting the first generative model). A second generative model can be selected. This is because the user attributes B and C match those used when training the second generative model.

Figure 1 also shows the generative model training engine 120 and the training instance engine 122. The training instance engine 122 creates a training instance and stores the training instance in the training instance database 164. For example, the training instance engine 122 may generate multiple training instances based on a submitted query database 162 that stores past query submissions for a large population of users. The generative model training engine 120 trains the generative model 152 based on the stored training instances in database 164. As described herein, in some embodiments, optionally, one or more of the generative models 152 utilize a training instance-independent reinforcement learning technique in the training instance database 164. Can be further trained. Additional instructions for embodiments of engines 120, 122, and databases 162 and 164 are provided in the description with respect to FIG. 2 below.

When provided, the controller engine 114 works in concert with the mutant engine 112 to control whether the mutant engine 112 produces variants and / or to generate parameters that influence mutation generation. , Provided to the mutant engine 112. Optionally, the controller engine 114 trains one or more in controlling whether the mutant engine 112 produces mutations and / or in generating parameters that affect mutation generation. Use the completed control model 154. In some embodiments, the variant engine 112 comprises one or more CPUs, GPUs, and / or TPUs operating via a trained control model 154.

In some embodiments, the controller engine 114 determines for a submitted query whether any variant should be generated by the variant engine 112 for the submitted query. For example, the controller engine 114 may make such a decision based on the submitted query itself and / or on the response (if any) from the search system 140 for the submitted query. For example, the controller engine 114 has a search system-provided score where the response is not returned by the search system 140, or some returned response is of inadequate quality (eg, the threshold cannot be met). ) Only in case can it be decided to generate a variant. In some of those embodiments, the controller engine 114 applies the token of the submitted query and / or the response characteristics to the submitted query to one of the control models 154 and the variant is It produces output through a control model 154 that indicates whether it should be generated. In some additional or alternative embodiments, the controller engine 114 applies the response characteristics to one of the submitted query tokens and / or control model 154 and generates a generative model in generating the variant. Generates output through the control model 154 provided to the mutation engine 112 for application as an input to (thus affecting mutation generation).

As described herein, in some embodiments, the variant engine 112 produces variants of the submitted query at each of the plurality of time steps. In some of those embodiments, the controller engine 114 determines when mutation generation should stop. In other words, whether the variant engine 112 produces variants in a given time step is subject to permission from the controller engine 114. In addition, the controller engine 114 may provide features that influence mutation generation at each time step for each time step. The controller engine 114 is at least one of one or more control models 154 in deciding whether mutagenesis should be stopped and / or in generating features that affect mutagenesis. Can be used.

As an example, the controller engine 114 is a search system response to the original query as input to one of the control models 154, a search system for variants of the original query generated by the variant engine 112 in the previous time step. Responses, variants of the original query generated by the variant engine in the previous time step, and / or features based on the original query can be applied. The controller engine 114 mutates to generate an output through a control model based on the applied input and use the output to generate another variant, or instead stop mutating. It can be decided whether to instruct the shape engine 112. When mutagenesis is stopped, the controller engine 114 instead provides as output a response to previously generated mutagens and / or previously generated mutagens in response to the submitted query. Can be. In this way, within the actor-critic environment, the controller engine 114 can act as a "critic" and the mutant engine 112 can act as an "actor". An additional description of an embodiment of the interaction of the controller engine 114, one of the control models 154, and / or the variant engine 112 of the controller engine 114 is described below with respect to FIG.

With reference to FIG. 2, an example of training the generative model 152A of the generative model 152 is shown. Training instance 164A is retrieved from the training instance database 164. Training instance 164A can be generated by the training instance engine 122 (Figure 1) based on a pair of queries previously submitted, for example by the user, and stored in the submitted query database 162 (Figure 1). As an example, the query pair is the user's query of the previous time, "did roger moore drive an aston martin in the persuaders", and (previous). A synonym for the time query) "what car did roger moore drive in the persuaders" (for example, before) Can include the user's query (immediately after the time query). As another example, the query pair is the previous time user's query "did leonardo da vinci paint mona lisa" and (a supplementary type to the previous time query). It can include a user's query for a later time, "who commissioned leonardo da vinci to paint the mona lisa".

Training instance 164A contains training instance inputs that include queries (eg, submitted queries for the previous time of a pair), attributes, and types. Attributes can include, for example, the attributes of the user who submitted the query, the temporal attributes of the query (eg, the day of the week of submission), the characteristics of the search system response to the query, and so on. The type can be a type value that indicates what type of variant is included in the training instance output. In some embodiments, the type can be assigned by human labeling, or based on the characteristics of the query pair used to generate training instance 164A (for example, the temporal separation of query submissions in the query pair). Can be inferred by the training instance engine 122 (based on size, comparison of search system responses for query-to-query queries). Training instance 164A also includes training instance output containing variants (eg, those submitted at a later time in a pair).

The generative model training engine 120 applies the training instance input of the training instance as the input to the generative model 152A. The generative model training engine 120 further generates output through the generative model 152A based on the applied inputs and the currently learned parameters of the generative model 152A. The generative model training engine 120 further generates a gradient based on the comparison of the generated output to the training instance output of the training instance 164A and updates the generative model 152A based on the gradient (eg, gradient through the generative model 152A as a whole). Backpropagation).

In generating the output based on the applied inputs, the generative model training engine 120 applies all or part of the inputs to the encoder layer 153A of the generative model 152A and produces the coding through the encoder layer 153A. For example, the token of the original query of the input may be applied to encoder layer 153A. The engine 120 may further apply the coding to the decoder layer 154A of the generative model 152A and generate a decoding of the coding through the decoder layer 154A. The engine 120 then applies the generated coding to the softmax layer 155A and produces an output through the softmax layer 155A based on the application of the generated coding. In some embodiments, the engine 120 applies input attributes and / or types to other layers and / or as "side inputs" of encoder layer 153A, decoder layer 154A, and / or softmax layer 155A. Applies to one of them. In some of those embodiments, engine 120 applies attributes and / or types to other layers downstream of encoder layer 153A rather than upstream of decoder layer 154A.

Figure 2 shows only a single training instance 164A, but it should be understood that many additional training instances are utilized in the training generation model 152A. Note that in some embodiments, a single training instance 164A and additional training instances are selected, thereby training the generative model 152A to specifically fit some attributes. For example, the generative model 152A can be trained by selecting (or biasing towards) training instances that are generated based on the user's past submissions with several attributes. For example, user attributes explicitly contained within the training instance input of a training instance can be utilized in such a selection. Further, for example, the generative model 152A can be trained by selecting (or biasing towards) training instances only related to some task attributes. For example, selection can be biased towards queries submitted in relation to several tasks involved (or should be involved). It should also be noted that in some embodiments, the generative model 152A is trained utilizing a training instance that contains multiple different "types" within the training instance input. As described herein, this creates a multitasking model that can generate multiple different types of variants and, at run time, can be biased towards a particular type by applying the corresponding type values as inputs. Is possible.

Figure 3 shows an example of using a generative model to generate one or more variants of a query. In Figure 3, the original query and user attributes are sent from the client device 106 to the variant engine 112. In some other embodiments, one or more of the attributes (eg, all) may not be sent with the query by the client device 106, or even not at all by the client device 106. For example, a user's attributes can be stored remotely from a client device. For example, attributes are stored remotely and can be accessed from remote storage by the variant engine 112 based on the user's past interactions (eg, through other client devices).

The variant engine 112 utilizes at least one of the generative models 152 to generate one or more variants of the original query. In generating the variant, the variant engine 112 may utilize the attribute in selecting one of the generative models 152 and / or of the attributes as an input to one of the generative models. One or more may be applied. The variant engine 112 may further apply the tokens of the original query to the generative model and / or other features (eg, previously generated variants when multiple variants are generated iteratively). ..

In some embodiments, the variant engine 112 sends the variant to the client device 106 as output that should be provided based on the original query. In some embodiments, as an addition or alternative, the variant engine 112 provides the search system 140 with one or more of the variants, which in turn provides one or more responses to the variant. Determine (for example, a single answer search result, or multiple search results) and send the response to the client device as output that should be provided based on the original query.

Figure 4 shows another example of using a generative model to generate one or more variants of a query. Specifically, FIG. 4 shows an example in which a control model is used to control the generation of variants.

In Figure 4, the original query and user attributes are sent from the client device 106 to the controller engine 114. As in FIG. 3, in some other embodiments, one or more of the attributes (eg, all) may not be sent with the query by the client device 106, or even by the client device 106 at all. It may not be sent.

In some embodiments, the controller engine 114 utilizes one or more control models 154 to determine whether to generate a variant of the original query. For example, the controller engine 114 applies the token of the original query, the search system response to the original query, and / or the user's attributes to one of the control models 154 to determine whether to generate a variant. Can be done. In some other embodiments, the controller engine 114 may determine by default that at least one variant or original query should be generated.

The controller engine 114 provides the mutant engine 112 with a reward signal determined based on the output through one or more control models 154, and also provides the current state. The current state can include, for example, the original query, the user's attributes, and / or feature vectors based on one or both, and the feature vectors are also based on the output through one or more control models 154.

The mutation engine may utilize at least one of the generative models 152 to generate one or more variants of the original query. In generating the variant, the variant engine 112 may utilize the reward signal in the provided state and optionally. For example, the variant engine 112 may apply a reward signal to a learned reward function to determine the reward for generating a query variant. The variant engine 112 provides variants to the search system 140. In response, the search system 140 generates one or more responses and provides the response to the controller engine 114.

The controller engine 114 utilizes the mutants generated so far and / or their corresponding responses to determine if another variant should be generated by the mutant engine 112. For example, the controller engine 114 applies the previously generated mutant tokens and / or corresponding response features as inputs to one of control model 154, and based on the inputs via the control model. The output can be used to determine if another variant should be generated. In some embodiments, the controller engine 114 further applies the token of the original query, the search system response to the original query, and / or the user's attributes as part of the input.

If the controller engine 114 determines that another variant should be generated, the controller engine 114 will receive an updated reward signal and an updated current state (eg, the mutants generated so far and / /. Or updated based on the corresponding variant response). The variant engine 112 may then generate one or more alternative variants and provide the variant to search system 140, again providing the corresponding response. The controller engine 114 may then redetermine whether another variant should be generated based on another variant and the corresponding response.

If, in a given iteration, the controller engine 114 determines that another variant should not be generated, the controller engine 114 will have one or more search system responses and / or one or more generated mutations. Send the form to client device 106 as output that should be provided based on the original query. For example, the controller engine 114 remembers all the responses provided and provides only one of the responses as a response output (for example, the highest quality response, or the highest quality response confirmed by other responses). obtain. As another example, the controller engine 114 may provide multiple of the responses (eg, N best responses, a diverse set of responses).

In some embodiments, the control model 154, the generative model 152, the controller engine 114, and / or the variant engine 112 can be trained utilizing reinforcement learning. In some of those embodiments, control model 154 and / or generative model 152 can be initially trained using other techniques and improved through reinforcement learning. For example, the generative model 152 is initially trained as described with respect to FIG. 2 and can be further trained through reinforcement learning.

In some of those embodiments, within the actor-critic algorithm, the actor controller engine 114 and control model 154 are considered "critics," and the mutant engine 112 and generative model 152 are considered "actors." Can be done. In general, actors generate variants and probe the environment with the variants. The environment can be, for example, search system 140. In general, critics accumulate evidence from the environment (for example, responses such as answer strings, or their ranked list) to generate global actions / judgments d, maintain global state s, and actors. Provides a reward signal r and a context c to.

The behavior of actors and critics can be driven by reinforcements on two different time scales. Actors can operate on a finer time scale (indexed by t'). At each step, the actor produces the following variants, subject to context. Critics accumulate evidence from the environment within the global state s. In some situations, the state at least gives the original query, the mutants generated, and the observations (eg, the search system response to the mutants generated), and the vector summary h used to feed the network. Includes (s = ({q _t , o _t } 1 _{.. T} , h _t )). Assuming a global state, critics make a global judgment d at each time step, release a response, or continue the cycle of mutagenesis and accumulation of more evidence. Critics also provide context to actors to coordinate mutation generation and reward signals. Critics directly model the value of state-action vs. "Q function" Q (s _t , d _t). This value of the Q function is passed to the actor as a reward signal. The Q function is trained using the global rewards defined for the response (eg, the response to the original query), and the sequence of decisions d made. Time-scale separation allows the two tasks of mutation generation and global decision-making to be modeled separately, but trained together to optimize end-to-end performance.

A terminal state is reached when critics release a response instead of continuing the cycle of mutagenesis and accumulation of more evidence. The action space for an actor can be defined as A: = {(a, <w>): a ∈ {question, answer}, <w> ∈ Strings}. In the above equation, a can be a variant probing the environment or releasing a response. The action is paired with the string <w> that defines the variant or answer (the released response). In some embodiments, the "probing the environment in a variant" action does not receive rewards, and the "responding action" receives rewards that are proportional to the quality of the response. Critics can learn a Q function that maps an action (a, <w>) to the expected return E [Gs] from the current state. If only the "action that emits a response" is rewarded, the expected return is ^{expressed as E [γ κ} R], where [0,1] is the discount and k is the number of iterations to the final state. , R is the final reward.

Q-function training can be achieved using the Monte Carlo Q-learning technique. Mutants can be sampled until the final state is reached, rewards are determined, and all intermediate predictions of the Q function can be updated towards ^{γ κ.} The actor produces variants, for example, as input, taking more features subject to the original query, the last variant, and, in some cases, the history of variants and responses, one or more distinctions. May include a sequence-to-sequence model that returns a variant of. Actors can be trained with the Monte Carlo policy gradient technique. The response set received from the environment acts as a memory of the answers that have been seen so far. The response set can be used to provide features for actors and Q functions and / or allow critics to return the answers seen in some intermediate iterations.

Next, with reference to FIG. 5, a flowchart showing a method 500 for training the generative model according to various embodiments disclosed herein is provided. For convenience, the operation of the flow diagram will be described with reference to the system performing the operation. The system may include one or more components such as one or more processors (eg, CPU, GPU, and / or TPU). The behavior of Method 500 is shown in a particular order, but this does not imply a limitation. One or more actions can be sorted, omitted, or added.

At block 552, the system selects a group of training instances. For example, when the generative model is trained in method 500 to be a multitasking model, the system may select the group to include training instances that demonstrate multiple types of variant generation. In addition, for example, if the generative model is trained to be personalized to a particular group of users, as an addition or alternative, the system will submit the training instance to a past query by a user who conforms to that particular group. Groups may be selected to include only training instances based on, or to include significant amounts (eg, more than half, more than 70%) of training instances. In addition, for example, if the generative model is trained to be personalized to a particular task, for example, as an addition or alternative, the system will make the training instance a training instance based on past submissions of queries related to the particular task. Groups may be selected to include only or to include significant amounts (eg, more than half, more than 70%) of training instances.

At block 554, the system selects a training instance for the group.

At block 556, the system applies the training instance input of the training instance as input to the generative model. The training instance input can include, for example, the terms and attributes of the original query (eg, the attributes of the user who submitted the original query), and the type values (indicating the type of variant of the original query).

At block 558, the system generates variants through a generative model based on the applied training instance inputs.

At block 560, the system determines an error for the training instance based on a comparison of the generated variants to the training instance output (ie, the variants shown within the training instance output).

At block 562, the system updates the generative model based on the error. For example, the error can be a gradient that is backpropagated through the generative model to update the generative model.

At block 564, the system determines if there are any additional raw training instances in the group. If so, the system goes to block 554 and selects additional training instances. The system then implements blocks 556, 558, 560, and 562 based on additional training instances.

In the iteration of block 564, if the system determines that there are no additional unprocessed training instances in the group (or other training criteria have been met), the system proceeds to block 566 and training ends.

Figure 5 shows a particular non-batch training technique, but understands that batch training (for example, when an error is determined and backpropagated based on a batch of training instances) can be used in training as an addition or alternative. I want to be done. Further, it should be understood that in various embodiments, the trained generative model based on Method 500 can be further trained according to the techniques disclosed herein. For example, a generative model can be further trained using reinforcement learning techniques, and can be further trained separately from a separate control model, but in combination with a separate control model. Further, if multiple generative models are generated, method 500 can be repeated with different selection criteria in block 552 to generate another model.

Then, with reference to FIG. 6, a flowchart is provided showing a method 600 of utilizing the generative model to generate one or more variants of the query, according to various embodiments disclosed herein. For convenience, the operation of the flow diagram will be described with reference to the system performing the operation. The system may include one or more components such as one or more processors (eg, CPU, GPU, and / or TPU). The behavior of Method 600 is shown in a particular order, but this does not imply a limitation. One or more actions can be sorted, omitted, or added.

At block 652, the system receives the query.

At block 654, the system selects a generative model from a plurality of candidate generative models. In some embodiments, the system selects a generative model based on the attributes of one or more of the users who submitted the query in block 652. For example, the system may choose a generation model based on the fact that the generation model is stored in relation to an attribute that matches one or more of the user's attributes. For example, generative models can be stored in relation to such attributes based on being trained based on training instances based on past query submissions of users with those attributes. In some embodiments, block 654 may be omitted (for example, only a single generative model may be available).

At block 656, the system applies the query token and additional values as input to the generative model. Various additional values may be applied, such as the attributes of the user who submitted the query, the temporal attributes, and / or the attributes of the search system response for the received query. As one particular example, additional values may include the expected task attributes of the user who submitted the query. The predicted task attributes can be predicted, for example, based on the content recently viewed by the user on the computing device, the user's stored calendar entries, and / or the user's electronic communications.

At block 658, the system generates one or more variants through a generative model based on the input applied.

At block 660, the system determines whether to generate another variant. In some embodiments, the system is based on the properties of the previously generated variant and / or the response from the search system for the previously generated variant. Decide whether to generate. For example, the system may decide whether to generate another variant based on whether the response to the previously generated variant was found by the search system and / or the quality scale of the response. For example, if no response is found and / or the quality scale fails to meet one or more quality criteria, the system may generate another variant.

If, at the iteration of block 660, the system decides to generate another variant, the system proceeds to block 662 and one or more additions that should be applied as inputs to the generative model in subsequent iterations of block 656. Update the value of. For example, the system may change the type value for the next iteration of block 658 to reflect the response to the variant, to reflect the variant generated in the latest iteration of block 658, and / or to reflect the response to the variant. Additional values can be updated for. The system then performs another iteration of block 656 with the updated additional values, and then proceeds to blocks 658 and 660.

If, in the iteration of block 660, the system decides not to generate another variant, the system proceeds to block 664 and provides output based on one or more of the generated variants. The output may include a search system response to one or more of the variants and / or one or more of the variants.

FIG. 7 is a flow chart showing how the generative model is used to generate one or more variants of a query, where the control model is used to control the generation of variants. For convenience, the operation of the flow diagram will be described with reference to the system performing the operation. The system may include one or more components such as one or more processors (eg, CPU, GPU, and / or TPU). The behavior of Method 700 is shown in a particular order, but this does not imply a limitation. One or more actions can be sorted, omitted, or added.

At block 752, the system receives the query.

At block 754, the system produces a control output via a control model based on the current state. For example, the current state can be based on the token of the current query, the search system response to the current query, and / or other features.

At block 756, the system decides whether to generate a variant of the received query based on the control output. In some embodiments, the first iteration of block 754 and block 756 may be omitted. In other words, in those embodiments, the system may decide to always generate variants (eg, to check the validity of the search system response to the received query).

If, in the iteration of block 756, the system decides not to generate a variant, the system proceeds to block 766 to provide the current search system response and / or output based on the mutant generated.

If, in the iteration of block 756, the system decides to generate a variant, the system proceeds to block 758.

At block 758, the system determines the reward signal and / or context based on the control output generated in the latest iteration of block 754. The reward signal can be obtained based on the learned Q function described herein, and the context can include, for example, a current state and / or a vector outline of the current state.

At block 760, the system generates variants through a generative model based on the received queries and reward signals and / or the context of block 758.

At block 762, the system determines the response to the variants generated at block 760. For example, the system may submit a variant to a search system and receive a response from a search system that responds to the variant. In some situations, the search system does not return a response and / or produces a "null", each of which indicates that a response (eg, an answer) is not available.

At block 764, the system updates the current state based on the variant and the response to the variant. The system then returns to block 754 to generate control output via the control model based on the current state, including updates to block 764. In this way, in subsequent iterations of block 764, previously generated variants and responses (ie, those generated in previous iterations of blocks 760 and 762) are considered in the next iteration of block 754. obtain. The system then returns to block 756 to determine whether to generate another variant of the received query based on the control output. When the system decides to generate another variant, the reward signal and context provided in the next iteration of block 758 will likewise be the previously generated variant and response (ie, blocks 760 and 762). Note that it can be conditional on (generated in the previous iteration). In this way, the next iteration of mutation generation in block 760 is affected by the resulting previously generated mutations and responses.

Then, with reference to FIGS. 8A and 8B, exemplary graphical user interfaces 800A and 800B are shown to provide output based on the variants generated according to the embodiments disclosed herein. The graphical user interfaces 800A and 800B may be presented on the client device 106 (eg, in a browser running on the client device 106 and / or in another application running on the client device 106).

In Figure 8A, the user provided query 891A: "Did leonardo da vinci paint mona lisa". In response, an output is provided that includes response 892A and also contains two variants 893A. Two variants 893A can be generated according to the embodiments disclosed herein. In some embodiments, each of the variants is selectable, and in response to the selection, the corresponding variant is submitted as a new query. In some embodiments, response 892A is also based on the variants produced according to the embodiments disclosed herein. For example, in some situations, response 892A can be a response to a variant of query 891A (a variant different from variant 893A), and / or response 892A can be to query 891A, It can be validated based on the response to the variant of the query (for example, by ensuring that those variants also produced a positive response).

In Figure 8B, the user provided the query 891A, "Did michelangelo paint the mona lisa." In response, an output containing response 892B "No" is provided. Optionally, box 895B in FIG. 8B may not be visible for display, but as an example of a variant that can be generated according to the techniques described herein to generate a "no" response 892B. The box 895B presented displays the original query (indicated by an "O") and an "Y" in parentheses to indicate that the response response was generated by the search system in response to the original query. including. For example, the response could be "yes, Michelangelo did paint the Mona Lisa." However, instead of providing a response response, multiple variants that are "supplementary" variants are generated to verify the accuracy of the response of the original query. Specifically, variants V1, V2, and V3 are generated. The search system generated an "no answer" response in response to each of those complementary variants, as indicated by the "N" in parentheses. Given that the answer is not available for those multiple supplements, the controller engine can determine that the "answer response" to the original query is inaccurate (because the supplement does not lead to any answer). .. As a result, the controller engine provides a "no" response 892B.

Examples of graphical interfaces are presented in Figures 8A and 8B, but as an addition or alternative, a query may be received based on the user's utterance input, and / or, as an addition or alternative, its variants and / or responses. It should be understood that it can be audibly provided for presentation to the user via the client device.

In situations where the systems described herein may collect or utilize personal information about a user, the program or feature may be user information (eg, information about the user's social network, social actions or activities, etc. Controls whether to collect occupations, user preferences, or the user's current geographic location, or whether and / or how to receive content from the content server that may be more relevant to the user. The user may be given the opportunity to control. In addition, certain data may be treated in one or more ways so that personally identifiable information is removed before it is stored or used. For example, a user's geography from which geolocation information is obtained so that the user's identity can be treated so that personally identifiable information about the user can be determined, or the user's specific geographic location cannot be determined. Geolocation can be generalized (city, ZIP code, state level, etc.). Therefore, the user may have control over how information is collected and / or used for the user.

FIG. 9 is a block diagram of an exemplary computing device 910 that can optionally be utilized to implement one or more aspects of the techniques described herein. The computing device 910 includes at least one processor 914 (eg, CPU, GPU, and / or TPU) that communicates with several peripheral devices via the bus subsystem 912. These peripheral devices may include, for example, a storage subsystem 924 including a memory subsystem 925 and a file storage subsystem 926, a user interface output device 920, a user interface input device 922, and a network interface subsystem 915. Input and output devices allow user interaction with the computing device 910. The network interface subsystem 915 provides an interface to the external network and is coupled to the corresponding interface device within other computing devices.

The user interface input device 922 includes a keyboard, a pointing device such as a mouse, trackball, touchpad, graphics tablet, a scanner, a touch screen built into the display, a voice recognition system, a microphone, and / or others. It may include an audio input device such as an input device of the type. In general, the use of the term "input device" shall include all possible types of devices, and methods for inputting information within the computing device 910 or over communication networks.

The user interface output device 920 may include a non-visual display such as a display subsystem, printer, fax machine, or audio output device. The display subsystem may include a flat panel device such as a cathode ray tube (CRT), a liquid crystal display (LCD), a projection device, or any other mechanism for creating a normal image. The display subsystem may also provide a non-visual display, such as through an audio output device. In general, the use of the term "output device" shall include all possible types of devices, and methods for outputting information from the computing device 910 to the user or another machine or computing device.

The storage subsystem 924 stores programming and data structures that provide some or all of the functionality of the modules described herein. For example, the storage subsystem 924 may include logic for implementing selected aspects of the methods described herein.

These software modules are generally executed by processor 914 alone or in combination with other processors. The memory 925 used in the storage subsystem 924 consists of a main random access memory (RAM) 930 for storing instructions and data during program execution, and a read-only memory (ROM) 932 for storing fixed instructions. May include some memory including. The file storage subsystem 926 may provide persistent storage for programs and data files, including hard disk drives, solid state drives, floppy disk drives and related removable media, CD-ROM drives, optical drives, or removable media. May include cartridges. Modules that perform the functions of some embodiments may be stored by file storage subsystem 926 in storage subsystem 924 or in other machines accessible by processor 914.

The bus subsystem 912 provides various components and subsystems of the computing device 910 to allow them to communicate with each other as desired. Although the bus subsystem 912 is schematically shown as a single bus, alternative implementations of the bus subsystem may use multiple buses.

The computing device 910 can be of various types, including workstations, servers, computing clusters, blade servers, server farms, or any other data processing system or computing device. Due to the ever-changing nature of computers and networks, the description of the computing device 910 shown in FIG. 9 shall be merely a specific example to illustrate some embodiments. Many other configurations of the computing device 910 are possible, with more or fewer components than the computing device shown in FIG.

106 Client device
110 query system
112 Mutant engine
114 controller engine
120 Generative model training engine
122 Training Instance Engine
140 search system
152 Generative model
152A generative model
153A encoder layer
154 Control model
154A decoder layer
155A Softmax layer
162 database
164 database
164A training instance
166 resources
800A graphical user interface
800B graphical user interface
891A query
892A response
892B response
893A variant
895B box
910 computing device
912 Bus subsystem
914 processor
915 Network Interface Subsystem
920 user interface output device
922 User Interface Input Device
924 Storage subsystem
925 memory
926 File storage subsystem
930 Main Random Access Memory (RAM)
932 Read-only memory (ROM)

Claims (36)

A method implemented by one or more processors
A step that receives the original query, wherein the original query is generated based on the user's user interface input through the client device.
The step of applying the token of the original query as input to the trained generative model,
A step of generating at least one variant of the original query based on the application of the token of the original query to the trained generative model.
A step of generating output based on at least one of the at least one variant and the response of at least one search system to the at least one variant.
In response to the original query, viewing including the step of providing said output for presentation via said client device,
Prior to the step of generating the at least one variant, the method is performed by a controller engine utilizing a trained control model, based on at least one response from the search system for the original query. A step of determining whether the form should be generated for the original query and a step of determining whether any variant should be generated for the original query.
The step of applying the at least one response feature as a controller input to the trained control model, and
A step of generating a controller output through the trained control model based on the controller input,
A method comprising the step of determining to generate the at least one variant based on the controller output. The method of claim 1, further comprising applying one or more attributes related to the user as part of the input to the trained generative model. The method of claim 2, further comprising generating at least one variant of the original query based on the one or more attributes to the trained generative model. The method of claim 2 or 3, wherein the one or more attributes include one or more of the user's location, the task the user is currently engaged in, and the weather at the user's location. .. Claims further include the step of applying one or more temporal attributes, including at least one of the current time, the current day of the week, and the current date, as part of the input to the trained generative model. The method according to any one of 1 to 4. Steps to determine the expected task for the user, and
As an input to the trained generative model
Further including the step of applying one or more task attributes of the predicted task for the user.
Luz step generates at least one variant of the original query, based on the application of the one or more tasks attributes for the trained generative model, according to any one of claims 1 to 5 Method. Luz step to determine the task to be the prediction of the user, based on one or more interactions with the user via the client device or additional client devices A method according to claim 6. Wherein one or more interaction, including calendar entries created by electronic communication, or the user sent by the previous SL user The method of claim 7. The absence steps to determine which tasks the expected users, based on calendar entries stored in electronic communication, or the user sent to the user A method according to claim 6. A step to generate a training instance that contains a training instance input and a training instance output.
The training instance input is
The first query token of the first query and
Including task attributes
The training instance output is
Contains the second query token of the second query
The training instance, along with the task attributes, is based on determining that the past submits of the first query followed by the past submits of the second query are related to the predicted task. Steps and steps generated as training instance inputs
The method of claim 6, further comprising training the generative model based on the generated training instance. A claim comprising the step of selecting a trained generative model from a plurality of trained generative models based on the trained generative model being trained based on past query submissions associated with the predicted task. The method according to any one of 6 to 10. The step of selecting a training instance generated based on the past query submission related to the predicted task, and
11. The method of claim 11, further comprising training the generative model based on the selected training instance. The step of determining that a group of two or more previously submitted queries is relevant to the predicted task, and
With the step of generating one of the training instances based on the previously submitted query of the group.
Further including the step of labeling the one of the training instances as related to the predicted task.
The step of selecting the training instance generated based on the previously submitted query related to the predicted task includes the step of selecting the one of the training instances based on the labeling. , The method of claim 12. Luz steps to determine that the group of the submitted query more than one previously associated with the task to be the expected, the computing-based actions previously carried out following the submission of the submitted query 13. The method of claim 13. A trained generative model from multiple trained generative models based on the trained generative model being trained based on past query submissions of a group of users with one or more attributes in common with the user. Steps to select and
The method of any one of claims 1-5, further comprising applying the token of the original query as input to the selected trained generative model. The method according to any one of claims 1 to 15, wherein the trained generative model is a deep neural network model having one or more storage layers. Luz step to produce the variants of the query comprises the step of producing the variants based on the learned parameters of the trained generative model,
A step of applying additional inputs to the trained generative model, wherein the additional inputs make at least one of the token of the original query and the variant token of the variant of the original query. Including, steps and
Based on the additional input, a step of generating an additional variant of the original query through the trained generative model, wherein the additional variant is the variant and the original query. Differently, the step of generating the additional variant of the original query comprises the step of generating the additional variant based on the learned parameters of the trained generative model.
A step of determining an additional variant response for the additional variant of the original query based on the submission of the additional variant of the original query to the search system.
A step of generating an output based on at least one of the mutant response and the additional mutant response.
The method of any one of claims 1-16, further comprising the step of providing the output for presentation through the client device in response to the original query. The trained generative model is trained to generate a plurality of types of query variants, wherein the variant is the first type of the plurality of types of query variants, the additional variant. The method of claim 17, wherein is the second type of the plurality of types of query variants. The first type is one of a synonym query, a supplementary query, a generalized query, a normalized query, an implication query, a specification query, a clarification query, and a language translation query, and the second type is Claim 18, which is another one of the synonymous query, the supplementary query, the generalized query, the normalized query, the implication query, the specification query, the clarification query, and the language translation query. The method described. The variant is generated as the first type through the trained generative model based on the application of the first type value as part of the input to the trained generative model, said addition. Is generated as the second type through the trained generative model based on the application of the second type value as part of the additional input to the trained generative model. , The method of claim 18. 17. The method of claim 17, wherein the additional input applied to the trained generative model comprises said token of the original query. 21. The method of claim 21, wherein the additional input further comprises said variant token of said variant of said original query. 21. The method of claim 21, wherein the additional input further comprises a mutant response feature based on the mutant response for the variant of the original query. It further comprises the step of determining whether to provide the variant response as the output or to generate the additional variant based on the variant response prior to generating the additional variant.
Luz step to generate the additional mutations shape, instead of providing the variant response, provided that determines to generate the additional variants A method according to claim 17. Or providing the variant response as the output, or the additional absence steps to determine whether to generate the variants is further based on the variant of the original query, the method according to claim 24. On the basis of the variant response, or providing the variant response as the output, or absent steps to determine whether to generate the additional mutations shape,
The step of applying the variant response feature as a controller input to the trained control model, and
A step of generating a controller output through the trained control model based on the controller input,
Based on the controller output, and determining to generate the additional variants The method of claim 24. On the basis of the variant response, or providing the variant response as the output, or absent steps to determine whether to generate the additional mutations shape,
26. The method of claim 26, further comprising applying the variant of the original query as part of the input to the trained control model. 26. The method of claim 26, wherein the trained control model is trained on reinforcement learning based on rewards determined based on past interactions with the search system or additional search systems. 26. The method of claim 26, wherein the trained control model is trained in combination with the trained generative model, although it is separate from the trained generative model. 26. The method of claim 26, wherein the trained control model is a feedforward neural network or a recurrent neural network. The method of any one of claims 17-26, wherein the input further comprises one or more attributes associated with the user, and the additional input further comprises the one or more attributes. .. A step of sending an original request to the search system over a network, wherein the original request includes the original query.
Further including the step of receiving the original response from the search system in response to the original request.
And Luz steps to apply the token of the original query as an input to the trained generation model, the and away step to generate variants is based on the original response from the retrieval system, it claims 17 to 31 The method according to any one of the above. The method according to any one of claims 1 to 32, wherein the trained generative model is a deep neural network model having a storage layer. The method of any one of claims 1-33, further comprising the step of training a sequence-to-sequence neural machine translation model to generate the trained generative model. A system comprising one or more processors and memory operably coupled to one or more processors, configured to perform the method of any one of claims 1-34. A computer program configured to perform the method according to any one of claims 1-34 when executed by a processor.

Images