KR20200003132A - Generate query variants using a trained generation model - Google Patents

Abstract

Systems, methods, and computer readable media related to generating query variants for a submitted query are disclosed. In many implementations, query variants are generated using a generation model. The generation model is productive in that it can be used to actively generate variations of the query based on the application of tokens of the query to the generated model and optionally based on the application of additional input features to the generation model.

Description

Generate query variants using a trained generation model

The present invention is directed to generating query variants using a trained generation model.

Rule-based rewrites of search queries have been used in query processing components of search systems. For example, some rule-based rewrites can generate a rewrite of a query by removing certain stop words from the query, such as "the", "a", and so on. The rewritten query can then be submitted to the search system and the search results responsive to the rewritten query can be returned.

In addition, collections of similar queries have been used in the search system to recommend additional queries (eg, "people also search for X") associated with the submitted query, for example. Queries similar to a given query are often determined by search clustering. For example, for the query "funny cat pictures", a similar query of "funny cat pictures with subtitles" may be determined based on similar queries that users frequently submit after submitting "funny cat pictures". . Thus, similar queries for a particular query are often predefined.

Implementations herein relate to systems, methods, and computer readable media related to generating query variants for a submitted query. In many implementations, query variants are generated at run time using a trained generation model. The generation model is productive in that it can be used to actively generate variations of the query based on the application of tokens of the query to the generated model, and optionally based on the application of additional input features to the generation model. In this way, even if the generation model is not trained based on the query, the generation model can be used to generate variant (s) of any query. Thus, the generation model can be used to generate new queries and variants for so-called "tail" queries (ie, queries whose submission frequency and / or submission quantity are below a threshold). As a result, rich query input makes it possible to more efficiently identify relevant results, thus making queries more efficient. For example, queries are not simply excluded due to low submission frequency and / or submission quantity. If the initial query does not produce relevant results, there may be an improved efficiency in the speed at which the relevant results can be obtained since the user does not have to resubmit the modified query. The disclosed implementations allow multiple query variants to be tested automatically. Convergence of the results can also be ensured through the training of the model used to generate the variations, thus improving efficiency through the generation of target query variants rather than processing multiple queries simultaneously. The use of technical resources required for query processing, including processing power and power consumption of processors implementing the disclosed methods, is therefore optimized through implementations of the present invention.

In some implementations, the generation model is a neural network model, such as a neural network model with one or more "memory hierarchies". The memory hierarchy includes one or more repeating neural network (RNN) units, such as long term memory (“LSTM”) units and / or gated repeat units (“GRUs”).

In some implementations where the generation model is a neural network model with memory layers, the generation model is a sequence-to-sequence model. For example, a sequence-to-sequence model may allow the tokens of a query to be applied as input to the model (eg, token-to-token-based or combination-based), and encoding of tokens may be generated across the layers of the network. Can be. In addition, the generated encoding can be decoded through additional layers of the network, with the resulting decoding indicating (directly or indirectly) modification of the query. For example, the resulting decoding can be applied to the softmax layer (s) of the network to generate a modification of the query. In some versions of these implementations, the generation model has the same or similar structure as the sequence-to-sequence neural machine translation model and is trained using query variant specific training data. The query variant specific training data may be, for example, query pairs each having "clicks" in the same documents (eg, training for generating equivalent query variants); Consecutively submitted query pairs (eg, training to generate subsequent query variants); And / or original standard query pairs (eg, training to generate standard query variants). Such a model can be optionally pre-trained based on translation training data.

In some implementations, the generation model is trained as a “multitask” model in that it is trained to produce a query variant of any one of multiple types of query variants. In some of these implementations, the type of query variant to be generated for a particular pass of the generation model may be indicated based on the type value input applied to the model in that particular pass. Types of query variants are, for example, an equivalent query, subsequent query, generalized query, normalized query, language translation query, implicit query, specification query, and / or explanatory query (i.e., providing output to the user to prompt explanation). Query). Additional and / or alternative types may be defined, including greater or smaller granularity. In the training of the reproductive model, various types of training data can be utilized, where each instance of the training data includes a type value input that indicates the type of query variant for that instance and can be used as a training instance input during training. do. In some of these implementations, training a multitasking model in this manner can utilize information sharing across various types of training data, which can lead to more powerful performance of the trained multitasking model.

Once trained, the multitask model generates a first type of variation for the query in the first pass (eg, based on applying the first type value as input), and (eg, Based on applying the second type value as input) may be used to generate a second type of variation for the query in the second pass. Further types of further variants may be created in further passes. As described herein, the amount of additional passes generated using the multitasking model may vary from query to query. For example, the amount of additional passes may be controlled in an ad hoc manner, for example, based on the deformation (s) generated in the path (s) and / or response (s) to such modification (s). In addition, subsequent passes may produce variations of the same type as previous variants created in previous passes. In some of these situations, as described herein, the subsequent pass may be based on previous modifications (eg, the previous variation itself and / or response (s) to the previous variation) and / or other previous variations. Can be used, where the deformation of the subsequent pass can be different than the previous deformation.

In some implementations, multiple variants of the original query are generated using a generation model, each of the multiple variants is submitted to a search system, and the corresponding response (s) are received for each of the multiple variants. The output can be generated based on one or more responses, and the output is provided in response to the original query. For example, the output may include a "best" response (eg, indicated by response scores provided by a search system), a number of "best" responses and / or variations and corresponding response (s) (eg, If the variant is of a subsequent type). In this and other ways, the response (s) for the modification (s) of the original query can be used to provide an output in response to the original query, which output directly answers the original query. In addition, the response (s) to the modification (s) of the original query may be used to verify / confirm the response (s) to the original query and / or the response (s) to other modification (s). For example, the accuracy of the "answer" to the original query may be determined depending on whether positive answers are provided for variations of the original query. For example, the accuracy of the "answer" to the original query is equivalent, depending on whether other positive answers are provided for subsequent types of variation (s) and / or positive similar answers (similar to the original query answer). , Generalization and / or language translation type (s) may be determined according to availability. In this and other ways, the unproven / unguaranteed response (s) may be determined and not used at the provided output and / or used when provided at the provided output (eg, marked "potentially fake"). Can be flagged as unconfirmed.

In some implementations and / or situations, multiple responses are returned by the search system in response to the modification. In some other implementations and / or situations, the search system provides a single response in response to the modification. In some of these implementations, the single response includes an “answer” (eg, the response considered by the search system is the answer to the variant) or an indication that the answer is unknown. In other implementations, an indication that the answer is unknown can be a lack of a response by the search system. The search system may be a search system that operates across multiple domains or is specialized for one or more specific domains (eg, an online shopping domain). The response returned from the search system may include, for example, search results (e.g., content snippets from the document and links to the document), answers (e.g., content deemed a formal answer by the search system), It can be an image, video or knowledge graph entity, a "null" response (eg, a "no answer" response). In some situations, the generated variant may additionally or alternatively be provided to the user as an output (submitted to the original query) to prompt the explanation, and clarify the user interface input provided by the user in response to the prompt. Can be used as a "response" to the transformation. This user-provided response can be used to influence the generation of further variations. For example, such user-provided response can be used to generate a context vector that is passed to the generation model in further iterations of generating the transformation.

In some implementations, multiple generation models can be generated, each generation model being trained based on training data based on past query submission of a group of unique users. For example, the first generation model may be generated based on training data based on past query submissions of users with attributes A and B. A second generation model may be generated based on training data based on past query submissions of users with attributes B and C. For a user's submitted query with attributes B and C (but not A), the second generation model is transformed to the user because user attributes B and C match those used to train the second generation model. Can be selected for use in generating them (without selecting the first generation model). In this way, a generation model can be selected from a plurality of available generation models so that the selected generation model is tailored to the user's attributes. As a result, query variants more suitable to the user may be generated using the selected generation model. For example, very different variations can be created for a scientific researcher compared to a freelance writer.

In some implementations, several generation models can be generated, each generation model being based on training data based on past query submissions related to particular attributes, such as the user's specific attributes, specific temporal attributes, and / or other attributes. Trained on the basis of For example, the first generation model may be generated based on training data based on past query submissions related to the online shopping task. For example, past query submissions are shopping-centric search results based on users who select shopping content (eg, specific advertisements) with respect to those submitted based on submissions to an online shopping search system. Can be identified based on the users who completed the transaction after submission. The second generation model may be generated based on training data based on past query submissions associated with different specific attributes. For example, a second generation model may be generated based on training data based on past query submissions related to a trip to a location task (eg, any location, any restaurant location, meeting location, etc.). For example, past query submissions may be identified based on those submitted before and / or during the trip to the location, based on being submitted in proximity to the scheduled calendar entry. For a user's submitted query, the user's task can be predicted, and a generation model corresponding to the selected predicted task can be provided to generate variants for the submitted query. For example, if a user's calendar entry and / or electronic communication indicates that the user is traveling to or will soon be traveling to a location, the second generation model in the previous example is based on the model associated with the trip to the location task. Can be selected. In this way, the generated model can be selected from a plurality of available generation models, such that the selected generation model is tailored to the user's task such that the predicted task will or will be involved. This allows query variants that are more suitable for the user's current task using the selected generation model. As described above and elsewhere herein, in various implementations, the generation model may be a multitasking model and may generate various heterogeneous types of query variants. Some of these various implementations can use the generation model to generate variants that can expand the user query and explore multiple paths that extend the query. Such variations may be provided for presentation to the user (eg, optionally without first issuing a query based on those variations) to allow the user to explore various paths to expand the query. Additionally or alternatively, the response to these variations may be obtained from a search system and provided to present to the user so that the user may explore the various responses to extensions to the query.

Some implementations described herein can be used to generate variations of queries submitted by users that may have difficulty formulating queries (eg, due to physical damage). For example, a query can be formulated by a user using a gaze guidance (or other low effort) user interface input and query variants created in accordance with the techniques described herein.

As described herein, a generation model may be used to actively generate variations of a query based on applying tokens of the query to the generation model and optionally based on applying additional input features to the generation model. . In some of these implementations, additional input features may include attributes, temperature attributes, and / or other features associated with the user who submitted the query. Attributes associated with a user may include, for example, the user's location (eg, KY Lewisville; "restaurant"; southeastern United States), tasks associated with the user (eg, cooking, car repairs, travel planning), and / Or weather at the user's location. The task associated with the user may be a task that will be engaged by the current user or by the user. In some implementations, the task may include, for example, the user's stored calendar entry, the user's electronic communication (eg, chat messages or other communication sent to or by the user), a past query submitted by the user. Prediction based on various signals such as Time attributes may include, for example, current time, current day of week, and / or current date. In this way, query variant generation using the generation model may be personalized to the user and / or current context based on the application of additional input features to the generation model.

In some implementations, the generation model can be used to generate advertisements or other content provided to the client device that generated the query, based on variations of the query and such content assigned to one or more variations. In some of these implementations, the variations generated by the generation model can be tailored to the client device and / or user of the client device using techniques such as those described herein. For example, the generation model may be selected based on the user's attribute (s) and / or the attributes associated with the user may be provided as input to the generation model and used to generate variations.

In some implementations, the generation model is used to generate a variant (and / or token of a variant) of the original query in each of the plurality of time steps. In some of these implementations, at a particular time step, whether a variation is generated and / or which variation is generated may be based on the current state feature (s). The current state feature (s) can be, for example, search system response (s) for the original query; Search system response (s) for the modification (s) of the original query generated in the previous time step (s); Modification (s) of the original query generated in the previous time step (s); User response (s) to the modification (s) of the original query (eg, an explanatory variant provided as a prompt to the user); And / or features based on the original query. In this way, the generation of a variant for a query during the session is dynamically based on the variant (s) of the query previously generated during the session, the response (s) for the previously generated variant (s) and / or the original query. May be affected. For example, in some implementations, one or more of these current state features can be used to determine if additional variations should be generated, and, alternatively, transfer in response to the original query without generating additional variations. It may be used to determine if the response (s) to the variant (s) (and / or original query) should be provided instead. Further, for example, in some additional or alternative implementations, one or more of these current state features may be applied (directly or indirectly) as input to the generation model to affect the generation of deformations in a time step. . For example, a vector summary of the current state features can be generated and applied as input to the generation model to affect the generated transformation.

In some implementations, the trained control model determines, in each of the plurality of time steps, whether a deformation is generated and / or to determine features to be provided as input to the generation model to influence the deformation generation in the time step. To be used. For example, the trained control model can be a feedforward neural network model or an iterative neural network (RNN) model. Current state features can be applied as input to a trained control model, through which the features to be provided to the generation model to affect whether additional deformations are generated and / or deformation generation (if additional deformations are generated). Generate value (s) indicative of the (s) (eg, a vector summary of current state features and / or compensation signal). In this way, the control model can act as a "criteria" and the generation model acts as an "acter" in the actor critique environment. Thus, a trained control model can be used to determine whether additional variations are generated and / or affect feature (s) affecting such generation based on the observed current state feature (s). In this way, the trained control model can control the quantity of additional variations created for a particular query. The amount of variants generated by this control may vary from query to query, for example, the control model is based on the variant (s) generated in previous iterations for a given query and / or the response to such variant (s). This is because it dynamically determines the amount of iterations of variant generation for a particular query. Such dynamic control often results in relatively large amounts (e.g., more than 5, more than 10, or more than 15) of modifications being generated and / or relatively large amounts of response to such variations being considered. I understand.

In some implementations, the control model and / or generation model can be trained based at least in part on reinforcement learning. In some of these implementations, the control and generation models are trained separately but in combination with each other. When training a control model and / or a generation model based on reinforcement learning, the generated modifications may be submitted to a search system, and responses from the search system (and optionally no responses) may represent a reward. For example, for a 'response' response that is a response to a query variant, relative to the quality of the response response (eg, as indicated by the response score provided by the search system for a "answer" response) Rewards may be assigned (or otherwise). In some examples, no reward is provided in response to a query variant and / or when no response is considered to be a "answer" response (eg, based on output from a search system), assign any reward Will not be. In other words, only the last "answer" response will be rewarded, and the intermediate actions are updated based on this reward (eg Monte-Carlo Q learning style). In this way, Q function learning or other reinforcement learning may occur based on rewards based on the responses provided by the search system interacting during the reinforcement learning. In implementations of reinforcement learning described herein, a state at a particular time step is represented by one or more state features (eg, those described above), and the behavior is a query variant (ie, creates additional query variants). ) Or provide a "answer" response. Each action in the action space can be paired with a string that defines the corresponding question or "answer" response.

In some implementations, a method is provided that is implemented by one or more processors that includes receiving an original query generated based on a user interface input of a user through a client device. The method further includes applying as inputs to the trained generation model, tokens of the original query, and one or more attributes associated with the user. The trained generation model is a sequence-to-sequence deep neural network model with one or more memory layers. The method further includes generating at least one variant of the original query based on the application of the tokens and one or more attributes for the trained generation model. The method further includes generating an output based on at least one of at least one variant and at least one search system response to the at least one variant. The method further includes providing an output for presentation via the client device in response to the original query.

In some implementations, a method implemented by one or more processors can include receiving an original query; Applying tokens of the original query as input to the trained generation model; And generating a plurality of variations of the original query based on the application of tokens of the original query to the trained generation model. The original query may be generated based on the user interface input of the user through the client device. Each of the generated variants is different from the original query and creating the variants involves creating the variants based on the learned parameters of the trained generation model. The trained generation model is trained to produce several types of query variants, and the generated variants are the first of the many types of query variants and the second type of the many types of query variants A second variant that is phosphorus. The method comprises: generating an output based on at least one search system response for at least one of the plurality of variants and / or at least one of the plurality of variations; And in response to the original query, providing an output for presentation via the client device.

In some implementations, a method implemented by one or more processors includes receiving an original query generated based on a user interface input of a user through a client device. The method selects a trained generation model from a plurality of trained generation models based on the trained generation model trained based on past query submissions of a group of users with one or more attributes in common with the user. It further comprises a step. The method includes applying tokens of the original query as input to the selected trained generation model; Generating at least one variant of the original query based on the application of tokens of the original query to the trained generation model; And generating an output based on at least one variant and / or at least one search system response for the at least one variant. The method further includes providing an output for presentation via the client device in response to the original query.

In some implementations, a method implemented by one or more processors can include receiving an original query; Applying tokens of the original query as input to the trained generation model; And generating a variant of the original query for the trained generation model based on the input. The original query may be generated based on the user interface input of the user through the client device. Variations generated over a trained generation model are different from the original query, and generating a variant of the query includes generating a variant based on learned parameters of the trained generation model. The method includes determining a modification response to a modification of the query based on submitting a modification of the query to a search system; Applying additional input to the trained generation model; And generating further modifications of the original query to the trained generation model based on the further input. The additional input applied to the trained generation model includes at least one of tokens of the original query and variant tokens of the modification of the original query. The generated further variant is different from the variant and the original query, and generating further variant of the original query includes generating further variant based on the learned parameters of the trained generation model. The method further includes determining an additional modification response to the further modification of the original query based on submitting further modifications of the original query to the search system. The method includes generating an output based on a modification response and / or a further modification response; And in response to the original query, providing an output for presentation via the client device.

In some implementations, a method implemented by one or more processors includes receiving an original query generated based on a user interface input of a user via a client device. The method includes determining a predicted task for a user; And applying, as input to the trained generation model, one or more task attributes of the original task's tokens and the predicted task for the user. The method further includes generating at least one variant of the original query based on applying the tokens and one or more task attributes to the trained generation model. The method comprises: generating an output based on at least one variant and / or at least one search system response for at least one variant; And in response to the original query, providing an output for presentation via the client device.

A method implemented by one or more processors includes receiving, via a client device, an original query generated based on a user interface input of a user. The method further includes determining a predicted task for the user. The method further includes selecting a trained generation model from the plurality of trained generation models based on the trained generation model trained based on past query submissions associated with the predicted task. The method includes applying tokens of the original query as input to the selected trained generation model; Generating at least one variant of the original query based on the application of tokens of the original query to the trained generation model; And generating an output based on at least one variant and / or at least one search system response for the at least one variant. The method further includes providing an output in response to the original query.

Various implementations disclosed herein may be embodied by a processor (eg, central processing unit (CPU), graphics processing unit (GPU), and / or TPU (Tensor) to perform a method such as one or more methods described herein. One or more non-transitory computer readable storage media storing instructions executable by the unit. Still other various implementations can include a system of one or more computers that include one or more processors operable to execute stored instructions to perform a method such as one or more methods described herein.

It is to be understood that the combination of the foregoing concepts and additional concepts described in more detail herein are considered to be part of the subject matter disclosed herein. For example, all combinations of claimed subject matter appearing at the end of the present invention are considered to be part of the subject matter disclosed herein.

1 is a block diagram of an example environment in which implementations disclosed herein may be implemented.
2 illustrates an example of training a generation model in accordance with implementations disclosed herein.
3 illustrates an example of using a generation model to generate one or more variations of a query.
4 shows another example of using a generation model to generate one or more variants of a query, wherein a control model is used to control the generation of variants.
5 is a flow diagram illustrating a method of training a generation model in accordance with implementations disclosed herein.
6 is a flow diagram illustrating a method of using a generation model to generate one or more variations of a query.
7 is a flow diagram illustrating a method of generating one or more variations of a query using a model of generation, wherein a control model is used to control the generation of variations.
8A and 8B each show an example graphical user interface for providing output based on variant (s) generated in accordance with implementations disclosed herein.
9 illustrates an example architecture of a computing device.

1 illustrates an example environment in which implementations disclosed herein may be implemented. The example environment of FIG. 1 includes client device 106, query system 110, search system 140, generation model training engine 120, and training instance engine 122. Such systems and engines may each be implemented in one or more computing devices that communicate, for example, via a communication network. The communication network may include a wide area network (WAN), such as the Internet, one or more intranets, and / or one or more bus subsystems. The communication network may optionally use one or more standard communication technologies, protocols and / or interprocess communication technologies.

The query system 110, the retrieval system 140, the generation model training engine 120, and the training instance engine 122 may be implemented with and / or have the systems, components, and techniques described herein. Example components that the technologies may interface with. The operations performed by one or more systems 110, 140 and engines 120, 122 of FIG. 1 may each be distributed across several multiple computer systems. In some implementations, one or more aspects of the systems 110, 140 and engines 120, 122 can be combined in a single system and / or one or more aspects can be implemented in the client device 106. For example, in some of these implementations, aspects of query system 110 may be combined with aspects of search system 140.

A user of client device 106 may formulate a query through client device 106 by providing user interface input through one or more user interface input devices of client device 106. Client device 106 submits a query to query system 110. In some situations, the query is in text form. In other situations, the query may be submitted in audio and / or other form and may be converted to text form by the query system 110 (or other component).

For the received query, the query system 110 generates one or more variations of the received query, and causes the output to be provided to the client device 106, the output based on the one or more variations. In some implementations, the output includes one or more variations to be provided as alternative variations proposed for consideration by the user. In some implementations, the output further or alternatively includes content based on one or more responses from search system 140, where the response (s) are based on submitting one or more variations to search system 140. do. Search system 140 may determine responses based on access of one or more resources 166 and may use various techniques, such as traditional information retrieval techniques. Content based on the response (s) may be, for example, graphical and / or audible “answers” or other search results based on (eg, the same) based on the response. When content based on the response (s) is provided, the query system 110 can provide the content directly to the client device 106 or allow the search system 140 to provide the content to the client device 106. Can be. In some implementations, query system 110 and search system 140 can optionally be controlled by the same party and / or operate in cooperation with each other. Additional and / or alternate output may be provided based on generated variations, such as an advertisement assigned to a variant created in one or more databases.

In FIG. 1, query system 110 includes a transformation engine 112 and a controller engine 114. In some implementations, one or more aspects of the transformation engine 112 and the controller engine 114 can be combined and / or implemented in a component separate from the query system 110, such as the client device 106. In some implementations, the controller engine 114 can be omitted.

Transform engine 112 uses one or more trained generation models 152 to generate one or more query variants for a submitted query. In some implementations, deformation engine 112 includes one or more CPUs, GPUs, and / or TPUs that operate via trained generation models 152. The transformation engine 112 generates the transformation for the submitted query by applying the tokens of the query as input to one of the generation models 152 and generating a modification to the generation model based on the input. In many implementations, when generating a variant, the deformation engine 112 further applies additional input features as input to the generation model and generates a variant based on the additional input features.

In some implementations, additional input features can include attributes, temporal attributes, and / or other features associated with the user who submitted the query. For example, when generating a variation on the original query, the transformation engine 112 may, as input to one of the generation models 152, tokens of the original query, attributes of the user who submitted the query (eg For example, the user's location, the work the user participated in, and temporal attributes (eg, current day of the week, current time) can be applied, and create a variant to the generation model based on the applied input.

In some implementations, additional input features that are applied in a particular iteration that produces a variation on the original query may, additionally or alternatively, modify (s) and / or the original query generated in the previous iteration (s). Include features based on the search system response (s) for the variation (s). For example, when generating a variation on the original query, the transformation engine 112 may generate the variation in each of a number of time steps. At a particular time step, the transformation engine 112 may include, as input to one of the generation models 152, a search system response (s) for the original query; Search system response (s) for the modification (s) of the original query generated in the previous time step (s); Modification (s) of the original query generated in the previous time step (s); And / or apply features based on the original query. In this way, the generation of a variant of a particular time step may be influenced by previously generated variant (s), response (s) to previously generated variant (s) and / or original query.

In some implementations, additional input features applied in a particular iteration that produces a variation on the original query can additionally or alternatively include a type value. For example, in some implementations, one of the generation models 152 can be a “multitask” model, which is trained to enable the generation of any of multiple types of query variants. In some of these implementations, the transformation engine 112 may apply, as input to one of the generation models 152, a type value that indicates the type of query variant to be generated. Query variants types may include, for example, an equivalent query, a subsequent query, a generalized query, a normalized query, a language translation query, and / or an implicit query. In some implementations, the deformation engine 112 generates multiple variations of the individual types using the same generation model by selecting a different type value in each of the plurality of iterations generating the variation.

In some implementations, multiple generation models 152 are accessible to deformation engine 112, and deformation engine 112 can generate modification (s) for a submitted query based on one or more parameters. Select a subset of one or more multiple generation models 152. For example, a number of generation models 152 can be provided, each generation model being trained based on training data based on past query submission of a group of unique users. For example, the first generation model may be generated based on training data based on past query submissions of users with attributes A and B. The second generation model may be generated based on training data based on past query submissions of users with attributes B and C. For a user's submitted query with attributes B and C (but not A), transformation engine 112 matches the query because user attributes B and C match those used to train the second generation model. In generating the transforms for the second generation model, the second generation model may be selected (rather than the first generation model).

The generation model training engine 120 and training instance engine 122 are also shown in FIG. 1. The training instance engine 122 creates a training instance and stores the training instances in the training instancetm database 164. For example, training instance engine 122 may generate a plurality of training instances based on submitted query database 162 storing past query submissions of multiple users. The generation model training engine 120 trains the generation models 152 based on the stored training instances of the database 164. As described herein, in some implementations, one or more generation models 152 can be further trained using reinforcement learning techniques that do not optionally rely on training instances of training instance database 164. Further description of the implementation of engines 120, 122 and databases 162 and 164 is provided below in the description associated with FIG. 2.

The controller engine 114, when provided, operates in conjunction with the deformation engine 112 and controls whether the deformation engine 112 generates deformation; And / or generate and provide parameters to the deformation engine 112 that affect the deformation generation. The controller engine 114 optionally uses one or more trained control models 154 to control whether the deformation engine 112 generates deformations and / or to generate parameters that affect deformation generation. In some implementations, the deformation engine 112 includes one or more CPUs, GPUs and / or TPUs that operate via the trained control models 154.

In some implementations, the controller engine 114 determines, for a submitted query, whether any variations will be generated by the transformation engine 112 for the submitted query. For example, the controller engine 114 may make such a determination based on the submitted query itself and / or based on the response (s) (if any) from the search system 140 for the submitted query. have. For example, the controller engine 114 may not respond if the response is not returned by the search system 140 or if the quality of the returned response is insufficient (eg, the search system provides a score to satisfy the threshold). Only if it does not). In some of these implementations, the controller engine 114 applies the features of the response (s) to the submitted query for one of the tokens and / or control models 154 of the submitted query, and variants are generated. Generate outputs for control models 154 that indicate whether or not to be. In some further or alternative implementations, the controller engine 114 applies the features of the tokens and / or response (s) of the submitted query to one of the control models 154, and when generating a variant ( The result is an output to the control models 154 provided to the deformation engine 112 to apply as input to the generation model.

As described herein, in some implementations, the transformation engine 112 generates a variation of the submitted query in each of a number of time steps. In some of these implementations, the controller engine 114 determines when the strain generation should be stopped. That is, whether the deformation engine 112 generates a deformation at a particular time step may be determined according to authentication from the controller engine 114. In addition, the controller engine 114 may provide, for each time step, features that affect deformation generation in the time step. The controller engine 114 may use at least one of the one or more control models 154 when determining whether to stop the deformation generation and / or when generating features that affect the deformation generation.

As one example, the controller engine 114 may include, as input to one of the control models 154, search system responses to the original query; Retrieval system response (s) for the modification (s) of the original query generated by the transformation engine 112 in the previous time step (s); The modification (s) of the original query generated by the transformation engine in the previous time step (s); And / or apply features based on the original query. The controller engine 114 may generate an output for the control model based on the input applied, and use the output to determine whether to instruct the deformation engine 112 to generate additional deformation or to stop generating the deformation. If the variant generation is stopped, the controller engine 114 may instead provide a previously generated variant and / or a response to the previously generated variant as an output that is a response to the submitted query. In this way, controller engine 114 may act as a "computer" and transformation engine 112 may act as an "acter" in an actor-critical environment. Further description of the implementation of the controller engine 114, one of the control model (s) 154, and the interactions of the controller engine 114 with the deformation engine 112 is described below with respect to FIG. 4.

Referring to FIG. 2, an example of training the generation model 152A among the generation models 152 is shown. Training instance 164A is retrieved from training instances database 164. Training instance 164A is based on training instance engine 122 (FIG. 1), for example, based on a pair of queries previously submitted by a user and stored in submitted query database 162 (FIG. 1). Can be generated by As an example, a pair of queries may be a user's initial query, "What did Roger Moore drive Aston Martin in the faders," and "What difference did Roger Moore drive in the faders?" And a subsequent query (i.e. immediately after the initial query) of the user. As another example, a pair of queries may include a user's initial query, "Is Leonardo da Vinci drew a Mona Lisa" and a follow-up query from the user, "Who asked Leonardo da Vinci to draw the Mona Lisa" (a subsequent type related to the initial query). It may include.

Training instance 164A includes a training instance input that includes a query (eg, a pair of initially submitted queries), attributes, and type. The attributes may include, for example, the attribute of the user who submitted the query, the temporal attributes of the query (eg, day of submission), and features of the search system response (s) for the query. The type may be a type value indicating the type of variant included in the training instance output. In some implementations, the type can be assigned by human labeling or based on the characteristics of the query pair used to generate the training instance 164A (eg, of temporal separation of submissions of queries of the query pair). Size, based on a comparison of search system responses to queries of the query pair), and can be inferred by the training instance engine 122. Training instance 164A also includes a training instance output that includes a modification (eg, after the pair of submitted times).

The generation model training engine 120 applies the training instance input of the training instance as input to the generation model 152A. The generation model training engine 120 further generates an output for the generation model 152A based on the applied input of the generation model 152A and the currently learned parameters. The generation model training engine 120 also generates a slope further based on the comparison of the generated output with the training instance output of the training instance 164A, and updates the generation model 152A based on the slope (eg, Backpropagates the slope for the entire generation model 152A).

When generating an output based on the applied input, the generation model training engine 120 may apply all or part of the input to the encoder layers 153A of the generation model 152A and to the encoder layers 153A. Can generate an encoding for the For example, the tokens of the original query of the input can be applied to the encoder layers 153A. Engine 120 may further apply encoding to decoder layers 154A of generation model 152A and generate a decoding of the encoding for decoder layers 154A. The engine 120 may then apply the generated encoding to the softmax layers 155A and generate output through the softmax layers 155A based on the application of the generated encoding. In some implementations, engine 120 applies attributes and / or type of input to other layers and / or as encoder side 153A, decoder layers 154A and / or softmax layers (“side input”). Applies to one of 155A). In some of these implementations, engine 120 applies the attributes and / or type to other layers downstream of encoder layers 153A but upstream of decoder layers 154A.

2 shows only a single training instance 164A, however, it is understood that many additional training instances will be used in the training generation model 152A. In some implementations, it is noted that the single training instance 164A and additional training instances are selected such that the generation model 152A is trained to be specifically adapted to certain attributes. For example, generation model 152A may be trained to select (or bias towards training instances) only training instances that are generated based on past submissions of users with particular attribute (s). For example, the attributes of users explicitly included in training instance inputs of training instances may be used for such selection. Also, for example, generation model 152A may be trained by selecting only (or biasing towards) training instances associated with particular work attributes. For example, the selection may be biased against queries submitted in relation to the particular task (s) involved (or to be involved). It is also noted that in some implementations, the generation model 152A is trained using training instances that include a plurality of different “types” in the training instance input. As described herein, this may create a variant of multiple heterogeneous types and, at runtime, enable the generation of a multitask model that may be biased towards a particular type by applying a corresponding type value as input. do.

3 illustrates an example of using a generation model to generate one or more variations of a query. In FIG. 3, the user's original queries and attributes are sent from the client device 106 to the transformation engine 112. In some other implementations, one or more (eg, all) of the attributes may not be sent by the client device 106 with the query, or even not by the client device 106 at all. For example, the user's attributes can be stored remotely from the client device. For example, the attributes can be stored remotely and based on the user's past interactions (eg, via other client devices), and accessed by the transformation engine 112 from the remote storage device. Can be.

The transformation engine 112 uses at least one of the generation models 152 to generate one or more variations of the original query. In generating the transformation (s), the transformation engine 112 may use the attributes to select one of the generation models 152 and / or apply one or more attributes as input to one of the generation models. have. The transformation engine 112 may further apply the tokens of the original query to the generation model and / or other features (eg, variations created in the past in which multiple variations are generated in an iterative manner).

In some implementations, the transformation engine 112 sends the transformations to the client device 106 as output to be provided based on the original query. In some implementations, the transformation engine 112 may additionally or alternatively perform one or more variations of the search system 140 (which may be one or more response (s) (e.g., For example, determining a single answer search result or several search results) and sending the response to the client device as output to be provided based on the original query.

4 illustrates another example of using a generation model to generate one or more variations of a query. In particular, FIG. 4 shows an example in which a control model is used to control the generation of variants.

In FIG. 4, the user's original queries and attributes are sent from the client device 106 to the controller engine 114. As in FIG. 3, in other implementations, one or more attributes (eg, all of the attributes) may not be sent by the client device 106 with the query, or even by the client device 106. It may not be transmitted at all.

In some implementations, the controller engine 114 uses one or more control models 154 to determine whether to produce a variation of the original query. For example, the controller engine 114 may add tokens of the original query, search system response (s) for the original query, and / or the user's attributes to one of the control models 154 to determine whether to generate a variant. Applicable In some other implementations, the controller engine 114 can basically determine that at least one variant or original query should be generated.

The controller engine 114 provides the deformation engine 112 with a compensation signal determined based on the output through the one or more control models 154, and also provides the current state. The current state may include, for example, a feature vector based on the original query, the user's attributes, and / or one or both, where the feature vector is based on output through one or more control models 154. do.

The transformation engine uses at least one generation model 152 to generate one or more variations of the original query. In generating the deformation (s), the deformation engine 112 may use the provided state and optionally the compensation signal. For example, the deformation engine 112 may apply the compensation signal to the learned compensation function to determine the compensation when generating the query deformation. The transformation engine 112 provides the transformation (s) to the search system 140. In response, search system 140 generates one or more response (s) and provides the response (s) to controller engine 114.

The controller engine 114 uses the variant (s) created so far and / or their corresponding response (s) to determine whether further variants should be generated by the deformation engine 112. For example, the controller engine 114 may apply tokens of features of the variation (s) and / or corresponding response (s) generated so far as input to one of the control models 154, and You can generate output through the control model based on the input, and use the output to determine if you are creating additional variants. In some implementations, the controller engine 114 further applies as part of the input to the tokens of the original query, the search system response (s) for the original query, and / or the user's attributes.

If the controller engine 114 determines that additional variants should be generated, based on the updated compensation signal and the updated current state (eg, the variant (s) generated so far and / or the corresponding variant response (s)). Updated). The transformation engine 112 may generate one or more additional variations, provide the modification (s) to the search system 140, and provide the corresponding response (s) again. The controller engine 114 may again determine whether additional variables should be generated based on the further modification (s) and corresponding response (s).

At a given iteration, if the controller engine 114 determines that no additional variations should be generated, the controller engine 114 generates one or more search system response (s) and / or one or more outputs as output to be provided based on the original query. The modified variant to the client device 106. For example, the controller engine 114 stores all provided response (s), and only one of the response (s) is responsive output (eg, highest quality response identified by the highest quality response or other responses). It can be provided as. As another example, the controller engine 114 may provide a number of responses (eg, N best responses, various sets of responses).

In some implementations, control model (s) 154, generation model (s) 152, controller engine 114, and / or transformation engine 112 may be trained using reinforcement learning. In some of these implementations, control model (s) 154 and / or generation model (s) 152 may be initially trained using other techniques and may be improved through reinforcement learning. For example, generation model (s) 152 may be initially trained as described in connection with FIG. 2, and may be further trained through reinforcement learning.

In some of these implementations, controller engine 114 and control model (s) 154 may be viewed as “critical” in the actor-critical algorithm, and transformation engine 112 and generation model (s) 152. Can be viewed as an "actor". In general, an actor creates variants and uses them to examine the environment. The environment may be, for example, a search system 140. In general, critics accumulate evidence from the environment (eg responses such as answer strings or corresponding rank lists) to generate overall actions / decisions d, maintain overall states, and Provide compensation signal r and context c.

The actions of the actor and the critic can be driven by consolidation on two different time scales. An actor can run on a finer time scale (indexed by t '). At each stage, the actor creates the next variant, depending on the context. Critics accumulate evidence as a whole state in the environment. In some situations, in some situations, the state may contain at least the original query, generated variants, and observations (eg, a retrieval system response to the generated variants) and a vector summary h used to supply the network. S = ({q _t , o _t } _1..T , h _t ). Given the overall condition, the critic makes a full decision at each stage. That is, to respond or continue the cycle of generating strain and more evidence. The critic also provides context to the actor to coordinate the variant generation and compensation signals. The critic directly models the value of the state-behavior pair "Q-function" Q (s _t , d _t ). This value of the Q function is passed to the actor as a reward signal. The Q function is trained using the response (s) (eg, response (s) to the original query) and the overall compensation defined in the sequence of decisions d. By separating the time scales, you can separately model the two tasks of creating transformations and making overall decisions, but you can combine and train them to optimize end-to-end performance.

로 표현될 수 있고, [0,1]은 디스카운트이고, k는 최종 상태에 대한 반복 회수들이고, R은 최종 보상이다.Instead of continuing to generate variants and cycles of more evidence, the final state is reached when the critic responds. The actor's space of action can be defined as A: = {(a, <w>): a ∈ {question, answer}, <w> ∈ strings}, where a uses a variant to examine the environment Can respond or respond. An action is paired with a string <w> that defines a variant or an answer. In some implementations, the “investigate the environment with deformation” actions are not compensated, and the “release of response actions” is rewarded in proportion to the quality of the answer. The critic can learn a Q function that maps actions (a, <w>) from the current state to the expected return E [Gs]. If only "release of response actions" is compensated, the expected return is

Where [0,1] is a discount, k is the number of iterations for the final state, and R is the final compensation.

으로 업데이트될 수 있다. 행위자는 변형들을 생성하며, 그리고 예를 들어, 원래 질의, 최신 변형, 그리고 변형들 및 응답들의 이력에 대한 조건으로 하는 더 많은 피처들을, 입력으로 받아 하나 이상의 추가 변형을 리턴하는 시퀀스 대 시퀀스 모델을 포함할 수 있다. 행위자는 몬테-카를로 정책 그라디언트 접근 방식으로 훈련될 수 있다. 환경으로부터 수신된 응답 세트는 지금까지 보여진 답변들의 메모리 역할을 한다. 행위자 및 Q 함수에 피처들을 제공하고 그리고/또는 비평가가 중간 반복에서 보여지는 답변들을 리턴가능하도록 사용될 수 있다.Q function training can be accomplished using the Monte-Carlo Q-learning approach. The variants can be sampled until the final compensation is reached, the compensation can be determined, and all intermediate predictions of the Q function

Can be updated. An actor generates variants, and for example, has a sequence-to-sequence model that takes as input, more features subject to the original query, the latest variant, and the history of the variants and responses, and returns one or more additional variants. It may include. Actors can be trained in the Monte-Carlo policy gradient approach. The response set received from the environment serves as a memory for the answers shown so far. The features may be provided to the actor and the Q function and / or the critic may be used to return the answers shown in the intermediate iteration.

Referring now to FIG. 5, a flow diagram illustrating a method 500 of training a generation model in accordance with various implementations disclosed herein is provided. For convenience, the operations of the flowcharts are described with reference to a system that performs the operations. Such a system may include one or more components, such as one or more processors (eg, CPU (s), GPU (s), and / or TPU (s)). Although the operation of method 500 is shown in a particular order, this is not limiting. One or more actions may be rearranged, omitted, or added.

At block 552, the system selects a group of training instances. For example, when a generation model is trained to be a multitasking model in method 500, the system may select a group such that the group includes training instances representing multiple types of variant generation. In addition, for example, if the generation model is additionally or alternatively trained to be specific to particular users group (s), the system may be configured to query queries by users whose training sequences conform to the particular group (s). The group may be selected to include only training instances based on past submissions or to include a significant amount (eg, more than half, more than 70%) of the training instances. Also, for example, if the generation model is additionally or alternatively trained to be specific to a particular task (s), the system may provide a training instance in which the training instances are based on past submissions of queries with respect to the particular task (s). Groups can be selected to include only those, or include a significant amount of the training instances (eg, first half, greater than 70%).

At block 554, the system selects a training instance of the group.

At block 556, the system applies the training instance input of the training instance as input to the generation model. The training instance input may include, for example, terms of the original query, attributes (eg, attributes of the user who submitted the original query), and type value (indicating the variant type of the original query).

At block 558, the system generates a modification to the generation model based on the applied training instance input.

At block 560, the system determines an error for the training instance based on the comparison of the generated modification with the training instance output (ie, the variation indicated in the training instance output).

At block 562, the system updates the generation model based on the error. For example, the error may be a gradient that is propagated back to the generation model to update the generation model.

At block 564, the system determines if there are additional raw training instances in the group. If so, the system proceeds to block 554 to select additional training instances. The system then performs blocks 556, 558, 560, and 562 based on the additional training instance.

In an iteration of block 564, if the system determines that there are no further training instances in the group (or other training criteria have been met), the system proceeds to block 566 where the training ends.

5 illustrates a particular non-batch training approach, but batch training (eg when an error is determined and back propagated based on a batch of training instances) is additional or alternative in training. It is understood that it can be used as. In addition, it is understood that in various implementations, a generation model trained based on the method 500 may be further trained in accordance with the techniques disclosed herein. For example, the generation model can be further trained using reinforcement learning techniques, and can be further trained separately from the separate control model, but with the separate control model. In addition, if multiple generation models are generated, the method 500 may be repeated at block 552 with different selection criteria to generate additional model (s).

Referring to FIG. 6, a flow diagram illustrating a method 600 of generating one or more variations of a query using a generation model in accordance with various implementations disclosed herein is provided. For convenience, the operations of the flowcharts are described with reference to a system that performs the operations. The system may include one or more components such as one or more processors (eg, CPU (s), GPU (s) and / or TPU (s)). Although the operations of method 600 are shown in a particular order, this is not limiting. One or more actions may be rearranged, omitted, or added.

At block 652, the system receives a query.

At block 654, the system selects a generation model from the plurality of candidate generation models. In some implementations, the system selects a generation model based on one or more attributes of the user who submitted the query of block 652. For example, the system may select a generation model based on a model stored in association with an attribute that matches one or more attributes of a user. For example, one may store in association with these attributes based on what is trained based on training instances based on past query submissions of users with those attributes. In some implementations, block 654 can be omitted (eg, only a single generation model can be used).

At block 656, the system applies the tokens and additional values of the query as input to the generation model. Various additional values may be applied, such as the attributes of the user who submitted the query, the temporal attributes, and / or the attributes for the search system response (s) for the received query. As one specific example, the additional values may include the predicted task attribute of the user who submitted the query. The predicted task attribute may be predicted based on, for example, content recently viewed by the user at the computing device, the user's stored calendar entry, and / or the user's electronic communication (s).

At block 658, the system generates one or more variations on the generation model based on the applied input.

At block 660, the system determines whether to generate additional variations. In some implementations, the system determines whether to generate additional variants based on the properties of the variants created so far and / or the response (s) from the search system for the variants created so far. For example, the system may generate additional variables based on whether the response (s) for the variation (s) generated so far have been found by the search system and / or the quality measure (s) of the response (s). Can be determined. For example, the system may generate additional variations if there are no responses and / or if the quality measurement (s) do not meet one or more quality criteria.

At an iteration of block 660, if the system determines to generate additional variations, the system proceeds to block 662 to update one or more additional values to be applied as inputs to the generation model at a subsequent iteration of block 656. For example, the system may reflect the type (s) generated at the most recent iteration of block 658, to reflect the response (s) to the variant (s), and the type value for the next iteration of block 658. You can update additional values to change. The system then performs another iteration of block 656 using the updated additional values and then proceeds to blocks 658 and 660.

At an iteration of block 660, if the system determines not to generate additional variations, the system proceeds to block 664 to provide an output based on one or more generated variations. The output may include one or more variant (s) and / or search system response (s) for one or more variant (s).

7 is a flow diagram illustrating a method 700 of generating one or more variations of a query using a generation model, wherein a control model is used to control the generation of variations. For convenience, the operations of the flowcharts are described with reference to a system that performs the operations. The system may include one or more components such as one or more processors (eg, CPU (s), GPU (s) and / or TPU (s)). Although the operations of method 700 are shown in a particular order, this is not intended to be limiting. One or more actions may be rearranged, omitted, or added.

In block 752, the system receives a query.

At block 754, the system generates a control output via the control model based on the current state. For example, the current state may be based on tokens of the current query, search system responses to the current query, and / or other features.

At block 756, the system determines whether to generate a variation of the received query based on the control output. In some implementations, initial iteration of block 754 and block 756 can be omitted. In other words, in such implementations, the system may decide to always generate a variant (eg, to validate a search system response to a received query).

At an iteration of block 756, if the system determines not to generate a variant, the system proceeds to block 766 to provide an output based on the current search system response (s) and / or generated variant (s).

At an iteration of block 756, if the system determines to generate a variant, the system proceeds to block 758.

At block 758, the system determines a compensation signal and / or context based on the control output generated at the most recent iteration of block 754. The compensation signal may be based on the learned Q-function as described herein, and the context may include, for example, a vector summary of the current state and / or the current state.

At block 760, the system generates a variation on the generation model based on the received query and the compensation signal and / or context of block 758.

At block 762, the system determines the response (s) for the transformation generated at block 760. For example, the system can submit a variant to the search system and receive response (s) from the search system responsive to the variant. In some situations, the search system does not return a response and / or generates a "null", each indicating that no response (eg, answer) is available.

At block 764, the system updates the current state and response (s) for the variant based on the variant. The system then proceeds to block 754 to generate a control output for the control model based on the current state including the update of block 764. In this way, in subsequent iterations of block 764, the modification (s) and response (s) previously generated (ie, generated in the previous iterations of blocks 760 and 762) are considered in the iteration of the next block 754. Can be. The system then returns to block 756 to determine whether to generate another variant of the received query based on the control output. If the system decides to generate another variant, the compensation signal and context provided in the next iteration of block 758 is likewise the variant (s) and response previously generated (ie, generated in the previous iteration of blocks 760 and 762). It is noted that it can be adjusted according to the (s). In this way, the deformation generation of the next iteration of block 760 is consequently affected by the previously generated deformation (s) and response (s).

8A and 8B, example graphical user interfaces 800a and 800b are shown to provide an output based on variant (s) generated in accordance with implementations disclosed herein. Graphical user interfaces 800A and 800B may be provided at client device 106 (eg, in a browser running on client device 106 and / or in another application running on client device 106). .

In FIG. 8A, the user provided a query 891A that "Da Vinci drew the Mona Lisa". In response, an output is provided that includes a response 892A and also includes two variants 893A. Two variants 893A can be generated in accordance with the implementations disclosed herein. In some implementations, each variant is selectable and, in response to the selection, causes the corresponding variant to be submitted as a new query. In some implementations, response 892A is also based on the modification (s) generated in accordance with the implementations disclosed herein. For example, in some situations, response 892A may be a response to a variant of query 891A (a variant different from variants 893A) and / or response 892A may be a response to query 891A. But may be verified based on the response (s) for the variant (s) of the query (eg, by ensuring more generated positive responses of these variants).

In FIG. 8B, the user provided a query 891B that "Miquel Angelo drew the Mona Lisa". In response, an output is provided that includes a response 892B of "no". Box 895B of FIG. 8B may optionally not be provided for display, but is presented as an example of variations that may be generated in accordance with the techniques described herein to generate a “no” response 892B. Box 895B displays the original query (indicated by "O") and includes "Y" in parentheses to indicate that an answer response was generated by the search system in response to the original query. For example, "Yes, Michelangelo drew Mona Lisa." However, instead of providing an answer, a number of variants are created, which are "subsequent" variants, to verify the correctness of the response to the original query. In particular, variants VI, V2 and V3 are created. As indicated by "N" in parentheses, "no answer" responses were generated by the search system in response to each subsequent modification. If there is no answer available for a number of subsequent variants, the controller engine may determine that the "answer response" to the original query is incorrect (since subsequent variants do not lead to any answers). As a result, the controller engine may provide a response 892B with "no".

Although examples of the graphical interface are presented in FIGS. 8A and 8B, queries may be additionally or alternatively received based on a user's voice input and / or variations and / or responses are audited via the client device. In addition, it is to be understood that the present invention may be additionally or alternatively provided to the user.

In situations where the systems described herein may collect or use personal information about users, programs or features may be used to identify user information (eg, a user's social network, social activities or behaviors, occupations). , Users' preferences, or information about the user's current geographic location) or provide users with an opportunity to control whether and / or how to receive content from content servers that may be more relevant to the user. I can receive it. In addition, certain data may be processed in one or more ways prior to storage or use so that personally identifiable information is removed. For example, the user's identity may be processed such that no personally identifiable information can be determined for the user, or the user's geographic location may be generalized if geographic location information (such as city, ZIP code, or state level) is obtained. As such, a particular geographic location of the user cannot be determined. Thus, the user can control how information about the user is collected and / or used.

9 is a block diagram of an example computing device 910 that can optionally be used to perform one or more aspects of the techniques described herein. Computing device 910 includes at least one processor 914 (eg, CPU, GPU, and / or TPU) that communicates with a number of peripheral devices via bus subsystem 912. Such peripheral devices may include, for example, storage subsystem 1024 including memory subsystem 925 and file storage subsystem 926, user interface output devices 920, user interface input devices 922. And network interface subsystem 915. The input device and output device enable user interaction with the computing device 910. Network interface subsystem 915 provides an interface to external networks and is coupled to corresponding interface devices in other computing devices.

User interface input devices 922 may be a pointing device such as a keyboard, mouse, trackball, touch pad or graphics tablet, scanner, touch screen integrated into a display, audio input devices such as voice recognition systems, microphones, and And / or other types of input devices. In general, use of the term “input device” is intended to include all possible types of devices and manners of entering information into computing device 910 or on a communication network.

User interface output devices 920 may include non-visual displays, such as a display subsystem, printer, fax machine, or audio output devices. The display subsystem may include a cathode ray tube (CRT), a flat panel device such as a liquid crystal display (LCD), a projection device, or other mechanism for generating a visible image. The display subsystem can also provide non-visual display via audio output devices. In general, the use of the term “output device” is intended to include all possible types of devices and ways of outputting information from computing device 910 to a user or other machine or computing device.

Storage subsystem 924 stores programming and data structures that provide the functionality of some or all of the modules described herein. For example, storage subsystem 924 can include logic to perform selected aspects of the methods described herein.

These software modules are generally executed by the processor 914 alone or in combination with other processors. Memory 925 used in storage subsystem 924 includes main random access memory (RAM) 930 for storage of instructions and data during program execution and read-only memory (ROM) in which fixed instructions are stored ( And multiple memories, including 932. File storage subsystem 926 can provide permanent storage for program and data files, and can include a floppy disk drive, CD-ROM drive, optical drive, or removable media cartridge along with a hard disk drive, associated removable media. Can include them. Modules that implement the functionality of certain implementations may be stored in the file storage subsystem 926 of the storage subsystem 924 or other machines accessible by the processor (s) 914.

Bus subsystem 912 provides a mechanism that allows various components and subsystems of computing device 910 to communicate with each other as intended. Although bus subsystem 912 is schematically depicted as a single bus, other implementations of the bus subsystem may use multiple buses.

Computing device 910 may be of various types, including a workstation, server, computing cluster, blade server, server farm, or any other data processing system or computing device. Because of the ever-changing nature of computers and networks, the description of computing device 910 shown in FIG. 9 is intended only as a specific example for describing some implementations. Many other configurations of the computing device 910 are possible with more or fewer components than the computing device shown in FIG. 9.

Claims (36)

A method implemented by one or more processors,
Receiving via the client device an original query generated based on the user interface input of the user;
Applying the tokens of the original query as input to a trained generation model;
Generating at least one variant of the original query based on applying tokens of the original query to the trained generation model;
Generating an output based on at least one of the at least one variant and at least one search system response to the at least one variant; And
In response to the original query, providing an output for presentation through the client device.
A method implemented by one or more processors. The method of claim 1,
As part of the input, further comprising applying one or more attributes associated with the user to the trained generation model.
A method implemented by one or more processors. The method of claim 2,
Generating at least one variant of the original query based on one or more attributes for the trained generation model.
A method implemented by one or more processors. The method according to claim 2 or 3,
The one or more attributes include one or more of a user's location, a task that the user currently participates in, and weather at the user's location.
A method implemented by one or more processors. The method according to any one of claims 1 to 4,
As part of the input, further comprising applying one or more time attributes to the trained generation model including at least one of a current time, a current day of the week, and a current date.
A method implemented by one or more processors. The method according to any one of claims 1 to 5,
Determining a predicted task for the user; And
As an input, applying one or more task attributes of a task predicted for the user to the trained generation model,
Generating at least one variant of the original query is based on the application of the one or more task attributes to the trained generation model.
A method implemented by one or more processors. The method of claim 6,
Determining a predicted task for the user is based on one or more interactions with the user via the client device or an additional client device.
A method implemented by one or more processors. The method of claim 7, wherein
Based on the predicted task being determined, the one or more interactions comprise an electronic communication sent by the user or a calendar entry generated by the user.
A method implemented by one or more processors. The method of claim 6,
Determining a task predicted for the user is based on an electronic communication sent to the user or a stored calendar entry of the user.
A method implemented by one or more processors. The method of claim 6,
Creating a training instance comprising a training instance input and a training instance output; And
Training the generation model based on the generated training instance;
The training instance input is:
First query tokens of the first query, and
Contains task attributes,
The training instance output is:
Includes second query tokens of the second query,
The training instance is generated with the task attribute as a training instance input based on a determination that past submission of the first query and subsequent past submission of the second query are related to the predicted task.
A method implemented by one or more processors. The method according to any one of claims 6 to 10,
Selecting a trained generation model from a plurality of trained generation models based on the trained generation model trained based on past query submissions associated with the predicted task.
A method implemented by one or more processors. The method of claim 11,
Selecting training instances generated based on past query submissions associated with the predicted task; And
Training the generation model based on the selected training instances.
A method implemented by one or more processors. The method of claim 12,
Determining that a group of two or more previously submitted queries is associated with the predicted task;
Creating one of the training instances based on the group of previously submitted queries; And
Labeling one of the training instances associated with the predicted task,
Selecting training instances generated based on past query submissions associated with the predicted task comprises selecting one of the training instances based on the labeling.
A method implemented by one or more processors. The method of claim 13,
Determining that a group of two or more previously submitted queries is related to the predicted task, based on a computer-based operation performed after submission of the previously submitted queries.
A method implemented by one or more processors. The method according to any one of claims 1 to 5,
Selecting a trained generation model from a plurality of trained generation models based on the trained generation model trained based on past query submissions of a group of users having one or more attributes in common with the user; And
As an input, applying tokens of an original query to the selected trained generation model
A method implemented by one or more processors. The method according to any one of claims 1 to 15,
The trained generation model is a deep neural network model having one or more memory layers.
A method implemented by one or more processors. The method according to any one of claims 1 to 16,
Generating at least one variant of the original query comprises generating the variant based on learned parameters of the trained generation model,
The method is:
Applying an additional input to the trained generation model, the additional input comprising at least one of tokens of the original query and variant tokens of the modification of the original query;
Generating the original query further variant for the trained generation model based on the further input, wherein the further variant is different from the variant and the original query, and generating the original query further variant Generating the further modification based on the learned parameters of the trained generation model;
Determining a further modification response to the further modification of the original query based on submitting the further modification of the original query to the search system;
Generating an output based on at least one of the modification response and the further modification response; And
In response to the original query, providing an output for presentation through the client device.
A method implemented by one or more processors. The method of claim 17,
The trained generation model is trained to produce multiple types of query variants, and the variant is a first type of the multiple types of query variants and the further variant is a second of the multiple types of query variants. Characterized in that the type
A method implemented by one or more processors. The method of claim 18,
The first type is one of an equivalent query, a subsequent query, a generalized query, a normalized query, an implicit query, a specification query, a description query, and a language translation query, and the second type is an equivalent query, a subsequent query, a generalized query, Characterized in that the other one of the normalized query, implicit query, specification query, description query
A method implemented by one or more processors. The method of claim 18,
The modification is generated for the trained generation model as the first type, based on a first type value applied as part of the input to the trained generation model, and
The further modification is generated for the trained generation model as the second type based on a second type value applied as part of the further input to the trained generation model.
A method implemented by one or more processors. The method of claim 17,
The additional input applied to the trained generation model comprises tokens of the original query
A method implemented by one or more processors. The method of claim 21,
The further input further comprises variant tokens of the variant of the original query
A method implemented by one or more processors. The method of claim 21,
The additional input further includes modification response features based on the transformation response to the transformation of the original query.
A method implemented by one or more processors. The method of claim 17,
Before generating the further modification, further comprising determining, based on the deformation response, whether to provide the deformation response as an output or to generate the additional deformation,
Generating the further variation depends on determining to create the further variation instead of providing the variation response.
A method implemented by one or more processors. The method of claim 24,
Determining whether to provide the variant response as an output or generate the further variant, further based on a variant of the original query
A method implemented by one or more processors. The method of claim 24,
Based on the modification response, determining whether to provide the modification response as an output or generate the further modification:
Applying a feature of the deformation response to a trained control model as a controller input;
Generating a controller output for the trained control model based on the controller input; And
Determining to generate the further modification based on the controller output.
A method implemented by one or more processors. The method of claim 26,
Based on the modification response, determining whether to provide the modification response as an output or generate the further modification:
Applying a modification of the original query to the trained control model as part of the input.
A method implemented by one or more processors. The method of claim 26,
The trained control model is trained based on reinforcement learning based on rewards determined based on past interactions with the search system or further search system.
A method implemented by one or more processors. The method of claim 26,
The trained control model is trained separately from the trained generation model but in combination with the trained generation model.
A method implemented by one or more processors. The method of claim 26,
The trained control model is characterized in that the feed forward (feed forward) neural network or iterative neural network
A method implemented by one or more processors. The method according to any one of claims 17 to 26,
The input further comprises one or more attributes associated with the user, and wherein the additional input further comprises the one or more attributes.
A method implemented by one or more processors. The method of any one of claims 17 to 31,
Sending an original request to the search system over a network, the original request including the original query; And
Receiving an original response from the search system in response to the original request,
Applying the tokens of the original query to a trained generation model as input and generating the variant are based on the original response from the search system.
A method implemented by one or more processors. The method according to any one of claims 1 to 32,
The trained generation model is a deep neural network model with memory layers
A method implemented by one or more processors. The method according to any one of claims 1 to 33,
Training sequence-to-sequence neural network machine translation to produce the trained generation model.
A method implemented by one or more processors. 35. A system comprising at least one processor configured to perform the method of any one of claims 1 to 34 and a memory operably coupled with the at least one processor. 35. A computer program configured to perform the method of any one of claims 1 to 34 when executed by a processor.

Images