CN111078842A - Method, device, server and storage medium for determining query result - Google Patents
Method, device, server and storage medium for determining query result Download PDFInfo
- Publication number
- CN111078842A CN111078842A CN201911408075.2A CN201911408075A CN111078842A CN 111078842 A CN111078842 A CN 111078842A CN 201911408075 A CN201911408075 A CN 201911408075A CN 111078842 A CN111078842 A CN 111078842A
- Authority
- CN
- China
- Prior art keywords
- word
- sample
- query
- vector
- semantic vector
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 80
- 239000013598 vector Substances 0.000 claims abstract description 498
- 238000012549 training Methods 0.000 claims description 59
- 230000015654 memory Effects 0.000 claims description 22
- 230000006399 behavior Effects 0.000 claims description 8
- 238000012512 characterization method Methods 0.000 claims description 6
- 238000012545 processing Methods 0.000 abstract description 20
- 230000008569 process Effects 0.000 abstract description 19
- 238000003058 natural language processing Methods 0.000 abstract description 3
- 238000005516 engineering process Methods 0.000 description 12
- 230000006870 function Effects 0.000 description 11
- 238000012935 Averaging Methods 0.000 description 8
- 238000010586 diagram Methods 0.000 description 8
- 230000002457 bidirectional effect Effects 0.000 description 7
- 235000013399 edible fruits Nutrition 0.000 description 7
- 238000004891 communication Methods 0.000 description 5
- 239000003814 drug Substances 0.000 description 4
- 230000011218 segmentation Effects 0.000 description 4
- PCTMTFRHKVHKIS-BMFZQQSSSA-N (1s,3r,4e,6e,8e,10e,12e,14e,16e,18s,19r,20r,21s,25r,27r,30r,31r,33s,35r,37s,38r)-3-[(2r,3s,4s,5s,6r)-4-amino-3,5-dihydroxy-6-methyloxan-2-yl]oxy-19,25,27,30,31,33,35,37-octahydroxy-18,20,21-trimethyl-23-oxo-22,39-dioxabicyclo[33.3.1]nonatriaconta-4,6,8,10 Chemical compound C1C=C2C[C@@H](OS(O)(=O)=O)CC[C@]2(C)[C@@H]2[C@@H]1[C@@H]1CC[C@H]([C@H](C)CCCC(C)C)[C@@]1(C)CC2.O[C@H]1[C@@H](N)[C@H](O)[C@@H](C)O[C@H]1O[C@H]1/C=C/C=C/C=C/C=C/C=C/C=C/C=C/[C@H](C)[C@@H](O)[C@@H](C)[C@H](C)OC(=O)C[C@H](O)C[C@H](O)CC[C@@H](O)[C@H](O)C[C@H](O)C[C@](O)(C[C@H](O)[C@H]2C(O)=O)O[C@H]2C1 PCTMTFRHKVHKIS-BMFZQQSSSA-N 0.000 description 3
- 230000007787 long-term memory Effects 0.000 description 3
- 230000003287 optical effect Effects 0.000 description 3
- 239000013589 supplement Substances 0.000 description 3
- 230000009286 beneficial effect Effects 0.000 description 2
- 238000004364 calculation method Methods 0.000 description 2
- 238000010276 construction Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 235000011475 lollipops Nutrition 0.000 description 2
- 238000013139 quantization Methods 0.000 description 2
- 230000004044 response Effects 0.000 description 2
- 230000006403 short-term memory Effects 0.000 description 2
- 239000007787 solid Substances 0.000 description 2
- 238000004590 computer program Methods 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
- 230000001960 triggered effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/33—Querying
- G06F16/3331—Query processing
- G06F16/334—Query execution
- G06F16/3344—Query execution using natural language analysis
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/33—Querying
- G06F16/3331—Query processing
- G06F16/334—Query execution
- G06F16/3347—Query execution using vector based model
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q30/00—Commerce
- G06Q30/06—Buying, selling or leasing transactions
- G06Q30/0601—Electronic shopping [e-shopping]
- G06Q30/0623—Item investigation
- G06Q30/0625—Directed, with specific intent or strategy
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Artificial Intelligence (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The application discloses a method and a device for determining a query result, a server and a storage medium, and belongs to the field of natural language processing. The method comprises the following steps: acquiring a query word contained in the query request; inputting a query word into a semantic vector generation model to obtain a first semantic vector corresponding to the query word, wherein the semantic vector generation model is used for dividing input contents according to word granularity and vectorially expressing the semantics of a division result; determining at least one candidate semantic vector from the second semantic vectors according to the vector similarity between the first semantic vector and each second semantic vector; and determining the commodity words corresponding to the candidate semantic vectors as query results responding to the query requests. By adopting the method, the query word can be vectorized by directly utilizing the semantic vector generation model to obtain the first semantic vector, and in addition, the process of segmenting the query word is reduced by segmenting the query word according to the word granularity, so that the model processing efficiency is improved on the basis of improving the accuracy of the query result.
Description
Technical Field
The embodiment of the application relates to the field of natural language processing, in particular to a method and a device for determining a query result, a server and a storage medium.
Background
Text matching is a core problem in natural language processing, for example, in the e-commerce field, a commodity search task is a matching process of a query word and a commodity word, and the core problem of the commodity search task is matching the query word input by a user with an optimal candidate commodity word. When a user initiates a commodity search task once, related candidate commodity sets are found out from massive commodities according to query words provided by the user, then the candidate commodity sets are scored according to different rules, all the candidate commodity sets are sorted according to the scoring results, and finally the query results are presented on a terminal interface.
In the related art, there are various methods for completing a task of searching for a commodity. If so, constructing a synonym dictionary and a core word dictionary, performing synonym expansion and core word extraction on the query words provided by the user, taking the expanded and extracted words as new query words to perform query, obtaining query results of different paths, and merging the query results of the paths to obtain a final query result; for another example, vectorizing the query word and all commodity words to obtain word vectors of the query word and the commodity words, averaging the word vectors to obtain sentence vectors of the query word and the commodity words, and solving the commodity with the closest vector distance in the query word and all commodities as a query result.
Then, the method for completing the commodity search task excessively depends on the word segmentation result, the corresponding word vector is generated according to the word segmentation result, for the commodity search task in a specific field, such as a fresh field with poor word segmentation effect, the query result is not ideal, and in addition, the averagely obtained sentence vector cannot reflect the weight of the special characters or words in the whole sentence, so that the query result is not accurate.
Disclosure of Invention
The embodiment of the application provides a method, a device, a server and a storage medium for determining a query result, and the technical scheme is as follows:
in one aspect, a method for determining query results is provided, where the method includes:
acquiring a query word contained in the query request;
inputting the query word into a semantic vector generation model to obtain a first semantic vector corresponding to the query word, wherein the semantic vector generation model is used for dividing input contents according to word granularity and vectorially representing the semantics of a division result;
determining at least one candidate semantic vector from the second semantic vectors according to the vector similarity between the first semantic vector and each second semantic vector, wherein the second semantic vector is a semantic vector corresponding to a commodity word in a commodity library;
and determining the commodity words corresponding to the candidate semantic vectors as the query results responding to the query requests.
In another aspect, an apparatus for determining a query result is provided, the apparatus comprising:
the query term acquisition module is used for acquiring query terms contained in the query request;
the semantic vector generation module is used for inputting the query word into a semantic vector generation model to obtain a first semantic vector corresponding to the query word, and the semantic vector generation model is used for dividing input content according to word granularity and vectorizing and expressing the semantics of a division result;
the semantic vector determining module is used for determining at least one candidate semantic vector from the second semantic vectors according to the vector similarity between the first semantic vector and each second semantic vector, wherein the second semantic vector is a semantic vector corresponding to a commodity word in a commodity library;
and the query result determining module is used for determining the commodity words corresponding to the candidate semantic vectors as the query results responding to the query requests.
In another aspect, a server is provided, the server comprising a processor and a memory; the memory stores at least one instruction for execution by the processor to implement the method of determining a query result as described in the above aspect.
In another aspect, a computer-readable storage medium is provided, the storage medium storing at least one instruction for execution by a processor to implement the method for determining a query result as described in the above aspect.
In another aspect, a computer program product is provided, which stores at least one instruction that is loaded and executed by the processor to implement the method for determining the query result according to the above aspect.
In the embodiment of the application, the server obtains a first semantic vector corresponding to a query word through a semantic vector generation model, calculates the vector similarity between the first semantic vector and a second semantic vector corresponding to a commodity word in a commodity library, and determines at least one candidate semantic vector from the second semantic vector as a query result for responding to a query request. Based on the determination method of the query result provided by the embodiment of the application, the query word can be directly vectorized by using the semantic vector generation model, and the first semantic vector corresponding to the query word is obtained, so that the semantic vector generation process in the related technology is simplified; in addition, the semantic vector generation model directly divides the query words according to the word granularity, and reduces the process of dividing the query words, so that the efficiency of processing the input content by the semantic vector generation model is improved on the basis of enabling the semantic representation of the semantic vector to be more accurate, and especially in the field with higher requirement on word division precision, compared with the word granularity division method in the related technology, the embodiment of the application is beneficial to improving the accuracy of the query result.
Drawings
FIG. 1 illustrates a schematic diagram of a computer system provided by an exemplary embodiment of the present application;
FIG. 2 illustrates a flow chart of a method of determining query results provided by an exemplary embodiment of the present application;
FIG. 3 illustrates a flow chart of a method of determining query results provided by another exemplary embodiment of the present application;
FIG. 4 illustrates a flow chart of semantic vector generation model training provided by an exemplary embodiment of the present application;
FIG. 5 illustrates a flow diagram of semantic vector generation model training provided by another exemplary embodiment of the present application;
FIG. 6 is a schematic diagram illustrating the structure of a semantic vector generation model provided by an exemplary embodiment of the present application;
FIG. 7 illustrates a flow chart of a method of determining query results provided by another exemplary embodiment of the present application;
FIG. 8 is a block diagram illustrating an apparatus for determining query results according to an exemplary embodiment of the present application;
fig. 9 is a block diagram illustrating a server according to an exemplary embodiment of the present application.
Detailed Description
To make the objects, technical solutions and advantages of the present application more clear, embodiments of the present application will be described in further detail below with reference to the accompanying drawings.
Reference herein to "a plurality" means two or more. "and/or" describes the association relationship of the associated objects, meaning that there may be three relationships, e.g., a and/or B, which may mean: a exists alone, A and B exist simultaneously, and B exists alone. The character "/" generally indicates that the former and latter associated objects are in an "or" relationship.
The method for determining the query result can be widely applied to scenes needing target object searching, such as in the field of e-commerce, the target object is a commodity which is purchased by a user in a commodity library and also such as a candidate person meeting the available position in a personnel system. For convenience of explanation of the following embodiments, the various embodiments of the present application are schematically illustrated in the e-commerce field, but are not limited to the present application.
Referring to FIG. 1, a schematic diagram of a computer system provided by an exemplary embodiment of the present application is shown. The computer system includes a terminal 110 and a server 120, wherein the terminal 110 and the server 120 perform data communication through a communication network, optionally, the communication network may be a wired network or a wireless network, and the communication network may be at least one of a local area network, a metropolitan area network, and a wide area network.
The terminal 110 has an application program with a commodity search function installed therein, such as various shopping software. Optionally, the terminal 110 may be a mobile terminal such as a smart phone, a tablet computer, a laptop portable notebook computer, or may also be a terminal such as a desktop computer, a projection computer, and the like, which is not limited in this embodiment of the present application.
The server 120 may be implemented as one server, or may be implemented as a server cluster formed by a group of servers, which may be physical servers, or may be implemented as a cloud server. In one possible implementation, the server 120 is a backend server for applications in the terminal 110.
In one example, as shown in fig. 1, a piece of fruit purchasing software is installed in the terminal 110, and at this time, the terminal 110 receives a query operation triggered by a user in a search engine, where the query operation is used to indicate that the user has an intention to search for a commodity corresponding to the query word a at this time, or when the commodity corresponding to the query word a does not exist, the user intends to search for a commodity close to the commodity corresponding to the query word a. The terminal 110 generates a query request according to the query operation and sends the query request to the server 120 in the background of the fruit purchasing software.
On the server 120 side, the server 120 obtains a query word a according to the query request, and inputs the query word a into the semantic vector generation model to obtain a first semantic vector, the server 120 obtains a semantic index of the commodity library, the semantic index includes second semantic vectors of all commodities included in the commodity library, if the semantic index corresponds to the fruit purchasing software, the commodity library includes all fruit names of fruits which can be currently provided by the software, each fruit name corresponds to a respective second semantic vector, and the semantic index in this example is a set of all second semantic vectors. Further, the server 120 calculates the vector similarity between the first semantic vector and each of the second semantic vectors, determines at least one candidate semantic vector from the second semantic vectors, and determines the commodity word corresponding to the candidate semantic vector as a query result in response to the query request.
Accordingly, after receiving the query result including at least one commodity word, the terminal 110 displays each commodity word, such as the commodity words 1 to 5 shown in fig. 1, on the search interface of the fruit purchase software. In one possible implementation, the commodity words in the query result are arranged based on the vector similarity, for example, the commodity word (e.g., commodity word 1 in fig. 1) corresponding to the second semantic vector with the highest vector similarity is displayed at the first position, and so on.
In other possible embodiments, the semantic vector generation model and the semantic index of the commodity library may also be implemented as part or all of the application program, and accordingly, the terminal 110 may determine the query result locally without the aid of the server 120, which is not limited in this application.
For convenience of description, the following embodiments are described as examples in which the determination method of the query result is performed by the server.
Referring to fig. 2, a flowchart of a method for determining a query result according to an exemplary embodiment of the present application is shown. The method comprises the following steps:
Optionally, the query request is initiated by a user, the user inputs a query term to an input box corresponding to a search engine displayed on the terminal interface in a text input, picture input, or voice input manner, and the server generates the query request corresponding to the query term according to the input operation.
When the content input by the user is in a non-text form, the server converts the input content in the non-text form into the input content in the text form through a voice recognition technology or a picture recognition technology, and acquires the query word according to the converted input content.
In a possible implementation manner, when the determination method of the query result of the present application is executed by the terminal side, the terminal may directly obtain the query term according to the input operation of the user, without obtaining the query term by generating the query request; in another possible implementation manner, when the determination method of the query result of the present application is executed by the terminal side and the server side in a matching manner, as shown in fig. 1, the terminal side generates a query request according to an input operation of a user, and the server receives the query request sent by the terminal side and then acquires a query term according to the query request.
In the present application, the determination method of the query result is implemented based on calculating the vector similarity between the semantic vectors, and therefore, the server needs to obtain the first semantic vector corresponding to the query word.
The semantic vector generation model is used for dividing input content according to word granularity and vectorizing and expressing the semantics of a division result.
In a possible implementation manner, the server inputs the query word into a semantic vector generation model, divides the query word, and uses the division unit as word granularity, namely, obtains each word included in the query word, and represents each word included in the query word in a vectorization form.
In one example, a user inputs "organic yam" in a search engine, a server divides the query word into four divided words, namely "there", "mountain" and "medicine", and vectorizes each word through a semantic vector generation model, and if the representation result is S ═ S (S1, S2, S3, S4), S represents the query word, S1 represents the vectorized result corresponding to "there", S2 represents the vectorized result corresponding to "there", S3 represents the vectorized result corresponding to "mountain", and S4 represents the vectorized result corresponding to "medicine". Furthermore, the semantics of each word are obtained through a semantic vector generation model, and finally a first semantic vector corresponding to the query word is obtained.
Each commodity word in the commodity library corresponds to a semantic vector, and in order to distinguish the semantic vector from the first semantic vector corresponding to the query word, the semantic vector corresponding to the commodity word in the commodity library is represented as a second semantic vector, namely the second semantic vector is the semantic vector corresponding to the commodity word in the commodity library.
Alternatively, the query term input by the user does not necessarily refer to a specific commodity name, and may be a general name of such a commodity, or a local feature of a certain commodity, and so on, and therefore, the server needs to obtain an appropriate commodity term from the commodity library as a query result in response to the query request.
Therefore, in order to obtain a proper commodity word as a query result for responding to the query request, the server calculates the vector similarity between a first semantic vector corresponding to the current query word and a second semantic vector corresponding to each commodity word, and determines at least one candidate semantic vector from the second semantic vectors.
In a possible implementation manner, the obtaining number of the candidate semantic vectors may be a fixed value, and if the application program corresponding to the current search engine sets that the query result displays at most 5 commodity words, the server obtains the second semantic vector of the top five of the ranking as the candidate semantic vector based on the vector similarity.
In another possible implementation, the number of acquired candidate semantic vectors may be a variable, and if the application program corresponding to the current search engine sets all second semantic vectors whose acquired vector similarity is greater than a certain threshold as the candidate semantic vectors, the number of acquired candidate semantic vectors is not fixed. The number of candidate semantic vectors obtained is not limited in the present application.
And step 204, determining the commodity words corresponding to the candidate semantic vectors as query results for responding to the query requests.
Optionally, each second semantic vector is a semantic vector corresponding to a commodity word, and the candidate semantic vectors are a part of all the second semantic vectors, that is, each candidate semantic vector also corresponds to a commodity word, so that after determining the candidate semantic vectors, the server obtains the commodity words corresponding to each candidate semantic vector, where the commodity words are query results responding to the query request.
To sum up, in the embodiment of the present application, the server obtains a first semantic vector corresponding to a query word through the semantic vector generation model, calculates the vector similarity between the first semantic vector and a second semantic vector corresponding to a commodity word in the commodity library, and determines at least one candidate semantic vector from the second semantic vector as a query result for responding to the query request. Based on the determination method of the query result provided by the embodiment of the application, the query word can be directly vectorized by using the semantic vector generation model, and the first semantic vector corresponding to the query word is obtained, so that the semantic vector generation process in the related technology is simplified; in addition, the semantic vector generation model directly divides the query words according to the word granularity, and reduces the process of dividing the query words, so that the efficiency of processing the input content by the semantic vector generation model is improved on the basis of enabling the semantic representation of the semantic vector to be more accurate, and especially in the field with higher requirement on word division precision, compared with the word granularity division method in the related technology, the embodiment of the application is beneficial to improving the accuracy of the query result.
Further, the construction of the semantic vector generation model is further illustrated by the following embodiments.
Referring to fig. 3, a flowchart of a method for determining a query result according to another exemplary embodiment of the present application is shown. The method comprises the following steps:
Please refer to step 201, and the embodiments of the present application are not described herein again.
In the application, after the semantic vector generation model inputs contents, such as query words and/or commodity words, the input contents can be divided according to word granularity, each divided word is subjected to vectorization expression, and then the process of segmenting the input contents is reduced, so that the vectorization expression of the input contents is realized under the condition of high segmentation precision; in addition, the input content represented by vectorization is not used as a direct output result, and the semantic vector generation model is further used for sequentially vectorizing and coding the input content represented by vectorization and obtaining a semantic vector corresponding to the input content, for example, after a query word is input by the semantic vector generation model, a first semantic vector corresponding to the query word is obtained.
In a possible embodiment, the functions that the semantic vector generation model can be used to implement are implemented by a presentation layer and an encoding layer, i.e. the semantic vector generation model comprises a presentation layer and an encoding layer. The presentation layer is used for implementing vectorization presentation of input content according to word granularity, outputting the input content after vectorization presentation and then inputting the input content into the coding layer, and the coding layer is used for coding the input content after vectorization presentation and obtaining semantic vectors corresponding to each input content.
Optionally, the representation layer is constructed based on a bidirectional encoder representation model, and the encoding layer is constructed based on a long-term and short-term memory network.
The bidirectional encoder characterization model is used for realizing vector representation of input content according to word granularity. In a possible implementation manner, the server inputs the number of input contents such as query words and/or commodity words into the bidirectional encoder characterization model, the bidirectional encoder characterization model divides each input content according to the word granularity to obtain words contained in each input content, and further, the bidirectional encoder characterization model converts the words contained in each input content into one-dimensional vectors through the query word vector library, namely the word vectors corresponding to each word.
Alternatively, the word vector library may be a word vector library of the bidirectional encoder characterization model itself. In the embodiment of the present application, the server and/or a technician may also supplement the word vector library, such as obscure words in a specific application field, and the like, and may supplement the word vector library for the current application field before implementing the method for determining the query result of the present application.
The long-time and short-time memory network is used for coding the content output by the presentation layer and obtaining the semantic vector corresponding to the input content. In the related technology, the input content is divided according to word granularity to obtain word vectors of the input content, the word vectors are subjected to averaging operation and then processed to obtain semantic vectors corresponding to the input content, such as a query word "ABCD" containing four characters, the word vectors are divided according to the word granularity to obtain "AB" and "CD", the vectorized query word "ABCD" has two corresponding word vectors, and the word vectors are subjected to averaging operation and then processed to obtain the semantic vectors corresponding to the query word "ABCD". However, the method provided by the related art has the problem of reducing the weight of the keyword. For example, the query word "tasty lollipop" is a word vector obtained by dividing the word granularity in the related technology, and the word vector is processed after averaging operation is performed on each word vector to obtain a semantic vector, wherein the word vector corresponding to "tasty" or "tasty" reduces the weight of the keyword "lollipop" after averaging operation is performed on the word vector, and finally has a certain influence on the accuracy of the semantic vector.
In the application, word granularity division of input contents is realized through a long-time memory network, and weight promotion of keywords is realized in the process of generating corresponding semantic vectors, so that semantic vectors with more accurate semantic representations are obtained.
It is necessary that the fine-tuning of the representation model and the long-term memory network structure of the bi-directional encoder for optimizing the data processing capability of the representation layer and the encoding layer is within the protection scope of the present application.
Therefore, in a possible implementation manner, after obtaining the query term included in the query request, the server inputs the query term into the presentation layer to obtain a first word vector group, where the first word vector group includes word vectors corresponding to each word in the query term.
As an example in the above embodiment, after the user inputs "organic yam" in the search engine, the server obtains the query word "organic yam", inputs the query word "organic yam" in the presentation layer of the semantic vector generation model, the server divides the query word by the word granularity through the presentation layer to obtain four divided words, i.e., "there", "mountain", and "medicine", and vectorizes and represents each word through the presentation layer, and the first word vector group may be represented as S ═ S (S1, S2, S3, S4), where S represents the query word, S1 represents a word vector corresponding to "there", S2 represents a word vector corresponding to "there", S3 represents a word vector corresponding to "mountain", and S4 represents a word vector corresponding to "medicine".
The first word vector group contains word vectors corresponding to each word in the query term.
The server obtains a first word vector corresponding to the query word through the presentation layer, and further the computer inputs the first word vector group into the coding layer to obtain a first semantic vector.
In a possible implementation manner, after a coding layer constructed by a long-short memory network is provided to obtain a first word vector group, vectorization coding is performed on each word vector according to the sequence of each word vector in the first word vector group, and a semantic vector corresponding to a query word is obtained through processing, wherein the semantic vector is different from the semantic vector obtained through averaging operation in the related art, for example, the processing of the long-short memory network improves the weight of keywords in the query word, and then weakens the weight of non-keywords, and for example, the long-short memory network can generate a semantic vector closest to the semantic representation of a specific field based on an application scene of the specific field.
Please refer to step 203, which is not described herein again in this embodiment.
And 305, determining commodity words corresponding to the candidate semantic vectors as query results responding to the query requests.
Please refer to step 204, which is not described herein again in this embodiment.
In the embodiment of the application, on the basis of the embodiment, a further explanation function is performed on the construction of a semantic vector generation model, the semantic vector generation model comprises a presentation layer and a coding layer, optionally, query words are divided by the presentation layer according to word granularity and then vectorized to obtain a first vector group corresponding to the query words, and compared with a method for dividing the query words according to word granularity in the related art, the method has the advantages that the query words are directly divided by the presentation layer according to the word granularity, so that the process of dividing the query words is reduced, and the efficiency of processing input contents by the semantic vector generation model is improved on the basis of more accurate semantic representation of the semantic vector; in addition, the first vector group output by the presentation layer is coded and processed by the coding layer to obtain the first semantic vector corresponding to the query word, and compared with a method for obtaining the semantic vector by carrying out averaging operation processing on the result after the vector quantization in the related technology, the method and the device for generating the semantic vector have the advantages that the keyword weight in the query word can be improved through the coding layer, the non-keyword weight can be weakened, and the semantic vector closest to the semantic representation in the specific field can be generated based on the application scene in the specific field.
Optionally, the semantic vector generation model shown in the above embodiment is a model obtained by training based on a training sample set, that is, before obtaining the query word included in the query request (step 201), a training process of the semantic vector generation model is further included.
Referring to fig. 4, a flowchart of training a semantic vector generation model provided by an exemplary embodiment of the present application is shown. The method comprises the following steps:
The semantic vector generation model can generate a first semantic vector of the query word and a second semantic vector of the commodity word. Therefore, the training sample set for training the semantic vector generation model includes not only the training samples related to the query word but also the training samples related to the commodity word.
Optionally, the training sample set includes sample query words, positive sample commodity words and negative sample commodity words, and the positive sample commodity words are commodity words of the relevant commodities corresponding to the sample query words. In addition, compared with a triple training sample format in the related art (that is, one training sample is composed of one sample query word, one positive sample commodity word and one negative sample commodity word), in the embodiment of the present application, in order to improve the accuracy of obtaining the commodity word of the related commodity corresponding to the query word by the semantic vector generation model, each training sample of the training sample set includes at least one negative sample commodity word, for example, one training sample is composed of one sample query word, one positive sample commodity word and four negative sample commodity words.
In one possible implementation, step 401 includes the following.
And content I, acquiring historical query terms, and determining the historical query terms as sample query terms.
Optionally, in order to make the query result more conform to the actual expectation of the user, the training sample set for training the semantic vector model in the present application is obtained based on the historical operation of the user.
Therefore, in one possible implementation, the server and/or the technician obtains the historical query terms based on the historical operations of the user from the log file of the search engine, and determines the historical query terms as the sample query terms.
And determining positive sample commodity words and negative sample commodity words from historical behavior data corresponding to the historical query words.
Wherein the historical behavior data comprises historical purchase data and historical browse data. Historical purchased commodities corresponding to the sample query terms can be obtained according to the historical purchase data, namely the commodities corresponding to the sample commodity terms are historical purchased commodities corresponding to the sample query terms; in addition, the unviewed commodities corresponding to the sample query word can be obtained according to the historical browsing data, namely the commodities corresponding to the negative sample commodity word are the unviewed commodities corresponding to the sample query word.
In one example, a server acquires a log file from a terminal, the log file corresponds to an operation record of a user at a search engine, the server acquires that a query word A input by a user A initiates a query request to the search engine in each log file corresponding to the search engine, the server provides 10 commodity words (commodity word 1 to commodity word 10) as query results to the user A according to the query request, and the user A browses commodities corresponding to the commodity words 1 to 3 and purchases commodities corresponding to the commodity word 1. The server marks the commodity word 1 as a positive sample commodity word corresponding to the query word a, and marks the commodity words 4 to 10 as negative sample commodity words corresponding to the query word a.
In addition, for the historical browsing data, the commodity word corresponding to the commodity which is browsed but not purchased can also be used as a positive sample commodity word corresponding to the sample query word, which is not limited in the embodiment of the present application. As in the above example, the server may also mark both the purchased goods (goods corresponding to the goods word 1) and the browsed goods (goods corresponding to the goods words 1 to 3) as positive sample goods words corresponding to the query word a.
Optionally, the process of inputting the training sample set into the representation layer in step 402 to obtain the sample word vector group may refer to the process of inputting the query word into the representation layer in step 302 to obtain the first word vector group.
In addition, in the embodiment of the present application, that is, in the training phase of the semantic vector model, because each training sample included in the training sample set is input in the input format of the sample query word, the positive sample commodity word, and the at least one negative sample commodity word, the sample word vector group obtained by the training sample set input representation layer is not a word vector group that is a single object.
Optionally, the sample word vector group includes a first sample word vector group corresponding to the sample query word, a second sample word vector group corresponding to the positive sample commodity word, and a third sample word vector group corresponding to the negative sample commodity word.
Optionally, the process of inputting the sample word vector group into the coding layer in step 403 to obtain the sample semantic vector may refer to the process of inputting the first word vector group into the coding layer in step 303 to obtain the first semantic vector.
Similarly, the sample semantic vector obtained by inputting the sample word vector group into the encoding layer is not a semantic vector of a single object.
Optionally, the sample semantic vector includes a first sample semantic vector corresponding to the sample query word, a second sample semantic vector corresponding to the positive sample commodity word, and a third sample semantic vector corresponding to the negative sample commodity word.
In the actual model training process, because the training sample set is obtained based on the historical operation of the user, for some commodities with low purchase rate, the log file of the search engine contains not rich relevant historical behavior data, so that the positive sample commodity words and the negative sample commodity words corresponding to the sample query words based on the commodities are not necessarily accurate, that is, the corresponding positive sample commodity words and the negative sample commodity words do not represent the purchase intention and non-purchase intention of the user in a larger range.
Therefore, in order to determine that the training sample set includes a positive sample commodity word and a corresponding sample query word that have a higher similarity and are not an erroneous positive sample commodity word, the embodiment of the present application further includes calculating the content of the positive sample vector similarity between the first sample semantic vector and the second sample semantic vector after the first sample semantic vector and the second sample semantic vector are output by the encoding layer.
Correspondingly, in order to determine that the training sample set has low similarity between the negative sample commodity word and the corresponding sample query word and is not an erroneous negative sample commodity word, the embodiment of the present application further includes calculating the content of the negative sample vector similarity between the first sample semantic vector and each of the third sample semantic vectors after the first sample semantic vector and the third sample semantic vector are output by the encoding layer.
Further, after the server calculates the positive sample vector similarity and the negative sample vector similarity, the semantic vector generation model is trained according to the positive sample vector similarity and the negative sample vector similarity.
In one possible implementation, step 405 includes the following.
And content I, determining the sample loss of the training sample set according to the positive sample vector similarity and the negative sample vector similarity.
For each training sample in the training sample set, after the output of the coding layer, there is a corresponding positive sample vector similarity and a corresponding negative sample vector similarity, and in a possible implementation, the sample loss of each training sample is calculated through a loss function, and the sample loss of the whole training sample set is finally obtained through summation.
And secondly, training a semantic vector generation model according to sample loss.
Optionally, after the loss of the sample of the training sample set is obtained through the loss function calculation, the semantic vector generation model is continuously trained according to a preset loss threshold until the loss of the sample of the training sample set is not greater than the loss threshold, that is, the server repeatedly trains the semantic vector generation model, continuously minimizes the loss function until the semantic vector generation model converges, and then the training of the semantic vector generation model is finished.
In the embodiment of the application, a training process of a semantic vector generation model before acquiring a query word contained in a query request is disclosed. The method comprises the steps that a server obtains a training sample set according to historical behavior data, the training sample set is input into a presentation layer to obtain a sample word vector group, the sample word vector group is input into a coding layer to obtain a sample semantic vector, and a trained semantic vector generation model can simultaneously generate semantic vectors with different input contents (such as query words and commodity words); in addition, the semantic vector generation model is trained by calculating the positive sample vector similarity of the first sample semantic vector and the second sample semantic vector and the negative sample vector similarity of the first sample semantic vector and each third sample semantic vector, and a loss function is introduced, so that the accuracy of the semantic vector generation model is improved.
For the semantic vector generation model, the coding layer comprises a first coding network and a second coding network, and network parameters of the first coding network and the second coding network are different. Based on the difference between the query term and the commodity term (including the positive sample commodity term and the negative sample commodity term), in one possible embodiment, the first coding network corresponds to the generation of a semantic vector of the query term, and the first coding network corresponds to the generation of a semantic vector of the commodity term.
Referring to fig. 5, a flowchart of training a semantic vector generation model provided by another exemplary embodiment of the present application is shown. The method comprises the following steps:
Please refer to step 401, which is not described herein again in this embodiment.
Please refer to step 402, which is not described herein again in this embodiment.
Further illustratively, as shown in fig. 6, the semantic vector generation model may further include an input layer and an output layer.
Optionally, the input layer may be configured to directly input the input content of the semantic vector generation model to the presentation layer, or adjust a plurality of input contents to a certain data format and then input the adjusted input contents to the presentation layer, or the input layer has a voice recognition function and an image recognition function, and may convert the input content in a non-text form to obtain the content in a text form. The input layer is shown for illustrative purposes only and is not intended to limit the present application.
Optionally, the output layer may be configured to directly output the processing result of the coding layer in the semantic vector generation model, or adjust a plurality of processing contents to a certain data format and then output the processing result, or, when the processing results of the coding layer are of different types, the output layer has a function of evaluating the processing results of different types, as shown in fig. 6, the output layer is configured to calculate the similarity of each sample vector in the foregoing embodiment, or output the processing results sequentially according to a certain arrangement rule, and so on. The output layer is shown for illustrative purposes only and is not intended to limit the present application.
In a possible implementation, after the query word is quantized through the presentation layer, a first sample word vector group is generated, and further, as shown in fig. 6, the server inputs the first sample word vector group into a first coding network corresponding to the query word in the coding layer, and generates a first sample semantic vector corresponding to the first sample word vector group through network parameters of the first coding network.
In the embodiment of the application, the semantic vectors of the commodity words corresponding to the second coding network are generated, that is, after the positive sample commodity and the negative sample commodity are vectorized and represented by the representation layer, the corresponding semantic vectors are generated by the second coding network.
In a possible embodiment, after the positive sample commodity is quantized through the presentation layer, a second sample word vector group is generated, and further, as shown in fig. 6, the server inputs the second sample word vector group into a second coding network corresponding to the commodity word in the coding layer, and generates a second sample semantic vector corresponding to the second sample word vector group through network parameters of the second coding network.
And 505, inputting the third sample word vector group into a second coding network to obtain a third sample semantic vector.
Correspondingly, after the negative sample commodity is quantized through the presentation layer, a third sample word vector group is generated, and further, as shown in fig. 6, the server inputs the third sample word vector group into a second coding network corresponding to the commodity word in the coding layer, and generates a third sample semantic vector corresponding to the third sample word vector group through network parameters of the second coding network.
Please refer to step 404, which is not described herein again in this embodiment.
And step 507, training a semantic vector generation model according to the positive sample vector similarity and the negative sample vector similarity.
Please refer to step 405, which is not described herein again in this embodiment.
In the embodiment of the application, on the basis of the above embodiment, the structure of the semantic vector generation model is further described, that is, the coding layer includes a first coding network and a second coding network, wherein the first coding network corresponds to the generation of the semantic vector of the query word, the first coding network corresponds to the generation of the semantic vector of the commodity word, and the network parameters of the first coding network and the second coding network are different; compared with the method for obtaining the semantic vector by carrying out averaging operation processing on the result after opposite quantization in the related technology, the embodiment of the application can realize generation of the semantic vector of the query word and the commodity word through different coding networks of a coding layer, and further improve the training effect of the semantic vector generation model, so that the trained semantic vector generation model can more accurately obtain the positive sample commodity word corresponding to the query word, and reduce or avoid the obtaining of the negative sample commodity word.
Optionally, for the method for obtaining the query result of the present application, obtaining the query term is a real-time task performed by the server, and obtaining the query term, and generating a first semantic vector corresponding to the query term through a semantic vector model obtained through training. The generation of the second semantic vector corresponding to each commodity word in the commodity library is not a real-time task, that is, after the training of the semantic vector generation model is completed, a set of the second semantic vectors corresponding to the current commodity library, for example, a semantic index corresponding to the commodity library, is established in advance.
Referring to fig. 7, a flowchart of a method for determining a query result according to another exemplary embodiment of the present application is shown. The method comprises the following steps:
and 701, inputting each commodity word in the commodity library into the presentation layer to obtain a second character vector group.
The second character vector group comprises character vectors corresponding to all characters in the commodity words.
Optionally, based on the application of the present application in different fields, the present application does not limit the types of the commodity libraries, for example, different application fields correspond to different commodity libraries.
In a possible implementation manner, the server inputs commodity words included in the commodity library into a presentation layer of a semantic vector generation model, the presentation layer divides each commodity word according to word granularity to obtain words included in each commodity word, further, the presentation layer converts the words included in each commodity word into one-dimensional vectors by querying a word vector library to obtain a second word vector group corresponding to the commodity word, wherein the second word vector group includes word vectors corresponding to the words in the commodity word.
The server obtains second word vectors corresponding to all commodity words in the commodity library through the presentation layer, and further the computer inputs the second word vector group into the coding layer, vectorially codes each word vector according to the sequence of each word vector in the second word vector group, and processes the vectorized word vectors to obtain second semantic vectors corresponding to the commodity words.
Since the coding layer includes the first coding network and the second coding network, where the second coding network corresponds to generation of semantic vectors of commodity words, in one possible implementation, the server inputs the second word vector group into the second coding network of the coding layer to obtain the second semantic vector.
And 703, establishing a semantic index corresponding to the commodity library according to the second semantic vector corresponding to each commodity word.
In a possible implementation manner, a server acquires second semantic vectors corresponding to a current commodity library, loads the second semantic vectors corresponding to all commodity words to a search engine, so as to establish semantic indexes corresponding to the commodity library, generates a first semantic vector corresponding to a query word when a user inputs the query word through the search engine, and then acquires at least one candidate semantic vector from the semantic indexes according to the similarity between the first semantic vector and each second semantic vector in the semantic indexes.
Please refer to step 301, which is not described herein again in this embodiment.
Please refer to step 302, which is not described herein again in this embodiment.
Please refer to step 303, which is not described herein again in this embodiment.
And step 707, determining at least one candidate semantic vector from the second semantic vectors according to the vector similarity between the first semantic vector and each second semantic vector.
Please refer to step 304, which is not described herein again in this embodiment.
Please refer to step 305, which is not described herein again in this embodiment.
On the basis of the embodiment, the embodiment of the application supplements the content of establishing the semantic index corresponding to the commodity library, thereby avoiding the addition of an operation process for generating a second semantic vector in the application process of the semantic vector generation model by the server and realizing the high efficiency of obtaining the query result.
Referring to fig. 8, a block diagram of a query result determining apparatus according to an exemplary embodiment of the present application is shown, where the apparatus includes:
a query term obtaining module 801, configured to obtain a query term included in the query request;
a semantic vector generation module 802, configured to input the query word into a semantic vector generation model to obtain a first semantic vector corresponding to the query word, where the semantic vector generation model is configured to divide input content according to word granularity and perform vectorization expression on semantics of a division result;
a semantic vector determining module 803, configured to determine at least one candidate semantic vector from the second semantic vectors according to the vector similarity between the first semantic vector and each second semantic vector, where the second semantic vector is a semantic vector corresponding to a commodity word in a commodity library;
and a query result determining module 804, configured to determine the commodity word corresponding to the candidate semantic vector as a query result for responding to the query request.
Optionally, the semantic vector generation model includes a representation layer and an encoding layer;
optionally, the semantic vector generating module 802 includes:
the first generating unit is used for inputting the query term into the representation layer to obtain a first word vector group, and the first word vector group comprises word vectors corresponding to all words in the query term;
and the second generating unit is used for inputting the first word vector group into the coding layer to obtain the first semantic vector.
Optionally, the representation layer is constructed based on a bidirectional encoder representation model, and the coding layer is constructed based on a long-term and short-term memory network.
Optionally, the apparatus further comprises:
the system comprises a first acquisition module, a second acquisition module and a third acquisition module, wherein the first acquisition module is used for acquiring a training sample set, the training sample set comprises sample query words, positive sample commodity words and at least one negative sample commodity word, and the positive sample commodity words are commodity words of related commodities corresponding to the sample query words;
a second obtaining module, configured to input the training sample set into the presentation layer to obtain a sample word vector group, where the sample word vector group includes a first sample word vector group corresponding to the sample query word, a second sample word vector group corresponding to the positive sample commodity word, and a third sample word vector group corresponding to the negative sample commodity word;
a third obtaining module, configured to input the sample word vector group into the coding layer to obtain a sample semantic vector, where the sample semantic vector includes a first sample semantic vector corresponding to the sample query word, a second sample semantic vector corresponding to the positive sample commodity word, and a third sample semantic vector corresponding to the negative sample commodity word;
a similarity calculation module, configured to calculate a positive sample vector similarity between the first sample semantic vector and the second sample semantic vector, and a negative sample vector similarity between the first sample semantic vector and each of the third sample semantic vectors;
and the model training model is used for training the semantic vector generation model according to the positive sample vector similarity and the negative sample vector similarity.
Optionally, the coding layer includes a first coding network and a second coding network, and network parameters of the first coding network and the second coding network are different;
optionally, the third obtaining module includes:
a first obtaining unit, configured to input the first sample word vector group into the first coding network, so as to obtain the first sample semantic vector;
a second obtaining unit, configured to input the second sample word vector group into the second coding network, so as to obtain a second sample semantic vector;
and the third acquisition unit is used for inputting the third sample word vector group into the second coding network to obtain the third sample semantic vector.
Optionally, the second generating unit is further configured to:
inputting the first word vector group into the first coding network to obtain the first semantic vector;
optionally, the apparatus further comprises:
the first input module is used for inputting each commodity word in the commodity library into the presentation layer to obtain a second character vector group, and the second character vector group comprises character vectors corresponding to each character in the commodity words;
a second input module, configured to input the second word vector group into the second coding network of the coding layer to obtain the second semantic vector;
and the semantic index establishing module is used for establishing the semantic index corresponding to the commodity library according to the second semantic vector corresponding to each commodity word.
Optionally, the first obtaining module includes:
the first determining unit is used for acquiring a historical query word by an acquisition module and determining the historical query word as the sample query word;
the second determining unit is configured to determine the positive sample commodity word and the negative sample commodity word from historical behavior data corresponding to the historical query word, where the historical behavior data includes historical purchase data and historical browsing data, a commodity corresponding to the positive sample commodity word is a historical purchase commodity, and a commodity corresponding to the negative sample commodity word is an unviewed commodity.
Referring to fig. 9, a schematic structural diagram of a server 900 according to an embodiment of the present application is shown. The server 900 may be used to implement the determination method of the query result provided in the above-described embodiments. The server 900 may be the server 120 described in the embodiment of fig. 1. Specifically, the method comprises the following steps:
the server 900 includes a Central Processing Unit (CPU)901, a system memory 904 including a Random Access Memory (RAM)902 and a Read Only Memory (ROM)903, and a system bus 905 connecting the system memory 904 and the central processing unit 901. The server 900 also includes a basic input/output system (I/O system) 906 for facilitating the transfer of information between devices within the computer, and a mass storage device 907 for storing an operating system 913, application programs 914, and other program modules 915.
The basic input/output system 906 includes a display 908 for displaying information and an input device 909 such as a mouse, keyboard, etc. for user input of information. Wherein the display 908 and the input device 909 are connected to the central processing unit 901 through an input output controller 910 connected to the system bus 905. The basic input/output system 906 may also include an input/output controller 910 for receiving and processing input from a number of other devices, such as a keyboard, mouse, or electronic stylus. Similarly, input-output controller 910 also provides output to a display screen, a printer, or other type of output device.
The mass storage device 907 is connected to the central processing unit 901 through a mass storage controller (not shown) connected to the system bus 905. The mass storage device 907 and its associated computer-readable media provide non-volatile storage for the server 900. That is, the mass storage device 907 may include a computer-readable medium (not shown) such as a hard disk or CD-ROM drive.
Without loss of generality, the computer-readable media may comprise computer storage media and communication media. Computer storage media includes volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. Computer storage media includes RAM, ROM, EPROM, EEPROM, flash memory or other solid state memory technology, CD-ROM, DVD, or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices. Of course, those skilled in the art will appreciate that the computer storage media is not limited to the foregoing. The system memory 904 and mass storage device 907 described above may be collectively referred to as memory.
The server 900 may also operate as a remote computer connected to a network via a network, such as the internet, according to various embodiments of the present application. That is, the server 900 may be connected to the network 912 through the network interface unit 911 coupled to the system bus 905, or the network interface unit 911 may be used to connect to other types of networks or remote computer systems (not shown).
The memory also includes one or more programs stored in the memory and configured to be executed by one or more processors. The one or more programs include instructions for implementing the method for determining the result of the query on the server side.
The memory has stored therein at least one instruction, at least one program, set of codes, or set of instructions configured to be executed by one or more processors to implement the functions of the various steps in the method of determining a query result described above.
Embodiments of the present application further provide a computer-readable storage medium, in which at least one instruction, at least one program, a code set, or a set of instructions is stored, and the at least one instruction, the at least one program, the code set, or the set of instructions is loaded and executed by a processor to implement the method for determining a query result provided in the foregoing embodiments.
Optionally, the computer-readable storage medium may include: a Read Only Memory (ROM), a Random Access Memory (RAM), a Solid State Drive (SSD), or an optical disc. The Random Access Memory may include a resistive Random Access Memory (ReRAM) and a Dynamic Random Access Memory (DRAM).
The above-mentioned serial numbers of the embodiments of the present application are merely for description and do not represent the merits of the embodiments.
It will be understood by those skilled in the art that all or part of the steps for implementing the above embodiments may be implemented by hardware, or may be implemented by a program instructing relevant hardware, where the program may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a magnetic disk or an optical disk, etc.
The above description is only exemplary of the present application and should not be taken as limiting, as any modification, equivalent replacement, or improvement made within the spirit and principle of the present application should be included in the protection scope of the present application.
Claims (10)
1. A method for determining query results, the method comprising:
acquiring a query word contained in the query request;
inputting the query word into a semantic vector generation model to obtain a first semantic vector corresponding to the query word, wherein the semantic vector generation model is used for dividing input contents according to word granularity and vectorially representing the semantics of a division result;
determining at least one candidate semantic vector from the second semantic vectors according to the vector similarity between the first semantic vector and each second semantic vector, wherein the second semantic vector is a semantic vector corresponding to a commodity word in a commodity library;
and determining the commodity words corresponding to the candidate semantic vectors as the query results responding to the query requests.
2. The method of claim 1, wherein the semantic vector generation model comprises a presentation layer and an encoding layer;
the step of inputting the query word into a semantic vector generation model to obtain a first semantic vector corresponding to the query word includes:
inputting the query word into the presentation layer to obtain a first character vector group, wherein the first character vector group comprises character vectors corresponding to all characters in the query word;
and inputting the first word vector group into the coding layer to obtain the first semantic vector.
3. The method of claim 2, wherein the representation layer is constructed based on a bi-directional encoder characterization model, and the encoding layer is constructed based on a long-and-short-term memory network.
4. The method of claim 2, wherein before obtaining the query term included in the query request, the method further comprises:
acquiring a training sample set, wherein the training sample set comprises a sample query word, a positive sample commodity word and at least one negative sample commodity word, and the positive sample commodity word is a commodity word of a related commodity corresponding to the sample query word;
inputting the training sample set into the presentation layer to obtain a sample word vector group, wherein the sample word vector group comprises a first sample word vector group corresponding to the sample query word, a second sample word vector group corresponding to the positive sample commodity word and a third sample word vector group corresponding to the negative sample commodity word,
inputting the sample word vector group into the coding layer to obtain a sample semantic vector, wherein the sample semantic vector comprises a first sample semantic vector corresponding to the sample query word, a second sample semantic vector corresponding to the positive sample commodity word and a third sample semantic vector corresponding to the negative sample commodity word;
calculating the positive sample vector similarity of the first sample semantic vector and the second sample semantic vector and the negative sample vector similarity of the first sample semantic vector and each third sample semantic vector;
and training the semantic vector generation model according to the positive sample vector similarity and the negative sample vector similarity.
5. The method of claim 4, wherein the coding layer comprises a first coding network and a second coding network, and wherein the first coding network and the second coding network have different network parameters;
inputting the sample word vector group into the coding layer to obtain a sample semantic vector, including:
inputting the first sample word vector group into the first coding network to obtain the first sample semantic vector;
inputting the second sample word vector group into the second coding network to obtain a second sample semantic vector;
and inputting the third sample word vector group into the second coding network to obtain the third sample semantic vector.
6. The method of claim 5, wherein inputting the first set of word vectors into the coding layer to obtain the first semantic vector comprises:
inputting the first word vector group into the first coding network to obtain the first semantic vector;
the method further comprises the following steps:
inputting each commodity word in the commodity library into the presentation layer to obtain a second character vector group, wherein the second character vector group comprises character vectors corresponding to each character in the commodity words;
inputting the second word vector group into the second coding network of the coding layer to obtain the second semantic vector;
and establishing a semantic index corresponding to the commodity library according to the second semantic vector corresponding to each commodity word.
7. The method of claim 4, wherein the obtaining a training sample set comprises:
acquiring historical query terms, and determining the historical query terms as the sample query terms;
and determining the positive sample commodity words and the negative sample commodity words from historical behavior data corresponding to the historical query words, wherein the historical behavior data comprises historical purchase data and historical browsing data, commodities corresponding to the positive sample commodity words are historical purchase commodities, and commodities corresponding to the negative sample commodity words are unviewed commodities.
8. An apparatus for determining query results, the apparatus comprising:
the query term acquisition module is used for acquiring query terms contained in the query request;
the semantic vector generation module is used for inputting the query word into a semantic vector generation model to obtain a first semantic vector corresponding to the query word, and the semantic vector generation model is used for dividing input content according to word granularity and vectorizing and expressing the semantics of a division result;
the semantic vector determining module is used for determining at least one candidate semantic vector from the second semantic vectors according to the vector similarity between the first semantic vector and each second semantic vector, wherein the second semantic vector is a semantic vector corresponding to a commodity word in a commodity library;
and the query result determining module is used for determining the commodity words corresponding to the candidate semantic vectors as the query results responding to the query requests.
9. A server, comprising a processor and a memory; the memory stores at least one instruction for execution by the processor to implement the method of determining a query result of any of claims 1 to 7.
10. A computer-readable storage medium having stored thereon at least one instruction for execution by a processor to perform a method of determining a result of a query as claimed in any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911408075.2A CN111078842A (en) | 2019-12-31 | 2019-12-31 | Method, device, server and storage medium for determining query result |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911408075.2A CN111078842A (en) | 2019-12-31 | 2019-12-31 | Method, device, server and storage medium for determining query result |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111078842A true CN111078842A (en) | 2020-04-28 |
Family
ID=70320437
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911408075.2A Pending CN111078842A (en) | 2019-12-31 | 2019-12-31 | Method, device, server and storage medium for determining query result |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111078842A (en) |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111737418A (en) * | 2020-07-20 | 2020-10-02 | 北京每日优鲜电子商务有限公司 | Method, apparatus and storage medium for predicting relevance of search term and commodity |
| CN111914551A (en) * | 2020-07-29 | 2020-11-10 | 北京字节跳动网络技术有限公司 | Language representation model system, pre-training method, device, equipment and medium |
| CN112182144A (en) * | 2020-12-01 | 2021-01-05 | 震坤行网络技术(南京)有限公司 | Search term normalization method, computing device, and computer-readable storage medium |
| CN112632406A (en) * | 2020-10-10 | 2021-04-09 | 咪咕文化科技有限公司 | Query method and device, electronic equipment and storage medium |
| CN113868370A (en) * | 2021-08-20 | 2021-12-31 | 深延科技(北京)有限公司 | Text recommendation method and device, electronic equipment and computer-readable storage medium |
| CN114416940A (en) * | 2021-12-28 | 2022-04-29 | 北京百度网讯科技有限公司 | Phrase generalization method and device in table question and answer, electronic equipment and storage medium |
| CN118394892A (en) * | 2024-07-01 | 2024-07-26 | 浪潮电子信息产业股份有限公司 | Question answering method, device, equipment and computer readable storage medium |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20070131094A1 (en) * | 2005-11-09 | 2007-06-14 | Sony Deutschland Gmbh | Music information retrieval using a 3d search algorithm |
| CN102880645A (en) * | 2012-08-24 | 2013-01-16 | 上海云叟网络科技有限公司 | Semantic intelligent search method |
| CN106445915A (en) * | 2016-09-14 | 2017-02-22 | 科大讯飞股份有限公司 | New word discovery method and device |
| CN106815252A (en) * | 2015-12-01 | 2017-06-09 | 阿里巴巴集团控股有限公司 | A kind of searching method and equipment |
| CN107491547A (en) * | 2017-08-28 | 2017-12-19 | 北京百度网讯科技有限公司 | Searching method and device based on artificial intelligence |
| CN108287858A (en) * | 2017-03-02 | 2018-07-17 | 腾讯科技（深圳）有限公司 | The semantic extracting method and device of natural language |
-
2019
- 2019-12-31 CN CN201911408075.2A patent/CN111078842A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20070131094A1 (en) * | 2005-11-09 | 2007-06-14 | Sony Deutschland Gmbh | Music information retrieval using a 3d search algorithm |
| CN102880645A (en) * | 2012-08-24 | 2013-01-16 | 上海云叟网络科技有限公司 | Semantic intelligent search method |
| CN106815252A (en) * | 2015-12-01 | 2017-06-09 | 阿里巴巴集团控股有限公司 | A kind of searching method and equipment |
| CN106445915A (en) * | 2016-09-14 | 2017-02-22 | 科大讯飞股份有限公司 | New word discovery method and device |
| CN108287858A (en) * | 2017-03-02 | 2018-07-17 | 腾讯科技（深圳）有限公司 | The semantic extracting method and device of natural language |
| CN107491547A (en) * | 2017-08-28 | 2017-12-19 | 北京百度网讯科技有限公司 | Searching method and device based on artificial intelligence |
Non-Patent Citations (1)
| Title |
|---|
| 李岩;张博文;郝红卫;: "基于语义向量表示的查询扩展方法" * |
Cited By (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111737418A (en) * | 2020-07-20 | 2020-10-02 | 北京每日优鲜电子商务有限公司 | Method, apparatus and storage medium for predicting relevance of search term and commodity |
| CN111737418B (en) * | 2020-07-20 | 2021-05-14 | 北京每日优鲜电子商务有限公司 | Method, apparatus and storage medium for predicting relevance of search term and commodity |
| CN111914551A (en) * | 2020-07-29 | 2020-11-10 | 北京字节跳动网络技术有限公司 | Language representation model system, pre-training method, device, equipment and medium |
| CN112632406A (en) * | 2020-10-10 | 2021-04-09 | 咪咕文化科技有限公司 | Query method and device, electronic equipment and storage medium |
| CN112632406B (en) * | 2020-10-10 | 2024-04-09 | 咪咕文化科技有限公司 | Query method, query device, electronic equipment and storage medium |
| CN112182144A (en) * | 2020-12-01 | 2021-01-05 | 震坤行网络技术(南京)有限公司 | Search term normalization method, computing device, and computer-readable storage medium |
| CN112182144B (en) * | 2020-12-01 | 2021-03-05 | 震坤行网络技术(南京)有限公司 | Search term normalization method, computing device, and computer-readable storage medium |
| CN113868370A (en) * | 2021-08-20 | 2021-12-31 | 深延科技(北京)有限公司 | Text recommendation method and device, electronic equipment and computer-readable storage medium |
| CN114416940A (en) * | 2021-12-28 | 2022-04-29 | 北京百度网讯科技有限公司 | Phrase generalization method and device in table question and answer, electronic equipment and storage medium |
| CN118394892A (en) * | 2024-07-01 | 2024-07-26 | 浪潮电子信息产业股份有限公司 | Question answering method, device, equipment and computer readable storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111078842A (en) | Method, device, server and storage medium for determining query result | |
| CN108536852B (en) | Question-answer interaction method and device, computer equipment and computer readable storage medium | |
| US20180158078A1 (en) | Computer device and method for predicting market demand of commodities | |
| CN105989040B (en) | Intelligent question and answer method, device and system | |
| US20180336193A1 (en) | Artificial Intelligence Based Method and Apparatus for Generating Article | |
| CN111797210A (en) | Information recommendation method, device and equipment based on user portrait and storage medium | |
| US20230130006A1 (en) | Method of processing video, method of quering video, and method of training model | |
| US20220318275A1 (en) | Search method, electronic device and storage medium | |
| CN111581926B (en) | Document generation method, device, equipment and computer readable storage medium | |
| CN111581923A (en) | Method, device and equipment for generating file and computer readable storage medium | |
| CN110347908B (en) | Voice shopping method, device, medium and electronic equipment | |
| US11238050B2 (en) | Method and apparatus for determining response for user input data, and medium | |
| CN114861889B (en) | Deep learning model training method, target object detection method and device | |
| CN112818091A (en) | Object query method, device, medium and equipment based on keyword extraction | |
| CN115080742B (en) | Text information extraction method, apparatus, device, storage medium, and program product | |
| CN111198939A (en) | Statement similarity analysis method and device and computer equipment | |
| CN110633398A (en) | Method for confirming central word, searching method, device and storage medium | |
| CN114238689A (en) | Video generation method, video generation device, electronic device, storage medium, and program product | |
| US20090327877A1 (en) | System and method for disambiguating text labeling content objects | |
| US12061611B2 (en) | Search method, apparatus, electronic device, storage medium and program product | |
| CN117076636A (en) | Information query method, system and equipment for intelligent customer service | |
| CN116662495A (en) | Question-answering processing method, and method and device for training question-answering processing model | |
| CN117494815A (en) | File-oriented credible large language model training and reasoning method and device | |
| CN112906368B (en) | Industry text increment method, related device and computer program product | |
| CN117609612A (en) | Resource recommendation method and device, storage medium and electronic equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20200428 |
|
| WD01 | Invention patent application deemed withdrawn after publication |