CN114706894A - Information processing method, apparatus, device, storage medium, and program product - Google Patents
Information processing method, apparatus, device, storage medium, and program product Download PDFInfo
- Publication number
- CN114706894A CN114706894A CN202210417765.XA CN202210417765A CN114706894A CN 114706894 A CN114706894 A CN 114706894A CN 202210417765 A CN202210417765 A CN 202210417765A CN 114706894 A CN114706894 A CN 114706894A
- Authority
- CN
- China
- Prior art keywords
- query
- mapping relation
- target
- feature
- actual
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 230000010365 information processing Effects 0.000 title claims abstract description 57
- 238000003672 processing method Methods 0.000 title claims abstract description 31
- 238000013507 mapping Methods 0.000 claims abstract description 311
- 238000004364 calculation method Methods 0.000 claims abstract description 133
- 238000000034 method Methods 0.000 claims abstract description 45
- 238000012545 processing Methods 0.000 claims description 35
- 230000001960 triggered effect Effects 0.000 claims description 30
- 238000005516 engineering process Methods 0.000 claims description 19
- 230000004044 response Effects 0.000 claims description 15
- 238000004590 computer program Methods 0.000 claims description 14
- 230000001360 synchronised effect Effects 0.000 claims description 12
- 230000006399 behavior Effects 0.000 claims description 9
- 230000010354 integration Effects 0.000 claims description 8
- 238000004458 analytical method Methods 0.000 claims description 7
- 238000000605 extraction Methods 0.000 claims description 5
- 238000013135 deep learning Methods 0.000 abstract description 2
- 238000012821 model calculation Methods 0.000 description 15
- 230000008859 change Effects 0.000 description 12
- 238000004891 communication Methods 0.000 description 8
- 238000007726 management method Methods 0.000 description 8
- 238000010801 machine learning Methods 0.000 description 7
- 230000008569 process Effects 0.000 description 7
- 238000010586 diagram Methods 0.000 description 6
- 230000000694 effects Effects 0.000 description 4
- 230000006870 function Effects 0.000 description 3
- 230000003993 interaction Effects 0.000 description 3
- 230000004048 modification Effects 0.000 description 3
- 238000012986 modification Methods 0.000 description 3
- 230000003287 optical effect Effects 0.000 description 3
- 238000013473 artificial intelligence Methods 0.000 description 2
- 238000013461 design Methods 0.000 description 2
- 238000009826 distribution Methods 0.000 description 2
- 230000008713 feedback mechanism Effects 0.000 description 2
- 230000007246 mechanism Effects 0.000 description 2
- 238000011144 upstream manufacturing Methods 0.000 description 2
- 230000009471 action Effects 0.000 description 1
- 238000003491 array Methods 0.000 description 1
- 230000008901 benefit Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 230000001413 cellular effect Effects 0.000 description 1
- 239000002131 composite material Substances 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 238000013523 data management Methods 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000000977 initiatory effect Effects 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 230000002085 persistent effect Effects 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 230000001953 sensory effect Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 238000012549 training Methods 0.000 description 1
- 230000007704 transition Effects 0.000 description 1
- 238000009827 uniform distribution Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2455—Query execution
- G06F16/24552—Database cache management
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
- G06F16/2228—Indexing structures
- G06F16/2255—Hash tables
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/23—Updating
- G06F16/2308—Concurrency control
- G06F16/2315—Optimistic concurrency control
- G06F16/2322—Optimistic concurrency control using timestamps
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/23—Updating
- G06F16/2365—Ensuring data consistency and integrity
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/23—Updating
- G06F16/2379—Updates performed during online database operations; commit processing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2458—Special types of queries, e.g. statistical queries, fuzzy queries or distributed queries
- G06F16/2468—Fuzzy queries
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Databases & Information Systems (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Software Systems (AREA)
- Fuzzy Systems (AREA)
- Mathematical Physics (AREA)
- Computational Linguistics (AREA)
- Computer Security & Cryptography (AREA)
- Automation & Control Theory (AREA)
- Probability & Statistics with Applications (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The disclosure provides an information processing method, an information processing device, information processing equipment, a storage medium and a program product, and relates to the technical field of deep learning and data synchronization. The method comprises the following steps: extracting actual query words and actual query features from the acquired query request; respectively extracting time stamps corresponding to the actual query words from the mapping relation data stored in the local storage space and the cache; determining mapping relation data containing the latest timestamp as target mapping relation data; when the actual query features are consistent with the query features in the target mapping relation data and the target mapping relation data are obtained from local storage, taking the calculation results in the target mapping relation data as target calculation results; updating the mapping relation data which is stored in the cache and contains the same model number by using the target mapping relation data; and generating a query result corresponding to the query request based on the target calculation result. The cache hit rate is improved through management of model granularity, and the feedback efficiency of results is improved.
Description
Technical Field
The present disclosure relates to the field of data processing technologies, and in particular, to the field of deep learning and data synchronization technologies, and in particular, to an information processing method and apparatus, an electronic device, a computer-readable storage medium, and a computer program product.
Background
With the gradual development of artificial intelligence technology, understanding of user search intention or processing of user query request based on natural language understanding is increasingly emphasized, so that feedback accuracy and feedback efficiency of search results are improved by better identifying real search or query intention of a user.
In the case of fuzzy search, a user search intention understanding system or a user query request processing system cannot accurately determine which vertical class a user search intention or a query request belongs to, and therefore, it is a typical computation-intensive service to mount hundreds of machine learning models which belong to different vertical classes (i.e., each machine learning model corresponds to user search intention understanding or user query request processing under one vertical class).
As model magnitude and strategy complexity continue to increase, system concurrency and stability are both greatly challenged. Therefore, how to accurately and efficiently feed back search results or query results in such a scenario is a technical problem to be urgently solved by those skilled in the art.
Disclosure of Invention
The embodiment of the disclosure provides an information processing method, an information processing device, an electronic device, a computer readable storage medium and a computer program product.
In a first aspect, an embodiment of the present disclosure provides an information processing method, including: extracting actual query words and actual query features from the acquired query request; respectively extracting time stamps corresponding to the actual query words from the mapping relation data stored in the local storage and the cache; the mapping relation data comprises corresponding relations among query words, model numbers, time stamps, query characteristics and calculation results, and the mapping relation data stored in the cache is synchronous with the mapping relation data generated by local storage; determining mapping relation data containing the latest timestamp as target mapping relation data; in response to the fact that the actual query features are consistent with the query features in the target mapping relation data and the target mapping relation data are obtained from local storage, taking a calculation result in the target mapping relation data as a target calculation result; updating the mapping relation data which is stored in the cache and contains the same model number by using the target mapping relation data; and generating a query result corresponding to the query request based on the target calculation result.
In a second aspect, an embodiment of the present disclosure provides another information processing method, including: extracting actual query words and actual query features from the acquired query request; respectively extracting time stamps corresponding to the actual query words from the mapping relation data stored in the local storage and the cache; the mapping relation data comprises corresponding relations among query words, model numbers, time stamps, query characteristics and calculation results, and the mapping relation data stored in the cache is synchronous with mapping relation data generated by local storage; determining mapping relation data containing the latest timestamp as target mapping relation data; in response to the fact that the actual query features are not consistent with the query features in the target mapping relation, calculating a target calculation result based on the model corresponding to the target mapping relation data and the actual query features; determining a new timestamp according to the generation time of the target calculation result, and generating new mapping relation data based on the actual query word, the model number, the new timestamp, the actual query feature and the target calculation result; updating the mapping relation data which is stored in the cache and contains the same model number by using the new mapping relation data; and generating a query result corresponding to the query request based on the target calculation result.
In a third aspect, an embodiment of the present disclosure provides an information processing apparatus, including: the query request analysis unit is configured to extract actual query words and actual query characteristics from the acquired query request; the time stamp extracting unit is configured to extract time stamps corresponding to the actual query words from the mapping relation data stored in the local storage and the cache respectively; the mapping relation data comprises corresponding relations among query words, model numbers, time stamps, query characteristics and calculation results, and the mapping relation data stored in the cache is synchronous with mapping relation data generated by local storage; a target mapping relationship data determination unit configured to determine mapping relationship data containing the latest time stamp as target mapping relationship data; a first processing unit configured to take a calculation result in the target mapping relationship data as a target calculation result in response to the actual query feature having consistency with a query feature in the target mapping relationship data and the target mapping relationship data being taken from a local storage; a first updating unit configured to update the mapping relationship data containing the same model number stored in the cache using the target mapping relationship data; and a query result generation unit configured to generate a query result corresponding to the query request based on the target calculation result.
In a fourth aspect, an embodiment of the present disclosure provides another information processing apparatus, including: the query request analysis unit is configured to extract actual query words and actual query features from the acquired query request; the time stamp extracting unit is configured to extract time stamps corresponding to the actual query words from the mapping relation data stored in the local storage and the cache respectively; the mapping relation data comprises corresponding relations among query words, model numbers, time stamps, query characteristics and calculation results, and the mapping relation data stored in the cache is synchronous with mapping relation data generated by local storage; a target mapping relationship data determination unit configured to determine mapping relationship data containing the latest time stamp as target mapping relationship data; a second processing unit configured to calculate a target calculation result based on the model corresponding to the target mapping data and the actual query feature in response to the actual query feature not being consistent with the query feature in the target mapping; a new mapping relation generating unit configured to determine a new timestamp according to a generation time of the target calculation result and generate new mapping relation data based on the actual query word, the model number, the new timestamp, the actual query feature, and the target calculation result; a second updating unit configured to update the mapping relationship data containing the same model number stored in the cache with the new mapping relationship data; and the query result generation unit is configured to generate a query result corresponding to the query request based on the target calculation result.
In a fifth aspect, an embodiment of the present disclosure provides an electronic device, including: at least one processor; and a memory communicatively coupled to the at least one processor; wherein the memory stores instructions executable by the at least one processor to cause the at least one processor to perform the method of information as described in any implementation of the first aspect or any implementation of the second aspect when executed.
In a fourth aspect, the present disclosure provides a non-transitory computer-readable storage medium storing computer instructions for enabling a computer to implement, when executed, the information processing method as described in any one of the implementation manners of the first aspect or any one of the implementation manners of the second aspect.
In a fifth aspect, the present disclosure provides a computer program product including a computer program, where the computer program is capable of implementing the information processing method as described in any one of the implementation manners of the first aspect or any one of the implementation manners of the second aspect when executed by a processor.
It should be understood that the statements in this section do not necessarily identify key or critical features of the embodiments of the present disclosure, nor do they limit the scope of the present disclosure. Other features of the present disclosure will become apparent from the following description.
Drawings
Other features, objects and advantages of the disclosure will become more apparent upon reading of the following detailed description of non-limiting embodiments thereof, made with reference to the accompanying drawings in which:
fig. 1 is a flowchart of an information processing method according to an embodiment of the present disclosure;
fig. 2 is a flowchart of another information processing method provided by the embodiment of the present disclosure;
fig. 3 is a flowchart of another information processing method provided by the embodiment of the present disclosure;
fig. 4 is a flowchart of a method for determining whether an actual query feature is consistent with a query feature in target mapping relationship data according to an embodiment of the present disclosure;
fig. 5 is a schematic flowchart of an information processing method in an application scenario according to an embodiment of the present disclosure;
fig. 6 is a block diagram of an information processing apparatus according to an embodiment of the present disclosure;
fig. 7 is a block diagram of another information processing apparatus according to an embodiment of the present disclosure;
fig. 8 is a schematic structural diagram of an electronic device adapted to execute an information processing method according to an embodiment of the present disclosure.
Detailed Description
Exemplary embodiments of the present disclosure are described below with reference to the accompanying drawings, in which various details of the embodiments of the disclosure are included to assist understanding, and which are to be considered as merely exemplary. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the present disclosure. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness. It should be noted that, in the present disclosure, the embodiments and features of the embodiments may be combined with each other without conflict.
In the technical scheme of the disclosure, the collection, storage, use, processing, transmission, provision, disclosure and other processing of the personal information of the related user are all in accordance with the regulations of related laws and regulations and do not violate the good customs of the public order.
Referring to fig. 1, fig. 1 is a flowchart of an information processing method according to an embodiment of the disclosure, where the process 100 includes the following steps:
step 101: extracting actual query words and actual query features from the acquired query request;
the present step is intended to extract, by an execution subject (for example, a data storage server or a user terminal device) of the information processing method, an actual query term (hereinafter, collectively referred to as an actual query term, which is used to distinguish query terms from other sources) and an actual query feature (hereinafter, collectively referred to as an actual query feature, which is used to distinguish query features from other sources) from a received query request.
The actual query term is usually given directly by the object initiating the query request, and may be physically typed in directly by means of a keyboard and a touch screen, or may be obtained by analyzing an incoming voice signal, such as "Yuanmingyuan", "XXX (a movie name)"; the actual query feature is used for representing a relevant feature accompanying an actual query term, and is used for embodying a search or query intention of different users based on different starting points in different scenes, and may include at least one of a client feature when a query request is triggered, a behavior feature when the query request is triggered, a public sentiment feature when the query request is triggered, and a sound feature when the query request is triggered, for example.
Taking the client characteristics when triggering the query request as an example, the current clients can be roughly divided into: the method comprises the following steps that a Personal Computer (PC) end, an Android (Android) mobile phone end and an apple (IOS) mobile phone end are classified, the same object possibly reflects different search intentions of the object to a query word through different types of client ends based on the query request initiated by the same query word, and for example, when the query word is strongly associated with a certain type of client end, a search result can be obviously influenced; taking the public sentiment characteristics when triggering the query request as an example, if a certain query word has different interpretation ways in the past and the present or implicit associations are newly added due to the change of the public sentiment characteristics, the accuracy of the search result is inevitably caused. Other characteristics are similar, and influence on accuracy of search results is caused to different degrees in different dimensions, so that scientific and effective analysis is necessary to be performed on non-query word data included in a query request, so that as much related data as possible is obtained, and finally, query characteristics as comprehensive as possible are obtained through analysis.
Furthermore, when the comprehensive query features are formed by the multiple specific features, corresponding feature comprehensive weights can be preset for each feature forming the query features, so that the finally generated query features are more in line with the actual situation by combining all the weights.
Step 102: extracting time stamps corresponding to the actual query words from the mapping relation data stored in the local storage and the cache respectively;
on the basis of step 101, this step is intended to extract, by the execution subject described above, timestamps corresponding to actual query terms from the mapping relationship data stored in the local storage and the cache, respectively.
The term "local storage" and "cache" described in the present application refer to storage media for storing data or information, and the difference is that the read-write speed of data stored in the "cache" is significantly higher than the read-write speed of data stored in the "local storage", so that the feedback efficiency of query is improved by means of the "cache" storage medium with high-speed read-write characteristics.
Specifically, in some scenarios, "local storage" and "Cache (Cache)" may simply refer to "Read Only Memory (ROM)" and "Random Access Memory (RAM)", where ROM generally represents a non-power-off volatile persistent storage hard disk (having a larger capacity compared to the capacity of the Memory), and RAM generally represents a power-off volatile storage medium Memory (having a smaller capacity compared to the capacity of the hard disk); in other scenarios, a Cache may be understood as a memory located between a Central Processing Unit (CPU) and a memory, and has a faster data read-write speed than the memory, and when the CPU reads or writes data into the memory, the data may also be stored in the Cache, and when the CPU needs the data again, the data may be directly read from the Cache instead of the memory, thereby further increasing the read-write speed. However, neither of the above concepts does not affect the understanding of the two concepts of "local storage" and "cache" described in the present application, and the core is that "cache" will be used as a storage medium with a data read-write speed faster than that of "local storage" to improve the data feedback efficiency by virtue of the high-speed data read-write capability of "cache".
The mapping relation data comprises corresponding relations among query words, model numbers, time stamps, query features and calculation results, namely under one query word, each numbered model is calculated to obtain one calculation result based on the corresponding query features and the query words, and corresponding time is generated to be the time stamps. However, in general, the mapping relationship data stored in the cache is synchronized with the mapping relationship data generated from the local storage (i.e., the original mapping relationship data is originally generated in the local storage), and therefore, when the model structure and the query feature are not changed, the mapping relationship data stored in the cache should be identical with the mapping relationship data in the local storage, and are all the latest.
Step 103: determining mapping relation data containing the latest timestamp as target mapping relation data;
on the basis of step 102, this step is intended to determine mapping relationship data containing the latest timestamp as target mapping relationship data finally by the execution subject comparing the corresponding times of the two timestamps from the local storage and the cache respectively.
Step 104: in response to the fact that the actual query features are consistent with the query features in the target mapping relation data and the target mapping relation data are obtained from local storage, taking a calculation result in the target mapping relation data as a target calculation result;
on the basis of step 103, this step is intended to take the calculation result in the target mapping relationship as the target calculation result in the case that the above-mentioned execution subject finds that the actual query feature is consistent with the query feature in the target mapping relationship data, and the target mapping relationship data is taken from the local storage (i.e. there is mapping relationship data corresponding to the same model number in the local storage, which is new to the cache).
The actual query features are consistent with the query features in the target mapping relation data, which indicates that the actual query features corresponding to the current query request are consistent with the query features recorded in the stored mapping relation data, so that the calculation result obtained by calculation based on the query features recorded in the mapping relation data is still effective and available, and the calculation result in the target mapping relation data can be used as the target calculation result.
Since the local storage has mapping relationship data corresponding to the same model number, which is new to the cache, the rough rate indicates that after the mapping relationship data in the local storage is synchronized to the cache at the last time, the model corresponding to the model number is locally changed (for example, the model structure and the model parameters are changed), and the change causes that the query result corresponding to the same query word is different from that before the change, but a new synchronization operation has not yet been triggered. Therefore, by determining whether the target mapping relation data is taken from the local storage or the cache, it can be further determined whether the corresponding mapping relation data stored in the cache is still in an available state.
Step 105: updating the mapping relation data which is stored in the cache and contains the same model number by using the target mapping relation data;
on the basis of step 104, in this step, when the corresponding mapping relationship data stored in the cache by the execution main body is in an unavailable state, the mapping relationship data stored in the cache and containing the same model number is updated by using the target mapping relationship data, so that subsequent identical query requests can be directly named for the cache, and an accurate query result can be quickly returned.
Step 106: and generating a query result corresponding to the query request based on the target calculation result.
On the basis of step 105, this step is intended to generate a query result corresponding to the query request by the execution principal described above based on the target calculation result. The query result is typically generated based on the target settlement result in the updated mapping relationship data already stored in the cache, via step 105. In some cases, however, the target computation results in the local store may actually be used to generate the query results.
In the scenario addressed by the present application, it is necessary to return a sufficiently comprehensive and accurate query result by means of a plurality of machine learning models respectively corresponding to different vertical classes, so that it is generally necessary to determine whether to obtain target calculation results corresponding to all model numbers, or target calculation results corresponding to sufficiently large vertical classes of machine learning models, or target calculation results corresponding to sufficiently important vertical classes of machine learning models, when step 106 is executed, and further generate a query result corresponding to the query request based on the target calculation results.
According to the information processing method provided by the embodiment of the disclosure, the fine management granularity of the model granularity is provided through the mapping relation data formed based on the model number, the timestamp and the query characteristic, and the cache data which are unavailable and need to be updated pertinently are determined based on the comparison of the timestamps in the mapping relation data stored in the local storage and the cache, so that the latest model calculation results stored in the cache are controlled as much as possible through a scientific updating strategy, the hit rate of directly hitting the query result corresponding to the query request into the cache is further improved, and the feedback efficiency of the query result is finally improved.
Referring to fig. 2, fig. 2 is a flowchart of another information processing method according to an embodiment of the disclosure, where the process 200 includes the following steps:
step 201: extracting actual query words and actual query features from the acquired query request;
step 202: extracting time stamps corresponding to the actual query words from the mapping relation data stored in the local storage and the cache respectively;
step 203: determining mapping relation data containing the latest timestamp as target mapping relation data;
Step 204: in response to the fact that the actual query features are not consistent with the query features in the target mapping relation, calculating a target calculation result based on the model corresponding to the target mapping relation data and the actual query features;
on the basis of step 203, this step is intended to calculate a target calculation result based on the model corresponding to the target mapping relationship data and the actual query feature when the execution subject finds that the actual query feature does not have consistency with the query feature in the target mapping relationship data.
The actual query features are not consistent with the query features in the target mapping relationship data, which indicates that the actual query features corresponding to the current query request are not consistent with the query features recorded in the stored mapping relationship data, so that it can be considered that the calculation result obtained by calculation based on the original query features recorded in the mapping relationship data no longer has a use value, and therefore, an available target calculation result needs to be obtained by recalculation based on the model corresponding to the target mapping relationship data and the actual query features.
Step 205: determining a new timestamp according to the generation time of the target calculation result, and generating new mapping relation data based on the actual query word, the model number, the new timestamp, the actual query feature and the target calculation result;
on the basis of step 204, this step is intended to determine a new time stamp by the execution subject according to the generation time of the target calculation result, and generate new mapping relation data based on the actual query word, the model number, the new time stamp, the actual query feature, and the target calculation result. That is, the new mapping relationship data updates the calculation result, the query feature, and the timestamp compared to the target mapping relationship data.
The new timestamp may be directly the generation time of the target calculation result, or may be a time calculated according to the generation time.
Step 206: updating the mapping relation data which is stored in the cache and contains the same model number by using the new mapping relation data;
on the basis of step 205, this step is intended to update, by the execution subject described above, the mapping relationship data stored in the cache that contains the same model number with the new mapping relationship data.
Step 207: and generating a query result corresponding to the query request based on the target calculation result.
Different from the solution given by the process 100 when the actual query feature and the query feature in the target mapping relationship data have consistency and the target mapping relationship data is obtained from the local storage, the present embodiment is another solution given when the actual query feature and the query feature in the target mapping relationship data do not have consistency, but core points of both embodiments provide a fine management granularity of the model granularity through the mapping relationship data formed based on the model number, the timestamp and the query feature, and determine which cache data are unavailable and need to be updated in a targeted manner based on comparing timestamps in the mapping relationship data stored in the local storage and the cache, so as to control the latest model calculation result stored in the cache as much as possible through a scientific updating strategy and further improve the hit rate of directly hitting the query result corresponding to the query request into the cache, and finally, the query result feedback efficiency is improved.
In order to better understand all the possible branches in practical situations, in this embodiment, when the processing branches of fig. 1 and fig. 2 are combined, a flowchart of another information processing method is further shown by fig. 3, which further embodies a third processing branch, where the flowchart 300 includes the following steps:
step 301: extracting actual query words and actual query features from the acquired query request;
step 302: respectively extracting time stamps corresponding to the actual query words from the mapping relation data stored in the local storage and the cache, and determining the mapping relation data containing the latest time stamp as target mapping relation data;
the above steps 301 and 302 are the same as the previous steps shown in fig. 1 and 2, and the contents of the same portions refer to the corresponding portions of the previous embodiment, which are not described herein again.
Step 303: judging whether the actual query features are consistent with the query features in the target mapping relation data or not, if not, executing a step 304, and if so, executing a step 307;
the step aims to judge whether the actual query features are consistent with the query features in the target mapping relation data or not by the execution main body, and select different subsequent processing branches according to the judgment result.
In consideration of the fact that there are rarely two different query requests accompanied by exactly the same query features, when making a decision as to whether there is consistency, the decision is usually not made based on whether they are exactly the same. One implementation, including and not limited to, may refer to the flowchart shown in fig. 4:
step 401: calculating the difference degree between the actual query feature and the query feature in the target mapping relation data according to the preset dimension;
the preset dimension is used for representing a feature dimension which can be used for distinguishing the difference degree, such as a behavior feature dimension, a public opinion feature dimension, a client type dimension and the like, and can also be other higher or lower dimensions, and is mainly used for representing a numerical value which influences the calculated difference degree in an actual application scene, so that the numerical value is closer to the requirement in the actual application scene.
Step 402: judging whether the difference degree exceeds a preset difference threshold value, if so, executing a step 404, otherwise, executing a step 403;
step 403: determining that the actual query features are consistent with the query features in the target mapping relation data;
step 404: and calculating the difference degree between the actual query feature and the query feature in the target mapping relation data according to the preset dimension.
Based on the implementation shown in fig. 4, in the case that the difference between the actual query feature and the query feature in the target mapping relationship data is small, it can still be considered that the actual query feature and the query feature in the target mapping relationship data have consistency, and in the case that the actual query feature and the query feature in the target mapping relationship data have consistency, it is not necessary to re-calculate based on the actual query feature to obtain a new calculation result, that is, the calculation result recorded in the target mapping relationship data can still be continuously used.
Step 304: calculating a target calculation result based on the model corresponding to the target mapping relation data and the actual query characteristics;
in this step, if the determination result in step 303 is that the actual query feature does not have consistency with the query feature in the target mapping relationship data, since the actual query feature and the query feature do not have consistency, that is, the calculation result recorded in the target mapping relationship data cannot be used continuously, the calculation needs to be recalculated based on the model corresponding to the target mapping relationship data and the actual query feature to obtain the usable target calculation result.
Step 305: determining a new timestamp according to the generation time of the target calculation result, and generating a new mapping relation based on the actual query word, the model number, the new timestamp, the actual query feature and the target calculation result;
step 306: updating the mapping relation data which is stored in the cache and contains the same model number by using the new mapping relation;
steps 304-306 are identical to steps 204-206 of the process 200, and will not be described herein.
Step 307: judging whether the target mapping relation data is taken from a cache, if so, executing step 308, and if so, executing step 310;
in this step, when the determination result in step 303 is that there is consistency between the actual query feature and the query feature in the target mapping relationship data, it is intended that the execution subject determines whether the target mapping relationship data is fetched from the cache, and may be equivalently understood as "determining where or from which the target mapping relationship data is fetched". It should be noted that "where to fetch" described in this step refers to determining where mapping relationship data including the latest "timestamp" is stored between "local storage" and "cache", and when the timestamp in the mapping relationship data stored in "local storage" is not newer than that in "cache", it is considered that target mapping relationship data is fetched from "cache" because "cache" has higher data read-write capability than "local storage"; on the contrary, the target mapping relation data is considered to be taken from the local storage only when the time stamp in the mapping relation data stored in the local storage is new to the cache.
Step 308: taking a calculation result in the target mapping relation data as a target calculation result;
this step is established in order for the execution subject to take the calculation result in the target mapping relationship data as the target calculation result in the case where the determination result in step 307 is that the target mapping relationship data is not taken from the cache or is taken from the local storage.
Step 309: updating mapping relation data corresponding to the same model number in the cache by using the target mapping relation data;
on the basis of step 308, this step is intended to update, by the execution subject, mapping data corresponding to the same model number in the cache by using the target mapping data, and achieve the purpose of synchronizing the latest mapping to the cache.
Step 310: taking a calculation result in the target mapping relation data as a target calculation result;
this step is established in that, in the case that the determination result in step 307 is that the target mapping relationship data is taken from the cache, it indicates that the data stored in the cache can be directly used, and the execution subject directly uses the calculation result in the target mapping relationship data as the target calculation result.
Step 311: and generating a query result corresponding to the query request based on the target calculation result.
On the basis of the embodiment shown in fig. 2, the embodiment provides a two-layer judgment type processing scheme performed in sequence through steps 303 to 310, further comprehensively considers more other situations including the situations described in steps 104 to 105 shown in the flow 100 and the situations described in steps 204 to 206 shown in the flow 200, and further comprehensively considers a synchronization update policy between local storage and cache of fine management granularity and mapping relationship data specific to a model, so as to finally ensure that a query request can have a higher hit rate in the cache, and also improve the feedback efficiency of a query result.
On the basis of any of the above embodiments, this embodiment mainly provides a specific implementation manner for how to construct mapping relationship data that records correspondence between query terms, model numbers, timestamps, query features, and calculation results, so as to facilitate understanding of an actual expression form and a use manner of the mapping relationship data:
creating models with different model numbers into different model sub-buckets by utilizing a sub-bucket technology;
loading different model buckets into a unified HashMap through a HashMap technology, and establishing a mapping relation table from model numbers to time stamps;
and generating mapping relation data based on the mapping relation table and the corresponding query words, query characteristics and calculation results.
Among them, the hash method (chinese is also called as Hashing method) is a method of converting a character string composed of characters into a numerical value or an index value of a fixed length (generally, a shorter length). Since database searching is faster with shorter hash values than with the original values, this method is typically used to index and search databases, as well as in various decryption algorithms.
Whereas HashMap is an asynchronous implementation of the Map interface based on hash tables. This implementation provides all optional mapping operations and allows null values and null keys to be used. HashMap stores key-value pairs (key-values), which are fast. Such does not guarantee the order of the mapping, in particular it does not guarantee that the order is constant. The internal structure of the HashMap can be regarded as a composite structure formed by combining an array and a linked list, the array is divided into buckets (buckets), each bucket stores one or more Entry objects, each Entry object comprises three parts of key (key), value and next (point to the next Entry), and the addressing of the Entry object in the array is determined through a hash value; entry objects (key value pairs) with the same hash value are stored in a linked list. If the size of the linked list exceeds the THRESHOLD of tree transition (tree _ THRESHOLD 8), the linked list is modified to a tree structure.
In order to deepen understanding, the present disclosure further provides a caching technology based on model granularity in combination with a specific application scenario, so as to mark a time mark on a model in a time dimension of model updating, and realize incremental calculation of model granularity and cache incremental updating by calculating differences (also called diff calculation) of the model in a time sequence, thereby achieving the effects of high cache hit rate, high timeliness, consistency, low latency and saving of calculation resources.
The model granularity caching technology is realized based on diff calculation design between a model time sequence and a query characteristic time sequence: the system (namely the execution main body) locally (namely locally stores) maintains time sequences of all models and query characteristics, a cache also stores output data (namely the calculation results described above) and query characteristics calculated by all models, maintains a set of time sequences of the models and the query characteristics, carries out diff calculation on the two time sequences to obtain an updated model and characteristic list, carries out model calculation only on the list, and feeds back calculation result data to the cache to form a forward circulation feedback mechanism to realize incremental updating. Specific system components and interactions are shown in fig. 5:
to achieve the above objects, the following is set forth in detail in connection with the key problems to be solved:
1. model change and feature change awareness issues
The system needs to identify the demand of a query request (query), needs a large amount of online calculation of machine learning models, thousands of model changes and feature changes, has large change frequency difference of different models, and has half an hour level, a day level and a week level.
In this embodiment, time series markers are used to sense changes in the model. When the model offline training is completed, a timestamp (timestamp) is synchronously marked on the model, the offline model is distributed to online service instances through a uniform distribution platform, the instances complete the loading of the model and complete the loading of the full or incremental model timestamps through cold start and hot loading technologies, the model timestamps in sub-buckets are loaded into a uniform Hash diagram, a mapping relation table from model numbers to timestamps is established, and the system refreshes the latest timestamps into the mapping relation table in time along with the change of the model. A set of specific data structures is maintained in the cache, and the structures are used for storing the output data calculated by the model and the mapping relation table.
When a first query request comes, feature query and full-scale model calculation are triggered, data calculated by the model and a corresponding mapping relation table are written into the data structure, feature data are stored into the data structure, time stamps are printed on the feature data, a time sequence is maintained by the local system and the cache system, and therefore efficient and accurate sensing models and feature change are achieved.
2. Model-granular cache update problem
In the traditional caching technology, the caching granularity is in the level of a query request (or a query word), the effective control cannot be refined, only the whole effect can be achieved, the whole is invalid, and differential incremental updating cannot be carried out.
When a second query request comes, firstly querying a cache system, analyzing cache data, and acquiring a mapping relation table, wherein local mapping tables and mapping tables in the cache are based on time series, and the two tables are subjected to diff calculation of the time series to acquire a model list of the latest time series; then, analyzing the feature data to obtain the feature data in an effective period, immediately performing feature query on the invalid feature data, and fusing the two feature data to form a feature list; and then, putting the latest model list and the latest feature list into a model calculation center for model calculation, writing the model calculation result data and the latest feature data back into the cached data structure, and simultaneously writing the timestamps of the latest model and the latest feature list into a cached mapping relation table.
And subsequent query requests continuously perform time-series diff calculation to cause data to be continuously written back to the cache, so that a forward circulating feedback mechanism is formed, and finally, the cache data management of model granularity is realized. Due to the fact that the difference between the model and the characteristic changing frequency is large, most models do not need model calculation, cache data are directly used, the cache hit rate is greatly improved, meanwhile due to the fact that diff calculation of the time sequence is conducted, the models and the characteristics which are accurately perceived and changed can be immediately subjected to model calculation, and the timeliness of the models is greatly improved.
3. Consistency problem
There are two consistency problems, the first is the consistency problem caused by the inability to identify traffic characteristics, which is caused by conventional extensive management caching techniques. The query request under the search flow carries a large number of query features related to query behaviors besides the query word itself, and the query features can be used for model calculation, guidance strategies, flow end identification and small flow mechanisms, while for a complex strategy system, the calculation results of the same query word under different query features are different greatly. In the traditional caching technology, because query requests or query words are used as management granularity, all query characteristics are difficult to accurately and timely completely identify, particularly, in the upstream caching technology, the upstream is seen from the downstream, the downstream belongs to a black box state, and how the downstream system applies the flow characteristics cannot be known, so that the problem of cache consistency is caused, and the typical problem is that small flow and full flow are mutually interfered.
In the embodiment, the cache is refined to the model granularity and extends to the model calculation level, all query features can be identified with low cost and high efficiency, the features influencing the model calculation are extracted in a targeted manner, and the extracted features are added into the cache key signature calculation, so that the cache key contains the query features, and the mapping unicity from the key to the value result is ensured. For the small flow characteristics, the corresponding small flow characteristics are added to the model level, in the time series diff calculation process, the small characteristic matching operation of the model and the flow level is carried out, the model calculation is carried out only when the matching is successful, namely the small flow and the full flow model are calculated and separated, the problem of mutual interference of the small flow and the full flow is solved, and the problem of consistency of the flow characteristic level is integrally solved.
The second consistency problem is caused by the fact that effective period control is extensive and model change is asynchronous on-line large-scale instances, in a large-scale distributed system for searching, hundreds of instances can exist in each subsystem and module, model data change is completed through a distribution platform, and a hierarchical distribution mechanism is generally available in consideration of stability and complexity of an on-line environment, so that all instances are basically impossible to change synchronously, a certain time difference exists, and tens of minutes to hours are possible. Because the instance changes are asynchronous, the cache data written in different instances are different, so that mutual interference is formed, and the problem of cache inconsistency is caused.
In this embodiment, diff calculation of a model time sequence is adopted, a latest model in the time sequence is screened out and is synchronously updated into a cache structure, when an inquiry request falls to an instance in which model change is not completed, model data in the cache structure, which is in front of the time sequence, is screened out according to the time sequence diff and is directly used as a calculation result of the model, and the model does not participate in model calculation any more. In short, when the actual query instance to which the query request belongs is in the mapping update state, the query result corresponding to the query request can be obtained from the same query instance in the mapping update completion state.
With further reference to fig. 6, as an implementation of the methods shown in the above figures, the present disclosure provides an embodiment of an information processing apparatus, which corresponds to the embodiment of the method shown in fig. 1, and which is particularly applicable in various electronic devices.
As shown in fig. 6, the information processing apparatus 600 of the present embodiment may include: query request analyzing section 601, timestamp extracting section 602, target mapping relation data determining section 603, first processing section 604, first updating section 605, and query result generating section 606. The query request analyzing unit 601 is configured to extract an actual query term and an actual query feature from the obtained query request; a timestamp extraction unit 602 configured to extract timestamps corresponding to actual query terms from mapping relationship data stored in the local storage and the cache, respectively; the mapping relation data comprises corresponding relations among query words, model numbers, time stamps, query characteristics and calculation results, and the mapping relation data stored in the cache is synchronous with the mapping relation data generated by local storage; a target mapping relationship data determining unit 603 configured to determine mapping relationship data containing the latest time stamp as target mapping relationship data; a first processing unit 604 configured to take the calculation result in the target mapping relationship data as a target calculation result in response to that the actual query feature has consistency with the query feature in the target mapping relationship data and the target mapping relationship data is taken from a local storage; a first updating unit 605 configured to update the mapping relationship data containing the same model number stored in the cache with the target mapping relationship data; a query result generation unit 606 configured to generate a query result corresponding to the query request based on the target calculation result.
In the present embodiment, in the information processing apparatus 600: the detailed processing and the technical effects brought by the query request analyzing unit 601, the timestamp extracting unit 602, the target mapping relationship data determining unit 603, the first processing unit 604, the first updating unit 605 and the query result generating unit 606 can refer to the related description of step 101 and step 106 in the embodiment corresponding to fig. 1, and are not repeated herein.
In some optional implementations of this embodiment, the information processing apparatus 600 may further include:
and the second processing unit is configured to respond to the fact that the actual query features are consistent with the query features in the target mapping relation and the target mapping relation is taken from the cache, and take the calculation results in the target mapping relation as target calculation results.
In some optional implementations of this embodiment, the query result generating unit 606 may be further configured to:
and generating a query result corresponding to the query request based on the target calculation results corresponding to all the acquired model numbers.
In some optional implementations of this embodiment, the query feature includes: at least one of a client characteristic when the query request is triggered, a behavior characteristic when the query request is triggered, a public sentiment characteristic when the query request is triggered, and a sound characteristic when the query request is triggered.
In some optional implementations of this embodiment, the information processing apparatus 600 may further include:
and a feature integration weight setting unit configured to set a corresponding feature integration weight for each feature constituting the query feature.
In some optional implementations of this embodiment, the information processing apparatus 600 may further include:
a difference degree calculation unit configured to calculate a difference degree of the actual query feature and the query feature in the target mapping relationship data according to a preset dimension;
and the consistency judging unit is configured to respond to the difference degree not exceeding a preset difference threshold value, and determine that the actual query feature is consistent with the query feature in the target mapping relation data.
In some optional implementations of this embodiment, the information processing apparatus 600 may further include:
a model subbucket creation unit configured to create models of different model numbers as different model subbuckets using a subbucket technique;
the mapping relation table creating unit is configured to load different model sub-buckets into a unified hash map through a HashMap technology of the HashMap, and establish a mapping relation table from a model number to a timestamp;
and the mapping relation generation unit is configured to generate mapping relation data based on the mapping relation table and the corresponding query words, the query features and the calculation results.
In some optional implementations of this embodiment, the information processing apparatus 600 may further include:
and the updating state processing unit is configured to respond to the fact that the actual query instance to which the query request belongs is in the mapping relation updating state currently, and obtain the query result corresponding to the query request from the same query instance in the mapping relation updating completion state.
This embodiment exists as an apparatus embodiment corresponding to the method embodiment as shown in fig. 1 corresponding to flow 100.
With further reference to fig. 7, as an implementation of the methods shown in the above figures, the present disclosure provides an embodiment of an information processing apparatus, which corresponds to the method embodiment shown in fig. 2, and which is particularly applicable in various electronic devices.
As shown in fig. 7, the information processing apparatus 700 of the present embodiment may include: a query request analysis unit 701, a timestamp extraction unit 702, a target mapping relationship data determination unit 703, a second processing unit 704, a new mapping relationship generation unit 705, a second updating unit 706, and a query result generation unit 707. The query request analyzing unit 701 is configured to extract an actual query term and an actual query feature from the obtained query request; a timestamp extraction unit 702 configured to extract timestamps corresponding to actual query terms from mapping relationship data stored in the local storage and the cache, respectively; the mapping relation data comprises corresponding relations among query words, model numbers, time stamps, query characteristics and calculation results, and the mapping relation data stored in the cache is synchronous with mapping relation data generated by local storage; a target mapping relationship data determining unit 703 configured to determine mapping relationship data containing the latest time stamp as target mapping relationship data; a second processing unit 704 configured to calculate a target calculation result based on the model corresponding to the target mapping data and the actual query feature in response to the actual query feature not being consistent with the query feature in the target mapping; a new mapping relationship generating unit 705 configured to determine a new timestamp according to a generation time of the target calculation result, and generate a new mapping relationship based on the actual query word, the model number, the new timestamp, the actual query feature, and the target calculation result; a second updating unit 706 configured to update the mapping relationship data stored in the cache and containing the same model number with the new mapping relationship; a query result generation unit 707 configured to generate a query result corresponding to the query request based on the target calculation result.
In the present embodiment, in the information processing apparatus 700: the specific processing and the technical effects of the query request analyzing unit 701, the timestamp extracting unit 702, the target mapping relationship data determining unit 703, the second processing unit 704, the new mapping relationship generating unit 705, the second updating unit 706 and the query result generating unit 707 can refer to the related descriptions of step 201 and step 207 in the corresponding embodiment of fig. 2, which are not described herein again.
In some optional implementations of this embodiment, the information processing apparatus 700 may further include:
and the second processing unit is configured to respond to the fact that the actual query features are consistent with the query features in the target mapping relation and the target mapping relation is taken from the cache, and take the calculation results in the target mapping relation as target calculation results.
In some optional implementations of this embodiment, the query result generating unit 706 may be further configured to:
and generating a query result corresponding to the query request based on the target calculation results corresponding to all the acquired model numbers.
In some optional implementations of this embodiment, the query feature includes: at least one of a client characteristic when the query request is triggered, a behavior characteristic when the query request is triggered, a public sentiment characteristic when the query request is triggered, and a sound characteristic when the query request is triggered.
In some optional implementations of this embodiment, the information processing apparatus 700 may further include:
and a feature integration weight setting unit configured to set a corresponding feature integration weight for each feature constituting the query feature.
In some optional implementations of this embodiment, the information processing apparatus 700 may further include:
a difference degree calculation unit configured to calculate a difference degree of the actual query feature and the query feature in the target mapping relationship data according to a preset dimension;
and the non-consistency judging unit is configured to respond to the difference degree exceeding a preset difference threshold value, and determine that the actual query feature is not consistent with the query feature in the target mapping relation data.
In some optional implementations of this embodiment, the information processing apparatus 700 may further include:
a model sub-bucket creation unit configured to create models of different model numbers as different model sub-buckets using a sub-bucket technique;
the mapping relation table creating unit is configured to load different model sub-buckets into a unified hash map through a HashMap technology of the HashMap, and establish a mapping relation table from a model number to a timestamp;
and the mapping relation generation unit is configured to generate mapping relation data based on the mapping relation table and the corresponding query words, the query features and the calculation results.
In some optional implementations of this embodiment, the information processing apparatus 700 may further include:
and the updating state processing unit is configured to respond to that the actual query instance to which the query request belongs is currently in a mapping relation updating state, and obtain a query result corresponding to the query request from the same query instance in the mapping relation updating completion state.
The two embodiments described above exist as device embodiments corresponding to the method embodiment shown in the flowchart 100 corresponding to fig. 1 and the method embodiment shown in the flowchart 200 corresponding to fig. 2, respectively.
The information processing device provided by the two device embodiments provides a fine management granularity of the model granularity through the mapping relation formed based on the model number, the timestamp and the query characteristics, and determines which cache data are unavailable and need to be updated in a targeted manner based on comparing the timestamps in the mapping relations stored in the local cache and the cache, so that the latest model calculation results stored in the cache are controlled as much as possible through a scientific updating strategy, the hit rate of directly hitting the cache by the query result corresponding to the query request is improved, and the feedback efficiency of the query result is improved finally.
According to an embodiment of the present disclosure, the present disclosure also provides an electronic device including: at least one processor; and a memory communicatively coupled to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor, so that the at least one processor can implement the information processing method described in any of the above embodiments when executing the instructions.
According to an embodiment of the present disclosure, the present disclosure further provides a readable storage medium storing computer instructions for enabling a computer to implement the information processing method described in any of the above embodiments when executed.
According to an embodiment of the present disclosure, there is also provided a computer program product, which when executed by a processor is capable of implementing the information processing method described in any of the above embodiments.
FIG. 8 illustrates a schematic block diagram of an example electronic device 800 that can be used to implement embodiments of the present disclosure. Electronic devices are intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device may also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be examples only, and are not meant to limit implementations of the disclosure described and/or claimed herein.
As shown in fig. 8, the apparatus 800 includes a computing unit 801 which can perform various appropriate actions and processes according to a computer program stored in a Read Only Memory (ROM)802 or a computer program loaded from a storage unit 808 into a Random Access Memory (RAM) 803. In the RAM 803, various programs and data required for the operation of the device 800 can also be stored. The calculation unit 801, the ROM 802, and the RAM 803 are connected to each other by a bus 804. An input/output (I/O) interface 805 is also connected to bus 804.
A number of components in the device 800 are connected to the I/O interface 805, including: an input unit 806, such as a keyboard, a mouse, or the like; an output unit 807 such as various types of displays, speakers, and the like; a storage unit 808, such as a magnetic disk, optical disk, or the like; and a communication unit 809 such as a network card, modem, wireless communication transceiver, etc. The communication unit 809 allows the device 800 to exchange information/data with other devices via a computer network such as the internet and/or various telecommunication networks.
Various implementations of the systems and techniques described here above may be implemented in digital electronic circuitry, integrated circuitry, Field Programmable Gate Arrays (FPGAs), Application Specific Integrated Circuits (ASICs), Application Specific Standard Products (ASSPs), system on a chip (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include: implemented in one or more computer programs that are executable and/or interpretable on a programmable system including at least one programmable processor, which may be special or general purpose, receiving data and instructions from, and transmitting data and instructions to, a storage system, at least one input device, and at least one output device.
Program code for implementing the methods of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable data processing apparatus, such that the program codes, when executed by the processor or controller, cause the functions/operations specified in the flowchart and/or block diagram to be performed. The program code may execute entirely on the machine, partly on the machine, as a stand-alone software package partly on the machine and partly on a remote machine or entirely on the remote machine or server.
In the context of this disclosure, a machine-readable medium may be a tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user; and a keyboard and a pointing device (e.g., a mouse or a trackball) by which a user may provide input to the computer. Other kinds of devices may also be used to provide for interaction with a user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form, including acoustic, speech, or tactile input.
The systems and techniques described here can be implemented in a computing system that includes a back-end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back-end, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local Area Networks (LANs), Wide Area Networks (WANs), and the Internet.
The computer system may include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The Server may be a cloud Server, which is also called a cloud computing Server or a cloud host, and is a host product in a cloud computing service system, so as to solve the defects of high management difficulty and weak service extensibility in the conventional physical host and Virtual Private Server (VPS) service.
According to the technical scheme of the embodiment of the disclosure, the fine management granularity of the model granularity is provided through the mapping relation formed based on the model number, the timestamp and the query characteristics, and the cache data which are unavailable and need to be updated pertinently are determined based on the comparison of the timestamps in the mapping relations stored in the local cache and the cache, so that the latest model calculation results stored in the cache are controlled as much as possible through a scientific updating strategy, the hit rate of directly hitting the cache by the query result corresponding to the query request is improved, and the feedback efficiency of the query result is improved finally.
It should be understood that various forms of the flows shown above may be used, with steps reordered, added, or deleted. For example, the steps described in the present disclosure may be executed in parallel, sequentially, or in different orders, as long as the desired results of the technical solutions disclosed in the present disclosure can be achieved, and the present disclosure is not limited herein.
The above detailed description should not be construed as limiting the scope of the disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions may be made in accordance with design requirements and other factors. Any modification, equivalent replacement, and improvement made within the spirit and principle of the present disclosure should be included in the scope of protection of the present disclosure.
Claims (31)
1. An information processing method comprising:
extracting actual query words and actual query features from the acquired query request;
extracting timestamps corresponding to the actual query words from the mapping relation data stored in the local storage and the cache respectively; the mapping relation data comprises corresponding relations among query words, model numbers, time stamps, query characteristics and calculation results, and the mapping relation data stored in the cache is synchronous with the mapping relation data generated by the local storage;
determining mapping relation data containing the latest timestamp as target mapping relation data;
taking a calculation result in the target mapping relation data as a target calculation result in response to that the actual query feature is consistent with a query feature in the target mapping relation data and the target mapping relation data is taken from the local storage;
updating the mapping relation data which is stored in the cache and contains the same model number by using the target mapping relation data;
and generating a query result corresponding to the query request based on the target calculation result.
2. The method of claim 1, wherein the generating a query result corresponding to the query request based on the target computation result comprises:
and generating a query result corresponding to the query request based on the target calculation results corresponding to all the acquired model numbers.
3. The method of claim 1, wherein the query features comprise:
at least one of a client characteristic when the query request is triggered, a behavior characteristic when the query request is triggered, a public sentiment characteristic when the query request is triggered, and a sound characteristic when the query request is triggered.
4. The method of claim 3, further comprising:
and setting corresponding feature comprehensive weight for each feature forming the query feature.
5. The method of claim 1, further comprising:
calculating the difference degree between the actual query feature and the query feature in the target mapping relation data according to a preset dimension;
and determining that the actual query feature is consistent with the query feature in the target mapping relation data in response to the difference degree not exceeding a preset difference threshold.
6. The method of claim 1, further comprising:
creating models with different model numbers into different model sub-buckets by utilizing a sub-bucket technology;
loading different model buckets into a unified HashMap through a HashMap technology, and establishing a mapping relation table from the model numbers to the timestamps;
and generating the mapping relation data based on the mapping relation table and the corresponding query words, query characteristics and calculation results.
7. The method of any of claims 1-6, further comprising:
and responding to the current mapping relation updating state of the actual query instance to which the query request belongs, and acquiring a query result corresponding to the query request from the same query instance in the mapping relation updating completion state.
8. An information processing method comprising:
extracting actual query words and actual query features from the acquired query request;
extracting timestamps corresponding to the actual query words from the mapping relation data stored in the local storage and the cache respectively; the mapping relation data comprises corresponding relations among query words, model numbers, time stamps, query characteristics and calculation results, and the mapping relation data stored in the cache is synchronous with the mapping relation data generated by the local storage;
determining mapping relation data containing the latest timestamp as target mapping relation data;
in response to the actual query feature not being consistent with the query feature in the target mapping relationship, calculating a target calculation result based on a model corresponding to the target mapping relationship data and the actual query feature;
determining a new timestamp according to the generation time of the target calculation result, and generating new mapping relation data based on the actual query word, the model number, the new timestamp, the actual query feature and the target calculation result;
updating the mapping relation data which is stored in the cache and contains the same model number by using the new mapping relation data;
and generating a query result corresponding to the query request based on the target calculation result.
9. The method of claim 8, wherein the generating a query result corresponding to the query request based on the target computation result comprises:
and generating a query result corresponding to the query request based on the target calculation results corresponding to all the acquired model numbers.
10. The method of claim 8, wherein the query features comprise:
at least one of a client characteristic when the query request is triggered, a behavior characteristic when the query request is triggered, a public sentiment characteristic when the query request is triggered, and a sound characteristic when the query request is triggered.
11. The method of claim 10, further comprising:
and setting corresponding characteristic comprehensive weight for each characteristic forming the query characteristic.
12. The method of claim 8, further comprising:
calculating the difference degree between the actual query feature and the query feature in the target mapping relation data according to a preset dimension;
and determining that the actual query feature does not have consistency with the query feature in the target mapping relationship data in response to the degree of difference exceeding a preset difference threshold.
13. The method of claim 8, further comprising:
creating models with different model numbers into different model sub-buckets by utilizing a sub-bucket technology;
loading different model buckets into a unified HashMap through a HashMap technology, and establishing a mapping relation table from the model numbers to the timestamps;
and generating the mapping relation data based on the mapping relation table and the corresponding query words, query characteristics and calculation results.
14. The method according to any one of claims 8-13, further comprising:
and responding to the current mapping relation updating state of the actual query instance to which the query request belongs, and acquiring a query result corresponding to the query request from the same query instance in the mapping relation updating completion state.
15. An information processing apparatus includes:
the query request analysis unit is configured to extract actual query words and actual query features from the acquired query request;
the timestamp extraction unit is configured to extract timestamps corresponding to the actual query words from the mapping relation data stored in the local storage and the cache respectively; the mapping relation data comprises corresponding relations among query words, model numbers, time stamps, query characteristics and calculation results, and the mapping relation data stored in the cache is synchronous with the mapping relation data generated by the local storage;
a target mapping relationship data determination unit configured to determine mapping relationship data containing the latest time stamp as target mapping relationship data;
a first processing unit configured to take a calculation result in the target mapping relation data as a target calculation result in response to the actual query feature having consistency with a query feature in the target mapping relation data and the target mapping relation data being taken from the local storage;
a first updating unit configured to update the mapping relationship data stored in the cache, which contains the same model number, with the target mapping relationship data;
a query result generation unit configured to generate a query result corresponding to the query request based on the target calculation result.
16. The apparatus of claim 15, the query result generation unit further configured to:
and generating a query result corresponding to the query request based on the target calculation results corresponding to all the acquired model numbers.
17. The apparatus of claim 15, wherein the query features comprise: at least one of a client characteristic when the query request is triggered, a behavior characteristic when the query request is triggered, a public sentiment characteristic when the query request is triggered, and a sound characteristic when the query request is triggered.
18. The apparatus of claim 17, further comprising:
a feature integration weight setting unit configured to set a corresponding feature integration weight for each feature constituting the query feature.
19. The apparatus of claim 15, further comprising:
a difference degree calculation unit configured to calculate a difference degree of the actual query feature and the query feature in the target mapping relationship data according to a preset dimension;
and the consistency judging unit is configured to respond to the difference degree not exceeding a preset difference threshold value, and determine that the actual query feature is consistent with the query feature in the target mapping relation data.
20. The apparatus of claim 15, further comprising:
a model sub-bucket creation unit configured to create models of different model numbers as different model sub-buckets using a sub-bucket technique;
the mapping relation table creating unit is configured to load different model sub-buckets into a unified hash map through a HashMap technology of the HashMap, and establish a mapping relation table from the model number to the timestamp;
and the mapping relation generation unit is configured to generate the mapping relation data based on the mapping relation table and the corresponding query terms, the query features and the calculation results.
21. The apparatus of any of claims 15-20, further comprising:
and the updating state processing unit is configured to respond to that the actual query instance to which the query request belongs is currently in a mapping relation updating state, and obtain a query result corresponding to the query request from the same query instance in a mapping relation updating completion state.
22. An information processing apparatus comprising:
the query request analysis unit is configured to extract actual query words and actual query characteristics from the acquired query request;
the timestamp extraction unit is configured to extract timestamps corresponding to the actual query words from the mapping relation data stored in the local storage and the cache respectively; the mapping relation data comprises corresponding relations among query words, model numbers, time stamps, query characteristics and calculation results, and the mapping relation data stored in the cache is synchronous with the mapping relation data generated by the local storage;
a target mapping relationship data determination unit configured to determine mapping relationship data containing the latest time stamp as target mapping relationship data;
a second processing unit configured to calculate a target calculation result based on a model corresponding to the target mapping data and the actual query feature in response to the actual query feature not being consistent with the query feature in the target mapping;
a new mapping relation generating unit configured to determine a new timestamp according to a generation time of the target calculation result, and generate new mapping relation data based on the actual query word, a model number, the new timestamp, the actual query feature, and the target calculation result;
a second updating unit configured to update the mapping relationship data stored in the cache, which contains the same model number, with the new mapping relationship data;
a query result generation unit configured to generate a query result corresponding to the query request based on the target calculation result.
23. The apparatus of claim 22, the query result generation unit further configured to:
and generating a query result corresponding to the query request based on the target calculation results corresponding to all the acquired model numbers.
24. The apparatus of claim 22, wherein the query features comprise: at least one of a client characteristic when the query request is triggered, a behavior characteristic when the query request is triggered, a public sentiment characteristic when the query request is triggered, and a sound characteristic when the query request is triggered.
25. The apparatus of claim 24, further comprising:
a feature integration weight setting unit configured to set a corresponding feature integration weight for each feature constituting the query feature.
26. The apparatus of claim 22, further comprising:
a difference degree calculation unit configured to calculate a difference degree between the actual query feature and a query feature in the target mapping relationship data according to a preset dimension;
a non-consistency determination unit configured to determine that the actual query feature is not consistent with the query feature in the target mapping relationship data in response to the degree of difference exceeding a preset difference threshold.
27. The apparatus of claim 22, further comprising:
a model sub-bucket creation unit configured to create models of different model numbers as different model sub-buckets using a sub-bucket technique;
the mapping relation table creating unit is configured to load different model sub-buckets into a unified hash map through a HashMap technology and establish a mapping relation table from the model numbers to the timestamps;
and the mapping relation generation unit is configured to generate the mapping relation data based on the mapping relation table and the corresponding query terms, the query features and the calculation results.
28. The apparatus of any of claims 22-27, further comprising:
and the updating state processing unit is configured to respond to that the actual query instance to which the query request belongs is currently in a mapping relation updating state, and obtain a query result corresponding to the query request from the same query instance in a mapping relation updating completion state.
29. An electronic device, comprising:
at least one processor; and
a memory communicatively coupled to the at least one processor; wherein,
the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the information processing method of any one of claims 1 to 7 or any one of claims 8 to 14.
30. A non-transitory computer-readable storage medium storing computer instructions for causing the computer to execute the information processing method of any one of claims 1 to 7 or any one of claims 8 to 14.
31. A computer program product comprising a computer program which, when executed by a processor, carries out the steps of the information processing method according to any one of claims 1 to 7 or any one of claims 8 to 14.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202210417765.XA CN114706894A (en) | 2022-04-20 | 2022-04-20 | Information processing method, apparatus, device, storage medium, and program product |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202210417765.XA CN114706894A (en) | 2022-04-20 | 2022-04-20 | Information processing method, apparatus, device, storage medium, and program product |
Publications (1)
Publication Number | Publication Date |
---|---|
CN114706894A true CN114706894A (en) | 2022-07-05 |
Family
ID=82174222
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN202210417765.XA Pending CN114706894A (en) | 2022-04-20 | 2022-04-20 | Information processing method, apparatus, device, storage medium, and program product |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN114706894A (en) |
Cited By (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN115146217A (en) * | 2022-09-01 | 2022-10-04 | 国网信息通信产业集团有限公司 | Method, system and equipment for solving data cycle calculation of comprehensive energy system |
CN117009755A (en) * | 2023-10-07 | 2023-11-07 | 国仪量子(合肥)技术有限公司 | Waveform data processing method, computer-readable storage medium, and electronic device |
-
2022
- 2022-04-20 CN CN202210417765.XA patent/CN114706894A/en active Pending
Cited By (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN115146217A (en) * | 2022-09-01 | 2022-10-04 | 国网信息通信产业集团有限公司 | Method, system and equipment for solving data cycle calculation of comprehensive energy system |
CN117009755A (en) * | 2023-10-07 | 2023-11-07 | 国仪量子(合肥)技术有限公司 | Waveform data processing method, computer-readable storage medium, and electronic device |
CN117009755B (en) * | 2023-10-07 | 2023-12-19 | 国仪量子(合肥)技术有限公司 | Waveform data processing method, computer-readable storage medium, and electronic device |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US20210312139A1 (en) | Method and apparatus of generating semantic feature, method and apparatus of training model, electronic device, and storage medium | |
TW202020691A (en) | Feature word determination method and device and server | |
EP2988230A1 (en) | Data processing method and computer system | |
CN114706894A (en) | Information processing method, apparatus, device, storage medium, and program product | |
CN111753914A (en) | Model optimization method and device, electronic equipment and storage medium | |
EP2874077A2 (en) | Stateless database cache | |
CN112926298B (en) | News content identification method, related device and computer program product | |
US20230012642A1 (en) | Method and device for snapshotting metadata, and storage medium | |
CN112559717B (en) | Search matching method, device, electronic equipment and storage medium | |
CN112860811B (en) | Method and device for determining data blood relationship, electronic equipment and storage medium | |
CN115249043A (en) | Data analysis method and device, electronic equipment and storage medium | |
US20220107949A1 (en) | Method of optimizing search system | |
CN115631273A (en) | Big data duplicate removal method, device, equipment and medium | |
US20220198358A1 (en) | Method for generating user interest profile, electronic device and storage medium | |
CN115510247A (en) | Method, device, equipment and storage medium for constructing electric carbon policy knowledge graph | |
CN115145924A (en) | Data processing method, device, equipment and storage medium | |
CN111291192A (en) | Triple confidence degree calculation method and device in knowledge graph | |
CN110737432A (en) | script aided design method and device based on root list | |
CN111736929A (en) | Method, device and equipment for creating task instance and readable storage medium | |
CN116578646A (en) | Time sequence data synchronization method, device, equipment and storage medium | |
US20220269659A1 (en) | Method, device and storage medium for deduplicating entity nodes in graph database | |
CN115454971A (en) | Data migration method and device, electronic equipment and storage medium | |
US20220198301A1 (en) | Method and apparatus for update processing of question answering system | |
US20210209166A1 (en) | Relationship network generation method and device, electronic apparatus, and storage medium | |
CN110851517A (en) | Source data extraction method, device and equipment and computer storage medium |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination |