CN119441566A - Data processing method and device, non-volatile storage medium, and electronic device - Google Patents
Data processing method and device, non-volatile storage medium, and electronic device Download PDFInfo
- Publication number
- CN119441566A CN119441566A CN202510049360.9A CN202510049360A CN119441566A CN 119441566 A CN119441566 A CN 119441566A CN 202510049360 A CN202510049360 A CN 202510049360A CN 119441566 A CN119441566 A CN 119441566A
- Authority
- CN
- China
- Prior art keywords
- target
- document
- information
- language model
- large language
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/93—Document management systems
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/903—Querying
- G06F16/9032—Query formulation
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/903—Querying
- G06F16/9035—Filtering based on additional data, e.g. user or group profiles
Landscapes
- Engineering & Computer Science (AREA)
- Databases & Information Systems (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computational Linguistics (AREA)
- Mathematical Physics (AREA)
- Business, Economics & Management (AREA)
- General Business, Economics & Management (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The application discloses a data processing method and device, a nonvolatile storage medium and electronic equipment. The method comprises the steps of receiving a query question input by a target object, identifying semantic information corresponding to the query question, determining identity characteristic information and behavior characteristic information of the target object, determining a target large language model matched with the semantic information, the identity characteristic information and the behavior characteristic information of the target object, searching a document set in a database in the target large language model, searching document details of the document set, determining the frequency of query keywords, document update time and a document quality scoring index as sorting factors, sorting the document set, and taking the sorted document set as an output result of the target large language model. The application solves the technical problem that the related technology can not combine the semantic information, the identity characteristic information and the behavior characteristic information of the user to select a specific large language model to process the query input by the user.
Description
Technical Field
The present application relates to the field of natural language processing technologies, and in particular, to a data processing method and apparatus, a nonvolatile storage medium, and an electronic device.
Background
Natural language processing (Natural Language Processing, NLP) is an important branch of the field of artificial intelligence, which focuses on how computers understand, interpret and generate human language. In recent years, with the development of deep learning technology, the NLP field has made remarkable progress.
The prior art often employs keyword-based search algorithms to match user queries or uses shallower natural language processing techniques to understand user input. Such methods have limited performance in handling complex semantic structures and understanding implicit user intent, and cannot fully exploit the deep understanding and generating capabilities of large language models.
In addition, the prior art cannot dynamically adjust the response strategy of the model by comprehensively considering the identity characteristics and behavior characteristics of the user. This means that the information provided by the system, whether new staff or senior specialists, is at a premium, lacks personalization and pertinency, whether in handling conventional tasks or coping with emergency situations.
In view of the above problems, no effective solution has been proposed at present.
Disclosure of Invention
The application provides a data processing method and device, a nonvolatile storage medium and electronic equipment, which at least solve the technical problem that related technologies cannot combine semantic information, identity characteristic information and behavior characteristic information of a user, and select a specific large language model to process query input by the user.
According to one aspect of the application, a data processing method is provided, which comprises the steps of receiving a query question input by a target object, identifying semantic information corresponding to the query question, determining identity characteristic information and behavior characteristic information of the target object, determining a target large language model matched with the semantic information, the identity characteristic information and the behavior characteristic information of the target object, searching a document set in a database in the target large language model through an inverted index, wherein the document set comprises query keywords in the query question, searching document details of the document set through a forward index, wherein the document details comprise document update time and document quality scoring indexes, determining the frequency of the query keywords in the inverted index, the document update time and the document quality scoring indexes in the forward index as ranking factors, ranking the document set according to the ranking factors, and taking the ranked document set as an output result of the target large language model.
The method comprises the steps of determining a target large language model matched with semantic information, identity characteristic information and behavior characteristic information of a target object, inputting the semantic information, the identity characteristic information and the behavior characteristic information of the target object into a routing model, wherein the routing model is used for executing the steps of carrying out characteristic extraction processing on the semantic information to obtain a first characteristic vector, carrying out characteristic extraction processing on the identity characteristic information of the target object to obtain a second characteristic vector, carrying out characteristic extraction processing on the behavior characteristic information of the target object to obtain a third characteristic vector, carrying out merging processing on the first characteristic vector, the second characteristic vector and the third characteristic vector to obtain a target characteristic vector, inputting the target characteristic vector into a rule base, wherein a preset routing rule based on data characteristics and service scenes is stored in the rule base, the routing rule is presented in the form of a decision tree, each decision node in the decision tree corresponds to a threshold judgment of one characteristic dimension, the branch node in the decision tree corresponds to a different model selection path, and carrying out characteristic extraction processing on the identity characteristic information of the target characteristic vector to obtain a target model path corresponding to the target characteristic vector, and carrying out merging processing on the first characteristic vector, the second characteristic vector and the third characteristic vector.
Optionally, the method further comprises the steps that after the routing selection is completed, the routing selection module sends identification information of the target large language model to the task planning module, after the task planning module receives the identification information, the resource use efficiency of the target large language model corresponding to the identification information is detected, wherein the resource use efficiency comprises CPU use rate and memory occupancy rate, and when the fact that the CPU use rate of the target large language model corresponding to the identification information is smaller than a first preset percentage and/or the memory occupancy rate is smaller than a second preset percentage is detected, the task planning module sends a target instruction to the routing selection module, and the target instruction is used for indicating the routing selection module to redefine routing rules.
Optionally, the method further comprises extracting sample data from the historical data storage library according to a preset time interval, taking feature vectors of the sample data as input, taking corresponding target model paths as labels, and training a routing model.
The method comprises the steps of determining task urgency of a query problem according to semantic information, determining target objects with at least one preset condition that the number of credit accounts is smaller than a first preset threshold value, the number of times of credit reports queried by financial institutions is smaller than a second preset threshold value, the number of times of transactions of bank accounts in the past preset time is smaller than a third preset threshold value, determining target objects with at least one preset condition that the number of times of credit accounts is not smaller than the first preset threshold value, the number of times of credit reports queried by financial institutions is not smaller than the second preset threshold value, the number of times of transactions of bank accounts in the past preset time is smaller than the third preset threshold value, determining target language models with the first type of target objects with the urgency according to preset matching rules, determining target language models with the first type of tasks with the urgency level is different than the first type of target language models with the first type of urgency.
Optionally, the response speed of the first class target large language model is greater than that of the second class target large language model, the response speed of the third class target large language model is greater than that of the fourth class target large language model, the response speed of the third class target large language model is greater than that of the first class target large language model, and the response speed of the fourth class target large language model is greater than that of the second class target large language model.
According to the application, the data processing device comprises a receiving module, a first determining module, a second determining module, an output module and a sorting module, wherein the receiving module is used for receiving a query problem input by a target object, the first determining module is used for identifying semantic information corresponding to the query problem and determining identity characteristic information and behavior characteristic information of the target object, the second determining module is used for determining a target large language model matched with the semantic information, the identity characteristic information and the behavior characteristic information of the target object, the output module is used for searching a document set in a database in the target large language model through an inverted index, wherein the document set comprises query keywords in the query problem, the document detailed content of the document set is searched through a forward index, the document detailed content comprises document update time and document quality scoring index, the frequency of the query keywords in the inverted index, the document update time and the document quality scoring index are determined to be sorting factors, the document set is sorted according to the sorting factors, and the sorted document set is used as an output result of the target large language model.
According to still another aspect of the present application, there is also provided a nonvolatile storage medium including a stored program, wherein the program, when run, controls a device in which the storage medium is located to execute the above data processing method.
According to still another aspect of the present application, there is also provided an electronic device including a memory and a processor for executing a program stored in the memory, wherein the program executes the above data processing method when running.
According to yet another aspect of the present application, there is also provided a computer program, wherein the computer program implements the above data processing method when being executed by a processor.
According to yet another aspect of the present application, there is also provided a computer program product comprising a non-volatile computer readable storage medium, wherein the non-volatile computer readable storage medium stores a computer program which, when executed by a processor, implements the above data processing method.
The application adopts a query problem input by a target object, identifies semantic information corresponding to the query problem, determines identity characteristic information and behavior characteristic information of the target object, determines a target large language model matched with the semantic information, the identity characteristic information and the behavior characteristic information of the target object, searches a document set in a database in the target large language model through an inverted index, wherein the document set comprises query keywords in the query problem, searches document detailed contents of the document set through the inverted index, wherein the document detailed contents comprise document update time and document quality scoring index, determines the frequency of the query keywords in the inverted index and the document update time and the document quality scoring index in the inverted index as sorting factors, sorts the document set according to the sorting factors, and uses the sorted document set as an output result of the target large language model, thereby achieving the purposes of combining the semantic information, the identity characteristic information and the behavior characteristic information of a user and selecting the specific large language model to process the query problem input by the user, further achieving the technical effect of efficiently processing the query problem of different types, and further solving the technical problem that the related technical problem that the semantic information, the identity characteristic information and the behavior characteristic information of the user cannot be combined to input the specific language model.
Drawings
The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this specification, illustrate embodiments of the application and together with the description serve to explain the application and do not constitute a limitation on the application. In the drawings:
FIG. 1 is a flow chart of a data processing method according to an embodiment of the present application;
FIG. 2 is a block diagram of a data processing apparatus according to an embodiment of the present application;
fig. 3 is a block diagram of a hardware structure of a computer terminal of a data processing method according to an embodiment of the present application.
Detailed Description
In order that those skilled in the art will better understand the present application, a technical solution in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in which it is apparent that the described embodiments are only some embodiments of the present application, not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the present application without making any inventive effort, shall fall within the scope of the present application.
It should be noted that the terms "first," "second," and the like in the description and the claims of the present application and the above figures are used for distinguishing between similar objects and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used may be interchanged where appropriate such that the embodiments of the application described herein may be implemented in sequences other than those illustrated or otherwise described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
According to an embodiment of the present application, there is provided a method embodiment of a data processing method, it being noted that the steps shown in the flowcharts of the figures may be performed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowcharts, in some cases the steps shown or described may be performed in an order different from that herein.
FIG. 1 is a flow chart of a data processing method according to an embodiment of the present application, as shown in FIG. 1, the method includes the steps of:
step S102, receiving a query question input by a target object.
Receiving a query request from a user, the user may enter their questions or needs in natural language form through a variety of channels, such as a mobile device, PC end, or counter terminal. The system immediately captures these inputs and prepares them for further processing.
Step S104, identifying semantic information corresponding to the query problem, and determining identity characteristic information and behavior characteristic information of the target object.
Upon receiving a query question, the question is subjected to deep semantic parsing using natural language processing techniques including, but not limited to, word vectors, attention mechanisms, and transducer architecture, understanding the core intent and context information of the question. Meanwhile, analysis can be performed according to the identity characteristics and the behavior characteristics of the user, and a foundation is laid for subsequent model selection and personalized service.
Step S106, determining a target large language model matched with the semantic information, the identity characteristic information of the target object and the behavior characteristic information.
The behavior characteristic information is, for example, the number of times the credit report is queried by a financial institution, and the number of times the bank account has been transacted within the past preset time length.
Based on the semantic information, the identity characteristic information and the behavior characteristic information collected and analyzed in the previous step, one model or a group of models are intelligently selected from a plurality of preset large language models, and the models can be optimally adapted to the characteristics of the current query scene and the target object. The selection process involves the steps of performance assessment, domain correlation analysis, personalized parameter adjustment, etc. of the model, ensuring that the selected model not only can accurately understand the question, but also can provide the most appropriate and accurate answer based on the specific requirements of the target object.
Step S108, searching a document set in a database in the target large language model through an inverted index, wherein the document set comprises query keywords in a query problem, searching document details of the document set through a forward index, wherein the document details comprise document updating time and document quality grading indexes, determining the frequency of the query keywords in the inverted index, the document updating time and the document quality grading indexes in the forward index as sorting factors, sorting the document set according to the sorting factors, and taking the sorted document set as an output result of the target large language model.
Once the target large language model is selected, the query question is passed to the model for processing with its powerful natural language understanding and generating capabilities. The model may generate one or more answers based on semantic information of the question, characteristics of the target object, and its own knowledge base. The answers are not only directly aimed at the questions posed by the user, but also consider the identity, the preference and the background information of the query of the user, so that more personalized service fitting the actual demands is provided. After the processing is completed, the output result of the model is presented to the target object, and one-time complete query service is completed.
In the above step S108, the document set including the query keyword is first quickly located in the database of the target large language model by using the inverted index technique. The inverted index is an efficient data structure that associates keywords in documents with a list of document IDs that contain the keywords, so that all relevant documents can be found instantaneously when a keyword is queried, greatly improving search speed. By the technology, the system can quickly identify all documents matched with the keywords in the user query problem, and a candidate document set is constructed. After the candidate document set is obtained, the detailed information of each document is further searched through a forward index technology, wherein the detailed information comprises metadata such as update time of the document, document quality scoring indexes and the like. The forward index is a data structure taking document ID as a key and taking document content, attribute and the like as values, and can provide complete information of the document, so that the system can conveniently and comprehensively evaluate the document later. And taking the frequency of the query keywords in the inverted index, the updating time of the documents in the forward index and the document quality scoring index as key factors of the sorting. The frequency of the query keywords reflects the tightness degree of the document and the query, the update time of the document ensures the timeliness of the information, is particularly important for the supervision policy and the internal regulation of the frequently-changed information, ensures the authority and the reliability of the search result by the document quality scoring index, and avoids the interference of low-quality documents. And finally, comprehensively sorting the candidate document sets according to the sorting factors. This process involves a variety of ranking algorithms, such as weight-based ranking, TF-IDF algorithm, etc., to calculate the composite score of the document. The documents with the high scores will be preferentially presented to the user, ensuring that the user receives the most relevant, most current and highest quality information first. The system takes the ordered document set as an output result, provides the output result for users to view, helps the users to quickly and accurately acquire required knowledge, and supports business operation and decision making processes.
By combining the reverse index and the forward index, the searching strategy can not only rapidly locate related documents, but also ensure high quality and timeliness of searching results, and provide more intelligent and efficient information searching experience for users. The mechanism fully embodies the application potential of the large model technology in banking business scenes, and provides powerful business support for users through intelligent searching and sequencing.
According to the method, the device and the system for processing the query problems comprise the steps of receiving the query problems input by a target object, identifying semantic information corresponding to the query problems, determining identity characteristic information and behavior characteristic information of the target object, determining a target large language model matched with the semantic information, the identity characteristic information and the behavior characteristic information of the target object, searching a document set in a database in the target large language model through an inverted index, wherein the document set comprises query keywords in the query problems, searching document details of the document set through the forward index, wherein the document details comprise document update time and document quality scoring indexes, determining the frequency of the query keywords in the inverted index, the document update time and the document quality scoring indexes in the forward index as ranking factors, ranking the document set according to the ranking factors, and using the ranked document set as an output result of the target large language model, so that the purposes of combining the semantic information, the identity characteristic information and the behavior characteristic information of a user and selecting a specific large language model to process the query problems input by the user are achieved, and the technical effects of efficiently processing the query problems of different types are achieved.
The steps shown in fig. 1 are exemplarily illustrated and explained below.
According to some optional embodiments of the application, determining the target large language model matched with the identity feature information and the behavior feature information of the target object can be achieved by inputting semantic information, the identity feature information and the behavior feature information of the target object into a routing model, wherein the routing model is used for executing the following steps of carrying out feature extraction processing on the semantic information to obtain a first feature vector, carrying out feature extraction processing on the identity feature information of the target object to obtain a second feature vector, carrying out feature extraction processing on the behavior feature information of the target object to obtain a third feature vector, carrying out merging processing on the first feature vector, the second feature vector and the third feature vector to obtain a target feature vector, inputting the target feature vector into a rule base, wherein a preset routing rule based on data features and service scenes is stored in the rule base, the routing rule is presented in a form of a decision tree, each decision node in the decision tree corresponds to a threshold judgment of one feature dimension, and a branch node in the decision tree corresponds to a different model selection path, carrying out matching on the target feature vector according to the routing rule to obtain a target feature vector, and carrying out merging processing on the first feature vector, wherein the target feature vector and the target language model is pointed to one or more target language models.
In the embodiment, the semantic information input by the user is subjected to deep analysis, the key semantic features are extracted, and the extracted key semantic features are converted into vector representations which can be understood by a computer. The techniques of word embedding, sentence vector and the like can be utilized to ensure that the vector can reflect the complex semantics and intent of the problem. The identity characteristic information of the target object is subjected to characteristic extraction, and the behavior characteristic information of the target object is subjected to characteristic extraction, including historical query records, preference setting, common query types and the like, and is also converted into vector representation. The behavior feature vector reflects the personalized needs and habits of the user. And combining the three feature vectors to obtain a comprehensive target feature vector. The vector integrates all the semantic information, the identity features and the behavior features, and provides comprehensive basis for subsequent model selection.
The rule base stores routing rules based on data characteristics and service scenes, and the rules are presented in the form of decision trees. Each decision node in the decision tree is set with a threshold decision for a particular feature dimension, such as complexity of the problem, role level of the user, etc. And inputting the target feature vector into a decision tree of a rule base, and matching according to a preset routing rule. The branch nodes in the decision tree correspond to different model selection paths that specify one or more large language models to be selected based on feature vector matching results. Finally, the decision tree will output path information for one or more target large language models, which ensures that the most appropriate model is selected to deal with the problem based on the nature, identity and historical behavior of the user query. In the case of multiple model paths, the system may further use ensemble learning or model fusion techniques to achieve a better response depending on the specific needs of the problem.
The routing module is used for receiving the identification information, detecting the resource utilization efficiency of the target large language model corresponding to the identification information, wherein the resource utilization efficiency comprises CPU utilization rate and memory occupancy rate, and sending a target instruction to the routing module by the task planning module when the CPU utilization rate of the target large language model corresponding to the identification information is detected to be smaller than a first preset percentage and/or the memory occupancy rate is detected to be smaller than a second preset percentage.
It can be appreciated that the task planning module, upon receiving the target large language model identification information sent by the routing model, is primarily tasked with monitoring the resource usage efficiency of the selected model. The method comprises the steps of detecting CPU utilization rate and memory occupancy rate in real time, and evaluating the hardware resource consumption condition of a model in current task execution. The first preset percentage and the second preset percentage are preset thresholds according to system performance, hardware specification and business requirements, and are used for judging whether the resource use of the model is in an ideal state or not.
If the detection result shows that the CPU utilization rate of the target large language model is lower than the first preset percentage and/or the memory occupancy rate is lower than the second preset percentage, this means that the selected model has lower resource consumption when processing the current task, and may have insufficient resource utilization. In this case, the mission planning module may send a "target instruction" to the routing model, instructing it to re-evaluate the routing rules to select a more appropriate model, ensuring efficient utilization of resources. And sending a target instruction, prompting the routing model to start the selection process again, dynamically adjusting the selection of the model based on the latest resource state and service requirement, and optimizing the resource allocation.
The resource use condition of the model is monitored in real time, the selection of the model is dynamically adjusted, the resource waste caused by the selection of the fixed model is avoided, and the efficiency and the flexibility of the system in resource utilization are ensured. By selecting the model which is most suitable for the current demand and the resource state, the quick and accurate response can be provided, so that the satisfaction and experience of the user in the query process are remarkably improved.
Further, sample data are extracted from the historical data storage library according to preset time intervals, feature vectors of the sample data are used as input, corresponding target model paths are used as labels, and a routing model is trained.
It is noted that the system automatically extracts sample data from the historical data store at predetermined time intervals. The sample data includes historical query questions, corresponding query results, user feedback, and information of the selected large language model. The extraction of the sample data not only considers the distribution on the time sequence, but also ensures the diversity and the representativeness of the data and covers various service scenes and user characteristic combinations. For each item of sample data extracted, the system will reconstruct its feature vectors, including those of semantic information, those of identity feature information, and those of behavioral feature information. These feature vectors collectively reflect the entire background information of the historical query. Meanwhile, based on the information of the 'target large language model' actually used by the sample data in the history inquiry process, the system sets a corresponding label, namely a target model path, for each sample data. This label indicates which large language model or combination of models has proven to be the best choice under a particular feature vector. Using these tagged feature vector data, the system begins training the routing model. During the training process, the model is continually learned and adjusted to improve its ability to predict the correct target model path given the feature vector. Through training of a large amount of historical data, the routing model can gradually grasp association rules between different feature vectors and target model paths, and optimize decision logic of the routing model, so that when the routing model faces a new user query, the routing model can more accurately and more quickly select a large language model which is most suitable for processing the query. After training, performance evaluation is performed on the routing model, and the prediction accuracy and resource efficiency of the routing model on new data are checked. If the performance of the model does not reach the expected standard or the selection in certain specific scenes has deviation, the system automatically adjusts the training strategy, introduces new sample data or adjusts the construction mode of the feature vector, retrains, realizes iterative optimization of the model, and ensures continuous improvement of the model so as to adapt to the continuously changed business demands and user behaviors.
The routing model training mechanism based on the historical data can ensure that the decision process of the system for selecting a large language model is established on a large amount of actual business data and user feedback, and continuously improves the efficiency and accuracy of routing through continuous learning and optimization, thereby providing more intelligent, personalized and efficient service experience for the inside and outside of a bank.
According to other optional embodiments of the application, the target large language model matched with the semantic information, the identity characteristic information and the behavior characteristic information of the target object can be achieved by determining the task emergency degree of the query problem according to the semantic information, wherein the task emergency degree comprises emergency and non-emergency, determining the target object with at least one preset condition that the number of credit accounts is smaller than a first preset threshold value, the number of times of the credit report queried by a financial institution is smaller than a second preset threshold value and the number of times of the transaction of a bank account in the past preset time period is smaller than a third preset threshold value as a first type target object, determining the target object with the task emergency degree as a first type target large language model matched with the task emergency degree according to preset matching rules, determining the target object with the task emergency degree as a second type target object according to the first type target language model matched with the task emergency degree and the first type target language model according to the preset emergency degree, and determining the target language model matched with the task emergency degree as the first type target large language model according to the emergency degree.
Preferably, the response speed of the first class target large language model is greater than that of the second class target large language model, the response speed of the third class target large language model is greater than that of the fourth class target large language model, the response speed of the third class target large language model is greater than that of the first class target large language model, and the response speed of the fourth class target large language model is greater than that of the second class target large language model.
In the above-described embodiment, first, the task urgency of the query problem is evaluated based on semantic information, and it is classified as "urgent" or "not urgent". This classification may be based on keywords, mood, or context information associated with time sensitivity contained in the question. For example, queries involving instant transaction confirmation or urgent compliance consultation may be labeled "urgent", while queries concerning business process guidance or general policy interpretation may be labeled "not urgent".
Next, the system divides the target objects (i.e., query initiators) into a "first type of target object" and a "second type of target object". This classification is based on preset conditions such as the number of credit accounts for the target object, the number of times the credit report is queried, and the transaction frequency of the bank account. Specifically, the first type of target objects are users with fewer credit accounts, lower credit report query times and lower recent transaction frequency. Such users may have little dependence on banking services or their account activity may be relatively small. And the second type of target objects are users with more credit accounts, higher credit report inquiry times or higher recent transaction frequency. Such users may have a higher demand for banking services, frequent account activity, and a higher expectation of timely response.
And matching large language models of corresponding types for different types of users and task emergency degrees according to preset matching rules. The selection of these models takes into account the response speed and the resource utilization efficiency to ensure the service efficiency, namely, the first class target large language model is matched with the urgent task and the first class target object, and the models are characterized by higher response speed and can rapidly process and reply to urgent queries, but can be balanced in terms of the resource utilization and the capability of processing complex problems. The second class of target large language models are matched with the non-urgent tasks and the first class of target objects, and the response speed of the models is slower, but better balance between processing capacity and resource efficiency is possible. And the third class of target large language model is matched with the emergency task and the second class of target object, so that the model has high response speed and higher capability of processing complex problems, and is suitable for high-demand users and emergency scenes. And a fourth class of target large language model which is matched with the non-urgent task and the second class of target object, wherein the models find balance points among response speed, resource efficiency and processing capacity and are suitable for processing daily query of the second class of target object.
Fig. 2 is a block diagram of a data processing apparatus according to an embodiment of the present application, as shown in fig. 2, including:
The receiving module 22 is configured to receive a query question input by the target object.
The first determining module 24 is configured to identify semantic information corresponding to the query problem, and determine identity feature information and behavior feature information of the target object.
A second determining module 26 is configured to determine a target large language model that matches the semantic information, the identity feature information of the target object, and the behavior feature information.
The output module 28 is configured to search a document set in a database in the target large language model through an inverted index, where the document set includes query keywords in a query problem, search document details of the document set through a forward index, where the document details include a document update time and a document quality score index, determine a frequency of the query keywords in the inverted index and the document update time and the document quality score index in the forward index as ranking factors, rank the document set according to the ranking factors, and use the ranked document set as an output result of the target large language model.
Optionally, the second determining module 26 is further configured to perform the steps of inputting semantic information, identity feature information of the target object, and behavior feature information into a routing model, where the routing model is configured to perform the steps of performing feature extraction processing on the semantic information to obtain a first feature vector, performing feature extraction processing on the identity feature information of the target object to obtain a second feature vector, performing feature extraction processing on the behavior feature information of the target object to obtain a third feature vector, performing merging processing on the first feature vector, the second feature vector, and the third feature vector to obtain a target feature vector, inputting the target feature vector into a rule base, where a preset routing rule based on data features and service scenes is stored in the rule base, the routing rule is presented in a form of a decision tree, each decision node in the decision tree corresponds to a threshold judgment of one feature dimension, and branch nodes in the decision tree correspond to different model selection paths, and matching the target feature vector according to the routing rule to obtain a target model path corresponding to the target feature vector, where the target model path points to one or more target language models.
Optionally, the second determining module 26 is further configured to perform the steps that the routing module sends the identification information of the target large language model to the task planning module after the routing is completed, the task planning module detects the resource usage efficiency of the target large language model corresponding to the identification information after receiving the identification information, where the resource usage efficiency includes a CPU usage rate and a memory occupancy rate, and the task planning module sends a target instruction to the routing module when detecting that the CPU usage rate of the target large language model corresponding to the identification information is less than a first preset percentage and/or the memory occupancy rate is less than a second preset percentage, where the target instruction is used to instruct the routing module to redefine the routing rule.
Optionally, the second determining module 26 is further configured to extract sample data from the historical data repository at preset time intervals, and train the routing model with feature vectors of the sample data as input and corresponding target model paths as labels.
Optionally, the second determining module 26 is further configured to determine, according to the semantic information, a task urgency degree of the query problem, where the task urgency degree includes urgency and nonemergency, determine, as a first type target object, a target object having at least one of the identity feature information and the behavior feature information satisfying a preset condition that a credit account number is smaller than a first preset threshold, a credit report queried number of times by the financial institution is smaller than a second preset threshold, a target object having at least one of the identity feature information and the behavior feature information satisfying a preset condition that a credit account number is not smaller than the first preset threshold, a credit report queried number of times by the financial institution is smaller than the third preset threshold, determine, according to a preset matching rule, a first type target language model matching with a first type target object having the task urgency degree of urgency, determine, according to the preset matching rule, a second type target language model matching with a task urgency degree of the first type target object having the task urgency degree of nonemergency, determine, according to the preset matching rule, a second type target language model matching with the task language model having the task urgency degree of the first type target language model, and determine, according to the preset language model having the urgency degree of the first type target language model matching with the first type target language model.
Optionally, the response speed of the first class target large language model is greater than that of the second class target large language model, the response speed of the third class target large language model is greater than that of the fourth class target large language model, the response speed of the third class target large language model is greater than that of the first class target large language model, and the response speed of the fourth class target large language model is greater than that of the second class target large language model.
It should be noted that each module in fig. 2 may be a program module (for example, a set of program instructions for implementing a specific function), or may be a hardware module, and for the latter, it may be expressed in a form, but is not limited to, that each module is expressed in a form of one processor, or the functions of each module are implemented by one processor.
It should be noted that, the preferred implementation manner of the embodiment shown in fig. 2 may refer to the related description of the embodiment shown in fig. 1, which is not repeated herein.
Fig. 3 shows a block diagram of a hardware structure of a computer terminal for implementing a data processing method. As shown in fig. 3, the computer terminal 30 may include one or more processors 302 (shown in the figures as 302a, 302b, 302 n), which processor 302 may include, but is not limited to, a processing device such as a microprocessor MCU or a programmable logic device FPGA, a memory 304 for storing data, and a transmission module 306 for communication functions. Among other things, a display, an input/output interface (I/O interface), a Universal Serial BUS (USB) port (which may be included as one of the ports of the BUS BUS), a network interface, a power supply, and/or a camera. It will be appreciated by those of ordinary skill in the art that the configuration shown in fig. 3 is merely illustrative and is not intended to limit the configuration of the electronic device described above. For example, the computer terminal 30 may also include more or fewer components than shown in FIG. 3, or have a different configuration than shown in FIG. 3.
It should be noted that the one or more processors 302 and/or other data processing circuits described above may be referred to generally herein as "data processing circuits. The data processing circuit may be embodied in whole or in part in software, hardware, firmware, or any other combination. Furthermore, the data processing circuitry may be a single stand-alone processing module or incorporated, in whole or in part, into any of the other elements in the computer terminal 30. As referred to in embodiments of the application, the data processing circuit acts as a processor control (e.g., selection of the path of the variable resistor termination connected to the interface).
The memory 304 may be used to store software programs and modules of application software, such as program instructions/data storage devices corresponding to the data processing method in the embodiment of the present application, and the processor 302 executes the software programs and modules stored in the memory 304, thereby performing various functional applications and data processing, that is, implementing the data processing method described above. Memory 304 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 304 may further include memory remotely located relative to the processor 302, which may be connected to the computer terminal 30 via a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.
The transmission module 306 is used to receive or transmit data via a network. The specific examples of the network described above may include a wireless network provided by a communication provider of the computer terminal 30. In one example, the transmission module 306 includes a network adapter (Network Interface Controller, NIC) that can connect to other network devices through a base station to communicate with the internet. In one example, the transmission module 306 may be a Radio Frequency (RF) module for communicating with the internet wirelessly.
The display may be, for example, a touch screen type Liquid Crystal Display (LCD) that may enable a user to interact with a user interface of the computer terminal 30.
It should be noted here that, in some alternative embodiments, the computer terminal shown in fig. 3 may include hardware elements (including circuits), software elements (including computer code stored on a computer readable medium), or a combination of both hardware and software elements. It should be noted that fig. 3 is only one example of a specific example, and is intended to illustrate the types of components that may be present in the computer terminal described above.
It should be noted that, the computer terminal shown in fig. 3 is configured to execute the data processing method shown in fig. 1, so that the explanation of the method for executing the command is also applicable to the electronic device, and is not repeated herein.
The embodiment of the application also provides a nonvolatile storage medium, which comprises a stored program, wherein the program controls the equipment where the storage medium is located to execute the data processing method when running.
The non-volatile storage medium executes a program for receiving a query question input by a target object, identifying semantic information corresponding to the query question, determining identity characteristic information and behavior characteristic information of the target object, determining a target large language model matched with the semantic information, the identity characteristic information and the behavior characteristic information of the target object, searching a document set in a database in the target large language model through an inverted index, wherein the document set comprises query keywords in the query question, searching document details of the document set through a forward index, wherein the document details comprise document update time and document quality scoring indexes, determining the frequency of the query keywords in the inverted index, the document update time and the document quality scoring indexes in the forward index as ranking factors, ranking the document set according to the ranking factors, and taking the ranked document set as an output result of the target large language model.
The embodiment of the application also provides electronic equipment, which comprises a memory and a processor, wherein the processor is used for running a program stored in the memory, and the data processing method is executed when the program runs.
The processor is used for running a program for executing the following functions of receiving a query question input by a target object, identifying semantic information corresponding to the query question, determining identity characteristic information and behavior characteristic information of the target object, determining a target large language model matched with the semantic information, the identity characteristic information and the behavior characteristic information of the target object, searching a document set in a database in the target large language model through an inverted index, wherein the document set comprises query keywords in the query question, searching document details of the document set through a forward index, wherein the document details comprise document update time and document quality scoring indexes, determining the frequency of the query keywords in the inverted index, the document update time and the document quality scoring indexes in the forward index as ranking factors, ranking the document set according to the ranking factors, and taking the ranked document set as an output result of the target large language model.
The foregoing embodiment numbers of the present application are merely for the purpose of description, and do not represent the advantages or disadvantages of the embodiments.
In the foregoing embodiments of the present application, the descriptions of the embodiments are emphasized, and for a portion of this disclosure that is not described in detail in this embodiment, reference is made to the related descriptions of other embodiments.
In the above embodiment of the present application, the collected information is information and data authorized by the user or sufficiently authorized by each party, and the processes of collection, storage, use, processing, transmission, provision, disclosure, application, etc. of the related data all comply with the related laws and regulations and standards, necessary protection measures are taken without violating the public welfare, and corresponding operation entries are provided for the user to select authorization or rejection.
In the several embodiments provided in the present application, it should be understood that the disclosed technology may be implemented in other manners. The above-described embodiments of the apparatus are merely exemplary, and the division of the units, for example, may be a logic function division, and may be implemented in another manner, for example, a plurality of units or components may be combined or may be integrated into another system, or some features may be omitted, or not performed. Alternatively, the coupling or direct coupling or communication connection shown or discussed with each other may be through some interfaces, units or modules, or may be in electrical or other forms.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
In addition, each functional unit in the embodiments of the present application may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit. The integrated units may be implemented in hardware or in software functional units.
The integrated units, if implemented in the form of software functional units and sold or used as stand-alone products, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present application may be embodied in essence or a part contributing to the related art or all or part of the technical solution, in the form of a software product stored in a storage medium, including several instructions for causing a computer device (which may be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the method according to the embodiments of the present application. The storage medium includes a U disk, a Read-Only Memory (ROM), a random access Memory (RAM, random Access Memory), a removable hard disk, a magnetic disk, or an optical disk, etc. which can store the program code.
The foregoing is merely a preferred embodiment of the present application and it should be noted that modifications and adaptations to those skilled in the art may be made without departing from the principles of the present application, which are intended to be comprehended within the scope of the present application.
Claims (10)
1. A method of data processing, comprising:
Receiving a query question input by a target object;
identifying semantic information corresponding to the query problem, and determining identity characteristic information and behavior characteristic information of the target object;
determining a target large language model matched with the semantic information, the identity characteristic information and the behavior characteristic information of the target object;
Searching a document set in a database in the target large language model through an inverted index, wherein the document set comprises query keywords in the query questions, searching document details of the document set through the inverted index, wherein the document details comprise document updating time and document quality grading indexes, determining the frequency of the query keywords in the inverted index, the document updating time and the document quality grading indexes in the forward index as sorting factors, sorting the document set according to the sorting factors, and taking the sorted document set as an output result of the target large language model.
2. The method of claim 1, wherein determining a target large language model that matches the semantic information, the identity feature information and the behavioral feature information of the target object comprises:
The method comprises the steps of inputting semantic information, identity characteristic information and behavior characteristic information of a target object into a routing model, wherein the routing model is used for executing the following steps of carrying out characteristic extraction processing on the semantic information to obtain a first characteristic vector, carrying out characteristic extraction processing on the identity characteristic information of the target object to obtain a second characteristic vector, carrying out characteristic extraction processing on the behavior characteristic information of the target object to obtain a third characteristic vector, carrying out merging processing on the first characteristic vector, the second characteristic vector and the third characteristic vector to obtain a target characteristic vector, inputting the target characteristic vector into a rule base, wherein a preset routing rule based on data characteristics and service scenes is stored in the rule base, each decision node in the decision tree is presented in a form of a decision tree, each decision node in the decision tree corresponds to a threshold judgment of one characteristic dimension, a branch node in the decision tree corresponds to a different model selection path, matching the target characteristic vector according to the routing rule to obtain a target characteristic vector, and the target path corresponding to the target characteristic vector is pointed to one or more target language models.
3. The method according to claim 2, wherein the method further comprises:
After the routing is completed, the routing model sends the identification information of the target large language model to a task planning module;
after the task planning module receives the identification information, detecting the resource utilization efficiency of the target large language model corresponding to the identification information, wherein the resource utilization efficiency comprises CPU utilization rate and memory occupancy rate;
And under the condition that the CPU utilization rate of the target large language model corresponding to the identification information is detected to be smaller than a first preset percentage and/or the memory occupancy rate is detected to be smaller than a second preset percentage, the task planning module sends a target instruction to the routing model, wherein the target instruction is used for indicating the routing model to re-determine the routing rule.
4. The method according to claim 2, wherein the method further comprises:
and extracting sample data from a historical data storage library according to a preset time interval, taking a characteristic vector of the sample data as input, taking a corresponding target model path as a label, and training the routing model.
5. The method of claim 1, wherein determining a target large language model that matches the semantic information, the identity feature information and the behavioral feature information of the target object comprises:
Determining the task emergency degree of the query problem according to the semantic information, wherein the task emergency degree comprises emergency and non-emergency;
Determining target objects of which the identity characteristic information and the behavior characteristic information meet at least one preset condition as first type target objects, wherein the number of credit accounts is smaller than a first preset threshold value, the number of times of inquiring the credit report by a financial institution is smaller than a second preset threshold value, and the number of times of trading the bank account in the past preset time period is smaller than a third preset threshold value;
Determining target objects of the identity characteristic information and the behavior characteristic information meeting at least one preset condition as second type target objects, wherein the number of credit accounts is not less than the first preset threshold, the number of times that a credit report is queried by a financial institution is not less than the second preset threshold, and the number of times that the transaction of the bank account is in the past preset time period is less than the third preset threshold;
According to a preset matching rule, determining a first type target large language model matched with a first type target object with the emergency degree of the task as emergency;
Determining a second-class target large language model matched with the first-class target object with the task emergency degree of non-emergency according to the preset matching rule;
According to a preset matching rule, determining a third class target large language model matched with a second class target object with the emergency degree of the task as emergency;
And determining a fourth type of target large language model matched with the second type of target object with the task emergency degree of non-emergency according to the preset matching rule.
6. The method of claim 5, wherein the first class of target large language models have a response speed greater than the second class of target large language models, wherein the third class of target large language models have a response speed greater than the fourth class of target large language models, wherein the third class of target large language models have a response speed greater than the first class of target large language models, and wherein the fourth class of target large language models have a response speed greater than the second class of target large language models.
7. A data processing apparatus, comprising:
The receiving module is used for receiving the query problem input by the target object;
The first determining module is used for identifying semantic information corresponding to the query problem and determining identity characteristic information and behavior characteristic information of the target object;
the second determining module is used for determining a target large language model matched with the semantic information, the identity characteristic information and the behavior characteristic information of the target object;
the output module is used for searching a document set in a database in the target large language model through an inverted index, wherein the document set comprises query keywords in the query questions, searching document details of the document set through a forward index, wherein the document details comprise document updating time and document quality grading indexes, determining the frequency of the query keywords in the inverted index, the document updating time and the document quality grading indexes in the forward index as sorting factors, sorting the document set according to the sorting factors, and taking the sorted document set as an output result of the target large language model.
8. A non-volatile storage medium, characterized in that the non-volatile storage medium comprises a stored program, wherein the program, when run, controls a device in which the non-volatile storage medium is located to perform the data processing method of any one of claims 1 to 6.
9. An electronic device comprising a memory and a processor for executing a program stored in the memory, wherein the program is executed to perform the data processing method of any one of claims 1 to 6.
10. A computer program product comprising a computer program, characterized in that the computer program, when executed by a processor, implements the data processing method of any of claims 1 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202510049360.9A CN119441566B (en) | 2025-01-13 | 2025-01-13 | Large language model financial operation business question-answering method based on index algorithm |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202510049360.9A CN119441566B (en) | 2025-01-13 | 2025-01-13 | Large language model financial operation business question-answering method based on index algorithm |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN119441566A true CN119441566A (en) | 2025-02-14 |
| CN119441566B CN119441566B (en) | 2025-04-18 |
Family
ID=94507842
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202510049360.9A Active CN119441566B (en) | 2025-01-13 | 2025-01-13 | Large language model financial operation business question-answering method based on index algorithm |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN119441566B (en) |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US11971914B1 (en) * | 2023-07-21 | 2024-04-30 | Cure A1, LLC | Artificial intelligence systems and methods |
| CN118228102A (en) * | 2024-03-14 | 2024-06-21 | 北京未来式智能科技有限公司 | Method for generating multiple large language models to cooperate with agents |
| KR102687013B1 (en) * | 2023-12-11 | 2024-07-22 | (주)유알피 | Method for intelligent document search through analyzing search keyword based on generative ai model |
| CN118394890A (en) * | 2024-06-28 | 2024-07-26 | 宏景科技股份有限公司 | Knowledge retrieval enhancement generation method and system based on large language model |
| CN118822705A (en) * | 2024-09-13 | 2024-10-22 | 北京银行股份有限公司 | Financial artificial intelligence management service platform and financial business processing method |
| CN119166763A (en) * | 2024-08-15 | 2024-12-20 | 平安科技（深圳）有限公司 | Query processing method, device, computer equipment and medium based on artificial intelligence |
| CN119226501A (en) * | 2024-12-02 | 2024-12-31 | 北京银行股份有限公司 | Data processing method and device, non-volatile storage medium, and electronic device |
| CN119273443A (en) * | 2024-12-09 | 2025-01-07 | 北京银行股份有限公司 | Financial information query method, device, storage medium and computer program product |
-
2025
- 2025-01-13 CN CN202510049360.9A patent/CN119441566B/en active Active
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US11971914B1 (en) * | 2023-07-21 | 2024-04-30 | Cure A1, LLC | Artificial intelligence systems and methods |
| KR102687013B1 (en) * | 2023-12-11 | 2024-07-22 | (주)유알피 | Method for intelligent document search through analyzing search keyword based on generative ai model |
| CN118228102A (en) * | 2024-03-14 | 2024-06-21 | 北京未来式智能科技有限公司 | Method for generating multiple large language models to cooperate with agents |
| CN118394890A (en) * | 2024-06-28 | 2024-07-26 | 宏景科技股份有限公司 | Knowledge retrieval enhancement generation method and system based on large language model |
| CN119166763A (en) * | 2024-08-15 | 2024-12-20 | 平安科技（深圳）有限公司 | Query processing method, device, computer equipment and medium based on artificial intelligence |
| CN118822705A (en) * | 2024-09-13 | 2024-10-22 | 北京银行股份有限公司 | Financial artificial intelligence management service platform and financial business processing method |
| CN119226501A (en) * | 2024-12-02 | 2024-12-31 | 北京银行股份有限公司 | Data processing method and device, non-volatile storage medium, and electronic device |
| CN119273443A (en) * | 2024-12-09 | 2025-01-07 | 北京银行股份有限公司 | Financial information query method, device, storage medium and computer program product |
Non-Patent Citations (1)
| Title |
|---|
| 杨旸;刘佳;蔡圣暐;杨书略;: "云计算中保护数据隐私的快速多关键词语义排序搜索方案", 计算机学报, no. 06, 1 June 2017 (2017-06-01) * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN119441566B (en) | 2025-04-18 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106815252B (en) | Searching method and device | |
| US11429405B2 (en) | Method and apparatus for providing personalized self-help experience | |
| CN117271767A (en) | Operation and maintenance knowledge base establishing method based on multiple intelligent agents | |
| CN111625715B (en) | Information extraction method and device, electronic equipment and storage medium | |
| CN113254711B (en) | Interactive image display method and device, computer equipment and storage medium | |
| CN119202214A (en) | Financial knowledge intelligent question answering method, device, storage medium and program product | |
| CN118822705B (en) | Financial artificial intelligence management service platform and financial business processing method | |
| CN118410152B (en) | Information processing method, question-answering method and question-answering system | |
| US12212473B2 (en) | Methods and systems for dynamic re-clustering of nodes in computer networks using machine learning models | |
| CN119226536A (en) | Financial business processing method and system, storage medium, and computer program product | |
| CN119226501A (en) | Data processing method and device, non-volatile storage medium, and electronic device | |
| CN119273443B (en) | Financial information query method and device based on artificial intelligence | |
| CN120234398A (en) | Intelligent question-answering method and system for low-altitude economic education on interactive learning platform | |
| CN119646166A (en) | Customer service reply method, device and equipment based on business intelligence | |
| CN118093839B (en) | Knowledge operation question-answer dialogue processing method and system based on deep learning | |
| CN118245528B (en) | Intelligent training service data mining method and system applying deep learning algorithm | |
| CN119441566B (en) | Large language model financial operation business question-answering method based on index algorithm | |
| CN118035398A (en) | Intention recognition method, device, equipment and storage medium | |
| CN117333191A (en) | Complaint event association method and device and complaint event association system | |
| CN116956183A (en) | Multimedia resource recommendation method, model training method, device and storage medium | |
| CN117216256A (en) | Intention recognition method, device, equipment and computer storage medium | |
| CN116701752A (en) | News recommendation method and device based on artificial intelligence, electronic equipment and medium | |
| CN120805905A (en) | Processing method for through mark data | |
| CN119128164A (en) | Data weaving method, device and storage medium | |
| CN118227751A (en) | Target searching method, target searching device, electronic equipment and readable storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |