CN117520819A - Algorithm recommendation method, device, equipment and medium based on data waveform characteristics - Google Patents
Algorithm recommendation method, device, equipment and medium based on data waveform characteristics Download PDFInfo
- Publication number
- CN117520819A CN117520819A CN202311481646.1A CN202311481646A CN117520819A CN 117520819 A CN117520819 A CN 117520819A CN 202311481646 A CN202311481646 A CN 202311481646A CN 117520819 A CN117520819 A CN 117520819A
- Authority
- CN
- China
- Prior art keywords
- algorithm
- data
- target
- recommendation
- scene
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000004422 calculation algorithm Methods 0.000 title claims abstract description 271
- 238000000034 method Methods 0.000 title claims abstract description 51
- 238000004458 analytical method Methods 0.000 claims abstract description 30
- 238000012545 processing Methods 0.000 claims abstract description 28
- 230000000694 effects Effects 0.000 claims description 21
- 238000012549 training Methods 0.000 claims description 12
- 238000004590 computer program Methods 0.000 claims description 9
- 238000000605 extraction Methods 0.000 claims description 9
- 238000012544 monitoring process Methods 0.000 claims description 9
- 238000012795 verification Methods 0.000 claims description 4
- 238000004364 calculation method Methods 0.000 abstract description 27
- 238000005516 engineering process Methods 0.000 abstract description 7
- 239000002699 waste material Substances 0.000 abstract description 6
- 238000012821 model calculation Methods 0.000 abstract description 2
- 238000010586 diagram Methods 0.000 description 6
- 230000006870 function Effects 0.000 description 5
- 230000007774 longterm Effects 0.000 description 4
- 238000004519 manufacturing process Methods 0.000 description 4
- 230000008569 process Effects 0.000 description 4
- 238000007637 random forest analysis Methods 0.000 description 4
- 230000006872 improvement Effects 0.000 description 3
- 239000013598 vector Substances 0.000 description 3
- 230000008859 change Effects 0.000 description 2
- 230000008676 import Effects 0.000 description 2
- 230000010355 oscillation Effects 0.000 description 2
- 230000001360 synchronised effect Effects 0.000 description 2
- 230000000007 visual effect Effects 0.000 description 2
- 208000000044 Amnesia Diseases 0.000 description 1
- 208000026139 Memory disease Diseases 0.000 description 1
- 238000013473 artificial intelligence Methods 0.000 description 1
- 238000013528 artificial neural network Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 230000003111 delayed effect Effects 0.000 description 1
- 238000011156 evaluation Methods 0.000 description 1
- 230000003203 everyday effect Effects 0.000 description 1
- 238000009499 grossing Methods 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 230000006984 memory degeneration Effects 0.000 description 1
- 208000023060 memory loss Diseases 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000000737 periodic effect Effects 0.000 description 1
- 238000012216 screening Methods 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 238000007619 statistical method Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/213—Feature extraction, e.g. by transforming the feature space; Summarisation; Mappings, e.g. subspace methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/903—Querying
- G06F16/9035—Filtering based on additional data, e.g. user or group profiles
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/25—Fusion techniques
- G06F18/259—Fusion by voting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N20/00—Machine learning
- G06N20/20—Ensemble learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N5/00—Computing arrangements using knowledge-based models
- G06N5/01—Dynamic search techniques; Heuristics; Dynamic trees; Branch-and-bound
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Physics & Mathematics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Artificial Intelligence (AREA)
- Evolutionary Computation (AREA)
- Software Systems (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Evolutionary Biology (AREA)
- Bioinformatics & Computational Biology (AREA)
- Computational Linguistics (AREA)
- Computing Systems (AREA)
- Mathematical Physics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Databases & Information Systems (AREA)
- Medical Informatics (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The application relates to the field of financial science and technology, and discloses an algorithm recommendation method, device, equipment and medium based on data waveform characteristics, comprising the following steps: acquiring target data in a first preset time period of a target scene; extracting characteristics of target data by utilizing a waveform analysis algorithm to obtain data waveform characteristics; inputting the data waveform characteristics into a target recommendation model, and evaluating the processing capacity of each candidate algorithm on target data by using the target recommendation model to obtain a recommendation algorithm for processing a target scene. According to the recommendation method and the device, the recommendation model is used for determining the recommendation algorithm of the target scene according to the data waveform characteristics, the data waveform characteristics are smaller in data quantity compared with the target data, the recommendation model is convenient to rapidly infer and calculate, the model calculation cost is reduced, repeated calculation of various algorithms is avoided, and the prediction accuracy of the service scene is improved. Particularly, in the field of financial science and technology, the business scenes are numerous, and the waste of computing resources and storage resources of a financial system can be effectively reduced.
Description
Technical Field
The application relates to the technical field of artificial intelligence and financial science and technology, in particular to an algorithm recommendation method, device, equipment and medium based on data waveform characteristics.
Background
In order to compare which algorithm is optimal, a full amount of prediction needs to be performed on each algorithm, and then an optimal algorithm or an optimal prediction result is selected according to the prediction results of all algorithms, which requires more and more computing resources. Only one result is formally used after the calculation of all algorithms is completed, redundant calculation results are discarded, and the waste of calculation resources and storage resources is intangibly caused. Especially in the fields of financial science and technology, such as banks, etc., the business scenario is numerous, and the algorithms that need to be selected are also many, and the burden of the financial system is increased in intangibly.
Disclosure of Invention
The main purpose of the application is to provide an algorithm recommendation method, device, equipment and medium based on data waveform characteristics, which can solve the technical problem of resource waste caused by respectively carrying out full prediction by utilizing a plurality of algorithms in the prior art.
To achieve the above object, a first aspect of the present application provides an algorithm recommendation method based on data waveform characteristics, the method comprising:
acquiring target data in a first preset time period of a target scene;
extracting characteristics of target data by utilizing a waveform analysis algorithm to obtain data waveform characteristics;
inputting the data waveform characteristics into a target recommendation model, and predicting and evaluating the processing capacity of each candidate algorithm on target data according to the data waveform characteristics by using the target recommendation model to obtain a recommendation algorithm for processing a target scene.
To achieve the above object, a second aspect of the present application provides an algorithm recommendation device based on data waveform characteristics, the device comprising:
the first data acquisition module is used for acquiring target data in a first preset time period of a target scene;
the feature extraction module is used for extracting features of the target data by utilizing a waveform analysis algorithm to obtain data waveform features;
and the recommendation algorithm determining module is used for inputting the data waveform characteristics into the target recommendation model, and predicting and evaluating the processing capacity of each candidate algorithm on the target data according to the data waveform characteristics by utilizing the target recommendation model to obtain a recommendation algorithm for processing the target scene.
To achieve the above object, a third aspect of the present application provides a computer-readable storage medium storing a computer program, which when executed by a processor, causes the processor to perform the steps of:
acquiring target data in a first preset time period of a target scene;
extracting characteristics of target data by utilizing a waveform analysis algorithm to obtain data waveform characteristics;
inputting the data waveform characteristics into a target recommendation model, and predicting and evaluating the processing capacity of each candidate algorithm on target data according to the data waveform characteristics by using the target recommendation model to obtain a recommendation algorithm for processing a target scene.
To achieve the above object, a fourth aspect of the present application provides a computer device, including a memory and a processor, the memory storing a computer program, which when executed by the processor causes the processor to perform the steps of:
acquiring target data in a first preset time period of a target scene;
extracting characteristics of target data by utilizing a waveform analysis algorithm to obtain data waveform characteristics;
inputting the data waveform characteristics into a target recommendation model, and predicting and evaluating the processing capacity of each candidate algorithm on target data according to the data waveform characteristics by using the target recommendation model to obtain a recommendation algorithm for processing a target scene.
By adopting the embodiment of the application, the method has the following beneficial effects:
according to the method, the waveform analysis algorithm is utilized to conduct feature extraction on the target data to obtain the data waveform features, the recommendation model is utilized to determine the recommendation algorithm of the target scene according to the data waveform features, the data waveform features are smaller in data quantity compared with the target data, quick reasoning and calculation of the recommendation model are more convenient, model calculation cost and calculation quantity are reduced, and the generalization capability of scene identification can be improved by the data waveform features; the algorithm recommendation scheme based on the data waveform characteristics can select a proper algorithm for any scene, can effectively avoid repeated calculation by using multiple algorithms in the prior art, and can improve the prediction accuracy of the service scene by selecting a recommended optimal algorithm from multiple candidate algorithms through a model. Particularly, in the fields of financial science and technology, such as banks, insurance and the like, the business scenes are numerous, the waste of computing resources and storage resources of a financial system can be effectively avoided or reduced, and the burden of the financial system is reduced.
Drawings
In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings that are required in the embodiments or the description of the prior art will be briefly described below, it being obvious that the drawings in the following description are only some embodiments of the present application, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
Wherein:
FIG. 1 is an application environment diagram of an algorithm recommendation method based on data waveform characteristics in an embodiment of the present application;
FIG. 2 is a flowchart of an algorithm recommendation method based on data waveform characteristics in an embodiment of the present application;
FIG. 3 is a block diagram of an algorithm recommendation device based on data waveform characteristics in an embodiment of the present application;
fig. 4 is a block diagram of a computer device in an embodiment of the present application.
Detailed Description
The following description of the embodiments of the present application will be made clearly and fully with reference to the accompanying drawings, in which it is evident that the embodiments described are only some, but not all, of the embodiments of the present application. All other embodiments, which can be made by one of ordinary skill in the art based on the embodiments herein without making any inventive effort, are intended to be within the scope of the present application.
FIG. 1 is an application environment diagram of an algorithm recommendation method based on data waveform characteristics in one embodiment. Referring to fig. 1, the algorithm recommendation method based on the data waveform characteristics is applied to an algorithm recommendation system based on the data waveform characteristics. The algorithm recommendation system based on the data waveform characteristics includes a terminal 110 and a server 120. The terminal 110 and the server 120 are connected through a network, and the terminal 110 may be a desktop terminal or a mobile terminal, and the mobile terminal may be at least one of a mobile phone, a tablet computer, a notebook computer, and the like. The server 120 may be implemented as a stand-alone server or as a server cluster composed of a plurality of servers. The terminal 110 is configured to send an algorithm recommendation request or instruction to the server 120, where the server 120 is configured to obtain target data in a first preset time period of a target scene; extracting characteristics of target data by utilizing a waveform analysis algorithm to obtain data waveform characteristics; inputting the data waveform characteristics into a target recommendation model, and predicting and evaluating the processing capacity of each candidate algorithm on target data according to the data waveform characteristics by using the target recommendation model to obtain a recommendation algorithm for processing a target scene.
As shown in FIG. 2, in one embodiment, an algorithm recommendation method based on data waveform characteristics is provided. The algorithm recommendation based on the data waveform characteristics specifically comprises the following steps:
s100: and acquiring target data in a first preset time period of the target scene.
Specifically, the algorithm recommendation method based on the data waveform characteristics is applied to the algorithm recommendation system based on the data waveform characteristics, and the target scene can be any application scene in any service system.
The target data within the first preset time period may be, for example, any one of the last 2 days, the last 24 hours, the last 12 hours, the last 6 hours, and the like, which is not limited in this application, specifically according to the actual scene configuration.
S200: and extracting characteristics of the target data by utilizing a waveform analysis algorithm to obtain data waveform characteristics.
Specifically, in one embodiment, the waveform analysis algorithm is utilized to extract waveform characteristics by utilizing the open source library of pywavelets, and the waveform characteristics of specific data can be analyzed after the data in the corresponding data format is directly input into the library.
For example, the code is as follows:
import pywt
coeffs＝pywt.wavedec(data,"db4",level＝5)
ca5,cd5,cd4,cd3,cd2,cd1＝coeffs
the ca5 obtained according to the method is a low-frequency characteristic, the cd5 is a high-frequency characteristic, the data of the angle ca5 which is convenient to understand represent the long-term trend of a data curve, and the cd5 represents the period and oscillation in the short term of the data.
The data waveform feature can be at least one of the features of ca5, cd4, cd3, cd2, cd1, etc.
S300: inputting the data waveform characteristics into a target recommendation model, and predicting and evaluating the processing capacity of each candidate algorithm on target data according to the data waveform characteristics by using the target recommendation model to obtain a recommendation algorithm for processing a target scene.
Specifically, the target recommendation model is used for evaluating the processing capacity of various candidate algorithms on the same group of target data, obtaining the estimated score corresponding to each candidate algorithm, and screening the optimal algorithm according to the estimated score. The estimated score is used for indicating the processing capacity of the corresponding candidate algorithm on the target data, and the higher the estimated score is, the stronger the processing capacity of the candidate algorithm on the target data is.
In one particular embodiment, the target recommendation model is a random forest classifier (random forest) constructed based on a random forest algorithm.
How many trees are included in the target recommendation model can be specified according to specific application scenes, but the more the trees are, the more the result is represented. For example, 10 trees are constructed, each having a height of 8, and an optimal result is selected according to the selection results of the 10 weak classifiers. Meanwhile, due to the random sample selection capability of the random forest model, the model has certain fitting resistance.
For example, 10 trees represent 10 votes, and if 5 votes are selected for 10 votes, algorithm a is ultimately selected as the optimal algorithm. Overall the target recommendation model will select the choice with the most votes as the best algorithm or recommendation algorithm.
The target recommendation model needs to be trained before being put into use. The training samples of the recommendation model include data waveform features (e.g., one or more of low frequency features, high frequency features, etc.) of scene data of a plurality of scenes, and corresponding data tags, each data tag being: the algorithm identification corresponding to the candidate algorithm with the highest scene score (e.g., each candidate algorithm is identified using a label or number).
The training samples are obtained specifically as follows: the current n candidate algorithms perform algorithm calculation on scene data of a service scene every day, and generate n different algorithm calculation results based on the same service scene. After waiting for the actual data to be generated, calculating errors of the result and the actual result by the batch calculation algorithm generates an error rate, so that a score is generated, the smaller the error rate is, the higher the score is, and the algorithm identification of the candidate algorithm with the highest score is selected as the label of the training sample. And simultaneously, extracting the characteristics of the scene data of the service scene by a waveform analysis algorithm to obtain the data waveform characteristics of the scene data of the service scene. And obtaining a training sample according to the data waveform characteristics of the scene data and the algorithm identification of the candidate algorithm with the highest score.
The n candidate algorithms may be, for example, 6, 7, etc., and are merely illustrative, and the number of candidate algorithms is determined according to the actual application scenario in practice, which is not limited in this application.
The types of the candidate algorithms are specifically determined according to the service scenarios, for example, the candidate algorithms in one service scenario include: at least 2 of a moving average algorithm, a fourier transform algorithm, an exponential smoothing algorithm, a neural network algorithm, a depth autoregressive algorithm, a limiting gradient hint tree algorithm, and the like.
Examples of application scenarios of candidate algorithms:
scene 1: and predicting key business scenes, such as predicting indexes such as payment number, success rate, failure rate, success number and the like of each APP in each minute under some payment scenes.
Scene 2: application log monitoring, for example, predicts the number of occurrences of a certain uri of a certain application in a log.
Scene 3: basic data monitoring, e.g., prediction of CPU, memory loss data of a computer device.
Prediction result:
for example, taking the indicator of the number of pays in scenario 1 as an example, the candidate algorithm predicts the number of pays for the APP today or tomorrow based on the APP's number of pays in the history.
The input data is historical data of APP payment number, and the output data is forecast data of APP payment number.
The obtained optimal algorithm is a recommended algorithm for processing the target scene.
In the prior art, algorithm configuration parameters are complex, operators cannot accurately judge which algorithm is more suitable for the current service scene in a short time, and the algorithm used in the possible configuration is not the best algorithm, so that the effect and the accuracy of an offline calculation result are reduced. In the embodiment, the recommendation algorithm based on the data waveform characteristics of the recommendation model selects a proper algorithm for any scene, so that repeated calculation can be avoided, and the prediction precision is improved.
According to the method, the characteristics of the target data are extracted by utilizing the waveform analysis algorithm to obtain the data waveform characteristics, the recommendation algorithm of the target scene is determined by utilizing the recommendation model according to the data waveform characteristics, the data waveform characteristics are smaller in data quantity compared with the target data, the quick reasoning and calculation of the recommendation model are more convenient, the calculation cost and calculation quantity of the model are reduced, and the generalization capability of scene identification can be improved by the data waveform characteristics; the algorithm recommendation scheme based on the data waveform characteristics can select a proper algorithm for any scene, can effectively avoid repeated calculation by using multiple algorithms in the prior art, and can improve the prediction accuracy of the service scene by selecting a recommended optimal algorithm from multiple candidate algorithms through a model. Particularly, in the fields of financial science and technology, such as banks, insurance and the like, the business scenes are numerous, and the waste of computing resources and storage resources of a financial system can be effectively avoided or reduced.
In one embodiment, the target data includes target sub-data for at least one indicator within a first preset time period;
in step S200, feature extraction is performed on the target data by using a waveform analysis algorithm to obtain waveform features of the data, including:
and respectively extracting the characteristics of the target sub-data of different indexes by utilizing a waveform analysis algorithm to obtain the waveform characteristics of the sub-data of each index.
Specifically, for the target scene, there may be at least one representative index, so the target data includes target sub-data of each index in the first preset time period in at least one index of the target scene, and the selection of the index is determined according to the actual application scene, which is not limited in this application. In addition, there may be a large difference in the index from one scene to another.
The first preset time period is a historical time period before the current moment. For example, the last 24 hours, the last 12 hours, the last 2 days, the last 5 days, etc. from the current time, which is not limited by the present application.
The target data comprises target sub-data of at least one index in a first preset time period, waveforms corresponding to each index can be obtained according to the target sub-data, and waveform data characteristics of each index are obtained through a waveform analysis algorithm.
According to the method and the device, the data waveform characteristics of the target sub-data of at least one index are extracted, the data waveform characteristics of each index are obtained and used as the characteristic data of the target scene, and the accuracy of algorithm recommendation of the target scene can be improved.
In one embodiment, the waveform analysis algorithm is a wavelet analysis algorithm;
the data waveform features include a low frequency data waveform feature and a high frequency data waveform feature.
Specifically, for example, the code is as follows:
import pywt
coeffs＝pywt.wavedec(data,"db4",level＝5)
ca5,cd5,cd4,cd3,cd2,cd1＝coeffs
the ca5 obtained according to the method is a low-frequency characteristic, the cd5 is a high-frequency characteristic, the data of the angle ca5 which is convenient to understand represent the long-term trend of a data curve, and the cd5 represents the period and oscillation in the short term of the data.
The data waveform feature may select the low frequency feature ca5 as the low frequency data waveform feature and the high frequency feature cd5 as the high frequency data waveform feature.
The low frequency characteristic represents a long-term trend characteristic of the data, and the high frequency characteristic represents a periodic characteristic of the data. Both low frequency features and high frequency features need to be input into the target recommendation model. For example, the encoded vectors corresponding to the high-frequency features and the low-frequency features are spliced and input into the target recommendation model, and each line of data input into the target recommendation model is a vector, for example, the first 52 numbers represent the low-frequency features, and the last 52 numbers represent the high-frequency features.
In addition, the low frequency features represent a long-term trend of the data, so the low frequency features can be used as main features and the high frequency features can be used as auxiliary features.
The output result of the target recommendation model is the algorithm identification or number representing the candidate algorithm with the highest score.
For example, a moving average algorithm is recommended if the data trend is relatively smooth, and a fourier transform algorithm is recommended if the data periodicity is strong.
Various characteristics of target data can be obtained through a waveform analysis algorithm, wherein the high-frequency characteristic and the low-frequency characteristic are the most representative characteristics, the data size of the high-frequency characteristic and the low-frequency characteristic is smaller than other characteristics, the calculation cost of a model can be effectively reduced, and the reasoning speed of the model is improved.
In the embodiment, the most representative high-frequency characteristic and low-frequency characteristic are selected to serve as the characteristics of the target data to be input into the recommendation model, so that the input of invalid characteristics can be reduced, and the calculation cost of the model is reduced.
In one embodiment, the obtained optimal algorithm is used for the target scene of the next second preset time period of the current second preset time period, and the optimal algorithms used for the target scenes in different second preset time periods are not necessarily the same.
Specifically, the current time is in the current second preset time period, and the embodiment is used for calculating a recommendation algorithm (i.e., an optimal algorithm) of the target scene in the next second preset time period at the current time.
Thus, the optimal algorithm within a second, different preset time period may be different for the same target scene. The reason for this is that the target data of the target scene is dynamically changed during the different first preset time period, and the optimal algorithm is obtained based on the target data, so that the optimal algorithm of the target scene may be changed along with the change of the target data.
For example, today, the optimal algorithm to be used by the target tomorrow is obtained and stored. In the same scenario, the algorithm is not fixed, and the optimal algorithm is dynamically changed every second preset duration according to the changed data. The second preset time is a duration of a second preset time period.
It should be noted that, the target data acquired in the current second preset period is the history data before the current data acquisition time.
And (3) for any target scene, the optimal algorithm of the target scene in any second preset time period can be obtained through the steps S100-S300 and related steps.
The recommendation algorithm of the target scene every second preset time length is the latest obtained optimal algorithm.
According to the method, the device and the system, the newly generated target data of the target scene are obtained regularly, the new optimal algorithm is obtained accordingly, the optimal algorithm of the same scene is updated, the recommended optimal algorithm can be changed according to the change of the scene data, the actual scene needs are met, and the accuracy of the recommended algorithm is achieved.
In one embodiment, the target recommendation model is one of at least 2 candidate recommendation models,
the method further comprises the steps of: if the time length of the currently used target recommendation model exceeds the time length threshold value and/or at least one backup recommendation model passes through new data iterative training and passes through accuracy verification, one of the backup recommendation models is used for replacing the currently used target recommendation model, wherein the backup recommendation model is other candidate recommendation models.
Specifically, there are at least 2 candidate recommendation models similar to the target recommendation model, and only one candidate recommendation model can be selected as the target recommendation model from all candidate recommendation models at each moment. The target recommendation model is a candidate recommendation model which is currently being used, and other unused candidate recommendation models can be continuously and iteratively trained and updated according to the newly collected training samples.
In general, the performance of the candidate recommendation model continuously trained in an iteration mode is better and better, so that after the continuous use time of each target recommendation model exceeds a time threshold, each target recommendation model is replaced by other candidate recommendation models, and the recommendation model with better performance is used, so that the accuracy of optimal algorithm selection is improved.
The backup recommendation model replacing the currently used target recommendation model is one of the other candidate recommendation models in addition to the target recommendation model. And the backup recommendation model is a model which passes through accuracy verification after iterative training.
Specifically, one backup recommendation model with optimal performance can be selected from a plurality of backup recommendation models to replace the target recommendation model with overtime current use.
Taking the example of maintaining 2 sets of candidate recommendation models, the first set of candidate recommendation models serves as the target recommendation model that is currently being applied in production (the model may not be updated based on data for one to two weeks). The second set of candidate recommendation models is not used in the production application, but is iteratively updated according to the latest data which are continuously acquired, and when the accuracy of the second set of candidate recommendation models is verified to exceed a certain standard and the continuous use time of the old first set of candidate recommendation models exceeds a certain time (such as one week and the like), the new second set of candidate recommendation models are used for replacing the old first set of candidate recommendation models, and the new second set of candidate recommendation models are used as target recommendation models in the production application and are used in the sound field environment. Thereby realizing the iterative switching of the candidate recommendation model.
According to the method and the device for learning the new data of the candidate recommendation model, new data are learned regularly on the candidate recommendation model which is not used temporarily, and the adaptability and generalization capability of the recommendation model can be effectively improved.
In the embodiment, multiple candidate recommendation models are maintained and managed, under the condition that one candidate recommendation model is normally used, other candidate recommendation models continue to carry out iterative training according to new data, under the condition that the opportunity is proper, the latest trained candidate recommendation model is used for replacing the old candidate recommendation model, so that the accuracy of optimal algorithm recommendation can be effectively improved, meanwhile, the selection of algorithms and operation of processes in a normal service scene can be prevented from being delayed by model training, and synchronization of model training and algorithm recommendation is achieved without delay.
In one embodiment, the method further comprises:
monitoring the application effect of the current recommended optimal algorithm in the target scene;
if the application effect does not meet the preset effect, notifying relevant personnel.
Specifically, acquiring an algorithm calculation result of an optimal algorithm recommended currently in a target scene; and obtaining an actual result of the target scene through statistical analysis of actual production data of the target scene. And taking the error or the loss function between the algorithm calculation result and the actual result as an application effect or an evaluation index. If the error is not in the allowable error range or the loss function is higher than the loss function threshold, judging that the application effect of the currently recommended optimal algorithm in the target scene does not meet the preset effect. And if the error is within the allowable error range or the loss function is lower than the loss function threshold, judging that the application effect of the currently recommended optimal algorithm in the target scene meets the preset effect.
If it is determined that the application effect of the currently recommended optimal algorithm does not meet the preset effect, the relevant person may be notified in an alarm manner. The purpose of informing related personnel can enable the related personnel to know the actual application effect of the recommended optimal algorithm in various scenes in time, and the algorithm can be adjusted in time under the condition of poor application effect, for example, the algorithm is designated for the target scene, so as to ensure that the flow operation of various scenes is smooth. The alarm method may be by means of mail, short message, system message, etc., which is not limited in this application.
According to the method and the device for monitoring the application effect of the recommendation algorithm, the defects of the recommendation algorithm can be found in time through monitoring the application effect of the recommendation algorithm which is currently applied, and related personnel can conveniently adjust and process the algorithm which does not meet expectations in time.
In one embodiment, the method further comprises:
determining a priority level of use of available algorithms in the target scene according to the configuration data;
the available algorithm of the highest priority is called according to the priority usage level, wherein the available algorithm comprises an optimal algorithm and a user-specified algorithm.
Specifically, in this embodiment, the algorithm recommendation system and the service system where the scene is located are the same system.
The embodiment is that an algorithm recommendation system or a service system is configured with a visual configuration interface capable of realizing man-machine interaction, and related personnel can assign an algorithm to any scene of the service system through the visual configuration interface.
For example, after any current service scenario is configured, a service person is required to manually select a corresponding algorithm according to the characteristics of data. A configuration interface is provided in the vector service monitoring system, and is used for configuring an offline algorithm used in each scene, wherein the offline algorithm can be configured as a specific specified algorithm or a recommended algorithm. If a specific algorithm is configured, the calculation is performed according to a policy specific to the specific algorithm. The recommended algorithm may be selected for use directly after the recommendation without requiring and understanding of a specific algorithm.
If the priority level of the designated algorithm is higher than that of the recommended optimal algorithm, the designated algorithm is preferentially used; if the recommended optimal algorithm priority is higher than the specified algorithm, the recommended optimal algorithm is preferentially used.
Of course, the algorithm recommendation system and the business system may be two different systems. In this case, the algorithm recommendation system is configured to recommend an optimal algorithm to a target service system corresponding to the target scene; the target service system is used for determining the priority use level of the available algorithms in the target scene according to the configuration data, and calling the available algorithm with the highest priority according to the priority use level, wherein the available algorithms comprise the optimal algorithm recommended by the algorithm recommendation system and the user-specified algorithm.
Or,
the algorithm recommendation system is used for storing the algorithm identification of the target scene optimal algorithm obtained each time and distributing versions for the algorithm identification stored each time; the target service system is used for acquiring an algorithm identifier of an optimal algorithm of the latest version of the target scene stored by the algorithm recommendation system under the condition that the algorithm of the target scene needs to be updated, determining a priority use level of an available algorithm in the target scene according to the configuration data, and calling the available algorithm with the highest priority according to the priority use level, wherein the available algorithm comprises the optimal algorithm corresponding to the acquired algorithm identifier and a user-specified algorithm.
The algorithm recommendation system can specifically persist the algorithm identification of the target scene optimal algorithm obtained each time in a database (such as hbase) to provide for online computing services and other service use or call.
The embodiment provides a plurality of optional modes such as user specification, system recommendation and the like for carrying out algorithm configuration on any scene, and flexible algorithm selection is realized.
Referring to fig. 3, the present application further provides an algorithm recommendation device based on data waveform characteristics, the device includes:
the first data acquisition module 100 is configured to acquire target data in a first preset time period of a target scene;
the feature extraction module 200 is configured to perform feature extraction on the target data by using a waveform analysis algorithm to obtain a data waveform feature;
the recommendation algorithm determining module 300 is configured to input the data waveform characteristics to a target recommendation model, and evaluate the processing capability of each candidate algorithm on the target data according to the data waveform characteristics by using the target recommendation model, so as to obtain a recommendation algorithm for processing the target scene.
According to the method, the characteristics of the target data are extracted by utilizing the waveform analysis algorithm to obtain the data waveform characteristics, the recommendation algorithm of the target scene is determined by utilizing the recommendation model according to the data waveform characteristics, the data waveform characteristics are smaller in data quantity compared with the target data, the quick reasoning and calculation of the recommendation model are more convenient, the calculation cost and calculation quantity of the model are reduced, and the generalization capability of scene identification can be improved by the data waveform characteristics; the algorithm recommendation scheme based on the data waveform characteristics can select a proper algorithm for any scene, can effectively avoid repeated calculation by using multiple algorithms in the prior art, and can improve the prediction accuracy of the service scene by selecting a recommended optimal algorithm from multiple candidate algorithms through a model. Particularly, in the fields of financial science and technology, such as banks, insurance and the like, the business scenes are numerous, and the waste of computing resources and storage resources of a financial system can be effectively avoided or reduced.
In one embodiment, the target data includes target sub-data for at least one indicator within a first preset time period;
the feature extraction module 200 is specifically configured to perform feature extraction on the target sub-data of different indexes by using a waveform analysis algorithm, so as to obtain waveform features of the sub-data of each index.
In one embodiment, the waveform analysis algorithm is a wavelet analysis algorithm;
the data waveform features include a low frequency data waveform feature and a high frequency data waveform feature.
In one embodiment, the obtained optimal algorithm is used for the target scene of the next second preset time period of the current second preset time period, and the optimal algorithms used for the target scenes in different second preset time periods are not necessarily the same.
In one embodiment, the target recommendation model is one of at least 2 candidate recommendation models,
the apparatus further comprises:
and the model replacement module is used for replacing the currently used target recommendation model by using one of the backup recommendation models if the time length of the currently used target recommendation model exceeds a time length threshold value and/or at least one backup recommendation model passes new data iterative training and passes accuracy verification, wherein the backup recommendation model is other candidate recommendation models.
In one embodiment, the apparatus further comprises:
the monitoring module is used for monitoring the application effect of the current recommended optimal algorithm in the target scene;
and the notification module is used for notifying related personnel if the application effect does not meet the preset effect.
In one embodiment, the apparatus further comprises:
the priority determining module is used for determining the priority use level of the available algorithm in the target scene according to the configuration data;
and the algorithm calling module is used for calling the available algorithm with the highest priority according to the priority using level, wherein the available algorithm comprises an optimal algorithm and a user-specified algorithm.
Compared with the prior art, the algorithm recommendation scheme can save huge calculation resources, and compared with the prior art, the original architecture needs the same service scene to be calculated for a plurality of times, and only needs to be calculated once after improvement, so that 83.3% of calculation power resources are saved. The algorithm recommendation scheme saves huge disk resources, multiple original architecture calculation results need to be saved, and only one part needs to be saved after the algorithm recommendation scheme is improved. The method and the device greatly facilitate the selection of the algorithm of operators, and the original offline algorithm configuration needs to be adjusted according to different algorithm parameters to see the accuracy degree of the prediction result. Only the recommended mode is needed to be selected after improvement, so that the cost of familiarity with different algorithms is saved. The method improves the accuracy of the final use prediction result, the original architecture is an algorithm selected according to the experience of service personnel, and under the condition that a new better algorithm exists after a plurality of configurations select one algorithm, the configuration personnel do not know or update in time, so that the result of the used algorithm is not the best algorithm. And the newly added algorithm can be obtained according to the recommendation model, and the optimal algorithm is selected from the historical algorithm and the newly added algorithm.
FIG. 4 illustrates an internal block diagram of a computer device in one embodiment. The computer device may specifically be a terminal or a server. As shown in fig. 4, the computer device includes a processor, a memory, and a network interface connected by a system bus. The memory includes a nonvolatile storage medium and an internal memory. The non-volatile storage medium of the computer device stores an operating system, and may also store a computer program which, when executed by a processor, causes the processor to implement the steps of the method embodiments described above. The internal memory may also have stored therein a computer program which, when executed by a processor, causes the processor to perform the steps of the method embodiments described above. Those skilled in the art will appreciate that the structures shown in FIG. 4 are block diagrams only and do not constitute a limitation of the computer device on which the present aspects apply, and that a particular computer device may include more or less components than those shown, or may combine some of the components, or have a different arrangement of components.
In one embodiment, a computer device is provided comprising a memory and a processor, the memory storing a computer program that, when executed by the processor, causes the processor to perform the steps of:
acquiring target data in a first preset time period of a target scene;
extracting characteristics of target data by utilizing a waveform analysis algorithm to obtain data waveform characteristics;
inputting the data waveform characteristics into a target recommendation model, and predicting and evaluating the processing capacity of each candidate algorithm on target data according to the data waveform characteristics by using the target recommendation model to obtain a recommendation algorithm for processing a target scene.
In one embodiment, a computer readable storage medium is provided, storing a computer program which, when executed by a processor, causes the processor to perform the steps of:
acquiring target data in a first preset time period of a target scene;
extracting characteristics of target data by utilizing a waveform analysis algorithm to obtain data waveform characteristics;
inputting the data waveform characteristics into a target recommendation model, and predicting and evaluating the processing capacity of each candidate algorithm on target data according to the data waveform characteristics by using the target recommendation model to obtain a recommendation algorithm for processing a target scene.
Those skilled in the art will appreciate that the processes implementing all or part of the methods of the above embodiments may be implemented by a computer program for instructing relevant hardware, and the program may be stored in a non-volatile computer readable storage medium, and the program may include the processes of the embodiments of the methods as above when executed. Any reference to memory, storage, database, or other medium used in the various embodiments provided herein may include non-volatile and/or volatile memory. The nonvolatile memory can include Read Only Memory (ROM), programmable ROM (PROM), electrically Programmable ROM (EPROM), electrically Erasable Programmable ROM (EEPROM), or flash memory. Volatile memory can include Random Access Memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms such as Static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double Data Rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous Link DRAM (SLDRAM), memory bus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), among others.
The technical features of the above embodiments may be arbitrarily combined, and all possible combinations of the technical features in the above embodiments are not described for brevity of description, however, as long as there is no contradiction between the combinations of the technical features, they should be considered as the scope of the description.
The foregoing examples represent only a few embodiments of the present application, which are described in more detail and are not thereby to be construed as limiting the scope of the present application. It should be noted that it would be apparent to those skilled in the art that various modifications and improvements could be made without departing from the spirit of the present application, which would be within the scope of the present application. Accordingly, the scope of protection of the present application is to be determined by the claims appended hereto.
Claims (10)
1. An algorithm recommendation method based on data waveform characteristics, the method comprising:
acquiring target data in a first preset time period of a target scene;
extracting characteristics of the target data by utilizing a waveform analysis algorithm to obtain data waveform characteristics;
inputting the data waveform characteristics into a target recommendation model, and predicting and evaluating the processing capacity of each candidate algorithm on the target data according to the data waveform characteristics by using the target recommendation model to obtain a recommendation algorithm for processing the target scene.
2. The method of claim 1, wherein the target data comprises target sub-data for at least one indicator over a first preset time period;
the step of extracting the characteristics of the target data by using a waveform analysis algorithm to obtain the waveform characteristics of the data comprises the following steps:
and respectively extracting the characteristics of the target sub-data of different indexes by utilizing a waveform analysis algorithm to obtain the waveform characteristics of the sub-data of each index.
3. The method of claim 1, wherein the waveform analysis algorithm is a wavelet analysis algorithm;
the data waveform features include low frequency data waveform features and high frequency data waveform features.
4. The method according to claim 1, wherein the obtained optimal algorithm is used for a target scene of a second preset time period next to the current second preset time period, and the optimal algorithm used for the target scene in different second preset time periods is not necessarily the same.
5. The method of claim 1, wherein the target recommendation model is one of at least 2 candidate recommendation models,
the method further comprises the steps of: if the time length of the currently used target recommendation model exceeds a time length threshold value and/or at least one backup recommendation model passes new data iterative training and passes accuracy verification, one of the backup recommendation models is used for replacing the currently used target recommendation model, wherein the backup recommendation model is other candidate recommendation models.
6. The method according to claim 1, wherein the method further comprises:
monitoring the application effect of the current recommended optimal algorithm in the target scene;
and if the application effect does not meet the preset effect, notifying related personnel.
7. The method according to claim 1, wherein the method further comprises:
determining a priority level of use of available algorithms in the target scene according to configuration data;
and calling the available algorithm with the highest priority according to the priority using level, wherein the available algorithm comprises the optimal algorithm and a user-specified algorithm.
8. An algorithm recommendation device based on data waveform characteristics, the device comprising:
the first data acquisition module is used for acquiring target data in a first preset time period of a target scene;
the feature extraction module is used for carrying out feature extraction on the target data by utilizing a waveform analysis algorithm to obtain data waveform features;
and the recommendation algorithm determining module is used for inputting the data waveform characteristics into a target recommendation model, and predicting and evaluating the processing capacity of each candidate algorithm on the target data according to the data waveform characteristics by utilizing the target recommendation model to obtain a recommendation algorithm for processing the target scene.
9. A computer readable storage medium storing a computer program, which when executed by a processor causes the processor to perform the steps of the method according to any one of claims 1 to 7.
10. A computer device comprising a memory and a processor, wherein the memory stores a computer program which, when executed by the processor, causes the processor to perform the steps of the method of any of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311481646.1A CN117520819A (en) | 2023-11-08 | 2023-11-08 | Algorithm recommendation method, device, equipment and medium based on data waveform characteristics |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311481646.1A CN117520819A (en) | 2023-11-08 | 2023-11-08 | Algorithm recommendation method, device, equipment and medium based on data waveform characteristics |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117520819A true CN117520819A (en) | 2024-02-06 |
Family
ID=89761944
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311481646.1A Pending CN117520819A (en) | 2023-11-08 | 2023-11-08 | Algorithm recommendation method, device, equipment and medium based on data waveform characteristics |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117520819A (en) |
-
2023
- 2023-11-08 CN CN202311481646.1A patent/CN117520819A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| KR102178633B1 (en) | User credit evaluation method and device, and storage medium | |
| CN107040397B (en) | Service parameter acquisition method and device | |
| CN110188910B (en) | Method and system for providing online prediction service by using machine learning model | |
| CN111191871A (en) | Project baseline data generation method and device, computer equipment and storage medium | |
| CN111314173A (en) | Monitoring information abnormity positioning method and device, computer equipment and storage medium | |
| CN111311286A (en) | Intelligent customer service data processing method and device, computing equipment and storage medium | |
| CN112508695A (en) | Financial risk prediction method and device based on market risk and electronic equipment | |
| CN117632905B (en) | Database management method and system based on cloud use records | |
| CN112801434A (en) | Method, device, equipment and storage medium for monitoring performance index health degree | |
| CN114595970A (en) | Resource scheduling intelligent decision method and device, electronic equipment and storage medium | |
| CN114446019A (en) | Alarm information processing method, device, equipment, storage medium and product | |
| CN116756522B (en) | Probability forecasting method and device, storage medium and electronic equipment | |
| CN114997879B (en) | Payment routing method, device, equipment and storage medium | |
| CN117520819A (en) | Algorithm recommendation method, device, equipment and medium based on data waveform characteristics | |
| CN113298120B (en) | Fusion model-based user risk prediction method, system and computer equipment | |
| CN116126859A (en) | Data management method and device, electronic equipment and storage medium | |
| CN112925723B (en) | Test service recommendation method and device, computer equipment and storage medium | |
| CN111967806B (en) | User risk updating method and device based on periodic retrace and electronic equipment | |
| CN115271277A (en) | Power equipment portrait construction method and system, computer equipment and storage medium | |
| CN115600818A (en) | Multi-dimensional scoring method and device, electronic equipment and storage medium | |
| CN111737319B (en) | User cluster prediction method, device, computer equipment and storage medium | |
| CN113296951A (en) | Resource allocation scheme determination method and equipment | |
| CN110874612B (en) | Time interval prediction method and device, computer equipment and storage medium | |
| CN116149971B (en) | Equipment fault prediction method and device, electronic equipment and storage medium | |
| CN111815442B (en) | Link prediction method and device and electronic equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |