CN113283677B - Index data processing method, device, equipment and storage medium - Google Patents
Index data processing method, device, equipment and storage medium Download PDFInfo
- Publication number
- CN113283677B CN113283677B CN202110723271.XA CN202110723271A CN113283677B CN 113283677 B CN113283677 B CN 113283677B CN 202110723271 A CN202110723271 A CN 202110723271A CN 113283677 B CN113283677 B CN 113283677B
- Authority
- CN
- China
- Prior art keywords
- index
- data
- analysis
- network layer
- model
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/04—Forecasting or optimisation specially adapted for administrative or management purposes, e.g. linear programming or "cutting stock problem"
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
- G06F16/2282—Tablespace storage structures; Management thereof
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/242—Query formulation
- G06F16/2433—Query languages
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/06—Resources, workflows, human or project management; Enterprise or organisation planning; Enterprise or organisation modelling
- G06Q10/063—Operations research, analysis or management
- G06Q10/0635—Risk analysis of enterprise or organisation activities
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/06—Resources, workflows, human or project management; Enterprise or organisation planning; Enterprise or organisation modelling
- G06Q10/063—Operations research, analysis or management
- G06Q10/0639—Performance analysis of employees; Performance analysis of enterprise or organisation operations
- G06Q10/06393—Score-carding, benchmarking or key performance indicator [KPI] analysis
Abstract
The invention relates to artificial intelligence and provides an index data processing method, device, equipment and storage medium. The method can determine an analysis index according to an index processing request, determine an index object to which the analysis index belongs according to the index processing request, determine an index field to which the analysis index belongs according to the index object, obtain an object model corresponding to the index object, obtain a field model corresponding to the index field, process the object model and the field model based on sample information of the analysis index to obtain a data prediction model, obtain analysis data of the analysis index according to the index processing request, preprocess the analysis data to obtain standard data, and process the standard data based on the data prediction model to obtain an index result. The method and the device can improve the accuracy of the index result and the generation efficiency of the index result. In addition, the invention also relates to a block chain technology, and the index result can be stored in the block chain.
Description
Technical Field
The present invention relates to the field of data processing technologies, and in particular, to a method, an apparatus, a device, and a storage medium for processing index data.
Background
For risk screening, index data prediction is usually performed on index data of a product planned to invest to determine the risk of the index.
In the current index data prediction scheme, index data are usually directly analyzed through a universal investment model on the market, and inaccurate data prediction is caused by the fact that the mode does not combine the product characteristics of the index to analyze the index data, so that risk investigation is not facilitated.
Disclosure of Invention
In view of the above, it is necessary to provide an index data processing method, apparatus, device and storage medium, which can not only improve the accuracy of index results, but also improve the efficiency of generating index results.
In one aspect, the present invention provides an index data processing method, where the index data processing method includes:
when an index processing request is received, determining an analysis index according to the index processing request;
determining an index object to which the analysis index belongs according to the index processing request, and determining an index field to which the analysis index belongs according to the index object;
acquiring an object model corresponding to the index object, and acquiring a field model corresponding to the index field;
processing the object model and the field model based on the sample information of the analysis index to obtain a data prediction model;
acquiring analysis data of the analysis index according to the index processing request;
preprocessing the analysis data to obtain standard data;
and processing the standard data based on the data prediction model to obtain an index result of the analysis index.
According to a preferred embodiment of the present invention, the determining, according to the index processing request, an index object to which the analysis index belongs includes:
determining a sending terminal of the index processing request, and determining a user corresponding to the sending terminal as a trigger user;
acquiring an object library corresponding to the trigger user;
traversing the object library based on the analysis index, and determining an object containing the analysis index in the object library as the index object.
According to a preferred embodiment of the present invention, the processing the object model and the domain model based on the sample information of the analysis index to obtain a data prediction model includes:
obtaining the sample information of the analysis index from a configuration library;
cutting the object model to obtain an object network layer, and cutting the field model to obtain a field network layer;
splicing the domain network layer, the object network layer and a preset network layer to obtain an initial model;
and adjusting parameters in the initial model based on the sample information until the initial model converges to obtain the data prediction model.
According to a preferred embodiment of the present invention, the obtaining the sample information of the analysis index from the configuration library includes:
acquiring a preset query template, wherein a time tag and a query object tag are stored in the preset query template;
determining the receiving time of the index processing request, and calculating the sum of the receiving time and a preset time period to obtain a query time period;
writing the query time interval into a position corresponding to the time tag in the preset query template, and writing the analysis index into a position corresponding to the query object in the preset query template to obtain a first query statement;
running the first query statement based on the configuration library to obtain a sample result;
acquiring sub-nodes of the analysis indexes from a preset decision tree as sub-indexes;
updating the analysis index in the first query statement based on the sub-index to obtain a second query statement, and operating the second query statement based on the configuration library to obtain sample data;
and determining the sample result and the sample data as the sample information.
According to the preferred embodiment of the present invention, the cutting the object model to obtain the object network layer includes:
determining all network layers in the object model as a first network layer;
extracting network parameters of each first network layer;
for each arbitrary first network layer, determining a first network layer except the arbitrary first network layer in the first network layers as a characteristic network layer;
comparing a first network parameter in the arbitrary first network layer with a second network parameter in the feature network layer;
if the second network parameter is the same as the first network parameter, determining the characteristic network layer as a repeated network layer;
and deleting the repeated network layer from the object model to obtain the object network layer.
According to a preferred embodiment of the present invention, the adjusting parameters in the initial model based on the sample information until the initial model converges to obtain the data prediction model includes:
inputting the sample data into the domain network layer to obtain a domain prediction result, and inputting the sample data into the object network layer to obtain an object prediction result;
processing the field prediction result and the object prediction result based on the preset network layer to obtain a target prediction result;
calculating a difference value between the target prediction result and the sample result to obtain a loss value;
and adjusting the parameters based on the loss value until the loss value is not reduced any more, so as to obtain the data prediction model.
According to a preferred embodiment of the present invention, the preprocessing the analysis data to obtain standard data includes:
identifying a data type of the analysis data;
calculating the data amount of the analysis data in each data type;
determining the data type with the maximum data volume as a target type;
determining the analysis data of which the data type is not the target type as feature data, and determining the type of the feature data as a feature type;
acquiring a relation mapping table of the feature type and the target type;
and converting the characteristic data based on the relational mapping table to obtain the standard data.
In another aspect, the present invention further provides an index data processing apparatus, including:
a determination unit configured to determine an analysis index according to an index processing request when the index processing request is received;
the determining unit is further configured to determine an index object to which the analysis index belongs according to the index processing request, and determine an index field to which the analysis index belongs according to the index object;
an obtaining unit, configured to obtain an object model corresponding to the index object, and obtain a domain model corresponding to the index domain;
the processing unit is used for processing the object model and the field model based on the sample information of the analysis index to obtain a data prediction model;
the acquisition unit is further used for acquiring analysis data of the analysis index according to the index processing request;
the preprocessing unit is used for preprocessing the analysis data to obtain standard data;
the processing unit is further configured to process the standard data based on the data prediction model to obtain an index result of the analysis index.
In another aspect, the present invention further provides an electronic device, including:
a memory storing computer readable instructions; and
a processor executing computer readable instructions stored in the memory to implement the metric data processing method.
In another aspect, the present invention further provides a computer-readable storage medium, in which computer-readable instructions are stored, and the computer-readable instructions are executed by a processor in an electronic device to implement the index data processing method.
According to the technical scheme, the object model and the field model are processed through the sample information, so that the generated data prediction model has the object characteristics of the object and the field characteristics of the field, the prediction precision of the data prediction model is improved, and the accuracy of the index result is improved. In addition, the standard data can be generated in a standardized manner by preprocessing the analysis data, so that the efficiency of processing the analysis indexes by the data prediction model is improved, and the generation efficiency of the index result is improved.
Drawings
FIG. 1 is a flowchart illustrating a method for processing index data according to a preferred embodiment of the present invention.
FIG. 2 is a functional block diagram of an index data processing apparatus according to a preferred embodiment of the present invention.
Fig. 3 is a schematic structural diagram of an electronic device implementing the method for processing index data according to a preferred embodiment of the invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention will be described in detail with reference to the accompanying drawings and specific embodiments.
FIG. 1 is a flow chart of a pointer data processing method according to a preferred embodiment of the present invention. The order of the steps in the flow chart may be changed and some steps may be omitted according to different needs.
The index data processing method is applied to one or more electronic devices, which are devices capable of automatically performing numerical calculation and/or information processing according to computer readable instructions set or stored in advance, and the hardware thereof includes, but is not limited to, a microprocessor, an Application Specific Integrated Circuit (ASIC), a Programmable Gate Array (FPGA), a Digital Signal Processor (DSP), an embedded device, and the like.
The electronic device may be any electronic product capable of performing human-computer interaction with a user, for example, a Personal computer, a tablet computer, a smart phone, a Personal Digital Assistant (PDA), a game machine, an interactive Internet Protocol Television (IPTV), a smart wearable device, and the like.
The electronic device may include a network device and/or a user device. Wherein the network device includes, but is not limited to, a single network electronic device, an electronic device group consisting of a plurality of network electronic devices, or a Cloud Computing (Cloud Computing) based Cloud consisting of a large number of hosts or network electronic devices.
The network in which the electronic device is located includes, but is not limited to: the internet, a wide area Network, a metropolitan area Network, a local area Network, a Virtual Private Network (VPN), and the like.
S10, when receiving the index processing request, determining the analysis index according to the index processing request.
In at least one embodiment of the present invention, the index processing request may be triggered by any user, and the information carried in the index processing request includes, but is not limited to: index identification code, analysis period, etc.
The analysis index refers to an index which needs to be subjected to data prediction analysis in the index processing request. The analysis index may be an index on any product in any field, for example, the analysis index may be an index corresponding to an a financial product in the financial field.
In at least one embodiment of the present invention, the electronic device determining an analysis metric according to the metric processing request includes:
analyzing the message of the index processing request to obtain message information carried by the message;
extracting information indicating indexes from the message information as index identification codes;
and determining the index corresponding to the index identification code as the analysis index.
The index identification code refers to an identifier of the analysis index, and the index identification code can be unique to the analysis index.
The index identification code can be quickly acquired through the message, and the analysis index can be accurately determined through the mapping relation between the index identification code and the index.
S11, determining the index object to which the analysis index belongs according to the index processing request, and determining the index field to which the analysis index belongs according to the index object.
In at least one embodiment of the present invention, the index object refers to a product where the analysis index is located, and the index field refers to a field where the analysis index is located.
In at least one embodiment of the present invention, the electronic device determining, according to the index processing request, an index object to which the analysis index belongs includes:
determining a sending terminal of the index processing request, and determining a user corresponding to the sending terminal as a trigger user;
acquiring an object library corresponding to the trigger user;
traversing the object library based on the analysis index, and determining an object containing the analysis index in the object library as the index object.
The sending terminal is a terminal for sending the index processing request to the electronic equipment.
The triggering user refers to a user who triggers the sending of the index processing request, and the triggering user may also be a user who triggers the generation of the index processing request.
The object library stores the mapping relation between the object associated with the trigger user and the index.
Through the implementation mode, the object library can be accurately determined, and further through traversing the object library, the index object to which the analysis index belongs can be accurately determined.
In at least one embodiment of the present invention, the electronic device determines a domain corresponding to the object and the analysis index as an index domain.
And S12, acquiring an object model corresponding to the index object and acquiring a field model corresponding to the index field.
In at least one embodiment of the present invention, the object model refers to a model generated based on object characteristics of the index object, the domain model refers to a model generated based on domain characteristics of the index domain, and the domain model is generally a model commonly used in the index domain.
And S13, processing the object model and the field model based on the sample information of the analysis index to obtain a data prediction model.
In at least one embodiment of the present invention, the sample information refers to information for training the data prediction model, and the sample information includes sample data and a sample result.
The data prediction model comprises a field network layer, an object network layer and a preset network layer, wherein the preset network layer is used for determining an index result. The parameters in the preset network layer can be determined according to the last network layer in the object model and the field model.
In at least one embodiment of the present invention, the processing, by the electronic device, the object model and the domain model based on the sample information of the analysis index to obtain a data prediction model includes:
obtaining the sample information of the analysis index from a configuration library;
cutting the object model to obtain an object network layer, and cutting the field model to obtain a field network layer;
splicing the domain network layer, the object network layer and a preset network layer to obtain an initial model;
and adjusting parameters in the initial model based on the sample information until the initial model converges to obtain the data prediction model.
Wherein, the configuration library stores related information of a plurality of indexes.
The object network layer refers to a model obtained by removing repeated network layers in the object model, and the field network layer refers to a model obtained by removing network layers with the same parameters in the field model.
The parameters comprise all configuration parameters on the domain network layer, the object network layer and a preset network layer.
The configuration library can be used for acquiring sample information of data standardization, the training efficiency of the data prediction model can be improved due to no need of preprocessing the sample information, the training efficiency of the data prediction model can be further improved by cutting an object model and the field model, and the prediction precision of the data prediction model can be ensured by adjusting the parameters through the sample information.
Specifically, the obtaining, by the electronic device, the sample information of the analysis indicator from the configuration library includes:
acquiring a preset query template, wherein a time tag and a query object tag are stored in the preset query template;
determining the receiving time of the index processing request, and calculating the sum of the receiving time and a preset time period to obtain a query time period;
writing the query time interval into a position corresponding to the time tag in the preset query template, and writing the analysis index into a position corresponding to the query object in the preset query template to obtain a first query statement;
running the first query statement based on the configuration library to obtain a sample result;
acquiring sub-nodes of the analysis indexes from a preset decision tree as sub-indexes;
updating the analysis index in the first query statement based on the sub-index to obtain a second query statement, and operating the second query statement based on the configuration library to obtain sample data;
and determining the sample result and the sample data as the sample information.
The preset query template stores the association relationship between the time tag and a certain position and the association relationship between the query object tag and the certain position.
The receiving time refers to a time when the electronic device receives the index processing request.
The preset time period can be configured arbitrarily.
The preset decision tree stores the connection relation of a plurality of indexes.
The first query statement can be generated quickly through the preset query template, so that the sample result can be obtained quickly, the first query statement is updated based on the sub-indexes, the second query statement can be generated quickly, and the sample data can be obtained quickly. In addition, the sub-indexes can be accurately obtained through the preset decision tree, and therefore sample data corresponding to the sample result can be accurately determined.
Specifically, the cutting the object model by the electronic device to obtain the object network layer includes:
determining all network layers in the object model as a first network layer;
extracting network parameters of each first network layer;
for each arbitrary first network layer, determining a first network layer except the arbitrary first network layer in the first network layers as a characteristic network layer;
comparing a first network parameter in the arbitrary first network layer with a second network parameter in the feature network layer;
if the second network parameter is the same as the first network parameter, determining the characteristic network layer as a repeated network layer;
and deleting the repeated network layer from the object model to obtain the object network layer.
Wherein the network parameter is a parameter that affects an output result of the object network layer.
By deleting the duplicate network layer, the repeated processing of the analysis data can be avoided, and the generation efficiency of the index result is improved.
Specifically, the manner in which the electronic device cuts the domain model is the same as the manner in which the electronic device cuts the object model, which is not described in detail herein.
Specifically, the sample information includes sample data and a sample result, the initial model includes the domain network layer, the object network layer and a preset network layer, the electronic device adjusts parameters in the initial model based on the sample information until the initial model converges, and obtaining the data prediction model includes:
inputting the sample data into the domain network layer to obtain a domain prediction result, and inputting the sample data into the object network layer to obtain an object prediction result;
processing the field prediction result and the object prediction result based on the preset network layer to obtain a target prediction result;
calculating a difference value between the target prediction result and the sample result to obtain a loss value;
and adjusting the parameters based on the loss value until the loss value is not reduced any more, so as to obtain the data prediction model.
The parameters are adjusted through the loss values, so that the prediction precision of the data prediction model can be ensured, and the accuracy of index results is improved.
And S14, acquiring the analysis data of the analysis index according to the index processing request.
In at least one embodiment of the present invention, the analysis data refers to data required for predicting an index result of the analysis index.
In at least one embodiment of the present invention, the acquiring, by the electronic device, analysis data of the analysis index according to the index processing request includes:
determining the trigger user according to the index processing request;
acquiring a user database corresponding to the trigger user;
acquiring information indicating time from the index processing request as an analysis period;
writing the analysis time interval into a position corresponding to the time tag in the preset query template, and writing the analysis index into a position corresponding to the query object in the preset query template to obtain a target query statement;
and running the target query statement based on the user database to obtain the analysis data.
Through the implementation mode, the data source required for predicting the analysis index can be accurately determined, so that the analysis data can be accurately acquired.
And S15, preprocessing the analysis data to obtain standard data.
In at least one embodiment of the present invention, the standard data refers to analysis data with the same data type.
In at least one embodiment of the present invention, the electronic device pre-processes the analysis data to obtain standard data, including:
identifying a data type of the analysis data;
calculating the data amount of the analysis data in each data type;
determining the data type with the maximum data volume as a target type;
determining the analysis data of which the data type is not the target type as feature data, and determining the type of the feature data as a feature type;
acquiring a relation mapping table of the feature type and the target type;
and converting the characteristic data based on the relational mapping table to obtain the standard data.
Wherein the data type refers to a type to which the analysis data belongs, for example, the data type includes, but is not limited to: binary value type, character type, etc.
The relation mapping table stores the conversion relation between the feature type and the target type.
The feature data corresponding to the data type with small data volume are subjected to standardization processing, so that the data volume of the standardization processing can be reduced, the preprocessing efficiency is improved, and the feature data can be accurately converted into the standard data corresponding to the target type through the relational mapping table.
And S16, processing the standard data based on the data prediction model to obtain an index result of the analysis index.
In at least one embodiment of the present invention, the index result refers to a result predicted based on the analysis data and corresponding to the analysis index.
It is emphasized that the index result can also be stored in a node of a blockchain in order to further ensure the privacy and security of the index result.
In at least one embodiment of the present invention, after obtaining the index result of the analysis index, the method further includes:
acquiring a request number of the index processing request;
generating prompt information according to the request number and the index result;
encrypting the prompt information by adopting a symmetric encryption technology to obtain a ciphertext;
and sending the ciphertext to the sending terminal.
According to the embodiment, the index result can be prevented from being tampered, and the safety of the index result can be improved.
According to the technical scheme, the object model and the field model are processed through the sample information, so that the generated data prediction model has the object characteristics of the object and the field characteristics of the field, the prediction precision of the data prediction model is improved, and the accuracy of the index result is improved. In addition, the standard data can be generated in a standardized manner by preprocessing the analysis data, so that the efficiency of processing the analysis indexes by the data prediction model is improved, and the generation efficiency of the index result is improved.
FIG. 2 is a functional block diagram of an index data processing apparatus according to a preferred embodiment of the present invention. The index data processing apparatus 11 includes a determination unit 110, an acquisition unit 111, a processing unit 112, a preprocessing unit 113, a generation unit 114, an encryption unit 115, and a transmission unit 116. A module/unit as referred to herein is a series of computer readable instruction segments capable of being retrieved by the processor 13 and performing a fixed function, and stored in the memory 12. In the present embodiment, the functions of the modules/units will be described in detail in the following embodiments.
When receiving the index processing request, the determination unit 110 determines an analysis index according to the index processing request.
In at least one embodiment of the present invention, the index processing request may be triggered by any user, and the information carried in the index processing request includes, but is not limited to: index identification code, analysis period, etc.
The analysis index refers to an index which needs to be subjected to data prediction analysis in the index processing request. The analysis index may be an index on any product in any field, for example, the analysis index may be an index corresponding to a financial product a in a financial field.
In at least one embodiment of the present invention, the determining unit 110 determines the analysis index according to the index processing request includes:
analyzing the message of the index processing request to obtain message information carried by the message;
extracting information indicating indexes from the message information as index identification codes;
and determining the index corresponding to the index identification code as the analysis index.
The index identification code refers to an identifier of the analysis index, and the index identification code can be unique to the analysis index.
The index identification code can be quickly obtained through the message, and the analysis index can be accurately determined through the mapping relation between the index identification code and the index.
The determination unit 110 determines an index object to which the analysis index belongs according to the index processing request, and determines an index field to which the analysis index belongs according to the index object.
In at least one embodiment of the present invention, the index object refers to a product where the analysis index is located, and the index field refers to a field where the analysis index is located.
In at least one embodiment of the present invention, the determination unit 110, according to the index processing request, determines the index object to which the analysis index belongs, including:
determining a sending terminal of the index processing request, and determining a user corresponding to the sending terminal as a trigger user;
acquiring an object library corresponding to the trigger user;
traversing the object library based on the analysis index, and determining an object containing the analysis index in the object library as the index object.
The sending terminal is a terminal which sends the index processing request to the electronic equipment.
The triggering user refers to a user who triggers the sending of the index processing request, and the triggering user may also be a user who triggers the generation of the index processing request.
The object library stores the mapping relation between the object associated with the trigger user and the index.
Through the implementation mode, the object library can be accurately determined, and further through traversing the object library, the index object to which the analysis index belongs can be accurately determined.
In at least one embodiment of the present invention, the determination unit 110 determines a domain corresponding to the object and the analysis index as an index domain.
The obtaining unit 111 obtains an object model corresponding to the index object, and obtains a domain model corresponding to the index domain.
In at least one embodiment of the present invention, the object model refers to a model generated based on object characteristics of the index object, the domain model refers to a model generated based on domain characteristics of the index domain, and the domain model is generally a model commonly used in the index domain.
The processing unit 112 processes the object model and the field model based on the sample information of the analysis index to obtain a data prediction model.
In at least one embodiment of the present invention, the sample information refers to information for training the data prediction model, and the sample information includes sample data and a sample result.
The data prediction model comprises a field network layer, an object network layer and a preset network layer, wherein the preset network layer is used for determining an index result. The parameters in the preset network layer can be determined according to the last network layer in the object model and the field model.
In at least one embodiment of the present invention, the processing unit 112 processes the object model and the field model based on the sample information of the analysis index, and obtaining a data prediction model includes:
obtaining the sample information of the analysis index from a configuration library;
cutting the object model to obtain an object network layer, and cutting the field model to obtain a field network layer;
splicing the domain network layer, the object network layer and a preset network layer to obtain an initial model;
and adjusting parameters in the initial model based on the sample information until the initial model converges to obtain the data prediction model.
Wherein, the configuration library stores related information of a plurality of indexes.
The object network layer is a model obtained by removing repeated network layers in the object model, and the field network layer is a model obtained by removing network layers with the same parameters in the field model.
The parameters comprise all configuration parameters on the domain network layer, the object network layer and a preset network layer.
The configuration library can be used for acquiring sample information of data standardization, the training efficiency of the data prediction model can be improved due to no need of preprocessing the sample information, the training efficiency of the data prediction model can be further improved by cutting an object model and the field model, and the prediction precision of the data prediction model can be ensured by adjusting the parameters through the sample information.
Specifically, the obtaining, by the processing unit 112, the sample information of the analysis index from the configuration library includes:
acquiring a preset query template, wherein a time tag and a query object tag are stored in the preset query template;
determining the receiving time of the index processing request, and calculating the sum of the receiving time and a preset time period to obtain a query time period;
writing the query time interval into a position corresponding to the time tag in the preset query template, and writing the analysis index into a position corresponding to the query object in the preset query template to obtain a first query statement;
running the first query statement based on the configuration library to obtain a sample result;
acquiring sub-nodes of the analysis indexes from a preset decision tree as sub-indexes;
updating the analysis index in the first query statement based on the sub-index to obtain a second query statement, and operating the second query statement based on the configuration library to obtain sample data;
and determining the sample result and the sample data as the sample information.
The preset query template stores the association relationship between the time tag and a certain position and the association relationship between the query object tag and the certain position.
The receiving time refers to a time when the electronic device receives the index processing request.
The preset time period can be configured arbitrarily.
The preset decision tree stores the connection relation of a plurality of indexes.
The first query statement can be generated quickly through the preset query template, so that the sample result can be obtained quickly, the first query statement is updated based on the sub-indexes, the second query statement can be generated quickly, and the sample data can be obtained quickly. In addition, the sub-indexes can be accurately obtained through the preset decision tree, and therefore sample data corresponding to the sample result can be accurately determined.
Specifically, the cutting the object model by the processing unit 112 to obtain the object network layer includes:
determining all network layers in the object model as a first network layer;
extracting network parameters of each first network layer;
for each arbitrary first network layer, determining a first network layer except the arbitrary first network layer in the first network layers as a characteristic network layer;
comparing a first network parameter in the arbitrary first network layer with a second network parameter in the feature network layer;
if the second network parameter is the same as the first network parameter, determining the characteristic network layer as a repeated network layer;
and deleting the repeated network layer from the object model to obtain the object network layer.
Wherein the network parameter is a parameter that affects an output result of the object network layer.
By deleting the duplicate network layer, the repeated processing of the analysis data can be avoided, and the generation efficiency of the index result is improved.
Specifically, the manner of clipping the domain model by the processing unit 112 is the same as the manner of clipping the object model by the processing unit 112, which is not described in detail herein.
Specifically, the sample information includes sample data and a sample result, the initial model includes the domain network layer, the object network layer, and a preset network layer, the processing unit 112 adjusts parameters in the initial model based on the sample information until the initial model converges, and obtaining the data prediction model includes:
inputting the sample data into the domain network layer to obtain a domain prediction result, and inputting the sample data into the object network layer to obtain an object prediction result;
processing the field prediction result and the object prediction result based on the preset network layer to obtain a target prediction result;
calculating a difference value between the target prediction result and the sample result to obtain a loss value;
and adjusting the parameters based on the loss value until the loss value is not reduced any more, so as to obtain the data prediction model.
The parameters are adjusted through the loss values, so that the prediction precision of the data prediction model can be ensured, and the accuracy of index results is improved.
The acquisition unit 111 acquires analysis data of the analysis index according to the index processing request.
In at least one embodiment of the present invention, the analysis data refers to data required for predicting an index result of the analysis index.
In at least one embodiment of the present invention, the acquiring unit 111 acquires analysis data of the analysis index according to the index processing request includes:
determining the trigger user according to the index processing request;
acquiring a user database corresponding to the trigger user;
acquiring information indicating time from the index processing request as an analysis period;
writing the analysis time interval into a position corresponding to the time tag in the preset query template, and writing the analysis index into a position corresponding to the query object in the preset query template to obtain a target query statement;
and running the target query statement based on the user database to obtain the analysis data.
Through the implementation mode, the data source required for predicting the analysis index can be accurately determined, so that the analysis data can be accurately acquired.
The preprocessing unit 113 preprocesses the analysis data to obtain standard data.
In at least one embodiment of the present invention, the standard data refers to analysis data with the same data type.
In at least one embodiment of the present invention, the preprocessing unit 113 preprocesses the analysis data to obtain standard data, and includes:
identifying a data type of the analysis data;
calculating the data amount of the analysis data in each data type;
determining the data type with the maximum data volume as a target type;
determining the analysis data of which the data type is not the target type as feature data, and determining the type of the feature data as a feature type;
acquiring a relation mapping table of the feature type and the target type;
and converting the characteristic data based on the relational mapping table to obtain the standard data.
Wherein the data type refers to a type to which the analysis data belongs, for example, the data type includes, but is not limited to: binary value type, character type, etc.
The relation mapping table stores the conversion relation between the feature type and the target type.
By standardizing the characteristic data corresponding to the data type with smaller data volume, the data volume of the standardized processing can be reduced, so that the preprocessing efficiency is improved, and the characteristic data can be accurately converted into the standard data corresponding to the target type through the relational mapping table.
The processing unit 112 processes the standard data based on the data prediction model to obtain an index result of the analysis index.
In at least one embodiment of the present invention, the index result refers to a result predicted based on the analysis data and corresponding to the analysis index.
It is emphasized that the index result can also be stored in a node of a blockchain in order to further ensure the privacy and security of the index result.
In at least one embodiment of the present invention, after obtaining the index result of the analysis index, the obtaining unit 111 obtains the request number of the index processing request;
the generating unit 114 generates prompt information according to the request number and the index result;
the encryption unit 115 encrypts the prompt message by using a symmetric encryption technology to obtain a ciphertext;
the transmission unit 116 transmits the ciphertext to the transmitting terminal.
According to the embodiment, the index result can be prevented from being tampered, and the safety of the index result is improved.
According to the technical scheme, the object model and the field model are processed through the sample information, so that the generated data prediction model has the object characteristics of the object and the field characteristics of the field, the prediction precision of the data prediction model is improved, and the accuracy of the index result is improved. In addition, the standard data can be generated in a standardized manner by preprocessing the analysis data, so that the efficiency of processing the analysis indexes by the data prediction model is improved, and the generation efficiency of the index result is improved.
Fig. 3 is a schematic structural diagram of an electronic device according to a preferred embodiment of the method for processing pointer data according to the present invention.
In one embodiment of the present invention, the electronic device 1 includes, but is not limited to, a memory 12, a processor 13, and computer readable instructions, such as an index data processing program, stored in the memory 12 and executable on the processor 13.
It will be appreciated by those skilled in the art that the schematic diagram is merely an example of the electronic device 1, and does not constitute a limitation of the electronic device 1, and may include more or less components than those shown, or combine some components, or different components, for example, the electronic device 1 may further include an input-output device, a network access device, a bus, etc.
The Processor 13 may be a Central Processing Unit (CPU), other general purpose Processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA) or other Programmable logic device, discrete Gate or transistor logic device, discrete hardware component, etc. The processor 13 is an operation core and a control center of the electronic device 1, and is connected to each part of the whole electronic device 1 by various interfaces and lines, and executes an operating system of the electronic device 1 and various installed application programs, program codes, and the like.
Illustratively, the computer readable instructions may be partitioned into one or more modules/units that are stored in the memory 12 and executed by the processor 13 to implement the present invention. The one or more modules/units may be a series of computer readable instruction segments capable of performing specific functions, which are used for describing the execution process of the computer readable instructions in the electronic device 1. For example, the computer readable instructions may be divided into a determination unit 110, an acquisition unit 111, a processing unit 112, a preprocessing unit 113, a generation unit 114, an encryption unit 115, and a transmission unit 116.
The memory 12 may be used for storing the computer readable instructions and/or modules, and the processor 13 implements various functions of the electronic device 1 by executing or executing the computer readable instructions and/or modules stored in the memory 12 and invoking data stored in the memory 12. The memory 12 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application program required by at least one function (such as a sound playing function, an image playing function, etc.), and the like; the storage data area may store data created according to use of the electronic device, and the like. The memory 12 may include non-volatile and volatile memories, such as: a hard disk, a memory, a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card), at least one magnetic disk storage device, a Flash memory device, or other storage device.
The memory 12 may be an external memory and/or an internal memory of the electronic device 1. Further, the memory 12 may be a memory having a physical form, such as a memory stick, a TF Card (Trans-flash Card), or the like.
The integrated modules/units of the electronic device 1 may be stored in a computer-readable storage medium if they are implemented in the form of software functional units and sold or used as separate products. Based on such understanding, all or part of the flow of the method according to the above embodiments may be implemented by hardware that is configured to be instructed by computer readable instructions, which may be stored in a computer readable storage medium, and when the computer readable instructions are executed by a processor, the steps of the method embodiments may be implemented.
Wherein the computer readable instructions comprise computer readable instruction code which may be in source code form, object code form, an executable file or some intermediate form, and the like. The computer-readable medium may include: any entity or device capable of carrying said computer readable instruction code, recording medium, U-disk, removable hard disk, magnetic disk, optical disk, computer Memory, Read-Only Memory (ROM), Random Access Memory (RAM).
The block chain is a novel application mode of computer technologies such as distributed data storage, point-to-point transmission, a consensus mechanism, an encryption algorithm and the like. A block chain (Blockchain), which is essentially a decentralized database, is a series of data blocks associated by using a cryptographic method, and each data block contains information of a batch of network transactions, so as to verify the validity (anti-counterfeiting) of the information and generate a next block. The blockchain may include a blockchain underlying platform, a platform product service layer, an application service layer, and the like.
With reference to fig. 1, the memory 12 in the electronic device 1 stores computer-readable instructions to implement a metric data processing method, and the processor 13 can execute the computer-readable instructions to implement:
when an index processing request is received, determining an analysis index according to the index processing request;
determining an index object to which the analysis index belongs according to the index processing request, and determining an index field to which the analysis index belongs according to the index object;
acquiring an object model corresponding to the index object, and acquiring a field model corresponding to the index field;
processing the object model and the field model based on the sample information of the analysis index to obtain a data prediction model;
acquiring analysis data of the analysis index according to the index processing request;
preprocessing the analysis data to obtain standard data;
and processing the standard data based on the data prediction model to obtain an index result of the analysis index.
Specifically, the processor 13 may refer to the description of the relevant steps in the embodiment corresponding to fig. 1 for a specific implementation method of the computer readable instructions, which is not described herein again.
In the embodiments provided in the present invention, it should be understood that the disclosed system, apparatus and method may be implemented in other ways. For example, the above-described apparatus embodiments are merely illustrative, and for example, the division of the modules is only one logical functional division, and other divisions may be realized in practice.
The computer readable storage medium has computer readable instructions stored thereon, wherein the computer readable instructions when executed by the processor 13 are configured to implement the steps of:
when an index processing request is received, determining an analysis index according to the index processing request;
determining an index object to which the analysis index belongs according to the index processing request, and determining an index field to which the analysis index belongs according to the index object;
acquiring an object model corresponding to the index object, and acquiring a field model corresponding to the index field;
processing the object model and the field model based on the sample information of the analysis index to obtain a data prediction model;
acquiring analysis data of the analysis index according to the index processing request;
preprocessing the analysis data to obtain standard data;
and processing the standard data based on the data prediction model to obtain an index result of the analysis index.
The modules described as separate parts may or may not be physically separate, and parts displayed as modules may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the present embodiment.
In addition, functional modules in the embodiments of the present invention may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, or in a form of hardware plus a software functional module.
The present embodiments are therefore to be considered in all respects as illustrative and not restrictive, the scope of the invention being indicated by the appended claims rather than by the foregoing description, and all changes which come within the meaning and range of equivalency of the claims are therefore intended to be embraced therein. Any reference signs in the claims shall not be construed as limiting the claim concerned.
Furthermore, it is obvious that the word "comprising" does not exclude other elements or steps, and the singular does not exclude the plural. The plurality of units or devices may also be implemented by one unit or device through software or hardware. The terms first, second, etc. are used to denote names, but not any particular order.
Finally, it should be noted that the above embodiments are only for illustrating the technical solutions of the present invention and not for limiting, and although the present invention is described in detail with reference to the preferred embodiments, it should be understood by those skilled in the art that modifications or equivalent substitutions may be made on the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention.
Claims (8)
1. An index data processing method, characterized by comprising:
when an index processing request is received, determining an analysis index according to the index processing request;
determining an index object to which the analysis index belongs according to the index processing request, including: determining a sending terminal of the index processing request, and determining a user corresponding to the sending terminal as a trigger user; acquiring an object library corresponding to the trigger user; traversing the object library based on the analysis index, determining an object containing the analysis index in the object library as the index object, and determining an index field to which the analysis index belongs according to the index object, including: determining a field corresponding to the index object and the analysis index as an index field;
acquiring an object model corresponding to the index object, and acquiring a field model corresponding to the index field;
processing the object model and the field model based on the sample information of the analysis index to obtain a data prediction model, including: obtaining the sample information of the analysis index from a configuration library; cutting the object model to obtain an object network layer, and cutting the field model to obtain a field network layer; splicing the domain network layer, the object network layer and a preset network layer to obtain an initial model; adjusting parameters in the initial model based on the sample information until the initial model converges to obtain the data prediction model;
acquiring analysis data of the analysis index according to the index processing request;
preprocessing the analysis data to obtain standard data;
and processing the standard data based on the data prediction model to obtain an index result of the analysis index.
2. The index data processing method of claim 1, wherein the obtaining the sample information of the analysis index from a configuration library comprises:
acquiring a preset query template, wherein a time tag and a query object tag are stored in the preset query template;
determining the receiving time of the index processing request, and calculating the sum of the receiving time and a preset time period to obtain a query time period;
writing the query time interval into a position corresponding to the time tag in the preset query template, and writing the analysis index into a position corresponding to the query object in the preset query template to obtain a first query statement;
running the first query statement based on the configuration library to obtain a sample result;
acquiring sub-nodes of the analysis indexes from a preset decision tree as sub-indexes;
updating the analysis index in the first query statement based on the sub-index to obtain a second query statement, and operating the second query statement based on the configuration library to obtain sample data;
and determining the sample result and the sample data as the sample information.
3. The method of claim 1, wherein the clipping the object model to obtain an object network layer comprises:
determining all network layers in the object model as a first network layer;
extracting network parameters of each first network layer;
for each arbitrary first network layer, determining a first network layer except the arbitrary first network layer in the first network layers as a characteristic network layer;
comparing a first network parameter in the arbitrary first network layer with a second network parameter in the feature network layer;
if the second network parameter is the same as the first network parameter, determining the characteristic network layer as a repeated network layer;
and deleting the repeated network layer from the object model to obtain the object network layer.
4. The method of claim 2, wherein the adjusting parameters in the initial model based on the sample information until the initial model converges to obtain the data prediction model comprises:
inputting the sample data into the domain network layer to obtain a domain prediction result, and inputting the sample data into the object network layer to obtain an object prediction result;
processing the field prediction result and the object prediction result based on the preset network layer to obtain a target prediction result;
calculating a difference value between the target prediction result and the sample result to obtain a loss value;
and adjusting the parameters based on the loss value until the loss value is not reduced any more, so as to obtain the data prediction model.
5. The index data processing method of claim 1, wherein the preprocessing the analysis data to obtain standard data comprises:
identifying a data type of the analysis data;
calculating the data amount of the analysis data in each data type;
determining the data type with the maximum data volume as a target type;
determining the analysis data of which the data type is not the target type as feature data, and determining the type of the feature data as a feature type;
acquiring a relation mapping table of the feature type and the target type;
and converting the characteristic data based on the relational mapping table to obtain the standard data.
6. An index data processing apparatus characterized by comprising:
a determination unit configured to determine an analysis index according to an index processing request when the index processing request is received;
the determining unit is further configured to determine, according to the index processing request, an index object to which the analysis index belongs, and includes: determining a sending terminal of the index processing request, and determining a user corresponding to the sending terminal as a trigger user; acquiring an object library corresponding to the trigger user; traversing the object library based on the analysis index, determining an object containing the analysis index in the object library as the index object, and determining an index field to which the analysis index belongs according to the index object, including: determining a field corresponding to the index object and the analysis index as an index field;
an obtaining unit, configured to obtain an object model corresponding to the index object, and obtain a domain model corresponding to the index domain;
the processing unit is used for processing the object model and the field model based on the sample information of the analysis index to obtain a data prediction model, and comprises: obtaining the sample information of the analysis index from a configuration library; cutting the object model to obtain an object network layer, and cutting the field model to obtain a field network layer; splicing the domain network layer, the object network layer and a preset network layer to obtain an initial model; adjusting parameters in the initial model based on the sample information until the initial model converges to obtain the data prediction model;
the acquisition unit is further used for acquiring analysis data of the analysis index according to the index processing request;
the preprocessing unit is used for preprocessing the analysis data to obtain standard data;
and the processing unit is used for processing the standard data based on the data prediction model to obtain an index result of the analysis index.
7. An electronic device, characterized in that the electronic device comprises:
a memory storing computer readable instructions; and
a processor executing computer readable instructions stored in the memory to implement the index data processing method of any one of claims 1 to 5.
8. A computer-readable storage medium characterized by: the computer-readable storage medium stores therein computer-readable instructions that are executed by a processor in an electronic device to implement the index data processing method according to any one of claims 1 to 5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110723271.XA CN113283677B (en) | 2021-06-29 | 2021-06-29 | Index data processing method, device, equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110723271.XA CN113283677B (en) | 2021-06-29 | 2021-06-29 | Index data processing method, device, equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113283677A CN113283677A (en) | 2021-08-20 |
| CN113283677B true CN113283677B (en) | 2022-08-26 |
Family
ID=77285956
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110723271.XA Active CN113283677B (en) | 2021-06-29 | 2021-06-29 | Index data processing method, device, equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113283677B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115048613B (en) * | 2022-08-16 | 2023-05-12 | 四川大学华西医院 | Index homogenization conversion method and device, electronic equipment and storage medium |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110930078A (en) * | 2019-12-31 | 2020-03-27 | 中国银行股份有限公司 | Business object identification method, device and equipment |
| CN111260142A (en) * | 2020-01-19 | 2020-06-09 | 杭州网易再顾科技有限公司 | Commodity index data prediction method and device, storage medium and electronic equipment |
| WO2020249017A1 (en) * | 2019-06-10 | 2020-12-17 | 中国移动通信有限公司研究院 | Service processing method and device for access network |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109376275A (en) * | 2018-10-29 | 2019-02-22 | 上海点融信息科技有限责任公司 | For monitoring the method, apparatus and medium of the operational indicator on block chain |
| CN112215424A (en) * | 2020-10-16 | 2021-01-12 | 平安国际智慧城市科技股份有限公司 | Medical index prediction method, device, electronic equipment and storage medium |
-
2021
- 2021-06-29 CN CN202110723271.XA patent/CN113283677B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2020249017A1 (en) * | 2019-06-10 | 2020-12-17 | 中国移动通信有限公司研究院 | Service processing method and device for access network |
| CN110930078A (en) * | 2019-12-31 | 2020-03-27 | 中国银行股份有限公司 | Business object identification method, device and equipment |
| CN111260142A (en) * | 2020-01-19 | 2020-06-09 | 杭州网易再顾科技有限公司 | Commodity index data prediction method and device, storage medium and electronic equipment |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113283677A (en) | 2021-08-20 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111488363B (en) | Data processing method, device, electronic equipment and medium | |
| CN112669138B (en) | Data processing method and related equipment | |
| CN113032528B (en) | Case analysis method, case analysis device, case analysis equipment and storage medium | |
| CN113283675B (en) | Index data analysis method, device, equipment and storage medium | |
| CN111639487A (en) | Classification model-based field extraction method and device, electronic equipment and medium | |
| CN114035827A (en) | Application program updating method, device, equipment and storage medium | |
| CN113536770B (en) | Text analysis method, device and equipment based on artificial intelligence and storage medium | |
| CN112541009A (en) | Data query method and device, electronic equipment and storage medium | |
| CN113283677B (en) | Index data processing method, device, equipment and storage medium | |
| CN113918467A (en) | Financial system testing method, device, equipment and storage medium | |
| CN114372060A (en) | Data storage method, device, equipment and storage medium | |
| CN113268597A (en) | Text classification method, device, equipment and storage medium | |
| CN112947911A (en) | Interface script generation method, device, equipment and storage medium | |
| CN112948418A (en) | Dynamic query method, device, equipment and storage medium | |
| CN113065947A (en) | Data processing method, device, equipment and storage medium | |
| CN112711398A (en) | Method, device and equipment for generating buried point file and storage medium | |
| CN112434062A (en) | Quasi-real-time data processing method, device, server and storage medium | |
| CN114360732B (en) | Medical data analysis method, device, electronic equipment and storage medium | |
| CN112949305B (en) | Negative feedback information acquisition method, device, equipment and storage medium | |
| CN115205032A (en) | Credit investigation query method, apparatus, device and storage medium | |
| CN113269179B (en) | Data processing method, device, equipment and storage medium | |
| CN114942749A (en) | Development method, device and equipment of approval system and storage medium | |
| CN113434273B (en) | Data processing method, device, system and storage medium | |
| CN113343700B (en) | Data processing method, device, equipment and storage medium | |
| CN114238296A (en) | Product index data display method, device, equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |