CN115062705A - Data verification method, data verification device, electronic equipment and storage medium - Google Patents

Abstract

The embodiments of the present application provide a data verification method, a data verification device, an electronic device and a storage medium, which belong to the technical field of artificial intelligence. The method includes: acquiring original business data at a target stage; classifying and processing the original business data according to a preset label to obtain the label business data; extracting features from the label business data according to a preset extraction method to obtain key business features; Set an algorithm to predict and process key business features to obtain business deviation data; perform status verification on original business data according to business deviation data to obtain verification results; output task scheduling instructions or perform data correction on original business data according to the verification results. The embodiments of the present application can improve the efficiency of data verification.

Description

Data verification method, data verification device, electronic equipment and storage medium

technical field

The present application relates to the technical field of artificial intelligence, and in particular, to a data verification method, a data verification device, an electronic device and a storage medium.

Background technique

At present, in the data verification, manual verification is often used, and this method often requires manual verification of each stage of data one by one. The workload is large and affects the efficiency of data verification. Therefore, how to improve data verification? efficiency has become an urgent technical problem to be solved.

SUMMARY OF THE INVENTION

The main purpose of the embodiments of the present application is to provide a data verification method, a data verification device, an electronic device, and a storage medium, so as to improve the efficiency of data verification.

In order to achieve the above purpose, a first aspect of the embodiments of the present application provides a data verification method, the method includes:

Obtain the original business data of the target stage;

Classify the original business data according to the preset label to obtain label business data;

Perform feature extraction on the label business data according to a preset extraction method to obtain key business features;

Perform prediction processing on the key features of the business through a preset algorithm to obtain business deviation data;

Perform status verification on the original business data according to the business deviation data to obtain a verification result;

According to the verification result, output a task scheduling instruction or perform data correction on the original service data.

In some embodiments, the step of classifying and processing the original service data according to the preset label to obtain the label service data includes:

obtaining attribute data of the target stage;

Screening the preset tags in the preset tag library according to the attribute data to obtain the target tag;

Perform label probability calculation on the original service data by using a preset function and the target label to obtain a label probability value;

Classify the original service data according to the tag probability value to obtain the tag service data.

In some embodiments, the target stage includes a first stage, the extraction method includes a first method, and the step of performing feature extraction on the tag service data according to a preset extraction method to obtain business key features includes:

If the label service data is the service data of the first stage, obtain the data category segment corresponding to the first method;

Perform keyword extraction on the tag service data according to the data category segment to obtain initial service keywords;

The initial business keywords are filtered to obtain the business key features.

In some embodiments, the target stage includes a second stage, the extraction method includes a second method, and the step of performing feature extraction on the tag service data according to a preset extraction method to obtain business key features includes:

If the tag service data is the service data of the second stage, the algorithm configuration table corresponding to the second mode is obtained according to the preset data dimension, and the algorithm configuration table stores preset tags and preset algorithms, so There is a mapping relationship between the preset label and the preset algorithm;

extracting the target tag corresponding to the tag service data from the preset tag;

According to the mapping relationship, extract the preset algorithm corresponding to the target tag from the algorithm configuration table;

Perform field extraction on the tag service data through the preset algorithm to obtain an initial key field;

Perform field value calculation on the initial key field to obtain the business key feature.

In some embodiments, the step of performing prediction processing on the business key features by using a preset algorithm to obtain business deviation data includes:

Perform regression prediction on the key features of the business through the least squares method to obtain initial prediction data;

The initial prediction data is calibrated according to a preset deviation parameter to obtain the service deviation data.

In some embodiments, the verification result includes a first result and a second result, and the step of performing status verification on the original business data according to the business deviation data to obtain the verification result includes:

If the business deviation data is within a preset deviation range, determine that the original business data is in a normal state, and output a first result;

If the service deviation data is outside the preset deviation range, it is determined that the original service data is in an abnormal state, and a second result is output.

In some embodiments, the verification result includes a first result and a second result, and the step of outputting a task scheduling instruction or performing data correction on the original service data according to the verification result includes:

If the check result is the first result, output a task scheduling instruction to check the business data according to the task scheduling instruction;

If the verification result is the second result, anomaly is located on the original service data, anomaly information is generated, and data correction is performed on the original service data according to the anomaly information.

In order to achieve the above object, a second aspect of the embodiments of the present application provides a data verification device, the device includes:

The data acquisition module is used to acquire the original business data of the target stage;

A classification module, configured to classify and process the original business data according to the preset label to obtain label business data;

a feature extraction module, configured to perform feature extraction on the label business data according to a preset extraction method to obtain key business features;

A prediction module, configured to perform prediction processing on the key business features through a preset algorithm to obtain business deviation data;

a verification module, configured to perform status verification on the original business data according to the business deviation data, and obtain a verification result;

A processing module, configured to output a task scheduling instruction or perform data correction on the original service data according to the verification result.

To achieve the above object, a third aspect of the embodiments of the present application provides an electronic device, the electronic device includes a memory, a processor, a program stored on the memory and executable on the processor, and a program for A data bus for connection and communication between the processor and the memory is implemented, and the method described in the first aspect above is implemented when the program is executed by the processor.

To achieve the above purpose, a fourth aspect of the embodiments of the present application provides a computer-readable storage medium, where the computer-readable storage medium stores one or more computer programs, and the one or more computer programs can be or executed by multiple processors to implement the steps of the data verification method of the first aspect

The data verification method, data verification device, electronic equipment and storage medium proposed in the present application obtain the original business data of the target stage by obtaining the original business data; Obtain business data of different target stages, so as to conduct data verification in different business stages, and improve the data accuracy of the whole process. Further, feature extraction is performed on the label business data according to a preset extraction method to obtain business key features; the business key features are predicted and processed by a preset algorithm to obtain business deviation data, which can take different measures for business data at different target stages. The extraction method is used to extract, so that the obtained key business features are more accurate. Compared with the manual verification method, the embodiment of the present application performs data prediction through a preset algorithm, which can effectively shorten the data processing time, and can reduce the calculation error of business deviation data, thereby improving the efficiency and verification accuracy of data verification. Finally, check the status of the original business data according to the business deviation data, and obtain the check result; output task scheduling instructions or perform data correction on the original business data according to the check result, and analyze whether the original business data is in a normal state or an abnormal state according to the business deviation data. , so as to take corresponding measures in a timely manner according to the verification results, thereby improving the timeliness of data processing.

Description of drawings

1 is a flowchart of a data verification method provided by an embodiment of the present application;

Fig. 2 is the flowchart of step S102 in Fig. 1;

Fig. 3 is the flow chart of step S103 in Fig. 1;

Fig. 4 is another flowchart of step S103 in Fig. 1;

Fig. 5 is the flow chart of step S104 in Fig. 1;

Fig. 6 is the flowchart of step S105 in Fig. 1;

Fig. 7 is the flow chart of step S106 in Fig. 1;

8 is a schematic structural diagram of a data verification device provided by an embodiment of the present application;

FIG. 9 is a schematic diagram of a hardware structure of an electronic device provided by an embodiment of the present application.

Detailed ways

In order to make the purpose, technical solutions and advantages of the present application more clearly understood, the present application will be described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application, but not to limit the present application.

It should be noted that although the functional modules are divided in the schematic diagram of the device, and the logical sequence is shown in the flowchart, in some cases, the modules may be divided differently from the device, or executed in the order in the flowchart. steps shown or described. The terms "first", "second" and the like in the description and claims and the above drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence.

Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the technical field to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of the present application, and are not intended to limit the present application.

First, some terms involved in this application are analyzed:

Artificial intelligence (AI): It is a new technical science that studies and develops theories, methods, technologies and application systems for simulating, extending and expanding human intelligence; artificial intelligence is a branch of computer science, artificial intelligence. Intelligence attempts to understand the essence of intelligence and produce a new intelligent machine that can respond in a similar way to human intelligence. Research in this field includes robotics, language recognition, image recognition, natural language processing, and expert systems. Artificial intelligence can simulate the information process of human consciousness and thinking. Artificial intelligence is also a theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.

Natural language processing (NLP): NLP uses computers to process, understand and use human languages (such as Chinese, English, etc.). NLP belongs to a branch of artificial intelligence and is an interdisciplinary subject between computer science and linguistics. It is called computational linguistics. Natural language processing includes syntax analysis, semantic analysis, text understanding, etc. Natural language processing is often used in technical fields such as machine translation, handwritten and printed character recognition, speech recognition and text-to-language conversion, information intent recognition, information extraction and filtering, text classification and clustering, public opinion analysis, and opinion mining. Deal with related data mining, machine learning, knowledge acquisition, knowledge engineering, artificial intelligence research, and linguistic research related to language computing, etc.

Information Extraction: A text processing technology that extracts specified types of entities, relationships, events and other factual information from natural language texts, and forms structured data output. Information extraction is a technique for extracting specific information from text data. Text data is composed of some specific units, such as sentences, paragraphs, chapters, and text information is composed of some small specific units, such as words, words, phrases, sentences, paragraphs, or a combination of these specific units. . Extracting noun phrases, person names, place names, etc. from text data is all text information extraction. Of course, the information extracted by text information extraction technology can be various types of information.

MP (Mass-Production): mass production.

The least squares method (generalized least squares): Also known as the least squares method, is a mathematical optimization technique. It finds the best functional match for the data by minimizing the sum of squared errors. The unknown data can be easily obtained by the least squares method, and the sum of squares of the errors between the obtained data and the actual data can be minimized. The least squares method can also be used for curve fitting. Other optimization problems can also be formulated by least squares methods that minimize energy or maximize entropy.

Data fitting: also known as curve fitting, commonly known as pull curve, is a way of substituting existing data into a mathematical formula through mathematical methods. For scientific and engineering problems, several discrete data can be obtained through methods such as sampling and experiments. According to these data, we often hope to obtain a continuous function (that is, a curve) or a denser discrete equation that is consistent with the known data. This process It's called fitting.

At present, when verifying data in insurance business scenarios, manual verification is often used, and this method often requires manual verification of data at each stage one by one. The workload is large and affects the efficiency of data verification. Therefore, how to improve the efficiency of data verification has become an urgent technical problem to be solved.

Based on this, the embodiments of the present application provide a data verification method, a data verification device, an electronic device, and a storage medium, so as to improve the efficiency of data verification.

The data verification method, data verification device, electronic device, and storage medium provided by the embodiments of the present application are specifically described by the following embodiments. First, the data verification method in the embodiments of the present application is described.

The embodiments of the present application may acquire and process related data based on artificial intelligence technology. Among them, artificial intelligence (AI) is a theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. .

The basic technologies of artificial intelligence generally include technologies such as sensors, special artificial intelligence chips, cloud computing, distributed storage, big data processing technology, operation/interaction systems, and mechatronics. Artificial intelligence software technology mainly includes computer vision technology, robotics technology, biometrics technology, speech processing technology, natural language processing technology, and machine learning/deep learning.

The data verification method provided by the embodiment of the present application relates to the technical field of artificial intelligence. The data verification method provided in the embodiment of the present application can be applied to the terminal, also can be applied to the server, and can also be software running in the terminal or the server. In some embodiments, the terminal may be a smart phone, a tablet computer, a notebook computer, a desktop computer, etc.; the server side may be configured as an independent physical server, or may be configured as a server cluster or distributed system composed of multiple physical servers, or A cloud that can be configured to provide basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms The server; the software can be an application that implements the data verification method, etc., but is not limited to the above forms.

The present application may be used in numerous general purpose or special purpose computer system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, including A distributed computing environment for any of the above systems or devices, and the like. The application may be described in the general context of computer-executable instructions, such as program modules, being executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform particular tasks or implement particular abstract data types. The application may also be practiced in distributed computing environments where tasks are performed by remote processing devices that are linked through a communications network. In a distributed computing environment, program modules may be located in computer-readable storage media, including storage devices, in both local and remote computers.

FIG. 1 is an optional flowchart of a data verification method provided by an embodiment of the present application. The method in FIG. 1 may include, but is not limited to, steps S101 to S106.

Step S101, obtaining the original business data of the target stage;

Step S102, classifying and processing the original service data according to the preset label to obtain label service data;

Step S103, perform feature extraction on the tag service data according to a preset extraction method to obtain key service features;

Step S104, predicting and processing key business features by a preset algorithm to obtain business deviation data;

Step S105, performing a status check on the original business data according to the business deviation data to obtain a check result;

Step S106, output a task scheduling instruction according to the verification result or perform data correction on the original service data.

Steps S101 to S106 shown in the embodiment of the present application are to obtain the original business data of the target stage; to classify and process the original business data according to the preset label, to obtain the labelled business data, and to obtain the business of different target stages according to business requirements. Data, so as to conduct data verification at different business stages and improve the data accuracy of the whole process. Further, feature extraction is performed on the label business data according to a preset extraction method to obtain business key features; the business key features are predicted and processed by a preset algorithm to obtain business deviation data, which can take different measures for business data at different target stages. The extraction method is used to extract, so that the obtained key business features are more accurate. Compared with the manual verification method, the embodiment of the present application performs data prediction through a preset algorithm, which can effectively shorten the data processing time, and can reduce the calculation error of business deviation data, thereby improving the efficiency and verification accuracy of data verification. Finally, check the status of the original business data according to the business deviation data, and obtain the check result; output task scheduling instructions or perform data correction on the original business data according to the check result, and analyze whether the original business data is in a normal state or an abnormal state according to the business deviation data. , so as to take corresponding measures in a timely manner according to the verification results, thereby improving the timeliness of data processing.

In step S101 of some embodiments, a web crawler can be written, and after the data source is set, the data can be crawled in a targeted manner, so as to obtain the original business data of the target stage. The original business data can also be obtained in other ways, but not limited to this. It should be noted that in the insurance business, the target stage mainly includes the policy summary stage, the calculation stage and the data compression stage, and each stage will generate corresponding business result data. For example, in the policy summary stage, the original business data includes multiple types. The insurance data includes the number of policies, premiums, etc.

Referring to FIG. 2, in some embodiments, step S102 may include but is not limited to including steps S201 to S204:

Step S201, acquiring attribute data of the target stage;

Step S202, performing screening processing on the preset tags in the preset tag library according to the attribute data to obtain the target tag;

Step S203, performing label probability calculation on the original business data by using a preset function and a target label to obtain a label probability value;

Step S204: Classify the original service data according to the tag probability value to obtain tag service data.

In step S201 of some embodiments, the target stage includes a policy summary stage, a calculation stage, and a data compression stage, and a large amount of business data will be generated in each stage. Different stages can be distinguished by different attribute data. For example, when the target stage is the policy summary stage, the attribute data includes policy types, etc.; when the target stage is the data compression stage, the attribute data includes product types and so on. Therefore, by writing a web crawler, setting up the data source, and crawling the data in a targeted manner, the attribute data of the target stage can be obtained. The attribute data can also be obtained in other ways, but not limited to this.

In step S202 of some embodiments, the preset tag library stores data tags of different target stages, ie preset tags, wherein the preset tags include insurance category tags, time tags, quantity tags or value tags, and so on. Through the attribute data of different stages, the preset tags that meet the needs of the current stage can be selected from multiple preset tags, and this type of preset tags can be used as the target tags. For example, in the policy summary stage, the attribute data includes the policy type, According to the policy type, the quantity label and the value label can be filtered out from the preset labels, so that the quantity of different types of policies can be obtained according to the quantity label, and the insurance amount of different types of policies can be obtained according to the numerical label.

In step S203 of some embodiments, the preset function is the softmax function, and the probability distribution of the original business data on the target label can be easily created through the softmax function, so as to obtain the label probability value of each target label, the label probability value It can be used to reflect the possibility that the original business data belongs to the target label.

In step S204 in some embodiments, since the tag probability value can be used to reflect the possibility that the original service data belongs to the target tag, the original service data is classified according to the tag probability value, and the original service data is divided into tag probability values. The category of the largest target label, so as to distinguish the original business data at different stages, and obtain different types of label business data at different stages. The data is divided according to the insurance category, and the label business data belonging to the policy summary stage is obtained.

Referring to FIG. 3 , in some embodiments, the target stage includes the first stage, the extraction method includes the first method, and step S103 may include but is not limited to including steps S301 to S303:

Step S301, if the tag service data is the service data of the first stage, obtain the data category segment corresponding to the first mode;

Step S302, performing keyword extraction on the tag service data according to the data category segment to obtain initial service keywords;

Step S303, filtering the initial business keywords to obtain business key features.

In step S301 of some embodiments, the first stage is the policy summary stage. If the tagged business data is the business data of the policy summary stage, the data category segment corresponding to the first method is obtained. The first method is generally for different types of The label segment corresponding to the data setting, that is, the data type segment, wherein the data type segment is specifically the number of policies, the amount of the policy, and so on.

In step S302 of some embodiments, the tag service data may be processed into several sentence nodes through the TF-IDF algorithm. Specifically, the frequency of occurrence of the data category segment of each sentence in the tag service data may be calculated through the TF-IDF algorithm. , obtain the term frequency (Term Frequency, TF) of the data category word segment, where TF=the number of occurrences of the data category word segment/the number of word segments in the tag service data; further, calculate the inverse file frequency (InverseDocument Frequency, IDF), where, IDF=log (the total number of word segments of the tag business data/(the number of word segments of the label business data including the data category word segment+1)), and finally, the word segment is calculated according to the word frequency and the reverse file frequency The comprehensive frequency value of , the comprehensive frequency value = word frequency * reverse file frequency. In each target text, the word segment with the largest comprehensive frequency value is selected as the initial business keyword.

In step S303 of some embodiments, other word segments within the preset character range of the initial business keyword are extracted, word segments with higher correlation with the initial business keyword are retained, and word segments with lower correlation are filtered The key features of the business are obtained. For example, if the initial business keyword is the policy amount, the word segment that is close to the policy amount field is extracted, and the word segment containing the value and the unit of measurement of the amount is reserved in the extracted word segment. , and get the key features of the business corresponding to the policy amount: 50,000 yuan.

Referring to FIG. 4, in some embodiments, the target stage includes the second stage, and the extraction method includes the second method. Step S103 may include, but is not limited to, including steps S401 to S405:

Step S401, if the tag service data is the service data of the second stage, obtain an algorithm configuration table corresponding to the second mode according to a preset data dimension, and the algorithm configuration table stores a preset tag and a preset algorithm, and the preset tag and the preset algorithm are stored. There is a mapping relationship between the preset algorithms;

Step S402, extracting the target label corresponding to the label service data from the preset label;

Step S403, according to the mapping relationship, extract the preset algorithm corresponding to the target tag from the algorithm configuration table;

Step S404, performing field extraction on the tag service data through a preset algorithm to obtain an initial key field;

In step S405, field value calculation is performed on the initial key fields to obtain business key features.

In step S401 of some embodiments, the second stage may refer to the calculation stage or the data compression stage, the second mode is generally an algorithm configuration mode, and the data dimension can be set according to actual business requirements. When in the calculation stage, the preset data The dimension is the data analysis at the policy level. When in the data compression stage, the preset data dimension is the data analysis at the product level. When the label service is actually the service data in the calculation stage or the service data in the data compression stage. The algorithm configuration table corresponding to the second mode is acquired, wherein the algorithm configuration table stores a preset label and a preset algorithm, and there is a mapping relationship between the preset label and the preset algorithm. It should be noted that this mapping relationship can be in the form of one-to-one or one-to-many.

In step S402 of some embodiments, since the target tag is one of the preset tags, the same tag can be easily extracted from the preset tags according to the tag name or tag value of the target tag.

In step S403 of some embodiments, after determining which tag in the preset tags is the target tag, the preset algorithm corresponding to the preset tag and the mapping relationship between the two can be directly extracted from the algorithm configuration table data. For example, the algorithm configuration table is traversed according to the target tag, and the preset algorithm of the target tag pair is extracted, as well as the mapping data between the target tag and the preset algorithm, wherein the mapping data includes the target tag, the preset algorithm, the target tag and the The relevance of the preset algorithm.

In step S404 in some embodiments, the preset algorithm may be a slice function, a start function, and a stop function to extract fields from the tag service data to obtain an initial key field, where the preset algorithm may be a TF-IDF algorithm Wait, no restrictions.

In step S405 of some embodiments, when calculating the field value of the initial key field, first extract the field value of the initial key field by using the tan function, and then sum up the field values of all the calculated initial key fields Or take the average to get the business critical characteristics.

In a specific embodiment, when the data verification is in the calculation stage, a preset algorithm configuration table is obtained, and the target label and the preset algorithm corresponding to the target label are configured in the algorithm configuration table. field and the calculation method to calculate the field. In the actual calculation of business key features, field extraction and field value calculation can be performed on the tag business data according to a corresponding preset algorithm, so as to obtain business key features.

Referring to FIG. 5, in some embodiments, step S104 may include but is not limited to including steps S501 to S502:

Step S501, perform regression prediction on the key features of the business through the least squares method to obtain initial prediction data;

Step S502, performing calibration processing on the initial prediction data according to preset deviation parameters to obtain service deviation data.

In step S501 of some embodiments, the least squares method can minimize the sum of squares of errors to find the best function matching of the data. Therefore, the least squares method can be used to perform regression prediction on key business features, generate a regression line, and obtain initial prediction data, The initial forecast data can be represented in the form of a curve.

In step S502 of some embodiments, the preset deviation parameter can be set according to actual business requirements without limitation. For example, if the deviation parameter is ±5%, the preset deviation parameter is used to determine the regression prediction range corresponding to the initial prediction data. A calibration process is performed to obtain business deviation data, where the business deviation data is a regression prediction value interval.

Referring to FIG. 6, in some embodiments, the verification result includes the first result and the second result, and step S105 includes but is not limited to including steps S601 to S602:

Step S601, if the business deviation data is within a preset deviation range, determine that the original business data is in a normal state, and output a first result;

Step S602, if the service deviation data is outside the preset deviation range, it is determined that the original service data is in an abnormal state, and a second result is output.

In step S601 of some embodiments, if the service deviation data is within the preset deviation range, that is, the deviation of the field information corresponding to the tag service data is within the preset deviation range, the original service data is normal, and the output The first result, wherein the specific semantic content of the first result includes: the current original service data is in a normal state, and the verification is passed. Therefore, the normal original service data can be easily represented by the first result.

In step S602 of some embodiments, if the service deviation data is outside the preset deviation range, that is, the deviation of the field information corresponding to the tag service data is outside the preset deviation range, the original service data is abnormal, and output The second result, where the specific semantic content of the second result includes: the current original business data is in an abnormal state, and the verification fails, and it is necessary to perform anomaly analysis on the current original business data, determine the cause of the anomaly, and perform data analysis on the current original business data. Corrective action. Therefore, the abnormal original service data can be easily represented by the second result.

Referring to FIG. 7, in some embodiments, the verification result includes a first result and a second result, and step S106 may include, but is not limited to, including steps S701 to S702:

Step S701, if the check result is the first result, output a task scheduling instruction to check the business data according to the task scheduling instruction;

Step S702, if the verification result is the second result, perform abnormal location on the original service data, generate abnormal information, and perform data correction on the original service data according to the abnormal information.

In step S701 of some embodiments, if the verification result is the first result, it means that the original service data of the current stage is normal, and the data verification of the next stage can be performed. Therefore, the task scheduling instruction is output and extracted according to the task scheduling instruction. For the original business data of the next stage, the original business data of the next stage is verified by the same verification process.

In step S702 of some embodiments, if the verification result is the second result, it means that the original business data at the current stage is abnormal, and the abnormal data needs to be processed. Therefore, according to the comparison between the original business data and the historical business data, it is determined that the The location of the original business data in the abnormal state, and the abnormal information is generated, and the abnormal information includes the abnormal location, the deviation, the abnormal cause, etc., so that the data correction is performed on the original business data according to the preset solution, wherein the preset The solution can be formulated according to the abnormal cause and solution measures determined by the abnormal problem in the historical investigation, wherein, the solution can be stored in the form of a configuration table, so that after determining the original business data in an abnormal state, traverse the configuration table to extract corresponding solution.

In a specific embodiment, the process of checking the insurance business data of a certain month includes three stages, that is, a policy summary stage, a DCS calculation stage, and a data compression stage.

First, the original business data in the policy summary stage is obtained, which is generally presented in the form of result table data, and the original business data is classified and counted according to the attribute data of the insurance type to obtain the label business data belonging to different insurance types. Further, key fields are extracted from the label business data according to the data type fields such as the number of policies and the policy amount to obtain key business features. Further, perform regression prediction on these key business features through the least squares method, construct a regression curve corresponding to each key business feature, and assign a deviation of ±5% to each key business feature to obtain the business deviation of each key business feature. Compare the business deviation data with the key business feature of the month in the historical records. If it exceeds the preset deviation range, it is determined that the original business data corresponding to the key business feature is abnormal. Perform anomaly analysis on the original business data. Specifically, traverse the preset problem configuration table according to the key business characteristics, and retrieve the abnormal cause and solution determined according to the abnormal problems in the historical investigation from the problem configuration table, so as to analyze the abnormal problems in the abnormal state. The business data is corrected and processed until all the original business data in the policy summary stage are in normal state, the task scheduling instruction is output, and the original business data of the next stage is extracted according to the task scheduling instruction.

Further, after the verification of the original business data in the policy summary stage is completed, the original business data in the DCS calculation stage is verified and processed according to the task scheduling instruction. Specifically, the label service data in the DCS calculation stage is obtained, and a preset algorithm configuration table is obtained, in which the target label and the preset algorithm corresponding to the target label are configured, that is, the label to be calculated can be specified through the algorithm configuration table. The service data and the calculation method for calculating the tag service data, extract the preset algorithm corresponding to each tag service data through the algorithm configuration table, and perform field extraction and field value calculation on the tag service data according to the corresponding preset algorithm, so as to obtain Business key characteristics, where, the business key characteristics. Furthermore, the least squares method is used to perform regression prediction on these key business features, and a regression curve corresponding to each key business feature is constructed, and a deviation of ±5% is assigned to each key business feature to obtain the business deviation data of each key business feature. , compare the business deviation data with the key business feature of the month in the historical records. If it exceeds the preset deviation range, determine that the original business data corresponding to the key business feature is in an abnormal state. Perform anomaly analysis on the business data, specifically, traverse the preset problem configuration table according to the key business characteristics, and retrieve the abnormal cause and solution determined according to the abnormal problem through historical investigation from the problem configuration table, so as to analyze the original business in an abnormal state. The data is corrected and processed until all the original business data in the policy summary stage are normal, the task scheduling instruction is output, and the original business data of the next stage is extracted according to the task scheduling instruction.

Further, after the verification of the original service data in the DSC calculation phase is completed, the verification processing is performed on the original service data in the data compression phase according to the task scheduling instruction. Specifically, the data verification in the data compression stage is mainly to verify the same type of data. First, group by operation is performed on the same type of tag service data according to the preset dimensions, so as to realize the aggregation of tag service data, and obtain the corresponding tag service data. Field value. Therefore, the data of these fields are summed or averaged to obtain the key business features of this stage. Similarly, the least squares method is used to check the abnormality of the key business features of this stage, so as to correct the original business data in an abnormal state. deal with.

It should be noted that, in the embodiment of the present application, if the original business data of the current stage is not checked, the original business data of the next stage will not be checked. When the original business data of the current stage is checked, the computer program Automatically verify the qualified data and enter the next stage of data verification. In this way, the original business data of each stage can be checked one by one, which can effectively save the time for business personnel to verify the result data and improve the efficiency of data verification. The stage can be automatically verified by the computer program. When the verification is qualified, it can automatically enter the data verification of the next stage, so as to realize the seamless connection of data verification in each stage. In addition, when the original business data in an abnormal state is found, the computer program can query the cause of the possible problem from the preset problem configuration table according to the combination of parameters and historical problems, so as to facilitate the business personnel to deal with abnormal problems. Quickly locate and improve the timeliness of problem solving.

The data verification method of the embodiment of the present application obtains the original business data of the target stage; classifies and processes the original business data according to the preset label to obtain the labelled business data, and can obtain the business data of different target stages according to business requirements, so that the Data verification is carried out at different business stages to improve the data accuracy of the whole process. Further, feature extraction is performed on the label business data according to a preset extraction method to obtain business key features; the business key features are predicted and processed by a preset algorithm to obtain business deviation data, which can take different measures for business data at different target stages. The extraction method is used to extract, so that the obtained key business features are more accurate. Compared with the manual verification method, the embodiment of the present application performs data prediction through a preset algorithm, which can effectively shorten the data processing time, and can reduce the calculation error of business deviation data, thereby improving the efficiency and verification accuracy of data verification. Finally, check the status of the original business data according to the business deviation data, and obtain the check result; output task scheduling instructions or perform data correction on the original business data according to the check result, and analyze whether the original business data is in a normal state or an abnormal state according to the business deviation data. , so as to take corresponding measures in a timely manner according to the verification results, thereby improving the timeliness of data processing.

Referring to FIG. 8 , an embodiment of the present application further provides a data verification apparatus, which can implement the above data verification method, and the apparatus includes:

A data acquisition module 801, configured to acquire original business data of the target stage;

A classification module 802, configured to classify and process the original service data according to the preset label to obtain the label service data;

A feature extraction module 803, configured to perform feature extraction on the tag service data according to a preset extraction method to obtain key service features;

A prediction module 804, configured to perform prediction processing on the key features of the business through a preset algorithm to obtain business deviation data;

A verification module 805, configured to perform status verification on the original business data according to the business deviation data, and obtain a verification result;

The processing module 806 is configured to output a task scheduling instruction according to the verification result or perform data correction on the original service data.

In some embodiments, classification module 802 includes:

The attribute data acquisition unit is used to acquire the attribute data of the target stage;

The screening unit is used for screening the preset tags in the preset tag library according to the attribute data to obtain the target tag;

a probability calculation unit, configured to perform tag probability calculation on the original business data through a preset function and a target tag to obtain a tag probability value;

The data classification unit is used for classifying and processing the original service data according to the tag probability value to obtain the tag service data.

In some embodiments, the target stage includes the first stage, the extraction method includes the first method, and the feature extraction module 803 includes:

a segment acquiring unit, configured to acquire the data category segment corresponding to the first mode if the tag service data is the first-stage service data;

A keyword extraction unit, configured to perform keyword extraction on the tag service data according to the data category segment to obtain initial service keywords;

The filtering unit is used for filtering the initial business keywords to obtain business key features.

In some embodiments, the target stage includes the second stage, the extraction method includes the second method, and the feature extraction module 803 further includes:

The configuration table obtaining unit is configured to obtain the algorithm configuration table corresponding to the second mode according to the preset data dimension if the tag service data is the service data of the second stage, and the algorithm configuration table stores the preset tag and the preset algorithm, There is a mapping relationship between preset tags and preset algorithms;

The label extraction unit is used to extract the target label corresponding to the label service data from the preset label;

The algorithm extraction unit is used for extracting the preset algorithm corresponding to the target tag from the algorithm configuration table according to the mapping relationship;

The field extraction unit is used to extract the fields of the tag service data through a preset algorithm to obtain the initial key fields;

The field value calculation unit is used to calculate the field value of the initial key field to obtain the business key feature.

In some embodiments, prediction module 804 includes:

The regression prediction unit is used to perform regression prediction on key business features through the least squares method to obtain initial prediction data;

The calibration unit is configured to perform calibration processing on the initial prediction data according to the preset deviation parameter to obtain service deviation data.

In some embodiments, the verification result includes the first result and the second result, and the verification module 805 includes:

a first judging unit, configured to determine that the original business data is in a normal state if the business deviation data is within a preset deviation range, and output a first result;

The second judgment unit is configured to determine that the original service data is in an abnormal state if the service deviation data is outside the preset deviation range, and output a second result.

In some embodiments, the verification result includes the first result and the second result, and the processing module 806 includes:

an output unit, configured to output a task scheduling instruction if the verification result is the first result, so as to verify the business data according to the task scheduling instruction;

The abnormality processing unit is configured to, if the verification result is the second result, locate the abnormality of the original service data, generate abnormality information, and perform data correction on the original service data according to the abnormality information.

The specific implementation of the data verification apparatus is basically the same as the specific embodiment of the data verification method described above, and will not be repeated here.

An embodiment of the present application also provides an electronic device, the electronic device includes: a memory, a processor, a program stored in the memory and running on the processor, and a data bus for realizing connection and communication between the processor and the memory , the above-mentioned data checking method is implemented when the program is executed by the processor. The electronic device may be any intelligent terminal including a tablet computer, a vehicle-mounted computer, and the like.

Please refer to FIG. 9. FIG. 9 illustrates a hardware structure of an electronic device according to another embodiment. The electronic device includes:

The processor 901 can be implemented by a general-purpose CPU (Central Processing Unit, central processing unit), a microprocessor, an application specific integrated circuit (Application Specific Integrated Circuit, ASIC), or one or more integrated circuits, etc., for executing relevant programs to achieve The technical solutions provided by the embodiments of the present application;

The memory 902 may be implemented in the form of a read-only memory (ReadOnlyMemory, ROM), a static storage device, a dynamic storage device, or a random access memory (Random Access Memory, RAM). The memory 902 can store the operating system and other application programs. When the technical solutions provided by the embodiments of this specification are implemented through software or firmware, the relevant computer program codes are stored in the memory 902 and invoked by the processor 901 to execute the application. The data verification method of the embodiment;

Input/output interface 903, used to realize information input and output;

The communication interface 904 is used to realize the communication interaction between the device and other devices, and the communication can be realized by wired means (such as USB, network cable, etc.), or by wireless means (such as mobile network, WIFI, Bluetooth, etc.);

a bus 905 that transfers information between the various components of the device (eg, processor 901, memory 902, input/output interface 903, and communication interface 904);

The processor 901 , the memory 902 , the input/output interface 903 and the communication interface 904 realize the communication connection between each other within the device through the bus 905 .

Embodiments of the present application also provide a computer-readable storage medium, where one or more computer programs are stored in the computer-readable storage medium, and the one or more computer programs can be executed by one or more processors to implement the above data verification method.

As a non-transitory computer-readable storage medium, the memory can be used to store non-transitory software programs and non-transitory computer-executable programs. Additionally, the memory may include high-speed random access memory, and may also include non-transitory memory, such as at least one magnetic disk storage device, flash memory device, or other non-transitory solid state storage device. In some embodiments, the memory may optionally include memory located remotely from the processor, which may be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

The data verification method, data verification device, electronic device, and storage medium provided by the embodiments of the present application obtain the original business data of the target stage; According to the requirements, the business data of different target stages can be obtained, so as to conduct data verification in different business stages and improve the data accuracy of the whole process. Further, feature extraction is performed on the label business data according to a preset extraction method to obtain business key features; the business key features are predicted and processed by a preset algorithm to obtain business deviation data, which can take different measures for business data at different target stages. The extraction method is used to extract, so that the obtained key business features are more accurate. Compared with the manual verification method, the embodiment of the present application performs data prediction through a preset algorithm, which can effectively shorten the data processing time, and can reduce the calculation error of business deviation data, thereby improving the efficiency and verification accuracy of data verification. Finally, check the status of the original business data according to the business deviation data, and obtain the check result; output task scheduling instructions or perform data correction on the original business data according to the check result, and analyze whether the original business data is in a normal state or an abnormal state according to the business deviation data. , so as to take corresponding measures in a timely manner according to the verification results, thereby improving the timeliness of data processing.

The embodiments described in the embodiments of the present application are for the purpose of illustrating the technical solutions of the embodiments of the present application more clearly, and do not constitute a limitation on the technical solutions provided by the embodiments of the present application. With the emergence of application scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

It can be understood by those skilled in the art that the technical solutions shown in FIGS. 1-7 do not constitute limitations to the embodiments of the present application, and may include more or less steps than those shown in the drawings, or combine certain steps, or different A step of.

The apparatus embodiments described above are only illustrative, and the units described as separate components may or may not be physically separated, that is, may be located in one place, or may be distributed to multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution in this embodiment.

Those of ordinary skill in the art can understand that all or some of the steps in the methods disclosed above, functional modules/units in the systems, and devices can be implemented as software, firmware, hardware, and appropriate combinations thereof.

The terms "first", "second", "third", "fourth", etc. (if any) in the description of the present application and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It is to be understood that data so used may be interchanged under appropriate circumstances so that the embodiments of the application described herein can be practiced in sequences other than those illustrated or described herein. Furthermore, the terms "comprising" and "having" and any variations thereof, are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those expressly listed Rather, those steps or units may include other steps or units not expressly listed or inherent to these processes, methods, products or devices.

It should be understood that, in this application, "at least one (item)" refers to one or more, and "a plurality" refers to two or more. "And/or" is used to describe the relationship between related objects, indicating that there can be three kinds of relationships, for example, "A and/or B" can mean: only A, only B, and both A and B exist , where A and B can be singular or plural. The character "/" generally indicates that the associated objects are an "or" relationship. "At least one item(s) below" or similar expressions thereof refer to any combination of these items, including any combination of single item(s) or plural items(s). For example, at least one (a) of a, b or c, can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c" ", where a, b, c can be single or multiple.

In the several embodiments provided in this application, it should be understood that the disclosed apparatus and method may be implemented in other manners. For example, the apparatus embodiments described above are only illustrative. For example, the division of the above units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components may be combined or may be Integration into another system, or some features can be ignored, or not implemented. On the other hand, the shown or discussed mutual coupling or direct coupling or communication connection may be through some interfaces, indirect coupling or communication connection of devices or units, and may be in electrical, mechanical or other forms.

The units described above as separate components may or may not be physically separated, and components shown as units may or may not be physical units, that is, may be located in one place, or may be distributed to multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution in this embodiment.

In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically alone, or two or more units may be integrated into one unit. The above-mentioned integrated units may be implemented in the form of hardware, or may be implemented in the form of software functional units.

The integrated unit, if implemented in the form of a software functional unit and sold or used as a stand-alone product, may be stored in a computer-readable computer-readable storage medium. Based on such understanding, the technical solutions of the present application can be embodied in the form of software products in essence, or the parts that contribute to the prior art, or all or part of the technical solutions, which are stored in a computer-readable The storage medium includes multiple instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods of the various embodiments of the present application. The aforementioned computer-readable storage medium includes: U disk, mobile hard disk, Read-Only Memory (ROM for short), Random Access Memory (RAM for short), magnetic disk or optical disk, etc. A medium on which a computer program is stored.

The preferred embodiments of the embodiments of the present application have been described above with reference to the accompanying drawings, which are not intended to limit the scope of rights of the embodiments of the present application. Any modifications, equivalent replacements and improvements made by those skilled in the art without departing from the scope and essence of the embodiments of the present application shall fall within the scope of the rights of the embodiments of the present application.

Claims (10)

1. a data verification method, is characterized in that, described method comprises: Obtain the original business data of the target stage; Classify the original business data according to the preset label to obtain label business data; Perform feature extraction on the label business data according to a preset extraction method to obtain key business features; Perform prediction processing on the key features of the business through a preset algorithm to obtain business deviation data; Perform status verification on the original business data according to the business deviation data to obtain a verification result; According to the verification result, output a task scheduling instruction or perform data correction on the original service data. 2. The data verification method according to claim 1, wherein the step of classifying and processing the original service data according to a preset label to obtain label service data comprises: obtaining attribute data of the target stage; Screening the preset tags in the preset tag library according to the attribute data to obtain the target tag; Perform label probability calculation on the original service data by using a preset function and the target label to obtain a label probability value; Classify the original service data according to the tag probability value to obtain the tag service data. 3 . The data verification method according to claim 1 , wherein the target stage includes a first stage, the extraction method includes a first method, and the tag service data is processed according to a preset extraction method. 4 . Feature extraction, the steps to obtain key business features, including: If the label service data is the service data of the first stage, obtain the data category segment corresponding to the first method; Perform keyword extraction on the tag service data according to the data category segment to obtain initial service keywords; The initial business keywords are filtered to obtain the business key features. 4 . The data verification method according to claim 1 , wherein the target stage includes a second stage, the extraction method includes a second method, and the tag service data is processed according to a preset extraction method. 5 . Feature extraction, the steps to obtain key business features, including: If the tag service data is the service data of the second stage, the algorithm configuration table corresponding to the second mode is obtained according to the preset data dimension, and the algorithm configuration table stores preset tags and preset algorithms, so There is a mapping relationship between the preset label and the preset algorithm; extracting the target tag corresponding to the tag service data from the preset tag; According to the mapping relationship, extract the preset algorithm corresponding to the target tag from the algorithm configuration table; Perform field extraction on the tag service data through the preset algorithm to obtain an initial key field; Perform field value calculation on the initial key field to obtain the business key feature. 5. The data verification method according to claim 1, wherein the step of performing prediction processing on the business key features by using a preset algorithm to obtain business deviation data comprises: Perform regression prediction on the key features of the business through the least squares method to obtain initial prediction data; The initial prediction data is calibrated according to a preset deviation parameter to obtain the service deviation data. 6 . The data verification method according to claim 1 , wherein the verification result includes a first result and a second result, and the status verification is performed on the original business data according to the business deviation data, and the verification is obtained. 7 . The resulting steps include: If the business deviation data is within a preset deviation range, determine that the original business data is in a normal state, and output a first result; If the service deviation data is outside the preset deviation range, it is determined that the original service data is in an abnormal state, and a second result is output. 7. The data verification method according to any one of claims 1 to 6, wherein the verification result includes a first result and a second result, and the task scheduling instruction is output according to the verification result or the The steps of data correction for original business data include: If the check result is the first result, output a task scheduling instruction to check the business data according to the task scheduling instruction; If the verification result is the second result, anomaly is located on the original service data, anomaly information is generated, and data correction is performed on the original service data according to the anomaly information. 8. A data verification device, wherein the device comprises: The data acquisition module is used to acquire the original business data of the target stage; A classification module, configured to classify and process the original business data according to the preset label to obtain label business data; a feature extraction module, configured to perform feature extraction on the label business data according to a preset extraction method to obtain key business features; A prediction module, configured to perform prediction processing on the key business features through a preset algorithm to obtain business deviation data; a verification module, configured to perform status verification on the original business data according to the business deviation data, and obtain a verification result; A processing module, configured to output a task scheduling instruction or perform data correction on the original service data according to the verification result. 9. An electronic device, characterized in that the electronic device comprises a memory, a processor, a program stored on the memory and executable on the processor, and a program for implementing the processor and the memory A data bus connecting and communicating between the programs, when the program is executed by the processor, implements the steps of the data checking method according to any one of claims 1 to 7. 10. A computer-readable storage medium, wherein the computer-readable storage medium stores one or more computer programs, and the one or more computer programs can be executed by one or more processors to achieve The steps of the data verification method of any one of claims 1 to 7.

Images