CN115344564A

CN115344564A - Data verification method and device, computer equipment and storage medium

Info

Publication number: CN115344564A
Application number: CN202210995257.XA
Authority: CN
Inventors: 吕端端
Original assignee: Ping An Technology Shenzhen Co Ltd
Current assignee: Ping An Technology Shenzhen Co Ltd
Priority date: 2022-08-18
Filing date: 2022-08-18
Publication date: 2022-11-15

Abstract

The application discloses a data verification method, a data verification device, computer equipment and a storage medium, and belongs to the technical field of artificial intelligence. The method comprises the steps of constructing training data by combining historical verification data and a historical data configuration table, training an initial neural network model by utilizing the training data to obtain a data configuration table generation model, receiving a data verification instruction, obtaining to-be-verified data corresponding to the data verification instruction, importing the to-be-verified data into the data configuration table generation model to obtain an initial data configuration table corresponding to the to-be-verified data, importing the to-be-verified data into the initial data configuration table to obtain a target data configuration table, verifying the target data configuration table by executing a preset data verification script, and outputting a data verification result of the to-be-verified data. In addition, the application also relates to a block chain technology, and the data to be checked can be stored in the block chain. According to the method and the device, data verification configuration can be carried out according to different user requirements, and meanwhile, the accuracy and timeliness of data verification are improved.

Description

Data verification method and device, computer equipment and storage medium

Technical Field

The application belongs to the technical field of artificial intelligence, and particularly relates to a data verification method and device, computer equipment and a storage medium.

Background

With the arrival of the big data era, the data volume is exponentially increased, data is the root of each enterprise unit, the enterprise units pay more attention to the protection and backup of the data, and data collection, data verification and data storage are further developed in the data processing technology level.

However, the accuracy verification of data is basically implemented by means of manual verification, and the following problems exist in the manual verification manner:

first, manual verification generally has a higher error rate, and even repeated work can cause various problems; secondly, the manual verification period is long, at present, aiming at the verification of mass data, the manual verification mode generally needs to be verified once in months, and the verification in a long period can reach half a year or even once a year, so that the verification in a long period obviously cannot meet the requirements of enterprise units; thirdly, manual verification is not comprehensive enough, verification is performed according to historical error places in many times, and potential problems are difficult to find; finally, manual verification is time-consuming, and for some scenes which often need verification, a large amount of manpower is arranged for long-time verification each time, so that the significance is not great and expected results cannot be achieved necessarily.

Disclosure of Invention

An embodiment of the application aims to provide a data verification method, a data verification device, computer equipment and a storage medium, so as to solve the technical problems of high error rate, long period, long time consumption and incomplete data verification existing in the existing manual verification data verification mode.

In order to solve the above technical problem, an embodiment of the present application provides a data verification method, which adopts the following technical solutions:

a method of data validation, comprising:

obtaining historical verification data and a historical data configuration table corresponding to the historical verification data;

combining the historical verification data and the historical data configuration table to construct training data;

importing training data into a preset initial neural network model, and training the initial neural network model by using the training data to obtain a data configuration table generation model;

receiving a data verification instruction, acquiring to-be-verified data corresponding to the data verification instruction, importing the to-be-verified data into a data configuration table generation model, and obtaining an initial data configuration table corresponding to the to-be-verified data;

importing the data to be checked into an initial data configuration table to obtain a target data configuration table;

and verifying the target data configuration table by executing a preset data verification script, and outputting a data verification result of the data to be verified.

Further, the preset initial neural network model is a convolutional neural network model, the initial neural network model comprises an embedding unit, a convolutional unit and a full-connection unit, training data are led into the preset initial neural network model, the training data are used for training the initial neural network model, and a data configuration table generation model is obtained, and the method specifically comprises the following steps:

importing training data into an initial neural network model, wherein historical verification data in the training data are input into an embedding unit of the initial neural network model, and a historical data configuration table corresponding to the historical verification data is input into a full-connection unit of the initial neural network model;

performing feature extraction and vector conversion processing on the historical verification data through an embedding unit of an initial neural network model to obtain a first initial vector;

convolving the first initial vector through a convolution unit of the initial neural network model to obtain first initial characteristic data;

similarity calculation is carried out on the first initial characteristic data through a full-connection unit of the initial neural network model, and a first similarity calculation result is obtained;

and iteratively updating the initial neural network model based on the first similarity calculation result until the model is fitted to obtain a data configuration table generation model.

Further, the similarity calculation is performed on the first initial feature data through a full-connection unit of the initial neural network model to obtain a first similarity calculation result, and the method specifically includes:

performing similarity calculation on the first initial characteristic data by using a pre-configured classifier in a full-connection unit of the initial neural network model to obtain first initial characteristic similarity;

and sequencing the obtained first initial feature similarities, and combining all the first initial feature similarities of which the similarity values are larger than a preset threshold value to obtain a first similarity calculation result.

Further, iteratively updating the initial neural network model based on the first similarity calculation result until the model is fitted to obtain a data configuration table generation model, and specifically including:

determining first initial characteristic data corresponding to the first similarity calculation result;

constructing an intermediate data configuration table based on the first initial feature data corresponding to the first similarity calculation result;

comparing the intermediate data configuration table with the historical data configuration table to obtain a prediction error;

and comparing the prediction error with a preset error threshold, and iteratively updating the initial neural network model according to the error comparison result until the model is fitted to obtain a data configuration table generation model.

Further, comparing the prediction error with a preset error threshold, and iteratively updating the initial neural network model according to the error comparison result until the model is fitted to obtain a data configuration table generation model, which specifically comprises:

transmitting a prediction error in the initial neural network model based on a preset back propagation algorithm;

obtaining error values of each network layer in the initial neural network model;

comparing the error value of each network layer in the initial neural network model with a preset error threshold value;

and if the error value of any network layer is larger than the preset error threshold value, iteratively updating the initial neural network model until the error values of all the network layers of the initial neural network model are smaller than or equal to the preset threshold value, and obtaining a data configuration table generation model.

Further, the data configuration table generation model includes an embedding unit, a convolution unit and a full connection unit, receives the data verification instruction, obtains the data to be verified corresponding to the data verification instruction, and introduces the data to be verified into the data configuration table generation model to obtain the initial data configuration table corresponding to the data to be verified, and specifically includes:

acquiring to-be-checked data corresponding to the data checking instruction, and importing the to-be-checked data into a data configuration table generation model;

performing feature extraction and vector conversion processing on the data to be checked through an embedding unit of the data configuration table generation model to obtain a second initial vector;

convolving the second initial vector through a convolution unit of a data configuration table generation model to obtain second initial characteristic data;

performing similarity calculation on the second initial characteristic data through a full-connection unit of the data configuration table generation model to obtain a second similarity calculation result;

determining second initial characteristic data corresponding to the second similarity calculation result;

and constructing a data configuration table based on second initial characteristic data corresponding to the second similarity calculation result to obtain an initial data configuration table.

Further, the target data configuration table is verified by executing a preset data verification script, and a data verification result of the data to be verified is output, which specifically includes:

acquiring a preset data verification script and a data verification requirement file of data to be verified;

executing the data verification script to analyze the data verification requirement file to obtain a data verification requirement;

and traversing the target data configuration table, and verifying the data in the target data configuration table according to the data verification requirement to obtain a data verification result of the data to be verified.

In order to solve the above technical problem, an embodiment of the present application further provides a data verification apparatus, which adopts the following technical solutions:

a data validation device, comprising:

the historical data acquisition module is used for acquiring historical verification data and a historical data configuration table corresponding to the historical verification data;

the training data construction module is used for combining the historical verification data and the historical data configuration table to construct training data;

the model iteration training module is used for importing training data into a preset initial neural network model, and training the initial neural network model by using the training data to obtain a data configuration table generation model;

the data configuration table generating module is used for receiving the data verification instruction, acquiring to-be-verified data corresponding to the data verification instruction, and importing the to-be-verified data into the data configuration table generating model to obtain an initial data configuration table corresponding to the to-be-verified data;

the system comprises a to-be-verified data importing module, a target data configuration table and a verification module, wherein the to-be-verified data importing module is used for importing the to-be-verified data into an initial data configuration table to obtain the target data configuration table;

and the data automatic verification module is used for verifying the target data configuration table by executing a preset data verification script and outputting a data verification result of the data to be verified.

In order to solve the above technical problem, an embodiment of the present application further provides a computer device, which adopts the following technical solutions:

a computer device comprising a memory having computer readable instructions stored therein and a processor which when executed implements the steps of a data validation method as claimed in any preceding claim.

In order to solve the above technical problem, an embodiment of the present application further provides a computer-readable storage medium, which adopts the following technical solutions:

a computer readable storage medium having computer readable instructions stored thereon which, when executed by a processor, implement the steps of a data validation method as claimed in any one of the preceding claims.

Compared with the prior art, the embodiment of the application mainly has the following beneficial effects:

the application discloses a data verification method, a data verification device, computer equipment and a storage medium, and belongs to the technical field of artificial intelligence. The method comprises the steps of combining historical verification data and a historical data configuration table to construct training data, training an initial neural network model by utilizing the training data to obtain a data configuration table generation model, receiving a data verification instruction, obtaining to-be-verified data corresponding to the data verification instruction, importing the to-be-verified data into the data configuration table generation model to obtain an initial data configuration table corresponding to the to-be-verified data, importing the to-be-verified data into the initial data configuration table to obtain a target data configuration table, verifying the target data configuration table by executing a preset data verification script, and outputting a data verification result of the to-be-verified data. According to the method and the device, a data configuration table generation model is trained, the data configuration table generation model is used for generating the data configuration table of the data to be checked, and finally the target data configuration table is checked through the data checking script, so that the data checking of different user requirements can be met, and meanwhile, the accuracy and the timeliness of the data checking are improved.

Drawings

In order to more clearly illustrate the solution of the present application, the drawings needed for describing the embodiments of the present application will be briefly described below, and it is obvious that the drawings in the following description are some embodiments of the present application, and that other drawings can be obtained by those skilled in the art without inventive effort.

FIG. 1 illustrates an exemplary system architecture diagram to which the present application may be applied;

FIG. 2 illustrates a flow diagram of one embodiment of a method of data validation according to the present application;

FIG. 3 illustrates a schematic structural diagram of one embodiment of a data validation device according to the present application;

FIG. 4 shows a schematic block diagram of one embodiment of a computer device according to the present application.

Detailed Description

Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs; the terminology used in the description of the application herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the application; the terms "including" and "having," and any variations thereof in the description and claims of this application and the description of the figures above, are intended to cover non-exclusive inclusions. The terms "first," "second," and the like in the description and claims of this application or in the above-described drawings are used for distinguishing between different objects and not for describing a particular order.

Reference herein to "an embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment can be included in at least one embodiment of the application. The appearances of the phrase in various places in the specification are not necessarily all referring to the same embodiment, nor are separate or alternative embodiments mutually exclusive of other embodiments. It is explicitly and implicitly understood by one skilled in the art that the embodiments described herein may be combined with other embodiments.

In order to make the technical solutions better understood by those skilled in the art, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings.

As shown in fig. 1, the system architecture 100 may include

terminal devices

101, 102, 103, a network 104, and a server 105. The network 104 serves as a medium for providing communication links between the

terminal devices

101, 102, 103 and the server 105. Network 104 may include various connection types, such as wired, wireless communication links, or fiber optic cables, among others.

The user may use the

terminal devices

101, 102, 103 to interact with the server 105 via the network 104 to receive or send messages or the like. The

terminal devices

101, 102, 103 may have various communication client applications installed thereon, such as a web browser application, a shopping application, a search application, an instant messaging tool, a mailbox client, social platform software, and the like.

The

terminal devices

101, 102, 103 may be various electronic devices having a display screen and supporting web browsing, including but not limited to smart phones, tablet computers, e-book readers, MP3 players (Moving Picture experts Group Audio Layer III, mpeg compression standard Audio Layer 3), MP4 players (Moving Picture experts Group Audio Layer IV, mpeg compression standard Audio Layer 4), laptop portable computers, desktop computers, and the like.

The server 105 may be a server that provides various services, for example, a background server that provides support for pages displayed on the

terminal devices

101, 102, and 103, and may be an independent server, or a cloud server that provides basic cloud computing services such as a cloud service, a cloud database, cloud computing, a cloud function, cloud storage, a web service, cloud communication, a middleware service, a domain name service, a security service, a Content Delivery Network (CDN), and a big data and artificial intelligence platform.

It should be noted that, the data verification method provided in the embodiments of the present application is generally executed by a server, and accordingly, the data verification apparatus is generally disposed in the server.

It should be understood that the number of terminal devices, networks, and servers in fig. 1 is merely illustrative. There may be any number of terminal devices, networks, and servers, as desired for implementation.

With continued reference to FIG. 2, a flow diagram of one embodiment of a data validation method according to the present application is shown. The embodiment of the application can acquire and process related data based on an artificial intelligence technology. Among them, artificial Intelligence (AI) is a theory, method, technique and application system that simulates, extends and expands human Intelligence using a digital computer or a machine controlled by a digital computer, senses the environment, acquires knowledge and uses the knowledge to obtain the best result.

The artificial intelligence infrastructure generally includes technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technologies, operation/interaction systems, mechatronics, and the like. The artificial intelligence software technology mainly comprises a computer vision technology, a robot technology, a biological recognition technology, a voice processing technology, a natural language processing technology, machine learning/deep learning and the like. The data verification method comprises the following steps:

s201, obtaining historical verification data and a historical data configuration table corresponding to the historical verification data.

Specifically, the server obtains historical verification data from a preset database and a historical data configuration table corresponding to the historical verification data. The historical verification data is data subjected to a data verification program, and the historical data configuration table corresponding to the historical verification data is a data configuration table used by the historical verification data in the data verification program. The data configuration table is configured according to different data verification requirements, and the data configuration table is used for realizing the rapid verification of various data.

S202, combining the historical verification data and the historical data configuration table to construct training data.

Specifically, the server acquires historical verification data from a preset database, and combines a historical data configuration table corresponding to the historical verification data to obtain a training data set for model training. In a specific embodiment of the present application, a combination of multiple sets of historical verification data and historical data configuration tables may be obtained to construct a training data set.

And S203, importing the training data into a preset initial neural network model, and training the initial neural network model by using the training data to obtain a data configuration table generation model.

The initial Neural network model may adopt a CNN Convolutional Neural network model, and a Convolutional Neural Network (CNN) is a kind of feed forward Neural network (fed Neural network) that includes convolution calculation and has a deep structure, and is one of the representative algorithms of deep learning (deep learning). Convolutional Neural Networks have a feature learning (rendering) capability, and can perform Shift-Invariant classification (Shift-Invariant classification) on input information according to a hierarchical structure thereof, and are therefore also called "Shift-Invariant Artificial Neural Networks (SIANN)". The convolutional neural network is constructed by imitating a visual perception (visual perception) mechanism of a living being, can perform supervised learning and unsupervised learning, and has the advantages that the convolutional neural network can learn grid-like topologic features such as pixels and audio with small calculation amount, has stable effect and has no additional feature engineering (feature engineering) requirement on data due to the fact that convolutional kernel parameter sharing in an implicit layer and sparsity of connection between layers.

Specifically, the server imports training data into a preset initial neural network model, trains the initial neural network model by using the training data, and obtains a data configuration table generation model. Wherein, the initial neural network model comprises an embedding unit, a convolution unit and a full connection unit,

specifically, the server performs feature extraction and vector conversion processing on the historical verification data through an embedding unit of an initial neural network model to obtain an initial vector of the historical verification data, performs convolution on the initial vector of the historical verification data through a convolution unit of the initial neural network model to obtain initial feature data of the historical verification data, performs similarity calculation on the initial feature data of the historical verification data through a full-connection unit of the initial neural network model to obtain a similarity calculation result of the historical verification data, performs iterative update on the initial neural network model based on the similarity calculation result of the historical verification data until the model is fitted to obtain a data configuration table generation model, and the data configuration table generation model can be used for generating a data configuration table of input data.

S204, receiving the data verification instruction, obtaining the data to be verified corresponding to the data verification instruction, importing the data to be verified into the data configuration table generation model, and obtaining an initial data configuration table corresponding to the data to be verified.

Specifically, the server receives the data verification instruction, obtains to-be-verified data corresponding to the data verification instruction, and imports the to-be-verified data into the data configuration table generation model, the data configuration table generation model processes the output to-be-verified data, the data configuration table generation model finally outputs an initial data configuration table corresponding to the to-be-verified data, and the initial data configuration table is used for realizing rapid verification of the to-be-verified data.

In this embodiment, the electronic device (for example, the server shown in fig. 1) on which the data verification method operates may receive the data verification instruction through a wired connection manner or a wireless connection manner. It is noted that the wireless connection means may include, but is not limited to, a 3G/4G connection, a WiFi connection, a bluetooth connection, a WiMAX connection, a Zigbee connection, a UWB (ultra wideband) connection, and other wireless connection means now known or developed in the future.

And S205, importing the data to be checked into the initial data configuration table to obtain a target data configuration table.

Specifically, the server analyzes the initial data configuration table, performs data division on the data to be checked according to the information of the initial data configuration table obtained through analysis to obtain a plurality of data segments, and sequentially fills the plurality of obtained data segments into the initial data configuration table to obtain a target data configuration table.

And S206, verifying the target data configuration table by executing a preset data verification script, and outputting a data verification result of the data to be verified.

Specifically, the server acquires a preset data verification script and a verification requirement file of the data to be verified, executes the data verification script to analyze the data verification requirement file to obtain a data verification requirement, traverses a target data configuration table corresponding to the data to be verified, and performs automatic verification on the data in the target data configuration table according to the data verification requirement to obtain a data verification result of the data to be verified.

In the embodiment of the application, after the data checking result is obtained, the data checking result is filled into the test result table, the test result table can display the verification result, a user can obtain the data test result by checking the test result table by himself or can display the test result in an online manner by creating an online report, the data accuracy verification result is displayed clearly and visually, and the reading requirement of the user is met.

In the embodiment, the training data is constructed by combining the historical verification data and the historical data configuration table, the initial neural network model is trained by using the training data to obtain a data configuration table generation model, the data verification instruction is received, the to-be-verified data corresponding to the data verification instruction is obtained, the to-be-verified data is imported into the data configuration table generation model to obtain an initial data configuration table corresponding to the to-be-verified data, the to-be-verified data is imported into the initial data configuration table to obtain a target data configuration table, the target data configuration table is verified by executing a preset data verification script, and the data verification result of the to-be-verified data is output. According to the method and the device, a data configuration table generation model is trained, the data configuration table generation model is used for generating the data configuration table of the data to be checked, and finally the target data configuration table is checked through the data checking script, so that the data checking of different user requirements can be met, and meanwhile, the accuracy and the timeliness of the data checking are improved.

Specifically, the preset initial neural network model is a convolutional neural network model, and the initial neural network model comprises an embedding unit, a convolutional unit and a full-connection unit. The server imports the training data into the initial neural network model, wherein the historical verification data in the training data is input into the embedding unit of the initial neural network model so as to carry out the characteristic processing of the data, and the historical data configuration table corresponding to the historical verification data is input into the full-connection unit of the initial neural network model for the model iteration.

The server conducts feature extraction and vector conversion processing on historical verification data through an embedding unit of an initial neural network model to obtain a first initial vector, conducts convolution on the first initial vector through a convolution unit of the initial neural network model to obtain first initial feature data, conducts similarity calculation on the first initial feature data through a full-connection unit of the initial neural network model to obtain a first similarity calculation result, conducts iteration updating on the initial neural network model based on the first similarity calculation result until the model is fitted to obtain a data configuration table generation model.

It should be noted that, the weights and biases of each network layer in the convolutional neural network model are all preset with an initial parameter, so that the convolutional neural network model can perform feature extraction and vector conversion processing on training data according to the initial parameter, where the weights and biases are model parameters used for performing refraction transformation calculation on input data in the network, so that the result output by the network through calculation can be consistent with the actual situation.

The convolution calculation process is to construct an x × n convolution kernel for an m × n matrix, taking 1-dimensional convolution as an example, and the convolution kernel operates on the original matrix in a sliding manner. For example, if m is 5, x is 1, the convolution kernel slides from top to bottom, x is multiplied by the n-dimensional vector of the first row and summed to obtain a value, and then x slides downwards continuously to obtain a matrix of 5 x 1, namely a convolution result, wherein the matrix of 2 nd row and 3 rd row is 8230and the convolution operation is carried out.

The full connection layer comprises a preset classifier, when the full connection layer receives the feature data, the preset classifier is used for calculating the similarity of the feature data, and the similarity calculation result is output.

and sequencing the obtained first initial feature similarities, and combining all the first initial feature similarities of which the similarity values are greater than a preset threshold value to obtain a first similarity calculation result.

Specifically, the server calculates similarity of first initial feature data by using a pre-configured classifier in a full-connection unit of the initial neural network model to obtain first initial feature similarity, sorts the obtained first initial feature similarity, and combines all the first initial feature similarities with similarity values larger than a preset threshold value to obtain a first similarity calculation result. The preset threshold may be set in advance, for example, the preset threshold is set to 0.5, and at this time, the server combines the first initial feature similarities whose similarity values are greater than 0.5 in the first similarity calculation result.

Specifically, after the server completes similarity calculation through the full-connection unit of the initial neural network model, all first initial feature data corresponding to a first similarity calculation result are determined, an intermediate data configuration table is built based on the first initial feature data corresponding to the first similarity calculation result, the intermediate data configuration table is compared with a historical data configuration table to obtain a prediction error, the prediction error is transmitted according to a back propagation algorithm, the prediction error is compared with a preset error threshold value, the initial neural network model is iteratively updated according to an error comparison result until the model is fitted, and a data configuration table generation model is obtained.

if the error value of any network layer is larger than the preset error threshold value, the initial neural network model is updated iteratively until the error values of all the network layers of the initial neural network model are smaller than or equal to the preset threshold value, and a data configuration table generation model is obtained.

The back propagation Algorithm (Backpropagation Algorithm) is suitable for a learning Algorithm of a multilayer neuron network, and is based on a gradient descent method. The input-output relationship of the back propagation algorithm network is essentially a mapping relationship: an n-input m-output BP neural network performs the function of continuous mapping from n-dimensional euclidean space to a finite field in m-dimensional euclidean space, which is highly non-linear.

Specifically, the server transmits a prediction error in the initial neural network model based on a preset back propagation algorithm, obtains error values of each network layer in the initial neural network model, compares the error values of each network layer in the initial neural network model with a preset error threshold, and if the error value of any network layer is larger than the preset error threshold, iteratively updates the initial neural network model until the error values of all network layers of the initial neural network model are smaller than or equal to the preset threshold, so as to obtain a data configuration table generation model.

performing feature extraction and vector conversion processing on data to be checked through an embedding unit of the data configuration table generation model to obtain a second initial vector;

Specifically, when data verification is performed, to-be-verified data corresponding to a data verification instruction is obtained, the to-be-verified data is imported into a data configuration table generation model, feature extraction and vector conversion processing are performed on the to-be-verified data through an embedding unit of the data configuration table generation model to obtain a second initial vector, the second initial vector is convolved through a convolution unit of the data configuration table generation model to obtain second initial feature data, similarity calculation is performed on the second initial feature data through a full connection unit of the data configuration table generation model to obtain a second similarity calculation result, a data configuration table is established based on the second initial feature data corresponding to the second similarity calculation result to determine second initial feature data corresponding to the second similarity calculation result, and an initial data configuration table is obtained.

Further, the target data configuration table is verified by executing the preset data verification script, and a data verification result of the data to be verified is output, which specifically includes:

acquiring a preset data verification script and a data verification demand file of data to be verified;

Specifically, the server acquires a preset data verification script and a data verification requirement file of the data to be verified, wherein the data verification requirement file of the data to be verified is a file which is uploaded by a user and records the data verification requirement of the data to be verified, then the data verification script is executed to analyze the data verification requirement file to obtain the data verification requirement in the data verification requirement file, finally, the target data configuration table is traversed, the data in the target data configuration table is verified according to the data verification requirement, and a data verification result of the data to be verified is obtained.

In the embodiment, the application discloses a data verification method, and belongs to the technical field of artificial intelligence. The method comprises the steps of combining historical verification data and a historical data configuration table to construct training data, training an initial neural network model by using the training data to obtain a data configuration table generation model, receiving a data verification instruction, obtaining to-be-verified data corresponding to the data verification instruction, importing the to-be-verified data into the data configuration table generation model to obtain an initial data configuration table corresponding to the to-be-verified data, importing the to-be-verified data into the initial data configuration table to obtain a target data configuration table, verifying the target data configuration table by executing a preset data verification script, and outputting a data verification result of the to-be-verified data. According to the method and the device, a data configuration table generation model is trained, the data configuration table generation model is used for generating the data configuration table of the data to be checked, and finally the target data configuration table is checked through the data checking script, so that the data checking of different user requirements can be met, and meanwhile, the accuracy and the timeliness of the data checking are improved.

It is emphasized that, in order to further ensure the privacy and security of the data to be verified, the data to be verified may also be stored in a node of a block chain.

The block chain referred by the application is a novel application mode of computer technologies such as distributed data storage, point-to-point transmission, a consensus mechanism, an encryption algorithm and the like. A block chain (Blockchain), which is essentially a decentralized database, is a string of data blocks associated by using a cryptographic method, and each data block contains information of a batch of network transactions, which is used for verifying the validity (anti-counterfeiting) of the information and generating a next block. The blockchain may include a blockchain underlying platform, a platform product service layer, an application service layer, and the like.

It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by hardware associated with computer readable instructions, which can be stored in a computer readable storage medium, and when executed, can include processes of the embodiments of the methods described above. The storage medium may be a non-volatile storage medium such as a magnetic disk, an optical disk, a Read-Only Memory (ROM), or a Random Access Memory (RAM).

It should be understood that, although the steps in the flowcharts of the figures are shown in order as indicated by the arrows, the steps are not necessarily performed in order as indicated by the arrows. The steps are not performed in the exact order shown and may be performed in other orders unless explicitly stated herein. Moreover, at least a portion of the steps in the flow chart of the figure may include multiple sub-steps or multiple stages, which are not necessarily performed at the same time, but may be performed at different times, which are not necessarily performed in sequence, but may be performed alternately or alternately with other steps or at least a portion of the sub-steps or stages of other steps.

With further reference to fig. 3, as an implementation of the method shown in fig. 2, the present application provides an embodiment of a data verification apparatus, which corresponds to the embodiment of the method shown in fig. 2, and which can be applied to various electronic devices.

As shown in fig. 3, the data verification apparatus according to this embodiment includes:

the historical data acquisition module 301 is configured to acquire historical verification data and a historical data configuration table corresponding to the historical verification data;

a training data construction module 302 for combining the historical verification data and the historical data configuration table to construct training data;

the model iteration training module 303 is configured to import training data into a preset initial neural network model, train the initial neural network model by using the training data, and obtain a data configuration table generation model;

the data configuration table generating module 304 is configured to receive the data verification instruction, obtain to-be-verified data corresponding to the data verification instruction, and import the to-be-verified data into the data configuration table generating model to obtain an initial data configuration table corresponding to the to-be-verified data;

a to-be-verified data importing module 305, configured to import the to-be-verified data into the initial data configuration table to obtain a target data configuration table;

and the data automatic verification module 306 is configured to verify the target data configuration table by executing a preset data verification script, and output a data verification result of the data to be verified.

Further, the preset initial neural network model is a convolutional neural network model, the initial neural network model includes an embedding unit, a convolutional unit and a full-connection unit, and the model iteration training module 303 specifically includes:

the training data import submodule is used for importing the training data into the initial neural network model, wherein historical verification data in the training data are input into the embedding unit of the initial neural network model, and a historical data configuration table corresponding to the historical verification data is input into the full-connection unit of the initial neural network model;

the first feature vector conversion submodule is used for performing feature extraction and vector conversion processing on the historical verification data through an embedding unit of the initial neural network model to obtain a first initial vector;

the first convolution operation submodule is used for performing convolution on the first initial vector through a convolution unit of the initial neural network model to obtain first initial characteristic data;

the first similarity operator module is used for carrying out similarity calculation on the first initial characteristic data through a full-connection unit of the initial neural network model to obtain a first similarity calculation result;

and the model iteration submodule is used for iteratively updating the initial neural network model based on the first similarity calculation result until the model is fitted to obtain a data configuration table generation model.

Further, the similarity operator module specifically includes:

the similarity calculation unit is used for calculating the similarity of the first initial characteristic data by using a pre-configured classifier in the full-connection unit of the initial neural network model to obtain the first initial characteristic similarity;

and the similarity sorting unit is used for sorting the obtained first initial feature similarities and combining all the first initial feature similarities with similarity values larger than a preset threshold value to obtain a first similarity calculation result.

Further, the model iteration submodule specifically includes:

the characteristic data determining unit is used for determining first initial characteristic data corresponding to the first similarity calculation result;

the configuration table construction unit is used for constructing an intermediate data configuration table based on the first initial characteristic data corresponding to the first similarity calculation result;

the configuration table comparison unit is used for comparing the intermediate data configuration table with the historical data configuration table to obtain a prediction error;

and the model iteration unit is used for comparing the prediction error with a preset error threshold value, and carrying out iteration updating on the initial neural network model according to the error comparison result until the model is fitted to obtain a data configuration table generation model.

Further, the model iteration unit specifically includes:

the error transmission subunit is used for transmitting the prediction error in the initial neural network model based on a preset back propagation algorithm;

the error value obtaining subunit is used for obtaining the error values of all network layers in the initial neural network model;

the error value comparison subunit is used for comparing the error value of each network layer in the initial neural network model with the preset error threshold value;

and the model iteration subunit is used for performing iteration updating on the initial neural network model when the error value of any network layer is larger than a preset error threshold value, until the error values of all the network layers of the initial neural network model are smaller than or equal to the preset threshold value, and obtaining a data configuration table generation model.

Further, the data configuration table generation model includes an embedding unit, a convolution unit, and a full connection unit, and the data configuration table generation module 304 specifically includes:

the to-be-verified data import submodule is used for acquiring to-be-verified data corresponding to the data verification instruction and importing the to-be-verified data into the data configuration table generation model;

the second feature vector conversion submodule is used for performing feature extraction and vector conversion processing on the data to be checked through the embedding unit of the data configuration table generation model to obtain a second initial vector;

the second convolution operation submodule is used for performing convolution on the second initial vector through a convolution unit of the data configuration table generation model to obtain second initial characteristic data;

the second similarity calculation submodule is used for calculating the similarity of the second initial characteristic data through the full-connection unit of the data configuration table generation model to obtain a second similarity calculation result;

the characteristic data determining submodule is used for determining second initial characteristic data corresponding to a second similarity calculation result;

and the configuration table constructing submodule is used for constructing a data configuration table based on second initial characteristic data corresponding to the second similarity calculation result to obtain an initial data configuration table.

Further, the data automatic verification module 306 specifically includes:

the script file acquisition submodule is used for acquiring a preset data verification script and a data verification demand file of data to be verified;

the requirement file analysis submodule is used for executing the data verification script to analyze the data verification requirement file to obtain a data verification requirement;

and the data automatic verification sub-module is used for traversing the target data configuration table, verifying the data in the target data configuration table according to the data verification requirement and obtaining the data verification result of the data to be verified.

In the above embodiment, the application discloses a data verification device, and belongs to the technical field of artificial intelligence. The method comprises the steps of combining historical verification data and a historical data configuration table to construct training data, training an initial neural network model by utilizing the training data to obtain a data configuration table generation model, receiving a data verification instruction, obtaining to-be-verified data corresponding to the data verification instruction, importing the to-be-verified data into the data configuration table generation model to obtain an initial data configuration table corresponding to the to-be-verified data, importing the to-be-verified data into the initial data configuration table to obtain a target data configuration table, verifying the target data configuration table by executing a preset data verification script, and outputting a data verification result of the to-be-verified data. According to the method and the device, a data configuration table generation model is trained, the data configuration table generation model is used for generating the data configuration table of the data to be checked, and finally the target data configuration table is checked through the data checking script, so that the data checking of different user requirements can be met, and meanwhile, the accuracy and the timeliness of the data checking are improved.

In order to solve the technical problem, the embodiment of the application further provides computer equipment. Referring to fig. 4, fig. 4 is a block diagram of a basic structure of a computer device according to the present embodiment.

The computer device 4 comprises a memory 41, a processor 42, a network interface 43 communicatively connected to each other via a system bus. It is noted that only computer device 4 having components 41-43 is shown, but it is understood that not all of the shown components are required to be implemented, and that more or fewer components may be implemented instead. As will be understood by those skilled in the art, the computer device is a device capable of automatically performing numerical calculation and/or information processing according to a preset or stored instruction, and the hardware includes, but is not limited to, a microprocessor, an Application Specific Integrated Circuit (ASIC), a Programmable Gate Array (FPGA), a Digital Signal Processor (DSP), an embedded device, and the like.

The computer device can be a desktop computer, a notebook, a palm computer, a cloud server and other computing devices. The computer equipment can carry out man-machine interaction with a user in a keyboard mode, a mouse mode, a remote controller mode, a touch panel mode or a voice control equipment mode.

The memory 41 includes at least one type of readable storage medium including a flash memory, a hard disk, a multimedia card, a card type memory (e.g., SD or DX memory, etc.), a Random Access Memory (RAM), a Static Random Access Memory (SRAM), a Read Only Memory (ROM), an Electrically Erasable Programmable Read Only Memory (EEPROM), a Programmable Read Only Memory (PROM), a magnetic memory, a magnetic disk, an optical disk, etc. In some embodiments, the memory 41 may be an internal storage unit of the computer device 4, such as a hard disk or a memory of the computer device 4. In other embodiments, the memory 41 may also be an external storage device of the computer device 4, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card), and the like, provided on the computer device 4. Of course, the memory 41 may also include both internal and external storage devices of the computer device 4. In this embodiment, the memory 41 is generally used for storing an operating system installed in the computer device 4 and various types of application software, such as computer readable instructions of a data verification method. Further, the memory 41 may also be used to temporarily store various types of data that have been output or are to be output.

The processor 42 may be a Central Processing Unit (CPU), controller, microcontroller, microprocessor, or other data Processing chip in some embodiments. The processor 42 is typically used to control the overall operation of the computer device 4. In this embodiment, the processor 42 is configured to execute computer readable instructions stored in the memory 41 or process data, such as executing computer readable instructions of the data verification method.

The network interface 43 may comprise a wireless network interface or a wired network interface, and the network interface 43 is generally used for establishing a communication connection between the computer device 4 and other electronic devices.

The application discloses computer equipment belongs to artificial intelligence technical field. The method comprises the steps of combining historical verification data and a historical data configuration table to construct training data, training an initial neural network model by utilizing the training data to obtain a data configuration table generation model, receiving a data verification instruction, obtaining to-be-verified data corresponding to the data verification instruction, importing the to-be-verified data into the data configuration table generation model to obtain an initial data configuration table corresponding to the to-be-verified data, importing the to-be-verified data into the initial data configuration table to obtain a target data configuration table, verifying the target data configuration table by executing a preset data verification script, and outputting a data verification result of the to-be-verified data. According to the method and the device, a data configuration table generation model is trained, the data configuration table generation model is used for generating the data configuration table of the data to be checked, and finally the target data configuration table is checked through the data checking script, so that the data checking of different user requirements can be met, and meanwhile, the accuracy and the timeliness of the data checking are improved.

The present application further provides another embodiment, which is to provide a computer-readable storage medium storing computer-readable instructions executable by at least one processor to cause the at least one processor to perform the steps of the data verification method as described above.

The application discloses a storage medium belongs to artificial intelligence technical field. The method comprises the steps of combining historical verification data and a historical data configuration table to construct training data, training an initial neural network model by utilizing the training data to obtain a data configuration table generation model, receiving a data verification instruction, obtaining to-be-verified data corresponding to the data verification instruction, importing the to-be-verified data into the data configuration table generation model to obtain an initial data configuration table corresponding to the to-be-verified data, importing the to-be-verified data into the initial data configuration table to obtain a target data configuration table, verifying the target data configuration table by executing a preset data verification script, and outputting a data verification result of the to-be-verified data. According to the method and the device, a data configuration table generation model is trained, the data configuration table generation model is used for generating the data configuration table of the data to be checked, and finally the target data configuration table is checked through the data checking script, so that the data checking of different user requirements can be met, and meanwhile, the accuracy and the timeliness of the data checking are improved.

Through the above description of the embodiments, those skilled in the art will clearly understand that the method of the above embodiments can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware, but in many cases, the former is a better implementation manner. Based on such understanding, the technical solutions of the present application may be embodied in the form of a software product, which is stored in a storage medium (such as ROM/RAM, magnetic disk, optical disk) and includes instructions for enabling a terminal device (such as a mobile phone, a computer, a server, an air conditioner, or a network device) to execute the method according to the embodiments of the present application.

The application is operational with numerous general purpose or special purpose computing system environments or configurations. For example: personal computers, server computers, hand-held or portable devices, tablet-type devices, multiprocessor systems, microprocessor-based systems, set top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments that include any of the above systems or devices, and the like. The application may be described in the general context of computer-executable instructions, such as program modules, being executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform particular tasks or implement particular abstract data types. The application may also be practiced in distributed computing environments where tasks are performed by remote processing devices that are linked through a communications network. In a distributed computing environment, program modules may be located in both local and remote computer storage media including memory storage devices.

It is to be understood that the above-described embodiments are merely illustrative of some, but not restrictive, of the broad invention, and that the appended drawings illustrate preferred embodiments of the invention and do not limit the scope of the invention. This application is capable of embodiments in many different forms and the embodiments are provided so that this disclosure will be thorough and complete. Although the present application has been described in detail with reference to the foregoing embodiments, it will be apparent to one skilled in the art that the present application may be practiced without modification or with equivalents of some of the features described in the foregoing embodiments. All equivalent structures made by using the contents of the specification and the drawings of the present application are directly or indirectly applied to other related technical fields and are within the protection scope of the present application.

Claims

1. A method of data verification, comprising:

importing the training data into a preset initial neural network model, and training the initial neural network model by using the training data to obtain a data configuration table generation model;

receiving a data verification instruction, acquiring to-be-verified data corresponding to the data verification instruction, and importing the to-be-verified data into the data configuration table generation model to obtain an initial data configuration table corresponding to the to-be-verified data;

importing the data to be checked into the initial data configuration table to obtain a target data configuration table;

2. The data verification method of claim 1, wherein the preset initial neural network model is a convolutional neural network model, the initial neural network model includes an embedding unit, a convolutional unit and a full-connection unit, the training data is imported into the preset initial neural network model, the training data is used for training the initial neural network model, and a data configuration table generation model is obtained, specifically including:

importing the training data into the initial neural network model, wherein historical verification data in the training data are input into an embedding unit of the initial neural network model, and a historical data configuration table corresponding to the historical verification data is input into a full-connection unit of the initial neural network model;

performing feature extraction and vector conversion processing on the historical verification data through an embedding unit of the initial neural network model to obtain a first initial vector;

and iteratively updating the initial neural network model based on the first similarity calculation result until the model is fitted to obtain the data configuration table generation model.

3. The data verification method of claim 2, wherein the performing similarity calculation on the first initial feature data through the fully connected unit of the initial neural network model to obtain a first similarity calculation result specifically comprises:

and sequencing the obtained first initial feature similarities, and combining all the first initial feature similarities with similarity values larger than a preset threshold value to obtain the first similarity calculation result.

4. The data verification method of claim 3, wherein the iteratively updating the initial neural network model based on the first similarity calculation result until model fitting is performed to obtain the data configuration table generation model specifically comprises:

determining first initial feature data corresponding to the first similarity calculation result;

constructing an intermediate data configuration table based on first initial feature data corresponding to the first similarity calculation result;

and comparing the prediction error with a preset error threshold, and iteratively updating the initial neural network model according to an error comparison result until the model is fitted to obtain the data configuration table generation model.

5. A data verification method as claimed in claim 4, wherein the comparing of the prediction error with a preset error threshold and the iterative updating of the initial neural network model according to the error comparison result until model fitting to obtain the data configuration table generation model specifically comprises:

transmitting the prediction error in the initial neural network model based on a preset back propagation algorithm;

comparing the error value of each network layer in the initial neural network model with the preset error threshold value;

and if the error value of any network layer is larger than the preset error threshold value, iteratively updating the initial neural network model until the error values of all the network layers of the initial neural network model are smaller than or equal to the preset threshold value, and obtaining the data configuration table generation model.

6. The data verification method according to any one of claims 1 to 5, wherein the data configuration table generation model includes an embedding unit, a convolution unit and a full connection unit, the receiving a data verification instruction, obtaining to-be-verified data corresponding to the data verification instruction, and importing the to-be-verified data into the data configuration table generation model to obtain an initial data configuration table corresponding to the to-be-verified data specifically includes:

acquiring to-be-checked data corresponding to the data checking instruction, and importing the to-be-checked data into the data configuration table generation model;

convolving the second initial vector through a convolution unit of the data configuration table generation model to obtain second initial characteristic data;

and constructing a data configuration table based on second initial characteristic data corresponding to the second similarity calculation result to obtain the initial data configuration table.

7. The data verification method of claim 6, wherein the verifying the target data configuration table by executing a preset data verification script and outputting a data verification result of the data to be verified specifically comprises:

acquiring a preset data verification script and a data verification demand file of the data to be verified;

8. A data verification apparatus, comprising:

the model iteration training module is used for importing the training data into a preset initial neural network model, and training the initial neural network model by using the training data to obtain a data configuration table generation model;

the data configuration table generation module is used for receiving a data verification instruction, acquiring to-be-verified data corresponding to the data verification instruction, and importing the to-be-verified data into the data configuration table generation model to obtain an initial data configuration table corresponding to the to-be-verified data;

the to-be-checked data import module is used for importing the to-be-checked data into the initial data configuration table to obtain a target data configuration table;

9. A computer device comprising a memory having computer readable instructions stored therein and a processor which when executed implements the steps of the data verification method of any one of claims 1 to 7.

10. A computer readable storage medium having computer readable instructions stored thereon which, when executed by a processor, carry out the steps of the data verification method of any one of claims 1 to 7.