CN117762916A

CN117762916A - Data verification method, device, computer equipment and storage medium

Info

Publication number: CN117762916A
Application number: CN202410010948.9A
Authority: CN
Inventors: 李冠志
Original assignee: Ping An Property and Casualty Insurance Company of China Ltd
Current assignee: Ping An Property and Casualty Insurance Company of China Ltd
Priority date: 2024-01-03
Filing date: 2024-01-03
Publication date: 2024-03-26

Abstract

The application belongs to the field of big data and the field of financial science and technology, and relates to a data verification method, which comprises the following steps: acquiring service metadata in a service database; carrying out standardized processing on the business metadata to obtain target data; determining dimension information and measurement information corresponding to the target data; constructing a fact table based on the dimension information and the measurement information; acquiring source pasting service data corresponding to the service metadata from the data lake; acquiring data to be reported corresponding to service metadata from an application layer; and carrying out data verification processing on the source service data and the data to be reported based on the fact table, and generating a corresponding data verification result. The application also provides a data verification device, computer equipment and a storage medium. In addition, the application relates to a blockchain technology, and data verification results can be stored in the blockchain. The method and the device can be applied to business data comparison scenes in the financial field, improve the processing efficiency of data verification, and ensure the data accuracy of the generated data verification result.

Description

Data verification method, device, computer equipment and storage medium

Technical Field

The present disclosure relates to the field of big data technologies and financial technologies, and in particular, to a data verification method, a data verification device, a computer device, and a storage medium.

Background

In recent years, a data platform team newly adds a plurality of supervision report projects, so that the aim is to effectively identify financial risks and improve the important means of data quality of financial enterprises. At present, an Oracle is adopted in a business system database of an insurance company, financial insurance business is high in complexity, a plurality of management systems for managing a large number of claims, underwriting and the like exist, products deduced among different management systems are diversified, data enter a lake, a large number of tasks are not timely adapted and modified in the process of processing a large data processing link, data cannot be effectively and timely communicated from a source end to an application data reporting end to end, and how to effectively improve accuracy, integrity and consistency of reporting data and business data becomes an important subject to be urgently solved in supervision and report in supervision and management of data management. After Oracle business data enters a lake, the data is preprocessed by multiple layers of data, and special processing exists at the downstream, and the fields of the surface model are completely different, so that in order to meet the data consistency requirement of supervision, a supervision data management project needs to be established, and the consistency of the reported data and business system data double-verification comparison is ensured.

The verification mode between the data to be reported and the original business data adopted by the prior insurance company is to check the data quantity and the summarized data of the data to be reported and the original business data by a manual mode, and check the detailed data by manual check to obtain the corresponding data verification result.

Disclosure of Invention

An embodiment of the present application is directed to a data verification method, device, computer equipment and storage medium, so as to solve the technical problems that a verification method between data to be reported and original service data adopted by an existing insurance company needs to consume a lot of manpower, has low processing efficiency, and cannot guarantee the accuracy of a generated data verification result.

In order to solve the above technical problems, the embodiments of the present application provide a data verification method, which adopts the following technical schemes:

acquiring service metadata in a preset service database;

carrying out standardization processing on the service metadata to obtain corresponding target data;

determining dimension information and measurement information corresponding to the target data;

Constructing a corresponding fact table based on the dimension information and the measurement information;

acquiring source-attached service data corresponding to the service metadata from a preset data lake;

acquiring data to be reported corresponding to the service metadata from a preset application layer;

and carrying out data verification processing on the patch source service data and the data to be reported based on the fact table, and generating a data verification result between the patch source service data and the service data.

Further, the step of performing data verification processing on the patch source service data and the data to be reported based on the fact table to generate a data verification result between the patch source service data and the service data specifically includes:

acquiring data range information and granularity information of the fact table;

processing the source pasting service data based on the data range information and the granularity information to obtain processed source pasting service data;

processing the data to be reported based on the data range information and the granularity information to obtain processed data to be reported;

and calling a preset comparison code to perform data verification processing on the processed source service data and the processed data to be reported, and generating a data verification result between the source service data and the service data.

Further, the step of determining dimension information and metric information corresponding to the target data specifically includes:

acquiring service caliber information corresponding to the target data;

determining dimension information corresponding to the target data based on the service caliber information; the method comprises the steps of,

and determining measurement information corresponding to the target data based on the service caliber information.

Further, the step of performing standardization processing on the service metadata to obtain corresponding target data specifically includes:

acquiring a preset field standardization rule;

carrying out standardization processing on the service metadata based on the field standardization rule to obtain processed service metadata;

and taking the processed business metadata as the target data.

Further, after the step of performing data verification processing on the patch source service data and the data to be reported based on the fact table to generate a data verification result between the patch source service data and the service data, the method further includes:

judging whether the data verification result is verification passing or not;

if yes, determining a target reporting mechanism corresponding to the data to be reported;

Acquiring a delivery address of the target delivery mechanism;

and sending the report data to the target report mechanism based on the report address.

Further, after the step of performing data verification processing on the patch source service data and the data to be reported based on the fact table to generate a data verification result between the patch source service data and the service data:

judging whether the data verification result is that the verification fails;

if so, acquiring difference data between the patch source service data and the service data;

and storing the difference data.

Further, after the step of obtaining the difference data between the patch source service data and the service data, the method further includes:

acquiring communication information of preset service personnel;

generating corresponding difference repair information based on the difference data;

and sending the difference repairing information to the business personnel based on the communication information.

In order to solve the above technical problems, the embodiments of the present application further provide a data verification device, which adopts the following technical scheme:

the first acquisition module is used for acquiring service metadata in a preset service database;

The processing module is used for carrying out standardized processing on the business metadata to obtain corresponding target data;

the first determining module is used for determining dimension information and measurement information corresponding to the target data;

the construction module is used for constructing a corresponding fact table based on the dimension information and the measurement information;

the second acquisition module is used for acquiring the source-attached service data corresponding to the service metadata from a preset data lake;

the third acquisition module is used for acquiring data to be reported corresponding to the service metadata from a preset application layer;

and the verification module is used for carrying out data verification processing on the source service data and the data to be reported based on the fact table and generating a data verification result between the source service data and the service data.

In order to solve the above technical problems, the embodiments of the present application further provide a computer device, which adopts the following technical schemes:

acquiring service metadata in a preset service database;

In order to solve the above technical problems, embodiments of the present application further provide a computer readable storage medium, which adopts the following technical solutions:

acquiring service metadata in a preset service database;

Compared with the prior art, the embodiment of the application has the following main beneficial effects:

firstly, acquiring service metadata in a preset service database, and carrying out standardized processing on the service metadata to obtain corresponding target data; then determining dimension information and measurement information corresponding to the target data; then constructing a corresponding fact table based on the dimension information and the measurement information; subsequently, source pasting service data corresponding to the service metadata are obtained from a preset data lake; acquiring data to be reported corresponding to the service metadata from a preset application layer; and finally, carrying out data verification processing on the source service data and the data to be reported based on the fact table, and generating a data verification result between the source service data and the service data. According to the method and the device, the service metadata in the service database are subjected to standardized processing to obtain the corresponding target data, then the dimension information and the measurement information corresponding to the target data construct the fact table, based on the fact table, the automatic and rapid data verification processing of the source pasting service data corresponding to the service metadata obtained from the preset data lake and the data to be submitted corresponding to the service metadata obtained from the preset application layer is realized, so that the data verification result between the source pasting service data and the service data can be accurately generated, the processing efficiency and the processing standardization of the data verification between the source pasting service data and the service data are improved, and the data accuracy of the generated data verification result is ensured.

Drawings

For a clearer description of the solution in the present application, a brief description will be given below of the drawings that are needed in the description of the embodiments of the present application, it being obvious that the drawings in the following description are some embodiments of the present application, and that other drawings may be obtained from these drawings without inventive effort for a person of ordinary skill in the art.

FIG. 1 is an exemplary system architecture diagram in which the present application may be applied;

FIG. 2 is a flow chart of one embodiment of a data verification method according to the present application;

FIG. 3 is a schematic diagram of a structure of one embodiment of a data verification device according to the present application;

FIG. 4 is a schematic structural diagram of one embodiment of a computer device according to the present application.

Detailed Description

Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs; the terminology used in the description of the applications herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the application; the terms "comprising" and "having" and any variations thereof in the description and claims of the present application and in the description of the figures above are intended to cover non-exclusive inclusions. The terms first, second and the like in the description and in the claims or in the above-described figures, are used for distinguishing between different objects and not necessarily for describing a sequential or chronological order.

Reference herein to "an embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment may be included in at least one embodiment of the present application. The appearances of such phrases in various places in the specification are not necessarily all referring to the same embodiment, nor are separate or alternative embodiments mutually exclusive of other embodiments. Those of skill in the art will explicitly and implicitly appreciate that the embodiments described herein may be combined with other embodiments.

In order to better understand the technical solutions of the present application, the following description will clearly and completely describe the technical solutions in the embodiments of the present application with reference to the accompanying drawings.

As shown in fig. 1, a system architecture 100 may include terminal devices 101, 102, 103, a network 104, and a server 105. The network 104 is used as a medium to provide communication links between the terminal devices 101, 102, 103 and the server 105. The network 104 may include various connection types, such as wired, wireless communication links, or fiber optic cables, among others.

The user may interact with the server 105 via the network 104 using the terminal devices 101, 102, 103 to receive or send messages or the like. Various communication client applications, such as a web browser application, a shopping class application, a search class application, an instant messaging tool, a mailbox client, social platform software, etc., may be installed on the terminal devices 101, 102, 103.

The terminal devices 101, 102, 103 may be various electronic devices having a display screen and supporting web browsing, including but not limited to smartphones, tablet computers, electronic book readers, MP3 players (Moving Picture Experts Group Audio Layer III, dynamic video expert compression standard audio plane 3), MP4 (Moving Picture Experts Group Audio Layer IV, dynamic video expert compression standard audio plane 4) players, laptop and desktop computers, and the like.

The server 105 may be a server providing various services, such as a background server providing support for pages displayed on the terminal devices 101, 102, 103.

It should be noted that, the data verification method provided in the embodiments of the present application is generally executed by a server/terminal device, and accordingly, the data verification device is generally disposed in the server/terminal device.

It should be understood that the number of terminal devices, networks and servers in fig. 1 is merely illustrative. There may be any number of terminal devices, networks, and servers, as desired for implementation.

With continued reference to FIG. 2, a flow chart of one embodiment of a data verification method according to the present application is shown. The order of the steps in the flowchart may be changed and some steps may be omitted according to various needs. The data verification method provided by the embodiment of the application can be applied to any scene needing service data verification, and can be applied to products of the scenes, for example, data verification between source-attached service data and data to be reported in the field of financial insurance. The data verification method comprises the following steps:

Step S201, obtaining service metadata in a preset service database.

In this embodiment, the electronic device (for example, the server/terminal device shown in fig. 1) on which the data verification method operates may acquire the image to be checked through a wired connection manner or a wireless connection manner. It should be noted that the wireless connection may include, but is not limited to, 3G/4G/5G connection, wiFi connection, bluetooth connection, wiMAX connection, zigbee connection, UWB (ultra wideband) connection, and other now known or later developed wireless connection. The carrier of the service database may be Oracle. In the scenario of data reporting in the financial insurance industry, the service database may refer to a car insurance service system, a life insurance service system, a medical insurance service system, and so on. The service metadata is original source service data without any processing, and the category of the service metadata may include transaction data, payment data, policy data, and the like.

Step S202, the service metadata are standardized to obtain corresponding target data.

In this embodiment, the foregoing specific implementation process of normalizing the service metadata to obtain the corresponding target data will be described in further detail in the following specific embodiments, which are not described herein.

Step S203, determining dimension information and metric information corresponding to the target data.

In this embodiment, the specific implementation process of determining the dimension information and the metric information corresponding to the target data will be described in further detail in the following specific embodiments, which are not described herein.

Step S204, constructing a corresponding fact table based on the dimension information and the measurement information.

In this embodiment, a corresponding table model may be designed according to the dimension information and the metric information, and then a corresponding fact table may be constructed according to the table model.

Step S205, source pasting service data corresponding to the service metadata are obtained from a preset data lake.

In this embodiment, after the service metadata is stored in the service database, the service metadata is synchronized to the data lake, and the data to be submitted is obtained after processing the data layers such as the data warehouse, the data mart, the submitting application layer, and the like. The source service data is the synchronized service metadata stored in the data lake.

Step S206, obtaining the data to be reported corresponding to the business metadata from a preset application layer.

In this embodiment, the application layer refers to a reporting application layer, and the reporting application layer generates corresponding data to be reported after processing the synchronized service metadata.

Step S207, performing data verification processing on the source service data and the data to be reported based on the fact table, and generating a data verification result between the source service data and the service data.

In this embodiment, the fact table is used to perform data verification processing on the source service data and the data to be reported, so that verification on the source service data and the data to be reported can be implemented according to the same table model, dimension, measurement and granularity. The specific implementation process of performing the data verification process on the source service data and the data to be reported based on the fact table to generate the data verification result between the source service data and the service data will be described in further detail in the following specific embodiments, which will not be described herein.

Firstly, acquiring service metadata in a preset service database, and carrying out standardization processing on the service metadata to obtain corresponding target data; then determining dimension information and measurement information corresponding to the target data; then constructing a corresponding fact table based on the dimension information and the measurement information; subsequently, source pasting service data corresponding to the service metadata are obtained from a preset data lake; acquiring data to be reported corresponding to the service metadata from a preset application layer; and finally, carrying out data verification processing on the source service data and the data to be reported based on the fact table, and generating a data verification result between the source service data and the service data. According to the method and the device, the service metadata in the service database are subjected to standardized processing to obtain the corresponding target data, then the dimension information and the measurement information corresponding to the target data construct the fact table, based on the fact table, the automatic and rapid data verification processing of the source pasting service data corresponding to the service metadata obtained from the preset data lake and the data to be submitted corresponding to the service metadata obtained from the preset application layer is realized, so that the data verification result between the source pasting service data and the service data can be accurately generated, the processing efficiency and the processing standardization of the data verification between the source pasting service data and the service data are improved, and the data accuracy of the generated data verification result is ensured.

In some alternative implementations, step S207 includes the steps of:

and acquiring the data range information and the granularity information of the fact table.

In this embodiment, in the fact table constructed based on the dimension information and the metric information, the table field model of the fact table may be extracted to extract corresponding data range information and granularity information applicable to the table field of the fact table.

And processing the source pasting service data based on the data range information and the granularity information to obtain processed source pasting service data.

In this embodiment, the data range information and the granularity information of the source service data are normalized by processing the source service data according to the data range information and the granularity information, so that the processed source service data and the processed data to be reported can have the same data range information and granularity information.

And processing the data to be reported based on the data range information and the granularity information to obtain processed data to be reported.

In this embodiment, the data to be reported is processed according to the data range information and the granularity information, so that the data range information and the granularity information of the data to be reported are normalized, so that the processed data to be reported and the processed source service data can have the same data range information and granularity information.

In this embodiment, the comparison code is a pre-constructed code for automatically executing a data verification process between different data, where the data verification process includes data amount, statistics value, and attribute code value comparison. The comparison code can be called and operated, so that the data verification processing of the processed source service data and the processed data to be reported is automatically realized, and a data verification result between the source service data and the service data is generated. In addition, if the difference between the source service data and the service data is detected in the running process of the comparison code, the difference data between the source service data and the service data is synchronously recorded.

The data range information and the granularity information of the fact table are acquired; processing the source pasting service data based on the data range information and the granularity information to obtain processed source pasting service data; processing the data to be reported based on the data range information and the granularity information to obtain processed data to be reported; and subsequently calling a preset comparison code to perform data verification processing on the processed source service data and the processed data to be reported, and generating a data verification result between the source service data and the service data. According to the data verification method and device, the data range information and the granularity information of the fact table are respectively processed on the source pasting service data and the data to be submitted, the processed data to be submitted and the processed source pasting service data can have the same data range information and granularity information, and further the preset comparison code is called to automatically and rapidly carry out data verification processing on the processed source pasting service data and the processed data to be submitted, so that a data verification result between the source pasting service data and the service data can be accurately generated, the processing efficiency and the processing standardization of data verification between the source pasting service data and the service data are improved, and the data accuracy of the generated data verification result is guaranteed.

In some alternative implementations of the present embodiment, step S203 includes the steps of:

and acquiring service caliber information corresponding to the target data.

In this embodiment, service caliber information corresponding to target data may be queried by performing service caliber query on the target data. Where service aperture is a standard that ensures the validity and efficiency of service operations, typically specified and established by the relevant departments or teams.

And determining dimension information corresponding to the target data based on the service caliber information.

In this embodiment, after service caliber information corresponding to the target data is obtained, the fields in the target data may be dimension identified according to the service caliber information, so as to generate corresponding dimension information.

In this embodiment, after service caliber information corresponding to the target data is obtained, measurement and identification can be performed on fields in the target data according to the service caliber information, so as to generate corresponding measurement information.

The service caliber information corresponding to the target data is acquired; then determining dimension information corresponding to the target data based on the service caliber information; and determining metric information corresponding to the target data based on the service caliber information. According to the method and the device, the service caliber information corresponding to the target data is obtained, and then dimension identification and measurement identification processing can be carried out on the target data based on the service caliber information, so that the dimension information and the measurement information corresponding to the target data can be rapidly and accurately determined, the generation efficiency of the dimension information and the measurement information of the target data is improved, and the data accuracy of the generated dimension information and measurement information is ensured.

In some alternative implementations, step S202 includes the steps of:

and acquiring a preset field standardization rule.

In this embodiment, the above-mentioned field standardization rule is a rule of field naming and field definition corresponding to fields of different field types predefined according to actual service usage requirements.

And carrying out standardization processing on the service metadata based on the field standardization rule to obtain the processed service metadata.

In this embodiment, the service metadata may be normalized according to field normalization processing logic included in the field normalization rule, so as to obtain corresponding processed service metadata.

And taking the processed business metadata as the target data.

The method comprises the steps of obtaining a preset field standardization rule; then, carrying out standardization processing on the service metadata based on the field standardization rule to obtain processed service metadata; and taking the processed business metadata as the target data. According to the method and the device, the standardized processing of the business metadata is achieved based on the usage of the obtained field standardized rules, so that required target data can be generated rapidly, and the processing intelligence of the business metadata is improved.

In some alternative implementations, after step S207, the electronic device may further perform the following steps:

and judging whether the data verification result is verification passing or not.

In this embodiment, the content of the data verification result includes verification passing or verification failing. If the data verification result is that the verification is passed, the data consistency between the source service data and the data to be reported is met; and if the data verification result is that the verification is not passed, the data difference exists between the source service data and the data to be reported.

If yes, determining a target reporting mechanism corresponding to the data to be reported.

In this embodiment, the target reporting mechanism corresponding to the data to be reported may be determined by querying reporting information corresponding to the data to be reported.

And acquiring the reporting address of the target reporting mechanism.

In this embodiment, the mechanism identifier of the target delivery mechanism may be obtained, and then the delivery address of the target delivery mechanism may be queried according to the mechanism identifier. The address may refer to a communication address for data transmission with the target reporting mechanism.

Judging whether the data verification result is verification passing or not; if yes, determining a target reporting mechanism corresponding to the data to be reported; then acquiring the report address of the target report mechanism; and then, based on the report address, sending the report data to the target report mechanism. According to the method and the device, the report address of the target report mechanism corresponding to the data to be reported is obtained only when the data verification result between the patch source service data and the service data is detected to be the verification passing, and the report data is sent to the target report mechanism, so that the data quality of the data to be reported, which is sent to the target report mechanism, is effectively guaranteed, and the processing standardization and the processing intelligence of the data report are improved.

In some optional implementations of this embodiment, after step S207, the electronic device may further perform the following steps:

and judging whether the data verification result is that the verification fails.

In this embodiment, the content of the data verification result includes verification passing or verification failing.

And if so, acquiring difference data between the paste source service data and the service data.

In this embodiment, in a process of performing data verification processing on the source service data and the to-be-reported data by using a preset comparison code and the fact table, if a difference between the source service data and the service data is detected, difference data between the source service data and the service data is also recorded synchronously.

And storing the difference data.

In this embodiment, the storage mode of the difference data is not specifically limited, and for example, a blockchain storage mode, a local database storage mode, a cloud disk storage mode, and the like may be adopted.

Judging whether the data verification result is that verification fails; if so, acquiring difference data between the patch source service data and the service data; the difference data is subsequently stored. According to the method and the device, after the fact table is based on data verification processing is carried out on the source pasting service data and the data to be reported, after the data verification result between the source pasting service data and the service data is generated, if the obtained data verification result is detected to be that the verification is failed, the difference data between the source pasting service data and the service data are intelligently extracted and stored, so that the safety of the obtained difference data can be guaranteed, the follow-up repair processing of the data to be reported can be completed according to the difference data, and the repaired accurate data to be reported is reported so that the data quality of the data to be reported is effectively guaranteed.

In some optional implementations of this embodiment, after the step of obtaining the difference data between the source service data and the service data, the electronic device may further perform the following steps:

and acquiring communication information of preset service personnel.

In this embodiment, the service personnel may be operation maintenance personnel for the auxiliary data reporting work of the service system. The communication information may include a mail address or a telephone number.

And generating corresponding difference repair information based on the difference data.

In this embodiment, the difference data may be filled into a preset repair information template to generate corresponding difference repair information. The repair information template is an information template for reminding a relevant user of repairing data, the content of the repair information template is not particularly limited, and the repair information template can be set according to actual requirements.

The method comprises the steps of obtaining communication information of preset service personnel; then generating corresponding difference repair information based on the difference data; and then, based on the communication information, sending the difference repair information to the business personnel. When the data verification result between the source service data and the service data is that verification fails, the method and the device can also intelligently generate corresponding difference repair information based on the difference data, and send the difference repair information to the service personnel according to the communication information of the service personnel, so that the service personnel can be timely reminded of repairing the difference data by using the difference repair information, and the repaired accurate data to be reported can be reported later, so that the data quality of the data to be reported can be effectively guaranteed.

It should be understood that the sequence number of each step in the foregoing embodiment does not mean that the execution sequence of each process should be determined by the function and the internal logic, and should not limit the implementation process of the embodiment of the present invention.

It should be emphasized that, to further ensure the privacy and security of the data verification results, the data verification results may also be stored in a node of a blockchain.

The blockchain referred to in the application is a novel application mode of computer technologies such as distributed data storage, point-to-point transmission, consensus mechanism, encryption algorithm and the like. The Blockchain (Blockchain), which is essentially a decentralised database, is a string of data blocks that are generated by cryptographic means in association, each data block containing a batch of information of network transactions for verifying the validity of the information (anti-counterfeiting) and generating the next block. The blockchain may include a blockchain underlying platform, a platform product services layer, an application services layer, and the like.

The embodiment of the application can acquire and process the related data based on the artificial intelligence technology. Among these, artificial intelligence (Artificial Intelligence, AI) is the theory, method, technique and application system that uses a digital computer or a digital computer-controlled machine to simulate, extend and extend human intelligence, sense the environment, acquire knowledge and use knowledge to obtain optimal results.

Artificial intelligence infrastructure technologies generally include technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technologies, operation/interaction systems, mechatronics, and the like. The artificial intelligence software technology mainly comprises a computer vision technology, a robot technology, a biological recognition technology, a voice processing technology, a natural language processing technology, machine learning/deep learning and other directions.

Those skilled in the art will appreciate that implementing all or part of the above described methods may be accomplished by computer readable instructions stored in a computer readable storage medium that, when executed, may comprise the steps of the embodiments of the methods described above. The storage medium may be a nonvolatile storage medium such as a magnetic disk, an optical disk, a Read-Only Memory (ROM), or a random access Memory (Random Access Memory, RAM).

It should be understood that, although the steps in the flowcharts of the figures are shown in order as indicated by the arrows, these steps are not necessarily performed in order as indicated by the arrows. The steps are not strictly limited in order and may be performed in other orders, unless explicitly stated herein. Moreover, at least some of the steps in the flowcharts of the figures may include a plurality of sub-steps or stages that are not necessarily performed at the same time, but may be performed at different times, the order of their execution not necessarily being sequential, but may be performed in turn or alternately with other steps or at least a portion of the other steps or stages.

With further reference to fig. 3, as an implementation of the method shown in fig. 2, the present application provides an embodiment of a data verification apparatus, where an embodiment of the apparatus corresponds to the embodiment of the method shown in fig. 2, and the apparatus may be specifically applied to various electronic devices.

As shown in fig. 3, the data verification apparatus 300 according to this embodiment includes: a first acquisition module 301, a processing module 302, a first determination module 303, a construction module 304, a second acquisition module 305, a third acquisition module 306, and a verification module 307. Wherein:

a first obtaining module 301, configured to obtain service metadata in a preset service database;

the processing module 302 is configured to perform standardization processing on the service metadata to obtain corresponding target data;

a first determining module 303, configured to determine dimension information and metric information corresponding to the target data;

a construction module 304, configured to construct a fact table corresponding to the measurement information based on the dimension information;

a second obtaining module 305, configured to obtain patch source service data corresponding to the service metadata from a preset data lake;

a third obtaining module 306, configured to obtain data to be reported corresponding to the service metadata from a preset application layer;

And the verification module 307 is configured to perform data verification processing on the patch source service data and the data to be reported based on the fact table, and generate a data verification result between the patch source service data and the service data.

In this embodiment, the operations performed by the modules or units respectively correspond to the steps of the data verification method in the foregoing embodiment one by one, which is not described herein again.

In some alternative implementations of the present embodiment, the verification module 307 includes:

the first acquisition submodule is used for acquiring data range information and granularity information of the fact table;

the first processing sub-module is used for processing the source pasting service data based on the data range information and the granularity information to obtain processed source pasting service data;

the second processing sub-module is used for processing the data to be reported based on the data range information and the granularity information to obtain processed data to be reported;

and the verification sub-module is used for calling a preset comparison code to carry out data verification processing on the processed source service data and the processed data to be reported, and generating a data verification result between the source service data and the service data.

In some alternative implementations of the present embodiment, the first determining module 303 includes:

the second acquisition sub-module is used for acquiring service caliber information corresponding to the target data;

and the first determining submodule is used for determining dimension information corresponding to the target data based on the service caliber information.

And the second determining submodule is used for determining measurement information corresponding to the target data based on the service caliber information.

In some alternative implementations of the present embodiment, the processing module 302 includes:

the third acquisition sub-module is used for acquiring a preset field standardization rule;

the processing sub-module is used for carrying out standardized processing on the service metadata based on the field standardized rule to obtain the processed service metadata;

and the determining submodule is used for taking the processed business metadata as the target data.

In some optional implementations of this embodiment, the data verification apparatus further includes:

the first judging module is used for judging whether the data verification result is that the verification fails;

the second determining module is used for determining a target reporting mechanism corresponding to the data to be reported if yes;

a fourth acquisition module, configured to acquire a delivery address of the target delivery mechanism;

and the first sending module is used for sending the report data to the target report mechanism based on the report address.

the second judging module is used for judging whether the data verification result is verification passing or not;

a fifth obtaining module, configured to obtain difference data between the patch source service data and the service data if the patch source service data is positive;

And the storage module is used for storing the difference data.

a sixth acquisition module, configured to acquire communication information of a preset service person;

the generation module is used for generating corresponding difference repair information based on the difference data;

and the second sending module is used for sending the difference repairing information to the business personnel based on the communication information.

In order to solve the technical problems, the embodiment of the application also provides computer equipment. Referring specifically to fig. 4, fig. 4 is a basic structural block diagram of a computer device according to the present embodiment.

The computer device 4 comprises a memory 41, a processor 42, a network interface 43 communicatively connected to each other via a system bus. It should be noted that only computer device 4 having components 41-43 is shown in the figures, but it should be understood that not all of the illustrated components are required to be implemented and that more or fewer components may be implemented instead. It will be appreciated by those skilled in the art that the computer device herein is a device capable of automatically performing numerical calculations and/or information processing in accordance with predetermined or stored instructions, the hardware of which includes, but is not limited to, microprocessors, application specific integrated circuits (Application Specific Integrated Circuit, ASICs), programmable gate arrays (fields-Programmable Gate Array, FPGAs), digital processors (Digital Signal Processor, DSPs), embedded devices, etc.

The computer equipment can be a desktop computer, a notebook computer, a palm computer, a cloud server and other computing equipment. The computer equipment can perform man-machine interaction with a user through a keyboard, a mouse, a remote controller, a touch pad or voice control equipment and the like.

The memory 41 includes at least one type of readable storage medium including flash memory, hard disk, multimedia card, card memory (e.g., SD or DX memory, etc.), random Access Memory (RAM), static Random Access Memory (SRAM), read Only Memory (ROM), electrically Erasable Programmable Read Only Memory (EEPROM), programmable Read Only Memory (PROM), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the storage 41 may be an internal storage unit of the computer device 4, such as a hard disk or a memory of the computer device 4. In other embodiments, the memory 41 may also be an external storage device of the computer device 4, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash Card (Flash Card) or the like, which are provided on the computer device 4. Of course, the memory 41 may also comprise both an internal memory unit of the computer device 4 and an external memory device. In this embodiment, the memory 41 is typically used to store an operating system and various application software installed on the computer device 4, such as computer readable instructions of a data verification method. Further, the memory 41 may be used to temporarily store various types of data that have been output or are to be output.

The processor 42 may be a central processing unit (Central Processing Unit, CPU), controller, microcontroller, microprocessor, or other data processing chip in some embodiments. The processor 42 is typically used to control the overall operation of the computer device 4. In this embodiment, the processor 42 is configured to execute computer readable instructions stored in the memory 41 or process data, such as computer readable instructions for executing the data verification method.

The network interface 43 may comprise a wireless network interface or a wired network interface, which network interface 43 is typically used for establishing a communication connection between the computer device 4 and other electronic devices.

in the embodiment of the application, firstly, service metadata in a preset service database are acquired, and the service metadata are subjected to standardized processing to obtain corresponding target data; then determining dimension information and measurement information corresponding to the target data; then constructing a corresponding fact table based on the dimension information and the measurement information; subsequently, source pasting service data corresponding to the service metadata are obtained from a preset data lake; acquiring data to be reported corresponding to the service metadata from a preset application layer; and finally, carrying out data verification processing on the source service data and the data to be reported based on the fact table, and generating a data verification result between the source service data and the service data. According to the method and the device, the service metadata in the service database are subjected to standardized processing to obtain the corresponding target data, then the dimension information and the measurement information corresponding to the target data construct the fact table, based on the fact table, the automatic and rapid data verification processing of the source pasting service data corresponding to the service metadata obtained from the preset data lake and the data to be submitted corresponding to the service metadata obtained from the preset application layer is realized, so that the data verification result between the source pasting service data and the service data can be accurately generated, the processing efficiency and the processing standardization of the data verification between the source pasting service data and the service data are improved, and the data accuracy of the generated data verification result is ensured.

The present application also provides another embodiment, namely, a computer-readable storage medium storing computer-readable instructions executable by at least one processor to cause the at least one processor to perform the steps of the data verification method as described above.

From the above description of the embodiments, it will be clear to those skilled in the art that the above-described embodiment method may be implemented by means of software plus a necessary general hardware platform, but of course may also be implemented by means of hardware, but in many cases the former is a preferred embodiment. Based on such understanding, the technical solution of the present application may be embodied essentially or in a part contributing to the prior art in the form of a software product stored in a storage medium (such as ROM/RAM, magnetic disk, optical disk), comprising several instructions for causing a terminal device (which may be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to perform the method described in the embodiments of the present application.

It is apparent that the embodiments described above are only some embodiments of the present application, but not all embodiments, the preferred embodiments of the present application are given in the drawings, but not limiting the patent scope of the present application. This application may be embodied in many different forms, but rather, embodiments are provided in order to provide a more thorough understanding of the present disclosure. Although the present application has been described in detail with reference to the foregoing embodiments, it will be apparent to those skilled in the art that modifications may be made to the embodiments described in the foregoing, or equivalents may be substituted for elements thereof. All equivalent structures made by the specification and the drawings of the application are directly or indirectly applied to other related technical fields, and are also within the protection scope of the application.

Claims

1. A data verification method, comprising the steps of:

acquiring service metadata in a preset service database;

2. The data verification method according to claim 1, wherein the step of performing data verification processing on the patch source service data and the data to be reported based on the fact table to generate a data verification result between the patch source service data and the service data specifically includes:

acquiring data range information and granularity information of the fact table;

3. The data verification method according to claim 1, wherein the step of determining dimension information and metric information corresponding to the target data specifically includes:

acquiring service caliber information corresponding to the target data;

4. The data verification method according to claim 1, wherein the step of performing standardization processing on the service metadata to obtain corresponding target data specifically includes:

Acquiring a preset field standardization rule;

and taking the processed business metadata as the target data.

5. The data verification method according to claim 1, wherein after the step of performing data verification processing on the patch source service data and the data to be reported based on the fact table to generate a data verification result between the patch source service data and the service data, further comprises:

judging whether the data verification result is verification passing or not;

acquiring a delivery address of the target delivery mechanism;

6. The data verification method according to claim 1, wherein after the step of performing data verification processing on the patch source service data and the data to be reported based on the fact table to generate a data verification result between the patch source service data and the service data:

Judging whether the data verification result is that the verification fails;

and storing the difference data.

7. The data verification method according to claim 6, further comprising, after the step of acquiring the difference data between the patch source service data and the service data:

acquiring communication information of preset service personnel;

8. A data verification apparatus, comprising:

9. A computer device comprising a memory having stored therein computer readable instructions which when executed by a processor implement the steps of the data verification method of any one of claims 1 to 7.

10. A computer readable storage medium having stored thereon computer readable instructions which when executed by a processor implement the steps of the data verification method according to any of claims 1 to 7.