CN115718728A

CN115718728A - Data checking method, device, equipment and medium of distributed system

Info

Publication number: CN115718728A
Application number: CN202211504784.2A
Authority: CN
Inventors: 王平殿
Original assignee: China Construction Bank Corp; CCB Finetech Co Ltd
Current assignee: China Construction Bank Corp; CCB Finetech Co Ltd
Priority date: 2022-11-28
Filing date: 2022-11-28
Publication date: 2023-02-28

Abstract

The method sets the same data structure for all record files generated under the same business operation so as to determine a file to be checked according to a file identifier of a target file in a data checking stage, and then determines a first field value of a checking field in the target file and a second field value of the checking field in all files to be processed according to data structure information of the target file. And generating a checking record meeting the checking condition according to the comparison result of the first field value and each second field value. The checking process does not need manual participation, and has the advantages of higher checking efficiency and avoidance of human errors.

Description

Data checking method, device, equipment and medium of distributed system

Technical Field

The present application relates to the field of data processing technologies, and in particular, to a method, an apparatus, a device, and a medium for data checking in a distributed system.

Background

A Distributed System (Distributed System) is a loosely coupled System in which a plurality of processors are interconnected via a communication line, and is widely used because it has many advantages such as resource sharing, high reliability, and quick communication, as compared with a single-machine System.

Different jobs, job steps in a distributed system may be executed on different node servers. The operation record file generated in the operation process is divided into a plurality of parts and stored in different storage spaces. In business processing, it is often necessary to check and analyze job record files, for example, when counting problem records generated in a certain job flow, it is necessary to search all job record files generated in the job flow, and check whether target fields representing the problem records are recorded in the job record files.

At present, data checking for a distributed system is mainly performed manually, and specifically, a checking person needs to find a corresponding checking file according to an index indicating a checking range, and perform manual checking on the checking file. The checking mode has the problems of long checking period and large error.

Disclosure of Invention

The embodiment of the application provides a data checking method, a data checking device, data checking equipment and a data checking medium of a distributed system, and is used for solving the problems of long checking period and large error in the current checking mode.

In order to achieve the above purpose, the technical solution of the embodiment of the present application is implemented as follows:

in a first aspect, an embodiment of the present application provides a data checking method for a distributed system, where the method includes:

receiving file information and checking information of the target file in response to the checking indication; the target file is a record file generated when any business operation is executed in batch; the file information comprises file identification and data structure information representing a target file data structure, the checking information comprises checking fields and checking conditions related to the checking fields, and the checking fields are at least one field in the data structure corresponding to the target file;

determining a storage space to be checked based on the file identification, and determining each file to be checked corresponding to the target file from the storage space; the file to be checked and the target file are recording files generated under the same service operation, and the data structures of the recording files generated under the same service operation are the same;

determining a first field value of the checking field in the target file and a second field value of the checking field in each file to be checked according to the data structure information; and generating a checking record meeting the checking condition according to the comparison result of the first field value and each second field value.

In some possible embodiments, the file identifier is a storage index of the target file in the storage space, and the file identifier includes a first identifier characterizing the target file, a second identifier characterizing a business job corresponding to the target file, and a third identifier characterizing a batch processing number of the business job.

In some possible embodiments, the distributed system comprises a plurality of storage partitions, each storage partition corresponding to a unique data platform; the data platform is used for storing generated record files in each storage partition corresponding to the data platform after batch processing of service execution operation;

the file information also comprises a check range; the determining the storage space to be checked based on the file identifier includes:

determining a storage index of the target file based on the file identification;

and determining the storage space to be checked according to the checking range and the storage index.

In some possible embodiments, the checking range includes a first checking range characterizing checking in units of data platforms, and a second checking range characterizing checking in units of storage partitions; the determining the storage space to be checked according to the checking range and the storage index includes:

if the checking range is the first checking range, determining a data platform for generating the target file according to the storage index, and taking all storage partitions under the data platform as the storage space;

and if the checking range is the second checking range, determining a target storage partition for storing the target file according to the storage index, and taking the target storage partition as the storage space.

In some possible embodiments, the check condition includes a review check and a range check; the generating a checking record meeting the checking condition according to the comparison result of the first field value and each second field value includes:

if the checking condition is a re-checking check, determining a first target field value from the second field values according to the comparison result; wherein the first target field value is the same as the first field value;

if the checking condition is range checking, determining a second target field value from the second field values according to the comparison result; wherein a second target field value is within a specified numerical range, the specified numerical range being determined from the first field value;

generating the check record based on the number of target field values; wherein the target field value is the first target field value or the second target field value.

In some possible embodiments, the data structure information is used to record fields included in a corresponding data structure of the target file, and an offset and a field length of each field.

In a second aspect, an embodiment of the present application provides a data checking apparatus for a distributed system, where the apparatus includes:

the information acquisition module is configured to receive file information and check information of the target file in response to the check instruction; the target file is a record file generated when any business operation is executed in batch; the file information comprises file identification and data structure information representing a target file data structure, the checking information comprises a checking field and a checking condition associated with the checking field, and the checking field is at least one field in the data structure corresponding to the target file;

the file acquisition module is configured to execute the determination of the storage space to be checked based on the file identification, and determine each file to be checked corresponding to the target file from the storage space; the file to be checked and the target file are recording files generated under the same service operation, and the data structures of the recording files generated under the same service operation are the same;

the checking and recording module is configured to determine a first field value of the checking field in the target file and a second field value of the checking field in each file to be checked according to the data structure information; and generating a checking record meeting the checking condition according to the comparison result of the first field value and each second field value.

the file information also comprises a check range; executing the determination of the storage space to be checked based on the file identifier, wherein the file acquisition module is configured to:

In some possible embodiments, the checking range includes a first checking range characterizing checking in units of data platforms, and a second checking range characterizing checking in units of storage partitions; executing the determining of the storage space to be checked according to the checking range and the storage index, wherein the file acquisition module is configured to:

In some possible embodiments, the check condition includes a review check and a range check; executing the comparison result between the first field value and each second field value to generate a check record meeting the check condition, wherein the check record module is configured to:

In a third aspect, an embodiment of the present application further provides an electronic device, which includes a memory and a processor, where the memory stores a computer program that is executable on the processor, and when the computer program is executed by the processor, the processor is caused to implement any one of the methods in the first aspect.

In a fourth aspect, this application further provides a computer-readable storage medium, in which a computer program is stored, and when the computer program is executed by a processor, the computer program implements any one of the methods in the first aspect.

In a fifth aspect, an embodiment of the present application is a computer program product, which includes computer instructions stored in a computer-readable storage medium; when the processor of the computer device reads the computer instructions from the computer-readable storage medium, the processor executes the computer instructions, causing the computer device to implement any of the methods of the first aspect.

In the embodiment of the application, the same data structure is set for each record file generated under the same service operation, so that after a file to be checked is determined according to a file identifier of a target file in a data checking stage, a first field value of a checking field in the target file and a second field value of the checking field in each file to be processed are determined according to data structure information of the target file. And generating a checking record meeting the checking condition according to the comparison result of the first field value and each second field value. The checking process does not need manual participation, and has the advantages of higher checking efficiency and avoidance of human errors.

Additional features and advantages of the present application will be set forth in the description which follows, and in part will be obvious from the description, or may be learned by the practice of the present disclosure. The objectives and other advantages of the application may be realized and attained by the structure particularly pointed out in the written description and claims hereof as well as the appended drawings.

Drawings

FIG. 1 is a schematic view of a manual checking process provided in an embodiment of the present application;

fig. 2 is an overall flowchart of a data checking method of a distributed system according to an embodiment of the present application;

fig. 3 is a schematic diagram of a storage structure in a distributed system according to an embodiment of the present application;

FIG. 4 is a schematic diagram of a formatted storage index path according to an embodiment of the present application;

FIG. 5 is a schematic diagram of review and range checking provided by embodiments of the present application;

fig. 6 is a block diagram of a data check 600 of a distributed system according to an embodiment of the present application;

fig. 7 is a structural diagram of an electronic device according to an embodiment of the present application.

Detailed Description

In order to make the objects, technical solutions and advantages of the present application more apparent, the technical solutions in the embodiments of the present application will be described clearly and completely with reference to the accompanying drawings in the embodiments of the present application, and it is obvious that the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments. All other embodiments obtained by a person of ordinary skill in the art based on the embodiments in the present application without making any creative effort belong to the protection scope of the present application. In the present application, the embodiments and features of the embodiments may be arbitrarily combined with each other without conflict. Also, while a logical order is shown in the flow diagrams, in some cases, the steps shown or described may be performed in an order different than here.

The terms "first" and "second" in the description and claims of the present application and the above-described drawings are used for distinguishing between different objects and not for describing a particular order. Furthermore, the term "comprises" and any variations thereof are intended to cover non-exclusive protection. For example, a process, method, system, article, or apparatus that comprises a list of steps or elements is not limited to only those steps or elements listed, but may alternatively include other steps or elements not listed, or inherent to such process, method, article, or apparatus. The "plurality" in the present application may mean at least two, for example, two, three or more, and the embodiments of the present application are not limited.

As mentioned above, at present, data checking for a distributed system is mainly performed manually, and a checking person needs to find a corresponding checking file according to an index indicating a checking range, and perform manual checking on the checking file. Specifically, as shown in fig. 1, the checking personnel needs to log in the database in advance, and search the file index according to the job batch number and the record file name to obtain the storage path of the checking file. And then calling the corresponding check file from the script directory according to the storage path. And finally, manually checking the content of each check file. Therefore, the processes from the query and the calling of the file to the checking need manual participation, the checking period is long, and human errors exist.

In order to solve the above problems, the inventive concept of the present application is: the method comprises the steps of setting the same data structure for all record files generated under the same business operation, determining a file to be checked according to a file identifier of a target file in a data checking stage, and then determining a first field value of a checking field in the target file and a second field value of the checking field in all files to be processed according to data structure information of the target file. And generating a checking record meeting the checking condition according to the comparison result of the first field value and each second field value. The checking process does not need manual participation, and has the advantages of higher checking efficiency and avoidance of human errors.

Next, as shown in fig. 2, fig. 2 shows an overall flow of a data checking method of a distributed system provided in the embodiment of the present application, which specifically includes:

step 201: receiving file information and checking information of a target file in response to a checking instruction; the target file is a record file generated when any business operation is executed in batch; the file information comprises file identification and data structure information representing a target file data structure, the checking information comprises a checking field and a checking condition associated with the checking field, and the checking field is at least one field in the data structure corresponding to the target file;

in order to facilitate understanding of the technical solution of the present application, a bank scenario is taken as an example for description, and it should be understood that the bank scenario is merely an introduction of the solution, and is not a limitation on an application scenario. The technical scheme is suitable for application scenes of any distributed system.

In a banking scenario, it is usually necessary to batch process a business, for example, when executing a "interest bearing" job, it is necessary to batch process "interest bearing" to a list of interest bearing lists. When any business job is executed in batch, a record file for recording the flow of the job is generated, and the record file generated by batch in the distributed system can be stored in different storage spaces.

In the embodiment of the application, the same data structure is set for the record files generated by the same service job, that is, the number of fields, the positions of the fields and the meanings of the fields contained in the files of each record file generated by any service job are the same. If the business needs to check the error reporting times in the certain branch balance business, a record file in any branch balance business can be selected as a target file, and the file to be checked is determined according to the file information and the checking information of the target file.

Specifically, the file information in this embodiment includes a file identifier and data structure information representing a data structure of the target file. The file identification is a storage index of the target file in the storage space, and the storage space for storing the target file in the distributed system can be found according to the file information. The file identifier may include a first identifier characterizing the target file, a second identifier characterizing the business job corresponding to the target file, and a third identifier characterizing the batch processing times of the business job.

The first identification is the file name of the target file, and the file name is unique in each record file generated by any business operation. The second identifier is the service job for generating the target file, and taking the above interest bearing job as an example, the second identifier here represents the interest bearing job. The third identifier represents the batch processing times of the service job, and the first to third identifiers indicate the storage index of the target file.

The data structure information in the embodiment of the present application is used to record fields included in the data structure corresponding to the target file, and offset and field length of each field. That is, the position of each field in the target file can be determined through the data structure information, and then the field value of each field in the target file is determined.

Step 202: determining a storage space to be checked based on the file identification, and determining each file to be checked corresponding to the target file from the storage space; the file to be checked and the target file are recording files generated under the same service operation, and the data structures of the recording files generated under the same service operation are the same;

the distributed system in the embodiment of the application comprises a plurality of storage partitions, and each storage partition corresponds to a unique data platform. And any data platform is used for storing the generated record file in each storage partition corresponding to the data platform after batch processing of the service operation. The bank scenario is described above, a server or a server cluster of each branch is erected in the distributed system, and a server or a server cluster corresponding to each branch corresponds to a plurality of storage partitions, that is, each storage partition corresponds to a unique branch.

The file information in the embodiment of the application further comprises a checking range, wherein the checking range comprises a first checking range for representing checking by taking the data platform as a unit and a second checking range for carrying out checking by taking the storage partition as a unit; and determining the storage space to be checked according to the target storage partition and the checking range.

When the step 202 is executed, the storage index of the target file needs to be determined based on the file identifier, and then the storage space to be checked is determined according to the checking range and the storage index.

As mentioned above, the record file generated by the same service job in the distributed system is divided into a plurality of records stored in different storage partitions, and the storage distribution of the banking scenario in the distributed system can be as shown in fig. 3, that is, any branch (corresponding to a unique server or a server cluster) includes a plurality of storage partitions, and each storage partition stores a plurality of record files. In implementation, the storage path of each record file can be stored by setting the database form, so as to determine the storage index of the record file according to the identification information of the record file.

The first checking range in the embodiment of the application is used for representing checking in a unit of a region where a bank branch is located, and corresponding to a distributed system, that is, checking in a unit of a server or a server cluster corresponding to the branch, and the second checking range is used for representing checking in a unit of a storage partition. Assuming that the Qinghai branch conducts error reporting statistics on the balance service A, any record file with error reporting records is selected from a plurality of batches of processing records of the balance service A and is used as a target file. And then determining the storage index of the target file according to the file identification of the target file. Assume that the target FILE belongs to a record FILE generated when the Qinghai branch holds the job, and its FILE name is FILE. If the job batch number of the generated target FILE is 0O1, the identification information of the target FILE is obtained, which is the first identification FILE, the second identification a, and the third identification 001. According to the FILE identification, the storage index Qh/Qh1/C _001/FILE _ A _001 of the target FILE in the unit of the result operation is obtained at this time. The first bit "Qh" in Qh/Qh1/C _001/FILE _ a _001 represents the Qinghai branch, the second bit "Qh1" represents the corresponding server, the third bit "C _001" represents the target storage partition storing the target FILE, and the last bit "FILE _ a _001" represents the FILE identifier of the target FILE.

Next, still in the example of performing error reporting statistics on the result service a by the Qinghai branch, it is assumed that the Qinghai province has a server cluster including the server Qh in the distributed system ₁ ～Qh _N If the checking range of the target file is the first checking range, the storage space to be checked is the server Qh ₁ All memory partitions below. Correspondingly, if the checking range is the second checking range, the storage space to be checked is the server Qh ₁ ～Qh _N In the lower storage partitions, the target storage partition of the target file is stored.

As shown in fig. 4 in particular, in the above example, the first checking range indicates that all the storage partitions under the corresponding servers in the Qinghai branch are checked. Thus, the storage index Qh/Qh of the target file is determined according to the identification information of the target file ₁ after/C _001/FILE _ a _001, the index is formatted, and the processed index is Qh/C/FILE.Wherein "Qh" represents all servers in the Qinghai branch, "C" represents all storage partitions under the server Qh, and "FILE" represents all record FILEs stored in the storage partition C. At this time, all the storage partitions C in the server cluster Qh of the Qinghai branch are called from the storage space, and the record files stored in all the storage partitions C are used as the to-be-checked files.

Correspondingly, if the second checking range is used for checking, after the storage index Qh/Qh1/C _001/FILE _ a _001 of the target FILE is determined according to the identification information of the target FILE, the storage index is formatted, and the processed storage index is Qh/Qh1/C _ 001/FILE. Namely, representing all record files stored in the target storage partition C _001 where the target file is located as the files to be checked.

Step 203: determining a first field value of the checking field in the target file and a second field value of the checking field in each file to be checked according to the data structure information; and generating a checking record meeting the checking condition according to the comparison result of the first field value and each second field value.

As mentioned above, the data structure information in the embodiment of the present application indicates the fields included in the corresponding data structure of the target file, and the offset and field length of each field. That is, the position of each field in the target file can be determined through the data structure information, and then the field value of each field in the target file is determined. In the embodiment of the present application, each record file generated by the same service job has the same data structure, so that the first field value of the search field in the target file and the second field value in each file to be checked can be determined according to the data structure information of the target file 1.

Therefore, the checking records meeting the checking conditions are generated according to the comparison result of the first field value and each second field value. The checking conditions in the embodiment of the present application include a review check and a range check. If the checking condition is a re-checking check, determining a first target field value from all second field values according to the comparison result; wherein the first target field value is the same as the first field value. If the checking condition is range checking, determining a second target field value from all second field values according to the comparison result; wherein the second target field value is within a specified numerical range, the specified numerical range being determined based on the first field value.

It is assumed that there are field a and field B in the data structure of the parsing service, where field a represents error reporting and the field value may be 1 or 0 (i.e. 1 is error reporting and 0 is not error reporting). The field B characterizes the specific reason for the error. When the error reporting statistics is performed for the result services, any record file with an error reporting may be used as a target file, and the error checking and checking may be performed for the field a or the field B to determine the statistics of the number of errors reported for the result services within the checking range.

Specifically, as shown in fig. 5, the review check process first obtains a first field value 1 of a field a in the target file, and then obtains a second field value in the field a in each target file. And counting the number of the second field values to be 1. And the range checking determines a checking range of a field value according to the first field value in the target file. In the example of the error number statistics shown in fig. 5, since the target file is a file with an error, the field B in the target file for characterizing the cause of the error is necessarily a non-empty field, and at this time, the field B may be used as a search field to check whether the field B in each file to be checked is a non-empty field, that is, whether the second field value of the field B is Null. If the second field value is not null, it indicates that the file to be checked has error reporting, and at this time, the number of error reporting recorded currently can be accumulated, so as to complete the error reporting number statistics for the balance service.

The search field in the embodiment of the present application is any field or a combination of multiple fields in the data structure corresponding to the target file. In the checking stage, the data structure information and the storage index of the target file can be automatically acquired only by manually selecting a search field from the form of the record data structure and inputting the file identification of the target file. And determining the file to be checked according to a preset checking range, and checking the file to be checked according to a preset checking condition. The checking process has the advantages of convenience in operation, high checking efficiency and avoidance of human errors.

Based on the same inventive concept, an embodiment of the present application provides a data checking apparatus 600 of a distributed system, specifically as shown in fig. 6, including:

an information obtaining module 601 configured to perform receiving file information and check information of a target file in response to a check instruction; the target file is a record file generated when any business operation is executed in batch; the file information comprises file identification and data structure information representing a target file data structure, the checking information comprises checking fields and checking conditions related to the checking fields, and the checking fields are at least one field in the data structure corresponding to the target file;

a file obtaining module 602 configured to execute determining a storage space to be checked based on the file identifier, and determine each file to be checked corresponding to the target file from the storage space; the file to be checked and the target file are recording files generated under the same service operation, and the data structures of the recording files generated under the same service operation are the same;

a checking record module 603 configured to perform determining, according to the data structure information, a first field value of the checking field in the target file and a second field value of the checking field in each file to be checked; and generating a checking record meeting the checking condition according to the comparison result of the first field value and each second field value.

the file information also comprises a check range; executing the determining of the storage space to be checked based on the file identifier, the file obtaining module 602 is configured to:

In some possible embodiments, the checking range includes a first checking range characterizing checking in units of data platforms, and a second checking range characterizing checking in units of storage partitions; executing the determining of the storage space to be checked according to the checking range and the storage index, where the file obtaining module 602 is configured to:

In some possible embodiments, the check condition includes a review check and a range check; executing the comparison result between the first field value and each second field value to generate the checking record meeting the checking condition, where the checking record module 603 is configured to:

The electronic device 130 according to this embodiment of the present application is described below with reference to fig. 7. The electronic device 130 shown in fig. 7 is only an example, and should not bring any limitation to the functions and the scope of use of the embodiments of the present application.

As shown in fig. 7, the electronic device 130 is represented in the form of a general electronic device. The components of the electronic device 130 may include, but are not limited to: the at least one processor 131, the at least one memory 132, and a bus 133 that connects the various system components (including the memory 132 and the processor 131).

Bus 133 represents one or more of any of several types of bus structures, including a memory bus or memory controller, a peripheral bus, a processor, or a local bus using any of a variety of bus architectures.

The memory 132 may include readable media in the form of volatile memory, such as Random Access Memory (RAM) 1321 and/or cache memory 1322, and may further include Read Only Memory (ROM) 1323.

Memory 132 may also include a program/utility 1325 having a set (at least one) of program modules 1324, such program modules 1324 including, but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which, or some combination thereof, may comprise an implementation of a network environment.

The electronic device 130 may also communicate with one or more external devices 134 (e.g., keyboard, pointing device, etc.), with one or more devices that enable a user to interact with the electronic device 130, and/or with any devices (e.g., router, modem, etc.) that enable the electronic device 130 to communicate with one or more other electronic devices. Such communication may occur via input/output (I/O) interfaces 135. Also, the electronic device 130 may communicate with one or more networks (e.g., a Local Area Network (LAN), a Wide Area Network (WAN), and/or a public network, such as the internet) via the network adapter 136. As shown, network adapter 136 communicates with other modules for electronic device 130 over bus 133. It should be understood that although not shown in the figures, other hardware and/or software modules may be used in conjunction with electronic device 130, including but not limited to: microcode, device drivers, redundant processors, external disk drive arrays, RAID systems, tape drives, and data backup storage systems, among others.

In an exemplary embodiment, a computer-readable storage medium comprising instructions, such as the memory 132 comprising instructions, executable by the processor 131 of the apparatus to perform the method described above is also provided. Alternatively, the computer readable storage medium may be a ROM, a Random Access Memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, and the like.

In an exemplary embodiment, there is also provided a computer program product comprising computer programs/instructions which, when executed by the processor 131, implement any of the data checking methods of a distributed system as provided herein.

In an exemplary embodiment, the various aspects of the data checking method of the distributed system provided in the present application may also be implemented in the form of a program product, which includes program code for causing a computer device to perform the steps in the data checking method of the distributed system according to various exemplary embodiments of the present application described above in this specification when the program product runs on the computer device.

The program product may employ any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples (a non-exhaustive list) of the readable storage medium include: an electrical connection having one or more wires, a portable disk, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

The program product for data checking of a distributed system of embodiments of the present application may employ a portable compact disc read only memory (CD-ROM) and include program code, and may be run on an electronic device. However, the program product of the present application is not limited thereto, and in this document, a readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.

A readable signal medium may include a propagated data signal with readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take any of a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A readable signal medium may be any readable medium that is not a readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device.

Program code embodied on a readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

Program code for carrying out operations of the present application may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, C + + or the like and conventional procedural programming languages, such as the PowerPC programming language or similar programming languages. The program code may execute entirely on the consumer electronic device, partly on the consumer electronic device, as a stand-alone software package, partly on the consumer electronic device and partly on a remote electronic device, or entirely on the remote electronic device or server. In the case of remote electronic devices, the remote electronic devices may be connected to the consumer electronic device through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external electronic device (e.g., through the internet using an internet service provider).

It should be noted that although several units or sub-units of the apparatus are mentioned in the above detailed description, such division is merely exemplary and not mandatory. Indeed, the features and functions of two or more units described above may be embodied in one unit, according to embodiments of the application. Conversely, the features and functions of one unit described above may be further divided into embodiments by a plurality of units.

Further, while the operations of the methods of the present application are depicted in the drawings in a particular order, this does not require or imply that these operations must be performed in this particular order, or that all of the illustrated operations must be performed, to achieve desirable results. Additionally or alternatively, certain steps may be omitted, multiple steps combined into one step execution, and/or one step broken down into multiple step executions.

As will be appreciated by one skilled in the art, embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and so forth) having computer-usable program code embodied therein.

The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the application. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable image scaling apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable image scaling apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable image scaling apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be loaded onto a computer or other programmable image scaling device to cause a series of operational steps to be performed on the computer or other programmable device to produce a computer implemented process such that the instructions which execute on the computer or other programmable device provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

While the preferred embodiments of the present application have been described, additional variations and modifications in those embodiments may occur to those skilled in the art once they learn of the basic inventive concepts. Therefore, it is intended that the appended claims be interpreted as including preferred embodiments and all alterations and modifications as fall within the scope of the application.

It will be apparent to those skilled in the art that various changes and modifications may be made in the present application without departing from the scope of the application. Thus, if such modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is intended to include such modifications and variations as well.

Claims

1. A method for data verification in a distributed system, the method comprising:

receiving file information and checking information of a target file in response to a checking instruction; the target file is a record file generated when any business operation is executed in batch; the file information comprises file identification and data structure information representing a target file data structure, the checking information comprises a checking field and a checking condition associated with the checking field, and the checking field is at least one field in the data structure corresponding to the target file;

2. The method according to claim 1, wherein the file identifier is a storage index of the target file in a storage space, and the file identifier includes a first identifier representing the target file, a second identifier representing a corresponding business job of the target file, and a third identifier representing a batch processing number of the business job.

3. The method of claim 2, wherein the distributed system comprises a plurality of storage partitions, each storage partition corresponding to a unique data platform; the data platform is used for storing generated record files in each storage partition corresponding to the data platform after batch processing of service operation;

4. The method of claim 3, wherein the checking range comprises a first checking range characterizing checking in units of data platforms, and a second checking range characterizing checking in units of storage partitions; the determining the storage space to be checked according to the checking range and the storage index includes:

5. The method of claim 1, wherein the checking conditions comprise a review check and a range check; the generating of the checking record meeting the checking condition according to the comparison result of the first field value and each second field value includes:

6. The method according to any of claims 1-5, wherein the data structure information is used to record fields contained in the corresponding data structure of the target file, and an offset and a field length of each field.

7. An apparatus for data checking of a distributed system, the apparatus comprising:

an information acquisition module configured to perform receiving file information and check information of a target file in response to a check instruction; the target file is a record file generated when any business operation is executed in batch; the file information comprises file identification and data structure information representing a target file data structure, the checking information comprises a checking field and a checking condition associated with the checking field, and the checking field is at least one field in the data structure corresponding to the target file;

8. An electronic device, comprising:

a memory for storing program instructions;

a processor for calling program instructions stored in said memory and for executing the steps comprised by the method of any one of claims 1 to 6 in accordance with the obtained program instructions.

9. A computer-readable storage medium, characterized in that the computer-readable storage medium stores a computer program comprising program instructions that, when executed by a computer, cause the computer to perform the method according to any one of claims 1-6.

10. A computer program product, the computer program product comprising: computer program code which, when run on a computer, causes the computer to perform the method according to any of the preceding claims 1-6.