CN112084183A - Data verification method and device, electronic equipment and computer readable medium - Google Patents
Data verification method and device, electronic equipment and computer readable medium Download PDFInfo
- Publication number
- CN112084183A CN112084183A CN202010956359.1A CN202010956359A CN112084183A CN 112084183 A CN112084183 A CN 112084183A CN 202010956359 A CN202010956359 A CN 202010956359A CN 112084183 A CN112084183 A CN 112084183A
- Authority
- CN
- China
- Prior art keywords
- data
- data set
- resource
- verified
- verification
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/21—Design, administration or maintenance of databases
- G06F16/215—Improving data quality; Data cleansing, e.g. de-duplication, removing invalid entries or correcting typographical errors
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
- G06F11/0703—Error or fault processing not based on redundancy, i.e. by taking additional measures to deal with the error or fault not making use of redundancy in operation, in hardware, or in data representation
- G06F11/0706—Error or fault processing not based on redundancy, i.e. by taking additional measures to deal with the error or fault not making use of redundancy in operation, in hardware, or in data representation the processing taking place on a specific hardware platform or in a specific software environment
- G06F11/0727—Error or fault processing not based on redundancy, i.e. by taking additional measures to deal with the error or fault not making use of redundancy in operation, in hardware, or in data representation the processing taking place on a specific hardware platform or in a specific software environment in a storage system, e.g. in a DASD or network based storage system
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2458—Special types of queries, e.g. statistical queries, fuzzy queries or distributed queries
Abstract
The embodiment of the disclosure discloses a data verification method, a data verification device, electronic equipment and a computer readable medium. One embodiment of the method comprises: performing data repeated verification on at least one piece of data contained in the data set to be verified to obtain a verified data set; performing resource verification on data in the verified data set based on a first comparison result of the first data volume of the data set to be verified and the second data volume of the verified data set to obtain a resource verification data set; and generating verification information based on a second comparison result of the second data volume of the verified data set and the third data volume of the resource verification data set. The embodiment improves the accuracy and the effectiveness of data verification.
Description
Technical Field
The embodiment of the disclosure relates to the technical field of computers, in particular to a data verification method, a data verification device, electronic equipment and a computer readable medium.
Background
With the development of network technology, various services are handled through a network. Data processing devices on the network may need to provide various data processing services. With the increase of the complexity of data processing and networks, the complexity of data transmission and processing in the networks also increases, and the problem of data loss or data processing errors inevitably occurs. Therefore, data needs to be verified. The existing data verification usually verifies the data amount of the data, and does not verify the data processing process, but the data processing process may also have data loss or data processing errors. Therefore, the accuracy and the validity of the verification information obtained by the existing verification method are not high.
Disclosure of Invention
This summary is provided to introduce a selection of concepts in a simplified form that are further described below in the detailed description. This summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used to limit the scope of the claimed subject matter.
Some embodiments of the present disclosure propose data verification methods, apparatuses, electronic devices, and computer readable media to solve the technical problems mentioned in the background section above.
In a first aspect, some embodiments of the present disclosure provide a data verification method, including: performing data repeated verification on at least one piece of data contained in the data set to be verified to obtain a verified data set; performing resource verification on data in the verified data set based on a first comparison result of the first data volume of the data set to be verified and the second data volume of the verified data set to obtain a resource verification data set; and generating verification information based on a second comparison result of the second data volume of the verified data set and the third data volume of the resource verification data set.
In a second aspect, some embodiments of the present disclosure provide a data verification apparatus, including: the repeated checking unit is configured to perform data repeated checking on at least one piece of data contained in the data set to be checked to obtain a checked data set; a resource checking unit configured to perform resource checking on data in the checked data set based on a first comparison result of a first data amount of the data set to be checked and a second data amount of the checked data set, so as to obtain a resource checking data set; and a check information generating unit configured to generate check information based on a second comparison result of the second data amount of the checked data set and the third data amount of the resource check data set.
In a third aspect, some embodiments of the present disclosure provide an electronic device, comprising: one or more processors; a memory having one or more programs stored thereon, which when executed by the one or more processors, cause the one or more processors to perform the data verification method of the first aspect.
In a fourth aspect, some embodiments of the present disclosure provide a computer readable medium on which a computer program is stored, wherein the program, when executed by a processor, implements the data verification method of the first aspect.
One of the above-described various embodiments of the present disclosure has the following advantageous effects: firstly, data in the data set to be verified are repeatedly verified, and the data in the data set to be verified are verified from the data volume level. The method is beneficial to quickly determining whether the data set to be checked has the data repetition problem, and the data checking efficiency is improved. And then, carrying out resource verification on the data in the verified data set, realizing verification on the data in the verified data set from the hierarchy of the data content, and improving the effectiveness of verification on the resource verification data. Therefore, the verification information can reflect the data loss problem and the processing error problem, and the accuracy and the effectiveness of data verification are improved.
Drawings
The above and other features, advantages and aspects of various embodiments of the present disclosure will become more apparent by referring to the following detailed description when taken in conjunction with the accompanying drawings. Throughout the drawings, the same or similar reference numbers refer to the same or similar elements. It should be understood that the drawings are schematic and that elements and elements are not necessarily drawn to scale.
Fig. 1 is a schematic diagram of an application scenario of a data verification method of some embodiments of the present disclosure;
FIG. 2 is a flow diagram of some embodiments of a data verification method according to the present disclosure;
FIG. 3 is a flow diagram of further embodiments of a data verification method according to the present disclosure;
FIG. 4 is a flow diagram of still further embodiments of a data verification method according to the present disclosure;
FIG. 5 is a schematic block diagram of some embodiments of a data verification device according to the present disclosure;
FIG. 6 is a schematic structural diagram of an electronic device suitable for use in implementing some embodiments of the present disclosure.
Detailed Description
Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. While certain embodiments of the present disclosure are shown in the drawings, it is to be understood that the disclosure may be embodied in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided for a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the disclosure are for illustration purposes only and are not intended to limit the scope of the disclosure.
It should be noted that, for convenience of description, only the portions related to the related invention are shown in the drawings. The embodiments and features of the embodiments in the present disclosure may be combined with each other without conflict.
It should be noted that the terms "first", "second", and the like in the present disclosure are only used for distinguishing different devices, modules or units, and are not used for limiting the order or interdependence relationship of the functions performed by the devices, modules or units.
It is noted that references to "a", "an", and "the" modifications in this disclosure are intended to be illustrative rather than limiting, and that those skilled in the art will recognize that "one or more" may be used unless the context clearly dictates otherwise.
The names of messages or information exchanged between devices in the embodiments of the present disclosure are for illustrative purposes only, and are not intended to limit the scope of the messages or information.
The present disclosure will be described in detail below with reference to the accompanying drawings in conjunction with embodiments.
Fig. 1 is a schematic diagram of an application scenario of a data verification method according to some embodiments of the present disclosure.
In fig. 1, after receiving a data set 101 to be verified, an electronic device 100 may verify the data set 101 to be verified through a data repetition verification 102 to obtain a verified data set 103. The data set 101 to be verified may be various types of data. For example, the data set 101 to be verified may be information such as article display information, article transaction information, etc. processed in a certain period of time on the data server. Under normal conditions, the data contained in the data set 101 to be verified should be different from each other, and if at least two identical data exist in the data set 101 to be verified, it indicates that there is a problem in the data set 101 to be verified. The electronic device 100 may query whether the data set 101 to be verified contains multiple identical data through the data duplicate verification 102. Data duplication checking 102 may be various applications for checking data duplication. After data repetition check 102, the obtained checked data set 103 has no repeated data. The electronic device 100 may compare the total amount of data in the data set to be verified 101 (i.e., the first amount of data) with the total amount of data in the verified data set 103 (i.e., the second amount of data), and obtain a first comparison result. The first comparison result is then: the first data amount and the second data amount are the same or the first data amount and the second data amount are different. The electronic device 100 may perform resource check 104 on the checked data set 103 according to the first comparison result, to obtain a resource check data set 105. The resource check 104 may be various applications for resource processing (e.g., may be a resource deduction program, etc.). The resource check data set 105 is formed by deducting resources from the checked data set 103. Electronic device 100 may compare the total amount of data in verified data set 103 (i.e., the second amount of data) with the total amount of data in resource verification data set 105 (i.e., the third amount of data) to obtain a second comparison result. The second comparison result is: the second data amount and the third data amount are the same or the second data amount and the third data amount are different. The electronic device 100 may generate the verification information according to the second comparison result. Then, the check information may be that there is data loss in the resource check data set 105 or that the resource check data set 105 is data complete. The check information may also be generated when the first amount of data and the second amount of data are different. At this time, the verification information may be that there is data duplication in the data set 101 to be verified.
As can be seen from the above description, the data duplication check 102 processes the to-be-checked data set 101 in terms of data amount. Resource check 104 processes the to-be-checked data set 101 from the data content. Then, after the data repetition check 102 and the resource check 104, the number of the data sets 101 to be checked and the data content are checked, and the accuracy and the validity of the data check are submitted.
It should be understood that the number of electronic devices 100 in FIG. 1 is merely illustrative. There may be any number of electronic devices 100, as desired for implementation.
With continued reference to fig. 2, fig. 2 illustrates a flow 200 of some embodiments of a data verification method according to the present disclosure. The data verification method comprises the following steps:
In some embodiments, an execution subject of the data verification method (e.g., the electronic device 100 shown in fig. 1) may receive a web browsing request from a terminal with which a user browses a web page through a wired connection manner or a wireless connection manner. It should be noted that the wireless connection means may include, but is not limited to, a 3G/4G connection, a WiFi connection, a bluetooth connection, a WiMAX connection, a Zigbee connection, a uwb (ultra wideband) connection, and other wireless connection means now known or developed in the future.
After receiving the data set to be verified, the execution main body may perform data repeated verification on each piece of data included in the data set to be verified, so as to obtain a verified data set. For example, the execution subject may select a piece of data from the data set to be checked, and then compare the piece of data with other pieces of data in the data set to be checked one by one, thereby determining whether the data has duplicate data in the data set to be checked. If the data set to be checked has the repeated data, the execution main body can delete the repeated data in the data set to be checked, and then the checked data set is obtained. The data quantity of the data set to be verified is verified through repeated data verification, and the detection of the data set to be verified is realized from the hierarchy of the data quantity. The method is beneficial to quickly determining whether the data set to be checked has the data repetition problem, and the data checking efficiency is improved.
After the verified data set is obtained, the execution main body may compare a first data volume of the data set to be verified with a second data volume of the verified data set, and obtain a first comparison result. The first comparison result may be that the first data amount is the same as the second data amount or the first data amount is different from the second data amount. After the first comparison result is obtained, the execution main body may perform resource verification on the data in the verified data set to obtain a resource verification data set. Here, the resource may be the number of items, the amount of the items, and the like. When the execution main body performs resource verification on the data in the verified data set, corresponding data processing can be performed on the resources of the data, and the obtained data is the data in the resource verification data set. For example, a certain data in the checked data set may be: consuming XX elements, the resulting data in the resource check data set may be: XX elements have been deducted. The resource check realizes the check aiming at the data processing process, and is beneficial to detecting the possibility of data loss or errors in the processing process.
The execution subject may compare the second data size of the checked data set with the third data size of the resource check data set, to obtain a second comparison result. The second comparison result may be that the second data amount is the same as the third data amount or the second data amount is different from the third data amount. The execution subject may generate the verification information according to the second comparison result. For example, when the second comparison result is that the second data amount is the same as the third data amount, the verification information may be: the data is correct; when the second comparison result is that the second data amount is different from the third data amount, the check information may be: data errors. Therefore, the verification aiming at the data processing process is realized through the resource verification, and the accuracy and the effectiveness of the verification information are improved.
According to the data verification method disclosed by some embodiments of the disclosure, data repetition verification is performed on data in the data set to be verified at first, and verification of the data in the data set to be verified is achieved from the data volume level. The method is beneficial to quickly determining whether the data set to be checked has the data repetition problem, and the data checking efficiency is improved. And then, carrying out resource verification on the data in the verified data set, realizing verification on the data in the verified data set from the hierarchy of the data content, and improving the effectiveness of verification on the resource verification data. Therefore, the verification information can reflect the data loss problem and the processing error problem, and the accuracy and the effectiveness of data verification are improved.
With continued reference to fig. 3, fig. 3 illustrates a flow 300 of some embodiments of a data verification method according to the present disclosure. The data verification method comprises the following steps:
Each piece of data in the data set to be verified may include a respective data number. The data number may be represented by a number of combinations or a character string composed of numbers and characters. The execution subject may check whether there are multiple data with the same data number in the data set to be checked. If the data exists, the fact that the repeated data exists in the data set to be checked is indicated.
Similarly, data duplication check can be performed on data by using time information (for example, the generation time or modification time of the data can be obtained), data size information (for example, the data size of a certain data is 1,804KB), and the like. The execution body may also repeatedly check the data by combining various information (e.g., data number, time information, data size information). Therefore, the accuracy of repeated data verification can be greatly improved.
The execution subject may combine the data in the data set to be verified, which are different from each other, to form a verified data set.
The contents of steps 303 to 304 are the same as those of steps 202 to 203, and are not described in detail here.
The embodiment corresponding to fig. 3 can improve the accuracy of data verification, and is beneficial to improving the accuracy and effectiveness of verification information.
With continued reference to fig. 4, fig. 4 illustrates a flow 400 of some embodiments of a data verification method according to the present disclosure. The data verification method comprises the following steps:
The content of step 401 is the same as that of step 201, and is not described in detail here.
In some optional implementation manners of some embodiments, the performing resource check on the data in the checked data set based on a first comparison result between the first data amount of the data set to be checked and the second data amount of the checked data set to obtain a resource check data set includes: and responding to the first comparison result that the first data volume is different from the second data volume, sending a first alarm signal, and generating first verification information through the repeated data in the data set to be verified.
When the first data amount is different from the second data amount, it indicates that there is an error in terms of the data amount alone, i.e., the first data amount is larger than the second data amount. At this time, the execution main body may issue a first alarm signal. Then, the first check information is generated by repeating the data. The first check information may be: repeating the data numbering of the data. Therefore, after the first alarm signal is acquired, the user can acquire the repeated data in the data set to be checked through the first checking information.
In some optional implementation manners of some embodiments, the performing resource check on the data in the checked data set based on a first comparison result between the first data amount of the data set to be checked and the second data amount of the checked data set to obtain a resource check data set includes: and in response to the first comparison result that the first data volume is the same as the second data volume, performing resource verification on the data in the verified data set to obtain a resource verification data set.
When the first data volume is the same as the second data volume, it is described that, in terms of data volume, there is no problem in the data set to be checked. At this time, the execution main body may further perform resource check on the data in the checked data set to obtain a resource check data set.
In some optional implementation manners of some embodiments, the performing resource check on the data in the checked data set to obtain a resource check data set may include: and for each piece of data in the verified data set, importing the data into a preset resource processing system to obtain resource verification data corresponding to each piece of data.
The predetermined resource processing system may be a fee deduction system or other resource deduction system (for example, a card swiping machine, etc.). The execution main body can import the data in the verified data set into the resource processing system to obtain the resource verification data.
And step 404, in response to the second comparison result indicating that the second data volume is different from the third data volume, sending a second alarm signal.
When the second data amount is different from the third data amount, it is described that data loss occurs in the resource checking process, that is, the second data amount is greater than the third data amount. At this time, the execution main body may issue a second alarm signal.
The execution main body may compare the verified data set with the resource verification data set, and use data in the verified data set that does not correspond to data in the resource verification data set as lost data.
The execution subject may count the missing data to obtain the second check-up information. The second check-up information may be: the data lost during the resource check process includes: XXX.
In some optional implementation manners of some embodiments, the generating second check-up information based on the lost data may include:
firstly, log data corresponding to the lost data in the resource processing system is inquired.
The resource processing system may comprise at least one resource processing unit. Each resource processing unit is configured with tag information. The execution subject may query log data corresponding to the lost data in the resource processing system. The log data includes data processing records of each resource processing unit in the resource processing system.
And secondly, determining at least one target resource processing unit for processing the lost data according to the log data.
Lost data is typically lost after processing by one resource handling unit and before processing by another resource handling unit. The execution subject may query each resource processing unit through which a certain verified data passes, and further determine at least one target resource processing unit.
And a third step of generating the second check-up information based on the at least one target resource processing unit.
The execution main body may determine, according to at least one target resource processing unit, information such as sequence information, processing time information, and failure information of the verified data passing through each target resource processing unit, and generate second verification information according to the information.
The execution subject may generate third check information when the second data amount is the same as the third data amount. At this time, the content of the third verification information may be: the data in the data set to be verified is error-free.
The embodiment corresponding to fig. 4 may accurately acquire the repeated data in the data set to be checked and the lost data in the resource checking process, and generate corresponding checking information according to the repeated data or the lost data. The accuracy and the validity of the verification information are improved.
With further reference to fig. 5, as an implementation of the methods shown in the above figures, the present disclosure provides some embodiments of a data verification apparatus, which correspond to those shown in fig. 2, and which may be applied in various electronic devices.
As shown in fig. 5, the data verification apparatus 500 of some embodiments includes: a repetition check unit 501, a resource check unit 502, and a check information generation unit 503. The repeated checking unit 501 is configured to perform data repeated checking on at least one piece of data included in the data set to be checked to obtain a checked data set; a resource checking unit 502 configured to perform resource checking on data in the checked data set based on a first comparison result between a first data amount of the to-be-checked data set and a second data amount of the checked data set, so as to obtain a resource checking data set; a check information generating unit 503 configured to generate check information based on a second comparison result of the second data amount of the checked data set and the third data amount of the resource check data set.
In an optional implementation manner of some embodiments, the data includes a data number, and the duplication checking unit 501 may include: a repeated data determining subunit (not shown in the figure) and a verified data set constructing subunit (not shown in the figure). The repeated data determining subunit is configured to, for each piece of data in the data set to be checked, determine repeated data with the same data number as that of the data by comparing the data number of the piece of data with data numbers of other pieces of data in the data set to be checked; and the verified data set constructing subunit is configured to construct a verified data set by the data in the data set to be verified except the repeated data.
In an optional implementation manner of some embodiments, the resource checking unit 502 may include: and a first check information generating subunit (not shown in the figure), configured to send a first alarm signal in response to that the first comparison result is that the first data amount is different from the second data amount, and generate first check information by using the repeated data in the to-be-checked data set.
In an optional implementation manner of some embodiments, the resource checking unit 502 may include: and a resource checking subunit (not shown in the figure), configured to perform resource checking on the data in the checked data set to obtain a resource checking data set, in response to that the first comparison result is that the first data amount is the same as the second data amount.
In an optional implementation manner of some embodiments, the resource checking subunit may include: a resource verification data obtaining module (not shown in the figure), configured to, for each piece of data in the verified data set, import the data into a preset resource processing system, so as to obtain resource verification data corresponding to each piece of data.
In an optional implementation manner of some embodiments, the check information generating unit 503 may include: an alarm signal issuing subunit (not shown in the figure), a lost data determining subunit (not shown in the figure), and a check information generating subunit (not shown in the figure). Wherein the alarm signal issuing subunit, in response to the second comparison result being that the second data amount is different from the third data amount, is configured to issue a second alarm signal; a lost data determining subunit configured to determine lost data by the checked data set and the resource check data set; and a check information generating subunit configured to generate second check information based on the lost data.
In an optional implementation manner of some embodiments, the resource processing system includes at least one resource processing unit, each of the at least one resource processing unit is configured with tag information, and the check information generating subunit may include: a log data query module (not shown in the figure), a target resource processing unit determination module (not shown in the figure), and a second check information generation module (not shown in the figure). The log data query module is configured to query log data corresponding to the lost data in the resource processing system; a target resource processing unit determining module configured to determine at least one target resource processing unit that processes the lost data according to the log data; a second check-up information generating module configured to generate the second check-up information based on the at least one target resource processing unit.
In an optional implementation manner of some embodiments, the check information generating unit 503 may include: and a third check information generating module (not shown in the figure) configured to generate third check information through the resource check data set in response to the second comparison result indicating that the second data amount is the same as the third data amount.
It will be understood that the elements described in the apparatus 500 correspond to various steps in the method described with reference to fig. 2. Thus, the operations, features and resulting advantages described above with respect to the method are also applicable to the apparatus 500 and the units included therein, and are not described herein again.
As shown in fig. 6, electronic device 600 may include a processing means (e.g., central processing unit, graphics processor, etc.) 601 that may perform various appropriate actions and processes in accordance with a program stored in a Read Only Memory (ROM)602 or a program loaded from a storage means 608 into a Random Access Memory (RAM) 603. In the RAM603, various programs and data necessary for the operation of the electronic apparatus 600 are also stored. The processing device 601, the ROM 602, and the RAM603 are connected to each other via a bus 604. An input/output (I/O) interface 605 is also connected to bus 604.
Generally, the following devices may be connected to the I/O interface 605: input devices 606 including, for example, a touch screen, touch pad, keyboard, mouse, camera, microphone, accelerometer, gyroscope, etc.; output devices 607 including, for example, a Liquid Crystal Display (LCD), a speaker, a vibrator, and the like; storage 608 including, for example, tape, hard disk, etc.; and a communication device 609. The communication means 609 may allow the electronic device 600 to communicate with other devices wirelessly or by wire to exchange data. While fig. 6 illustrates an electronic device 600 having various means, it is to be understood that not all illustrated means are required to be implemented or provided. More or fewer devices may alternatively be implemented or provided. Each block shown in fig. 6 may represent one device or may represent multiple devices as desired.
In particular, according to some embodiments of the present disclosure, the processes described above with reference to the flow diagrams may be implemented as computer software programs. For example, some embodiments of the present disclosure include a computer program product comprising a computer program embodied on a computer readable medium, the computer program comprising program code for performing the method illustrated in the flow chart. In some such embodiments, the computer program may be downloaded and installed from a network through the communication device 609, or installed from the storage device 608, or installed from the ROM 602. The computer program, when executed by the processing device 601, performs the above-described functions defined in the methods of some embodiments of the present disclosure.
It should be noted that the computer readable medium described above in some embodiments of the present disclosure may be a computer readable signal medium or a computer readable storage medium or any combination of the two. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples of the computer readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In some embodiments of the disclosure, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In some embodiments of the present disclosure, however, a computer readable signal medium may include a propagated data signal with computer readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: electrical wires, optical cables, RF (radio frequency), etc., or any suitable combination of the foregoing.
In some embodiments, the clients, servers may communicate using any currently known or future developed network Protocol, such as HTTP (HyperText Transfer Protocol), and may interconnect with any form or medium of digital data communication (e.g., a communications network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), the Internet (e.g., the Internet), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any currently known or future developed network.
The computer readable medium may be embodied in the electronic device; or may exist separately without being assembled into the electronic device. The computer readable medium carries one or more programs which, when executed by the electronic device, cause the electronic device to: performing data repeated verification on at least one piece of data contained in the data set to be verified to obtain a verified data set; performing resource verification on data in the verified data set based on a first comparison result of the first data volume of the data set to be verified and the second data volume of the verified data set to obtain a resource verification data set; and generating verification information based on a second comparison result of the second data volume of the verified data set and the third data volume of the resource verification data set.
Computer program code for carrying out operations for embodiments of the present disclosure may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C + +, and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet service provider).
The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
The units described in some embodiments of the present disclosure may be implemented by software, and may also be implemented by hardware. The described units may also be provided in a processor, and may be described as: a processor includes a repetition check unit, a resource check unit, and a check information generation unit. Here, the names of these units do not constitute a limitation to the unit itself in some cases, and for example, the verification information generation unit may also be described as "a unit for generating verification information".
The functions described herein above may be performed, at least in part, by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field Programmable Gate Arrays (FPGAs), Application Specific Integrated Circuits (ASICs), Application Specific Standard Products (ASSPs), systems on a chip (SOCs), Complex Programmable Logic Devices (CPLDs), and the like.
According to one or more embodiments of the present disclosure, there is provided a data verification method including: performing data repeated verification on at least one piece of data contained in the data set to be verified to obtain a verified data set; performing resource verification on data in the verified data set based on a first comparison result of the first data volume of the data set to be verified and the second data volume of the verified data set to obtain a resource verification data set; and generating verification information based on a second comparison result of the second data volume of the verified data set and the third data volume of the resource verification data set.
According to one or more embodiments of the present disclosure, the data includes a data number, and the data repeated verification is performed on at least one piece of data included in the data set to be verified to obtain a verified data set, including: for each piece of data in the data set to be verified, comparing the data number of the piece of data with the data numbers of other pieces of data in the data set to be verified, and determining repeated data with the same data number as the data number of the piece of data; and forming a verified data set by the data in the data set to be verified except the repeated data.
According to one or more embodiments of the present disclosure, performing resource check on data in the checked data set based on a first comparison result between a first data amount of the data set to be checked and a second data amount of the checked data set to obtain a resource check data set includes: and responding to the first comparison result that the first data volume is different from the second data volume, sending a first alarm signal, and generating first verification information through the repeated data in the data set to be verified.
According to one or more embodiments of the present disclosure, performing resource check on data in the checked data set based on a first comparison result between a first data amount of the data set to be checked and a second data amount of the checked data set to obtain a resource check data set includes: and in response to the first comparison result that the first data volume is the same as the second data volume, performing resource verification on the data in the verified data set to obtain a resource verification data set.
According to one or more embodiments of the present disclosure, the performing resource check on the data in the checked data set to obtain a resource check data set includes: and for each piece of data in the verified data set, importing the data into a preset resource processing system to obtain resource verification data corresponding to each piece of data.
According to one or more embodiments of the present disclosure, the generating of the check information based on the second comparison result between the second data amount of the checked data set and the third data amount of the resource check data set includes: sending a second alarm signal in response to the second comparison result that the second data volume is different from the third data volume; determining lost data through the checked data set and the resource check data set; and generating second check-up information based on the lost data.
According to one or more embodiments of the present disclosure, the resource processing system includes at least one resource processing unit, each of the at least one resource processing unit is configured with tag information, and the generating second check-up information based on the lost data includes: inquiring log data corresponding to the lost data in the resource processing system; determining at least one target resource processing unit for processing the lost data according to the log data; and generating the second check-up information based on the at least one target resource processing unit.
According to one or more embodiments of the present disclosure, the generating of the check information based on the second comparison result between the second data amount of the checked data set and the third data amount of the resource check data set includes: and generating third check information through the resource check data set in response to the second comparison result that the second data volume is the same as the third data volume.
According to one or more embodiments of the present disclosure, there is provided a data verification apparatus including: the repeated checking unit is configured to perform data repeated checking on at least one piece of data contained in the data set to be checked to obtain a checked data set; a resource checking unit configured to perform resource checking on data in the checked data set based on a first comparison result of a first data amount of the data set to be checked and a second data amount of the checked data set, so as to obtain a resource checking data set; and a check information generating unit configured to generate check information based on a second comparison result of the second data amount of the checked data set and the third data amount of the resource check data set.
According to one or more embodiments of the present disclosure, the data includes a data number, and the duplication checking unit includes: a repeated data determining subunit, configured to, for each piece of data in the data set to be verified, determine repeated data having the same data number as that of the piece of data by comparing the data number of the piece of data with data numbers of other pieces of data in the data set to be verified; and the verified data set constructing subunit is configured to construct a verified data set by the data in the data set to be verified except the repeated data.
According to one or more embodiments of the present disclosure, the resource checking unit includes: and the first checking information generating subunit is configured to send a first alarm signal in response to that the first comparison result is that the first data volume is different from the second data volume, and generate first checking information through the repeated data in the data set to be checked.
According to one or more embodiments of the present disclosure, the resource checking unit includes: and the resource checking subunit, in response to the first comparison result that the first data volume is the same as the second data volume, is configured to perform resource checking on the data in the checked data set to obtain a resource checking data set.
According to one or more embodiments of the present disclosure, the resource check subunit includes: and the resource verification data acquisition module is configured to import each piece of data in the verified data set into a preset resource processing system to obtain resource verification data corresponding to each piece of data.
According to one or more embodiments of the present disclosure, the verification information generating unit includes: an alarm signal issuing subunit configured to issue a second alarm signal in response to the second comparison result being that the second data amount is different from the third data amount; a lost data determining subunit configured to determine lost data by the checked data set and the resource check data set; and a check information generating subunit configured to generate second check information based on the lost data.
According to one or more embodiments of the present disclosure, the resource processing system includes at least one resource processing unit, each of the at least one resource processing unit is configured with tag information, and the check information generating subunit includes: a log data query module configured to query log data corresponding to the lost data in the resource processing system; a target resource processing unit determining module configured to determine at least one target resource processing unit that processes the lost data according to the log data; a second check-up information generating module configured to generate the second check-up information based on the at least one target resource processing unit.
According to one or more embodiments of the present disclosure, the verification information generating unit includes: and the third check information generating module is configured to generate third check information through the resource check data set in response to the second comparison result indicating that the second data amount is the same as the third data amount.
The foregoing description is only exemplary of the preferred embodiments of the disclosure and is illustrative of the principles of the technology employed. It will be appreciated by those skilled in the art that the scope of the invention in the embodiments of the present disclosure is not limited to the specific combination of the above-mentioned features, but also encompasses other embodiments in which any combination of the above-mentioned features or their equivalents is made without departing from the inventive concept as defined above. For example, the above features and (but not limited to) technical features with similar functions disclosed in the embodiments of the present disclosure are mutually replaced to form the technical solution.
Claims (11)
1. A method of data verification, comprising:
performing data repeated verification on at least one piece of data contained in the data set to be verified to obtain a verified data set;
performing resource verification on data in the verified data set based on a first comparison result of the first data volume of the data set to be verified and the second data volume of the verified data set to obtain a resource verification data set;
and generating verification information based on a second comparison result of the second data volume of the verified data set and the third data volume of the resource verification data set.
2. The method of claim 1, wherein the data comprises a data number; and
the data repetition check of at least one piece of data included in the data set to be checked is performed to obtain a checked data set, and the data repetition check includes:
for each piece of data in the data set to be verified, comparing the data number of the piece of data with the data numbers of other pieces of data in the data set to be verified, and determining repeated data with the same data number as the data number of the piece of data;
and forming a checked data set by the data in the data set to be checked except the repeated data.
3. The method of claim 1, wherein the performing resource check on the data in the checked data set based on the first comparison result between the first data amount of the data set to be checked and the second data amount of the checked data set to obtain a resource check data set comprises:
and sending a first alarm signal in response to the first comparison result that the first data volume is different from the second data volume, and generating first verification information through repeated data in the data set to be verified.
4. The method of claim 1, wherein the performing resource check on the data in the checked data set based on the first comparison result between the first data amount of the data set to be checked and the second data amount of the checked data set to obtain a resource check data set comprises:
and in response to the first comparison result that the first data volume is the same as the second data volume, performing resource verification on the data in the verified data set to obtain a resource verification data set.
5. The method of claim 4, wherein the performing resource check on the data in the checked data set to obtain a resource check data set comprises:
and for each piece of data in the verified data set, importing the data into a preset resource processing system to obtain resource verification data corresponding to each piece of data.
6. The method of claim 5, wherein the generating verification information based on a second comparison of a second amount of data of the verified data set to a third amount of data of the resource verification data set comprises:
sending a second alarm signal in response to the second comparison result that the second data volume is different from the third data volume;
determining lost data through the checked data set and the resource check data set;
second check-up information is generated based on the missing data.
7. The method of claim 6, wherein the resource processing system comprises at least one resource processing unit, each of the at least one resource processing unit configured with tag information; and
generating second check-up information based on the lost data, comprising:
inquiring log data corresponding to the lost data in the resource processing system;
determining at least one target resource processing unit for processing the lost data according to the log data;
generating the second check-up information based on the at least one target resource processing unit.
8. The method of claim 1, wherein the generating verification information based on a second comparison of a second amount of data of the verified data set to a third amount of data of the resource verification data set comprises:
and generating third check information through the resource check data set in response to the second comparison result that the second data volume is the same as the third data volume.
9. A data verification apparatus, comprising:
the repeated checking unit is configured to perform data repeated checking on at least one piece of data contained in the data set to be checked to obtain a checked data set;
the resource checking unit is configured to perform resource checking on data in the checked data set based on a first comparison result of a first data volume of the data set to be checked and a second data volume of the checked data set to obtain a resource checking data set;
a check information generating unit configured to generate check information based on a second comparison result of a second data amount of the checked data set and a third data amount of the resource check data set.
10. An electronic device, comprising:
one or more processors;
a storage device having one or more programs stored thereon,
when executed by the one or more processors, cause the one or more processors to implement the method of any one of claims 1-8.
11. A computer-readable medium, on which a computer program is stored, wherein the program, when executed by a processor, implements the method of any one of claims 1 to 8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010956359.1A CN112084183A (en) | 2020-09-11 | 2020-09-11 | Data verification method and device, electronic equipment and computer readable medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010956359.1A CN112084183A (en) | 2020-09-11 | 2020-09-11 | Data verification method and device, electronic equipment and computer readable medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112084183A true CN112084183A (en) | 2020-12-15 |
Family
ID=73737627
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010956359.1A Pending CN112084183A (en) | 2020-09-11 | 2020-09-11 | Data verification method and device, electronic equipment and computer readable medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112084183A (en) |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106708648A (en) * | 2015-07-14 | 2017-05-24 | 阿里巴巴集团控股有限公司 | Text data storage checking method and system |
| CN108683668A (en) * | 2018-05-18 | 2018-10-19 | 腾讯科技（深圳）有限公司 | Resource checksum method, apparatus, storage medium and equipment in content distributing network |
| CN110390069A (en) * | 2019-07-31 | 2019-10-29 | 中国工商银行股份有限公司 | The method, apparatus of data check calculates equipment and medium |
| US20200004644A1 (en) * | 2018-06-29 | 2020-01-02 | International Business Machines Corporation | Data corruption source and timeline analysis |
| CN110674116A (en) * | 2019-09-25 | 2020-01-10 | 四川长虹电器股份有限公司 | System and method for checking and inserting data repetition of database based on swoole |
| CN110851539A (en) * | 2019-10-25 | 2020-02-28 | 东软集团股份有限公司 | Metadata verification method and device, readable storage medium and electronic equipment |
| CN111127178A (en) * | 2019-11-26 | 2020-05-08 | 泰康保险集团股份有限公司 | Data processing method and device, storage medium and electronic equipment |
| CN111144697A (en) * | 2019-11-29 | 2020-05-12 | 泰康保险集团股份有限公司 | Data processing method, data processing device, storage medium and electronic equipment |
| CN111626865A (en) * | 2020-05-22 | 2020-09-04 | 泰康保险集团股份有限公司 | Data processing method and device, electronic equipment and storage medium |
-
2020
- 2020-09-11 CN CN202010956359.1A patent/CN112084183A/en active Pending
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106708648A (en) * | 2015-07-14 | 2017-05-24 | 阿里巴巴集团控股有限公司 | Text data storage checking method and system |
| CN108683668A (en) * | 2018-05-18 | 2018-10-19 | 腾讯科技（深圳）有限公司 | Resource checksum method, apparatus, storage medium and equipment in content distributing network |
| US20200004644A1 (en) * | 2018-06-29 | 2020-01-02 | International Business Machines Corporation | Data corruption source and timeline analysis |
| CN110390069A (en) * | 2019-07-31 | 2019-10-29 | 中国工商银行股份有限公司 | The method, apparatus of data check calculates equipment and medium |
| CN110674116A (en) * | 2019-09-25 | 2020-01-10 | 四川长虹电器股份有限公司 | System and method for checking and inserting data repetition of database based on swoole |
| CN110851539A (en) * | 2019-10-25 | 2020-02-28 | 东软集团股份有限公司 | Metadata verification method and device, readable storage medium and electronic equipment |
| CN111127178A (en) * | 2019-11-26 | 2020-05-08 | 泰康保险集团股份有限公司 | Data processing method and device, storage medium and electronic equipment |
| CN111144697A (en) * | 2019-11-29 | 2020-05-12 | 泰康保险集团股份有限公司 | Data processing method, data processing device, storage medium and electronic equipment |
| CN111626865A (en) * | 2020-05-22 | 2020-09-04 | 泰康保险集团股份有限公司 | Data processing method and device, electronic equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109447641B (en) | Method and apparatus for transmitting blockchain data to blockchain browser | |
| CN112463729B (en) | Data file warehousing method and device, electronic equipment and medium | |
| CN110851139A (en) | Method and device for checking codes and electronic equipment | |
| CN109325744B (en) | Payment processing method, payment processing device, payment processing medium and electronic equipment | |
| CN113254813B (en) | Short link generation method, device, server and computer readable medium | |
| CN110516463B (en) | Method and apparatus for generating information | |
| CN109614327B (en) | Method and apparatus for outputting information | |
| CN110928594A (en) | Service development method and platform | |
| CN111787041B (en) | Method and device for processing data | |
| CN112559024A (en) | Method and device for generating transaction code change list | |
| CN111858381A (en) | Application program fault tolerance capability test method, electronic device and medium | |
| CN114428815A (en) | Data storage method and device, electronic equipment and computer readable medium | |
| CN112084183A (en) | Data verification method and device, electronic equipment and computer readable medium | |
| CN111460020B (en) | Method, device, electronic equipment and medium for resolving message | |
| CN114490718A (en) | Data output method, data output device, electronic equipment and computer readable medium | |
| CN112184406A (en) | Data processing method, system, electronic device and computer readable storage medium | |
| CN112367266A (en) | Current limiting method, current limiting device, electronic equipment and computer readable medium | |
| CN112163176A (en) | Data storage method and device, electronic equipment and computer readable medium | |
| CN112667119A (en) | Measuring point batch correlation method and device, electronic equipment and computer readable medium | |
| CN112579428A (en) | Interface testing method and device, electronic equipment and storage medium | |
| CN113807854B (en) | Method, apparatus, system, electronic device and medium for electronic payment | |
| CN110262756B (en) | Method and device for caching data | |
| CN115098453A (en) | Information storage method and device, electronic equipment and computer readable medium | |
| CN116012158A (en) | Service processing method, device, equipment and storage medium | |
| CN114444064A (en) | Account processing method and device, electronic equipment and computer readable medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |