CN114611155A - Data management node verification method, device, equipment and medium - Google Patents
Data management node verification method, device, equipment and medium Download PDFInfo
- Publication number
- CN114611155A CN114611155A CN202210251022.XA CN202210251022A CN114611155A CN 114611155 A CN114611155 A CN 114611155A CN 202210251022 A CN202210251022 A CN 202210251022A CN 114611155 A CN114611155 A CN 114611155A
- Authority
- CN
- China
- Prior art keywords
- data
- node
- content
- data management
- management node
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
- G06F21/60—Protecting data
- G06F21/64—Protecting data integrity, e.g. using checksums, certificates or signatures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/23—Updating
- G06F16/2365—Ensuring data consistency and integrity
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Computer Security & Cryptography (AREA)
- General Physics & Mathematics (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Databases & Information Systems (AREA)
- Data Mining & Analysis (AREA)
- Health & Medical Sciences (AREA)
- Bioethics (AREA)
- General Health & Medical Sciences (AREA)
- Computer Hardware Design (AREA)
- Software Systems (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The embodiment of the invention discloses a method, a device, equipment and a medium for verifying a data management node. Wherein, the method comprises the following steps: when the data management node operates on line, determining the node updating time of the data management node and the data table updating time of a management data table; if at least one of the node updating time and the data table updating time is longer than the pre-stored verification time of the data management node, generating a content abstract based on the configuration content of the data management node, and recording the content abstract generating time; and if the generated content abstract is consistent with the information of the pre-stored content abstract, determining that the data management node passes the verification. According to the technical scheme, the verification speed of the data management node can be improved, and the system computing power is saved.
Description
Technical Field
The invention relates to the technical field of data verification, in particular to a verification method, a verification device, verification equipment and verification media for data management nodes.
Background
With the rapid development of internet technology, data interaction through the internet has become a common business requirement. The data management is a key link in the data management and data application process. Data information which is messy, poor in quality or unclear in semantics can be converted into a data form meeting the requirements through data management, the value of data is improved, and a computing system can be used conveniently.
However, in the current business application scenario, one data governance process often includes multiple governance nodes, when the process runs and comes online, the validity of the content of each node needs to be checked, the checking logic of each node is different, and part of the nodes (such as import/export tasks) also involve checking of a third-party data source, which results in increased system pressure and long operation waiting time.
Therefore, how to increase the verification speed of the data management node is a technical problem to be solved urgently by those skilled in the art.
Disclosure of Invention
The invention provides a method, a device, equipment and a medium for verifying a data management node, which can improve the verification speed of the data management node and save the system calculation power.
According to an aspect of the present invention, there is provided a method for verifying a data administration node, the method including:
determining node updating time of the data management nodes and data table updating time of a management data table when the data management nodes are in online operation;
if at least one of the node updating time and the data table updating time is longer than the pre-stored verification time of the data management node, generating a content abstract based on the configuration content of the data management node, and recording the content abstract generating time;
and if the generated content abstract is consistent with the information of the pre-stored content abstract, determining that the data management node passes the verification.
Optionally, before determining the node update time of the data abatement node and the data table update time of the abatement data table, the method further includes:
and if the data management node is detected to be verified to be complete, generating a pre-stored content abstract by adopting a preset format for the configuration content of the data management node, and recording the pre-stored verification time.
Optionally, after the configuration content of the data governance node is generated into a pre-stored content abstract by using a preset format, the method further includes:
and if the configuration content updating operation of the data management node is detected, processing the configuration content by adopting a preset format based on the updated configuration content so as to update the abstract of the prestored content, and updating the prestored verification time based on the occurrence time of the updating operation.
Optionally, the preset format is that the hash value is calculated by using an SHA-256 algorithm for the configuration content of the data management node.
Optionally, the method further includes:
if the generated content abstract is inconsistent with the information of the pre-stored content abstract, performing logic verification based on the configuration content of the data management node and the content of a management data table;
and after the verification is passed, generating a content abstract based on the verified configuration content so as to update the pre-stored content abstract, and updating the pre-stored verification time based on the occurrence time of the updating operation.
Optionally, performing logic verification based on the configuration content of the data governance node and the content of the governance data table includes:
and performing logic check on at least one of connectivity, field content integrity, field type correspondence, field mapping matching and SQL condition validity of the data table source node and the data table target node based on the configuration content of the data management node and the content of the management data table.
According to another aspect of the present invention, there is provided a data administration node verification apparatus, including:
the updating time determining module is used for determining the node updating time of the data management node and the data table updating time of the management data table when the data management node operates on line;
the content abstract determining module is used for generating a content abstract based on the configuration content of the data management node and recording the content abstract generating time if at least one of the node updating time and the data table updating time is greater than the pre-stored verification time of the data management node;
and the data management node checking module is used for determining that the data management node passes the checking if the generated content abstract is consistent with the information of the pre-stored content abstract.
According to another aspect of the present invention, there is provided an electronic apparatus including:
at least one processor; and
a memory communicatively coupled to the at least one processor; wherein the content of the first and second substances,
the memory stores a computer program executable by the at least one processor, the computer program being executable by the at least one processor to enable the at least one processor to perform a method of verifying a data abatement node according to any one of the embodiments of the invention.
According to another aspect of the present invention, there is provided a computer readable storage medium storing computer instructions for causing a processor to implement a method of verifying a data governance node according to any one of the embodiments of the present invention when executed.
According to the technical scheme of the embodiment of the invention, the node updating time of the data management node and the data table updating time of the management data table are determined when the data management node operates on line; if at least one of the node updating time and the data table updating time is longer than the pre-stored verification time of the data management node, generating a content abstract based on the configuration content of the data management node, and recording the content abstract generating time; and if the generated content abstract is consistent with the information of the pre-stored content abstract, determining that the data management node passes the verification. According to the technical scheme, the verification speed of the data management node can be improved, and the system computing power is saved.
It should be understood that the statements in this section do not necessarily identify key or critical features of the embodiments of the present invention, nor do they necessarily limit the scope of the invention. Other features of the present invention will become apparent from the following description.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings needed to be used in the description of the embodiments will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.
Fig. 1 is a flowchart of a method for checking a data administration node according to an embodiment of the present invention;
FIG. 2 is a flow chart of a method for verifying a data administration node according to a second embodiment of the present invention;
FIG. 3 is a flow chart of a method for verifying a data administration node according to a third embodiment of the present invention;
fig. 4 is a schematic structural diagram of a checking apparatus for a data administration node according to a fourth embodiment of the present invention;
fig. 5 is a schematic structural diagram of an electronic device implementing the data governance node verification method according to the embodiment of the present invention.
Detailed Description
In order to make the technical solutions of the present invention better understood, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
It should be noted that the terms "pre-stored", "pre-set", etc. in the description and claims of the present invention and the above-mentioned drawings are used for distinguishing similar objects and are not necessarily used for describing a specific order or sequence. It is to be understood that the data so used is interchangeable under appropriate circumstances such that the embodiments of the invention described herein are capable of operation in sequences other than those illustrated or described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
Example one
Fig. 1 is a flowchart of a method for verifying a data management node according to an embodiment of the present invention, where this embodiment is applicable to a situation where a data management node is verified, and the method may be implemented by a verification apparatus of the data management node, and the verification apparatus of the data management node may be implemented in a hardware and/or software manner, and the verification apparatus of the data management node may be configured in an electronic device with data processing capability. As shown in fig. 1, the method includes:
s110, when the data governance nodes are operated on line, determining node updating time of the data governance nodes and data table updating time of the governance data table.
It will be appreciated that the data administration nodes are connected to administration data tables, which may include data source tables and target tables, which may be generated by SQL conditions configured by the data administration nodes. The data source table can be a source table established according to metadata information acquired by a data source, the data source can be from a plurality of mainstream databases, data interaction is carried out among the databases through a connector, and the corresponding databases are connected according to the provided database information; the target table may be a table established according to SQL conditions configured by the data management node, and the SQL conditions may be set according to actual needs.
It should be noted that the table information in the abatement data table mainly includes fields and field types. The fields may include names, ages, pictures, etc., and the corresponding field types may be texts, numbers, graphics.
In the embodiment of the invention, when the process runs on line at the data management node, the legality of the content of the data management node needs to be checked, and in order to avoid checking the nodes one by one for each on-line operation, the node updating time of the data management node and the data table updating time of the management data table are introduced, and whether the data management node can be checked to pass or not is determined by comparing the data management node with the management data table and judging whether the data management node changes from the last check to the current check.
The node update time may be a timestamp for updating the configuration information of the data management node, and the data table update time may be a timestamp for updating the data source table or a timestamp for updating the target table. Wherein, the update can be deletion, addition, modification, etc. A timestamp may be data that can indicate the complete verifiability of a piece of data that already exists at a particular point in time to demonstrate the time of generation of certain data.
In the embodiment of the invention, the node updating time of the data management node and the data table updating time of the management data table are determined, and the node updating time and the data table updating time can be determined by establishing a mapping relation with a timestamp through a timestamp protocol, a linear link protocol, a distributed protocol or a binary protocol and the like.
And S120, if at least one of the node updating time and the data table updating time is longer than the pre-stored verification time of the data management node, generating a content abstract based on the configuration content of the data management node, and recording the content abstract generating time.
The pre-stored check time can be a timestamp saved when the data management node completes the validity check last time. The configuration content can be set according to actual needs, and the data management node can be configured by setting SQL conditions. The content digest may be digest information obtained by converting configuration content of the data administration node through a message digest algorithm. The message digest algorithm can change a message with any length into a digital digest of a short message with a fixed length, namely, a one-way Hash function is adopted to convert a plaintext to be encrypted into a string of ciphertext with a fixed length. The message digest algorithm may be an MD algorithm, an SHA algorithm, or an MAC algorithm, and the like, and the type of the message digest algorithm is not limited in the embodiment of the present invention.
In the embodiment of the invention, at least one of the node updating time and the data table updating time is greater than the pre-stored checking time of the data management node, which means that the data management node and the data table are updated again after the data management node completes the validity check last time. In this case, the updated content digest and the content digest generated when the data governance node completes the validity check last time need to be compared to determine whether the configuration content of the data governance node and the data information in the governance data table are updated. Therefore, if at least one of the node updating time and the data table updating time is greater than the pre-stored verification time of the data management node, generating a content summary based on the configuration content of the data management node, and recording the content summary generating time.
And S130, if the generated content abstract is consistent with the information of the pre-stored content abstract, determining that the data management node passes the verification.
The pre-stored content abstract can be a content abstract generated based on configuration content of the data management node at the time when the data management node completes validity check last time.
In the embodiment of the invention, the generated content abstract is consistent with the information of the pre-stored content abstract, which means that the configuration content of the data management node and the data information in the management data table are not updated, and the data management node passes the verification.
According to the technical scheme of the embodiment of the invention, the node updating time of the data management node and the data table updating time of the management data table are determined when the data management node operates on line; if at least one of the node updating time and the data table updating time is longer than the pre-stored verification time of the data management node, generating a content abstract based on the configuration content of the data management node, and recording the content abstract generating time; and if the generated content abstract is consistent with the information of the pre-stored content abstract, determining that the data management node passes the verification. According to the technical scheme, the verification speed of the data management node can be improved, and the system computing power is saved.
Example two
Fig. 2 is a flowchart of a verification method for a data management node according to a second embodiment of the present invention, and the second embodiment is optimized based on the foregoing embodiment. The concrete optimization is as follows: before determining the node update time for a data abatement node and the data table update time for an abatement data table, the method further comprises: and if the data management node is detected to be verified to be complete, generating a pre-stored content abstract by adopting a preset format for the configuration content of the data management node, and recording the pre-stored verification time.
As shown in fig. 2, the method of this embodiment specifically includes the following steps:
s210, if the data management node is detected to be verified to be complete, generating a pre-stored content abstract by adopting a preset format for the configuration content of the data management node, and recording pre-stored verification time.
The validity check of the data management nodes can be completed by the means that the pre-stored content summaries of the data management nodes are consistent with the content summaries generated after the data management nodes or the management data table are updated. The preset format may be a content digest generation rule determined according to different message digest algorithms. The pre-stored check time may be a timestamp saved by the data administration node when detecting that the data administration node completes validity check this time.
And S220, if the configuration content updating operation of the data management node is detected, processing the configuration content based on the updated configuration content by adopting a preset format so as to update the abstract of the prestored content, and updating the prestored verification time based on the occurrence time of the updating operation.
The updating operation may include operations of screening, adding, deleting or modifying data or data types in the data source table according to the configuration content of the data governance node.
In the embodiment of the invention, if the configuration content updating operation of the data management node is detected, processing is carried out by adopting a preset format based on the updated configuration content to obtain an updated content abstract, and the occurrence time of the updating operation is recorded; and replacing the pre-stored content abstract with the updated content abstract, and replacing the pre-stored verification time with the occurrence time of the updating operation so as to update the configuration content of the data management node.
In this embodiment, optionally, the preset format is that the hash value is calculated by using an SHA-256 algorithm for the configuration content of the data governance node.
The SHA-256 algorithm is one of secure hash algorithms, and the SHA-256 algorithm generates a 256-bit hash value, i.e., a content digest, for a message of any length, which is equivalent to an array of 32 bytes in length. Specifically, the configuration content length of the data management node can be subjected to bit complementing by adopting an SHA-256 algorithm, so that the remainder of the length after modulus is taken for 512 is 448; after the data after bit complementing is added with the length of 64 bits, the data is divided into n data blocks according to 512 bits and is circularly processed for n times, and finally a 256-bit information abstract, namely a content abstract corresponding to the configuration content of the data management node is obtained.
Through the arrangement, the content abstract integrity can be guaranteed, and meanwhile, the safety of the content abstract integrity is improved.
And S230, determining the node updating time of the data governance node and the data table updating time of the governance data table when the data governance node operates on line.
S240, if at least one of the node updating time and the data table updating time is larger than the pre-stored verification time of the data management node, generating a content abstract based on the configuration content of the data management node, and recording the content abstract generating time.
And S250, if the generated content abstract is consistent with the information of the pre-stored content abstract, determining that the data management node passes the verification.
According to the technical scheme of the embodiment of the invention, before the node updating time of the data management node and the data table updating time of the data management data table are determined, if the data management node is detected to be verified to be legal, the configuration content of the data management node is generated into the pre-stored content abstract in a preset format, and the pre-stored verification time is recorded. According to the technical scheme, the verification speed and the safety of the data management node can be improved, and the system calculation power is saved.
EXAMPLE III
Fig. 3 is a flowchart of a verification method for a data management node according to a second embodiment of the present invention, and the second embodiment of the present invention is optimized based on the above embodiments. The concrete optimization is as follows: if the generated content abstract is inconsistent with the information of the pre-stored content abstract, performing logic verification based on the configuration content of the data management node and the content of a management data table; and after the verification is passed, generating a content abstract based on the verified configuration content so as to update the pre-stored content abstract, and updating the pre-stored verification time based on the occurrence time of the updating operation.
As shown in fig. 3, the method of this embodiment specifically includes the following steps:
s310, when the data governance nodes are operated on line, determining node updating time of the data governance nodes and data table updating time of the governance data tables.
S320, if at least one of the node updating time and the data table updating time is larger than the pre-stored verification time of the data management node, generating a content abstract based on the configuration content of the data management node, and recording the content abstract generating time.
And S330, if the generated content abstract is consistent with the information of the pre-stored content abstract, determining that the data management node passes the verification.
And S340, if the generated content abstract is inconsistent with the information of the pre-stored content abstract, performing logic verification based on the configuration content of the data management node and the content of the management data table.
The generated content abstract is inconsistent with the information of the pre-stored content abstract, the field type is inconsistent, the field mapping type is inconsistent, and the configured SQL condition is illegal. The logical check may be a process of checking according to corresponding data in the data source table and the target table. In this embodiment of the present invention, optionally, performing logic verification based on the configuration content of the data governance node and the content of the governance data table includes: and performing logic check on at least one of connectivity, field content integrity, field type correspondence, field mapping matching and SQL condition validity of the data table source node and the data table target node based on the configuration content of the data management node and the content of the management data table.
The connectivity of the data table source node and the target node can be that data can be successfully connected to the target node through the data table node and the data governance node. The field content integrity may be the length of the field. For example, there are 5 fields in the data source table, and there are 4 fields corresponding to the data source table in the target table, which indicate that the data administration node fails to check. The field type correspondence may be that the type of a column in the data source table and the target table and the type of all data of the column are corresponding. For example, if a column name in the target table is "age", the column data field type should be a character string type, and if the data in the column is actually recorded as a text type such as "zhangsan", "liquan", etc., it indicates that the data administration node check fails. The field mapping match may be a correspondence of a data source table and a target table. For example, if the data source table is of a character string type, the corresponding target table is also of the character string type, otherwise, the data governance node check is not passed. The SQL conditional validity can be that the SQL condition matches the corresponding data type. For example, the target table is 50 pieces of data in a certain data source table, and a constraint condition is added at this time, that is, the number of data is greater than 50 pieces, and the constraint condition is SQL conditional validity check. Through the arrangement, the checking accuracy of the data management node can be improved.
And S350, after the verification is passed, generating a content abstract based on the verified configuration content so as to update the pre-stored content abstract, and updating the pre-stored verification time based on the occurrence time of the updating operation.
According to the technical scheme of the embodiment of the invention, the node updating time of the data management node and the data table updating time of the management data table are determined when the data management node operates on line; if at least one of the node updating time and the data table updating time is longer than the pre-stored verification time of the data management node, generating a content abstract based on the configuration content of the data management node, and recording the content abstract generating time; if the generated content abstract is consistent with the information of the pre-stored content abstract, determining that the data management node passes the verification; if the generated content abstract is inconsistent with the information of the pre-stored content abstract, performing logic verification based on the configuration content of the data management node and the content of a management data table; and after the verification is passed, generating a content abstract based on the verified configuration content so as to update the pre-stored content abstract, and updating the pre-stored verification time based on the occurrence time of the updating operation. According to the technical scheme, the verification speed of the data management node can be increased, the verification accuracy and safety can be improved, and the system calculation power can be saved.
Example four
Fig. 4 is a schematic structural diagram of a data management node verification apparatus according to a fifth embodiment of the present invention, which is capable of executing a data management node verification method according to any embodiment of the present invention, and has functional modules and beneficial effects corresponding to the execution method. As shown in fig. 4, the apparatus includes:
an update time determination module 410, configured to determine node update time of a data management node and data table update time of a management data table when the data management node operates online;
a content abstract determining module 420, configured to generate a content abstract based on the configuration content of the data management node and record content abstract generation time if at least one of the node update time and the data table update time is greater than pre-stored verification time of the data management node;
and the data management node checking module 430 is configured to determine that the data management node passes checking if the generated content abstract is consistent with the information of the pre-stored content abstract.
According to the technical scheme of the embodiment of the invention, the node updating time of the data management node and the data table updating time of the management data table are determined when the data management node operates on line; if at least one of the node updating time and the data table updating time is longer than the pre-stored verification time of the data management node, generating a content abstract based on the configuration content of the data management node, and recording the content abstract generating time; and if the generated content abstract is consistent with the information of the pre-stored content abstract, determining that the data management node passes the check. According to the technical scheme, the verification speed of the data management node can be improved, and the system computing power is saved.
Further, the apparatus further comprises:
and the pre-stored content abstract determining module is used for generating a pre-stored content abstract by adopting a preset format for the configuration content of the data management node and recording the pre-stored verification time if the data management node is detected to be verified to be complete in validity before the node updating time of the data management node and the data table updating time of the data management data table are determined.
Further, the apparatus further comprises:
and the configuration updating module is used for generating a pre-stored content abstract by adopting a preset format for the configuration content of the data management node, and if the configuration content updating operation of the data management node is detected, processing the pre-stored content abstract by adopting the preset format based on the updated configuration content so as to update the pre-stored content abstract, and updating the pre-stored verification time based on the occurrence time of the updating operation.
Further, the preset format is that the hash value is calculated by adopting an SHA-256 algorithm for the configuration content of the data management node.
Further, the apparatus further comprises:
the logic verification module is used for performing logic verification based on the configuration content of the data management node and the content of the management data table if the generated content abstract is inconsistent with the information of the pre-stored content abstract;
and the pre-stored content abstract updating module is used for generating a content abstract based on the verified configuration content after the verification is passed so as to update the pre-stored content abstract, and updating the pre-stored verification time based on the occurrence time of the updating operation.
Further, the logic check module includes:
and the logic checking unit is used for carrying out logic checking on at least one of connectivity, field content integrity, field type correspondence, field mapping matching and SQL condition validity of the source node and the target node of the data table based on the configuration content of the data governance node and the content of the governance data table.
The data management node verifying device provided by the embodiment of the invention can execute the data management node verifying method provided by any embodiment of the invention, and has the corresponding functional modules and beneficial effects of the executing method.
EXAMPLE five
FIG. 5 illustrates a schematic diagram of an electronic device 10 that may be used to implement an embodiment of the invention. Electronic devices are intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device may also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smart phones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be exemplary only, and are not meant to limit implementations of the inventions described and/or claimed herein.
As shown in fig. 5, the electronic device 10 includes at least one processor 11, and a memory communicatively connected to the at least one processor 11, such as a Read Only Memory (ROM)12, a Random Access Memory (RAM)13, and the like, wherein the memory stores a computer program executable by the at least one processor, and the processor 11 may perform various suitable actions and processes according to the computer program stored in the Read Only Memory (ROM)12 or the computer program loaded from the storage unit 18 into the Random Access Memory (RAM) 13. In the RAM 13, various programs and data necessary for the operation of the electronic apparatus 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other via a bus 14. An input/output (I/O) interface 15 is also connected to bus 14.
A number of components in the electronic device 10 are connected to the I/O interface 15, including: an input unit 16 such as a keyboard, a mouse, or the like; an output unit 17 such as various types of displays, speakers, and the like; a storage unit 18 such as a magnetic disk, an optical disk, or the like; and a communication unit 19 such as a network card, modem, wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information/data with other devices via a computer network such as the internet and/or various telecommunication networks.
In some embodiments, the verification of the method data abatement nodes may be implemented as a computer program tangibly embodied in a computer readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program may be loaded and/or installed onto the electronic device 10 via the ROM 12 and/or the communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of the verification of the method data administration nodes described above may be performed. Alternatively, in other embodiments, the processor 11 may be configured to perform the verification method of the data abatement node by any other suitable means (e.g., by means of firmware).
Various implementations of the systems and techniques described here above may be implemented in digital electronic circuitry, integrated circuitry, Field Programmable Gate Arrays (FPGAs), Application Specific Integrated Circuits (ASICs), Application Specific Standard Products (ASSPs), system on a chip (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include: implemented in one or more computer programs that are executable and/or interpretable on a programmable system including at least one programmable processor, which may be special or general purpose, receiving data and instructions from, and transmitting data and instructions to, a storage system, at least one input device, and at least one output device.
A computer program for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus, such that the computer programs, when executed by the processor, cause the functions/acts specified in the flowchart and/or block diagram block or blocks to be performed. A computer program can execute entirely on a machine, partly on a machine, as a stand-alone software package partly on a machine and partly on a remote machine or entirely on a remote machine or server.
In the context of the present invention, a computer-readable storage medium may be a tangible medium that can contain, or store a computer program for use by or in connection with an instruction execution system, apparatus, or device. A computer readable storage medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. Alternatively, the computer readable storage medium may be a machine readable signal medium. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
To provide for interaction with a user, the systems and techniques described here can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user; and a keyboard and a pointing device (e.g., a mouse or a trackball) by which a user can provide input to the electronic device. Other kinds of devices may also be used to provide for interaction with a user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form, including acoustic, speech, or tactile input.
The systems and techniques described here can be implemented in a computing system that includes a back-end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back-end, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local Area Networks (LANs), Wide Area Networks (WANs), blockchain networks, and the internet.
The computing system may include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The server can be a cloud server, also called a cloud computing server or a cloud host, and is a host product in a cloud computing service system, so that the defects of high management difficulty and weak service expansibility in the traditional physical host and VPS service are overcome.
It should be understood that various forms of the flows shown above may be used, with steps reordered, added, or deleted. For example, the steps described in the present invention may be executed in parallel, sequentially, or in different orders, and are not limited herein as long as the desired results of the technical solution of the present invention can be achieved.
The above-described embodiments should not be construed as limiting the scope of the invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions may be made in accordance with design requirements and other factors. Any modification, equivalent replacement, and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims (10)
1. A method for verifying a data administration node, the method comprising:
when the data management node operates on line, determining the node updating time of the data management node and the data table updating time of a management data table;
if at least one of the node updating time and the data table updating time is longer than the pre-stored verification time of the data management node, generating a content abstract based on the configuration content of the data management node, and recording the content abstract generating time;
and if the generated content abstract is consistent with the information of the pre-stored content abstract, determining that the data management node passes the verification.
2. The method of claim 1, wherein prior to determining the node update time for a data abatement node and the data table update time for an abatement data table, the method further comprises:
and if the data management node is detected to be verified to be complete, generating a pre-stored content abstract by adopting a preset format for the configuration content of the data management node, and recording the pre-stored verification time.
3. The method of claim 2, wherein after generating the pre-stored content summary of the configuration content of the data governance node in the preset format, the method further comprises:
and if the configuration content updating operation of the data management node is detected, processing the configuration content by adopting a preset format based on the updated configuration content so as to update the abstract of the prestored content, and updating the prestored verification time based on the occurrence time of the updating operation.
4. The method of claim 2 or 3, wherein the preset format is that the hash value is calculated by using SHA-256 algorithm for the configuration content of the data governance node.
5. The method of claim 1, further comprising:
if the generated content abstract is inconsistent with the information of the pre-stored content abstract, performing logic verification based on the configuration content of the data management node and the content of a management data table;
and after the verification is passed, generating a content abstract based on the verified configuration content so as to update the pre-stored content abstract, and updating the pre-stored verification time based on the occurrence time of the updating operation.
6. The method of claim 5, wherein performing a logical check based on the configuration content of the data abatement node and the content of an abatement data table comprises:
and performing logic check on at least one of connectivity, field content integrity, field type correspondence, field mapping matching and SQL condition validity of the data table source node and the data table target node based on the configuration content of the data management node and the content of the management data table.
7. A data administration node verification apparatus, the apparatus comprising:
the updating time determining module is used for determining the node updating time of the data management node and the data table updating time of the management data table when the data management node operates on line;
the content abstract determining module is used for generating a content abstract based on the configuration content of the data management node and recording the content abstract generating time if at least one of the node updating time and the data table updating time is greater than the pre-stored verification time of the data management node;
and the data management node checking module is used for determining that the data management node passes the checking if the generated content abstract is consistent with the information of the pre-stored content abstract.
8. The apparatus of claim 7, further comprising:
and the pre-stored content abstract determining module is used for generating a pre-stored content abstract by adopting a preset format for the configuration content of the data management node and recording the pre-stored verification time if the data management node is detected to be verified to be complete in validity before the node updating time of the data management node and the data table updating time of the data management data table are determined.
9. An electronic device, characterized in that the electronic device comprises:
at least one processor; and
a memory communicatively coupled to the at least one processor; wherein the content of the first and second substances,
the memory stores a computer program executable by the at least one processor to enable the at least one processor to perform the method of verifying a data abatement node of any one of claims 1 to 7.
10. A computer readable storage medium having stored thereon computer instructions for causing a processor to perform the method of checking a data governance node of any one of claims 1 to 7 when executed.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210251022.XA CN114611155B (en) | 2022-03-15 | 2022-03-15 | Data management node verification method, device, equipment and medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210251022.XA CN114611155B (en) | 2022-03-15 | 2022-03-15 | Data management node verification method, device, equipment and medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114611155A true CN114611155A (en) | 2022-06-10 |
| CN114611155B CN114611155B (en) | 2022-12-13 |
Family
ID=81862884
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210251022.XA Active CN114611155B (en) | 2022-03-15 | 2022-03-15 | Data management node verification method, device, equipment and medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114611155B (en) |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20200007343A1 (en) * | 2018-06-28 | 2020-01-02 | Blockchain Integrated Partners, Llc | Systems and methods for data validation and assurance |
| CN111563130A (en) * | 2020-07-15 | 2020-08-21 | 成都无右区块链科技有限公司 | Data credible data management method and system based on block chain technology |
| CN112597485A (en) * | 2021-03-01 | 2021-04-02 | 腾讯科技（深圳）有限公司 | Information checking method, device and equipment based on block chain and storage medium |
| CN112800095A (en) * | 2021-04-13 | 2021-05-14 | 腾讯科技（深圳）有限公司 | Data processing method, device, equipment and storage medium |
| CN112966311A (en) * | 2019-03-15 | 2021-06-15 | 创新先进技术有限公司 | Intelligent contract checking method and device and electronic equipment |
| CN113286004A (en) * | 2021-03-29 | 2021-08-20 | 腾讯科技（深圳）有限公司 | Node management method, device, equipment and storage medium based on block chain network |
| WO2021232590A1 (en) * | 2020-05-22 | 2021-11-25 | 平安国际智慧城市科技股份有限公司 | User verification method and apparatus based on private chain, and computer device and storage medium |
| CN114048206A (en) * | 2021-10-29 | 2022-02-15 | 济南浪潮数据技术有限公司 | Configuration consistency checking method, system and device for large-scale cluster |
-
2022
- 2022-03-15 CN CN202210251022.XA patent/CN114611155B/en active Active
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20200007343A1 (en) * | 2018-06-28 | 2020-01-02 | Blockchain Integrated Partners, Llc | Systems and methods for data validation and assurance |
| CN112966311A (en) * | 2019-03-15 | 2021-06-15 | 创新先进技术有限公司 | Intelligent contract checking method and device and electronic equipment |
| WO2021232590A1 (en) * | 2020-05-22 | 2021-11-25 | 平安国际智慧城市科技股份有限公司 | User verification method and apparatus based on private chain, and computer device and storage medium |
| CN111563130A (en) * | 2020-07-15 | 2020-08-21 | 成都无右区块链科技有限公司 | Data credible data management method and system based on block chain technology |
| CN112597485A (en) * | 2021-03-01 | 2021-04-02 | 腾讯科技（深圳）有限公司 | Information checking method, device and equipment based on block chain and storage medium |
| CN113286004A (en) * | 2021-03-29 | 2021-08-20 | 腾讯科技（深圳）有限公司 | Node management method, device, equipment and storage medium based on block chain network |
| CN112800095A (en) * | 2021-04-13 | 2021-05-14 | 腾讯科技（深圳）有限公司 | Data processing method, device, equipment and storage medium |
| CN114048206A (en) * | 2021-10-29 | 2022-02-15 | 济南浪潮数据技术有限公司 | Configuration consistency checking method, system and device for large-scale cluster |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114611155B (en) | 2022-12-13 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN115114599A (en) | Method, device and equipment for processing database watermark and storage medium | |
| CN115617888A (en) | Data import method, device, equipment, storage medium and product | |
| CN115329759B (en) | Information processing method, device, equipment and storage medium | |
| CN114611155B (en) | Data management node verification method, device, equipment and medium | |
| CN116089985A (en) | Encryption storage method, device, equipment and medium for distributed log | |
| CN115632866A (en) | Message desensitization method, device, equipment and medium based on FPGA | |
| CN116361591A (en) | Content auditing method, device, electronic equipment and computer readable storage medium | |
| CN115454971A (en) | Data migration method and device, electronic equipment and storage medium | |
| CN115639966A (en) | Data writing method and device, terminal equipment and storage medium | |
| CN112887426B (en) | Information stream pushing method and device, electronic equipment and storage medium | |
| CN115328898A (en) | Data processing method and device, electronic equipment and medium | |
| CN115567624A (en) | Message processing method and device, electronic equipment and medium | |
| CN115827588A (en) | Method, device, equipment and storage medium for generating business global serial number | |
| CN117056310A (en) | Data migration method and device, electronic equipment and storage medium | |
| CN115994119A (en) | File access processing method, device, equipment and storage medium | |
| CN115168372A (en) | Data updating method, device, equipment, storage medium and product | |
| CN116841982A (en) | Database migration method, device, equipment, storage medium and program product | |
| CN118171676A (en) | Graphic code generation and storage method, device, equipment, medium and product | |
| CN113722316A (en) | Model processing method and device, electronic equipment and computer readable medium | |
| CN117331475A (en) | Task creation method, device, equipment and storage medium | |
| CN116050351A (en) | Structure comparison method, device and equipment of logic expression and storage medium | |
| CN115981657A (en) | Code generation method and device, electronic equipment and readable medium | |
| CN116400956A (en) | Code merging information determining method and device, electronic equipment and storage medium | |
| CN116841549A (en) | Layer processing method and device, electronic equipment and storage medium | |
| CN114416662A (en) | File comparison method and device, electronic equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |