CN117033362A - Data self-consistency monitoring method and device and electronic equipment - Google Patents

Data self-consistency monitoring method and device and electronic equipment Download PDF

Info

Publication number
CN117033362A
CN117033362A CN202310996731.5A CN202310996731A CN117033362A CN 117033362 A CN117033362 A CN 117033362A CN 202310996731 A CN202310996731 A CN 202310996731A CN 117033362 A CN117033362 A CN 117033362A
Authority
CN
China
Prior art keywords
data
sql script
checking
sql
script
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202310996731.5A
Other languages
Chinese (zh)
Inventor
吴闯硕
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Ping An Bank Co Ltd
Original Assignee
Ping An Bank Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Ping An Bank Co Ltd filed Critical Ping An Bank Co Ltd
Priority to CN202310996731.5A priority Critical patent/CN117033362A/en
Publication of CN117033362A publication Critical patent/CN117033362A/en
Pending legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/21—Design, administration or maintenance of databases
    • G06F16/215—Improving data quality; Data cleansing, e.g. de-duplication, removing invalid entries or correcting typographical errors
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q40/00—Finance; Insurance; Tax strategies; Processing of corporate or income taxes
    • G06Q40/02—Banking, e.g. interest calculation or account maintenance
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
    • Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management

Landscapes

  • Engineering & Computer Science (AREA)
  • Business, Economics & Management (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Finance (AREA)
  • Accounting & Taxation (AREA)
  • General Physics & Mathematics (AREA)
  • Databases & Information Systems (AREA)
  • Quality & Reliability (AREA)
  • General Engineering & Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Development Economics (AREA)
  • Economics (AREA)
  • Marketing (AREA)
  • Strategic Management (AREA)
  • Technology Law (AREA)
  • General Business, Economics & Management (AREA)
  • Debugging And Monitoring (AREA)

Abstract

The application provides a method, a device and electronic equipment for monitoring data self-consistency, comprising the following steps: acquiring SQL scripts written based on a data inspection specification and configuration information of each SQL script; when the target SQL script reaches the corresponding monitoring time, automatically checking the data item corresponding to the target SQL script by adopting the target SQL script to obtain a checking result; and sending the information of the abnormal data item in the checking result to a processor corresponding to the target SQL script, and sending the information of the abnormal data item to a problem pool according to the configuration information of whether the target SQL script needs to be tracked. According to the data self-consistency monitoring method, the automatic monitoring platform can automatically check each data item in the data checking standard by adopting the SQL script, the data item in the data checking standard is full, namely the data checking range is large enough, the accuracy of the automatic checking is high, and the automatic monitoring method is intelligent, reliable and high in efficiency.

Description

Data self-consistency monitoring method and device and electronic equipment
Technical Field
The present application relates to the field of data inspection, and in particular, to a method and an apparatus for monitoring data self-consistency, and an electronic device.
Background
With the development of a financial business system, daily transaction amount of a bank core system is continuously improved, system functions are gradually huge, and developers need to meet various functions which are gradually enriched and cannot influence the original functions, so that after each time of requirement development, the integrity, accuracy, consistency and reliability of data are required to be checked. Data self-consistency checking becomes a necessary skill for most developers.
The data self-consistency checking mode currently used by developers is manual checking, and before checking, the information of the checking range, the data type, the size, the format and the like needs to be confirmed, a tool for checking the data is selected, and a checking method is determined. When in inspection, necessary processing and modification are needed according to the inspection requirement, and inspection results are recorded in a log, so that subsequent data analysis and optimization are facilitated. After the inspection result is generated, an experienced person needs to be searched for data review.
Most of the conventional inspection methods require manual operation, and are inevitably omitted, such as insufficient inspection range, missing inspection data, defective inspection methods, and the like. The core system is used as a full row A system, so that the data inspection is more careful.
In conclusion, the existing data self-consistency checking method has the technical problems of poor intelligence and poor accuracy.
Disclosure of Invention
In view of the above, the application aims to provide a data self-consistency monitoring method, a device and electronic equipment, so as to solve the technical problems of poor intelligence and poor accuracy of the existing data self-consistency checking method.
In a first aspect, an embodiment of the present application provides a method for monitoring data self-consistency, which is applied to an automated monitoring platform, and includes:
acquiring SQL scripts written based on data inspection specifications and configuration information of the SQL scripts, wherein the data inspection specifications are summarized in advance and are used for automatically inspecting corresponding data items in the data inspection specifications, and the configuration information comprises: monitoring time, responsible person, processing person and whether tracking is needed;
when the target SQL script reaches the corresponding monitoring time, automatically checking the data item corresponding to the target SQL script by adopting the target SQL script to obtain a checking result;
and sending the information of the abnormal data item in the checking result to a processor corresponding to the target SQL script, and sending the information of the abnormal data item to a problem pool according to the configuration information of whether the target SQL script needs to be tracked.
Further, the data items in the data inspection specification may be updated as needed.
Further, the SQL script includes: basic information of corresponding data items, expected values of corresponding data items and comparison rules of actual values.
Further, after acquiring the SQL scripts written based on the data inspection specification, before acquiring the configuration information of each SQL script, the method further includes:
analyzing and pre-executing the SQL script to determine the accuracy of the SQL script.
Further, after determining the accuracy of the SQL script, the method further comprises:
a unique script number is added to each SQL script.
Further, each SQL script corresponds to a custom priority, wherein the priority of the SQL script is custom defined according to the priority of the data item to be checked and the priority of the scene.
Further, sending the information of the abnormal data item in the inspection result to a handler corresponding to the target SQL script, including:
and sending the information of the abnormal data items in the inspection results to a processor corresponding to the target SQL script in a mail form, wherein the same mail can comprise the information of the abnormal data items in the inspection results corresponding to a plurality of SQL scripts.
In a second aspect, an embodiment of the present application further provides a device for monitoring data self-consistency, which is applied to an automated monitoring platform, and includes:
the system comprises an acquisition unit, a configuration information generation unit and a configuration information generation unit, wherein the acquisition unit is used for acquiring SQL scripts written based on data inspection specifications and configuration information of the SQL scripts, the data inspection specifications are summarized in advance and are used for automatically inspecting corresponding data items in the data inspection specifications, and the configuration information comprises: monitoring time, responsible person, processing person and whether tracking is needed;
the automatic checking unit is used for automatically checking the data item corresponding to the target SQL script by adopting the target SQL script when the target SQL script reaches the corresponding monitoring time, so as to obtain a checking result;
and the sending unit is used for sending the information of the abnormal data item in the checking result to a processor corresponding to the target SQL script and sending the information of the abnormal data item to a problem pool according to the configuration information of whether the target SQL script needs to be tracked.
In a third aspect, an embodiment of the present application further provides an electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor, where the processor implements the steps of the method according to any one of the first aspects when the processor executes the computer program.
In a fourth aspect, embodiments of the present application also provide a computer-readable storage medium storing machine-executable instructions which, when invoked and executed by a processor, cause the processor to perform the method of any one of the first aspects.
In an embodiment of the present application, a method for monitoring data self-consistency is provided, which is applied to an automated monitoring platform, and includes: acquiring SQL scripts written based on data inspection specifications and configuration information of the SQL scripts, wherein the data inspection specifications are summarized in advance and are used for automatically inspecting corresponding data items in the data inspection specifications, and the configuration information comprises: monitoring time, responsible person, processing person and whether tracking is needed; when the target SQL script reaches the corresponding monitoring time, automatically checking the data item corresponding to the target SQL script by adopting the target SQL script to obtain a checking result; and sending the information of the abnormal data item in the checking result to a processor corresponding to the target SQL script, and sending the information of the abnormal data item to a problem pool according to the configuration information of whether the target SQL script needs to be tracked. According to the data self-consistency monitoring method disclosed by the application, the automatic monitoring platform can automatically check each data item in the data checking specification by adopting the SQL script, the data item in the data checking specification is sufficiently comprehensive, namely the data checking range is sufficiently large, the automatic checking accuracy is high, the intelligent and reliable performance is improved, the efficiency is high, and the technical problems of poor intelligence and poor accuracy of the existing data self-consistency checking method are solved.
Drawings
In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings that are needed in the description of the embodiments or the prior art will be briefly described, and it is obvious that the drawings in the description below are some embodiments of the present application, and other drawings can be obtained according to the drawings without inventive effort for a person skilled in the art.
Fig. 1 is a flowchart of a method for monitoring data self-consistency according to an embodiment of the present application;
FIG. 2 is a schematic diagram of a process flow of an automated monitoring platform according to an embodiment of the present application;
fig. 3 is a schematic diagram of a data self-consistency monitoring device according to an embodiment of the present application;
fig. 4 is a schematic diagram of an electronic device according to an embodiment of the present application.
Detailed Description
The technical solutions of the present application will be clearly and completely described in connection with the embodiments, and it is apparent that the described embodiments are some embodiments of the present application, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the application without making any inventive effort, are intended to be within the scope of the application.
The existing data self-consistency inspection needs manual operation, and has poor intelligence and poor accuracy.
Based on the above, in the data self-consistency monitoring method, the automatic monitoring platform can automatically check each data item in the data checking specification by adopting the SQL script, and the data item in the data checking specification is comprehensive enough, namely the data checking range is large enough, and the automatic checking accuracy is high, and the automatic checking method is more intelligent and reliable and has high efficiency.
For the sake of understanding the present embodiment, first, a method for monitoring data self-consistency disclosed in the present embodiment is described in detail.
Embodiment one:
according to an embodiment of the present application, there is provided an embodiment of a method of monitoring data self-consistency, it being noted that the steps shown in the flowchart of the figures may be performed in a computer system, such as a set of computer executable instructions, and that although a logical order is shown in the flowchart, in some cases the steps shown or described may be performed in an order other than that shown or described herein.
Fig. 1 is a flowchart of a method for monitoring data self-consistency according to an embodiment of the present application, as shown in fig. 1, the method includes the following steps:
step S102, acquiring SQL scripts written based on data inspection specifications and configuration information of the SQL scripts, wherein the data inspection specifications are summarized in advance and need to be inspected, the SQL scripts are used for automatically inspecting corresponding data items in the data inspection specifications, and the configuration information comprises: monitoring time, responsible person, processing person and whether tracking is needed;
in the embodiment of the application, the data self-consistency monitoring method can be applied to an automatic monitoring platform, and the data inspection specifications are summarized in advance and need to be inspected.
Specifically, the conventional data item checking range is only to realize data item checking processing within the range according to the updating requirement of the data item, after analyzing, sorting, summarizing and summarizing the existing codes, the inventor roughly knows important data items and the consistency problem of the related data items by combining the actual data use conditions, forms a data checking specification (considering self-consistency of fields, self-consistency of data logic of the same row, self-consistency of data of different rows, self-consistency of data of different tables and the like), performs full checking on the summarized data items (including the data items and tables related to the data items), and the checking range is wider and the related data is more comprehensive.
In addition, the traditional checking mode has very high precision required for the early preparation of the data item to be checked, and meanwhile, the capability and the careful degree of the recheck person are also particularly checked, so that the work becomes an irreplaceable matter, the recheck person cannot draw a body to do deeper thinking, and other people cannot help to process. At present, by combining the functions of automatic monitoring and developing the sensitivity to the data items, what is seemingly irreplaceable is a turning machine, namely, automatic checking is realized by developing SQL scripts written based on the data checking specification.
Specifically, after classifying and summarizing based on the data inspection specifications of the pre-arrangement, a developer writes an SQL script for automatically inspecting the data items in the SQL script.
When the method is implemented, the SQL scripts are imported into the automatic monitoring platform, and after the pre-execution of the SQL scripts is completed, a user can configure configuration information of each SQL script in the automatic monitoring platform. For example, the monitoring time is configured, different monitoring time can be set for data items in different service time periods, monitoring can be configured to be performed every half an hour for data items with higher timeliness, and for data items generated in batch or data items pushed at regular time, time periods of 18 days, 8 days in the interval, 12 hours in the early morning and the like can be set, so that timeliness of monitoring is guaranteed, and the fact that the system cannot receive excessive requests at the same time to cause downtime is also guaranteed, and the pressure of the system is reduced. For some special scenarios, such as interest, reservation, moistening, etc. The date is also required, and the method can be realized through the configuration of an automatic monitoring platform. When the configuration of the monitoring time is carried out, the configuration selection is carried out at the corresponding position of each SQL script in the automatic monitoring platform, the responsible person and the processing person of the SQL script can be configured, after the information of the abnormal data item is checked, the processing person is required to be immediately notified to carry out analysis, and when the SQL script is configured, the responsible person and the processing person of the SQL are added, and the responsible person is mainly responsible for rewriting and maintaining the SQL script; the processor mainly analyzes and cleans the abnormal data items. The notification mechanism can use a mail notification mode, and can present records of 200 abnormal data items in the mail, thereby meeting most of scenes. Meanwhile, the same mail can display the inspection results of a plurality of SQL scripts, so that the problem that the mail is too many and is unopposed is avoided.
In actual work, abnormal data items which are monitored may not be processed timely, so that when SQL is configured, whether the SQL needs to be tracked (namely, the abnormal data items are put into a problem pool when the SQL needs to be tracked) can be selected, and after the SQL is put into the problem pool, all the abnormal data items which are not analyzed can be queried through a query interface provided by a platform for subsequent analysis.
Step S104, when the target SQL script reaches the corresponding monitoring time, automatically checking the data item corresponding to the target SQL script by adopting the target SQL script to obtain a checking result;
step S106, the information of the abnormal data item in the checking result is sent to a processor corresponding to the target SQL script, and the information of the abnormal data item is sent to the problem pool according to the configuration information of whether the target SQL script needs to be tracked.
The process flow of the automated monitoring platform is shown in fig. 2.
In an embodiment of the present application, a method for monitoring data self-consistency is provided, which is applied to an automated monitoring platform, and includes: acquiring SQL scripts written based on data inspection specifications and configuration information of the SQL scripts, wherein the data inspection specifications are summarized in advance and are used for automatically inspecting corresponding data items in the data inspection specifications, and the configuration information comprises: monitoring time, responsible person, processing person and whether tracking is needed; when the target SQL script reaches the corresponding monitoring time, automatically checking the data item corresponding to the target SQL script by adopting the target SQL script to obtain a checking result; and sending the information of the abnormal data item in the checking result to a processor corresponding to the target SQL script, and sending the information of the abnormal data item to a problem pool according to the configuration information of whether the target SQL script needs to be tracked. According to the data self-consistency monitoring method disclosed by the application, the automatic monitoring platform can automatically check each data item in the data checking specification by adopting the SQL script, the data item in the data checking specification is sufficiently comprehensive, namely the data checking range is sufficiently large, the automatic checking accuracy is high, the intelligent and reliable performance is improved, the efficiency is high, and the technical problems of poor intelligence and poor accuracy of the existing data self-consistency checking method are solved.
In an alternative embodiment of the application, the data items in the data inspection specification may be updated as required.
In particular, data items in the data inspection specification may be incrementally updated according to business requirements.
In an alternative embodiment of the present application, the SQL script comprises: basic information of corresponding data items, expected values of corresponding data items and comparison rules of actual values.
Specifically, the basic information may include: customer account number, name, currency, number, etc. to facilitate locating and finding the detected anomalous data item. The comparison rule of the expected value and the actual value of the corresponding data item is used for intuitively finding the abnormal data item during inspection.
In an alternative embodiment of the present application, after acquiring the SQL scripts written based on the data inspection specification, before acquiring the configuration information of each SQL script, the method further includes:
and analyzing and pre-executing the SQL script to determine the accuracy of the SQL script.
Specifically, the SQL script is imported into the automatic monitoring platform to perform analysis operation of the SQL script, an SQL pre-execution function is added in the automatic monitoring platform, and if analysis fails, an SQL execution result is prompted for a developer to analyze. If the execution is successful, the execution success is displayed, and the execution result is returned.
In an alternative embodiment of the present application, after determining the accuracy of the SQL script, the method further comprises:
a unique script number is added to each SQL script.
Specifically, the automated monitoring platform adds a corresponding script number to each SQL script, and uses the corresponding script number as a unique index of SQL for subsequent query.
In an alternative embodiment of the present application, each SQL script corresponds to a custom priority, where the priority of the SQL script is custom defined according to the priority of the data item it is to examine and the priority of the scene.
Specifically, in order to facilitate the prioritization of data items, priority levels L1-L4 are defined for SQL scripts, and the smaller the number is, the higher the priority level is, and the faster the corresponding request processing ages.
In an alternative embodiment of the present application, the method for sending the information of the abnormal data item in the inspection result to the handler corresponding to the target SQL script specifically includes:
and sending the information of the abnormal data items in the inspection results to a processor corresponding to the target SQL script in the form of a mail, wherein the same mail can comprise the information of the abnormal data items in the inspection results corresponding to a plurality of SQL scripts.
The data self-consistency monitoring method has the following advantages:
(1) The accuracy and the reliability of data item inspection are greatly improved, the artificial errors and missing data are reduced, and the reliability and the accuracy of the data are improved;
(2) In the actual use process, the automatic monitoring can automatically clean and classify data, filter useless information and improve analysis precision;
(3) The automatic monitoring can analyze mass data, so that abnormal and error information in data items can be quickly found, and the analysis capability of team data is improved;
(4) The automatic monitoring reduces manual operation and errors, reduces the working complexity of developers, enables the developers to be better focused on code development, and reduces maintenance cost.
Embodiment two:
the embodiment of the application also provides a data self-consistency monitoring device which is mainly used for executing the data self-consistency monitoring method provided in the first embodiment of the application, and the data self-consistency monitoring device provided in the embodiment of the application is specifically introduced below.
Fig. 3 is a schematic diagram of a data self-consistency monitoring device according to an embodiment of the present application, as shown in fig. 3, the device mainly includes: an acquisition unit 10, an automatic inspection unit 20, and a transmission unit 30, wherein:
the system comprises an acquisition unit, a configuration information generation unit and a configuration information generation unit, wherein the acquisition unit is used for acquiring SQL scripts written based on data inspection standards and configuration information of the SQL scripts, the data inspection standards are summarized in advance and are used for automatically inspecting corresponding data items in the data inspection standards, and the configuration information comprises: monitoring time, responsible person, processing person and whether tracking is needed;
the automatic checking unit is used for automatically checking the data items corresponding to the target SQL script by adopting the target SQL script when the target SQL script reaches the corresponding monitoring time, so as to obtain a checking result;
and the sending unit is used for sending the information of the abnormal data item in the checking result to a processor corresponding to the target SQL script and sending the information of the abnormal data item to the problem pool according to the configuration information of whether the target SQL script needs to be tracked.
In an embodiment of the present application, a data self-consistency monitoring device is provided, which is applied to an automated monitoring platform, and includes: acquiring SQL scripts written based on data inspection specifications and configuration information of the SQL scripts, wherein the data inspection specifications are summarized in advance and are used for automatically inspecting corresponding data items in the data inspection specifications, and the configuration information comprises: monitoring time, responsible person, processing person and whether tracking is needed; when the target SQL script reaches the corresponding monitoring time, automatically checking the data item corresponding to the target SQL script by adopting the target SQL script to obtain a checking result; and sending the information of the abnormal data item in the checking result to a processor corresponding to the target SQL script, and sending the information of the abnormal data item to a problem pool according to the configuration information of whether the target SQL script needs to be tracked. According to the data self-consistency monitoring device disclosed by the application, the automatic monitoring platform can automatically check each data item in the data checking specification by adopting the SQL script, the data item in the data checking specification is sufficiently comprehensive, namely the data checking range is sufficiently large, the automatic checking accuracy is high, the data self-consistency monitoring device is more intelligent and reliable and has high efficiency, and the technical problems of poor intelligence and poor accuracy of the existing data self-consistency checking method are solved.
Optionally, the data items in the data inspection specification may be updated as needed.
Optionally, the SQL script comprises: basic information of corresponding data items, expected values of corresponding data items and comparison rules of actual values.
Optionally, the device is further configured to: and analyzing and pre-executing the SQL script to determine the accuracy of the SQL script.
Optionally, the device is further configured to: a unique script number is added to each SQL script.
Optionally, each SQL script corresponds to a custom priority, where the priority of the SQL script is custom defined according to the priority of the data item to be checked and the priority of the scene.
Optionally, the sending unit is further configured to: and sending the information of the abnormal data items in the inspection results to a processor corresponding to the target SQL script in the form of a mail, wherein the same mail can comprise the information of the abnormal data items in the inspection results corresponding to a plurality of SQL scripts.
The device provided by the embodiment of the present application has the same implementation principle and technical effects as those of the foregoing method embodiment, and for the sake of brevity, reference may be made to the corresponding content in the foregoing method embodiment where the device embodiment is not mentioned.
As shown in fig. 4, an electronic device 600 provided in an embodiment of the present application includes: the system comprises a processor 601, a memory 602 and a bus, wherein the memory 602 stores machine-readable instructions executable by the processor 601, the processor 601 and the memory 602 communicate through the bus when the electronic device is running, and the processor 601 executes the machine-readable instructions to perform the steps of the data self-consistency monitoring method.
Specifically, the memory 602 and the processor 601 can be general-purpose memories and processors, which are not limited herein, and the method for monitoring data self-consistency can be performed when the processor 601 runs a computer program stored in the memory 602.
The processor 601 may be an integrated circuit chip with signal processing capabilities. In implementation, the steps of the above method may be performed by integrated logic circuits of hardware in the processor 601 or instructions in the form of software. The processor 601 may be a general-purpose processor, including a central processing unit (Central Processing Unit, CPU for short), a network processor (Network Processor, NP for short), etc.; but may also be a digital signal processor (Digital Signal Processing, DSP for short), application specific integrated circuit (Application Specific Integrated Circuit, ASIC for short), off-the-shelf programmable gate array (Field-Programmable Gate Array, FPGA for short), or other programmable logic device, discrete gate or transistor logic device, discrete hardware components. The disclosed methods, steps, and logic blocks in the embodiments of the present application may be implemented or performed. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like. The steps of the method disclosed in connection with the embodiments of the present application may be embodied directly in the execution of a hardware decoding processor, or in the execution of a combination of hardware and software modules in a decoding processor. The software modules may be located in a random access memory, flash memory, read only memory, programmable read only memory, or electrically erasable programmable memory, registers, etc. as well known in the art. The storage medium is located in a memory 602, and the processor 601 reads information in the memory 602 and performs the steps of the above method in combination with its hardware.
Corresponding to the above method for monitoring data self-consistency, the embodiment of the application also provides a computer readable storage medium, wherein the computer readable storage medium stores machine executable instructions, and the computer executable instructions cause a processor to execute the steps of the above method for monitoring data self-consistency when the computer executable instructions are called and executed by the processor.
The monitoring device for data self-consistency provided by the embodiment of the application can be specific hardware on equipment or software or firmware installed on the equipment. The device provided by the embodiment of the present application has the same implementation principle and technical effects as those of the foregoing method embodiment, and for the sake of brevity, reference may be made to the corresponding content in the foregoing method embodiment where the device embodiment is not mentioned. It will be clear to those skilled in the art that, for convenience and brevity, the specific operation of the system, apparatus and unit described above may refer to the corresponding process in the above method embodiment, which is not described in detail herein.
In the embodiments provided in the present application, it should be understood that the disclosed apparatus and method may be implemented in other manners. The above-described apparatus embodiments are merely illustrative, for example, the division of the units is merely a logical function division, and there may be other manners of division in actual implementation, and for example, multiple units or components may be combined or integrated into another system, or some features may be omitted, or not performed. Alternatively, the coupling or direct coupling or communication connection shown or discussed with each other may be through some communication interface, device or unit indirect coupling or communication connection, which may be in electrical, mechanical or other form.
As another example, the flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of apparatus, methods and computer program products according to various embodiments of the present application. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
The units described as separate units may or may not be physically separate, and units shown as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
In addition, each functional unit in the embodiments provided in the present application may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit.
The functions, if implemented in the form of software functional units and sold or used as a stand-alone product, may be stored in a computer-readable storage medium. Based on such understanding, the technical solution of the present application may be embodied essentially or in a part contributing to the prior art or in a part of the technical solution, in the form of a software product stored in a storage medium, comprising several instructions for causing an electronic device (which may be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the vehicle marking method according to the embodiments of the present application. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a random access Memory (Random Access Memory, RAM), a magnetic disk, or an optical disk, or other various media capable of storing program codes.
It should be noted that: like reference numerals and letters in the following figures denote like items, and thus once an item is defined in one figure, no further definition or explanation of it is required in the following figures, and furthermore, the terms "first," "second," "third," etc. are used merely to distinguish one description from another and are not to be construed as indicating or implying relative importance.
Finally, it should be noted that: the above examples are only specific embodiments of the present application, and are not intended to limit the scope of the present application, but it should be understood by those skilled in the art that the present application is not limited thereto, and that the present application is described in detail with reference to the foregoing examples: any person skilled in the art may modify or easily conceive of the technical solution described in the foregoing embodiments, or perform equivalent substitution of some of the technical features, while remaining within the technical scope of the present disclosure; such modifications, changes or substitutions do not depart from the spirit of the corresponding technical solutions. Are intended to be encompassed within the scope of the present application. Therefore, the protection scope of the application is subject to the protection scope of the claims.

Claims (10)

1. The data self-consistency monitoring method is characterized by being applied to an automatic monitoring platform and comprising the following steps of:
acquiring SQL scripts written based on data inspection specifications and configuration information of the SQL scripts, wherein the data inspection specifications are summarized in advance and are used for automatically inspecting corresponding data items in the data inspection specifications, and the configuration information comprises: monitoring time, responsible person, processing person and whether tracking is needed;
when the target SQL script reaches the corresponding monitoring time, automatically checking the data item corresponding to the target SQL script by adopting the target SQL script to obtain a checking result;
and sending the information of the abnormal data item in the checking result to a processor corresponding to the target SQL script, and sending the information of the abnormal data item to a problem pool according to the configuration information of whether the target SQL script needs to be tracked.
2. The method of claim 1, wherein data items in the data inspection specification are updated as needed.
3. The method of claim 1, wherein the SQL script comprises: basic information of corresponding data items, expected values of corresponding data items and comparison rules of actual values.
4. The method of claim 1, wherein after acquiring the SQL scripts written based on the data inspection specification, before acquiring the configuration information of each of the SQL scripts, the method further comprises:
analyzing and pre-executing the SQL script to determine the accuracy of the SQL script.
5. The method of claim 4, wherein after determining the accuracy of the SQL script, the method further comprises:
a unique script number is added to each SQL script.
6. The method of claim 1, wherein each of the SQL scripts has a custom priority, wherein the priority of the SQL script is custom defined according to the priority of the data item it is to examine and the priority of the scene.
7. The method of claim 1, wherein sending the information of the abnormal data item in the inspection result to the handler corresponding to the target SQL script comprises:
and sending the information of the abnormal data items in the inspection results to a processor corresponding to the target SQL script in a mail form, wherein the same mail can comprise the information of the abnormal data items in the inspection results corresponding to a plurality of SQL scripts.
8. The utility model provides a monitoring device of data self-consistency which characterized in that is applied to automatic monitoring platform, includes:
the system comprises an acquisition unit, a configuration information generation unit and a configuration information generation unit, wherein the acquisition unit is used for acquiring SQL scripts written based on data inspection specifications and configuration information of the SQL scripts, the data inspection specifications are summarized in advance and are used for automatically inspecting corresponding data items in the data inspection specifications, and the configuration information comprises: monitoring time, responsible person, processing person and whether tracking is needed;
the automatic checking unit is used for automatically checking the data item corresponding to the target SQL script by adopting the target SQL script when the target SQL script reaches the corresponding monitoring time, so as to obtain a checking result;
and the sending unit is used for sending the information of the abnormal data item in the checking result to a processor corresponding to the target SQL script and sending the information of the abnormal data item to a problem pool according to the configuration information of whether the target SQL script needs to be tracked.
9. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the processor implements the steps of the method of any of the preceding claims 1 to 7 when the computer program is executed.
10. A computer readable storage medium storing machine executable instructions which, when invoked and executed by a processor, cause the processor to perform the method of any one of the preceding claims 1 to 7.
CN202310996731.5A 2023-08-08 2023-08-08 Data self-consistency monitoring method and device and electronic equipment Pending CN117033362A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202310996731.5A CN117033362A (en) 2023-08-08 2023-08-08 Data self-consistency monitoring method and device and electronic equipment

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202310996731.5A CN117033362A (en) 2023-08-08 2023-08-08 Data self-consistency monitoring method and device and electronic equipment

Publications (1)

Publication Number Publication Date
CN117033362A true CN117033362A (en) 2023-11-10

Family

ID=88638554

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202310996731.5A Pending CN117033362A (en) 2023-08-08 2023-08-08 Data self-consistency monitoring method and device and electronic equipment

Country Status (1)

Country Link
CN (1) CN117033362A (en)

Similar Documents

Publication Publication Date Title
CN110287052B (en) Root cause task determination method and device for abnormal task
CN106656536B (en) Method and equipment for processing service calling information
CN108984389A (en) A kind of applied program testing method and terminal device
CN114116496A (en) Automatic testing method, device, equipment and medium
WO2018233170A1 (en) Method, device, computer device, and storage medium for recording a log
CN113590432A (en) Database inspection method and device
CN108512726B (en) Data monitoring method and equipment
CN110471912B (en) Employee attribute information verification method and device and terminal equipment
CN113448862A (en) Software version testing method and device and computer equipment
CN116126843A (en) Data quality evaluation method and device, electronic equipment and storage medium
CN113806343B (en) Evaluation method and system for Internet of vehicles data quality
CN111435327B (en) Log record processing method, device and system
CN116738091A (en) Page monitoring method and device, electronic equipment and storage medium
CN117033362A (en) Data self-consistency monitoring method and device and electronic equipment
CN116185706A (en) Data backup method and device, computing and storage medium and electronic equipment
CN113791980B (en) Conversion analysis method, device and equipment for test cases and storage medium
CN113868137A (en) Method, device and system for processing buried point data and server
CN109840213B (en) Test data creating method, device, terminal and storage medium for GUI test
CN113656318A (en) Software version testing method and device and computer equipment
CN111352825B (en) Data interface testing method and device and server
CN113238940A (en) Interface test result comparison method, device, equipment and storage medium
CN112631905A (en) Execution process data management method and device, computer equipment and storage medium
CN112363944A (en) Method and equipment for comparing return values of multiple environment interfaces
CN111767222A (en) Data model verification method and device, electronic equipment and storage medium
TWI778634B (en) Method for classifying faults, electronic equipment and storage medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination