CN116126913A - Feedback type data management method, device, equipment and computer readable storage medium - Google Patents

Feedback type data management method, device, equipment and computer readable storage medium Download PDF

Info

Publication number
CN116126913A
CN116126913A CN202211729602.1A CN202211729602A CN116126913A CN 116126913 A CN116126913 A CN 116126913A CN 202211729602 A CN202211729602 A CN 202211729602A CN 116126913 A CN116126913 A CN 116126913A
Authority
CN
China
Prior art keywords
data
abnormal
feedback
processing
governance
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202211729602.1A
Other languages
Chinese (zh)
Inventor
罗龙
刘栋梁
张健博
谈卫华
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Guangdong Digital Information Technology Co ltd
Original Assignee
Guangdong Digital Information Technology Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Guangdong Digital Information Technology Co ltd filed Critical Guangdong Digital Information Technology Co ltd
Priority to CN202211729602.1A priority Critical patent/CN116126913A/en
Publication of CN116126913A publication Critical patent/CN116126913A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/24Querying
    • G06F16/245Query processing
    • G06F16/2455Query execution
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/21Design, administration or maintenance of databases
    • G06F16/215Improving data quality; Data cleansing, e.g. de-duplication, removing invalid entries or correcting typographical errors
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/21Design, administration or maintenance of databases
    • G06F16/219Managing data history or versioning
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/23Updating

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Databases & Information Systems (AREA)
  • Data Mining & Analysis (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Computational Linguistics (AREA)
  • Quality & Reliability (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

The invention discloses a feedback type data management method, a device, equipment and a computer readable storage medium, wherein the feedback type data management method comprises the following steps: when abnormal data fed back by a user is received, confirming the data source of the abnormal data; if the abnormal data is from the data center, feeding back the abnormal data to a data center; after the data center station receives the abnormal data, confirming a data deviation point; the data center station adopts corresponding processing measures to correct the abnormal data according to the data deviation points; and after the correction of the abnormal data is completed, the data is acquired again and updated by the data management platform. The method has the advantages of more comprehensive problem discovery, low cost, high treatment efficiency, clear responsibility and the like.

Description

Feedback type data management method, device, equipment and computer readable storage medium
Technical Field
The invention relates to the technical field of data management in universities, in particular to a feedback type data management method, a device, equipment and a computer readable storage medium.
Background
The related departments such as an information center (or a network information center) and the like carry out check on the data quality according to service specifications and requirements, the found quality problems are fed back to the service department, and the service part is responsible for the work such as verification, data error correction and the like. And finally, entering the next quality optimization period, and iteratively optimizing the data quality.
However, the conventional data management process has the following problems:
1) Data anomalies are not easy to find: under the conditions of meeting the data specification requirements and service requirements, the data quality problem system cannot find out, so that the data is inaccurate, and the upper layer decision is influenced;
2) The process is complicated and the treatment efficiency is low: the reasons for causing the data quality problem are various, the traditional processing mode is based on the gradual reverse pushing of the data processing flow, the checking cost is high, the communication cost is high, and the efficiency is low.
3) The cost of multiparty cooperative processing and communication is higher: the data flow among the business systems constructed by multiple manufacturers is problematic, the manufacturers are required to be pulled through to assist in positioning the problem and correcting abnormal data, joint debugging and communication between the manufacturers are high in cost, and the failure processing efficiency is low.
Therefore, there is a need to provide a data management method to solve the above problems of the conventional data management process.
Disclosure of Invention
The embodiment of the application aims to solve the problems existing in the traditional data management flow by providing a feedback type data management method.
In order to achieve the above objective, an embodiment of the present application provides a feedback data management method, including:
when abnormal data fed back by a user is received, confirming the data source of the abnormal data;
if the abnormal data is from the data center, feeding back the abnormal data to a data center;
after the data center station receives the abnormal data, confirming a data deviation point;
the data center station adopts corresponding processing measures to correct the abnormal data according to the data deviation points;
and after the correction of the abnormal data is completed, the data is acquired again and updated by the data management platform.
In one embodiment, the step of identifying the data deviation point after the data center station receives the abnormal data includes:
the data center station acquires a data link of the abnormal data according to the blood edge condition of the abnormal data;
and determining the data deviation point according to the data change condition in the data link.
In one embodiment, the data center station adopts corresponding processing measures to correct the abnormal data according to the data deviation point, and the method comprises the following steps:
if the data processing process is abnormal according to the processing deviation point, analyzing the abnormal reason of the data processing process;
correcting a data processing mechanism corresponding to the data according to the abnormal reason;
the exception data is corrected based on the corrected data processing mechanism.
In an embodiment, the data center station adopts corresponding processing measures to correct the abnormal data according to the data deviation point, and further includes:
if the source data is abnormal according to the processing deviation point, analyzing the data version of the abnormal data to inquire the target data version of the abnormal data;
confirming a responsible person for data processing according to the target data version;
sending an analysis result and a modification suggestion of the abnormal data to the responsible person;
correcting the abnormal data according to the information of the responsible person based on the analysis result and the modification suggestion feedback.
In one embodiment, the data re-acquisition and data update by the data management platform comprises:
the data management platform re-collects corrected data and detects whether the corrected data accords with a set data specification;
if yes, the data management platform performs historical data backup and updates the data of the data center based on the corrected data.
In an embodiment, the method further comprises:
and if the abnormal data is locally sourced from the user terminal, correcting the abnormal data at the user terminal.
In an embodiment, the method further comprises:
and feeding back an abnormal data processing result to the user.
In order to achieve the above objective, an embodiment of the present application further provides a feedback data management device, including:
the user terminal is used for confirming the data source of the abnormal data when the abnormal data fed back by the user are received, and feeding back the abnormal data to the data center when the abnormal data are from the data center;
the data center is used for confirming a data deviation point after the abnormal data is received, and the data deviation point adopts corresponding processing measures to correct the abnormal data;
and the data management platform is used for carrying out data re-acquisition and data updating after the correction of the abnormal data is completed.
To achieve the above objective, an embodiment of the present application further provides a feedback data management device, including a memory, a processor, and a feedback data management program stored in the memory and capable of running on the processor, where the processor implements the feedback data management method according to any one of the above when executing the feedback data management program.
To achieve the above object, an embodiment of the present application further provides a computer readable storage medium, where a feedback data management program is stored, where the feedback data management program when executed by a processor implements a feedback data management method according to any one of the above.
Compared with the traditional data management flow, the feedback data management method has the following advantages:
(1) The data problem can be found by the data owner, so that abnormal data which cannot be detected through the data quality checking rule and the service specification can be solved, and the comprehensiveness of abnormal data finding can be increased.
(2) The method combines the semi-automatic analysis of the abnormal reasons of the data treatment platform, can compress the data processing flow, reduces the labor cost, shortens the data processing time and improves the processing efficiency.
(3) The whole links between the system (data center platform) and the system (data management platform) and between the user and the system are pulled through, the whole process record of the data processing process can be tracked, the communication cost of the multiparty cooperative processing problem is reduced, and the problem of unclear responsibility is solved.
(4) And the processing process is traceable and auditable, so that the data security and the data processing efficiency are improved.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings that are required in the embodiments or the description of the prior art will be briefly described, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and other drawings may be obtained according to the structures shown in these drawings without inventive effort for a person skilled in the art.
FIG. 1 is a block diagram of one embodiment of a feedback data management apparatus according to the present invention;
FIG. 2 is a flow chart of an embodiment of a feedback data management method according to the present invention;
FIG. 3 is a block diagram of a feedback data management apparatus according to an embodiment of the present invention.
The achievement of the objects, functional features and advantages of the present invention will be further described with reference to the accompanying drawings, in conjunction with the embodiments.
Detailed Description
It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the scope of the invention.
In order that the above-described aspects may be better understood, exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be embodied in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.
It should be noted that in the claims, any reference signs placed between parentheses shall not be construed as limiting the claim. The word "comprising" does not exclude the presence of elements or steps other than those listed in a claim. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The invention may be implemented by means of hardware comprising several distinct elements, and by means of a suitably programmed computer. In the unit claims enumerating several means, several of these means may be embodied by one and the same item of hardware. And the use of "first," "second," and "third," etc. do not denote any order, and the terms may be construed as names.
As shown in fig. 1, fig. 1 is a schematic structural diagram of a server 1 (also called a feedback type data management device) of a hardware running environment according to an embodiment of the present invention.
The server provided by the embodiment of the invention is equipment with display function, such as 'Internet of things equipment', intelligent air conditioner with networking function, intelligent electric lamp, intelligent power supply, AR/VR equipment with networking function, intelligent sound box, automatic driving automobile, PC, intelligent mobile phone, tablet personal computer, electronic book reader, portable computer and the like.
As shown in fig. 1, the server 1 includes: memory 11, processor 12 and network interface 13.
The memory 11 includes at least one type of readable storage medium including flash memory, a hard disk, a multimedia card, a card memory (e.g., SD or DX memory, etc.), a magnetic memory, a magnetic disk, an optical disk, etc. The memory 11 may in some embodiments be an internal storage unit of the server 1, such as a hard disk of the server 1. The memory 11 may in other embodiments also be an external storage device of the server 1, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash Card (Flash Card) or the like, which are provided on the server 1.
Further, the memory 11 may also include an internal storage unit of the server 1 as well as an external storage device. The memory 11 may be used not only for storing application software installed in the server 1 and various types of data, such as codes of the feedback data administration program 10, but also for temporarily storing data that has been output or is to be output.
Processor 12 may in some embodiments be a central processing unit (Central Processing Unit, CPU), controller, microcontroller, microprocessor or other data processing chip for executing program code or processing data stored in memory 11, such as executing feedback data governance program 10, etc.
The network interface 13 may optionally comprise a standard wired interface, a wireless interface (e.g. WI-FI interface), typically used to establish a communication connection between the server 1 and other electronic devices.
The network may be the internet, a cloud network, a wireless fidelity (Wi-Fi) network, a Personal Area Network (PAN), a Local Area Network (LAN), and/or a Metropolitan Area Network (MAN). Various devices in a network environment may be configured to connect to a communication network according to various wired and wireless communication protocols. Examples of such wired and wireless communication protocols may include, but are not limited to, at least one of the following: transmission control protocol and internet protocol (TCP/IP), user Datagram Protocol (UDP), hypertext transfer protocol (HTTP), file Transfer Protocol (FTP), zigBee, EDGE, IEEE 802.11, light fidelity (Li-Fi), 802.16, IEEE 802.11s, IEEE 802.11g, multi-hop communications, wireless Access Points (APs), device-to-device communications, cellular communication protocol and/or bluetooth (bluetooth) communication protocol, or combinations thereof.
Optionally, the server may further comprise a user interface, which may comprise a Display (Display), an input unit such as a Keyboard (Keyboard), and optionally a standard wired interface, a wireless interface. Alternatively, in some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, an OLED (Organic Light-Emitting Diode) touch, or the like. The display may also be referred to as a display screen or a display unit, for displaying information processed in the server 1 and for displaying a visual user interface.
Fig. 1 shows only a server 1 having components 11-13 and a feedback data governance program 10, it will be understood by those skilled in the art that the configuration shown in fig. 1 is not limiting of server 1 and may include fewer or more components than shown, or may combine certain components, or a different arrangement of components.
In this embodiment, the processor 12 may be configured to call a feedback data governance program stored in the memory 11 and perform the following operations:
when abnormal data fed back by a user is received, confirming the data source of the abnormal data;
if the abnormal data is from the data center, feeding back the abnormal data to a data center;
after the data center station receives the abnormal data, confirming a data deviation point;
the data center station adopts corresponding processing measures to correct the abnormal data according to the data deviation points;
and after the correction of the abnormal data is completed, the data is acquired again and updated by the data management platform.
In one embodiment, processor 12 may be configured to invoke the feedback data governance program stored in memory 11 and perform the following operations:
after the data center station receives the abnormal data, confirming a data deviation point, comprising the following steps:
the data center station acquires a data link of the abnormal data according to the blood edge condition of the abnormal data;
and determining the data deviation point according to the data change condition in the data link.
In one embodiment, processor 12 may be configured to invoke the feedback data governance program stored in memory 11 and perform the following operations:
the data center station adopts corresponding processing measures to correct the abnormal data according to the data deviation points, and comprises the following steps:
if the data processing process is abnormal according to the processing deviation point, analyzing the abnormal reason of the data processing process;
correcting a data processing mechanism corresponding to the data according to the abnormal reason;
the exception data is corrected based on the corrected data processing mechanism.
In one embodiment, processor 12 may be configured to invoke the feedback data governance program stored in memory 11 and perform the following operations:
the data center station adopts corresponding processing measures to correct the abnormal data according to the data deviation points, and the method further comprises the following steps:
if the source data is abnormal according to the processing deviation point, analyzing the data version of the abnormal data to inquire the target data version of the abnormal data;
confirming a responsible person for data processing according to the target data version;
sending an analysis result and a modification suggestion of the abnormal data to the responsible person;
correcting the abnormal data according to the information of the responsible person based on the analysis result and the modification suggestion feedback.
In one embodiment, processor 12 may be configured to invoke the feedback data governance program stored in memory 11 and perform the following operations:
the data re-acquisition and data updating by the data management platform comprises the following steps:
the data management platform re-collects corrected data and detects whether the corrected data accords with a set data specification;
if yes, the data management platform performs historical data backup and updates the data of the data center based on the corrected data.
In one embodiment, processor 12 may be configured to invoke the feedback data governance program stored in memory 11 and perform the following operations:
the method further comprises the steps of:
and if the abnormal data is locally sourced from the user terminal, correcting the abnormal data at the user terminal.
In one embodiment, processor 12 may be configured to invoke the feedback data governance program stored in memory 11 and perform the following operations:
the method further comprises the steps of:
and feeding back an abnormal data processing result to the user.
Based on the hardware architecture of the feedback data management device, the embodiment of the feedback data management method is provided. The feedback type data treatment method aims to solve the problems of difficult discovery of data abnormality, low processing efficiency, higher communication cost and the like in the traditional data treatment process.
Referring to fig. 2, fig. 2 is a schematic diagram showing an embodiment of a feedback data management method according to the present invention, wherein the feedback data management method includes the following steps:
s10, when abnormal data fed back by a user are received, confirming the data source of the abnormal data.
Specifically, the user may feed back the abnormal data problem through the user terminal, where the user terminal specifically refers to software or a web page applied on a computer, a mobile phone, or other devices. On the user terminal, each user has a separate personal information center.
Specifically, when the user finds that there is a problem with the data recorded in the personal information center, the user can perform feedback of the abnormal data through the user terminal.
Further, when the personal information center receives abnormal data fed back by the user, the personal information center can confirm the data source by inquiring the data source, verifying the data meta-information, tracking the data flow and the like. The purpose of this confirmation of the origin of the data is to confirm whether the data originates locally at the terminal or at the data center.
And S20, if the abnormal data is sourced from the data center, feeding back the abnormal data to the data center.
Where data centers refer to the information technology infrastructure used to store, process, and distribute data.
The data center is a core part of a data management system and is responsible for monitoring and managing the quality and integrity of data, and can provide a standardized data interface for a service system to use.
Specifically, if it is determined that the abnormal data originates from the data center, the terminal calls a data abnormality feedback API to feed back the abnormal data to the data center.
Optionally, the data exception feedback API supports HTTP, webServices, JMS, kafaka, databases (mainstream databases Oracle, mysql, and mainstream home databases, etc.).
S30, after the data center station receives the abnormal data, confirming a data deviation point.
The data deviation point is the source or symptom of the data quality problem. The data deviation points may take many forms, such as data entry errors, data conversion problems, data summarization problems, etc. For each data deviation point, the data management system needs to take corresponding processing measures to solve the problem. For example, for data input errors, a manual auditing mode can be adopted for verification; for data conversion problems, a data conversion tool can be used for conversion; for data summarization problems, a data summarization tool may be used for summarization.
Illustratively, the data center station confirms the data deviation point after receiving the abnormal data, and comprises the following steps:
s31, the data center station acquires a data link of the abnormal data according to the blood edge condition of the abnormal data.
The data blood-source refers to the circulation condition of data in the data processing process, and comprises links such as data source, conversion, summarization and the like. Data blood-rims can help determine how data was generated and find the source of the data problem.
Data blood-lineage can be managed using a data blood-lineage map or a data blood-lineage table, among other tools. The data blood-edge graph refers to a chart for showing the blood-edge relation of the data, and can help a user to intuitively understand the circulation condition of the data. The data blood-edge table is a table for displaying data blood-edge information, and can help a user to clearly record the circulation condition of data.
A data link refers to a flow path that data traverses during data processing. It represents the process by which data arrives from a source to a final destination. The data link is typically composed of a plurality of nodes, each representing a data processing link, and a connecting segment representing the process of data flow. For example, a data link may include nodes for data collection, data cleansing, data conversion, data storage, and the like.
S32, determining the data deviation point according to the data change condition in the data link.
Specifically, after the data center station receives abnormal data feedback, the data link can be analyzed according to the blood edge condition of the abnormal data, and the data deviation point can be determined according to the data change condition in the link.
It can be understood that the data blood margin mode is adopted to confirm the data deviation point of the abnormal data, so that the root of the data deviation point can be quickly found, the data blood margin can be used for conveniently recording the information of the data deviation point, and the subsequent management is convenient.
It should be noted that, the design of the present application is not limited thereto, and in other embodiments, the deviation point of the abnormal data may be confirmed by manual auditing, data analysis, and the like.
S40, the data center station adopts corresponding processing measures to correct the abnormal data according to the data deviation points.
Specifically, after determining the data deviation point, the data center station may confirm the reason for the occurrence of the abnormal data according to the data deviation point, and correct the abnormal data according to the corresponding processing measure, for example, modify the data processing flow, adjust the data processing parameters, and so on.
Optionally, the step of the data center station adopting corresponding processing measures to correct the abnormal data according to the data deviation point includes the following steps:
s41, if the data processing process is abnormal according to the processing deviation point, analyzing the abnormality reason of the data processing process.
Specifically, it can be determined from the characteristics of the deviation points that the data deviation points are caused by the abnormality of the data processing process.
For example, if a data deviation point occurs during a data processing process and the data characteristics of the deviation point are clearly not in compliance with the requirements of the data processing process, it may be determined that the data deviation point is caused by an abnormality in the data processing process.
Further, when it is determined that the data processing process is abnormal, the following method may be adopted to find the cause of the abnormality:
1. viewing a log of data processing procedures: the log of the data processing process may record the abnormal information in the data processing process, and the reasons of the abnormal information can be analyzed by looking up the log.
2. Checking the data processing code: anomalies in the data processing often result from errors in the code, which may be viewed to execute the data processing, analyzing the code for logical errors or other problems.
3. Analyzing parameters of a data processing process: the parameter settings of the data processing process can be checked to analyze whether there is a parameter setting error.
4. Analyzing the dependency relationship of the data processing process: the dependency relationship of the data processing process can be checked, and whether the dependency relationship error exists or not is analyzed.
S42, correcting a data processing mechanism corresponding to the data according to the abnormal reason.
Where a data processing mechanism refers to a rule, procedure, or system used to process data. The data processing mechanism may be used to perform operations on the data such as cleaning, converting, aggregating, analyzing, etc., thereby making the data more useful and accurate. Common data processing mechanisms include: data processing algorithms, data processing flows, data processing systems, etc.
Specifically, after determining the cause of the data abnormality, the corresponding data processing mechanism may be corrected in a targeted manner according to the cause of the abnormality. For example, if the cause of the exception is an error in the data processing code, the code may be modified to correct the error; if the reason for the abnormality is that the setting of the data processing parameters is not reasonable, the data processing parameters can be adjusted to correct the error; if the reason for the abnormality is that the design of the data processing flow is not reasonable, the data processing flow can be modified to correct the error; if the reason for the anomaly is that the design of the data processing algorithm is not reasonable, the data processing algorithm may be modified to correct the error.
S43, correcting the abnormal data based on the corrected data processing mechanism.
Specifically, after correcting the problematic data processing mechanism, the corrected data processing mechanism may be used to process the source data of the abnormal data again to correct the abnormal data. For example, if the data acquisition server is abnormal, the data acquisition service can be restored, and the missed data can be acquired again.
Optionally, the data center station adopts corresponding processing measures to correct the abnormal data according to the data deviation point, and further comprises the following steps:
and S44, if the source data is abnormal according to the processing deviation point, analyzing the data version of the abnormal data to inquire the target data version of the abnormal data.
Where the data versions refer to different versions of the data over different time periods. The data version management refers to the process of recording, managing and analyzing version information of data. Data version management can help organizations manage data better, ensuring accuracy and consistency of data. For example, in data version management, version information for each data change may be recorded to trace back and compare different versions of data. In addition, the data version management can also help the organization to control the data quality, and ensure the accuracy and consistency of the data.
Specifically, if the inspection data processing flow fails to find the cause of the data deviation point, then the anomalous data may be considered to be caused by the source data anomaly.
Further, if it is determined that the source data is abnormal based on the processing deviation point, this means that the data is abnormal in the process from the source data to the data center. In this case, the following steps may be considered to analyze the data version of the abnormal data:
1. version information of the abnormal data is recorded: version information of the abnormal data may be recorded as it is fed back for later analysis.
2. Querying data version information: version information of the exception data may be queried using a data version management system.
3. Analyzing the data version information: and analyzing the version of abnormal data according to the data version information obtained by inquiry.
S45, confirming a responsible person for data processing according to the target data version.
Specifically, the responsible person can be confirmed by the UC matrix (data creator Create, user). Among these, the UC matrix (data creator Create, user) is a data management method for determining creator and User of data. The UC matrix is mainly used for solving the problems of data redundancy, conflict and untimely updating and determining the responsible person of the data.
S46, sending an analysis result and a modification suggestion of the abnormal data to the responsible person.
The analysis result refers to a conclusion or a result obtained after data is analyzed. When the abnormal data is analyzed, the analysis result can indicate the reason of the abnormal data and corresponding modification suggestions are provided. For example, if the analysis of the anomalous data indicates that the data is due to human error, then the modification advice may be to adjust the data processing flow to avoid the anomaly of the data due to human error.
Specifically, the analysis result and the modification suggestion can be sent to the relevant responsible person through short messages, application messages, telephones, mails and the like.
S47, correcting the abnormal data according to the information fed back by the responsible person based on the analysis result and the modification advice.
Specifically, after the analysis result and the modification suggestion are sent to the relevant responsible person, if feedback information of the responsible person is received, the data can be corrected according to the feedback information.
It can be understood that by the above manner, the abnormal data can be corrected in a targeted manner according to the reason of occurrence of the abnormal data.
S50, after the correction of the abnormal data is completed, the data management platform performs data re-acquisition and data updating.
And updating the data of the data center based on the corrected data.
Wherein the data governance platform is a tool or system for managing and governance data. The data governance platform may help businesses collect, store, process, and use data, and ensure the accuracy, integrity, and availability of the data. Data governance platforms typically include a variety of functions such as data collection, data storage, data processing, data cleansing, data visualization, data quality management, and the like. Through the data management platform, enterprises can better utilize the value brought by the data and realize the decision of data driving.
Specifically, after the correction of the abnormal data is completed, the abnormal data can be fed back to the data management platform, and the data management platform can carry out data acquisition and data update again.
Optionally, the data re-acquisition and data updating by the data management platform comprises:
s51, the data management platform re-collects corrected data and detects whether the corrected data accords with the set data specification.
Wherein the data specification is a rule or standard for unifying data format, content and quality. The data specification can help enterprises to better manage and use the data, and improve the accuracy and usability of the data.
Specifically, after the abnormal data is corrected, the data management platform needs to re-acquire the corrected data. In the process of acquiring the corrected data, the data management platform also needs to detect whether the corrected data accords with the set data specification. This may be done by quality checking the data to ensure that the data quality (data quality including data specifications, etc.) meets the requirements.
And S52, if so, the data management platform performs historical data backup and updates the data of the data center based on the corrected data.
Specifically, the data administration platform also needs to perform a historical data backup before updating the data of the data center. The method can help enterprises to keep original data, and can restore the original data when the data update has problems.
Finally, the data management platform may update the data of the data center based on the corrected data. This may be accomplished by directly overlaying the original data, or by combining the original data with the updated data.
In some embodiments, the feedback data governance method of the present application further comprises:
and if the abnormal data is locally sourced from the user terminal, correcting the abnormal data at the user terminal.
The user terminal locally refers to devices used by a user, such as a personal computer, a notebook computer, a mobile phone, and the like. These devices typically store user data and applications and may be accessed and controlled directly by the user. The data local to the user terminal may be managed and maintained by the user himself and is typically not accessible to other persons.
Specifically, if the abnormal data is stored in the local of the user terminal, it can be judged that the abnormal data originates from the local of the user terminal, and at this time, the updating of the abnormal data can be completed without reporting the data center and directly receiving the modified data provided by the user to replace the abnormal data.
It can be understood that by checking whether the abnormal data originates from the local of the user terminal, the updating of the abnormal data can be directly completed when the abnormal data originates from the local of the terminal, so that unnecessary resource waste can be reduced, and the cost can be saved.
In some embodiments, the feedback data governance method of the present application further comprises: and feeding back an abnormal data processing result to the user.
Specifically, after the data management platform or the user terminal processes the abnormal data locally, the processing result should be fed back to the user. The feedback mode can be feedback mode of message notification or mail of the user terminal. The feedback content includes, but is not limited to, the processing condition of the abnormal data, the influence condition of the abnormal data, the precaution measure of the abnormal data, and the like.
It can be appreciated that the feedback of the abnormal data processing result to the user is helpful for the user to understand the state of the data, and the trust degree of the user on the data can be improved.
It can be appreciated that the feedback data management method of the present application has the following advantages compared to the conventional data management flow:
(1) The data problem can be found by the data owner, so that abnormal data which cannot be detected through the data quality checking rule and the service specification can be solved, and the comprehensiveness of abnormal data finding can be increased.
(2) The method combines the semi-automatic analysis of the abnormal reasons of the data treatment platform, can compress the data processing flow, reduces the labor cost, shortens the data processing time and improves the processing efficiency.
(3) The whole links between the system (data center platform) and the system (data management platform) and between the user and the system are pulled through, the whole process record of the data processing process can be tracked, the communication cost of the multiparty cooperative processing problem is reduced, and the problem of unclear responsibility is solved.
(4) And the processing process is traceable and auditable, so that the data security and the data processing efficiency are improved.
In addition, referring to fig. 3, an embodiment of the present invention further provides a feedback data management device, where the feedback data management device includes:
the user terminal 110 is configured to confirm a data source of the abnormal data when abnormal data fed back by a user is received, and feed back the abnormal data to a data center when the abnormal data is derived from a data center;
a data center 120 for confirming a data deviation point after receiving the abnormal data, and the data deviation point adopting corresponding processing measures to correct the abnormal data;
the data management platform 130 is used for data re-acquisition and data update after the correction of the abnormal data is completed.
The steps of implementing each functional module of the feedback data management device may refer to each embodiment of the feedback data management method of the present invention, which is not described herein.
In addition, the embodiment of the invention also provides a computer readable storage medium, which can be any one or any combination of a plurality of hard disk, a multimedia card, an SD card, a flash memory card, an SMC, a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a portable compact disc read-only memory (CD-ROM), a USB memory and the like. The computer readable storage medium includes the feedback data management program 10, and the embodiment of the computer readable storage medium of the present invention is substantially the same as the above-mentioned feedback data management method and the embodiment of the server 1, and will not be described herein.
It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as a method, system, or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
While preferred embodiments of the present invention have been described, additional variations and modifications in those embodiments may occur to those skilled in the art once they learn of the basic inventive concepts. It is therefore intended that the following claims be interpreted as including the preferred embodiments and all such alterations and modifications as fall within the scope of the invention.
It will be apparent to those skilled in the art that various modifications and variations can be made to the present invention without departing from the spirit or scope of the invention. Thus, it is intended that the present invention also include such modifications and alterations insofar as they come within the scope of the appended claims or the equivalents thereof.

Claims (10)

1. A method of feedback data governance comprising:
when abnormal data fed back by a user is received, confirming the data source of the abnormal data;
if the abnormal data is from the data center, feeding back the abnormal data to a data center;
after the data center station receives the abnormal data, confirming a data deviation point;
the data center station adopts corresponding processing measures to correct the abnormal data according to the data deviation points;
and after the correction of the abnormal data is completed, the data is acquired again and updated by the data management platform.
2. The feedback data governance method of claim 1, wherein identifying a data deviation point after the data center station receives the anomaly data comprises:
the data center station acquires a data link of the abnormal data according to the blood edge condition of the abnormal data;
and determining the data deviation point according to the data change condition in the data link.
3. The feedback data governance method of claim 1, wherein the data center station taking corresponding processing measures to correct the abnormal data according to the data deviation point, comprising:
if the data processing process is abnormal according to the processing deviation point, analyzing the abnormal reason of the data processing process;
correcting a data processing mechanism corresponding to the data according to the abnormal reason;
the exception data is corrected based on the corrected data processing mechanism.
4. The feedback data governance method of claim 3, wherein the data center station takes corresponding processing measures to correct the abnormal data according to the data deviation point, and further comprising:
if the source data is abnormal according to the processing deviation point, analyzing the data version of the abnormal data to inquire the target data version of the abnormal data;
confirming a responsible person for data processing according to the target data version;
sending an analysis result and a modification suggestion of the abnormal data to the responsible person;
correcting the abnormal data according to the information of the responsible person based on the analysis result and the modification suggestion feedback.
5. The feedback data governance method of claim 4, wherein the data re-acquisition and data update by the data governance platform comprises:
the data management platform re-collects corrected data and detects whether the corrected data accords with a set data specification;
if yes, the data management platform performs historical data backup and updates the data of the data center based on the corrected data.
6. The feedback data remediation method of claim 1, wherein the method further comprises:
and if the abnormal data is locally sourced from the user terminal, correcting the abnormal data at the user terminal.
7. The feedback data remediation method of any one of claims 1 to 6, further comprising:
and feeding back an abnormal data processing result to the user.
8. A feedback data governance device, comprising:
the user terminal is used for confirming the data source of the abnormal data when the abnormal data fed back by the user are received, and feeding back the abnormal data to the data center when the abnormal data are from the data center;
the data center is used for confirming a data deviation point after the abnormal data is received, and the data deviation point adopts corresponding processing measures to correct the abnormal data;
and the data management platform is used for carrying out data re-acquisition and data updating after the correction of the abnormal data is completed.
9. A feedback data governance device comprising a memory, a processor and a feedback data governance program stored on the memory and operable on the processor, the processor implementing a feedback data governance method according to any of claims 1 to 7 when executing the feedback data governance program.
10. A computer readable storage medium, wherein a feedback data governance program is stored on the computer readable storage medium, which when executed by a processor implements the feedback data governance method of any of claims 1 to 7.
CN202211729602.1A 2022-12-30 2022-12-30 Feedback type data management method, device, equipment and computer readable storage medium Pending CN116126913A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202211729602.1A CN116126913A (en) 2022-12-30 2022-12-30 Feedback type data management method, device, equipment and computer readable storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202211729602.1A CN116126913A (en) 2022-12-30 2022-12-30 Feedback type data management method, device, equipment and computer readable storage medium

Publications (1)

Publication Number Publication Date
CN116126913A true CN116126913A (en) 2023-05-16

Family

ID=86300313

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202211729602.1A Pending CN116126913A (en) 2022-12-30 2022-12-30 Feedback type data management method, device, equipment and computer readable storage medium

Country Status (1)

Country Link
CN (1) CN116126913A (en)

Similar Documents

Publication Publication Date Title
US9712385B2 (en) Managing configurations of distributed devices
US10013339B2 (en) System and method for automating testing without scripting
TWI540424B (en) Method, device and system of repairing software run-time error
US7721158B2 (en) Customization conflict detection and resolution
CN102571403B (en) The implementation method of general data quality control adapter and device
CN104092718A (en) Distributed system and configuration information updating method in distributed system
US20130339933A1 (en) Systems and methods for quality assurance automation
US8275739B2 (en) User interface display for monitoring a database load engine
CN112328619A (en) Data quality monitoring method, device, system, electronic device and storage medium
US20200293310A1 (en) Software development tool integration and monitoring
CN110063042A (en) A kind of response method and its terminal of database failure
CN110706800A (en) Automatic management method and device for equipment maintenance, computer equipment and storage medium
WO2019148657A1 (en) Method for testing associated environments, electronic device and computer readable storage medium
US20170011302A1 (en) Action correlation framework
US11715496B2 (en) Systems and methods for processing video data
CN113468143A (en) Data migration method, system, computing device and storage medium
CN114841678B (en) Post data exchange method, data exchange system, server and storage medium
CN116126913A (en) Feedback type data management method, device, equipment and computer readable storage medium
US9354971B2 (en) Systems and methods for data storage remediation
US10877450B2 (en) Workflow-based change management and documentation system and method
CN113094345A (en) Method and equipment for importing table data file
CN108566293B (en) Electronic device, zk node information notification method, and storage medium
JP7359218B2 (en) Management device, management method and management program
CN111324374A (en) Application program registration method and device based on application performance management system
CN116071046A (en) Equipment inspection maintenance data processing method and system based on industrial Internet

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination