LU505740B1 - Data monitoring method and system - Google Patents
Data monitoring method and system Download PDFInfo
- Publication number
- LU505740B1 LU505740B1 LU505740A LU505740A LU505740B1 LU 505740 B1 LU505740 B1 LU 505740B1 LU 505740 A LU505740 A LU 505740A LU 505740 A LU505740 A LU 505740A LU 505740 B1 LU505740 B1 LU 505740B1
- Authority
- LU
- Luxembourg
- Prior art keywords
- data
- fusion
- model
- classified
- grading
- Prior art date
Links
- 238000012544 monitoring process Methods 0.000 title claims abstract description 122
- 238000000034 method Methods 0.000 title claims abstract description 39
- 230000004927 fusion Effects 0.000 claims abstract description 131
- 238000007405 data analysis Methods 0.000 claims description 22
- 238000012550 audit Methods 0.000 claims description 17
- 238000004364 calculation method Methods 0.000 claims description 17
- 238000013145 classification model Methods 0.000 claims description 14
- 238000004590 computer program Methods 0.000 claims description 12
- 238000000605 extraction Methods 0.000 claims description 12
- 241000475481 Nebula Species 0.000 claims description 5
- 230000001052 transient effect Effects 0.000 claims description 4
- 238000012216 screening Methods 0.000 claims description 3
- 238000003908 quality control method Methods 0.000 abstract description 5
- 238000010586 diagram Methods 0.000 description 9
- 238000004891 communication Methods 0.000 description 4
- 238000012545 processing Methods 0.000 description 3
- 230000003287 optical effect Effects 0.000 description 2
- 238000011161 development Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- ZZUFCTLCJUWOSV-UHFFFAOYSA-N furosemide Chemical compound C1=C(Cl)C(S(=O)(=O)N)=CC(C(O)=O)=C1NCC1=CC=CO1 ZZUFCTLCJUWOSV-UHFFFAOYSA-N 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/21—Design, administration or maintenance of databases
- G06F16/215—Improving data quality; Data cleansing, e.g. de-duplication, removing invalid entries or correcting typographical errors
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/901—Indexing; Data structures therefor; Storage structures
- G06F16/9024—Graphs; Linked lists
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/906—Clustering; Classification
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N5/00—Computing arrangements using knowledge-based models
- G06N5/02—Knowledge representation; Symbolic representation
- G06N5/022—Knowledge engineering; Knowledge acquisition
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Databases & Information Systems (AREA)
- General Engineering & Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Software Systems (AREA)
- Quality & Reliability (AREA)
- Artificial Intelligence (AREA)
- Computational Linguistics (AREA)
- Evolutionary Computation (AREA)
- Computing Systems (AREA)
- Mathematical Physics (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
- Testing And Monitoring For Control Systems (AREA)
Abstract
The embodiment of the present disclosure provides a data monitoring method and a system, wherein the method includes: determining fusion data of a current data governance platform to be monitored; inputting the fusion data of the current data governance platform to be monitored into a data monitoring model to obtain a data monitoring result output by the data monitoring model; wherein the data monitoring model is obtained by classifying and grading the sample fusion data of the current data governance platform and then generating a knowledge map and a quality knowledge base. The present disclosure effectively solves the problem that it is difficult to govern data in data classification and grading and quality control work existing in the current data governance platform.
Description
DATA MONITORING METHOD AND SYSTEM LUS05740
[0001] The present disclosure relates to the technical field of computers, in particular to a data monitoring method and a system.
[0002] With the development of computer technology, government data are complicated and varied. In order to share and exchange data effectively and legally, a lot of data classification and grading work is needed in government data governance. The function of classifying and grading data and managing data quality provided by the traditional data governance platform needs a lot of expert knowledge, and the threshold of use is high, so that it is difficult for ordinary maintainers to be competent after a data governance project is transferred to a normal operation stage in the later stage, and finally it is difficult for the data governance project to end or even stop. Therefore, it is urgent to provide a data monitoring method and a system to solve the problem that it is difficult to govern data in data classification and grading and quality control work existing in the current data governance platform.
[0003] The embodiment of the present disclosure provides a data monitoring method and a system to solve the problem that it is difficult to govern data in data classification and grading and quality control work existing in the current data governance platform.
[0004] In a first aspect, an embodiment of the present disclosure provides a data monitoring method, including:
[0005] determining fusion data of a current data governance platform to be monitored,
[0006] inputting the fusion data of the current data governance platform to be monitored into a data monitoring model to obtain a data monitoring result output by the data monitoring model;
[0007] wherein the data monitoring model is obtained by classifying and grading the sample fusion data of the current data governance platform and then generating a knowledge map and a quality knowledge base.
[0008] Preferably, the data monitoring model includes a data analysis model and a quality audit model;
[0009] inputting the fusion data of the current data governance platform to be monitored into a data monitoring model to obtain a data monitoring result output by the data monitoring model includes: LU505740
[0010] inputting the fusion data of the current data governance platform to be monitored into the data analysis model, and outputting classified and graded data of the fusion data based on the knowledge map;
[0011] inputting the classified and graded data of the fusion data into the quality audit model, and outputting the data monitoring result based on the quality knowledge base.
[0012] Preferably, the data analysis model includes a data feature extraction model, a data classification model and a data grading model; the knowledge map includes a classification map and a grading map;
[0013] inputting the fusion data of the current data governance platform to be monitored into the data analysis model and outputting classified and graded data of the fusion data based on the knowledge map includes:
[0014] inputting the fusion data of the current data governance platform to be monitored into the data feature extraction model, and outputting data feature values corresponding to the fusion data;
[0015] inputting the fusion data and the corresponding data feature values into the data classification model, and outputting classified data based on the classification map;
[0016] inputting the classified data into the data grading model, and outputting the classified and graded data of the fusion data based on the grading map.
[0017] Preferably, inputting the fusion data and the corresponding data feature values into the data classification model and outputting classified data based on the classification map includes:
[0018] weighting the fusion data by using the data feature values corresponding to the fusion data as weights to obtain the weighted fusion data;
[0019] clustering weighted value indexes of the weighted fusion data based on the classification map to obtain the classified data.
[0020] Preferably, inputting the classified data into the data grading model and outputting the classified and graded data of the fusion data based on the grading map includes:
[0021] multiplying a weight value corresponding to the classified data by a preset grading coefficient to obtain a grading value corresponding to the classified data;
[0022] performing data division on the grading value corresponding to the classified data based on the grading map to obtain the classified and graded data of the fusion data.
[0023] Preferably, inputting the classified and graded data of the fusion data into the quality audit model and outputting the data monitoring result based on the quality knowledge base includes:
[0024] retrieving, screening, deleting or retaining the classified and graded data of the fusid/505740 data based on the quality knowledge base to obtain the data monitoring result.
[0025] Preferably, the data monitoring model is obtained by classifying and grading the sample fusion data of the current data governance platform and then generating a knowledge map and a quality knowledge base, including:
[0026] acquiring the sample fusion data of the current data governance platform;
[0027] performing graph calculation Nebula, off-line calculation Spark or real-time calculation
Flink on the sample fusion data through a knowledge calculation engine to generate a knowledge map and a quality knowledge base.
[0028] In a second aspect, an embodiment of the present disclosure provides a data monitoring system, including:
[0029] a data determining unit, which is configured to determine fusion data of a current data governance platform to be monitored,
[0030] a data monitoring unit, which is configured to input the fusion data of the current data governance platform to be monitored into a data monitoring model to obtain a data monitoring result output by the data monitoring model;
[0031] wherein the data monitoring model is obtained by classifying and grading the sample fusion data of the current data governance platform and then generating a knowledge map and a quality knowledge base.
[0032] In a third aspect, an embodiment of the present disclosure provides an electronic device, including a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the program, implements steps of the data monitoring method as provided by the first aspect described above.
[0033] In a fourth aspect, an embodiment of the present disclosure provides a non-transient computer-readable storage medium in which a computer program is stored, wherein the computer program, when executed by a processor, implements step of the data monitoring method as provided by the first aspect described above.
[0034] The embodiment of the present disclosure provides a data monitoring method and a system, wherein the method includes inputting the fusion data of the current data governance platform to be monitored into a data monitoring model to obtain a data monitoring result output by the data monitoring model; wherein the data monitoring model is obtained by classifying and grading the sample fusion data of the current data governance platform and then generating a knowledge map and a quality knowledge base. The present disclosure effectively solves the problem that it is difficult to govern data in data classification and grading and quality control work existing in the current data governance platform. LUS05740
[0035] In order to explain the technical scheme of the present disclosure or the prior art more clearly, the drawings needed in the description of the embodiments or the prior art will be briefly introduced hereinafter. Obviously, the drawings in the following description are some embodiments of the present disclosure. Other drawings can be obtained according to these drawings without creative labor for those skilled in the art.
[0036] FIG. 1 is a flow diagram of a data monitoring method according to the present disclosure.
[0037] FIG. 2 is a block diagram of a data monitoring model according to the present disclosure.
[0038] FIG. 3 is a block diagram of a data analysis model according to the present disclosure.
[0039] FIG. 4 is a schematic structural diagram of a knowledge map according to the present disclosure.
[0040] FIG. 5 is a schematic structural diagram of a data monitoring system according to the present disclosure.
[0041] FIG. 6 is a schematic structural diagram of an electronic device according to the present disclosure.
[0042] In order to make the purpose, technical scheme and advantages of the present disclosure more clear, the technical scheme in the present disclosure will be described clearly and completely with reference to the drawings in the present disclosure hereinafter. Obviously, the described embodiments are some embodiments of the present disclosure, rather than all of the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by those skilled in the art without creative labor belong to the scope of protection of the present disclosure.
[0043] A data monitoring method and a system according to the present disclosure will be described with reference to FIG. 1 to FIG. 6 hereinafter.
[0044] The embodiment of the present disclosure provides a data monitoring method. FIG. 1 is a flow diagram of a data monitoring method according to an embodiment of the present disclosure. As shown in FIG. 1, the method includes:
[0045] Step 110, determining fusion data of a current data governance platform to be monitored,
[0046] Step 120: inputting the fusion data of the current data governance platform to b&/505740 monitored into a data monitoring model to obtain a data monitoring result output by the data monitoring model;
[0047] wherein the data monitoring model is obtained by classifying and grading the sample 5 fusion data of the current data governance platform and then generating a knowledge map and a quality knowledge base.
[0048] Specifically, the present disclosure obtains the data monitoring model by classifying and grading the sample fusion data of the current data governance platform first, and then inputs the fusion data of the current data governance platform into the data monitoring model to obtain the data monitoring result of the current data governance platform.
[0049] Compared with the prior art, the method according to the embodiment of the present disclosure inputs the fusion data of the current data governance platform into the data monitoring model obtained after classifying and grading processing, which effectively improves the data governance level of the current data governance platform.
[0050] Based on any of the above embodiments, as shown in FIG. 2, the data monitoring model 200 includes a data analysis model 210 and a quality audit model 220;
[0051] inputting the fusion data of the current data governance platform to be monitored into a data monitoring model to obtain a data monitoring result output by the data monitoring model includes:
[0052] inputting the fusion data of the current data governance platform to be monitored into the data analysis model, and outputting classified and graded data of the fusion data based on the knowledge map;
[0053] inputting the classified and graded data of the fusion data into the quality audit model, and outputting the data monitoring result based on the quality knowledge base.
[0054] Specifically, the data monitoring model is realized by the data analysis model and the quality audit model, wherein the data analysis model classifies and grades the fusion data to be monitored in sequence based on the knowledge map including a classification map and a grading map. The quality audit model monitors the quality of the classified and graded fusion data based on the quality knowledge base to obtain the monitoring result.
[0055] Based on any of the above embodiments, as shown in FIG. 3, the data analysis model 300 includes a data feature extraction model 310, a data classification model 320 and a data grading model 330.
[0056] As shown in FIG. 4, the knowledge map 400 includes a classification map 410 and a grading map 420;
[0057] inputting the fusion data of the current data governance platform to be monitored into th&/505740 data analysis model 300 and outputting classified and graded data of the fusion data based on the knowledge map 400 includes:
[0058] inputting the fusion data of the current data governance platform to be monitored into the data feature extraction model 310, and outputting data feature values corresponding to the fusion data;
[0059] inputting the fusion data and the corresponding data feature values into the data classification model 320, and outputting classified data based on the classification map 410;
[0060] inputting the classified data into the data grading model 330, and outputting the classified and graded data of the fusion data based on the grading map 420.
[0061] Specifically, the data analysis model is realized by the data feature extraction model, the data classification model and the data grading model, wherein the data feature extraction model calculates the data feature values of the fusion data to be monitored, takes the corresponding data feature values as the weight values of the fusion data, inputs the weight values into the data classification model based on the classification map, and outputs the classified data after clustering according to the weighted value indexes and the weighted fusion data in the classification map; and the data grading model grades each category of the classified data based on the grading map, so as to obtain the classified and graded data of the fusion data.
[0062] Based on any of the above embodiments, as shown in FIG. 3 and FIG. 4, inputting the fusion data and the corresponding data feature values into the data classification model 320 and outputting classified data based on the classification map 410 includes:
[0063] weighting the fusion data by using the data feature values corresponding to the fusion data as weights to obtain the weighted fusion data;
[0064] clustering weighted value indexes of the weighted fusion data based on the classification map to obtain the classified data.
[0065] Specifically, the data classification model classifies the fusion data based on the classification map, that is, the data feature values corresponding to the fusion data are used as the weight values of the fusion data, the fusion data is weighted, and the classified data is obtained further based on the weight value indexes of the classification map.
[0066] Based on any of the above embodiments, as shown in FIG. 3 and FIG. 4, inputting the classified data into the data grading model 330 and outputting the classified and graded data of the fusion data based on the grading map 420 includes:
[0067] multiplying a weight value corresponding to the classified data by a preset grading coefficient to obtain a grading value corresponding to the classified data;
[0068] performing data division on the grading value corresponding to the classified data baséd/505740 on the grading map to obtain the classified and graded data of the fusion data.
[0069] Specifically, the data grading model finishes grading processing on the classified data based on the grading system of the grading map and according to the grading value corresponding to the classified data, wherein the grading value corresponding to the classified data is obtained by multiplying the weight value corresponding to the classified data by the preset grading coefficient.
[0070] Based on any of the above embodiments, inputting the classified and graded data of the fusion data into the quality audit model and outputting the data monitoring result based on the quality knowledge base includes:
[0071] retrieving, screening, deleting or retaining the classified and graded data of the fusion data based on the quality knowledge base to obtain the data monitoring result.
[0072] Specifically, the quality audit model retrieves, screens, deletes or retains the classified and graded data of the fusion data based on the quality knowledge base, wherein the quality knowledge base can be iteratively updated according to the quality experience.
[0073] Based on any of the above embodiments, the data monitoring model is obtained by classifying and grading the sample fusion data of the current data governance platform and then generating a knowledge map and a quality knowledge base, including:
[0074] acquiring the sample fusion data of the current data governance platform;
[0075] performing graph calculation Nebula, off-line calculation Spark or real-time calculation
Flink on the sample fusion data through a knowledge calculation engine to generate a knowledge map and a quality knowledge base.
[0076] Specifically, the knowledge map and the quality knowledge base in the data monitoring model are obtained by calculating the sample fusion data in the manner of Nebula, Spark or
Flink of the existing knowledge calculation engine.
[0077] A data monitoring system according to the present disclosure is described hereinafter.
The data monitoring system described hereinafter can correspond to the data monitoring method described above.
[0078] FIG. 5 is a schematic structural diagram of a data monitoring system according to an embodiment of the present disclosure. As shown in FIG. 5, the system includes a data determining unit 510 and a data monitoring unit 520;
[0079] the data determining unit 510 is configured to determine fusion data of a current data governance platform to be monitored,
[0080] the data monitoring unit 520 is configured to input the fusion data of the current data governance platform to be monitored into a data monitoring model to obtain a data monitorirt4505740 result output by the data monitoring model;
[0081] wherein the data monitoring model is obtained by classifying and grading the sample fusion data of the current data governance platform and then generating a knowledge map and a quality knowledge base.
[0082] Compared with the prior art, the system according to the embodiment of the present disclosure inputs the fusion data of the current data governance platform into the data monitoring model obtained after classifying and grading processing, which effectively improves the data governance level of the current data governance platform.
[0083] Based on any of the above embodiments, the data monitoring unit includes a data analysis module and a quality audit module;
[0084] the data monitoring model includes a data analysis model and a quality audit model,
[0085] the data analysis module is configured to input the fusion data of the current data governance platform to be monitored into the data analysis model, and output classified and graded data of the fusion data based on the knowledge map;
[0086] the quality audit module is configured to input the classified and graded data of the fusion data into the quality audit model, and output the data monitoring result based on the quality knowledge base.
[0087] Based on any of the above embodiments, the data analysis module includes a feature extraction module, a data classification module and a data grading module;
[0088] the data analysis model includes a data feature extraction model, a data classification model and a data grading model,
[0089] the feature extraction module is configured to input the fusion data of the current data governance platform to be monitored into the data feature extraction model, and output data feature values corresponding to the fusion data;
[0090] the data classification module is configured to input the fusion data and the corresponding data feature values into the data classification model, and output classified data based on the classification map;
[0091] the data grading module is configured to input the classified data into the data grading model, and output the classified and graded data of the fusion data based on the grading map.
[0092] Based on any of the above embodiments, the data classification module is specifically configured to:
[0093] weight the fusion data by using the data feature values corresponding to the fusion data as weights to obtain the weighted fusion data;
[0094] cluster weighted value indexes of the weighted fusion data based on the classificatid>05740 map to obtain the classified data.
[0095] Based on any of the above embodiments, the data grading module is specifically configured to:
[0096] multiply a weight value corresponding to the classified data by a preset grading coefficient to obtain a grading value corresponding to the classified data;
[0097] perform data division on the grading value corresponding to the classified data based on the grading map to obtain the classified and graded data of the fusion data.
[0098] Based on any of the above embodiments, the quality audit module is specifically configured to:
[0099] retrieve, screen, delete or retain the classified and graded data of the fusion data based on the quality knowledge base to obtain the data monitoring result.
[00100] Based on any of the above embodiments, the data monitoring model is obtained by classifying and grading the sample fusion data of the current data governance platform and then generating a knowledge map and a quality knowledge base, including:
[00101] acquiring the sample fusion data of the current data governance platform;
[00102] performing graph calculation Nebula, off-line calculation Spark or real-time calculation
Flink on the sample fusion data through a knowledge calculation engine to generate a knowledge map and a quality knowledge base.
[00103] To sum up, the embodiment of the present disclosure inputs the fusion data of the current data governance platform into a data monitoring model to obtain a data monitoring result output by the data monitoring model; wherein the data monitoring model is obtained by classifying and grading the sample fusion data of the current data governance platform and then generating a knowledge map and a quality knowledge base. The present disclosure effectively solves the problem that it is difficult to govern data in data classification and grading and quality control work existing in the current data governance platform.
[00104]FIG. 6 is a schematic structural diagram of an electronic device according to an embodiment of the present disclosure. As shown in FIG. 6, the electronic device can include a processor 610, a communication interface 620, a memory 630 and a communication bus 640, wherein the processor 610, the communication interface 620 and the memory 630 communicate with each other through the communication bus 640. The processor 610 can call logic instructions in the memory 630 to execute the data monitoring method. The method includes: determining fusion data of a current data governance platform to be monitored; inputting the fusion data of the current data governance platform to be monitored into a data monitoring model to obtain a data monitoring result output by the data monitoring model; wherein the datà/505740 monitoring model is obtained by classifying and grading the sample fusion data of the current data governance platform and then generating a knowledge map and a quality knowledge base.
[00105] In addition, the above logic instructions in the memory 630 can be realized in the form of software functional units and can be stored in a computer-readable storage medium when being sold or used as independent products. Based on this understanding, the technical scheme of the present disclosure can be embodied in the form of software products in essence or in the part that contributes to the prior art or in the part of the technical scheme. The computer software products are stored in a storage medium and include several instructions, so that a computer device (which can be a personal computer, a server, a network device, etc.) executes all or part of the steps of the method described in various embodiments of the present disclosure. The above storage medium includes: a USB flash disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other media that can store program codes.
[00106]In another aspect, the embodiment of the present disclosure further provides a computer program product. The computer program product includes a computer program stored on a non-transient computer-readable storage medium. The computer program includes program instructions. When the program instructions are executed by a computer, the computer can execute the data monitoring method provided by the above methods. The method includes: determining fusion data of a current data governance platform to be monitored; inputting the fusion data of the current data governance platform to be monitored into a data monitoring model to obtain a data monitoring result output by the data monitoring model; wherein the data monitoring model 1s obtained by classifying and grading the sample fusion data of the current data governance platform and then generating a knowledge map and a quality knowledge base.
[00107]In yet another aspect, the embodiment of the present disclosure further provides a non-transient computer-readable storage medium in which a computer program is stored. The computer program, when executed by a processor, executes the data monitoring method provided by the above methods. The method includes: determining fusion data of a current data governance platform to be monitored; inputting the fusion data of the current data governance platform to be monitored into a data monitoring model to obtain a data monitoring result output by the data monitoring model; wherein the data monitoring model 1s obtained by classifying and grading the sample fusion data of the current data governance platform and then generating a knowledge map and a quality knowledge base.
[00108] The device embodiment described above is only schematic, in which the units described as separate components may or may not be physically separated. The components displayed 4505740 units may or may not be physical units, that is, the components may be located in one place or distributed to a plurality of network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the scheme of the embodiment. Those skilled in the art can understand and implement the purpose without creative labor.
[00109] From the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be realized by means of software plus a necessary general hardware platform, and of course it can also be realized by hardware. Based on this understanding, the above technical scheme can be embodied in the form of software products in essence or in the part that contributes to the prior art. The computer software products can be stored in a computer-readable storage medium, such as ROM/RAM, a magnetic disk, an optical disk, etc., and include several instructions, so that a computer device (which can be a personal computer, a server, a network device, etc.) executes the method described in various embodiments or in some parts of the embodiments.
[00110]Finally, it should be explained that the above embodiments are only used to illustrate the technical scheme of the present disclosure, rather than limit the technical scheme. Although the present disclosure has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that it is still possible to modify the technical scheme described in the foregoing embodiments, or to substitute some technical features equivalently.
However, these modifications or substitutions do not make the essence of the corresponding technical schemes deviate from the spirit and scope of the technical schemes of various embodiments of the present disclosure.
Claims (10)
1. À data monitoring method, wherein the method comprises: determining fusion data of a current data governance platform to be monitored, inputting the fusion data of the current data governance platform to be monitored into a data monitoring model to obtain a data monitoring result output by the data monitoring model; wherein the data monitoring model is obtained by classifying and grading the sample fusion data of the current data governance platform and then generating a knowledge map and a quality knowledge base.
2. The data monitoring method according to claim 1, wherein the data monitoring model comprises a data analysis model and a quality audit model, inputting the fusion data of the current data governance platform to be monitored into a data monitoring model to obtain a data monitoring result output by the data monitoring model comprises: inputting the fusion data of the current data governance platform to be monitored into the data analysis model, and outputting classified and graded data of the fusion data based on the knowledge map; inputting the classified and graded data of the fusion data into the quality audit model, and outputting the data monitoring result based on the quality knowledge base.
3. The data monitoring method according to claim 2, wherein the data analysis model comprises a data feature extraction model, a data classification model and a data grading model; the knowledge map comprises a classification map and a grading map; inputting the fusion data of the current data governance platform to be monitored into the data analysis model and outputting classified and graded data of the fusion data based on the knowledge map comprises: inputting the fusion data of the current data governance platform to be monitored into the data feature extraction model, and outputting data feature values corresponding to the fusion data; inputting the fusion data and the corresponding data feature values into the data classification model, and outputting classified data based on the classification map; inputting the classified data into the data grading model, and outputting the classified and graded data of the fusion data based on the grading map.
4. The data monitoring method according to claim 3, wherein inputting the fusion data and the corresponding data feature values into the data classification model and outputting classified data based on the classification map comprises: LU505740 weighting the fusion data by using the data feature values corresponding to the fusion data as weights to obtain the weighted fusion data; clustering weighted value indexes of the weighted fusion data based on the classification map to obtain the classified data.
5. The data monitoring method according to claim 3, wherein inputting the classified data into the data grading model and outputting the classified and graded data of the fusion data based on the grading map comprises: multiplying a weight value corresponding to the classified data by a preset grading coefficient to obtain a grading value corresponding to the classified data; performing data division on the grading value corresponding to the classified data based on the grading map to obtain the classified and graded data of the fusion data.
6. The data monitoring method according to claim 2, wherein inputting the classified and graded data of the fusion data into the quality audit model and outputting the data monitoring result based on the quality knowledge base comprises: retrieving, screening, deleting or retaining the classified and graded data of the fusion data based on the quality knowledge base to obtain the data monitoring result.
7. The data monitoring method according to claim 1, wherein the data monitoring model is obtained by classifying and grading the sample fusion data of the current data governance platform and then generating a knowledge map and a quality knowledge base, comprising: acquiring the sample fusion data of the current data governance platform; performing graph calculation Nebula, off-line calculation Spark or real-time calculation Flink on the sample fusion data through a knowledge calculation engine to generate a knowledge map and a quality knowledge base.
8. A data monitoring system, comprising: a data determining unit, which is configured to determine fusion data of a current data governance platform to be monitored, a data monitoring unit, which is configured to input the fusion data of the current data governance platform to be monitored into a data monitoring model to obtain a data monitoring result output by the data monitoring model; wherein the data monitoring model is obtained by classifying and grading the sample fusion data of the current data governance platform and then generating a knowledge map and a quality knowledge base.
9. An electronic device, comprising a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor, when executing th&505740 program, implements steps of the data monitoring method according to any of claims 1 to 7.
10. A non-transient computer-readable storage medium in which a computer program is stored, wherein the computer program, when executed by a processor, implements step of the data monitoring method according to any of claims 1 to 7.
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210913441.5A CN114969018B (en) | 2022-08-01 | 2022-08-01 | Data monitoring method and system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| LU505740A1 LU505740A1 (en) | 2024-02-19 |
| LU505740B1 true LU505740B1 (en) | 2024-06-17 |
Family
ID=82969442
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| LU505740A LU505740B1 (en) | 2022-08-01 | 2022-12-13 | Data monitoring method and system |
Country Status (3)
| Country | Link |
|---|---|
| CN (1) | CN114969018B (en) |
| LU (1) | LU505740B1 (en) |
| WO (1) | WO2024027071A1 (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114969018B (en) * | 2022-08-01 | 2022-11-08 | 太极计算机股份有限公司 | Data monitoring method and system |
Family Cites Families (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109086458A (en) * | 2018-09-12 | 2018-12-25 | 杭州格原信息技术有限公司 | A kind of search engine system applied to reconnaissance projecting trade |
| CN111460167A (en) * | 2020-03-19 | 2020-07-28 | 平安国际智慧城市科技股份有限公司 | Method for positioning pollution discharge object based on knowledge graph and related equipment |
| CN111858236B (en) * | 2020-06-23 | 2022-12-16 | 深圳精匠云创科技有限公司 | Knowledge graph monitoring method and device, computer equipment and storage medium |
| CN112182234B (en) * | 2020-07-29 | 2022-06-28 | 长江勘测规划设计研究有限责任公司 | Basin flood control planning data knowledge graph construction method |
| CN112256887B (en) * | 2020-10-28 | 2022-06-24 | 福建亿榕信息技术有限公司 | Intelligent supply chain management method based on knowledge graph |
| CN112801164B (en) * | 2021-01-22 | 2024-02-13 | 北京百度网讯科技有限公司 | Training method, device, equipment and storage medium of target detection model |
| CN112990656B (en) * | 2021-02-05 | 2023-05-05 | 南方电网调峰调频发电有限公司信息通信分公司 | Health evaluation system and health evaluation method for IT equipment monitoring data |
| CN113505241B (en) * | 2021-07-15 | 2023-06-30 | 润建股份有限公司 | Intelligent diagnosis method for potential safety hazards of electricity consumption based on knowledge graph |
| CN114818707A (en) * | 2022-03-02 | 2022-07-29 | 北京航空航天大学 | Automatic driving decision method and system based on knowledge graph |
| CN114706994A (en) * | 2022-03-21 | 2022-07-05 | 华迪计算机集团有限公司 | Operation and maintenance management system and method based on knowledge base |
| CN114969018B (en) * | 2022-08-01 | 2022-11-08 | 太极计算机股份有限公司 | Data monitoring method and system |
-
2022
- 2022-08-01 CN CN202210913441.5A patent/CN114969018B/en active Active
- 2022-12-13 LU LU505740A patent/LU505740B1/en active
- 2022-12-13 WO PCT/CN2022/138505 patent/WO2024027071A1/en unknown
Also Published As
| Publication number | Publication date |
|---|---|
| CN114969018A (en) | 2022-08-30 |
| CN114969018B (en) | 2022-11-08 |
| LU505740A1 (en) | 2024-02-19 |
| WO2024027071A1 (en) | 2024-02-08 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10606862B2 (en) | Method and apparatus for data processing in data modeling | |
| CN105718490A (en) | Method and device for updating classifying model | |
| LU505740B1 (en) | Data monitoring method and system | |
| CN111754044A (en) | Employee behavior auditing method, device, equipment and readable storage medium | |
| CN115576834A (en) | Software test multiplexing method, system, terminal and medium for supporting fault recovery | |
| CN115392592B (en) | Storage product parameter configuration recommendation method, device, equipment and medium | |
| CN115063035A (en) | Customer evaluation method, system, equipment and storage medium based on neural network | |
| CN116467171A (en) | Automatic test case construction device, method, electronic equipment and storage medium | |
| CN109977977B (en) | Method for identifying potential user and corresponding device | |
| US11227288B1 (en) | Systems and methods for integration of disparate data feeds for unified data monitoring | |
| CN111858236A (en) | Knowledge graph monitoring method and device, computer equipment and storage medium | |
| CN116226108A (en) | Data management method and system capable of realizing different management degrees | |
| CN116185684A (en) | Construction method and application of server fault root cause analysis model | |
| CN114881521A (en) | Service evaluation method, device, electronic equipment and storage medium | |
| CN115204501A (en) | Enterprise evaluation method and device, computer equipment and storage medium | |
| CN114493379B (en) | Enterprise evaluation model automatic generation method, device and system based on government affair data | |
| CN112966031A (en) | Data processing method and device, electronic equipment and computer readable storage medium | |
| CN113886579B (en) | Construction method and system, identification method and system for positive and negative surface models of industry information | |
| CN116541382B (en) | Data management method and system based on data security identification level | |
| Fernandes et al. | Impact of Non-Fitting Cases for Remaining Time Prediction in a Multi-Attribute Process-Aware Method. | |
| US20220237388A1 (en) | Method and apparatus for generating table description text, device and storage medium | |
| CN115758135A (en) | Functional requirement tracing method and device for rail transit signal system and electronic equipment | |
| CN115408378A (en) | Data verification method and device | |
| CN116069979A (en) | Metadata type dividing method, electronic device and storage medium | |
| CN118227507A (en) | Method and system for testing suitability of information creation equipment |